AI Fundamentals: The Evolution of AI in Drone Systems

Artificial intelligence and deep machine learning are allowing UAS to scale and intensify their impact across many sectors.

Every now and then, the world experiences significant bursts of innovation thanks to random concurrence of favorable conditions in technology, business and society. The explosive growth in innovative applications of Unmanned Aerial Vehicles (UAVs) over the last 10 years can only be matched by the World Wide Web and the iPhone.

Remote controlled flight had existed since the Royal Air Force launched the Queen Bee aircraft in 1935. After that, drones evolved to higher levels of sophistication, but remained, for the most part, limited to military applications. Perhaps the two watershed moments that turned drones into true consumer products happened in 2010, when Parrot flew its AR Drone with an iPhone app at CES, and in 2013, when DJI equipped its Phantom drone with a camera.

Ease of navigation, image capture capabilities and miniaturization of ARM processors provided fertile ground for AI applications across all sectors. This powerful combination allowed drones to permeate an impressive variety of domains, including ISR (intelligence, surveillance and reconnaissance) and targeting, film, agriculture, logistics, engineering and disaster response. The iPhone, the GoPro camera, GPS and AI were technology kindling. The number of problems that have been waiting for a UAV solution is only bound by imagination.

Today’s drones come prepackaged with GPS sensing and navigation capabilities, video capture, and command and control applications, as well as several interfaces that allow the implementation of AI and special purpose programs. This combination puts the “system” in UAS.

THE AI ADVANTAGE

Computers enabled massive automation and scaling of information processing. AI, on the other hand, moved computation to a much higher level, enabling massive automation of reasoning, or more precisely, solution discovery and parameter optimization. These two operations allowed machines to learn.

An AI thermostat, for instance, can “learn” the optimal temperature setting that minimizes the number of trips a person takes to warm or cool a home or office. To do this, a machine learning algorithm collects many data points such as room temperature and the action the person takes every time she or he sets the thermostat. This gives the algorithm valuable feedback on what the proper setting should be given ambient temperature, time of day, etc. All machine learning algorithms follow the same principle, except they use hundreds or thousands of “thermostats,” each of which learns an obscure parameter inside the algorithm. The accumulation of learning from these units results in machine intelligence such as recognizing a cat in an image, parallel parking a trailer truck, translating text from Mandarin to Arabic with no prior knowledge of vocabulary or grammar, or identifying terrorist cells in a social network.

The most successful and practical applications of drones today leverage AI for image recognition and image stitching. While this is far short of the full promise of AI, it is making a tremendous impact, helping to automate and scale a vast array of applications, at minimal cost. Drones are flying inside Europe’s cathedrals and architectural treasures to build comprehensive 3D models. Drones count sheep in Israel, and, surprisingly, stay awake. Civil engineers use drones to constantly scan large structures like bridges, dams and oil rigs to detect structural faults before they grow into serious problems. And, of course, the military has been using drones for decades now to collect field intelligence. Drone imaging is becoming so prevalent that it will be virtually impossible to find a domain that has not been permeated.

CONDITIONS MATTER

Drones that scan the insides of a cathedral, for instance, enjoy many luxuries that are rarely afforded to many in other domains. The drones can easily fly back to charging stations every time they need power. They can use broadband to upload their high-resolution videos to the cloud to be processed by gigantic server farms. They are protected from wind, rain, theft and attacks, and run little cybersecurity risk. AI cannot do its job without some of these luxuries.

The challenge of operating applications in hostile environments such as war zones, or forest fires, requires a great deal of support infrastructure as well as bigger, more powerful drones. This challenge provides a great opportunity for AI to provide “operational support” functions. Albeit less glamorous than object identification, navigation and context determination, these support functions optimize and preprocess the data to mitigate the lack of readily available powerful computing and high-speed data transfer.

For example, rather than upload high- resolution images to the server, or process them on the drone, AI algorithms are used to sample the images and only capture relevant features, such as edges, position of each eye relative to the nose, etc. These features can then be uploaded to a server requiring much lower bandwidth and using much less power. Another powerful approach, which is frequently used in movies, consists of taking a few high-resolution static images and low-resolution video; AI algorithms then use the two to construct high-resolution video. Once high-resolution video is on the server, then virtually endless computing power can be used by advanced AI. A drone equipped with FHD video, for instance, captures images that contain 1920×1080 pixels (about 2 million). Using images, an eighth the FHD resolution produces 240×135 (or 32,400 pixels), which still provides recognizable images and results in an order of a 64-fold reduction in power required for capturing, storing and transmitting the images, as well as required bandwidth. Many AI applications only use 64×32 images, which results in a 1024-fold reduction.

Advances in computing and battery technology as well as increased availability of bandwidth will make these support functions less relevant in many applications. However, the drive for miniaturization and continued expansion of drones onto new domains of application with hostile environments will always require additional support functions.