Share
The primary mission of many UAVs is to provide intelligence in the form of geo-reference information for both civilian and defence users. Architecturally, they can be viewed as a mobile network of connected objects, each object being a dedicated computer with a special function, such as the attitude heading and altitude reference system (AHARS), autopilot or flight management computer (FMC), and mission computer.
Each of these dedicated computers is connected using a different network or link interface using different topologies and communication protocols. Mission computers, in particular, are constantly gaining in computing power per unit of electrical power and weight, as the growth in sensor data means algorithms constantly require more power.
Many applications, such as military intelligence surveillance and reconnaissance (ISR), oil and gas infrastructure surveillance or maritime surveillance, require UAVs to fly long-endurance, long-range missions at low altitude. These applications share several common points but also have specific requirements based on the operation’s location, type of threats, environment as well as other factors.
• For many governments and users, life-cycle costs have become of paramount importance. These must be reined in line with capabilities, performance and the ability to maintain equipment and personnel in good operating conditions. This important factor has multiple implications for technology and design.
• Systems need to be deployed beyond line of sight so operators remain at a safe distance from potentially harmful areas.
• UAVs are in demand to monitor, identify and recon specific objects of interest – mission commanders need intelligence not raw data
• UAVs need to be operated at safe but low altitude to provide the necessary sensor data resolution.
• With endurance reaching more than 20 hours in some cases, the cost of personnel on the ground is of outmost importance and sometimes greatly outgrows the cost of acquisition after a few years.
• Data dissemination at short and long distance is extremely important to ground personnel operating with the UAVs’ results and to supervisory personnel in need of data to make appropriate decisions.
One common attribute that affects all the needs listed above is the ability to process sensor data on board. This trend has been clear since the middle of the last decade, but difficult to implement in small aircraft. In addition, the need for an open architecture to support software reusability and on-board sensor data processing has been fighting an uphill battle as few serious standards have emerged since 2005.
Automatic threat detection
Real-time image processing for automatic threat detection has been tested since the mid 2000s in the oil and gas industry and has proven to be very useful. The airborne system components used include a rugged computer integrated into the UAV, a gyro-stabilised gimbal integrating one or two cameras in the visible and the infrared spectrum, as well as an attitude and heading reference system to generate metadata. The rugged computer acts as a mission computer and includes many functions to manage the hardware on board, interface with navigation systems as well as data-link. The ground control computer software includes an alarm logging function that receives alarms from the aircraft and displays them on the flight course.
To detect threats, ‘change detection’ is used. This involves analysing images from video or still imagery and detecting groups of pixels that are different to their immediate surroundings. As the UAV has real-time information about its position and attitude, as well as the time when the image was generated, the image can then be geo-referenced with appropriate processing.
In Figure 1, the truck and its traces can clearly be differentiated from the rest of the picture, making change detection relatively simple. However, given the fact that UAV operations can take place in many different land and maritime locations, at different seasons, day or night, and in different meteorological conditions, the sensors and algorithms need to be configured to handle a wide variety of backgrounds.
One method of doing this involves the use of the scale-invariant feature transform, which computes the image intensities around interesting locations in the image domain – aka ‘interest points’ or ‘key points’.
The analysis of primary components is a statistical method that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called ‘principal components’. The number of principal components is less than or equal to the number of original variables. This transformation is defined in such a way that the first principal component has the largest possible variance and each following component in turn has the highest variance possible under the constraint that it is orthogonal to the preceding components. The resulting vectors are an uncorrelated orthogonal basis set. The principal components are orthogonal because they are the eigenvectors of the covariance matrix, which is symmetric. One of the downside of this method is the sensitivity to the relative scaling of the original variables.
Medium complexity
Another example of imagery is showed in
Figure 2, which highlights a background of medium complexity. The background includes different kinds of objects that can easily be identified as ‘objects of interest’, as well as similar objects that are no threats. This situation introduces the need for classification of objects and learning capabilities as the imaging software processes backgrounds and new objects, and the figure shows preliminary annotations, with red showing a positive object, green a false object, the identification being based on a database check that prevents false alarms.
Once detected objects are confirmed, an alarm is generated by the UAV on-board mission computer and transmitted using a low-bandwidth data link. Once the operator is made aware of the alarm, he or she can request an image sample and receive an annotated image with geo-referenced information. That information is usually generated as the result of an underground pipeline inspection. Image annotations includes the type of objects, actual location and time of survey.
While these techniques represent a clear advance in airborne surveillance on board UAVs, there is plenty of room to keep improving detection performance.
Maritime environments
Additional work in maritime environments demonstrates the capabilities of this approach with a different background and for the detection of different objects, such as boats and even fish, for environmental applications. The maritime environment changes as the result of factors, such as the time of operation (day or night), meteorological conditions (wind, sun, cloud cover, etc.), sea state and winds. Depending on the changes, this can make the detection of objects either more or less complicated.
The examples in figures 3, 4 and 5
demonstrate the level of false alarm augmenting or diminishing needed, based on the image background or scene specifics.
Summary
The ability to process data on board a UAV improves detection capabilities and therefore surveillance efficiency, and reduces workload and consequently operating costs. While I discussed only one method in this article, other methods can be considered, especially when linking on-board processing and data link capabilities.
Aero Surveillance has pioneered the implementation of some of these technologies and implemented such capabilities on UAVs weighing less than 25kg, in order to enhance positive detection in broader environments, reduce false alarms and increase overall surveillance performance.
Philippe Roy is president and CEO of Aero Surveillance (www.aerosurveillance.com)
