Swarm Unmanned Aerial Vehicles (SUAVs): A Comprehensive Analysis of Localization, Recent Aspects, and Future Trends
Abstract
Within the last decade, Swarm Unmanned Aerial Vehicles (SUAVs) are booming and growing at a surprisingly rapid pace. From military combat, environmental surveillance, and air transport to the blossoming public entertainment sector, there is a wide range of UAV applications. For these above use cases, the accurate location of the target of interest can be requested/queried, which is very important for their mission accomplishment. In the case of GPS-armed SUAVs, this is an easy task. However, the GPS signal can be obscured, affected by environmental conditions, or suppressive jamming. Therefore, location information needs to be improved/assisted by some other localization techniques, which constitutes the main scope of this article. Besides, with the advancements in localization, guidance, and communication technologies; future SUAVs will be operating autonomously by distributing tasks and coordinating the operation of many UAVs. Thus, UAV to UAV communication is presented as it constitutes the SUAV’s autonomous coordination ability. In addition, future research directions and open challenges that need to be addressed, including autonomous SUAVs, are also discussed.
1. Introduction
Drones have emerged from toys to sophisticated aerial devices, and many application areas followed [1, 2], such as advanced vision at high altitude, surveillance, recognition capabilities, and rescue and logistics management. One of the most spectacular pieces of news came from Amazon Seattle R&D laboratory when it announced that drones are being tested (and awaited by FAA regulations to publicize) for package delivery in the USA [3] through the Amazon Prime Air program, in order to provide faster delivery, low costs, low emissions, and happy customers.
Unmanned aerial vehicles (UAVs), sometimes mistakenly referred to as drones and contrary to this common misconception, are more sophisticated and advanced aircraft. UAVs have an extremely long-range operation capability (both altitude and distance) when compared to drones. They are piloted by remote control or embedded computer programs without a human on-board and can operate for many hours on air without physical intervention. Swarms of such aerial autonomous unmanned aerial vehicles (SUAVs) are emerging as they provide advantages during combat operations [4, 5]. Further, as the agriculture industry is approaching an important milestone nowadays, an interesting application field of SUAV shortly soon would be smart agriculture (e.g., see Figure 1). The traditional methods in agriculture are evolving into cognitive and autonomous ways of cultivation, which are also referred to as “AgriFood 4.0” [6, 7]. Within this context, by cooperatively scanning the whole field, SUAV can execute the assigned task (observation, seeding, applying pesticides, etc.) in minutes instead of an hour-long job that can be fulfilled via a single UAV. AgriFood 4.0 will transform traditional ways of farming into innovative ones by leveraging all the benefits of information, computing, electronics, mechatronics, engineering, and physics sciences altogether, according to us, foremost SUAVs.
Therefore, to effectively accomplish either of these missions, a swarm must allow all the team to safely and reliably complete their tasks. Not only must each UAV of the swarm maintain formation and avoid collisions with each other but they must also prevent collisions with different obstacles in the environment [8]. In addition, SUAVs move and change their positions frequently; thus, they must repeatedly estimate them based on the new flight conditions. However, flying several UAVs at once mainly led to significant challenges in localizing and monitoring them in a safe and convenient way [9, 10]. Hence, it is required to design new localization schemes to tackle this challenging topic.
- (1)
The fundamental requirements of every localization scheme are analysed by describing the major prospects to obtain accurate localization information by SUAVs
- (2)
Thereafter, the related work on localization techniques used by SUAVs is discussed, while an up-to-date comprehensive overview is provided and the challenges and issues surrounding the studied schemes are delineated
- (3)
Furthermore, the open problems and potential future research directions are highlighted
The rest of this paper is organized as follows: Section 2 presents a taxonomy of SUAV localization requirements, whereas Section 3 provides state-of-the-art localization techniques and systems and their potentials for SUAVs. Research challenges and future directions are presented in Section 4. Finally, Section 5 concludes the paper and the topic.
2. Localization Requirement of the SUAVs
Despite the great progress achieved in the field of UAVs, localization is one of the most challenging problems to tackle, as it raises several issues and plays a key role in many applications [9]. For example, in air transportation missions, monitoring targets and areas of interest, search and rescue operations, natural disasters, forest fires, etc. In fact, early localization prediction helps to plan appropriate emergency system interventions that can prevent such events from occurring or mitigate resulting damage. In addition, during localization, some important challenges that UAVs have to face are as follows: UAV velocity (causing Doppler shift and thus causing positioning errors), strong multipath (causing RF signals to attenuate), non-LOS (LOS positioning schemes cannot operate under this condition), and computational load (causing onboard CPU units to have bottlenecks) [10].
Moreover, the architecture of UAV communications is significantly affected by the antenna layout and the asset delivery platforms used for their transmissions. The design of antennas for UAV localization is an important research avenue and can be achieved using several techniques, such as 3D MIMO [11]. To date, researchers in the field of UAV localization have studied different use cases for cellular networks. Thus, various research works have been proposed on UAV-assisted cellular networks, as the wireless cellular infrastructure promises to solve many of the limiting factors in frequency planning and resource utilization. For example, the recent developments in cellular systems allow UAVs to be remotely controlled beyond the visual Line of Sight (LoS) [12]. The Industrial, Scientific, and Medical (ISM) frequency band is employed by most UAV civil applications [13]. However, the main challenge with the usage of ISM bands is that they are very limited in terms of frequency bandwidths and therefore result in limited flight coverage. To overcome this problem, the exploitation of cellular frequency bands and therefore the use of cellular networks for communications are considered. For instance, [14] shows how the 4G-LTE systems can be successfully utilized for UAV-based real-time video surveillance and streaming.
Further, localization via triangulation by using GPS signals has been successfully executed, especially for the stationary or slowly moving receivers, for the last two decades, and is not something new [15]. What is challenging here is, executing the same operation on fast-moving and unstable platforms such as UAVs. In the aviation industry, this is solved again with the inclusion of another location estimation unit called INS (Inertial Navigation System), and today’s aircraft (airplanes, helicopters, etc.) successfully use INS-GPS couple to estimate their locations. Again, this does not apply to the UAVs, due to the heavy installation weight of these devices which is against the strict lightweight equipment requirements of the UAVs.
In the following, we provide a broad overview of key requirements critical to any localization approaches and discuss main prospects encountered in obtaining localization information by SUAVs (Figure 2 illustrates some applications such as “air-based network” and “ground-based network”):
2.1. Localization Accuracy
Localization accuracy refers to how correct is the location estimation relative to the actual position by describing the consistency of the estimates. In fact, it depends on the range measurement errors, the UAV noise calibration, its position in the network, the effects of random actions, inaccurate models, and the environmental conditions.
2.2. Reliability
UAVs are inherently unreliable to failures since they are prone to physical damages, energy depletion, and harsh deployment environments. Thus, the localization schemes designed for UAV swarm must be resilient to failures and have the ability to provide error control and correction mechanisms.
2.3. The Environment Monitoring
The localization process is entirely dependent on the properties of the environment in which the UAVs are intended to operate.
2.4. Noisy Measurements
This might cause wrong positioning and lead to uncertainty in the localization of the intended target position by the UAV.
2.5. Flexibility
It is related to the number of UAVs that a remote control system can handle. In addition, it also can be thought of as interrelated to noisy measurements and errors of the UAVs. Deals with how ill effects of noisy measurements and errors in the UAV systems would be handled. For instance, the higher the noise is, the less the accurate acquired position read by the UAV is.
2.6. Heterogeneity and Interoperability
Particularly in hybrid settings, two or more localization systems/algorithms are involved. Handshaking and interoperability of the signals among those systems is a challenge to tackle, for instance, what parity system is preferred, what is the chosen message size, what a message consists of, what endianness is being used (big vs. little), and what measurement system (metric vs. imperial) is chosen.
2.7. Cost
It is in terms of infrastructure installation requirements, computation capacity, energy supply, the cost per UAV, and any additional hardware required to establish its position.
3. Key Techniques for Localization of the SUAVs
The focus of researchers on designing localization systems for UAVs has resulted in a very high number of schemes. These schemes are distinguished by their performance and also by their adaptations to the applications considered. However, though numerous, as can be seen from existing literature [16–18], only a few localization techniques for swarm have resulted in products ready for deployment in the real world. It is then important to carry out an analysis to better understand the factors that must be solved to successfully develop localization methods for SUAVs. Hence, a literature review was necessary.
In this section, we present an overview of different categories of techniques that are the most popular in the literature while comparing and emphasizing the advantages of each.
3.1. Computer Vision Techniques
The augmentation of onboard technological equipment such as external cameras, multiple types of sensors, and other systems while designing UAVs has demonstrated the potential to improve their positioning and mapping. In fact, with such systems, more real-time aerial images and information are obtained by them and under different environmental conditions [3].
Considering the triggering nature of UAV swarm tasks and their objectives, current vision-based localization methods can be roughly divided into three categories (see Figure 3).
The first refers to push-button approaches that incorporate visual services, target recognition, and exploration missions. The second is the predictable tasks relating to mapping and obstacle detection, and the third is the controlled tasks that hinge on flight control, navigation, and tracking. Push-button methods generally require fulfilling tasks from take-off to landing, with limited or no human intervention. They enable us to estimate accurate positions of UAVs in the environment using visual information, auxiliary visual markers, and aerial images obtained from the vision system. In contrast, controlled approaches provide more flexible solutions in which UAVs perform flights through navigation towards a specific target, where they are visually guided to follow a path defined by images or visual features that they need to detect and track. Predictable methods are broadly referred to as the process of computing the specific optimal distribution of objects in space, according to certain constraints or requirements of the environment. Therefore, there is a use of a mapping process in which UAVs can navigate in order to avoid obstacles. Furthermore, as they come usually with a variety of visual odometry systems, captured data and images can be used for localization and creating accurate navigation maps [2, 4].
One of the most recent works dealing with aerial surveillance systems based on SUAVs was proposed by Trotta et al. [19]. In this work, a swarm of UAVs is used in the permanent monitoring of the activities of mobile ground targets (MGT) via continuous video capture. Indeed, the tracking of several MGTs requires jointly addressing research issues related to computer vision invoking target location, energy, and mobility management. As a result, a new framework called “PERCEIVE” was proposed for continuous video surveillance of crowds using UAVs. The latter are periodically replenished by Mobile Charging Stations (MCSs). The PERCEIVE architecture is inspired by recent software-defined networking (SDN) since it separates these two functionalities: (1) UAV data acquisition and micromobility plan, which is carried out on the edge, and (2) the swarm control plan, which is realized on the cloud by a chain of services using different modules.
3.2. Cooperative Localization Methods
Cooperative localization (CL) has been widely studied in mobile robotics and UAVs. The basic concept behind this technique is to use several UAVs in a team that collaborates to set their positions and/or one of the targets [2]. This team cooperation enhances location accuracy by fusing various measurements from multiple position estimates. Further, applying CL not only increases the position accuracy of each UAV but also that of the entire group, which can operate effectively as a swarm. For instance, Bryson and Sukkarieh [20] presented a CL method of multiple UAVs to construct a global shared map, without using GPS. UAVs are distributed over a large area where each one builds its map locally and sends it to the others via a central node. Map information provided by others is used to assess each UAV position, which in turn enhances map accuracy. This is mainly due to the strong coupling of localization and mapping in Simultaneous Localization and Mapping (SLAM) using the Extended Kalman Filter (EKF). In contrast to centralized MAP-based, computing can be distributed among the CL team such as using consensus theory, where a team of UAVs can manage the processing system to converge towards a common agreement and value in a distributed manner. Facinelli et al. [21] presented an interesting example where an SUAV was envisaged to localize and detect the source of a gas leak in a chemical factory. The authors proposed three distributed exploration strategies based on the consensus theory which exploits UAVs’ position information. Furthermore, this UAV team is controlled by a dynamic distributed localization process triggered when the source of the gas dispersion is detected.
3.3. Centralized vs. Distributed
According to the level of cooperation between the team of UAVs, SUAV localization techniques can be classified into two categories: centralized vs. distributed. Compared to distributed strategies, in centralized mechanisms, a single unit controls all the swarm [2]. This central entity requires high-power computing as it is responsible to treat the team of a swarm as one system by combining their measurements to update the state estimate of all UAVs [22]. Therefore, it can plan its actions accordingly to deliver the best localization accuracy. The planning can be performed online and/or ahead of time. In contrast, in distributed systems, UAVs independently calculate their positions using location information collected from their neighbours and make their decisions locally. However, the methods by how the measurement information is combined, collected, and shared between UAVs vary based on the controller processing. There are many formation strategies for holding a team of UAVs, of which two are prominent, the leader-follower approach [23] and the behavior-based method [2, 22], as illustrated in Figure 4.
In the leader-follower method, each UAV member refers to a single leader that controls the movement, position, and formation of the whole swarm. Nevertheless, the main drawback of this method is the problem of reliability and robustness. Indeed, when the swarm depends on a physical leader that might fail, the system will be unreliable and less scalable. To overcome this issue, a virtual leader will be selected in the formation, or multiple leaders can be chosen to guarantee a UAV consensus to generate their position estimates and consequently, avoid collisions [23].
The behavior-based method, on the other hand, was inspired by the behavior of migratory birds (such as holding a V-formation) in terms of animals’ collective movement [22]. It is a natural self-organized structure solution since each member follows specific rules to perform group behavior. In addition, each UAV should guarantee a predefined safety distance of intraswarm and interswarm with its neighbours, to avoid collision and obstacles. In fact, the safety distance or relative positioning between neighbours during a flight allows them to move as a single unit. This can be achieved by ordered manual or automatic flight control commands. A large level of detail of the swarm behavior design task can be found in [2, 22].
3.4. Outdoor vs. Indoor Localization
Localization techniques can also be grouped into two major categories due to where the localization takes place: indoor vs. outdoor. The most distinguished among these is the usage of GNSS aid during the localization outdoors for precise estimation. However, GNSS connectivity in congested urban areas with a dense distribution of skyscrapers might be challenging even for outdoor navigation. In the same manner, localization is a real challenge for indoor environments. Hence, GNSS signals are blocked by the thick walls, concrete, and steel; some other methods are used for indoor localization.
For outdoor localization, other than very well-known techniques/technologies, some other methodologies are also being proposed. An example would be a UAV-embedded microphone array system for sound source localization in outdoor environments. The concept is based on exploiting sound information as an aid for localization of the UAVs in search and rescue activities to compensate for poor visual information [24, 25].
Indoor localization and navigation of UAVs constitute a critical part of autonomous flight and automated visual inspection of elements in continuously changing environments such as construction sites.
In order to enhance the precision for indoor/outdoor localization and navigation, usage of SUAVs in which UAVs share useful information in a collaborative way to enhance data of overall consensus is a good candidate solution.
3.5. IA/ML-Based Localization Strategies
Localization techniques based on artificial intelligence (AI) using Machine Learning (ML) have emerged as a promising approach to provide SUAV and target positions [26]. These methods can learn the properties of the environment and quickly adapt to the behavior of SUAVs [27]. Indeed, despite their high mobility, IA/ML methods allow them to independently improve their performance and thus give better accuracy of their estimated positions. However, while their benefits are invaluable, one of the main challenges of these techniques is the cumbersome process of adapting to dynamic conditions as they have to manage the increasing volume of all training data [28, 29]. For example, in [30], a fuzzy, multiobjective cooperative surveillance method is conceived in order to effectively track targets. Their states are predicted using the EKF combinatorial method and probability estimation. Then, distributed predictive control is used to obtain the optimal local path of each UAV within collision avoidance, minimum turning radius, and control input constraints. Meanwhile, the multiobjective model with preemptive priorities is solved by using fuzzy satisfying optimization to balance the multiobjective optimization requirement and the preemptive priorities.
4. Research Challenges and Future Directions
Some of the requirements of the SUAVs stated in Section 2 are still valid today and constitute obstacles to be tackled in this research area, namely, measurement errors due to a noisy environment, interoperability, and scalability. Moreover, several other relevant and interesting topics deserve further investigation as UAVs represent a huge market opportunity for manufacturers of equipment, investors, and business service suppliers. According to the CompTIA research report [31], the addressable global market for a UAV will reach $43.1 billion by 2024 vs. only $14.1 billion in 2018. The potential market is expected to be dominated by construction (civilian infrastructure) and agriculture sectors as shown in Figure 5 (the plotted data is collected from [31]).
In the following subsections, we will set some of the SUAV potential future research directions.
4.1. Localization over 5G, B5G, and 6G
We are in the era of mobile communications, and developments in these technologies (LTE, 4G/5G/6G, NB-IoT, etc.) will pave the way for easy and seamless M2M communications. These technologies are based on wireless communication standards such as WiMAX, CDMA, OFDM, MC-CDMA, and UWB. Besides providing communications, in the recent past, cellular technologies have been also employed for localization. Particularly, before GPS was introduced and vastly deployed, cellular technology was one of the most trusted and reliable sources of localization techniques from which humankind benefited in the search and rescue operations. On the other hand, quantum communications will play a key role in these future technologies as they will far exceed the capabilities and performance of their traditional counterparts [32]. Not only do they enable long-distance communications but also do offer very high security and resistance to signal degradation. Therefore, a deep investigation is required on this topic.
4.2. Energy Harvesting and Battery-Free UAVs
Energy limitation is a fundamental constraint in SUAVs due to the limited onboard battery capacity of every UAV that restricts its flight time and autonomy [9]. Therefore, this delicate subject can be oriented towards two main questions: global (i.e., the best trade-off and scheduling of energy use between the SUAV for increasing the flight time of the whole team) and local (i.e., the design of hardware for each UAV to improve its rechargeable battery or endurance). However, batteries powered with energy harvesting depend on the energy scavenged from their environment, which is uncontrollable and spatially and temporally dynamic. Consequently, significant research efforts must be devoted to developing new technologies for energy delivery.
4.3. Cooperative Localization in the Internet of Everything (IoE)
Cooperation between SUAVs should always be guaranteed despite the harsh environmental conditions and failures of their hardware [33]. Besides, collision avoidance and the fulfilment of their tasks and decision-making rely on the flow of information shared with/among the UAVs and their coordination. However, cloud robotics and autonomous multirobot systems [34] are nowadays a rising field in the robotics community. Thus, their use in sharing data and resources of distributed robotic systems is leading to the design and development of cloud robotic systems (CRS). Ontologies present a promising solution in identifying formal concepts and their relationships in an interoperable environment. They aimed to define and facilitate access to various types of shared CRS resources and content representations. Moreover, with the emergence of the Internet of Everything (IoE) [35], more information can be stored, accessed, and collected intelligently. In fact, IoE connects ubiquitous devices to the Internet, by analysing the massive data generated by the connected end nodes using artificial intelligence. Hence, the spread of the use of SUAV-enabled IoE is a future research direction.
4.4. Mobile Fog and Edge Computing (MFEC)
Compared to the centralized cloud computing service, Mobile Fog and Edge Computing (MFEC) improved real-time service delivery and solved issues of long delays and excessive aggregated traffic. Therefore, future MFEC is aimed at pushing computing and data services to the edge/fog of the network, which is closer to the source of the request to alleviate its load and enhance the service quality. However, some applications such as monitoring critical industrial areas with cooperative autonomous SUAV require unprecedented high accuracy and ultralow latency. Hence, multiple issues and challenges need to be investigated to make mobile/intelligent fog/edge computing practical for SUAVs.
4.5. Mobile Crowd Sensing and Monitoring
Crowd surveillance and control (monitoring) are expected to play a key role in monitoring and tracking applications [36, 37]. In addition, new emerging mobile systems, particularly SUAVs, offer great flexibility by covering larger search/surveillance areas and making real-time dynamic data readily available. However, modelling the dynamics of crowds is becoming essential to predict their movement for control and decision-making purposes. Therefore, there is a significant challenge on how to guarantee the efficiency of the crowd through dynamic models while ensuring the autonomous and safe flight of SUAVs to achieve good service. For example, the work presented in [38] enables multi-UAVs to periodically monitor a group of moving targets simulating the movement of people walking in crowded environments. The law enforcement applications, in assisting authorities to monitor crowds to identify and track suspicious individuals with the intent to vandalize or link to attempted terrorist attacks, might execute this system outfitted with a group of UAVs, thus answering the question of how to continuously monitor a large group of individuals (targets) in a crowd, by alternately visiting each of them while trying to keep them in sight. Thus, the use of mobile crowd-sensing strategies in the design of complex and robust control systems with accurate observations from SUAVs is still in its infancy and remains a topic for future direction.
4.6. Safety, Security, and Location Privacy of SUAVs
As discussed in [39], the cyberattacks on the UAV systems have been reported and cybersecurity is still a significant challenge to be overcome for UAVs and SUAVs. Moreover, [1] discussed the risk exposed by the impersonation and Sybil attacks against UAVs, especially towards military application scenarios such as engagements on the battlefield. Developing timely strategies and countermeasures are needed to divert malicious cyber threats. Location privacy is an important issue that is directly related to localization. It has been shown that several obfuscation techniques can be used to hide the exact location of the users from nonintended parties. Example application scenarios are from the Cyber-Physical Systems (CPSs) and Internet of Things (IoT). Furthermore, authentication of the UAV/SUAV users (a.k.a. pilots) is also an important issue and constitutes an attack surface for the hackers that need to be considered seriously. This might include employing counteractions such as IPS (Intrusion Prevention System)/IDS (Intrusion Detection System) that are tailored for the SUAVs. Finally, alternative approaches can also be employed as discussed in [1] (identity and location validation scheme), which proposes a combination of a public-key-based authentication mechanism along with a movement plausibility check (position validation by using previously validated coordinates to classify and identify whether the UAV being investigated is legitimate or not) for groups of UAVs.
5. Conclusion
Today’s connected and automated world nonsurprisingly brings many technological advancements that are thriving people’s lives. SUAV is such an advancement and upper step of the flying things (drones, UAVs, toys, etc.) that offers a fully autonomous operation by distributing tasks and coordination among many UAVs.
As such, intense usage of SUAVs is expected in many application areas of modern life, such as in disaster relief, survey and rescue operations, and military operations (C4ISR, etc.). In fact, the proliferation of UAVs can be seen as a nightmare for individuals who dare for their privacy, especially when it is breached from the air. On the other hand, when these UAVs are ought to be controlled via sophisticated and automated manner to drive preassigned humanitarian tasks (e.g., wildfire tracking and avalanche/flood monitoring), such as in the case of SUAVs, then, they will become more acceptable and welcomed by the society.
These advanced tasks will be made possible by the usage of cooperative localization techniques, which not only ease the burden on each UAV by delegating and distributing the task over multiple peers but also decrease the overall battery consumption of the nodes. This is due to consideration of the per effort of each peer in these tasks when compared to individual efforts to handle those tasks. With the help of another technological trend, UAVs can be supported and equipped with energy harvesting tools such as fuselage-attached photovoltaic cells, in order to enhance the overall operation/fly time of the swarm.
Another aspect is that the advancements in the field of AI and ML will have a positive impact on the various problems and challenges that SUAVs are facing today, for instance, effective and efficient path planning\finding, networking (packet aggregation and dumping), and cyber security.
Speaking of cyber security, the cyber defence of the SUAVs against cyber threats also possesses importance, as the operation of the swarm is dependent on the legitimacy and authenticity of each participating UAV. Centrally located UAVs might act as a “single-point-of-failure” in the cyberattacks and as such should constitute an open target for the adversaries. Therefore, traditional cyber defence mechanisms should be tailored according to the needs of the SUAVs during the development and implementation phases.
The authors foresee that the usage of mobile cellular (4G, 5G, B5G, 6G, variants, and beyond) technology would relieve many limiting factors that prevent the utility of UAVs such as communication range and depth, networking challenges, and size/weight/power calculations. Additionally, machine-to-machine (M2M) communications (within the swarm) would take advantage of the robust and reliable infrastructure proposed by the cellular systems. The use of cellular networks would drastically enhance the efficiency and commercial utility of SUAVs especially in the presence of upcoming cellular networks with M2M communication capabilities.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
Publication of this paper is made possible by the authors’ efforts within the same journal in terms of reviewing and editorial services.
