Advancing Capabilities With UAVs: A Comprehensive Review of Sensor Fusion, Path Planning, Energy Management, and Deep Learning Algorithms
Abstract
The current engineering and construction of buildings has led to an increase in building height, which has created difficulties for search and rescue teams working with fire departments. These challenges are exacerbated by the limited reach of the fire ladder and water hose restrictions. Furthermore, traffic jams might make it difficult for the fire brigade to get to the site quickly, which could lead to the loss of life and property. To combat that issue, unmanned aerial vehicles (UAVs) can be employed for environment mapping in buildings during fire emergencies which will help detect fire and identify trapped humans. That is possible by equipping UAVs with path-planning algorithms, control strategies, and image-processing algorithms coupled with a robust sensor fusion algorithm. Well-designed UAVs can help overcome the limitations of traditional firefighting methods. This paper presents an elaborate review of the development of UAVs in terms of their data handling, control management, and energy storage. For the data handling and control management systems, the paper discusses effective sensor fusion algorithms, optimal path-planning strategies, control techniques, and fire detection algorithms. The use of hybrid energy storage for UAVs along with its energy management is elaborated. Through this review, researchers will acquire knowledge about the existing development in the application of UAVs in firefighting. Finally, the paper highlights advancements and limitations of various sensor fusion, path planning and energy management strategies to increase the flight time.
1. Introduction
Unmanned aerial vehicles (UAVs) have gained immense popularity because of the advancement of artificial intelligence (AI) [1]. It is an airborne robot that does not require a human pilot on board. The most common forms of UAVs are drones whose operation is controlled remotely by pilots situated at a ground station [2]. These UAVs can have different levels of automation ranging from total manual operation to complete autonomy with the help of AI. UAVs were first created in the twentieth century for military purposes that were too dangerous for humans. By the twenty-first century, however, they had become indispensable assets for many industries [3–8]. Control techniques gradually developed over the years. Drones became more affordable, making them cost-effective in a variety of applications such as surveillance of fires in forests, firefighting, aerial photography, package deliveries, and agriculture. Today, drones are deployed to inspect infrastructure, monitor defense activities, and assist in construction. Interestingly, their popularity has even led to its penetration in the entertainment industry, with drone racing entering mainstream media [9–13]. Scientific studies on drones, both technical and nontechnical, have been the focus of the last 10 years. Depending on how they were used, the public’s perception of drones varied, including support for their usage in battling fires. Other factors to consider were logic and economic viability. To ensure fire safety, technical aspects included fire suppressants, remote sensing, simulation studies, and the application of fire detection algorithms with unmanned aerial systems (UASs) [14]. The use of drones has been increasing in firefighting [15–20]. The sensor fusion algorithm methodologies for accurate localization are given in [21–25]. Human injuries are frequently the consequence of inadequate information regarding the fire front. Here’s where UAVs come into play, saving money, and taking over dangerous fire tracking duties [26–30]. The algorithms can be implemented on a drone along with multiple algorithms designed to accomplish the functions of take-off and landing autonomy, planning of trajectory, and fire tracking [31–34]. The UAVs equipped with infrared and RGB cameras, communication modules, and temperature sensors can provide complete details on the fire as well as the UAV position and velocity. The UAV then delivers the collected data to the ground station in real-time [35, 36].
Research assessing the employment of a path-planning algorithm by a group of UAVs to prevent collisions with obstacles discovered that the UAVs were able to reach their mission region successfully [37]. This demonstrates the use of path-planning algorithms in the deployment of autonomous drones, which can be used to provide virtual reality support to first responders and aid in the fight against spreading forest fires. The advancement of technology that allows autonomous drone operation is not only applicable to drones, but also to marine, space, and service robots, as well as unmanned vehicles [38]. Experimental research with the informed RRT ∗ algorithm revealed that it outperformed the classic RRT ∗ algorithm under diverse scenarios [39]. The informed RRT ∗ algorithm shows benefits such as less computation time, the capacity to generate paths through difficult situations and less dependency on the planning problem’s state dimension and spectrum. Furthermore, research has been conducted on the use of light detection and ranging (LiDAR)-mounted UAVs for remote sensing, which is important in forest management [40].
Precision agriculture (PA) has progressed because of the advent of UAV–based remote sensing systems [41]. When compared to older methods, using UAVs to monitor crops provides a gateway to gather data from agricultural fields in a simple, quick, and cost-effective manner. PA using IoT technology based on UAVs is the future of remote sensing [42]. Another aspect of UAVs is their capability to have low-altitude flights which are advantageous for taking very-detailed photographs of crops. The efficiency of the monitoring system and its ability to collect usable and accurate data has been greatly improved [43–47]. Several sectors and businesses, such as agriculture, construction, mining, and infrastructure inspection, benefit from drone mapping. The maps are very useful in catastrophe management through monitoring, charting, and assessing postdisaster damage [48]. The use of drones for search and rescue missions has increased recently, particularly for missions in inhospitable areas. A case study highlighted the deployment of drones to locate a person in a slot canyon that was rugged and hard to access by foot [49]. The study mentioned that the drones sent the search and rescue team images of difficult-to-access places. That shows that drones add incredible advantages to human efforts. Drones substantially speed up rescue operations while keeping the rescuers out of significant danger. UAVs detected an at-risk human across a larger area in less time than first responders and rescuers without assistance, according to a prospective randomized simulation investigation of the effectiveness of drone technology [50]. Drones using UAV technology in combination with the global navigation satellite system (GNSS), deep learning, and real-time computer vision could offer error-free data to rescuers and personnel in the ground station in a short duration of time [48].
The paper provides a comprehensive review of the advancements in UAVs, focusing on their data handling, control management systems, and hybrid energy storage for self-powering. The data handling and control management systems encompass an efficient sensor fusion algorithm, an optimized path planning strategy, and various control and fire detection algorithms for detecting and extinguishing a fire. The concept of utilizing hybrid energy storage for UAVs along with its energy management is also elaborated. The paper highlights the numerous possible solutions to make UAVs smart and reliable with intelligent algorithms developed to date. The rest of this paper is structured as follows: Section 2 explains the qualitative comparison of sensor fusion algorithms of the firefighting UAV and the advantages and limitations of different path planning algorithms. Section 3 compares various control algorithms and discusses the pros and cons of image processing deep learning algorithms. Section 4 evaluates energy storage to power the UAVs along with the various hybrid energy storage combinations. Sections 5 and 6 summarize the insights after reviewing the literature and conclude with major outcomes and areas for future research.
2. Firefighting UAV Architecture, Sensor Fusion, and Path Planning
2.1. General Architecture of Firefighting UAVs
The general architecture of a smart UAV is illustrated in Figure 1, which shows the on-board hybrid energy storage, the on-board computer that runs the various algorithms, flight controller, global positioning system (GPS), a variety of sensors (thermal imaging camera (TIC), LiDAR, temperature, inertial measurement unit (IMU), and ultrasonic) for fire detection and tracking and the motors. The data collected by the TIC, depth camera, temperature sensor, IMU sensor, and ultrasonic distance sensor is processed in the onboard computer. It houses the SLAM, path-planning, and proprietary fusion algorithms, which calculate and transfer the results to the flight controller. Additionally, data from the GPS sensor is sent directly to the flight controller, which processes data and gives signals to the drone about motor speed. Stabilization and tracking algorithms are used to keep drones on their predicted pathways. With the limited power supplied by the energy source on board, power distribution coupled with energy management is utilized to predict flying time for the given battery state of charge.
Four pillars make up the brain of a firefighting UAV—sensor fusion, path planning, flight control, and fire detection.
2.2. UAV Sensor Fusion
For the fire-tracking and path-finding algorithms to work, it is necessary to have sensor fusion algorithms [51]. Figure 2 illustrates the types of sensor fusion algorithms. The algorithms fuse data from the sensors and other sources to provide crucial information to the processor [52, 53]. If data from those sources were processed separately instead of being fused, the communication of information would not be as effective, increasing computational time and reducing accuracy. In the UAV’s sensor fusion, GPS, IMU, the visual camera, LiDAR, and radar sensors play very crucial roles. For example, the visual camera can provide information about the kind of object it sees. The radar and LiDAR sensors will provide the distance of the object to the drone. With sensor fusion, their information is combined, allowing the computer to make the appropriate calculation for the speed and angle of the rotor blades [54, 55]. Apart from that, other types of data are fused to give meaningful input to the controller.
The Brovey transform (BT) is a simple technique that improves the visual contrast at both ends of the histogram of satellite data. Each moderate-resolution band is multiplied by the ratio of the higher-resolution panchromatic band divided by the sum of all three multispectral bands in this method [56–58]. Filtering algorithms find their use in applications that require some degree of segmentation from the preferred and nonpreferred types of data. This separation can be done by setting a certain threshold. This can be applied to various parameters such as the voltage output of the sensor, temperature reading from analog sensors, and sound frequency. There are different types of filtering algorithms such as high pass filter (HPF), low pass filter (LPF), Kalman filter, and extended Kalman filter [59], developed a 2D median filtering algorithm for image processing and it has the limitation of in preserving edges in highly textured areas [60], introduced a constraint satisfaction problem, which can handle difference constraints. The collaborative filtering algorithm with an evaluation metric that closely models user experience, enhancing recommendation accuracy was proposed in [61], adaptive filtering algorithm, and a model-based filtering algorithm was presented for to provide better performance in noisy environments and gives better accuracy [62, 63]. A filtering algorithm was introduced for enhancing the detection of variable stars in [64], multidirectional ground filtering for LiDAR to improve the mapping accuracy was proposed in [65]. A median filtering algorithm of better noise reduction [66] and UAVs safety authentication, behavior modeling, and safety supervision was developed in [67]. The practice of integrating two or more photos into a single image while preserving certain important features from each is known as wavelet image fusion for effective multiresolution analysis [68–70], which is a widely used technology in a variety of domains, including remote sensing, robotics and medical applications [71, 72], introduced image fusion technique using neutrosophic-based wavelet transform, improving image clarity and focus. Principal component analysis (PCA) is an unsupervised learning method used for reducing complexity in machine learning. Correlated attributes are converted into a set of linearly uncorrelated properties through the statistical method of orthogonal transformation [73, 74]. By integrating standard PCA with rotated wavelet transform to fuse multispectral images to improve the image resolution is developed in [75–77].
The values of parameters between data points are predicted using geostatistical analysis techniques. Geostatistical methods are only applicable to data that is spatially dependent, that is, not random [78–80]. The fusion technique is one of the most widely used procedures for sharpening the HIS approach. Color enhancement, feature enhancement, spatial resolution improvement, and the merging of different data sets have become common procedures in image analysis [81–83]. A hybrid algorithm mixes two or more different algorithms to solve the same problem and is commonly used in programming languages such as C++, picking one (depending on the input) or switching between them throughout the procedure [84]. The elaborative qualitative comparison of the above types of sensor fusion algorithms is illustrated in Table 1.
| Algorithm | Algorithm types | Advantages | Disadvantages |
|---|---|---|---|
| Brovey transform [56–58] | — |
|
|
| Filtering algorithm [59–67] |
|
|
|
| Wavelet-based fusion [68–72] | Multivariate wavelet denoising (MWD) |
|
|
| Statistical fusion [73, 74] |
|
|
|
| PCA [75–77] |
|
|
|
| Geostatistical based fusion [78–80] | Kringing BME |
|
|
| HIS [81–83] |
|
|
|
| Hybrid [84] | MQQA-BME |
|
|
The fusion algorithms are compared based on statistical parameters. Prediction accuracy measures such as correlation (R), standard deviation (STD), mean absolute difference (MAD), and root mean square error (RMSE) are among them. These terms can be used to evaluate the overall accuracy of the anticipated data by comparing the predicted results with real outcomes at each specified pixel at specific times. In addition to the R, STD, MAD, and RMSE, the error relative global dim synthesis (ERGAS) and the spectral angle mapper (SAM) are considered. The ERGAS can be used to determine the spectral similarity of merged data to actual values, whereas the SAM may be utilized to extract the merged or fused image’s amount of distortion [85].
Based on the comparison, the accuracy of the fusion algorithms was evaluated and the algorithms were sorted on the bases of the RMSE accuracy. The lower the RMSE score, the better the fused data produced, that is, the accuracy of the fused data is inversely proportional to the value of the RMSE value.
Figure 3 illustrates the application of sensor fusion algorithms for each sensor and how it is interlinked in a firefighting UAV (drone).
2.3. Path Planning
The process of constructing a path from a starting point to a destination is known as path planning. It allows an autonomous robot or vehicle to discover the best path to its goal [86–88]. The map needed for this process can be in the form of a grid map, state space, or topological roadmap. Grid-based search and sampling-based search algorithms are the two main types of algorithms that can obtain an optimum planned path [89].
Grid-based search algorithms seek and produce a path in a grid map based on the lowest travel cost which is the amount of energy required to travel from point A to point B. In a 2D environment, they can be used for path planning for mobile robots. However, the amount of memory required to perform grid-based algorithms rises as the dimension number increases, such as in the case of a six-degree-of-freedom robot [90–92]. With the aid of randomly selecting new nodes or robot configurations from a state space, sampling-based search algorithms develop a tree structure to produce a searchable tree. They can be used in low- and high-dimensional search spaces [93–95].
- •
be capable of always finding an optimal or near-optimal path in real-time static environments;
- •
be able to adapt and be implemented in dynamic environments;
- •
remain compatible with and enhance the chosen self-referencing approach; and
- •
minimize computation time, complexity, and data storage.
The survey suggests that rapidly exploring random tree (RRT), RTT ∗, and informed RTT ∗ are the most effective. In RRT, points are produced at random and linked to the nearest accessible node. When a vertex is created, it must be checked to ensure that it is not in the path of an obstacle. In addition, the vertex must be chained to its nearest neighbor while avoiding impediments. The method terminates when a node inside the desired region is created or a limit is reached. A design decision is made while choosing a random position [96]. If there is an obstruction between the random node and the nearest neighbor node, it will not link to the random node. Otherwise, if the random node is closer to the intended target, it will connect to the random node. In unexplored state spaces, the likelihood of random node selection rises. Simple methods, including using internal random number generators, can be used. We can determine the path from the starting node to the end objective using this fast-exploring random trees approach. This method, however, will not produce an ideal route. Such techniques are adequate only for simple applications [97].
RRT ∗ is a streamlined type of RRT [98]. The RRT ∗ method identifies the shortest path to the destination when the number of nodes approaches infinity. RRT ∗ has the same fundamental premise as RRT, but two major changes to the method result in drastically different outcomes. RRT ∗ begins by measuring the separation between each vertex and its parent vertex. That is referred to as the vertex’s cost function. A neighborhood of vertices within a certain radius of the new node is then investigated after the nearest node in the graph has been identified. If a node is found that is less expensive than the proximal node, the proximal node is replaced with the cheaper node. That feature’s effect is illustrated by the addition of fan-shaped twigs to the tree structure. The second distinction made by RRT ∗ is the rewiring of the tree. The neighbors are reverified when a vertex is connected to the least expensive neighbor. RRT ∗ then examines whether rewiring to the newly added vertex will reduce the cost of neighboring nodes. The neighbor is connected to the newly added vertex if the price drops. This feature smoothens the route.
Informed RRT ∗ is an enhancement to the RRT ∗ technique that accelerates the convergence of the discovered solution to the optimum [99]. That approach, developed expressly for path-length issues, directly samples a subset of a planning problem that contains all feasible improvements to a given solution. To discover a better solution, one should confine the search to the entire state space, which can increase the pace of convergence and locate the best path in a finite time. Informed RRT ∗ samples the subset directly (inside the ellipse) and concentrates on the path between start and target. As a result, with each sample, the answer will improve. Even in a huge state–space, it is computationally cheap. Table 2 illustrates the advantages and limitations of informed RRT ∗ compared with RRT ∗ and RRT in terms of their path-finding factors. It was found that for a given number of nodes, informed RRT ∗ was 88% faster than RRT ∗ [99].
| Advantages of informed RRT ∗ | |||
|---|---|---|---|
| Factors | Informed RRT ∗ | RRT ∗ | RRT |
| Convergence rate [96, 97] | Excellent convergence rate with the improved and feasible initial solution | Better convergence rate than RRT but still suffers from a costly initial solution | Satisfactory, but often takes time to reach convergence. Plus requires a costly initial solution |
| Computational cost [96, 98] | Consumes the least amount of computational resource as it samples the subset directly unlike RRT and RRT ∗ | Consumes a lesser amount of computational resources compared to RRT as path finding is more confined toward the finish | Consumes a lot of computational resources as it calculates all the paths from the starting nodes regardless of their direction toward the finish |
| Ability to find difficult paths [97, 98] | It can find the feasible path in the least amount of time as it samples the subset directly | As it samples paths based on the nearest nodes to the start, the path finding is quicker than RRT | As it does its sampling randomly, it takes a long time to determine a feasible path |
| Optimization speed [97, 98] | Optimization speed is the fastest provided it calculates an optimum initial solution | Optimization is faster than RRT but slower than informed RRT ∗ | Slowest compared to RRT ∗ and informed RRT ∗ as it finds paths randomly |
| Limitations of informed RRT ∗ | |||
| Factors | Informed RRT ∗ | RRT ∗ | RRT |
| Dependency on the initial solution [99] | Because the search is intrinsically relying on the present solution, it cannot focus when the related prolate hyper-spheroid is bigger than the planning problem | It is dependent on the initial solution thus if it deviates, the whole solution could bring on increased lapse time | It is not dependent on the initial solution as it goes about calculating paths randomly |
3. UAV Control and Deep Learning Algorithms
3.1. UAV Control Algorithms
A control algorithm is a strategy that receives inputs from various sensors and feedback (if the system is closed loop) to calculate an optimal control signal and send it to the actuator. It is important to employ the right type of control algorithm for the UAVs. There are various types of control algorithms, some of the popular ones are illustrated in Figure 4.
A proportional integral derivative (PID) controller comprises proportional, integral, and derivative terms that work in harmony to calculate a control signal. The controller calculates an error value e (t) in real-time as the difference between a desired setpoint (SP) and a measured process variable (PV), then, corrects the error using proportional, integral, and derivative terms. The proportional term amplifies or reduces the control value by multiplying a gain to the error value. The gain can either be an integer or a rational number. With every iteration, the error will get closer to zero as the measured value will reach closer to the desired value. However, with only the proportional term, the controller will never get the system to the appropriate set point. Thus, an integral term is added to keep the memory of the previous iteration to add it to the control value. The PI controller may overshoot the desired set point for certain conditions of the system. To monitor the rate of change, a derivative term is added that reduces the control value after a certain threshold of the rate of change. PID can be further classified into normal PID and intelligent PID [104, 105].
Model predictive controller (MPC) is a feedback control algorithm that uses a mathematical model to predict future outputs of the process. It can accept multiple inputs and produce multiple outputs (MIMO) to accommodate interactions among different control loops. This prevents a complex design for a multi-PID controller. The principle of MPC is to predict the output for a set of time steps. To predict, the controller receives an input (e.g., the position of the drone) as a reference value. Based on this reference and the various input and output constraints, it calculates various output predictions. The prediction that provides the least cost function, that is, the error value associated with the shortest path between the set point and the PV, would be chosen as the final output. After having the prediction output chosen, the set of inputs that were used to acquire the output is supplied in real-time [106, 107]. MPC can be further classified into deterministic MPC, adaptive MPC, telematics MPC, and stochastic MPC.
Sliding mode control (SMC) is a nonlinear control strategy that changes the dynamics of a nonlinear system by providing a discontinuous control signal. It is a set-valued control signal that compels the system to “slide” along a cross section of its usual behavior. The feedback control law is not a time-reliant function. Instead, depending on where it is in the state space, it can move from one continuous structure to another. As a result, SMC is a way of variable structure control. As a legion of control structures is built in such a manner that trajectories always advance toward an adjacent region with a different control structure, the end trajectory will not reside wholly inside one control structure. Rather, it will glide along the control structures’ limits. The system’s movement as it slides along those limits is referred to as a sliding mode and the boundaries formed by the geometrical locus are referred to as the sliding surface. SMC can be classified into integral SMC, terminal SMC, fractional order SMC, higher order SMC, SMC with feed-forward disturbance compensation, boundary layer SMC, SMC with reaching law modification, and SMC with AI [108, 109].
Fuzzy logic (FL) control is a popular type of control strategy. The term “fuzzy” alludes to how the logic involved may cope with things that cannot be represented as “true” or “false,” but rather as “partially true” or “partially false.” With traditional controls, the inputs and outputs are binary, that is, they are communicated in 0 and 1. FL, however, considers decimal values between 0 and 1, allowing judgments to be made based on estimations of how partially true or partially false an event is. FL has the benefit of allowing the solution to the issue to be expressed in words that humans can understand, allowing for the use of their expertise in the controller’s design. That simplifies the automation of tasks humans do well. The tasks consist of input, processing, and output stages. The input stage transforms inputs from sensors or other sources, including switches and thumbwheels, into the appropriate membership functions and truth values. Each applicable rule is put through its paces in the processing stage, which produces a result for each one before combining them. The combined result is then reconverted into the desired control output value in the output stage. FL control can be classified into adaptive FL and predictive FL [100, 101, 110]. Table 3 tabulates the pros and cons of each algorithm.
| Control algorithm | Advantages | Disadvantages |
|---|---|---|
| PID [104, 105] |
|
|
| Intelligent PID [104, 105] |
|
|
| MPC [106, 107] |
|
|
| Adaptive MPC [106, 107] |
|
|
| Stochastic MPC (SMPC) [106, 107] |
|
|
| SMC [108, 109] |
|
|
| Integral SMC [108, 109] |
|
|
| Fractional order SMC [108, 109] |
|
|
| Terminal SMC [108, 109] |
|
|
| Higher order SMC [108, 109] |
|
|
| Boundary layer SMC [108, 109] |
|
|
| SMC with reaching law modification [108, 109] |
|
|
| SMC with feed-forward disturbance compensation [108, 109] |
|
|
| SMC with AI [108, 109] |
|
|
| FLC [100, 110] |
|
|
| Adaptive FLC [100, 101] |
|
|
3.2. Deep Learning Algorithms
To effectively determine the location of a fire, detection devices have been combined with AI, which is beneficial for UAV pilots. A system was designed with one IR camera that would work with the Raspberry Pi platform and detect images using various image detection algorithms. Some of the algorithms are discussed in detail below.
The Harris Corner and Canny detector can be defined as a metric that defines corner quality N as the difference between a matrix’s determinant and trace that is expressed in terms of an image’s gradients. If N is greater than zero, there is a corner present. A canny detector works on a similar principle but instead of finding corners, it finds edges [102, 103]. Scale-invariant feature transform (SIFT) uses the difference of Gaussians (DoGs), which is determined by first convolving a picture with a Gaussian kernel. Using a series of reference pictures, the SIFT key points of objects are extracted and kept in a database. An object is detected by comparing each feature in the new image to this database and discovering matching features [111–113].
SIFT, when coupled with the speeded up robust feature (SURF) algorithm, becomes closely related descriptors. SIFT and SURF are founded on the same concepts, but each stage of each algorithm has its own set of specifics. Square-shaped filters are employed for SURF, which is several times faster than SIFT, as well as being more durable [114–116]. Oriented features from accelerated segment test (FAST) and rotated binary robust independent elementary features (BRIEF; ORB) creates a binary string for image description using two algorithms—FAST and BRIEF. ORB compares pixels to differentiate between an object and its surroundings. FAST declares a corner if it finds a pixel with a higher or lower position than its center pixel. The BRIEF algorithm uses that information as well, to perform a series of threshold tests on the intensities of pixels in a given patch image [117]. The binary robust invariant scalable key (BRISK) points algorithm uses an optimized variant of FAST called adaptive and generic accelerated segment test (AGAST). To ensure invariance against scale variation, BRISK uses a pyramid-based technique [118]. Histograms of oriented gradient (HOG) is a gradient-based descriptor made up of local histograms of picture grid gradient orientations. It divides the image into many grids and then checks local gradient directions. The HOG descriptor is created by normalizing the collected histograms from all grids [119–121]. And last, a convolutional neural network (CNN) is a composition of convolutional layers, pooling, and rectified linear unit (ReLU) which are fully connected layers between an input and output layer [122, 123].
A qualitative comparison of the advantages and disadvantages of the types of object detection algorithms appears in Table 4.
| Algorithm | Advantages | Disadvantages |
|---|---|---|
| Harris Corner [102, 103] |
|
|
| SIFT and SURF [111–116] |
|
|
| ORB [117] |
|
|
| BRISK [118] |
|
|
| HOG [119–121] |
|
|
| CNN [122, 123] |
|
|
4. Energy Storage Solutions for UAVs
The preceding sections primarily discussed the data handling and control management systems of a UAV. Along with understanding the methods to control a UAV, it is equally important to address the source that provides the necessary amount of energy to the UAV. Energy storage coupled with the other systems in the UAV determines the device’s overall weight and distribution. Moreover, finding the right type of energy storage is necessary as it dictates flight endurance. Factors to keep in mind while selecting energy storage for the UAVs are: (a) provide a long duration of charge, (b) have a good energy to weight ratio, (c) be environmentally friendly, (d) be readily available, and (e) produce little to no noise and vibration.
4.1. Single-Source Energy Storage
Some popular single-source energy storage solutions are explained below.
Batteries—Batteries are a very popular form of energy storage. Their applications can be seen in various consumer electronics, electric vehicles, lawnmowers, et cetera. They work on the principle of reduction oxidation (REDOX) reaction where chemical energy is converted into electric energy. Using batteries to store energy improves a propulsion system’s functionality and capabilities. Battery-powered platforms can meet a variety of hobbyist needs in terms of flight duration and cost-effectiveness. Many small UAVs, particularly quadcopters, are powered by batteries. Typical tiny UAVs powered by batteries have restricted endurance because of battery pack weight constraints. Using lithium polymer (LiPo) batteries, they can have a flight time of up to 90 min [124].
Hydrogen fuel cells (HFCs)—HFCs generate electricity by combining hydrogen (H2) with oxygen (O2) through a spontaneous chemical reaction where water is eliminated as a byproduct. Constant supplies of hydrogen are needed to run the fuel cell. HFCs have several advantages. They do not emit direct pollution, and they do not make noise. They are fueled by an energy-dense fuel that is the most prevalent molecule on the planet. HFCs are more energy efficient than other UAV power sources. They can power the UAVs for hours in the air. HFCs perform well in cold temperatures, unlike batteries whose chemical activity weakens at low temperatures. The refueling time of a hydrogen-fueled UAV is quick (can be refueled in under a minute). While HFCs may look promising, it is important to address their disadvantages as well. The cost of using HFCs is quite high. In addition, the generation, storage, and transportation of hydrogen is complicated and expensive [125]. Another problem with HFCs is that they generate a lot of heat. Given the widespread usage of plastic in UAVs such as drones, the generation of heat could melt some of the drone’s parts.
Super capacitors (SCs)—SCs have a high specific power ranging from 10 W/g which is much higher than a lithium-ion battery 1–3 W/g, thus, allowing for fast discharge and charge [126]. This is especially useful when a rapid recharge is necessary to recover boost capability from the secondary power source. In addition, supercapacitors have the advantage of providing power over the entire voltage range from 0 V to the maximum nominal voltage, unlike batteries that must operate within a specific range. Unlike an electrochemical battery, which has a fixed cycle life, cycling a supercapacitor causes little wear and tear which makes it age better. Under typical conditions, a supercapacitor’s capacity degrades from 100% to 80% after 10 years. Another advantage is that the supercapacitor is forgiving in both hot and cold climates, which batteries cannot match. Though they are advantageous in some respects, supercapacitors have their cons. First, a supercapacitor has low specific energy which means the flight time of the UAV is significantly low. Second, a full energy spectrum cannot be achieved because of its linear discharge. A supercapacitor also suffers from high self-discharge. And last, it is expensive to manufacture, thus, will raise the cost of the UAV [127].
4.2. Hybrid Energy Storage
Because of its dynamic load response capabilities and energy efficiency, a fuel cell and battery hybrid energy solution (Figure 5) for UAVs has lately gained interest. Backed by multiple studies, a fuel cell and battery hybrid provide an average energy efficiency of 45%, which is extremely effective for operating UAVs such as drones [128–130]. A disadvantage, however, concerns safety-related issues with storing hydrogen in UAVs used for firefighting applications as hydrogen is very flammable and can catch fire easily in the hot environment it will be subjected to. In addition, the UAVs will have to be equipped with multiple accessories to keep hydrogen in its optimal state, which will add weight to the UAV and downgrade its maneuverability and flight time along with increasing its price.
Another disadvantage fuel cells have is that they have low instantaneous current supply capacity. Coupling a supercapacitor with the fuel cell (Figure 6) can overcome this disadvantage. A couple of tests and analyses were performed on fuel cells and supercapacitor hybrid, which report that the performance of the UAVs significantly improves. By not depleting the fuel cell, its longevity can be extended [131, 132].
Batteries can store huge amounts of energy, but they require a significant amount of time to charge. Whereas capacitors recharge quickly but only store a little energy. Combining the properties of a lithium battery with the fast charging of a capacitor, supercapacitor hybrid UAVs greatly boost the efficiency levels and power of Li batteries. They guarantee to save a huge amount of weight while doubling the distance of flight and flight time of the UAVs [133] (Figure 7).
To make the energy delivery operation more efficient and reliable, the battery, fuel cell, and supercapacitor can be coupled in the order of two or three. The HFC excels at low power and long endurance, whereas the supercapacitor excels at brief bursts of high power and rapid recharging. Because the supercapacitor has limited energy storage, a battery is necessary to meet high power demand. The battery supplies the additional power essential for flight phases like takeoff and ascent and it is sized to last for the desired length of boost power [134]. This combination is reported to be extremely efficient and reliable [135] (Figure 8).
A solar-powered UAV requires it to be equipped with a vast span of solar cells along its surface. Such UAVs can fly without issue during sunny conditions. However, at night or on a cloudy day, a solar-powered UAV becomes inoperable. A solar battery hybrid drone is equipped with a battery backup along with solar panels. Thus, during sunny hours, the solar panels would harness energy to power the drone and charge the battery. This stored energy can then be used during nonsunny hours [136] (Figure 9).
The attributes of different available energy sources are discussed in Table 5.
| Energy storage | Advantages | Disadvantages |
|---|---|---|
| Battery-powered UAVs [124] |
|
|
| Fuel cell-powered UAVs [125] |
|
|
| Supercapacitor as an auxiliary power source [126, 127] |
|
|
| Fuel cell and battery [128–130] |
|
|
| Fuel cell and supercapacitor [131, 132] |
|
|
| Battery and supercapacitor [133] |
|
|
| Battery, fuel cell, and supercapacitor [134, 135] |
|
|
| Solar cell and battery [136] |
|
|
4.3. Power Management Strategies for UAVs
Hybridization is the ideal design for powering a UAV propulsion system. It combines the benefits and capabilities of several power sources while balancing their drawbacks. Power must be distributed as required from a variety of sources to produce an optimal result of energy efficiency and good performance while increasing lifespan as much as possible. As a result, a power management strategy (PMS) must be implemented. The essentials to consider while implementing PMS in real-time are good efficiency, quick response time, and fuel usage along with power requirements and flight circumstances.
Table 6 presents a short comparison between two PMSs—rule-based strategies and FL strategies.
| PMSs | Advantages | Limitations |
|---|---|---|
| Rule-based strategies [137, 138] |
|
|
| Fuzzy logic strategies [139, 140] |
|
|
5. Applications of Deep Learning Algorithms in UAVs
Future directions of promising research in UAVs hybrid energy management systems and deep learning applications: AI driven reinforcement leaning algorithms can be used to establish energy management strategies to improve flight time for different applications like agriculture, disaster management, surveillance by optimizing the required energy sources like batteries, fuel cells, and solar panels, and SCs. The deep learning techniques like long short-term memory (LSTM) and other ML models can be used for state-of-charge (SOC) and state-of-health (SOH) estimation and prediction of system failures, UAVs endurance. CNN algorithms can be used for network optimization in swarm UAVs applications like logistics and surveillance and disaster management. The recent technology in advanced deep learning algorithms in communication of swarm robotics and key findings are given in Table 7. Many industries are focusing on the development of digital twin technology to improve the prediction accuracy of deep learning algorithms about the system faults, lifetime by analyzing the sensors data in simulations.
| Research focus | Methodology | Key findings and contributions | Year of publication | Applications |
|---|---|---|---|---|
| Optimization using multi access edge computing (MEC) [141] | Convex and nonconvex optimization algorithms | The iterative algorithm improves 5G network experience by 19%–34% compared to benchmark | 2024 | 5G communication, and latency sensitive applications |
| Wireless coverage optimization with constraints with multiple UAVs [142] | Multi and improved multiobjective gray wolf optimizer for wireless coverage optimization | Optimized energy and balanced coverage with energy optimization is developed | 2024 | Disaster management applications and wireless communication multi-UAV based applications |
| Multi access edge computing in IOT networks [143] | Deep reinforcement learning and Lyapunov optimization in IOT networks | An online learning-based algorithm to reduce service delay | 2023 | Future transportation, smart cities, and IOT networks |
| Assisted data collection using multi-UAVs for optimization [144] | Three step ant colony optimization algorithm is used based on density | Proposed a strategy to improve the network performance by data accuracy | 2023 | Real time monitoring with UAV, sensor, and IOT networks |
| Reduce mission time and cost using multi-UAVs for real-time data collection [145] | Heuristic approximations and successive convex optimizations algorithms | Optimized UAV trajectory for optimal performance of the allocated missions | 2023 | UAV based surveillance and monitoring applications and IOT networks |
6. Discussion
For an effective operation and functioning of a UAV in fire disaster management, a suitable sensor fusion algorithm and path planning algorithm are necessary along with control strategies and a precise fire detection algorithm. Those can be considered the four prongs of the control management of the UAV. Numerous types of each prong are compared based on the merits and demerits they offer in the tables. A UAV requires an efficient energy storage and management system of which there are multifarious types. This section summarizes the various aspects of each type of sensor fusion algorithms, path planning algorithms, control strategies, fire detection algorithms, and hybrid energy storage solutions.
6.1. Sensor Fusion Algorithms
Optimal data fusion and its computational time are the primary aspects one must study when analyzing sensor fusion algorithms. BT offers advantages such as better spatial resolution and simple development. Though it suffers from distortion of color and cannot be used on a broad scale. It also reduces the amount of spectral data while combining RGB images. Unlike BT, filtering algorithms such as HPF, LPF, and Kalman decrease spectral asymmetry. They eliminate the accelerometer spike and frequency smoothing during the fusion of sensor data. Those algorithms can be relied upon for good calculation because of their iterative nature. But the iterative method produces a huge amount of useless data. This may lead to the formation of blurred images as output. Wavelet-based fusion can be considered for the fusion of images if the spectrum must be changed before the combining process. One major advantage it holds is that it retains the initial color and statistical parameters. However, it suffers from insufficient spatial resolution and is limited to only the linear environment. If the application demands a high spatial resolution fusion, one must use PCA. It can be used in a nonlinear environment, unlike wavelet-based fusion. It, however, suffers from spectral distortion and low accuracy if the number of sensors is low. Another type of fusion is statistical-based. Spatial and temporal adaptive reflectance fusion model (STARFM), spatialtemporal adaptive algorithm for mapping reflectance change (STAARCH), and spatiotemporal data fusion method (STDFM) are adaptable to nature. The issue with those algorithms, however, is that they avoid BRDF, which compromises accuracy in heterogeneous scenarios. If robustness and simplicity are demanded, geostatistical-based fusion such as Kringing and BME must be used. But they are limited to hard data. BME can be combined with MQQA to form a hybrid that will provide an integrated fusion algorithm. However, that requires in-depth knowledge about using hybrid or it provides erroneous results. HIS algorithm is proven to provide satisfying spatial augmentation, but at the cost of spectral distortion.
6.2. Path Planning Algorithms
The path planning algorithm must be fast, efficient, and accurate to provide the shortest path in the least amount of time without being excessively computationally expensive. The three important path-planning algorithms are RRT, RRT ∗, and informed RRT ∗. Factors used to compare the three are convergence rate, computational cost, ability to locate difficult paths, and optimization speed. In terms of convergence rate, RRT is slow compared to RRT ∗ and informed RRT ∗. Along with that, it requires a high-cost initial solution. RRT ∗ offers a better convergence rate compared to RRT, but suffers from a costly initial solution, an issue found in RRT. However, that is solved in informed RRT ∗, which acquires a feasible solution and enjoys an excellent convergence rate. Regarding computational cost, the RRT algorithm consumes a huge amount of computational resources as it calculates all the paths from the starting nodes regardless of their direction toward the finish point. Unlike RRT, RRT ∗’s path-finding method is more confined toward the finish point because it needs to evaluate less data. Informed RRT ∗, however, consumes the least amount of computational resources as it samples the subset directly, unlike RRT and RRT ∗. RRT takes a long time to determine a feasible path as it does its sampling randomly. Because of that, its optimization speed is slow. RRT ∗ samples paths based on the nearest nodes to the start point, making the pathfinding and time to optimization quicker than RRT. Informed RRT ∗ samples the subset directly and finds the feasible path in the least amount of time with the fastest optimization speed. However, informed RRT ∗’s search intrinsically relies on the present solution, hence, it cannot focus when the related prolate hyper spheroid is bigger than the planning problem. Figure 10 shows the comparison of cost and run time of different path planning algorithms, and in Table 8, the comparison of these path planning algorithms is given.
6.3. Control Algorithms
Algorithms that control UAVs should be fast, accurate, and reliable along with being robust to uncertainties in the system and environment. PID controllers are advantageous if the application cannot handle a high computational effort. It is a simple structured strategy codified based on the difference between a desired SP and a measured PV. Prior knowledge of the plant is not required and it does not depend on complex mathematical models. However, setting up the PID controller is time-consuming as the tuning parameter can only be determined by trial and error. It does not hold any robustness against uncertainties and external disturbances. It does not support the MIMO system. To combat some of the issues with PID, intelligent PID is developed which has better robustness for control systems. But its adaptability and parameter handling capacity are low. Unlike PID, MPC controllers can support MIMO systems along with having good parameter handling capacity. They provide zero state error and decrease peak overshoot. However, operating an MPC demands intensive computational resources along with being tedious to integrate as it requires a nonlinear optimization problem to be solved online. Also, its ability to handle uncertainties is poor. Compared to the offset-free MPC’s static disturbance correction components, adaptive MPC may be able to handle time-varying disturbances and address plant-model mismatch across larger prediction horizons. Another type of control algorithm is SMC which is high-speed and robust. It offers a simple architecture with immense accuracy as it does not depend on a complex mathematical model and is easy to implement. It is suitable for plants that work in the presence of parametric uncertainties and MIMO systems. Although SMC is robust, it is vulnerable to plant parametric instability and sliding mode disturbances. It also suffers from the chattering phenomenon along with having zero constraint handling capacity. There are several variations of SMCs developed such as integral SMC, terminal SMC, fractional order SMC, higher order SMC, SMC with feed-forward disturbance compensation, boundary layer SMC, SMC with reaching law modification, and SMC with AI. Additionally, a FL controller (FLC) can improve transient and steady-state performance over a PI controller. It is robust toward speed control applications along with being compatible with MIMO systems. But to use FLC, prior knowledge of the plant is necessary. Like PID, it does not have good constraint-handling capacity. It requires a huge amount of computational power.
6.4. Fire Detection Image Processing Algorithms
Fire detection algorithms are expected to detect the location of fire effectively. The Harris Corner algorithm is known for not being influenced by disturbances during rough flights because of its simple and independent rotation and motion. The algorithm is limited to a single scale and the results of detection overly depend on the threshold. For an under-the-scale scenario and rotation changes, SIFT, SURF, and BRISK algorithms are proven to be effective. However, their performance is not up to the mark when subjected to an illumination change. BRISK has many outliers as it uses a binary feature and matching approach. The ORB algorithm offers advantages such as high computational efficiency and stability. It is, however, ineffective in detecting scale invariance in real-time feature points. Another algorithm that is popular amongst developers is CNN. It holds the ability to recognize relevant features without requiring human intervention. It also learns unique characteristics for each class on its own. But to implement CNN, a huge amount of training data is required. Additionally, it cannot position and orient an object. The HOG algorithm is subjected to low susceptibility to external influences as environmental changes. However, the gears used to operate HOG are expensive and disturbances on the surface erode the method’s statistical dependability.
6.5. Energy Storage Systems and Management
Hybrid energy storage systems can increase the flight time of a UAV. There are several types of hybrid energy systems. Solar cell and battery is a much-discussed combination that can be used. Photovoltaic cells (PVCs) can help reduce the weight of the UAV and achieve up to 59% of fuel savings. However, the UAVs should have a large wingspan to maximize the amount of light energy captured which will increase weight. Coupling fuel cells with batteries can provide excellent dynamic load response. The combination is energy efficient and offers a longer flight time. Because the battery is coupled with fuel cells, the amount of hydrogen required is reduced which improves space efficiency. The remaining hydrogen is still very flammable. Also, the onboard H2 generation system makes the system complex. If the UAV requires frequent impulses of power, the fuel cell can be coupled with a supercapacitor. It improves the performance of the UAV while increasing the longevity of the fuel cell. But the whole system is costly and an increased amount of complexity is involved in its implementation. To eliminate the risks associated with hydrogen as fuel, the fuel cell can be replaced by a battery-supercapacitor hybrid. With this combination, the efficiency and power of UAVs can be increased significantly. Along with that, the supercapacitor hybrid is lighter compared to other hybrid energy solutions. However, complex EMS is required. To acquire the best of everything, a tri-energy storage system can be used which comprises the battery, fuel cell, and supercapacitor. It is highly efficient, fulfilling the energy requirement for all operating conditions effectively. Though it is expensive because of its increased complexity. In Table 9, the endurance time with different energy sources is discussed with various energy source combinations.
| Type of energy source | Fuel cell (kW) | Battery (Ah) | Super capacitor (F) | Endurance time (min) |
|---|---|---|---|---|
| Fuell cell + battery | 0.55 | 5.3 | — | 140 |
| Fuel cell + super capacitor | 0.5 | — | 3500 | 100 |
| Battery + super capacitor | — | 4.5 | 120 | 60 |
| Battery + solar cell | — | 4.3 | 150 | 80 |
| Battery + fuel cell + super capacitor | 12.5 | 40 | 88 | 1800 |
Figure 11 shows the endurance times of some UAVs with hybrid energy system. Power must be distributed as required to produce an optimal result of energy efficiency and good performance with a lifespan of as long as possible. Rule-based strategies offer advantages such as low computational cost because of their simplistic nature, prolonged performance, and ease of accessibility. However, they are not the best performers in terms of prediction quality. One can also consider FL strategies as they use less memory, and thus, offer high energy efficiency. Though the rules of FL require constant monitoring and need to be updated frequently.
7. Conclusions and Future Work
An elaborate review was conducted on the advancement in the technology of UAVs for firefighting applications. The paper initially dealt with the current scenario of firefighting techniques where various issues like delay in fire extinguishment, risk of injury for victims and firefighters, et cetera, were exposed. The review focused on the following aspects of the development of UAVs: control management systems involved the discussion of an effective sensor fusing algorithm, optimal path planning strategy, and various control and hybrid energy storage solutions, fire detection algorithms to detect and extinguish a fire. Successively, various path planning algorithms were reviewed and RRT, RRT ∗, and informed RRT ∗ were extensively discussed. Informed RRT ∗ required the least amount of time for the convergence of the discovered solution to the optimum which made it 88% faster than RRT ∗ and RRT. Regarding the control algorithms, several of them were discussed based on their working methodology and the advantages and disadvantages they concurred. It was found that simple algorithms such as PID required less computational energy compared to the more sophisticated control strategies though at a cost to the robustness of the system. Complex algorithms such as SMC with AI offered enhanced performance and robustness, but required a large amount of computational resources. Concerning the fire detection algorithms, the review of various methods was discussed. The HOG algorithm was proven to be the most effective in discerning the fire in the most optimal and precise manner. Following this, the hybrid energy storages for UAVs were discussed and compared to understand which type of energy storage combination is most optimal for increasing flight time and to distribute the energy effectively across all the components, efficient energy management was discussed.
In the future, one can expect to see even more widespread use of these techniques, particularly in applications where the system dynamics are complex or hard to model. Adaptive control algorithms can adjust their parameters in real time based on the system’s current behavior. This makes them well-suited for use in systems where the operating conditions change frequently or are hard to predict. In the future, there is a potential to see more research on developing control algorithms that can handle distributed systems, including techniques for coordinating the actions of the different components. Hybrid energy storage systems, which combine multiple types of storage technologies (such as batteries, supercapacitors, and fuel cells), have the potential to greatly improve the performance and endurance of UAVs. It would be a crucial step towards the development of long-duration UAVs that can be deployed for a variety of tasks such as surveillance, disaster management, transportation, and scientific research.
Disclosure
We certify that the submission is original work and is not under review at any other publication.
Conflicts of Interest
The authors declare no conflicts of interest.
Author Contributions
All coauthors have seen and agree with the contents of the manuscript.
Funding
This work was funded by Vellore Institute of Technology (VIT), Vellore. The Seed Grant Financial Committee (SGEC) has given the project under process/product development File No: SG 20210132. The authors extend their appreciation to the Deanship of Research and Graduate Studies at King Khalid University for funding this work through Large Research Project under Grant RGP2/239/46. All authors would like to express their gratitude to the United Arab Emirates University, Al Ain, UAE, for providing financial support with Grant 12R283.
Acknowledgments
This work was funded by Vellore Institute of Technology (VIT), Vellore. The Seed Grant Financial Committee (SGEC) has given the project under process/product development File No: SG 20210132. The authors extend their appreciation to the Deanship of Research and Graduate Studies at King Khalid University for funding this work through Large Research Project under Grant RGP2/239/46. All authors would like to express their gratitude to the United Arab Emirates University, Al Ain, UAE, for providing financial support with Grant 12R283.
Open Research
Data Availability Statement
Data sharing is not applicable to this article as no new data were created or analyzed in this study.
