International Journal of Aerospace Engineering

Volume 2023, Issue 1 7147168

Research Article

Open Access

TTA-GEP: Terrain Traversability Analysis with Geometry and Environmental Perception for the Path Planning of Planetary Rovers

Li Yang

Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province, College of Information Engineering, China Jiliang University, Hangzhou 310018, China cjlu.edu.cn

Search for more papers by this author

Chao Liang,

Chao Liang

orcid.org/0009-0003-3309-8614

Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province, College of Information Engineering, China Jiliang University, Hangzhou 310018, China cjlu.edu.cn

Search for more papers by this author

Ximing He,

Corresponding Author

Ximing He

[email protected]

orcid.org/0009-0004-9375-1723

Beijing Aerospace Control Center (BACC), Beijing 100094, China

Science and Technology on Aerospace Flight Dynamics Laboratory, Beijing 100094, China

Search for more papers by this author

Dengyang Zhao,

Dengyang Zhao

Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province, College of Information Engineering, China Jiliang University, Hangzhou 310018, China cjlu.edu.cn

Search for more papers by this author

Li Yang,

Li Yang

Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province, College of Information Engineering, China Jiliang University, Hangzhou 310018, China cjlu.edu.cn

Search for more papers by this author

Chao Liang,

Chao Liang

orcid.org/0009-0003-3309-8614

Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province, College of Information Engineering, China Jiliang University, Hangzhou 310018, China cjlu.edu.cn

Search for more papers by this author

Ximing He,

Corresponding Author

Ximing He

[email protected]

orcid.org/0009-0004-9375-1723

Beijing Aerospace Control Center (BACC), Beijing 100094, China

Science and Technology on Aerospace Flight Dynamics Laboratory, Beijing 100094, China

Search for more papers by this author

Dengyang Zhao,

Dengyang Zhao

Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province, College of Information Engineering, China Jiliang University, Hangzhou 310018, China cjlu.edu.cn

Search for more papers by this author

First published: 31 October 2023

https://doi.org/10.1155/2023/7147168

Academic Editor: Chuang Liu

Share a link

Email
Wechat
Bluesky

Abstract

Terrain traversability analysis (TTA), the key to the navigation of planetary rovers, is significant to the safety of the rover. Therefore, owing to its complexity, the Martian terrain is worth analysing comprehensively based on the terrain variability and hazard level. In this work, we propose a novel method for terrain traversability analysis for the path planning of planetary rovers by integrating Martian terrain geometry features with terrain semantic information, which includes geometry and environmental perception (GEP). Specifically, we deploy semantic segmentation to classify common terrain types, such as rocks, bedrocks, and sand, obtaining semantic information as one part of terrain traversability analysis at the same time. Simultaneously, the point cloud is generated by using binocular images from the planetary rover navigation camera (Navcam) to construct a 2.5D elevation map of the environment to analyse the geometric characteristics of the terrain. Besides, we implement path planning based on the results of TTA-GEP. Overall, our proposed method improves the performance of the terrain traversability analysis and reduces the risk of planetary rovers while detecting in an unstructured environment.

1. Introduction

Extraterrestrial planet exploration is an important research direction in the field of aerospace, and patrol detection is one of the important forms of exploration [1]. In particular, when planetary rovers conduct exploration on the surface of Mars, the extraterrestrial navigation of autonomous robots faces severe challenges due to the great unknown, complexity, and uncertainty of the unstructured Martian surface [2]. On the one hand, the terrain on the Martian surface is more complicated and unstructured. On the other hand, due to the limitations of technology and distance, we cannot obtain all the information of the extraterrestrial environment. Hence, hazardous and unstructured environments may pose threats to planetary rover operations. Especially, sandy terrain can cause the rover to sink into it and make it impossible to continue driving. For example, Courage was so deep in it because of the sand that NASA had to abandon the exploration mission. Furthermore, steep area can cause the rover to roll over. Meanwhile, Curiosity’s wheels were damaged by wear and tear due to the impact of rocks [3]. Therefore, to guarantee the safety and extend the service life of the planetary rover, the rover must bypass hazardous areas during its travel, meaning that it will not have any collision with obstacles. Thus, it is essential to enhance the ability of the planetary rover to perceive and analyse its environment. Since the geometric analysis of the terrain cannot determine the type of terrain, the planetary rover is unable to avoid risky terrain types, leaving the rover in danger. Therefore, our proposed method, integrating terrain geometry features with terrain semantic information, can ensure the rover’s own safety and reduce the dependence on human commands by assessing the terrain around the rover autonomously.

In general, terrain traversability is employed to measure the difficulty of crossing a particular area and can indicate the geometric characteristics of the terrain like slope, roughness, and height difference, which lays the foundation for proper path planning in sophisticated environments; these geometric metrics are related to the optimality of the paths as well [4]. Several studies have proposed geometric methods to estimate terrain traversability. For instance, Tanaka et al. [5] analyse terrain traversability based on fuzzy logic and use a fuzzy inference module to convert traversability values into risk values, avoiding uncertainty in sensor data and generating travelable directions for mobile robots. Some researchers [6, 7] apply stereo vision to the visual odometer and navigation system of the Mars Exploration Rovers (MERs), using stereo vision to perceive the geometric aspects of the environment. In another work, Meng et al. [8] generate elevation grid maps using point cloud data acquired by LiDAR, combining RANSAC and least squares methods to estimate the geometric features of the terrain. Some algorithms [9, 10] similarly construct a 2.5D elevation map using the data from a radar-acquired point cloud and an odometer, which is subsequently used to derive traversability by evaluating roughness, slope, and step height. In addition, mobility is important in path planning. To satisfy the field mobility of the lunar rover, two metrics of terrain, terrain slope and terrain roughness, were considered by Ishigami [4]. However, in some cases, the sandy terrain and the flat terrain are similar in height. Thus, the planetary rover can easily cross the flat terrain while it is unable to cross the sandy terrain. Apparently, the geometric approach ignores the semantic information of its environment and does not incorporate semantic information in traversability analysis. The pure geometric analysis ignores this important information. Consequently, it cannot meet the actual mission requirements very well.

Meanwhile, some studies have acquired scene data from sensors [11, 12] for semantic segmentation [13, 14] to obtain semantic maps [15]. For example, Hosseinpoor et al. [16] use aerial RGB images to segment images by different threshold values of height. However, it is not sufficient to analyse traversability by semantic information alone. For instance, areas with relatively low altitude but uneven terrain can equally pose difficulties to the traversability of rovers.

Hence, in this paper, to analyse the terrain traversability of Mars’ unstructured terrain, we propose a novel fusion method called TTA-GEP, in which we integrate terrain geometric features with terrain semantic information, for planetary rover path planning. Firstly, we build 2.5D elevation maps by stereo vision, which are used to evaluate the geometric features of the terrain. Secondly, a terrain classifier is used to enhance the terrain traversability analysis via terrain types. Specifically, the semantic information not only helps the planetary rover to identify the terrain type of the scene but also improves the ability to perceive risks to guarantee the safety of the mission. Finally, we use TTA-GEP for path planning. As shown in Figure 1, we use the Navcam images and camera matrix as the input of our framework. The Navcam images are sent to the terrain classifier for semantic segmentation which is used to obtain the terrain types and generate the semantic map for the terrain traversability analysis. The point cloud of the place where the planetary rover is located is generated by Navcam images and camera matrix, which is projected into a 2.5D elevation map. Then, we analyse the terrain traversability by integrating geometry analysis based on the 2.5D elevation map with semantic information by terrain classifier. Finally, we use the result of the traversability analysis to plan the path for planetary rovers.

Details are in the caption following the image — **Figure 1**
Open in figure viewer PowerPoint

The architecture of our proposed method.

In summary, our main contributions of this work are as follows:

(1)
A terrain classifier of the Martian terrain is realized to identify the Martian terrain types and generate a semantic map of Mars as well
(2)
A method called TTA-GEP to estimate traversability by integrating terrain semantic information with terrain geometric features, including terrain types and the geometric information acquired from elevation data such as slope, elevation difference, and roughness into the evaluation function is proposed
(3)
We use TTA-GEP for rovers’ path planning, and experiments indicate that the proposed method improves the ability to perceive the environment and reduces risks for the rovers

The remainder of this paper is organized as follows. In Section 2, we describe related work. Section 3 introduces our proposed TTA-GEP method. The simulation results are reported in Section 4, followed by conclusions at last.

2. Related Work

2.1. Terrain Classification

For the planetary rover, the unknown and unstructured environment makes its task quite challenging. Besides, most of the Martian surface is covered by loose soil; hence, planetary rovers driving on this ground are prone to slippage, which affects operational safety [17]. Furthermore, the planetary rover can perceive the environment while driving. As a result, classification based on terrain characteristics can further improve its ability to analyse its own traversability and consequently, avoid the traversable risks arising from the terrain promptly. Therefore, terrain classification plays an essential role in improving the safety and operational efficiency of planetary rovers. For example, Liu et al. [18] proposed a hybrid attention semantic segmentation (HASS) network, which aggregates both local interclass and global intraclass contextual information, comparing the terrain consistency of the same class and considering the relationship between neighbouring terrains simultaneously, to further improve the accuracy of segmented classes. In another study, Rothrock et al. [19] proposed soil property and object classification (SPOC), which can classify planetary orbital images and planetary surface images, respectively. The approach is successfully applied in the traversability estimation for the rover and Mars Science Laboratory (MSL) sliding prediction missions. Additionally, Goh et al. [20] proposed a semisupervised learning framework to improve the robustness of image segmentation. Meanwhile, Swan et al. [21] created a large-scale terrain classification dataset called Al4Mars for Mars. Brooks and Iagnemma [22] classified terrain through vibrations caused by the interaction between the wheels of the planetary rover and soil. This technique avoids the instability of classification generated by changes in light intensity in vision-based classification methods. Further, Manduchi et al. [23] classified terrain and detected obstacles based on the inherent characteristics of different terrain types using a stereo camera and a single-line radar. Finally, Halatci et al. [24] integrated vision classification methods with sensor classification to improve classification accuracy.

2.2. Path Planning

Planetary rovers are capable of traversing the planetary surface autonomously and rapidly, avoiding obstacles effectively, and conducting exploration missions, which is important to maximize their scientific value. In recent years, with the development of semantic segmentation, there exist an increasing number of techniques that combine terrain classification with path planning. For example, Egan and Göktogan [25] proposed a traversability estimation algorithm for path planning and control (TEAPAC) by integrating obstacle-aware information and terrain classification results. In another study, Ebadi et al. [26] used the DeepLabv3+ framework to segment the skyline in Mars images to automatically estimate the planetary rover’s global position. Besides, autonomous robots need to perform inspection tasks in a nuclear storage environment. Hence, Wang et al. [27] proposed to convert the obstacles distribution into a two-dimensional binary map that includes the location and orientation of the target points, which constructs maps that may be used for path planning on the purpose of check. Additionally, in [28], semantic information is considered during path planning. More specifically, the semantic information is integrated into the navigation task to construct cost maps using semantic information. Meanwhile, Chiodini et al. [29] proposed a technique to generate the 3D semantic map of the Martian environment for trajectory planning and target identification by using the stereo images acquired by the planetary rover as input. In addition, Sadat et al. [30] proposed a neural network architecture, which uses voxelized radar data and a priori mapping scheme, to provide a probabilistic semantic occupancy layer containing the current and predicted positions of obstacles and vehicles for autonomous vehicles in urban environments. Subsequently, the model selects vehicle trajectories from a set of motion primitives by optimizing a cost function that includes safety-related penalty terms computed via predicted semantic segmentation, driving comfort, and other terms related to traffic rules independent of semantic information. The two-dimensional semantic grid in [31] is also used for traversability estimation to reach the target location specified by the human operator in the rescue mission through the D ^∗ path planning algorithm.

3. Technical Approach

In this section, our proposed method is described by modules. In Section 3.1, the method to generate the 2.5D elevation map from the point cloud is presented. In Section 3.2, we present the method to classify the terrain by using deep learning. Terrain traversability analysis is introduced in Section 3.3 while path planning is in Section 3.4.

3.1. Environment Representation

When constructing the configuration space of a robot using grid maps, the conventional approach divides the entire map into the obstacle and obstacle-free areas. To be more specific, for one grid there are two states, including both the obstacle and obstacle-free [32]. Thus, if there is an obstacle in the grid, this grid is marked as an impassable area; conversely, the grid is marked as an obstacle-free area that is passable. However, the planetary surface terrain is complex with significant unstructured features, and the above map representation obviously cannot sufficiently reflect the planetary surface environment to meet the requirements of planetary rover operation in the Mars scenario. To fully represent the terrain features of the planetary surface, we adopt a 2.5D grid map for the map representation of the Martian surface.

Furthermore, the planetary rover is equipped with Navcam including left one and right one. The binocular images are obtained by the Navcam. Besides, the point cloud is generated by stereo matching [33] when the stereo images (I_NCL, I_NCR) are rectified. After that, the point cloud is mapped to the grid map after filtering the noise and downsampling to reduce the number of point clouds by using RANSAC [34]. However, since the 3D point cloud generated based on binocular vision recovery is sparse in some regions far away from the Navcam, the data needs to be preprocessed. Hence, we adopt the inverse distance weighting method [35] for the interpolation operation, which is a weighted average interpolation method to reduce the impact of missing information on the accuracy of the map. Specifically, this method considers that any observation has an impact on the neighbourhood, and the smaller the distance, the greater the impact. It can be expressed by

(1)

where Z is the estimated value of the height; Z_i is the height of the samples (i = 1, 2, 3 ⋯ , N); D_i is the distance of the sample from the unsampled point; and p is the power of the distance.

3.2. Terrain Classifier

The semantic segmentation of the terrain is one of the important parts of the whole framework by semantic segmentation. Specifically, through the classifier, each pixel gets a label with a predefined terrain class when inputting a Navcam image. It means that the planetary rover can classify the terrain and achieve a more comprehensive understanding of the terrain where the rover is located, such as sand, rocks, and soil. For planetary rovers, mountains, rocks, and sand are obstacles; sand is geologically soft and can cause wheels to sink, which threatens the safety of planetary rovers, all of which are impassable. In this work, three different deep neural network architectures, HRNet [36], SegFormer [37], and DeepLabv3+ [38], were tested and compared. HRNet maintains high-resolution information of images while reducing the loss of details. Deeplabv3+, including hole convolution with multiple hole rates, increases the receptive field and improves the ability to extract semantic features. Segformer, introducing the architecture of transformers [39, 40], has a global receptive field, which is more robust but more difficult to train. We selected HRNet, and the results of our experiments in Section 4.1 show that it achieves better performance. In addition, the dataset is S⁵Mars [41], which has 6000-labeled images containing nine labels of soil, bedrock, rock, rover, sky, ridge, trace, sand, and hole. The detailed descriptions of these nine classes are given in Table 1.

Table 1. Mars terrain types.

Class	Description
Soil	Martian soil, flat and yellow-black in color.
Bedrock	Stones that are relatively flat and not exposed to the ground.
Rock	Stones that are large in size and completely exposed to the ground.
Rover	The body of the planetary rover.
Sky	Martian sky.
Ridge	Mountains in Mars.
Trace	Wheel trajectory.
Sand	Sandy region.
Hole	Traces left by the planetary rover during sampling.

The label of each point P_i(x, y, z) of the point cloud is generated from the pixel (u, v) of the Navcam images. More so, the scale of the elevation map and the point cloud is not the same. When projecting the point cloud, each grid will contain several points P_i, which correspond to different labels L_i, and each grid only has one height value and one label; thus, it is necessary to match the point cloud data and labels in one grid. Furthermore, we divide it into the following three cases. First, if there is only one point in the grid, the height of the grid is the height of this point, and the grid label is the label of this point. Second, if there are two points in the grid, the height of the grid is taken as the height of the higher point, and the grid label is the label of the corresponding point. Third, if there are more than two points in the grid, the height of each grid is sorted, and the middle height is taken as the height of the grid; and the grid label is the label of the corresponding point. Currently, the above-proposed strategy can meet our requirements. However, how to handle the grid data more reasonably and match the labels of each grid needs our further research.

3.3. Terrain Traversability Analysis

Terrain traversability analysis is an indispensable part for the path planning of planetary rovers. To analyse terrain traversability, we can obtain three terrain features [42] from the elevation data, which includes slope [43], roughness, and elevation difference. However, when analysing digital elevation maps, topographic features cannot be precisely reflected if only one grid is considered at a time. Therefore, we utilize the window method to analyse a grid as the center and analyse the grid in the window area of this grid as a whole to reduce the error and improve the accuracy. First, we need to fit the window area to the plane shown in Figure 2.

Suppose that there are N grids in each window, the coordinate of each grid G_ij is (x_k, y_k), and the corresponding height of the grid is z_k; then, the fitting plane is shown in Eq.(2) as follows:

(2)

that is

(3)

where z is the height of the fitting plane, and by the least squares method, we should find a₀, a₁, a₂ to minimize S.

(4)

where M means the total number of grids in each window.

The partial derivatives of a₀, a₁, a₂ are given as follows:

(5)

convert to matrix form as follows:

(6)

Using Cramer’s rule, we can find a₀, a₁, a₂, and the normal vector of the fitting plane is

Since the normal vector of the horizontal plane Z = 0 is

, the angle θ between the fitting plane and the horizontal plane, which is the slope angle θ_slope, is given by

(7)

Specifically, the roughness of the terrain may be characterized as the root mean square of the fitting plane, and the window method is used to evaluate the roughness of each grid as well. The distance from each grid in the window to the fitting plane is given by

(8)

Thus, the roughness R_ij of each gird is given by

(9)

The height difference H of the terrain is obtained from the maximum value H_max and the minimum value H_min of the elevation in the fitting plane, which is given by

(10)

Before analysing terrain traversability, we set thresholds for slope, roughness, and elevation difference, respectively. If any geometric feature of any grid exceeds the corresponding threshold, it will be marked as impassable.

When the planetary rover moves, it is necessary to maintain a safe attitude. It is known that the current attitude of the planetary rover is P_cur = (c_pitch, c_roll); the safe attitude does not exceed P_safe = (s_pitch, s_roll); and the safe attitude of each grid is judged by

(11)

If either of the two equations above is satisfied, the current grid is impassable and the maximum cost is assigned to this grid.

Furthermore, the hazard level of different terrain types for planetary rover operation determines the different terrain costs of each grid. Particularly, soil offers the least resistance to the wheels; thus, it is the most desirable and the least risky terrain for planetary rovers. Specifically, a bedrock, which is not exposed to the ground, will cause wear and tear on the wheels. In addition, rocks, ridges, and sand are not considered as traversable regions, giving them the highest cost. Thus, these areas should be avoided when running. Additionally, different terrain types mean different terrain costs, and the cost is assigned to each type according to our requirements; hence, the terrain semantic cost S_tc of each grid is matched with the label of each grid.

Meanwhile, planetary rovers work in complicated environments, where a single metric of traversability cannot represent traversability well. Therefore, we integrate the terrain geometric metrics with metric of terrain types in the traversability evaluation function. Additionally, when planning the path, this function integrates the influence of different indicators in each grid, and the evaluation function is given by

(12)

where C_GEP represents the cost of each grid, P_s, P_r, P_h, and P_tc are the weights to set the priority of terrain slope, terrain roughness, height difference, and terrain semantic information, respectively, and α, β, γ, δ ∈ [0, 1] are the normalization coefficients. For example, the planetary terrain is complex and when slope is large, there is a possibility of side-swiping. Therefore, slope cost has the greatest effect on the cost of the whole grid compared to other types of cost. Then, P_s should be set to the maximum of the four values. Consequently, the sum of this weight is 100%.

3.4. TTA-GEP for Path Planning

Typically, the classical A ^∗ algorithm [44] only finds an optimal path given the start and end points but does not consider the kinematic constraints of the vehicle. Conversely, hybrid A ^∗ [45] combines the kinematic constraints of the vehicle with the classical A ^∗ algorithm to satisfy the nonholonomic constraints of the vehicle. In this work, the planning algorithm we adopt is based on a variation of hybrid A ^∗. Additionally, we use space-contiguous to extend the child node as shown in Figure 3. Specifically, the grid in the center denotes the parent node while the other points surrounded represent the subnodes. Figure 3(a) shows the eight subnodes to be extended in classical A ^∗ algorithm, which are in eight directions of the center grid. Shown in Figure 3(b) are the six child nodes of hybrid A ^∗, whose positions are located anywhere around the center grid, meaning that they satisfy kinematic constraints. The extension of the nodes is based on

(13)

where g(n) is the accumulated cost of the path from the start to the current node n; and h(n) is the estimated distance between node n and the end, which can provide information about the distance between the node to be extended and the end; thus, improving the search efficiency and avoiding blind search. Furthermore, we do not consider the situation of reversing according to the requirements of our mission, and we use the heuristic distance of holonomic with obstacles by

(14)

Moreover, the complex terrain of the Martian surface requires the consideration of terrain traversability in addition to the optimality of the paths in terms of the distance and extension of the nodes while satisfying the nonholonomic constraints. Therefore, g(n) is redesigned in conjunction with our proposed TTA-GEP. It is given by

(15)

where g(p) is the cost of the parent node n_p; C_GEP(n_p, n_c) is the terrain fusion analysis cost from the parent node n_p to the child node n_c, obtained from Eq.(12); and S(n_p, n_c, φ) is the steering cost from the parent node n_p to the child node n_c, and φ is the angle between the node n_p and n_c. Since the motion of the planetary rover is constrained by the minimum radius R_mr of turning, there exists a corresponding maximum angle θ_ma of steering for its change of heading. The formula is as follows:

(16)

where L means the length of a rover.

Meanwhile, when extending a child node, it is necessary to determine whether the angle between the parent node and the node to be extended exceeds the maximum steering angle. when exceeding the maximum angle of steering, the node is given a high cost. Therefore, the steering cost is given by

(17)

where

is the normalization coefficient and inf represents the high cost.

4. Experiments

4.1. Terrain Classification

We trained our terrain classifier on NVIDIA GeForce RTX 3090 GPU by using Pytorch. Besides, the training set and test set are 80% and 20%, respectively. In addition, the source images are RGB images with a resolution of 1200 × 1200. Specifically, the image is cropped to a resolution of 512 × 512 as the input to the network. The output of the network is a color semantic mask with the same resolution as the input. Meanwhile, we applied data enhancement techniques such as cropping, mirroring, and resizing to improve the accuracy of training and the robustness of the model against changes in the styles of images. For the semantic segmentation, we tested on the three networks mentioned above; and we evaluated the performance of the network using the mean intersection over union (mIoU), given by

(18)

where TP_c, FP_c, and FN_c denote the number of true positive, false positive, and false negative predictions for each prediction class, respectively, and C is the total number of classes.

Table 2 displays the test results of the three network architectures on S⁵Mars. Compared with other classes, the three networks perform better on soil, bedrock, sky, ridge, and sand, which have a better predictive accuracy. The reason for this is that these terrain types have distinct features, and the textures are easy to distinguish. For the prediction of soil, rover, sky, ridge, and sand, HRNet performs better than the other two networks. Besides, in terms of mIoU, the overall performance of HRNet is significantly better than the other two networks. Due to the complexity of the transformer architecture, training Segformer can be more challenging than the other two segmentation models. As shown in Figure 4, the first line shows the source images while the images predicted by HRNet are on the second line. Furthermore, the IoU of rock is low, because there are more samples of bedrock and fewer samples of rock in the dataset. Meanwhile, the IoU of hole is 0, which is caused by the obvious lack of samples as well.

Table 2. IoU(%) of SegFormer, DeepLabv3+, and HRNet on S⁵Mars.

Network	SegFormer	DeepLabv3+	HRNet
Soil	63.91	79.29	81.47
Bedrock	85.2	91.08	90.97
Rock	2.41	19.28	14.84
Rover	40.5	79.37	86.51
Sky	78.57	89.38	90.53
Ridge	73.93	88.78	89.95
Trace	6.7	75.72	75.71
Sand	66.73	71.39	74.2
Hole	0	0	0
mIoU	46.44	66.03	67.13

4.2. Environmental Construction

Based on the method of environment construction mentioned above, we input the camera matrix and binocular images acquired by the Navcam to construct a 2.5D elevation map with the point cloud generated by stereo matching. The size of each grid map is 2000 × 2000 and the resolution of each grid is 0.02 × 0.02 m². In addition, we visualize the map by OpenGL [46], and the top view of the 2.5D elevation map is shown in Figure 5(a). Specifically, the methods we adopt clearly reflect the information regarding the Martian surface terrain and contribute to subsequent traversability analysis and path planning. Figure 5(b) shows the semantic map, which integrates the labels with the 2.5D elevation map. The green area is the region of bedrock; and the fusion of terrain labels and elevation maps can help us further identify terrain types and what the environment is like while the rover is running.

4.3. TTA-GEP and Path Planning

We validate the effectiveness of our proposed method in the environment constructed in Section 4.2. In this experiment, parameters needed are recorded in Table 3. More specifically, the cost of traversability is displayed in [0, 1] with different colors as shown in Figure 6, where the value of the red area is one, which represents completely impassable. The remaining areas are passable and the boundaries are indicated by yellow and green.

Table 3. The configuration of parameters.

Parameter	Value
L (cm)	150
R_mr (m)	5
P_s, P_h, P_ts , and P_r	0.5, 0.25, 0.15, and 0.1
s_pitch and s_roll (deg)	15, 15
Threshold of height (cm), slope (deg), and roughness (cm)	15, 10, 3

In Figures 6(a) and 6(b), the results of the terrain traversability analysis are obviously different under the same configuration of parameters. Particularly, the red area in Figure 6(b) is obviously larger than that in Figure 6(a). Combining the results of Figure 5(b) semantic and elevation map fusion, the concentrated green bedrock region near the rover is reflected in the corresponding position in Figure 6(b). It means that our method has the ability to complement the performance of terrain traversability analysis. But for some of the green areas of bedrock, which are small and far away from the planetary rover, they are not fully represented in Figure 6(b), because the point cloud generated is affected by the accuracy of stereo matching and lack of data caused by sparse of point clouds. However, the impact of this problem may be solved by improving the accuracy of the stereo matching or updating the map in time after the planetary rover travels a certain distance. Overall, the better performance of our proposed method improves the ability to perceive the environmental risks and reduce possible risks.

What is shown in Figure 7 is the results of classical A ^∗ algorithm and our proposed method for path planning with the setting in Figure 6(b), where S represents the starting point and D represents the end. In Figure 7, we can see that classical A ^∗ algorithm only considers the cost of distance and has no kinematic constraints. Thus, the planned path is shorter and takes the form of straight lines. Our proposed method meets kinematic constraints and considers not only the geometric features of the terrain but also the influence of different terrain types. Table 4 shows the expected time of execution of planning and the length of paths for both methods at the same starting and ending points. Compared to classical A ^∗, our method has a 4.32% decrease in planning time and a 12.2% increase in the length of the path. However, the increase in the length of the path makes the operation of the planetary rover safer, which is worthwhile. Thus, TTA-GEP is a conservatively analytical method to analyse terrain traversability. In addition, it can improve the safety of the path and extend the service life of rovers.

Table 4. The comparison of the numerical results between the classical A ^∗ algorithm and our method in the environment (Figure 6(b)). The time here refers to the expected running time of the planetary rovers from the start point to the end point.

Method	Time (s)	Length of path (m)
A ^∗ algorithm	364.21	12.13
Our method	348.47	13.61
Change (%)	-4.32	+12.20

5. Conclusion

In this paper, we propose a novel method called TTA-GEP to analyse terrain traversability by integrating terrain geometry with terrain semantic information. The terrain classifier is built on HRNet, and the model is trained by using a dataset called S⁵Mars containing nine labels of soil, bedrock, rock, rover, sky, ridge, trace, sand, and hole. When inputting a Navcam image, the classifier gives each pixel of the image a terrain type which has an impact on the terrain traversability analysis. This method enables autonomous terrain classification to identify hazardous terrains and integrate terrain semantic information and terrain geometric features to analyse terrain traversability, compensating for the shortcomings of analysing traversability relying only on geometric analysis or terrain semantic analysis. Furthermore, we plan paths based on TTA-GEP by using a variant of hybrid A ^∗. Since the real environment is more complicated, the validation process simplifies the analysis by imposing constraints. Overall, we achieved the expected results in the experiments. A series of experiments indicate that TTA-GEP is an effective method for terrain traversability analysis in unstructured Martian surface, which improves the safety not only for the path but also for rovers. In future work, our method may be combined with spacecraft attitude controllers [47] to further enhance the safety of spacecrafts. However, in more complex environments, our approach may face more challenges; and thus, we need to continuously improve the validity and robustness of our approach in our future work.

Conflicts of Interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Acknowledgments

This work is supported by the Key Laboratory of Space Flight Dynamics Technology (KJW6142210210309) and the Key Research and Development Projects in Zhejiang Province (2022C01204).

Open Research

Data Availability

The data supporting this are from previously reported studies and datasets, which have been cited.

References

1 Yang J., Sun Z., Zheng Y., Li H., Tao Z., Zhang T., and Tian H., Overview on development of planetary rover technology, Journal of Deep Space Exploration. (2020) 7, no. 5, 419–427, https://doi.org/10.15982/j.issn.2096-9287.2020.20200031.
10.15982/j.issn.2096-9287.2020.20200031
Google Scholar
2 Sonsalla R., Cordes F., Christensen L., Roehr T. M., Stark T., Planthaber S., Maurus M., Mallwitz M., and Kirchner E. A., Field testing of a cooperative multi-robot sample return mission in Mars analogue environment, Proceedings of the 14th symposium on advanced space technologies in robotics and automation (ASTRA), 2017, Leiden, Netherlands, https://www.dfki.de/web/forschung/projekte-publikationen/publikation/9091.
Google Scholar
3 Arvidson R. E., DeGrosse P., Grotzinger J. P., Heverly M. C., Shechet J., Moreland S. J., Newby M. A., Stein N., Steffy A. C., Zhou F., Zastrow A. M., Vasavada A. R., Fraeman A. A., and Stilly E. K., Relating geologic units and mobility system kinematics contributing to curiosity wheel damage at gale crater, Mars, Journal of Terramechanics. (2017) 73, 73–93, https://doi.org/10.1016/j.jterra.2017.03.001, 2-s2.0-85017418543.
10.1016/j.jterra.2017.03.001
Web of Science® Google Scholar
4 Ishigami G., Nagatani K., and Yoshida K., Path planning and evaluation for planetary rovers based on dynamic mobility index, 2011 IEEE/RSJ international conference on intelligent robots and systems, 2011, San Francisco, CA, USA, 601–606, https://doi.org/10.1109/IROS.2011.6094768.
10.1109/IROS.2011.6094768
Google Scholar
5 Tanaka Y., Ji Y., Yamashita A., and Asama H., Fuzzy based traversability analysis for a mobile robot on rough terrain, 2015 IEEE International Conference on Robotics and Automation (ICRA), 2015, Seattle, WA, USA, 3965–3970, https://doi.org/10.1109/ICRA.2015.7139753, 2-s2.0-84938254514.
10.1109/ICRA.2015.7139753
Google Scholar
6 Goldberg S. B., Maimone M. W., and Matthies L., Stereo vision and rover navigation software for planetary exploration, Proceedings, IEEE Aerospace Conference, 2002, Big Sky, MT, USA, 5–5, https://doi.org/10.1109/AERO.2002.1035370, 2-s2.0-84879375888.
10.1109/AERO.2002.1035370
Google Scholar
7 Matthies L., Maimone M., Johnson A., Yang C., Willson R., Villalpando C., Goldberg S., Huertas A., Stein A., and Angelova A., Computer vision on Mars, International Journal of Computer Vision. (2007) 75, no. 1, 67–92, https://doi.org/10.1007/s11263-007-0046-z, 2-s2.0-34547193294.
10.1007/s11263-007-0046-z
Web of Science® Google Scholar
8 Meng X., Cao Z., Liang S., Pang L., Wang S., and Zhou C., A terrain description method for traversability analysis based on elevation grid map, International Journal of Advanced Robotic Systems. (2018) 15, no. 1, https://doi.org/10.1177/1729881417751530, 2-s2.0-85041377175.
10.1177/1729881417751530
Web of Science® Google Scholar
9 Wermelinger M., Fankhauser P., Diethelm R., Krüsi P., Siegwart R., and Hutter M., Navigation planning for legged robots in challenging terrain, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016, Daejeon, Republic of Korea, 1184–1189, https://doi.org/10.1109/IROS.2016.7759199, 2-s2.0-85006339416.
10.1109/IROS.2016.7759199
Google Scholar
10 Fankhauser P., Bloesch M., and Hutter M., Probabilistic terrain mapping for mobile robots with uncertain localization, IEEE Robotics and Automation Letters. (2018) 3, no. 4, 3019–3026, https://doi.org/10.1109/lra.2018.2849506, 2-s2.0-85056442529.
10.1109/LRA.2018.2849506
Web of Science® Google Scholar
11 Dargazany A. R., Stereo-based terrain traversability analysis using normal-based segmentation and superpixel surface analysis, 2019, https://arxiv.org/abs/1907.06823.
Google Scholar
12 Suger B., Steder B., and Burgard W., Traversability analysis for mobile robots in outdoor environments: a semi-supervised learning approach based on 3D-lidar data, 2015 IEEE International Conference on Robotics and Automation (ICRA), 2015, Seattle, WA, USA, 3941–3946, https://doi.org/10.1109/ICRA.2015.7139749, 2-s2.0-84938254540.
10.1109/ICRA.2015.7139749
Google Scholar
13 Guan T., Kothandaraman D., Chandra R., Sathyamoorthy A. J., Weerakoon K., and Manocha D., GA-Nav: efficient terrain segmentation for robot navigation in unstructured outdoor environments, IEEE Robotics and Automation Letters. (2022) 7, no. 3, 8138–8145, https://doi.org/10.1109/LRA.2022.3187278.
10.1109/LRA.2022.3187278
Web of Science® Google Scholar
14 Maturana D., Chou P.-W., Uenoyama M., and Scherer S., Real-time semantic mapping for autonomous off-road navigation, Field and Service Robotics, 2018, Springer, 335–350, https://doi.org/10.1007/978-3-319-67361-5_22.
10.1007/978-3-319-67361-5_22
Google Scholar
15 Vincent G., Yepremyan A., Chen J., and Goh E., Mixed-domain training improves multi-mission terrain segmentation, Computer Vision–ECCV 2022 Workshops: Tel Aviv, Israel, 2023, Springer Nature Switzerland, Cham, Switzerland, 96–111, https://doi.org/10.1007/978-3-031-25056-9_7.
10.1007/978-3-031-25056-9_7
Google Scholar
16 Hosseinpoor S., Mantelli M., Pittol D., Kolberg M., Maffei R., Prestes E., and Torresen J., Traversability analysis by semantic terrain segmentation for mobile robots, 2021 IEEE 17th international conference on automation science and engineering (CASE), 2021, Lyon, France, 1407–1413, https://doi.org/10.1109/CASE49439.2021.9551629.
10.1109/CASE49439.2021.9551629
Google Scholar
17 Ishigami G., Miwa A., Nagatani K., and Yoshida K., Terramechanics-based model for steering maneuver of planetary exploration rovers on loose soil, Journal of Field Robotics. (2007) 24, no. 3, 233–250, https://doi.org/10.1002/rob.20187, 2-s2.0-34247489270.
10.1002/rob.20187
Web of Science® Google Scholar
18 Liu H., Yao M., Xiao X., and Cui H., A hybrid attention semantic segmentation network for unstructured terrain on Mars, Acta Astronautica. (2023) 204, 492–499, https://doi.org/10.1016/j.actaastro.2022.08.002.
10.1016/j.actaastro.2022.08.002
ADS Web of Science® Google Scholar
19 Rothrock B., Kennedy R., Cunningham C., Papon J., Heverly M., and Ono M., SPOC: deep learning-based terrain classification for Mars rover missions, AIAA SPACE 2016, 2016, Long Beach, CA, USA, https://doi.org/10.2514/6.2016-5539.
10.2514/6.2016-5539
Google Scholar
20 Goh E., Chen J., and Wilson B., Mars terrain segmentation with less labels, 2022 IEEE Aerospace Conference (AERO), 2022, Big Sky, MT, USA, 1–10, https://doi.org/10.1109/AERO53065.2022.9843245.
10.1109/AERO53065.2022.9843245
Google Scholar
21 Swan R. M., Atha D., Leopold H. A., Gildner M., Oij S., Chiu C., and Ono M., AI4MARS: a dataset for terrain-aware autonomous driving on Mars, 2021 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW), 2021, Nashville, TN, USA, 1982–1991, https://doi.org/10.1109/CVPRW53098.2021.00226.
10.1109/CVPRW53098.2021.00226
Google Scholar
22 Brooks C. A. and Iagnemma K., Vibration-based terrain classification for planetary exploration rovers, IEEE Transactions on Robotics. (2005) 21, no. 6, 1185–1191, https://doi.org/10.1109/tro.2005.855994, 2-s2.0-29844444582.
10.1109/TRO.2005.855994
Web of Science® Google Scholar
23 Manduchi R., Castano A., Talukder A., and Matthies L., Obstacle detection and terrain classification for autonomous off-road navigation, Autonomous Robots. (2005) 18, no. 1, 81–102, https://doi.org/10.1023/b:auro.0000047286.62481.1d, 2-s2.0-9144265616.
10.1023/B:AURO.0000047286.62481.1d
Web of Science® Google Scholar
24 Halatci I., Brooks C. A., and Iagnemma K., Terrain classification and classifier fusion for planetary exploration rovers, 2007 IEEE Aerospace Conference, 2007, Big Sky, MT, USA, 1–11, https://doi.org/10.1109/AERO.2007.352692, 2-s2.0-34548731283.
10.1109/AERO.2007.352692
Google Scholar
25 Egan R. and Göktogan A. H., Deep Learning Based Terrain Classification for Traversability Analysis, Path Planning and Control of a Mars Rover, Australasian Conference on Robotics and Automation (ACRA-2021), 2021, Melbourne, Australia, https://www.researchgate.net/publication/356833048_Deep_Learning_based_Terrain_Classification_for_Traversability_Analysis_Path_Planning_and_Control_of_a_Mars_Rover.
Google Scholar
26 Ebadi K., Coble K., Kogan D., Atha D., Schwartz R., Padgett C., and Hook J. V., Semantic mapping in unstructured environments: toward autonomous localization of planetary robotic explorers, 2022 IEEE Aerospace Conference., March 2022, Big Sky, Montana, USA, https://russ-stuff.com/wp-content/uploads/2022/01/IEEE_AERO___Semantic_Segmentation_TRN_on_Mars.pdf.
Google Scholar
27 Wang M., Long X., Chang P., and Padlr T., Autonomous robot navigation with rich information mapping in nuclear storage environments, 2018 IEEE international symposium on safety, security, and rescue robotics (SSRR), 2018, Philadelphia, PA, USA, 1–6, https://doi.org/10.1109/SSRR.2018.8468634, 2-s2.0-85055539385.
10.1109/SSRR.2018.8468634
Google Scholar
28 Achat S., Marzat J., and Moras J., Path planning incorporating semantic information for autonomous robot navigation, 19th International Conference on Informatics in Control, Automation and Robotics (ICINCO), 2022, Lisbon, Portugal, 285–295, https://doi.org/10.5220/0011134300003271.
10.5220/0011134300003271
Google Scholar
29 Chiodini S., Torresin L., Pertile M., and Debei S., Evaluation of 3D CNN semantic mapping for rover navigation, 2020 IEEE 7th international workshop on metrology for AeroSpace (MetroAeroSpace), 2020, Pisa, Italy, 32–36, https://doi.org/10.1109/MetroAeroSpace48742.2020.9160157.
10.1109/MetroAeroSpace48742.2020.9160157
Google Scholar
30 Sadat A., Casas S., Ren M., Wu X., Dhawan P., and Urtasun R., Perceive, predict, and plan: safe motion planning through interpretable semantic representations, Computer Vision–ECCV, 2020, Springer, 414–430, https://doi.org/10.1007/978-3-030-58592-1_25.
10.1007/978-3-030-58592-1_25
Google Scholar
31 Delmerico J., Mueggler E., Nitsch J., and Scaramuzza D., Active autonomous aerial exploration for ground robot path planning, IEEE Robotics and Automation Letters. (2017) 2, no. 2, 664–671, https://doi.org/10.1109/lra.2017.2651163, 2-s2.0-85061596484.
10.1109/LRA.2017.2651163
Web of Science® Google Scholar
32 Ghosh D., Nandakumar G., Narayanan K., Honkote V., and Sharma S., Kinematic constraints based bi-directional RRT (KB-RRT) with parameterized trajectories for robot path planning in cluttered environment, 2019 International Conference on Robotics and Automation (ICRA), 2019, Montreal, QC, Canada, 8627–8633, https://doi.org/10.1109/ICRA.2019.8793896, 2-s2.0-85071439836.
10.1109/ICRA.2019.8793896
Google Scholar
33 Zhang C., Li Z., Cheng Y., Cai R., Chao H., and Rui Y., Meshstereo: a global stereo model with mesh alignment regularization for view interpolation, 2015 IEEE International Conference on Computer Vision (ICCV), 2015, Santiago, Chile, 2057–2065, https://doi.org/10.1109/iccv.2015.238, 2-s2.0-84973866076.
10.1109/iccv.2015.238
Google Scholar
34 Fischler M. A. and Bolles R. C., Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Communications of the ACM. (1981) 24, no. 6, 381–395, https://doi.org/10.1145/358669.358692, 2-s2.0-0019574599.
10.1145/358669.358692
Web of Science® Google Scholar
35 Bartier P. M. and Keller C. P., Multivariate interpolation to incorporate thematic surface data using inverse distance weighting (IDW), Computers & Geosciences. (1996) 22, no. 7, 795–799, https://doi.org/10.1016/0098-3004(96)00021-0, 2-s2.0-0030430467.
10.1016/0098-3004(96)00021-0
CAS Web of Science® Google Scholar
36 Wang J., Sun K., Cheng T., Jiang B., Deng C., Zhao Y., Liu D., Mu Y., Tan M., Wang X., and Liu W., Deep high-resolution representation learning for visual recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence. (2021) 43, no. 10, 3349–3364, https://doi.org/10.1109/TPAMI.2020.2983686.
10.1109/TPAMI.2020.2983686
PubMed Web of Science® Google Scholar
37 Xie E., Wang W., Yu Z., Anandkumar A., Alvarez J. M., and Luo P., SegFormer: simple and efficient design for semantic segmentation with transformers, Advances in Neural Information Processing Systems. (2021) 34, 12077–12090.
Google Scholar
38 Chen L.-C., Zhu Y., Papandreou G., Schroff F., and Adam H., Encoder-decoder with atrous separable convolution for semantic image segmentation, Computer Vision – ECCV 2018, 2018, Springer, 801–818, https://doi.org/10.1007/978-3-030-01234-2_49, 2-s2.0-85055083443.
10.1007/978-3-030-01234-2_49
Google Scholar
39 Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser L., and Polosukhin I., Attention is all you need, Advances in Neural Information Processing Systems. (2017) 30.
Google Scholar
40 Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D., Zhai X., Unterthiner T., Dehghani M., Minderer M., Heigold G., Gelly S., Uszkoreit J., and Houlsby N., An image is worth 16x16 words: transformers for image recognition at scale, 2020, https://arxiv.org/abs/2010.11929.
Google Scholar
41 Zhang J., Lin L., Fan Z., Wang W., and Liu J., S⁵ Mars: self-supervised and semi-supervised learning for Mars segmentation, 2002, https://arxiv.org/abs/2207.01200.
Google Scholar
42 Yu X., Huang Q., Wang P., and Guo J., Comprehensive global path planning for lunar rovers, 2020 3rd international conference on unmanned systems (ICUS), 2020, Harbin, China, 505–510, https://doi.org/10.1109/ICUS50048.2020.9274967.
10.1109/ICUS50048.2020.9274967
Google Scholar
43 Brooks R., A robust layered control system for a mobile robot, IEEE Journal on Robotics and Automation. (1986) 2, no. 1, 14–23, https://doi.org/10.1109/JRA.1986.1087032, 2-s2.0-0022688781.
10.1109/JRA.1986.1087032
Web of Science® Google Scholar
44 Hart P. E., Nilsson N. J., and Raphael B., A formal basis for the heuristic determination of minimum cost paths, IEEE Transactions on Systems Science and Cybernetics. (1968) 4, no. 2, 100–107, https://doi.org/10.1109/TSSC.1968.300136, 2-s2.0-84899829959.
10.1109/TSSC.1968.300136
Web of Science® Google Scholar
45 Dolgov D., Thrun S., Montemerlo M., and Diebel J., Practical search techniques in path planning for autonomous driving, Ann Arbor. (2008) 1001, article 48105.
Google Scholar
46 Kessenich J. M., Sellers G., and Shreiner D., OpenGL® Programming Guide: The Official Guide to Learning Opengl®, Version 4.5 with Spir-V, 2016, Addison-Wesley, Boston, MA, USA.
Google Scholar
47 Liu C., Yue X., Shi K., and Sun Z., Spacecraft Attitude Control: A Linear Matrix Inequality Approach, 2022, Elsevier, Amsterdam, UK.
Google Scholar

All articles

TTA-GEP: Terrain Traversability Analysis with Geometry and Environmental Perception for the Path Planning of Planetary Rovers

Abstract

1. Introduction

2. Related Work

2.1. Terrain Classification

2.2. Path Planning

3. Technical Approach

3.1. Environment Representation

3.2. Terrain Classifier

3.3. Terrain Traversability Analysis

3.4. TTA-GEP for Path Planning

4. Experiments

4.1. Terrain Classification

4.2. Environmental Construction

4.3. TTA-GEP and Path Planning

5. Conclusion

Conflicts of Interest

Acknowledgments

Open Research

Data Availability

References

Figures

References

Information

About Wiley Online Library

Help & Support

Opportunities

Connect with Wiley