2.1. Embodied Navigation

Since the introduction of the first mobile robot in the 1960s, navigation technology has garnered significant attention as a pivotal capability of robots. As a core technology, it establishes the groundwork for robots to accomplish diverse complex tasks and finds extensive application across various scenarios, actively engaging in production and daily life. Mobile robot navigation technology encompasses multilayered architectures such as perception, decision-making, and control. Robot navigation can be categorized based on system framework, the presence or absence of environmental priors, and signal input into modular framework-based navigation, end-to-end framework-based navigation, map-based navigation, mapless navigation, visual navigation, and VLN.

2.2. Embodied Exploration

In embodied exploration tasks, agents acquire the ability to navigate unfamiliar environments to gather information essential for future task deployments in those environments [76]. When an agent finds itself in uncharted territory, active exploration and foundational knowledge acquisition about the surroundings are crucial for efficiently undertaking new tasks. Taking a home robot as an example, if it is expected to execute commands such as “bring some fruits from the kitchen to the owner,” it must explore the room in advance, figure out what its purpose is, the path to take, and where the items are located.

Navigation is central to embodied visual exploration. Unlike task-oriented navigation, embodied visual exploration lacks a definitive goal but aims to maximize environmental data collection and form pertinent map structure models, accurately depicting semantic and geometric environmental information for downstream tasks. Historically, SLAM technology has been used for map construction. Agents use this technology for localization and path planning, though it remains a well-established yet passive exploration method requiring advanced environment traversal [77]. Additionally, SLAM heavily relies on sensors, often leading to noise interference issues. Currently, learning-based techniques dominate embodied visual exploration tasks. Compared to traditional SLAM, the learning-based approach constructs maps using visual information less susceptible to noise and actively explores the environment. It employs unsupervised methods, such as RL, to learn within new environments, minimizing manual effort and enhancing efficiency. However, learning-based methods for embodied visual exploration often face sparse rewards challenges. In RL, rewards are granted solely upon reaching a specific state. In complex new environments, an agent’s state cannot be predetermined, resulting in sparse or absent rewards [78]. Hence, exploration strategies are a focal research area in embodied visual exploration. Four distinct exploration methods from the literature—curiosity, novelty, coverage, and reconstruction—can establish reward systems aiding agents in environmental exploration. Agents guided by curiosity are attracted to unexpected states, earning rewards when actual states diverge from predicted ones. The larger the discrepancy, the greater the reward, focusing on intrinsic rewards over external ones [56, 79, 80]. In the novelty mechanism, the goal is to discover unencountered states, rewarding agents based on the infrequency of current state accesses [58, 81, 82]. Coverage mechanisms incentivize agents to observe their environments comprehensively, rewarding them based on the increment of observed objects of interest [59]. Finally, in the reconstruction mechanism, agents attempt to reconstruct scenes using their central view and sensor information. The closer the reconstructed views are to the originals, the higher the reward [61, 83].

2.3. Embodied Question Answering

Malinowski et al. [84] pioneered the visual question answering task. This challenge entails providing an image and a natural language question, the model then answering by interpreting both the natural language and visual information. They subsequently proposed the Neural Image Question Answering model [85] to address the task, modeling the VQA problem as a generation problem, extracting image and problem features stitching as inputs, and then feeding into the LSTM network to output the answer, and it is a classic VQA model type based on multimodal fusion. The key to such a method is how to integrate the vision and text features. A number of academics have devoted their research to it. The most common strategies for fusing visual and text features involve methods like affine transformation [86], vector outer product [87], matrix decomposition [88], etc. Researches have shown that traditional VQA methods possess elements that are not pertinent to providing answers during feature fusion. Therefore, in an effort to raise the accuracy of answers, researchers have optimized the VQA model with attention mechanisms in order to augment the relevant beneficial data in visual and textual information [89–91]. Although the accuracy has been augmented through the utilization of attention mechanisms, the opaque, noninterpretable black box model for answering questions remains a mystery. Scholars have begun to question whether VQA models are using data bias in their answers, instead of making true deductions. How to establish an effective and interpretable universal model has always been a prominent research area. Inspired by the fact that humans divide questions into several steps when answering them, the reasoning process of visual question answering can be serialized. Neural module network (NMN) [92] model constructs multiple submodules required for answering questions, such as “Find,” “Transform,” “Combine,” “Describe,” and “Measure”. Afterward, a combination of submodules is automatically generated according to the structure of question for collaborative learning. In 2016, Stanford and Facebook jointly released the CLEVR dataset [93] and proposed a new modular visual reasoning method for the CLEVR dataset [94]. Due to the simple scenario of the dataset, only attention needs to be paid to the reasoning itself. As a result, research on this composite dataset has grown rapidly. Yi et al. [95] proposed neural-symbolic visual question answering (NS-VQA) system, where a scene parser renders an image to obtain a structured scene representation. NS-VQA is a typical model composed of scene parsers (de-renderers), problem parsers (program generators), and program executors, laying the foundation for subsequent visual reasoning models. The central idea of this type of visual question answering method is task decomposition, which is not only applicable to question answering tasks but also to robot navigation tasks, laying the foundation for subsequent embodied question answering.

The long-term aspiration of embodied question answering is to create an agent that is able to perceive its environment, communicate using natural language, and take action to assist humans. Specifically, the task involves placing the agent in an environment, allowing it to experience the environment from a first-person point of view, and having the capability to perform a series of atomic actions, the agent has to explore the environment to gain visual information to answer relevant questions. To answer questions in an embodied form, an agent requires a wide range of AI capabilities, including visual recognition, language understanding, object-oriented navigation, task planning, and commonsense reasoning. Consequently, embodied question answering is deemed the more difficult and complex challenge in current research on embodied AI.

The method employed initially utilized a hierarchical model comprising four modules: vision, language, navigation, and answering. The adaptative computation time (ACT), a navigator of the navigation module, was integrated with encoding visual and linguistic information to choose the next action, predict its duration, and answer the question on the basis of the collected images from multiple frames [64]. Reference [96] proposes a hierarchical-level embodied question answering method based on neural module control, which divides the problem into multiple subtasks, and then specialized systems complete each subtask through navigation and observation. Reference [65] suggested the interactive question answering (IQA) task, which necessitates an augmented level of interaction between the agent and its environment in terms of embodied question answering. In EQA task, the agent mainly collects information through navigation and observation of the environment to answer questions, while in IQA, the agent needs to take action to act on the environment and change its state, which requires more complex planning and reasoning. In addition, there are multitarget embodied question answering tasks and multiagent embodied question answering tasks. In multitarget tasks, the problem involves not only one target, but also multiple targets distributed in different locations in the environment, which adds complexity to reasoning and navigation [66]. In multiagent embodied question answering, multiple agents work in unison to address the question, thus avoiding overburdening a single agent with complex multitarget tasks. Working out how to distribute tasks to each agent in multiagent embodied question answering to avoid duplication of work is vital. Reference [67] introduces a scalable optimization-based planner to assign different viewpoints to each agent and execute tasks on their respective viewpoints.

As we mentioned earlier, these embodied tasks are not independent of each other. As shown in Figure 1, embodied exploration tasks can serve as preparatory work for downstream tasks, providing environmental prior knowledge. Both embodied navigation and exploration tasks can serve the comprehensive task of embodied question and answer.

We have summarized the typical work of three embodied tasks in Table 1, which facilitates readers to clearly understand the types of methods, frameworks, architectures, input data (sensors), benchmarks, and other related information, and Table 2 provides a supplementary key to the abbreviations and specialized symbols used therein. Additionally, training intelligent agents capable of performing embodied tasks requires a suitable environment (embodied environment), while the cost of training real robots in the real world is expensive that requires a lot of time and is difficult to provide environmental diversity. Therefore, most works choose to train in a virtual environment first and then deploy it to the real environment. We summarized some commonly used simulation environments in Table 3, where users can directly control the state of objects in the environment through APIs and interact with objects in the environment through an agent or VR devices.

3.1. LLM

One of the challenges of visual language tasks lies in comprehending natural language and overcoming the bad generalization issues brought about by language diversity. Transformer [111] is the most preferable NLP model at present. What sets Transformer apart from earlier RNN and LSTM models is its reliance on attention mechanism instead of recursion and convolutions for capturing semantic dependencies in sentences, thereby achieving remarkable performance in various NLP tasks. Its computational efficiency and scalability enable model parameters to be expanded to over 100B. Transformer, as an early LLM, had a significant impact on later models. Currently, most LLMs are based on Transformer and are essentially variants of Transformer, for instance, BERT [112] and GPT-3 [113]. BERT is a pretrained language representation model based on the Transformer structure, which is trained on a large corpus using two unsupervised tasks, masked LM and next sentence prediction, and can be finely tuned to downstream tasks. While GPT is an auto-recursive model, GPT-1 was originally developed based on a generative, decoder-only Transformer architecture, using a hybrid method of unsupervised pretraining and supervised fine tuning. Subsequent research into the scaling effect of the model revealed that when the parameters of the model were expanded to a larger size, the model’s performance significantly improved; therefore, when the parameter size of GPT-3 reached 175B, the model showed astonishing emergence ability [114]. Aside from GPT, there are other language models with large-scale parameters such as PaLM [115] and LLaMA [116]. Although there are differences in training process, parameter size, and model structure details (activation functions, embedding layers, etc.), most of them are still based on the transformer structure. We have summarized information about them in Table 4.

3.2. MLM

The LLM has in context learning ability, which is trained through prompt learning and RL from human feedback to learn from human preferences, enabling it to cope with various natural language tasks. Moreover, it also exhibits certain abilities in code generation and solving mathematical problems. Although LLMs exhibit surprising zero/new shot inference performance in most NLP tasks, they are only restricted to discrete text. At the same time, large-scale visual models have experienced accelerated development in terms of visual perception. ViT [120] is a large visual model (LVM) inspired by Transformer, which also eliminates convolution and calculates attention between image regions. It divides the image into small pieces and uses the linear embedding sequence of these small pieces as input to the Transformer, achieving excellent results in image classification tasks and can be widened to multiple other tasks. CLIP [121] is a LVM that combines natural language; during the training process, image text pairs are used instead of image labels to enable the model to learn the correlation between image features and text features, leveraging contrastive learning to hone the model’s general feature representation skill which through the way that boosting the similarity between images and their corresponding text, and decreasing the similarity between images and unrelated text. CLIP’s supervised information acquired via natural language demonstrates how cross-modal learning enhances model training generalization, and its capacity to relate images with text makes it especially suitable for downstream visual language tasks. Visual information and textual information are often complementary by joining together LLMs and LVMs, and it is possible to further enhance the capability of vision–language models in tackling various challenging multimodal tasks. There are two key points in LLM that combined with LVM: (1) The zero-shot/feed-shot capability of the model and (2) the cross-modal alignment capability. Most of the current LVM models utilize pretrained single-mode models, such as pure visual models and pure language models, which can provide models with strong zero-shot/feed-shot capabilities to generalize to new tasks. However, single-mode visual/language models are trained within their respective modes, and how to align cross-modal information is the most critical problem. Presently, two main approaches are direct combination, such as KOSMOS-1 [122], where Transformer is used straightaway to merge textual and visual features, or an additional cross-modal alignment network, like Flamingo [123], and it deploys the Perceiver Resampler and Gated XATTN-dense modules to promote cross-modal information interaction. The Perceiver Resampler is used to generate a fixed number of visual outputs and perform cross-modal alignment through the cross-attention layer in the Gated XATTN-dense module. BLIP-2 [124] fills the modal gap through a lightweight Querying Transformer (Q-Transformer) that directly connects image encoder and LLM. Through two stages of training, the Q-Transformer learns the visual representation extracting and vision-to-language generation of the corresponding text. The Q-Transformer is composed of two modules that share self-attention layers, and it utilizes the cross-attention layer to achieve cross-modal interaction between vision and text as well. ML-MFSL [125] defines a new multimodal few-shot meta learning method to bridge the gap between vision and language patterns. It consists of three components: a vision encoder, a meta mapper, and a language model. The meta mapper, which consists of multiple attention modules, maps the visual encoding to the late space of the language model. During the training process, a set of visual prefixes are added before the visual features extracted from the vision encoder; then, it is fed into the meta mapper to obtain the output results and concatenate them with text features, and sent to LLM later. Overall, the key to cross-modal information alignment lies in finding an appropriate mechanism to deliver visual concepts to language models by accumulating shared knowledge from multimodal tasks. From the perspective of AGI, MLM is more consistent with the way humans comprehend the world and can handle a greater variety of tasks, which has a higher promoting influence on embodied intelligence. We have provided a summary of some widely used MLMs in Table 4.

3.3. Chain of Thought (CoT)

Prompting engineering and fine-tuning LLM has stimulated the model’s emergency capabilities, but these are still insufficient to reach outstanding performance on challenging tasks. OpenAI proposed CoT in 2020, a way that enhances language model reasoning ability and has been validated to be effective in intricate reasoning tasks. It can improve the model performance in a range of mathematical, common sense, and symbolic reasoning tasks [130]. The main idea of CoT seeks to promote LLMs to not only generate the final result, but also to present the thinking process that leads to the answer, just like human mental processes. The construction of CoT includes two modes: (1) filling-based mode and (2) prediction-based mode. Specifically, padding-based patterns require inferring steps between the context (previous and subsequent steps) to fill logical blanks, while prediction-based patterns require extending the reasoning chain with given conditions (instructions and previous reasoning history) [131]. Both patterns are striving for the same goal, which is to ensure that the produced steps are accurate and consistent; moreover, several investigations regarding CoT have proposed broadening single-mode CoT to multimodal CoT (M-CoT), making use of visual data to enhance reasoning [132].

5.1. Task Complexity

A direct indicator of the ability of an embodied intelligent system is to observe how complex the system can perform tasks. Based on results, we categorize current embodied tasks into two types. The first type, embodied actions, necessitates the agent to execute a series of actions culminating in a completion signal. The second, embodied question and answer, similarly requires a series of actions but necessitates further analysis, ultimately returning an answer. Various embodied schemes often establish embodied environments based on virtual environments and agents, exemplified by the task “Bring me the rice chips from the drawer.” Here, the mobile agent needs to: (1) go to the drawers, (2) open the top drawer, and (3) retrieve the rice chip bag and place it on the counter. These tasks are relatively straightforward, focusing on navigation and retrieval without extensive environmental interaction or analysis. Taking SayCan as an example, PaLM-SayCan performed worst on the most challenging long-horizon tasks, where most failures were a result of early termination by the LLM. Therefore, such systems exhibit limited tolerance for unexpected situations, such as items being hidden or environmental settings contradicting the task description. Hence, during training, it is essential to increase task complexity by incorporating these unexpected factors, enabling embodied systems to overcome obstacles and execute tasks smoothly. Completing long-horizon tasks requires long-horizon reasoning over a required order, an abstract understanding of the instruction, and knowledge of both the environment and robot’s capabilities. Understanding complex environments is paramount, yet current schemes hinge on LLM/MLM-based task decomposition mechanisms that harness common sense for basic task planning but fail to grasp specific scenarios. An exemplary embodied intelligent system must function seamlessly across diverse, unforecasted environments, highlighting the necessity of interpreting and executing natural language instructions to augment knowledge transfer and generalization capabilities within intricate settings.

5.2. Model Reasoning Ability

LLM contains substantial real-world knowledge driven by a large amount of data, but lacks robust reasoning ability. CoT can enhance LLM’s reasoning ability by adding gradual reasoning text to the dataset, improving answer generation accuracy. However, acquiring this prompt is not straightforward, necessitating zero-shot training strategies. Model reasoning methods include probability-based and symbol-based methods. LLM’s reasoning stems from a probability-based methodology, where decisions are formulated based on the inherent correlations within data. In contrast to symbol-based methods, this approach boasts robust generalization abilities. However, it fails to endow the model with a genuine understanding of the causal nexus among knowledge, behavior, and environment, thereby yielding biased outcomes influenced by data. Consequently, these models struggle to operate interpretably, robustly, and reliably in genuine contexts. Reasoning types encompass common sense, relational, logical, and counterfactual reasoning, each requiring distinct solutions. Neither probability-based nor symbol-based methods can cover all reasoning types, suggesting a hybrid approach as a valuable paradigm exploration.

5.3. Migration From Simulation to Reality

In comparison with the extensive language or visual language datasets available, robot data remain notably sparse. Acquiring such data is both time-consuming and resource-intensive. To establish a versatile entity model that spans various scenarios and tasks within robot technology, the construction of large-scale datasets and the utilization of high-fidelity simulated environment data as a supplementary to real-world data are imperative. Nonetheless, the paramount issue with relying solely on simulated data lies in the simulation-to-reality gap. Simulation-to-reality adaptation entails the transposition of competencies or behaviors acquired in a simulated environment (cyberspace) to real-world scenarios (physical world). This process entails validating and refining the efficacy of algorithms, models, and control strategies developed in simulations to guarantee their resilient and dependable performance in physical environments. For instance, reference [135] notes that in a simulated kitchen, PaLM-SayCan attained a planning success rate of 84% and an execution rate of 74%, whereas in an authentic kitchen setting, there was a decrement of 3% in planning performance and 14% in execution. The question then arises: How can models trained in simulated environments be generalized to the real world without a significant decrement in execution performance? Crafting world models that closely mirror real-world environments and procuring high-quality data are pivotal to enhancing generalization capabilities.

5.4. Unified Evaluation Criteria

Although numerous benchmarks exist for assessing low-level control strategies, they frequently exhibit substantial disparities in the evaluated skills. Moreover, the objects and scenes incorporated into these benchmark tests are frequently constrained by simulator limitations. To conduct a thorough evaluation of the physical model, it is imperative to employ realistic simulators for benchmark tests that encompass a wide array of skills. Regarding advanced task planners, many benchmarks concentrate on assessing planning capabilities through question answering tasks. However, a more effective approach would be to simultaneously evaluate both high-level task planners and low-level control strategies to execute long-term tasks and measure success rates, rather than merely relying on isolated evaluations of the planners. This holistic approach offers a more comprehensive assessment of the capabilities of embedded intelligence systems.

5.5. Underlying Skill Expansion

In the realm of multimodal LLMs, current embodied intelligence adopts the PPA paradigm, integrating MLM with thought chains for task planning and reasoning. Essentially, it constitutes an integrated and invoked methodology. The primary limitation of the system lies in the scope and capabilities of the underlying skills. Future endeavors to broaden the skill repertoire and enhance their robustness would alleviate this constraint.

PPA, as the cornerstone paradigm of embodied intelligence, is not the definitive framework for general AI. Yet, it marks a promising beginning. Until we unravel the mysteries of the brain, exploring embodied intelligence from the ground up offers valuable insights for cognitive science.