Understanding Embodied Plans

This repository contains the code that implements the computational model from the paper "A computational framework for understanding embodied plans". In addition, it contains information about the experiment and analysis scripts that were used to generate the data for the paper.

Getting Started

The codebase contains two sub-repositories, each implementing different realizations of the GenEA framework depending on the task, namely block building and navigation. This separation is necessary because each sub-repository has different dependencies. Despite the separation, this codebase has been designed to provide a consistent workflow. We use the placeholder <task> to specify the task domain: use bb for block building and nav for navigation.

Working with Apptainer (or Singularity)

We have set up the entire codebase so that it can conveniently be run via Apptainer (or Singularity).

Install Apptainer

sudo apt install -y software-properties-common
sudo add-apt-repository -y ppa:apptainer/ppa
sudo apt update
sudo apt install -y apptainer

🖥️ Setup

Download the Apptainer containers (~ 5 mins)

./setup.sh cont_pull  

Download and install all the necessary Python packages (~ 5 mins)

./setup.sh <task> python

Download data for running the experiments and to perform analysis (~ 5 mins)

./setup.sh data

🖥️ System Requirements

This project has been tested on Linux (Ubuntu 18.04, 20.04, and 24.04). However, you should be able to use any operating system supported by Apptainer.

BB: Block Building

Compile the codebase

When running the code for the first time, you will need to compile the CPP files in the codebase. This is done by running the following command (~ 5 mins):

singularity exec cont_init.simg bash -c "cd mlr/share && make cleanAll"
singularity exec cont_init.simg bash -c "cd mlr/share && make"

NOTE: A lot of targets under mlr/share will fail to compile with messages such as ***** FAILED. This is completely fine and expected!

Continue with the following commands:

./compile.sh makefile_make_g_file
./compile.sh makefile_run_physx
./compile.sh makefile_run_risk_test
./compile.sh makefile_run_stability
./compile.sh makefile_run_viewer
./compile.sh makefile_run_ppm_generator

Define block configurations

Each trial of a study is described via a pair of .g files.

The files under the library/init_files directory define initial block configurations whereas those under the library/fin_files directory specify final block configurations.

To view any block configuration, run the following command:

./run.sh bb python run_viewer.py -p <absolute path of the .g file>

You can also generate images of any block configuration by running the following command:

./run.sh bb python run_image_generator.py -p <absolute path of the .g file> -i

Run the model

All key model variables and parameters are defined in the core_utils.py file, including ConfigUtils.IS_PHYSICS_ON which, when set to False, runs the standard TAMP ablation model.

Use the library/exp_data/trials_list.csv file to specify the trials you want to run the model on.

• The first column in this file states the study name (e.g., "action" for the action inference study),
• The second column is the number associated with the trial.
• The last column specifies the first actions that the model should consider (e.g., ['red'] means picking the red block; ['blue','yellow'] implies that the first action will involve picking both the blue and the yellow blocks). ""

To run the model on the studies specified in the trials_list.csv file, run the following command (~ 20-30 mins, depending on the experiment):

./run.sh bb python run_experiment.py -a   # run action inference trials
./run.sh bb python run_experiment.py -g   # run goal inference trials
./run.sh bb python run_experiment.py -e   # run embodiment strategies inference (how many hands) trials

The output of the model is stored in the library/out_robot_data directory. Data from the action, goal, and embodiment strategies inference studies will be stored in the overall_out.csv file. All intermediate files generated during the model run will be stored in the library/intermediate_files directory.

Analyze the results

./run.sh bb python perform_analysis.py -ag  # analyze action-goal inference data
./run.sh bb python perform_analysis.py -e   # analyze embodiment strategies inference data

NAV: Navigation

Define platform configurations

While the platforms themselves are created in Blender (see blender_utils.py), their configurations can be generated programmatically. Please refer to the stimuli.py file in the navigation repository for examples.

Run the model

All key model variables and parameters are defined in the config_utils.py file, including the boolean constant RUN_DYNAMICS that, when set to False, runs the standard TAMP ablation model.

To run the model:

./run.sh nav python main.py -g -si <stim_index>  # run navigation trials

The output of the model is stored in the library/out/model directory. To generate a single CSV file compiling all the data generated by the model, run ModelRunner.combine_results in main.py.

Analyze the results

./run.sh nav python perform_analysis.py -d  # analyze navigation data

Behavioral Studies

The source code for the three experiments reported in the paper can be found using the links below:

• Goal and Action Inference: https://github.com/CNCLgithub/genea_goal_action_psiturk
• Embodiment Strategies: https://github.com/CNCLgithub/genea_hands_psiturk
• Navigation: https://github.com/CNCLgithub/genea_nav_psiturk

All stimuli used in these experiments can be found via this link.

---