PyraMotion

Source Code of NIPS2025 paper "PyraMotion: Attentional Pyramid-Structured Motion Integration for Co-Speech 3D Gesture Synthesis"

Release Status

• Source Code Release
• Dataset Release
• Pretrained APVQVAE Parameters Released
• Pretrained PyraMotion Parameters Released

Installation

Environment Configuration

Python version: 3.8 Cuda Version: 12.2

# Download Source Code
git clone https://github.com/Williamy946/PyraMotion.git
conda create -n pyramo python==3.8
conda activate pyramo
# Install ffmpeg for media processing and libstdcxx-ng for rendering
conda install -c conda-forge libstdcxx-ng ffmpeg
# Install with pip:
pip install -r ./scripts/requirements.txt
pip install 'git+https://github.com/facebookresearch/pytorch3d.git@stable'

Download weights

Download the parameters files and place them into the folder /pretrained

These weights should be orgnized as follows:

<your root>/PyraMotion/
|-- pretrained
|   |-- hands.bin
|   |-- face.bin
|   |-- foot.bin
|   |-- lowerfoot.bin
|   |-- upper_vertex_1layer_710.bin
|   |-- last_420.bin
|   |-- smplx_models
|   |   `-- smplx/SMPLX_NEUTRAL_2020.npz
`   `-- test_sequences

Inference

python pyramotion_demo.py --config ./configs/pyramotion.yaml

Train

Data Preparation

Download the unzip version BEAT2 in path <your root>/PyraMotion/:

Evaluation of Pretrianed Weights

Once you downloaded full BEAT2 dataset, run:

python test.py --config ./configs/pyramotion.yaml

Training of EMAGE

python train.py --config ./configs/pyramotion.yaml

Training of APVQVAE

python train.py --config ./configs/cnn_vqvae_face_30.yaml 

python train.py --config ./configs/cnn_vqvae_hands_30.yaml 

python train.py --config ./configs/cnn_vqvae_lower_30.yaml 

python train.py --config ./configs/cnn_vqvae_lower_foot_30.yaml 

python train.py --config ./configs/cnn_vqvae_upper_30.yaml