MPEcho: A Melody and Phoneme-Aware Generative Framework for Controllable Cover Song Generation

Arxiv | Web | MPEcho | MPEcho Ckpt | Phonsa | Phonsa Ckpt

This is a cover song generation framework based on Ace-Step.

The paper is accepted by ISMIR 2026.

Data preprocessing

Lyrics Transcription

Please refer SongPrep for lyrics transcription and structure prediction.

Phoneme Alignment

Please refer our proposed Phonsa for lyrics alignment.

---

Precompute for training

The following default result will saved in ./data_preprocessing/training_dataset/precompute_*.

You can manually edit the target path in each processing file.

1. Audio latent and text embedding

   python extract_ace.py
   

2. REPA embedding

   python extract_encoder.py
   

3. Melody

   cd ./data_preprocessing
   python RMVPEF0Predictor.py
   

Training

python trainer_all.py --exp_name [name of the exp] --dataset [path of the training dataset]

Inference

python inference_all.py --ckpt_path [path of the model ckpt] --exp_name [name of the exp]

Citation

If you use MPEcho or Phonsa in your research, please cite:

@misc{lee2026mpechomelodyphonemeawaregenerative,
      title={MPEcho: A Melody and Phoneme-Aware Generative Framework for Controllable Cover Song Generation}, 
      author={Wei-Jaw Lee and Hsuan-Yu Yeh and Ting-Yi Hu and Chih-Pin Tan and Fang-Duo Tsai and Yi-Hsuan Yang},
      year={2026},
      eprint={2607.26698},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2607.26698}, 
}

Acknowledgements

We would like to express our gratitude to the following open-source projects made this project possible:

• Ace-Step
• SongEcho
• RMVPE