YuE

A closer look at YuE

YuE, from HKUST's Multimodal Art Projection group, is a family of open foundation models for full-song generation: it tackles long-form music generation, particularly the lyrics-to-song problem, generating up to five minutes of music while maintaining lyrical alignment, coherent musical structure and engaging vocal melodies with appropriate accompaniment. Its technical approach uses track-decoupled next-token prediction, structural progressive conditioning for long-context lyrical alignment, and a multitask, multiphase pre-training recipe, plus redesigned in-context learning for versatile style transfer, such as converting Japanese city pop into an English rap while preserving the accompaniment, and bidirectional generation. YuE2 produces 48 kHz stereo audio without quantization, with model files downloaded from Hugging Face on first use, and tooling including an inference and serving toolkit, ComfyUI nodes with a text-to-music workflow, and Maestro for local song generation.

Editorial guidance based on published documentation. The workflow below is a suggested starting point, not a measured product result.

When to choose YuE

• Researchers and builders that want open full-song generation from lyrics with vocals, instruments and style matching.
• Users that want long-form songs up to five minutes with lyrical alignment and coherent structure.

When to choose something else

• Users without an NVIDIA GPU with BF16 support and 24 GB VRAM, since that is the quick-start hardware requirement.
• Users on Windows, AMD, Apple Silicon or Intel that need a supported path, since those paths are presented as experimental only.

A practical starting workflow

1. Set up Linux with Python 3.12 and an NVIDIA GPU with BF16 support and 24 GB VRAM.
2. Install YuE from the repository and let model files download from Hugging Face on first use.
3. Generate a song from lyrics with the example script, keeping the score, semantic tokens and acoustic latents in the output directory.
4. Use the inference toolkit, ComfyUI nodes or Maestro for serving, workflows or local generation.

Check before committing

• YuE2 needs Linux, Python 3.12 and an NVIDIA GPU with BF16 support and 24 GB VRAM for the quick start.
• Model files download from Hugging Face on first use, so first use needs a download step.
• Windows, AMD, Apple Silicon and Intel paths are experimental, with hardware setup and validation notes to check.

Common questions

Compare approaches for this task

• Suno — 需要把想法快速变成完整歌曲的人：短视频创作者找背景音乐或主题曲、独立音乐人做 demo 小样、播客/品牌方定制片头曲、给朋友写生日歌的普通用户。尤其是需要导出音频文件、发到 Spotify/YouTube 等平台并商用的人，Suno 是目前同类里链路最完整的（下载、分轨、商用授权都可用）。
• Udio — 把 AI 作曲当灵感实验和打磨练习的人：想逐段雕琢一首歌的词曲爱好者、喜欢对比两个版本再选的创作者、想体验 Voices（可复用的人声风格）和风格库混搭的玩家。免费版每天能做 3 首完整长度的歌，不绑卡，适合先免费玩明白再决定。