ComfyUI-LongCat-AudioDiT is a custom extension for ComfyUI that integrates the LongCat-AudioDiT model, enabling high-quality text-to-speech (TTS) and voice cloning capabilities. This tool facilitates the generation of audio from text input and allows users to clone voices from short audio samples without any prior fine-tuning.
- Provides zero-shot TTS capabilities, allowing for immediate audio generation from text.
- Supports voice cloning from short audio clips, enabling users to create customized speech outputs.
- Integrates seamlessly into ComfyUI with advanced features like multi-speaker conversation synthesis and various model precision options.
Context
This extension serves as a bridge between the LongCat-AudioDiT model and ComfyUI, enhancing the platform's functionality by adding advanced audio synthesis capabilities. It allows users to generate high-quality speech audio and clone voices based on short reference audio clips, making it a powerful tool for various applications in audio production and creative projects.
Key Features & Benefits
The tool's primary features include zero-shot TTS, which allows users to input text and receive corresponding audio output without needing extensive training data. Voice cloning capabilities enable the replication of specific voices from brief audio samples, making it useful for personalized applications. Additionally, the multi-speaker synthesis allows for the creation of dialogues with multiple distinct voices, enhancing the realism of audio outputs.
Advanced Functionalities
The extension utilizes a diffusion-based architecture known as DiT (Diffusion Transformer) and incorporates an ODE Euler solver for generating high-fidelity audio. It supports multiple quantization formats (FP8, BF16, FP16, FP32), providing flexibility in performance and quality based on the user's hardware capabilities. The integration of smart caching and automatic model downloading further streamlines the user experience.
Practical Benefits
By incorporating this tool into their workflows, users can significantly enhance their audio generation processes. It improves control over the output quality and allows for efficient voice cloning, which can save time and resources in projects requiring personalized audio content. The seamless integration with ComfyUI also means users can leverage existing workflows while adding sophisticated audio capabilities.
Credits/Acknowledgments
The LongCat-AudioDiT model is developed by Meituan, and the repository is available under the MIT License. Contributions to the project can be found on the original GitHub repository, which serves as a reference for the model and its functionalities.
