This custom node for ComfyUI offers a streamlined method to enhance VRAM utilization by allowing users to offload model layers to system RAM or other GPUs. It supports multi-GPU setups and provides a flexible way to manage the loading of components like UNet and CLIP, ultimately maximizing the latent space available for processing.
- Enables one-click "Virtual VRAM" management for UNet and CLIP loaders.
- Facilitates multi-GPU integration through the bespoke WanVideoWrapper.
- Offers advanced model-driven allocation options for precise resource management across devices.
Context
This tool, known as ComfyUI-MultiGPU v2, is designed to optimize memory management within the ComfyUI framework. Its primary goal is to enhance the handling of large models by distributing their components across multiple GPUs or system RAM, thereby improving the overall efficiency of image generation workflows.
Key Features & Benefits
The extension provides several practical features that significantly improve the user experience:
- Universal .safetensors Support: Allows seamless integration of all
.safetensorsmodels, facilitating easy model management. - Faster GGUF Inference: Offers improved inference speeds for GGUF models, potentially increasing productivity during image generation tasks.
- Bespoke WanVideoWrapper Integration: Ensures stable and efficient support for video processing tasks, making it easier for users to manage complex workflows.
Advanced Functionalities
ComfyUI-MultiGPU v2 includes advanced functionalities such as two operational modes for resource allocation:
- Normal Mode: Users can easily select a donor device to offload model components, making it user-friendly.
- Expert Mode: For advanced users, this mode allows precise control over how model components are distributed across devices using specific parameters like bytes or ratios, enabling optimal resource utilization.
Practical Benefits
This tool enhances the workflow within ComfyUI by:
- Allowing users to free up GPU VRAM quickly and efficiently, enabling the execution of larger models without performance degradation.
- Facilitating the use of all available VRAM for actual computations, thus improving processing speed and quality of generated images.
- Simplifying the management of multi-GPU setups, which can significantly enhance the capabilities of users working with complex models.
Credits/Acknowledgments
The tool is currently maintained by pollockjj and was originally developed by Alexander Dzhoganov. Special thanks are extended to City96 for their contributions.
