This custom node for ComfyUI enhances memory management by enabling one-click "Virtual VRAM" for UNet and CLIP loaders, as well as MultiGPU integration in WanVideoWrapper. It efficiently manages the offloading of model layers to either DRAM or VRAM, optimizing the use of available GPU resources.
- Supports universal .safetensors and GGUF models for seamless integration.
- Offers intuitive model-driven allocation options, allowing for precise control over resource distribution across devices.
- Includes both Normal and Expert modes for layer offloading, catering to different user needs and expertise levels.
Context
This tool is an extension for ComfyUI designed to improve memory management and resource utilization when generating images or videos. By allowing users to offload parts of their models to different devices, it maximizes the available latent space, facilitating more complex computations without overwhelming a single GPU.
Key Features & Benefits
The extension introduces several practical features, including support for universal model formats like .safetensors and GGUF, which simplifies the integration of various models. The model-driven allocation options, including Normal and Expert modes, provide users with flexibility in managing how resources are distributed across their available hardware, enhancing performance without requiring complex configurations.
Advanced Functionalities
The tool features two distinct operational modes: Normal Mode, which allows users to offload layers easily by selecting a donor device, and Expert Mode, which provides advanced users with granular control over how the model is split across devices. This includes specifying exact allocations in bytes, ratios, or fractions, enabling highly customized setups for optimized performance.
Practical Benefits
By utilizing this tool, users can significantly improve their workflow efficiency in ComfyUI. It allows for the execution of larger models by freeing up GPU VRAM, ensuring that the main GPU can focus on actual computation rather than being bogged down by memory constraints. This results in faster processing times and the ability to handle more complex tasks seamlessly.
Credits/Acknowledgments
The current maintenance of this tool is handled by pollockjj, with its original creation credited to Alexander Dzhoganov. Special thanks are extended to City96 for their contributions.
