Mastering ComfyUI: Setting Up Workflows, Models, and the Environment

ComfyUI is a free application that enables you to generate AI images and videos locally on your machine. It is designed for users who seek greater control than standard prompt-and-generate tools offer, providing visibility and adjustability for each stage of the generation process. While this flexibility introduces a learning curve, this guide supports your journey from installation through to executing and customizing your first workflow.

Prerequisites for Getting Started

To run ComfyUI effectively, your computer requires sufficient GPU resources to handle your chosen models and workflows. More complex models and demanding workflows typically necessitate higher VRAM capacity.

You must also secure the specific model files required by your workflow. Depending on the architecture, this may involve checkpoints, diffusion models, VAEs, text encoders, LoRAs, or other components. These files are typically stored within the ComfyUI/models directory.

Should your local hardware lack the necessary GPU capabilities, you can execute ComfyUI on a remote GPU-powered desktop. This approach shifts the computational load to the remote GPU while allowing you to access the interface from your standard computer.

Installing ComfyUI

For Windows and macOS users, the ComfyUI desktop application is recommended as the most straightforward method for new users. Alternative options include manual installation or using the ComfyUI command-line tool, with the best choice depending on your specific operating system and environment.

Once installation is complete, launch the application to access the interface. Here, you will find the workflow canvas and the tools necessary for creating and managing your workflows.

The Importance of ComfyUI Workflows

A ComfyUI workflow defines the generation process for images or videos, controlling the models, settings, and processing steps that produce the final output.

This structure offers significantly more control than a simple prompt box. You can switch models, integrate LoRAs, utilize input images, tweak generation parameters, upscale results, or insert additional processing steps.

Workflows are also designed to be saved and reused. Rather than reconstructing the same setup repeatedly, you can maintain a preferred workflow and adjust individual settings as needed. Additionally, you can download and adapt workflows created by other community members to fit your requirements.

Understanding ComfyUI Workflows

Workflows consist of interconnected nodes. Each node manages a specific aspect of the generation process, while the connections dictate how data flows between them.

A standard text-to-image workflow might include nodes for loading the model, entering the prompt, initializing image data, generating the image, decoding the result, and saving the file.

• Model loader: Retrieves the model used for generation.
• Text encoder: Translates your prompt into data understandable by the model.
• Sampler: Executes the generation process based on the selected settings.
• VAE: Converts between latent data and the visible image.
• Save Image: Writes the final generated image to your disk.

You are not required to build every workflow from scratch. ComfyUI offers built-in workflow templates, and numerous community workflows are available for download and immediate use.

Loading an Existing Workflow

Using a pre-existing workflow is often the simplest way to begin. ComfyUI includes example workflows for various models and tasks, and community platforms provide extensive additional options.

Many workflow images embed the workflow data within their metadata. You can drag such an image directly into ComfyUI or use the Workflows → Open menu to load it. The nodes and settings will then appear pre-configured on the canvas.

After loading the workflow, verify which models it requires. If files are missing, ComfyUI can identify gaps in supported workflow templates. For other workflows, you may need to locate and install the models independently.

Sourcing Models for ComfyUI

Models are commonly found on repositories such as Hugging Face and Civitai, or on the model's official project page. The critical factor is ensuring the model is compatible with your intended workflow.

Avoid assuming that any model file will work in any workflow. Different model architectures often require specific loaders and supporting files.

Before downloading, verify the following:

• The model architecture and version
• The specific ComfyUI workflow required
• The model file format
• Recommended VRAM and hardware requirements
• Any necessary VAE, text encoder, LoRA, or other supporting files
• The model's license and usage restrictions

ComfyUI supports various model file types, each residing in specific folders. For instance, checkpoints belong in models/checkpoints, LoRAs in models/loras, and VAEs in models/vae. Newer models may utilize folders such as models/diffusion_models and models/text_encoders.

Installing a Model

After downloading a model, place it in the directory specified by the workflow. You can then select it within the corresponding model loader.

For example, a checkpoint might be located in:

ComfyUI/models/checkpoints/

A LoRA might be stored in:

ComfyUI/models/loras/

If the new model does not appear in the list, refresh the interface or restart ComfyUI.

Installing Custom Nodes

Many advanced workflows rely on custom nodes not included in the standard installation. If these dependencies are missing, the workflow may display missing node errors.

ComfyUI includes a Manager for installing custom nodes. Alternatively, nodes can be installed manually by placing their repositories in the custom_nodes directory and installing their respective dependencies.

Only install custom nodes from trusted sources. Custom nodes may contain executable code and carry their own dependencies and security implications.

Running and Modifying Your Workflow

Once the necessary models and custom nodes are installed, review the key settings in your workflow. Begin by checking the model, prompt, image dimensions, and sampling parameters.

When prepared, click the Queue button to execute the workflow. ComfyUI will process each step and generate the output defined by the workflow structure.

You can subsequently modify individual components without rebuilding the workflow. This includes adding LoRAs, connecting input images, changing samplers, adding upscalers, or adjusting other settings to alter the result.

Saving Your Workflows

Save workflows you intend to reuse. A workflow file contains the node graph and settings but does not typically include the model files themselves. Keep a record of the specific models and custom nodes each workflow requires.

This is particularly important when transferring workflows to another computer or cloud desktop. You will likely need to install the same models and custom nodes before the workflow can run correctly.

Try ComfyUI on DaDesktop

There is no need to purchase a new GPU solely to run ComfyUI. If your current hardware lacks sufficient GPU resources, you can utilize a cloud desktop instead, accessing the power only when needed.

DaDesktop offers cloud desktops with dedicated GPU resources optimized for workloads like AI image and video generation. You can install ComfyUI, download desired models, and build custom workflows without adding a dedicated GPU to your local machine.

Learn more about AI image and video generation on DaDesktop. You can also view the available GPUs and select a configuration suited to your models and workflows.