diff --git a/built-in-nodes/BasicScheduler.mdx b/built-in-nodes/BasicScheduler.mdx new file mode 100755 index 000000000..ac4928595 --- /dev/null +++ b/built-in-nodes/BasicScheduler.mdx @@ -0,0 +1,74 @@ +--- +title: "BasicScheduler - ComfyUI Built-in Node Documentation" +description: "The BasicScheduler node is used to compute a sequence of sigma values for diffusion models based on the provided scheduler, model, and denoising parameters." +sidebarTitle: "BasicScheduler" +icon: "circle" +--- + +The `BasicScheduler` node is designed to compute a sequence of sigma values for diffusion models based on the provided scheduler, model, and denoising parameters. It dynamically adjusts the total number of steps based on the denoise factor to fine-tune the diffusion process, providing precise "recipes" for different stages in advanced sampling processes that require fine control (such as multi-stage sampling). + +## Inputs + +| Parameter | Data Type | Input Type | Default | Range | Metaphor Description | Technical Purpose | +| ----------- | ------------- | ---------- | ------- | --------- | ------------------------------ | ---------------------------- | +| `model` | MODEL | Input | - | - | **Canvas Type**: Different canvas materials need different paint formulas | Diffusion model object, determines sigma calculation basis | +| `scheduler` | COMBO[STRING] | Widget | - | 9 options | **Mixing Technique**: Choose how paint concentration changes | Scheduling algorithm, controls noise decay mode | +| `steps` | INT | Widget | 20 | 1-10000 | **Mixing Count**: 20 mixes vs 50 mixes precision difference | Sampling steps, affects generation quality and speed | +| `denoise` | FLOAT | Widget | 1.0 | 0.0-1.0 | **Creation Intensity**: Control level from fine-tuning to repainting | Denoising strength, supports partial repainting scenarios | + +### Scheduler Types + +Based on source code `comfy.samplers.SCHEDULER_NAMES`, supports the following 9 schedulers: + +| Scheduler Name | Characteristics | Use Cases | Noise Decay Pattern | +| -------------------- | -------------------- | ---------------------------- | ---------------------------- | +| **normal** | Standard linear | General scenarios, balanced | Uniform decay | +| **karras** | Smooth transition | High quality, detail-rich | Smooth non-linear decay | +| **exponential** | Exponential decay | Fast generation, efficiency | Exponential rapid decay | +| **sgm_uniform** | SGM uniform | Specific model optimization | SGM optimized decay | +| **simple** | Simple scheduling | Quick testing, basic use | Simplified decay | +| **ddim_uniform** | DDIM uniform | DDIM sampling optimization | DDIM specific decay | +| **beta** | Beta distribution | Special distribution needs | Beta function decay | +| **linear_quadratic** | Linear quadratic | Complex scenario optimization| Quadratic function decay | +| **kl_optimal** | KL optimal | Theoretical optimization | KL divergence optimized decay| + +## Outputs + +| Parameter | Data Type | Output Type | Metaphor Description | Technical Meaning | +| --------- | --------- | ----------- | ---------------------- | -------------------------------- | +| `sigmas` | SIGMAS | Output | **Paint Recipe Chart**: Detailed paint concentration list for step-by-step use | Noise level sequence, guides diffusion model denoising process | + +## Node Role: Artist's Color Mixing Assistant + +Imagine you are an artist creating a clear image from a chaotic mixture of paint (noise). `BasicScheduler` acts like your **professional color mixing assistant**, whose job is to prepare a series of precise paint concentration recipes: + +### Workflow + +- **Step 1**: Use 90% concentration paint (high noise level) +- **Step 2**: Use 80% concentration paint +- **Step 3**: Use 70% concentration paint +- **...** +- **Final Step**: Use 0% concentration (clean canvas, no noise) + +### Color Assistant's Special Skills + +**Different mixing methods (scheduler)**: + +- **"karras" mixing method**: Paint concentration changes very smoothly, like professional artist's gradient technique +- **"exponential" mixing method**: Paint concentration decreases rapidly, suitable for quick creation +- **"linear" mixing method**: Paint concentration decreases uniformly, stable and controllable + +**Fine control (steps)**: + +- **20 mixes**: Quick painting, efficiency priority +- **50 mixes**: Fine painting, quality priority + +**Creation intensity (denoise)**: + +- **1.0 = Complete new creation**: Start completely from blank canvas +- **0.5 = Half transformation**: Keep half of original painting, transform half +- **0.2 = Fine adjustment**: Only make subtle adjustments to original painting + +### Collaboration with Other Nodes + +`BasicScheduler` (Color Assistant) → Prepare Recipe → `SamplerCustom` (Artist) → Actual Painting → Completed Work diff --git a/built-in-nodes/Canny.mdx b/built-in-nodes/Canny.mdx new file mode 100755 index 000000000..86153878d --- /dev/null +++ b/built-in-nodes/Canny.mdx @@ -0,0 +1,47 @@ +--- +title: "Canny - ComfyUI Built-in Node Documentation" +description: "The Canny node used to extract edge lines from photos." +sidebarTitle: "Canny" +icon: "circle" +--- + +Extract all edge lines from photos, like using a pen to outline a photo, drawing out the contours and detail boundaries of objects. + +## Working Principle + +Imagine you are an artist who needs to use a pen to outline a photo. The Canny node acts like an intelligent assistant, helping you decide where to draw lines (edges) and where not to. + +This process is like a screening job: + +- **High threshold** is the "must draw line standard": only very obvious and clear contour lines will be drawn, such as facial contours of people and building frames +- **Low threshold** is the "definitely don't draw line standard": edges that are too weak will be ignored to avoid drawing noise and meaningless lines +- **Middle area**: edges between the two standards will be drawn together if they connect to "must draw lines", but won't be drawn if they are isolated + +The final output is a black and white image, where white parts are detected edge lines and black parts are areas without edges. + +## Inputs + +| Parameter Name | Data Type | Input Type | Default | Range | Function Description | +|------------------|-----------|------------|---------|-----------|----------------------| +| `image` | IMAGE | Input | - | - | Original photo that needs edge extraction | +| `low_threshold` | FLOAT | Widget | 0.4 | 0.01-0.99 | Low threshold, determines how weak edges to ignore. Lower values preserve more details but may produce noise | +| `high_threshold` | FLOAT | Widget | 0.8 | 0.01-0.99 | High threshold, determines how strong edges to preserve. Higher values only keep the most obvious contour lines | + +## Outputs + +| Output Name | Data Type | Description | +|-------------|-----------|-------------| +| `image` | IMAGE | Black and white edge image, white lines are detected edges, black areas are parts without edges | + +## Parameter Comparison + +![Original Image](/images/built-in-nodes/canny/input.webp) + +![Parameter Comparison](/images/built-in-nodes/canny/compare.webp) + +**Common Issues:** + +- Broken edges: Try lowering high threshold +- Too much noise: Raise low threshold +- Missing important details: Lower low threshold +- Edges too rough: Check input image quality and resolution diff --git a/built-in-nodes/CheckpointLoaderSimple.mdx b/built-in-nodes/CheckpointLoaderSimple.mdx new file mode 100755 index 000000000..68f7d6cd2 --- /dev/null +++ b/built-in-nodes/CheckpointLoaderSimple.mdx @@ -0,0 +1,28 @@ +--- +title: "CheckpointLoaderSimple - ComfyUI Built-in Node Documentation" +description: "The CheckpointLoaderSimple node is used to load model files from specified locations and decompose them into three core components: the main model, text encoder, and image encoder/decoder." +sidebarTitle: "CheckpointLoaderSimple" +icon: "circle" +--- + +This is a model loader node that loads model files from specified locations and decomposes them into three core components: the main model, text encoder, and image encoder/decoder. + +This node automatically detects all model files in the `ComfyUI/models/checkpoints` folder, as well as additional paths configured in your `extra_model_paths.yaml` file. + +1. **Model Compatibility**: Ensure the selected model is compatible with your workflow. Different model types (such as SD1.5, SDXL, Flux, etc.) need to be paired with corresponding samplers and other nodes +2. **File Management**: Place model files in the `ComfyUI/models/checkpoints` folder, or configure other paths through extra_model_paths.yaml +3. **Interface Refresh**: If new model files are added while ComfyUI is running, you need to refresh the browser (Ctrl+R) to see the new files in the dropdown list + +## Inputs + +| Parameter | Data Type | Input Type | Default | Range | Description | +|----------------|-----------|------------|---------|-------|-------------| +| `ckpt_name` | STRING | Widget | null | All model files in checkpoints folder | Select the checkpoint model file name to load, which determines the AI model used for subsequent image generation | + +## Outputs + +| Output Name | Data Type | Description | +|-------------|-----------|-------------| +| `MODEL` | MODEL | The main diffusion model used for image denoising generation, the core component of AI image creation | +| `CLIP` | CLIP | The model used for encoding text prompts, converting text descriptions into information that AI can understand | +| `VAE` | VAE | The model used for image encoding and decoding, responsible for converting between pixel space and latent space | diff --git a/built-in-nodes/CheckpointSave.mdx b/built-in-nodes/CheckpointSave.mdx new file mode 100755 index 000000000..2d4c292fc --- /dev/null +++ b/built-in-nodes/CheckpointSave.mdx @@ -0,0 +1,37 @@ +--- +title: "CheckpointSave - ComfyUI Built-in Node Documentation" +description: "The CheckpointSave node is used to save the complete Stable Diffusion model (including UNet, CLIP, and VAE components) as a **.safetensors** format checkpoint file." +sidebarTitle: "CheckpointSave" +icon: "circle" +--- + +The `Save Checkpoint` node is designed to save a complete Stable Diffusion model (including UNet, CLIP, and VAE components) as a **.safetensors** format checkpoint file. + +The Save Checkpoint is primarily used in model merging workflows. After creating a new merged model through nodes like `ModelMergeSimple`, `ModelMergeBlocks`, etc., you can use this node to save the result as a reusable checkpoint file. + +## Inputs + +| Parameter | Data Type | Description | +|-----------|-----------|-------------| +| `model` | MODEL | The model parameter represents the primary model whose state is to be saved. It is essential for capturing the current state of the model for future restoration or analysis. | +| `clip` | CLIP | The clip parameter is intended for the CLIP model associated with the primary model, allowing its state to be saved alongside the main model. | +| `vae` | VAE | The vae parameter is for the Variational Autoencoder (VAE) model, enabling its state to be saved for future use or analysis alongside the main model and CLIP. | +| `filename_prefix` | STRING | This parameter specifies the prefix for the filename under which the checkpoint will be saved. | + +Additionally, the node has two hidden inputs for metadata: + +**prompt (PROMPT)**: Workflow prompt information +**extra_pnginfo (EXTRA_PNGINFO)**: Additional PNG information + +## Outputs + +This node will output a checkpoint file, and the corresponding output file path is `output/checkpoints/` directory + +## Architecture Compatibility + +- Currently fully supported: SDXL, SD3, SVD and other mainstream architectures, see [source code](https://github.com/comfyanonymous/ComfyUI/blob/master/comfy_extras/nodes_model_merging.py#L176-L189) +- Basic support: Other architectures can be saved but without standardized metadata information + +## Related Links + +Related source code: [nodes_model_merging.py#L227](https://github.com/comfyanonymous/ComfyUI/blob/master/comfy_extras/nodes_model_merging.py#L227) diff --git a/built-in-nodes/ClipLoader.mdx b/built-in-nodes/ClipLoader.mdx new file mode 100755 index 000000000..3206d29fb --- /dev/null +++ b/built-in-nodes/ClipLoader.mdx @@ -0,0 +1,79 @@ +--- +title: "ClipLoader - ComfyUI Built-in Node Documentation" +description: "The ClipLoader node is used to load CLIP text encoder models independently." +sidebarTitle: "ClipLoader" +icon: "circle" +--- + +This node is primarily used for loading CLIP text encoder models independently. +The model files can be detected in the following paths: + +- "ComfyUI/models/text_encoders/" +- "ComfyUI/models/clip/" + +> If you save a model after ComfyUI has started, you'll need to refresh the ComfyUI frontend to get the latest model file path list + +Supported model formats: + +- `.ckpt` +- `.pt` +- `.pt2` +- `.bin` +- `.pth` +- `.safetensors` +- `.pkl` +- `.sft` + +For more details on the latest model file loading, please refer to [folder_paths](https://github.com/comfyanonymous/ComfyUI/blob/master/folder_paths.py) + +## Inputs + +| Parameter | Data Type | Description | +|---------------|---------------|-------------| +| `clip_name` | COMBO[STRING] | Specifies the name of the CLIP model to be loaded. This name is used to locate the model file within a predefined directory structure. | +| `type` | COMBO[STRING] | Determines the type of CLIP model to load. As ComfyUI supports more models, new types will be added here. Please check the `CLIPLoader` class definition in [node.py](https://github.com/comfyanonymous/ComfyUI/blob/master/nodes.py) for details. | +| `device` | COMBO[STRING] | Choose the device for loading the CLIP model. `default` will run the model on GPU, while selecting `CPU` will force loading on CPU. | + +### Device Options Explained + +**When to choose "default":** + +- Have sufficient GPU memory +- Want the best performance +- Let the system optimize memory usage automatically + +**When to choose "cpu":** + +- Insufficient GPU memory +- Need to reserve GPU memory for other models (like UNet) +- Running in a low VRAM environment +- Debugging or special purpose needs + +**Performance Impact** + +Running on CPU will be much slower than GPU, but it can save valuable GPU memory for other more important model components. In memory-constrained environments, putting the CLIP model on CPU is a common optimization strategy. + +### Supported Combinations + +| Model Type | Corresponding Encoder | +|------------|---------------------| +| stable_diffusion | clip-l | +| stable_cascade | clip-g | +| sd3 | t5 xxl/ clip-g / clip-l | +| stable_audio | t5 base | +| mochi | t5 xxl | +| cosmos | old t5 xxl | +| lumina2 | gemma 2 2B | +| wan | umt5 xxl | + +As ComfyUI updates, these combinations may expand. For details, please refer to the `CLIPLoader` class definition in [node.py](https://github.com/comfyanonymous/ComfyUI/blob/master/nodes.py) + +## Outputs + +| Parameter | Data Type | Description | +|-----------|-----------|-------------| +| `clip` | CLIP | The loaded CLIP model, ready for use in downstream tasks or further processing. | + +## Additional Notes + +CLIP models play a core role as text encoders in ComfyUI, responsible for converting text prompts into numerical representations that diffusion models can understand. You can think of them as translators, responsible for translating your text into a language that large models can understand. Of course, different models have their own "dialects," so different CLIP encoders are needed between different architectures to complete the text encoding process. diff --git a/built-in-nodes/ClipMergeSimple.mdx b/built-in-nodes/ClipMergeSimple.mdx new file mode 100755 index 000000000..c97e4d0af --- /dev/null +++ b/built-in-nodes/ClipMergeSimple.mdx @@ -0,0 +1,47 @@ +--- +title: "ClipMergeSimple - ComfyUI Built-in Node Documentation" +description: "The ClipMergeSimple node is used to combine two CLIP text encoder models based on a specified ratio." +sidebarTitle: "ClipMergeSimple" +icon: "circle" +--- + +`CLIPMergeSimple` is an advanced model merging node used to combine two CLIP text encoder models based on a specified ratio. + +This node specializes in merging two CLIP models based on a specified ratio, effectively blending their characteristics. It selectively applies patches from one model to another, excluding specific components like position IDs and logit scale, to create a hybrid model that combines features from both source models. + +## Inputs + +| Parameter | Data Type | Description | +|-----------|-----------|-------------| +| `clip1` | CLIP | The first CLIP model to be merged. It serves as the base model for the merging process. | +| `clip2` | CLIP | The second CLIP model to be merged. Its key patches, except for position IDs and logit scale, are applied to the first model based on the specified ratio. | +| `ratio` | FLOAT | Range `0.0 - 1.0`, determines the proportion of features from the second model to blend into the first model. A ratio of 1.0 means fully adopting the second model's features, while 0.0 retains only the first model's features. | + +## Outputs + +| Parameter | Data Type | Description | +|-----------|-----------|-------------| +| `clip` | CLIP | The resulting merged CLIP model, incorporating features from both input models according to the specified ratio. | + +## Merging Mechanism Explained + +### Merging Algorithm + +The node uses weighted averaging to merge the two models: + +1. **Clone Base Model**: First clones `clip1` as the base model +2. **Get Patches**: Obtains all key patches from `clip2` +3. **Filter Special Keys**: Skips keys ending with `.position_ids` and `.logit_scale` +4. **Apply Weighted Merge**: Uses the formula `(1.0 - ratio) * clip1 + ratio * clip2` + +### Ratio Parameter Explained + +- **ratio = 0.0**: Fully uses clip1, ignores clip2 +- **ratio = 0.5**: 50% contribution from each model +- **ratio = 1.0**: Fully uses clip2, ignores clip1 + +## Use Cases + +1. **Model Style Fusion**: Combine characteristics of CLIP models trained on different data +2. **Performance Optimization**: Balance strengths and weaknesses of different models +3. **Experimental Research**: Explore combinations of different CLIP encoders diff --git a/built-in-nodes/ClipSave.mdx b/built-in-nodes/ClipSave.mdx new file mode 100755 index 000000000..d4c6a0a15 --- /dev/null +++ b/built-in-nodes/ClipSave.mdx @@ -0,0 +1,38 @@ +--- +title: "ClipSave - ComfyUI Built-in Node Documentation" +description: "The ClipSave node is used to save CLIP text encoder models in SafeTensors format." +sidebarTitle: "ClipSave" +icon: "circle" +--- + +The `CLIPSave` node is designed for saving CLIP text encoder models in SafeTensors format. This node is part of advanced model merging workflows and is typically used in conjunction with nodes like `CLIPMergeSimple` and `CLIPMergeAdd`. The saved files use the SafeTensors format to ensure security and compatibility. + +## Inputs + +| Parameter | Data Type | Required | Default Value | Description | +|-----------|-----------|----------|---------------|-------------| +| clip | CLIP | Yes | - | The CLIP model to be saved | +| filename_prefix | STRING | Yes | "clip/ComfyUI" | The prefix path for the saved file | +| prompt | PROMPT | Hidden | - | Workflow prompt information (for metadata) | +| extra_pnginfo | EXTRA_PNGINFO | Hidden | - | Additional PNG information (for metadata) | + +## Outputs + +This node has no defined output types. It saves the processed files to the `ComfyUI/output/` folder. + +### Multi-file Saving Strategy + +The node saves different components based on the CLIP model type: + +| Prefix Type | File Suffix | Description | +|------------|-------------|-------------| +| `clip_l.` | `_clip_l` | CLIP-L text encoder | +| `clip_g.` | `_clip_g` | CLIP-G text encoder | +| Empty prefix | No suffix | Other CLIP components | + +## Usage Notes + +1. **File Location**: All files are saved in the `ComfyUI/output/` directory +2. **File Format**: Models are saved in SafeTensors format for security +3. **Metadata**: Includes workflow information and PNG metadata if available +4. **Naming Convention**: Uses the specified prefix plus appropriate suffixes based on model type diff --git a/built-in-nodes/ClipSetLastLayer.mdx b/built-in-nodes/ClipSetLastLayer.mdx new file mode 100755 index 000000000..c12947a59 --- /dev/null +++ b/built-in-nodes/ClipSetLastLayer.mdx @@ -0,0 +1,40 @@ +--- +title: "ClipSetLastLayer - ComfyUI Built-in Node Documentation" +description: "The ClipSetLastLayer node is used to control the processing depth of CLIP models." +sidebarTitle: "ClipSetLastLayer" +icon: "circle" +--- + +`CLIP Set Last Layer` is a core node in ComfyUI for controlling the processing depth of CLIP models. It allows users to precisely control where the CLIP text encoder stops processing, affecting both the depth of text understanding and the style of generated images. + +Imagine the CLIP model as a 24-layer intelligent brain: + +- Shallow layers (1-8): Recognize basic letters and words +- Middle layers (9-16): Understand grammar and sentence structure +- Deep layers (17-24): Grasp abstract concepts and complex semantics + +`CLIP Set Last Layer` works like a **"thinking depth controller"**: + +-1: Use all 24 layers (complete understanding) +-2: Stop at layer 23 (slightly simplified) +-12: Stop at layer 13 (medium understanding) +-24: Use only layer 1 (basic understanding) + +## Inputs + +| Parameter | Data Type | Default | Range | Description | +|-----------|-----------|---------|--------|-------------| +| `clip` | CLIP | - | - | The CLIP model to be modified | +| `stop_at_clip_layer` | INT | -1 | -24 to -1 | Specifies which layer to stop at, -1 uses all layers, -24 uses only the first layer | + +## Outputs + +| Output Name | Data Type | Description | +|-------------|-----------|-------------| +| clip | CLIP | The modified CLIP model with the specified layer set as the last one | + +## Why Set the Last Layer + +- **Performance Optimization**: Like not needing a PhD to understand simple sentences, sometimes shallow understanding is enough and faster +- **Style Control**: Different levels of understanding produce different artistic styles +- **Compatibility**: Some models might perform better at specific layers diff --git a/built-in-nodes/ClipTextEncode.mdx b/built-in-nodes/ClipTextEncode.mdx new file mode 100755 index 000000000..1b3e3e926 --- /dev/null +++ b/built-in-nodes/ClipTextEncode.mdx @@ -0,0 +1,54 @@ +--- +title: "ClipTextEncode - ComfyUI Built-in Node Documentation" +description: "The ClipTextEncode node is used to convert text prompts into AI-understandable 'language' for image generation." +sidebarTitle: "ClipTextEncode" +icon: "circle" +--- + +`CLIP Text Encode (CLIPTextEncode)` acts like a translator, converting your creative text prompts into a special "language" that AI can understand, helping the AI accurately interpret what kind of image you want to create. + +Imagine communicating with a foreign artist - you need a translator to help accurately convey the artwork you want. This node acts as that translator, using the CLIP model (an AI model trained on vast amounts of image-text pairs) to understand your text descriptions and convert them into "instructions" that the AI art model can understand. + +## Inputs + +| Parameter | Data Type | Input Method | Default | Range | Description | +|-----------|-----------|--------------|---------|--------|-------------| +| text | STRING | Text Input | Empty | Any text | Like detailed instructions to an artist, enter your image description here. Supports multi-line text for detailed descriptions. | +| clip | CLIP | Model Selection | None | Loaded CLIP models | Like choosing a specific translator, different CLIP models are like different translators with slightly different understandings of artistic styles. | + +## Outputs + +| Output Name | Data Type | Description | +|-------------|-----------|-------------| +| CONDITIONING | CONDITIONING | These are the translated "painting instructions" containing detailed creative guidance that the AI model can understand. These instructions tell the AI model how to create an image matching your description. | + +## Usage Tips + +1. **Basic Text Prompt Usage** + - Write detailed descriptions like you're writing a short essay + - More specific descriptions lead to more accurate results + - Use English commas to separate different descriptive elements + +2. **Special Feature: Using Embedding Models** + - Embedding models are like preset art style packages that can quickly apply specific artistic effects + - Currently supports .safetensors, .pt, and .bin file formats, and you don't necessarily need to use the complete model name + - How to use: + 1. Place the embedding model file (in .pt format) in the `ComfyUI/models/embeddings` folder + 2. Use `embedding:model_name` in your text + Example: If you have a model called `EasyNegative.pt`, you can use it like this: + + ``` + a beautiful landscape, embedding:EasyNegative, high quality + ``` + +3. **Prompt Weight Adjustment** + - Use parentheses to adjust the importance of certain descriptions + - For example: `(beautiful:1.2)` will make the "beautiful" feature more prominent + - Regular parentheses `()` have a default weight of 1.1 + - Use keyboard shortcuts `ctrl + up/down arrow` to quickly adjust weights + - The weight adjustment step size can be modified in settings + +4. **Important Notes** + - Ensure the CLIP model is properly loaded + - Use positive and clear text descriptions + - When using embedding models, make sure the file name is correct and compatible with your current main model's architecture diff --git a/built-in-nodes/ClipTextEncodeFlux.mdx b/built-in-nodes/ClipTextEncodeFlux.mdx new file mode 100644 index 000000000..31ef456d9 --- /dev/null +++ b/built-in-nodes/ClipTextEncodeFlux.mdx @@ -0,0 +1,49 @@ +--- +title: "ClipTextEncodeFlux - ComfyUI Built-in Node Documentation" +description: "The ClipTextEncodeFlux node is used to encode text prompts into Flux-compatible conditioning embeddings." +sidebarTitle: "ClipTextEncodeFlux" +icon: "circle" +--- + +`CLIPTextEncodeFlux` is an advanced text encoding node in ComfyUI, specifically designed for the Flux architecture. It uses a dual-encoder mechanism (CLIP-L and T5XXL) to process both structured keywords and detailed natural language descriptions, providing the Flux model with more accurate and comprehensive text understanding for improved text-to-image generation quality. + +This node is based on a dual-encoder collaboration mechanism: +1. The `clip_l` input is processed by the CLIP-L encoder, extracting style, theme, and other keyword features—ideal for concise descriptions. +2. The `t5xxl` input is processed by the T5XXL encoder, which excels at understanding complex and detailed natural language scene descriptions. +3. The outputs from both encoders are fused, and combined with the `guidance` parameter to generate unified conditioning embeddings (`CONDITIONING`) for downstream Flux sampler nodes, controlling how closely the generated content matches the text description. + +## Inputs + +| Parameter | Data Type | Input Method | Default | Range | Description | +|-----------|----------|-------------|---------|-------|-------------| +| `clip` | CLIP | Node input | None | - | Must be a CLIP model supporting the Flux architecture, including both CLIP-L and T5XXL encoders | +| `clip_l` | STRING | Text box | None | Up to 77 tokens | Suitable for concise keyword descriptions, such as style or theme | +| `t5xxl` | STRING | Text box | None | Nearly unlimited | Suitable for detailed natural language descriptions, expressing complex scenes and details | +| `guidance`| FLOAT | Slider | 3.5 | 0.0 - 100.0 | Controls the influence of text conditions on the generation process; higher values mean stricter adherence to the text | + +## Outputs + +| Output Name | Data Type | Description | +|--------------|-------------|-------------| +| `CONDITIONING` | CONDITIONING | Contains the fused embeddings from both encoders and the guidance parameter, used for conditional image generation | + +## Usage Examples + +### Prompt Examples + +- **clip_l input** (keyword style): + - Use structured, concise keyword combinations + - Example: `masterpiece, best quality, portrait, oil painting, dramatic lighting` + - Focus on style, quality, and main subject + +- **t5xxl input** (natural language description): + - Use complete, fluent scene descriptions + - Example: `A highly detailed portrait in oil painting style, featuring dramatic chiaroscuro lighting that creates deep shadows and bright highlights, emphasizing the subject's features with renaissance-inspired composition.` + - Focus on scene details, spatial relationships, and lighting effects + +### Notes + +1. Make sure to use a CLIP model compatible with the Flux architecture +2. It is recommended to fill in both `clip_l` and `t5xxl` to leverage the dual-encoder advantage +3. Note the 77-token limit for `clip_l` +4. Adjust the `guidance` parameter based on the generated results diff --git a/built-in-nodes/overview.mdx b/built-in-nodes/overview.mdx index c6378f620..50473aa5f 100644 --- a/built-in-nodes/overview.mdx +++ b/built-in-nodes/overview.mdx @@ -6,6 +6,10 @@ sidebarTitle: "Overview" Built-in nodes are ComfyUI's default nodes. They are core functionalities of ComfyUI that you can use without installing any third-party custom node packages. -As we have just started updating this section, the content is not yet complete. We will gradually add more content in the future. +## About built-in node document -If you find any errors in the content, you can submit an issue or PR in this [repo](https://github.com/Comfy-Org/docs) to help us improve. \ No newline at end of file +We have now added built-in node help documentation, so the content of this section is periodically synced from [this repo](https://github.com/Comfy-Org/embedded-docs). We will update the content manually once a week. + +## Contribute + +If you find any errors in the content, or want to contribute missing content, please submit an issue or PR in [this repo](https://github.com/Comfy-Org/embedded-docs) to help us improve. \ No newline at end of file diff --git a/docs.json b/docs.json index 43ab81f25..fbda44846 100644 --- a/docs.json +++ b/docs.json @@ -235,19 +235,57 @@ { "group": "conditioning", "pages": [ - "built-in-nodes/conditioning/video-models/wan-vace-to-video" + "built-in-nodes/ClipSetLastLayer", + "built-in-nodes/ClipTextEncode", + { + "group": "Video Models", + "pages": [ + "built-in-nodes/conditioning/video-models/wan-vace-to-video" + ] + } + ] + }, + { + "group": "Image", + "pages": [ + "built-in-nodes/Canny" + ] + }, + { + "group": "Loader", + "pages": [ + "built-in-nodes/CheckpointLoaderSimple", + "built-in-nodes/ClipLoader" ] }, { - "group": "latent", + "group": "Latent", "pages": [ "built-in-nodes/latent/video/trim-video-latent" ] + },{ + "group": "Advanced", + "pages": [ + "built-in-nodes/ClipMergeSimple", + "built-in-nodes/ClipSave", + { + "group": "Conditioning", + "pages": [ + "built-in-nodes/ClipTextEncodeFlux" + ] + } + ] }, { - "group": "samling", + "group": "Sampling", "pages": [ - "built-in-nodes/samling/ksampler" + "built-in-nodes/samling/ksampler", + { + "group": "Custom Sampling", + "pages": [ + "built-in-nodes/BasicScheduler" + ] + } ] }, { @@ -697,28 +735,66 @@ "pages": [ "zh-CN/built-in-nodes/overview", { - "group": "conditioning", + "group": "条件", "pages": [ - "zh-CN/built-in-nodes/conditioning/video-models/wan-vace-to-video" + "zh-CN/built-in-nodes/ClipSetLastLayer", + "zh-CN/built-in-nodes/ClipTextEncode", + { + "group": "视频模型", + "pages": [ + "zh-CN/built-in-nodes/conditioning/video-models/wan-vace-to-video" + ] + } + ] + }, + { + "group": "图像", + "pages": [ + "zh-CN/built-in-nodes/Canny" + ] + }, + { + "group": "加载器", + "pages": [ + "zh-CN/built-in-nodes/CheckpointLoaderSimple", + "zh-CN/built-in-nodes/ClipLoader" ] }, { - "group": "latent", + "group": "潜变量", "pages": [ "zh-CN/built-in-nodes/latent/video/trim-video-latent" ] + },{ + "group": "高级", + "pages": [ + "zh-CN/built-in-nodes/ClipMergeSimple", + "zh-CN/built-in-nodes/ClipSave", + { + "group": "条件", + "pages": [ + "zh-CN/built-in-nodes/ClipTextEncodeFlux" + ] + } + ] }, { - "group": "samling", + "group": "采样", "pages": [ - "zh-CN/built-in-nodes/sampling/ksampler" + "zh-CN/built-in-nodes/sampling/ksampler", + { + "group": "自定义采样", + "pages": [ + "zh-CN/built-in-nodes/BasicScheduler" + ] + } ] }, { - "group": "API Node", + "group": "API 节点", "pages": [ { - "group": "Image", + "group": "图像", "pages": [ { "group": "BFL", diff --git a/images/built-in-nodes/canny/compare.webp b/images/built-in-nodes/canny/compare.webp new file mode 100644 index 000000000..305dc9f7e Binary files /dev/null and b/images/built-in-nodes/canny/compare.webp differ diff --git a/images/built-in-nodes/canny/input.webp b/images/built-in-nodes/canny/input.webp new file mode 100644 index 000000000..471ef2ad9 Binary files /dev/null and b/images/built-in-nodes/canny/input.webp differ diff --git a/zh-CN/built-in-nodes/BasicScheduler.mdx b/zh-CN/built-in-nodes/BasicScheduler.mdx new file mode 100644 index 000000000..e0520ab59 --- /dev/null +++ b/zh-CN/built-in-nodes/BasicScheduler.mdx @@ -0,0 +1,74 @@ +--- +title: "BasicScheduler - ComfyUI 原生节点文档" +description: "`BasicScheduler` 节点旨在根据提供的调度器、模型和去噪参数为扩散模型计算一系列 sigma 值。它根据去噪因子动态调整总步骤数,以微调扩散过程,在一些需要精细控制的高级的采样过程" +sidebarTitle: "BasicScheduler" +icon: "circle" +--- + +`BasicScheduler` 节点旨在根据提供的调度器、模型和去噪参数为扩散模型计算一系列 sigma 值。它根据去噪因子动态调整总步骤数,以微调扩散过程,在一些需要精细控制的高级的采样过程(比如分步采样)等等提供了精细的不同阶段的“配方” + +## 输入 + +| 参数名称 | 数据类型 | 输入类型 | 默认值 | 范围 | 比喻说明 | 技术作用 | +| ----------- | ------------- | -------- | ------ | ------- | ---------------------------------------------- | ------------------------------ | +| `模型` | MODEL | Input | - | - | **画布类型**:不同材质的画布需要不同的颜料配方 | 扩散模型对象,决定sigma值的计算基础 | +| `调度器` | COMBO[STRING] | Widget | - | 9种选项 | **调色技法**:选择颜料浓度变化的方式 | 调度算法,控制噪声衰减模式 | +| `步数` | INT | Widget | 20 | 1-10000 | **调色次数**:调色20次 vs 50次的精细度差异 | 采样步数,影响生成质量与速度 | +| `降噪` | FLOAT | Widget | 1.0 | 0.0-1.0 | **创作强度**:从微调到重绘的控制力度 | 去噪强度,支持部分重绘场景 | + +### 调度器类型详解 + +基于源码 `comfy.samplers.SCHEDULER_NAMES`,支持以下9种调度器: + +| 调度器名称 | 特点 | 适用场景 | 噪声衰减特性 | +| -------------------- | ------------ | -------------------- | -------------- | +| **normal** | 标准线性调度 | 通用场景,平衡效果 | 均匀递减 | +| **karras** | 平滑过渡调度 | 高质量生成,细节丰富 | 平滑非线性递减 | +| **exponential** | 指数递减调度 | 快速生成,效率优先 | 指数型快速递减 | +| **sgm_uniform** | SGM均匀调度 | 特定模型优化 | SGM优化递减 | +| **simple** | 简单调度 | 快速测试,基础应用 | 简化递减 | +| **ddim_uniform** | DDIM均匀调度 | DDIM采样优化 | DDIM特定递减 | +| **beta** | Beta分布调度 | 特殊分布需求 | Beta函数递减 | +| **linear_quadratic** | 线性二次调度 | 复杂场景优化 | 二次函数递减 | +| **kl_optimal** | KL最优调度 | 理论最优化 | KL散度优化递减 | + +## 输出结果 + +| 参数名称 | 数据类型 | 输出类型 | 比喻说明 | 技术含义 | +| -------- | -------- | -------- | -------------------------------------------------- | ------------------------------ | +| `sigmas` | SIGMAS | Output | **调色配方表**:详细的颜料浓度清单,供画家逐步使用 | 噪声水平序列,指导扩散模型的去噪过程 | + +## 节点角色:画家的调色助手 + +想象你是一位画家,正在从一团混乱的颜料(噪声)创作清晰的图像。`BasicScheduler` 就像是的**专业调色助手**,它的工作是为您准备一系列精确的颜料浓度配方: + +### 工作流程 + +- **第1步**:使用90%浓度的颜料(高噪声水平) +- **第2步**:使用80%浓度的颜料 +- **第3步**:使用70%浓度的颜料 +- **...** +- **最后一步**:使用0%浓度(纯净画布,无噪声) + +### 调色助手的特殊技能 + +**不同的调色方法(scheduler)**: + +- **"karras"调色法**:颜料浓度变化非常平滑,像专业画家的渐变技巧 +- **"exponential"调色法**:颜料浓度快速递减,适合快速创作 +- **"linear"调色法**:颜料浓度均匀递减,稳定可控 + +**精细控制(steps)**: + +- **20次调色**:快速作画,效率优先 +- **50次调色**:精细作画,质量优先 + +**创作强度(denoise)**: + +- **1.0 = 全新创作**:完全从空白画布开始 +- **0.5 = 半改造**:保留原画一半,改造一半 +- **0.2 = 微调优化**:只对原画进行细微调整 + +### 与其他节点的配合 + +`BasicScheduler`(调色助手)→ 准备配方 → `SamplerCustom`(画家)→ 实际绘画 → 完成作品 diff --git a/zh-CN/built-in-nodes/Canny.mdx b/zh-CN/built-in-nodes/Canny.mdx new file mode 100644 index 000000000..4c473932d --- /dev/null +++ b/zh-CN/built-in-nodes/Canny.mdx @@ -0,0 +1,47 @@ +--- +title: "Canny - ComfyUI 原生节点文档" +description: "Canny 节点是 ComfyUI 中用于提取图像边缘的节点。" +sidebarTitle: "Canny" +icon: "circle" +--- + +从照片中提取所有边缘线条,就像用钢笔为照片描边一样,把物体的轮廓和细节边界都画出来。 + +## 工作原理 + +想象您是一位画家,要用钢笔为一张照片描边。Canny节点就像一个智能助手,帮您决定哪些地方需要画线(边缘),哪些地方不需要。 + +这个过程就像筛选工作: + +- **高阈值**是"必须画线的标准":只有非常明显、清晰的轮廓线才会被画出来,比如人物脸部轮廓、建筑物边框 +- **低阈值**是"完全不画线的标准":太微弱的边缘会被忽略,避免画出噪点和无意义的线条 +- **中间区域**:介于两个标准之间的边缘,如果连着"必须画的线"就一起画出来,如果是孤立的就不画 + +最终输出一张黑白图像,白色部分是检测到的边缘线条,黑色部分是没有边缘的区域。 + +## 输入 + +| 参数名称 | 数据类型 | 输入方式 | 默认值 | 取值范围 | 功能说明 | +| -------- | -------- | -------- | ------ | --------- | ---------------------------------------------------------- | +| 图像 | IMAGE | 连接 | - | - | 需要提取边缘的原始照片 | +| 低阈值 | FLOAT | 手动输入 | 0.4 | 0.01-0.99 | 低阈值,决定忽略多弱的边缘。数值越小保留的细节越多,但可能产生噪点 | +| 高阈值 | FLOAT | 手动输入 | 0.8 | 0.01-0.99 | 高阈值,决定保留多强的边缘。数值越大只保留最明显的轮廓线 | + +## 输出 + +| 输出名称 | 数据类型 | 说明 | +| -------- | -------- | ---------------------------------------------------------- | +| 图像 | IMAGE | 黑白边缘图像,白色线条为检测到的边缘,黑色区域为无边缘部分 | + +## 参数对比 + +![原图](/images/built-in-nodes/canny/input.webp) + +![参数对比](/images/built-in-nodes/canny/compare.webp) + +**常见问题:** + +- 边缘断断续续:尝试降低高阈值 +- 出现很多噪点:提高低阈值 +- 丢失重要细节:降低低阈值 +- 边缘太粗糙:检查输入图像质量和分辨率 diff --git a/zh-CN/built-in-nodes/CheckpointLoaderSimple.mdx b/zh-CN/built-in-nodes/CheckpointLoaderSimple.mdx new file mode 100644 index 000000000..69a366487 --- /dev/null +++ b/zh-CN/built-in-nodes/CheckpointLoaderSimple.mdx @@ -0,0 +1,29 @@ +--- +title: "CheckpointLoaderSimple - ComfyUI 原生节点文档" +description: "CheckpointLoaderSimple 节点是 ComfyUI 中用于加载模型的节点。" +sidebarTitle: "CheckpointLoaderSimple" +icon: "circle" +--- + +这是一个模型加载器节点,用于从指定位置加载模型文件,并将其分解为三个核心组件:主模型、文本编码器和图像编解码器。 + +这个节点会自动检测`ComfyUI/models/checkpoints`文件夹下的所有模型文件,以及你在extra_model_paths.yaml文件中配置的额外路径。 + +1. **模型兼容性**:确保所选模型与你的工作流程兼容,不同类型的模型(如SD1.5、SDXL、Flux等)需要配合相应的采样器和其他节点 +2. **文件管理**:将模型文件放在`ComfyUI/models/checkpoints`文件夹中,或通过extra_model_paths.yaml配置其他路径 +3. **界面刷新**:如果在ComfyUI运行期间添加了新的模型文件,需要刷新浏览器(Ctrl+R)才能在下拉列表中看到新文件 + +## 输入 + +| 参数名称 | 数据类型 | 输入方式 | 默认值 | 取值范围 | 功能说明 | +|----------|----------|----------|---------|----------|----------| +| checkpoint名称 | STRING | 下拉选择 | null | checkpoints文件夹中的所有模型文件 | 选择要加载的检查点模型文件名称,决定了后续生图使用的AI模型 | + +## 输出 + +| 输出名称 | 数据类型 | 说明 | +|----------|----------|------| +| 模型 | MODEL | 用于图像去噪生成的主要扩散模型,是AI绘画的核心组件 | +| CLIP | CLIP | 用于编码文本提示词的模型,将文字描述转换为AI能理解的信息 | +| VAE | VAE | 用于图像编解码的模型,负责在像素空间和潜在空间之间转换 | + diff --git a/zh-CN/built-in-nodes/CheckpointSave.mdx b/zh-CN/built-in-nodes/CheckpointSave.mdx new file mode 100644 index 000000000..18d64925f --- /dev/null +++ b/zh-CN/built-in-nodes/CheckpointSave.mdx @@ -0,0 +1,39 @@ +--- +title: "保存Checkpoint - ComfyUI 原生节点文档" +description: "保存Checkpoint 节点是 ComfyUI 中用于保存模型的节点。" +sidebarTitle: "保存Checkpoint" +icon: "circle" +--- + + +`保存Checkpoint` 节点的作用是将完整的 Stable Diffusion 模型(包括 UNet、CLIP 和 VAE 组件)保存为 **.safetensors** 格式的检查点文件。 + +CheckpointSave 主要用于模型合并工作流中,当你通过 `ModelMergeSimple`、`ModelMergeBlocks` 等节点创建了新的合并模型后,可以使用此节点将结果保存为可重复使用的检查点文件。 + +## 输入 + +| 参数名称 | 数据类型 | 作用 | +| ------- | -------- | ------------------------------------------------------------ | +| `模型` | MODEL | 表示要保存状态的主要模型。捕获模型的当前状态以供将来恢复或分析至关重要。 | +| `clip` | CLIP | 与主要模型相关联的 CLIP 模型的参数,允许其状态与主模型一起保存。 | +| `vae` | VAE | 变分自编码器(VAE)模型的参数,使其状态可以与主模型和 CLIP 一起保存以供将来使用或分析。 | +| `文件名前缀` | STRING | 指定保存检查点的文件名前缀。 | + +另外节点还存在两个隐藏输入用于元数据: + +**prompt (PROMPT)**: 工作流提示信息 +**extra_pnginfo (EXTRA_PNGINFO)**: 额外的 PNG 信息 + +## 输出 + +这个节点将会输出一个 checkpoint 文件, 对应的输出文件路径为 `output/checkpoints/` 目录 + +## 架构兼容性 + +- 目前完全支持: SDXL、SD3、SVD 等主流架构,查看[源码](https://github.com/comfyanonymous/ComfyUI/blob/master/comfy_extras/nodes_model_merging.py#L176-L189) +- 基础支持:其它架构认可保存,但不会添加标准化的元数据信息 + +## 相关链接 + +相关源码: [nodes_model_merging.py#L227](https://github.com/comfyanonymous/ComfyUI/blob/master/comfy_extras/nodes_model_merging.py#L227) + diff --git a/zh-CN/built-in-nodes/ClipLoader.mdx b/zh-CN/built-in-nodes/ClipLoader.mdx new file mode 100644 index 000000000..5fd0b58d2 --- /dev/null +++ b/zh-CN/built-in-nodes/ClipLoader.mdx @@ -0,0 +1,79 @@ +--- +title: "加载CLIP - ComfyUI 原生节点文档" +description: "`加载CLIP ` 节点主要用于单独加载 CLIP 文本编码器模型。" +sidebarTitle: "加载CLIP" +icon: "circle" +--- + +该节点主要用于单独加载 CLIP 文本编码器模型。 +支持检测以下路径的模型文件检测: + +- “ComfyUI/models/text_encoders/” +- “ComfyUI/models/clip/” + +> 如果你是在 ComfyUI 启动后才保存的模型则需要刷新 ComfyUI 前端来获取最新的模型文件路径列表 + +支持的模型格式有: + +- `.ckpt` +- `.pt` +- `.pt2` +- `.bin` +- `.pth` +- `.safetensors` +- `.pkl` +- `.sft` + +更多最新模型文件加载详情请查阅[folder_paths](https://github.com/comfyanonymous/ComfyUI/blob/master/folder_paths.py) + +## 输入 + +| 参数名称 | 数据类型 | 作用 | +| ------------ | -------- | ------------------------------------------------------------ | +| `CLIP名称` | COMBO[STRING] | 指定要加载的 CLIP 模型的名称。此名称用于在预定义的目录结构内定位模型文件。 | +| `类型` | COMBO[STRING] | 确定要加载的 CLIP 模型类型,随着 ComfyUI 支持的模型数量增加这里的类型也会新增对应的类型,请查看[node.py](https://github.com/comfyanonymous/ComfyUI/blob/master/nodes.py)中源码里关于`CLIPLoader` 类的相关定义| +| `设备` | COMBO[STRING] |选择加载 CLIP 模型的设备,`default` 将会将对应的模型在 GPU 上运行,如果选择`CPU` 将强制在 CPU上进行加载| + +### 不同`设备`选项的说明 + +**选择 "default" 的情况**: + +- 有足够的 GPU 内存 +- 希望获得最佳性能 +- 让系统自动优化内存使用 + +**选择 "cpu" 的情况**: + +- GPU 内存不足 +- 需要为其他模型(如 UNet)保留 GPU 内存 +- 在低 VRAM 环境下运行 +- 调试或特殊用途需要 + +**性能影响** + +CPU 运行会比 GPU 运行慢很多,但可以节省宝贵的 GPU 内存供其他更重要的模型组件使用。在内存受限的环境中,将 CLIP 模型放在 CPU 上是一个常见的优化策略。 + +### 支持的搭配 + +| 模型类型 | 对应编码器 | +|----------|------------| +| stable_diffusion | clip-l | +| stable_cascade | clip-g | +| sd3 | t5 xxl/ clip-g / clip-l | +| stable_audio | t5 base | +| mochi | t5 xxl | +| cosmos | old t5 xxl | +| lumina2 | gemma 2 2B | +| wan | umt5 xxl | + +未来随着 ComfyUI 的更新,这个更新搭配可能会新增,详情请参考 [node.py](https://github.com/comfyanonymous/ComfyUI/blob/master/nodes.py)中源码里关于`CLIPLoader` 类的相关定义说明 + +## 输出 + +| 参数名称 | 数据类型 | 作用 | +| -------- | -------- | ------------------------------------------ | +| `clip` | CLIP | 加载的 CLIP 模型,准备用于下游任务或进一步处理。 | + +## 其它扩展 + +CLIP 模型在 ComfyUI 中扮演着文本编码器的核心角色,负责将文本提示转换为可供扩散模型理解的数值表示,你可以把它理解成翻译官,负责将你的文本翻译成大模型可以理解的语言,当然不同模型也存在着 “方言” ,所以在不同架构的模型之间需要不同的 CLIP 模型来完成文本编码的这一过程。 diff --git a/zh-CN/built-in-nodes/ClipMergeSimple.mdx b/zh-CN/built-in-nodes/ClipMergeSimple.mdx new file mode 100644 index 000000000..a934c8495 --- /dev/null +++ b/zh-CN/built-in-nodes/ClipMergeSimple.mdx @@ -0,0 +1,47 @@ +--- +title: "CLIP融合简易 - ComfyUI 原生节点文档" +description: "`CLIP融合简易` 是一个高级模型合并节点,用于将两个 CLIP 文本编码器模型按指定比例进行合并." +sidebarTitle: "CLIP融合简易" +icon: "circle" +--- + +`CLIP融合简易` 是一个高级模型合并节点,用于将两个 CLIP 文本编码器模型按指定比例进行合并. + +此节点专门用于根据指定比例合并两个CLIP模型,有效地混合它们的特性。它有选择性地将一个模型的补丁应用到另一个模型上,排除像位置ID和对数尺度这样的特定组件,以创建一个结合了两个源模型特征的混合模型。 + +## 输入 + +| 参数名称 | 数据类型 | 作用 | +|----------|----------|------| +| `clip1` | CLIP | 要合并的第一个CLIP模型。它作为合并过程的基础模型。 | +| `clip2` | CLIP | 要合并的第二个CLIP模型。根据指定的比例,除位置ID和对数尺度外,其关键补丁将应用于第一个模型。 | +| `比例` | FLOAT | 取值范围 `0.0 - 1.0` 确定从第二个模型融合到第一个模型中的特性比例。比例为1.0意味着完全采用第二个模型的特性,而0.0则仅保留第一个模型的特性。 | + +## 输出 + +| 参数名称 | 数据类型 | 作用 | +|----------|----------|------| +| `clip` | CLIP | 结果合并的CLIP模型,根据指定的比例整合了两个输入模型的特性。 | + +## 合并机制详解 + +### 合并算法 + +节点使用加权平均的方式合并两个模型: + +1. **克隆基础模型**: 首先克隆 `clip1` 作为基础模型 +2. **获取补丁**: 从 `clip2` 获取所有键值补丁 (key patches) +3. **过滤特殊键**: 跳过 `.position_ids` 和 `.logit_scale` 结尾的键 +4. **应用加权合并**: 使用公式 `(1.0 - 比例) * clip1 + 比例 * clip2` + +### 比例 参数说明 + +- **比例 = 0.0**: 完全使用 clip1,忽略 clip2 +- **比例 = 0.5**: 两个模型各占 50% +- **比例 = 1.0**: 完全使用 clip2,忽略 clip1 + +## 使用场景 + +1. **模型风格融合**: 结合不同训练数据的 CLIP 模型特点 +2. **性能优化**: 平衡不同模型的优缺点 +3. **实验研究**: 探索不同 CLIP 编码器的组合效果 diff --git a/zh-CN/built-in-nodes/ClipSave.mdx b/zh-CN/built-in-nodes/ClipSave.mdx new file mode 100644 index 000000000..1d738d5d1 --- /dev/null +++ b/zh-CN/built-in-nodes/ClipSave.mdx @@ -0,0 +1,31 @@ +--- +title: "保存CLIP - ComfyUI 原生节点文档" +description: "`CLIP保存` 节点用于将 CLIP 文本编码器模型保存为 SafeTensors 格式文件, 该节点属于高级模型合并工作流的一部分,通常与 `CLIPMergeSimple`、`CLIPMergeAdd` 等节点配合使用。保存的文件采用 SafeTensors 格式,确保安全性和兼容性。" +sidebarTitle: "保存CLIP" +icon: "circle" +--- + +`CLIP保存` 节点用于将 CLIP 文本编码器模型保存为 SafeTensors 格式文件, 该节点属于高级模型合并工作流的一部分,通常与 `CLIPMergeSimple`、`CLIPMergeAdd` 等节点配合使用。保存的文件采用 SafeTensors 格式,确保安全性和兼容性。 + +## 输入 + +| 参数名 | 类型 | 是否必需 | 默认值 | 描述 | +|--------|------|----------|--------|------| +| `clip` | CLIP | 必需 | - | 要保存的 CLIP 模型 | +| `文件名前缀` | STRING | 必需 | `"clip/ComfyUI"` | 保存文件的前缀路径 | +| `prompt` | PROMPT | 隐藏参数 | - | 工作流提示信息(用于元数据) | +| `extra_pnginfo` | EXTRA_PNGINFO | 隐藏参数 | - | 额外的PNG信息(用于元数据) | + +## 输出 + +该节点没有定义输出类型, 会将处理后的文件保存到 `ComfyUI/output/` 文件夹下 + +### 多文件保存策略 + +节点会根据 CLIP 模型类型分别保存不同组件: + +| 前缀类型 | 文件名后缀 | 说明 | +|----------|------------|------| +| `clip_l.` | `_clip_l` | CLIP-L 文本编码器 | +| `clip_g.` | `_clip_g` | CLIP-G 文本编码器 | +| 空前缀 | 无后缀 | 其他 CLIP 组件 | diff --git a/zh-CN/built-in-nodes/ClipSetLastLayer.mdx b/zh-CN/built-in-nodes/ClipSetLastLayer.mdx new file mode 100644 index 000000000..ee27c1d50 --- /dev/null +++ b/zh-CN/built-in-nodes/ClipSetLastLayer.mdx @@ -0,0 +1,41 @@ +--- +title: "设置CLIP最后一层 - ComfyUI 原生节点文档" +description: "`设置CLIP最后一层` 是 ComfyUI 中用于控制 CLIP 模型处理深度的核心节点。" +sidebarTitle: "设置CLIP最后一层" +icon: "circle" +--- + +`设置CLIP最后一层` 是 ComfyUI 中用于控制 CLIP 模型处理深度的核心节点。它允许用户精确控制 CLIP 文本编码器在哪一层停止处理,从而影响文本理解的深度和生成图像的风格。 + +想象 CLIP 模型是一个24层的智能大脑: + +- 浅层 (1-8层):识别基本字母、单词 +- 中层 (9-16层):理解语法、句子结构 +- 深层 (17-24层):掌握抽象概念、复杂语义 + +`设置CLIP最后一层` 就像一个**"思考深度调节器"**: + +-1: 使用全部24层(最完整理解) +-2: 停在第23层(稍微简化) +-12: 停在第13层(中等理解) +-24: 只用第1层(最基础理解) + +## 输入 + +| 参数名称 | 数据类型 | 默认值 | 取值范围 | 功能说明 | +|----------|----------|---------|----------|----------| +| `clip` | CLIP | - | - | 要修改的CLIP模型 | +| `停止在 CLIP层` | INT | -1 | -24 到 -1 | 指定停止处理的层级,-1表示使用全部层级,-24表示只用第一层 | + +## 输出 + +| 输出名称 | 数据类型 | 说明 | +|----------|----------|------| +| clip | CLIP | 已修改的CLIP模型,指定的层被设置为最后一层 | + +## 为什么需要设置最后一层 + +- **性能优化**:就像不需要博士学位来理解简单句子一样,有时浅层理解就够了,速度更快 +- **风格控制**:不同层次的理解会产生不同的艺术风格 +- **兼容性**:某些模型可能在特定层次上表现更好 + diff --git a/zh-CN/built-in-nodes/ClipTextEncode.mdx b/zh-CN/built-in-nodes/ClipTextEncode.mdx new file mode 100644 index 000000000..18894fa18 --- /dev/null +++ b/zh-CN/built-in-nodes/ClipTextEncode.mdx @@ -0,0 +1,54 @@ +--- +title: "CLIP文本编码 - ComfyUI 原生节点文档" +description: "`CLIP文本编码` 这个节点就像一位翻译官,它能将你用文字描述的创作想法转换成AI能够理解的特殊语言,帮助AI准确理解你想要创作的图像内容。" +sidebarTitle: "CLIP文本编码" +icon: "circle" +--- + +`CLIP文本编码 (CLIPTextEncode)` 这个节点就像一位翻译官,它能将你用文字描述的创作想法转换成AI能够理解的特殊"语言",帮助AI准确理解你想要创作的图像内容。 + +想象你在和一位外国画家沟通,你需要一位翻译来帮助你准确传达你想要的画作效果。这个节点就像那位翻译,它使用CLIP模型(一个经过大量图文训练的AI模型)来理解你的文字描述,并将其转换成AI绘画模型能理解的"语言指令"。 + +## 输入 + +| 参数名称 | 数据类型 | 输入方式 | 默认值 | 取值范围 | 功能说明 | +|----------|----------|----------|---------|----------|----------| +| text | STRING | 文本输入框 | 空 | 任意文本 | 就像给画家的详细说明,在这里输入你想要生成的图像的文字描述。支持多行文本,可以非常详细地描述你想要的效果。 | +| clip | CLIP | 模型选择 | 无 | 已加载的CLIP模型 | 相当于选择特定的翻译官,不同的CLIP模型就像不同的翻译官,他们对艺术风格的理解略有不同。 | + +## 输出 + +| 输出名称 | 数据类型 | 说明 | +|----------|----------|------| +| 条件 | CONDITIONING | 这是转换后的"绘画指令",包含了AI能够理解的详细创作指引。这些指令会告诉AI模型应该如何绘制符合你描述的图像。 | + +## 使用建议 + +1. **文本提示的基本用法** + - 可以像写作文一样详细描述你想要的图像 + - 越具体的描述,生成的图像越符合预期 + - 可以使用英文逗号分隔不同的描述要素 + +2. **特殊功能:使用Embedding模型** + - Embedding模型就像预设的艺术风格包,可以快速应用特定的艺术效果 + - 目前支持 .safetensors、.pt、.bin 这三种文件格式,你不一定需要在使用的时候用完整的模型名称 + - 使用方法: + 1. 将embedding模型文件(.pt格式)放入`ComfyUI/models/embeddings`文件夹 + 2. 在文本中使用`embedding:模型名称`来调用 + 例如:如果你有一个叫`EasyNegative.pt`的模型,可以这样使用: + + ``` + a beautiful landscape, embedding:EasyNegative, high quality + ``` + +3. **提示词权重调整** + - 可以用括号来调整某些描述的重要程度 + - 例如:`(beautiful:1.2)`会让"beautiful"这个特征更突出 + - 普通括号`()`的默认权重是1.1 + - 使用键盘快捷键 `ctrl + 上/下方向键` 头可以快速调整权重 + - 对应权重快速调整步长可以在设置中进行修改 + +4. **注意事项** + - 确保CLIP模型已正确加载 + - 文本描述尽量使用正面、明确的词语 + - 如果使用embedding模型,确保文件名称输入正确并且和当前主模型的架构吻合 diff --git a/zh-CN/built-in-nodes/ClipTextEncodeFlux.mdx b/zh-CN/built-in-nodes/ClipTextEncodeFlux.mdx new file mode 100644 index 000000000..f371447ba --- /dev/null +++ b/zh-CN/built-in-nodes/ClipTextEncodeFlux.mdx @@ -0,0 +1,49 @@ +--- +title: "CLIP文本编码Flux - ComfyUI 原生节点文档" +description: "CLIP文本编码Flux 节点是 ComfyUI 中专为 Flux 架构设计的高级文本编码节点。" +sidebarTitle: "CLIP文本编码Flux" +icon: "circle" +--- + +`CLIP文本编码Flux` 是 ComfyUI 中专为 Flux 架构设计的高级文本编码节点。它采用双文本编码器(CLIP-L 与 T5XXL)协同机制,能够同时处理结构化关键词和详细自然语言描述,为 Flux 模型提供更精准、更丰富的文本理解能力,提升文本到图像的生成质量。 + +该节点基于双编码器协作机制: +1. `clip_l` 输入会被 CLIP-L 编码器处理,提取风格、主题等关键词特征,适合简洁描述。 +2. `t5xxl` 输入由 T5XXL 编码器处理,擅长理解复杂、细致的自然语言场景描述。 +3. 两路编码结果融合后,结合"引导"参数,生成统一的条件嵌入(CONDITIONING),用于下游的 Flux 采样器节点,控制生成内容与文本描述的契合度。 + +## 输入 + +| 参数名称 | 数据类型 | 输入方式 | 默认值 | 取值范围 | 功能说明 | +|---------|---------|----------|--------|----------|----------| +| `clip` | CLIP | 节点输入 | 无 | - | 必须是支持 Flux 架构的 CLIP 模型,包含 CLIP-L 和 T5XXL 两个编码器 | +| `clip_l` | STRING | 文本框 | 无 | 最多77个token | 适合输入简洁的关键词描述,如风格、主题等 | +| `t5xxl` | STRING | 文本框 | 无 | 近乎无限制 | 适合输入详细的自然语言描述,表达复杂场景和细节 | +| `引导` | FLOAT | 滑块 | 3.5 | 0.0 - 100.0 | 控制文本条件对生成过程的影响强度,数值越大越严格遵循文本描述 | + +## 输出 + +| 输出名称 | 数据类型 | 说明 | +|---------|---------|------| +| `条件` | CONDITIONING | 包含双编码器处理后的条件嵌入和引导参数,用于条件图像生成 | + +## 使用示例 + +### 提示词示例 + +- **clip_l 输入框**(关键词风格): + - 使用结构化、简洁的关键词组合 + - 示例:`masterpiece, best quality, portrait, oil painting, dramatic lighting` + - 重点描述风格、质量、主题等核心元素 + +- **t5xxl 输入框**(自然语言描述): + - 使用完整、流畅的场景描述 + - 示例:`A highly detailed portrait in oil painting style, featuring dramatic chiaroscuro lighting that creates deep shadows and bright highlights, emphasizing the subject's features with renaissance-inspired composition.` + - 重点描述场景细节、空间关系、光影效果 + +### 注意事项 + +1. 确保使用兼容的 Flux 架构 CLIP 模型 +2. 建议同时填写 clip_l 和 t5xxl,以发挥双编码器优势 +3. 注意 clip_l 的词元数量限制(77个token) +4. 根据生成效果调整"引导"参数 diff --git a/zh-CN/built-in-nodes/overview.mdx b/zh-CN/built-in-nodes/overview.mdx index 39b817282..03caff49e 100644 --- a/zh-CN/built-in-nodes/overview.mdx +++ b/zh-CN/built-in-nodes/overview.mdx @@ -6,6 +6,10 @@ sidebarTitle: "总览" 内置节点是 ComfyUI 的默认节点,它们是 ComfyUI 的核心功能,你无需额外安装第三方自定义节点包,就可以使用的节点。 -由于此部分内容我们才开始更新中,所以内容不够完善,未来将逐步补充相应的内容。 +## 节点文档更新说明 -如果有内容错误等,你可以在这个 [repo](https://github.com/Comfy-Org/docs) 中提交 issue 或 pr 来帮助我们改进。 \ No newline at end of file +我们目前已经支持了内置的节点帮助文档,所以此部分的文档内容是定期从 [这个仓库](https://github.com/Comfy-Org/embedded-docs) 中同步过来的,目前我们会每周进行一次人工同步和内容更新。 + +## 贡献内容 + +如果你发现我们的内容错误,或者想要补充我们缺失的内容,请在 [这个仓库](https://github.com/Comfy-Org/embedded-docs) 中提交 issue 或 pr 来帮助我们改进。 \ No newline at end of file