-
Notifications
You must be signed in to change notification settings - Fork 17
Description
Overview
We want to add support for sub-graph workflows in the workflow-to-code transpiler of our library.
⚠️ Note: This feature is only for the transpiler. The originalClientAPI will not support sub-graph logic (i.e., nodes with IDs like"xxx:xxx"). Supporting sub-graphs in the runtime API is unnecessary because sub-graphs can be easily implemented using standard JavaScript functions when controlling execution via code.
Although ComfyUI currently has some bugs when loading and editing sub-graphs, the implementation is now approaching usability. By supporting sub-graph JSON in the transpiler, we allow users to convert workflows with sub-graphs into modular, human-readable JavaScript functions.
Example Input
{
"9": {"inputs":{"filename_prefix":"z-image","images":["58:43",0]}, "class_type":"SaveImage","_meta":{"title":"Save Image"}},
"57": {"inputs":{"text":""}, "class_type":"Text Multiline","_meta":{"title":"Text Multiline"}},
"58:39": {"inputs":{"clip_name":"qwen_3_4b.safetensors","type":"lumina2","device":"default"}, "class_type":"CLIPLoader","_meta":{"title":"Load CLIP"}},
"58:40": {"inputs":{"vae_name":"ae.safetensors"}, "class_type":"VAELoader","_meta":{"title":"Load VAE"}},
"58:46": {"inputs":{"unet_name":"z_image_turbo_bf16.safetensors","weight_dtype":"default"}, "class_type":"UNETLoader","_meta":{"title":"UNet Loader"}},
"58:47": {"inputs":{"shift":3,"model":["58:46",0]}, "class_type":"ModelSamplingAuraFlow","_meta":{"title":"Sampler (AuraFlow)"}},
"58:41": {"inputs":{"width":1024,"height":1024,"batch_size":1}, "class_type":"EmptySD3LatentImage","_meta":{"title":"Empty Latent Image (SD3)"}},
"58:45": {"inputs":{"text":["57",0],"clip":["58:39",0]}, "class_type":"CLIPTextEncode","_meta":{"title":"CLIP Text Encode"}},
"58:42": {"inputs":{"conditioning":["58:45",0]}, "class_type":"ConditioningZeroOut","_meta":{"title":"Conditioning Zero Out"}},
"58:43": {"inputs":{"samples":["58:44",0],"vae":["58:40",0]}, "class_type":"VAEDecode","_meta":{"title":"VAE Decode"}},
"58:44": {"inputs":{"seed":533303727624653,"steps":9,"cfg":1,"sampler_name":"res_multistep","scheduler":"simple","denoise":1,"model":["58:47",0],"positive":["58:45",0],"negative":["58:42",0],"latent_image":["58:41",0]}, "class_type":"KSampler","_meta":{"title":"K Sampler"}}
}Expected Output
The transpiler should generate modular JavaScript functions, encapsulating sub-graphs:
/* Text Multiline */
const [STRING_1] = cls["Text Multiline"]({ "text": "" });
const [IMAGE_1] = sub58(STRING_1);
/* Save Image */
const [] = cls.SaveImage({
"filename_prefix": "z-image",
"images": IMAGE_1
});
/* Subgraph: 58 */
function sub58(...args) {
const [STRING_1] = args;
/* Empty Latent Image (SD3) */
const [LATENT_1] = cls.EmptySD3LatentImage({ "width": 1024, "height": 1024, "batch_size": 1 });
/* UNet Loader */
const [MODEL_1] = cls.UNETLoader({ "unet_name": "z_image_turbo_bf16.safetensors", "weight_dtype": "default" });
/* Sampler (AuraFlow) */
const [MODEL_2] = cls.ModelSamplingAuraFlow({ "shift": 3, "model": MODEL_1 });
/* Load VAE */
const [VAE_1] = cls.VAELoader({ "vae_name": "ae.safetensors" });
/* Load CLIP */
const [CLIP_1] = cls.CLIPLoader({ "clip_name": "qwen_3_4b.safetensors", "type": "lumina2", "device": "default" });
/* CLIP Text Encode */
const [CONDITIONING_1] = cls.CLIPTextEncode({ "text": STRING_1, "clip": CLIP_1 });
/* Conditioning Zero Out */
const [CONDITIONING_2] = cls.ConditioningZeroOut({ "conditioning": CONDITIONING_1 });
/* K Sampler */
const [LATENT_2] = cls.KSampler({
"seed": 533303727624653,
"steps": 9,
"cfg": 1,
"sampler_name": "res_multistep",
"scheduler": "simple",
"denoise": 1,
"model": MODEL_2,
"positive": CONDITIONING_1,
"negative": CONDITIONING_2,
"latent_image": LATENT_1
});
/* VAE Decode */
const [IMAGE_1] = cls.VAEDecode({ "samples": LATENT_2, "vae": VAE_1 });
return [IMAGE_1];
}Goals
- Parse sub-graph nodes with IDs like
"58:44"or"58:39". - Generate modular, human-readable JavaScript functions for each sub-graph.
- Preserve input/output references between main graph and sub-graphs.
- Maintain compatibility with the transpiler only; runtime
ClientAPI calls do not need sub-graph support. - Ensure clear comments for each node for maintainability.