extract workflows from images

[geroldmeisinger]

Describe the solution you'd like

I published a custom node with workflow images created using pythongosssss workflow image. To make it work work with workflow_templates however I had to extract the workflow json for each image file. I also wanted to change the image afterwards but then the metadata gets removed by GIMP and I needed a way to re-add the workflow from the original file.

As comfy-cli is the tool to support with everything comfy I think a feature should be included which enables to extract, insert, and copy workflows from image files (and all the other supported medias: webp, mp4 etc.).

comfy workflow extract [image] [workflow_json]
comfy workflow insert [workflow_json] [image]
comfy workflow copy [src_image] [dst_image]

options:
--prompt --workflow
default to workflow
--indent=4
json pretty printing

Describe alternatives you've considered

This is what I used using exiftool:

extract workflow

for f in *.png; do exiftool -b -workflow "$f" | jq --indent 4 '.' > "${f%.png}.json"; done

copy workflow
ComfyUI.config:

%Image::ExifTool::UserDefined = (
    'Image::ExifTool::PNG::TextualData' => {
        workflow => { %unreg },
        prompt => { %unreg },
    },
);
1;

exiftool -config ComfyUI.config -TagsFromFile source.png -workflow -prompt target.png

as the workflow gets added to a non-standard tag it has to be defined as an %unreg otherwise exiftool refuses to add it.

there are some tools:
https://github.com/set-soft/Comfy_UI_prompt_extractor (CLI tool)
https://github.com/ibmace/comfyui-prompt-extractor (CLI tool)
https://github.com/bleeckerj/comfyui-workflow-extractor (python library)
https://github.com/cocktailpeanut/comfymeta (UI tool)
https://github.com/mamorett/comfyui_prompt_extractor (UI and CLI tool)

but none of them actually allowed me to copy or insert workflows

Additional context

There are 2 different workflows versions documented: https://docs.comfy.org/specs/workflow_json
I don't know if there are actually more. There are also API workflows. But all of this should be independent of the extraction. but maybe an extra option to convert between api and non-api.

maybe allow wildcards in src and dst filenames (see imagemagick convert syntax)

comfy workflow extract *.png
comfy workflow insert *.json *.png

according to https://github.com/set-soft/Comfy_UI_prompt_extractor:

Notes about the difference between the prompt and workflow files
The workflow file contains all the nodes and its sizes and status. So this file is the most important. Loading it in ComfyUI will reproduce the worflow as it was displayed when saving the image.
The prompt file contains only the nodes executed to generate the image, without the sizes and status. So this file is much smaller and is enough to regenerate the image, but isn't nice to modify.

nice2have: render a json file to png or svg with or without workflow included, just as pythongosssss' workflow image

[geroldmeisinger]

the official metadata parsing happens in comfyui-frontend-package in javascript context: https://github.com/Comfy-Org/ComfyUI_frontend/tree/main/src/scripts/metadata