-
Notifications
You must be signed in to change notification settings - Fork 12.2k
Node Replacement API #12014
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Node Replacement API #12014
Changes from 22 commits
Commits
Show all changes
33 commits
Select commit
Hold shift + click to select a range
7024486
Create helper classes for node replace registration
Kosinkadink c9dbe13
Add public register_node_replacement function to node_replace, add No…
Kosinkadink 588bc6b
Added old_widget_ids param to NodeReplace
Kosinkadink 04f89c7
Rename UseValue to SetValue
Kosinkadink d6b217a
Create some test replacements for frontend testing purposes
Kosinkadink 8bbd8f7
Fix test ndoe replacement for resize_type.multiplier field
Kosinkadink d5b3da8
feat: add legacy node replacements from frontend hardcoded patches (#…
viva-jinyi a2d4c0f
refactor: process isolation support for node replacement API (#12298)
christian-byrne 739ed21
fix: use direct PromptServer registration instead of ComfyAPI class
christian-byrne 8d0da49
feat: add node_replacements server feature flag (#12362)
christian-byrne a6d691d
Merge branch 'master' into jk/node-replace-api
Kosinkadink 9e758b5
Refactored _node_replace.py InputMap/OutputMap to use a TypedDict ins…
Kosinkadink 1ef4c6e
Merge branch 'master' into jk/node-replace-api
Kosinkadink b1c69ed
Improve NodeReplace docstring
Kosinkadink e2f7eaf
Merge branch 'master' into jk/node-replace-api
viva-jinyi e50ab67
Merge branch 'master' into jk/node-replace-api
Kosinkadink 14184a0
Add apply_replacements to NodeReplaceManager to apply registered node…
Kosinkadink 23338b5
Move node replacement registration for core nodes into nodes_replacem…
Kosinkadink 5b4ba29
Merge branch 'master' into jk/node-replace-api
Kosinkadink aac40e4
Skip replacements that lead to a missing node_id anyway
Kosinkadink 0132eb0
Slight fix to a type annotation
Kosinkadink 3ddcc43
Fixed redundant comment
Kosinkadink 962f62d
Merge branch 'master' into jk/node-replace-api
Kosinkadink 1768392
Use is_link for checking if an input is a link instead of looking for…
Kosinkadink 1ab9d4a
Remove unnecessary empty check
Kosinkadink 07d1ee2
Add a public API function for registering node replacements, refactor…
Kosinkadink 5f409b6
Call extension.on_load() in load_custom_node function, arttempt at us…
Kosinkadink 3caf811
Make the API call work, not sure if this is the intended way
Kosinkadink a9cc84b
Fix ComfyAPI initialization for base async case, move on_load to be c…
Kosinkadink 2d992eb
Removed node_replace files, moved NodeReplace to _io.py and exposed it
Kosinkadink d4d2798
Merge branch 'master' into jk/node-replace-api
Kosinkadink 2605c24
Change NodeReplace import slightly in case we will want to move the c…
Kosinkadink 0a77da9
Merge branch 'jk/node-replace-api' of https://github.com/Comfy-Org/Co…
Kosinkadink File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| from __future__ import annotations | ||
|
|
||
| from aiohttp import web | ||
|
|
||
| from typing import TYPE_CHECKING, TypedDict | ||
| if TYPE_CHECKING: | ||
| from comfy_api.latest._node_replace import NodeReplace | ||
|
|
||
| from nodes import NODE_CLASS_MAPPINGS | ||
|
|
||
| class NodeStruct(TypedDict): | ||
| inputs: dict[str, str | int | float | bool | tuple[str, int]] | ||
| class_type: str | ||
| _meta: dict[str, str] | ||
|
|
||
| def copy_node_struct(node_struct: NodeStruct, empty_inputs: bool = False) -> NodeStruct: | ||
| new_node_struct = node_struct.copy() | ||
| if empty_inputs: | ||
| new_node_struct["inputs"] = {} | ||
| else: | ||
| new_node_struct["inputs"] = node_struct["inputs"].copy() | ||
| new_node_struct["_meta"] = node_struct["_meta"].copy() | ||
| return new_node_struct | ||
|
|
||
|
|
||
| class NodeReplaceManager: | ||
| """Manages node replacement registrations.""" | ||
|
|
||
| def __init__(self): | ||
| self._replacements: dict[str, list[NodeReplace]] = {} | ||
|
|
||
| def register(self, node_replace: NodeReplace): | ||
| """Register a node replacement mapping.""" | ||
| self._replacements.setdefault(node_replace.old_node_id, []).append(node_replace) | ||
|
|
||
| def get_replacement(self, old_node_id: str) -> list[NodeReplace] | None: | ||
| """Get replacements for an old node ID.""" | ||
| return self._replacements.get(old_node_id) | ||
|
|
||
| def has_replacement(self, old_node_id: str) -> bool: | ||
| """Check if a replacement exists for an old node ID.""" | ||
| return old_node_id in self._replacements | ||
|
|
||
| def apply_replacements(self, prompt: dict[str, NodeStruct]): | ||
| connections: dict[str, list[tuple[str, str, int]]] = {} | ||
| need_replacement: set[str] = set() | ||
| for node_number, node_struct in prompt.items(): | ||
| class_type = node_struct["class_type"] | ||
| # need replacement if not in NODE_CLASS_MAPPINGS and has replacement | ||
| if class_type not in NODE_CLASS_MAPPINGS.keys() and self.has_replacement(class_type): | ||
| need_replacement.add(node_number) | ||
| # keep track of connections | ||
| for input_id, input_value in node_struct["inputs"].items(): | ||
| if isinstance(input_value, list): | ||
| conn_number = input_value[0] | ||
| connections.setdefault(conn_number, []).append((node_number, input_id, input_value[1])) | ||
| if len(need_replacement) > 0: | ||
Kosinkadink marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| for node_number in need_replacement: | ||
| node_struct = prompt[node_number] | ||
| class_type = node_struct["class_type"] | ||
| replacements = self.get_replacement(class_type) | ||
| if replacements is None: | ||
| continue | ||
| # just use the first replacement | ||
| replacement = replacements[0] | ||
| new_node_id = replacement.new_node_id | ||
Kosinkadink marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| # if replacement is not a valid node, skip trying to replace it as will only cause confusion | ||
| if new_node_id not in NODE_CLASS_MAPPINGS.keys(): | ||
| continue | ||
| # first, replace node id (class_type) | ||
| new_node_struct = copy_node_struct(node_struct, empty_inputs=True) | ||
| new_node_struct["class_type"] = new_node_id | ||
| # TODO: consider replacing display_name in _meta as well for error reporting purposes; would need to query node schema | ||
| # second, replace inputs | ||
| if replacement.input_mapping is not None: | ||
| for input_map in replacement.input_mapping: | ||
| if "set_value" in input_map: | ||
| new_node_struct["inputs"][input_map["new_id"]] = input_map["set_value"] | ||
| elif "old_id" in input_map: | ||
| new_node_struct["inputs"][input_map["new_id"]] = node_struct["inputs"][input_map["old_id"]] | ||
| # finalize input replacement | ||
| prompt[node_number] = new_node_struct | ||
| # third, replace outputs | ||
| if replacement.output_mapping is not None: | ||
| # re-mapping outputs requires changing the input values of nodes that receive connections from this one | ||
| if node_number in connections: | ||
| for conns in connections[node_number]: | ||
| conn_node_number, conn_input_id, old_output_idx = conns | ||
| for output_map in replacement.output_mapping: | ||
| if output_map["old_idx"] == old_output_idx: | ||
| new_output_idx = output_map["new_idx"] | ||
| previous_input = prompt[conn_node_number]["inputs"][conn_input_id] | ||
| previous_input[1] = new_output_idx | ||
|
|
||
| def as_dict(self): | ||
| """Serialize all replacements to dict.""" | ||
| return { | ||
| k: [v.as_dict() for v in v_list] | ||
| for k, v_list in self._replacements.items() | ||
| } | ||
|
|
||
| def add_routes(self, routes): | ||
| @routes.get("/node_replacements") | ||
| async def get_node_replacements(request): | ||
| return web.json_response(self.as_dict()) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| from __future__ import annotations | ||
|
|
||
| from typing import Any, TypedDict | ||
|
|
||
|
|
||
| class InputMapOldId(TypedDict): | ||
| """Map an old node input to a new node input by ID.""" | ||
| new_id: str | ||
| old_id: str | ||
|
|
||
|
|
||
| class InputMapSetValue(TypedDict): | ||
| """Set a specific value for a new node input.""" | ||
| new_id: str | ||
| set_value: Any | ||
|
|
||
|
|
||
| InputMap = InputMapOldId | InputMapSetValue | ||
| """ | ||
| Input mapping for node replacement. Type is inferred by dictionary keys: | ||
| - {"new_id": str, "old_id": str} - maps old input to new input | ||
| - {"new_id": str, "set_value": Any} - sets a specific value for new input | ||
| """ | ||
|
|
||
|
|
||
| class OutputMap(TypedDict): | ||
| """Map outputs of node replacement via indexes.""" | ||
| new_idx: int | ||
| old_idx: int | ||
|
|
||
|
|
||
| class NodeReplace: | ||
| """ | ||
| Defines a possible node replacement, mapping inputs and outputs of the old node to the new node. | ||
|
|
||
| Also supports assigning specific values to the input widgets of the new node. | ||
|
|
||
| Args: | ||
| new_node_id: The class name of the new replacement node. | ||
| old_node_id: The class name of the deprecated node. | ||
| old_widget_ids: Ordered list of input IDs for widgets that may not have an input slot | ||
| connected. The workflow JSON stores widget values by their relative position index, | ||
| not by ID. This list maps those positional indexes to input IDs, enabling the | ||
| replacement system to correctly identify widget values during node migration. | ||
| input_mapping: List of input mappings from old node to new node. | ||
| output_mapping: List of output mappings from old node to new node. | ||
| """ | ||
Kosinkadink marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| def __init__(self, | ||
| new_node_id: str, | ||
| old_node_id: str, | ||
| old_widget_ids: list[str] | None=None, | ||
| input_mapping: list[InputMap] | None=None, | ||
Kosinkadink marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| output_mapping: list[OutputMap] | None=None, | ||
| ): | ||
| self.new_node_id = new_node_id | ||
| self.old_node_id = old_node_id | ||
| self.old_widget_ids = old_widget_ids | ||
| self.input_mapping = input_mapping | ||
| self.output_mapping = output_mapping | ||
|
|
||
| def as_dict(self): | ||
| """Create serializable representation of the node replacement.""" | ||
| return { | ||
| "new_node_id": self.new_node_id, | ||
| "old_node_id": self.old_node_id, | ||
| "old_widget_ids": self.old_widget_ids, | ||
| "input_mapping": list(self.input_mapping) if self.input_mapping else None, | ||
| "output_mapping": list(self.output_mapping) if self.output_mapping else None, | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| from ._node_replace import * # noqa: F403 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| from comfy_api.latest import ComfyExtension, io, node_replace | ||
| from server import PromptServer | ||
|
|
||
| def _register(nr: node_replace.NodeReplace): | ||
| """Helper to register replacements via PromptServer.""" | ||
| PromptServer.instance.node_replace_manager.register(nr) | ||
Kosinkadink marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| async def register_replacements(): | ||
| """Register all built-in node replacements.""" | ||
| register_replacements_longeredge() | ||
| register_replacements_batchimages() | ||
| register_replacements_upscaleimage() | ||
| register_replacements_controlnet() | ||
| register_replacements_load3d() | ||
| register_replacements_preview3d() | ||
| register_replacements_svdimg2vid() | ||
| register_replacements_conditioningavg() | ||
|
|
||
| def register_replacements_longeredge(): | ||
| # No dynamic inputs here | ||
| _register(node_replace.NodeReplace( | ||
| new_node_id="ImageScaleToMaxDimension", | ||
| old_node_id="ResizeImagesByLongerEdge", | ||
| old_widget_ids=["longer_edge"], | ||
| input_mapping=[ | ||
| {"new_id": "image", "old_id": "images"}, | ||
| {"new_id": "largest_size", "old_id": "longer_edge"}, | ||
| {"new_id": "upscale_method", "set_value": "lanczos"}, | ||
| ], | ||
| # just to test the frontend output_mapping code, does nothing really here | ||
| output_mapping=[{"new_idx": 0, "old_idx": 0}], | ||
| )) | ||
|
|
||
| def register_replacements_batchimages(): | ||
| # BatchImages node uses Autogrow | ||
| _register(node_replace.NodeReplace( | ||
| new_node_id="BatchImagesNode", | ||
| old_node_id="ImageBatch", | ||
| input_mapping=[ | ||
| {"new_id": "images.image0", "old_id": "image1"}, | ||
| {"new_id": "images.image1", "old_id": "image2"}, | ||
| ], | ||
| )) | ||
|
|
||
| def register_replacements_upscaleimage(): | ||
| # ResizeImageMaskNode uses DynamicCombo | ||
| _register(node_replace.NodeReplace( | ||
| new_node_id="ResizeImageMaskNode", | ||
| old_node_id="ImageScaleBy", | ||
| old_widget_ids=["upscale_method", "scale_by"], | ||
| input_mapping=[ | ||
| {"new_id": "input", "old_id": "image"}, | ||
| {"new_id": "resize_type", "set_value": "scale by multiplier"}, | ||
| {"new_id": "resize_type.multiplier", "old_id": "scale_by"}, | ||
| {"new_id": "scale_method", "old_id": "upscale_method"}, | ||
| ], | ||
| )) | ||
|
|
||
| def register_replacements_controlnet(): | ||
| # T2IAdapterLoader → ControlNetLoader | ||
| _register(node_replace.NodeReplace( | ||
| new_node_id="ControlNetLoader", | ||
| old_node_id="T2IAdapterLoader", | ||
| input_mapping=[ | ||
| {"new_id": "control_net_name", "old_id": "t2i_adapter_name"}, | ||
| ], | ||
| )) | ||
|
|
||
| def register_replacements_load3d(): | ||
| # Load3DAnimation merged into Load3D | ||
| _register(node_replace.NodeReplace( | ||
| new_node_id="Load3D", | ||
| old_node_id="Load3DAnimation", | ||
| )) | ||
|
|
||
| def register_replacements_preview3d(): | ||
| # Preview3DAnimation merged into Preview3D | ||
| _register(node_replace.NodeReplace( | ||
| new_node_id="Preview3D", | ||
| old_node_id="Preview3DAnimation", | ||
| )) | ||
|
|
||
| def register_replacements_svdimg2vid(): | ||
| # Typo fix: SDV → SVD | ||
| _register(node_replace.NodeReplace( | ||
| new_node_id="SVD_img2vid_Conditioning", | ||
| old_node_id="SDV_img2vid_Conditioning", | ||
| )) | ||
|
|
||
| def register_replacements_conditioningavg(): | ||
| # Typo fix: trailing space in node name | ||
| _register(node_replace.NodeReplace( | ||
| new_node_id="ConditioningAverage", | ||
| old_node_id="ConditioningAverage ", | ||
| )) | ||
|
|
||
| class NodeReplacementsExtension(ComfyExtension): | ||
| async def get_node_list(self) -> list[type[io.ComfyNode]]: | ||
| return [] | ||
|
|
||
| async def comfy_entrypoint() -> NodeReplacementsExtension: | ||
| await register_replacements() | ||
| return NodeReplacementsExtension() | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.