Skip to content

Commit 58c4411

Browse files
Chenlei Huwebfiltered
Chenlei Hu
and
webfiltered
authored
RFC: Widget Input Socket (#9)
* rfc-04: Widget Input Socket * Update rfcs/0004-widget-input-socket.md Co-authored-by: filtered <[email protected]> * Update rfcs/0004-widget-input-socket.md Co-authored-by: filtered <[email protected]> * Update rfcs/0004-widget-input-socket.md Co-authored-by: filtered <[email protected]> * Update rfcs/0004-widget-input-socket.md Co-authored-by: filtered <[email protected]> * Add grab link example * Update rfcs/0004-widget-input-socket.md Co-authored-by: filtered <[email protected]> * Update and rename 0004-widget-input-socket.md to 0003-widget-input-socket.md --------- Co-authored-by: filtered <[email protected]>
1 parent c20aad5 commit 58c4411

File tree

1 file changed

+240
-0
lines changed

1 file changed

+240
-0
lines changed

‎rfcs/0003-widget-input-socket.md‎

Lines changed: 240 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
1+
# RFC: Widget Input Socket
2+
3+
- Start Date: 2025-01-13
4+
- Target Major Version: Frontend v1.16
5+
- Implementation PRs:
6+
7+
- https://github.com/Comfy-Org/ComfyUI_frontend/pull/3326
8+
- https://github.com/Comfy-Org/litegraph.js/pull/891
9+
10+
- Reference Issues:
11+
12+
- <https://github.com/Comfy-Org/ComfyUI_frontend/pull/1021>
13+
14+
## Summary
15+
16+
This RFC proposes replacing ComfyUI's current widget-to-socket conversion system with a simpler, more intuitive "widget input socket" design. Instead of requiring users to manually convert widgets to sockets through context menus, widgets will automatically display an input socket when hovered over. This socket behaves like any other input socket - allowing users to drag connections to and from it. When connected, the widget becomes disabled (grayed out) to indicate it's receiving an external input.
17+
18+
This change aims to:
19+
20+
- Simplify the codebase by removing complex conversion logic
21+
- Improve usability by making connections more discoverable and intuitive
22+
- Align with industry standards seen in tools like Blender
23+
- Reduce the number of clicks needed to make connections
24+
- Eliminate the need to maintain conversion states in workflow files
25+
26+
The proposal represents a breaking change that would be implemented in [workflow schema v2.0](https://github.com/Comfy-Org/rfcs/pull/2), though existing workflows would continue to function without modification.
27+
28+
## Basic example
29+
30+
### Current conversion mechanism between widget and socket
31+
32+
#### Convert from widget to socket
33+
34+
To convert a widget to a socket, the user needs to first right click the node, and then select the corresponding conversion option from the context menu.
35+
36+
![conversion_context_menu](https://github.com/user-attachments/assets/522f163b-8aad-42b2-a899-817c1a0bae75)
37+
38+
![converted_socket](https://github.com/user-attachments/assets/ab6c50a3-ee33-443c-89ca-73d3a3c67042)
39+
40+
#### Convert from socket to widget
41+
42+
There are two ways to convert a socket to a widget:
43+
44+
1\. **Option1**: Right click the node, and select the corresponding conversion option from the context menu.
45+
46+
![conversion_context_menu](https://github.com/user-attachments/assets/4e47f740-d607-44da-b49c-4a9bea548656)
47+
48+
2\. **Option2**: Drag a link of correct type from an output socket on another node to the widget (Implemented in <https://github.com/Comfy-Org/ComfyUI_frontend/pull/1021>).
49+
50+
https://github.com/user-attachments/assets/360013b2-d350-4fb0-bbce-cb860178d9ed
51+
52+
### Proposed design for widget input socket
53+
54+
When the cursor hovers over a widget, a socket will be shown on the left side of the widget. The socket will be interacted the same way as the current input socket, i.e.
55+
56+
- user can drag a link from the socket and drop to another node's output socket to create a new link
57+
- user can drag a link from another node's output socket and drop to the socket to create a new link
58+
59+
![cursor_hover_on_widget](https://github.com/user-attachments/assets/953867dc-f27c-47de-a06f-aa94a29350a4)
60+
61+
When grabbing a link, the widget with a matching type will be highlighted. Here in the example, the link type is `INT`,
62+
so the widget `seed` and `steps` are highlighted.
63+
64+
![grab_link](https://github.com/user-attachments/assets/9f44ff15-37ae-4fda-af77-24048098582c)
65+
66+
When connected, the widget will be disabled (grayed out) and the socket will be highlighted.
67+
68+
![connected_widget_socket](https://github.com/user-attachments/assets/fe17d9b2-01c6-441a-adc6-f869f7aa3cbf)
69+
70+
## Motivation
71+
72+
1. **Simplified State Management**
73+
74+
- Current implementation requires complex state tracking for widget/socket conversion status
75+
- Eliminates need to persist conversion state in workflow files
76+
- Removes the `default_input` configuration complexity from node definitions
77+
- Reduces potential for bugs related to state synchronization
78+
79+
2. **Improved Discoverability**
80+
81+
- New users often struggle to discover the widget-to-socket conversion feature
82+
- Hover-based socket visibility provides immediate visual feedback
83+
- Makes connection capabilities self-evident without requiring documentation
84+
- Follows established UI patterns where hover reveals additional functionality
85+
86+
3. **Industry Standard Alignment**
87+
88+
- Matches behavior in popular node-based tools like Blender
89+
- Reduces learning curve for users coming from other platforms
90+
- Leverages existing mental models from the visual programming community
91+
- Makes ComfyUI feel more familiar to experienced node-based workflow users
92+
93+
4. **Cognitive Simplification**
94+
95+
- Eliminates the artificial distinction between widgets and sockets
96+
- Treats all inputs as potentially connectable by default
97+
- Removes need to teach users about "conversion" as a concept
98+
- Provides a more intuitive "what you see is what you can do" interface
99+
100+
5. **Workflow Optimization**
101+
102+
- Reduces actions needed to create connections from 3+ clicks to 1 drag
103+
- Eliminates context menu navigation time
104+
- Speeds up workflow creation and modification
105+
- Particularly beneficial for complex workflows with many connections
106+
107+
6. **Enhanced Accessibility**
108+
109+
- Reduces fine motor control requirements compared to context menu usage
110+
- Provides larger hit areas for connection interactions
111+
- More forgiving of slight cursor movement during interaction
112+
- Supports users with various input devices more effectively
113+
114+
7. **Technical Benefits**
115+
116+
- Simplifies the codebase by removing conversion logic
117+
- Makes widget behavior more predictable and easier to test
118+
- Reduces potential edge cases in the connection system
119+
- Easier to maintain and extend in the future
120+
121+
The primary goal is to make ComfyUI more intuitive and efficient to use while reducing implementation complexity. This change would bring the interface more in line with user expectations and industry standards, while simultaneously simplifying the codebase.
122+
123+
## Detailed design
124+
125+
### Component Updates
126+
127+
#### LGraphCanvas
128+
129+
1. **Widget Socket Rendering**
130+
131+
- Modify `drawNodeWidgets()` to render an input socket for a widget when:
132+
133+
- The widget is being hovered
134+
- The socket has an active connection
135+
- A compatible link is being dragged
136+
137+
- Update `drawNode()` to skip rendering duplicate sockets for widget inputs
138+
139+
2. **Interaction Handling**
140+
141+
- Extend `isOverNodeInput()` to detect cursor position over widget input sockets
142+
- Return `true` when cursor is within the socket's hit area
143+
144+
#### LGraphWidget
145+
146+
1. **Disabled State Management**
147+
148+
- Add `isDisabled` getter property
149+
150+
- Returns `true` when the widget has a connected input socket
151+
152+
- Used to control widget interactivity and visual state
153+
154+
2. **Visual Styling**
155+
156+
- Apply 0.5 opacity to widgets in disabled state
157+
- Maintain visual consistency with standard disabled UI elements
158+
159+
### Data Structure Changes
160+
161+
#### LGraphNode
162+
163+
1. **Input Management**
164+
165+
- Extend `inputs` array to include widget input sockets
166+
- Add widget reference to each input socket object:
167+
168+
```typescript
169+
interface InputSocket {
170+
widget?: LGraphWidget;
171+
// ... existing input socket properties
172+
}
173+
```
174+
175+
2. **Serialization**
176+
177+
- Maintain compatibility with existing serialization format
178+
179+
- Reference: [RFC #2](https://github.com/Comfy-Org/rfcs/pull/2)
180+
181+
- No changes required to current workflow file structure
182+
183+
## Drawbacks
184+
185+
1. **Implementation Challenges**
186+
187+
- Need to modify core rendering logic in LGraphCanvas
188+
- Potential edge cases with complex layouts for custom DOM widgets
189+
- Additional complexity in handling hover states during link dragging
190+
- Need to maintain backward compatibility with existing workflows
191+
192+
2. **User Experience Trade-offs**
193+
194+
- Loss of explicit user control over widget/socket conversion
195+
- Hover-based interactions may be less reliable on touch devices
196+
- May be less discoverable than context menu options for some users
197+
198+
3. **Migration Concerns**
199+
200+
- Existing tutorials and documentation will need updates
201+
- Users familiar with the current system will need to adapt
202+
- Custom nodes using the current widget conversion system may need modifications
203+
204+
## Adoption strategy
205+
206+
The transition to the new widget input socket design will be implemented in a single phase:
207+
208+
### Implementation Phase (v2.0)
209+
210+
1. **Breaking Changes**
211+
212+
- Replace the existing conversion system with the new widget input socket design
213+
- Remove all conversion-related APIs and context menu options
214+
- Remove `force_input` configurations from node definitions
215+
- Clean up legacy conversion code from the codebase
216+
217+
2. **Documentation & Communication**
218+
219+
- Update official documentation with the new interaction model
220+
- Provide migration guides for node developers
221+
- Create visual tutorials demonstrating the new connection workflow
222+
- Issue clear communication about the breaking changes
223+
224+
3. **Ecosystem Impact**
225+
226+
- Custom node developers will need to:
227+
228+
- Remove any conversion-specific code
229+
- Update widget definitions to work with new socket system
230+
- Test existing nodes with the new connection behavior
231+
232+
4. **User Impact**
233+
234+
- All existing workflows will continue to work as expected
235+
- Users will immediately benefit from the simplified interaction model
236+
- No manual migration steps required for end users
237+
238+
This direct approach allows us to quickly realize the benefits of the new design while minimizing the complexity of maintaining two parallel systems. Since the new design is more intuitive and requires less user education, the transition cost is justified by the immediate improvements in usability.
239+
240+
## Unresolved questions

0 commit comments

Comments
 (0)