ComfyUI-itsB34ST-Nodes

A set of nodes and utilities for use with Comfy UI.

prerelease stage

nodes should show up in a category called BSTnodes

Sprite Transformer Node

A sprite is a two-dimensional image or animation that is integrated into a larger scene, typically used in video games to represent characters, objects, and other visual elements.

The Sprite Transformer Node offers the capability to render a sprite on a transparent plane within a 3D space. This node is versatile, equipped with a variety of settings that allow for extensive customization and dynamic transformations of the sprite.

Users can:

• Move the sprite to any location within the 3D space; via the ST Mover node.
• Rotate/Spin the sprite, facilitating full 360-degree orientation changes; via the ST Spinner node.
• Scale the sprite, adjusting its size to fit specific requirements; via the ST Scaler node.
• Light the sprite, adjusting the scene lighting in order to better match the intended background; via the ST Lighter node.
• Extrude the sprite to create a compelling illusion of depth, enhancing the visual impact; via the ST 3D Maker node.

These features make the Sprite Transformer Node a powerful tool for animators looking for more direct control over their animations directly within ComfyUI. The intended use is to build composites for further refinement with AnimateDiff.

The input can be a batch of any size, with various options as to handle it with the sprite loop setting:

• restart will restart the batch from frame id 0 if the desired number of frames exceeds it.
• reflect will reverse the batch when the final index has been reached for the remainder of the desired frames.
• random will randomly extend the batch to fill the duration.
• lock will freeze frame on the final index of the batch.
• more coming soon

The resolution and field of view can be set, also the "flat" option toggles orthographic view mode. Warning if using the ST 3D Maker node, resolution = geometry, hence a very large image size would lead to a very large polygon count and long render times. Steps in this case are equal to number of frames to be rendered.

Universal settings options within the settings nodes are as follows:

• Loop will attempt to split the steps in half and complete the desired transformation in half the steps and then reverse.
• Restart will attempt to split the steps and perform the transform X number of times.
• These can be used in unison for various motion effects, also this will be improved upon later.

The ST Mover node also posseses a setting called "Ts_Factor" this is the translation scale factor, it has been precalibrated. The ST 3D Maker node has been preset for average middle of the road performance, increase or decrease smoothing for your purposes; note for our purposes extrusion is on the negative scale. The ST Lighter node is self explanatory.

The exptected outputs of the node are as follows:

• rgba A batch of 4-channel RGBA images of the newly transformed sprite.
• mask A batch of binary masks to facilitate easily compositing; to be improved later.
• rgb A batch of RGB images of the newly transformed sprite on a black background.

Troubleshooting common problems:

• Black Output: Double check your settings, you must be moving too far, and it is out of view.
• Slow Ouput: Scale down the input batch before the Sprite Transformer node, especially if using ST 3D Maker.
• Others?: Open an issue and let me know whats wrong.

Basic Sprite Transformer Tutorial

Step 1: Prepare your sprite for transformation. Example process only. For the sake of simplicity I used 2 images to create a 16 frame walk cycle. Feel free to build/create sprites of actors or objects any way you want.

Step 2: Render your sprite with desired transforms. Basic motion example. Here we have our sprite move from the back of the viewport towards the screen, with a slight rotation along the Y-axis to give it a more lifelike gait. In this instance I used rife to smooth out and slow down the animation slightly. Optionally we select an image to use as an IP Adapter target image.

Step 3: Vid2Vid your composite through AnimateDiff. Optimal settings may vary. We feed our newly rendered composite through AnimateDiff. Experiment with various denoising values, on average it seems to work well between 0.3 and 0.65 depending on what you are going for. AnimateDiff will halucinate the finer details of the scene due to being biased on video, as such most things that should move, will. And the sprite/actor will follow the predetermined trajectory.