Refocus - ComfyUI Nodes for Generative Refocusing

ComfyUI custom nodes for the Genfocus generative refocusing pipeline.

Transform any photo into a professional-looking image with controllable depth-of-field (bokeh) effects!

Features

DeblurNet: Restore all-in-focus images from blurry/defocused inputs
BokehNet: Generate realistic depth-of-field effects with controllable focus
DepthPro: High-quality metric depth estimation (Apple ml-depth-pro)
Depth Utilities: Convert between depth formats, focal length calculations

Example Workflows

Ready-to-use workflow files are in the workflows/ folder. Download the JSON files or drag the PNG workflow images directly into ComfyUI!

🔄 Refocus Workflow

Files:

Refocus-Eric-1225.json / Refocus-Eric-1225.png (workflow)
Input: HunyuanImage3_2025-12-11_17-30-15_000.png

This workflow demonstrates the complete refocusing pipeline: deblur an image first, then selectively refocus to create professional depth-of-field effects.

Workflow Diagram

Input Image

The starting image was generated with Hunyuan Image 3 (NF4). It has a soft, natural background blur — similar to what you'd get from a small format camera at f/8.

Results

Result	Description
	Deblurred — Extended depth of field from foreground to infinity. Notice how both the subject and background are now sharp.
	Refocus 1 — Focus on subject's face using a small mask mark. The background is blurred to make the subject stand out. Adjust `blur_strength` and `max_coc` for more dramatic separation.
	Refocus 2 — Creative example: focus shifted to the 2nd car in the background. The subject is now artistically blurred.

Key Concepts

Deblur sharpens everything — extends depth of field from foreground to infinity
Refocus shifts the focal plane to wherever you mark with a mask or set with [x,y] coordinates
max_coc and blur_strength in the Compute Defocus node control how much blur is applied outside the depth of field

🔍 Simple Deblur Workflow

Files:

Deblur_Eric_1225_simple.json / Deblur_Eric_1225_simple.png (workflow)
Input: HunyuanImage3_2025-12-20_00-59-02_000.png

A minimal workflow for deblurring images with very few adjustable parameters — perfect for automation or batch processing.

Workflow Diagram

Before & After

Input	Output

Original image with natural background blur	Deblurred result with sharp foreground and background

Notes

The input image (generated with Hunyuan Image 3) has a subtle, natural-looking background blur
The deblurred result brings both foreground and background elements into sharp focus
For additional sharpening, consider using a Richardson-Lucy deconvolution node as a final step (after upscaling)
Minimal configuration required — great for automation pipelines
Works best with images around 1 megapixel (FLUX.1-dev native resolution), though FLUX handles larger sizes reasonably well
Increasing the number of steps can provide slight quality improvements

Installation

1. Clone the Repository

cd ComfyUI/custom_nodes
git clone <this-repo> Refocus

2. Install Base Dependencies

pip install safetensors

3. Install DepthPro (Optional - for depth estimation)

pip install git+https://github.com/apple/ml-depth-pro.git --no-deps
pip install pillow_heif

4. Install Native Genfocus Dependencies (Recommended)

pip install diffusers transformers accelerate peft

Model Setup

Folder Structure

ComfyUI/
├── models/
│   ├── genfocus/                    ← Genfocus LoRAs
│   │   ├── deblurNet.safetensors
│   │   └── bokehNet.safetensors
│   ├── diffusers/                   ← FLUX model (diffusers format)
│   │   └── FLUX.1-dev/
│   │       ├── model_index.json
│   │       ├── transformer/
│   │       └── ...
│   └── checkpoints/
│       └── depth_pro.pt             ← DepthPro weights

Download Links

Model	Size	License	Download
Genfocus LoRAs	~35MB	Apache 2.0	HuggingFace
DepthPro	~500MB	Apple Sample Code	HuggingFace
FLUX.1-dev	~23GB	Non-Commercial	HuggingFace

Node Reference

Genfocus Nodes (diffusers-based)

These nodes use the full Genfocus implementation with proper multi-conditional LoRA support.

Genfocus Model Loader (Native)

Loads FLUX.1-dev with Genfocus LoRA adapters.

Input	Type	Description
`flux_model`	dropdown	Local diffusers model or HuggingFace ID
`precision`	dropdown	bf16 (recommended), fp16, fp32
`deblur_lora`	dropdown	DeblurNet LoRA from models/genfocus/
`bokeh_lora`	dropdown	BokehNet LoRA from models/genfocus/
`offload_to_cpu`	boolean	Enable CPU offloading to save VRAM

Output	Type	Description
`pipeline`	GENFOCUS_PIPELINE	Loaded pipeline for generation

Genfocus Unload Models

Manually unload Genfocus models from VRAM/RAM. Useful for freeing memory between runs.

Input	Type	Description
`trigger`	*	Any input to trigger execution
`unload_flux`	BOOLEAN	Unload FLUX pipeline
`unload_depth_pro`	BOOLEAN	Unload DepthPro model
`clear_cuda_cache`	BOOLEAN	Clear CUDA cache after unload

Output	Type	Description
`status`	STRING	Status message

Genfocus Deblur (Native)

Convenience node - takes a blurry image and outputs a sharp, all-in-focus result.

Input	Type	Description
`pipeline`	GENFOCUS_PIPELINE	From Genfocus Model Loader
`image`	IMAGE	Blurry/defocused input image
`steps`	INT	Denoising steps (default: 28)
`seed`	INT	Random seed

Output	Type	Description
`deblurred`	IMAGE	Sharp, all-in-focus result

Genfocus Bokeh (Native)

Convenience node - applies realistic bokeh to a sharp image using a defocus map.

Input	Type	Description
`pipeline`	GENFOCUS_PIPELINE	From Genfocus Model Loader
`image`	IMAGE	Sharp input image
`defocus_map`	IMAGE	Grayscale blur intensity map
`steps`	INT	Denoising steps (default: 28)
`seed`	INT	Random seed
`max_coc`	FLOAT	Maximum circle of confusion (blur strength)

Output	Type	Description
`bokeh_result`	IMAGE	Image with bokeh effect

Genfocus Generate (Native)

Advanced node - full control over multi-conditional generation.

Input	Type	Description
`pipeline`	GENFOCUS_PIPELINE	From Genfocus Model Loader
`prompt_preset`	dropdown	deblur, bokeh, or custom
`width` / `height`	INT	Output size (default: 1024x1024)
`steps`	INT	Denoising steps
`guidance_scale`	FLOAT	CFG scale (1.0 = no CFG)
`seed`	INT	Random seed
`condition_1/2/3`	GENFOCUS_CONDITION	Image conditions (optional)
`latents`	LATENT	Starting latents (optional, leave empty)
`main_adapter`	dropdown	auto, deblurring, bokeh, none

Output	Type	Description
`image`	IMAGE	Generated result

Genfocus Condition

Creates a condition from an image for use with Genfocus Generate.

Input	Type	Description
`image`	IMAGE	Input image
`adapter`	dropdown	deblurring or bokeh
`position_delta_x/y`	INT	Position offset for the condition
`scale`	FLOAT	Condition strength

Output	Type	Description
`condition`	GENFOCUS_CONDITION	Condition for generation

Genfocus Defocus Map Condition

Creates a condition from a defocus map for bokeh generation.

Input	Type	Description
`defocus_map`	IMAGE	Grayscale defocus map
`max_coc`	FLOAT	Maximum blur radius

Output	Type	Description
`condition`	GENFOCUS_CONDITION	Defocus condition

Depth & Utility Nodes

These standalone utility nodes work independently and can be used with any workflow.

Load DepthPro Model

Loads Apple's ml-depth-pro model for depth estimation.

Input	Type	Description
`model_path`	dropdown	Path to depth_pro.pt
`use_relative_depth`	boolean	Output relative (0-1) or metric depth

DepthPro Estimate

Generates depth maps from images.

Input	Type	Description
`depth_model`	DEPTH_PRO_MODEL	From Load DepthPro Model
`image`	IMAGE	Input image

Output	Type	Description
`depth_map`	IMAGE	Depth visualization (colorized)
`disparity_map`	IMAGE	Inverse depth visualization
`raw_depth`	IMAGE	Unnormalized metric depth (meters)
`raw_inverse`	IMAGE	Unnormalized inverse depth
`visualization`	IMAGE	3-panel analysis (image, depth, histogram)
`focal_length`	FLOAT	Estimated focal length in pixels

Depth Metric to Relative

Convert raw metric depth to normalized 0-1 range. Works with any depth source.

Input	Type	Description
`depth`	IMAGE	Raw depth map (from DepthPro, Marigold, etc.)
`per_image`	BOOLEAN	Normalize each batch image independently
`invert`	BOOLEAN	Flip near/far (near=bright for ControlNet)
`gamma`	FLOAT	Brightness bias (>1 = brighter midtones)

Output	Type	Description
`relative_depth`	IMAGE	Normalized 0-1 depth map

Depth Metric to Inverse

Simple inverse depth conversion: 1/(1+depth). Compresses far distances.

Input	Type	Description
`depth`	IMAGE	Raw metric depth

Output	Type	Description
`inverse_depth`	IMAGE	Inverse depth map

Focal Length PX to MM

Convert DepthPro's pixel-based focal length to millimeters.

Input	Type	Description
`focal_px`	FLOAT	Focal length in pixels
`sensor_mm`	FLOAT	Sensor size in mm (default: 24mm)
`image_width/height`	INT	Image dimensions

Output	Type	Description
`focal_mm`	FLOAT	Focal length in mm
`focal_str`	STRING	Formatted string (e.g., "50.00mm")

Focal Length MM to PX

Convert real camera focal length to pixels for DepthPro input.

Input	Type	Description
`focal_mm`	FLOAT	Focal length in mm (from EXIF)
`sensor_mm`	FLOAT	Sensor size in mm
`image_width/height`	INT	Image dimensions

Output	Type	Description
`focal_px`	FLOAT	Focal length in pixels
`focal_str`	STRING	Formatted string

Compute Defocus Map

Calculates blur intensity from depth and focus settings.

Input	Type	Description
`disparity`	IMAGE	From DepthPro
`focus_disparity`	FLOAT	Focus plane (0-1)
`aperture`	FLOAT	Blur strength
`max_blur`	FLOAT	Maximum blur radius

Select Focus Point

Select focus point using sliders OR a mask input for interactive selection.

Input	Type	Description
`image`	IMAGE	Input image
`mask`	MASK	Optional - from SAM, MaskPainter, etc. Centroid becomes focus point
`x_percent` / `y_percent`	FLOAT	Percentage coordinates (0-100), ignored if mask provided

Output	Type	Description
`focus_point`	FOCUS_POINT	Focus coordinates for defocus map
`preview`	IMAGE	Image with crosshair at focus point

💡 Tip: Connect a mask from Impact Pack's SAM Detector or MaskPainter for interactive point-and-click focus selection!

Focus Point From Mask

Extract focus point from any mask source. Great for interactive workflows!

Input	Type	Description
`image`	IMAGE	Input image
`mask`	MASK	Mask from SAM, MaskPainter, or any source
`fallback_x/y`	FLOAT	Fallback position if mask is empty

Output	Type	Description
`focus_point`	FOCUS_POINT	Focus coordinates
`preview`	IMAGE	Preview with mask overlay and crosshair
`x` / `y`	INT	Pixel coordinates (for debugging)

Interactive Focus Selection Workflow

┌─────────────┐     ┌─────────────────────┐     ┌─────────────────────┐
│ Load Image  │────▶│ SAM Detector        │────▶│ Focus Point From    │
│             │     │ (Impact Pack)       │     │ Mask                │
└─────────────┘     │                     │     │                     │
                    │ Click on your       │     │ Extracts centroid   │
                    │ focus target!       │     │ from mask           │
                    └─────────────────────┘     └─────────────────────┘
                                                          │
                                                          ▼
                                                ┌─────────────────────┐
                                                │ Compute Defocus Map │
                                                └─────────────────────┘

Understanding the Defocus Map

The defocus map controls how much blur is applied to each pixel based on its distance from the focus plane. This is the key to creating realistic bokeh effects.

How the Defocus Map Works

The defocus map is computed using this formula:

blur = K × |depth - depth_at_focus|

Where:

K = blur_strength parameter
depth = normalized depth value at each pixel (0-1)
depth_at_focus = depth value at your selected focus point

The result is clamped to max_coc and normalized to 0-1 for the Genfocus network.

Compute Defocus Map Outputs

Output	Description
defocus_preview	Normalized 0-1 for visualization (full contrast stretch)
defocus_map	What Genfocus actually uses (clamped at max_coc)
defocus_raw	Raw blur values before normalization

Parameter Reference

Parameter	Effect	Higher Value	Lower Value
blur_strength (K)	How fast blur increases with depth difference	More aggressive depth-dependent blur	Gentler blur gradient
max_coc	Maximum blur radius cap (Circle of Confusion)	Allows extreme blur far from focus	Caps blur earlier (subtler effect)

Recommended Settings

Effect	blur_strength	max_coc	Notes
Subtle portrait	10-20	64	Gentle background softening
Standard bokeh	20-30	100	Good for most photos
Dramatic tilt-shift	40-60	150	Strong miniature effect
Extreme separation	60-100	200+	Very shallow DOF look

What is Circle of Confusion (CoC)?

In real photography, the Circle of Confusion is the maximum size of a point of light that will still appear "sharp" to the human eye. In Genfocus:

max_coc sets the upper limit on blur intensity (in normalized units)
Higher max_coc = allows more extreme blur for objects far from focus
Lower max_coc = gentler, more subtle blur even at depth extremes

Troubleshooting Low Contrast Defocus Maps

If your defocus map appears very dark or low-contrast, this may be intentional!

Original Genfocus behavior: The original implementation does NOT normalize the defocus map to full 0-1 range. Looking at demo.py:

dmf = K * |disparity - disparity_focus|
cond_map = (dmf / MAX_COC).clamp(0, 1)  # Often results in 0-0.2 range!

The network was trained with this convention, so dark defocus maps are normal for subtle bokeh effects.

Options to control intensity:

Method	Description
Increase blur_strength (K)	Try 30-60 for more visible effect
Enable `normalize_for_genfocus`	NEW! Stretches output to full 0-1 range
Use `defocus_preview`	Already normalized for visualization

normalize_for_genfocus option:

False (default): Original Genfocus behavior - dark maps, subtle bokeh
True: Stretches to full 0-1 range - may produce stronger/different bokeh effects (experimental)

Does the Bokeh Node Change the Image If Focus Matches Original?

Yes, slightly. Even if you select a focus point matching the original photo's focus:

The FLUX diffusion process inherently modifies the image during generation
Very small depth differences still create tiny blur values
The result may appear slightly softer overall

Tip: If you want NO change to already-sharp areas, ensure your defocus map is truly zero there (check the preview).

Tips for Better Quality

Getting Finer Details

Increase inference steps - Try 32-50 steps for more refined output
Use higher resolution - FLUX works best at 1024x1024 or higher
Run deblur first - Always deblur before applying bokeh for sharpest results
Lower guidance_scale - Try 2.5-3.0 for a more natural look (less "AI" appearance)
Post-process with Richardson-Lucy - The deconvolution node can recover additional detail

Understanding the Workflow Order

For best results, follow this order:

Original Image → Deblur → Depth Estimation → Focus Selection → Defocus Map → Bokeh

Deblur restores a sharp base image (important!)
Depth Estimation analyzes the scene structure
Focus Selection picks where you want sharpness
Defocus Map calculates blur intensity per-pixel
Bokeh applies the learned depth-of-field effect

Why Deblur First?

The Genfocus bokeh network expects a sharp input. If you skip deblurring:

The network has to work harder to generate sharp in-focus areas
The original blur mixes with the generated bokeh unpredictably
Results will be softer than intended

Tips & Troubleshooting

VRAM Usage

FLUX.1-dev requires ~24GB VRAM at bf16
Enable offload_to_cpu to reduce VRAM (slower)
Use fp16 if bf16 is not supported

Image Dimensions

FLUX native resolution: 1024x1024
Other sizes work but may have quality differences
Keep dimensions multiples of 16

Common Issues

Problem	Solution
"diffusers not found"	`pip install diffusers transformers accelerate peft`
"FLUX model not found"	Place in `models/diffusers/FLUX.1-dev/` or use HF login
"LoRA not loading"	Check files are in `models/genfocus/` folder
Out of memory	Enable `offload_to_cpu` or use smaller resolution

Licenses

Component	License	Commercial Use
Refocus nodes	Apache 2.0	✅ Yes
Genfocus LoRAs	Apache 2.0	✅ Yes
Apple ml-depth-pro	Apple Sample Code	⚠️ Research only
FLUX.1-dev	Non-Commercial	❌ Requires license

Technical Notes

Why Diffusers Instead of Native ComfyUI FLUX?

We initially attempted to create a native ComfyUI sampler that would allow using any FLUX fine-tune loaded in ComfyUI with Genfocus LoRAs. This would have been ideal — no need for a separate diffusers FLUX download, and compatibility with all FLUX variants.

The challenge: Genfocus LoRAs were trained on the diffusers FLUX architecture, which uses separate Q, K, V projection layers:

transformer.single_transformer_blocks.0.attn.to_q
transformer.single_transformer_blocks.0.attn.to_k
transformer.single_transformer_blocks.0.attn.to_v

ComfyUI's FLUX implementation uses fused QKV layers for efficiency:

single_blocks.0.linear1  ← Contains Q, K, V, and MLP fused together

Applying LoRA weights trained for separate Q/K/V to a fused layer requires:

Splitting the LoRA weights to the correct offsets within the fused layer
Ensuring the math works out correctly for the different projections

We implemented key mapping and offset-based patching using ComfyUI's built-in mechanisms, but the results showed characteristic spotted corruption patterns, indicating the LoRA weights weren't being applied correctly to the fused architecture.

Current status: The native ComfyUI sampler approach is abandoned for now. The diffusers-based nodes work correctly because they use the exact architecture the LoRAs were trained on.

Future possibility: If someone trains Genfocus-style LoRAs directly on ComfyUI's FLUX architecture, or develops proper weight conversion utilities, native sampling could become viable.

Credits

Genfocus: NYCU CP Lab - Core algorithm and LoRAs
Paper: Generative Refocusing of Radiance Fields
ml-depth-pro: Apple - Depth estimation
FLUX: Black Forest Labs - Base model

Citation

@article{Genfocus2025,
  title={Generative Refocusing: Flexible Defocus Control from a Single Image},
  author={Tuan Mu, Chun-Wei and Huang, Jia-Bin and Liu, Yu-Lun},
  journal={arXiv preprint arXiv:2512.16923},
  year={2025}
}

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
nodes		nodes
pipeline		pipeline
scripts		scripts
utils		utils
workflows		workflows
.gitignore		.gitignore
LICENSE		LICENSE
LICENSES.md		LICENSES.md
README.md		README.md
__init__.py		__init__.py
download_models.bat		download_models.bat
package.json		package.json
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt

License

EricRollei/comfyui-refocus

Folders and files

Latest commit

History

Repository files navigation

Refocus - ComfyUI Nodes for Generative Refocusing

Features

Example Workflows

🔄 Refocus Workflow

Workflow Diagram

Input Image

Results

Key Concepts

🔍 Simple Deblur Workflow

Workflow Diagram

Before & After

Notes

Installation

1. Clone the Repository

2. Install Base Dependencies

3. Install DepthPro (Optional - for depth estimation)

4. Install Native Genfocus Dependencies (Recommended)

Model Setup

Folder Structure

Download Links

Node Reference

Genfocus Nodes (diffusers-based)

Genfocus Model Loader (Native)

Genfocus Unload Models

Genfocus Deblur (Native)

Genfocus Bokeh (Native)

Genfocus Generate (Native)

Genfocus Condition

Genfocus Defocus Map Condition

Depth & Utility Nodes

Load DepthPro Model

DepthPro Estimate

Depth Metric to Relative

Depth Metric to Inverse

Focal Length PX to MM

Focal Length MM to PX

Compute Defocus Map

Select Focus Point

Focus Point From Mask

Interactive Focus Selection Workflow

Understanding the Defocus Map

How the Defocus Map Works

Compute Defocus Map Outputs

Parameter Reference

Recommended Settings

What is Circle of Confusion (CoC)?

Troubleshooting Low Contrast Defocus Maps

Does the Bokeh Node Change the Image If Focus Matches Original?

Tips for Better Quality

Getting Finer Details

Understanding the Workflow Order

Why Deblur First?

Tips & Troubleshooting

VRAM Usage

Image Dimensions

Common Issues

Licenses

Technical Notes

Why Diffusers Instead of Native ComfyUI FLUX?

Credits

Citation

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 1

Packages 0

Languages

Packages