Skip to content

Conversation

@polvalente
Copy link
Collaborator

@polvalente polvalente commented Oct 23, 2025

With Nx.Defn.Evaluator's debugging features from Nx 0.10 I was able to track this down.
This was the last mile for getting Stable Diffusion to generate a valid image, tested by running
the issue's script as a standalone exs script.

output

closes #70 (finally!)

Test script to be placed at the root of emlx

Mix.install([
   {:bumblebee, github: "elixir-nx/bumblebee", override: true},
  {:emlx, path: __DIR__},
  {:stb_image, "~> 0.1"},
  {:kino_bumblebee, "~> 0.5.1"}
])

Nx.global_default_backend({EMLX.Backend, device: :gpu})

repository_id = "CompVis/stable-diffusion-v1-4"
{:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, "openai/clip-vit-large-patch14"})
{:ok, clip} = Bumblebee.load_model({:hf, repository_id, subdir: "text_encoder"})
{:ok, unet} = Bumblebee.load_model({:hf, repository_id, subdir: "unet"})

{:ok, vae} =
  Bumblebee.load_model({:hf, repository_id, subdir: "vae"}, architecture: :decoder)

{:ok, scheduler} = Bumblebee.load_scheduler({:hf, repository_id, subdir: "scheduler"})

{:ok, featurizer} =
  Bumblebee.load_featurizer({:hf, repository_id, subdir: "feature_extractor"})

{:ok, safety_checker} =
  Bumblebee.load_model({:hf, repository_id, subdir: "safety_checker"})

serving =
  Bumblebee.Diffusion.StableDiffusion.text_to_image(clip, unet, vae, tokenizer, scheduler,
    num_steps: 40,
    num_images_per_prompt: 1,
    safety_checker: safety_checker,
    safety_checker_featurizer: featurizer,
    compile: [batch_size: 1, sequence_length: 50]
  )

IO.puts("running...")
%{results: [%{image: tensor}]} = Nx.Serving.run(serving, %{prompt: "numbat, forest, high quality, detailed, digital art", seed: 1})

tensor |> StbImage.from_nx() |> StbImage.write_file("/tmp/output.png")

@polvalente polvalente self-assigned this Oct 23, 2025
@polvalente polvalente merged commit 4e927d1 into main Oct 23, 2025
7 checks passed
@polvalente polvalente deleted the fix/mlx-scalar-item-bug branch October 23, 2025 01:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Always grey images when using Stable Diffusion with Bumblebee

2 participants