SAM3 onnx infer no result

[demo-telo]

Here is my process:

• First, I used the transform library to convert sam3.pt（sam3 offi） into the Transform library's format.
  result like:

-> Second,I'm used the "exportv2.py" file you provided to get onnx-format model, the parameters follow those provided in your example.
like:
--all --model-path /home/ubuntu/program/usls-main/checkpoints/converted_sam3_model --output-dir /home/ubuntu/program/usls-main/onnx-models-v2 --device cuda --image-height 1008 --image-width 1008
result（no error to report）:

->But,when I ues “inference_v2.py” to test the onnx-format model， it get on result in kid.jpg using "shoe" prompt. I have noticed the name of onnx-format model used in “inference_v2.py” is not the same with "exportv2.py"'s result.
like:
.
My command like:
"python3 /home/ubuntu/program/usls-main/scripts/sam3-image/inference_v2.py --image ../../assets --text kids --model-dir /home/ubuntu/program/usls-main/onnx-models-v2 --tokenizer /home/ubuntu/program/usls-main/checkpoints/converted_sam3_model/tokenizer.json --output output-text-v2.png --device cpu --image-height 1008 --image-width 1008 "

Besides, I have tried the v1 version process, but it didn't work either.

So, where might the problem lie? I hope you can guide me. Thank you.

[demo-telo]

To add to the above, there were no errors during the process, only no results (neither boxes nor masks). The inference output looked like this: /home/ubuntu/anaconda3/envs/sam3/bin/python3 /home/ubuntu/program/usls-main/scripts/sam3-image/inference_v2.py --image ../../assets --text shoe --model-dir /home/ubuntu/program/usls-main/onnx-models-v2 --tokenizer /home/ubuntu/program/usls-main/checkpoints/converted_sam3_model/tokenizer.json --output output-text-v2.png --device cpu --image-height 1008 --image-width 1008
../../assets/kids.jpg

进程已结束，退出代码为 0

[jamjamjon]

@demo-telo This is only a demo for now. You can run it first to check whether everything works correctly.
I’ve tested it and it runs fine on my side.

V1

--image must be a single image file (directories are not supported).

uv run inference.py \
    --image ../../assets/kids.jpg \
    --text "shoe" \
    --model-dir ./onnx-models \
    --tokenizer /path/to/tokenizer.json \
    --output output-text-v1.png

V2

--image accepts a directory of images.

uv run inference_v2.py \
    --image ../../assets/kids.jpg \
    --text "shoe" \
    --model-dir ./onnx-models-v2 \
    --tokenizer /path/to/tokenizer.json \
    --output output-text-v2.png \
    --device cuda \
    --image-height 1008 \
    --image-width 1008

You may require a GPU or more CPU memory, especially when processing high-resolution images or directories.

[demo-telo]

Everything looks normal, except for the no detection results. I can provide the output of v1,like:

which clearly shows that the expected objects were not found (the print statement for v2 simply tells me that it ran normally and saved the image, but similarly, the expected objects were not found on the image).
Due to concerns about CUDA version compatibility, I am using CPUExecutionProvider.
results:

[jamjamjon]

@demo-telo I can run inference normally using the CPU execution provider as well, so this is likely an environment issue on your side.

1. Download the weights directly from here:
   https://github.com/jamjamjon/assets/releases/tag/sam3
2. Use the same Python environment as mine:
   

   usls/scripts/sam3-image/pyproject.toml

   Lines 1 to 14 in 1e8e89c

   	[project]
   	name = "sam3-tools"
   	version = "0.1.0"
   	requires-python = ">=3.10"
   	dependencies = [
   	"torch",
   	"onnx",
   	"onnxruntime-gpu",
   	"transformers @ git+https://github.com/huggingface/transformers@2a61590a479d3b1f77059f75caee7cc22760019d",
   	"numpy>=2.2.6",
   	"opencv-python>=4.11.0.86",
   	"torchvision>=0.24.1",
   	]

[demo-telo]

Thank you for your reply. However, if the pipeline doesn't work, I can't convert my fine-tuned sam3 model into the final format that can run faster (pt->onnx->trt). So I will check my environment and try again later.

[jamjamjon]

If you have fine-tuned SAM3, please refer to #227.