Skip to content

support forge models#668

Open
rfatimaTT wants to merge 6 commits intodevfrom
rumeza/forge-models
Open

support forge models#668
rfatimaTT wants to merge 6 commits intodevfrom
rumeza/forge-models

Conversation

@rfatimaTT
Copy link
Collaborator

Process

  • Add Forge-specific docker runtime config (MODEL_RUNNER, device env, mesh descriptor selection, hugepage mount behavior).
  • Build tt-forge-server:latest during setup in run.py from artifact tt-media-server/Dockerfile.forge and then models create containers from it

@rfatimaTT rfatimaTT requested a review from anirudTT March 2, 2026 22:55
@anirudTT
Copy link
Collaborator

@rfatimaTT : can you add some screengrabs on what this should look like ?

@anirudTT
Copy link
Collaborator

Current UI for forge models working on n300

Screenshot 2026-03-17 at 2 19 09 PM

Keep build_forge_docker_image() from forge-models branch alongside
the stricter sys.exit(1) error handling introduced in dev.

Made-with: Cursor
Resolved conflict in run.py by keeping the Forge CNN Docker image build
step that was added on this branch.

Made-with: Cursor
@anirudTT
Copy link
Collaborator

Tested the models to work.

Screenshot 2026-03-24 at 11 41 04 AM

return device_name


def _get_tt_device_ids():
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we adding this

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

infer server version was lacking device mappings so it only showed the models as compatible for n300 or n150, when in reality they are supported in QB as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants