Skip to content
This repository was archived by the owner on Sep 11, 2025. It is now read-only.

Conversation

@kevinmingtarja
Copy link
Contributor

@kevinmingtarja kevinmingtarja commented Jan 14, 2025

Description

This PR adds meta-llama/Llama-3.2-3B-Instruct to the list of models available during local development.

Edit: This PR removes the localHypermodeModels list, so that we don't need to push new commits and version updates when making new models available for local dev.

Instead, we let the users specify any model they want, and if it's not available, our models gateway will return a 404, and the Runtime code will handle it accordingly.

Screenshot 2025-01-17 at 1 45 33 AM

Checklist

All PRs should check the following boxes:

  • I have given this PR a title using the
    Conventional Commits syntax, leading with fix:,
    feat:, chore:, ci:, etc.
    • The title should also be used for the commit message when the PR is squashed and merged.
  • I have formatted and linted my code with Trunk, per the instructions in
    the contributing guide.

If the PR includes a code change, then also check the following boxes. (If not, then delete the
next section.)

  • I have added an entry to the CHANGELOG.md file.
    • Add to the "UNRELEASED" section at the top of the file, creating one if it doesn't yet exist.
    • Be sure to include the link to this PR, and please sort the section numerically by PR number.
  • I have manually tested the new or modified code, and it appears to behave correctly.

@kevinmingtarja kevinmingtarja requested a review from a team January 14, 2025 19:42
@kevinmingtarja kevinmingtarja enabled auto-merge (squash) January 14, 2025 19:45
@mattjohnsonpint
Copy link
Contributor

Approved, but I really think we should just remove this list entirely. Let the user try any model. Fail with the appropriate HTTP code if it's not available, then handle that error accordingly. Then we won't need to push new versions to update the list of models.

@kevinmingtarja
Copy link
Contributor Author

Approved, but I really think we should just remove this list entirely. Let the user try any model. Fail with the appropriate HTTP code if it's not available, then handle that error accordingly. Then we won't need to push new versions to update the list of models.

yeah let's just do that in this PR actually.

@kevinmingtarja kevinmingtarja changed the title chore: add meta-llama/llama-3.2-3b-instruct to localHypermodeModels chore: remove localHypermodeModels list and handle 404s properly instead in local dev Jan 16, 2025
@kevinmingtarja kevinmingtarja merged commit d7bc852 into main Jan 16, 2025
49 checks passed
@kevinmingtarja kevinmingtarja deleted the kevinm/add-llama-3.2 branch January 16, 2025 22:25
@mattjohnsonpint
Copy link
Contributor

Thanks! 🚀

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants