Compose mode and think flags

[ilopezluna]

Adds mode and think flags into Compose integration to allow run embedding models and define the reasoning budged

[gemini-code-assist]

Summary of Changes

Hello @ilopezluna, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the compose up command by introducing new command-line flags that provide greater control over model behavior and backend operations. Users can now explicitly define the backend's mode, such as completion or embedding, and fine-tune the reasoning budget for models, which is particularly useful for optimizing performance and resource usage. These changes make the CLI more flexible and powerful for deploying and interacting with various model types.

Highlights

• New CLI Flags: Introduced two new flags to the compose up command: --mode for specifying the backend operation mode (completion, embedding, reranking) and --think for controlling the reasoning budget of models (true, false, high, medium, low).
• Backend Configuration: Integrated the new --mode and --think parameters into the backend configuration, allowing users to configure the model's operation mode and set a specific reasoning budget for llama.cpp models.
• Parsing Logic and Validation: Added robust parsing functions (parseBackendMode and parseThinkToReasoningBudget) to convert string inputs from the new flags into appropriate internal types, including error handling for invalid values.
• Comprehensive Unit Tests: Included a new test file (compose_test.go) with extensive unit tests for the new parsing logic, ensuring the correct behavior and validation of the --mode and --think flag inputs.
• Documentation Update: Updated the CLI reference documentation (docker_model_compose_up.yaml) to reflect the addition of the new --mode and --think flags, providing clear descriptions and accepted values.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces mode and think flags to the compose up command, allowing users to specify the backend operation mode and reasoning budget. The changes include adding new flags, parsing logic, and updating the backend configuration. New tests are added to verify the parsing logic for the new flags, and the command's documentation is updated accordingly. My review includes a couple of suggestions to improve code robustness and reduce duplication. Overall, the changes are well-implemented and tested.

[doringeman]

This is used for the provider approach which is deprecated: https://docs.docker.com/ai/compose/models-and-compose/#alternative-configuration-with-provider-services.
We need to do the same for configure as this is the one currently used: https://github.com/docker/compose/blob/dc66e6bad1439f8152a1ab3cf7a831f1f6454027/pkg/compose/model.go#L172.