Initial Mistral implementation by OriPekelman · Pull Request #43 · crmne/ruby_llm

OriPekelman · 2025-03-18T13:48:18Z

Tried to follow existing implementation as much as possible and hopefully integrated correctly.

crmne · 2025-03-23T16:00:28Z

Thank you @OriPekelman for your contribution!

Two new features to consider in your provider implementation:

Model Aliases: Please add entries for your provider in aliases.json:

"claude-3-5-sonnet": {
  "anthropic": "claude-3-5-sonnet-20241022",
  "your-provider": "your-provider-specific-id"
}

Provider Selection: Users will be able to specify your provider:

chat = RubyLLM.chat(model: 'claude-3-5-sonnet', provider: 'your-provider')

Docs: https://rubyllm.com/guides/models#using-model-aliases

Please ask me for a code review when you're ready.

crmne · 2025-03-25T10:47:47Z

Added configuration requirements handling in 75f99a1

Each provider now specifies what configuration is required via a simple configuration_requirements method (you will need to implement this in your main provider file) that returns an array of config keys as symbols. The Provider module uses this to:

Determine if a provider is properly configured
Throw an error if you're trying to use that provider without configuration
Include ready-to-paste configuration code in the error message
Skip unconfigured providers during model refresh while preserving their models

Example of the new error messages:

RubyLLM::ConfigurationError: anthropic provider is not configured. Add this to your initialization:

RubyLLM.configure do |config|
  config.anthropic_api_key = ENV['ANTHROPIC_API_KEY']
end

crmne · 2025-04-17T12:06:12Z

@OriPekelman is this still on your radar? I'd love to merge Mistral support soon. Whenever you are ready, could you resolve the conflicts and rerequest a review? Thanks!

OriPekelman · 2025-04-17T12:50:53Z

Yup, I'll try to get to it very soon.

crmne · 2025-04-23T20:14:03Z

Hi @OriPekelman is Mistral OpenAI compatible?

OriPekelman · 2025-04-27T21:20:24Z

Hi @OriPekelman is Mistral OpenAI compatible?

Well, like most providers, it has some basic OpenAI compatability ... but only for chat completions - and even there you are going to have quite a bit of differences in tool calling and multi-turn behaviour.

I rebased the whole thing and cleaned it up quite a bit + added the configure.

There are a number of tests I had to skip (for example Mistral does not support custom dimensions for embedding, the tool calling in multi-turn conversations doesn't support adding tools mid conversation, only the last chunk in a stream would have token count).

Hopefully this is useful.

crmne

Good stuff! I left you some comments

crmne · 2025-04-28T13:56:25Z

lib/ruby_llm/providers/mistral/chat.rb

+        def completion_url
+          "#{Mistral.api_base(RubyLLM.config)}/chat/completions"
+        end


you don't need to pass the api base here

Check out the other provider's code. You only need the path part of the url.

crmne · 2025-04-28T13:56:44Z

lib/ruby_llm/providers/mistral/chat.rb

+              Array(tools)
+            end
+
+          puts "\n[DEBUG] Available tools: #{tools_array&.map { |t| t.name.to_s }}" if ENV["DEBUG"]


Use RubyLLM.logger.debug

crmne · 2025-04-28T13:56:53Z

lib/ruby_llm/providers/mistral/chat.rb

+              "none"
+            end
+
+          puts "[DEBUG] Tool choice: #{effective_tool_choice.inspect}" if ENV["DEBUG"]


crmne · 2025-04-28T13:56:59Z

lib/ruby_llm/providers/mistral/chat.rb

+            frequency_penalty: frequency_penalty,
+          }.compact
+
+          puts "[DEBUG] Full payload: #{payload.inspect}" if ENV["DEBUG"]


crmne · 2025-04-28T13:57:16Z

lib/ruby_llm/providers/mistral/chat.rb

+              arguments: tool_call.arguments,
+            },
+          }
+          puts "[DEBUG] Rendered tool call: #{tool_call_spec.inspect}" if ENV["DEBUG"]


crmne · 2025-04-28T14:00:01Z

spec/ruby_llm/chat_spec.rb


      it "#{provider}/#{model} successfully uses the system prompt" do
        skip 'System prompt can be flaky for Ollama models' if provider == :ollama
+        skip 'Mistral API does not allow system messages after assistant messages' if provider == :mistral


but in this case the chat should start with the system message, right?

crmne · 2025-04-28T14:00:33Z

spec/ruby_llm/chat_streaming_spec.rb

+        skip 'Mistral API only returns token count on the last chunk.' if provider == :mistral
+


also the other APIs, so don't worry

crmne · 2025-04-28T14:00:54Z

spec/ruby_llm/chat_tools_spec.rb

      model = model_info[:model]
      provider = model_info[:provider]
      it "#{provider}/#{model} can use tools" do # rubocop:disable RSpec/MultipleExpectations
+        skip 'Mistral does not reliably support tool usage' if provider == :mistral


really? that would be super bad! in what way is it not reliable?

OK I addressed the mistral too hallucination issue - from what I can see it only happens with the small model. At any rate it no longer breaks the implementation. It tends to believe it has a "web_search" tool whether you have one or not.

crmne · 2025-04-28T14:02:05Z

spec/ruby_llm/embeddings_spec.rb

      end

      it "#{provider}/#{model} can handle a single text with custom dimensions" do # rubocop:disable RSpec/MultipleExpectations
+        skip "Mistral embed does not support custom dimensions" if model == "mistral-embed"


you can do if provider == :mistral

Went for provider, but if I am not mistaken it is a string here.

crmne · 2025-04-28T14:02:12Z

spec/ruby_llm/embeddings_spec.rb

      end

      it "#{provider}/#{model} can handle multiple texts with custom dimensions" do # rubocop:disable RSpec/MultipleExpectations
+        skip "Mistral embed does not support custom dimensions" if model == "mistral-embed"


OriPekelman · 2025-05-05T20:29:56Z

Oh and now i'll try to do a clean rebase.

OriPekelman · 2025-05-05T20:43:41Z

I see there are some new tests, I'll see if I can implement the local image stuff.

OriPekelman · 2025-05-07T21:06:54Z

OK this should work now (given that models.json is up-to-date) for the failing vision tests.

OriPekelman · 2025-05-29T15:27:17Z

OK rebased on main I did not commit the changes to models.json (as it is a source of possible merge conflicts) - but the update seems to work .. and the tests look green after I run rake models:update I ran the test with both mistral small and medium.

crmne

I'm confused as to why we need so much code for the Mistral provider. Isn't it OpenAI compatible? In that case, please take a look at already existing OpenAI compatible providers like Ollama and OpenRouter.

crmne · 2025-06-10T10:54:24Z

lib/ruby_llm.rb

  'deepseek' => 'DeepSeek',
  'bedrock' => 'Bedrock',
  'openrouter' => 'OpenRouter',
-  'pdf' => 'PDF'


why remove this inflection?

OOps. That was part of me removing the PDF support for mistral (I added it with their dedicated end-point https://docs.mistral.ai/capabilities/OCR/basic_ocr/ but decided against it before the main functionality gets merged. My excuses.

crmne · 2025-06-10T10:55:03Z

lib/ruby_llm.rb

  'bedrock' => 'Bedrock',
  'openrouter' => 'OpenRouter',
-  'pdf' => 'PDF'
+  'mistral' => 'Mistral'


I realize we didn't need bedrock as an inflection, and so we don't need mistral as one either

OK will remove that one too.

crmne · 2025-06-10T10:56:09Z

lib/ruby_llm/providers/mistral/chat.rb

+        def completion_url
+          "#{Mistral.api_base(RubyLLM.config)}/chat/completions"
+        end


Check out the other provider's code. You only need the path part of the url.

crmne · 2025-06-10T11:11:46Z

lib/ruby_llm/providers/mistral.rb

I'm confused as to why we need so much code for the Mistral provider. Isn't it OpenAI compatible? In that case, please take a look at already existing OpenAI compatible providers like Ollama and OpenRouter.

crmne · 2025-06-10T11:12:31Z

lib/tasks/capability_scraper.rb

I guess this is a merge error, but we don't need this anymore thanks to Parsera.

crmne · 2025-06-10T11:12:44Z

lib/tasks/model_updater.rb

This looks like a merge error too.

…ration - Updated model aliases in aliases.json to include specific paths for Mistral models. - Added `mistral_api_base` configuration option in configuration.rb. - Introduced new models in models.json, including Mistral Tiny, Pixtral Large, and others. - Refactored Mistral provider methods to support new API structure and error handling. - Removed deprecated image generation methods from the Mistral provider. - Updated tests to reflect changes in model handling and API responses.

…chat provider - Updated debug output from `puts` to `RubyLLM.logger.debug` for better logging management. - Commented out skip conditions in tests for Mistral provider to avoid flaky behavior during testing.

… cassettes - Refactored the chat provider to be closer to the other implementations. - No longer ignoring specs except custom dimensions. - Refactored error message formatting in the Mistral provider. - Mistral (small) may hallucinate tools which explains previous failures. Added robustness for that. - Updated chat and streaming files to include necessary modules and methods. - Removed outdated VCR cassettes related to Mistral functionality. - Updated test fixtures to reflect changes in API responses and model handling. - Added frozen string literal to Mistral provider files for performance.

crmne · 2025-07-31T11:20:16Z

Went ahead and implemented it in 92d5451

OriPekelman mentioned this pull request Mar 18, 2025

Include Mistral AI #7

Closed

crmne added the new provider New provider integration label Mar 21, 2025

crmne linked an issue Mar 23, 2025 that may be closed by this pull request

Include Mistral AI #7

Closed

OriPekelman force-pushed the mistral branch from 53be515 to 883ecfc Compare April 27, 2025 21:12

crmne requested changes Apr 28, 2025

View reviewed changes

OriPekelman force-pushed the mistral branch from 8b2bc04 to 433ec40 Compare May 5, 2025 20:41

OriPekelman marked this pull request as draft May 5, 2025 20:43

OriPekelman marked this pull request as ready for review May 5, 2025 20:44

OriPekelman force-pushed the mistral branch from 775f377 to 31a5cc9 Compare May 29, 2025 15:23

OriPekelman requested a review from crmne May 29, 2025 15:24

crmne requested changes Jun 10, 2025

View reviewed changes

OriPekelman added 9 commits June 10, 2025 16:04

Base mistral implementation

d2ee337

refactor(logging): Replace debug prints with logger calls in Mistral …

13f1493

…chat provider - Updated debug output from `puts` to `RubyLLM.logger.debug` for better logging management. - Commented out skip conditions in tests for Mistral provider to avoid flaky behavior during testing.

Added Mistral to model update tasks

53451dd

resolve image test failures

580ff8d

Try to make model capabilities more robust. Add to model update task

2076f27

Apply the attachments refactoring. Rebase on origin/main

a396bd6

Update models.json

83adc4e

OriPekelman force-pushed the mistral branch from 31a5cc9 to 83adc4e Compare June 10, 2025 14:10

Add back pdf and remove mistral

96001a7

OriPekelman force-pushed the mistral branch from 43d9496 to 96001a7 Compare June 10, 2025 14:13

crmne closed this Jul 31, 2025

		skip 'Mistral API only returns token count on the last chunk.' if provider == :mistral

Uh oh!

Conversation

OriPekelman commented Mar 18, 2025

Uh oh!

crmne commented Mar 23, 2025

Uh oh!

crmne commented Mar 25, 2025

Uh oh!

crmne commented Apr 17, 2025

Uh oh!

OriPekelman commented Apr 17, 2025

Uh oh!

crmne commented Apr 23, 2025

Uh oh!

OriPekelman commented Apr 27, 2025

Uh oh!

crmne left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

OriPekelman commented May 5, 2025

Uh oh!

OriPekelman commented May 5, 2025

Uh oh!

OriPekelman commented May 7, 2025

Uh oh!

OriPekelman commented May 29, 2025

Uh oh!

crmne left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

crmne commented Jul 31, 2025

Uh oh!