Enabling Claude Sonnet thinking with Bedrock

[brickZA]

Hi,

My work provides LLM access through Bedrock. That seems to be working great with gptel, thanks!

However, I'd like to use Sonnet's thinking mode. I see there is a way to set up a custom backend when using Anthropic's APIs directly in #663, but it also seems like the Bedrock request look quite different and so I'm guessing that specific backend stanza won't work? How could I do something similar with bedrock? At the very least, how can I generate the equivalent of the default bedrock equivalent as a new backend so I can try and hack it in myself?

This is my current config:

  (gptel-make-bedrock "AWS"
    ;; optionally enable streaming
    :stream t
    :region "eu-west-1"
    ;; subset of gptel--bedrock-models
    :models '(claude-sonnet-4-20250514)
    ;; Model region for cross-region inference profiles. Required for models such
    ;; as Claude without on-demand throughput support. One of 'apac, 'eu or 'us.
    ;; https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-use.html
    :model-region 'eu)
  ;; OPTIONAL configuration
  (setq gptel-model   'claude-sonnet-4-20250514
        gptel-backend
        (gptel-make-bedrock "AWS"
          ;; optionally enable streaming
          :stream t
          :region "eu-west-1"
          ;; subset of gptel--bedrock-models
          :models '(claude-sonnet-4-20250514)
          ;; Model region for cross-region inference profiles. Required for models such
          ;; as Claude without on-demand throughput support. One of 'apac, 'eu or 'us.
          ;; https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-use.html
          :model-region 'eu))
  )

[karthink]

Does adding some :request-params work?

(gptel-make-bedrock "AWS"
  :stream t
  :region "eu-west-1"
  :models '(claude-sonnet-4-20250514)
  :model-region 'eu
  :request-params '( :thinking (:type "enabled"
                                :budget_tokens 2048)
                     :max_tokens 4096))

[brickZA]

Hi,

Nope, I seem to get this:

Debugger entered--Lisp error: (error "Keyword argument :request-params not one of (:region :models :model-region :stream :curl-args :protocol)")
  error("Keyword argument %s not one of (:region :models :model-region :stream :curl-args :protocol)" :request-params)
  gptel-make-bedrock("AWS" :stream t :region "eu-west-1" :models (claude-sonnet-4-20250514) :model-region eu :request-params (:thinking (:type "enabled" :budget_tokens 2048) :max_tokens 4096))
  eval((gptel-make-bedrock "AWS" :stream t :region "eu-west-1" :models '(claude-sonnet-4-20250514) :model-region 'eu :request-params '(:thinking (:type "enabled" :budget_tokens 2048) :max_tokens 4096)) nil)
  elisp--eval-last-sexp(nil)
  #f(compiled-function () #<bytecode 0xbeeb6604d37a>)()
  #<subr eval-last-sexp>(nil)
  #f(compiled-function (&rest _it) #<bytecode 0xab10489a4eb3d2>)()
  eval-sexp-fu-flash-doit-simple(#f(compiled-function (&rest _it) #<bytecode 0xab10489a4eb3d2>) #f(compiled-function (&rest args2) #<bytecode -0x3fa935570039c84>) #f(compiled-function (&rest args2) #<bytecode -0x3fc37ae401d9c84>))
  eval-sexp-fu-flash-doit(#f(compiled-function (&rest _it) #<bytecode 0xab10489a4eb3d2>) #f(compiled-function (&rest args2) #<bytecode -0x3fa935570039c84>) #f(compiled-function (&rest args2) #<bytecode -0x3fc37ae401d9c84>))
  esf-flash-doit(#f(compiled-function (&rest _it) #<bytecode 0xab10489a4eb3d2>) #f(compiled-function (&rest args2) #<bytecode -0x3fa935570039c84>) #f(compiled-function (&rest args2) #<bytecode -0x3fc37ae401d9c84>) #f(compiled-function (&rest args2) #<bytecode 0x1b7d1ada5f8e6314>))
  ad-Advice-eval-last-sexp(#<subr eval-last-sexp> nil)
  apply(ad-Advice-eval-last-sexp #<subr eval-last-sexp> nil)
  eval-last-sexp(nil)
  funcall-interactively(eval-last-sexp nil)
  command-execute(eval-last-sexp)

I'll spend a couple of minutes to see if I can understand the code well enough to hack it in in the meantime, but I'll probably need help :)

[karthink]

gptel-make-bedrock was not setting request-params, I added it. You can update gptel and try again.

[brickZA]

Hi,

Looks like the update to gptel did not plumb the parameters all the way through. I think I've manage to fix it with this PR: #990. Furthermore the request parameters are slightly different, this config worked:

  (setq gptel-model   'claude-3-7-sonnet-20250219 ;'claude-sonnet-4-20250514
        gptel-backend
        (gptel-make-bedrock "AWS"
          ;; optionally enable streaming
          :stream t
          :region "eu-west-1"
          ;; subset of gptel--bedrock-models
          :models '(claude-sonnet-4-20250514 claude-3-7-sonnet-20250219)
          ;; Model region for cross-region inference profiles. Required for models such
          ;; as Claude without on-demand throughput support. One of 'apac, 'eu or 'us.
          ;; https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-use.html
          :request-params '(:additionalModelRequestFields (:thinking (:type "enabled" :budget_tokens 2048))
                      :inferenceConfig (:max_tokens 4096))
          :model-region 'eu)

In fact, I was quite close to having it work with my previous hack -- I think the only issue was that the total token budget needs to be larger than the thinking budget and the thinking budget must be at least 1024 IIRC.

It does not look like the Bedrock parser understands thinking yet, but I can confirm that it worked by looking at the debug output. The Bedrock response now contains reasoningContent events (after running through the strings command):

event{"contentBlockIndex":0,"delta":{"reasoningContent":{"text":"The"}},"p":"abcdef"}
...
event{"contentBlockIndex":0,"delta":{"reasoningContent":{"text":" user is just saying \""}},"p":"abcdefghijklmnopqrstuvw"}
...
event{"contentBlockIndex":0,"delta":{"reasoningContent":{"text":"Testing!\". This looks like they"}},"p":"abcdefghijkl"}
...

[brickZA]

Just to test it out I hacked some changes into gptel-bedrock.el
Reverse-engineering the Java SDK example from https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-runtime_example_bedrock-runtime_Converse_AnthropicClaudeReasoning_section.html and cross-referencing the API docs, I really thought this should work:

--- a/gptel-bedrock.el
+++ b/gptel-bedrock.el
@@ -60,13 +60,61 @@
 (cl-defmethod gptel--request-data ((backend gptel-bedrock) prompts)
   "Prepare request data for AWS Bedrock BACKEND from PROMPTS."
   (nconc
-   `(:messages [,@prompts] :inferenceConfig (:maxTokens ,(or gptel-max-tokens 500)))
+   `(:messages [,@prompts] :inferenceConfig (:maxTokens ,(or gptel-max-tokens 500))
+                :additionalModelRequestFields(:thinking (:type "enabled" :budget_tokens 2048))
    (when gptel--system-message `(:system [(:text ,gptel--system-message)]))
    (when gptel-temperature `(:temperature ,gptel-temperature))
    (when (and gptel-use-tools gptel-tools)
      `(:toolConfig (:toolChoice ,(if (eq gptel-use-tools 'force) '(:any '()) '(:auto '()))
                     :tools ,(gptel--parse-tools backend gptel-tools))))))

but I just get AWS error: (HTTP/2 400) Could not parse HTTP response. when I add that.

OK, now I'm very confused, back to unmodified, and still getting the 400s. I guess I'm not fixing this today ;)