Skip to content

Conversation

serialx
Copy link
Contributor

@serialx serialx commented Sep 22, 2025

Summary
In rare cases, Claude Sonnet 4 with extended thinking emits multiple thinking blocks, each with its own signature. Our converter previously kept only the last signature. This change collects all signatures in order and joins them with \n, enabling proper reconstruction/verification of each block.

Before
Input: two thinking blocks with signatures → output stored only the last signature.

After
Input: two+ thinking blocks with signatures → output stores all signatures joined by newline (e.g., "sig1\nsig2\nsig3").

Implementation

  • Accumulate signatures during parsing instead of overwriting.
  • Store the result as a newline-delimited string at encrypted_content

Tests

  • multiple blocks all w/ signatures (order preserved)

Compatibility / Risk

  • No behavior change for single-block responses.
  • Low risk; perf impact negligible.

…erter

Previously only the last signature was preserved when multiple thinking blocks
had signatures. Now all signatures are collected and stored with newline
delimiters, ensuring proper reconstruction of each thinking block.
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex Review: Here are some suggestions.

About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".

Comment on lines 530 to +535
"type": "thinking",
"thinking": content_item.get("text", ""),
}
# Add signature if available
if signature:
thinking_block["signature"] = signature
# Add signatures if available
if signatures:
thinking_block["signature"] = signatures.pop(0)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve signature alignment when some thinking blocks lack signatures

message_to_output_items now collects only the signatures that are present and stores them as a newline‑joined string, but items_to_messages reconstructs thinking blocks by popping from that list for every reasoning block in order. If a response contains multiple thinking blocks where some early blocks do not have signatures, this logic shifts the later signatures onto earlier blocks and drops any trailing ones (e.g., blocks [no sig, sigA, sigB] become [sigA, sigB, none]). This produces incorrect signature/block pairing and will break downstream verification whenever not all blocks carry a signature. Consider storing signature placeholders (e.g., None) for blocks without signatures or associating signatures with their block index before reconstruction.

Useful? React with 👍 / 👎.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reasoning blocks should always contain signatures, this is basically a N:1 conversion and 1:N back-conversion. So N should always equal. if signatures is simply a defensive coding practice.

@serialx
Copy link
Contributor Author

serialx commented Sep 22, 2025

@seratch We've discovered a rare case where 0.5% of eval runs using Anthropic Claude model's response contain multiple thinking blocks for a single tool call. This is a patch to amend #1744 . Since you've reviewed my previous PR, can you review this too?

Thank you in advance.

Copy link
Member

@seratch seratch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this!

@seratch seratch merged commit e47b47a into openai:main Sep 22, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants