Skip to content

fix(server): send Gemini thought signatures back with tool calls - #103

Open
Aj-Niplex wants to merge 3 commits into
CopilotKit:mainfrom
Aj-Niplex:fix/gemini-tool-call-thought-signatures
Open

Aj-Niplex wants to merge 3 commits into
CopilotKit:mainfrom
Aj-Niplex:fix/gemini-tool-call-thought-signatures

Conversation

@Aj-Niplex

Copy link
Copy Markdown

Gemini 3 through Google's OpenAI-compatible endpoint fails on every tool-calling turn. Google returns a thought signature on each tool call (tool_calls[].extra_content.google.thought_signature) and answers the next request with 400 Function call is missing a thought_signature unless it is sent back. The chat-completions adapter drops extra_content, so the chat ends with "The current turn returned no response".

Direct calls to the endpoint confirm it: replaying the tool call without the signature gives 400; with the returned signature or the documented placeholder skip_thought_signature_validator it gives 200.

Change

  • src/server/gemini-compat.ts: a fetch wrapper that remembers signatures from responses (by tool call id) and restores them on later requests, using the placeholder for tool calls it has not seen (e.g. history after a restart).
  • src/server/dot-agent.ts: uses it only when the base URL host is generativelanguage.googleapis.com; other providers are unchanged.
  • docs/SETUP.md: one paragraph.

Tests: wrapper unit tests, plus a full tool-call round trip through DotAgent against a Google base URL (and a check that other providers get no Gemini fields).

check-format, lint, typecheck, test (261) and build pass on Node 24. Reproduced on a build at c2569bb; the patch is based on main at 565bf78.

@jerelvelarde jerelvelarde left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This addresses a real Gemini tool-loop incompatibility and fits the existing provider adapter, but hold merge for two signature-preservation failures reproduced through the installed adapter and DotAgent. All 21 focused compatibility/agent/channel tests pass. Synthetic fragmented-stream and repeated-ID fixtures reproduce the findings; no live Google API calls were used. The official contract requires preserving received signatures: https://ai.google.dev/gemini-api/docs/generate-content/thought-signatures

Comment thread src/server/gemini-compat.ts Outdated
Comment thread src/server/gemini-compat.ts Outdated
for (const call of calls) {
const id = field(call, 'id');
const signature = signatureOf(call);
if (typeof id !== 'string' || !id || !signature) continue;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Assemble streamed tool-call deltas before associating the signature with an ID. This skips a signature when its delta omits the ID, even though an earlier delta for the same choice/tool-call index supplied it. An actual-adapter fixture emitting ID/name first and arguments/signature later sends the placeholder on continuation instead of the received signature. Track the deltas by choice and tool-call index and add a fragmented-SSE regression.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both points from the review are fixed in the latest push, with regression tests for each. I also squashed the history to two commits. Could you re-review, and approve the CI run when you get a chance? @jerelvelarde

Gemini 3 models reached through Google's OpenAI-compatible endpoint
return a thought signature on each tool call and reject the follow-up
request with HTTP 400 ("Function call is missing a thought_signature")
when it is not sent back. The chat-completions adapter drops
extra_content, so every turn that used a tool ended without a response.

Wrap fetch for Google endpoints so signatures seen in responses are
restored on the matching tool calls of later requests. Signatures are
cached per provider, model and thread, keyed by tool-call ID and matched
on arguments when an ID is reused. Streamed deltas are assembled per
choice and tool-call index before the signature is attached. Tool calls
that were never seen (for example history reloaded after a restart) get
Google's documented placeholder.
@Aj-Niplex
Aj-Niplex force-pushed the fix/gemini-tool-call-thought-signatures branch from 78d6e6c to 14cdf31 Compare October 8, 2026 01:17
@Aj-Niplex

Copy link
Copy Markdown
Author

Both points from the review are fixed in the latest push, with regression tests for each. I also squashed the history to two commits. Could you re-review, and approve the CI run when you get a chance? @jerelvelarde

@Aj-Niplex
Aj-Niplex requested a review from jerelvelarde October 8, 2026 11:26
Add tests for unindexed streamed tool calls, separate choices, malformed
stream events, failed responses, the per-ID and global cache limits,
tool calls without an ID, existing extra_content fields, and URL-object
requests.
@Aj-Niplex

Copy link
Copy Markdown
Author

Problem

Gemini 3 models, reached through Google's OpenAI-compatible endpoint, return a thought signature on every tool call (tool_calls[].extra_content.google.thought_signature). The next request must send it back. When it is missing, Google answers 400 Function call is missing a thought_signature.

The chat-completions adapter drops extra_content, so every turn that used a tool ends with "The current turn returned no response".

Direct calls to the endpoint confirm it: replaying the tool call without the signature gives 400. Replaying it with the returned signature, or with the documented placeholder skip_thought_signature_validator, gives 200.

How it works

sequenceDiagram
    participant D as DotAgent
    participant W as fetch wrapper
    participant G as Gemini endpoint
    D->>W: request 1 (user message)
    W->>G: forwarded unchanged
    G-->>W: tool call + thought_signature
    W-->>D: response (unchanged)
    Note over W: remember signature<br/>by thread, ID, arguments
    D->>W: request 2 (tool result, signature dropped by adapter)
    W->>G: same request + remembered signature
    G-->>W: 200 final answer
    W-->>D: response
Loading

Changes

File Change
src/server/gemini-compat.ts New fetch wrapper that remembers signatures from responses and restores them on later requests
src/server/dot-agent.ts Uses the wrapper only when the base URL host is generativelanguage.googleapis.com; other providers are unchanged
docs/SETUP.md Short note on Gemini 3
tests/gemini-compat.test.ts, tests/gemini-agent.test.ts Unit tests for the wrapper and DotAgent-level round trips

Behavior

Situation Result
Signature seen in an earlier response Sent back unchanged on the matching tool call
Tool call never seen (for example history reloaded after a restart) Documented placeholder is sent
Same tool-call ID in two conversations Signatures are never shared (cache is scoped to base URL, model and thread)
Same ID reused within a thread Matched on normalized arguments; if ambiguous, the placeholder is used instead of guessing
Streamed deltas split across chunks Assembled per choice and tool-call index before the signature is attached
Failed (non-2xx) responses Nothing is remembered
Memory At most 4 entries per ID and 5000 IDs; oldest are dropped first

Tests

  • 23 focused tests cover the wrapper and the DotAgent round trip, including fragmented SSE, parallel tool calls, multiple choices, ID reuse, scope isolation, malformed events, failed responses and the cache limits.
  • Each regression test was checked by breaking the code it covers and confirming the test fails.
  • check-format, lint, typecheck, test (277) and build pass on Node 24.
  • Reproduced on a build at c2569bb; the branch is based on main at 565bf78 and merges cleanly into the current main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants