Skip to content

fix(server-utils): Keep OpenAI streamed tool calls that have no index #25622

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
RulaKhaled merged 4 commits into from
Oct 28, 2026
Merged

fix(server-utils): Keep OpenAI streamed tool calls that have no index #25622

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f148c02
fix(server-utils): Keep OpenAI streamed tool calls that have no index
pcbeingused333 Oct 21, 2026
f0ac195
test(server-utils): cover an index-less OpenAI tool call followed by …
pcbeingused333 Oct 21, 2026
349e043
fix(server-utils): Handle null and reused indexes in OpenAI streamed …
pcbeingused333 Oct 26, 2026
0d1c6f7
fix(server-utils): Keep OpenAI streamed tool calls in the order they …
pcbeingused333 Oct 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions dev-packages/node-integration-tests/suites/tracing/openai/openai-tool-calls/scenario.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ class MockOpenAIToolCalls {

// If stream is requested, return an async generator
if (params.stream) {
if (params.model === 'gemini-compat') {
return this._createGeminiStyleToolCallsStream(params);
}
return this._createChatCompletionToolCallsStream(params);
}

Expand Down Expand Up @@ -139,6 +142,38 @@ class MockOpenAIToolCalls {
};
}

// Gemini's OpenAI-compatible endpoint streams whole, parallel tool calls without an `index`
async *_createGeminiStyleToolCallsStream(params) {

@RulaKhaled RulaKhaled Oct 28, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: thanks for adding it! ran it locally and the gemini case fails on develop, so it catches the bug 👍 we can keep it, but could you also add the same stream to scenario-chat.mjs? that one uses the real OpenAI client against an express server sending SSE, so the SDK actually parses the chunks

yield {
id: 'chatcmpl-stream-gemini-tools-123',
object: 'chat.completion.chunk',
created: 1677652310,
model: params.model,
choices: [
{
index: 0,
delta: {
role: 'assistant',
tool_calls: [
{
id: 'call_paris',
type: 'function',
function: { name: 'get_weather', arguments: '{"latitude":48.8566,"longitude":2.3522}' },
},
{
id: 'call_london',
type: 'function',
function: { name: 'get_weather', arguments: '{"latitude":51.5072,"longitude":-0.1276}' },
},
],
},
finish_reason: 'tool_calls',
},
],
usage: { prompt_tokens: 15, completion_tokens: 25, total_tokens: 40 },
};
}

// Create a mock streaming response for responses API with tool calls
async *_createResponsesApiToolCallsStream(params) {
const responseId = 'resp_stream_tools_789';
Expand Down Expand Up @@ -275,6 +310,15 @@ async function run() {
});
for await (const chunk of stream1) void chunk;

// Test 2b: Chat completion streaming whole parallel tool calls without an index (Gemini style)
const geminiStream = await client.chat.completions.create({
model: 'gemini-compat',
messages: [message],
tools: [weatherTool],
stream: true,
});
for await (const chunk of geminiStream) void chunk;

// Test 3: Responses API with tools (non-streaming)
await client.responses.create({
model: 'gpt-4',
Expand Down
38 changes: 36 additions & 2 deletions dev-packages/node-integration-tests/suites/tracing/openai/openai-tool-calls/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,25 @@ describe('OpenAI Tool Calls integration', () => {
},
]);

const CHAT_GEMINI_STREAM_TOOL_CALLS = JSON.stringify([
{
id: 'call_paris',
type: 'function',
function: {
name: 'get_weather',
arguments: '{"latitude":48.8566,"longitude":2.3522}',
},
},
{
id: 'call_london',
type: 'function',
function: {
name: 'get_weather',
arguments: '{"latitude":51.5072,"longitude":-0.1276}',
},
},
]);

const RESPONSES_TOOL_CALLS = JSON.stringify([
{
type: 'function_call',
Expand All @@ -84,7 +103,7 @@ describe('OpenAI Tool Calls integration', () => {
const segment = container.items.find(span => span.is_segment && span.name === 'main');
expect(segment).toBeDefined();
const spans = container.items.filter(span => span.attributes[SENTRY_ORIGIN]?.value === 'auto.ai.openai');
expect(spans).toHaveLength(4);
expect(spans).toHaveLength(5);
const chatToolsSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-tools-123',
);
Expand Down Expand Up @@ -193,6 +212,12 @@ describe('OpenAI Tool Calls integration', () => {
value: 40,
});

const geminiStreamSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-stream-gemini-tools-123',
);
expect(geminiStreamSpan).toBeDefined();
expect(geminiStreamSpan!.attributes[GEN_AI_RESPONSE_TOOL_CALLS]).toBeUndefined();

const responsesToolsSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'resp_tools_789',
);
Expand Down Expand Up @@ -318,7 +343,7 @@ describe('OpenAI Tool Calls integration', () => {
const segment = container.items.find(span => span.is_segment && span.name === 'main');
expect(segment).toBeDefined();
const spans = container.items.filter(span => span.attributes[SENTRY_ORIGIN]?.value === 'auto.ai.openai');
expect(spans).toHaveLength(4);
expect(spans).toHaveLength(5);
const chatToolsSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-tools-123',
);
Expand Down Expand Up @@ -452,6 +477,15 @@ describe('OpenAI Tool Calls integration', () => {
value: 40,
});

const geminiStreamSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-stream-gemini-tools-123',
);
expect(geminiStreamSpan).toBeDefined();
expect(geminiStreamSpan!.attributes[GEN_AI_RESPONSE_TOOL_CALLS]).toEqual({
type: 'string',
value: CHAT_GEMINI_STREAM_TOOL_CALLS,
});

const responsesToolsSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'resp_tools_789',
);
Expand Down
92 changes: 71 additions & 21 deletions dev-packages/node-integration-tests/suites/tracing/openai/scenario-chat.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,35 +22,74 @@ function startMockServer() {
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Connection', 'keep-alive');

const chunks = [
// Gemini's OpenAI-compatible endpoint streams whole, parallel tool calls without an `index`
const geminiToolCallChunks = [
{
id: 'chatcmpl-stream-123',
id: 'chatcmpl-stream-gemini-456',
object: 'chat.completion.chunk',
created: 1677652300,
created: 1677652305,
model: model,
choices: [{ delta: { role: 'assistant', content: '' }, index: 0 }],
},
{
id: 'chatcmpl-stream-123',
object: 'chat.completion.chunk',
created: 1677652300,
model: model,
choices: [{ delta: { content: 'Hello from OpenAI streaming!' }, index: 0 }],
},
{
id: 'chatcmpl-stream-123',
object: 'chat.completion.chunk',
created: 1677652300,
model: model,
choices: [{ delta: {}, index: 0, finish_reason: 'stop' }],
choices: [
{
index: 0,
delta: {
role: 'assistant',
tool_calls: [
{
id: 'call_paris',
type: 'function',
function: { name: 'get_weather', arguments: '{"city":"Paris"}' },
},
{
id: 'call_london',
type: 'function',
function: { name: 'get_weather', arguments: '{"city":"London"}' },
},
],
},
finish_reason: 'tool_calls',
},
],
usage: {
prompt_tokens: 12,
completion_tokens: 18,
total_tokens: 30,
prompt_tokens: 15,
completion_tokens: 25,
total_tokens: 40,
},
},
];

const chunks =
model === 'gemini-compat'
? geminiToolCallChunks
: [
{
id: 'chatcmpl-stream-123',
object: 'chat.completion.chunk',
created: 1677652300,
model: model,
choices: [{ delta: { role: 'assistant', content: '' }, index: 0 }],
},
{
id: 'chatcmpl-stream-123',
object: 'chat.completion.chunk',
created: 1677652300,
model: model,
choices: [{ delta: { content: 'Hello from OpenAI streaming!' }, index: 0 }],
},
{
id: 'chatcmpl-stream-123',
object: 'chat.completion.chunk',
created: 1677652300,
model: model,
choices: [{ delta: {}, index: 0, finish_reason: 'stop' }],
usage: {
prompt_tokens: 12,
completion_tokens: 18,
total_tokens: 30,
},
},
];

chunks.forEach((chunk, index) => {
setTimeout(() => {
res.write(`data: ${JSON.stringify(chunk)}\n\n`);
Expand Down Expand Up @@ -243,6 +282,17 @@ async function run() {
void chunk; // Prevent unused variable warning
}

// Streaming whole parallel tool calls without an index (Gemini style)
const geminiStream = await client.chat.completions.create({
model: 'gemini-compat',
messages: [{ role: 'user', content: 'Weather in Paris and London?' }],
stream: true,
});

for await (const chunk of geminiStream) {
void chunk;
}

// Fifth test: responses API streaming
const stream2 = await client.responses.create({
model: 'gpt-4',
Expand Down
37 changes: 34 additions & 3 deletions dev-packages/node-integration-tests/suites/tracing/openai/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import {
GEN_AI_RESPONSE_MODEL,
GEN_AI_RESPONSE_STREAMING,
GEN_AI_RESPONSE_TEXT,
GEN_AI_RESPONSE_TOOL_CALLS,
GEN_AI_SYSTEM_INSTRUCTIONS,
GEN_AI_USAGE_INPUT_TOKENS,
GEN_AI_USAGE_OUTPUT_TOKENS,
Expand Down Expand Up @@ -41,7 +42,7 @@ describe('OpenAI integration', () => {

const allSpans = await spansPromise;
const spans = allSpans.filter(span => span.attributes[SENTRY_ORIGIN]?.value === 'auto.ai.openai');
expect(spans).toHaveLength(8);
expect(spans).toHaveLength(9);
const chatCompletionSpan = spans.find(
span =>
span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-mock123' &&
Expand Down Expand Up @@ -173,6 +174,12 @@ describe('OpenAI integration', () => {
value: 'error-model',
});

const geminiStreamSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-stream-gemini-456',
);
expect(geminiStreamSpan).toBeDefined();
expect(geminiStreamSpan!.attributes[GEN_AI_RESPONSE_TOOL_CALLS]).toBeUndefined();

const streamingChatCompletionSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-stream-123',
);
Expand Down Expand Up @@ -444,7 +451,7 @@ describe('OpenAI integration', () => {

const allSpans = await spansPromise;
const spans = allSpans.filter(span => span.attributes[SENTRY_ORIGIN]?.value === 'auto.ai.openai');
expect(spans).toHaveLength(8);
expect(spans).toHaveLength(9);
const chatCompletionSpan = spans.find(
span =>
span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-mock123' &&
Expand Down Expand Up @@ -600,6 +607,18 @@ describe('OpenAI integration', () => {
value: '[{"role":"user","content":"This will fail"}]',
});

const geminiStreamSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-stream-gemini-456',
);
expect(geminiStreamSpan).toBeDefined();
expect(geminiStreamSpan!.attributes[GEN_AI_RESPONSE_TOOL_CALLS]).toEqual({
type: 'string',
value: JSON.stringify([
{ id: 'call_paris', type: 'function', function: { name: 'get_weather', arguments: '{"city":"Paris"}' } },
{ id: 'call_london', type: 'function', function: { name: 'get_weather', arguments: '{"city":"London"}' } },
]),
});

const streamingChatCompletionSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-stream-123',
);
Expand Down Expand Up @@ -902,7 +921,7 @@ describe('OpenAI integration', () => {

const allSpans = await spansPromise;
const spans = allSpans.filter(span => span.attributes[SENTRY_ORIGIN]?.value === 'auto.ai.openai');
expect(spans).toHaveLength(8);
expect(spans).toHaveLength(9);
const chatCompletionSpan = spans.find(
span =>
span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-mock123' &&
Expand All @@ -918,6 +937,18 @@ describe('OpenAI integration', () => {
value: expect.any(String),
});

const geminiStreamSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-stream-gemini-456',
);
expect(geminiStreamSpan).toBeDefined();
expect(geminiStreamSpan!.attributes[GEN_AI_RESPONSE_TOOL_CALLS]).toEqual({
type: 'string',
value: JSON.stringify([
{ id: 'call_paris', type: 'function', function: { name: 'get_weather', arguments: '{"city":"Paris"}' } },
{ id: 'call_london', type: 'function', function: { name: 'get_weather', arguments: '{"city":"London"}' } },
]),
});

const streamingChatCompletionSpan = spans.find(
span => span.attributes[GEN_AI_RESPONSE_ID]?.value === 'chatcmpl-stream-123',
);
Expand Down
Loading
Loading