Gemini Function Calling Token Usage Experiment

This repository contains a minimal script that helps determine whether a function response (tool output) in Gemini Function Calling is counted as input or output tokens. The default model is gemini-2.5-flash, which you can override via the MODEL environment variable.

How It Works

• Declare getTeamMembers(teamId) as a tool.
  • The function returns TEAM_MEMBER_COUNT members (defaults to 10). Set the environment variable to 1000 to simulate the “large payload” scenario. Names are strings generated on the fly.
• Ask the model, “How many members belong to team X?”
• The model proposes a function call → the host executes the function → the host sends the (possibly huge) member list back to the model and asks for the final answer.
• Log usageMetadata for both requests and compare token deltas.

Prerequisites

• Node.js 18+
• Set your API key in GOOGLE_API_KEY
• Optionally set MODEL to use a different Gemini model (defaults to gemini-2.5-flash)

Setup

npm install

How to Run

• Small payload (10 members):

GOOGLE_API_KEY=... TEAM_ID=abcd TEAM_MEMBER_COUNT=10 npm start

• Large payload (1000 members):

GOOGLE_API_KEY=... TEAM_ID=abc TEAM_MEMBER_COUNT=1000 npm start

Explicitly choosing a model (optional):

GOOGLE_API_KEY=... MODEL=gemini-2.5-flash TEAM_ID=abcd TEAM_MEMBER_COUNT=10 npm start

Execution prints the following:

• first: usageMetadata for the initial request (user question → model suggests a function call)
• second: usageMetadata for the second request where the function response is sent back
• answer: the model’s final textual answer

What to Observe

• When TEAM_MEMBER_COUNT=1000, the second request’s input-side tokens (e.g., promptTokenCount or inputTokenCount) should spike because the function response is part of that request’s input.
• The first request usually has fewer output tokens because the model only needs to emit a function call.

Field names in usageMetadata can differ by SDK/model version:

• promptTokenCount, candidatesTokenCount, totalTokenCount
• or inputTokenCount, outputTokenCount This sample logs multiple common fields for convenience.

Files

• src/experiment.js: main experiment script
• package.json: dependencies and npm scripts

Notes

• Network access is required; CI or sandbox environments may block outbound calls.
• Measured numbers vary with the model version and time of execution.

Result

• 10 team mebers

$ GOOGLE_API_KEY=... TEAM_ID=abcd TEAM_MEMBER_COUNT=10 npm start

> token-counter@0.1.0 start
> node src/experiment.js

==== Gemini Function Calling Token Usage Experiment ====
MODEL=gemini-2.5-flash
TEAM_ID=abcd (length=4)
TEAM_MEMBER_COUNT=10
[first] usage: {
  promptTokenCount: 87,
  candidatesTokenCount: 15,
  totalTokenCount: 179,
  inputTextTokenCount: undefined,
  inputTokenCount: undefined,
  outputTokenCount: undefined
}
[WARN] Function call missing id; responding without id may fail.
[second] usage: {
  promptTokenCount: 211,
  candidatesTokenCount: 2,
  totalTokenCount: 213,
  inputTextTokenCount: undefined,
  inputTokenCount: undefined,
  outputTokenCount: undefined
}

--- Answer ---
10

--- Summary ---
{
  teamId: 'abcd',
  first: {
    promptTokenCount: 87,
    candidatesTokenCount: 15,
    totalTokenCount: 179,
    promptTokensDetails: [ [Object] ],
    thoughtsTokenCount: 77
  },
  second: {
    promptTokenCount: 211,
    candidatesTokenCount: 2,
    totalTokenCount: 213,
    promptTokensDetails: [ [Object] ]
  }
}

• 1000 team members

$ GOOGLE_API_KEY=... TEAM_ID=abcd TEAM_MEMBER_COUNT=1000 npm start

> token-counter@0.1.0 start
> node src/experiment.js

==== Gemini Function Calling Token Usage Experiment ====
MODEL=gemini-2.5-flash
TEAM_ID=abcd (length=4)
TEAM_MEMBER_COUNT=1000
[first] usage: {
  promptTokenCount: 87,
  candidatesTokenCount: 15,
  totalTokenCount: 165,
  inputTextTokenCount: undefined,
  inputTokenCount: undefined,
  outputTokenCount: undefined
}
[WARN] Function call missing id; responding without id may fail.
[second] usage: {
  promptTokenCount: 11148,
  candidatesTokenCount: 4,
  totalTokenCount: 11262,
  inputTextTokenCount: undefined,
  inputTokenCount: undefined,
  outputTokenCount: undefined
}

--- Answer ---
1000

--- Summary ---
{
  teamId: 'abcd',
  first: {
    promptTokenCount: 87,
    candidatesTokenCount: 15,
    totalTokenCount: 165,
    promptTokensDetails: [ [Object] ],
    thoughtsTokenCount: 63
  },
  second: {
    promptTokenCount: 11148,
    candidatesTokenCount: 4,
    totalTokenCount: 11262,
    promptTokensDetails: [ [Object] ],
    thoughtsTokenCount: 110
  }
}

Hint: Run twice—once with TEAM_MEMBER_COUNT=10 and once with TEAM_MEMBER_COUNT=1000—for comparison.