up to $20

Blog

Will Claude Code use an MCP server the user did not name? We measured it

· by

TL;DR. At session start, Claude Code shows the model only MCP tool names and server instructions, so those are the only text the model has when it decides whether to search for a tool. In 240 runs of prompts that fit our server but did not name it, Claude Code handed 0 to the server with our old text and 21 after we rewrote the instructions and tool descriptions. Prompts it should keep were never handed over or offered. Research, file processing, browser and media prompts stayed at 0. These numbers come from one model (Claude Haiku 5.5), stub servers and a 10-turn cap.

Plori runs AI agents in cloud Workspaces, each with a shell, network access and files that persist, and offers them to clients through an MCP server. We wanted to know what Claude Code does when a user describes a task that fits but does not name the server.

How does Claude Code show MCP tools to the model?

Tool search is on by default, so Claude Code defers MCP tools. The Claude Code MCP docs say "Only tool names and server instructions load at session start". The model reads a description only after a ToolSearch call matches that tool. In every run of our eval, the first model request had no MCP tool definitions, contained our instructions, and offered ToolSearch.

The CHANGELOG gives the length limits. From 2.1.84, instructions and tool descriptions shown at session start were cut at 2,048 characters. Version 2.1.296 raised that to 4,096. From 2.1.295, descriptions loaded through tool search are cut at 16,384 characters.

How did we test it?

On October 13, 2026, Claude Code 2.1.296 ran Claude Haiku 5.5 through OpenRouter in a new Docker container per run, with all built-in tools allowed and a 10-turn limit. Stub MCP servers served Plori's 23 tools and text plus three decoy servers. Each of 90 prompts ran 6 times per text version: 10 direct prompts that name Plori, 40 indirect prompts that fit Plori but do not name it, and 40 negative prompts that should stay with Claude Code, including near misses such as "Write me a cron expression that runs every Monday at 9am."

A run delegated if it called one of the three tools that start a Plori run, checked in code. It offered if it started nothing but TypeSafe's Jev judge, a yes/no judgment with a probability, put the chance that its final reply offered to run the task on Plori above 0.5.

What did we change?

The old instructions described what a Workspace holds and how runs and approvals work, not when to use Plori. The new ones start with task classes: "Hand a task to Plori when it should run in the background, in parallel or after the chat ends". Because runs spend the user's prepaid balance, they also name what the client should keep: "Do it yourself for questions you can answer, short lookups, a few searches, edits to open files and anything you can finish in a few tool calls".

The three run-starting tools now open with a "Use this when" and a "Do not use for" sentence, as OpenAI's Apps SDK guidance recommends. We did not write "use this instead of WebSearch": Anthropic's connector review criteria reject descriptions that interfere with other tools.

What changed in the results?

Prompt set Old text New text
Direct, delegated 60 / 60 60 / 60
Indirect, delegated 0 / 240 21 / 240
Indirect, delegated or offered 18 / 240 45 / 240
Negative, delegated or offered 0 / 240 0 / 240

Background code prompts went from 0 to 17 delegations of 60 runs. Recurring prompts went from 0 to 4 delegations of 42, and from 14 to 25 delegations or offers. The instructions and the tool descriptions changed together, so the eval cannot split this gain between them.

Did the model search for tools at all?

With the new text, 87 of 240 indirect runs called ToolSearch, 37 loaded a Plori tool, and 21 started a run. The other 153 never searched. With the old text: 69, 17 and 0. By prompt type:

  • Background code: searches went from 0 of 60 runs to 20, and all 20 loaded a Plori tool. Tool names did not change, and descriptions are not visible before a search, so this increase must come from the instructions.
  • Recurring: 36 of 42 runs searched with the old text and 34 with the new. Plori loads stayed at 17. Here the new text changed what the model did after loading a tool.
  • Research saved as files: with the new text, 30 of 36 runs called ToolSearch, but every query selected Claude Code's own web tools by name (select:WebSearch,WebFetch or select:WebFetch). None used keywords that could match a Plori tool.
  • File processing, headless browser and media: no searches with the new text.

So a ToolSearch call does not mean the model looked for a new capability.

Where did the text not help?

Research, file processing, headless browser and media prompts got 0 delegations and 0 offers in every version, and long hand-off prompts at most 1 offer of 36. Part of this is a measurement limit: with the new text, all 12 media runs and 20 of 30 file-processing runs hit the 10-turn cap while still working locally.

For recurring prompts, Claude Code has its own scheduling tools. The model used its Cron tools or ScheduleWakeup in 18 of 240 indirect runs with the new text and 19 with the old, for example a cron job for a Friday report.

Did adding a skill help?

No. The settings schema in the Claude Code 2.1.296 binary describes skillListingBudgetFraction as the "Fraction of the context window (in characters) reserved for the skill listing sent to Claude (default: 0.01 = 1%)". Our container's 13 built-in skills already filled it with 6,051 characters. A 694-character Plori skill description was listed as the name only (- plori) in 540 of 540 runs. Claude Code listed a 189-character one in full. Delegation was 21 of 240 without a skill, 21 and 19 with one.

How can you measure and apply this?

  • Behind a proxy, set ENABLE_TOOL_SEARCH. When ANTHROPIC_BASE_URL is not a first-party host, Claude Code otherwise loads all tools at the start, which is not the deferred default.
  • Use direct, indirect and negative prompts, and report negatives next to the gain.
  • Record what each search asked for, and turn-cap cutoffs per prompt type.

What we did, which changed results for code and recurring prompts only:

  1. Started the instructions with the task classes the server is for.
  2. Named the tasks the client should keep.
  3. Opened run-starting tool descriptions with "Use this when" and "Do not use for".
  4. Shortened the skill description to 189 characters so it fits the listing budget. Claude Code shows a longer one as the name only. This did not change delegation.

What are the limits of this measurement?

  • One model, Claude Haiku 5.5 through OpenRouter. Other models, Claude Code with the first-party API, and real users were not measured.
  • Stub servers with Plori's text, not the production server.
  • A bare container. A real laptop has more tools and skills.
  • A 10-turn cap cut off 93 of 240 indirect runs with the new text and 95 with the old, about 40%. We cannot say what they would have done with more turns.
  • 6 runs per prompt. A difference of one or two runs in a single prompt can be noise: in our behaviour eval, two flagged differences disappeared on 10- and 20-run reruns.
  • The offer verdict. We agreed with the judge on 14 of 14 sampled replies. Replies near the 0.5 threshold could go either way.