A search-then-call gateway for MCP: let agents use hundreds of tools without loading them all into context.
Status: proposal. This repository currently contains design docs and plans only. There is no code yet. See the Roadmap for what gets built and when.
Every MCP server an agent connects to adds its tool definitions to the model's context on every request. That's fine for 10 tools. It breaks down at hundreds:
- Token cost. Loading 2,792 tool definitions would take about 206K tokens, which exceeds many models' context windows (Stacklok).
- Wrong tool choices. With 70 servers and 527 tools, retrieval errors account for nearly half of agent failures (LiveMCPBench).
- Look-alike tools. Several servers often expose tools with nearly the same name and purpose (
search,create_issue,read_file), and the model has to tell them apart.
ToolScout sits between an agent and many MCP servers. It's built in Python on LangChain and LangGraph, and it can be used in two ways:
- As an MCP server for any MCP client (Claude Code, Claude Desktop, …). To the client it looks like one server with two tools.
- As LangChain middleware for agents built with
create_agent. The agent sees only the tools that matter for the current step.
| Tool | What it does |
|---|---|
search_tools(query, k) |
Finds the k most relevant tools across all connected servers and returns their names, descriptions and input schemas |
call_tool(name, arguments) |
Checks the arguments against the tool's schema, forwards the call to the right server and returns the result |
flowchart LR
A["MCP client<br/>(Claude Code, Claude Desktop, …)"] -- "search_tools<br/>call_tool" --> F["FastMCP server"]
L["LangChain / LangGraph agent<br/>(create_agent)"] -- "ToolScoutMiddleware" --> C
F --> C["ToolScout core<br/>(Python)"]
C <--> I[("Tool index<br/>BM25F · embeddings")]
C -- "langchain.mcp<br/>MCPAdapter" --> S1["Server A"]
C --> S2["Server B"]
C --> S3["… Server N"]
A typical exchange over MCP:
Agent → search_tools({"query": "weather forecast for a city", "k": 3})
← [ wanderly_get_weather, … ] (names, descriptions, input schemas)
Agent → call_tool({"name": "wanderly_get_weather",
"arguments": {"latitude": 35.01, "longitude": 135.77}})
← (the upstream server's result, passed through unchanged)
Similar ideas already exist: Anthropic's tool search tool, Stacklok's MCP Optimizer, MCPProxy, and, inside the LangChain ecosystem, langgraph-bigtool, LLMToolSelectorMiddleware and ProviderToolSearchMiddleware. ToolScout's focus:
- Works with any client and any model. It's a plain MCP server first. The LangChain middleware is a second surface over the same core, not a replacement.
- Every design choice is measured. A retrieval benchmark ships in the repo and gates every change in CI. The end-to-end results report cost per successful task, not just retrieval accuracy, and compare against the LangChain-native options above.
- A strong lexical baseline first. Field-weighted BM25 (BM25F) over tool names, descriptions and parameters needs no API keys and no embedding service. Hybrid search (BM25F + embeddings, fused with LangChain's
EnsembleRetriever) is opt-in and has to earn its place in the benchmark. - Built on standard parts. Upstream connections use
langchain.mcp, retrievers implement LangChain'sBaseRetriever, the server surface uses FastMCP, and the evaluation agent runs on LangGraph.
| Layer | Question | Main metrics |
|---|---|---|
| Retrieval benchmark (runs in CI) | Does search put the right tools in the top k? | Recall@k, complete-recall@k, MRR, p50/p95 latency |
| End-to-end agent runs (LangGraph) | Do agents finish more tasks, more cheaply? | Task success, tokens per task, cost per successful task |
| Failure attribution | Why did a failed run fail? | Share of failures by cause (never retrieved, look-alike chosen, bad arguments, …) |
Details: docs/EVALUATION.md.
The interface below is a plan and will change as the build progresses.
# toolscout.yaml
servers:
- name: wanderly
command: ["./wanderly-mcp"]
- name: files
command: ["npx", "-y", "@modelcontextprotocol/server-filesystem", "~/notes"]
- name: remote-example
url: https://example.com/mcp
headers:
Authorization: "Bearer ${EXAMPLE_TOKEN}"
retrieval:
mode: lexical # lexical | hybrid
default_k: 5
embeddings: null # e.g. "ollama:nomic-embed-text" or "openai:text-embedding-3-small" for hybrid
pinned: [] # tools that are always visible to the agentAs an MCP server:
uvx toolscout serve --config toolscout.yaml # run as an MCP server over stdio
uvx toolscout search "create a calendar event" # debug: show ranked tools
uvx toolscout eval --suite retrieval # run the retrieval benchmarkAs LangChain middleware:
from langchain.agents import create_agent
from toolscout import ToolScout
from toolscout.langchain import ToolScoutMiddleware
async with ToolScout.from_config("toolscout.yaml") as scout:
agent = create_agent(
"claude-haiku-5-5",
tools=scout.tools, # every upstream tool, as LangChain tools
middleware=[ToolScoutMiddleware(scout, k=5)], # only the top k reach the model each step
)
await agent.ainvoke({"messages": [{"role": "user", "content": "What's the weather in Kyoto?"}]})| Milestone | Target | Outcome |
|---|---|---|
| M0 Proposal | Sep 28, 2026 ✅ | First docs (Go design) |
| M0.5 Re-plan | Oct 25, 2026 ✅ | These docs, moved to Python, LangChain and LangGraph |
| M1 Gateway skeleton | Nov 1 | Connect to servers via langchain.mcp, build the catalog, serve search_tools / call_tool with FastMCP |
| M2 Lexical search | Nov 8 | BM25F retriever, debug CLI, first benchmark numbers |
| M3 Hybrid, middleware, CI gate | Nov 15 | Embeddings, EnsembleRetriever, ToolScoutMiddleware, labeled benchmark, GitHub Actions gate |
| M4 End-to-end | Nov 22 | LangGraph agent runs on LiveMCPBench, cost and failure analysis |
| M5 v0.1 release | Nov 29 | Results in README, demo, setup guides, PyPI release |
| M6 Trained judge (optional) | After v0.1 | A small judge trained with SSL, SFT and RL, tested against hand labels |
| Phase 2 (optional) | Later | Research questions: who writes the query, doc2query, look-alike tools |
Full plan with checklists: docs/ROADMAP.md.
- Proposal: motivation, goals, prior work, success criteria, risks
- Design: architecture, components, search scoring, safety
- Evaluation: benchmark, end-to-end experiments, statistics
- Roadmap: milestones and checklists
- Decisions: decisions made so far and open questions
- Trained judge: optional track on training a small LLM judge with SSL, SFT and RL
- Anthropic, Introducing advanced tool use and the tool search tool
- LangChain,
langgraph-bigtooland the v1 agent middleware (LLMToolSelectorMiddleware,ProviderToolSearchMiddleware) - LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?
- RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation
- MCP-Zero: Active Tool Discovery for Autonomous LLM Agents
- Retrieval Models Aren't Tool-Savvy (ToolRet)
- Stacklok, MCP Optimizer vs. Anthropic's Tool Search Tool
- StackOne, Comparing BM25, TF-IDF, and Hybrid Search for MCP Tool Discovery