ToFu: Exploiting and Fuzzing the LLM-Orchestrator-Tool Trust Boundary

AI security research · arXiv cs.CR (AI security)

ToFu: Exploiting and Fuzzing the LLM-Orchestrator-Tool Trust Boundary

REDCORE assessment

The ToFu framework demonstrates that the trust boundary between LLM orchestrators and external tools is frequently misconfigured, leading to CWE-class vulnerabilities. While the research proves these systemic failures in existing tool servers, defenders should conclude that the LLM's output cannot serve as an implicit validation layer for external systems. Teams must assume that any LLM-driven process is susceptible to input-based abuse and enforce defense-in-depth at the API level.

Impact for lean teams

Small teams integrating AI agents must treat LLM-generated tool inputs as untrusted user input. You should audit any tool-calling logic to ensure strict input validation is enforced before execution.

Source summary

Large language models (LLMs) are increasingly deployed as agents that interact with external software through tools, with an orchestration layer translating model outputs into structured tool invocations. This architecture creates an LLM-orchestrator-tool trust boundary where tool semantics may go unenforced and LLM-generated arguments may be incorrectly treated as trusted. We present ToFu, a tool-aware framework that automatically derives semantic security checks for LLM tools from their existing metadata and a CWE-guided sanitizer corpus. ToFu specializes these checks to individual tools and automatically generates fuzzing harnesses and semantic oracles using runtime introspection and outcome sanitization. We evaluated ToFu across 273 MCP servers containing 3405 tools and identified verified bugs corresponding to CWE weakness classes in 343 tools across 118 of the servers. Our findings

Recommended actions

• Implement strict input schema validation for all tools called by LLM agents.
• Apply the principle of least privilege to the credentials used by AI agents when executing external tools.
• Use sandboxed execution environments for tool invocation to minimize the impact of successful exploitation.
• Establish automated security checks that verify tool input compliance against documented metadata.

This assessment was generated by REDCORE AI from the attributed source above and is provided for situational awareness. Validate against the original research and your own environment before acting.