Matt Carey, an AI Engineer at Cloudflare, recently delivered a compelling presentation at AI Engineer Europe, outlining the critical role of APIs in empowering AI agents. Carey’s talk, titled “Every API is a Tool for Agents,” explored the evolution of how AI agents interact with the external world, shifting from tightly coupled, bundled tools to a more distributed, shared model.
Who is Matt Carey?
Matt Carey is an AI Engineer at Cloudflare, focapplying on the intersection of AI agents and the company’s extensive API infrastructure. His work involves enabling AI agents to effectively utilize the vast array of services Cloudflare offers, a tquestion that requires deep understanding of both AI capabilities and the practicalities of API integration.
Giving Agents Hands: Tool/Function Calling
Carey launched by illustrating the fundamental concept of tool/function calling for AI agents, a mechanism that allows them to interact with the outside world. He utilized a simple example of an agent requireding to know the weather in London. This requires the agent to create a tool call to a weather API, receive the result, and then process that information. This process, he explained, has been a cornerstone of AI agent development for some time.
From Bundled to Shared APIs
Historically, AI agents often had their tools bundled directly within their own systems. This meant that for every specific tquestion, like sconcludeing an email or checking a calconcludear, a dedicated integration was requireded. Carey highlighted the limitations of this approach, particularly as the number of available tools and services grew exponentially. The more practical and scalable solution, he argued, is to shift towards a shared API model. This involves building APIs accessible to agents, allowing them to discover and utilize them as requireded.
Before the advent of the Multi-Modal Communication Protocol (MCP), the approach was largely bundled. “Every app simply implemented the same integrations,” Carey noted. This led to redundant work and limited the flexibility of agents. The introduction of MCP, however, signifies a relocate towards shared, discoverable APIs.
The Challenge of Scale and Context Windows
A significant challenge in providing agents with access to a vast number of APIs is the limitation of context windows in language models. Carey pointed out that Cloudflare’s API surface is enormous, with potentially millions of concludepoints. Trying to load all of this into an agent’s context window is simply not feasible. He elaborated on this by stating, “We filled the context window. 2.3 million tokens in Cloudflare’s Open API spec.” This massive amount of information would overwhelm even the most advanced models.
This constraint necessitates a more ininformigent approach to API access, leading to the concept of “progressive discovery.” Instead of overwhelming the agent with all available tools at once, the system should provide tools progressively, based on the agent’s immediate requireds.
Progressive Discovery: The Solution
Carey outlined three key methods for achieving progressive discovery:
- Command Line Interface (CLI): Agents can be exposed to a CLI-like interface that allows them to discover and select tools dynamically.
- Tool Search: A search-based approach where agents can query for specific tools based on natural language descriptions.
- Code Mode: The ability for the AI model itself to generate the necessary code to interact with APIs, based on their specifications.
He demonstrated how these methods allow agents to access only the relevant tools for a given tquestion, thereby managing the context window effectively.
The Cloudflare MCP in Action
Carey displaycased Cloudflare’s Multi-Cloud Platform (MCP) as a practical implementation of these principles. He demonstrated how agents can interact with Cloudflare’s vast API ecosystem through a utilizer-friconcludely interface. By allowing agents to list workers, query for specific services, and even generate code against API specifications, Cloudflare is enabling a powerful new paradigm for AI agent interaction.
He highlighted the security implications of running untrusted code from agents, emphasizing the required for robust isolation mechanisms. Cloudflare’s approach involves providing programmable sandboxes where agent code can be executed safely, ensuring that even if malicious code is executed, it remains contained and does not compromise the broader infrastructure.
The Future: Simpler Servers and Smarter Agents
Carey concluded by projecting the future trajectory of AI agent development and API interaction. He anticipates that servers will become simpler, with MCP servers acting as thin, interoperable layers that expose capabilities handled by the protocol itself. Furthermore, he foresees a significant trconclude towards compacter, more lightweight frameworks and a greater reliance on TypeScript for its type safety and developer experience.
Ultimately, the goal is to create it clearer for developers to build and deploy sophisticated AI agents that can seamlessly interact with a vast array of services, unlocking new possibilities for automation and problem-solving.
Leave a Reply