Tool Calling

1. Function Calling#

Function calling is the foundational mechanism that makes tool use possible in the first place, so it’s worth understanding clearly before anything else in this section. At its heart, the idea is this: instead of a language model only being able to respond with plain, free-form text, it can also be given the ability to respond by saying, in a very specific, structured way, “I want to call this particular function, with these particular input values.”

Here’s how this actually works in practice. When you set up an agent, you describe to the language model, in advance, exactly what functions (also often called “tools”) it’s allowed to use — giving each one a clear name, a description of what it does, and a precise specification of what inputs it needs (for example, a “get_weather” function might need a “city name” as its input). Then, during the conversation, if the model determines that using one of these functions would actually help answer the current question, instead of just generating a plain text reply, it generates a specially structured response indicating exactly which function it wants to call and with what specific input values. The surrounding system (not the model itself) then actually executes that function in the real world, takes whatever result comes back, and feeds that result back to the model, which can then use it to continue forming its final answer.

It’s worth being really clear about an important detail here, because it trips up a lot of beginners: the language model itself never directly runs any code or reaches out onto the internet on its own. It only ever generates text — but a very specific, structured kind of text that clearly says “please call this function with these inputs.” It’s entirely up to the surrounding application code to actually notice that structured request, genuinely go carry out the real action, and then report the result back. This separation matters a lot for safety and control — it means a developer always has a deliberate checkpoint where they can inspect, validate, or even block a requested function call before anything real actually happens, rather than the model having some kind of direct, unsupervised, unchecked ability to act in the world entirely on its own.

2. Tool Selection#

Tool selection refers to the process — and the real underlying challenge — of an agent correctly figuring out which specific tool, out of possibly many available ones, is actually the right one to use for a given situation, and this becomes a genuinely harder problem than it might sound like once an agent has more than just a couple of tools available to it.

Think about it from the model’s perspective: if an agent has been given access to twenty different tools (a web search tool, a calculator, a database lookup tool, a weather checker, a calendar tool, and so on), and a user asks a somewhat ambiguous question, the model needs to correctly reason about which of those twenty tools, if any, is actually the right fit for addressing that particular request — and it needs to get this right fairly reliably, because picking the wrong tool, or failing to recognize that a tool is needed at all, directly leads to a poor or completely wrong final answer.

A few practical things genuinely help make tool selection work more reliably in real systems. Clear, well-written tool descriptions matter enormously — a tool that’s vaguely or ambiguously described (like a function simply named “lookup” with a one-word description) gives the model far less to work with when deciding whether it’s actually the right fit, compared to a tool with a clear, specific name and a precise, well-written explanation of exactly what it does and doesn’t do. Not overwhelming the model with too many tools at once also matters — much like a human faced with an overwhelming wall of options, giving a model access to an excessive number of tools simultaneously, especially ones with overlapping or unclear purposes, tends to genuinely hurt its ability to reliably pick the correct one, which connects directly back to the Router pattern we discussed in the earlier Agent Patterns explanation, where narrowing down the relevant set of available tools or paths ahead of time can meaningfully improve overall reliability. And good examples and clear instructions in the surrounding prompt can further help guide a model toward correctly recognizing which situations genuinely call for which specific tool.

3. Structured Outputs#

Structured outputs refers to getting a language model to produce its response in a precise, predictable, well-defined format — like a specific structure with clearly named fields and specific data types — rather than in loose, free-flowing natural language text that a program would then have to try to interpret or parse afterward.

Why does this matter so much for tool calling specifically? Because for a tool call to actually work correctly, the surrounding application code needs to reliably know exactly what input values the model is providing, in a format the code can directly and confidently use — it can’t be left guessing or trying to loosely interpret something like “the model seems to want to search for something related to Paris weather, I think.” Structured outputs solve this by having the model’s response conform to a specific, predefined structure (very often something like a JSON format, which is a very common and simple way of representing structured data), so the exact input the model wants to send to a function is clearly, precisely, and unambiguously specified — for example, unambiguously specifying {"city": "Paris", "units": "celsius"} rather than a vague, informally-worded sentence that a program would then have to try to interpret.

This capability isn’t just useful for tool calling — it’s also broadly valuable any time you want a model’s output to reliably plug directly into some other piece of software without a human having to manually read and interpret free-form text first. Many modern language model APIs offer specific, dedicated features to help enforce this reliably, letting a developer specify the exact expected structure in advance and getting a strong, often technically enforced guarantee that the model’s response will actually conform to that structure, rather than just hoping the model happens to format its text response correctly through good instructions alone.

4. API Integration#

API integration refers to the general practice of connecting an agent to external APIs — the standard interfaces that different software services and companies expose, allowing separate pieces of software to communicate with each other and exchange data or trigger actions.

You’ve actually already learned a lot of the underlying concept here from the very first explanation in this whole series, on LLM APIs — an API, in general, is simply a defined, standardized way for one piece of software to request something from another piece of software over the internet, and get a structured response back. Tool calling, in a very real sense, is largely about giving an agent the ability to make exactly these kinds of API calls itself (through the function-calling mechanism we described above), rather than only a human developer manually writing code that calls a fixed, predetermined API at a fixed, predetermined moment.

In practice, integrating an agent with external APIs generally involves a few concrete pieces of work: defining a clear tool description that accurately explains what a particular API actually does and what inputs it specifically needs, handling authentication properly (many APIs require some kind of secret key or token to prove you’re an authorized, legitimate user, and this needs to be securely managed and never exposed improperly), and handling the response that comes back in a way the agent can actually make good use of (which sometimes means the raw response needs some reformatting or simplification before being handed back to the model, since raw API responses can occasionally be quite verbose, complex, or contain a lot of extra technical detail the model doesn’t actually need). Good API integration is really about creating a clean, reliable, well-documented bridge between the agent’s more abstract, flexible reasoning capability and the very concrete, specific, structured way that real external services and systems actually expect to be interacted with.

5. REST Tools#

REST is one of the most common and widely used standard styles for designing APIs across the software industry, and understanding it a little will help make sense of a huge number of the specific tools an agent might actually be given access to in practice.

Without going too deep into the technical weeds, REST is built around a few simple, consistent underlying ideas. It organizes everything around “resources” — specific pieces of data or things you can act upon, like “a customer,” “an order,” or “a product,” each typically identified by its own specific web address. And it uses a small, standard, consistent set of actions to interact with those resources — most commonly: fetching or reading a resource (called “GET”), creating a brand new resource (called “POST”), updating an existing resource (called “PUT” or “PATCH”), and removing a resource entirely (called “DELETE”). Because so many different companies and services have all generally settled on and adopted this same consistent underlying style, once you understand this basic pattern, a huge number of different real-world APIs across a huge range of different services all start to feel genuinely familiar and follow broadly the same general shape and logic.

When we talk about “REST tools” specifically in an agent context, we simply mean tools that are built to call these kinds of standard REST-style APIs on the agent’s behalf — for example, a tool that lets an agent fetch a customer’s order history by calling a company’s REST API, or a tool that lets an agent create a new support ticket by sending a properly formatted request to a REST endpoint designed specifically for that purpose. Because REST is such a widely adopted, well-understood, and consistent standard across the software industry, building tools that wrap and expose REST APIs to an agent is one of the single most common and foundational ways of actually connecting an agent to real, useful, external capabilities in the world.

6. Database Tools#

Database tools give an agent the ability to directly read from, and sometimes write to, a structured database — letting it look up specific stored records, run queries to find and retrieve particular pieces of information, or, in some cases, actually create, update, or delete data.

This is an especially powerful and valuable category of tool, because a huge amount of the world’s genuinely useful, day-to-day business information lives in structured databases rather than in loose documents or general web pages — think of a company’s customer records, their product inventory, their order history, or their internal analytics data. Giving an agent the ability to directly query this kind of information means it can answer very specific, precise, factual questions (“how many units of this specific product do we currently have in stock?”) that a general knowledge model, or even a document-based RAG system, would have absolutely no reliable way of answering on its own.

Database tools do come with some genuinely important design considerations worth knowing about, precisely because they can be significantly more powerful, and correspondingly significantly more risky, than a simple, harmless read-only search tool. Read versus write access matters enormously — a tool that only lets an agent look up and read existing information is generally much lower-risk than one that lets it actually modify or delete real, live data, and many well-designed production systems deliberately restrict agents to read-only database access specifically to avoid the very real risk of an agent, whether through a genuine reasoning mistake or an unexpected edge case, accidentally corrupting or deleting important, valuable data. Query safety also matters a lot — since agents ultimately construct their own database queries based on their own reasoning, well-designed systems need real, deliberate safeguards to prevent an agent (or a malicious actor deliberately trying to manipulate it through crafted, adversarial user input) from being able to construct a harmful, overly broad, or dangerously unrestricted query that exposes or damages far more data than was ever actually intended.

7. Search Tools#

Search tools give an agent the ability to look up current, external information — most commonly by actually searching the web, though this general category can also include searching within a specific, more narrowly defined document collection or knowledge base, connecting directly back to the semantic search and RAG concepts we covered in detail earlier in this whole series.

This is genuinely one of the single most common and valuable tools given to agents, and it directly addresses one of the core motivating problems behind RAG that we discussed way back at the start of this whole series: a language model’s own built-in training data has a fixed cutoff date, and it simply has no way of knowing about anything that’s happened more recently, or about anything genuinely current, like today’s news, current prices, or the current status of something happening right now. A search tool lets the agent reach outside its own frozen training knowledge and pull in fresh, current, real information at the actual moment it’s needed, rather than being limited entirely to whatever it happened to learn during its original training process.

In practice, a well-designed search tool typically takes a text query as its input (which the agent itself formulates, based on what it’s actually trying to find out), sends that query off to an actual underlying search engine or a specific, defined knowledge source, and returns a set of relevant results back to the agent — which the agent then needs to actually read through, interpret, and meaningfully incorporate into its final response. This connects very directly back to a number of the advanced RAG techniques we covered earlier, like query transformation (rewriting a vague or poorly-worded question into a more effective search query before actually running the search) and re-ranking (further refining and improving a batch of initial search results before actually using them) — these same underlying techniques apply just as well and just as usefully here, within the specific context of an agent actively deciding, on its own, exactly when and how to search for information as part of accomplishing some larger, broader task.

8. Browser Tools#

Browser tools give an agent the ability to actually control and interact with a real web browser — not just fetching and passively reading the raw content of a single webpage (which a simpler search or page-fetching tool can already do), but genuinely being able to click on links and buttons, fill out and submit forms, scroll down a page to reveal more content, and navigate from page to page across a website, much like a human actually would while manually browsing the web themselves.

This meaningfully expands what an agent can actually accomplish out in the real world, well beyond what a simple search tool alone would allow. A search tool can help an agent find and read relevant information — but a browser tool lets an agent actually take real action on a live, interactive website: filling in and submitting a genuine contact form, actually clicking “add to cart” on a specific product, logging into an actual account, or working through a live, multi-step process on a website that genuinely requires several sequential interactions to complete, rather than the agent being limited only to passively reading whatever static content happens to already be sitting there on a page.

Browser tools tend to come with some genuinely real added complexity, though, that’s worth understanding clearly. Modern websites are often quite complex and highly dynamic, meaning an agent controlling a browser needs a genuinely reliable way to correctly identify and interact with the right specific elements on a given page (the right specific button, the right specific input field), even as a page’s underlying layout and structure might change or update over time. There’s also a real, meaningfully elevated safety consideration here compared to a simple, purely read-only search tool — since a browser tool can potentially take genuinely consequential, real-world actions (submitting a real, live purchase, changing real account settings, and so on), the kind of human-in-the-loop safeguards we discussed earlier in the Agent Patterns explanation become especially important and especially relevant specifically when an agent has this level of real, powerful, interactive browser capability available to it.

9. File System Tools#

File system tools give an agent the ability to interact directly with files — reading the actual contents of an existing file, creating a brand new file, modifying or editing an existing file’s contents, or organizing and managing files and folders more generally.

This category of tool becomes especially important and especially relevant for agents that work with actual documents or code as a core, central part of their job — for example, a coding agent that genuinely needs to read through existing source code files, actually make specific edits directly to them, and then create new files as part of implementing some requested feature or fix, or a document-processing agent that needs to actually read through, meaningfully summarize, and organize a large batch of files into some more useful and manageable structure.

As with database tools, the specific scope of exactly what file system access an agent is actually granted matters a great deal in practice. A well-designed production system generally deliberately limits and clearly scopes exactly which specific files and folders an agent can actually access and modify (for example, restricting it to only a specific, designated project folder, rather than granting it broad, unrestricted access to an entire computer’s complete file system), specifically to prevent an agent from accidentally reading or modifying something sensitive, important, or entirely unrelated to its actual, intended task, whether that mistake arises from a genuine reasoning error on the agent’s part, or from a deliberately crafted, adversarial attempt by a malicious user to manipulate the agent into taking an unintended, harmful action outside its properly intended scope.

10. Code Execution#

Code execution tools give an agent the ability to actually write and run real, executable code, and then directly see the genuine results of running that code — rather than only being able to reason abstractly about what a piece of code would probably do, purely through its own internal language-based reasoning, without ever actually being able to verify that reasoning against real, concrete execution.

This is a genuinely powerful capability, and it’s worth understanding clearly why it matters so much. Language models, on their own, aren’t fundamentally calculators or code interpreters — they’re generating their text output based on learned patterns from their training data, which means they can sometimes make small logical or arithmetic mistakes, especially on more complex, multi-step calculations, purely because they’re reasoning about the answer in the abstract rather than actually, concretely computing it. Giving an agent the ability to actually write and run real code changes this dynamic substantially — instead of trying to solve a complex math problem, or process a large, complicated dataset, entirely “in its head” through pure language-based reasoning alone, it can write an actual, real script to genuinely perform that calculation with full precision, and directly, empirically observe the real, verified result, rather than just confidently generating what merely sounds like a plausible answer without any way to actually verify it’s correct.

Code execution tools are typically run inside a deliberately sandboxed, carefully isolated environment — a genuinely restricted, contained space specifically designed to prevent code the agent writes and runs from being able to cause any real, unintended damage to the broader, actual underlying system, or from being able to access things outside its own narrow, specifically intended scope. This sandboxing matters enormously for safety, precisely because code execution is inherently one of the most powerful and potentially high-risk categories of tool an agent can be given — genuinely arbitrary code, if left completely unrestricted, could in principle do almost anything at all on the underlying computer running it, so carefully containing and limiting exactly what that code is actually capable of doing and reaching is a genuinely critical design consideration.

11. Error Recovery#

Error recovery refers to how well an agent handles things going wrong during a tool call — because in any genuinely real-world system, things absolutely do go wrong with real regularity: a website might be temporarily unreachable, an API might return an unexpected error, a search might turn up no genuinely useful results at all, or a piece of code the agent wrote might have a bug and fail to actually run correctly.

A poorly designed agent simply breaks down completely the moment something unexpected happens — it either crashes outright, or it gets confused and effectively stuck, having no sensible, well-defined way of actually handling or working around the specific problem it just ran into. A genuinely well-designed agent, by contrast, is specifically built to anticipate that these kinds of failures will inevitably happen sometimes, and to actually handle them gracefully when they do, rather than treating every single failure as an unrecoverable, catastrophic dead end.

Good error recovery generally involves a few specific, practical things working together. The agent needs to be able to correctly recognize when an error has actually occurred in the first place — clearly distinguishing a genuine, real failure from a normal, valid, successful result (for example, correctly recognizing that a tool call returning “no results found” is fundamentally different from that same tool call genuinely crashing with a technical error). It then needs to reason sensibly about what to actually do next in response to that specific error — perhaps trying a meaningfully different approach altogether, perhaps trying the exact same action again in case the failure was only a temporary, transient issue, or perhaps concluding it genuinely can’t currently complete this specific part of the task and honestly, clearly communicating that limitation back to the user, rather than either quietly giving up without any real explanation, or worse, confidently making something up to paper over the actual underlying failure. This connects very directly and closely back to the Reflection pattern we covered earlier in the Agent Patterns explanation — a well-designed agent genuinely reflects on whether its last action actually succeeded as intended, and specifically uses that honest reflection to meaningfully inform what it sensibly does next, rather than simply barreling straight ahead as though every single action along the way had necessarily gone perfectly and exactly as originally planned.

12. Retry Strategies#

Retry strategies are a specific, more concrete piece of the broader error recovery puzzle we just discussed — specifically focused on the fairly common, practical question of exactly when and how an agent (or the surrounding system running it) should actually try a failed action again, rather than simply giving up entirely the very first time something doesn’t work.

This matters because not every single failure is genuinely the same kind of failure, and treating every one of them completely identically often isn’t actually the smartest, most effective way to handle things. Some failures are transient — meaning genuinely temporary, and quite likely to succeed just fine if you simply try again a short moment later (for example, a web service that’s very briefly, temporarily overloaded with a burst of traffic, or a rare, one-off momentary network hiccup). Other failures are persistent — meaning trying that exact same failed action again, completely unchanged, in exactly the same identical way, is very unlikely to actually produce any genuinely different or better result the next time (for example, if a tool call has fundamentally, structurally invalid or malformed input to begin with, or if a user is straightforwardly asking for something that a particular available tool is simply, fundamentally not capable of doing at all, no matter how many times you retry it).

A well-designed retry strategy takes this important distinction into careful account, rather than treating every possible failure completely identically. Common, well-established practices include only retrying failures that genuinely look transient in nature, based on the specific type or nature of the error encountered, rather than blindly retrying absolutely everything without any real discrimination. Using something called “backoff,” meaning waiting progressively longer between each successive retry attempt (rather than immediately and aggressively hammering away at the exact same failing action over and over again in rapid succession, which risks actually making an already-struggling underlying system’s problems meaningfully worse) is another very common, well-established practice. And critically, setting a firm, sensible limit on the maximum number of retry attempts matters a great deal too — without some kind of sensible cap in place, a genuinely persistently and permanently broken action could otherwise cause an agent to get stuck endlessly and pointlessly retrying forever, burning through real time and real cost without ever actually making any genuine progress, rather than the agent eventually, sensibly recognizing that this particular specific approach clearly isn’t working, and meaningfully moving on to try a genuinely different approach altogether, or honestly reporting the specific, persistent failure back to the user instead of silently looping forever.