Skip to main content
Function calling enables LLMs to interact with external systems by generating structured function calls that your application executes. Think of it like a voice assistant that can press buttons on your behalf: you say “check the weather in Tokyo,” the LLM decides to call get_weather(location="Tokyo"), your code actually fetches the weather, and the LLM weaves the result into a natural response. The model never actually executes code — it just decides which function to call and with what arguments. Your application is always in control of execution.

Function Schema Design

The schema is the menu you hand to the model. A clear, well-documented schema is the single biggest factor in reliable function calling. If your schema descriptions are vague, the model will guess wrong about which function to use and what arguments to pass. Practical tip: write function descriptions as if you are explaining to a new team member when to use each tool.

OpenAI Function Schema

Pydantic Schema Generation

Manually writing JSON schemas is tedious and error-prone. A better approach is to define your parameters as Pydantic models and auto-generate the OpenAI schema. This gives you validation on both sides: the model is constrained to the schema when generating arguments, and Pydantic validates the result before your function runs.

Function Execution Engine

The registry pattern decouples “what functions exist” from “how they get called.” This matters because the LLM returns a function name and arguments as strings — you need a clean way to look up the actual Python function, validate the arguments, execute it, and handle errors. Think of the registry as a switchboard operator connecting calls to the right department.

Parallel Function Execution


Argument Validation


Error Handling Patterns


Tool Choice Control


Streaming with Function Calls


Key Patterns


What is Next

LLM Orchestration

Learn to orchestrate multiple LLM providers with unified APIs

Interview Deep-Dive

Strong Answer:
  • The function calling loop is a multi-turn conversation between your application and the LLM. Here is the exact flow. Step one: you send the user’s message plus a list of tool schemas (JSON Schema definitions of your available functions) to the model. Step two: instead of returning a text response, the model returns one or more tool call objects, each containing a function name and JSON arguments. Critically, the model does not execute anything — it just generates structured data describing what it wants to call. Step three: your application parses the tool calls, validates the arguments, executes the actual functions against your APIs or databases, and collects the results. Step four: you append the tool call results as tool-role messages back into the conversation and send it to the model again. Step five: the model either generates another tool call (if it needs more information) or produces a final text response to the user.
  • The loop continues until the model decides it has enough information to answer, or until you hit a max-iterations safety limit. In production I always set a max of 5-10 iterations to prevent runaway loops where the model keeps calling tools without converging on an answer. I have seen cases where a model gets stuck in a cycle calling the same search function with slightly different queries because none of the results satisfy it.
  • The key architectural insight is that the model never directly touches your systems. It is always your code executing the functions and deciding what happens with the results. This is what makes it safe — you control validation, permissions, rate limiting, and error handling at the execution layer, not the LLM layer.
Red Flags: Candidate thinks the LLM executes the functions directly, does not mention the loop structure (thinks it is a single request-response), or does not mention the need for a max-iterations safeguard.Follow-up: The model generates a tool call with invalid arguments — maybe it hallucinates a parameter name or puts a string where a number should go. How do you handle this?I validate every tool call’s arguments against a Pydantic model before execution. If validation fails, I do not crash or silently drop the call. Instead, I send a tool-role message back to the model with a structured error explaining what went wrong: which parameter failed, what was expected, and a hint about how to fix it. Then the model gets another chance to generate correct arguments. I allow up to 2 validation retries before giving up and returning a user-friendly error. In practice, GPT-4o with strict: true in the function schema rarely generates invalid arguments because strict mode uses constrained decoding to guarantee schema conformance. But with non-strict mode or weaker models, I see validation failures on about 2-3% of calls, so the retry mechanism is essential. The Pydantic validation layer also acts as a security boundary — it prevents the model from injecting unexpected parameters that your function was not designed to handle.
Strong Answer:
  • Tool selection problems almost always trace back to one of three root causes: poor tool descriptions, overlapping tool purposes, or missing routing signals.
  • First, I audit every tool’s name and description. The description is the single most important factor in tool selection — it is the model’s only guide for when to use each tool. Vague descriptions like “search for data” cause confusion. I rewrite descriptions to be specific about when to use the tool and when not to: “Search the product catalog by keyword. Use this when the user asks about specific products, pricing, or availability. Do NOT use this for general knowledge questions.” Including explicit negative guidance (“do not use when…”) reduces false selections significantly.
  • Second, I look for overlapping tools. If I have both search_products and get_product_details, the model might call search_products when the user asks about a specific product ID, because the descriptions are not clear about the boundary. I either consolidate overlapping tools or add explicit disambiguation: “Use search_products for keyword searches across the catalog. Use get_product_details only when you have a specific product ID.”
  • Third, I reduce the tool set. With 15 tools, the model spends significant context on schema parsing and the probability of mis-selection increases. I segment tools by intent: a routing step first determines the user’s intent category, then only the 3-5 relevant tools for that category are passed to the model. This two-stage approach cut our tool mis-selection rate from 12% to under 2% at a company I worked at.
  • Finally, I use tool_choice strategically. For lookup intents where I know a tool must be called, I use tool_choice: "required" or even force a specific tool. For conversational turns where tools are optional, I use auto.
Red Flags: Candidate suggests adding more tools to solve the problem, does not mention tool descriptions as the primary lever, or does not consider reducing the tool set per request.Follow-up: How does parallel function calling work, and what gotchas should you watch out for?When the model generates multiple tool calls in a single response (parallel tool calling), your application receives an array of tool call objects. You should execute them concurrently using asyncio.gather rather than sequentially, because they are independent by definition — the model generated them in parallel specifically because they do not depend on each other. The main gotchas are: first, error isolation — if one tool call fails, you need to return the error for that specific call while still returning results for the successful ones. Do not let one failure abort the entire batch. Second, rate limiting — five parallel API calls might hit your external service’s rate limit. I use a semaphore to cap concurrency at 3-5 simultaneous outbound calls. Third, response assembly — each tool result must be sent back with the correct tool_call_id matching the original call. If you mix these up, the model gets confused about which result corresponds to which request. I have debugged this exact issue where swapped IDs caused the model to synthesize nonsensical answers because it attributed a weather API response to a database query.
Strong Answer:
  • Both APIs follow the same conceptual pattern (model suggests tool calls, you execute, you return results), but the implementation details differ in important ways.
  • OpenAI’s function calling uses a tools array with JSON Schema definitions and returns tool calls as a structured tool_calls array on the assistant message. The standout feature is strict: true mode, which uses constrained decoding to guarantee the generated arguments conform exactly to your JSON Schema. This eliminates argument validation errors at the cost of slightly higher latency. OpenAI also supports tool_choice with fine-grained control: auto, required, none, or force-a-specific-tool.
  • Anthropic’s tool use follows a similar pattern but with different ergonomics. The tool definitions go in a top-level tools parameter, and tool calls come back as tool_use content blocks within the response. One key difference is how system messages work: Anthropic separates the system prompt from message history, which affects how you structure tool instructions. Anthropic does not have an equivalent to OpenAI’s strict mode as of my last check, so argument validation on your side is more important.
  • The practical difference that matters most in production is how each handles multi-turn tool conversations and streaming. OpenAI streams tool calls as deltas that you need to accumulate (function name and arguments arrive in chunks), which requires careful buffer management. Anthropic streams tool use blocks more atomically.
  • For choosing between them: if argument schema compliance is critical (financial calculations, database queries), OpenAI’s strict mode is a significant advantage. If you need the model to handle ambiguous, open-ended tool selection with good reasoning about when not to use tools, Claude tends to be more conservative and less trigger-happy with tool calls, which can be either a pro or a con depending on your use case.
Red Flags: Candidate has only used one provider and cannot discuss tradeoffs, confuses function calling with assistants API or agent frameworks, or does not know about strict mode and its implications.Follow-up: How do you design a function calling system that works across multiple providers?I abstract the tool definition layer using Pydantic models as the source of truth. Each tool is defined as a Pydantic BaseModel with typed fields and descriptions, then I have converter functions that generate the provider-specific schema format: pydantic_to_openai_function() and pydantic_to_anthropic_tool(). The execution layer is provider-agnostic — it receives a function name and a dictionary of arguments regardless of which provider generated them. The tricky part is handling provider-specific behaviors: OpenAI might generate null for optional fields while Anthropic omits them entirely, and the tool response format differs between providers. I normalize these differences in a thin adapter layer so the rest of my application code never knows or cares which LLM generated the tool call. This abstraction paid off when we switched from OpenAI to Anthropic for our primary agent — the tool definitions and execution logic stayed identical, and we only changed the adapter layer.