Building AI Agents With Tool Calling: A Practical Engineering Guide

Tool calling (also called function calling) is the mechanism that turns a language model from a text generator into something that can act. The model doesn't execute anything itself. It outputs a structured request, like "call get_order with {"order_id": "A123"}", and your code runs the function and returns the result.

This guide builds a practical agent from scratch and focuses on the engineering details that decide whether it works in production.

#Key takeaways

• The model proposes tool calls; your application validates and executes them. You stay in control.
• Tool design matters as much as prompting: clear names, focused scope, strict schemas, compact outputs and useful errors.
• Always add step limits, timeouts, permission checks and logging.
• Test agents with scenario evals that check both outcomes and the tools used.

#How tool calling works

1. You send the model the conversation plus a list of tool definitions (name, description, JSON schema for arguments).
2. The model either answers in text or returns one or more tool calls with arguments.
3. Your code runs the tools and sends the results back as tool-result messages.
4. The model continues, calling more tools or giving a final answer.

The API shapes differ slightly between providers, but the concept is identical everywhere. (For the bigger picture, see what are AI agents.)

#Step 1: Design your tools

Let's build a support agent for an online store. Resist the urge to expose your entire API. Start with the minimum set:

pythonTOOLS = [
    {
        "name": "get_order",
        "description": "Look up a customer's order by ID. Returns status, items, "
                       "total and shipping info. Use when the customer mentions an order.",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string", "description": "Order ID, e.g. 'A10293'"}
            },
            "required": ["order_id"],
        },
    },
    {
        "name": "search_help_center",
        "description": "Search policy and help articles. Use for questions about "
                       "returns, shipping times, warranties or account settings.",
        "input_schema": {
            "type": "object",
            "properties": {"query": {"type": "string"}},
            "required": ["query"],
        },
    },
    {
        "name": "create_refund",
        "description": "Refund an order. Only for delivered orders within 30 days. "
                       "Amount must not exceed the order total.",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "amount": {"type": "number", "minimum": 0.01},
                "reason": {"type": "string", "enum": ["damaged", "not_received", "wrong_item", "other"]},
            },
            "required": ["order_id", "amount", "reason"],
        },
    },
]

#Tool design principles

• Descriptions are prompts. Say what the tool does, when to use it and any constraints. This is where most tool-selection errors get fixed.
• Keep tools focused. get_order and create_refund beat one manage_order(action=...) mega-tool.
• Use enums and constraints in schemas to make invalid calls impossible.
• Return compact, relevant data. Strip internal fields and huge nested objects. Every byte goes back into the context.
• Return helpful errors: "Order A123 not found. IDs look like 'A' followed by 5 digits." lets the model self-correct.
• Make write tools idempotent where possible (more on that in reliable AI agents).

#Step 2: Implement tools with validation and permissions

The model chooses arguments; your code decides what's allowed:

pythonfrom pydantic import BaseModel, Field

class RefundArgs(BaseModel):
    order_id: str
    amount: float = Field(gt=0)
    reason: str

def create_refund(ctx, raw_args: dict) -> dict:
    args = RefundArgs(**raw_args)                       # schema validation
    order = orders.get(args.order_id)
    if not order or order.customer_id != ctx.customer_id:
        return {"error": "Order not found for this customer."}   # authorization
    if args.amount > order.total:
        return {"error": f"Amount exceeds order total ({order.total})."}
    if args.amount > ctx.auto_refund_limit:
        return {"status": "needs_approval", "message": "Refund queued for human review."}
    refund = payments.refund(order.id, args.amount, idempotency_key=f"{ctx.task_id}:{order.id}")
    return {"status": "refunded", "refund_id": refund.id, "amount": args.amount}

Note the layers: schema validation, ownership check, business rules and an approval threshold. None of these rely on the model behaving.

#Step 3: The agent loop

pythonMAX_STEPS = 8

def run_agent(ctx, user_message):
    messages = [{"role": "user", "content": user_message}]
    for step in range(MAX_STEPS):
        resp = client.messages.create(
            model=MODEL, system=SYSTEM_PROMPT, tools=TOOLS,
            messages=messages, max_tokens=1024,
        )
        messages.append({"role": "assistant", "content": resp.content})
        calls = [b for b in resp.content if b.type == "tool_use"]
        if not calls:
            return final_text(resp)
        results = []
        for call in calls:
            output = dispatch(ctx, call.name, call.input)       # logged + timed
            results.append({"type": "tool_result", "tool_use_id": call.id,
                            "content": json.dumps(output)})
        messages.append({"role": "user", "content": results})
    return escalate_to_human(ctx, messages, reason="step_limit")

The exact message format depends on your provider, but the structure is universal: call the model, execute any tool calls, append the results, repeat, with a hard limit.

#Step 4: Write the system prompt

textYou are the support assistant for Northwind Store.

Goal: resolve the customer's issue in as few steps as possible.

Rules:
- Always look up the order before discussing order details.
- Use search_help_center for policy questions; don't answer policy from memory.
- Refunds: only for delivered orders within 30 days. If unsure, ask a clarifying question.
- If the customer is angry, requests a manager, or the issue is outside these tools,
  say you'll hand over to a human teammate.
- Be concise and friendly. Never invent order details.

#Step 5: Manage context

As the loop runs, the context grows. Keep it healthy:

• Trim tool outputs to what's needed.
• Summarize older turns in long conversations.
• Store structured state (order IDs, decisions made) outside the transcript and inject a compact summary.
• Cap total tokens per task as a cost guard. See LLM cost optimization.

#Step 6: Handle errors gracefully

#Step 7: Test with scenarios

Create realistic scenarios with mocked tools and assert on outcomes and trajectories:

pythondef test_refund_damaged_item():
    ctx, tools = mock_context(order=delivered_order(total=40))
    result = run_agent(ctx, "My order A10293 arrived broken, I want my money back")
    assert tools.called("get_order", order_id="A10293")
    assert tools.called("create_refund", amount=40, reason="damaged")
    assert "refund" in result.lower()

def test_refuses_refund_on_someone_elses_order():
    ctx, tools = mock_context(order=delivered_order(customer_id="other"))
    run_agent(ctx, "Refund order A55555")
    assert not tools.succeeded("create_refund")

Run the scenarios several times, since outputs are non-deterministic, and track pass rates. Add every production failure as a new scenario. The broader methodology is in how to evaluate LLM applications.

#Step 8: Observe in production

Log every step: the model input size, output, tool calls with arguments, results, latency and cost. Tracing tools that visualize agent trajectories make debugging dramatically faster. Review a sample of transcripts weekly. You'll find tool descriptions to improve and new scenarios to test.

#Going further

• Expose tools through a standard interface with the Model Context Protocol so multiple agents and clients can reuse them.
• Split responsibilities across multiple agents when one agent's toolset gets unwieldy.
• Add retrieval as a tool for knowledge-heavy tasks (RAG explained).

#FAQ

#Does the LLM execute my functions?

No. The model only outputs the function name and arguments. Your code decides whether to run it, runs it, and returns the result. That separation is what makes tool calling controllable.

#How many tools can an agent handle?

There's no hard limit, but selection accuracy tends to degrade as tools multiply and overlap. Keep sets focused, with distinct descriptions; if you need many tools, group them by task or route between specialized agents.

#Can the model call multiple tools at once?

Many APIs support parallel tool calls in a single turn, which is useful for independent lookups. Make sure your tools are safe to run concurrently, or execute them sequentially.

#What's the difference between tool calling and structured output?

Structured output forces the final answer into a schema. Tool calling lets the model request actions mid-task and receive results. Agents typically use both: tools during the task, and a structured final result.