Building Reliable Tool-Calling Agents: Lessons from the Field

If you've built an LLM-powered agent that calls external tools — a calculator, a database, an API — you've almost certainly encountered the parsing problem. The model generates a free-text response containing something that looks like a function call, and your code tries to extract the tool name and arguments from that text using string splitting, regex, or pattern matching.

It works in demos. It breaks in production.

In this article, I'll walk through exactly why naive output parsing is fragile, what failure modes you'll encounter at scale, and how modern structured tool-use APIs (OpenAI function calling, Anthropic tool use, Amazon Bedrock tool configuration) eliminate the problem entirely. This is drawn from hands-on experience building agentic systems — including the code examples in my book Building Agentic AI Systems.

The Allure of String Parsing

The ReAct pattern (Reason + Act) asks a model to interleave reasoning with tool invocations. In its simplest form, you prompt the model to output structured text:

Thought: I need to calculate (2+3)*4
Action: calculator((2+3)*4)

Then you parse the output:

action_line = response.split("Action:")[1].strip()
tool_name = action_line.split("(")[0].strip()
args = action_line.split("(")[1].split(")")[0]

This is elegant, readable, and pedagogically clear — which is why every tutorial (including Chapter 1 of my book) starts here. But it carries three categories of failure that compound at scale.

Three Ways String Parsing Breaks

1. Structural Ambiguity

The parser above uses split("(") to separate the tool name from arguments. But what happens when the argument itself contains parentheses?

# Model outputs: calculator((2+3)*4)
args = "calculator((2+3)*4)".split("(")[1].split(")")[0]
# Result: "2+3"  — silently wrong, not "(2+3)*4"

The parser grabs the text between the first open paren and the first close paren. Nested parentheses, quoted strings containing parens, JSON arguments — all silently truncated. This isn't a theoretical edge case; mathematical expressions, search queries with parenthetical qualifiers, and API payloads routinely contain nested delimiters.

2. Format Drift

LLMs are stochastic. Even with temperature=0, different inputs trigger different output patterns. Your parser expects Action: tool_name(args), but the model might produce:

• Action: tool_name( args ) — extra whitespace
• action: tool_name(args) — lowercase prefix
• Action: tool_name(args). — trailing punctuation
• Action:\ntool_name(args) — linebreak after colon
• I'll use the calculator. Action: tool_name(args) — preamble on the same line

Each variant requires a new special case in your parser. What begins as three lines of code becomes a 50-line regex monster — and it still breaks on the next novel format the model invents.

3. Silent Failures

The most dangerous failure mode: the parser extracts something, but it's the wrong thing. No exception is raised. The tool executes with incorrect arguments, returns a plausible-looking result, and the agent continues reasoning on corrupted data. The user sees a confident, wrong answer with no indication that a parsing error occurred upstream.

The Structured Output Solution

Modern LLM APIs solve this by moving tool-call extraction out of the text stream and into the API protocol itself. Instead of asking the model to write calculator(expression) as text, you declare tools as typed schemas and the API returns structured objects:

Approach	Provider	How It Works
Function Calling	OpenAI	Declare tools as JSON schemas; model returns tool_calls[] with name + JSON args
Tool Use	Anthropic	Declare tools in API request; model returns tool_use content blocks
Tool Configuration	Amazon Bedrock	Pass tool specs in Converse API; model returns toolUse in response content
Structured Outputs	OpenAI	Enforce full JSON schema compliance on model output — guaranteed valid structure

Here's what the production version looks like with OpenAI function calling:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=[{
        "type": "function",
        "function": {
            "name": "calculator",
            "description": "Evaluate a math expression",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {"type": "string"}
                },
                "required": ["expression"]
            }
        }
    }]
)

# No string parsing — structured extraction
tool_call = response.choices[0].message.tool_calls[0]
name = tool_call.function.name           # "calculator"
args = json.loads(tool_call.function.arguments)  # {"expression": "(2+3)*4"}

The parentheses in (2+3)*4 are now inside a JSON string value — they don't interfere with extraction. The tool name is a first-class field, not a substring. Argument types are validated against the schema. If the model doesn't want to call a tool, it simply doesn't populate tool_calls.

What Changes in Practice

Switching from string parsing to structured tool use isn't just a reliability improvement — it changes what you can build:

• Multi-tool calls per turn. The API can return multiple tool_calls in a single response. With string parsing, you'd need to parse multiple Action: lines — a combinatorial formatting nightmare.
• Typed arguments. The schema can enforce integer, boolean, enum, and nested object types. String parsing gives you one type: string.
• Parallel execution. Structured tool calls come with IDs, so you can dispatch tools concurrently and match results back to their calls unambiguously.
• No prompt engineering for format. You no longer spend tokens (and risk) on "Format your response as: Action: [tool_name(arguments)]". The model knows the schema from the API, not from prompt instructions.

When String Parsing Is Acceptable

That said, naive parsing has its place:

• Teaching. It makes the ReAct loop's mechanics visible. My Chapter 1 notebook uses it deliberately so readers can see thought → action → observation in plain text.
• Prototyping. If you're testing a new tool's description before committing to a schema, quick string parsing gets you to a working demo faster.
• Models without tool APIs. Some open-source or self-hosted models don't support structured tool use. String parsing (ideally with robust regex and validation) may be your only option.

The rule of thumb: if your system handles real user queries, use structured APIs. If you're learning or prototyping, string parsing is fine — just know its limits.

Key Takeaways

1. String parsing of LLM output is inherently fragile — nested delimiters, format drift, and silent failures compound at scale.
2. Structured tool-use APIs eliminate the parsing layer entirely — tool names, arguments, and types are first-class protocol objects.
3. The shift enables new patterns — multi-tool calls, typed arguments, parallel dispatch — that are impractical with text parsing.
4. Use string parsing only for teaching and quick prototyping, with clear caveats about its limitations.