Agentic AI architecture: the loop and what breaks

Strip away the diagrams and an agentA model in a loop: it decides which tool to call, sees the result, and decides again, until it decides it is done is about twenty lines of code.

def agent(task, tools):
    messages = [{"role": "user", "content": task}]
    while True:
        response = model(messages, tools=tools)
        messages.append({"role": "assistant", "content": response.content})

        if response.stop_reason != "tool_use":
            return response                       # model decided it is finished

        results = [execute(block) for block in tool_calls(response)]
        messages.append({"role": "user", "content": results})

That is the whole idea. Everything that makes agents hard is in what that loop accumulates, what it does when a step fails, and whether you should have written a workflow instead.

First: should this be an agent?

Most production LLM systems should not be agents. There is a ladder, and each rung is cheaper, faster and easier to debug than the one above it.

The test is one question: can you write down the steps in advance? If yes, write them down. A chain that always runs retrieve → draft → check → format is more reliable, cheaper, faster, and far easier to debug than an agent that usually chooses to do the same thing.

Four criteria are worth checking before you commit to the agent tier:

• Complexity — is the task genuinely open-ended? "Turn this design doc into a PR" is. "Extract the invoice total from this PDF" is not.
• Value — does the outcome justify 10× the tokens and 20× the latency of a single call?
• Viability — is the model actually good at this task type? An agent amplifies capability; it does not create it.
• Cost of error — can mistakes be caught and reversed? Tests, review, rollback, a dry-run mode.

If any answer is no, drop a tier. The most common architectural mistake in this space is not a bad agent — it is an agent where a chain belonged.

The four things that break

Assume you genuinely need the agent tier. These are what fail, in the order they will fail on you.

1. Context growth

The loop appends every tool result to the message history. A file read is 3,000 tokens. A search result is 2,000. A stack trace is 1,500. Twenty steps in, the model is re-reading forty thousand tokens of mostly-stale output on every single turn, and paying for it every time.

Two symptoms follow, and the second is worse than the first. The obvious one is cost — input tokens grow quadratically with steps, because each turn resends everything. The subtler one is that quality degrades before the context window fills. The signal the model needs is buried under twenty stale tool results, and it starts repeating work it already did.

Three mechanisms address this, and they are not interchangeable:

context editing  -->  DELETE old tool results outright
                      cheap, lossy, good for verbose noisy output

compaction       -->  SUMMARISE the history into a compact block
                      preserves the thread of the work, costs a call

memory / files   -->  WRITE findings to durable storage, keep the pointer
                      survives across sessions, needs explicit design

Context editing is the blunt instrument: drop tool results older than N turns. It works well when results are verbose and disposable — file listings, raw HTML, long logs. It is wrong when a result from step 3 is the reason step 18 makes sense.

Compaction summarises earlier context server-side rather than discarding it. The critical implementation detail: append the full response.content back to your message list, not just the extracted text. The compaction blocks in the response are what the API uses to replace the compacted history on the next request. Extracting only the text silently loses the compaction state, and you will not get an error — just a conversation that quietly stops compacting.

import anthropic

client = anthropic.Anthropic()
messages = []


def turn(user_message):
    messages.append({"role": "user", "content": user_message})
    resp = client.beta.messages.create(
        betas=["compact-2026-01-12"],
        model="claude-opus-5",
        max_tokens=16000,
        messages=messages,
        context_management={"edits": [{"type": "compact_20260112"}]},
    )
    # Append the whole content list. Compaction blocks must be preserved.
    messages.append({"role": "assistant", "content": resp.content})
    return resp

The third mechanism is the one that scales furthest: have the agent write to files and keep only paths in context. A research agent that appends findings to notes.md and holds one path in its history can run for hours. One that holds every fetched page in messages cannot.

2. Tool result size

Related but distinct: an individual result that is too big poisons the context in a single step.

# Bad: returns 40k tokens of JSON, most of it irrelevant
def search_orders(query: str) -> str:
    return json.dumps(db.search(query))          # 800 rows, every column

# Better: bounded, projected, and it tells the model what it truncated
def search_orders(query: str, limit: int = 20) -> str:
    rows = db.search(query)[:limit]
    out = [{"id": r.id, "customer": r.customer, "total": r.total,
            "status": r.status} for r in rows]
    more = max(0, len(db.search(query)) - limit)
    suffix = f"\n({more} more results — narrow the query to see them)" if more else ""
    return json.dumps(out) + suffix

Every tool should have a bounded output size. When you truncate, say so in the result — otherwise the model treats twenty rows as the complete answer and reasons confidently from a partial picture. That single line of truncation notice prevents a surprising share of wrong agent conclusions.

3. Error handling

The naive loop crashes on the first tool exception. The naive fix — swallow errors and return an empty string — is worse, because the model cannot distinguish "no results" from "the query was malformed".

def execute(block):
    try:
        result = TOOLS[block.name](**block.input)
        return {"type": "tool_result", "tool_use_id": block.id,
                "content": str(result)}
    except ValidationError as e:
        # Recoverable: tell the model exactly what to fix
        return {"type": "tool_result", "tool_use_id": block.id,
                "is_error": True,
                "content": f"Invalid arguments: {e}. Expected schema: {SCHEMA[block.name]}"}
    except PermissionError as e:
        # Not recoverable by retrying — say so, so it stops trying
        return {"type": "tool_result", "tool_use_id": block.id,
                "is_error": True,
                "content": f"Permission denied: {e}. Do not retry this tool."}
    except Exception as e:
        return {"type": "tool_result", "tool_use_id": block.id,
                "is_error": True, "content": f"Tool failed: {type(e).__name__}: {e}"}

Two rules that matter more than the code:

Return errors as tool results, not exceptions. The model is remarkably good at recovering from an error it can read. It cannot recover from a process that died.

Distinguish retryable from terminal. An agent that retries a permission error eleven times has burned your budget and your patience. Say "do not retry" in the message; models follow that instruction reliably.

There is one subtlety with parallel tool calls: when the model issues several tool_use blocks in one message, return all the tool_result blocks in a single user message — including the failed ones with is_error: true. Splitting them across messages, or dropping a failure, teaches the model to stop making parallel calls.

4. Termination

Agents get stuck. The loop above has no stopping condition except the model's own judgement, and that is not enough.

def agent(task, tools, max_steps=40, budget_usd=2.0):
    messages = [{"role": "user", "content": task}]
    spent, seen = 0.0, []

    for step in range(max_steps):
        resp = model(messages, tools=tools)
        spent += cost_of(resp)
        messages.append({"role": "assistant", "content": resp.content})

        if resp.stop_reason != "tool_use":
            return Done(resp)

        calls = tool_calls(resp)
        signature = tuple(sorted((c.name, json.dumps(c.input, sort_keys=True))
                                 for c in calls))
        if seen[-2:].count(signature) == 2:       # same call three times running
            return Stuck("repeating identical tool calls", messages)
        seen.append(signature)

        if spent > budget_usd:
            return OverBudget(spent, messages)

        messages.append({"role": "user", "content": [execute(c) for c in calls]})

    return OutOfSteps(messages)

Three independent limits, because each catches a different failure: step count catches slow drift, cost catches expensive loops, repeat detection catches the tight cycle where the agent calls the same failing tool forever.

A fourth technique is worth knowing: a task budget tells the model its own token ceiling so it paces itself and finishes gracefully rather than being cut off mid-thought.

with client.beta.messages.stream(
    model="claude-opus-5",
    max_tokens=64000,
    betas=["task-budgets-2026-03-13"],
    output_config={
        "effort": "high",
        "task_budget": {"type": "tokens", "total": 200000},
    },
    messages=messages,
    tools=tools,
) as stream:
    resp = stream.get_final_message()

This is advisory — the model sees a countdown and wraps up — whereas max_tokens is an enforced cut. Use both: the budget for graceful behaviour, the hard limits above for safety.

Designing the tool surface

Tool design has more effect on agent quality than prompt wording, and it gets a fraction of the attention.

Fewer, broader tools beat many narrow ones. Twenty tools with overlapping purposes produce selection errors. Five well-separated tools do not.

Bad:   get_user_by_id, get_user_by_email, get_user_by_phone,
       search_users, list_active_users, list_users_by_team
Good:  find_users(query, filters)  -- one entry point, structured filters

Write the description for a competent new colleague. Not "Searches users." Say what it returns, what it costs, when not to use it, and what a good query looks like. The description is the model's only documentation.

FIND_USERS = {
    "name": "find_users",
    "description": (
        "Search the user directory. Returns up to 20 matches with id, name, "
        "email, team and status. Use for questions about who someone is or "
        "which team they are on. Do NOT use for permission checks — use "
        "check_access for that, which reflects live policy. "
        "Queries match name and email substrings; filters are exact-match."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {"type": "string", "description": "Name or email substring"},
            "team": {"type": "string"},
            "status": {"type": "string", "enum": ["active", "suspended", "all"]},
        },
        "required": ["query"],
        "additionalProperties": False,
    },
    "strict": True,
}

Note strict: True with additionalProperties: False — this guarantees the arguments validate against the schema, which removes a whole category of runtime failure.

Make destructive tools ask. Anything irreversible should either be gated behind an explicit confirmation step or restricted to a dry-run that returns a plan.

def delete_records(ids: list[str], confirm_token: str = "") -> str:
    if confirm_token != EXPECTED_TOKEN:
        preview = [summarise(i) for i in ids[:10]]
        return (f"DRY RUN — would delete {len(ids)} records:\n"
                + "\n".join(preview)
                + "\nCall again with confirm_token to execute.")
    ...

The agent cannot delete anything by accident, and the human sees exactly what was about to happen.

Multi-agent, and when it is worth it

The usual multi-agent diagram — a planner, a researcher, a writer, a critic — is often theatre. Four models passing text is four times the cost and four times the places to lose information.

There is one shape where it genuinely pays: fan-out over independent sub-tasks with isolated context.

                +--> worker: read source A --> 200-token summary --+
orchestrator ---+--> worker: read source B --> 200-token summary --+--> synthesis
                +--> worker: read source C --> 200-token summary --+

each worker burns its own context on reading; the orchestrator never sees
the raw material, only the summaries

The value is not "specialisation". It is context isolation. Each worker fills its own window with raw material and returns a small result; the orchestrator's context stays clean. That is a real architectural benefit, and it maps onto a real constraint.

It is worth it when sub-tasks are genuinely independent and reading-heavy. It is not worth it when the "agents" are just a chain with extra steps, or when sub-tasks need to see each other's intermediate state — passing that state between agents costs more than doing the work in one loop.

What to log

Agents fail in ways that are invisible from the final output. Log enough to reconstruct a run.

log.info("agent_step", extra={
    "run_id": run_id,
    "step": step,
    "tools_called": [c.name for c in calls],
    "tool_args": [c.input for c in calls],
    "tool_result_tokens": [count(r) for r in results],
    "errors": [r.get("is_error", False) for r in results],
    "input_tokens": resp.usage.input_tokens,
    "cache_read_tokens": resp.usage.cache_read_input_tokens,
    "output_tokens": resp.usage.output_tokens,
    "context_total": total_context_tokens(messages),
})

Three of these fields answer most post-mortems. context_total over steps shows you whether context management is working. tool_result_tokens shows which tool is flooding the window. cache_read_tokens at zero across a run means a silent cache invalidator — a timestamp in the system prompt, a reordered tool list — and you are paying full price on every turn.

Frequently asked questions

When should I build an agent instead of a chain?

Only when you cannot enumerate the steps in advance. If the sequence is knowable, write it as code — a chain is cheaper, faster, more reliable and dramatically easier to debug. Agents earn their cost on open-ended tasks where the next step depends on what the previous one found.

How do I stop an agent from looping forever?

Three independent limits: a maximum step count, a cost budget, and repeat detection on the tool-call signature. Each catches a different failure mode, so use all three. Add a task budget so the model paces itself and finishes gracefully rather than being cut off.

Why does my agent get worse the longer it runs?

Context accumulation. Every tool result stays in the history, so the useful signal is progressively buried and cost grows quadratically. Fix it with context editing for disposable results, compaction for long threads, or by having the agent write findings to files and keep only paths in context.

How many tools should an agent have?

Fewer than you think. Five to ten well-separated tools with excellent descriptions outperform thirty overlapping ones. If two tools could plausibly answer the same request, merge them or make the boundary explicit in both descriptions.

Do multi-agent systems actually help?

For fan-out over independent, reading-heavy sub-tasks, yes — the benefit is context isolation, not specialisation. For a sequence of dependent steps, no; that is a chain wearing a costume, and passing state between agents loses more than it gains.

Next reading on Sythra Articles

• MCP and tool calling: designing tools an LLM can actually use
• LLM evaluation: grading a model with no right answer
• Production ML architecture