Building Efficient AI Agents — Part 1: The Mental Model & Core Architecture

The mental model that makes agent behavior predictable, and the reference architecture that survives contact with production.

This is the first installment of a four-part series:

1. Part 1: The Mental Model & Core Architecture
2. Part 2: Technology Stack, Deployment Patterns & Cost
3. Part 3: Code Reference & Agentic Workflows
4. Part 4: Security, Evaluation & Production Operations

The distinction that matters most is not intelligence — it is who controls the execution flow.

A chatbot answers. An agent works. The moment a task requires unpredictable multi-step reasoning — where you cannot enumerate the steps in advance — you are in agent territory.

A useful framing is a seven-level spectrum of agency, from a single LLM call up to autonomous tool creation. Most production systems deliberately sit somewhere in the middle: agent behavior at specific stages, workflow structure everywhere else. That hybrid is not a compromise — it is the right architecture.

Every agent, regardless of framework, is three things:

1. A model — the reasoning engine that decides what to do next.
2. Tools — the action space: search, code execution, APIs, file access.
3. A loop — the harness that carries context between reasoning steps until the task is done.

The LLM does not execute tools. It generates a structured specification of which tool to call and with what arguments. The host system executes the tool and feeds the result back. This loop continues until the LLM decides it has enough information to produce a final answer. That is the complete ReAct (Reason + Act) pattern that underlies every modern agent framework — LangGraph, CrewAI, AutoGen, and every from-scratch implementation all reduce to this.

The single most important insight in agent engineering: agent quality is determined by context quality, not model intelligence alone.

Context engineering is the discipline of providing the right information at the right time in the right form. It has five strategies:

Context rot is real. Published research (Chroma, 2025) shows model performance degrades well below advertised context window limits — the “lost in the middle” effect. The practical implication: minimize context to what is necessary. More tokens ≠ better answers.

flowchart LR
    classDef client fill:#fff3e0,stroke:#e65100
    classDef core fill:#e8f5e9,stroke:#2e7d32
    classDef ext fill:#e3f2fd,stroke:#1565c0
    classDef store fill:#f3e5f5,stroke:#6a1b9a

    C["Client<br/>App / Webhook / Automation platform"]:::client
    API["API Layer<br/>FastAPI / Express"]:::client

    subgraph CORE["Agent Core"]
        LOOP["ReAct Loop<br/>Think → Act → Observe"]:::core
        CTX["ExecutionContext<br/>Immutable event log"]:::core
        REQ["LlmRequest Builder<br/>Curated context projection"]:::core
    end

    LLM["LLM Providers<br/>via LiteLLM — OpenAI / Anthropic /<br/>Gemini / self-hosted"]:::ext
    TOOLS["Tool Layer<br/>MCP client → search, code exec,<br/>custom MCP servers"]:::ext

    SESS["Session Store<br/>Postgres / Redis"]:::store
    VEC["Vector Store<br/>pgvector / Qdrant"]:::store

    C --> API --> LOOP
    LOOP --> CTX
    CTX --> REQ
    REQ --> LLM
    LLM --> LOOP
    LOOP --> TOOLS
    TOOLS --> LOOP
    CTX --> SESS
    REQ --> VEC

The most important architectural decision — and the one most tutorials miss — is separating storage from presentation.

ExecutionContext is never modified by context engineering — it is the complete audit trail. LlmRequest is built fresh each step, applying whatever compression, summarization, or filtering strategy is appropriate. This means you can change context strategy without touching history, swap providers without touching agent logic, and debug failures with a complete event log.

Tools extend the agent’s action space. The Model Context Protocol (MCP) — introduced by Anthropic in November 2024 — standardizes tool definition and discovery so that tools built by any team are immediately usable by any MCP-compatible agent.

Why MCP matters for production: Without it, every tool is a bespoke integration maintained by whoever built it. With MCP, tool development and agent development are decoupled — server developers build reliable tools, agent developers focus on reasoning and orchestration.

Memory is what separates a stateless tool from an intelligent assistant. Three layers:

1. Working context — the curated window sent to the model each step (the LlmRequest projection).
2. Session memory — the durable transcript of the current conversation, persisted to a session store so a run survives restarts.
3. Long-term memory — cross-session knowledge stored as embeddings in a vector store, retrieved by similarity when relevant.

Context management hierarchy: measure tokens first → apply compaction (free, deterministic) → apply summarization (LLM cost, semantic) → raise error. Never invert this order.

Building Efficient AI Agents — Part 2: Technology Stack, Deployment Patterns & Cost →