Building Efficient AI Agents — Part 2: Technology Stack, Deployment Patterns & Cost

Choosing the stack and deploying it — with real cost figures rather than hand-waving.

This is the second installment of a four-part series:

1. Part 1: The Mental Model & Core Architecture
2. Part 2: Technology Stack, Deployment Patterns & Cost
3. Part 3: Code Reference & Agentic Workflows
4. Part 4: Security, Evaluation & Production Operations

Recommended strategy: Start with a closed LLM (OpenAI or Claude) to validate agent design. Use LiteLLM from day one so the provider is a config value, not a code dependency. Introduce open LLMs for auxiliary tasks (classification, extraction, summarization) once the core reasoning loop is stable.

Recommendation: pgvector for most production deployments. It collapses your session store and vector store into one Postgres instance, reduces operational surface area, and handles tens of millions of vectors adequately.

Understanding the from-scratch primitives means you can read any framework’s source and immediately know what it is doing underneath:

flowchart TD
    classDef dev fill:#e8f5e9,stroke:#2e7d32
    classDef prod fill:#e3f2fd,stroke:#1565c0
    classDef scale fill:#f3e5f5,stroke:#6a1b9a
    subgraph T1["Tier 1 — Development / Prototype"]
        D1["Single process<br/>Python + Jupyter"]:::dev
        D2["InMemorySessionManager"]:::dev
        D3["ChromaDB local"]:::dev
        D4["Direct LLM API calls"]:::dev
        D5["Search MCP server (npx)"]:::dev
    end
    subgraph T2["Tier 2 — Single-Server Production"]
        P1["FastAPI app server"]:::prod
        P2["Postgres + pgvector<br/>Session + vector store"]:::prod
        P3["Redis<br/>Session cache + task queue"]:::prod
        P4["Celery workers<br/>Async long-running agents"]:::prod
        P5["Nginx reverse proxy"]:::prod
        P6["Docker Compose"]:::prod
    end
    subgraph T3["Tier 3 — Distributed / Scale"]
        S1["Kubernetes<br/>Agent worker pods"]:::scale
        S2["Message queue<br/>RabbitMQ / SQS"]:::scale
        S3["Managed Postgres<br/>RDS / Cloud SQL"]:::scale
        S4["Managed vector DB<br/>Qdrant Cloud / Pinecone"]:::scale
        S5["API Gateway<br/>Auth + rate limiting"]:::scale
        S6["Observability stack<br/>OTel + Grafana / Datadog"]:::scale
    end
    T1 -->|"Validated agent design"| T2
    T2 -->|"Multi-tenant / volume growth"| T3

Move tiers only when forced: Tier 1 validates the agent design, Tier 2 serves real users on one machine with full observability, Tier 3 exists for multi-tenant scale. Most production agent systems live comfortably — and cheaply — in Tier 2.

The following prices were sourced from live provider APIs (as of 2026-07-01). These are indicative list prices — not quotes. Verify against each provider’s pricing calculator before any commitment.

VM Compute — ~2 vCPU / 8 GB (per vCPU-hour, us-east, on-demand Linux)

GCP is cheapest for VM compute at $0.0335/vCPU-hour on-demand. AWS and Azure are equivalent at $0.0480. Excludes reserved/committed-use discounts, attached storage, egress, and Windows licensing.

{"data":[{"type":"bar","x":["AWS EC2 m5.large","Azure D2s v5","GCP e2-standard-2"],"y":[0.048,0.048,0.0335],"marker":{"color":["#1976D2","#0288D1","#388E3C"]},"name":"$/vCPU-hour"}],"layout":{"title":"VM Compute — $/vCPU-hour (us-east, on-demand)","xaxis":{"title":"Provider / Service"},"yaxis":{"title":"USD per vCPU-hour"}}}

Object Storage (per GB-month, Standard tier, us-east)

Cloudflare R2 is cheapest at $0.015/GB-month, and critically charges zero egress fees — a significant advantage when agents frequently read stored documents or embeddings. Excludes request/operation charges, replication, and lifecycle transitions.

Serverless Functions (per 1M requests, after free tier)

AWS Lambda and Azure Functions are cheapest at $0.20/1M requests. All figures are invocation price only — excludes GB-second compute duration and egress; verify current pricing.

A Tier 2 single-server production baseline for a moderately active agent deployment (~5M API gateway requests/month, 100 GB document/embedding storage, two compute nodes running 24/7):

LLM API costs are not included above — they are the dominant variable cost. At GPT-4o pricing (~$2.50/1M input tokens, ~$10/1M output tokens), a moderately complex agent processing 10,000 tasks/month at ~5,000 tokens/task runs approximately $125–$300/month in LLM calls alone, depending on tool call depth. Use prompt caching aggressively — Anthropic and OpenAI both support it and can cut repeated-prefix costs by up to 90%.

Indicative, not a quote. Cloud list prices only; excludes committed-use/reserved discounts, support tiers, taxes, and egress. Verify against provider calculators before commitment.

← Building Efficient AI Agents — Part 1: The Mental Model & Core Architecture · Building Efficient AI Agents — Part 3: Code Reference & Agentic Workflows →