AI Model Architecture Landscape: A Decision Framework for Commercial vs. Open Source Deployment

AI model architecture has fragmented far beyond the “LLM” category that dominates most enterprise conversations. Thirteen distinct architecture classes now serve production workloads — each with different capability profiles, maturity levels, and economics. The conventional cost comparison between commercial APIs and open source models is systematically misleading because it omits infrastructure and labor from the open source side of the ledger. This whitepaper introduces a normalizing metric — Cost per Million Tokens Equivalent (CPMT-E) — that puts API spend and fully-loaded self-hosted costs on the same denominator, making the comparison actionable. The analysis concludes that no single deployment model wins across all architectures: commercial APIs dominate at low volume and for mid-tier LLMs where labor costs never amortize; self-hosted open source wins strongly for embeddings, rerankers, speech, and diffusion at scale; and a third path — hosted open-weight inference via providers like Together.ai and Fireworks.ai — is the underrated option that outperforms both at mid-market volumes for flagship LLMs.

Enterprise AI adoption has moved from experimentation into infrastructure. The question is no longer whether to use AI but how to procure, deploy, and operate it at acceptable unit economics. Two procurement paths dominate: commercial APIs (OpenAI, Anthropic, Google, Cohere) that abstract away all infrastructure in exchange for per-token pricing, and open source models (LLaMA, DeepSeek, Mistral, Stable Diffusion, Whisper) that offer full control in exchange for engineering ownership. The comparison between these paths is almost always made on the wrong terms — API rate cards against model weights that are labeled “free,” with hardware, operations, and labor treated as externalities rather than costs.

This whitepaper addresses that gap across all thirteen architecture classes that matter in production, establishes a fair common metric, and produces a decision framework grounded in live cloud pricing and realistic labor cost assumptions.

The eight-class taxonomy that circulates most widely in enterprise AI discussions is broadly accurate but contains meaningful gaps and nuances that affect procurement decisions.

The LLM category needs an internal split: decoder-only autoregressive architectures (the GPT lineage) behave differently from encoder-decoder architectures (the T5 lineage) in terms of hardware requirements, latency profiles, and appropriate use cases. Treating them as one class obscures the choice. Mixture of Experts is an architectural pattern, not a standalone class — GPT-4, Mixtral, and DeepSeek-V3 are all LLMs built with MoE routing, and conflating the pattern with a category leads to confused hardware planning. Large Concept Models remain research-stage with no production deployment path; including them in a procurement framework alongside production-ready architectures misrepresents readiness. Large Action Models are real and commercially available via Anthropic Computer Use and OpenAI Operator, but the category boundary with “agentic LLMs” is contested and the reliability characteristics are still maturing.

Five architecture classes that drive significant production workloads are absent from the standard eight-class taxonomy: embedding models, rerankers, diffusion models, speech and audio models, and code models. These are not peripheral — the embedding-to-reranker-to-generation pipeline is the dominant retrieval-augmented generation architecture in enterprise deployments, and diffusion models represent the entirety of the commercial image generation market.

The table below presents all thirteen architecture classes with their core function and production maturity.

flowchart TD
classDef genai fill:#e3f2fd,stroke:#1565c0,color:#000
classDef vision fill:#f3e5f5,stroke:#6a1b9a,color:#000
classDef retrieval fill:#e8f5e9,stroke:#2e7d32,color:#000
classDef action fill:#fff3e0,stroke:#e65100,color:#000
classDef audio fill:#fce4ec,stroke:#880e4f,color:#000

subgraph GEN["Generative Text"]
  LLM["LLM<br>Decoder-only<br>Autoregressive"]
  SLM["SLM<br>Compact LLM<br>Edge / Low-Cost"]
  MLM["MLM<br>Encoder Bidirectional<br>BERT-family"]
  MoE["MoE<br>Sparse Expert Routing<br>Pattern on LLM/VLM"]
  LCM["LCM<br>Concept-level<br>SONAR Embedding Space"]
  CODE["Code Model<br>Syntax-aware LLM<br>Code Corpora"]
end

subgraph VIS["Vision & Multimodal"]
  VLM["VLM<br>Vision + Language<br>Contrastive or Generative"]
  SAM["SAM<br>Universal Segmentation<br>Pixel-level Masks"]
  DIFF["Diffusion<br>Image/Video/Audio Gen<br>Latent Denoising"]
end

subgraph RET["Retrieval Layer"]
  EMB["Embedding Model<br>Dense Vectors<br>Semantic Search"]
  RNK["Reranker<br>Cross-encoder<br>Precision Scoring"]
end

subgraph ACT["Action & Agent"]
  LAM["LAM<br>System-level Actions<br>Tool Use + Planning"]
end

subgraph AUD["Audio / Speech"]
  SPK["Speech Model<br>ASR + TTS<br>Multilingual"]
end

EMB -->|"retrieves candidates"| RNK
RNK -->|"top-k to"| LLM
LLM -->|"drives"| LAM
VLM -->|"visual context to"| LLM

class LLM,SLM,MLM,MoE,LCM,CODE genai
class VLM,SAM,DIFF vision
class EMB,RNK retrieval
class LAM action
class SPK audio

The flagship LLM tier is where the commercial-vs-open-source tension is sharpest and where the marketing narrative most consistently misleads buyers. GPT-4o, Claude 3.5/4 Sonnet, and Gemini 2.5 Pro represent the commercial apex: state-of-the-art reasoning, multi-modal capability, simple API integration, and per-token pricing in the $5–15 per million input tokens range with output typically priced two to three times higher. The open source counterpart tier — DeepSeek-R1-0528, DeepSeek-V3-Pro, LLaMA 3.1/3.3 70B, Mistral Large, Qwen3 — has closed the capability gap substantially. DeepSeek-R1 in particular matches or exceeds GPT-4o on reasoning benchmarks at a fraction of the API cost when hosted externally, and at significant infrastructure investment when self-hosted.

Self-hosting a 70B parameter model requires two A100 80GB GPUs at minimum, representing approximately $5,986 per month in on-demand cloud compute before any reserved pricing. This figure is a hard floor, not a ceiling — it excludes storage, networking, redundancy, and the MLOps engineering to keep the system operational.

The mid-tier commercial offerings — Claude 3.5 Haiku, Gemini 2.0 Flash, GPT-4o Mini — are priced at $0.10–0.60 per million tokens and represent one of the clearest cases where the open source self-hosting economics never close. The labor cost to operate even a modest self-hosted stack exceeds the API spend at every realistic volume for this tier. Small language models (Phi-4 Mini, Qwen3-0.6B/1.7B, Gemma-3-1B, SmolLM2) break the pattern in the other direction: they run on a single A10G instance or even consumer hardware, and the crossover to self-hosting economics arrives very quickly — around 55 million tokens per month — making SLMs the strongest open source self-hosting case in the language model category.

The commercial VLM market is dominated by the same frontier providers extending their LLMs with vision encoders: GPT-4o Vision, Claude 3.5 Sonnet, Gemini 2.0 Flash/Pro all handle image-plus-text natively with image surcharges of roughly $0.002–0.01 per image on top of token costs. The open source tier — Qwen3-VL-4B, Gemma-4-26B-A4B-it, LLaVA-1.6, InternVL2 — is production-capable. The 4B class runs on a single consumer GPU; the 26B class requires two A100s. For organizations processing sensitive documents or medical images where data cannot leave the network perimeter, the self-hosted VLM is the only viable path regardless of economics.

MoE as an architectural pattern produces models with high total parameter counts but low active-parameter counts per inference pass, which is why Mixtral 8×22B can produce near-70B-quality output while activating only 22B parameters per token. The operational reality is that the full parameter set must reside in GPU memory, which means Mixtral 8×22B requires four A100 80GB GPUs and DeepSeek-V3-Pro requires eight — infrastructure that tips the economics firmly toward commercial APIs for all but the highest-volume deployments.

These two architecture classes represent the strongest and most consistent case for open source self-hosting in the entire taxonomy. Embedding models like nomic-embed-text-v1, mxbai-embed-large-v1, and BGE-M3 run on CPU for inference and on a single T4 GPU for batch ingestion. Rerankers like BAAI/bge-reranker-v2-m3 are similarly lightweight. The commercial options — OpenAI text-embedding-3, Cohere Embed v3, Cohere Rerank — are priced at $0.02–0.13 per million tokens for embeddings and $1.00–2.00 per million tokens for reranking. Self-hosted alternatives reach cost parity within the first month of moderate usage and then produce near-zero marginal cost for all subsequent volume. For any organization running a production RAG pipeline — which is most organizations deploying LLMs — the retrieval layer should default to self-hosted open source.

The commercial diffusion market has bifurcated into consumer products (Midjourney, Adobe Firefly) and API-accessible services (DALL-E 3 at $0.04–0.12 per image, Sora for video at $0.05–0.15 per second). The open source tier is mature and capable: Stable Diffusion 3.5 and FLUX.1 dev produce commercial-quality output and run on a single RTX 4090 or A100 40GB. At high image volumes — above roughly 90,000 images per month — self-hosted diffusion becomes significantly cheaper than API pricing.

OpenAI’s Whisper, available both as a commercial API ($0.006 per minute) and as open source weights, is the clearest example of a single model straddling both markets. The self-hosted version of Whisper large-v3 runs on a single GPU or fast CPU and reaches cost parity with the API at approximately 460,000 minutes of audio per month — a threshold that most production transcription workloads cross quickly. Code models follow the LLM flagship pattern: GitHub Copilot and Cursor (backed by Claude) are the commercial standard at $19 per seat per month, while DeepSeek-Coder-V2 and Qwen2.5-Coder-32B are the open source counterparts. Self-hosting becomes attractive for large engineering organizations above roughly 500 developer seats.

LAMs are the outlier in the taxonomy — the economics comparison is currently less relevant than the reliability question. Commercial LAMs (Anthropic Computer Use, OpenAI Operator, Microsoft Copilot Agents) are available but still maturing in terms of task completion rates on complex multi-step workflows. Open source agent frameworks (OpenHands, SWE-agent) exist but require very high engineering investment to deploy reliably. The recommendation for LAMs is not yet a cost optimization question; it is a capability and reliability question that the market has not yet resolved.

A comparison of API rate cards against “free” model weights is not a cost comparison — it is a comparison of one organization’s fully-loaded costs against another organization’s marginal material cost with all labor and capital excluded. The result consistently understates the true cost of open source deployment by a factor of two to ten at low and medium token volumes.

The correct approach normalizes total cost to output volume. The Cost per Million Tokens Equivalent (CPMT-E) is defined as:

CPMT-E = Monthly Total Cost ÷ Monthly Tokens Processed (in millions)

Monthly Total Cost:
  Commercial:   (Input tokens × input rate) + (Output tokens × output rate)
  Open Source:  Hardware + Hosting + Labor (ongoing) + Amortized Setup Cost

Three volume tiers anchor the analysis because the crossover point — not the absolute cost — is the actionable insight:

Live cloud compute pricing (us-east, on-demand Linux, no reserved discounts, as of July 2026) establishes the hardware floor:

General-purpose compute reference (live pricing):

GCP is cheapest at $0.0335/vCPU-hour. Indicative, assumptions-based — NOT a quote. Excludes reserved/committed-use discounts, support tiers, and taxes. Verify against the provider’s calculator before any commitment.

Internet data egress (published list prices):

Cloudflare is cheapest at $0.00/GB egress — relevant for high-throughput model output delivery. Indicative, assumptions-based — NOT a quote.

{"data":[{"type":"bar","x":["AWS","Azure","GCP"],"y":[0.048,0.048,0.03350571],"name":"vCPU-hour (USD)","marker":{"color":["#e57373","#64b5f6","#81c784"]}},{"type":"bar","x":["AWS","Azure","GCP","Cloudflare"],"y":[0.09,0.087,0.12,0.0],"name":"Egress per GB (USD)","marker":{"color":["#ef9a9a","#90caf9","#a5d6a7","#ffcc80"]}}],"layout":{"title":"Cloud Unit Pricing: Compute and Egress (us-east, on-demand, July 2026)","barmode":"group","yaxis":{"title":"USD"},"annotations":[{"x":2,"y":0.034,"text":"Cheapest compute","showarrow":true,"arrowhead":2,"ax":40,"ay":-30},{"x":3,"y":0.005,"text":"Cheapest egress ($0)","showarrow":true,"arrowhead":2,"ax":-60,"ay":-30}]}}

GPU instance pricing used in CPMT-E calculations (published list, us-east, author-modeled monthly cost):

Monthly figures are author-modeled estimates based on published on-demand GPU instance rates × 730 hours. Reserved or committed-use pricing reduces these figures by 30–60% at sustained workloads.

Infrastructure and operations labor is the cost that most open source cost analyses either omit or undercount. The roles involved are not interchangeable with general software engineering:

The CPMT-E calculations below use a conservative blended labor assumption: $8,000 per month representing 0.5 FTE MLOps at low volume (realistic when one engineer supports multiple models), scaling to $15,000 per month at medium and high volume where dedicated operations are necessary (author estimate).

The flagship LLM comparison — GPT-4o commercial API versus DeepSeek-R1 70B self-hosted versus hosted open-weight via third-party inference providers — illustrates the three-way dynamic that applies across most of the taxonomy.

GPT-4o’s blended rate at a 70/30 input/output split is approximately $8.00 per million tokens (author’s modeled estimate based on published $5.00 input / $15.00 output rates). DeepSeek-R1 70B self-hosted carries a fixed monthly cost of approximately $13,986 at low volume (hardware $5,986 + labor $8,000), which does not change with token volume — it is a fixed infrastructure cost. The commercial API is a pure variable cost. These two lines cross at approximately 1.75 billion tokens per month (author estimate). Below that threshold, the commercial API is cheaper in absolute terms. Above it, self-hosted wins on unit economics — but only if the MLOps capacity is already in place. The hosted open-weight path (Together.ai, Fireworks.ai pricing approximately $0.90 per million tokens for DeepSeek-class models) outperforms both commercial API and self-hosted across the full volume range below roughly 2,500 million tokens per month and should be the default choice for most organizations.

{"data":[{"type":"scatter","mode":"lines+markers","name":"GPT-4o API (Commercial)","x":[50,100,250,500,1000,2500,5000],"y":[400,800,2000,4000,8000,20000,40000],"line":{"color":"#c62828","width":3},"marker":{"size":8}},{"type":"scatter","mode":"lines+markers","name":"DeepSeek-R1 Self-Hosted (OS + Labor, author estimate)","x":[50,100,250,500,1000,2500,5000],"y":[13986,13986,13986,20986,20986,20986,20986],"line":{"color":"#1565c0","width":3},"marker":{"size":8}},{"type":"scatter","mode":"lines+markers","name":"Hosted Open-Weight ~$0.90/1M (e.g. Together.ai)","x":[50,100,250,500,1000,2500,5000],"y":[45,90,225,450,900,2250,4500],"line":{"color":"#2e7d32","width":3,"dash":"dash"},"marker":{"size":8}}],"layout":{"title":"LLM Flagship: Monthly Cost by Token Volume and Deployment Model","xaxis":{"title":"Monthly Token Volume (Millions)"},"yaxis":{"title":"Monthly Cost (USD)","type":"log"},"legend":{"x":0.02,"y":0.98},"annotations":[{"x":2500,"y":2250,"text":"Hosted open-weight crossover ~2,500M tokens","showarrow":true,"arrowhead":2,"ax":-100,"ay":-40,"bgcolor":"#fff9c4","bordercolor":"#f9a825"}]}}

All CPMT-E figures are author-modeled estimates based on published API rates and live cloud compute pricing. OS CPMT-E at 500M = (Hardware + Labor) ÷ 500. Indicative — not a quote.

One cost that almost never appears in AI architecture cost discussions is data egress. Every token of model output delivered over the internet carries an egress charge from the cloud provider hosting the inference. At high token volumes — 5 billion tokens per month, with an average output of four bytes per token — the egress volume approaches 20 terabytes per month. At GCP’s published rate of $0.12 per GB, that is $2,400 per month in egress alone (author estimate). Cloudflare’s zero-egress R2 and Workers architecture is worth evaluating as an output delivery layer for high-throughput self-hosted deployments, not as a compute platform.

The cost numbers tell part of the story. The full decision requires weighting additional dimensions that vary by organization.

{"data":[{"type":"scatterpolar","r":[9,2,8,7,9,3,8],"theta":["Setup Speed","Volume Cost Efficiency","Data Privacy Control","Customization Depth","Reliability SLA","Infra Complexity","Billing Predictability"],"fill":"toself","name":"Commercial API","line":{"color":"#c62828"}},{"type":"scatterpolar","r":[2,9,9,9,6,8,5],"theta":["Setup Speed","Volume Cost Efficiency","Data Privacy Control","Customization Depth","Reliability SLA","Infra Complexity","Billing Predictability"],"fill":"toself","name":"Self-Hosted Open Source","line":{"color":"#1565c0"}},{"type":"scatterpolar","r":[8,7,4,5,8,3,7],"theta":["Setup Speed","Volume Cost Efficiency","Data Privacy Control","Customization Depth","Reliability SLA","Infra Complexity","Billing Predictability"],"fill":"toself","name":"Hosted Open-Weight","line":{"color":"#2e7d32"}}],"layout":{"title":"Deployment Model Tradeoff Radar (1–10 scale, author assessment)","polar":{"radialaxis":{"visible":true,"range":[0,10]}},"legend":{"x":0.82,"y":1.1}}}

Data privacy is the dimension that overrides cost entirely in regulated industries. Healthcare organizations subject to data residency requirements, financial institutions under specific data handling obligations, and government workloads with sovereign cloud mandates cannot use commercial APIs that process data through third-party infrastructure — regardless of price. For these organizations, the self-hosted open source path is not an optimization choice; it is the only legal path. The CPMT-E math is secondary.

Reliability SLA is the corresponding dimension that favors commercial APIs. OpenAI, Anthropic, and Google publish uptime commitments and carry the on-call engineering burden internally. Self-hosted systems require the deploying organization to own that reliability, which is expensive and often underestimated at the outset.

The CPMT-E analysis produces clear directional recommendations across the thirteen architecture classes. These are organized into four groups based on the economic dynamics.

Open Source wins quickly — deploy self-hosted by default: Embeddings, rerankers, speech/ASR, and SAM all reach commercial cost parity within the first month of moderate production usage and produce near-zero marginal cost thereafter. The hardware requirements are modest — a single A10G instance handles all four — and the operational complexity is low relative to LLM serving. Any organization running a RAG pipeline should self-host its retrieval layer. Any organization transcribing significant audio volumes should self-host Whisper. The commercial APIs for these architecture classes are appropriate for prototyping and low-volume workloads, not production.

Commercial API wins — do not self-host: Mid-tier LLMs (Claude Haiku, Gemini Flash class) and large MoE models (Mixtral 8×22B, DeepSeek-V3 class) both represent cases where the self-hosting economics never close at realistic organizational scales. Mid-tier commercial LLMs are priced so aggressively that the labor cost to operate a self-hosted alternative exceeds the API spend at every volume below several billion tokens per month. Large MoE models require GPU memory configurations that are prohibitively expensive to maintain outside of hyperscaler environments. Both categories should use commercial APIs.

Hosted open-weight is the default for flagship LLMs: For the flagship LLM tier — the GPT-4o and Claude Sonnet class of models — the correct default is neither the commercial API nor self-hosted infrastructure. Hosted open-weight inference providers (Together.ai, Fireworks.ai, Groq) serve production-quality open source models at rates of $0.50–1.50 per million tokens, roughly 80–85% below commercial API pricing, with no infrastructure ownership. This path combines the operational simplicity of an API with open-weight model economics. Self-hosted infrastructure for this tier only makes sense above approximately 2.5 billion tokens per month and where data cannot leave the organization’s network perimeter.

Conditional or wait: Diffusion models and SAM at high image volumes follow the OS-wins-at-scale pattern, but the crossover requires meaningful image throughput (90,000+ images per month for diffusion, 8 million+ images per month for segmentation) that not all workloads reach. Code models follow the LLM pattern with a per-seat crossover around 500 developer seats. Large Action Models remain in the commercial-preferred category not because of economics but because the open source reliability story is not yet mature enough for production automation. Large Concept Models are research-only and should not be included in any current procurement plan.

flowchart TD
classDef commercial fill:#fce4ec,stroke:#880e4f,color:#000
classDef opensource fill:#e8f5e9,stroke:#2e7d32,color:#000
classDef hybrid fill:#fff3e0,stroke:#e65100,color:#000
classDef decision fill:#e3f2fd,stroke:#1565c0,color:#000

START["New AI Workload"]:::decision
Q1{"Data sensitivity or<br>regulatory constraint?"}
Q2{"Monthly volume above<br>crossover threshold?"}
Q3{"MLOps team<br>already in-house?"}
Q4{"Architecture class:<br>embedding / reranker<br>/ speech / SAM?"}
Q5{"Per-seat pricing<br>and large eng org<br>500+ developers?"}

COM["Commercial API<br>Low friction, variable cost<br>No infra overhead"]:::commercial
OS["Self-Hosted Open Source<br>Full control, fixed cost<br>Requires MLOps ownership"]:::opensource
HOW["Hosted Open-Weight<br>Together.ai / Fireworks.ai<br>Best economics at mid-scale"]:::hybrid
WAIT["Wait and Monitor<br>LCM, LAM not ready<br>for production commitment"]:::decision

START --> Q1
Q1 -->|"Yes — data must stay on-prem"| Q3
Q1 -->|"No"| Q4
Q4 -->|"Yes — retrieval or speech"| OS
Q4 -->|"No"| Q2
Q2 -->|"No — below crossover"| COM
Q2 -->|"Yes — above crossover"| Q3
Q3 -->|"Yes"| OS
Q3 -->|"No"| HOW
COM --> Q5
Q5 -->|"Yes"| OS

class COM commercial
class OS opensource
class HOW hybrid
class WAIT decision

The right answer also depends on where an organization sits on the AI infrastructure maturity curve.

Organizations without dedicated ML infrastructure capability should default to commercial APIs for language models and hosted open-weight for cost-sensitive workloads, with self-hosted open source limited to the low-complexity cases (embeddings, rerankers, Whisper) where the operational burden is manageable with general DevOps skills. The total cost of building MLOps capability from scratch — recruiting, tooling, and the inevitable early operational failures — is not captured in any CPMT-E calculation and should be factored as a strategic investment decision, not an infrastructure line item.

Organizations with existing MLOps capability should treat the crossover thresholds in the CPMT-E table as concrete budget triggers: once a workload crosses its architecture-specific volume threshold, the cost case for migrating to self-hosted infrastructure or hosted open-weight is established and should be actioned within the next budget cycle. The retrieval layer (embeddings and rerankers) should be migrated to self-hosted immediately regardless of volume — the economics are favorable from the first month and the operational complexity is low.

Organizations with mature ML platform infrastructure — dedicated GPU clusters, internal model serving platforms, established MLOps teams — should self-host aggressively across the high-value architecture classes (flagship LLMs, VLMs, diffusion, speech) and use commercial APIs only for workloads that are too small to justify dedicated serving capacity or where the commercial model has a genuine capability lead that the open source alternatives have not yet matched.

One pattern runs through every architecture class in this analysis: the cost of human expertise is larger than the cost of compute for most organizations at most volumes. The self-hosted option is not “open source = free + hardware.” It is “open source = hardware + a team of engineers who can operate GPU infrastructure, manage model versioning, build evaluation pipelines, handle quantization, maintain serving reliability, and respond to production incidents.” That team is scarce, expensive, and carries opportunity cost. The CPMT-E framework makes this explicit. Any procurement decision that does not include a realistic labor estimate on the open source side of the comparison is incomplete and will produce a budget surprise within the first six months of deployment.

The recommendation that applies universally, regardless of architecture class or volume tier: price the labor before pricing the hardware, and validate that the organizational capability to own the operational burden actually exists before committing to self-hosted infrastructure. For most organizations, the hosted open-weight path eliminates that constraint while capturing the majority of the cost savings — and that is where the default position should sit until the volume or data sensitivity case for full self-hosting is clearly established.