The best-documented enterprise agent pipeline in public. Uber published the full agent chain by name
— Intent Agent, Table Agent, Column Prune Agent — plus the three failure modes that forced the
redesign, the evaluation rubric, and the gateway layer every internal app uses to reach external models. It is
also unusually honest about what didn't ship, which is why two boxes on the L2 diagram are greyed out.
4LLM calls per query
1human gate in the path
10 → 3 minauthoring time
~300daily active users
12system workspaces
16M/mogateway LLM queries
Numbers you will see quoted that Uber never published
“80% → 86.1% accuracy”, “$120M annual productivity gains”, “140,000 hours
saved”, “40% of analyst time spent writing SQL.” None of these appear in any Uber source.
Uber publishes no end-to-end accuracy percentage for QueryGPT at all — only an evaluation rubric
(see below). The 86.1% figure in particular is a third-party blog artefact.
L1
System context
QueryGPT sits inside Uber's data platform, not its ML platform — a distinction most
reconstructions miss. The QueryGPT post never mentions Michelangelo, the Prompt Engineering Toolkit, Langfx or
the gateway. Those edges are drawn dashed.
QueryGPT is Data Platform. GenAI Gateway and
Genie are Michelangelo / AI Platform. Finch is FinTech. Enhanced Agentic-RAG is
Engineering Security & Privacy. They share infrastructure, not ownership.
The gateway naming trap
Uber has published five names for what is
probably one evolving service: Uber Generative AI API Gateway (2023), GenAI Gateway (2024), Michelangelo
Gateway (2024), Generative AI Gateway (2025), AI Gateway (2026). No post ever says they are the same thing.
Finch is the 2025 successor pattern
Its key delta over QueryGPT: it stores
natural-language aliases for both SQL columns AND their values in an OpenSearch index —
“a significant enhancement over traditional methods that rely solely on table schemas and a few sample
rows.” Plus curated single-table data marts so the LLM writes simpler SQL.
L2
The agent pipeline — every hop, as published
Uber shipped v1 at a May 2023 hackathon, then went through 20+ algorithm iterations
to reach this. Every named component below is verbatim from Uber's post. The two greyed boxes are components
Uber explicitly describes as needed or planned — not shipped.
Better RAG: similarity
search over CREATE TABLE DDL and SELECT statements “doesn't return relevant
results” for a natural-language question — embedding-space mismatch. Understanding user's
intent: “What we needed was an intermediate step, which classifies the user's prompt into an
'intent'.” Handling large schemas: tables spanning 200+ columns burned 40–60K tokens each;
three of them broke the 32K-context models of the day.
The human gate is real
Most agent diagrams show a fully autonomous chain.
Uber's has a mandatory ACK: the Table Agent's proposed list is rendered to the user, who clicks
“Looks Good” or edits it. It was added because users saw wrong tables being picked.
Intent classification is multi-label
“maps the user's question to
one or more business domains/workspaces.” Not a single-label router.
Component
Status
Uber's own words
Intent Agent, Table Agent, Column Prune Agent
shipped
All three named and diagrammed in the current design
Chat-style iteration mode
shipped
“introduced a chat style mode where users can iterate on the generated query” — past tense
Validation / self-correction agent
not shipped
“are also looking to include a 'Validation' agent… this remains an area that we haven't completely solved”
Prompt enhancer / expander
not shipped
“A 'prompt enhancer'… was needed to 'massage' the user question” — stated as a need, in Learnings
Query execution inside QueryGPT
inferred
Implied by the “Successful Run” eval metric; never drawn in the product flow
L2b
How agents reach external systems — gateway & identity
This is the layer the question “how are agents connected to external systems”
actually lives in. Top: the 2024 GenAI Gateway, the single egress point for every internal app.
Bottom: the 2026 agent-identity architecture that replaced it for agentic workloads — note the clean
split between AI Gateway (agent → model) and MCP Gateway (agent → Uber's own services).
uber — genai gateway (2024) and agent identity layer (2026)
“A pivotal design decision was to mirror the
HTTP/JSON interface of the OpenAI API… developers write code as if they're using native OpenAI client,
while being able to access LLMs from different vendors.” Rationale: a proprietary interface
“would risk becoming quickly outdated” and would break LangChain/LlamaIndex compatibility.
Cost attribution, concretely
From the Genie post — the only place Uber
explains the mechanism: “a UUID gets passed to Knowledge Service, which in turn passes the UUID through
the context header to Michelangelo Gateway… so that it can be added to an audit log used to
track costs by that UUID.”
2026: MCP over existing APIs
“Uber's microservices tech stack
comprising thousands of services was made AI-ready by enabling MCP support over existing service APIs.”
STS token exchange runs at P99 < 40 ms; the platform is adopted by thousands of internal agents.
L3
Critical path — question to executed SQL
Four LLM calls and one human gate, with the intermediate artefact at every hop.
The evaluation rubric Uber published instead of an accuracy number
Signal
What it measures
Intent
Was the workspace / business domain correctly inferred?
Table Overlap
Are the tables identified via Search + Table Agent correct? Scored 0–1 — the worked example scores 0.5 when 1 of 2 required tables is found. Uber's own text calls this “Search Overlap Score” in the example — an internal inconsistency
Successful Run
Does the generated SQL execute without error?
Run Has Output
Does it return rows? Catches the classic hallucination: WHERE status = "Finished" when the real value is "Completed"
Qualitative Query Similarity
LLM-scored 0–1 against the golden SQL
Two product flows in the harness
Vanilla — everything inferred end to end.
Decoupled — intent and datasets are supplied, isolating pure SQL-generation quality.
Their noise floor, stated openly
“we do not over-index decisions based on
~5% run-to-run changes.” A rare and useful admission.
Verified absent: there is no QueryGPT v2 post,
no QueryGPT conference talk, no InfoQ article and no podcast. Domain-restricted sweeps of InfoQ, QCon,
Data+AI Summit, Trino Summit and Presto Conference return nothing.
