back_office_ops · saas · workflow
Vercel removes 80% of agent tools and achieves 100% success rate with file system agent
Vercel's internal text-to-SQL agent (d0) was built with many specialized tools, heavy prompt engineering, and complex context management that made it fragile, slow, and expensive to maintain, achieving only an 80% success rate.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · Natural language question submitted
A team member submits a natural language question to get SQL-backed data answers without writing code.
Tools used
Claude Opus 4.5AI SDKNext.jsVercel Slack BoltCube
Outcome
The file system agent achieved 100% success rate (up from 80%), ran 3.5x faster, and used 37% fewer tokens, with the same previously failing query completing in 141 seconds with 19 steps and 67,483 tokens.
What failed first
The original multi-tool architecture constrained the model's reasoning by pre-filtering context and wrapping every interaction in validation logic. Its worst-case query took 724 seconds, 100 steps, and 145,463 tokens before failing.
Results
Time saved724 seconds
Volume100%
Grounding & classification
Source type: technical build writeup
28 fields verified against source quotes, 4 dropped as unverifiable.
agentic workflowdata extractionknowledge basefailure mode describedmetric backedproduction runtime claimedtools describedsoftwareaccuracy improvementcycle time reductionemployee productivitytechnical build writeupback office opsagentic task execution