Customer support · Production

Lessons from launching Enterprise-grade GenAI solutions at Coinbase

The problem

Coinbase initially expected to optimize only between cost and accuracy when adopting GenAI, but encountered additional enterprise challenges including trust and safety, model availability, latency, and a rapidly shifting LLM landscape.

Workflow diagram · grounded in source

Customer or employee submits request

trigger

“Coinbase's customer-facing conversational LLM chatbot, serving all our US consumers, was launched in June, 2024”

Route to appropriate LLM

routing

“The current CB-GPT architecture is truly multi-cloud (across AWS Bedrock, GCP VertexAI, Azure GPT and open-source LLMs) with different use cases routed to the appropriate destination.”

RAG grounds response in enterprise data

ai_action

“we extensively use a technique called RAG (Retrieval Augmented Generation) to ensure our responses are anchored in reliable sources of truth. For instance, the Coinbase Chatbot responses are based on our Help Center articles and the same…”

Guardrails evaluate input and output

validation

“guardrails should evaluate both input and output information to ensure that the LLM's responses adhere to the 'Three H Principle' (Helpful, Harmless, and Honest)”

Agentified LLM executes complex tasks

ai_action

“Agentified LLMs operate with minimal human intervention, and are ideal for automating repetitive but reasonably complex tasks such as email responses, scheduling, and data entry”

Reported outcome

Coinbase built CB-GPT, a unified multi-cloud GenAI platform with RAG, guardrails, and agentic capabilities; several dozen use cases have been built by non-ML teams, and a customer-facing conversational LLM chatbot serving all US consumers launched in June 2024.

Reported metrics

use cases built by non-ML teamsseveral dozen use cases

Reported stack

CB-GPTAWS BedrockGCP VertexAIAzure GPTRAGChatGPTGoogle Gemini

Source

https://www.coinbase.com/blog/lessons-from-launching-enterprise-grade-genAI-solutions-at-Coinbase

Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Coinbase built CB-GPT, a unified multi-cloud GenAI platform with RAG, guardrails, and agentic capabilities; several dozen use cases have been built by non-ML teams, and a customer-facing conversational LLM chatbot ser…

What tools did this team use?

CB-GPT, AWS Bedrock, GCP VertexAI, Azure GPT, RAG, ChatGPT, Google Gemini.

What results were reported?

use cases built by non-ML teams: several dozen use cases (source-reported, not independently verified).

How is this customer support AI workflow structured?

Customer or employee submits request → Route to appropriate LLM → RAG grounds response in enterprise data → Guardrails evaluate input and output → Agentified LLM executes complex tasks.