Back office ops · Production

Wealthsimple's LLM journey: building a secure internal generative AI platform for employee productivity

The problem

Wealthsimple needed a secure way for employees to leverage LLMs without inadvertently sharing sensitive financial data with external providers, while also overcoming low adoption of an early internal gateway that employees perceived as an inferior copy of ChatGPT.

First attempt

Early nudge mechanisms that sent Slack reminders to employees using ChatGPT directly failed to change behavior, and the PII redaction model introduced accuracy and relevancy problems that degraded the user experience.

Workflow diagram · grounded in source
1
Gateway receives and proxies requests
integration
“Our gateway was a tool that we made available for all employees behind a VPN, gated by Okta, and it would proxy the information from the conversation, send it to various LLM providers such as OpenAI, and track this information”
2
PII redaction before external send
validation
“We leveraged Microsoft residuals framework along with an NER model we developed internally to detect and redact any potentially sensitive information prior to sending to OpenAI or any external LLM providers”
3
RAG over company knowledge bases
ai_action
“introducing retrieval augmented generation as an API, which also included a very deliberate choice of our vector database. We built pipelines and DAGs in Airflow, our orchestration framework, to update and index our common knowledge base…”
4
Boosterpack personal assistant
ai_action
“we built a tool we called Boosterpack, which combines a lot of the reusable building blocks that I mentioned earlier. The idea of Boosterpack is we wanted to provide a personal assistant grounded against Wealthsimple context for all of o…”
5
Multi-modal document and image input
ai_action
“We added a feature within our gateway where our end users could upload either an image or a PDF, and the LLM would be able to drive the conversation with understanding what was being sent”
6
Client ticket triage routing
routing
“This is what our client experience triaging workflow used to look like. Every single day, we get a lot of tickets, both through text and through phone calls from our clients. A few years ago, we actually had a team dedicated to reading a…”
Reported outcome

Over 2200 messages are sent daily, close to a third of the company are weekly active users, slightly over half are monthly active users, and about 80% of all LLM usage flows through the internal gateway; almost everyone surveyed reported that LLMs significantly improved their productivity.

Reported metrics
Daily messages sentover 2200
Weekly active users (share of company)close to a third of the entire company
Monthly active users (share of company)slightly over half of the company
LLM usage through internal gatewayabout 80%
Show all 8 reported metrics
daily messages sentover 2200
weekly active users (share of company)close to a third of the entire company
monthly active users (share of company)slightly over half of the company
LLM usage through internal gatewayabout 80%
multi-modal feature weekly adoptionclose to a third of all of our end users
productivity improvement from LLMssignificantly increase or improve their productivity
share of usage for programming supportalmost half
applications on data platform in first two weeksover seven
Reported stack
LLM gatewayOpenAICohereOktaSlackNER modelllama.cppLlama 2MistralWhisperElasticsearchAirflowLangChainPythonStreamlitGeminiChatGPT
Source
https://www.infoq.com/presentations/genai-productivity
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Over 2200 messages are sent daily, close to a third of the company are weekly active users, slightly over half are monthly active users, and about 80% of all LLM usage flows through the internal gateway; almost everyo…

What tools did this team use?

LLM gateway, OpenAI, Cohere, Okta, Slack, NER model, llama.cpp, Llama 2, Mistral, Whisper.

What results were reported?

Daily messages sent: over 2200; Weekly active users (share of company): close to a third of the entire company; Monthly active users (share of company): slightly over half of the company; LLM usage through internal gateway: about 80% (source-reported, not independently verified).

What failed first in this deployment?

Early nudge mechanisms that sent Slack reminders to employees using ChatGPT directly failed to change behavior, and the PII redaction model introduced accuracy and relevancy problems that degraded the user experience.

How is this back office ops AI workflow structured?

Gateway receives and proxies requests → PII redaction before external send → RAG over company knowledge bases → Boosterpack personal assistant → Multi-modal document and image input → Client ticket triage routing.

Related back office ops cases