Back office ops · Production

Wix automates classification and conversion of 2,000+ Velo API code samples with GPT

The problem

A Velo syntax change rendered a huge number of Wix's 2,000+ API code samples outdated, and updating them manually would have required an enormous amount of tech writer work, halting the whole team's velocity.

Workflow diagram · grounded in source
1
Search source repos for samples
integration
“Our script used the GitHub CLI to search for code sample files throughout Wix's source repos and save them in our working repo. The script also created a mapping file to correlate the location of the sample files in our repo and the sour…”
2
LLM classifies samples
ai_action
“Another script used GPT to identify which code samples required rewriting for the new syntax. We accessed GPT via API and dynamically inserted the code sample into the prompt for each request. The script added a flag for the files that r…”
3
LLM converts code syntax
ai_action
“Next, a script sent each file that required rewriting to GPT with our conversion prompt. We stored the rewritten files near the originals in the repo for easy manual comparison.”
4
TypeScript compiler validation
validation
“we used a utility that retrieves the type definitions for all the Velo APIs and uses them to compile code samples using the TypeScript compiler”
5
Manual review of compile errors
human_review
“We divided the validation among a team of writers. For each error, the writer checked if it resulted from the code conversion and applied any necessary fixes.”
6
Apply changes and create PRs
output
“Our final script used the mapping file to apply all our code sample conversions to their respective source repos and returned a list of PRs. From here, tech writers could merge and publish the updated docs.”
Reported outcome

By automating with GPT, Wix limited team involvement to 6 writers for one morning, the LLM produced no conversion errors, and reusable knowledge and code have since helped the team complete similar tasks much faster.

Reported metrics
Code samples in scopeover 2,000
Writers involved6 writers who worked on the project for one morning
LLM conversion errorsnever saw a single example of the LLM failing to convert the code properly
Classification accuracyfailed only a small number of times
Reported stack
GPT-4GPT-3.5 turboGitHub CLITypeScript compiler
Source
https://www.wix.engineering/post/docs-automating-bulk-api-reference-rewrites-with-ai
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

By automating with GPT, Wix limited team involvement to 6 writers for one morning, the LLM produced no conversion errors, and reusable knowledge and code have since helped the team complete similar tasks much faster.

What tools did this team use?

GPT-4, GPT-3.5 turbo, GitHub CLI, TypeScript compiler.

What results were reported?

Code samples in scope: over 2,000; Writers involved: 6 writers who worked on the project for one morning; LLM conversion errors: never saw a single example of the LLM failing to convert the code properly; Classification accuracy: failed only a small number of times (source-reported, not independently verified).

How is this back office ops AI workflow structured?

Search source repos for samples → LLM classifies samples → LLM converts code syntax → TypeScript compiler validation → Manual review of compile errors → Apply changes and create PRs.

Related back office ops cases