Quality assurance · Production

How GitHub built GitHub Copilot: lessons from developing an enterprise LLM application

The problem

Developers were consistently crunched for time and needed to write code faster with less context switching, while existing AI coding assistants at the time could only complete a single line of code.

Workflow diagram · grounded in source
1
Focus on IDE coding functions
trigger
“we focused on one part of the software development lifecycle: coding functions in the IDE”
2
IDE modeless background integration
integration
“the team decided to focus on bringing GitHub Copilot to the IDE and making the AI capability modeless—or working in the background”
3
Neighboring tabs context processing
ai_action
“GitHub Copilot processes multiple files open in a developer's IDE instead of just the single one the developer is working on. Neighboring tabs helped to increase the acceptance rates of GitHub Copilot's suggestions by 5%”
4
Consistency and caching pipeline
validation
“the team applied two strategies: changing the parameters to reduce the randomness of outputs and caching responses”
5
Security and content filtering
validation
“the team integrated code security capabilities to filter out suggestions that could contain security vulnerabilities (e.g., SQL injections and hard coded credentials) and used natural language filters from Azure OpenAI Service to filter …”
6
Public code match filter
validation
“the GitHub Copilot team created a filter to block suggestions matching public source code in GitHub public repositories that were longer than 150 characters”
7
Scale to enterprise infrastructure
integration
“we scaled our use of Microsoft Azure's infrastructure to ensure GitHub Copilot had the quality, reliability, and responsible guardrails of a large-scale, enterprise-grade product”
Reported outcome

GitHub Copilot was scaled to a large-scale enterprise-grade product; the neighboring tabs technique increased suggestion acceptance rates by 5%, and caching reduced variability in suggestions while improving performance.

Reported metrics
Suggestion acceptance rate5%
Suggestion variabilityreduced variability in suggestions
Developer code speedwrite code faster with less context switching
Reported stack
GitHub CopilotOpenAI APIMicrosoft AzureAzure OpenAI ServiceMicrosoft Experimentation PlatformGitHub Copilot Chat
Source
https://github.blog/2023-09-06-how-to-build-an-enterprise-llm-application-lessons-from-github-copilot/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

GitHub Copilot was scaled to a large-scale enterprise-grade product; the neighboring tabs technique increased suggestion acceptance rates by 5%, and caching reduced variability in suggestions while improving performance.

What tools did this team use?

GitHub Copilot, OpenAI API, Microsoft Azure, Azure OpenAI Service, Microsoft Experimentation Platform, GitHub Copilot Chat.

What results were reported?

Suggestion acceptance rate: 5%; Suggestion variability: reduced variability in suggestions; Developer code speed: write code faster with less context switching (source-reported, not independently verified).

How is this quality assurance AI workflow structured?

Focus on IDE coding functions → IDE modeless background integration → Neighboring tabs context processing → Consistency and caching pipeline → Security and content filtering → Public code match filter → Scale to enterprise infrastructure.

Related quality assurance cases