Workflow · Production

How GitHub Copilot is getting better at understanding your code

The problem

GitHub Copilot's underlying transformers could process only about 6,000 characters at a time, meaning not all of a developer's code could be used as context; the first version was also limited to the single file a developer was actively working in.

First attempt

Prior to Fill-in-the-Middle, only code before the cursor was included in prompts; code after the cursor was entirely ignored. The original version could also only consider the single active file.

Workflow diagram · grounded in source
1
Developer codes in IDE
trigger
“GitHub Copilot will generate coding suggestions whether you're currently writing or just finished a comment, or in the middle of some gnarly code”
2
Prompt assembly
ai_action
“a series of algorithms first select relevant code snippets or comments from your current file and other sources. These snippets and comments are then prioritized, filtered, and assembled into the final prompt”
3
Neighboring tabs context
ai_action
“GitHub Copilot to comb through all of the data and find matching pieces of code between their open files and the code around their cursor—and add those matches to the prompt”
4
Fill-in-the-Middle paradigm
ai_action
“With FIM, we can tell the model which part of the prompt is the prefix, and which part is the suffix”
5
Semantic vector retrieval
ai_action
“Algorithms would create embeddings for all snippets in the repository (potentially billions of them), and keep them stored in the vector database. Then, as you're coding, algorithms would embed the snippets in your IDE. Algorithms would …”
6
Coding suggestion delivered
output
“GitHub Copilot is responding to your writing and edits in real time by generating prompts–or, in other words, prioritizing and sending relevant information to the model based on your actions in your IDE—to keep giving you the best coding…”
Reported outcome

Neighboring tabs increased user acceptance of GitHub Copilot's suggestions by 5% and Fill-in-the-Middle gave a 10% relative boost in performance; GitHub's quantitative research found developers code up to 55% faster while using the pair programmer.

Reported metrics
User acceptance increase from neighboring tabs5%
performance boost from Fill-in-the-Middle10%
Developer coding speed55% faster
Reported stack
GitHub CopilotCodexGPT-3vector databaseMicrosoft Azure AI-Platform
Source
https://github.blog/ai-and-ml/github-copilot/how-github-copilot-is-getting-better-at-understanding-your-code/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Neighboring tabs increased user acceptance of GitHub Copilot's suggestions by 5% and Fill-in-the-Middle gave a 10% relative boost in performance; GitHub's quantitative research found developers code up to 55% faster w…

What tools did this team use?

GitHub Copilot, Codex, GPT-3, vector database, Microsoft Azure AI-Platform.

What results were reported?

User acceptance increase from neighboring tabs: 5%; performance boost from Fill-in-the-Middle: 10%; Developer coding speed: 55% faster (source-reported, not independently verified).

What failed first in this deployment?

Prior to Fill-in-the-Middle, only code before the cursor was included in prompts; code after the cursor was entirely ignored.

How is this workflow AI workflow structured?

Developer codes in IDE → Prompt assembly → Neighboring tabs context → Fill-in-the-Middle paradigm → Semantic vector retrieval → Coding suggestion delivered.