Back office ops · Production

How Figma built AI-powered visual and semantic search

The problem

Designers at Figma and elsewhere were spending significant time searching for existing designs—tracking down source files from screenshots, navigating organizational structure, and asking teammates in Slack for help finding files. Finding designs between low-level component libraries and entire files was time-consuming and disruptive to creative flow.

First attempt

Figma first built a design autocomplete feature as its primary AI initiative, adding it to the product roadmap after a hackathon, before internal research revealed that improved search was a more immediate and impactful need and autocomplete was deprioritized.

Workflow diagram · grounded in source
1
Designer initiates AI search
trigger
“Visual search lets you search using a screenshot, selected frame, or even a quick sketch. Semantic search taps AI to understand the context behind text-based queries”
2
RAG retrieves similar designs
ai_action
“We knew based on Retrieval Augmented Generation (RAG) that we can improve AI outputs with examples from search”
3
Heuristics identify indexable UI frames
validation
“We solved this using heuristics like looking at common UI frame dimensions, and considering non-top level frames if they met the right conditions”
4
Edit-inactivity gate filters incomplete work
validation
“we'd only index once a file hadn't been edited for four hours. That approach kept unfinished work out of search results and increased the chances of surfacing completed designs. It also eased the load on our systems”
5
Eval plugin grades search quality
validation
“Using our public plugin API, we built a tool for grading search results on an infinite canvas”
6
Search results delivered with previews
output
“we added design details like 'peek' for quick previews, while letting designers hit CMD + Enter for a full-screen look at a result”
Reported outcome

Figma launched AI-powered visual and semantic search at Config 2024, enabling designers to search using a screenshot, sketch, or natural-language text query, after months of scoping, testing, and iterative quality evaluation.

Reported metrics
Share of canvas objects sourced from other files75%
Slack messages seeking design fileshundreds of messages in Slack
development duration for AI searchmonths of scoping and testing
Reported stack
RAGplugin API
Source
https://www.figma.com/blog/how-we-built-ai-search-in-figma/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Figma launched AI-powered visual and semantic search at Config 2024, enabling designers to search using a screenshot, sketch, or natural-language text query, after months of scoping, testing, and iterative quality eva…

What tools did this team use?

RAG, plugin API.

What results were reported?

Share of canvas objects sourced from other files: 75%; Slack messages seeking design files: hundreds of messages in Slack; development duration for AI search: months of scoping and testing (source-reported, not independently verified).

What failed first in this deployment?

Figma first built a design autocomplete feature as its primary AI initiative, adding it to the product roadmap after a hackathon, before internal research revealed that improved search was a more immediate and impactf…

How is this back office ops AI workflow structured?

Designer initiates AI search → RAG retrieves similar designs → Heuristics identify indexable UI frames → Edit-inactivity gate filters incomplete work → Eval plugin grades search quality → Search results delivered with previews.

Related back office ops cases