Incident management · Production

Operation Pale Fire: Block Red-Teams goose AI Agent for Prompt Injection Vulnerabilities

The problem

Block needed to proactively identify how attackers could exploit goose, their open-source AI agent, to compromise Block employees, because AI coding agents cannot effectively isolate untrusted context from their single context window.

First attempt

Two earlier attack campaigns failed before success was achieved: the first was hampered by Google Calendar API rate limits, an MCP update that broke date-range handling, and LLM non-determinism; the second was foiled by a typo in the prompt injection.

Workflow diagram · grounded in source
1
Identify calendar MCP attack vector
trigger
“We were looking for extensions that would pull in untrusted content into the context window. In our search we stumbled upon a Google Calendar MCP.”
2
Send stealthy calendar invite
trigger
“We utilized the Google Calendar API to opt out of sending an invitation email, so instead the invite would stealthily pop up on the victims calendar”
3
Goose decodes zero-width injection
ai_action
“goose had no protections against zero-width characters, and would happily interpret them when processing the invite”
4
Injection invokes developer shell
ai_action
“In that prompt injection payload we encourage goose to invoke its built-in developer shell tool to check for "updates" to the Calendar MCP. In actuality, this contacts our C2 server”
5
Poisoned recipe kicks off infostealer
ai_action
“As part of the debugging workflow the employee clicked and executed the recipe which proceeded to kick off the infostealer”
6
DART detects and isolates threat
human_review
“The Detection and Response team (DART) quickly engaged, isolating the affected environment and coordinating with the red team to analyze the event”
7
Mitigations merged into goose
feedback_loop
“Zero width character attacks have also been mitigated by stripping when loaded into the app. Finally we have been developing new ways to detect prompt injection, with our initial version of that work being merged into goose already.”
Reported outcome

The red team successfully compromised a Block employee's laptop using prompt injection hidden in invisible Unicode characters, DART identified and contained the simulated threat, and multiple security mitigations were subsequently merged into goose.

Reported metrics
Laptop compromise resultsuccessfully compromised a Block employee's laptop
Threat detection and containmentquickly identified and contained the simulated threat
Goose security posture improvementextremely valuable to improving the security posture of goose
Reported stack
gooseModel Context Protocol (MCP)Google Calendar MCPGoogle Calendar APIdeveloper shell toolGoogle MeetGoogle Calendar
Source
https://engineering.block.xyz/blog/how-we-red-teamed-our-own-ai-agent-
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The red team successfully compromised a Block employee's laptop using prompt injection hidden in invisible Unicode characters, DART identified and contained the simulated threat, and multiple security mitigations were…

What tools did this team use?

goose, Model Context Protocol (MCP), Google Calendar MCP, Google Calendar API, developer shell tool, Google Meet, Google Calendar.

What results were reported?

Laptop compromise result: successfully compromised a Block employee's laptop; Threat detection and containment: quickly identified and contained the simulated threat; Goose security posture improvement: extremely valuable to improving the security posture of goose (source-reported, not independently verified).

What failed first in this deployment?

Two earlier attack campaigns failed before success was achieved: the first was hampered by Google Calendar API rate limits, an MCP update that broke date-range handling, and LLM non-determinism; the second was foiled…

How is this incident management AI workflow structured?

Identify calendar MCP attack vector → Send stealthy calendar invite → Goose decodes zero-width injection → Injection invokes developer shell → Poisoned recipe kicks off infostealer → DART detects and isolates threat → Mitigations merged into goose.

Related incident management cases