Back office ops · Production

Architectural lessons from building Outropy: applying iterative software engineering to generative AI systems

The problem

Engineering managers and tech leads lacked intelligent tooling—relying on folders of spreadsheets and templates carried job to job—while most AI productivity products failed to deliver because they were built with either hype-driven or slow data-science-lab approaches rather than iterative engineering discipline.

First attempt

The initial single-step RAG approach to the daily briefing—sending all Slack messages directly to the LLM and asking for a summary—worked well in demos but failed in actual use, and the product ultimately failed commercially.

Workflow diagram · grounded in source
1
Collect Slack messages
trigger
“these are all the messages that happened in Slack within the last 24 hours”
2
Parse conversations and topics
ai_action
“can you break this down into discrete conversations and tell me what are the topics of these conversations?”
3
Build domain object model
ai_action
“build this object model, and that's really a domain model the same way that we have domain models elsewhere”
4
Fetch calendar context
integration
“One thing that was really important to us was whatever was in your calendar for the day”
5
Generate daily briefing
output
“Create the summary, the daily briefing for this person”
Reported outcome

Outropy reached a few thousand active users before failing commercially; the primary value delivered was architectural knowledge—treating AI workflows as data pipelines and agents as stateful objects communicating via semantic events.

Reported metrics
Time to first public betaabout six months
Active users reacheda few thousand people actually using the application
product quality vs Slack AImiles ahead in terms of quality than Slack AI
Product outcomeproduct failed miserably
Show all 5 reported metrics
time to first public betaabout six months
active users reacheda few thousand people actually using the application
product quality vs Slack AImiles ahead in terms of quality than Slack AI
product outcomeproduct failed miserably
user growth rategrowing like crazy
Reported stack
ChatGPTGPT-4LlamaSlackRedisSQLAlchemyKafkaRAGMCPAMR
Source
https://www.infoq.com/presentations/microservices-ai-systems/?utm_campaign=infoq_content&utm_source=infoq&utm_medium=feed&utm_term=AI%2C+ML+%26+Data+Engineering-presentations
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Outropy reached a few thousand active users before failing commercially; the primary value delivered was architectural knowledge—treating AI workflows as data pipelines and agents as stateful objects communicating via…

What tools did this team use?

ChatGPT, GPT-4, Llama, Slack, Redis, SQLAlchemy, Kafka, RAG, MCP, AMR.

What results were reported?

Time to first public beta: about six months; Active users reached: a few thousand people actually using the application; product quality vs Slack AI: miles ahead in terms of quality than Slack AI; Product outcome: product failed miserably (source-reported, not independently verified).

What failed first in this deployment?

The initial single-step RAG approach to the daily briefing—sending all Slack messages directly to the LLM and asking for a summary—worked well in demos but failed in actual use, and the product ultimately failed comme…

How is this back office ops AI workflow structured?

Collect Slack messages → Parse conversations and topics → Build domain object model → Fetch calendar context → Generate daily briefing.

Related back office ops cases