Finance ops · Production

ClimateAligned builds a RAG-based climate finance assessment system from zero to first users

The problem

Assessing the climate credentials of financial instruments required expert analysts to manually comb through large volumes of unstructured and highly variable company documents—a slow, expensive process incapable of scaling to the large number of companies producing relevant data.

Workflow diagram · grounded in source
1
Document ingestion and storage
integration
“we're consuming documents from the internet and storing them in a structured dataset within our system”
2
Hybrid vector and keyword search
ai_action
“adding a keyword search, we used BM25, and then doing a little bit of reranking using RRF, which is Reciprocal Rank Fusion... Combining that ranking with a similarity search, we were able to see significant increases in the quality of th…”
3
RAG question answering via LLM
ai_action
“the LLM is used to process and synthesize that information... you are using a search system to deliver some sources that get grouped in to the initial prompt along with whatever question you might be asking”
4
Classifier routes items for review
routing
“a simple classifier that's able to flag those things that do need review, so that Prashant can completely ignore those that do not”
5
Expert analyst human review
human_review
“we put Prashant, our analyst, our expert, insert whatever your expert knowledge source is between the model produced answers and our production... He's only having to make some judgment call, some additional assessment on top of that”
6
Structured output with cited sources
output
“It's structured format of a number of hierarchical questions. All these questions, if you look into them, have answers, and, very importantly, have sources”
7
Data flywheel from labeled reviews
feedback_loop
“the other benefit of having this initial human in the loop is that you do build up a fair dataset now of correct and incorrect examples produced by your core RAG system”
Reported outcome

The RAG system achieved 85% accuracy on its own; with expert human-in-the-loop review, effective accuracy reached 99% while per-company assessment time fell from about 2 hours to 20 minutes.
Adding a traditional ML classifier to route only items needing review delivered roughly another order of magnitude increase in throughput.

Reported metrics
RAG system accuracy85%
System accuracy with human-in-loop99%
Per-company assessment timefrom about 2 hours to 20 minutes
assessment speed-up (AI-assisted vs fully manual)loose order of magnitude in speed-up in time
Show all 5 reported metrics
RAG system accuracy85%
system accuracy with human-in-loop99%
per-company assessment timefrom about 2 hours to 20 minutes
assessment speed-up (AI-assisted vs fully manual)loose order of magnitude in speed-up in time
throughput improvement with classifierabout another order of magnitude increase in just throughput
Reported stack
RAGPostgresOpenAIGPT-4BM25RRFPythonvector database
Source
https://www.infoq.com/presentations/rag-llm/
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The RAG system achieved 85% accuracy on its own; with expert human-in-the-loop review, effective accuracy reached 99% while per-company assessment time fell from about 2 hours to 20 minutes.

What tools did this team use?

RAG, Postgres, OpenAI, GPT-4, BM25, RRF, Python, vector database.

What results were reported?

RAG system accuracy: 85%; System accuracy with human-in-loop: 99%; Per-company assessment time: from about 2 hours to 20 minutes; assessment speed-up (AI-assisted vs fully manual): loose order of magnitude in speed-up in time (source-reported, not independently verified).

How is this finance ops AI workflow structured?

Document ingestion and storage → Hybrid vector and keyword search → RAG question answering via LLM → Classifier routes items for review → Expert analyst human review → Structured output with cited sources → Data flywheel from labeled reviews.

Related finance ops cases