Legal document review · Production

Accelerating insurance policy reviews with generative AI: Verisk's Mozart companion

The problem

Verisk's insurance customers spend significant time regularly reviewing changes to policy forms, with change adoption taking days or weeks due to the complexity and unstructured nature of legal policy documents.

First attempt

The initial generative AI model results were not close to the desired level of accuracy and consistency, requiring iterative redesign, multiple foundation model calls, and testing of various foundation models before achieving acceptable quality.

Workflow diagram · grounded in source
1
Periodic document ingestion and embedding
integration
“An AWS Batch job reads these documents, chunks them into smaller slices, then creates embeddings of the text chunks using the Amazon Titan Text Embeddings model through Amazon Bedrock and stores them in an Amazon OpenSearch Service vecto…”
2
User selects documents to compare
trigger
“The user can pick the two documents that they want to compare. This action invokes an AWS Lambda function to retrieve the document embeddings from the OpenSearch Service database”
3
AI-generated change summary
ai_action
“retrieve the document embeddings from the OpenSearch Service database and present them to Anthropic's Claude 3 Sonnet FM, which is accessed through Amazon Bedrock”
4
Change details output to user
output
“The Mozart application rapidly compares policy documents and presents comprehensive change details, such as descriptions, locations, excerpts, in a tracked change format”
5
Expert evaluation and iterative feedback
feedback_loop
“Feedback from each round of tests was incorporated in subsequent tests”
Reported outcome

The Mozart companion generates over 90% good or acceptable summaries and reduces policy change adoption time from days or weeks to minutes, increasing productivity and enabling timely implementation of changes.

Reported metrics
Change adoption timefrom days or weeks to minutes
Summary quality rateover 90% good or acceptable summaries
Reported stack
Amazon BedrockAmazon S3AWS BatchAmazon Titan Text EmbeddingsAmazon OpenSearch ServiceAWS LambdaClaude 3 SonnetLangChain
Source
https://aws.amazon.com/blogs/machine-learning/accelerating-insurance-policy-reviews-with-generative-ai-verisks-mozart-companion?tag=soumet-20
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

The Mozart companion generates over 90% good or acceptable summaries and reduces policy change adoption time from days or weeks to minutes, increasing productivity and enabling timely implementation of changes.

What tools did this team use?

Amazon Bedrock, Amazon S3, AWS Batch, Amazon Titan Text Embeddings, Amazon OpenSearch Service, AWS Lambda, Claude 3 Sonnet, LangChain.

What results were reported?

Change adoption time: from days or weeks to minutes; Summary quality rate: over 90% good or acceptable summaries (source-reported, not independently verified).

What failed first in this deployment?

The initial generative AI model results were not close to the desired level of accuracy and consistency, requiring iterative redesign, multiple foundation model calls, and testing of various foundation models before a…

How is this legal document review AI workflow structured?

Periodic document ingestion and embedding → User selects documents to compare → AI-generated change summary → Change details output to user → Expert evaluation and iterative feedback.

Related legal document review cases