Back office ops · Production

ZURU improved floor plan generation accuracy by 109% using Amazon Bedrock and Amazon SageMaker

The problem

ZURU's Dreamcatcher platform needed to generate 2D floor plans from natural language prompts with high accuracy across two criteria: instruction adherence (understanding room types, purposes, and orientation) and mathematical correctness (dimensions, positioning, and orientation), but the baseline GPT2 model did not meet the required accuracy threshold.

Workflow diagram · grounded in source
1
User enters natural language prompt
trigger
“When a user query such as "Build me a house with three bedrooms and two bathrooms" is entered, the workflow follows these steps”
2
Prompt decomposition via Mistral 7B
ai_action
“The first step in our workflow is to understand the unique features of the house, which we can use as search criteria to find the most relevant examples in the subsequent steps. For this step, we use Amazon Bedrock, which provides a serv…”
3
Vector search for relevant examples
integration
“We use Amazon Bedrock Knowledge Bases backed by Amazon OpenSearch Serverless as a vector database to implement metadata filtering and hybrid search to retrieve the most relevant record identifiers. Amazon Simple Storage Service (Amazon S…”
4
Retrieve floor plan data via DynamoDB
integration
“we retrieve the actual floor plan data by record identifier using Amazon DynamoDB. By splitting the search and retrieval of floor plan examples into two steps, we were able to use purpose-built services with Amazon OpenSearch, allowing f…”
5
Generate floor plan with Claude 3.5 Sonnet
ai_action
“we use Amazon Bedrock and Anthropic's Claude 3.5 Sonnet as a model with leading benchmarks in deep reasoning and mathematics to generate our new floor plan”
6
Reflection self-correction
validation
“we use Amazon Bedrock with Anthropic's Claude 3.5 Sonnet on Amazon Bedrock again and pass the original prompt, instructions, examples and newly generated floor plan back with a final instruction for the model to reflect and double-check …”
Reported outcome

Prompt engineering with Claude 3.5 Sonnet and full fine-tuning with Llama 3.1 8B both achieved 109% improvement in instruction adherence over the baseline GPT2 model; full fine-tuning also delivered a 54% increase in mathematical correctness over the baseline.

Reported metrics
instruction adherence improvement over GPT2 baseline109%
mathematical correctness improvement (full fine-tuning vs GPT2 baseline)54%
Efficiency and quality improvement from data preparation techniquemore than 20%
LoRA instruction adherence vs full fine-tuning20% less
Show all 9 reported metrics
instruction adherence improvement over GPT2 baseline109%
mathematical correctness improvement (full fine-tuning vs GPT2 baseline)54%
efficiency and quality improvement from data preparation techniquemore than 20%
LoRA instruction adherence vs full fine-tuning20% less
LoRA mathematical correctness vs full fine-tuning50% lower
low-quality data filtered from dataset30%
full fine-tuning training time25 hours
LoRA training time2 hours
fine-tuning training samples (curated dataset)200,000
Reported stack
Amazon BedrockClaude 3.5 SonnetMistral 7BAmazon Bedrock Knowledge BasesAmazon OpenSearch ServerlessAmazon DynamoDBAmazon S3Llama 3.1 8BGPT2LoRA
Source
https://aws.amazon.com/blogs/machine-learning/how-zuru-improved-the-accuracy-of-floor-plan-generation-by-109-using-amazon-bedrock-and-amazon-sagemaker?tag=soumet-20
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Prompt engineering with Claude 3.5 Sonnet and full fine-tuning with Llama 3.1 8B both achieved 109% improvement in instruction adherence over the baseline GPT2 model; full fine-tuning also delivered a 54% increase in…

What tools did this team use?

Amazon Bedrock, Claude 3.5 Sonnet, Mistral 7B, Amazon Bedrock Knowledge Bases, Amazon OpenSearch Serverless, Amazon DynamoDB, Amazon S3, Llama 3.1 8B, GPT2, LoRA.

What results were reported?

instruction adherence improvement over GPT2 baseline: 109%; mathematical correctness improvement (full fine-tuning vs GPT2 baseline): 54%; Efficiency and quality improvement from data preparation technique: more than 20%; LoRA instruction adherence vs full fine-tuning: 20% less (source-reported, not independently verified).

How is this back office ops AI workflow structured?

User enters natural language prompt → Prompt decomposition via Mistral 7B → Vector search for relevant examples → Retrieve floor plan data via DynamoDB → Generate floor plan with Claude 3.5 Sonnet → Reflection self-correction.

Related back office ops cases