Marketing ops · Production
Insights Generation from Customer Feedback Using LLMs for a Leading Retailer
The problem
Organizations with high feedback volumes found manual review too time-consuming, and traditional NLP models proved ineffective at handling large, varied review comments.
First attempt
Traditional NLP models were found to be ineffective for handling large review comments, which prompted adoption of LLMs.
Workflow diagram · grounded in source
1
Data ingestion and enrichment
integration
“A data pipeline can be built to ingest the data from the data store for any data processing activities such as data cleansing and data enrichments before utilizing large language models to generate insights from the feedback”
2
Theme extraction via LLM
ai_action
“Themes Extraction and Sentiment Generator module does the job of calling the OpenAI to generate the themes from each of the review comments”
3
Sentiment generation per theme
ai_action
“For each of themes, OpenAI also generates the appropriate sentiment and the competitor comparisons”
4
Competitor comparison extraction
ai_action
“By analyzing feedback comments and identifying the competitors comparisons and sentiment, your company can address any negative aspects and enhance customer satisfaction”
5
Report and insight generation
output
“Various reports can be generated from this output. These reports may use graphs to visualize both positive and negative sentiment.”
Reported outcome
(not stated)
Reported stack
Azure OpenAIGPT
Source
https://devblogs.microsoft.com/ise/insights_generation_from_customer_feedback_using_llms/
Read source ↗Frequently asked questions
What did this team achieve with this AI workflow?
(not stated)
What tools did this team use?
Azure OpenAI, GPT.
What failed first in this deployment?
Traditional NLP models were found to be ineffective for handling large review comments, which prompted adoption of LLMs.
How is this marketing ops AI workflow structured?
Data ingestion and enrichment → Theme extraction via LLM → Sentiment generation per theme → Competitor comparison extraction → Report and insight generation.