Back office ops · Production
Serverless semantic search over academic papers using AWS Lambda, Qdrant, and GPT-3.5-turbo
The problem
The author needed a scalable, serverless way to perform semantic search over academic papers without provisioning or managing server infrastructure.
Workflow diagram · grounded in source
1
Academic paper fetch and chunk
trigger
“by selecting the corresponding paper number, the following code will fetch it and save it locally on your computer”
2
Embedding creation and vector store
ai_action
“I used LangChain and OpenAI's embeddings to create vector representations of document chunks and store them in Qdrant”
3
User query via Streamlit
trigger
“built a Streamlit application to interact with our Lambda function and Qdrant collection”
4
RAG retrieval and LLM generation
ai_action
“retrieval QA chain. It then invokes the chain with the provided query to retrieve context from Qdrant and generate an answer using GPT-3.5-turbo”
5
Lambda response via API Gateway
output
“Lambda function integrates seamlessly with AWS API Gateway (configured as a REST API) to handle HTTP requests, invoke the retrieval and generation logic, and return the results in a structured format”
Reported outcome
A working serverless RAG application was built allowing users to ask questions about academic papers via a Streamlit interface backed by AWS Lambda, Qdrant, and GPT-3.5-turbo, described as scalable and efficient.
Reported stack
QdrantLangChainOpenAIGPT-3.5-turboAWS ECRAPI GatewayStreamlitDockerAWS CloudWatch
Source
https://mlops.community/blog/building-a-serverless-application-with-aws-lambda-and-qdrant-for-semantic-search
Read source ↗Frequently asked questions
What did this team achieve with this AI workflow?
A working serverless RAG application was built allowing users to ask questions about academic papers via a Streamlit interface backed by AWS Lambda, Qdrant, and GPT-3.5-turbo, described as scalable and efficient.
What tools did this team use?
Qdrant, LangChain, OpenAI, GPT-3.5-turbo, AWS ECR, API Gateway, Streamlit, Docker, AWS CloudWatch.
How is this back office ops AI workflow structured?
Academic paper fetch and chunk → Embedding creation and vector store → User query via Streamlit → RAG retrieval and LLM generation → Lambda response via API Gateway.