Lenni's Technology Blog

Retrieval-Augmented Generation (RAG) is a compelling architectural pattern for solving a core limitation of large language models (LLMs): their inability to access or reason over your private, domain-specific data. By combining vector search with generative models, RAG enables systems that are both highly accurate and context-aware.

In this post, I’ll walk through the architecture and implementation of a custom RAG solution I built, which integrates Azure OpenAI with a variety of backend databases. It’s designed to be flexible, modular, and production-oriented — and the entire solution is available on GitHub.

Choosing a Domain

To showcase the solution, I selected a domain that’s both approachable and semantically rich: movies. A movie recommendation system makes it easy to demonstrate natural language interaction, embeddings, and similarity search — all in a context most users intuitively understand.

This scenario allows the RAG pattern to shine, enabling queries like:

• What are some underrated 90s sci-fi thrillers?
• Which movies are similar to Inception but more light-hearted?
• Suggest romantic comedies with a time travel twist.

Although the solution uses movies as a demonstration, it was built with extensibility in mind. You’re not limited to the movies dataset — the architecture fully supports plugging in your own domain.

Multi-Backend Architecture

One of the key goals was to design a solution that works across multiple data platforms. Rather than locking into a single backend, I built the system with a provider-based architecture. Each provider handles vector storage and similarity search using the most appropriate method for its platform, while the rest of the pipeline remains consistent.

Currently supported providers include:

• SQL Server 2022
• SQL Server 2025 (Private Preview)
• Azure SQL Database
• Azure SQL Database with native vector support (Public Preview)
• Azure Cosmos DB for NoSQL
• Azure Cosmos DB for MongoDB vCore

This design makes it straightforward to introduce additional platforms in the future.

A Look at the RAG Flow

Here’s how the end-to-end process works in practice.

The system begins by loading a dataset of movies, either from a local file or from Azure Blob Storage. Each movie includes a title, plot, genres, and other metadata.

Once loaded, the data is vectorized using Azure OpenAI’s embedding API. Each movie is transformed into a high-dimensional vector that captures its semantic meaning. These vectors are stored using backend-specific formats — as float arrays in SQL Server, vector types in Azure SQL, or JSON arrays in Cosmos DB.

The system also supports incremental updates. If a movie is added or modified, only the affected entries are re-vectorized, ensuring data remains consistent without redundant processing.

When a user submits a natural language question — such as “What are some action movies with a strong female lead?” — the system generates an embedding for the question, then performs a similarity search to retrieve the most relevant movie vectors.

The top results are summarized into a context string and sent to the Azure OpenAI chat completion API along with the user’s question. The final output is a coherent, conversational response — often detailed, specific, and tailored to the original query.

Optionally, the system can also generate visual content (such as movie posters) using DALL·E, which adds a creative and engaging touch.

Implementation Details

The solution is organized into a set of .NET projects, each with a distinct responsibility:

• Rag.AIClient.Engine: Core logic for orchestrating RAG operations (loading, vectorization, search, generation).
• Rag.AIClient.Engine.Custom: Extensible framework for domain-specific scenarios.
• Provider Projects: Contain platform-specific implementations (e.g., Rag.MoviesDatabase.SqlServer2022, Rag.MoviesFunction.CosmosDb).

Each provider implements the IVectorSearchProvider interface to encapsulate the mechanics of storing and querying vectors. This abstraction allows the rest of the system to remain unchanged regardless of backend.

Platform Examples

• SQL Server 2022 stores vectors in columnstore tables, and calculates cosine similarity using a scalar T-SQL function.
• SQL Server 2025 and Azure SQL Database use the new native vector type and VECTOR_DISTANCE function, and calls Azure OpenAI directly from T-SQL via sp_invoke_external_rest_endpoint.
• Azure Cosmos DB for NoSQL stores vectors as JSON arrays indexed with advanced DiskANN indexing, and performs similarity search in C# using the native vector_distance function.
• Azure Cosmos DB for MongoDB vCore utilizes IVF indexes to perform approximate nearest-neighbor search directly within queries.

This modularity ensures that every provider can take full advantage of its platform’s capabilities, without compromising the overall design.

Generating Responses

After retrieving the top vector matches, the system constructs a prompt for the chat model. It includes summaries of the retrieved movies and the original user question, like so:

You are a movie expert. Based on the following movies:
• Inception – A skilled thief uses dream-sharing technology…
• The Matrix – A hacker discovers reality is a simulation…

Answer the question: “What are some mind-bending action movies?”

The generated responses are surprisingly effective — often insightful, detailed, and well-structured.

Getting Started

To try out the solution:

git clone https://github.com/lennilobel/ai-demos-public

Pick a backend provider, load the sample data, and run the console application. The configuration is simple — just supply your Azure OpenAI endpoint and API key.

From there, you can begin submitting natural language questions and observing the full RAG pipeline in action.

Adapting the Solution to Your Own Data

To do this, use the Rag.AIClient.Engine.Custom project. It allows you to define your own entity type and implement the IRagClientCustom<T> interface. This gives you full control over how your data is loaded, how embeddings are generated, and how content is summarized before generating a final response.

Whether your use case involves legal case summaries, internal documentation, product descriptions, or support tickets, the engine remains unchanged. You only need to define how your domain data integrates into the pipeline.

This makes the solution ideal for enterprise scenarios, where RAG can be applied to internal knowledge bases, operational data, or other proprietary content.

What’s Next

There are several directions I’d like to explore next — including hybrid search, document chunking for large entries, and support for follow-up questions and conversation history.

Even in its current form, this project has proven to be a powerful and practical demonstration of how RAG can bridge the gap between LLMs and structured or semi-structured data systems. It’s helped me better understand embeddings, vector search, and prompt design — and I hope it helps you too.

You can explore the full solution on GitHub:
👉 https://github.com/lennilobel/ai-demos-public/tree/main/Rag

If you use this framework for your own project or extend it with a new provider, I’d love to hear about it!