Amit Choubey - Portfolio

Introduction

Retrieval-Augmented Generation (RAG) has emerged as a powerful approach to enhance Large Language Models (LLMs) with external knowledge. However, traditional RAG systems often use fixed, predetermined retrieval strategies that don't adapt to the complexity or specificity of user queries.

This guide introduces agentic patterns for RAG—techniques that make retrieval systems more intelligent, adaptive, and capable of reasoning about their own processes. By adding agency to RAG, we can create systems that understand when and how to retrieve information, iteratively refine their approach, and make intelligent decisions throughout the generation process.

A Refresher on RAG

Large Language Models (LLMs) or Foundational Models, are powerful but have a critical limitation: they can only generate responses based on their training data, which may be outdated or incomplete.

Retrieval-Augmented Generation (RAG) is a technique that addresses this limitation by giving models access to external knowledge. Instead of relying solely on their parameters, RAG-enhanced models retrieve relevant documents from databases, vector stores, or other sources to provide context to the LLM before generating a response. This makes the responses more accurate, up-to-date, and grounded in specific information.

The traditional RAG pipeline typically consists of:

Query Processing: Analyzing the user's question
Retrieval: Finding relevant documents based on semantic similarity
Context Construction: Preparing the retrieved information
Generation: Producing an answer using the LLM with the retrieved context

While effective, these basic RAG systems can be enhanced significantly by adding agentic capabilities.

What are Agentic Systems?

Agentic systems are AI systems that can:

Plan: Develop strategies to achieve goals
Reason: Consider alternatives and make informed decisions
Act autonomously: Take actions without explicit step-by-step instructions
Reflect: Evaluate their own performance and adapt

In the context of AI, agency refers to a system's ability to make intelligent decisions based on its environment, goals, and past experiences. An agentic system doesn't just follow predetermined steps; it adapts its approach based on the specific situation and desired outcomes.

What makes RAG Agentic?

Adding agency to RAG systems transforms them from rigid pipelines to adaptive, reasoning-driven processes. Agentic RAG systems can:

Analyze the query to determine the best retrieval strategy
Rewrite and decompose complex queries into more effective search queries
Plan multi-step retrieval journeys for complex information needs
Evaluate the quality of retrieved documents and decide whether to seek more information
Combine information from multiple sources intelligently
Adjust their approach based on success or failure

The key difference is that agentic RAG systems make decisions throughout the retrieval and generation process, rather than following a fixed, predetermined sequence of steps.

Agentic Patterns for RAG

Let's explore the key patterns that enable agency in RAG systems:

Query Analysis

Query analysis involves using the LLM to understand the intent, complexity, and knowledge domains of a user's question before attempting retrieval.

For example, when faced with the query "How has the EU's stance on agricultural subsidies evolved since the 2008 financial crisis?", an agentic RAG system might identify:

This is a complex, multi-part question
It requires information about EU policies before and after 2008
Multiple retrieval steps might be necessary
Documents about both EU policy and economic impacts should be retrieved

Based on this analysis, the system then selects appropriate retrieval strategies rather than simply embedding the whole query and finding similar documents.

Query Rewriting

Query rewriting transforms the original user query into more effective search queries. This might involve:

Simplification: Breaking down complex queries
Expansion: Adding related terms or context
Disambiguation: Clarifying vague terms
Specialization: Focusing on specific aspects

For instance, the query "What are the environmental impacts of NFTs?" might be rewritten as several search queries:

"Energy consumption of blockchain networks"
"Carbon footprint of NFT minting"
"Ethereum environmental impact"
"Green alternatives to proof-of-work NFTs"

Each rewritten query will likely retrieve more specific and relevant information than the original broad question.

Planning & Multi-Step Retrieval

For complex questions, a single retrieval step is often insufficient. Agentic RAG systems can plan a sequence of retrievals, where each step builds on previous information.

Consider the question: "How might quantum computing affect modern cryptography systems, and what measures are being developed to address these challenges?"

A multi-step retrieval plan might look like:

Retrieve information about current cryptography systems
Retrieve information about quantum computing capabilities and timelines
Retrieve specific information about quantum threats to cryptography
Retrieve information about post-quantum cryptography solutions
Synthesize the information to provide a comprehensive answer

This sequential approach ensures that each retrieval step is informed by the context gathered in previous steps.

Self Evaluation through Reflection

Agentic RAG systems can evaluate the quality of their retrievals and generated responses, then adapt accordingly. This involves:

Assessing relevance of retrieved documents
Identifying gaps in the retrieved information
Verifying factual consistency
Determining if additional retrieval is needed

For example, after generating a response, the system might reflect: "My answer about quantum-resistant algorithms doesn't mention recent NIST standardization efforts. I should retrieve more information about NIST's post-quantum cryptography standardization process to complete my response."

Bringing It All Together

These patterns can be combined to create highly effective agentic RAG systems. A typical flow might include:

Analyze the query to understand intent and complexity
Develop a retrieval plan (single or multi-step)
Rewrite queries for each retrieval step
Execute retrievals and evaluate results
If needed, refine and repeat retrievals
Generate response with the gathered context
Evaluate the response and improve if necessary

This integrated approach makes RAG systems more intelligent and capable of handling complex information needs.

Examples of Agentic RAG Pipelines

Single Agent Router

A single agent router uses the LLM to analyze the query and decide which retrieval approach to use from a set of available options:

Direct retrieval: For straightforward factual questions
Query decomposition: For complex, multi-part questions
Tool use: For questions requiring calculations or specific processing
No retrieval: For questions the LLM can answer from its parametric knowledge

This pattern allows the system to select the most appropriate strategy based on the query type, improving efficiency and accuracy.

Corrective RAG

Corrective RAG implements a feedback loop where the system:

Performs an initial retrieval and response generation
Evaluates the quality and completeness of the response
Identifies specific deficiencies or gaps
Performs targeted follow-up retrievals to address those gaps
Generates an improved response with the additional information

This pattern is particularly effective for ensuring comprehensiveness and accuracy in responses.

Adaptive RAG

Adaptive RAG adjusts its retrieval parameters and strategies based on ongoing performance:

Dynamically modifies the number of documents retrieved
Adjusts similarity thresholds based on retrieval quality
Switches between different embedding models or retrieval methods
Learns from successful and unsuccessful retrievals

This pattern makes the RAG system self-improving over time, as it learns which approaches work best for different types of queries.

Challenges & Mitigation Strategies in Agentic RAG

Implementing agentic RAG systems comes with several challenges:

Computational Overhead: Agentic patterns often require multiple LLM calls, increasing latency and costs.
Mitigation: Use caching, distilled models for simpler tasks, and efficient architectures.
Error Propagation: Mistakes in early stages (like query analysis) can cascade throughout the system.
Mitigation: Implement validation steps and fallback mechanisms for critical decision points.
Complexity Management: Agentic systems can become difficult to debug and maintain.
Mitigation: Use comprehensive logging, tracing, and visualization tools to understand system behavior.
Hallucination Risk: More complex pipelines can sometimes increase hallucination risks if not properly constrained.
Mitigation: Implement fact verification steps and provide clear sourcing in responses.

Conclusion

Agentic RAG represents the next evolution in retrieval-augmented generation systems. By incorporating planning, reasoning, and self-evaluation capabilities, these systems can handle more complex information needs with greater accuracy and adaptability.

The key takeaways from this guide are:

Agentic capabilities transform rigid RAG pipelines into adaptive, intelligent systems
Core patterns include query analysis, rewriting, multi-step retrieval, and self-evaluation
Different architectural approaches (router, corrective, adaptive) offer flexibility for various use cases
Challenges exist but can be mitigated with thoughtful system design

As LLMs continue to evolve, agentic RAG systems will play an increasingly important role in creating AI systems that can access, reason with, and effectively utilize external knowledge.

Agentic Patterns for RAG Systems

Contents