Multi-Agent Hierarchical with RAG Application Setup Guide

Table of Contents

• Overview
• Setting Up the Environment
• Vector Store (Milvus) Setup
• System Architecture
• Optional Usage & Configuration
• Instana Agent Setup
• Traceloop Integration
• Run the Application

This application is inspired by the tutorial from LangGraph Hierarchical Agent Teams.

Overview

LangGraph Hierarchical Agent Teams is a multi-agent hierarchical workflow integrated with a Retrieval-Augmented Generation (RAG) to manage research and writing tasks. The application delegates tasks to specialized agent teams, such as the research_team and writing_team, based on user queries.

The supervisor node is responsible for routing tasks to specific agents. The research supervisor handles queries related to research, while the writing supervisor handles tasks like blog or document creation.

---

Setting Up the Environment

1. Create a Conda Environment

First, create a new Conda environment with Python 3.11 and above:

conda create --name my_env python=3.11 -y

2. Activate the Environment

Activate the environment:

conda activate my_env

3. Install Dependencies

Install the required dependencies from requirements.txt:

pip install -r requirements.txt

4. Configure Environment Variables

Make sure to add necessary environment variables (e.g., API key, project, URL for WatsonX and Tavily key for search purpose.) to a .env file for the application to run. Follow this to create WatsonX API key.

---

Vector Store Types Used for RAG

This project utilizes Milvus as a vector store.

Setting up Milvus:

• Install and configure a Milvus standalone server running at http://localhost:19530. Follow the instructions for setup here.

• The milvus.py script implements a Milvus client with pymilvus and connects to the Milvus server to store documents related to top AI agents.

Running the Setup:

First, populate and index the Milvus datastore by running:

python milvus.py

This step will index documents related to top AI agents in the Milvus datastore.

---

LangGraph Task Supervisor System

System Architecture

The application implements a hierarchical agent system with the following roles:

1. Main Supervisor: Routes tasks based on user queries. It assigns tasks to either the research_team or writing_team.

2. Research Team: This team consists of a Research Supervisor that delegates tasks like searching for documents to the agent nodes rag_agent, search, or web_scraper based on the query. Here the rag_agent is the RAG agent which uses Milvus DB for retrieving the documents,

3. Writing Team: This team consists of a Writing Supervisor that delegates tasks such as blog or document writing to agents like doc_writer, note_taker, or chart_generator.

---

Usage (OPTIONAL)

Below is an example of how to create a supervisor node that handles task routing:

from langgraph import BaseChatModel, State, Command

def make_supervisor_node(llm: BaseChatModel, members: list[str]) -> str:
    system_prompt = (
        "You are a research supervisor managing a structured workflow involving {members}. "
        "Your role is to assign tasks efficiently and progress through a series of steps based on the user's request."
        "\n\n"
        "**Task Progression Rules:**\n"
        "1. If the query asks for research, delegate to the research team.\n"
        "2. If the query is related to writing tasks, delegate to the writing team.\n"
        "3. If the task is completed, finalize the process or route to the next task.\n"
    )
    # Create and return supervisor node logic here...

    return supervisor_node

---

Configuration (OPTIONAL)

• System Prompt: Modify the system_prompt to customize task delegation and routing rules.
• Team Members: Update the members list with the team members for both research and writing tasks.
• Working Directory: Currently, the files generated by the writing team are stored in files/.

Set up the Instana Agent

Instana Agent

• Download and unzip Instana agent:
• Install java 1.8 in system (Instana requirement)
• Set JAVA_HOME
• Run Instana by ./bin/karaf You might need to provide access in system settings
• Configure the port of agent by setting logging endpoint to 4317(grpc) or 4318(http) Now Instana Agent should be running

Setup Environment Variables

export TRACELOOP_BASE_URL=localhost:4317
export TRACELOOP_LOGGING_ENABLED=true
export TRACELOOP_LOGGING_ENDPOINT=$TRACELOOP_BASE_URL
export TRACELOOP_METRICS_ENABLED=false
export OTEL_EXPORTER_OTLP_INSECURE=true

Observability Frameworks Instrumented

Install Traceloop for observability:

Add the following lines to the application hierarchical-multi-agent.py:

from traceloop.sdk import Traceloop
Traceloop.init(app_name="AgenticRAG", api_endpoint="localhost:4317")

---

Run the Multi-agent RAG application:

python hierarchical-multi-agent.py