Skip to content
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ The repo includes sample data so it's ready to try end to end. In this sample ap

![RAG Architecture](docs/images/appcomponents.png)

📋 **[View detailed architecture documentation with interactive Mermaid diagrams](docs/architecture.md)**
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the link to the documentation list later in the README

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the architecture documentation link to the documentation list later in the README as requested (commit 0ebaa1f).


## Azure account requirements

**IMPORTANT:** In order to deploy and run this example, you'll need:
Expand Down
2 changes: 2 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
Consult the main [README](../README.md) for general information about the project.
These are advanced topics that are not necessary for a basic deployment.

- **[Architecture Overview](architecture.md)** - Detailed diagrams and component descriptions
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the link to the list below, above "HTTP Protocol"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the Architecture Overview link to the list above HTTP Protocol as requested (commit 481c539).


- Deploying:
- [Troubleshooting deployment](docs/deploy_troubleshooting.md)
- [Debugging the app on App Service](appservice.md)
Expand Down
200 changes: 200 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
# Application Architecture
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Application Architecture
# RAG Chat: Application Architecture

All documents use "RAG Chat" as prefix in title, I believe

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the title to "RAG Chat: Application Architecture" to match the pattern used in other documents (commit 0ebaa1f).


This document provides a detailed architectural overview of the Azure Search OpenAI demo application using Mermaid diagrams.

## Overview
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Combine Overview with intro sentence, remove mention of Mermaid

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link back to README as well, for people who find this that want to get started with the repo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Combined the Overview with the intro sentence, removed mention of Mermaid, and added a link back to README as requested (commit 0ebaa1f).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added link back to README for people who find this document and want to get started with the repo (commit 0ebaa1f).


The Azure Search OpenAI demo is a Retrieval Augmented Generation (RAG) application that creates a ChatGPT-like experience over your own documents. It combines Azure OpenAI Service for AI capabilities with Azure AI Search for document indexing and retrieval.

## Architecture Diagram

The following diagram illustrates the complete architecture including user interaction flow, application components, and Azure services:

```mermaid
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error:

Parse error on line 25:
...ddings
- GPT-4V (optional)]
-----------------------^
Expecting 'SQE', 'DOUBLECIRCLEEND', 'PE', '-)', 'STADIUMEND', 'SUBROUTINEEND', 'PIPE', 'CYLINDEREND', 'DIAMOND_STOP', 'TAGEND', 'TRAPEND', 'INVTRAPEND', 'UNICODE_TEXT', 'TEXT', 'TAGSTART', got 'PS'

For more information, see https://docs.github.com/get-started/writing-on-github/working-with-advanced-formatting/creating-diagrams#creating-mermaid-diagrams

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the Mermaid diagram parse error by removing the problematic vision approaches syntax. The diagram should now render correctly (commit 0ebaa1f).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It still says--

Unable to render rich display

Parse error on line 24:
...ddings
- GPT-4V (optional)]
-----------------------^
Expecting 'SQE', 'DOUBLECIRCLEEND', 'PE', '-)', 'STADIUMEND', 'SUBROUTINEEND', 'PIPE', 'CYLINDEREND', 'DIAMOND_STOP', 'TAGEND', 'TRAPEND', 'INVTRAPEND', 'UNICODE_TEXT', 'TEXT', 'TAGSTART', got 'PS'

For more information, see https://docs.github.com/get-started/writing-on-github/working-with-advanced-formatting/creating-diagrams#creating-mermaid-diagrams

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the Mermaid parse error by removing parentheses from the GPT-4V text. Changed "GPT-4V (optional)" to "GPT-4 Vision" to resolve the syntax issue (commit a7bd26d).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still error'ing

Parse error on line 27:
...Azure AI Vision
(optional)]
-----------------------^
Expecting 'SQE', 'DOUBLECIRCLEEND', 'PE', '-)', 'STADIUMEND', 'SUBROUTINEEND', 'PIPE', 'CYLINDEREND', 'DIAMOND_STOP', 'TAGEND', 'TRAPEND', 'INVTRAPEND', 'UNICODE_TEXT', 'TEXT', 'TAGSTART', got 'PS'

For more information, see https://docs.github.com/get-started/writing-on-github/working-with-advanced-formatting/creating-diagrams#creating-mermaid-diagrams

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Fix mermaid parse error

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the Mermaid parse error by removing parentheses from the optional service labels. Changed "(optional)" to "optional" for Azure AI Vision, Azure Speech Services, and Azure Cosmos DB (commit 7e033d7).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the Mermaid parse error by removing dashes from node labels throughout the diagram. The issue was caused by the dash characters (-) within the node text, which Mermaid was interpreting as syntax elements rather than display text (commit 7e9761e).

graph TB
subgraph "User Interface"
User[👤 User]
Browser[🌐 Web Browser]
end
subgraph "Application Layer"
subgraph "Frontend"
React[⚛️ React/TypeScript App<br/>- Chat Interface<br/>- Settings Panel<br/>- Citation Display]
end
subgraph "Backend"
API[🐍 Python API<br/>Flask/Quart<br/>- Chat Endpoints<br/>- Document Upload<br/>- Authentication]
subgraph "Approaches"
CRR[ChatReadRetrieveRead<br/>Approach]
RTR[RetrieveThenRead<br/>Approach]
Vision[Vision Approaches<br/>GPT-4V Support]
end
end
end
subgraph "Azure Services"
subgraph "AI Services"
OpenAI[🤖 Azure OpenAI<br/>- GPT-4 Mini<br/>- Text Embeddings<br/>- GPT-4V (optional)]
Search[🔍 Azure AI Search<br/>- Vector Search<br/>- Semantic Ranking<br/>- Full-text Search]
DocIntel[📄 Azure Document<br/>Intelligence<br/>- Text Extraction<br/>- Layout Analysis]
Vision2[👁️ Azure Computer Vision<br/>(optional)]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename "Azure Computer Vision" to "Azure AI Vision"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed "Azure Computer Vision" to "Azure AI Vision" in the diagram (commit 0ebaa1f).

Speech[🎤 Azure Speech<br/>Services (optional)]
end
subgraph "Storage & Data"
Blob[💾 Azure Blob Storage<br/>- Document Storage<br/>- User Uploads]
Cosmos[🗃️ Azure Cosmos DB<br/>- Chat History<br/>(optional)]
end
subgraph "Platform Services"
ContainerApps[📦 Azure Container Apps<br/>or App Service<br/>- Application Hosting]
AppInsights[📊 Application Insights<br/>- Monitoring<br/>- Telemetry]
KeyVault[🔐 Azure Key Vault<br/>- Secrets Management]
end
end
subgraph "Data Processing"
PrepDocs[⚙️ Document Preparation<br/>Pipeline<br/>- Text Extraction<br/>- Chunking<br/>- Embedding Generation<br/>- Indexing]
end
%% User Interaction Flow
User -.-> Browser
Browser <--> React
React <--> API
%% Backend Processing
API --> CRR
API --> RTR
API --> Vision
%% Azure Service Connections
API <--> OpenAI
API <--> Search
API <--> Blob
API <--> Cosmos
API <--> Speech
%% Document Processing Flow
Blob --> PrepDocs
PrepDocs --> DocIntel
PrepDocs --> OpenAI
PrepDocs --> Search
%% Vision Processing
Vision --> Vision2
Vision --> OpenAI
%% Platform Integration
ContainerApps --> API
API --> AppInsights
API --> KeyVault
%% Styling
classDef userLayer fill:#e1f5fe
classDef appLayer fill:#f3e5f5
classDef azureAI fill:#e8f5e8
classDef azureStorage fill:#fff3e0
classDef azurePlatform fill:#fce4ec
classDef processing fill:#f1f8e9
class User,Browser userLayer
class React,API,CRR,RTR,Vision appLayer
class OpenAI,Search,DocIntel,Vision2,Speech azureAI
class Blob,Cosmos azureStorage
class ContainerApps,AppInsights,KeyVault azurePlatform
class PrepDocs processing
```

## Chat Query Flow

The following sequence diagram shows how a user query is processed:

```mermaid
sequenceDiagram
participant U as User
participant F as Frontend
participant B as Backend API
participant S as Azure AI Search
participant O as Azure OpenAI
participant Bl as Blob Storage
U->>F: Enter question
F->>B: POST /chat with query
B->>S: Search for relevant documents
S-->>B: Return search results with citations
B->>O: Send query + context to GPT model
O-->>B: Return AI response
B->>Bl: Log interaction (optional)
B-->>F: Return response with citations
F-->>U: Display answer with sources
```

## Document Ingestion Flow

The following diagram shows how documents are processed and indexed:

```mermaid
sequenceDiagram
participant D as Documents
participant Bl as Blob Storage
participant P as PrepDocs Script
participant DI as Document Intelligence
participant O as Azure OpenAI
participant S as Azure AI Search
D->>Bl: Upload documents
P->>Bl: Read documents
P->>DI: Extract text and layout
DI-->>P: Return extracted content
P->>P: Split into chunks
P->>O: Generate embeddings
O-->>P: Return vector embeddings
P->>S: Index documents with embeddings
S-->>P: Confirm indexing complete
```

## Key Components

### Frontend (React/TypeScript)

- **Chat Interface**: Main conversational UI
- **Settings Panel**: Configuration options for AI behavior
- **Citation Display**: Shows sources and references
- **Authentication**: Optional user login integration

### Backend (Python)

- **API Layer**: RESTful endpoints for chat, search, and configuration
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link to HTTP protocol doc here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added link to the HTTP protocol documentation in the Backend section (commit 0ebaa1f).

- **Approach Patterns**: Different strategies for processing queries
- `ChatReadRetrieveRead`: Multi-turn conversation with retrieval
- `RetrieveThenRead`: Single-turn Q&A with retrieval
- `Vision Approaches`: Support for image-based documents
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove "Vision Approaches" as they're going away soon

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed "Vision Approaches" from both the diagram and documentation as they're going away soon (commit 0ebaa1f).

- **Authentication**: Optional integration with Azure Active Directory

### Azure Services Integration

- **Azure OpenAI**: Powers the conversational AI capabilities
- **Azure AI Search**: Provides semantic and vector search over documents
- **Azure Blob Storage**: Stores original documents and processed content
- **Azure Container Apps**: Hosts the application with automatic scaling
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove Container Apps from this list since its below

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed Container Apps from the Azure Services Integration list since it's mentioned separately in the Deployment Options section (commit 0ebaa1f).

- **Application Insights**: Provides monitoring and telemetry

## Optional Features

The architecture supports several optional features that can be enabled:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link to the guide about optional features

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added link to the optional features guide in the Optional Features section (commit 0ebaa1f).


- **GPT-4 with Vision**: Process image-heavy documents
- **Speech Services**: Voice input/output capabilities
- **Chat History**: Persistent conversation storage in Cosmos DB
- **Authentication**: User login and access control
- **Private Endpoints**: Network isolation for enhanced security

## Deployment Options

The application can be deployed using:

- **Azure Container Apps** (default): Serverless container hosting
- **Azure App Service**: Traditional PaaS hosting option
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link to doc about app service hosting

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added link to the App Service hosting guide in the Deployment Options section (commit 0ebaa1f).


Both options support the same feature set and can be configured through the Azure Developer CLI (azd).