From ae2045d91efa70b85bc74bd45af965ecf310e4fd Mon Sep 17 00:00:00 2001 From: Noor Chasib Date: Wed, 19 Mar 2025 15:43:38 -0700 Subject: [PATCH 1/3] create layout css for tables, rows and columns --- .../presentation/css/layout.scss | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/web/frontend-feedback-analytics/presentation/css/layout.scss b/web/frontend-feedback-analytics/presentation/css/layout.scss index 5d0536f94..6853f737c 100644 --- a/web/frontend-feedback-analytics/presentation/css/layout.scss +++ b/web/frontend-feedback-analytics/presentation/css/layout.scss @@ -68,3 +68,25 @@ .reveal .justify-start { justify-content: flex-start; } .reveal .justify-center { justify-content: center; } .reveal .justify-end { justify-content: flex-end; } + + +.reveal table { + font-size: 0.6em; + width: 100%; +} +.reveal th { + background-color: #4CAF50; + color: white; +} +.reveal tr:nth-child(even) { + background-color: #f2f2f2; +} + +.reveal .row { + display: flex; + width: 100%; +} +.reveal .col { + flex: 1; + padding: 0 10px; +} From b424fd5f8548e6cb1f86798b34dad357cae667d0 Mon Sep 17 00:00:00 2001 From: Noor Chasib Date: Wed, 19 Mar 2025 15:43:56 -0700 Subject: [PATCH 2/3] create image presentation --- .../presentation/index.html | 1064 +++++++++++------ 1 file changed, 681 insertions(+), 383 deletions(-) diff --git a/web/frontend-feedback-analytics/presentation/index.html b/web/frontend-feedback-analytics/presentation/index.html index e24fa8af4..519407781 100644 --- a/web/frontend-feedback-analytics/presentation/index.html +++ b/web/frontend-feedback-analytics/presentation/index.html @@ -1,389 +1,687 @@ - - - + + + + AI Technical Presentation + + + + + + - - -
-
-
-
-

Chat with BC Laws

-
-

Technical presentation

-
-
-

What we will cover

-
    -
  • Introduction
    • -
  • High level architecture
    • -
  • Technologies used
    • -
  • Preprocessing (HPC)
    • -
  • MLOps
    • -
  • Analytics
    • -
  • Active Feedback
    • -
  • Challenges
    • -
  • Q&A
    • -
-
-
- -
-
- -
-
-

RAG Pipeline

-
    -
  • Feed query to sentence transformer
    • -
  • Search the vector in Neo4j (vector search)
    • -
  • Capture Top 10 results based on similarity
    • -
  • - Optional: Re-rank results with cross-encoder -
    • -
  • - Generate prompt (query stack + Top K + current query) -
    • -
  • Feed prompt to generative AI
    • -
  • Receive AI-generated response
    • -
  • Return response to user
    • -
-
-
- -
-
- -
-
- -
-
-
- -
-
-

How we index the acts and regulations

- -
-
-

How data is stored in Neo4j

- -
-
-

Graph Database and vector store

- - - - - - - - - - - - - - - - - - - - -
Data storageNeo4j IntegrationAdvanced Querying
We use a graph database to store the data. - Neo4J is leveraged for both the graph database and the - vector store. - - Neo4J enables advanced queries, such as clustering the data - and finding communities. -
Vector store is utilized for storing embeddings. - These tasks are more efficient and easier to implement using - a graph database. -
-
-
-

HPC

-

High Performance Computing for pre-processing the data

-
-
- -
-
-
-
-

MLOps & Analytics

-
    -
  • Frontend analytics data
    • -
  • Backend RAG Chain tracking
    • -
  • Apache Airflow for orchestration
    • -
  • dbt for data transformation
    • -
  • - Integration with active learning pipeline -
    • -
-
-
- -
-
- -
-
- -
-
- -
-
-
-
-

Active Learning Integration

-
    -
  • - Analytics data feeds into active learning pipeline -
    • -
  • - Helps identify areas for model improvement -
    • -
  • - Informs data selection for model fine-tuning -
    • -
  • - Enables continuous improvement of the AI system -
    • -
-
-
- -
-
- -
-
- -
-
- -
-
-

Human in the Loop

-

- To improve the AI model we need to annotate and format the data - properly. After the data is annotated we can use it to train the - different models. -

-
-
-

Embedding Adaptors

-

- If the top sources are not accurate we can retrain the embedding - model based on human feedback. -

-
-
- -
-
-

Data Annotation (NER)

-

- For improving our retrieval and enhancing our result we are using - an AI technique called NER (Named Entity Recognition) to annotate - the data. -

-

- This can be done manually with tools such as Diffgram or Doccano - or can be automated using an AI model to pre-annotate. -

-
-
-

Doccano

- -
-
-

Assisted Annotation Process

-
    -
  • - Need large amounts of training data. Initial results suggest - thousands of samples would be needed for reliable results. -
    • -
  • - Manually annotating this data takes people resources, but AI - annotation is less accurate. For 5000 records: -
    • -
      -
    • - Manually: 8-10 days with high accuracy. -
      • -
    • - Automated with generative AI: only hours but is not accurate - so far. -
      • -
    -
-
-
-
-
-

Public Cloud

-
-
- -
-
-
-

Challenges

-
    -
  • - Getting this running in openshift and public cloud -
    • -
  • - Having a good understanding of the data, the AI algorithms, AI - workflows and performance compute is key -
    • -
-
-
-

Q&A

-

Questions?

-

- All of our presentation and diagrams can be found in our - - github repository. - -

-
-
-
- - - - - - - + .heading h1 span { + background: linear-gradient(60deg, #3e82ff, #295fff); + -webkit-background-clip: text; + background-clip: text; + color: transparent; + } + + + +
+
+
+
+

Chat with BC Laws

+
+

Technical presentation

+
+
+

What we will cover

+
    +
  • Introduction
    • +
  • High level architecture
    • +
  • Technologies used
    • +
  • Preprocessing (HPC)
    • +
  • MLOps
    • +
  • Analytics
    • +
  • Active Feedback
    • +
  • Challenges
    • +
  • Q&A
    • +
+
+
+ +
+
+ +
+
+

RAG Pipeline

+
    +
  • Feed query to sentence transformer
    • +
  • Search the vector in Neo4j (vector search)
    • +
  • Capture Top 10 results based on similarity
    • +
  • + Optional: Re-rank results with cross-encoder +
    • +
  • + Generate prompt (query stack + Top K + current query) +
    • +
  • Feed prompt to generative AI
    • +
  • Receive AI-generated response
    • +
  • Return response to user
    • +
+
+
+ +
+
+ +
+
+ +
+
+
+ +
+
+

How we index the acts and regulations

+ +
+
+

How data is stored in Neo4j

+ +
+
+

Graph Database and vector store

+ + + + + + + + + + + + + + + + + + + + +
Data storageNeo4j IntegrationAdvanced Querying
We use a graph database to store the data. + Neo4J is leveraged for both the graph database and the + vector store. + + Neo4J enables advanced queries, such as clustering the data + and finding communities. +
Vector store is utilized for storing embeddings. + These tasks are more efficient and easier to implement using + a graph database. +
+
+
+

HPC

+

High Performance Computing for pre-processing the data

+
+
+ +
+
+
+
+

MLOps & Analytics

+
    +
  • Frontend analytics data
    • +
  • Backend RAG Chain tracking
    • +
  • Apache Airflow for orchestration
    • +
  • dbt for data transformation
    • +
  • + Integration with active learning pipeline +
    • +
+
+
+ +
+
+ +
+
+ +
+
+ +
+
+
+
+

Active Learning Integration

+
    +
  • + Analytics data feeds into active learning pipeline +
    • +
  • + Helps identify areas for model improvement +
    • +
  • + Informs data selection for model fine-tuning +
    • +
  • + Enables continuous improvement of the AI system +
    • +
+
+
+ +
+
+ +
+
+ +
+
+ +
+
+

Human in the Loop

+

+ To improve the AI model we need to annotate and format the data + properly. After the data is annotated we can use it to train the + different models. +

+
+
+

Embedding Adaptors

+

+ If the top sources are not accurate we can retrain the embedding + model based on human feedback. +

+
+
+ +
+
+

Data Annotation (NER)

+

+ For improving our retrieval and enhancing our result we are using + an AI technique called NER (Named Entity Recognition) to annotate + the data. +

+

+ This can be done manually with tools such as Diffgram or Doccano + or can be automated using an AI model to pre-annotate. +

+
+
+

Doccano

+ +
+
+

Assisted Annotation Process

+
    +
  • + Need large amounts of training data. Initial results suggest + thousands of samples would be needed for reliable results. +
    • +
  • + Manually annotating this data takes people resources, but AI + annotation is less accurate. For 5000 records: +
    • +
      +
    • + Manually: 8-10 days with high accuracy. +
      • +
    • + Automated with generative AI: only hours but is not accurate + so far. +
      • +
    +
+
+
+
+
+

Public Cloud

+
+
+ +
+
+
+

Challenges

+
    +
  • + Getting this running in openshift and public cloud +
    • +
  • + Having a good understanding of the data, the AI algorithms, AI + workflows and performance compute is key +
    • +
+
+ +
+
+

Indexing Images

+

Making legal document images fully searchable

+
    +
  • Image Retrieval & Processing
    • +
  • AI-Based Image Summarization
    • +
  • Model Comparison: Open-Source vs Cloud
    • +
  • Vector Database Indexing
    • +
  • Semantic Search Capabilities
    • +
+
+ +
+

Image Retrieval & Processing

+
+
+

Retrieval Process

+
    +
  • Extraction from BC Laws website
    • +
  • + Hierarchical organization by document type +
    • +
  • + Custom path-based metadata extraction +
    • +
+
+
+

Processing Steps

+
    +
  • Format standardization (JPEG)
    • +
  • + Base64 encoding for API compatibility +
    • +
  • + Metadata preservation for traceability +
    • +
+
+
+
+ +
+

AI-Based Image Summarization

+

Standardized Prompt Structure

+
    +
  • Image type & category classification
    • +
  • Document context extraction
    • +
  • Content element identification
    • +
  • Structural component analysis
    • +
  • Technical specification documentation
    • +
+

+ Consistent prompt design ensures standardized extraction across + models +

+
+ +
+

Model Comparison: Infrastructure

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FeatureMOLMO 7BAmazon Nova ProClaude 3.5 Sonnet
DeploymentSelf-hosted (OpenShift)AWS BedrockAWS Bedrock
Parameters7 billionProprietaryProprietary (140B+)
Processing Time~24 hours batch2-3 sec/image2-4 sec/image
Input CostFree (self-hosted)$0.0008/1K tokens$0.003/1K tokens
Output CostFree (self-hosted)$0.0032/1K tokens$0.015/1K tokens
+
+ +
+

Model Comparison: Performance

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
AspectMOLMO 7BAmazon Nova ProClaude 3.5 Sonnet
Content AccuracyModerateGoodExcellent
Prompt FollowingInconsistentVariableHighly consistent
Output StructureOften deviatesSometimes deviatesFollows structure precisely
Legal DomainBasic understandingGood understandingStrong contextual grasp
Overall QualityAcceptableGoodSuperior
+
+ +
+

Why We Chose Claude 3.5 Sonnet

+
+
+

Key Decision Factors

+
    +
  • Superior prompt adherence
    • +
  • Consistent structured output
    • +
  • Better recognition of legal elements
    • +
  • Higher accuracy on technical content
    • +
+
+
+

Cost-Benefit Analysis

+
    +
  • Higher token cost offset by improved quality
    • +
  • Reduced need for manual corrections
    • +
  • Better downstream search performance
    • +
  • Substantially faster than self-hosted solution
    • +
+
+
+
+ +
+

Structured JSON Storage

+ 
+							{
+  "Acts": {
+    "Election Act": {
+      "96106_greatseal.gif": "Image Type: Official seal...",
+      // More images...
+    },
+    // More acts...
+  },
+  "Regulations": {
+    "Health Act": {
+      "diagram.png": "Image Type: Technical diagram...",
+      // More images...
+    }
+  }
+}
+
+
+
+

Structured JSON Storage Benefits

+
    +
  • Preserves document hierarchy
    • +
  • Maintains original context
    • +
  • Enables incremental updates
    • +
  • Simplifies downstream processing
    • +
+
+ +
+

Vector Database Indexing

+

Key Components

+
    +
  • + Text Chunking: 256 tokens with 20 token overlap +
    • +
  • + Embeddings: all-MiniLM-L6-v2 (384 dimensions) +
    • +
  • + Node Labels: ImageChunk, UpdatedChunksAndImagesv4 +
    • +
  • + Relationships: NEXT (sequential), PART_OF (document) +
    • +
  • + Metadata: Source path, document type, file references +
    • +
+
+ +
+

Semantic Search Capabilities

+
+
+

Search Features

+
    +
  • Vector similarity search
    • +
  • Cross-encoder reranking
    • +
  • Context retrieval via NEXT
    • +
  • Document relation traversal
    • +
+
+
+

Two-Stage Search Process

+
    +
  1. Initial vector similarity
    2. +
  2. Precision reranking
    3. +
  3. Context expansion
    4. +
  4. Result enrichment
    5. +
+
+
+
+ +
+

Lessons Learned

+
    +
  • + Larger models dramatically improve legal content extraction + quality +
    • +
  • + AWS Bedrock enables rapid iteration and production scaling +
    • +
  • + Standardized prompts are critical for consistent results +
    • +
  • + Costs for Claude 3.5 are justified by reduction in + post-processing +
    • +
  • + Chunking with relationships preserves critical context +
    • +
  • + Two-stage search delivers superior relevance +
    • +
+
+ +
+

Future Directions

+
    +
  • + Evaluate additional specialized models for niche content +
    • +
  • + Implement automated image change detection +
    • +
  • + Optimize chunking based on content characteristics +
    • +
  • + Enhance handling of complex tables and forms +
    • +
  • + Explore multimodal retrieval augmentation +
    • +
+
+
+
+

Q&A

+

Questions?

+

+ All of our presentation and diagrams can be found in our + + github repository. + +

+
+
+
+ + + + + + From ea2dc77785351655c4129c008d3c92321616eef9 Mon Sep 17 00:00:00 2001 From: Noor Chasib Date: Wed, 19 Mar 2025 16:01:33 -0700 Subject: [PATCH 3/3] Update index.html --- .../presentation/index.html | 92 +++++++++---------- 1 file changed, 42 insertions(+), 50 deletions(-) diff --git a/web/frontend-feedback-analytics/presentation/index.html b/web/frontend-feedback-analytics/presentation/index.html index 519407781..5c4ddd7a9 100644 --- a/web/frontend-feedback-analytics/presentation/index.html +++ b/web/frontend-feedback-analytics/presentation/index.html @@ -356,7 +356,8 @@

Challenges

- + +

Indexing Images

@@ -366,9 +367,9 @@

Indexing Images

  • AI-Based Image Summarization
  • Model Comparison: Open-Source vs Cloud
  • Vector Database Indexing
    • -
  • Semantic Search Capabilities
    • +

    Image Retrieval & Processing

    @@ -388,17 +389,16 @@

    Retrieval Process

    Processing Steps

      -
    • Format standardization (JPEG)
    • Base64 encoding for API compatibility
      • -
    • - Metadata preservation for traceability -
      • +
    • Legal document context preservation
      • +
    • Metadata tracking for traceability
    +

    AI-Based Image Summarization

    @@ -415,6 +415,7 @@

    Standardized Prompt Structure

    models

    +

    Model Comparison: Infrastructure

    @@ -436,15 +437,15 @@

    Model Comparison: Infrastructure

    Parameters - 7 billion + 7 Billion Proprietary - Proprietary (140B+) + Proprietary (175B+) Processing Time - ~24 hours batch - 2-3 sec/image - 2-4 sec/image + 15-20 sec/image
    (~ 24 hours) + 2-3 sec/image
    (~ 3-3.5 hours) + 2-4 sec/image
    (~ 3-3.5 hours) Input Cost @@ -461,6 +462,7 @@

    Model Comparison: Infrastructure

    +

    Model Comparison: Performance

    @@ -507,6 +509,7 @@

    Model Comparison: Performance

    +

    Why We Chose Claude 3.5 Sonnet

    @@ -514,28 +517,32 @@

    Why We Chose Claude 3.5 Sonnet

    Key Decision Factors

      -
    • Superior prompt adherence
      • -
    • Consistent structured output
      • -
    • Better recognition of legal elements
      • -
    • Higher accuracy on technical content
      • +
    • Superior prompt adherence
      • +
    • Consistent structured output
      • +
    • Better recognition of legal elements
      • +
    • Higher accuracy on technical content

    Cost-Benefit Analysis

      -
    • Higher token cost offset by improved quality
      • -
    • Reduced need for manual corrections
      • -
    • Better downstream search performance
      • -
    • Substantially faster than self-hosted solution
      • +
    • + Higher token cost offset by improved quality +
      • +
    • Reduced need for manual corrections
      • +
    • Better downstream search performance
      • +
    • + Substantially faster than OpenShift solution +
    +

    Structured JSON Storage

    - 
    -							{
+						
    {
   "Acts": {
     "Election Act": {
       "96106_greatseal.gif": "Image Type: Official seal...",
@@ -549,8 +556,7 @@ 
    Structured JSON Storage
    
       // More images...
     }
   }
-}    
-						
    
+}

    Structured JSON Storage Benefits

    @@ -561,6 +567,7 @@

    Structured JSON Storage Benefits

  • Simplifies downstream processing
    • +

    Vector Database Indexing

    @@ -583,30 +590,7 @@

    Key Components

    - -
    -

    Semantic Search Capabilities

    -
    -
    -

    Search Features

    -
      -
    • Vector similarity search
      • -
    • Cross-encoder reranking
      • -
    • Context retrieval via NEXT
      • -
    • Document relation traversal
      • -
    -
    -
    -

    Two-Stage Search Process

    -
      -
    1. Initial vector similarity
      2. -
    2. Precision reranking
      3. -
    3. Context expansion
      4. -
    4. Result enrichment
      5. -
    -
    -
    -
    +

    Lessons Learned

    @@ -621,18 +605,25 @@

    Lessons Learned

  • Standardized prompts are critical for consistent results
    • + +
    + +
    +

    Lessons Learned Cont.

    +
    • - Costs for Claude 3.5 are justified by reduction in + Costs for Claude 3.5 Sonnet are justified by reduction in post-processing
    • Chunking with relationships preserves critical context
    • - Two-stage search delivers superior relevance + Integration pipeline enables comprehensive legal image search
    +

    Future Directions

    @@ -650,11 +641,12 @@

    Future Directions

    Enhance handling of complex tables and forms
  • - Explore multimodal retrieval augmentation + Explore multilingual legal document support
    • +

    Q&A

    Questions?