diff --git a/web/frontend-feedback-analytics/presentation/css/layout.scss b/web/frontend-feedback-analytics/presentation/css/layout.scss index 5d0536f94..6853f737c 100644 --- a/web/frontend-feedback-analytics/presentation/css/layout.scss +++ b/web/frontend-feedback-analytics/presentation/css/layout.scss @@ -68,3 +68,25 @@ .reveal .justify-start { justify-content: flex-start; } .reveal .justify-center { justify-content: center; } .reveal .justify-end { justify-content: flex-end; } + + +.reveal table { + font-size: 0.6em; + width: 100%; +} +.reveal th { + background-color: #4CAF50; + color: white; +} +.reveal tr:nth-child(even) { + background-color: #f2f2f2; +} + +.reveal .row { + display: flex; + width: 100%; +} +.reveal .col { + flex: 1; + padding: 0 10px; +} diff --git a/web/frontend-feedback-analytics/presentation/index.html b/web/frontend-feedback-analytics/presentation/index.html index e24fa8af4..5c4ddd7a9 100644 --- a/web/frontend-feedback-analytics/presentation/index.html +++ b/web/frontend-feedback-analytics/presentation/index.html @@ -1,389 +1,679 @@ - - - + + + + AI Technical Presentation + + + + + + - - -
-
-
-
-

Chat with BC Laws

-
-

Technical presentation

-
-
-

What we will cover

-
    -
  • Introduction
    • -
  • High level architecture
    • -
  • Technologies used
    • -
  • Preprocessing (HPC)
    • -
  • MLOps
    • -
  • Analytics
    • -
  • Active Feedback
    • -
  • Challenges
    • -
  • Q&A
    • -
-
-
- -
-
- -
-
-

RAG Pipeline

-
    -
  • Feed query to sentence transformer
    • -
  • Search the vector in Neo4j (vector search)
    • -
  • Capture Top 10 results based on similarity
    • -
  • - Optional: Re-rank results with cross-encoder -
    • -
  • - Generate prompt (query stack + Top K + current query) -
    • -
  • Feed prompt to generative AI
    • -
  • Receive AI-generated response
    • -
  • Return response to user
    • -
-
-
- -
-
- -
-
- -
-
-
- -
-
-

How we index the acts and regulations

- -
-
-

How data is stored in Neo4j

- -
-
-

Graph Database and vector store

- - - - - - - - - - - - - - - - - - - - -
Data storageNeo4j IntegrationAdvanced Querying
We use a graph database to store the data. - Neo4J is leveraged for both the graph database and the - vector store. - - Neo4J enables advanced queries, such as clustering the data - and finding communities. -
Vector store is utilized for storing embeddings. - These tasks are more efficient and easier to implement using - a graph database. -
-
-
-

HPC

-

High Performance Computing for pre-processing the data

-
-
- -
-
-
-
-

MLOps & Analytics

-
    -
  • Frontend analytics data
    • -
  • Backend RAG Chain tracking
    • -
  • Apache Airflow for orchestration
    • -
  • dbt for data transformation
    • -
  • - Integration with active learning pipeline -
    • -
-
-
- -
-
- -
-
- -
-
- -
-
-
-
-

Active Learning Integration

-
    -
  • - Analytics data feeds into active learning pipeline -
    • -
  • - Helps identify areas for model improvement -
    • -
  • - Informs data selection for model fine-tuning -
    • -
  • - Enables continuous improvement of the AI system -
    • -
-
-
- -
-
- -
-
- -
-
- -
-
-

Human in the Loop

-

- To improve the AI model we need to annotate and format the data - properly. After the data is annotated we can use it to train the - different models. -

-
-
-

Embedding Adaptors

-

- If the top sources are not accurate we can retrain the embedding - model based on human feedback. -

-
-
- -
-
-

Data Annotation (NER)

-

- For improving our retrieval and enhancing our result we are using - an AI technique called NER (Named Entity Recognition) to annotate - the data. -

-

- This can be done manually with tools such as Diffgram or Doccano - or can be automated using an AI model to pre-annotate. -

-
-
-

Doccano

- -
-
-

Assisted Annotation Process

-
    -
  • - Need large amounts of training data. Initial results suggest - thousands of samples would be needed for reliable results. -
    • -
  • - Manually annotating this data takes people resources, but AI - annotation is less accurate. For 5000 records: -
    • -
      -
    • - Manually: 8-10 days with high accuracy. -
      • -
    • - Automated with generative AI: only hours but is not accurate - so far. -
      • -
    -
-
-
-
-
-

Public Cloud

-
-
- -
-
-
-

Challenges

-
    -
  • - Getting this running in openshift and public cloud -
    • -
  • - Having a good understanding of the data, the AI algorithms, AI - workflows and performance compute is key -
    • -
-
-
-

Q&A

-

Questions?

-

- All of our presentation and diagrams can be found in our - - github repository. - -

-
-
-
+ .heading h1 span { + background: linear-gradient(60deg, #3e82ff, #295fff); + -webkit-background-clip: text; + background-clip: text; + color: transparent; + } + + + +
+
+
+
+

Chat with BC Laws

+
+

Technical presentation

+
+
+

What we will cover

+
    +
  • Introduction
    • +
  • High level architecture
    • +
  • Technologies used
    • +
  • Preprocessing (HPC)
    • +
  • MLOps
    • +
  • Analytics
    • +
  • Active Feedback
    • +
  • Challenges
    • +
  • Q&A
    • +
+
+
+ +
+
+ +
+
+

RAG Pipeline

+
    +
  • Feed query to sentence transformer
    • +
  • Search the vector in Neo4j (vector search)
    • +
  • Capture Top 10 results based on similarity
    • +
  • + Optional: Re-rank results with cross-encoder +
    • +
  • + Generate prompt (query stack + Top K + current query) +
    • +
  • Feed prompt to generative AI
    • +
  • Receive AI-generated response
    • +
  • Return response to user
    • +
+
+
+ +
+
+ +
+
+ +
+
+
+ +
+
+

How we index the acts and regulations

+ +
+
+

How data is stored in Neo4j

+ +
+
+

Graph Database and vector store

+ + + + + + + + + + + + + + + + + + + + +
Data storageNeo4j IntegrationAdvanced Querying
We use a graph database to store the data. + Neo4J is leveraged for both the graph database and the + vector store. + + Neo4J enables advanced queries, such as clustering the data + and finding communities. +
Vector store is utilized for storing embeddings. + These tasks are more efficient and easier to implement using + a graph database. +
+
+
+

HPC

+

High Performance Computing for pre-processing the data

+
+
+ +
+
+
+
+

MLOps & Analytics

+
    +
  • Frontend analytics data
    • +
  • Backend RAG Chain tracking
    • +
  • Apache Airflow for orchestration
    • +
  • dbt for data transformation
    • +
  • + Integration with active learning pipeline +
    • +
+
+
+ +
+
+ +
+
+ +
+
+ +
+
+
+
+

Active Learning Integration

+
    +
  • + Analytics data feeds into active learning pipeline +
    • +
  • + Helps identify areas for model improvement +
    • +
  • + Informs data selection for model fine-tuning +
    • +
  • + Enables continuous improvement of the AI system +
    • +
+
+
+ +
+
+ +
+
+ +
+
+ +
+
+

Human in the Loop

+

+ To improve the AI model we need to annotate and format the data + properly. After the data is annotated we can use it to train the + different models. +

+
+
+

Embedding Adaptors

+

+ If the top sources are not accurate we can retrain the embedding + model based on human feedback. +

+
+
+ +
+
+

Data Annotation (NER)

+

+ For improving our retrieval and enhancing our result we are using + an AI technique called NER (Named Entity Recognition) to annotate + the data. +

+

+ This can be done manually with tools such as Diffgram or Doccano + or can be automated using an AI model to pre-annotate. +

+
+
+

Doccano

+ +
+
+

Assisted Annotation Process

+
    +
  • + Need large amounts of training data. Initial results suggest + thousands of samples would be needed for reliable results. +
    • +
  • + Manually annotating this data takes people resources, but AI + annotation is less accurate. For 5000 records: +
    • +
      +
    • + Manually: 8-10 days with high accuracy. +
      • +
    • + Automated with generative AI: only hours but is not accurate + so far. +
      • +
    +
+
+
+
+
+

Public Cloud

+
+
+ +
+
+
+

Challenges

+
    +
  • + Getting this running in openshift and public cloud +
    • +
  • + Having a good understanding of the data, the AI algorithms, AI + workflows and performance compute is key +
    • +
+
- - - - - - + +
+

Image Retrieval & Processing

+
+
+

Retrieval Process

+
    +
  • Extraction from BC Laws website
    • +
  • + Hierarchical organization by document type +
    • +
  • + Custom path-based metadata extraction +
    • +
+
+
+

Processing Steps

+
    +
  • + Base64 encoding for API compatibility +
    • +
  • Legal document context preservation
    • +
  • Metadata tracking for traceability
    • +
+
+
+
+ + +
+

AI-Based Image Summarization

+

Standardized Prompt Structure

+
    +
  • Image type & category classification
    • +
  • Document context extraction
    • +
  • Content element identification
    • +
  • Structural component analysis
    • +
  • Technical specification documentation
    • +
+

+ Consistent prompt design ensures standardized extraction across + models +

+
+ + +
+

Model Comparison: Infrastructure

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FeatureMOLMO 7BAmazon Nova ProClaude 3.5 Sonnet
DeploymentSelf-hosted (OpenShift)AWS BedrockAWS Bedrock
Parameters7 BillionProprietaryProprietary (175B+)
Processing Time15-20 sec/image
(~ 24 hours)		2-3 sec/image
(~ 3-3.5 hours)		2-4 sec/image
(~ 3-3.5 hours)
Input CostFree (self-hosted)$0.0008/1K tokens$0.003/1K tokens
Output CostFree (self-hosted)$0.0032/1K tokens$0.015/1K tokens
+
+ + +
+

Model Comparison: Performance

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
AspectMOLMO 7BAmazon Nova ProClaude 3.5 Sonnet
Content AccuracyModerateGoodExcellent
Prompt FollowingInconsistentVariableHighly consistent
Output StructureOften deviatesSometimes deviatesFollows structure precisely
Legal DomainBasic understandingGood understandingStrong contextual grasp
Overall QualityAcceptableGoodSuperior
+
+ + +
+

Why We Chose Claude 3.5 Sonnet

+
+
+

Key Decision Factors

+
    +
  • Superior prompt adherence
    • +
  • Consistent structured output
    • +
  • Better recognition of legal elements
    • +
  • Higher accuracy on technical content
    • +
+
+
+

Cost-Benefit Analysis

+
    +
  • + Higher token cost offset by improved quality +
    • +
  • Reduced need for manual corrections
    • +
  • Better downstream search performance
    • +
  • + Substantially faster than OpenShift solution +
    • +
+
+
+
+ + +
+

Structured JSON Storage

+ 
{
+  "Acts": {
+    "Election Act": {
+      "96106_greatseal.gif": "Image Type: Official seal...",
+      // More images...
+    },
+    // More acts...
+  },
+  "Regulations": {
+    "Health Act": {
+      "diagram.png": "Image Type: Technical diagram...",
+      // More images...
+    }
+  }
+}
+
+
+

Structured JSON Storage Benefits

+
    +
  • Preserves document hierarchy
    • +
  • Maintains original context
    • +
  • Enables incremental updates
    • +
  • Simplifies downstream processing
    • +
+
+ + +
+

Vector Database Indexing

+

Key Components

+
    +
  • + Text Chunking: 256 tokens with 20 token overlap +
    • +
  • + Embeddings: all-MiniLM-L6-v2 (384 dimensions) +
    • +
  • + Node Labels: ImageChunk, UpdatedChunksAndImagesv4 +
    • +
  • + Relationships: NEXT (sequential), PART_OF (document) +
    • +
  • + Metadata: Source path, document type, file references +
    • +
+
+ + +
+

Lessons Learned

+
    +
  • + Larger models dramatically improve legal content extraction + quality +
    • +
  • + AWS Bedrock enables rapid iteration and production scaling +
    • +
  • + Standardized prompts are critical for consistent results +
    • +
+
+ +
+

Lessons Learned Cont.

+
    +
  • + Costs for Claude 3.5 Sonnet are justified by reduction in + post-processing +
    • +
  • + Chunking with relationships preserves critical context +
    • +
  • + Integration pipeline enables comprehensive legal image search +
    • +
+
+ + +
+

Future Directions

+
    +
  • + Evaluate additional specialized models for niche content +
    • +
  • + Implement automated image change detection +
    • +
  • + Optimize chunking based on content characteristics +
    • +
  • + Enhance handling of complex tables and forms +
    • +
  • + Explore multilingual legal document support +
    • +
+
+ + +
+

Q&A

+

Questions?

+

+ All of our presentation and diagrams can be found in our + + github repository. + +

+
+
+
+ + + + + +