Add more documentation

Author: BenConstable9

text_2_sql/README.md

@@ -39,7 +39,7 @@ Three different iterations are presented and code provided for:
  - **Iteration 2:** Injection of a brief description of the available entities is injected into the prompt. This limits the number of tokens used and avoids filling the prompt with confusing schema information.
  - **Iteration 3:** Indexing the entity definitions in a vector database, such as AI Search, and querying it to retrieve the most relevant entities for the key terms from the query.
   - **Iteration 4:** Keeping an index of commonly asked questions and which schema / SQL query they resolve to - this index is generated by the LLM when it encounters a question that has not been previously asked. Additionally, indexing the entity definitions in a vector database, such as AI Search _(same as Iteration 3)_. First querying this index to see if a similar SQL query can be obtained _(if high probability of exact SQL query match, the results can be pre-fetched)_. If not, falling back to the schema index, and querying it to retrieve the most relevant entities for the key terms from the query.
-  - **Iteration 5:** Moves the Iteration 4 approach into a multi-agent approach for improved reasoning and query generation. With separation into agents, different agents can focus on one task only, and provide a better overall flow and response quality.
+  - **Iteration 5:** Moves the Iteration 4 approach into a multi-agent approach for improved reasoning and query generation. With separation into agents, different agents can focus on one task only, and provide a better overall flow and response quality. See more details below.
 
 All approaches limit the number of tokens used and avoids filling the prompt with confusing schema information.
 

text_2_sql/autogen/README.md

@@ -17,3 +17,62 @@ As the query cache is shared between users (no data is stored in the cache), a n
 ![Vector Based with Query Cache Logical Flow.](../images/Agentic%20Text2SQL%20Query%20Cache.png "Agentic Vector Based with Query Cache Logical Flow")
 
 ## Provided Notebooks & Scripts
+
+- `./agentic_text_2_sql.ipynb` provides example of how to utilise the Agentic Vector Based Text2SQL approach to query the database. The query cache plugin will be enabled or disabled depending on the environmental parameters.
+
+## Agents
+
+This approach builds on the the Vector Based SQL Plugin approach, but adds a agentic approach to the solution.
+
+This agentic system contains the following agents:
+
+- **Query Cache Agent:** Responsible for checking the cache for previously asked questions.
+- **Query Decomposition Agent:** Responsible for decomposing complex questions, into sub questions that can be answered with SQL.
+- **Schema Selection Agent:** Responsible for extracting key terms from the question and checking the index store for the queries.
+- **SQL Query Generation Agent:** Responsible for using the previously extracted schemas and generated SQL queries to answer the question. This agent can request more schemas if needed. This agent will run the query.
+- **SQL Query Verification Agent:** Responsible for verifying that the SQL query and results question will answer the question.
+- **Answer Generation Agent:** Responsible for taking the database results and generating the final answer for the user.
+
+The combination of this agent allows the system to answer complex questions, whilst staying under the token limits when including the database schemas. The query cache ensures that previously asked questions, can be answered quickly to avoid degrading user experience.
+
+All agents can be found in `/agents/`.
+
+## agentic_text_2_sql.py
+
+This is the main entry point for the agentic system. In here, the `Selector Group Chat` is configured with the termination conditions to orchestrate the agents within the system.
+
+A customer transition selector is used to automatically transition between agents dependent on the last one that was used. In some cases, this choice is delegated to an LLM to decide on the most appropriate action. This mixed approach allows for speed when needed (e.g. always calling Query Cache Agent first), but will allow the system to react dynamically to the events.
+
+## Utils
+
+### ai-search.py
+
+This util file contains helper functions for interacting with AI Search.
+
+### llm_agent_creator.py
+
+This util file creates the agents in the AutoGen framework based on the configuration files.
+
+### models.py
+
+This util file creates the model connections to Azure OpenAI for the agents.
+
+### sql.py
+
+#### get_entity_schema()
+
+This method is called by the AutoGen framework automatically, when instructed to do so by the LLM, to search the AI Search instance with the given text. The LLM is able to pass the key terms from the user query, and retrieve a ranked list of the most suitable entities to answer the question.
+
+The search text passed is vectorised against the entity level **Description** columns. A hybrid Semantic Reranking search is applied against the **EntityName**, **Entity**, **Columns/Name** fields.
+
+#### fetch_queries_from_cache()
+
+The vector based with query cache uses the `fetch_queries_from_cache()` method to fetch the most relevant previous query and injects it into the prompt before the initial LLM call. The use of Auto-Function Calling here is avoided to reduce the response time as the cache index will always be used first.
+
+If the score of the top result is higher than the defined threshold, the query will be executed against the target data source and the results included in the prompt. This allows us to prompt the LLM to evaluated whether it can use these results to answer the question, **without further SQL Query generation** to speed up the process.
+
+#### run_sql_query()
+
+This method is called by the AutoGen framework automatically, when instructed to do so by the LLM, to run a SQL query against the given database. It returns a JSON string containing a row wise dump of the results returned. These results are then interpreted to answer the question.
+
+Additionally, if any of the cache functionality is enabled, this method will update the query cache index based on the SQL query run, and the schemas used in execution.

text_2_sql/autogen/agents/__init__.py

@@ -6,7 +6,7 @@
 from autogen_agentchat.base import Response
 from autogen_agentchat.messages import AgentMessage, ChatMessage, TextMessage
 from autogen_core.base import CancellationToken
-from utils.sql_utils import fetch_queries_from_cache
+from utils.sql import fetch_queries_from_cache
 import json
 import logging
 

text_2_sql/autogen/custom_agents/sql_query_cache_agent.py

@@ -3,7 +3,7 @@
 import yaml
 from autogen_core.components.tools import FunctionTool
 from autogen_agentchat.agents import AssistantAgent
-from utils.sql_utils import (
+from utils.sql import (
     query_execution,
     get_entity_schemas,
 )

…t_2_sql/autogen/utils/ai_search_utils.py text_2_sql/autogen/utils/ai_search.pytext_2_sql/autogen/utils/ai_search_utils.py renamed to text_2_sql/autogen/utils/ai_search.py text_2_sql/autogen/utils/ai_search_utils.py renamed to text_2_sql/autogen/utils/ai_search.py

@@ -4,7 +4,7 @@
 import os
 import aioodbc
 from typing import Annotated
-from utils.ai_search_utils import run_ai_search_query
+from text_2_sql.autogen.utils.ai_search import run_ai_search_query
 import json
 import asyncio
 

text_2_sql/autogen/utils/llm_agent_creator.py

@@ -22,6 +22,10 @@ As the query cache is shared between users (no data is stored in the cache), a n
     - This setup is useful for a production application as the SQL Database is unlikely to be able to answer all the questions a user may ask.
 - `./time_comparison_script.py` provides a utility script for performing time based comparisons between the different approaches.
 
+### ai-search.py
+
+This util file contains helper functions for interacting with AI Search.
+
 ## Plugins
 
 ### prompt_based_sql_plugin.py
@@ -48,7 +52,6 @@ This method is called by the Semantic Kernel framework automatically, when instr
 
 The `./plugins/vector_based_sql_plugin/vector_based_sql_plugin.py` contains 3 key methods to power the Vector Based Text2SQL engine.
 
-
 #### system_prompt()
 
 This method simply returns a pre-made system prompt that contains optimised and working instructions for the LLM. This system prompt for the plugin is added to the main prompt file at runtime.

text_2_sql/autogen/utils/sql_utils.py text_2_sql/autogen/utils/sql.pytext_2_sql/autogen/utils/sql_utils.py renamed to text_2_sql/autogen/utils/sql.py

@@ -5,7 +5,7 @@
 import os
 import json
 import logging
-from utils.ai_search_utils import (
+from utils.ai_search import (
     add_entry_to_index,
     run_ai_search_query,
 )