update readme and process data

docs/AzureAccountSetUp.md

@@ -0,0 +1,14 @@
+## Azure account setup
+
+1. Sign up for a [free Azure account](https://azure.microsoft.com/free/) and create an Azure Subscription.
+2. Check that you have the necessary permissions:
+    * Your Azure account must have `Microsoft.Authorization/roleAssignments/write` permissions, such as [Role Based Access Control Administrator](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#role-based-access-control-administrator-preview), [User Access Administrator](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator), or [Owner](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#owner).
+    * Your Azure account also needs `Microsoft.Resources/deployments/write` permissions on the subscription level.
+
+You can view the permissions for your account and subscription by following the steps below: 
+- Navigate to the [Azure Portal](https://portal.azure.com/) and click on `Subscriptions` under 'Navigation' 
+- Select the subscription you are using for this accelerator from the list. 
+    - If you try to search for your subscription and it does not come up, make sure no filters are selected.
+- Select `Access control (IAM)` and you can see the roles that are assigned to your account for this subscription. 
+    - If you want to see more information about the roles, you can go to the `Role assignments`
+     tab and search by your account name and then click the role you want to view more information about.

docs/AzureSemanticSearchRegion.md

@@ -0,0 +1,7 @@
+## Select a region where Semantic Search Availability is available before proceeding with the deployment.
+
+Steps to Check Semantic Search Availability
+1. Open the [Semantic Search Availability](https://learn.microsoft.com/en-us/azure/search/search-region-support) page.
+2. Scroll down to the **"Availability by Region"** section.
+3. Use the table to find supported regions for **Azure AI Search** and its **Semantic Search** feature.
+4. If your target region is not listed, choose a supported region for deployment.

docs/CustomizingAzdParameters.md

@@ -0,0 +1,43 @@
+## [Optional]: Customizing resource names 
+
+By default this template will use the environment name as the prefix to prevent naming collisions within Azure. The parameters below show the default values. You only need to run the statements below if you need to change the values. 
+
+
+> To override any of the parameters, run `azd env set <key> <value>` before running `azd up`. On the first azd command, it will prompt you for the environment name. Be sure to choose 3-20 charaters alphanumeric unique name. 
+
+
+Change the Secondary Location (example: eastus2, westus2, etc.)
+
+```shell
+azd env set AZURE_ENV_SECONDARY_LOCATION eastus2
+```
+
+Change the Model Deployment Type (allowed values: Standard, GlobalStandard)
+
+```shell
+azd env set AZURE_ENV_MODEL_DEPLOYMENT_TYPE GlobalStandard
+```
+
+Set the Model Name (allowed values: gpt-4o, gpt-4o, gpt-4)
+
+```shell
+azd env set AZURE_ENV_MODEL_NAME gpt-4o
+```
+
+Change the Model Capacity (choose a number based on available GPT model capacity in your subscription)
+
+```shell
+azd env set AZURE_ENV_MODEL_CAPACITY 30
+```
+
+Change the Embedding Model 
+
+```shell
+azd env set AZURE_ENV_EMBEDDING_MODEL_NAME text-embedding-ada-002
+```
+
+Change the Embedding Deployment Capacity (choose a number based on available embedding model capacity in your subscription)
+
+```shell
+azd env set AZURE_ENV_EMBEDDING_MODEL_CAPACITY 80
+```

docs/DeleteResourceGroup.md

@@ -0,0 +1,53 @@
+# Deleting Resources After a Failed Deployment in Azure Portal
+
+If your deployment fails and you need to clean up the resources manually, follow these steps in the Azure Portal.
+
+---
+
+## **1. Navigate to the Azure Portal**
+1. Open [Azure Portal](https://portal.azure.com/).
+2. Sign in with your Azure account.
+
+---
+
+## **2. Find the Resource Group**
+1. In the search bar at the top, type **"Resource groups"** and select it.
+2. Locate the **resource group** associated with the failed deployment.
+
+![Resource Groups](Images/resourcegroup.png)
+
+![Resource Groups](Images/resource-groups.png)
+
+---
+
+## **3. Delete the Resource Group**
+1. Click on the **resource group name** to open it.
+2. Click the **Delete resource group** button at the top.
+
+![Delete Resource Group](Images/DeleteRG.png)
+
+3. Type the resource group name in the confirmation box and click **Delete**.
+
+📌 **Note:** Deleting a resource group will remove all resources inside it.
+
+---
+
+## **4. Delete Individual Resources (If Needed)**
+If you don’t want to delete the entire resource group, follow these steps:
+
+1. Open **Azure Portal** and go to the **Resource groups** section.
+2. Click on the specific **resource group**.
+3. Select the **resource** you want to delete (e.g., App Service, Storage Account).
+4. Click **Delete** at the top.
+
+![Delete Individual Resource](Images/deleteservices.png)
+
+---
+
+## **5. Verify Deletion**
+- After a few minutes, refresh the **Resource groups** page.
+- Ensure the deleted resource or group no longer appears.
+
+📌 **Tip:** If a resource fails to delete, check if it's **locked** under the **Locks** section and remove the lock.
+
+

infra/scripts/index_scripts/01_create_search_index.py

@@ -0,0 +1,97 @@
+from azure.keyvault.secrets import SecretClient  
+from azure.identity import DefaultAzureCredential
+
+key_vault_name = 'kv_to-be-replaced'
+index_name = "pdf_index"
+
+def get_secrets_from_kv(kv_name, secret_name):
+
+    # Set the name of the Azure Key Vault  
+    key_vault_name = kv_name 
+    credential = DefaultAzureCredential()
+
+    # Create a secret client object using the credential and Key Vault name  
+    secret_client =  SecretClient(vault_url=f"https://{key_vault_name}.vault.azure.net/", credential=credential)  
+
+    # Retrieve the secret value  
+    return(secret_client.get_secret(secret_name).value)
+
+search_endpoint = get_secrets_from_kv(key_vault_name,"AZURE-SEARCH-ENDPOINT")
+search_key =  get_secrets_from_kv(key_vault_name,"AZURE-SEARCH-KEY")
+
+# Create the search index
+def create_search_index():
+    from azure.core.credentials import AzureKeyCredential 
+    search_credential = AzureKeyCredential(search_key)
+
+    from azure.search.documents.indexes import SearchIndexClient
+    from azure.search.documents.indexes.models import (
+        SimpleField,
+        SearchFieldDataType,
+        SearchableField,
+        SearchField,
+        VectorSearch,
+        HnswAlgorithmConfiguration,
+        VectorSearchProfile,
+        SemanticConfiguration,
+        SemanticPrioritizedFields,
+        SemanticField,
+        SemanticSearch,
+        SearchIndex
+    )
+
+    # Create a search index 
+    index_client = SearchIndexClient(endpoint=search_endpoint, credential=search_credential)
+
+    # fields = [
+    #     SimpleField(name="id", type=SearchFieldDataType.String, key=True, sortable=True, filterable=True, facetable=True),
+    #     SearchableField(name="chunk_id", type=SearchFieldDataType.String),
+    #     SearchableField(name="content", type=SearchFieldDataType.String),
+    #     SearchableField(name="sourceurl", type=SearchFieldDataType.String),
+    #     SearchField(name="contentVector", type=SearchFieldDataType.Collection(SearchFieldDataType.Single),
+    #                 searchable=True, vector_search_dimensions=1536, vector_search_profile_name="myHnswProfile"),
+    # ]
+
+    fields = [
+        SimpleField(name="id", type=SearchFieldDataType.String, key=True),
+        SimpleField(name="chunk_id", type=SearchFieldDataType.String),
+        SearchField(name="content", type=SearchFieldDataType.String),
+        SimpleField(name="sourceurl", type=SearchFieldDataType.String),
+        SearchField(name="contentVector", type=SearchFieldDataType.Collection(SearchFieldDataType.Single), \
+        vector_search_dimensions=1536,vector_search_profile_name="myHnswProfile"
+        )
+    ]
+
+    # Configure the vector search configuration 
+    vector_search = VectorSearch(
+        algorithms=[
+            HnswAlgorithmConfiguration(
+                name="myHnsw"
+            )
+        ],
+        profiles=[
+            VectorSearchProfile(
+                name="myHnswProfile",
+                algorithm_configuration_name="myHnsw",
+            )
+        ]
+    )
+
+    semantic_config = SemanticConfiguration(
+        name="my-semantic-config",
+        prioritized_fields=SemanticPrioritizedFields(
+            keywords_fields=[SemanticField(field_name="chunk_id")],
+            content_fields=[SemanticField(field_name="content")]
+        )
+    )
+
+    # Create the semantic settings with the configuration
+    semantic_search = SemanticSearch(configurations=[semantic_config])
+
+    # Create the search index with the semantic settings
+    index = SearchIndex(name=index_name, fields=fields,
+                        vector_search=vector_search, semantic_search=semantic_search)
+    result = index_client.create_or_update_index(index)
+    print(f' {result.name} created')
+
+create_search_index()