Increase embedding TPM capacity and add note in cloud ingestion guide (#2846)

* Increase embedding TPM capacity and add note in cloud ingestion guide

* Update markdown lint

* Better config format

* Better config format

* Fix the config

* Ignore table style rule

Author: pamelafox

.github/workflows/lint-markdown.yml

@@ -14,9 +14,11 @@ jobs:
     steps:
       - name: Checkout repo
         uses: actions/checkout@v6
-      - name: Run markdownlint
-        uses: articulate/actions-markdownlint@v1
+      - name: Run markdownlint-cli2
+        uses: DavidAnson/markdownlint-cli2-action@v21
         with:
-          config: .github/workflows/markdownlint-config.json
-          files: '**/*.md'
-          ignore: data/
+          config: .markdownlint-cli2.jsonc
+          globs: |
+            **/*.md
+            !data/**
+            !.github/**

.github/workflows/markdownlint-config.json

@@ -0,0 +1,8 @@
+{
+    "config": {
+        "default": true,
+        "line-length": false,
+        "table-column-style": false,
+        "MD033": { "allowed_elements": ["br", "details", "summary"] }
+    }
+}

.markdownlint-cli2.jsonc

@@ -1,6 +1,5 @@
 <!-- BEGIN MICROSOFT SECURITY.MD V0.0.5 BLOCK -->
-
-## Security
+# Security
 
 Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/).
 

SECURITY.md

@@ -153,15 +153,21 @@ This project includes an optional feature to perform data ingestion in the cloud
 
 3. Open `azure.yaml` and un-comment the document-extractor, figure-processor, and text-processor sections. Those are the Azure Functions apps that will be deployed and serve as Azure AI Search skills.
 
-4. Provision the new Azure Functions resources, deploy the function apps, and update the search indexer with:
+4. (Recommended) Increase the capacity for the embedding model to the maximum quota allowed for your region/subscription, so that the Azure Functions can generate embeddings without hitting rate limits:
+
+    ```shell
+    azd env set AZURE_OPENAI_EMB_DEPLOYMENT_CAPACITY 400
+    ```
+
+5. Provision the new Azure Functions resources, deploy the function apps, and update the search indexer with:
 
     ```shell
     azd up
     ```
 
-5. That will upload the documents in the `data/` folder to the Blob storage container, create the indexer and skillset, and run the indexer to ingest the data. You can monitor the indexer status from the portal.
+6. That will upload the documents in the `data/` folder to the Blob storage container, create the indexer and skillset, and run the indexer to ingest the data. You can monitor the indexer status from the portal.
 
-6. When you have new documents to ingest, you can upload documents to the Blob storage container and run the indexer from the Azure Portal to ingest new documents.
+7. When you have new documents to ingest, you can upload documents to the Blob storage container and run the indexer from the Azure Portal to ingest new documents.
 
 ### Indexer architecture
 

docs/data_ingestion.md

@@ -231,7 +231,7 @@ var embedding = {
   deploymentName: !empty(embeddingDeploymentName) ? embeddingDeploymentName : 'text-embedding-3-large'
   deploymentVersion: !empty(embeddingDeploymentVersion) ? embeddingDeploymentVersion : (embeddingModelName == 'text-embedding-ada-002' ? '2' : '1')
   deploymentSkuName: !empty(embeddingDeploymentSkuName) ? embeddingDeploymentSkuName : (embeddingModelName == 'text-embedding-ada-002' ? 'Standard' : 'GlobalStandard')
-  deploymentCapacity: embeddingDeploymentCapacity != 0 ? embeddingDeploymentCapacity : 30
+  deploymentCapacity: embeddingDeploymentCapacity != 0 ? embeddingDeploymentCapacity : 200
   dimensions: embeddingDimensions != 0 ? embeddingDimensions : 3072
 }