cody docs: document why we skip files (#54165)

This adds a section to the FAQ and also documents some previously
undocumented site config settings.

Author: stefanhengl

doc/cody/explanations/code_graph_context.md

@@ -53,6 +53,20 @@ To use `excludedFilePathPatterns`, add it to your embeddings site config with a
 }
 ```
 
+By default, the following patterns are excluded from embeddings:
+
+- *ignore" // Files like .gitignore, .eslintignore
+- .gitattributes
+- .mailmap
+- *.csv
+- *.svg
+- *.xml
+- \_\_fixtures\_\_/
+- node_modules/
+- testdata/
+- mocks/
+- vendor/
+
 > NOTE: The `excludedFilePathPatterns` setting is only available in Sourcegraph version `5.0.1` and later.
 
 ### Storing embedding indexes
@@ -182,3 +196,18 @@ A negative value disables the limit and all repositories are selected.
   }
 }
 ```
+
+### Limitting the number of embeddings that can be generated
+
+The number of embeddings that can be generated per repo is limited to `embeddings.maxCodeEmbeddingsPerRepo` for code embeddings (default 3.072.000) or `embeddings.maxTextEmbeddingsPerRepo` (default 512.000) for text embeddings.
+
+Use the following site configuration to update the limits:
+
+```jsonc
+{
+  "embeddings": {
+    "maxCodeEmbeddingsPerRepo": 3072000,
+    "maxTextEmbeddingsPerRepo": 512000
+  }
+}
+```

doc/cody/faq.md

@@ -58,6 +58,13 @@ There can be several reasons why a job is not showing up in the list of jobs:
 ### How do I stop a running embeddings job?
 
 Jobs in state _QUEUED_ or _PROCESSING_ can be canceled by admins from the **Cody > Embeddings Jobs** page. To cancel a job, click on the _Cancel_ button of the job you want to cancel. The job will be marked for cancellation. Note that, depending on the state of the job, it might take a few seconds or minutes for the job to actually be canceled.
+#### What are the reasons files are skipped?
+
+Files are skipped for the following reasons:
+
+- The file is too large (1 MB)
+- The file path matches an [exclusion pattern](./explanations/code_graph_context.md#excluding-files-from-embeddings)
+- We have already generated more than [`embeddings.maxCodeEmbeddingsPerRepo`](./explanations/code_graph_context.md#limitting-the-number-of-embeddings-that-can-be-generated) or [`embeddings.maxTextEmbeddingsPerRepo`](./explanations/code_graph_context.md#limitting-the-number-of-embeddings-that-can-be-generated) embeddings for the repo.
 
 ### Third party dependencies