Implement dynamic community selection for global search (#1396)

* update gitignore

* add dynamic community sleection to updated main branch

* update SearchResult to record output_tokens.

* update search result

* dynamic search working

* format

* add llm_calls_categories and prompt_tokens and output_tokens cate

* update

* formatting

* log drift search output and prompt tokens separately

* update global_search.ipynb. update operate dulce dataset and add create_final_communities. update dynamic community selection init

* add .ipynb back to cspell.config.yaml

* format

* add notebook example on dynamic search

* rearrange

* update gitignore

* format code

* code format

* code format

* fix default variable

---------

Co-authored-by: Bryan Li <[email protected]>

Author: AlonsoGuevara

.gitignore

@@ -50,3 +50,9 @@ site/
 docsite/
 .yarn/
 .pnp*
+
+# PyCharm
+.idea/
+
+# Jupyter notebook
+.ipynb_checkpoints/

.semversioner/next-release/patch-20241106004612053719.json

@@ -0,0 +1,4 @@
+{
+  "type": "patch",
+  "description": "Implement dynamic community selection to global search"
+}