Overhaul v3 developer documentation (#1344)

* overhaul developer docs for v3

* pre-commit

* lay out new dev docs page structure

* add embedding model providers to EP API

* add personas documentation

* pre-commit

* remove duplicate content

* add section on using Jupyternaut's model

* remove unused inline_completions.md, already copied

* remove ampersands as suggested by @srdas

* title-case 'How-to' sections as suggested by @srdas

Author: dlqqq

docs/source/developers/entry_points_api/embeddings_providers_group.md

@@ -0,0 +1,46 @@
+# `'jupyter_ai.embeddings_model_providers'`
+
+```{contents} Contents
+```
+
+## Summary
+
+This entry point group allows packages to add custom embedding model providers
+to power the retrieval-augmented generation (RAG) capabilities of Jupyter AI.
+
+This group expects an **embedding model provider class**, a subclass of
+`BaseEmbeddingsProvider` from `jupyter-ai` that also inherits from an
+`Embeddings` class from LangChain. Instructions on defining one are given in the
+next section.
+
+:::{warning}
+This is a v2 extension point that may be removed in v3. In v3, we may explore
+updating the model API to make it easier for developers to add custom models.
+:::
+
+## How-to: Define a custom embedding model provider
+
+```python
+from jupyter_ai_magics import BaseEmbeddingsProvider
+from langchain.embeddings import FakeEmbeddings
+
+class MyEmbeddingsProvider(BaseEmbeddingsProvider, FakeEmbeddings):
+    id = "my_embeddings_provider"
+    name = "My Embeddings Provider"
+    model_id_key = "model"
+    models = ["my_model"]
+
+    def __init__(self, **kwargs):
+        super().__init__(size=300, **kwargs)
+```
+
+Jupyter AI uses entry points to discover embedding providers.
+In the `pyproject.toml` file, add your custom embedding provider to the
+`[project.entry-points."jupyter_ai.embeddings_model_providers"]` section:
+
+```toml
+[project.entry-points."jupyter_ai.embeddings_model_providers"]
+my-provider = "my_provider:MyEmbeddingsProvider"
+```
+
+[Embeddings]: https://api.python.langchain.com/en/stable/embeddings/langchain_core.embeddings.Embeddings.html

docs/source/developers/entry_points_api/index.md

@@ -0,0 +1,38 @@
+# Entry points API
+
+The [**entry points API**][entry_points] allows other packages to add custom
+objects to Jupyter AI's backend. Jupyter AI defines a set of **entry point
+groups**. Each entry point group is reserved for a specific type of object, e.g.
+one group may be reserved for personas and another may be reserved for model
+providers.
+
+A package can *provide* an **entry point** to an entry point group to add an
+object of the type reserved by the group. For example, providing an entry point
+to the personas group will add a new AI persona to Jupyter AI. A package can
+also provide multiple entry points to the same group.
+
+The entry points API currently includes the following entry point groups:
+
+- `'jupyter_ai.personas'`: allows developers to add AI personas.
+
+- `'jupyter_ai.model_providers'`: allows developers to add model providers.
+
+- `'jupyter_ai.embeddings_model_providers'`: allows developers to add embedding model providers.
+
+Each entry point group used by Jupyter AI is documented in a section below,
+which describes the type of object expected and how to define a custom
+implementation.
+
+At the end, we describe how to provide an entry point using a custom
+implementation in your package.
+
+```{toctree}
+:caption: Contents
+
+personas_group.md
+model_providers_group.md
+embeddings_providers_group.md
+providing_entry_points.md
+```
+
+[entry_points]: https://setuptools.pypa.io/en/latest/userguide/entry_point.html