Merge branch 'main' into DOC-800

Author: nerpaula

site/content/3.12/about-arangodb/features/core.md

@@ -150,11 +150,9 @@ available from v3.12.5 onward.
   threshold is reached.
 {{% /comment %}}
 
-{{% comment %}} Experimental feature
 - [**Vector search**](../../index-and-search/indexing/working-with-indexes/vector-indexes.md):
   Find items with similar properties by comparing vector embeddings generated by
   machine learning models.
-{{% /comment %}}
 
 - [**Search highlighting**](../../index-and-search/arangosearch/search-highlighting.md):
   Get the substring positions of matched terms, phrases, or _n_-grams.

site/content/3.12/aql/functions/geo.md

@@ -16,13 +16,13 @@ You can calculate vector embeddings using [ArangoDB's GraphML](../../data-scienc
 capabilities (available in ArangoGraph) or using external tools.
 
 {{< warning >}}
-The vector index is an experimental feature that you need to enable for the
-ArangoDB server with the `--experimental-vector-index` startup option.
+You need to enable the vector index feature for the
+ArangoDB server with the `--vector-index` startup option.
 Once enabled for a deployment, it cannot be disabled anymore because it
 permanently changes how the data is managed by the RocksDB storage engine
 (it adds an additional column family).
 
-To restore a dump that contains vector indexes, the `--experimental-vector-index`
+To restore a dump that contains vector indexes, the `--vector-index`
 startup option needs to be enabled on the deployment you want to restore to.
 {{< /warning >}}
 
@@ -56,21 +56,37 @@ be found depends on the data as well as the search effort (see the `nProbe` opti
 {{< info >}}
 - If there is more than one suitable vector index over the same attribute, it is
   undefined which one is selected.
-- You cannot have any `FILTER` operation between `FOR` and `LIMIT` for
-  pre-filtering.
+
+- In v3.12.4 and v3.12.5, you cannot have any `FILTER` operation between `FOR`
+  and `LIMIT` for pre-filtering. From v3.12.6 onward, you can add `FILTER`
+  operations between `FOR` and `SORT` that are then applied during the lookup in
+  the vector index. Example:
+
+  ```aql
+  FOR doc IN coll
+    FILTER doc.val > 3
+    SORT APPROX_NEAR_COSINE(doc.vector, @q) DESC
+    LIMIT 5
+    RETURN doc
+  ```
+
+  Note that e.g. `LIMIT 5` does not ensure that you get 5 results by searching
+  as many neighboring Voronoi cells as necessary, but it rather considers only as
+  many as configured via the `nProbes` parameter.
 {{< /info >}}
 
 ### APPROX_NEAR_COSINE()
 
 `APPROX_NEAR_COSINE(vector1, vector2, options) → similarity`
 
-Retrieve the approximate angular similarity using the cosine metric, accelerated
-by a matching vector index.
 
-The higher the cosine similarity value is, the more similar the two vectors
-are. The closer it is to 0, the more different they are. The value can also
-be negative, indicating that the vectors are not similar and point in opposite
-directions. You need to sort in descending order so that the most similar
+Retrieve the approximate cosine of the angle between two vectors, accelerated
+by a matching vector index with the `cosine` metric.
+
+The closer the similarity value is to 1, the more similar the two vectors
+are. The closer it is to 0, the more different they are. The value can also be
+negative up to -1, indicating that the vectors are not similar and point in opposite
+directions. You need to **sort in descending order** so that the most similar
 documents come first, which is what a vector index using the `cosine` metric
 can provide.
 
@@ -83,8 +99,8 @@ can provide.
     closest Voronoi cells to consider for the search results. The larger the number,
     the slower the search but the better the search results. If not specified, the
     `defaultNProbe` value of the vector index is used.
-- returns **similarity** (number): The approximate angular similarity between
-  both vectors.
+- returns **similarity** (number): The approximate cosine similarity of
+  both normalized vectors. The value range is `[-1, 1]`.
 
 **Examples**
 
@@ -126,15 +142,83 @@ FOR docOuter IN coll
   RETURN { key: docOuter._key, neighbors }
 ```
 
+### APPROX_NEAR_INNER_PRODUCT()
+
+<small>Introduced in: v3.12.6</small>
+
+`APPROX_NEAR_INNER_PRODUCT(vector1, vector2, options) → similarity`
+
+Retrieve the approximate dot product of two vectors, accelerated by a matching
+vector index with the `innerProduct` metric.
+
+The higher the similarity value is, the more similar the two vectors
+are. The closer it is to 0, the more different they are. The value can also
+be negative, indicating that the vectors are not similar and point in opposite
+directions. You need to **sort in descending order** so that the most similar
+documents come first, which is what a vector index using the `innerProduct`
+metric can provide.
+
+- **vector1** (array of numbers): The first vector. Either this parameter or
+  `vector2` needs to reference a stored attribute holding the vector embedding.
+- **vector2** (array of numbers): The second vector. Either this parameter or
+  `vector1` needs to reference a stored attribute holding the vector embedding.
+- **options** (object, _optional_):
+  - **nProbe** (number, _optional_): How many neighboring centroids respectively
+    closest Voronoi cells to consider for the search results. The larger the number,
+    the slower the search but the better the search results. If not specified, the
+    `defaultNProbe` value of the vector index is used.
+- returns **similarity** (number): The approximate dot product
+  of both vectors without normalization. The value range is unbounded.
+
+**Examples**
+
+Return up to `10` similar documents based on their closeness to the vector
+`@q` according to the inner product metric:
+
+```aql
+FOR doc IN coll
+  SORT APPROX_NEAR_INNER_PRODUCT(doc.vector, @q) DESC
+  LIMIT 10
+  RETURN doc
+```
+
+Return up to `5` similar documents as well as the similarity value,
+considering `20` neighboring centroids respectively closest Voronoi cells:
+
+```aql
+FOR doc IN coll
+  LET similarity = APPROX_NEAR_INNER_PRODUCT(doc.vector, @q, { nProbe: 20 })
+  SORT similarity DESC
+  LIMIT 5
+  RETURN MERGE( { similarity }, doc)
+```
+
+Return the similarity value and the document keys of up to `3` similar documents
+for multiple input vectors using a subquery. In this example, the input vectors
+are taken from ten random documents of the same collection:
+
+```aql
+FOR docOuter IN coll
+  LIMIT 10
+  LET neighbors = (
+    FOR docInner IN coll
+      LET similarity = APPROX_NEAR_INNER_PRODUCT(docInner.vector, docOuter.vector)
+      SORT similarity DESC
+      LIMIT 3
+      RETURN { key: docInner._key, similarity }
+  )
+  RETURN { key: docOuter._key, neighbors }
+```
+
 ### APPROX_NEAR_L2()
 
-`APPROX_NEAR_L2(vector1, vector2, options) → similarity`
+`APPROX_NEAR_L2(vector1, vector2, options) → distance`
 
 Retrieve the approximate distance using the L2 (Euclidean) metric, accelerated
-by a matching vector index.
+by a matching vector index with the `l2` metric.
 
 The closer the distance is to 0, the more similar the two vectors are. The higher
-the value, the more different the they are. You need to sort in ascending order
+the value, the more different the they are. You need to **sort in ascending order**
 so that the most similar documents come first, which is what a vector index using
 the `l2` metric can provide.
 
@@ -147,7 +231,7 @@ the `l2` metric can provide.
     for the search results. The larger the number, the slower the search but the
     better the search results. If not specified, the `defaultNProbe` value of
     the vector index is used.
-- returns **similarity** (number): The approximate L2 (Euclidean) distance between
+- returns **distance** (number): The approximate L2 (Euclidean) distance between
   both vectors.
 
 **Examples**

site/content/3.12/aql/functions/vector.md

@@ -117,6 +117,56 @@ All collections in the list that do not specify their own direction use the
 direction defined after `IN` (here: `OUTBOUND`). This allows using a different
 direction for each collection in your path search.
 
+### Graph path searches in a cluster
+
+Due to the nature of graphs, edges may reference nodes from arbitrary
+collections. Following the paths can thus involve documents from various
+collections and it is not possible to predict which are visited in a
+traversal. Which collections need to be loaded by the graph engine can only be
+determined at run time.
+
+Use the [`WITH` operation](../high-level-operations/with.md) to specify the
+node collections you expect to be involved. This is required for traversals
+using collection sets in cluster deployments. Declare the collection of the
+start node as well if it's not declared already (like by a `FOR` loop).
+
+{{< tip >}}
+From v3.12.6 onward, node collections are automatically deduced for graph
+queries using collection sets / anonymous graphs if there is a named graph with
+a matching edge collection in its edge definitions.
+
+For example, suppose you have two node collections, `person` and `movie`, and
+an `acts_in` edge collection that connects them. If you want to run a path search
+query that starts (and ends) at a person that you specify with its document ID,
+you need to declare both node collections at the beginning of the query:
+
+```aql
+WITH person, movie
+FOR p IN ANY ALL_SHORTEST_PATHS "person/1544" TO "person/52560" acts_in
+  RETURN p.vertices[*].label
+```
+
+However, if there is a named graph that includes an edge definition for the
+`acts_in` edge collection, with `person` as the _from_ collection and `movie`
+as the _to_ collection, you can omit `WITH person, movie`. That is, if you
+specify `acts_in` as an edge collection in an anonymous graph query, all
+named graphs are checked for this edge collection, and if there is a matching
+edge definition, its node collections are automatically added as data sources to
+the query.
+
+```aql
+FOR p IN ANY ALL_SHORTEST_PATHS "person/1544" TO "person/52560" acts_in
+  RETURN p.vertices[*].label
+
+// Chris Rock --> Dogma <-- Ben Affleck --> Surviving Christmas <-- Jennifer Morrison
+// Chris Rock --> The Longest Yard <-- Rob Schneider --> Big Stan <-- Jennifer Morrison
+// Chris Rock --> Down to Earth <-- John Cho --> Star Trek <-- Jennifer Morrison
+```
+
+You can still declare collections manually, in which case they are added as
+data sources in addition to automatically deduced collections.
+{{< /tip >}}
+
 ## Examples
 
 Load an example graph to get a named graph that reflects some possible

site/content/3.12/aql/graphs/all-shortest-paths.md

@@ -188,6 +188,57 @@ All collections in the list that do not specify their own direction use the
 direction defined after `IN` (here: `OUTBOUND`). This allows to use a different
 direction for each collection in your path search.
 
+### Graph path searches in a cluster
+
+Due to the nature of graphs, edges may reference nodes from arbitrary
+collections. Following the paths can thus involve documents from various
+collections and it is not possible to predict which are visited in a
+traversal. Which collections need to be loaded by the graph engine can only be
+determined at run time.
+
+Use the [`WITH` operation](../high-level-operations/with.md) to specify the
+node collections you expect to be involved. This is required for traversals
+using collection sets in cluster deployments. Declare the collection of the
+start node as well if it's not declared already (like by a `FOR` loop).
+
+{{< tip >}}
+From v3.12.6 onward, node collections are automatically deduced for graph
+queries using collection sets / anonymous graphs if there is a named graph with
+a matching edge collection in its edge definitions.
+
+For example, suppose you have two node collections, `person` and `movie`, and
+an `acts_in` edge collection that connects them. If you want to run a path search
+query that starts (and ends) at a person that you specify with its document ID,
+you need to declare both node collections at the beginning of the query:
+
+```aql
+WITH person, movie
+FOR p IN 4 ANY K_PATHS "person/1544" TO "person/52560" acts_in
+  LIMIT 2
+  RETURN p.vertices[*].label
+```
+
+However, if there is a named graph that includes an edge definition for the
+`acts_in` edge collection, with `person` as the _from_ collection and `movie`
+as the _to_ collection, you can omit `WITH person, movie`. That is, if you
+specify `acts_in` as an edge collection in an anonymous graph query, all
+named graphs are checked for this edge collection, and if there is a matching
+edge definition, its node collections are automatically added as data sources to
+the query.
+
+```aql
+FOR p IN 4 ANY K_PATHS "person/1544" TO "person/52560" acts_in
+  LIMIT 2
+  RETURN p.vertices[*].label
+
+// Chris Rock --> Dogma <-- Ben Affleck --> Surviving Christmas <-- Jennifer Morrison
+// Chris Rock --> The Longest Yard <-- Rob Schneider --> Big Stan <-- Jennifer Morrison
+```
+
+You can still declare collections manually, in which case they are added as
+data sources in addition to automatically deduced collections.
+{{< /tip >}}
+
 ## Examples
 
 You can load the `kShortestPathsGraph` example graph to get a named graph that

site/content/3.12/aql/graphs/k-paths.md

@@ -186,6 +186,57 @@ All collections in the list that do not specify their own direction use the
 direction defined after `IN` (here: `OUTBOUND`). This allows to use a different
 direction for each collection in your path search.
 
+### Graph path searches in a cluster
+
+Due to the nature of graphs, edges may reference nodes from arbitrary
+collections. Following the paths can thus involve documents from various
+collections and it is not possible to predict which are visited in a
+traversal. Which collections need to be loaded by the graph engine can only be
+determined at run time.
+
+Use the [`WITH` operation](../high-level-operations/with.md) to specify the
+node collections you expect to be involved. This is required for traversals
+using collection sets in cluster deployments. Declare the collection of the
+start node as well if it's not declared already (like by a `FOR` loop).
+
+{{< tip >}}
+From v3.12.6 onward, node collections are automatically deduced for graph
+queries using collection sets / anonymous graphs if there is a named graph with
+a matching edge collection in its edge definitions.
+
+For example, suppose you have two node collections, `person` and `movie`, and
+an `acts_in` edge collection that connects them. If you want to run a path search
+query that starts (and ends) at a person that you specify with its document ID,
+you need to declare both node collections at the beginning of the query:
+
+```aql
+WITH person, movie
+FOR p IN ANY K_SHORTEST_PATHS "person/1544" TO "person/52560" acts_in
+  LIMIT 2
+  RETURN p.vertices[*].label
+```
+
+However, if there is a named graph that includes an edge definition for the
+`acts_in` edge collection, with `person` as the _from_ collection and `movie`
+as the _to_ collection, you can omit `WITH person, movie`. That is, if you
+specify `acts_in` as an edge collection in an anonymous graph query, all
+named graphs are checked for this edge collection, and if there is a matching
+edge definition, its node collections are automatically added as data sources to
+the query.
+
+```aql
+FOR p IN ANY K_SHORTEST_PATHS "person/1544" TO "person/52560" acts_in
+  LIMIT 2
+  RETURN p.vertices[*].label
+
+// Chris Rock --> Dogma <-- Ben Affleck --> Surviving Christmas <-- Jennifer Morrison
+// Chris Rock --> The Longest Yard <-- Rob Schneider --> Big Stan <-- Jennifer Morrison
+```
+
+You can still declare collections manually, in which case they are added as
+data sources in addition to automatically deduced collections.
+{{< /tip >}}
+
 ## Examples
 
 You can load the `kShortestPathsGraph` example graph to get a named graph that