Pulling in findings from engineering Q&A

Author: HeidiSteen

articles/search/vector-search-how-to-create-index.md

@@ -82,7 +82,7 @@ The schema must include fields for the document key, vector fields, and any othe
    + Provide the name of the vector search algorithm configuration.
    + Provide the number of dimensions generated by the embedding model.
    + "searchable" must be "true".
-   + "retrievable" set to "true" allows you to display the raw vectors (for example, as a verification step), but doing so will increase storage usage. Set to "false" if you don't need to return raw vectors.
+   + "retrievable" set to "true" allows you to display the raw vectors (for example, as a verification step), but doing so will increase storage. Set to "false" if you don't need to return raw vectors. You don't need to return vectors for a query, but if you're passing a vector result to a downstream app then set "retrievable" to "true".
 
     ```http
     PUT https://my-search-service.search.windows.net/indexes/my-index?api-version=2023-07-01-Preview&allowIndexDowntime=true

articles/search/vector-search-how-to-query.md

@@ -37,7 +37,7 @@ If you aren't sure whether your search index already has vector fields, look for
 
 + In the fields collection, look for fields of type `Collection(Edm.Single)`, with a `dimensions` attribute and a `vectorSearchConfiguration` set to the name of the `vectorSearch` algorithm configuration used by the field.
 
-You can also send an empty query (`search=*`) against the index. Search documents containing vector data have fields containing many hundreds of floating point values.
+You can also send an empty query (`search=*`) against the index if the vector field is "retrievable". In a response, a vector field consists of an array of floating point values.
 
 ## Convert query input into a vector
 
@@ -79,19 +79,25 @@ The expected response is 202 for a successful call to the deployed model. The bo
 }
 ```
 
-## Design a query response
+## Configure a query response
 
-When you're setting up the vector query, think about how you want to structure the response. Search results are composed of either all "retrievable" fields (a REST API default) or the fields explicitly listed in a "select" parameter. In the query examples that follow, each one includes a "select" parameter that specifies text (non-vector) content for the response.
+When you're setting up the vector query, think about the response structure. Search results are composed of either all "retrievable" fields (a REST API default) or the fields explicitly listed in a "select" parameter on the query. In the examples that follow, each one includes a "select" statement that specifies text (non-vector) fields to include the response.
 
-Vector fields themselves aren't human readable, so avoid returning them in the response. Instead, choose non-vector fields that provide equivalent information from the same search document. For example, if the query is on a vector field ("descriptionVector"), return an equivalent text field ("description") in the response.
+Vectors aren't designed for readability, so avoid returning them in the response. Instead, choose non-vector fields that are representative of the search document. For example, if the query targets a "descriptionVector" field, return an equivalent text field ("description") in the response.
 
-The quantity of results are determines by query parameters. Quantity is either: 
+Size of the results is determined by the query parameters "k" and "top". Maximum results in a response are either: 
 
 + `"k": n` results for vector-only queries
+
 + `"top": n` results for hybrid queries
 
-> [!NOTE]
-> If you're familiar with full text search in Cognitive Search, you already know that a term or keyword, synonym, or filter criteria must match in order for a document to qualify as a match. Similarity search is less exacting because it's comparing vector compositions. It's possible for the HNSW model to sometimes return matches that don't seem especially relevant.
+Ranking of results is either:
+
++ Cosine similarity if the query is over a single vector field, assuming `cosine` is what you specified in the index `vectorConfiguration`. Azure OpenAI embedding models use cosine similarity metrics. Other supported ranking metrics include `euclidean` and `dotProduct`.
+
++ Reciprocal Rank Fusion (RRF) if there  multiple sets of search results. Multiple sets are created if the query targets multiple vector fields, or if the query is a hybrid of vector and full text search, with or withou the optional semantic re-ranking capabilities of [semantic search](semantic-search-overview.md).
+
+Within vector search, a vector query can only target one internal vector index. So for [multiple vector fields](#query-syntax-for-vector-query-over-multiple-fields) and [multiple vector queries](#query-syntax-for-multiple-vector-queries), the search engine generates parallel queries that target the respective vector indexes of each field. Output is a set of ranked results for each query, which are fused using RRF. For more information, see [Vector query execution and scoring](vector-search-ranking.md).
 
 ## Query syntax for vector search
 

articles/search/vector-search-overview.md

@@ -27,9 +27,9 @@ We recommend this article for background, but if you'd rather get started, follo
 
 ## What's vector search in Cognitive Search?
 
-Vector search is a new capability for indexing, storing, and retrieving vector embeddings. You can use it to power similarity search, multi-modal search, recommendations engines, or applications implementing the [Retrieval Augmented Generation (RAG) architecture](https://arxiv.org/abs/2005.11401).
+Vector search is a new capability for indexing, storing, and retrieving vector embeddings from a search index. You can use it to power similarity search, multi-modal search, recommendations engines, or applications implementing the [Retrieval Augmented Generation (RAG) architecture](https://arxiv.org/abs/2005.11401).
 
-Support for vector search is in public preview and available through the [**2023-07-01-Preview REST APIs**](/rest/api/searchservice/index-preview). To use vector search, define a *vector field* in the index definition and index documents with vector data. Then you can issue search request with a query vector, returning documents with the requested `k` nearest neighbors (kNN) according to the selected vector similarity metric.
+Support for vector search is in public preview and available through the [**2023-07-01-Preview REST APIs**](/rest/api/searchservice/index-preview). To use vector search, define a *vector field* in the index definition and index documents with vector data. Then you can issue a search request with a query vector, returning documents with the requested `k` nearest neighbors (kNN) according to the selected vector similarity metric.
 
 You can index vector data as fields in documents alongside textual and other types of content. Vector queries can be issued independently or in combination with other query types, including term queries (hybrid search) and filters in the same search request.
 
@@ -54,10 +54,9 @@ Scenarios for vector search include:
 
 + **Multi-lingual search**. Use a multi-lingual embeddings model to represent your document in multiple languages in a single vector space to find documents regardless of the language they are in.
 
-<!-- @Farzad, filterable is false on a vector field, so we need to explain what we mean here. I wonder if it goes with hybrid query? -->
-+ **Filtered vector search**. Use [filters](search-filters.md) with vector queries to select a specific category of indexed documents, or to implement document-level security, geospatial search, and more.
++ **Hybrid search**. Vector search is implemented at the field level, which means you can build qeuries that include vector fields and searchable text fields. The queries execute in parallel and the results are merged into a single reponse. Optionally, add [semantic search (preview)](semantic-search-overview.md) for even more accuracy with L2 reranking using the same language models that power Bing.
 
-+ **Hybrid search**. For text data, combine the best of vector retrieval and keyword retrieval to obtain the best results. Use with [semantic search (preview)](semantic-search-overview.md) for even more accuracy with L2 reranking using the same language models that power Bing.  
++ **Filtered vector search**. A query request can include a vector query and a [filter expression](search-filters.md). Filters apply to text and numeric fields, and are useful for including or excluding search documents based on filter criteria. Athough a vector field is not filterable itself, you can set up a filterable text or numeric field. The search engine processes the filter first, reducing the surface area of the search corpus before running the vector query.
 
 + **Vector database**. Use Cognitive Search as a vector store to serve as long-term memory or an external knowledge base for Large Language Models (LLMs), or other applications.
 

articles/search/vector-search-ranking.md

@@ -7,19 +7,19 @@ author: yahnoosh
 ms.author: jlembicz
 ms.service: cognitive-search
 ms.topic: conceptual
-ms.date: 07/07/2023
+ms.date: 07/14/2023
 ---
 
 # Vector query execution and scoring in Azure Cognitive Search
 
 > [!IMPORTANT]
 > Vector search is in public preview under [supplemental terms of use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). It's available through the Azure portal, preview REST API, and [alpha SDKs](https://github.com/Azure/cognitive-search-vector-pr#readme).
 
-This article is for developers who need a deeper understanding of ranking of vector queries in Azure Cognitive Search.
+This article is for developers who need a deeper understanding of vector query execution and ranking in Azure Cognitive Search.
 
 ## Vector similarity
 
-In a vector query, the search query is a vector as opposed to text in full-text queries. Documents which matched the vector query are ranked using vector similarity configured on the vector field defined in the index. A vector query specifies the `k` parameter which determines how many nearest neighbors of the query vector should be returned from the index. 
+In a vector query, the search query is a vector as opposed to text in full-text queries. Documents that match the vector query are ranked using vector similarity configured on the vector field defined in the index. A vector query specifies the `k` parameter which determines how many nearest neighbors of the query vector should be returned from the index. 
 
 > [!NOTE]
 > Full-text search queries could return fewer than the requested number of results if there are fewer or no matches, but vector search will return up to `k` matches as long as there are enough documents in the index. This is because with vector search, similarity is relative to the input query vector, not absolute. This means less relevant results have a worse similarity score, but they can still be the "nearest" vectors if there aren't any closer vectors. As such, a response with no meaningful results can still return `k` results, but each result's similarity score would be low.
@@ -30,13 +30,13 @@ If a query request is about dogs, the model maps the query into a vector that ex
 
 Commonly used similarity metrics include `cosine`, `euclidean` (also known as `l2 norm`), and `dotProduct`, which are summarized here:
 
-+ Cosine calculates the angle between two vectors.
++ cosine calculates the angle between two vectors. Cosine is the similarity metric used by [Azure OpenAI embedding models](/azure/cognitive-services/openai/concepts/understand-embeddings#cosine-similarity).
 
-+ Euclidean calculates the Euclidean distance between two vectors, which is the l2-norm of the difference of the two vectors.
++ euclidean calculates the Euclidean distance between two vectors, which is the l2-norm of the difference of the two vectors.
 
-+ Dot product is affected by both vectors' magnitudes and the angle between them.
++ dotProduct is affected by both vectors' magnitudes and the angle between them.
 
-For normalized embedding spaces, dot product is equivalent to the cosine similarity, but is more efficient.
+For normalized embedding spaces, dotProduct is equivalent to the cosine similarity, but is more efficient.
 
 ## Hybrid search