|
| 1 | +--- |
| 2 | +title: Assign narrow data types |
| 3 | +titleSuffix: Azure AI Search |
| 4 | +description: In vector search, assign narrow data types to vector fields to reduce the storage requirements of vector indexes. |
| 5 | + |
| 6 | +author: heidisteen |
| 7 | +ms.author: heidist |
| 8 | +ms.service: azure-ai-search |
| 9 | +ms.topic: how-to |
| 10 | +ms.date: 11/04/2024 |
| 11 | +--- |
| 12 | + |
| 13 | +# Assign narrow data types |
| 14 | + |
| 15 | +An easy way to reduce vector size is to store embeddings in a smaller data format. Most embedding models output 32-bit floating point numbers, but if you quantize your vectors, or if your embedding model supports it natively, output might be float16, int16, or int8, which is significantly smaller than float32. You can accommodate these smaller vector sizes by assigning a narrow data type to a vector field. In the vector index, narrow data types consume less storage. |
| 16 | + |
| 17 | +Data types are assigned to fields in an index definition. You can use the Azure portal, the [Search REST APIs](/rest/api/searchservice/indexes/create), or an Azure SDK package that provides the feature. |
| 18 | + |
| 19 | +## Prerequisites |
| 20 | + |
| 21 | +- An embedding model that output small data formats. |
| 22 | + |
| 23 | +## Supported narrow data types |
| 24 | + |
| 25 | +1. Review the [data types used for vector fields](/rest/api/searchservice/supported-data-types#edm-data-types-for-vector-fields) for recommended usage: |
| 26 | + |
| 27 | + - `Collection(Edm.Single)` 32-bit floating point (default) |
| 28 | + - `Collection(Edm.Half)` 16-bit floating point (narrow) |
| 29 | + - `Collection(Edm.Int16)` 16-bit signed integer (narrow) |
| 30 | + - `Collection(Edm.SByte)` 8-bit signed integer (narrow) |
| 31 | + - `Collection(Edm.Byte)` 8-bit unsigned integer (only allowed with packed binary data types) |
| 32 | + |
| 33 | +1. From that list, determine which data type is valid for your embedding model's output, or for vectors that undergo custom quantization. |
| 34 | + |
| 35 | + The following table provides links to several embedding models that can use a narrow data type (`Collection(Edm.Half)`) without extra quantization. You can cast from float32 to float16 (using `Collection(Edm.Half)`) with no extra work. |
| 36 | + |
| 37 | + | Embedding model | Native output | Assign this type in Azure AI Search | |
| 38 | + |------------------------|---------------|--------------------------------| |
| 39 | + | [text-embedding-ada-002](/azure/ai-services/openai/concepts/models#embeddings) | `Float32` | `Collection(Edm.Single)` or `Collection(Edm.Half)` | |
| 40 | + | [text-embedding-3-small](/azure/ai-services/openai/concepts/models#embeddings) | `Float32` | `Collection(Edm.Single)` or `Collection(Edm.Half)` | |
| 41 | + | [text-embedding-3-large](/azure/ai-services/openai/concepts/models#embeddings) | `Float32` | `Collection(Edm.Single)` or `Collection(Edm.Half)` | |
| 42 | + | [Cohere V3 embedding models with int8 embedding_type](https://docs.cohere.com/reference/embed) | `Int8` | `Collection(Edm.SByte)` | |
| 43 | + |
| 44 | + Other narrow data types can be used if your model emits embeddings in the smaller data format, or if you have custom quantization that converts vectors to a smaller format. |
| 45 | + |
| 46 | +1. Make sure you understand the tradeoffs of a narrow data type. `Collection(Edm.Half)` has less information, which results in lower resolution. If your data is homogenous or dense, losing extra detail or nuance could lead to unacceptable results at query time because there's less detail that can be used to distinguish nearby vectors apart. |
| 47 | + |
| 48 | +## Assign the data type |
| 49 | + |
| 50 | +[Define and build the index](vector-search-how-to-create-index.md). You can use the Azure portal, [Create or Update Index (REST API)](/rest/api/searchservice/indexes/create-or-update), or an Azure SDK package for this step. |
| 51 | + |
| 52 | +This field definition uses a narrow data type, `Collection(Edm.Half)`, that can accept a float32 embedding stored as a float16 value. As is true for all vector fields, `dimensions` and `vectorSearchProfile` are set. The specifics of the `vectorSearchProfile` are immaterial to the datatype. |
| 53 | + |
| 54 | +We recommend that you set `retrievable` and `stored` to true if you want to visually check the values of the field. On a subsequent rebuild, you can change these properties to false for reduced storage requirements. |
| 55 | + |
| 56 | +```json |
| 57 | +{ |
| 58 | + "name": "nameEmbedding", |
| 59 | + "type": "Collection(Edm.Half)", |
| 60 | + "searchable": true, |
| 61 | + "filterable": false, |
| 62 | + "retrievable": true, |
| 63 | + "sortable": false, |
| 64 | + "facetable": false, |
| 65 | + "key": false, |
| 66 | + "indexAnalyzer": null, |
| 67 | + "searchAnalyzer": null, |
| 68 | + "analyzer": null, |
| 69 | + "synonymMaps": [], |
| 70 | + "dimensions": 1536, |
| 71 | + "vectorSearchProfile": "myHnswProfile" |
| 72 | +} |
| 73 | +``` |
| 74 | + |
| 75 | +Recall that vector fields aren't filterable, sortable, or facetable. They can't be used as keys and don't use analyzers or synonym maps. |
| 76 | + |
| 77 | +### Working with a production index |
| 78 | + |
| 79 | +Data types are assigned on new fields when they're created. You can't change the data type of an existing field, and you can't drop a field without [rebuilding the index](search-howto-reindex.md). For established indexes already in production, it's common to work around this issue by creating new fields with the desired revisions and then removing obsolete fields during a planned index rebuild. |
| 80 | + |
| 81 | +## Check results |
| 82 | + |
| 83 | +1. Verify the field content matches the data type. Assuming the vector field is marked as retrievable, use [Search explorer](search-explorer.md) or [Search - POST](/rest/api/searchservice/documents/search-post?) to return vector field content. |
| 84 | + |
| 85 | +1. To check vector index size, refer to the vector index size column on the Indexes page in the Azure portal or use the [GET Statistics (REST API)](/rest/api/searchservice/indexes/get-statistics) or equivalent Azure SDK method to get the size. |
| 86 | + |
| 87 | +<!-- |
| 88 | + Evidence of choosing the wrong data type, for example choosing `int8` for a `float32` embedding, is a field that's indexed as an array of zeros. If you encounter this problem, start over. --> |
| 89 | + |
| 90 | +> [!NOTE] |
| 91 | +> The field's data type is used to create the physical data structure. If you want to change a data type later, either drop and rebuild the index, or create a second field with the new definition. |
0 commit comments