You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/azure-monitor/agents/azure-monitor-agent-extension-versions.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,6 +23,7 @@ We strongly recommended to always update to the latest version, or opt in to the
23
23
## Version details
24
24
| Release Date | Release notes | Windows | Linux |
25
25
|:---|:---|:---|:---|
26
+
| March 2024 |**Widows**<ul><li>**Breaking Change from Publict Preview to GA** Due to customer feedback, automatic parsing of JSON into column in your custom table in Log Analytic was added. You must take action to migrate your JSON DCR created prior to this release to prevent data loss. This is the last release of the JSON Log type in Public Preview an GA will be declared in a few weeks.</li><li>Fix AMA when resource ID contains non-ascii chars which is common when using some languages other than English. Errors would follow this pattern: … [HealthServiceCommon][][Error] … WinHttpAddRequestHeaders(x-ms-AzureResourceId: /subscriptions/{your subscription #} /resourceGroups/???????/providers/ … PostDataItems" failed with code 87(ERROR_INVALID_PARAMETER) </li></ul> | 1.25.0 | Comming Soon |
26
27
| February 2024 | **Known Issues**<ul><li>Occasional crash during startup in arm64 VMs. This is fixed in 1.30.3</li></uL>**Windows**<ul><li>Fix memory leak in Internet Information Service (IIS) log collection</li><li>Fix JSON parsing with Unicode characters for some ingestion endpoints</li><li>Allow Client installer to run on Azure Virtual Desktop (AVD) DevBox partner</li><li>Enable Transport Layer Security (TLS) 1.3 on supported Windows versions</li><li>Update MetricsExtension package to 2.2024.202.2043</li></ul>**Linux**<ul><li>Features<ul><li>Add EventTime to syslog for parity with OMS agent</li><li>Add more Common Event Format (CEF) format support</li><li>Add CPU quotas for Azure Monitor Agent (AMA)</li></ul><li>Fixes<ul><li>Handle truncation of large messages in syslog due to Transmission Control Protocol (TCP) framing issue</li><li>Set NO_PROXY for Instance Metadata Service (IMDS) endpoint in AMA Python wrapper</li><li>Fix a crash in syslog parsing</li><li>Add reasonable limits for metadata retries from IMDS</li><li>No longer reset /var/log/azure folder permissions</li></ul></ul> | 1.24.0 | 1.30.3<br>1.30.2 |
27
28
| January 2024 |**Known Issues**<ul><li>1.29.5 doesn't install on Arc-enabled servers because the agent extension code size is beyond the deployment limit set by Arc. **This issue was fixed in 1.29.6**</li></ul>**Windows**<ul><li>Added support for Transport Layer Security (TLS) 1.3</li><li>Reverted a change to enable multiple IIS subscriptions to use same filter. Feature is redeployed once memory leak is fixed</li><li>Improved Event Trace for Windows (ETW) event throughput rate</li></ul>**Linux**<ul><li>Fix error messages logged, intended for mdsd.err, that instead went to mdsd.warn in 1.29.4 only. Likely error messages: "Exception while uploading to Gig-LA: ...", "Exception while uploading to ODS: ...", "Failed to upload to ODS: ..."</li><li>Reduced noise generated by AMAs' use of semanage when SELinux is enabled</li><li>Handle time parsing in syslog to handle Daylight Savings Time (DST) and leap day</li></ul> | 1.23.0 | 1.29.5, 1.29.6 |
28
29
| December 2023 |**Known Issues**<ul><li>1.29.4 doesn't install on Arc-enabled servers because the agent extension code size is beyond the deployment limit set by Arc. Fix is coming in 1.29.6</li><li>Multiple IIS subscriptions cause a memory leak. feature reverted in 1.23.0</ul>**Windows** <ul><li>Prevent CPU spikes by not using bookmark when resetting an Event Log subscription</li><li>Added missing Fluent Bit executable to AMA client setup for Custom Log support</li><li>Updated to latest AzureCredentialsManagementService and DsmsCredentialsManagement package</li><li>Update ME to v2.2023.1027.1417</li></ul>**Linux**<ul><li>Support for TLS v1.3</li><li>Support for nopri in Syslog</li><li>Ability to set disk quota from Data Collection Rule (DCR) Agent Settings</li><li>Add ARM64 Ubuntu 22 support</li><li>**Fixes**<ul><li>SysLog</li><ul><li>Parse syslog Palo Alto CEF with multiple space characters following the hostname</li><li>Fix an issue with incorrectly parsing messages containing two '\n' chars in a row</li><li>Improved support for non-RFC compliant devices</li><li>Support Infoblox device messages containing both hostname and IP headers</li></ul><li>Fix AMA crash in Read Hat Enterprise Linux (RHEL) 7.2</li><li>Remove dependency on "which" command</li><li>Fix port conflicts due to AMA using 13000 </li><li>Reliability and Performance improvements</li></ul></li></ul>| 1.22.0 | 1.29.4|
Copy file name to clipboardExpand all lines: articles/cosmos-db/mongodb/vcore/vector-search-ai.md
+16-15Lines changed: 16 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,9 +21,9 @@ Another advantage of open-source vector databases is the strong community suppor
21
21
22
22
Some individuals opt for open-source vector databases because they are "free," meaning there's no cost to acquire or use the software. An alternative is using the free tiers offered by managed vector database services. These managed services provide not only cost-free access up to a certain usage limit but also simplify the operational burden by handling maintenance, updates, and scalability. Therefore, by using the free tier of managed vector database services, users can achieve cost savings while reducing management overhead. This approach allows users to focus more on their core activities rather than on database administration.
23
23
24
-
## Working mechanism of open-source vector databases
24
+
## Working mechanism of vector databases
25
25
26
-
Open-source vector databases are designed to store and manage vector embeddings, which are mathematical representations of data in a high-dimensional space. In this space, each dimension corresponds to a feature of the data, and tens of thousands of dimensions might be used to represent sophisticated data. A vector's position in this space represents its characteristics. Words, phrases, or entire documents, and images, audio, and other types of data can all be vectorized. These vector embeddings are used in similarity search, multi-modal search, recommendations engines, large languages models (LLMs), etc.
26
+
Vector databases are designed to store and manage vector embeddings, which are mathematical representations of data in a high-dimensional space. In this space, each dimension corresponds to a feature of the data, and tens of thousands of dimensions might be used to represent sophisticated data. A vector's position in this space represents its characteristics. Words, phrases, or entire documents, and images, audio, and other types of data can all be vectorized. These vector embeddings are used in similarity search, multi-modal search, recommendations engines, large languages models (LLMs), etc.
27
27
28
28
These databases' architecture typically includes a storage engine and an indexing mechanism. The storage engine optimizes the storage of vector data for efficient retrieval and manipulation, while the indexing mechanism organizes the data for fast searching and retrieval operations.
29
29
@@ -40,6 +40,14 @@ Vector databases are used in numerous domains and situations across analytical a
40
40
- Implement persistent memory for AI agents
41
41
- Enable retrieval-augmented generation (RAG)
42
42
43
+
### Integrated vector database vs pure vector database
44
+
45
+
There are two common types of vector database implementations - pure vector database and integrated vector database in a NoSQL or relational database.
46
+
47
+
A pure vector database is designed to efficiently store and manage vector embeddings, along with a small amount of metadata; it is separate from the data source from which the embeddings are derived.
48
+
49
+
A vector database that is integrated in a highly performant NoSQL or relational database provides additional capabilities. The integrated vector database in a NoSQL or relational database can store, index, and query embeddings alongside the corresponding original data. This approach eliminates the extra cost of replicating data in a separate pure vector database. Moreover, keeping the vector embeddings and original data together better facilitates multi-modal data operations, and enables greater data consistency, scale, and performance.
50
+
43
51
## Selecting the best open-source vector database
44
52
45
53
Choosing the best open-source vector database requires considering several factors. Performance and scalability of the database are crucial, as they impact whether the database can handle your specific workload requirements. Databases with efficient indexing and querying capabilities usually offer optimal performance. Another factor is the community support and documentation available for the database. A robust community and ample documentation can provide valuable assistance. Here are some popular open-source vector databases:
@@ -49,33 +57,26 @@ Choosing the best open-source vector database requires considering several facto
49
57
- Qdrant
50
58
- Weaviate
51
59
52
-
>[!NOTE]
53
-
>The most popular option may not be the best option for you. To find the best fit for your needs, you should compare different options based on features, supported data types, compatibility with existing tools and frameworks you use. Ease of installation, configuration, and maintenance should also be considered to ensure smooth integration into your workflow.
60
+
However, the most popular option may not be the best option for you. Thus, you should compare different options based on features, supported data types, compatibility with existing tools and frameworks you use. You should also keep in mind the challenges of open-source vector databases (below).
54
61
55
62
## Challenges of open-source vector databases
56
63
57
-
Open-source vector databases pose challenges that are typical of open-source software and ones that are specific to many vector databases.
64
+
Most open-source vector databases, including the ones listed above, are pure vector databases. In other words, they are designed to store and manage vector embeddings only, along with a small amount of metadata. Since they are independent of the data source from which the embeddings are derived, using them requires sending your data between service integrations, which adds extra cost, complexity, and bottlenecks for your production workloads.
58
65
59
-
### Challenges with open source:
66
+
They also pose the challenges that are typical of open-source databases:
60
67
61
68
- Setup: Users need in-depth knowledge to install, configure, and operate, especially for complex deployments. Optimizing resources and configuration while scaling up operation requires close monitoring and adjustments.
62
69
- Maintenance: Users must manage their own updates, patches, and maintenance. Thus, ML expertise wouldn't suffice; users must also have extensive experience in database administration.
63
70
- Support: Official support can be limited compared to managed services, relying more on community assistance.
64
71
65
72
Therefore, while free initially, open-source vector databases incur significant costs when scaling up. Expanding operations necessitates more hardware, skilled IT staff, and advanced infrastructure management, leading to higher expenses in hardware, personnel, and operational costs. Scaling open-source vector databases can be financially demanding despite the lack of licensing fees.
66
73
67
-
### Challenges that are specific to vector databases:
68
-
69
-
Most open-source vector databases nowadays are pure vector databases. In other words, they are designed to store and manage vector embeddings, along with a small amount of metadata; it is separate from the data source from which the embeddings are derived. Thus, using pure vector databases requires sending your data between service integrations, which adds extra cost, complexity, and bottlenecks for your production workloads.
70
-
71
74
## Addressing the challenges of open-source vector databases
72
75
73
-
A fully managed database service helps developers avoid the hassles from setting up, maintaining, and relying on community assistance for an open-source vector database; moreover, some managed vector database services offer a life-time free tier.
74
-
75
-
The extra cost and complexity of pure vector databases can be avoided by using a vector database that is integrated in a highly performant NoSQL or relational database, which stores, indexes, and queries embeddings alongside the corresponding original data. This approach eliminates the extra cost of replicating data in a separate pure vector database. Moreover, keeping the vector embeddings and original data together better facilitates multi-modal data operations, and enables greater data consistency, scale, and performance.
76
+
A fully managed vector database that is integrated in a highly performant NoSQL or relational database avoids the extra cost and complexity of open-source vector databases. Such a database stores, indexes, and queries embeddings alongside the corresponding original data. This approach eliminates the extra cost of replicating data in a separate pure vector database. Moreover, keeping the vector embeddings and original data together better facilitates multi-modal data operations, and enables greater data consistency, scale, and performance. Meanwhile, the fully managed service helps developers avoid the hassles from setting up, maintaining, and relying on community assistance for an open-source vector database. Moreover, some managed vector database services offer a life-time free tier.
76
77
77
-
An example is the Integrated Vector Database in Azure Cosmos DB for MongoDB. It allows developers to enjoy the same financial benefit associated with open-source vector databases, while the service provider handles maintenance, updates, and scalability. When it’s time to scale up operations, upgrading is quick and easy while keeping a low [total cost of ownership (TCO)](introduction.md#low-total-cost-of-ownership-tco).
78
+
An example is the Integrated Vector Database in Azure Cosmos DB for MongoDB. It allows developers to enjoy the same financial benefit associated with open-source vector databases, while the service provider handles maintenance, updates, and scalability. When it’s time to scale up operations, upgrading is quick and easy while keeping a low [total cost of ownership (TCO)](introduction.md#low-total-cost-of-ownership-tco). This service can also be used to conveniently [scale MongoDB](../reimagined.md) applications that are already in production.
78
79
79
80
## Next steps
80
81
> [!div class="nextstepaction"]
81
-
> [Create a lifetime free-tier vCore cluster for Azure Cosmos DB for MongoDB](free-tier.md)
82
+
> [Use lifetime freetier of Integrated Vector Database in Azure Cosmos DB for MongoDB](free-tier.md)
0 commit comments