Skip to content

Commit e1fd50b

Browse files
authored
Merge pull request #272117 from MicrosoftDocs/main
Publish to live, Sunday 4:00PM PDT, 04/14
2 parents 69ff432 + d453791 commit e1fd50b

File tree

8 files changed

+272
-268
lines changed

8 files changed

+272
-268
lines changed

articles/azure-monitor/agents/agents-overview.md

Lines changed: 116 additions & 136 deletions
Large diffs are not rendered by default.

articles/azure-monitor/agents/azure-monitor-agent-extension-versions.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ We strongly recommended to always update to the latest version, or opt in to the
2323
## Version details
2424
| Release Date | Release notes | Windows | Linux |
2525
|:---|:---|:---|:---|
26+
| March 2024 | **Widows**<ul><li>**Breaking Change from Publict Preview to GA** Due to customer feedback, automatic parsing of JSON into column in your custom table in Log Analytic was added. You must take action to migrate your JSON DCR created prior to this release to prevent data loss. This is the last release of the JSON Log type in Public Preview an GA will be declared in a few weeks.</li><li>Fix AMA when resource ID contains non-ascii chars which is common when using some languages other than English. Errors would follow this pattern: … [HealthServiceCommon] [] [Error] … WinHttpAddRequestHeaders(x-ms-AzureResourceId: /subscriptions/{your subscription #} /resourceGroups/???????/providers/ … PostDataItems" failed with code 87(ERROR_INVALID_PARAMETER) </li></ul> | 1.25.0 | Comming Soon |
2627
| February 2024 | **Known Issues**<ul><li>Occasional crash during startup in arm64 VMs. This is fixed in 1.30.3</li></uL>**Windows**<ul><li>Fix memory leak in Internet Information Service (IIS) log collection</li><li>Fix JSON parsing with Unicode characters for some ingestion endpoints</li><li>Allow Client installer to run on Azure Virtual Desktop (AVD) DevBox partner</li><li>Enable Transport Layer Security (TLS) 1.3 on supported Windows versions</li><li>Update MetricsExtension package to 2.2024.202.2043</li></ul>**Linux**<ul><li>Features<ul><li>Add EventTime to syslog for parity with OMS agent</li><li>Add more Common Event Format (CEF) format support</li><li>Add CPU quotas for Azure Monitor Agent (AMA)</li></ul><li>Fixes<ul><li>Handle truncation of large messages in syslog due to Transmission Control Protocol (TCP) framing issue</li><li>Set NO_PROXY for Instance Metadata Service (IMDS) endpoint in AMA Python wrapper</li><li>Fix a crash in syslog parsing</li><li>Add reasonable limits for metadata retries from IMDS</li><li>No longer reset /var/log/azure folder permissions</li></ul></ul> | 1.24.0 | 1.30.3<br>1.30.2 |
2728
| January 2024 |**Known Issues**<ul><li>1.29.5 doesn't install on Arc-enabled servers because the agent extension code size is beyond the deployment limit set by Arc. **This issue was fixed in 1.29.6**</li></ul>**Windows**<ul><li>Added support for Transport Layer Security (TLS) 1.3</li><li>Reverted a change to enable multiple IIS subscriptions to use same filter. Feature is redeployed once memory leak is fixed</li><li>Improved Event Trace for Windows (ETW) event throughput rate</li></ul>**Linux**<ul><li>Fix error messages logged, intended for mdsd.err, that instead went to mdsd.warn in 1.29.4 only. Likely error messages: "Exception while uploading to Gig-LA: ...", "Exception while uploading to ODS: ...", "Failed to upload to ODS: ..."</li><li>Reduced noise generated by AMAs' use of semanage when SELinux is enabled</li><li>Handle time parsing in syslog to handle Daylight Savings Time (DST) and leap day</li></ul> | 1.23.0 | 1.29.5, 1.29.6 |
2829
| December 2023 |**Known Issues**<ul><li>1.29.4 doesn't install on Arc-enabled servers because the agent extension code size is beyond the deployment limit set by Arc. Fix is coming in 1.29.6</li><li>Multiple IIS subscriptions cause a memory leak. feature reverted in 1.23.0</ul>**Windows** <ul><li>Prevent CPU spikes by not using bookmark when resetting an Event Log subscription</li><li>Added missing Fluent Bit executable to AMA client setup for Custom Log support</li><li>Updated to latest AzureCredentialsManagementService and DsmsCredentialsManagement package</li><li>Update ME to v2.2023.1027.1417</li></ul>**Linux**<ul><li>Support for TLS v1.3</li><li>Support for nopri in Syslog</li><li>Ability to set disk quota from Data Collection Rule (DCR) Agent Settings</li><li>Add ARM64 Ubuntu 22 support</li><li>**Fixes**<ul><li>SysLog</li><ul><li>Parse syslog Palo Alto CEF with multiple space characters following the hostname</li><li>Fix an issue with incorrectly parsing messages containing two '\n' chars in a row</li><li>Improved support for non-RFC compliant devices</li><li>Support Infoblox device messages containing both hostname and IP headers</li></ul><li>Fix AMA crash in Read Hat Enterprise Linux (RHEL) 7.2</li><li>Remove dependency on "which" command</li><li>Fix port conflicts due to AMA using 13000 </li><li>Reliability and Performance improvements</li></ul></li></ul>| 1.22.0 | 1.29.4|

articles/azure-monitor/agents/data-collection-text-log.md

Lines changed: 94 additions & 95 deletions
Large diffs are not rendered by default.

articles/azure-monitor/app/availability-azure-functions.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -206,6 +206,25 @@ To create a new file, right-click under your timer trigger function (for example
206206

207207
```
208208

209+
### Multi-Step Web Test Code Sample
210+
Follow the same instructions above and instead paste the following code into the **runAvailabilityTest.csx** file:
211+
212+
```csharp
213+
using System.Net.Http;
214+
215+
public async static Task RunAvailabilityTestAsync(ILogger log)
216+
{
217+
using (var httpClient = new HttpClient())
218+
{
219+
// TODO: Replace with your business logic
220+
await httpClient.GetStringAsync("https://www.bing.com/");
221+
222+
// TODO: Replace with your business logic for an additional monitored endpoint, and logic for additional steps as needed
223+
await httpClient.GetStringAsync("https://www.learn.microsoft.com/");
224+
}
225+
}
226+
```
227+
209228
## Next steps
210229

211230
* [Standard tests](availability-standard-tests.md)

articles/azure-monitor/toc.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -727,8 +727,8 @@ items:
727727
href: agents/data-collection-syslog.md
728728
- name: Configure a syslog forwarder
729729
href: ../sentinel/forward-syslog-monitor-agent.md?bc=%2fazure%2fazure-monitor%2fbreadcrumb%2ftoc.json&toc=%2fazure%2fazure-monitor%2ftoc.json
730-
- name: Send data to Event Hubs and Storage
731-
href: agents/azure-monitor-agent-send-data-to-event-hubs-and-storage.md
730+
- name: Log Analytics gateway
731+
href: agents/gateway.md
732732
- name: Define network settings
733733
href: agents/azure-monitor-agent-data-collection-endpoint.md
734734
- name: Configure Private Link for Azure Monitor Agent
@@ -763,8 +763,6 @@ items:
763763
href: ../virtual-machines/extensions/oms-windows.md?toc=/azure/azure-monitor/toc.json
764764
- name: VM Extension for Linux
765765
href: ../virtual-machines/extensions/oms-linux.md?toc=/azure/azure-monitor/toc.json
766-
- name: Log Analytics gateway
767-
href: agents/gateway.md
768766
- name: LA Agent Management
769767
href: agents/agent-manage.md
770768
- name: LA Agent Health
@@ -799,6 +797,8 @@ items:
799797
items:
800798
- name: Overview
801799
href: agents/diagnostics-extension-overview.md
800+
- name: Send data to Event Hubs and Storage
801+
href: agents/azure-monitor-agent-send-data-to-event-hubs-and-storage.md
802802
- name: Windows
803803
displayName: diagnostics extension
804804
items:
@@ -1591,4 +1591,4 @@ items:
15911591
- name: Pricing calculator
15921592
href: https://azure.microsoft.com/pricing/calculator/
15931593
- name: Regional availability
1594-
href: https://azure.microsoft.com/global-infrastructure/services/?regions=all&products=monitor
1594+
href: https://azure.microsoft.com/global-infrastructure/services/?regions=all&products=monitor

articles/cosmos-db/mongodb/vcore/vector-search-ai.md

Lines changed: 16 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,9 @@ Another advantage of open-source vector databases is the strong community suppor
2121

2222
Some individuals opt for open-source vector databases because they are "free," meaning there's no cost to acquire or use the software. An alternative is using the free tiers offered by managed vector database services. These managed services provide not only cost-free access up to a certain usage limit but also simplify the operational burden by handling maintenance, updates, and scalability. Therefore, by using the free tier of managed vector database services, users can achieve cost savings while reducing management overhead. This approach allows users to focus more on their core activities rather than on database administration.
2323

24-
## Working mechanism of open-source vector databases
24+
## Working mechanism of vector databases
2525

26-
Open-source vector databases are designed to store and manage vector embeddings, which are mathematical representations of data in a high-dimensional space. In this space, each dimension corresponds to a feature of the data, and tens of thousands of dimensions might be used to represent sophisticated data. A vector's position in this space represents its characteristics. Words, phrases, or entire documents, and images, audio, and other types of data can all be vectorized. These vector embeddings are used in similarity search, multi-modal search, recommendations engines, large languages models (LLMs), etc.
26+
Vector databases are designed to store and manage vector embeddings, which are mathematical representations of data in a high-dimensional space. In this space, each dimension corresponds to a feature of the data, and tens of thousands of dimensions might be used to represent sophisticated data. A vector's position in this space represents its characteristics. Words, phrases, or entire documents, and images, audio, and other types of data can all be vectorized. These vector embeddings are used in similarity search, multi-modal search, recommendations engines, large languages models (LLMs), etc.
2727

2828
These databases' architecture typically includes a storage engine and an indexing mechanism. The storage engine optimizes the storage of vector data for efficient retrieval and manipulation, while the indexing mechanism organizes the data for fast searching and retrieval operations.
2929

@@ -40,6 +40,14 @@ Vector databases are used in numerous domains and situations across analytical a
4040
- Implement persistent memory for AI agents
4141
- Enable retrieval-augmented generation (RAG)
4242

43+
### Integrated vector database vs pure vector database
44+
45+
There are two common types of vector database implementations - pure vector database and integrated vector database in a NoSQL or relational database.
46+
47+
A pure vector database is designed to efficiently store and manage vector embeddings, along with a small amount of metadata; it is separate from the data source from which the embeddings are derived.
48+
49+
A vector database that is integrated in a highly performant NoSQL or relational database provides additional capabilities. The integrated vector database in a NoSQL or relational database can store, index, and query embeddings alongside the corresponding original data. This approach eliminates the extra cost of replicating data in a separate pure vector database. Moreover, keeping the vector embeddings and original data together better facilitates multi-modal data operations, and enables greater data consistency, scale, and performance.
50+
4351
## Selecting the best open-source vector database
4452

4553
Choosing the best open-source vector database requires considering several factors. Performance and scalability of the database are crucial, as they impact whether the database can handle your specific workload requirements. Databases with efficient indexing and querying capabilities usually offer optimal performance. Another factor is the community support and documentation available for the database. A robust community and ample documentation can provide valuable assistance. Here are some popular open-source vector databases:
@@ -49,33 +57,26 @@ Choosing the best open-source vector database requires considering several facto
4957
- Qdrant
5058
- Weaviate
5159

52-
>[!NOTE]
53-
>The most popular option may not be the best option for you. To find the best fit for your needs, you should compare different options based on features, supported data types, compatibility with existing tools and frameworks you use. Ease of installation, configuration, and maintenance should also be considered to ensure smooth integration into your workflow.
60+
However, the most popular option may not be the best option for you. Thus, you should compare different options based on features, supported data types, compatibility with existing tools and frameworks you use. You should also keep in mind the challenges of open-source vector databases (below).
5461

5562
## Challenges of open-source vector databases
5663

57-
Open-source vector databases pose challenges that are typical of open-source software and ones that are specific to many vector databases.
64+
Most open-source vector databases, including the ones listed above, are pure vector databases. In other words, they are designed to store and manage vector embeddings only, along with a small amount of metadata. Since they are independent of the data source from which the embeddings are derived, using them requires sending your data between service integrations, which adds extra cost, complexity, and bottlenecks for your production workloads.
5865

59-
### Challenges with open source:
66+
They also pose the challenges that are typical of open-source databases:
6067

6168
- Setup: Users need in-depth knowledge to install, configure, and operate, especially for complex deployments. Optimizing resources and configuration while scaling up operation requires close monitoring and adjustments.
6269
- Maintenance: Users must manage their own updates, patches, and maintenance. Thus, ML expertise wouldn't suffice; users must also have extensive experience in database administration.
6370
- Support: Official support can be limited compared to managed services, relying more on community assistance.
6471

6572
Therefore, while free initially, open-source vector databases incur significant costs when scaling up. Expanding operations necessitates more hardware, skilled IT staff, and advanced infrastructure management, leading to higher expenses in hardware, personnel, and operational costs. Scaling open-source vector databases can be financially demanding despite the lack of licensing fees.
6673

67-
### Challenges that are specific to vector databases:
68-
69-
Most open-source vector databases nowadays are pure vector databases. In other words, they are designed to store and manage vector embeddings, along with a small amount of metadata; it is separate from the data source from which the embeddings are derived. Thus, using pure vector databases requires sending your data between service integrations, which adds extra cost, complexity, and bottlenecks for your production workloads.
70-
7174
## Addressing the challenges of open-source vector databases
7275

73-
A fully managed database service helps developers avoid the hassles from setting up, maintaining, and relying on community assistance for an open-source vector database; moreover, some managed vector database services offer a life-time free tier.
74-
75-
The extra cost and complexity of pure vector databases can be avoided by using a vector database that is integrated in a highly performant NoSQL or relational database, which stores, indexes, and queries embeddings alongside the corresponding original data. This approach eliminates the extra cost of replicating data in a separate pure vector database. Moreover, keeping the vector embeddings and original data together better facilitates multi-modal data operations, and enables greater data consistency, scale, and performance.
76+
A fully managed vector database that is integrated in a highly performant NoSQL or relational database avoids the extra cost and complexity of open-source vector databases. Such a database stores, indexes, and queries embeddings alongside the corresponding original data. This approach eliminates the extra cost of replicating data in a separate pure vector database. Moreover, keeping the vector embeddings and original data together better facilitates multi-modal data operations, and enables greater data consistency, scale, and performance. Meanwhile, the fully managed service helps developers avoid the hassles from setting up, maintaining, and relying on community assistance for an open-source vector database. Moreover, some managed vector database services offer a life-time free tier.
7677

77-
An example is the Integrated Vector Database in Azure Cosmos DB for MongoDB. It allows developers to enjoy the same financial benefit associated with open-source vector databases, while the service provider handles maintenance, updates, and scalability. When it’s time to scale up operations, upgrading is quick and easy while keeping a low [total cost of ownership (TCO)](introduction.md#low-total-cost-of-ownership-tco).
78+
An example is the Integrated Vector Database in Azure Cosmos DB for MongoDB. It allows developers to enjoy the same financial benefit associated with open-source vector databases, while the service provider handles maintenance, updates, and scalability. When it’s time to scale up operations, upgrading is quick and easy while keeping a low [total cost of ownership (TCO)](introduction.md#low-total-cost-of-ownership-tco). This service can also be used to conveniently [scale MongoDB](../reimagined.md) applications that are already in production.
7879

7980
## Next steps
8081
> [!div class="nextstepaction"]
81-
> [Create a lifetime free-tier vCore cluster for Azure Cosmos DB for MongoDB](free-tier.md)
82+
> [Use lifetime free tier of Integrated Vector Database in Azure Cosmos DB for MongoDB](free-tier.md)

0 commit comments

Comments
 (0)