-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Open
Description
I'm trying to integrate alerts from our system into alertmanager using the v2 API.
When I get an alert I generate the following payload:
{
"annotations": {
"id": "application-error/robot-cluster/s3-client",
"kind": "application",
"severity": "critical",
"alert": "application-error"
},
"labels": {
"application": "s3-client",
"application-version": "undefined",
"application-deployment": "s3-client",
"error-message": "container the-company.s3-client.s3-client-1.s3-client exited",
"alertname": "application-error/robot-cluster/s3-client"
},
"generatorURL": "avassa://application-error/robot-cluster/s3-client",
"startsAt": "2025-09-10T07:30:38.425Z"
}
Note, this is the alert being raised, hence I set startsAt but not endsAt.
A short while later the alert is cleared and I send the following payload
{
"annotations": {
"id": "application-error/robot-cluster/s3-client",
"kind": "application",
"severity": "critical",
"alert": "application-error"
},
"labels": {
"application": "s3-client",
"application-version": "undefined",
"application-deployment": "s3-client",
"alertname": "application-error/robot-cluster/s3-client"
},
"generatorURL": "avassa://application-error/robot-cluster/s3-client",
"endsAt": "2025-09-10T07:30:40.707Z"
}
Note, skipping startsAt but setting endsAt.
The problem is that when I check the alert list in alertmanager:
[
{
"annotations": {
"alert": "application-error",
"id": "application-error/robot-cluster/s3-client",
"kind": "application",
"severity": "critical"
},
"endsAt": "2025-09-10T07:35:38.434Z",
"fingerprint": "2ad165d8fe683c3c",
"receivers": [
{
"name": "web.hook"
}
],
"startsAt": "2025-09-10T07:30:38.425Z",
"status": {
"inhibitedBy": [],
"mutedBy": [],
"silencedBy": [],
"state": "active"
},
"updatedAt": "2025-09-10T07:30:38.434Z",
"generatorURL": "avassa://application-error/robot-cluster/s3-client",
"labels": {
"alertname": "application-error/robot-cluster/s3-client",
"application": "s3-client",
"application-deployment": "s3-client",
"application-version": "undefined",
"error-message": "container the-company.s3-client.s3-client-1.s3-client exited"
}
}
]
Note that endsAt is not updated by my last API call, which makes the alert stayng active too long.
At may very well be that I misunderstood something here, but my impression was that as long as I keep labels consistent, the other fields should be updated by consecutive API calls.
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
In progress