Add support for synthetics global parameters #1155

agunnerson-elastic · 2025-06-04T22:30:27Z

Add support for synthetics global parameters

This adds support for creating synthetics global parameters, which are
used to store secrets used for synthetics monitors.

This implementation is based on what we already do for synthetics
private locations. This is mostly just a simple wrapper around API calls, except we currently requery the resource after creating it or updating it. This is because Kibana (at least in 9.0.0) has some unexpected responses for /api/synthetics/params:

The POST request to create a parameter does not include the value field in the response.
The PUT request to update a parameter returns the new values for all fields, except for value, which contains the old secret.
None of the responses include the share_across_spaces value, which can only be inferred by checking of namespaces is ["*"].

We also use the undocumented DELETE /api/synthetics/params API that the Kibana UI uses due to many quirks with the deletion APIs in various versions of Kibana. I tested every single version of Kibana since the parameter APIs were introduced (8.12.0) and this one is the only one that works on all versions.

DELETE /api/synthetics/params/<id> (from documentation)
- Works on >=8.17.0.
- HTTP 404 on 8.12.x through 8.16.x.
DELETE /api/synthetics/params/_bulk_delete (from documentation)
- HTTP 400 on 9.0.x with error message about URL parameters and body provided at the same time (_bulk_delete interpreted as parameter ID?).
- HTTP 403 on 8.18.x with error message about missing uptime-read and uptime-write permissions despite having the superuser role.
- HTTP 403 with no details on 8.17.x.
- HTTP 404 on 8.12.x through 8.16.x.
POST /api/synthetics/params/_bulk_delete (from comment in documentation example)
- Works on >=8.17.0.
- HTTP 404 on 8.12.x through 8.16.x.
DELETE /api/synthetics/params (what the Kibana UI does)
- Works on >=8.12.0.

Finally, this updates the Github Actions workflow to run the tests using HTTP basic auth instead of API keys. On all versions of Kibana, the params endpoint fails to return the current value when using API keys, even if permissions are not restricted for the key. Explicitly granting the key all permissions to Kibana does not work either.

Fixes: #1152

agunnerson-elastic · 2025-06-04T22:41:27Z

Hmm, looking into the CI failures

This adds support for creating synthetics global parameters, which are used to store secrets used for synthetics monitors. This implementation is based on what we already do for synthetics private locations. This is mostly just a simple wrapper around API calls, except we currently requery the resource after creating it or updating it. This is because Kibana (at least in 9.0.0) has some unexpected responses for /api/synthetics/params: 1. The POST request to create a parameter does not include the `value` field in the response. 2. The PUT request to update a parameter returns the new values for all fields, except for `value`, which contains the old secret. 3. None of the responses include the `share_across_spaces` value, which can only be inferred by checking of `namespaces` is `["*"]`. We also use the undocumented `DELETE /api/synthetics/params` API that the Kibana UI uses due to many quirks with the deletion APIs in various versions of Kibana. I tested every single version of Kibana since the parameter APIs were introduced (8.12.0) and this one is the only one that works on all versions. * `DELETE /api/synthetics/params/<id>` (from documentation) - Works on >=8.17.0. - HTTP 404 on 8.12.x through 8.16.x. * `DELETE /api/synthetics/params/_bulk_delete` (from documentation) - HTTP 400 on 9.0.x with error message about URL parameters and body provided at the same time (`_bulk_delete` interpreted as parameter ID?). - HTTP 403 on 8.18.x with error message about missing `uptime-read` and `uptime-write` permissions despite having the `superuser` role. - HTTP 403 with no details on 8.17.x. - HTTP 404 on 8.12.x through 8.16.x. * `POST /api/synthetics/params/_bulk_delete` (from comment in documentation example) - Works on >=8.17.0. - HTTP 404 on 8.12.x through 8.16.x. * `DELETE /api/synthetics/params` (what the Kibana UI does) - Works on >=8.12.0. Finally, this updates the Github Actions workflow to run the tests using HTTP basic auth instead of API keys. On all versions of Kibana, the params endpoint fails to return the current value when using API keys, even if permissions are not restricted for the key. Explicitly granting the key all permissions to Kibana does not work either. Fixes: elastic#1152 Signed-off-by: Andrew Gunnerson <[email protected]>

agunnerson-elastic · 2025-06-05T02:29:59Z

CI is fixed! Turns out there are a lot of quirks with the deletion APIs. I tested all 39 Kibana versions between 8.12.0 and 9.0.2 and ended up using DELETE /api/synthetics/params, which works on all of them. Updated the PR description with more details.

tobio · 2025-06-05T19:53:41Z

Have we opened issues in the Kibana repo to cover these problems?

agunnerson-elastic · 2025-06-05T20:14:46Z

Have we opened issues in the Kibana repo to cover these problems?

Hey @tobio! Yep, I plan to do so, but haven't yet.

tobio

This is great! Thanks for adding this in, a couple of small comments but it would be great if we can base this off the open api client if possible.

internal/kibana/synthetics/parameter/create.go

internal/kibana/synthetics/parameter/read.go

tobio · 2025-06-06T01:03:18Z

libs/go-kibana-rest/kbapi/api._.go

+			Parameter: &KibanaSyntheticsParameterAPI{
+				Add:    newKibanaSyntheticsParameterAddFunc(c),
+				Delete: newKibanaSyntheticsParameterDeleteFunc(c),
+				Get:    newKibanaSyntheticsParameterGetFunc(c),
+				Update: newKibanaSyntheticsParameterUpdateFunc(c),
+			},


We're trying to move everything to a client generated from the Kibana openapi spec. It's not as smooth as I'd like at the moment, but I've done a bit of the work to get started in this commit. Would you be able to migrate this PR over to that client instead of updating this one?

Given the delete endpoint you're using here is undocumented we may need to leave that code in here, but at least for get/post/put we should be able to use the generated client.

Thanks for starting the work on that! I'll give it a shot.

I just pushed c7d42a3. The synthetics parameters stuff is working, but it looks like the fleet tests are failing. The only place I'm seeing any changes related to fleet is in generated/kbapi/kibana.gen.go, but I'm not sure why. It seems like the OAPI changes are additive. Any idea what might be going on?

Sorry for taking ages here. The failures in this PR are all fixed by #1189. We should just be able to merge main when it's fixed.

…update Signed-off-by: Andrew Gunnerson <[email protected]>

Co-Authored-By: Toby Brain <[email protected]> Signed-off-by: Andrew Gunnerson <[email protected]>

* origin/main: (23 commits) Regenerate Kibana client (elastic#1189) Addressing feedback on elastic#1178 (elastic#1180) Bump github.com/getkin/kin-openapi from 0.127.0 to 0.131.0 (elastic#1190) Remove explicit misspell (elastic#1182) Move tools to go.mod managed tools (elastic#1185) fix(deps): update module github.com/hashicorp/terraform-plugin-docs to v0.22.0 (elastic#1188) Bump github.com/cloudflare/circl from 1.6.0 to 1.6.1 in /tools (elastic#1187) Migrate the System User resource to the plugin framework. (elastic#1154) Bump github.com/go-viper/mapstructure/v2 from 2.2.1 to 2.3.0 in /tools (elastic#1184) Bump golang.org/x/net from 0.36.0 to 0.38.0 in /libs/go-kibana-rest (elastic#1183) chore(deps): update golang:1.24.4 docker digest to 20a022e (elastic#1177) Bump github.com/getkin/kin-openapi from 0.127.0 to 0.131.0 in /tools (elastic#1181) Add basic docs around the current kbapi generation (elastic#1178) chore(deps): update module github.com/golangci/golangci-lint to v2.2.1 (elastic#1176) chore(deps): update docker.elastic.co/elasticsearch/elasticsearch docker tag to v9.0.3 (elastic#1170) chore(deps): update docker.elastic.co/kibana/kibana docker tag to v9.0.3 (elastic#1171) chore(deps): update dependency go to v1.24.4 (elastic#1159) fix(deps): update module github.com/elastic/go-elasticsearch/v8 to v8.18.1 (elastic#1168) chore(deps): update golang docker tag to v1.24.4 (elastic#1161) fix(deps): update module github.com/goreleaser/goreleaser/v2 to v2.10.2 (elastic#1162) ...

* origin/main: Add description to kibana role resource (elastic#1186)

agunnerson-elastic force-pushed the synthetics-params branch from 57d4846 to 0f147fb Compare June 5, 2025 02:20

agunnerson-elastic self-assigned this Jun 5, 2025

tobio reviewed Jun 6, 2025

View reviewed changes

agunnerson-elastic and others added 5 commits June 6, 2025 17:34

synthetics/parameter: Remove duplicate code when rereading in create/…

13df2f0

…update Signed-off-by: Andrew Gunnerson <[email protected]>

synthetics/parameter: Switch to using OAPI client

c7d42a3

Co-Authored-By: Toby Brain <[email protected]> Signed-off-by: Andrew Gunnerson <[email protected]>

Merge remote-tracking branch 'origin/main' into synthetics-params

0c81847

* origin/main: Add description to kibana role resource (elastic#1186)

Changelog

49c82dd

tobio approved these changes Jul 8, 2025

View reviewed changes

tobio merged commit aafc5a1 into elastic:main Jul 8, 2025
23 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add support for synthetics global parameters #1155

Add support for synthetics global parameters #1155

agunnerson-elastic commented Jun 4, 2025 •

edited

Loading

Uh oh!

agunnerson-elastic commented Jun 4, 2025

Uh oh!

agunnerson-elastic commented Jun 5, 2025

Uh oh!

tobio commented Jun 5, 2025

Uh oh!

agunnerson-elastic commented Jun 5, 2025

Uh oh!

tobio left a comment

Uh oh!

Uh oh!

Uh oh!

tobio Jun 6, 2025 •

edited

Loading

Uh oh!

agunnerson-elastic Jun 6, 2025

Uh oh!

agunnerson-elastic Jun 7, 2025

Uh oh!

tobio Jul 8, 2025

Uh oh!

agunnerson-elastic Jul 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add support for synthetics global parameters #1155

Add support for synthetics global parameters #1155

Conversation

agunnerson-elastic commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agunnerson-elastic commented Jun 4, 2025

Uh oh!

agunnerson-elastic commented Jun 5, 2025

Uh oh!

tobio commented Jun 5, 2025

Uh oh!

agunnerson-elastic commented Jun 5, 2025

Uh oh!

tobio left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tobio Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agunnerson-elastic Jun 6, 2025

Choose a reason for hiding this comment

Uh oh!

agunnerson-elastic Jun 7, 2025

Choose a reason for hiding this comment

Uh oh!

tobio Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

agunnerson-elastic Jul 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

agunnerson-elastic commented Jun 4, 2025 •

edited

Loading

tobio Jun 6, 2025 •

edited

Loading