Skip to content

[VC-43793] Add debug roundtripper to discovery and identity clients for easier debugging #683

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 8, 2025

Conversation

wallrj-cyberark
Copy link
Contributor

@wallrj-cyberark wallrj-cyberark commented Aug 7, 2025

Closes: https://venafi.atlassian.net/browse/VC-43793

I want an easy way to see what requests are being made to the CyberArk API endpoints.
The dataupload wrapper already used transport.DebugRoundTripper, but the identity and discovery wrappers didn't.

In future perhaps we need to make all the API wrappers accept an HTTP client and pass in the venafi-connection-lib standard client. That will be in a followup PR.

Testing

Default:

$ go test ./pkg/internal/cyberark/dataupload/... -v -count 1 -run TestPostDataReadingsWithOptionsWithRealAPI
warning: both GOPATH and GOROOT are the same directory (/home/richard/go); see https://go.dev/wiki/InstallTroubleshooting
=== RUN   TestPostDataReadingsWithOptionsWithRealAPI
    identity.go:330: I0807 17:18:01.070011] made successful request to StartAuthentication source="Identity.doStartAuthentication" summary="NewPackage"
    identity.go:446: I0807 17:18:01.655762] successfully completed AdvanceAuthentication request to CyberArk Identity; login complete username="[email protected]"
--- PASS: TestPostDataReadingsWithOptionsWithRealAPI (2.75s)
PASS
ok      github.com/jetstack/preflight/pkg/internal/cyberark/dataupload  2.763s

Level 6: Responses are logged

$ go test ./pkg/internal/cyberark/dataupload/... -v -count 1 -run TestPostDataReadingsWithOptionsWithRealAPI -args -testing.v 6
warning: both GOPATH and GOROOT are the same directory (/home/richard/go); see https://go.dev/wiki/InstallTroubleshooting
=== RUN   TestPostDataReadingsWithOptionsWithRealAPI
    round_trippers.go:632: I0807 17:18:46.372021] Response verb="GET" url="https://platform-discovery.integration-cyberark.cloud/api/v2/services/subdomain/tlskp-test" status="200 OK" milliseconds=249
    round_trippers.go:632: I0807 17:18:46.767918] Response verb="POST" url="https://anb5751.id.integration-cyberark.cloud/Security/StartAuthentication" status="200 OK" milliseconds=395
    identity.go:330: I0807 17:18:46.768129] made successful request to StartAuthentication source="Identity.doStartAuthentication" summary="NewPackage"
    round_trippers.go:632: I0807 17:18:47.265313] Response verb="POST" url="https://anb5751.id.integration-cyberark.cloud/Security/AdvanceAuthentication" status="200 OK" milliseconds=497
    identity.go:446: I0807 17:18:47.265624] successfully completed AdvanceAuthentication request to CyberArk Identity; login complete username="[email protected]"
    round_trippers.go:632: I0807 17:18:47.869198] Response verb="POST" url="https://tlskp-test.inventory.integration-cyberark.cloud/api/ingestions/kubernetes/snapshot-links" status="200 OK" milliseconds=603
    round_trippers.go:632: I0807 17:18:48.353123] Response verb="PUT" url="https://kubernetes-snapshots-marshmallow-a1b2c3d4-integration.s3.amazonaws.com/8f08a102-58ca-49cd-960e-debc5e0d3cd4/bb068932-c80d-460d-88df-34bc7f3f3297/20250807_161847_799.snapshot?AWSAccessKeyId=ASIAR53UX6ITBG7EUJYX&Signature=zmfHXS%2BtfjAydyniuVYzWxgtGmI%3D&x-amz-tagging=agent_version%3Ddevelopment%26uploader_id%3Dbb068932-c80d-460d-88df-34bc7f3f3297%26vendor%3Dk8s%26checksum_sha256%3D7c4d8c99e7863a974de79a03da17c954b29a377b6f1cba60c7f9f569eb537529%26tenant_id%3D8f08a102-58ca-49cd-960e-debc5e0d3cd4%26username%3Drichard_wall%40cyberark.cloud.420375&x-amz-security-token=IQoJb3JpZ2luX2VjEFkaCXVzLWVhc3QtMSJGMEQCIHOt%2BimXGDex0pM1De92vFCdvJM6vCcOIWNdUNtu2SphAiAtZhfDDT9zw%2BXsFPyrL2aVwbsVtPqqJRb9k0jLE1FmdyrcAwiR%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAAaDDEzMjg1MTk1NDIxNCIM9nKMKeF0PycAlYO6KrADVcMNWxn9cBdjJ73C2HBP2uNHlHkXw1bgKTdhU08cfrIHGs8bdvybt%2F9MkzWya0NA96wCjTzxYgh7DilK8F%2FVyS6tqNz9OqQy%2B%2BzDC%2B00aPUkxWd1cjKvwZl7a6mGjUDkBBarZ2C45wn%2FoF09kQxNrHeYdy8DlUQtN%2Bz5YxGqBegXVeIuNA2gYvBCJQDiPIX4kgXgL8wK7H9iOUJBtc0J3JtEH%2FMCVMa%2FLENbB4ZOnEANjjRriicd4wAAFkq%2B9IzQwu1%2FARf8Rs3jQcoFAPaNdmRN%2BZGohrLZL%2BZBqAFy%2BtPZ6Akvnot%2BqodOF3QesmI%2FmItH7Ukw3EnrsK%2FF8JmrAMx%2BuBBGMMxgST4Pbr0oTaOa5SnhSUS4a4xmI5jKQ1TdS0iE4qCfVOQBOZXfZJXO28wNvTjn6JUm7ZfaFpk65APdIScCDpPvTnJ2i6GKqoGmwOmfPgmz%2BaHAwg4QhnOuC43QlRSYYI%2B9ClYoE0c6MiiNbwDkbFn7q1Zv1EDf8dyUAqUUPNWMqbTU67H%2FIT48ZsCqY8aQyMu1VIIpWVxtIZXghc%2BYi0%2B%2F7AEak0ZmiKvDMN%2Bh08QGOp8Bf6MrNf9q0vSucRJbo7Wu0je8bi7rQyW3yvKdtMKgt1%2F8ILSv02DkFH9Xoa0FWRCIqp%2BKvoQG9xhh1ixvsfTD8pQF0XtC88MtNyiedrIX4oBujcGAIfIDWrpJPHA023isZZxnxVV2h8C%2B2%2BQDc6I9DQkjaIS6MYq80y5B4CSp1rjZfMs%2FbYXphQSBLUB9phzw520SCcIUKRen4fE64zCv&Expires=1754583827" status="200 OK" milliseconds=483
--- PASS: TestPostDataReadingsWithOptionsWithRealAPI (2.23s)
PASS
ok      github.com/jetstack/preflight/pkg/internal/cyberark/dataupload  2.247s

Level 7: Request headers are logged, but tokens are masked.

$ go test ./pkg/internal/cyberark/dataupload/... -v -count 1 -run TestPostDataReadingsWithOptionsWithRealAPI -args -testing.v 7
warning: both GOPATH and GOROOT are the same directory (/home/richard/go); see https://go.dev/wiki/InstallTroubleshooting
=== RUN   TestPostDataReadingsWithOptionsWithRealAPI
    round_trippers.go:527: I0807 17:19:42.694642] Request verb="GET" url="https://platform-discovery.integration-cyberark.cloud/api/v2/services/subdomain/tlskp-test" headers=<
                Accept: application/json
                User-Agent: Mozilla/5.0 venafi-kubernetes-agent/development
         >
    round_trippers.go:632: I0807 17:19:42.899730] Response status="200 OK" milliseconds=204
    round_trippers.go:527: I0807 17:19:42.900104] Request verb="POST" url="https://anb5751.id.integration-cyberark.cloud/Security/StartAuthentication" headers=<
                Content-Type: application/json
                User-Agent: Mozilla/5.0 venafi-kubernetes-agent/development
                X-Idap-Native-Client: true
         >
    round_trippers.go:632: I0807 17:19:43.379186] Response status="200 OK" milliseconds=479
    identity.go:330: I0807 17:19:43.379929] made successful request to StartAuthentication source="Identity.doStartAuthentication" summary="NewPackage"
    round_trippers.go:527: I0807 17:19:43.380031] Request verb="POST" url="https://anb5751.id.integration-cyberark.cloud/Security/AdvanceAuthentication" headers=<
                Content-Type: application/json
                User-Agent: Mozilla/5.0 venafi-kubernetes-agent/development
                X-Idap-Native-Client: true
         >
    round_trippers.go:632: I0807 17:19:43.899467] Response status="200 OK" milliseconds=519
    identity.go:446: I0807 17:19:43.900346] successfully completed AdvanceAuthentication request to CyberArk Identity; login complete username="[email protected]"
    round_trippers.go:527: I0807 17:19:43.901228] Request verb="POST" url="https://tlskp-test.inventory.integration-cyberark.cloud/api/ingestions/kubernetes/snapshot-links" headers=<
                Authorization: Bearer <masked>
                Content-Type: application/json
                User-Agent: Mozilla/5.0 venafi-kubernetes-agent/development
         >
    round_trippers.go:632: I0807 17:19:44.583449] Response status="200 OK" milliseconds=682
    round_trippers.go:527: I0807 17:19:44.583961] Request verb="PUT" url="https://kubernetes-snapshots-marshmallow-a1b2c3d4-integration.s3.amazonaws.com/8f08a102-58ca-49cd-960e-debc5e0d3cd4/bb068932-c80d-460d-88df-34bc7f3f3297/20250807_161944_481.snapshot?AWSAccessKeyId=ASIAR53UX6ITBG7EUJYX&Signature=SsmW3mzs2x6w9rAwH1XZX%2FaLj94%3D&x-amz-tagging=agent_version%3Ddevelopment%26uploader_id%3Dbb068932-c80d-460d-88df-34bc7f3f3297%26vendor%3Dk8s%26checksum_sha256%3D7c4d8c99e7863a974de79a03da17c954b29a377b6f1cba60c7f9f569eb537529%26tenant_id%3D8f08a102-58ca-49cd-960e-debc5e0d3cd4%26username%3Drichard_wall%40cyberark.cloud.420375&x-amz-security-token=IQoJb3JpZ2luX2VjEFkaCXVzLWVhc3QtMSJGMEQCIHOt%2BimXGDex0pM1De92vFCdvJM6vCcOIWNdUNtu2SphAiAtZhfDDT9zw%2BXsFPyrL2aVwbsVtPqqJRb9k0jLE1FmdyrcAwiR%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAAaDDEzMjg1MTk1NDIxNCIM9nKMKeF0PycAlYO6KrADVcMNWxn9cBdjJ73C2HBP2uNHlHkXw1bgKTdhU08cfrIHGs8bdvybt%2F9MkzWya0NA96wCjTzxYgh7DilK8F%2FVyS6tqNz9OqQy%2B%2BzDC%2B00aPUkxWd1cjKvwZl7a6mGjUDkBBarZ2C45wn%2FoF09kQxNrHeYdy8DlUQtN%2Bz5YxGqBegXVeIuNA2gYvBCJQDiPIX4kgXgL8wK7H9iOUJBtc0J3JtEH%2FMCVMa%2FLENbB4ZOnEANjjRriicd4wAAFkq%2B9IzQwu1%2FARf8Rs3jQcoFAPaNdmRN%2BZGohrLZL%2BZBqAFy%2BtPZ6Akvnot%2BqodOF3QesmI%2FmItH7Ukw3EnrsK%2FF8JmrAMx%2BuBBGMMxgST4Pbr0oTaOa5SnhSUS4a4xmI5jKQ1TdS0iE4qCfVOQBOZXfZJXO28wNvTjn6JUm7ZfaFpk65APdIScCDpPvTnJ2i6GKqoGmwOmfPgmz%2BaHAwg4QhnOuC43QlRSYYI%2B9ClYoE0c6MiiNbwDkbFn7q1Zv1EDf8dyUAqUUPNWMqbTU67H%2FIT48ZsCqY8aQyMu1VIIpWVxtIZXghc%2BYi0%2B%2F7AEak0ZmiKvDMN%2Bh08QGOp8Bf6MrNf9q0vSucRJbo7Wu0je8bi7rQyW3yvKdtMKgt1%2F8ILSv02DkFH9Xoa0FWRCIqp%2BKvoQG9xhh1ixvsfTD8pQF0XtC88MtNyiedrIX4oBujcGAIfIDWrpJPHA023isZZxnxVV2h8C%2B2%2BQDc6I9DQkjaIS6MYq80y5B4CSp1rjZfMs%2FbYXphQSBLUB9phzw520SCcIUKRen4fE64zCv&Expires=1754583884" headers=<
                User-Agent: Mozilla/5.0 venafi-kubernetes-agent/development
         >
    round_trippers.go:632: I0807 17:19:45.066679] Response status="200 OK" milliseconds=482
--- PASS: TestPostDataReadingsWithOptionsWithRealAPI (2.37s)
PASS
ok      github.com/jetstack/preflight/pkg/internal/cyberark/dataupload  2.420s

@wallrj-cyberark wallrj-cyberark changed the title Add debug roundtripper to discovery and identity clients for easier debugging [VC-43793] Add debug roundtripper to discovery and identity clients for easier debugging Aug 7, 2025
@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-inventory-api-debug-roundtrippers branch from f13784c to 8cefefd Compare August 7, 2025 16:25
@@ -46,7 +46,7 @@ func NewCyberArkClient(trustedCAs *x509.CertPool, baseURL string, authenticateRe
if trustedCAs != nil {
tr.TLSClientConfig.RootCAs = trustedCAs
}
cyberClient.Transport = transport.DebugWrappers(tr)
cyberClient.Transport = transport.NewDebuggingRoundTripper(tr, transport.DebugByContext)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The DebugWrappers function is better in a way because it only wraps the transport if the global logger has been configured at level >=6....which avoids the inefficiency of looking up the log level for each request if the log level is at its default level.
But it makes it harder to control the logging verbosity from tests because the wrapper doesn't know about the verbosity setting on the ktesting logger.
I might revert back to DebugWrapper in future, but for now this makes it easier for me to see what's going on in the test logs.

@@ -12,6 +12,7 @@ import (
"github.com/stretchr/testify/require"
"k8s.io/klog/v2"
"k8s.io/klog/v2/ktesting"
_ "k8s.io/klog/v2/ktesting/init"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This registers the -testing.v and -testing.vmodule command line args for the tests.

// To enable verbose request logging:
//
// go test ./pkg/internal/cyberark/dataupload/... \
// -v -count 1 -run TestPostDataReadingsWithOptionsWithRealAPI -args -testing.v 6
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pasted some examples in the PR description.

@@ -137,7 +143,7 @@ func TestPostDataReadingsWithOptionsWithRealAPI(t *testing.T) {
return
}

logger := ktesting.NewLogger(t, ktesting.NewConfig())
logger := ktesting.NewLogger(t, ktesting.DefaultConfig)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DefaultConfig has the flag names which are registered by importing the init package (above)

@@ -165,7 +171,7 @@ func TestPostDataReadingsWithOptionsWithRealAPI(t *testing.T) {
cyberArkClient, err := dataupload.NewCyberArkClient(nil, serviceURL, identityClient.AuthenticateRequest)
require.NoError(t, err)

err = cyberArkClient.PostDataReadingsWithOptions(t.Context(), api.DataReadingsPost{}, dataupload.Options{
err = cyberArkClient.PostDataReadingsWithOptions(ctx, api.DataReadingsPost{}, dataupload.Options{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a mistake I made in #681

@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-inventory-api-debug-roundtrippers branch from 8cefefd to 102e566 Compare August 7, 2025 16:36
@@ -17,6 +17,8 @@ import (
"github.com/jetstack/preflight/pkg/internal/cyberark/dataupload"
"github.com/jetstack/preflight/pkg/internal/cyberark/identity"
"github.com/jetstack/preflight/pkg/internal/cyberark/servicediscovery"

_ "k8s.io/klog/v2/ktesting/init"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sets up the -testing.v and -testing.vmodule command line flags for the test.

@wallrj-cyberark wallrj-cyberark marked this pull request as ready for review August 7, 2025 16:38
Copy link
Contributor

@SgtCoDFish SgtCoDFish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

I've not dug too deep but this looks like a clear improvement for debugging, cheers!

@wallrj-cyberark wallrj-cyberark merged commit 4196ed9 into master Aug 8, 2025
2 checks passed
@wallrj-cyberark wallrj-cyberark deleted the VC-43403-inventory-api-debug-roundtrippers branch August 8, 2025 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants