discovery: Add AWS EKS audit log fetching for Access Graph by camscale · Pull Request #59568 · gravitational/teleport

camscale · 2025-09-25T03:06:28Z

Extend the static config for Access Graph discovery to be able to
specify the EKS cluster for which apiserver audit logs should be fetched
and sent to Access Graph.

Extend the Access Graph discovery resource syncer to identify clusters
that should have their apiserver audit logs fetched based on discovery
configuration. Send the list of clusters over a channel for a log
fetcher to receive so it can fetch the logs and send them to access
graph.

Add a watcher to start fetchers for all access graph EKS clusters that
are configured to have Kubernetes apiserver audit logs fetched and send
them to access graph. This receives the list of clusters from the resource
discovery syncer and reconciles it against the clusters currently having
their logs fetched, starting and stopping logs fetchers as required.

Issue: https://github.com/gravitational/access-graph/issues/1589

Note: This PR is split into logical commits that can be reviewed separately for
easier reviewing.

juliaogris

LGTM

lib/srv/discovery/fetchers/aws-sync/aws-sync.go

lib/config/fileconf.go

lib/srv/discovery/access_graph_aws.go

lib/srv/discovery/fetchers/aws-sync/eks.go

lib/srv/discovery/eks_audit_log_fetcher.go

lib/srv/discovery/eks_audit_log_watcher.go

lib/srv/discovery/eks_audit_log_fetcher.go

juliaogris

Still good, just a couple of little nits.

lib/srv/discovery/eks_audit_log_fetcher.go

lib/srv/discovery/eks_audit_log_watcher.go

lib/config/fileconf.go

lib/srv/discovery/fetchers/aws-sync/aws-sync.go

lib/srv/discovery/fetchers/aws-sync/cloudwatchlogs.go

lib/srv/discovery/fetchers/aws-sync/eks.go

Extend the static config for Access Graph discovery to be able to specify the EKS cluster for which apiserver audit logs should be fetched and sent to Access Graph.

Add a watcher to start fetchers for all access graph EKS clusters that are configured to have Kubernetes apiserver audit logs fetched and send them to access graph. It receives the set of clusters to fetch audit logs for from the AWS resource syncer as it discovers EKS clusters. Those clusters are reconciled against the current set of log fetchers, with no-longer-needed fetchers stopped and new fetchers started as needed. This commit requires go.mod be updated with: go get github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs@latest It is left out of this commit for now as it makes rebasing/merging master easier.

Run: go get github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs@latest make go-mod-tidy-all # Manually move the go.mod line back to the first section!?!? This commit is kept separate for easier merging/rebasing.

r0mant

Can we get some test coverage going here?

Refactor the eksAuditLog{Watcher,Fetcher} and the aws_sync.Fetcher cloudwatchlogs to be more testable: * factor away eksAuditLogFetcher from eksAuditLogWatcher. The watcher just needs a factory function to create a fetcher, and all the watcher needs from that fetcher is a `Run()` method. Lift the cancel func out of the watcher and store it directly in the watcher, as only the watcher uses it. * factor away aws_sync.Fetcher from eksAuditLogFetcher. All it needs from the sync fetcher it calls is one method to fetch cloudwatch logs. Make that an interface and use just that. This allows a fake source of cloudwatch logs to be provided for testing. While here, use protobuf getters rather than accessing fields directly. * Use protobuf getters in aws_sync.Fetcher cloudwatchlogs instead of accessing fields directly. In future, we could pass in an interface with those getters to make the code more testable.

Add tests for `eksAuditLogWatcher` and `eksAuditLogFetcher`. Copy the grpc stream testing util from the access graph repo into teleport as it is useful for the bidirectional streaming methods uses by access graph, and makes it easier to test on the client side.

camscale · 2025-11-11T02:34:46Z

Can we get some test coverage going here?

@r0mant PTAL. I've added tests for the watcher and fetcher. There are not many existing tests around the access graph sync stuff which really feels like it needs a bit of refactoring to be more testable. The overlap between Teleport discovery and access graph discovery also makes this all a bit confusing, but I feel cleaning this up is too much for this PR.

camscale requested review from juliaogris, r0mant and tigrato September 25, 2025 03:06

camscale force-pushed the camh/tag/kube-audit-logs-protos branch from 94a6def to 225048a Compare September 25, 2025 04:30

camscale force-pushed the camh/tag/kube-audit-logs branch 4 times, most recently from 53c7cf9 to e981c38 Compare September 25, 2025 07:01

camscale force-pushed the camh/tag/kube-audit-logs branch from e981c38 to 46ec433 Compare October 9, 2025 04:54

camscale force-pushed the camh/tag/kube-audit-logs-protos branch from 225048a to ffbb3b3 Compare October 14, 2025 07:48

camscale force-pushed the camh/tag/kube-audit-logs branch 4 times, most recently from 8141b3e to 3e19084 Compare October 15, 2025 03:31

camscale added the no-changelog Indicates that a PR does not require a changelog entry label Oct 15, 2025

camscale force-pushed the camh/tag/kube-audit-logs branch 2 times, most recently from 7fde12d to a492e96 Compare October 15, 2025 07:34

camscale force-pushed the camh/tag/kube-audit-logs-protos branch 2 times, most recently from 15db093 to eca596e Compare October 16, 2025 03:36

camscale force-pushed the camh/tag/kube-audit-logs branch 2 times, most recently from c359b0f to 3eba24d Compare October 16, 2025 03:37

camscale marked this pull request as ready for review October 16, 2025 03:38

camscale requested review from hugoShaka, rosstimothy, russjones, strideynet and zmb3 as code owners October 16, 2025 03:38

github-actions bot requested a review from marcoandredinis October 16, 2025 03:39

github-actions bot added discovery size/lg labels Oct 16, 2025

Base automatically changed from camh/tag/kube-audit-logs-protos to master October 17, 2025 01:26

camscale force-pushed the camh/tag/kube-audit-logs branch from c060d92 to 18680a9 Compare October 17, 2025 01:33

juliaogris approved these changes Oct 20, 2025

View reviewed changes

juliaogris approved these changes Oct 21, 2025

View reviewed changes

lib/srv/discovery/eks_audit_log_fetcher.go Outdated Show resolved Hide resolved

lib/srv/discovery/eks_audit_log_watcher.go Outdated Show resolved Hide resolved

tigrato reviewed Oct 27, 2025

View reviewed changes

camscale force-pushed the camh/tag/kube-audit-logs branch 4 times, most recently from 6c56bd5 to 9cc258f Compare October 28, 2025 08:26

tigrato approved these changes Nov 4, 2025

View reviewed changes

public-teleport-github-review-bot bot removed the request for review from r0mant November 4, 2025 21:46

camscale added 2 commits November 5, 2025 15:46

lib/config: Add static config for AccessGraph EKS audit logs

75f4d10

Extend the static config for Access Graph discovery to be able to specify the EKS cluster for which apiserver audit logs should be fetched and sent to Access Graph.

camscale force-pushed the camh/tag/kube-audit-logs branch from 7836522 to d5c9283 Compare November 5, 2025 05:03

camscale requested a review from r0mant as a code owner November 5, 2025 05:03

Add github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs to go.mod

a9eadec

Run: go get github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs@latest make go-mod-tidy-all # Manually move the go.mod line back to the first section!?!? This commit is kept separate for easier merging/rebasing.

camscale force-pushed the camh/tag/kube-audit-logs branch from d5c9283 to a9eadec Compare November 5, 2025 06:33

r0mant reviewed Nov 5, 2025

View reviewed changes

camscale force-pushed the camh/tag/kube-audit-logs branch 2 times, most recently from f580eef to 503a9e4 Compare November 10, 2025 08:53

camscale force-pushed the camh/tag/kube-audit-logs branch from 503a9e4 to a2f99a2 Compare November 10, 2025 11:24

r0mant approved these changes Nov 11, 2025

View reviewed changes

camscale added this pull request to the merge queue Nov 11, 2025

Merged via the queue into master with commit 5507d11 Nov 11, 2025
44 checks passed

camscale deleted the camh/tag/kube-audit-logs branch November 11, 2025 04:55

camscale mentioned this pull request Nov 21, 2025

[v18] identity activity center: Backport EKS audit logs #61651

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

discovery: Add AWS EKS audit log fetching for Access Graph#59568

discovery: Add AWS EKS audit log fetching for Access Graph#59568
camscale merged 5 commits intomasterfrom
camh/tag/kube-audit-logs

camscale commented Sep 25, 2025 •

edited

Loading

Uh oh!

juliaogris left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

juliaogris left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

r0mant left a comment

Uh oh!

camscale commented Nov 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

camscale commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

juliaogris left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

juliaogris left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

r0mant left a comment

Choose a reason for hiding this comment

Uh oh!

camscale commented Nov 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

camscale commented Sep 25, 2025 •

edited

Loading