Jakew/entra cert expiry alerts by nixpig · Pull Request #64504 · gravitational/teleport

nixpig · 2026-03-11T08:51:01Z

This change adds watch support for SAML connectors and a periodic SAML certificate expiry monitor. The monitor checks SAML connectors every 24 hours and on any connector create/update/delete for any with expiring signing certificates within the next 90 days. If any connectors have expiring/expired certificates then an alert is raised.

This addresses the "highest priority change" from #63604.

Single connector with expiring cert:

Multiple connectors with expiring certs:

Manual Test Plan

Test Environment

Local build of Teleport from branch jakew/entra-cert-expiry-alerts deployed to https://jwardtest01.cloud.gravitational.io/.

Test cert generation

Certificates are generated using the following command and updated against the connector via the web UI.

# Substituting the -days value per test requirement.
openssl req -x509 -newkey rsa:2048 -keyout /dev/null -days 1 -nodes -subj "/CN=test" -outform DER 2>/dev/null | base64 -w 0

Test Cases

Changelog: Display an alert when SAML connector signing certificates are within 90 days of expiry.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 80100c4587

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

lib/auth/saml.go

lib/auth/connector_monitor.go

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3c26a95e08

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

lib/auth/saml.go

lib/auth/connector_monitor.go

lib/services/saml.go

lib/auth/saml.go

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 52f105a66c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

lib/auth/connector_monitor.go

lib/services/local/events.go

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd622cc605

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

lib/auth/connector_monitor.go

api/proto/teleport/legacy/client/proto/event.proto

lib/auth/connector_monitor.go

lib/auth/connector_monitor_test.go

lib/auth/connector_monitor.go

lib/services/saml.go

smallinsky · 2026-03-12T10:20:12Z

lib/auth/connector_monitor.go

+// buildAlertMessage returns a SAML cert expiry alert message for the given connector names.
+func (m *SAMLCertExpiryMonitor) buildAlertMessage(connectorNames []string) string {
+	return fmt.Sprintf(
+		"The following connectors have one or more certificates that have expired or will expire in the next %d days: %s.",


can we distinguish the Expired and ExpiredSoon connectors in the alert message ?

Additionally this message should be somehow addressable. Where after reading The following connectors have one or more certificates that have expired or will expire in the next %d the customer will thing so what ? it just a cert.

They will not know what is the impact and what to resolve the issue.

We will need to align this. a bit more to User Friendly message describing a impact and provided a link or steps to resolve this issue.

Is something like this this more in line with what you're thinking?

The Learn More button currently linking to the SAML connector page in the docs, but I figure we should create a page specifically addressing connector cert expiry and how to resolve it.

I don't think the expired/expiring tacked on at the end is perfect, but from what I could see there isn't a way to have newline chars or lists in a cluster alert. Open to suggestions.

"The following" don't make to much sense because it is single line.

What about something like:

"SAML SSO users will no longer be able to authenticate to Teleport in X days" or in case of already expired "SAML SSO users are no longer able to authenticate to Teleport"

Connector "X" references an IdP certificate that expires in 3 days.

Connector "Y" references an IdP signing certificate that expired on 2026-03-10:14:00

Please rotate the IdP signing certificate. Refer to for more details

That looks good, but as mentioned in previous comment, it doesn't appear newlines are currently supported. The CheckMessage method rejects any message with control characters:

teleport/api/types/cluster_alert.go

Lines 186 to 190 in ddc3e93

for _, c := range c.Spec.Message {

if unicode.IsControl(c) {

return trace.BadParameter("control characters not supported in alerts")

}

}

I'm also not sure what we do when there are multiple connectors already expired and/or connectors expiring... e.g., if entra-connector-1 certs expire in 70 days, entra-connector-2 certts have already expired, and saml-connector-1 certs expire in 15 days, do we concat those messages like this:

"entra-connector-1 users will no longer be able to authenticate to Teleport in 70 days. entra-connector-2 users are no longer able to authenticate to Teleport. saml-connector-1 users will no longer be able to authenticate to Teleport in 15 days."

...which seems a bit confusing.

It's like we're trying to put too much and too granular information in an alert, which should just be a simple 'hey, you should be aware of this'.

Let's sync offline and come to agreement on how this should be presented.

lib/services/saml_test.go

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7026c2b248

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

lib/services/local/events.go

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 19d2e44ddc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-03-12T11:09:43Z

lib/auth/connector_monitor.go

+// and creates or updates an alert. If none are expiring, then any existing alert is deleted.
+func (m *SAMLCertExpiryMonitor) reconcileAlert(ctx context.Context) error {
+	var expiringConnectors []string
+	for connector, err := range m.Connectors.RangeSAMLConnectorsWithOptions(ctx, "", "", false, types.SAMLConnectorValidationFollowURLs(false)) {


Avoid clearing alerts after partial SAML scans

reconcileAlert relies on RangeSAMLConnectorsWithOptions(..., SAMLConnectorValidationFollowURLs(false)) to do a full scan, but connector unmarshalling can still fail for valid configs with mfa.entity_descriptor_url because ValidateSAMLConnector unconditionally fetches MFA metadata (lib/services/saml.go) and RangeSAMLConnectorsWithOptions drops those errors instead of returning them (lib/services/local/users.go). In that case the expiring connector is omitted from expiringConnectors, so this path can delete saml-cert-expiry-warning as a false negative until a later successful scan.

Useful? React with 👍 / 👎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 11c45fa585

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

lib/auth/connector_monitor.go

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eb9d2d8e0a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

lib/auth/connector_monitor.go

chatgpt-codex-connector · 2026-03-12T15:11:20Z

lib/auth/connector_monitor.go

+		if expiring, err := services.CheckSAMLCertExpiry(connector, samlCertExpiryTimeframe); err != nil {
+			return trace.Wrap(err)


Continue reconciliation after per-connector cert errors

This returns immediately on the first CheckSAMLCertExpiry error, so one malformed connector certificate can block scanning of all remaining connectors and suppress alert updates cluster-wide. Since connector validation accepts metadata XML without decoding each embedded cert, this failure mode is reachable with persisted connector data; in that state the monitor repeatedly fails and never creates/clears alerts for other connectors.

Useful? React with 👍 / 👎.

I think both chatgpt comments above are reasonable. I think we should log and continue here.

smallinsky

Mostly LGTM once the remaining comment is addressed.

I’d like to do a next pass after the dependency on the SAML connector event stream is removed from the context of this PR nit comment will be addressed.

lib/auth/connector_monitor.go

api/client/events.go

lib/auth/connector_monitor.go

smallinsky · 2026-03-13T12:12:46Z

lib/service/service.go

+	samlCertExpiryMonitor, err := auth.NewSAMLCertExpiryMonitor(auth.SAMLCertExpiryMonitorConfig{
+		Connectors: authServer.Services,
+		Alerts:     authServer.Services,
+		Events:     authServer.Services,
+		Clock:      process.Clock,
+		Backend:    process.backend,
+		Logger: logger.With(
+			teleport.ComponentKey, teleport.Component(teleport.ComponentAuth, "saml-cert-expiry-monitor"),
+		),
+	})
+	if err != nil {
+		return trace.Wrap(err)
+	}


Lets create a helper func for creation of samlCertExpirtyMonitor.

lib/service/service.go

lib/auth/connector_monitor.go

kopiczko · 2026-03-13T12:40:26Z

lib/auth/connector_monitor.go

+		if expiring, err := services.CheckSAMLCertExpiry(connector, samlCertExpiryTimeframe); err != nil {
+			return trace.Wrap(err)


I think both chatgpt comments above are reasonable. I think we should log and continue here.

lib/auth/connector_monitor.go

lib/auth/connector_monitor_test.go

kopiczko · 2026-03-13T12:47:46Z

lib/auth/connector_monitor_test.go

+		<-done
+	})
+
+	require.EventuallyWithT(t, func(t *assert.CollectT) {


Can this be rewritten to use synctest? Waiting for 1 second is almost guaranteed to be flaky in the CI.

This test spins up an auth server and uses the backend watch path for events, which doesn't feel like a great fit for synctest. Precedent in lib/auth for integration tests with newTLSTestServer seems to be Eventually/EventuallyWitT, e.g. lib/auth/tls_test.go and lib/auth/auth_with_roles_test.go.

I've bumped the timeout up to 5s to make it less sensitive to slow CI.

kopiczko · 2026-03-13T12:53:56Z

lib/auth/connector_monitor_test.go

+	require.NoError(t, err)
+
+	require.EventuallyWithT(t, func(t *assert.CollectT) {
+		alerts, err := srv.Auth().GetClusterAlerts(ctx, types.GetClusterAlertsRequest{


If this require.EventuallyWithT usage is to wait for the cache, then you can probably use the backend directly instead with srv.Auth().Services.GetClusterAlerts

It's waiting for the watcher to handle the event, which happens asynchronously. I don't think we can get away from the 'waiting' here.

…jakew/entra-cert-expiry-alerts" This reverts commit a2358ce, reversing changes made to 6597b9d.

github-actions · 2026-03-13T15:34:46Z

Amplify deployment status

Branch	Commit	Job ID	Status	Preview	Updated (UTC)
jakew/entra-cert-expiry-alerts	HEAD	1	✅SUCCEED	jakew-entra-cert-expiry-alerts	2026-03-13 15:36:33

This reverts commit 3591bda.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5f8eb96b1a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

lib/services/saml.go

nixpig requested a review from smallinsky March 11, 2026 08:51

github-actions bot added the size/md label Mar 11, 2026

github-actions bot requested review from Tener and aadc-dev March 11, 2026 08:51

chatgpt-codex-connector bot reviewed Mar 11, 2026

View reviewed changes

lib/auth/saml.go Outdated Show resolved Hide resolved

lib/auth/connector_monitor.go Outdated Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Mar 11, 2026

View reviewed changes

lib/auth/saml.go Outdated Show resolved Hide resolved

smallinsky reviewed Mar 11, 2026

View reviewed changes

lib/auth/connector_monitor.go Outdated Show resolved Hide resolved

lib/auth/connector_monitor.go Outdated Show resolved Hide resolved

lib/services/saml.go Outdated Show resolved Hide resolved

lib/auth/saml.go Outdated Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Mar 12, 2026

View reviewed changes

lib/auth/connector_monitor.go Outdated Show resolved Hide resolved

lib/services/local/events.go Outdated Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Mar 12, 2026

View reviewed changes

lib/auth/connector_monitor.go Outdated Show resolved Hide resolved

lib/auth/connector_monitor.go Outdated Show resolved Hide resolved

smallinsky requested a review from kopiczko March 12, 2026 08:15

smallinsky reviewed Mar 12, 2026

View reviewed changes

chatgpt-codex-connector bot reviewed Mar 12, 2026

View reviewed changes

lib/services/local/events.go Outdated Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Mar 12, 2026

View reviewed changes

lib/auth/connector_monitor.go Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Mar 12, 2026

View reviewed changes

smallinsky reviewed Mar 13, 2026

View reviewed changes

kopiczko reviewed Mar 13, 2026

View reviewed changes

nixpig mentioned this pull request Mar 13, 2026

Add event stream support for SAML connector #64564

Merged

nixpig requested review from avatus, doggydogworld, klizhentas, mjsmithnh, r0mant, ravicious, rosstimothy, russjones, ryanclark and zmb3 as code owners March 13, 2026 15:26

nixpig added 2 commits March 13, 2026 15:27

Show alert on SAML connector cert expiry

4c56340

Handle cert set directly on connector

34faa50

nixpig added 5 commits March 13, 2026 15:29

Further feedback addressing

511617a

Add backend event stream support for SAML connector

f7a82c6

Revert "Merge commit '611fabec9616940abb614d3aede41edafb05c1d2' into …

afa6797

…jakew/entra-cert-expiry-alerts" This reverts commit a2358ce, reversing changes made to 6597b9d.

Addressing further feedback

2692312

Remove events for other PR

344976e

nixpig force-pushed the jakew/entra-cert-expiry-alerts branch from 1e5e86a to 344976e Compare March 13, 2026 15:33

nixpig temporarily deployed to docs-amplify March 13, 2026 15:33 — with GitHub Actions Inactive

Align with master

1a3b7d1

nixpig temporarily deployed to docs-amplify March 13, 2026 15:34 — with GitHub Actions Inactive

nixpig added 6 commits March 13, 2026 15:38

Align with master

71f2bfa

Align with master

f1e250e

Align with master

fb1255c

Align with master

3591bda

Revert "Align with master"

55adfec

This reverts commit 3591bda.

Align with master

3da48aa

nixpig requested a review from gzdunek as a code owner March 13, 2026 15:49

nixpig removed request for Tener, aadc-dev, avatus, doggydogworld, klizhentas, mjsmithnh, r0mant, rosstimothy, russjones, ryanclark and zmb3 March 13, 2026 15:52

Remove odd files

5f8eb96

chatgpt-codex-connector bot reviewed Mar 15, 2026

View reviewed changes

lib/services/saml.go Show resolved Hide resolved

	for _, c := range c.Spec.Message {
	if unicode.IsControl(c) {
	return trace.BadParameter("control characters not supported in alerts")
	}
	}

		if expiring, err := services.CheckSAMLCertExpiry(connector, samlCertExpiryTimeframe); err != nil {
		return trace.Wrap(err)

Conversation

nixpig commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Manual Test Plan

Test Environment

Test cert generation

Test Cases

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smallinsky Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

nixpig commented Mar 11, 2026 •

edited

Loading

smallinsky Mar 13, 2026 •

edited

Loading

github-actions bot commented Mar 13, 2026 •

edited

Loading