Support safe removal of deprecated fields in the same version

[tobiasgiese]

Motivation

It would be useful to allow removal of deprecated fields without flagging them as breaking changes. In Kubernetes CRDs, deprecations are often indicated in the field description with Deprecated:. Recognizing this pattern can help reduce noise when checking for breaking changes and improve the developer experience.

Proposal

• Add an option to treat fields whose description contains Deprecated: as safe to remove.
• This could be controlled via a CLI flag (e.g., --allow-deprecated-removal) or configuration.
• The default behavior should remain unchanged for backward compatibility.

Example

# Previous CRD
spec:
  oldField:
    type: string
    description: "Deprecated: use `newField` instead"
  currentField:
    type: integer

# New CRD
spec:
  currentField:
    type: integer

With this feature, removal of oldField would not be considered a breaking change.

Benefits

• Reduces false positives when diffing CRDs.
• Aligns with Kubernetes deprecation conventions.
• Improves workflow for CRD maintainers when cleaning up deprecated fields.

[everettraven]

When you are referring to removal of deprecated fields in a CRD are you meaning across version boundaries like v1 --> v2 or in the same version like v1 --> v1?

[tobiasgiese]

Thanks for the super quick response!

I am referring to the removal of deprecated fields in the same version. Sorry, should have mentioned this.

Edited the issue title.

[everettraven]

I am referring to the removal of deprecated fields in the same version. Sorry, should have mentioned this.

This is never a non-breaking change. The same api version must never remove fields - deprecated or not.

For additional context for folks seeing this later, the Kubernetes deprecation conventions are here: https://kubernetes.io/docs/reference/using-api/deprecation-policy/

In order to properly deprecate and then remove a field from a CRD version you must:

• Create a new version of the CRD where that field is removed (i.e a v2)
• Create a conversion webhook to allow writing a v1 instance, reading it as v2 and converting it back to v1 without data loss
• Ship at least one release where the two versions and the associated conversion logic is present
• Remove the old version from the CRD in a later release (from an end user perspective, they will only ever be able to update to this after they have migrated all their stored instances to the newer API version) - likely a major release version increment.

Today, crdify will run validations across served versions for backwards compatibility checks but end users have the option to configure it such that those validations will not be run when the CRD specifies the Webhook conversion policy.

Because crdify cannot run the conversion logic to evaluate compatibility you are inherently trusting that you've correctly implemented the conversion logic in a backwards compatible and non-breaking way when setting the configuration to not run those validations.

[tobiasgiese]

We're aware of the benefits of having multiple API versions. For our project, however, we've decided not to introduce them to keep the operator simpler and reduce maintenance overhead.

What we really need is a reliable way to deprecate fields and remove them safely in later releases. We understand that removing deprecated fields will introduce breaking changes, which is why we plan to enforce checks in our CI pipeline to prevent accidental removal of non-deprecated fields.

Given this approach, supporting multiple API versions is not something we plan to implement, but proper deprecation handling is essential for our workflow.

[everettraven]

While I can appreciate wanting to reduce the maintenance burden, this is an anti-pattern that I do not anticipate us supporting natively.

We've designed this tool in a way that enables use of the common bits through importing the Go packages. If you want to do more custom logic here, you can certainly build your own binary that uses many of the common validations while dropping ones you don't agree with and/or adding your own custom validation implementations.

@JoelSpeed may have some other thoughts here, but I think we need to be careful here to ensure we are being opinionated in ways that leverages the Kubernetes conventions as the standard to be enforced by crdify.

[JoelSpeed]

I don't think this exists today, but have we considered an exceptions management configuration for crdify that would allow special use cases to create their own list of exceptions to the standard rules? I'm thinking how linters have a way to regex match a particular string and say "I don't care about warnings that match this particular rule".

I agree with what @everettraven is saying, really all projects should be going through the process of creating a v1 to v2 migration. However we know that many projects don't follow all guidelines strictly, so we probably do need some way to allow folks to use this in a manner that works for their project.

@tobiasgiese how are you using crdify? Is this the kind of thing where you could override a CI status check? Why is overriding CI not sufficient in cases where you know you are making breaking changes?

[tobiasgiese]

@tobiasgiese how are you using crdify? Is this the kind of thing where you could override a CI status check? Why is overriding CI not sufficient in cases where you know you are making breaking changes?

We implemented our own CLI using the crdify pkg. See https://github.com/NVIDIA/doca-platform/blob/public-main/hack/tools/dpfdev/cmd/crdify.go

Further, we added 2 more features to

• remove description warnings if the removed field has been deprecated in the old CRD (see impl)
• add an allowList to validations in the crdify config (see impl)

an example config will look like this (xref)

validations:
  - name: description
    enforcement: Warn # changing to warning is important for the deprecation feature
  - name: required
    enforcement: Error
    configuration:
      allowList:
        - apiVersion: "stable.example.com/v1"
          kind: "Shirt"
          field: ".spec"
          error: "new required fields: [color]"

This enables us to use cridfy in our CI pipeline already.

[tobiasgiese]

I agree with what @everettraven is saying, really all projects should be going through the process of creating a v1 to v2 migration. However we know that many projects don't follow all guidelines strictly, so we probably do need some way to allow folks to use this in a manner that works for their project.

I also agree that every controller/operator should stick to the Kubernetes policy. On the other hand, using alpha versions (like v1alpha1) implies that "The API may change in incompatible ways in a later software release without notice." (see Kubernetes API Overview). This means that breaking changes in alpha versions are allowed. As we still want to avoid most of the breaking changes we need a way to use tools like crdify even with a single alpha version.

[everettraven]

I also agree that every controller/operator should stick to the Kubernetes policy. On the other hand, using alpha versions (like v1alpha1) implies that "The API may change in incompatible ways in a later software release without notice."

AFAIK, Kubernetes still recommends increasing the version, in an alpha case it would be v1alpha1 -> v1alpha2 (and so on).

I don't think this exists today, but have we considered an exceptions management configuration for crdify that would allow special use cases to create their own list of exceptions to the standard rules? I'm thinking how linters have a way to regex match a particular string and say "I don't care about warnings that match this particular rule".

You are correct that this doesn't yet exist today, but I do agree that this is a reasonable configuration option to create. I've chatted offline about this with a couple folks in the past but don't think I've made any actionable issues related to this yet. I can plan to create one where we could continue discussion related to that. I'd like to think through more of the override use cases here to see what kind of operations we need to be able to support here.

Just as an example, I'm not sure how we would be able to use a simple regex matcher to handle the case @tobiasgiese outlined here. This case would probably be more suited to passing a schema in through a CEL expression to run for determining whether or not it should be ignored.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale