-
Notifications
You must be signed in to change notification settings - Fork 520
OPRUN-4133: OLMv1 single/own namespace install mode support #1849
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
c9b7c0f
to
ac0ba97
Compare
@anik120: This pull request references OPRUN-4133 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.21.0" version, but no target version was set. In response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
- Provide a configuration mechanism through the ClusterExtension API to specify watch namespaces | ||
- Maintain backward compatibility with existing OLM v0 operator content | ||
- Generate appropriate RBAC resources scoped to the target namespaces | ||
- Support seamless migration path for customers moving from OLM v0 to OLM v1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In future, support values
for Helm-based bundles
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And, future support for SubscriptionConfig type configuration on the registry+v1 bundles
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added the point about the support for helm-based bundles, but is conversation about SubscriptionConfig on registry+v1 bundles appropriate here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think its ok to mention as an example to further illustrate the direction things will go in. But, I don't think we need to go super deep. Just say that for registry+v1 the configuration surface will be expanded to support OLMv0 SubscriptionConfig type behavior
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done PTAL.
The ClusterExtension CRD used in the registry+v1 bundle installation process is enhanced to: | ||
- Parse watchNamespace configuration from the inline JSON configuration | ||
- Map install modes based on namespace relationships (install vs watch namespace) | ||
- Generate appropriate RBAC resources scoped to the target namespace |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Call out the default install mode to be used depending on the install modes supported by the bundle (maybe another table would be good here)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a table, PTAL.
ac0ba97
to
8d51ffd
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@perdasilva addressed your feedback, PTAL
- Provide a configuration mechanism through the ClusterExtension API to specify watch namespaces | ||
- Maintain backward compatibility with existing OLM v0 operator content | ||
- Generate appropriate RBAC resources scoped to the target namespaces | ||
- Support seamless migration path for customers moving from OLM v0 to OLM v1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added the point about the support for helm-based bundles, but is conversation about SubscriptionConfig on registry+v1 bundles appropriate here?
The ClusterExtension CRD used in the registry+v1 bundle installation process is enhanced to: | ||
- Parse watchNamespace configuration from the inline JSON configuration | ||
- Map install modes based on namespace relationships (install vs watch namespace) | ||
- Generate appropriate RBAC resources scoped to the target namespace |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a table, PTAL.
|
||
`Note`: With watch namespace as a configuration value that can (at times, must) be provided for installation, the user input will be validated based on the bundle's supported install modes: | ||
|
||
| AllNamespaces | SingleNamespace | OwnNamespace | WatchNamespace Configuration | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
<3
|
||
The system determines the actual install mode based on the bundle's supported install modes and the user's configuration: | ||
|
||
| Bundle Supported Modes | watchNamespace Config | Selected Install Mode | Rationale | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| Bundle Supported Modes | watchNamespace Config | Selected Install Mode | Rationale | | |
| Bundle Supported Modes | watchNamespace Config | Selected Install Mode | Rationale | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also <3
// | ||
// inline must be set if configType is 'Inline'. | ||
// | ||
// +kubebuilder:validation:Type=object |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be worth calling out that adding this annotation ensures that inputs that are valid json, but not key/value config, are rejected. E.g. something like just true
is valid JSON (a boolean), but it wouldn't be valid configuration. Enforcing an object ensures that we have something that resembles a configuration/values files, e.g. key: value
.
We should also think about empty object mechanics here, e.g. {}
. Maybe that should be treated as no configuration and defaults used (cc @joelanford wdyt?).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a comment about the key/value config.
We should also think about empty object mechanics here, e.g. {}. Maybe that should be treated as no configuration and defaults used
Or should we prevent empty objects in the first place?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's my initial instinct. I just don't know if there's an use-case where you'd want to specifically set an empty configuration. I can't think of one, but that doesn't mean there isn't one XD
8d51ffd
to
a93ffa3
Compare
#### Future Goals | ||
|
||
- Support similar watch namespace configuration of Helm-based bundles | ||
- Expand the configuration surface for registry+v1 bundles to support OLMv0 SubscriptionConfig-type behavior, enabling broader operator configuration compatibility during the migration from OLMv0 to OLMv1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
perfect
5. Generate appropriate RBAC resources (Roles/RoleBindings vs ClusterRoles/ClusterRoleBindings) based on the determined install mode | ||
6. Configure operator deployments with correct environment variables and RBAC for the target watch namespace | ||
|
||
**Specific Resource Changes:** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
perfect
#### Story 2: Security-Conscious Deployments | ||
As a security-conscious administrator, I want to install operators that explicitly limit their scope to specific namespaces so that I can maintain strict RBAC boundaries and reduce the attack surface of operator deployments. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would re-word this a little. The way this is written makes it sound like OLMv1's simplified operator management model worsens the overall security posture of OLM.
In reality, it is the registry+v1 format's expectation of automated generation and management by OLM of operator RBAC — especially around the treatment of AllNamespaces install mode — that is a security issue for some users. Ultimately, OLMv1 will not have first class features/opinions on RBAC management in its next-gen bundle formats.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about:
As a security-conscious administrator, I want to install registry+v1 operators in a way that explicitly limit their scope to specific namespaces so that I can replicate olmv0 behavior of strict RBAC boundaries to reduce the attack surface created by the automated RBAC generation inherent in the registry+v1 format, particularly around AllNamespaces install mode.
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually removed this story altogether.
As a security-conscious administrator, I want to install operators that explicitly limit their scope to specific namespaces so that I can maintain strict RBAC boundaries and reduce the attack surface of operator deployments. | ||
|
||
#### Story 2: Operator Author Requirements | ||
As an operator developer, I have operators designed for namespace isolation that only support Single or OwnNamespace modes, and I want my customers to be able to deploy these operators in OpenShift with OLM v1 so that I can maintain the intended isolation boundaries while benefiting from OLM v1's simplified management. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This makes it sound like we are lending our support to isolation boundary concerns in OLMv1, which we are not.
Overall, I think the tone of this enhancement should be:
- limited support for existing content (limited because we still generally won't allow multiple installs of a bundle)
- re-casting registry+v1 install mode support as configuration (not multi-tenancy concerns around security, RBAC, or isolation)
I think it is important that we go out of our way here to reiterate that OLMv1 is not changing its core stances around:
- no support for multi-tenancy
- no first-class knowledge or handling of watchNamespaces (we are treating watch namespace as opaque bundle-provided configuration, used only in order to render a bundle into manifests)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We may want to copy some/all of this text from our upstream design decisions docs:
https://operator-framework.github.io/operator-controller/project/olmv1_design_decisions/#watched-namespaces-cannot-be-configured-in-a-first-class-api
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That upstream design decision link is really helpful. Modified the Summary (and a few sections below that) to align it more with the design decisions capture in the upstream doc, PTAL. That should align the overall tone of the EP with the upstream design doc too.
|
||
#### Future Goals | ||
|
||
- Support similar watch namespace configuration of Helm-based bundles |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would remove this line. This makes it sound like we will give some special treatment for watch namespace configuration for other bundle formats. We won't.
- I think we'll likely recommend against having watch namespace specific configurations in future bundle formats.
- Even if Helm-based bundles do have configuration related to watch namespaces, OLMv1 will be completely unaware of it. All OLMv1 will do is execute the templating logic of the chart. OLMv1 will be completely unaware of configuration semantics.
- Re-introducing multi-tenancy features or supporting multiple installations of the same operator | ||
- Supporting MultiNamespace install mode (watching multiple namespaces) | ||
- Modifying the fundamental OLM v1 architecture or adding complex multi-tenancy logic | ||
- Supporting install mode switching after initial installation as a first-class feature |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since we are treating this as configuration, I think we would support install mode switching, right? Because, more generally, we're allowing configuration to be changed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@perdasilva could you chime in here (I'm going off of Per's instruction to include this point here)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there's a distinction that could be better worded here. Supporting install mode switching could mean a couple of things:
- you can change the value of/remove watchNamespace
- that change will work
I think 1 is in scope and 2 is not. It would be up to the author to make that happen, or to be able to recover from that switch. OLMv1 will just compute the correct resources and pivot towards them.
- Supporting install mode switching after initial installation as a first-class feature | ||
- Enabling this feature for bundle formats other than registry+v1 | ||
- Changes to OLM cache infrastructure: OLM will continue to watch bundle contents cluster-scope | ||
- First-class support for switching install modes: Limited support for mode transitions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Repeat of other non-goal I commented on above? (Also, like I said there, I think we would support that, simply because we will support changing the configuration)
## Proposal | ||
|
||
Update the OLMv1 operator-controller to: | ||
1. Validate bundle compatibility with the requested install mode during resolution |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Terminology is important. Correct me if I'm misinterpreting. Resolution is the process by which we choose a bundle to install/upgrade to. The process of consuming the configuration and applying to the bundle in order to derive plain manifests happens after resolution. The resolver is not aware of the bundle's configuration schema and will not try to choose a bundle based on valid alignment of watchNamespace and install mode support.
1. Validate bundle compatibility with the requested install mode during resolution | |
1. Validate bundle compatibility with the requested install mode after resolution |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't thinking about dependency resolution here. What I meant was "reconciliation". Updating the line with "during reconciliation", wdyt?
6. Configure operator deployments with correct environment variables and RBAC for the target watch namespace | ||
|
||
**Specific Resource Changes:** | ||
- **RBAC Conversion**: `clusterPermissions` entries in the CSV get converted to `Role` resources in the watch namespace (instead of `ClusterRole` resources) for Single/OwnNamespace modes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This isn't true, I don't think?
- CSV
clusterPermissions
are always created as ClusterRole and ClusterRoleBindings. - CSV
permissions
are created as Role and RoleBinding in every watched namespace. If the operator is configured to watch all namespaces, then these permissions are instead created in a single ClusterRole and ClusterRoleBinding.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not 100% sure either, @perdasilva could you confirm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup. Brainfart on my side. It's the other way around, permissions get elevated to clusterPermissions. My bad!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Switched it around, PTAL
3ae456e
to
f659697
Compare
f659697
to
68bd95b
Compare
The test keeps failing with "unable to run pod" - so the linting is not the thing that's actually failing. /test markdownlint |
@anik120: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
No description provided.