Skip to content

Commit 97358e3

Browse files
charludocharludo
committed
rfc: configuration file
1 parent 5fb4b4d commit 97358e3

File tree

2 files changed

+295
-0
lines changed

2 files changed

+295
-0
lines changed

‎rfc/013-configuration-file.md‎

Lines changed: 291 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,291 @@
1+
# RFC 013: Configuration File
2+
3+
## Background
4+
5+
Contrast is currently configured exclusively via a CLI with a large number of flags and arguments across multiple subcommands.
6+
While this provides flexibility, it leads to several practical issues:
7+
- Users who want to make changes to a deployment must reconstruct long command invocations.
8+
- In team settings, these invocations must be shared out-of-band.
9+
- Configuration intent isn't easily reviewable or version-controllable.
10+
11+
As a result, users are effectively forced to build their own configuration abstraction on top of the CLI.
12+
This RFC proposes introducing a first-class configuration file to address this, while preserving full backward compatibility with existing CLI usage.
13+
14+
## Requirements
15+
16+
1. Existing CLI invocations must continue to work without modification. More specifically:
17+
1. Command-line flags and arguments must always take precedence over configuration file values.
18+
2. Users must not be required to use a configuration file.
19+
2. The configuration format should easily be extensible for future use cases.
20+
3. The introduction of a configuration file must not significantly increase maintenance burden.
21+
22+
## Design
23+
24+
### Configuration format
25+
26+
We introduce a TOML-based configuration file.
27+
TOML is chosen for its readability and explicit structure.
28+
Additionally, parts of the codebase already interact with TOML files, allowing us to avoid additional dependencies.
29+
30+
The configuration file makes use of TOML tables for structure.
31+
At its most basic, the file only contains a `cli` table, with its keys corresponding to the existing CLI arguments:
32+
33+
```toml
34+
[cli]
35+
# path to the policy (.rego) file
36+
policy = "./path/to/policy.rego"
37+
38+
# path to the settings (.json) file
39+
settings = "./path/to/settings.json"
40+
41+
# path to the cache (.json) file containing the image layers
42+
genpolicy-cache-path = "./path/to/layers-cache.json"
43+
44+
# path to the manifest (.json) file
45+
manifest = "./path/to/manifest.json"
46+
47+
# set the default reference values used for attestation
48+
reference-values = "Metal-QEMU-SNP"
49+
50+
# ...
51+
```
52+
53+
All configuration relevant to the CLI is nested under this section.
54+
Additional sections may be introduced as needed, but a single configuration file is used for the entire project.
55+
56+
All fields are optional; if users specify a required field neither in the configuration file nor in their CLI arguments,
57+
the existing argument validation logic will inform the user in the same way it currently does.
58+
59+
### Mapping CLI flags to configuration
60+
61+
The configuration file mirrors the existing CLI surface:
62+
all flags, arguments, and options have corresponding fields in the configuration.
63+
64+
Command-line arguments always take precedence over values loaded from the configuration file.
65+
This ensures that:
66+
- Users who don't use a configuration file experience no behavior changes.
67+
- Existing tooling and scripts continue to work unchanged.
68+
- Users may selectively override configuration values for one-off invocations.
69+
70+
### Duplicate fields
71+
72+
Contrast subcommands do share a subset of their arguments.
73+
For example, `generate`, `set`, `verify` and `recover` all require setting the `manifest` argument.
74+
75+
We could adapt the structure of the configuration file to match this, for example:
76+
77+
```toml
78+
[cli.generate]
79+
# path to the manifest (.json) file
80+
manifest = "./path/to/manifest.json"
81+
# ...
82+
83+
[cli.set]
84+
# path to the manifest (.json) file
85+
manifest = "./path/to/manifest.json"
86+
# ...
87+
88+
[cli.verify]
89+
# path to the manifest (.json) file
90+
manifest = "./path/to/manifest.json"
91+
# ...
92+
93+
[cli.recover]
94+
# path to the manifest (.json) file
95+
manifest = "./path/to/manifest.json"
96+
# ...
97+
```
98+
99+
However, the need for this seems questionable: for all options shared between the commands, values aren't expected to differ.
100+
If a user does require overriding such a shared argument in a single invocation, they can always override the configuration file value in the CLI command.
101+
The procedure to obtain the relevant configuration options for a subcommand from the configuration file are shown below.
102+
103+
### Internal representation
104+
105+
Internally, the configuration is represented as a single, unified Go struct.
106+
This struct contains all configuration fields relevant to the application, and serves as the single source of truth for configuration options.
107+
More on this below.
108+
109+
```go
110+
type Config struct {
111+
CLI struct {
112+
// shared
113+
LogLevel string `toml:"log-level" comment:"set logging level (debug, info, warn, error, or a number)"`
114+
WorkspaceDir string `toml:"workspace-dir" comment:"directory to write files to, if not set explicitly to another location"`
115+
116+
ManifestPath string `toml:"manifest" comment:"path to manifest (.json) file"`
117+
Coordinator string `toml:"coordinator" comment:"endpoint the coordinator can be reached at"`
118+
PolicyPath string `toml:"policy-path" comment:"path to policy (.rego) file"`
119+
WorkloadOwnerKeyPath string `toml:"workload-owner-key" comment:"path to workload owner key (.pem) file"`
120+
// ...
121+
122+
// generate-specific
123+
SettingsPath string `toml:"settings" comment:"path to settings (.json) file"`
124+
GenpolicyCachePath string `toml:"genpolicy-cache-path" comment:"path to cache for the cache (.json) file containing the image layers"`
125+
ReferenceValuesPlatform string `toml:"reference-values" comment:"set the default reference values used for attestation"`
126+
WorkloadOwnerKeys []string `toml:"add-workload-owner-keys" comment:"add a workload owner key from a PEM file to the manifest (set more than once to add multiple keys)"`
127+
// ...
128+
129+
// set-specific
130+
// ...
131+
}
132+
}
133+
```
134+
135+
The struct makes use of the `toml` and `comment` annotations provided by [pelletier/go-toml](https://pkg.go.dev/github.com/pelletier/go-toml/v2#example-Marshal-Commented).
136+
A function `Default` is added to create a instance of the struct with default values set mirroring their current defaults in the `cobra.Command` subcommands:
137+
138+
```go
139+
func Default() Config {
140+
return Config{
141+
CLI: {
142+
// shared
143+
LogLevel: "warn",
144+
145+
ManifestPath: "manifest.json",
146+
PolicyPath: "rules.rego",
147+
WorkloadOwnerKeyPath: "workload-owner.pem",
148+
// ...
149+
150+
// generate-specific
151+
SettingsPath: "settings.json",
152+
GenpolicyCachePath: "layers-cache.json",
153+
WorkloadOwnerKeys: []string{"workload-owner.pem"},
154+
// ...
155+
156+
// set-specific
157+
// ...
158+
},
159+
}
160+
}
161+
```
162+
163+
The `Default` function has two main purposes.
164+
First, by creating a `Config` object through it, then unmarshaling a TOML configuration into the resulting struct, analogous behavior to the current one (that is, some missing flags fall back to defaults) is achieved.
165+
Secondly, we can trivially marshal the default `Config` object to obtain a commented, default-configured TOML file which users can use as a starting point for their configuration.
166+
167+
### `cobra.Command` derivations
168+
169+
The `Config` struct as depicted above uses the `toml` and `comment` annotations.
170+
An optional `short:"<char>"` annotation is added, that is,
171+
172+
```go
173+
type Config struct {
174+
CLI struct {
175+
// ...
176+
PolicyPath string `toml:"policy-path" short:"p" comment:"path to policy (.rego) file"`
177+
// ...
178+
}
179+
}
180+
```
181+
182+
and so forth.
183+
Together with the `Default` function, this provides all necessary information to derive the Cobra (sub-)commands.
184+
The functions currently used to create the `cobra.Command` structs for (sub-)commands get simplified as shown below for `NewGenerateCommand`.
185+
186+
```go
187+
func NewGenerateCmd(cfg Config) *cobra.Command {
188+
cmd := &cobra.Command{
189+
Use: "generate [flags] paths...",
190+
Short: "generate policies and inject into Kubernetes resources",
191+
Long: `Generate policies and inject into the given Kubernetes resources. [...]`,
192+
RunE: withTelemetry(runGenerate),
193+
}
194+
// ...
195+
196+
AddArgs(cfg, cmd, []string{
197+
"policy",
198+
"settings",
199+
"genpolicy-cache-path",
200+
"manifest",
201+
202+
"workload-owner-key",
203+
"disable-updates",
204+
// ...
205+
})
206+
207+
cmd.MarkFlagsMutuallyExclusive("add-workload-owner-key", "disable-updates")
208+
return cmd
209+
}
210+
```
211+
212+
Here, `AddArgs` takes the provided (default) `Config` and the `cobra.Command`, as well as a slice of argument names matching the ones used in the `toml` annotations.
213+
It then adds a `cobra.Command` argument for each one of those names, using the metadata from the `toml` annotations for short names and help texts.
214+
215+
Validation of the provided arguments continues to work just as it currently does (for example `parseGenerateFlags`), with the only difference being that this also runs on the new `Config` struct.
216+
217+
### Configuration loading and precedence
218+
219+
The `contrast` root command receives a new optional, persistent flag `--config`.
220+
Upon invocation of any CLI (sub-)command, the following steps are performed:
221+
- In the `OnInitialize` function of the root command:
222+
- Create a `Config` object via `Default()`
223+
- If the `--config` flag was set, load the specified configuration file.
224+
- If the file is specified, but missing or can't be parsed, exit with error.
225+
- Unmarshal the config file into the default configuration.
226+
- Pass the config struct to the subcommands.
227+
- In each subcommand:
228+
- For all flags set in the CLI invocation, override the corresponding `Config` struct field.
229+
- Apply the current validation logic.
230+
231+
### Parsing behavior
232+
233+
When parsing the configuration file, unknown fields intentionally result in an error to prevent typos or use of deprecated fields.
234+
Missing fields don't result in errors, that is, all fields are optional in the configuration file.
235+
236+
### Versioning
237+
238+
A version field may be included in the configuration file to allow explicit handling of breaking changes:
239+
240+
```toml
241+
version = 1.17
242+
```
243+
244+
This would allow us to perform compatibility checks when loading older configurations.
245+
However, versioning is considered optional at this stage and may be introduced later if required.
246+
247+
### Backward and Forward Compatibility
248+
249+
Backward compatibility is ensured by:
250+
- Making the configuration file entirely optional.
251+
- Giving precedence to CLI flags over configuration values.
252+
- Preserving the existing CLI interface.
253+
254+
## Further applications
255+
256+
In addition to using the configuration file for simplifying CLI uses, other applications could also be considered.
257+
The structure of the configuration file and the `Config` struct, that is, a nested `CLI` struct or table inside it,
258+
serves the purposes of compartmentalizing these different applications.
259+
260+
One such additional application is sketched out below.
261+
262+
### Reference values
263+
264+
Currently, a `--reference-values` argument needs to be passed to the CLI in (most) invocations of `generate`.
265+
Afterward, users need to manually fill in the actual reference values for the specified platforms in the manifest.
266+
267+
Allowing the users to instead set these values directly in the configuration file provides a dedicated place to store these values across manifest lifetimes,
268+
and to populate the manifest directly from this file, without additional user involvement.
269+
```toml
270+
[[reference_values]]
271+
platform = "Metal-QEMU-SNP"
272+
patch = '''
273+
[
274+
...
275+
]
276+
'''
277+
# ...
278+
```
279+
280+
The actual values here are JSON-patches, in keeping with how we handle our own reference value patches.
281+
282+
Again, these sections should be completely optional.
283+
Either passing `--reference-values` to the CLI *or* setting `cli.reference_values` in the CLI section of the configuration file should preclude the use of these values.
284+
285+
## Alternatives considered
286+
287+
### Using Viper
288+
289+
The [Viper](https://github.com/spf13/viper) library was considered due to its easy integration with Cobra and the built-in support for config files.
290+
However, Viper is geared more towards interacting with a configuration file from within an application, that is, loading, editing and saving a config file.
291+
Working with a single configuration struct and deriving the subcommands from this configuration also doesn't appear simpler than implementing the above suggestions manually.

‎tools/vale/styles/config/vocabularies/edgeless/accept.txt‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,7 @@ SNP
150150
SSD
151151
strongSwan
152152
subcommand
153+
subcommands
153154
Subresource
154155
substituters
155156
superset
@@ -173,6 +174,9 @@ undecryptable
173174
Undeploy
174175
underspecified
175176
unencrypted
177+
Unmarshal
178+
Unmarshaling
179+
unmarshaling
176180
unrepresentable
177181
unspoofable
178182
untrusted

0 commit comments

Comments
 (0)