Skip to content

Commit 32c4631

Browse files
yogi4yogi4
authored
Update README.md (#53)
* Refactored Readme.md to reflect latest run instructions based on Demo
1 parent f8f34b5 commit 32c4631

File tree

1 file changed

+174
-116
lines changed

1 file changed

+174
-116
lines changed

‎README.md‎

Lines changed: 174 additions & 116 deletions
Original file line numberDiff line numberDiff line change
@@ -1,174 +1,232 @@
1-
# OpenCHAMI cloud-init Server
1+
# OpenCHAMI Cloud-Init Server
2+
3+
## Summary of Repo
4+
The **OpenCHAMI cloud-init service** retrieves detailed inventory information from SMD and uses it to create cloud-init payloads customized for each node in an OpenCHAMI cluster.
5+
6+
## Table of Contents
7+
1. [About / Introduction](#about--introduction)
8+
2. [Build / Install](#build--install)
9+
- [Environment Variables](#environment-variables)
10+
- [Building Locally with GoReleaser](#building-locally-with-goreleaser)
11+
3. [Running the Service](#running-the-service)
12+
- [Cluster Name](#cluster-name)
13+
- [Fake SMD Mode](#fake-smd-mode)
14+
- [Impersonation](#impersonation)
15+
- [Nocloud-net Datasource](#nocloud-net-datasource)
16+
4. [Testing the Service](#testing-the-service)
17+
- [Basic Endpoint Testing](#basic-endpoint-testing)
18+
- [Meta-data](#meta-data)
19+
- [User-data](#user-data)
20+
- [Vendor-data](#vendor-data)
21+
5. [Group Handling and Overrides](#group-handling-and-overrides)
22+
- [Updating Group Data with a Simple Jinja Example](#updating-group-data-with-a-simple-jinja-example)
23+
- [Complex Base64 Example](#complex-base64-example)
24+
- [Cluster Defaults and Instance Overrides](#cluster-defaults-and-instance-overrides)
25+
- [Set Cluster Defaults](#set-cluster-defaults)
26+
- [Override Instance Data](#override-instance-data)
27+
6. [More Reading](#more-reading)
28+
29+
---
30+
31+
## About / Introduction
32+
The **OpenCHAMI Cloud-Init Service** is designed to generate cloud-init configuration for nodes in an OpenCHAMI cluster. The new design pushes the complexity of merging configurations into the cloud-init client rather than the server. This README provides instructions based on the [Demo.md](https://github.com/OpenCHAMI/cloud-init/blob/main/demo/Demo.md) file for running and testing the service.
33+
34+
This service provides configuration data to cloud-init clients via the standard nocloud-net datasource. The service merges configuration from several sources:
35+
- **SMD data** (or simulated data in development mode)
36+
- **User-supplied JSON** (for custom configurations)
37+
- **Cluster defaults and group overrides**
38+
39+
Cloud-init on nodes retrieves data in a fixed order:
40+
1. `/meta-data` – YAML document with system configuration.
41+
2. `/user-data` - a document which can be any of the [user data formats](https://cloudinit.readthedocs.io/en/latest/explanation/format.html#cloud-config-data)
42+
3. `/vendor-data` – Vendor-supplied configuration via include-file mechanisms.
43+
4. `/network-config` – An optional document in one of two [network configuration formats](https://cloudinit.readthedocs.io/en/latest/reference/network-config.html#network-config). This is only requested if configured to do so with a kernel parameter or through cloud-init configuration in the image. __NB__: __OpenCHAMI doesn't support delivering `network-config` via the cloud-init server today__
44+
45+
46+
---
47+
48+
## Build / Install
49+
50+
This project uses **[GoReleaser](https://goreleaser.com/)** for building and releasing, embedding additional metadata such as commit info, build time, and version. Below is a brief overview for local builds.
251

3-
The OpenCHAMI cloud-init server retrieves detailed inventory information from SMD and uses it to create cloud-init payloads customized for each node in an OpenCHAMI cluster.
52+
### Environment Variables
53+
To include detailed metadata in your builds, set the following:
454

5-
## cloud-init Server Setup
55+
- **GIT_STATE**: `clean` if your repo is clean, `dirty` if uncommitted changes exist
56+
- **BUILD_HOST**: Hostname of the build machine
57+
- **GO_VERSION**: Version of Go used (for consistent versioning info)
58+
- **BUILD_USER**: Username of the person/system performing the build
659

7-
OpenCHAMI utilizes the cloud-init platform for post-boot configuration. A custom cloud-init server container is included with this quickstart Docker Compose setup, but must be populated prior to use.
60+
```bash
61+
export GIT_STATE=$(if git diff-index --quiet HEAD --; then echo 'clean'; else echo 'dirty'; fi)
62+
export BUILD_HOST=$(hostname)
63+
export GO_VERSION=$(go version | awk '{print $3}')
64+
export BUILD_USER=$(whoami)
65+
```
866

9-
The cloud-init server provides multiple API endpoints, described in the sections below. Choose the appropriate option for your needs, or combine them as appropriate.
67+
### Building Locally with GoReleaser
68+
1. [Install GoReleaser](https://goreleaser.com/install/) following their documentation.
69+
2. Run in snapshot mode to build locally without releasing:
70+
71+
```bash
72+
goreleaser release --snapshot --clean
73+
```
74+
3. Check the `dist/` directory for compiled binaries, which will include the injected metadata.
1075

1176
> [!NOTE]
12-
> This guide assumes that the cloud-init server is exposed as `foobar.openchami.cluster`.
13-
> If your instance is named differently, adapt the examples accordingly.
77+
> If you encounter errors, ensure your GoReleaser version matches the one used in the [Release Action](.github/workflows/Release.yml).
1478
15-
### Unprotected Data
79+
---
1680

17-
#### Setup with `user-data`
81+
## Running the Service
1882

19-
User data can be injected into the cloud-init payload by making a PUT request to the `/cloud-init/{id}/user-data` endpoint. The request should contain a JSON-formatted body and can contain any arbritrary data desired.
83+
### Cluster Name
2084

21-
For example:
85+
Each instance of cloud-init is linked to a single SMD and operates for a single cluster. Until the cluster name is automatically available via your inventory system, you must specify it on the command line using the `-cluster-name` flag.
2286

87+
_Example:_
2388
```bash
24-
curl 'https://foobar.openchami.cluster/cloud-init/test/user-data' \
25-
-X PUT \
26-
-d '{
27-
"write_files": [{"content": "hello world", "path": "/etc/hello"}]
28-
}
29-
}}'
89+
-cluster-name venado
3090
```
3191

32-
...which will create a payload that will look like the example below. Notice that the `id` gets injected into the payload as the `name` property.
92+
### Fake SMD Mode
3393

34-
```json
35-
{
36-
"name": "IDENTIFIER",
37-
"cloud-init": {
38-
"userdata": {
39-
"write_files": [{"content": "hello world", "path": "/etc/hello"}]
40-
},
41-
"metadata": {...},
42-
}
43-
}
94+
For development purposes, you can run the cloud-init server without connecting to a real SMD instance. By setting the environment variable `CLOUD_INIT_SMD_SIMULATOR` to `true`, the service will generate a set of simulated nodes.
95+
96+
**Example command:**
97+
```bash
98+
CLOUD_INIT_SMD_SIMULATOR=true dist/cloud-init_darwin_arm64_v8.0/cloud-init-server -cluster-name venado -insecure -impersonation=true
4499
```
45100

46-
> [!NOTE]
47-
> Data can only be added manually into the `userdata` section of the payload. There is no way to add data directly to the `metadata` section.
101+
### Impersonation
102+
103+
By default, the service determines what configuration to return based on the IP address of the requesting node. For testing, impersonation routes can be enabled with the `-impersonation=true` flag.
104+
105+
**Sample commands:**
106+
```bash
107+
curl http://localhost:27777/cloud-init/admin/impersonation/x3000c1b1n1/meta-data
108+
```
48109

49-
`IDENTIFIER` can be:
110+
### Nocloud-net Datasource
50111

51-
- A node MAC address
52-
- A node xname
53-
- An SMD group name
112+
```bash
113+
cloud-init=enabled ds=nocloud-net;s=http://192.0.0.1/cloud-init
114+
```
115+
116+
---
117+
## Testing the Service
54118

55-
It may be easiest to add nodes to a group for testing, and upload a cloud-init configuration for that group to this server.
119+
The following testing steps (adapted from Demo.md) help you verify that the service is functioning correctly.
56120

57-
#### Generating Custom Metadata with Groups
121+
### Basic Endpoint Testing
58122

59-
The cloud-init server provides a group API to inject custom arbitrary data into the metadata section when requesting data with the specified `IDENTIFIER` mentioned above. The server will do a lookup for group labels from SMD and match the labels with the submitted data through the API.
123+
#### Start the Service in Fake SMD Mode (if desired):
60124

61-
For example, to add group data to the cloud-init server, we can make the following request:
125+
**Example:**
62126

63127
```bash
64-
curl -k http://127.0.0.1:27780/cloud-init/groups/install-nginx -d@install-nginx.json
128+
CLOUD_INIT_SMD_SIMULATOR=true dist/cloud-init_darwin_arm64_v8.0/cloud-init-server -cluster-name venado -insecure -impersonation=true
65129
```
66130

67-
The JSON data in `install-nginx.json`:
131+
#### Query the Standard Endpoints:
68132

69-
```json
70-
{
71-
"tier": "frontend",
72-
"application": "nginx"
73-
}
74-
```
75-
76-
Now, when we fetch data for a node in the `install-nginx` group, this data will be injected into the cloud-init metadata. Additionally, we can view all groups stored in cloud-init by making a GET request to the `/cloud-init/groups` endpoint.
133+
#### Meta-data:
77134

78135
```bash
79-
curl -k http://127.0.0.1:27780/cloud-init/x3000c1s7b53
136+
curl http://localhost:27777/cloud-init/meta-data
80137
```
81138

82-
And the response:
83-
84-
```json
85-
{
86-
"name": "x3000c1s7b53",
87-
"cloud-init": {
88-
"userdata": {...},
89-
"vendordata": {...},
90-
"metadata": {
91-
"groups": {
92-
"install-nginx": {
93-
"data": {
94-
"application": "nginx",
95-
"tier": "frontend"
96-
}
97-
}
98-
}
99-
}
100-
}
101-
}
102-
```
139+
You should see a YAML document with instance information (e.g., instance-id, cluster-name, etc.).
103140

104-
> [!NOTE]
105-
> The `IDENTIFIER` specified MUST be added to SMD for the data injection to work correctly.
141+
#### User-data:
106142

107-
#### Usage
143+
```bash
144+
curl http://localhost:27777/cloud-init/user-data
145+
```
108146

109-
Data is retrieved via HTTP GET requests to the `meta-data`, `user-data`, and `vendor-data` endpoints.
147+
For now, this returns a blank cloud-config document:
110148

111-
For example, one could download all cloud-init data for a node/group via `curl 'https://foobar.openchami.cluster/cloud-init/<IDENTIFIER>/{meta-data,user-data,vendor-data}'`.
149+
```yaml
150+
#cloud-config
151+
```
112152

113-
When retrieving data, `IDENTIFIER` can also be omitted entirely (e.g. `https://foobar.openchami.cluster/cloud-init/user-data`). In this case, the cloud-init server will attempt to look up the relevant xname based on the request's source IP address.
153+
#### Vendor-data:
114154

115-
Thus, the intended use case is to set nodes' cloud-init datasource URLs to `https://foobar.openchami.cluster/cloud-init/`, from which the cloud-init client will load its configuration data. Note that in this case, no `IDENTIFIER` is provided, so IP-based autodetection will be performed.
155+
```bash
156+
curl http://localhost:27777/cloud-init/vendor-data
157+
```
116158

117-
### JWT-Protected Data
159+
Vendor-data typically includes include-file directives pointing to group-specific YAML files:
118160

119-
#### Setup
161+
```yaml
162+
#include
163+
http://192.168.13.3:8080/all.yaml
164+
http://192.168.13.3:8080/login.yaml
165+
http://192.168.13.3:8080/compute.yaml
166+
```
120167

121-
The second endpoint, located at `/cloud-init-secure/`, restricts access to its cloud-init data behind a valid bearer token (i.e. a JWT).
122-
Storing data into this endpoint requires a valid access token, which we assume is stored in `$ACCESS_TOKEN`.
123-
The workflow described for unprotected data can be used, with the addition of the required authorization header, via e.g. `curl`'s `-H "Authorization: Bearer $ACCESS_TOKEN"`.
168+
## Group Handling and Overrides
124169

125-
#### Usage
170+
The service supports advanced configuration through group handling and instance overrides.
126171

127-
In order to access this protected data, nodes must also supply valid JWTs.
128-
Distribution of these tokens is out-of-scope for this repository, but may be handled via the [OpenCHAMI TPM-manager service](https://github.com/OpenCHAMI/TPM-manager).
172+
### Updating Group Data with a Simple Jinja Example
129173

130-
Once the JWT is known, it can be used to authenticate with the cloud-init server, via an invocation such as:
174+
This example sets a syslog aggregator via jinja templating. The group data is stored under the group name and then used in the vendor-data file.
131175

132176
```bash
133-
curl 'https://foobar.openchami.cluster/cloud-init-secure/<IDENTIFIER>/{meta-data,user-data,vendor-data}' \
134-
--create-dirs --output '/PATH/TO/DATA-DIR/#1' \
135-
--header "Authorization: Bearer $ACCESS_TOKEN"
177+
curl -X POST http://localhost:27777/cloud-init/admin/groups/ \
178+
-H "Content-Type: application/json" \
179+
-d '{
180+
"name": "x3001",
181+
"description": "Cabinet x3001",
182+
"data": {
183+
"syslog_aggregator": "192.168.0.1"
184+
},
185+
"file": {
186+
"content": "#template: jinja\n#cloud-config\nrsyslog:\n remotes: {x3001: {{ vendor_data.groups[\"x3001\"].syslog_aggregator }}}\n service_reload_command: auto\n",
187+
"encoding": "plain"
188+
}
189+
}'
136190
```
137191

138-
cloud-init (i.e. on the node) can then be pointed at `file:///PATH/TO/DATA-DIR/` as its datasource URL.
139-
140-
## Build/Install with goreleaser
192+
### Complex Base64 Example
141193

142-
This project uses [GoReleaser](https://goreleaser.com/) to automate releases and include additional build metadata such as commit info, build time, and versioning. Below is a guide on how to set up and build the project locally using GoReleaser.
194+
To add more sophisticated vendor-data (for example, installing the slurm client), you can encode a complete cloud-config in base64. (See the script in Demo.md for a complete example.)
143195

144-
### Environment Variables
196+
### Cluster Defaults and Instance Overrides
145197

146-
To include detailed build metadata, ensure the following environment variables are set:
198+
#### Set Cluster Defaults:
147199

148-
* __GIT_STATE__: Indicates whether there are uncommitted changes in the working directory. Set to clean if the repository is clean, or dirty if there are uncommitted changes.
149-
* __BUILD_HOST__: The hostname of the machine where the build is being performed.
150-
* __GO_VERSION__: The version of Go used for the build. GoReleaser uses this to ensure consistent Go versioning information.
151-
* __BUILD_USER__: The username of the person or system performing the build.
152-
153-
Set all the environment variables with:
154200
```bash
155-
export GIT_STATE=$(if git diff-index --quiet HEAD --; then echo 'clean'; else echo 'dirty'; fi)
156-
export BUILD_HOST=$(hostname)
157-
export GO_VERSION=$(go version | awk '{print $3}')
158-
export BUILD_USER=$(whoami)
201+
curl -X POST http://localhost:27777/cloud-init/admin/cluster-defaults/ \
202+
-H "Content-Type: application/json" \
203+
-d '{
204+
"cloud-provider": "openchami",
205+
"region": "us-west-2",
206+
"availability-zone": "us-west-2a",
207+
"cluster-name": "venado",
208+
"public-keys": [
209+
"ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEArV2...",
210+
"ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEArV3..."
211+
]
212+
}'
159213
```
160214

161-
### Building Locally with GoReleaser
162-
163-
Once the environment variables are set, you can build the project locally using GoReleaser in snapshot mode (to avoid publishing).
164-
215+
#### Override Instance Data:
165216

166-
Follow the installation instructions from [GoReleaser’s documentation](https://goreleaser.com/install/).
217+
```bash
218+
curl -X PUT http://localhost:27777/cloud-init/admin/instance-info/x3000c1b1n1 \
219+
-H "Content-Type: application/json" \
220+
-d '{
221+
"local-hostname": "compute-1",
222+
"instance-type": "t2.micro"
223+
}'
224+
```
225+
---
167226

168-
1. Run GoReleaser in snapshot mode with the --snapshot flag to create a local build without attempting to release it:
169-
```bash
170-
goreleaser release --snapshot --clean
171-
```
172-
2. Check the dist/ directory for the built binaries, which will include the metadata from the environment variables. You can inspect the binary output to confirm that the metadata was correctly embedded.
227+
## More Reading
173228

174-
__NOTE__ If you see errors, ensure that you are using the same version of goreleaser that is being used in the [Release Action](.github/workflows/Release.yml)
229+
- [Official cloud-init documentation](https://cloud-init.io/)
230+
- [OpenCHAMI TPM-manager service](https://github.com/OpenCHAMI/TPM-manager)
231+
- [GoReleaser Documentation](https://goreleaser.com/)
232+
- [SMD Documentation](https://github.com/OpenCHAMI/smd)

0 commit comments

Comments
 (0)