Skip to content

Commit c17d0e0

Browse files
minitrigafatih-acar
minitriga
and
fatih-acar
authored
Fix docs for configuration and update object storage docs (#7283)
* feat(docs): enhance configuration documentation with nested parameters and types * fix(docs): correct title and improve clarity in object storage documentation * fix(workflow): add pydantic-settings to Python environment setup * fix(docs): correct link formatting in object storage documentation * fix(docs): add missing period in object storage configuration reference link * Update docs/docs/topics/object-storage.mdx Co-authored-by: fatih-acar <[email protected]> --------- Co-authored-by: fatih-acar <[email protected]>
1 parent c28c016 commit c17d0e0

File tree

5 files changed

+378
-75
lines changed

5 files changed

+378
-75
lines changed

‎.github/workflows/keyword-scan.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,6 @@ jobs:
2020
with:
2121
python-version: 3.12
2222
- name: "Setup Python environment"
23-
run: "pip install invoke toml"
23+
run: "pip install invoke toml pydantic-settings"
2424
- name: Scan for prohibited keywords
2525
run: "invoke main.scan"

‎docs/_templates/infrahub_config.j2‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,4 +34,17 @@ Here are a few common methods of setting environmental variables:
3434
{% for parameter in section.parameters %}
3535
| `{{ parameter.env }}` | {{ parameter.description }} | {{ parameter.type }} | {{ parameter.default }} |
3636
{% endfor %}
37-
{% endfor %}
37+
38+
{%- for parameter in section.parameters %}
39+
{% if parameter.nested_parameters %}
40+
41+
### {{ parameter.env or parameter.name | upper }}
42+
43+
| Name | Description | Type | Default |
44+
|------|-------------|------|---------|
45+
{% for nested in parameter.nested_parameters %}
46+
| `{{ nested.env or nested.name | upper | replace(" ", "_") }}` | {{ nested.description }} | {{ nested.type }} | {{ nested.default }} |
47+
{% endfor -%}
48+
{% endif -%}
49+
{% endfor -%}
50+
{% endfor -%}

‎docs/docs/reference/configuration.mdx‎

Lines changed: 98 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ The HTTP settings control how Infrahub interacts with external HTTP servers. Thi
7676

7777
| Name | Description | Type | Default |
7878
|------|-------------|------|---------|
79-
| `INFRAHUB_DB_INFRAHUB_DB_TYPE` | None | None | neo4j |
79+
| `INFRAHUB_DB_INFRAHUB_DB_TYPE` | None | string (neo4j, memgraph) | neo4j |
8080
| `INFRAHUB_DB_PROTOCOL` | None | string | bolt |
8181
| `INFRAHUB_DB_USERNAME` | None | string | neo4j |
8282
| `INFRAHUB_DB_PASSWORD` | None | string | admin |
@@ -112,7 +112,7 @@ Configuration settings for the message bus.
112112
| `INFRAHUB_BROKER_MAXIMUM_MESSAGE_RETRIES` | The maximum number of retries that are attempted for failed messages | integer | 10 |
113113
| `INFRAHUB_BROKER_MAXIMUM_CONCURRENT_MESSAGES` | The maximum number of concurrent messages fetched by each worker | integer | 2 |
114114
| `INFRAHUB_BROKER_VIRTUALHOST` | The virtual host to connect to | string | / |
115-
| `INFRAHUB_BROKER_DRIVER` | None | None | rabbitmq |
115+
| `INFRAHUB_BROKER_DRIVER` | None | string (rabbitmq, nats) | rabbitmq |
116116

117117
## Cache
118118

@@ -122,7 +122,7 @@ Configuration settings for the message bus.
122122
| `INFRAHUB_CACHE_ADDRESS` | None | string | localhost |
123123
| `INFRAHUB_CACHE_PORT` | Specified if running on a non default port (6379) | None | None |
124124
| `INFRAHUB_CACHE_DATABASE` | Id of the database to use | integer | 0 |
125-
| `INFRAHUB_CACHE_DRIVER` | None | None | redis |
125+
| `INFRAHUB_CACHE_DRIVER` | None | string (redis, nats) | redis |
126126
| `INFRAHUB_CACHE_USERNAME` | None | string | |
127127
| `INFRAHUB_CACHE_PASSWORD` | None | string | |
128128
| `INFRAHUB_CACHE_TLS_ENABLED` | Indicates if TLS is enabled for the connection | boolean | False |
@@ -137,10 +137,10 @@ Configuration settings for the message bus.
137137
| `INFRAHUB_WORKFLOW_ADDRESS` | None | string | localhost |
138138
| `INFRAHUB_WORKFLOW_PORT` | Specified if running on a non default port. | None | None |
139139
| `INFRAHUB_WORKFLOW_TLS_ENABLED` | Indicates if TLS is enabled for the connection | boolean | False |
140-
| `INFRAHUB_WORKFLOW_DRIVER` | None | None | worker |
140+
| `INFRAHUB_WORKFLOW_DRIVER` | None | string (local, worker) | worker |
141141
| `INFRAHUB_WORKFLOW_DEFAULT_WORKER_TYPE` | None | string | infrahubasync |
142142
| `INFRAHUB_WORKFLOW_EXTRA_LOGGERS` | A list of additional logger that will be captured during task execution. | array[string] | None |
143-
| `INFRAHUB_WORKFLOW_EXTRA_LOG_LEVEL` | Log level applied to all extra loggers. | None | INFO |
143+
| `INFRAHUB_WORKFLOW_EXTRA_LOG_LEVEL` | Log level applied to all extra loggers. | string (CRITICAL, ERROR, WARNING, INFO, DEBUG) | INFO |
144144
| `INFRAHUB_WORKFLOW_WORKER_POLLING_INTERVAL` | Specify how often the worker should poll the server for tasks (sec) | integer | 2 |
145145

146146
## Miscellaneous
@@ -185,20 +185,109 @@ Configuration settings for the message bus.
185185
| `INFRAHUB_SECURITY_REFRESH_TOKEN_LIFETIME` | Lifetime of refresh token in seconds | integer | 2592000 |
186186
| `INFRAHUB_SECURITY_SECRET_KEY` | The secret key used to validate authentication tokens | string | None |
187187
| `INFRAHUB_SECURITY_OAUTH2_PROVIDERS` | The selected OAuth2 providers | array | None |
188-
| `INFRAHUB_SECURITY_OAUTH2_PROVIDER_SETTINGS` | None | None | None |
188+
| `INFRAHUB_SECURITY_OAUTH2_PROVIDER_SETTINGS` | None | object | Check nested parameters |
189189
| `INFRAHUB_SECURITY_OIDC_PROVIDERS` | The selected OIDC providers | array | None |
190-
| `INFRAHUB_SECURITY_OIDC_PROVIDER_SETTINGS` | None | None | None |
190+
| `INFRAHUB_SECURITY_OIDC_PROVIDER_SETTINGS` | None | object | Check nested parameters |
191191
| `INFRAHUB_SECURITY_RESTRICT_UNTRUSTED_JINJA2_FILTERS` | Indicates if untrusted Jinja2 filters should be disallowed for computed attributes | boolean | True |
192192
| `INFRAHUB_SECURITY_SSO_USER_DEFAULT_GROUP` | Name of the group to which users authenticated via SSO will belong if not provided by identity provider | None | None |
193193

194+
### INFRAHUB_SECURITY_OAUTH2_PROVIDER_SETTINGS
195+
196+
| Name | Description | Type | Default |
197+
|------|-------------|------|---------|
198+
| `INFRAHUB_OAUTH2_GOOGLE_ICON` | None | string | mdi:google |
199+
| `INFRAHUB_OAUTH2_GOOGLE_USERINFO_METHOD` | None | string (post, get) | get |
200+
| `INFRAHUB_OAUTH2_GOOGLE_CLIENT_ID` | Client ID of the application created in the auth provider | string | None |
201+
| `INFRAHUB_OAUTH2_GOOGLE_CLIENT_SECRET` | Client secret as defined in auth provider | string | None |
202+
| `INFRAHUB_OAUTH2_GOOGLE_AUTHORIZATION_URL` | None | string | https://accounts.google.com/o/oauth2/auth |
203+
| `INFRAHUB_OAUTH2_GOOGLE_TOKEN_URL` | None | string | https://oauth2.googleapis.com/token |
204+
| `INFRAHUB_OAUTH2_GOOGLE_USERINFO_URL` | None | string | https://www.googleapis.com/oauth2/v3/userinfo |
205+
| `INFRAHUB_OAUTH2_GOOGLE_SCOPES` | None | array[string] | None |
206+
| `INFRAHUB_OAUTH2_GOOGLE_DISPLAY_LABEL` | None | string | Google |
207+
| `INFRAHUB_OAUTH2_GOOGLE_FETCH_GROUPS` | Whether to use Cloud Identity API to fetch user groups. Note: requires additional scopes: https://www.googleapis.com/auth/cloud-identity.groups.readonly | boolean | False |
208+
| `INFRAHUB_OAUTH2_GOOGLE_CLOUDIDENTITY_URL` | Google Cloud endpoint for Cloud Identity. Using searchDirectGroups by default because it is available for the Free plan | string | https://cloudidentity.googleapis.com/v1/groups/-/memberships:searchDirectGroups |
209+
| `INFRAHUB_OAUTH2_PROVIDER1_ICON` | None | string | mdi:account-key |
210+
| `INFRAHUB_OAUTH2_PROVIDER1_USERINFO_METHOD` | None | string (post, get) | get |
211+
| `INFRAHUB_OAUTH2_PROVIDER1_CLIENT_ID` | Client ID of the application created in the auth provider | string | None |
212+
| `INFRAHUB_OAUTH2_PROVIDER1_CLIENT_SECRET` | Client secret as defined in auth provider | string | None |
213+
| `INFRAHUB_OAUTH2_PROVIDER1_AUTHORIZATION_URL` | None | string | None |
214+
| `INFRAHUB_OAUTH2_PROVIDER1_TOKEN_URL` | None | string | None |
215+
| `INFRAHUB_OAUTH2_PROVIDER1_USERINFO_URL` | None | string | None |
216+
| `INFRAHUB_OAUTH2_PROVIDER1_SCOPES` | None | array[string] | None |
217+
| `INFRAHUB_OAUTH2_PROVIDER1_DISPLAY_LABEL` | None | string | Single Sign on |
218+
| `INFRAHUB_OAUTH2_PROVIDER2_ICON` | None | string | mdi:account-key |
219+
| `INFRAHUB_OAUTH2_PROVIDER2_USERINFO_METHOD` | None | string (post, get) | get |
220+
| `INFRAHUB_OAUTH2_PROVIDER2_CLIENT_ID` | Client ID of the application created in the auth provider | string | None |
221+
| `INFRAHUB_OAUTH2_PROVIDER2_CLIENT_SECRET` | Client secret as defined in auth provider | string | None |
222+
| `INFRAHUB_OAUTH2_PROVIDER2_AUTHORIZATION_URL` | None | string | None |
223+
| `INFRAHUB_OAUTH2_PROVIDER2_TOKEN_URL` | None | string | None |
224+
| `INFRAHUB_OAUTH2_PROVIDER2_USERINFO_URL` | None | string | None |
225+
| `INFRAHUB_OAUTH2_PROVIDER2_SCOPES` | None | array[string] | None |
226+
| `INFRAHUB_OAUTH2_PROVIDER2_DISPLAY_LABEL` | None | string | Single Sign on |
227+
228+
### INFRAHUB_SECURITY_OIDC_PROVIDER_SETTINGS
229+
230+
| Name | Description | Type | Default |
231+
|------|-------------|------|---------|
232+
| `INFRAHUB_OIDC_GOOGLE_ICON` | None | string | mdi:google |
233+
| `INFRAHUB_OIDC_GOOGLE_DISPLAY_LABEL` | None | string | Google |
234+
| `INFRAHUB_OIDC_GOOGLE_USERINFO_METHOD` | None | string (post, get) | get |
235+
| `INFRAHUB_OIDC_GOOGLE_CLIENT_ID` | Client ID of the application created in the auth provider | string | None |
236+
| `INFRAHUB_OIDC_GOOGLE_CLIENT_SECRET` | Client secret as defined in auth provider | string | None |
237+
| `INFRAHUB_OIDC_GOOGLE_DISCOVERY_URL` | None | string | https://accounts.google.com/.well-known/openid-configuration |
238+
| `INFRAHUB_OIDC_GOOGLE_SCOPES` | None | array[string] | None |
239+
| `INFRAHUB_OIDC_GOOGLE_FETCH_GROUPS` | Whether to use Cloud Identity API to fetch user groups. Note: requires additional scope: https://www.googleapis.com/auth/cloud-identity.groups.readonly | boolean | False |
240+
| `INFRAHUB_OIDC_GOOGLE_CLOUDIDENTITY_URL` | Google Cloud endpoint for Cloud Identity. Using searchDirectGroups by default because it is available for the Free plan | string | https://cloudidentity.googleapis.com/v1/groups/-/memberships:searchDirectGroups |
241+
| `INFRAHUB_OIDC_PROVIDER1_ICON` | None | string | mdi:account-key |
242+
| `INFRAHUB_OIDC_PROVIDER1_DISPLAY_LABEL` | None | string | Single Sign on |
243+
| `INFRAHUB_OIDC_PROVIDER1_USERINFO_METHOD` | None | string (post, get) | get |
244+
| `INFRAHUB_OIDC_PROVIDER1_CLIENT_ID` | Client ID of the application created in the auth provider | string | None |
245+
| `INFRAHUB_OIDC_PROVIDER1_CLIENT_SECRET` | Client secret as defined in auth provider | string | None |
246+
| `INFRAHUB_OIDC_PROVIDER1_DISCOVERY_URL` | The OIDC discovery URL xyz/.well-known/openid-configuration | string | None |
247+
| `INFRAHUB_OIDC_PROVIDER1_SCOPES` | None | array[string] | None |
248+
| `INFRAHUB_OIDC_PROVIDER2_ICON` | None | string | mdi:account-key |
249+
| `INFRAHUB_OIDC_PROVIDER2_DISPLAY_LABEL` | None | string | Single Sign on |
250+
| `INFRAHUB_OIDC_PROVIDER2_USERINFO_METHOD` | None | string (post, get) | get |
251+
| `INFRAHUB_OIDC_PROVIDER2_CLIENT_ID` | Client ID of the application created in the auth provider | string | None |
252+
| `INFRAHUB_OIDC_PROVIDER2_CLIENT_SECRET` | Client secret as defined in auth provider | string | None |
253+
| `INFRAHUB_OIDC_PROVIDER2_DISCOVERY_URL` | The OIDC discovery URL xyz/.well-known/openid-configuration | string | None |
254+
| `INFRAHUB_OIDC_PROVIDER2_SCOPES` | None | array[string] | None |
255+
256+
## Storage
257+
258+
| Name | Description | Type | Default |
259+
|------|-------------|------|---------|
260+
| `INFRAHUB_STORAGE_DRIVER` | None | string (local, s3) | local |
261+
| `INFRAHUB_STORAGE_LOCAL` | None | object | Check nested parameters |
262+
| `INFRAHUB_STORAGE_S3` | None | object | Check nested parameters |
263+
264+
### INFRAHUB_STORAGE_LOCAL
265+
266+
| Name | Description | Type | Default |
267+
|------|-------------|------|---------|
268+
| `INFRAHUB_STORAGE_LOCAL_PATH` | None | string | /opt/infrahub/storage |
269+
270+
### INFRAHUB_STORAGE_S3
271+
272+
| Name | Description | Type | Default |
273+
|------|-------------|------|---------|
274+
| `AWS_ACCESS_KEY_ID` | None | string | |
275+
| `AWS_SECRET_ACCESS_KEY` | None | string | |
276+
| `INFRAHUB_STORAGE_BUCKET_NAME` | None | string | |
277+
| `INFRAHUB_STORAGE_ENDPOINT_URL` | None | string | |
278+
| `INFRAHUB_STORAGE_USE_SSL` | None | boolean | True |
279+
| `INFRAHUB_STORAGE_DEFAULT_ACL` | None | string | private |
280+
| `INFRAHUB_STORAGE_QUERYSTRING_AUTH` | None | boolean | False |
281+
| `INFRAHUB_STORAGE_CUSTOM_DOMAIN` | None | string | |
282+
194283
## Trace
195284

196285
| Name | Description | Type | Default |
197286
|------|-------------|------|---------|
198287
| `INFRAHUB_TRACE_ENABLE` | None | boolean | False |
199288
| `INFRAHUB_TRACE_INSECURE` | Use insecure connection (HTTP) if True, otherwise use secure connection (HTTPS) | boolean | True |
200-
| `INFRAHUB_TRACE_EXPORTER_TYPE` | Type of exporter to be used for tracing | None | console |
201-
| `INFRAHUB_TRACE_EXPORTER_PROTOCOL` | Protocol to be used for exporting traces | None | grpc |
289+
| `INFRAHUB_TRACE_EXPORTER_TYPE` | Type of exporter to be used for tracing | string (console, otlp) | console |
290+
| `INFRAHUB_TRACE_EXPORTER_PROTOCOL` | Protocol to be used for exporting traces | string (grpc, http/protobuf) | grpc |
202291
| `INFRAHUB_TRACE_EXPORTER_ENDPOINT` | OTLP endpoint for exporting traces | None | None |
203292

204293
## Experimental features

‎docs/docs/topics/object-storage.mdx‎

Lines changed: 53 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,53 +1,77 @@
11
---
2-
title: Object-storage
2+
title: Object storage
33
---
44

5-
# Object-storage
5+
# Object storage
66

7-
Infrahub provides an interface to store and retrieve files or objects in an object-storage. The object-storage is used internally within Infrahub to store rendered artifacts, but it can be used to store any text based file or content.
7+
Infrahub provides an interface to store and retrieve files or objects in object storage. Object storage is used internally within Infrahub to store rendered artifacts, but you can also use it to store any text-based file or content.
88

9-
You can interface with the object-storage through the REST API and Python SDK.
10-
See the [object-storage guide](../guides/object-storage) and [object-storage Python SDK]($(base_url)python-sdk/guides/object-storage) guide for more information.
9+
You can interact with object storage through the REST API and Python SDK. For practical steps, see the [object storage guide](../guides/object-storage) and the [object storage Python SDK guide]($(base_url)python-sdk/guides/object-storage).
1110

12-
## Supported backends
11+
## Supported storage backends
1312

14-
At this moment Infrahub supports using local storage, or AWS S3 storage backends. Support for other storage backends will be added in the future.
13+
Infrahub supports two storage backends:
1514

16-
### Local storage
15+
- Local storage (configured by default)
16+
- AWS S3 (or S3-compatible) storage
17+
18+
You configure the backend by setting the appropriate environment variables. These variables are typically set in your Docker Compose YAML file or in the deployment environment for your containers. See the [Infrahub configuration reference](../reference/configuration.mdx) for a full list of options.
19+
20+
---
1721

18-
Infrahub can use local storage as a storage backend. It can be any directory on a filesystem that is attached to the system on which Infrahub runs. The only requirement is that all the Infrahub API servers and Task workers need access to the filesystem.
22+
### Local storage
1923

20-
To setup Infrahub to use local storage backend you can use the following configuration:
24+
By default, Infrahub uses local storage for artifacts and files. No additional configuration is required unless you want to change the storage location.
2125

22-
```toml
23-
[storage]
24-
driver = "local"
26+
To explicitly configure local storage, set the following environment variables in your Docker Compose YAML file or deployment environment:
2527

26-
[storage.local]
27-
path = "/opt/infrahub/storage/"
28+
```yaml
29+
# ... other configuration ...
30+
INFRAHUB_STORAGE_DRIVER: "local"
31+
INFRAHUB_STORAGE_LOCAL_PATH: "/opt/infrahub/storage"
32+
# ... other configuration ...
2833
```
2934

30-
More details can be found in the [configuration documentation](../reference/configuration).
35+
All Infrahub API servers and task workers must have access to the specified directory.
36+
37+
---
3138

3239
### AWS S3 storage
3340

34-
Infrahub can use AWS S3 or any other S3 API compatible storage as a storage system.
41+
To use AWS S3 or any S3-compatible storage, set the following environment variables in your Docker Compose YAML file or deployment environment:
42+
43+
```yaml
44+
# ... other configuration ...
45+
INFRAHUB_STORAGE_DRIVER: "s3"
46+
AWS_ACCESS_KEY_ID: "my_access_key"
47+
AWS_SECRET_ACCESS_KEY: "secret_access_key"
48+
INFRAHUB_STORAGE_BUCKET_NAME: "my-infrahub-bucket"
49+
INFRAHUB_STORAGE_ENDPOINT_URL: "s3.eu-central-1.amazonaws.com"
50+
# ... other configuration ...
51+
```
52+
53+
For all available configuration options and environment variables, refer to the [Infrahub configuration reference](../reference/configuration.mdx#infrahub_storage_s3).
3554

36-
To setup Infrahub to use AWS S3 as a storage backend you can use the following configuration:
55+
---
3756

38-
```toml
39-
[storage]
40-
driver = "s3"
57+
:::info
58+
You can switch between storage backends by changing the value of `INFRAHUB_STORAGE_DRIVER` and updating the relevant environment variables in your deployment configuration.
59+
:::
4160

42-
[storage.s3]
43-
access_key_id = "my_access_key"
44-
secret_access_key = "secret_access_key"
45-
bucket_name = "my-infrahub-bucket"
46-
endpoint_url = "https://s3.eu-central-1.amazonaws.com"
47-
```
61+
:::warning
62+
If you change the storage driver (for example, from `local` to `s3` or vice versa), **existing artifacts will not be automatically migrated**.
63+
To regenerate artifacts:
4864

49-
More details can be found in the [configuration documentation](../reference/configuration).
65+
- You must either delete the old artifacts, or
66+
- Manually update their checksum values and regenerate them.
67+
68+
Failing to do so may result in missing or stale artifact files.
69+
:::
70+
71+
---
5072

5173
## Object model
5274

53-
Objects or files stored within the object-storage, are identified by an identifier (UUID). An identifier is assigned to the object at time of creation within the object-storage. Interactions with the object then happen leveraging that identifier.
75+
Objects or files stored within object storage are identified by a unique identifier (UUID). This identifier is assigned when the object is created. All interactions with the object use this identifier.
76+
77+
For more details on configuration options, see the [Infrahub configuration reference](../reference/configuration.mdx).

0 commit comments

Comments
 (0)