Skip to content

Commit 3a0c480

Browse files
wvandeunwvandeun
authored
Merge pull request #5697 from opsmill/wvd-20250202-docs-new-transform-artifact-guide
Rework documentation for atifacts and transformations
2 parents 48a18f4 + 4ae22c9 commit 3a0c480

File tree

8 files changed

+233
-137
lines changed

8 files changed

+233
-137
lines changed

‎docs/docs/guides/artifact.mdx‎

Lines changed: 58 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -8,15 +8,56 @@ import ReactPlayer from 'react-player/youtube'
88

99
:::note
1010

11-
This guide has the assumption that you followed the [Creating a Jinja Rendered File (Transform)](./jinja2-transform.mdx) or [Creating a Python transform](./python-transform.mdx) guide as well as [Creating a Group](./groups.mdx) guide
12-
This is a prerequisite.
11+
This guide has the assumption that you followed the [Creating a Jinja Rendered File (Transform)](./jinja2-transform.mdx) or [Creating a Python transform](./python-transform.mdx) guide. This is a prerequisite.
1312

1413
:::
1514

1615
The goal of this guide is to define an artifact for a Jinja rendered Transform or Python transform in Infrahub. We will be using the following steps.
1716

18-
1. Creating an artifact definition
19-
2. Accessing the artifacts
17+
1. Modifying the schema
18+
2. Creating a group and adding member nodes
19+
2. Creating an artifact definition
20+
3. Accessing the artifacts
21+
22+
## Modifying the node schema
23+
24+
Before we can generate an artifact for a node, we need to modify the schema. The node will need to inherit from the `CoreArtifactTarget` generic.
25+
26+
```yaml
27+
---
28+
version: "1.0"
29+
nodes:
30+
- name: Device
31+
namespace: Network
32+
display_labels:
33+
- name__value
34+
inherit_from:
35+
- CoreArtifactTarget
36+
attributes:
37+
- name: name
38+
kind: Text
39+
label: Name
40+
optional: false
41+
unique: true
42+
- name: description
43+
kind: Text
44+
label: Description
45+
optional: true
46+
```
47+
48+
We then have to load the modified schema definition into Infrahub.
49+
50+
```bash
51+
infrahubctl schema load /tmp/schema.yml
52+
```
53+
54+
## Creating a group and adding members
55+
56+
We need to create a group for every Artifact Definition that we will define. An artifact will be generated for every member that is part of that group.
57+
58+
Create a Standard Group with the name `DeviceGroup` and add `NetworkDevice` `switch1`, `switch2` and `switch3` as a member.
59+
60+
More information can be found in the [Creating a group guide](./groups.mdx).
2061

2162
<center>
2263
<ReactPlayer url='https://www.youtube.com/watch?v=ASGMKZVLCbY' light />
@@ -26,23 +67,23 @@ The goal of this guide is to define an artifact for a Jinja rendered Transform o
2667

2768
In the last step we need to define an Artifact definition, which groups together a [transformation](../topics/transformation.mdx) with a target group and forms the definition of the artifact. Artifact definitions can be created via the frontend, via GraphQL or via a [Git repository](../topics/repository.mdx). In this guide we will be using the Git repository.
2869

29-
Add the following contents to the end of the `.infrahub.yml` file at the root of the `tags_render` repository.
70+
Add the following contents to the end of the `.infrahub.yml` file at the root of the `device_config_render` repository.
3071

3172
```yaml
3273
artifact_definitions:
33-
- name: "tags_config_file"
34-
artifact_name: "Tags configuration file"
74+
- name: "device_configuration"
75+
artifact_name: "Device configuration file"
3576
parameters:
36-
tag: "name__value"
77+
name: "name__value"
3778
content_type: "text/plain"
38-
targets: "TagConfigGroup"
39-
transformation: "my-transform"
79+
targets: "DeviceGroup"
80+
transformation: "device_config_transform"
4081
```
4182
4283
This defines an artifact with the following properties:
4384
4485
- **name**: a unique name for the artifact
45-
- **parameters**: the parameter to pass to the transformation GraphQL query, in this case this we will pass the name of the object (tag) as the tag parameter
86+
- **parameters**: the parameter to pass to the transformation GraphQL query, in this case this we will pass the name of the object (device) as the name parameter
4687
- **content type**: the content type for the resulting artifact
4788
- **targets**: the name of a group of which the members will be a target for this artifact
4889
- **transformation**: the Jinja2 or Python transformation that should be used
@@ -53,11 +94,11 @@ Commit the changes to the repository and push them to the Git server
5394

5495
```shell
5596
git add .
56-
git commit -m "add tags_config_file artifact definition"
97+
git commit -m "add device_configuration artifact definition"
5798
git push origin main
5899
```
59100

60-
The artifact definition will be created in the database, when the Task worker(s) notice the change in the Git repository. The `tags_config_file` should now be visible in the Artifact Definition view in the web interface.
101+
The artifact definition will be created in the database, when the Task worker(s) notice the change in the Git repository. The `device_configuration` should now be visible in the Artifact Definition view in the web interface.
61102
![Artifact Definition](../media/guides/artifact/artifact_definition.png)
62103

63104
## Accessing the artifacts
@@ -69,6 +110,8 @@ The artifacts are generated by the git-agents. You can find the resulting artifa
69110
Open an artifact to get the result.
70111

71112
![Artifact detail](../media/guides/artifact/artifact_detail.png)
72-
You can download the artifact by clicking on the `Storage Id` or alternatively through the rest API endpoint `http://<INFRAHUB_HOST:INFRAHUB_PORT>/api/storage/object/<storage_id>`
113+
You can download the artifact by clicking on the download button, or by using the REST API endpoint `http://<INFRAHUB_HOST:INFRAHUB_PORT>/api/storage/object/<storage_id>`. The `Storage Id` can be copied from the menu next to the artifact name.
114+
115+
![Artifact menu](../media/guides/artifact/artifact_menu.png)
73116

74-
Optionally, Infrahub can create a relation between a node and an artifact, by inheriting from the CoreArtifactTarget generic in the schema of the node. This is outside the scope of this guide, since the BuiltinTag node does not inherit from CoreArtifactTarget. More information can be found in the [artifact](../topics/artifact.mdx) topic.
117+
Alternatively we can access the artifact of an object, by navigating to the object detail view. Navigate to device `switch1`'s detail page, from the *Artifacts* tab you can access all the artifacts that were generated for this object.

‎docs/docs/guides/jinja2-transform.mdx‎

Lines changed: 75 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -16,18 +16,48 @@ The goal of this guide is to develop a Jinja Transform and add it to Infrahub, w
1616
6. Add the repository to Infrahub as an external repository
1717
7. Validate that the transform works by using the render API endpoint
1818

19-
In this guide we are going to work with the builtin tag objects in Infrahub. It won't provide a rendered template that is very useful, the goal is instead to show how the Jinja Rendering works. Once you have mastered the basics you will be ready to go on to create more advanced template.
19+
## 1. Loading a schema
2020

21-
## 1. Creating a query to collect the desired data
21+
In this guide we are going to work with a very simplistic network device model. I won't provide a rendered template that is very useful, the goal is instead to show how the Jinja Rendering works. Once you have mastered the basics you will be ready to go on to create more advanced template.
22+
23+
```yaml
24+
---
25+
version: "1.0"
26+
nodes:
27+
- name: Device
28+
namespace: Network
29+
display_labels:
30+
- name__value
31+
attributes:
32+
- name: name
33+
kind: Text
34+
label: Name
35+
optional: false
36+
unique: true
37+
- name: description
38+
kind: Text
39+
label: Description
40+
optional: true
41+
```
42+
43+
Store the schema as a YAML file on your local disk, and load the schema into Infrahub using the following command
44+
45+
```bash
46+
infrahubctl schema load /path/to/schema.yml
47+
```
48+
49+
More information on loading schema files into Infrahub can be found [here](../guides/import-schema#load-a-schema-file).
50+
51+
## 2. Creating a query to collect the desired data
2252

2353
As the first step we need to have some data in the database to actually query.
2454

25-
Create three tags, called "red", "green", "blue", either using the frontend or by submitting three GraphQL mutations as per below (just swapping out the name of the color each time).
55+
Create three devices, called "switch1", "switch2", "switch3", either using the frontend or by submitting three GraphQL mutations as per below (just swapping out the name of the color each time).
2656

2757
```graphql
28-
mutation CreateTags {
29-
BuiltinTagCreate(
30-
data: {name: {value: "red"}, description: {value: "The red tag"}}
58+
mutation CreateDevice {
59+
NetworkDeviceCreate(
60+
data: {name: {value: "switch1"}, description: {value: "This is device switch1"}}
3161
) {
3262
ok
3363
object {
@@ -40,8 +70,8 @@ mutation CreateTags {
4070
The next step is to create a query that returns the data we just created. The rest of this guide assumes that the following query will return a response similar to the response below the query.
4171

4272
```graphql
43-
query TagsQuery {
44-
BuiltinTag {
73+
query DeviceQuery {
74+
NetworkDevice {
4575
edges {
4676
node {
4777
name {
@@ -61,35 +91,35 @@ Response to the tags query:
6191
```json
6292
{
6393
"data": {
64-
"BuiltinTag": {
94+
"NetworkDevice": {
6595
"edges": [
6696
{
6797
"node": {
6898
"name": {
69-
"value": "blue"
99+
"value": "switch1"
70100
},
71101
"description": {
72-
"value": "The blue tag"
102+
"value": "This is device switch1"
73103
}
74104
}
75105
},
76106
{
77107
"node": {
78108
"name": {
79-
"value": "green"
109+
"value": "switch2"
80110
},
81111
"description": {
82-
"value": "The green tag"
112+
"value": "This is device switch2"
83113
}
84114
}
85115
},
86116
{
87117
"node": {
88118
"name": {
89-
"value": "red"
119+
"value": "switch3"
90120
},
91121
"description": {
92-
"value": "The red tag"
122+
"value": "This is device switch3"
93123
}
94124
}
95125
}
@@ -99,19 +129,19 @@ Response to the tags query:
99129
}
100130
```
101131

102-
While it would be possible to create a transform that targets all of these tags, for example if you want to create a report, the goal for us is to be able to focus on one of these objects. For this reason we need to modify the query from above to take an input parameter so that we can filter the result to what we want.
132+
While it would be possible to create a transform that targets all of these devices, for example if you want to create a report, the goal for us is to be able to focus on one of these objects. For this reason we need to modify the query from above to take an input parameter so that we can filter the result to what we want.
103133

104134
Create a local directory on your computer.
105135

106136
```shell
107-
mkdir tags_render
137+
mkdir device_config_render
108138
```
109139

110-
Then save the below query as a text file named tags_query.gql.
140+
Then save the below query as a text file named `device_config.gql`.
111141

112142
```graphql
113-
query TagsQuery($tag: String!) {
114-
BuiltinTag(name__value: $tag) {
143+
query DeviceQuery($name: String!) {
144+
NetworkDevice(name__value: $name) {
115145
edges {
116146
node {
117147
name {
@@ -126,30 +156,31 @@ query TagsQuery($tag: String!) {
126156
}
127157
```
128158

129-
Here the query will require an input parameter called `$name` what will refer to the name of each tag. When we want to query for the red tag the input variables to the query would look like this:
159+
Here the query will require an input parameter called `$name` what will refer to the name of each device. When we want to query for device switch1, the input variables to the query would look like this:
130160

131161
```json
132162
{
133-
"tag": "red"
163+
"name": "switch1"
134164
}
135165
```
136166

137-
## 2. Create the Jinja template
167+
## 3. Create the Jinja template
138168

139-
The next step is to create the actual Jinja Template file. Create a file called tags_tpl.j2
169+
The next step is to create the actual Jinja Template file. Create a file called `device_config.j2`.
140170

141171
```jinja2
142-
{% if data.BuiltinTag.edges and data.BuiltinTag.edges is iterable %}
143-
{% for tag in data["BuiltinTag"]["edges"] %}
144-
{% set tag_name = tag.node.name.value %}
145-
{% set tag_description = tag.node.description.value %}
146-
{{ tag_name }}
147-
description: {{ tag_description }}
172+
{% if data.NetworkDevice.edges and data.NetworkDevice.edges is iterable %}
173+
{% for device in data["NetworkDevice"]["edges"] %}
174+
{% set device_name = device.node.name.value %}
175+
{% set device_description = device.node.description.value %}
176+
hostname {{ device_name }}
177+
description "{{ device_description }}"
178+
end
148179
{% endfor %}
149180
{% endif %}
150181
```
151182

152-
## 3. Create a .infrahub.yml file
183+
## 4. Create a .infrahub.yml file
153184

154185
In the .infrahub.yml file you define what transforms you have in your repository that you want to make available for Infrahub.
155186

@@ -158,26 +189,26 @@ Create a .infrahub.yml file in the root of the directory.
158189
```yaml
159190
---
160191
jinja2_transforms:
161-
- name: my-jinja2-transform # Unique name for your transform
162-
description: "short description" # (optional)
163-
query: "tags_query" # Name or ID of the GraphQLQuery
164-
template_path: "tags_tpl.j2" # Path to the main Jinja2 template
192+
- name: device_config_transform # Unique name for your transform
193+
description: "device config transform" # (optional)
194+
query: "device_config_query" # Name or ID of the GraphQLQuery
195+
template_path: "device_config.j2" # Path to the main Jinja2 template
165196

166197
queries:
167-
- name: tags_query # Name of the GraphQLQuery
168-
file_path: "tags_query.gql" # Path to the main Jinja2 template
198+
- name: device_config_query # Name of the GraphQLQuery
199+
file_path: "device_config.gql" # Path to the main Jinja2 template
169200
```
170201
171202
> The main Jinja2 template can import other templates
172203
173204
Three parts here are required, first the `name` of the transform which should be unique across Infrahub, `query` the GraphqlQuery linked to our transform and also the `template_path` that should point to the Jinja2 file within the repository.
174205

175-
## 4. Create a Git repository
206+
## 5. Create a Git repository
176207

177-
Within the `tags_render` folder you should now have tree files:
208+
Within the `device_config_render` folder you should now have tree files:
178209

179-
* tags_query.gql: Contains the GraphQL query
180-
* tags_tpl.j2: Contains the Jinja2 Template
210+
* device_config.gql: Contains the GraphQL query
211+
* device_config.j2: Contains the Jinja2 Template
181212
* .infrahub.yml: Contains the definition for the transform
182213

183214
Before we can test our transform we must add the files to a local Git repository.
@@ -188,7 +219,7 @@ git add .
188219
git commit -m "First commit"
189220
```
190221

191-
## 5. Test the render using infrahubctl
222+
## 6. Test the render using infrahubctl
192223

193224
Using infrahubctl you can first verify that the `.infrahub.yml` file is formatted correctly by listing available transforms.
194225

@@ -224,11 +255,11 @@ If `--branch` is not provided it will automatically use the name of the local br
224255

225256
:::
226257

227-
## 6. Adding the repository to Infrahub
258+
## 7. Adding the repository to Infrahub
228259

229260
In order to avoid having the same instructions over and over please refer to the guide [adding a repository to Infrahub](../guides/repository) in order to sync the repository you created and make it available within Infrahub.
230261

231-
## 7. Accessing the Transform from the API
262+
## 8. Accessing the Transform from the API
232263

233264
A transform can be rendered on demand via the REST API with the endpoint: `https://<host>/api/transform/jinja2/<transform name or ID>`
234265

0 commit comments

Comments
 (0)