Skip to content

Commit 7fbacbe

Browse files
authored
Merge pull request #100 from t0m3kz/tz-aci-adapter-feature
Aci adapter feature
2 parents a8ae669 + 3c725ee commit 7fbacbe

File tree

16 files changed

+2021
-3
lines changed

16 files changed

+2021
-3
lines changed

docs/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ This website is built using [Docusaurus](https://docusaurus.io/), a modern stati
88
$ yarn
99
```
1010

11-
## Local Development
11+
## Local development
1212

1313
```console
1414
$ yarn start

docs/docs/adapters/aci.mdx

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
---
2+
title: Cisco ACI adapter
3+
---
4+
import ReferenceLink from "../../src/components/Card";
5+
6+
<!-- vale off -->
7+
## What is Cisco ACI?
8+
<!-- vale on -->
9+
10+
The *Cisco ACI* is a software-defined networking (SDN) solution that provides a policy-based, application-centric approach to managing and orchestrating network infrastructure. It is commonly used in data centers for scalable, policy-driven networking.
11+
12+
## Sync direction
13+
14+
Cisco ACI Infrahub
15+
16+
:::note
17+
Currently, the Cisco ACI adapter supports only **one-way synchronization** from ACI to Infrahub. Syncing data back into ACI is not yet supported.
18+
:::
19+
20+
## Configuration
21+
22+
The adapter reads connection settings from the synchronization configuration and can be overridden by environment variables. Credentials should be provided via a secret manager or environment variables in production.
23+
24+
### Configuration parameters
25+
26+
```yaml
27+
---
28+
name: from-cisco-aci
29+
source:
30+
name: aci
31+
settings:
32+
url: "https://<APIC_CONTROLLER>"
33+
username: "<USER>"
34+
password: "<PASSWORD>"
35+
api_endpoint: "api" # optional, default: api
36+
verify: true # boolean or string ("false","0") accepted
37+
```
38+
39+
### Environment variables
40+
41+
- CISCO_APIC_URL: overrides settings.url
42+
- CISCO_APIC_USERNAME: overrides settings.username
43+
- CISCO_APIC_PASSWORD: overrides settings.password
44+
- CISCO_APIC_VERIFY: overrides settings.verify; accepts true/false/0/1 (strings are normalized)
45+
46+
Notes:
47+
48+
- Credentials must come from environment variables or a secret manager in production. Never commit secrets.
49+
- The adapter normalizes verify to a boolean (strings like false, 0, no are treated as False).
50+
- The adapter records login timestamps in UTC to avoid timezone related issues and ensure correct token refresh behavior.
51+
52+
### Schema mapping examples
53+
54+
#### Basic device mapping
55+
56+
```yaml
57+
- name: DcimPhysicalDevice
58+
mapping: "class/fabricNode.json"
59+
identifiers: ["name"]
60+
fields:
61+
- name: name
62+
mapping: "fabricNode.attributes.name"
63+
- name: serial
64+
mapping: "fabricNode.attributes.serial"
65+
- name: role
66+
mapping: "fabricNode.attributes.role"
67+
filters:
68+
- field: "fabricNode.attributes.fabricSt"
69+
operation: "=="
70+
value: "active"
71+
```
72+
73+
#### Interface mapping with ACI Jinja filter
74+
75+
```yaml
76+
- name: DcimPhysicalInterface
77+
mapping: "class/l1PhysIf.json"
78+
identifiers: ["device", "name"]
79+
fields:
80+
- name: name
81+
mapping: "l1PhysIf.attributes.id"
82+
- name: device
83+
mapping: "l1PhysIf.attributes.dn"
84+
reference: DcimPhysicalDevice
85+
- name: description
86+
mapping: "l1PhysIf.attributes.descr"
87+
transforms:
88+
- field: device
89+
expression: "{{ l1PhysIf.attributes.dn.split('/')[2].replace('node-', '') | aci_device_name }}"
90+
- field: status
91+
expression: "{{ 'active' if l1PhysIf.attributes.adminSt == 'up' else 'free' }}"
92+
filters:
93+
- field: "l1PhysIf.attributes.id"
94+
operation: "contains"
95+
value: "eth"
96+
```
97+
98+
## ACI-specific Jinja filters
99+
100+
The ACI adapter provides custom Jinja filters for data transformation:
101+
102+
### `aci_device_name` filter
103+
104+
The `aci_device_name` filter resolves ACI node IDs to device names automatically. This is particularly useful when mapping physical interfaces to their parent devices.
105+
106+
**Usage:**
107+
108+
```jinja2
109+
{{ node_id | aci_device_name }}
110+
```
111+
112+
**Example:**
113+
114+
- Input: `"102"` (ACI node ID)
115+
- Output: `"spine-102"` (actual device name from ACI)
116+
117+
**Common use case in transforms:**
118+
119+
```yaml
120+
transforms:
121+
- field: device
122+
expression: "{{ l1PhysIf.attributes.dn.split('/')[2].replace('node-', '') | aci_device_name }}"
123+
```
124+
125+
This transform:
126+
127+
1. Extracts the node ID from the ACI Distinguished Name (DN)
128+
2. Removes the `"node-"` prefix (for example: `"node-102"` `"102"`)
129+
3. Uses the `aci_device_name` filter to resolve the node ID to the actual device name
130+
131+
**How it works:**
132+
133+
- The adapter automatically queries the ACI `fabricNode` class during initialization
134+
- Builds a mapping of node IDs to device names
135+
- The filter performs a lookup with a fallback to the original node ID if not found
136+
137+
## Generating the models
138+
139+
Use the generate command to produce models from the schema mapping and examples:
140+
141+
```bash
142+
poetry run infrahub-sync generate --name from-cisco-aci --directory examples/
143+
```
144+
145+
## Common issues and troubleshooting
146+
147+
### Authentication and connectivity
148+
149+
- If you see token refresh errors, ensure the APIC response includes refreshTimeoutSeconds; the adapter forces re-login when refresh data is unavailable.
150+
- For TLS verification problems, set CISCO_APIC_VERIFY to false in a secure environment (use with caution).
151+
152+
### Device reference resolution
153+
154+
- **Interface-device relationship errors**: If you see "Unable to locate the node device" errors, ensure:
155+
- The `DcimPhysicalDevice` mapping runs before `DcimPhysicalInterface` in the `order` configuration
156+
- The device transform uses the `aci_device_name` filter correctly: `{{ node_id | aci_device_name }}`
157+
- The ACI fabric node query succeeds (check logs for "Built ACI device mapping" messages)
158+
159+
### Jinja filter issues
160+
161+
- **`aci_device_name` filter not found**: Ensure you're using the ACI adapter and the filter is correctly spelled
162+
- **Filter returns node ID instead of device name**: Check that the fabric node query succeeded during adapter initialization
163+
- **Transform expression errors**: Verify the DN parsing logic extracts the correct node ID:
164+
165+
```yaml
166+
expression: "{{ l1PhysIf.attributes.dn.split('/')[2].replace('node-', '') | aci_device_name }}"
167+
```
168+
169+
### General debugging
170+
171+
- Enable DEBUG logging for the adapter to see raw fetched objects and mapping decisions. Logs will not include secrets.
172+
- Check the device mapping build process in logs: look for "Built ACI device mapping with X entries"

docs/docs/adapters/local-adapters.mdx

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,13 +78,132 @@ For a more complete example, refer to the example in the repository:
7878

7979
<ReferenceLink title="Custom Adapter Example" url="https://github.com/opsmill/infrahub-sync/tree/main/examples/custom_adapter" openInNewTab />
8080

81+
### Adding custom Jinja filters
82+
83+
Custom adapters can provide their own Jinja filters for use in transform expressions. This is particularly useful for adapter-specific data transformations.
84+
85+
#### Implementing custom filters
86+
87+
To add custom filters to your adapter, implement the `_add_custom_filters` class method in your DiffSync model class:
88+
89+
```python
90+
from typing import Any, ClassVar
91+
from diffsync import DiffSyncModel
92+
from infrahub_sync import DiffSyncModelMixin
93+
94+
class MyCustomModel(DiffSyncModelMixin, DiffSyncModel):
95+
# Store any data needed by filters as class variables
96+
_my_mapping: ClassVar[dict[str, str]] = {}
97+
98+
@classmethod
99+
def set_my_mapping(cls, mapping: dict[str, str]) -> None:
100+
"""Set mapping data for use in filters."""
101+
cls._my_mapping = mapping
102+
103+
@classmethod
104+
def _add_custom_filters(cls, native_env: Any, item: dict[str, Any]) -> None:
105+
"""Add custom filters to the Jinja environment."""
106+
107+
def my_custom_filter(value: str) -> str:
108+
"""Custom filter that transforms values using stored mapping."""
109+
return cls._my_mapping.get(str(value), value)
110+
111+
def format_identifier(value: str) -> str:
112+
"""Another custom filter for formatting identifiers."""
113+
return f"ID-{value.upper()}"
114+
115+
# Register filters with the Jinja environment
116+
native_env.filters["my_custom_filter"] = my_custom_filter
117+
native_env.filters["format_identifier"] = format_identifier
118+
```
119+
120+
#### Setting up filter data
121+
122+
If your filters need data (like mappings, lookup values, etc.), initialize it in your adapter:
123+
124+
```python
125+
class MyCustomAdapter(DiffSyncMixin, Adapter):
126+
def __init__(self, target, adapter, config, *args, **kwargs):
127+
super().__init__(*args, **kwargs)
128+
# ... other initialization
129+
130+
# Build data needed by filters
131+
my_mapping = self._build_custom_mapping()
132+
133+
# Pass data to model class for filter use
134+
MyCustomModel.set_my_mapping(my_mapping)
135+
136+
def _build_custom_mapping(self) -> dict[str, str]:
137+
"""Build mapping data from your data source."""
138+
# Implementation depends on your data source
139+
return {"key1": "value1", "key2": "value2"}
140+
```
141+
142+
#### Using custom filters in configuration
143+
144+
Once implemented, use your custom filters in transform expressions:
145+
146+
```yaml
147+
schema_mapping:
148+
- name: MyModel
149+
mapping: "api/endpoint"
150+
fields:
151+
- name: identifier
152+
mapping: "raw_id"
153+
- name: formatted_name
154+
mapping: "name"
155+
transforms:
156+
- field: identifier
157+
expression: "{{ raw_id | my_custom_filter | format_identifier }}"
158+
- field: status
159+
expression: "{{ 'active' if enabled else 'inactive' }}"
160+
```
161+
162+
#### Filter implementation guidelines
163+
164+
1. **Keep filters focused**: Each filter should do one specific transformation
165+
2. **Handle edge cases**: Always provide fallback values for missing data
166+
3. **Use class variables**: Store filter data as class variables for efficient access
167+
4. **Document your filters**: Add Python documentation strings explaining what each filter does
168+
5. **Test thoroughly**: Ensure filters work with various input types and edge cases
169+
170+
#### Real-world example: ACI device name filter
171+
172+
Here's how the ACI adapter implements the `aci_device_name` filter:
173+
174+
```python
175+
class AciModel(DiffSyncModelMixin, DiffSyncModel):
176+
_device_mapping: ClassVar[dict[str, str]] = {}
177+
178+
@classmethod
179+
def set_device_mapping(cls, device_mapping: dict[str, str]) -> None:
180+
cls._device_mapping = device_mapping
181+
182+
@classmethod
183+
def _add_custom_filters(cls, native_env: Any, item: dict[str, Any]) -> None:
184+
def aci_device_name(node_id: str) -> str:
185+
"""Resolve ACI node IDs to device names."""
186+
return cls._device_mapping.get(str(node_id), node_id)
187+
188+
native_env.filters["aci_device_name"] = aci_device_name
189+
```
190+
191+
Used in configuration:
192+
193+
```yaml
194+
transforms:
195+
- field: device
196+
expression: "{{ l1PhysIf.attributes.dn.split('/')[2].replace('node-', '') | aci_device_name }}"
197+
```
198+
81199
### Best practices
82200

83201
1. **Package Structure**: Organize complex adapters as packages with `__init__.py`
84202
2. **Testing**: Include test data and documentation with your adapter
85203
3. **Configuration**: Use settings to make your adapter configurable
86204
4. **Error Handling**: Implement proper error handling and logging
87205
5. **Type Annotations**: Use type hints to make your code more maintainable
206+
6. **Custom Filters**: Implement adapter-specific Jinja filters for complex transformations
88207

89208
### Local adapter example
90209

docs/docs/reference/cli.mdx

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ $ infrahub-sync diff [OPTIONS]
3636
* `--directory TEXT`: Base directory to search for sync configurations
3737
* `--branch TEXT`: Branch to use for the diff.
3838
* `--show-progress / --no-show-progress`: Show a progress bar during diff [default: show-progress]
39+
* `--adapter-path TEXT`: Paths to look for adapters. Can be specified multiple times.
3940
* `--help`: Show this message and exit.
4041

4142
## `infrahub-sync generate`
@@ -54,6 +55,7 @@ $ infrahub-sync generate [OPTIONS]
5455
* `--config-file TEXT`: File path to the sync configuration YAML file
5556
* `--directory TEXT`: Base directory to search for sync configurations
5657
* `--branch TEXT`: Branch to use for the sync.
58+
* `--adapter-path TEXT`: Paths to look for adapters. Can be specified multiple times.
5759
* `--help`: Show this message and exit.
5860

5961
## `infrahub-sync list`
@@ -89,4 +91,5 @@ $ infrahub-sync sync [OPTIONS]
8991
* `--branch TEXT`: Branch to use for the sync.
9092
* `--diff / --no-diff`: Print the differences between the source and the destination before syncing [default: diff]
9193
* `--show-progress / --no-show-progress`: Show a progress bar during syncing [default: show-progress]
94+
* `--adapter-path TEXT`: Paths to look for adapters. Can be specified multiple times.
9295
* `--help`: Show this message and exit.

docs/sidebars.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,9 @@ const sidebars: SidebarsConfig = {
2525
'adapters/observium',
2626
'adapters/peering-manager',
2727
'adapters/slurpit',
28+
'adapters/aci',
2829
],
29-
},
30+
},
3031
{
3132
type: 'category',
3233
label: 'Reference',
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"""ACI sync adapter module."""
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
from __future__ import annotations
2+
3+
from infrahub_sync.plugin_loader import PluginLoader
4+
5+
from .sync_models import (
6+
DcimPhysicalDevice,
7+
DcimPhysicalInterface,
8+
LocationBuilding,
9+
LocationMetro,
10+
OrganizationCustomer,
11+
)
12+
13+
# Load adapter class dynamically at runtime
14+
15+
_AdapterBaseClass = PluginLoader().resolve("aci")
16+
17+
18+
# -------------------------------------------------------
19+
# AUTO-GENERATED FILE, DO NOT MODIFY
20+
# This file has been generated with the command `infrahub-sync generate`
21+
# All modifications will be lost the next time you reexecute this command
22+
# -------------------------------------------------------
23+
class AciSync(_AdapterBaseClass):
24+
DcimPhysicalDevice = DcimPhysicalDevice
25+
DcimPhysicalInterface = DcimPhysicalInterface
26+
LocationBuilding = LocationBuilding
27+
LocationMetro = LocationMetro
28+
OrganizationCustomer = OrganizationCustomer

0 commit comments

Comments
 (0)