Plugin development: List Interfaces if they're referred to by a plugin's model's ManyToManyField?

[pv2b]

Hi!

I'm currently working on a NetBox plugin to model MC-LAGs. See this repository for my current state of the code: https://github.com/pv2b/netbox-plugin-mclag

This is the data model of my plugin:

class McDomain(NetBoxModel):
    name = models.CharField(max_length=100)
    domain_id = models.CharField(max_length=20, blank=True, null=True)
    description = models.TextField(max_length=200, blank=True, null=True)
    devices = models.ManyToManyField(
        to='dcim.Device',
        related_name="mc_domains"
    )
    def __str__(self):
        return self.name
    def get_absolute_url(self):
        return reverse('plugins:netbox_plugin_mclag:mcdomain', args=[self.pk])
    class Meta:
        verbose_name="Multi-Chassis Domain"
        verbose_name_plural="Multi-Chassis Domains"

class McLag(NetBoxModel):
    name = models.CharField(max_length=100)
    lag_id = models.CharField(max_length=20, blank=True, null=True)
    description = models.TextField(max_length=200, blank=True, null=True)
    mc_domain = models.ForeignKey(
        to=McDomain,
        on_delete=models.CASCADE,
        related_name="mc_lags"
    )
    interfaces = models.ManyToManyField(
        to='dcim.Interface',
        related_name="mc_lags"
    )
    def __str__(self):
        return self.name
    def get_absolute_url(self):
        return reverse('plugins:netbox_plugin_mclag:mclag', args=[self.pk])
    class Meta:
        verbose_name="Multi-Chassis Link Aggregation Group"
        verbose_name_plural="Multi-Chassis Link Aggregation Groups"

To summarize, the McDomain model refers to one or more Interface objects using a ManyToManyField, and a McLag model refers to a McDomain using a ForeignKey.

I'm currently working on the form for McLag, and it currently looks like this:

class McLagForm(NetBoxModelForm):
    interfaces = DynamicModelMultipleChoiceField(
        queryset = Interface.objects.all(),
        selector = True,
        query_params = {
            'device__mc_domains': '$mc_domain'
        }
    )
    # Override the data-url to a custom API endpoint that gets a list of endpoints filtered by McDomain
    #interfaces.widget.attrs['data-url'] = "/asdf"
    class Meta:
        model = McLag
        fields = ('name', 'lag_id', 'description', 'mc_domain', 'tags', 'interfaces')

What I'm trying to do do right now is to make it so that only interfaces that belong to the devices referred to by the McDomain are offered to the user as selections. Or, in real-world terms, if you want to add an interface to a Multi-Chassis Link Aggregation Group, that interface has to be part of the Multi-Chassis Domain that the MC-LAG is associated with.

From what I've been able to figure out so far, DynamicModelMultipleChoiceField uses some client-side Javascript trickery to populate the form based on the results of an API call to list Interface objects, and also offers the query_params option to add parameters from this API call. Prefixing the value by '$' somehow sends a signal to the client-side Javascript to grab the ID of the current selection's object.

That means, right now, when I click on the selection button, my web browser sends a GET request to /api/dcim/interfaces/?brief=true&device__mc_domains=12 which is pretty cool, except even though I've created ManyToManyField, this query doesn't seem to work, it just returns all interfaces.

To recap, the following works in nbshell:

>>> Interface.objects.filter(device__mc_domains=12)
<RestrictedQuerySet [<Interface: Te1/0/1>, <Interface: Te1/0/1>]>

But the following is what I get from the API when I ask for http://localhost:8000/api/dcim/interfaces/?brief=true&device__mc_domains=12:

{
    "count": 3,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 3,
            "url": "http://localhost:8000/api/dcim/interfaces/3/",
            "display": "Coffee1/0/1",
            "device": {
                "id": 3,
                "url": "http://localhost:8000/api/dcim/devices/3/",
                "display": "Coffee Machine",
                "name": "Coffee Machine"
            },
            "name": "Coffee1/0/1",
            "cable": null,
            "_occupied": false
        },
        {
            "id": 1,
            "url": "http://localhost:8000/api/dcim/interfaces/1/",
            "display": "Te1/0/1",
            "device": {
                "id": 1,
                "url": "http://localhost:8000/api/dcim/devices/1/",
                "display": "Switch1",
                "name": "Switch1"
            },
            "name": "Te1/0/1",
            "cable": null,
            "_occupied": false
        },
        {
            "id": 2,
            "url": "http://localhost:8000/api/dcim/interfaces/2/",
            "display": "Te1/0/1",
            "device": {
                "id": 2,
                "url": "http://localhost:8000/api/dcim/devices/2/",
                "display": "Switch2",
                "name": "Switch2"
            },
            "name": "Te1/0/1",
            "cable": null,
            "_occupied": false
        }
    ]
}

As we can see we get the Coffee1/0/1 interface which we don't want (because we want Te...)

I feel I'm very close to the right solution now. So, with all this background my question is, how do I get all the way there?

Am I doing something wrong with my API parameter device__mc_domains? Do I need to format it differently somehow to do the filtering I want? Is this even possible with the stock API's? Or am I going to have to write a custom view to get this to work? (If you've noticed, I already discovered I can override widget.attrs['data-url'] to redirect the API Client to a different API URL, but if there's a simpler solution that's of course better.

[kkthxbye-code]

Am I doing something wrong with my API parameter device__mc_domains?

The ORM and the filtersets are not the same. Only explicitly defined filters are available for filtering in the API and to my knowledge it's not possible to modify the core netbox filtersets. So the solution would be to create your own API endpoint for the Interface model, with your own filterset.

[pv2b]

Great! That's half the solution. The other half is, if I want the DynamicModelMultipleChoiceField to use a custom API endpoint, is adding this line to the class the "correct" way of doing it?

interfaces.widget.attrs['data-url'] = "/asdf"

To me it seems a little off to be poking into "private" members of the DynamicModelMultipleChoiceField, but looking at the implementation I can't seem to find a "sanctioned" way to override this. I'm a little concerned I may be straying into "unsupported" parts of the plugin API here, so the question is, is there another way to do this in a more sanctioned way?

(I guess I could subclass DynamicModelMultipleChoiceField also....)

As an aside, overriding the API endpoint also gives me the opportunity to tweak the serializer for the interfaces to show the name of the device along with the interface name, which is going to be neccessary in this use case to get any kind of usability in the app, so... that's good I guess? Or is there another better way of doing that?

[kkthxbye-code]

This is how its done in netbox itself:

    user = DynamicModelMultipleChoiceField(
        queryset=User.objects.all(),
        required=False,
        label=_('User'),
        widget=APISelectMultiple(
            api_url='/api/users/users/',
        )
    )

As an aside, overriding the API endpoint also gives me the opportunity to tweak the serializer for the interfaces to show the name of the device along with the interface name, which is going to be neccessary in this use case to get any kind of usability in the app, so... that's good I guess? Or is there another better way of doing that?

Should be fine. Just be aware that the display field is what is used, so that's what you need to override.

[pv2b]

I made some more progress on this! I had to do some trickery to disable "brief mode" on the object serializer.

And now it almost works! When creating new McLags this works beautifully, including overriding the display names of the interfaces.

The only problem I have left here is that when editing an existing McLag, the widget display is pre-populated by static values coming in through the HTML rather than the tweaked values being provided by the serializer. I.e. the generated HTML code gives me the "wrong" interface names that it's pulled straight from the base Interface model, instead of from the API. I'm not sure how to override that part?

[pv2b]

Never mind, I figured it out. Thanks again for your help!