Is it thread-safe to get and assign IP?

[lotanaharoni]

Hi,
We are looking for a safe way to get and assign IPs via API calls.
I would like to get the next available IP, change its state, and assign it to some device.
Is it thread-safe to get an available IP and change its state?
For example, can one IP assign to two different devices? (from two different requests)?

Thank you.

[jeremystretch]

The available-ips REST API endpoint (and its counterparts) employ a database lock when provisioning new IP addresses, so it's safe to make multiple requests simultaneously.

[lotanaharoni]

Awesome, Thanks!

[elipsion]

However, that API does not have all the functionality present in the generic create-operations. So you can't create AND assign in a single operation.

[bmistry13]

Hi @jeremystretch ,

Netbox VERSION = '2.5.13'

What version of Netbox is Thread-safe?

I just tested with JMeter.

Three Separate device IDs got 2 of them the same IP address and one of them failed with an error via REST-API. I have done these API calls via JMeter and I can re-produce this:

Request:

POST http://localhost:8000/api/ipam/prefixes/1/available-ips/?contiguous=true

POST data:
[

{

    "vrf": 1,

    "status": 1 ,

    "vdFields": {

            "device_id" : "3000",

            "device_ip_type" :  "ESP"

        },

   "tags": ["VD"]

},

{

    "vrf": 1,

    "status": 1,

    "vdFields": {

            "device_id" : "3000",

            "device_ip_type" :  "VXLAN"

        },

    "tags": ["VD"]

}

]

POST http://localhost:8000/api/ipam/prefixes/1/available-ips/?contiguous=true

POST data:
[

{ 

    "vrf": 1,

    "status": 1 ,

    "vdFields": {

            "device_id" : "3001",

            "device_ip_type" :  "ESP"

        },

   "tags": ["VD"]

},

{ 

    "vrf": 1,

    "status": 1,

    "vdFields": {

            "device_id" : "3001",

            "device_ip_type" :  "VXLAN"

        },

    "tags": ["VD"]

}

]

Response:

1. User 1
   {"count":2,"ip_ranges":[{"prefix":"10.0.13.254/31","count":"2","netmask":"255.255.255.254"}],"results":[{"id":4741,"family":{"value":4,"label":"IPv4"},"address":"10.0.13.254/8","vrf":{"id":1,"url":http://localhost:8000/api/ipam/vrfs/1/,"name":"OverlayAddressPool","rd":"OverlayAddressPool"},"tenant":null,"status":{"value":1,"label":"Active"},"role":null,"interface":null,"description":"","nat_inside":null,"nat_outside":null,"tags":["VD"],"created":"2023-12-05","last_updated":"2023-12-05T00:11:52.110523Z"},{"id":4743,"family":{"value":4,"label":"IPv4"},"address":"10.0.13.255/8","vrf":{"id":1,"url":http://localhost:8000/api/ipam/vrfs/1/,"name":"OverlayAddressPool","rd":"OverlayAddressPool"},"tenant":null,"status":{"value":1,"label":"Active"},"role":null,"interface":null,"description":"","nat_inside":null,"nat_outside":null,"tags":["VD"],"created":"2023-12-05","last_updated":"2023-12-05T00:11:52.129938Z"}]}

2. User 2
   {"count":2,"ip_ranges":[{"prefix":"10.0.13.254/31","count":"2","netmask":"255.255.255.254"}],"results":[{"id":4742,"family":{"value":4,"label":"IPv4"},"address":"10.0.13.254/8","vrf":{"id":1,"url":http://localhost:8000/api/ipam/vrfs/1/,"name":"OverlayAddressPool","rd":"OverlayAddressPool"},"tenant":null,"status":{"value":1,"label":"Active"},"role":null,"interface":null,"description":"","nat_inside":null,"nat_outside":null,"tags":["VD"],"created":"2023-12-05","last_updated":"2023-12-05T00:11:52.127730Z"},{"id":4744,"family":{"value":4,"label":"IPv4"},"address":"10.0.13.255/8","vrf":{"id":1,"url":http://localhost:8000/api/ipam/vrfs/1/,"name":"OverlayAddressPool","rd":"OverlayAddressPool"},"tenant":null,"status":{"value":1,"label":"Active"},"role":null,"interface":null,"description":"","nat_inside":null,"nat_outside":null,"tags":["VD"],"created":"2023-12-05","last_updated":"2023-12-05T00:11:52.139129Z"}]}

3. user 3 (device id 3003)
   {"address":["Duplicate IP address found in VRF OverlayAddressPool (OverlayAddressPool): 10.0.13.254/8"]},{"address":["Duplicate IP address found in VRF OverlayAddressPool (OverlayAddressPool): 10.0.13.254/8"]}]

[candlerb]

What version of Netbox is Thread-safe?

The release notes are available here: https://github.com/netbox-community/netbox/tree/develop/docs/release-notes

EDIT: locking for the available-ips endpoint was first introduced in commit 2e83ce7 which made it into v2.7.7:

• #2519 - Avoid race condition when provisioning "next available" IPs/prefixes via the API

It was reimplemented in commit bace24b in v3.5.5 (I'm not sure if it had been lost before then).

For creation of specified IP addresses, it looks like this was implemented in v3.3.7:

• #10282 - Enforce advisory locks when allocating available IP addresses to prevent race conditions

Netbox VERSION = '2.5.13'

That was released four and a half years ago (May 31, 2019), so Netbox today has moved on a long way.

You can find other threads in this discussion group about how to go about such a major upgrade, but in short:

• Export the database and media from your current instance
• Build a new VM with 2.11.12
• Import the database into this new instance, run ./upgrade.sh, and test.
• Export the database from this instance
• Build a new VM with 3.6.6
• Import the database into this new instance, run ./upgrade.sh, and test.
• Point your users at the new VM

I would strongly recommend not doing an in-place upgrade for this: firstly because your existing system probably doesn't meet the necessary requirements (e.g. Python >=3.8), and secondly because there have been other major changes to how Netbox is installed, in particular using virtualenv (v2.7.9) and using systemd instead of supervisord (v2.7.0).

You must also check all the main release notes for intermediate versions (2.6.0, 2.7.0, 2.8.0 etc) to find out what changes have been made to the data model and REST API. These are not kept backwards compatible between versions, so you'll have to update any integrations you have to match.

[bmistry13]

@candlerb,

Thank you very much for your help! I was able to verify the fix with the latest netbox doker image with JMeter. Each IP was unique and did not throw an error message.

We are in a docker environment and the schema has significantly changed. I will find out how to upgrade, but can you please tell us how to set SECRET_KEY in the docker environment? Is Redis mandatory? We just need IPAM functionality.

Thank you for your help!

[candlerb]

I don't use docker for Netbox myself, but as I understand it:

1. Parameters are set in environment variables, or for certain "secret" values you can also set them in /run/secrets/XXX: reference configuration.py here and here. Any parameters which you can't set in environment variables you put in extra.py
2. Yes, Netbox requires Redis. You can see the necessary components in the docker-compose file. Redis is used for queuing background tasks and for caching.

If you're upgrading via docker whilst retaining data, the same thing applies: you'll first need to upgrade to 2.11.12, before you upgrade to any 3.x release. Running a newer docker container will apply the schema migrations on startup.

I strongly recommend you take a postgres backup first (but you are already backing up your postgres database regularly, aren't you?)