Azure IPAM upgrade to main is not working

[amitjattan]

Hello,

Whilst it's working in local, but not is Azure App server. This is we are trying to upgrade our ipam 3.1.0 version to new images 3.6.0. But it's failing with below error in APIs/Engine app

2025-08-04T20:45:36.095547981Z File "", line 1204, in _gcd_import
2025-08-04T20:45:36.095552972Z File "", line 1176, in _find_and_load
2025-08-04T20:45:36.095557900Z File "", line 1126, in _find_and_load_unlocked
2025-08-04T20:45:36.095562306Z File "", line 241, in _call_with_frames_removed
2025-08-04T20:45:36.095566035Z File "", line 1204, in _gcd_import
2025-08-04T20:45:36.095570257Z File "", line 1176, in _find_and_load
2025-08-04T20:45:36.095574189Z File "", line 1140, in _find_and_load_unlocked
2025-08-04T20:45:36.095578216Z ModuleNotFoundError: No module named 'app'/home/LogFiles/2025_08_04_ln1sdlwk000BIX_docker.log (https://xxxx.scm.azurewebsites.net/api/vfs/LogFiles/2025_08_04_ln1sdlwk000BIX_docker.log)
2025-08-04T20:47:42.665Z INFO - Waiting for response to warmup request for cont

[DCMattyG]

Hi @amitjattan, can you provide a bit more detail as to how you are upgrading your images?

Also, only v3.5.0 is available right now, though v3.6.0 is coming very shortly.

Are you running the update.ps1 script? Are you running in a Docker Compose based App Service or single container?

A few additional details would be wonderful so I can get you fixed up ASAP!

[amitjattan]

Hi Matt,

Thanks for the reply.

• So yeah actually we already has the solution deployed in App services pattern with version 3.1.0, where UI and Engine both apps services running parallel.
• We were deploying these apps via building images for both apps through code directly. And now I am trying to do same with branch 3.6.0 updating in my remote and then just running the comands to deploy new images like below
• az acr build -r -t ipam-engine:latest -f ./engine/Dockerfile.rhel ./engine
• After images are deployed, I just restarted my apps, UI seems to be loading fine, but running on blue circle which I understadn is because my engine app is not up and running. But engine app is failing in start up and show the logs as above.

[DCMattyG]

I think the problem is that you are using branch v3.6.0 which is not ready for production at all yet, whereas you should always be using the main branch which contains GA, tested code.

[amitjattan]

Ok yeah, I tried with main branch also and seems like same error occurring there also

2025-08-04T22:08:47.991308650Z File "/usr/lib64/python3.9/importlib/init.py", line 127, in import_module
2025-08-04T22:08:47.991320753Z return _bootstrap._gcd_import(name[level:], package, level)
2025-08-04T22:08:47.991324450Z File "", line 1030, in _gcd_import
2025-08-04T22:08:47.991328504Z File "", line 1007, in _find_and_load
2025-08-04T22:08:47.991332136Z File "", line 972, in _find_and_load_unlocked
2025-08-04T22:08:47.991348963Z File "", line 228, in _call_with_frames_removed
2025-08-04T22:08:47.991352831Z File "", line 1030, in _gcd_import
2025-08-04T22:08:47.991356424Z File "", line 1007, in _find_and_load
2025-08-04T22:08:47.991360115Z File "", line 984, in _find_and_load_unlocked
2025-08-04T22:08:47.991363700Z ModuleNotFoundError: No module named 'app'

[DCMattyG]

Hey @amitjattan, please shoot me an email at Matthew.Garrett@microsoft.com so we can hop on a quick call together and see what is going on. It should be as simple as building the new containers, assuming the environment was deployed from the original Azure IPAM scripts.

[amitjattan]

No actually we aren't using the deploy scripts from ipam code. We have created our infrastructure using terraform and then build the apps iamges directly via this code then deploy them to ACRs

After adding below line of code in ./engine/Dockerfile.Rhel worked

Line # 31
COPY ./app /ipam/app

[DCMattyG]

Actually @amitjattan, it appears there is a bug/fat finger in that file.

You can see here:

ipam/engine/Dockerfile.rhel

Line 28 in 6a86f14

ADD ./app ./appDockerfile

That line is currently:
ADD ./app ./appDockerfile

But it should be:
ADD ./app ./app

Somehow the Dockerfile got pasted in there by mistake and is likely the reason behind your issues. I've already corrected this for the code that will be released soon for v3.6.0.

Thanks for spotting this!!!

[DCMattyG]

Hey @amitjattan, just letting you know that v3.6.0 is now officially released and the above issue has been addressed in PR #364.

Hope you enjoy!

[amitjattan]

Thanks Matt, for the updates. It's actually a right timing, we have the plans to upgrade the IPAM in next Q. I had another question, do you know if IPAM can support multiple tenants this time. We have another tenant and wondering if we could use same instance for that? or we have to create new instance for the new tenant?

…

On Fri, Sep 12, 2025 at 10:04 AM DCMattyG ***@***.***> wrote: *DCMattyG* left a comment (Azure/ipam#360) <#360 (comment)> Hey @amitjattan <https://github.com/amitjattan>, just letting you know that v3.6.0 is now officially released and the above issue has been addressed in PR #364 <#364>. Hope you enjoy! — Reply to this email directly, view it on GitHub <#360 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGL3UQWL3AZLZCBOL53X3LD3SL4IJAVCNFSM6AAAAACDDEJNTGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEOBWGEYTINRWGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Thanks & Regards Amit Kumar 971-264-4425

[DCMattyG]

At this point in time it cannot?

WHY?

The way that the Azure IPAM tool gathers all of the data it uses is via Azure Resource Graph. The highest scope you can use for ARG is the Tenant level, which means we either need to call repeated ARG queries as we iterate through the list of Tenants or use something like Azure Lighthouse.

Azure Lighthouse makes the most sense because we ALSO have the issue of cross-tenant authentication and making sure the required Service Principals also have Reader access for each Tenants Root Management Group.

So, multi-tenant applications are complex and will require some significant re-work. I'll definitely be exploring what can be done here, but at this point in time that is a big rock to move. For now, the best advice is to use an Azure IPAM instance per Tenant.

Apologies for the inconvenience here @amitjattan.

[Survibharath]

Hi, I’d like to work on this issue. Could you please confirm if the expected fix is adjusting the Dockerfile/startup command to properly reference the app module?

[DCMattyG]

If you're referring to the typo above in the RHEL Dockerfile, you can see from the below screenshot that this has indeed been addressed in the current main branch of the repo as well as in the v3.6.0 release.

Hope that helps to clarify for you @Survibharath!