Provisioning: Lost ability to sync from git

[bitcollector1]

NetBox version

v3.6-beta1

Python version

3.8

Steps to Reproduce

Lost the ability to sync from GIT.

Here is the config template that synced successfully last week (I believe it was before the upgrade)

Expected Behavior

Template syncs successfully as before. We have another instance not on the newest beta that syncs just fine.

Observed Behavior

Error when clicking on the template:

[jeremystretch]

Moving this to a discussion as you have not provided any reproduction instructions.

[bitcollector1]

Not sure what else to add honestly, I just noticed that the sync failed today, it was previously working before I upgraded the beta version.

[bitcollector1]

ZTP just failed on the new box I just updated to 3.6......it was working previously.

[bitcollector1]

Really feeling like this is a bug....as you can see the successful provision happened earlier today, pre 3.6 update.

[chuegel]

I'm experiencing this on NetBox 3.5.7 on k8s. See below:

#13573

[bitcollector1]

Thanks for the heads up!

In my case the sync was working fine on 3.5.7 but the issue popped up the second I upgraded to 3.6. (on 2 different boxes with different data).

Hopefully it's an easy fix because I use the git sync pretty heavily.

[zndrr]

Ah I also have experienced it on v3.6-beta1, beta2 and now v3.6.0. Worked at least between v3.5.6-3.5.8, possibly before but can't remember.
Hadn't had time to collate info for a bug report sorry.

All I've done is re-add repos, but one of the ones I had a couple of the Netbox-community ones, such as:
https://github.com/netbox-community/customizations, branch master, no auth

EDIT: yeah, definitely okay on 3.5.8 . I cloned that for 3.6b1, then upgraded through 3.6b2 and finally 3.6.0. Not fresh, if that matters.

[kkthxbye-code]

For some reason this wasn't pointed out in the release notes, but dulwich (git) and boto3 (aws) were removed as default packages.

See https://docs.netbox.dev/en/stable/features/synchronized-data/

Data backends which connect to external sources typically require the installation of one or more supporting Python libraries. The Git backend requires the dulwich package, and the S3 backend requires the boto3 package. These must be installed within NetBox's environment to enable these backends.

And https://docs.netbox.dev/en/stable/installation/3-netbox/#remote-data-sources

NetBox supports integration with several remote data sources via configurable backends. Each of these requires the installation of one or more additional libraries.

Amazon S3: boto3
Git: dulwich
For example, to enable the Amazon S3 backend, add boto3 to your local requirements file:

sudo sh -c "echo 'boto3' >> /opt/netbox/local_requirements.txt"

This might be the issue you are facing.

[zndrr]

Yeah that did the trick thanks. Makes sense that that addendum might have been easy to miss.

[bitcollector1]

Thank you VERY much!

[chris240189]

ZTP just failed on the new box I just updated to 3.6......it was working previously.

I get exactly that error and I am not even using git, but a local datasource. Any ideas?
My files seem to get synced, but I cannot look at them through GUI and another plugin I was trying out (config diffs) cannot work with the datasource.

I created an issue for tracking it

[bitcollector1]

My sync jobs are still failing, even after installing the git package :(

[abhi1693]

It could be due to #13656

[bitcollector1]

Thank you for the assist, it ended up being a couple of different issues but happy to report it's now all working!

[bitcollector1]

I can now read the files when I click on them (after making the update in #13656

However, the SYNC is still failing :(

[bitcollector1]

I've given up for the day, I'll try to pull again later in the week :(

[bitcollector1]

seeing the following errors on install

Running migrations:
  No migrations to apply.
  Your models in app(s): 'circuits', 'core', 'dcim', 'extras', 'ipam', 'tenancy', 'users', 'virtualization', 'wireless' have changes that are not yet reflected in a migration, and so won't be applied.
  Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.

[bitcollector1]

Now it looks like the sync happens but the status still says failed!

[bitcollector1]

okay, I was able to confirm that the new templates have, in fact, been pulled. It appears the status message is incorrect now for whatever reason.

[bitcollector1]

okay, I FINALLY GOT IT ALL WORKING :)

It was a combo of the (2) issues and the order of operations of my hacks resulted in a mixed state.

After manually installing dulwich and also making the edit to the data file I'm finally back to working :)