Migrating from another system

[LqdBcnAtWork]

Hello, we currently are using Fishbowl inventory for our various needs. However, we're growing past our current 6 seat limit and do not want to switch to a subscription for our inventory control needs.

As a test to see if Inventree would be a viable option to switch to, I wrote some JS code which talks to FB's api and Inventree's REST api to start copying over data.

1. First and foremost: I'm getting some errors thrown while trying to add bom items to parts. Namely the self.tree_id == parent.tree_id check within Part.check_add_to_bom method. I don't quite understand why this error is being thrown. I think I understand what the tree_id reprersents, namely a unique hierarchy or tree of parts and boms. But the most confusing error I found happens to be with a particular screw being added as a bom item, but it doesn't exist on any other boms to begin with. Nor do we have any recursion in our current inventory system.

   Regardless, if someone could point me in the right direction on a Part's tree_id. Most notably if I knew some details on how it was generated or calculated etc I could probably figure out the rest. The next thing I'll probably try is to wipe the database and force the import of parts in a bottom up order. Maybe then the tree_id will play nice if I massage the data import order.

   Edit2: most likely a race condition. I had multiple requests open at the same time creating parts etc. Going to a single request at a time seems to have fixed this issue.

2. Secondly, one of the things I'm trying to do is pull all of the parts and bom items via the API. Literally every record, all of them, no filtering. But it's slow. Like 170 seconds to pull the data slow. It might be 20k records (parts and bom items), but still, there must be a better way...

   Could this just be a Python/Django problem? It's almost definitely not Node's fault, it handles the same qty of records from FB in just a second. I was getting a http 500 response when I asked for the whole thing in one go, so I'm using the pagination and asking with a limit of 1k. But waiting minutes to simply pull the data to run my script is hardly encouraging.

   How bad of an idea is it to bypass the REST API in favor of talking to the pgsql server directly? I only want to read the data from the DB, I do not want to write to it this way.

   Edit: I'm bypassing the API for reads, much faster.

If you're questioning why I want the data, it's so I can map against FB to apply changes. We want to have an environment like the demo but with our own data that's only a day stale kind of idea. Then once we've learned the new system and rebuilt tooling that was made around FB we'll actually switch.

[SchrodingersGat]

Hey @LqdBcnAtWork this is an interesting problem!

First and foremost: I'm getting some errors thrown while trying to add bom items to parts.

Can you please provide the full information on the API request you are making, and the full error message too.

Secondly, one of the things I'm trying to do is pull all of the parts and bom items via the API.

The API calls should not be as slow as that. Pulling all the records (rather than using pagination) is time consuming but should not be taking multiple minutes.

Can you compare how long a paginated call takes from your script, vs loading into a table in the UI?

[LqdBcnAtWork]

Not quite, since I commented out lines 6 lines in src/backend/InvenTree/part/models.py to make it work.

Namely the self.tree_id == parent.tree_id condition in Part.check_add_to_bom.

I used the install script.
invoke version:

Versions:
Python      3.9.5
Django      4.2.17
InvenTree   0.17.8
API         294

[SchrodingersGat]

This is a valid error message, and you should not work around it. If parts are added to their "own BOM" (or an assembly with the same tree_id value) it can cause recursion.

So, the issue is that the tree_id value is being set incorrectly. Are you manually setting that somehow in your import process?

[LqdBcnAtWork]

No, I didn't even know it existed until it was causing me issues.

I honestly don't know how they're getting the same tree_id. Which is why I was originally asking for information around it.

I'd much rather not touch the source code if need be.

However, for performance reasons I was allowing multiple requests to be open with the server at the same time. Due to other issues I disabled that 'feature'. In hindsight this seems like a likely candidate for this problem.

I'll try wiping and un-commenting that condition next.

[LqdBcnAtWork]

I uncommented the condition and did all 29,885 api requests without any tree_id related issues.

Note to self: don't have more than one request open at a time.

[matmair]

Just FYI: This is not recommended as it might mess with the MPTT order of elements and lead to unforeseen issues down the line as the system assumes that correct tree_id's were assigned during import / creation.