Skip to content

Data Import/Export API: Extend import API#1377

Merged
beesaferoot merged 12 commits intomainfrom
extend-import-api
Apr 2, 2026
Merged

Data Import/Export API: Extend import API#1377
beesaferoot merged 12 commits intomainfrom
extend-import-api

Conversation

@beesaferoot
Copy link
Copy Markdown
Contributor

Makes progress on #1232

Brief summary of the change made

Are there any other side effects of this change that we should be aware of?

Describe how you tested your changes?

Pull Request checklist

Please confirm you have completed any of the necessary steps below.

  • Meaningful Pull Request title and description
  • Changes tested as described above
  • Added appropriate documentation for the change.
  • Created GitHub issues for any relevant followup/future enhancements if appropriate.

@beesaferoot beesaferoot changed the title Data Import/Export API: Extend import api Data Import/Export API: Extend import API Mar 13, 2026
@beesaferoot beesaferoot requested a review from dmohns March 18, 2026 08:38
Three major changes here;
- Support async import by adding import jobs.
- Extend Import API to support more core mpm entities
- Update Import UI on settings page
@dmohns
Copy link
Copy Markdown
Member

dmohns commented Mar 30, 2026

I'm trying to import a customer + device. They both import fine, but the resulting objects are not linked 🤔

{
  "data": [
    {
      "title": "Mr",
      "name": "Daniel",
      "surname": "Test User",
      "birth_date": "2026-03-01",
      "gender": "male",
      "email": "",
      "phone": "",
      "city": "Mafia Village",
      "devices": "123456789",
      "agent": ""
    }
  ]
}
{
  "data": [
    {
      "id": 71,
      "customer": "Daniel Test User",
      "device_info": {
        "type": "solar_home_system",
        "manufacturer": "Spark SHS",
        "serial_number": "123456789",
        "appliance": "Spark Kit"
      },
      "tokens": [],
      "address": {
        "city": "Mafia Village",
        "street": "fadfkldsajlkf"
      },
      "created_at": "2026-03-04 11:34:57",
      "updated_at": "2026-03-04 11:35:31"
    }
  ]
}

But the result

image

@beesaferoot
Copy link
Copy Markdown
Contributor Author

I'm trying to import a customer + device. They both import fine, but the resulting objects are not linked 🤔

{
  "data": [
    {
      "title": "Mr",
      "name": "Daniel",
      "surname": "Test User",
      "birth_date": "2026-03-01",
      "gender": "male",
      "email": "",
      "phone": "",
      "city": "Mafia Village",
      "devices": "123456789",
      "agent": ""
    }
  ]
}
{
  "data": [
    {
      "id": 71,
      "customer": "Daniel Test User",
      "device_info": {
        "type": "solar_home_system",
        "manufacturer": "Spark SHS",
        "serial_number": "123456789",
        "appliance": "Spark Kit"
      },
      "tokens": [],
      "address": {
        "city": "Mafia Village",
        "street": "fadfkldsajlkf"
      },
      "created_at": "2026-03-04 11:34:57",
      "updated_at": "2026-03-04 11:35:31"
    }
  ]
}

But the result

image

Let me investigate that, thanks for providing the import data.

@beesaferoot
Copy link
Copy Markdown
Contributor Author

@dmohns The devices weren't being linked before; I have updated the logic to handle this. Please, can you test again?

@dmohns
Copy link
Copy Markdown
Member

dmohns commented Mar 31, 2026

Thanks for the fix! Proving further range of feedback here:

  1. Could we find a different place for the "Import"? Currently, it's in "Settings which I think it doesn't really belong. WDYT about: Rename the top menu to "Configuration", rename the Menu Item to Settings, e.g.
image
  1. Currently, the confirmation dialog only distinguishes between "Imported" and "Failed". From a user's PoV it would be great if we can say "Added" or "Modified". For example, re-importing the same JSON twice. First time it should say N entries added. Second time it should say N entries modified.

    PS: If this is too hard too add, we can keep as nice-to-have.

  2. Something is off with device geo. The sample contains an address field. But it should have a GeoInformation entry, right? Not sure if I'm missing something.

@dmohns
Copy link
Copy Markdown
Member

dmohns commented Mar 31, 2026

Another thing I just noticed:

  1. Phone number is not enforced to be unique. For example using the import it is possible to create two different users with the same Phone number. The database seems to allow this, but it causes problems mapping sms to the right customer.

@beesaferoot
Copy link
Copy Markdown
Contributor Author

@dmohns I've addressed your comments. Please let me know if anything else is left to be done on my end.

@dmohns
Copy link
Copy Markdown
Member

dmohns commented Apr 2, 2026

I'm trying to modify an existing user. It works, but the confirmation dialog looks like this:

image

Same when adding a new Device. The device gets added. But the dialog shows 0/0/0.

@dmohns
Copy link
Copy Markdown
Member

dmohns commented Apr 2, 2026

Is there a way to surface the import error to the users?

For example

image

is not very helpful.

But what I can see in dev tools is

image

@beesaferoot
Copy link
Copy Markdown
Contributor Author

beesaferoot commented Apr 2, 2026

Is there a way to surface the import error to the users?

For example

image is not very helpful.

But what I can see in dev tools is

image

There should be a way, but it will be quite tricky. Handling bulk uploads, the errors can become pretty large.

How about adding this as an improvement on subsequent PRs?

@beesaferoot
Copy link
Copy Markdown
Contributor Author

I'm trying to modify an existing user. It works, but the confirmation dialog looks like this:

image Same when adding a new Device. The device gets added. But the dialog shows 0/0/0.

This has now been fixed.

Copy link
Copy Markdown
Member

@dmohns dmohns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Let's keep the error handling as a future improvement.

@beesaferoot beesaferoot merged commit 5c9afcf into main Apr 2, 2026
17 checks passed
@beesaferoot beesaferoot deleted the extend-import-api branch April 2, 2026 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants