Skip to content

long entity lists load slow #3570

@sleidig

Description

@sleidig

If we have a looot of entities (i.e. >10k records) of a type, that list starts to get slow on initial loading. Pin down the causes (DB read? data filtering? UI - shouldn't be the problem as we have pagination?) and consider performance improvements


Analysis:

  • archiving records does not seem to help reduce the initial loading time. It does help when later filtering on the list, however
  • both app-filter (entities) and app-entities-table (records) receive the full array of entities of the given type. But removing <app-filter> does not improve performance. The bottleneck here really is the database access. This is also confirmed with benchmarking using our @PerformanceAnalysisLogging
  • using a pouchdb query index to skip inactive entities can improve performance hugely, if there are many inactive

possible solutions:

  • only load entities that are not archived (= !e.inactive) using an index
    • requires additional loading when the user toggles the "include inactive" toggle
    • users must archive less relevant records for this to help (for Notes this is not typically done yet)
  • paginate loading of entities. First load one page (<100 docs) and display these, then gradually load more
    • the flexible sorting and filtering is only possible in memory after loading all records. This means we may not be able to load the correct records that need to be displayed
    • we could sort by the "last updated at" date and initially show most recent entities, then load more and update the view with new filtered data as more and more data comes in.
  • similarly, only load active records in entity-select field initially. always display "show inactive" option in the dropdown and only load and search those once clicked initially

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

Waiting for External

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions