How to display several million points on the map

[treizexiii]

Hi,

I work in small startup who want to integrate martin in their solution.
The stack under consideration is a postgis database, martin as tiles server and maplibre in the front app.
The plan is to display several million points on the map. Is this possible with Martin, or should we consider other options?

[CommanderStorm]

Yes, that should be easily doable.

Depending on the exact requirments how you want to display things, for lower zoom levels, aggegation is benefitial, but as long as it is just points this should not be a problem.
Maplibre supports clustering on the client side for this, but if you run into problems with performance moving this to the server side is always an option.
I don't think you need to move things to the server side.

Dominik did something similar, but he used parquet instead of tiling MVTs.
If you want the utmost level of performance (better than MVT, better than parquet), converting to MLT is an option in general (not currently implemented in martin 😉).
I don't think you need this.

Millions is in the end not that much.

[treizexiii]

Thanks for your reactivity.

However, I've tried serving a table containing 500,000 rows and displaying them individually, but my front-end application freezes frequently.

I don't have much knowledge of GIS, I simply configured Martin on a database for testing purposes.

[CommanderStorm]

That is unfortunate.
What is the cause of the stutters?
Is it the frontend lagging or the server not delivering tiles fast enough?

It kind of depends what you want to achive, so which way is right.
I named a few optimisations (which come with tradeoffs) that are possible such as clustering, ..
Lets step back: What would you like to do in the first place? it is fairly hard to give good recomendations otherwise.

Lets rule out the simple things first:

• please go to /_/metrics or on the network tab to ensure that what is stuttering is not postgres.
• what is the tile size (the size of the network packets going to your client)?
• that the points don't come with metadata (all other collumns) that you are not rendering at the zoom level requested

[treizexiii]

I checked the metrics and I don't have any particular problems. The packet received from Martin is 10MB. However, when it has to be rendered in the front-end application, it takes a long time to display and navigating the map is very slow. And it's only 300,000 points.

[CommanderStorm]

Yea, 10MB is too much for most network connections.
Maybe also the frontend, but at 3s download times this is the first bottleneck.

Do you need all of this data displayed at the zoom level?
Can you aggregate on the server side?
We support this via postgrres functions.

[treizexiii]

Unfortunately, I need to project all the points individually with their attributes. I tried clustering but customers need to work with all the data for pattern detection. I wonder if martin/maplibre is the good way to provide ao much data.

[CommanderStorm]

At higher zoom levels sure. there you need attributes.
For the lower ones? Is your monitor really able to display the difference between the points?
-> where you might actually load hundreds of thousands of points.
I think here a heatmap style aggegation is likely what you want instead.

Our support for heatmaps in martin currently is not great.
I think it can be hacked together, but it is not quite documented.

Here is for example how this looks by our friends at deck-gl
https://deck.gl/examples/heatmap-layer

Alternatively, maybe you want an index like A5 or H3 instead?
Here is the example from our friends at deck-gl
https://deck.gl/examples/hexagon-layer

Again, martin does not quite support this in the user friendlyest of ways.
We would be open to contributions on both ^^