Possibilities of extending the Maplibre-gl-js library while developing a weather application

[patrikschiller]

Hi,

I am developing a weather application on top of Maplibre library as a custom WebGL renderer. I already have one fully working version combining radar, satellite, land, and country-borders layers (with some occasional bugs) and now I would like to rewrite it from scratch (to fix bugs, make the code cleaner, etc.). However, the current version of my application uses Maplibre only minimally (basically only for handling map events, everything else is implemented as a whole new renderer using CustomLayerInterface - as I inspected the code of MeteoBlue app, I found out, that they do it similarly).

The thing is that some parts of my app's functionality are duplicate with the code which is already part of Maplibre library (WebGL-related stuff, tile handling, etc.). So I was thinking about trying to rewrite the application in a way to reuse as much of Maplibre code as possible so that some of the new functionality I implement could be potentially integrated into Maplibre later. However I am not sure, whether it is possible, and if so, then where to start.

I did some rough code analysis of Maplibre and came out with two solutions:
i) Try to extend RasterTileSource and RasterTileLayer, since the weather data are tile-based rasters - however, I am not sure, how hard it would be and whether it is possible and a good approach.
ii) Stick with the old solution implementing the application using CustomLayerInterface and just use some minor parts of Maplibre somehow... However in this case I am aware of the need to implement a major part of the core functionality by myself again since some parts of Maplibre code are private functions and properties that should not be accessed from outer scope.

While developing the application using Maplibre components, I could also contribute some code to the library. For example, I was thinking about extending the RasterTileSource / RasterTileLayer to support custom fragment (and vertex) shaders (also that was the first thing I was trying to figure out when I was starting with the app implementation - did not find any solution so I started with the mentioned CustomLayerInterface and did all from scratch - including tile handling and all stuff, which is also implemented somewhere in Maplibre). I think that it would be great if one could supply a tile source parametric URL and fragment shader source, which renders the tile images with some custom transformation. The reason is that not in all cases the fetched images are meant to be directly rendered on screen - they need some color transform / pre-process.

To make it more clear, here are some functional requirements:

• i) render raster tiles from parametric z/x/y source [supported by Maplibre]
• ii) render raster tiles from parametric z/x/y source using custom fragment shader / with a possibility to perform some operations on the input raster [NOT supported by Maplibre as I know]
• iii) -||- using custom fragment AND vertex shader (for example, the radar and satellite layers perform postprocessing on tile composites, and instead of reading per-tile textures, they read one tile-composite texture -> extra computation must be performed also in the tile Vertex shader) [NOT supported by Maplibre as I know]
• iv) render not-tile-based data - this is too specific, and probably should be implemented using CustomLayerInterface

The current weather application uses the mentioned FRs as follows:
i) for rendering country borders (no raster transform needed),
ii) for rendering land (dual-channel data transform to quad-channel),
iii) for rendering radar and satellite (before the raster-data are rendered, they must be decoded, put into a composite, preprocessed, and interpolated based on time -- this could be done in the preRender method as fully custom functionality not related to Maplibre, because it performs multiple offscreen render passes. The final result could be read by the custom tile Fragment Shader as defined in ii) ),
iv) for some non-tile-based data

Based on the requirements, do you think that I should try to implement the renderer in a way that it extends the current Maplibre functionality, or it is too much specific problem that it would be better to implement it using CustomLAyerInterface and do not care about extending Maplibre?

Thank you for any recommendations :)

PS: Sorry for such a long post, it's hard to express myself and my thoughts.

[HarelM]

I don't fully know what parts of maplibre would be helpful, but your can do a lot of stuff related to tiles with addProtocol.
You can also add a custom source, which is more complicated I think.
If you think some parts of the code should be added to the public API that's another thing we can consider.
If the button line is that the things you need are too specific to your problem, it might be that you'll need to roll your own, IDK, I guess I need more concrete examples.
Also it's worth considering which hooks can be added to maplibre to allow you to extend it better.

[patrikschiller]

Hi,

thank you for your fast reply.

We had a discussion today and came out with a set of files we need to edit and create, which will enable us to implement our application using Maplibre. Some of our use-cases are very specific, however, some of them could be potentially useful for other developers. The specific functionality would be part of passed callbacks, while the general functionality would be part of the code changes in Maplibre.

The main requirement from us is the ability to extend the raster tile renderer, which is currently implemented in Maplibre. It currently does not provide support for any edits of the input raster (fetched from z/x/y source), only some minor tweak parameters such as contrast and brightness - however we need to perform more complex operations, read more textures, and work with different set of uniforms - this includes passing our custom shaders to the rendering process. However, the rest of the tile functionality does not need any changes (such as fetch, store in cache, load, unload, etc.) - it is implemented very well and there is no need to implement it again by us (although we already have it, but not that robust).

The problem is, that we cannot implement the mentioned changes without making minor changes in the already existing files in the Maplibre library. I'll try to list them all and explain why those changes should be made. Also, a mix of a class/dependency/flow diagram is provided to make it clearer (the orange color represents files to edit, green color represents new files). We created the diagram by backtracking from the gl.drawElements() call, which is the final call before rendering onto the screen.

• raster_custom_program.ts [new file] - probably won't be used (depends on the final decision on the architecture of custom rendering), could be an alternative for setting custom uniform variables
• draw_custom_raster.ts [new file] - this file replaces the original draw_raster.ts and accepts user-defined "callback", which contains its own rendering method (mix of bound uniforms, buffers, textures, etc.). So instead of defining our use-case-specific logic (which is basically the case of the current draw_raster.ts = "hardcoded" parameters and flow of rendering), this method will only accept callback (which handles the use-case-specific rendering logic). This means, that every developer can supply his own rendering method of a tile. This is also related to creating custom shaders with custom variables (uniforms).
  For example, as I mentioned in the previous post, we need to read a composite texture with edited texture coordinates.
• painter.ts [edited file] - we need to append the current switch-case (row 506+) by our new layer type (for example "raster-custom")
• shader.ts [edited file] - we need to be able to set new shaders into the "shaders" table. To make it clean and reusable by other developers, we propose an appendShader(vsSource, fsSource) method, which appends the new shader program into the "shaders" array/object. Hardcoding the shader right into the table (including the shader imports) is not a good and reusable solution.
• custom_raster_style_layer.ts [new file] - since we need to perform some preprocessing (such as creating the composite textures), we need to perform some offscreen rendering before the actual rendering. The "prerender" method is an ideal candidate for this, however, it is available only in the "custom" layer type. So we came out with creating a new style layer, which also contains the user-supplied prerender method while extending the raster_style_layer.
• create_style_layer.ts [edited file] - to be able to create the new mentioned layer type, we also need to extend the switch-case statement by the new layer type key ("raster-custom").
• custom_raster_tile_source.ts [new file] - since our requests can contain more URL parameters than "zoom/x/y", we also need to construct different URLs. The easiest way to do this is to create a new raster tiles provider (via loadTile()) containing a custom request transformer (we already have it) - instead of trying to reimplement the current requestManager module with our specific parameters.
• source.ts [edited file] - the probably last file we will need to edit. Once again, we need to append the current switch-case statement by the new layer type "raster-custom"

We tried to design the changes to have minimal impact on the current Maplibre code while providing benefits to all developers who would need to render raster tiles using their own method / workflow and data.

If you have any notes / ideas / recommendations, feel free to share them.

Thank you :)

[HarelM]

Thanks for the detailed explanation!
This seems a bit too specific to raster.
While I'm not sure a new shader can be loaded ATM in maplibre and appendShader might be missing, I know there are three extensions that seems very similar to what is written here: the addSource, CustomLayerInterface and addProtocol.
I would prefer to focus on extending or complication those instead of narrowing things to custom raster source and layers.
This is also what I wrote in my previous post and I didn't see any references to in in the last post you wrote, it would be best to explain why these are not solving the problem at hand.

[patrikschiller]

Hi,

sorry for not mentioning the addProtocol method. I inspected its functionality, and in my opinion, it does not fulfill all our requirements (if I understand it correctly). It looks like, that it can be used to modify per-tile requests (which could be useful for adding our custom URL parameters) and also perform some tile preprocess since it returns tile raw data (e.g. using canvas and then return back-read data). However, the preprocessing in this way won't be performing very well in my opinion (we need to transfer the transformed data back from GPU to CPU / perform the whole preprocess on CPU, so we can return the ArrayBuffer). So yeah, it can be useful to a certain degree, but it won't solve all our requirements.

I am a bit confused by what do you mean by the addSource method and CustomLayerInterface. The CustomLayerInterface is what we currently use - it is an interface between Maplibre and our renderer, but as I already said, using this approach we have a duplicate functionality for the tile rendering (since the "custom" layer does provide only WebGLContext and map transform matrix, however everything related to tiles is not available and must be implemented by us). Well, theoretically it could be used, if we imported the required methods (however some of them are not exported, like shaders.ts->compile()) and modules from the library but I don't think, it would be a good solution... It still would be a hybrid combining Maplibre and our code and still introduce some duplicate code.

And I am not sure, what did you mean by the "addSource" method. Yeah, we can add more raster sources, but I don't know, how it is related to solving our problem. Btw, the mentioned new "custom_raster_tile_source.ts" would be added to the map by the addSource method.

Back to your first point, yeah, our problem is probably too specific, however since our data are raster-tile-based, it is a good spot where to extend the functionality.

However, if you think that our problem is too specific and complicated to try to extend the Maplibre functionality and that our current solution is a better choice, please, let me know. We are still going to rewrite the application from scratch, maybe combining our code with Maplibre code to a certain degree (the mentioned hybrid), maybe rewriting Maplibre code in a way described in the previous post, but as an internal custom fork. The reason, why I started this discussion was to communicate the best approach so we could implement our application in a way, that some general functionality could be merged into Maplibre. And also to minimize duplicate code / functionality.

Thank you for your patience :)

[HarelM]

addSource allows you to add a custom source to maplibre, which might be similar to things you wrote here - this is one of the hooks I have yet to see someone use properly, but I would like to see if it helps or not and how in this specific case.
If you say that CustomeLayerInterface is lacking some data, let's try and address that.
If there's a need to extend the ability to compile and run shaders let's address that as well.
If there are methods or classes that you think might be beneficial to export let's do that as well.
What I'm trying to say is that I would love to take this use case to see how the current extension points address your specific need.
From what I understand by the explanation you provided, the solution is specific to "custom raster" kind of layer/source and your domain problem. I would like to see if we can generalize this to any kind of layer/source, not just raster so that other can utilize these hooks.
I would also love to know what the limitation of the current hooks are and how to improve them to solve this specific need.

[patrikschiller]

Hmm, I forgot about the fact, that our data are also time-dependent, so we also need to load different tile sets when using a time slider for the same viewport state.

After some discussion, we think, that the easiest solution for us would be to implement the application using Maplibre in a way we need and which should be a best-practice approach, while trying to minimize changes in the original Maplibre library, and with respect to your advice. We are still talking on a theoretical level and there might be more caveats that are not apparent right now and which we would need to resolve, which also means contacting you / Maplibre developers again.

After we make some major progress we can provide some feedback and after the application is completed, we can retrospectively share changes, we had to perform inside the original Maplibre code. After that, we can have some discussion on the possible integration / merge.

Thank you for your cooperation, I'll let you know when something is new.