Enabling 3d lightning for extrusion shadows

[yanik-recke]

Hey,

I already spent so much time on this which is why I am looking for help here now. Super grateful for any kind of comment or nudge in the right direction, my head is spinning.

I would like to integrate shadows into my map. I found this gist by Ted, the creator of shademap: https://gist.github.com/ted-piotrowski/420a31bf3c157664fdda14bf45692785

This kinda still works but only for an older version of mapbox. I tried to port it to MapLibre but I am having trouble since my knowledge about shaders and such is rather limited.

Initially, the problem was that the color of the buildings is the only thing that seemed to change (get darker / lighter). It felt like they are acting as a mask and the shadows are only working on the buildings. Additionally, I noticed that setting the transparency to 1 causes the whole map to be overlayed by black so it seemed like the shadow layer is on the whole map:

Through debugging I found out that it's probably the first pass that is broken at the moment. When rendering directly after the first pass and removing the second pass, the whole map is black so it seems like the first pass is just returning the whole viewport.

Then I found out that at some point the attributes seem to have changed from vec4 a_pos_normal_ed; into in vec2 a_pos; and in vec4 a_normal_ed; which might be one of the reason why it is not working anymore.

maplibre-gl-js/src/shaders/fill_extrusion.vertex.glsl

Line 9 in 61396b0

in vec2 a_pos;

By changing from a_pos_normal_ed into a_pos and a_normal_ed I was able to get some kind of shadows back on the map but they are super displaced and vary in different sizes. I would be super grateful for any help regarding this.

Here is a video example of the current state: https://youtu.be/vcQIz_87Tcw

Here is a gist of my current try: https://gist.github.com/yanik-recke/e7d4cd4763adbeff302ad114845e7c78
I am suspecting an issue with the binding maybe?

It includes the style.json and the config.json. If you want to run it yourself you probably have to provide your own tiles, but I can also send you the ones that I am using at the moment.

Best regards

[HarelM]

CC: @mwilsnd maybe you could take a look?
If this is something that looks nice, maybe we can also create an example out of it...?

[mwilsnd]

If I'm understanding what this is trying to do correctly, it seems to be rendering displaced building geometry in a custom layer? It doesn't look like conventional perspective shadow mapping where a depth pass would be rendered from the light's perspective. Without debugging it myself, I suspect your main issue might be in the vertices and indices being sent to your first shader that draws the flattened out buildings.

The displacement seems to require correct normals and height information, make sure the data you're binding to those vertex attributes looks correct. I would also consider disabling back face culling, it's hard to tell in the video but it seems like some building shadows have holes that make me suspect that might be an issue. Are you able to get it working with the older mapbox version to create some comparison images? If the height/base calculations don't evaluate to zero at the base of extrusions, you'll get some amount of XY displacement giving that floating look.

I would double check your draw calls making sure they match what MapLibre does. Just going off the video provided, it does look like broken vertices or indices as some buildings are rendering correctly underneath the broken geometry. For debugging, I'd give something like spector.js a try which should let you see all the GL state and buffers that are bound for a draw call. With that you should be able to compare the base fill extrusion draw calls with the draws for your custom layer.

This does look like a rather fragile way to add building shadows, but it's pretty impressive it can be done without having to modify the library. This isn't going to be compatible with terrain or the globe projection mode without extra work in the shaders, but if you can get the vertex binding right then it could be a neat effect for the standard 2D projection.

[yanik-recke]

Thanks for your response I will take a look at spector.js and try debugging it further. At the moment I inspected the buckets and the buffers using console logging entries.

Here is a video of the working mapbox example: https://youtu.be/EbS80UsJRYY
The code is available here: https://gist.github.com/ted-piotrowski/420a31bf3c157664fdda14bf45692785

I will keep trying, thanks!