bugfix/overscaled 3d tile raycast

[dwayneparton]

Issue

When using terrain with a DEM source that has a lower maxzoom than the current display zoom (e.g., DEM maxzoom of 10 while viewing at zoom 14), the pointCoordinate raycast was not consistent with the rendered terrain. This caused issues where clicking on the visible terrain surface would return incorrect 3D coordinates.

{"type":"raster-dem","tiles":["https://example.tile.server/v1/terrain/{z}/{x}/{y}.png"],"tileSize":256,"maxzoom":10,"encoding":"terrarium"}

bug-raycast-on-overscaled-tiles.mov

Solution

The raycast now uses a proxy tile-based approach when overzoomed, which matches the exact geometry rendered by the terrain shader.

fix-raycast-on-proxied-tiles.mov

Launch Checklist

• Make sure the PR title is descriptive and preferably reflects the change from the user's perspective.
• Add additional detail and context in the PR description (with screenshots/videos if there are visual changes).
• Manually test the debug page.
• Write tests for all new functionality and make sure the CI checks pass.
• Document any changes to public APIs.
• Post benchmark scores if the change could affect performance.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ dwayneparton
❌ ThibaudLopez
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[github-actions]

Hey, @dwayneparton 👋 Thanks for your contribution to Mapbox GL JS!

Important: This repository does not accept direct merges. All changes go through our internal review process.

What happens next:

1. A team member will review your PR here first
2. If it looks good, they will import it to our internal repository for further review
3. If approved, changes will be synced back here via our release process

Please respond to any review comments on this PR. For more details, see CONTRIBUTING.md.