WebGPU/WebGL rendering

[cmdcolin]

JBrowse (both 1 and 2) have been "optimized for side scrolling". The "static blocks" based methodology reflects this: tiles of data are prerendered side by side. However this is not really the most important type of scrolling! Smooth zoom is arguably more important!

In order to get true smooth zooming, we likely need to adopt webgl. The reason webgl is needed is that we can provide all the "coordinate data" to the webgl renderer, and the GPU just does transformations on zoom level changes or side scroll. Contrast this with canvas, where on zoom level change, we have to completely, on the CPU, re-draw everything with javascript

This is something that can not be done smoothly via canvas. Every single frame needs redrawing. The only "hacky" way to incorporate it into canvas is just applying canvas 'transforms' which distorts. We also see this in our current code with the css 'scaleX' which also distorts

We have to think quite seriously to try to overcome our limitations and get to true smooth scrolling...and that likely requires webgl

I don't know a lot about webgl and there may be some hard challenges to come up against, but this is a very early demonstration of what it might look like to use webgl for the alignments track.

Possible challenges

There are many possible challenges, that range from known challenges to unknown challenges.

Some challenges i currently forsee include:

1. My current thinking is to try to make this just apply on a track by track basis. This means that there might be many webgl contexts on a single screen. This actually is somewhat dangerous because browsers limit the number of webgl contexts that are available. WebGL only allows 16 contexts at a time. However, it is hard to even have 16 tracks open at once. I think that if we combine this with IntersectionObserver, we will be able to make it work 99% of the time. We can consider looking into things like https://github.com/greggman/virtual-webgl

2. It may be challenging to make our code work with our web worker architecture. There is the concept of web workers rendering to main thread canvases, but our current web worker code renders a static bitmap, sends that result to main thread. We don't dynamically control it on a per-frame basis. Furthermore, it is known that serializing too much data between the main thread and webworker is also very slow, so we can't just get automatic gains by parsing on the web worker and serializing drawing to the main thread. This demo is just main thread only.

3. There may some clients that have browser compat issues with webgl. We may need canvas fallback

4. Our svg export will need to have a custom mapping as we use the fact that we rendered to canvas as a mechanism to serialize those canvas commands to svg. that drawing code will need to be persistently probably maintained separately, which isn't very fun, but extreme unit testing might help cover here

Video

This video demo should be compelling to anyone who has ever zoomed in or out in jbrowse. It simply demonstrates smooth zooming. Compare against master where the entire screen blanks and re-renders at each zoom level, and the smooth zoom feels much more natural

out.mp4

Share link

this likely still has glitches and weirdness. only bam files on the alignments track are demonstrated for the time

you can use "mouse wheel vertical scroll" in the area of the bam track to scroll zoom

https://jbrowse.org/code/jb2/webgl-poc/?config=test_data%2Fvolvox%2Fconfig.json&session=share-1eTgCoKkSa&password=x4Byl

Context

This was motivated by work with Pratik on the lorax project https://lorax.ucsc.edu/ this project is deeply webgl driven and is interested in integrating with jbrowse (meeting with Pratik later this week)

Caveats

This PR is very hackily implemented, and largely claude coded. It is unclear if we will run into any blockers, however, even seeing some amount of progress i hope helps to see that it could be quite compelling to implement this