Backgroundable rendering

[akre54]

Hubble currently relies on Luma's Timeline and AnimationLoop classes which utilize the browser's requestAnimationFrame timer.

This is higher-precision and more realtime than alternatives (notably setTimeout) but requires that the tab be foregrounded in order for the timer to fire. The AnimationLoop class also, as far as I can tell, utilizes "wall time" to update animation state (i.e. Date.now for engineTime) instead of rendering frame-by-frame.

Because browsers only throttle timers running in the main thread and not on Worker threads, it should be possible to use a package like worker-timers to ensure that timers are always rendered.

[chrisgervang]

Do you think this would be a good replacement or configurable alternative to use within luma?

As you touch on, I don't think hubble needs a consistent timer to move through the timeline, but it does benefit from a fast one that isn't throttled in the background.

Tangentially related to this discussion, Deck.gl internal animations (like those found on a ScenegraphLayer) use a separate luma Timeline/AnimationLoop per deck instance. Hubble originally tied it's renderer to that loop, but later detached from it after experiencing issues without clear causes, and because it unintentionally took over GLTF animations (like a rotating helicopter rotor). Hubble should offer a solution for keying baked GLTF animations, but I think there is some investigation necessary before recommending a solution.

But back to the timer discussion, it's possible there are a few usages of requestAnimationFrame between hubble, deck, and mapbox (or maplibrae). Rendering doesn't require wall time (just a loop marching through frames), so I wonder how we might make these adjustments.

Do you think using a non-throttled timer necessary for offscreen rendering? And/or is it necessary to solve the "switching tabs can pause rendering" problem?

[akre54]

Yes, it might be a good fit for a config option in Luma, but maybe more generally or philosophically, I'm curious if Hubble wants to be more focused on the non-realtime use case, while Luma, like most web / webgl stuff, wants to be more realtime and interactive.

If it's an easy add for luma, loaders, etc. I'd say let's try to extend those packages, otherwise I do think there's a niche to be carved out for a full-fledged webgl animation package: keyframes, timelines, and encoding being the main tasks. Luma has support for these, but seems more focused on its specific use cases. I think there's a case to be made for avoiding duplicate work (and not adding yet another rAF timer), but I also can see a lot of benefit from making Hubble renderer-agnostic, and even adding in some UI elements for timelines and animation curves, say.

What are your thoughts here?

Do you think using a non-throttled timer necessary for offscreen rendering? And/or is it necessary to solve the "switching tabs can pause rendering" problem?

Offscreen rendering, no. Background tabs, yes.

It's a battery / cpu optimization thing, but IMO it prevents serious use of non-realtime rendering if you can't use your computer for other tasks while your work is being encoded. Again I sorta think this is one of those borderline features where it wouldn't really benefit most Luma users but would make a massive difference to the rendered use case.

[chrisgervang]

I've thought of Hubble as only focused on the non-realtime use case, and am comfortable making design decisions that sacrifice the typical interactive web concerns in return for "pixel-perfect" results (like, never dropping a frame in a beauty render pass). Other unique capabilities emerge in this design space too like being able to completely swap data sources on every frame if the user wanted to (On the extreme of that I imagine creating a time lapse containing GBs of data paged in from a somewhere during render).

What are your thoughts here? [extend vs own]

I do think about this a good amount. In the past I generally leaned on extension within existing vis.gl packages if there's mutual buy-in for it. When we're unsure, I've hashed it out within hubble and later reevaluated if the result a good fit to be moved.

I see #236 as the encoders going through this kind of re-evaluation to see if loaders.gl is a good fit for the core encoder code, or not. Another evaluation coming up is if the hubble/kepler react components should remain in hubble, be completely moved into kepler, or a mix (the thesis being some components will be generic enough to be reused outside of a kepler app context).

Background tab support is a compelling feature. I assume users are confused when they encounter this limitation, so it should be addressed and, by extension, luma.gl timeline and keyframe re-evaluated. They were originally written by Tarek based on initial assumptions of what he and I thought might be useful for hubble and luma, but as you point out their design principles are pretty different in some cases.

I do think there's a niche to be carved out for a full-fledged webgl animation package: keyframes, timelines, and encoding being the main tasks... making Hubble renderer-agnostic, and even adding in some UI elements for timelines and animation curves, say.

This resonates for sure. It'd be great to use hubble with deck, kepler, and luma, even three.js - in theory anything webgl-based or canvas-based could use it. I've also thought about how nice it would be to support full-browser window capture with puppeteer or electron so that HTML can be used for what it's good at. Of course, I've focused on deck (and kepler) to do a few things well rather than everything poorly - but there's a lot of potential.

[charlieforward9]

Sending a flare in here to understand why this feature never made it further on the roadmap??

[akre54]

@charlieforward9 shifted priorities. Might make for a nice first pull though! Looks straightforward enough to implement