Support for custom ShaderMaterials

[Charles-by]

Is your feature request related to a problem? Please describe.

I am currently working with a pipeline where materials are authored in the Babylon.js material system and subsequently exported/generated as Three.js ShaderMaterial objects.

The primary frustration is that three-gpu-pathtracer is currently optimized for standard material types (e.g., MeshStandardMaterial, MeshPhysicalMaterial). Since ShaderMaterial is essentially a "black box" to the path tracer's kernel, it is impossible to path-trace these custom shaders or procedurally generated materials. This limits the engine's ability to handle complex, non-standard visual effects that go beyond the fixed PBR parameters.

Describe the solution you'd like

I would like to see a mechanism that allows the path tracer to "collect" or integrate custom shader logic into its evaluation loop. Specifically:

Custom Shader Integration: A way to parse or inject the logic within a ShaderMaterial so that the path tracer can evaluate its BSDF/sampling logic.

WebGPU Implementation: As the project migrates to WebGPU, I’m curious if there are plans to implement a more modular architecture—perhaps something akin to Blender Cycles' SVM (Shader Virtual Machine) or a node-based shader translation layer.

Extensibility: If a full SVM is too out of scope, perhaps a middle-ground solution where users can provide custom GLSL/WGSL snippets that the path tracer can invoke for material evaluation during the bounce calculation.

[Charles-by]

If you're thinking along the same lines, maybe I could help you build it.

[gkjohnson]

Supporting material definitions has been of interest for awhile (see #479) but unfortunately adding this to the WebGL implementation (and maintaining it in general) has proven to be impractical for a number of reasons. With the WebGPU version coming up and nodes being available in three.js I think this is looking a lot more reasonable, though. Realistically the WebGLPathTracer will be deprecated and removed from the project at some point after WebGPUPathTracer is complete so it would be best to focus on how WebGPU and nodes can achieve this rather than ShaderMaterials.

Implementation-wise I think the most simple way to think about this is slotting a TSL function (TSL, wgslFn, color node, etc) to be called for a BSDF, color input, etc and rely on TSL's shader construction logic to build the final kernel for us. Ergonomically it would be ideal if a TSL node material could "just work" but there are some open questions regarding implementation:

• At what level of granularity can custom nodes be used? As a full replacement BRDF? As just replacement input nodes for existing material definitions (eg MeshStandardNodeMaterial.color node)?
• How procedurally inspectable is the TSL graph? How will we handle nodes like those for fetching world space fragment values? Or texture sampling?

I don't expect we can make this work with every type of node but it would be good to know what the scope and limitations should be, so that might be a good place to start. In practice the webgpu-pathtracer branch only has support for a basic lambertian model so far but even so there should be enough in place to prototype something like the ability to declare a custom color node to see how that would look.

perhaps something akin to Blender Cycles' SVM (Shader Virtual Machine)

The SVM is interesting but also sounds significantly more complicated. Using TSL function slotting seems more approachable to me right now, at least.

cc @TheBlek