Inspector v2: Add lightweight runtime "instrumentation" helpers and a usage example (#16748)

One of the problems we currently have in tooling (and especially in
inspector v1) is having the UI react to scene state changes. Ideally
we'd have observables for every mutable state of a scene, but the
runtime overhead would be too high to make this practical. We currently
work around this in one of a few ways:

1. In some of the tools, we use a global observable, where any part of
the tool UI that mutates any scene state is supposed to fire the
observable. This way all the different parts of the tool UI can
communicate state changes that they initiate.
2. When state changes don't come from the tools (e.g. Playground, or
really any app that uses Inspector), we just poll for state changes
(e.g. continuously re-render react components every ~500ms for example
to detect changes).
3. Provide a refresh button, where the user must click the button to see
changes.

This PR introduces a new idea, which I refer to as "lightweight runtime
instrumentation." The idea is to temporarily hook function calls and
property setters to produce a transient observable. This would not scale
if we tried to hook every property on every object of the scene
simultaneously, but in reality only a very small part of the scene state
is bound to the inspector UI at any time, e.g.:
1. The properties of a single object (in the properties pane).
2. The names of as many nodes as fit into scene explorer (since it
virtualizes, so there are not that many live react components for tree
view nodes bound to scene entities).
When a different entity is bound to the properties pane, or the scene
explorer is scrolled, the temporary hooks are removed and the entity
objects are restored to their original state.

With this in mind, hooking functions/properties to observe scene state
changes specifically for tooling should be quite low overhead, so this
PR introduces the idea and has one example of the usage. There are
helper functions for "intercepting" function calls and property setters,
and then a React hook that translates this to React state, e.g.:
`const computeBonesUsingShadersObservable =
useInterceptObservable("property", mesh, "computeBonesUsingShaders")`
This observable (and the temporary instrumentation of the
`computeBonesUsingShaders` property) is reverted as soon as the
consuming react component is unmounted.

Take a look in this order:

1. The `useObservableState` in
[meshGeneralProperties.tsx](https://github.com/BabylonJS/Babylon.js/pull/16748/files#diff-dca74d1795e7e366439f24e66a2a26fe0b14ef5f3fc4ceb3b05585033b781955R10)
- this is just showing an existing concept where given an observable, we
can produce React state that the component can use to re-render.
2. The `useInterceptObservable` used along with `useObservableState` in
[meshAdvancedProperties.tsx](https://github.com/BabylonJS/Babylon.js/pull/16748/files#diff-1fc176dd99343ae0ff259bf520c6504813ab7c02fbbb06292280310d7a2e588dR11)
- this shows using the new `useInterceptObservable` to create a
temporary observable that lets us immediately respond to state changes
of a property that doesn't actually have an observable.
4. The `useObservableState` implementation in
[instrumentationHooks.ts](https://github.com/BabylonJS/Babylon.js/pull/16748/files#diff-92b79d9fbaaf8f9eef52315919e1ae5640553f1262ca2c43618f091ebb29de4dR19).
5. The `InterceptFunction` helper function in
[functionInstrumentation.ts](https://github.com/BabylonJS/Babylon.js/pull/16748/files#diff-0124bc224339ce336c700bf1128d94268f9abcc1ad92cebeb171e825e250d07cR20).
6. The `InterceptProperty` helper function in
[propertyInstrumentation.ts](https://github.com/BabylonJS/Babylon.js/pull/16748/files#diff-ec2332f5239be4ddf8eab5cb2faa5bb1cf36da855b214cbc44a25f43981a6496R20).

Open to feedback on this idea. So far it seems like it works quite well.

Author: ryantrem

packages/dev/inspector-v2/src/hooks/instrumentationHooks.ts

@@ -0,0 +1,50 @@
+// eslint-disable-next-line import/no-internal-modules
+import type { IDisposable, IReadonlyObservable } from "core/index";
+
+import { useEffect, useMemo } from "react";
+
+import { Observable } from "core/Misc/observable";
+
+import { InterceptFunction } from "../instrumentation/functionInstrumentation";
+import { InterceptProperty } from "../instrumentation/propertyInstrumentation";
+
+/**
+ * Provides an observable that fires when a specified function/property is called/set.
+ * @param type The type of the interceptor, either "function" or "property".
+ * @param target The object containing the function/property to intercept.
+ * @param propertyKey The key of the function/property to intercept.
+ * @returns An observable that fires when the function/property is called/set.
+ */
+// eslint-disable-next-line @typescript-eslint/naming-convention
+export function useInterceptObservable<T extends object>(type: "function" | "property", target: T, propertyKey: keyof T): IReadonlyObservable<void> {
+    // Create a cached observable. It effectively has the lifetime of the component that uses this hook.
+    const observable = useMemo(() => new Observable<void>(), []);
+
+    // Whenver the type, target, or property key changes, we need to set up a new interceptor.
+    useEffect(() => {
+        let interceptToken: IDisposable;
+
+        if (type === "function") {
+            interceptToken = InterceptFunction(target, propertyKey, {
+                afterCall: () => {
+                    observable.notifyObservers();
+                },
+            });
+        } else if (type === "property") {
+            interceptToken = InterceptProperty(target, propertyKey, {
+                afterSet: () => {
+                    observable.notifyObservers();
+                },
+            });
+        } else {
+            throw new Error(`Unknown interceptor type: ${type}`);
+        }
+
+        // When the effect is cleaned up, we need to dispose of the interceptor.
+        return () => {
+            interceptToken.dispose();
+        };
+    }, [type, target, propertyKey, observable]);
+
+    return observable;
+}

packages/dev/inspector-v2/src/instrumentation/functionInstrumentation.ts

@@ -0,0 +1,102 @@
+// eslint-disable-next-line import/no-internal-modules
+import type { IDisposable } from "core/index";
+
+export type FunctionHooks = {
+    /**
+     * This function will be called after the hooked function is called.
+     */
+    afterCall?: () => void;
+};
+
+const InterceptorHooksMaps = new WeakMap<object, Map<PropertyKey, FunctionHooks[]>>();
+
+/**
+ * Intercepts a function on an object and allows you to add hooks that will be called during function execution.
+ * @param target The object containing the function to intercept.
+ * @param propertyKey The key of the property that is a function (this is the function that will be intercepted).
+ * @param hooks The hooks to call during the function execution.
+ * @returns A disposable that removes the hooks when disposed and returns the object to its original state.
+ */
+export function InterceptFunction<T extends object>(target: T, propertyKey: keyof T, hooks: FunctionHooks): IDisposable {
+    if (!hooks.afterCall) {
+        throw new Error("At least one hook must be provided.");
+    }
+
+    const originalFunction = Reflect.get(target, propertyKey, target) as (...args: any) => any;
+    if (typeof originalFunction !== "function") {
+        throw new Error(`Property "${propertyKey.toString()}" of object "${target}" is not a function.`);
+    }
+
+    // Make sure the property is configurable and writable, otherwise it is immutable and cannot be intercepted.
+    const propertyDescriptor = Reflect.getOwnPropertyDescriptor(target, propertyKey);
+    if (propertyDescriptor) {
+        if (!propertyDescriptor.configurable) {
+            throw new Error(`Property "${propertyKey.toString()}" of object "${target}" is not configurable.`);
+        }
+
+        if (propertyDescriptor.writable === false || (propertyDescriptor.writable === undefined && !propertyDescriptor.set)) {
+            throw new Error(`Property "${propertyKey.toString()}" of object "${target}" is readonly.`);
+        }
+    }
+
+    // Get or create the hooks map for the target object.
+    let hooksMap = InterceptorHooksMaps.get(target);
+    if (!hooksMap) {
+        InterceptorHooksMaps.set(target, (hooksMap = new Map()));
+    }
+
+    // Get or create the hooks array for the property key.
+    let hooksForKey = hooksMap.get(propertyKey);
+    if (!hooksForKey) {
+        hooksMap.set(propertyKey, (hooksForKey = []));
+        if (
+            // Replace the function with a new one that calls the hooks in addition to the original function.
+            !Reflect.set(target, propertyKey, (...args: any) => {
+                const result = Reflect.apply(originalFunction, target, args);
+                for (const { afterCall } of hooksForKey!) {
+                    afterCall?.();
+                }
+                return result;
+            })
+        ) {
+            throw new Error(`Failed to define new function "${propertyKey.toString()}" on object "${target}".`);
+        }
+    }
+    hooksForKey.push(hooks);
+
+    let isDisposed = false;
+    return {
+        dispose: () => {
+            if (!isDisposed) {
+                // Remove the hooks from the hooks array for the property key.
+                hooksForKey.splice(hooksForKey.indexOf(hooks), 1);
+
+                // If there are no more hooks for the property key, remove the property from the hooks map.
+                if (hooksForKey.length === 0) {
+                    hooksMap.delete(propertyKey);
+
+                    // If there are no more hooks for the target object, remove the hooks map from the WeakMap.
+                    if (hooksMap.size === 0) {
+                        InterceptorHooksMaps.delete(target);
+                    }
+
+                    if (propertyDescriptor) {
+                        // If we have a property descriptor, it means the property was defined directly on the target object,
+                        // in which case we replaced it and the original property descriptor needs to be restored.
+                        if (!Reflect.defineProperty(target, propertyKey, propertyDescriptor)) {
+                            throw new Error(`Failed to restore original function "${propertyKey.toString()}" on object "${target}".`);
+                        }
+                    } else {
+                        // Otherwise, the property was inherited through the prototype chain, and so we can simply delete it from
+                        // the target object to allow it to fall back to the prototype chain as it did originally.
+                        if (!Reflect.deleteProperty(target, propertyKey)) {
+                            throw new Error(`Failed to delete transient function "${propertyKey.toString()}" on object "${target}".`);
+                        }
+                    }
+                }
+
+                isDisposed = true;
+            }
+        },
+    };
+}

packages/dev/inspector-v2/src/instrumentation/propertyInstrumentation.ts

@@ -0,0 +1,119 @@
+// eslint-disable-next-line import/no-internal-modules
+import type { IDisposable } from "core/index";
+
+export type PropertyHooks = {
+    /**
+     * This function will be called after the hooked property is set.
+     */
+    afterSet?: () => void;
+};
+
+const InterceptorHooksMaps = new WeakMap<object, Map<PropertyKey, PropertyHooks[]>>();
+
+/**
+ * Intercepts a property on an object and allows you to add hooks that will be called when the property is get or set.
+ * @param target The object containing the property to intercept.
+ * @param propertyKey The key of the property to intercept.
+ * @param hooks The hooks to call when the property is get or set.
+ * @returns A disposable that removes the hooks when disposed and returns the object to its original state.
+ */
+export function InterceptProperty<T extends object>(target: T, propertyKey: keyof T, hooks: PropertyHooks): IDisposable {
+    // Find the property descriptor and note the owning object (might be inherited through the prototype chain).
+    let propertyOwner: object | null = target;
+    let propertyDescriptor: PropertyDescriptor | undefined;
+    while (propertyOwner) {
+        if ((propertyDescriptor = Reflect.getOwnPropertyDescriptor(propertyOwner, propertyKey))) {
+            break;
+        }
+        propertyOwner = Reflect.getPrototypeOf(propertyOwner);
+    }
+
+    if (!propertyDescriptor) {
+        throw new Error(`Property "${propertyKey.toString()}" not found on "${target}" or in its prototype chain.`);
+    }
+
+    // Make sure the property is configurable and writable, otherwise it is immutable and cannot be intercepted.
+    if (!propertyDescriptor.configurable) {
+        throw new Error(`Property "${propertyKey.toString()}" of object "${target}" is not configurable.`);
+    }
+    if (propertyDescriptor.writable === false || (propertyDescriptor.writable === undefined && !propertyDescriptor.set)) {
+        throw new Error(`Property "${propertyKey.toString()}" of object "${target}" is readonly.`);
+    }
+
+    // Get or create the hooks map for the target object.
+    let hooksMap = InterceptorHooksMaps.get(target);
+    if (!hooksMap) {
+        InterceptorHooksMaps.set(target, (hooksMap = new Map()));
+    }
+
+    // Get or create the hooks array for the property key.
+    let hooksForKey = hooksMap.get(propertyKey);
+    if (!hooksForKey) {
+        hooksMap.set(propertyKey, (hooksForKey = []));
+
+        let { get: getValue, set: setValue } = propertyDescriptor;
+
+        // We already checked that the property is writable, so if there is no setter, then it must be a value property.
+        // In this case, getValue can return the direct value, and setValue can set the direct value.
+        if (!setValue) {
+            getValue = () => propertyDescriptor.value;
+            setValue = (value: any) => (propertyDescriptor.value = value);
+        }
+
+        if (
+            // Replace the property with a new one that calls the hooks in addition to the original getter and setter.
+            !Reflect.defineProperty(target, propertyKey, {
+                configurable: true,
+                get: getValue ? () => getValue.call(target) : undefined,
+                set: (newValue: any) => {
+                    setValue.call(target, newValue);
+                    for (const { afterSet } of hooksForKey!) {
+                        afterSet?.();
+                    }
+                },
+            })
+        ) {
+            throw new Error(`Failed to define new property "${propertyKey.toString()}" on object "${target}".`);
+        }
+    }
+    hooksForKey.push(hooks);
+
+    // Take note of whether the property is owned by the target object or inherited from its prototype chain.
+    const isOwnProperty = propertyOwner === target;
+
+    let isDisposed = false;
+    return {
+        dispose: () => {
+            if (!isDisposed) {
+                // Remove the hooks from the hooks array for the property key.
+                hooksForKey.splice(hooksForKey.indexOf(hooks), 1);
+
+                // If there are no more hooks for the property key, remove the property from the hooks map.
+                if (hooksForKey.length === 0) {
+                    hooksMap.delete(propertyKey);
+
+                    // If there are no more hooks for the target object, remove the hooks map from the WeakMap.
+                    if (hooksMap.size === 0) {
+                        InterceptorHooksMaps.delete(target);
+                    }
+
+                    if (isOwnProperty) {
+                        // If the property is owned by the target object, it means the property was defined directly on the target object,
+                        // in which case we replaced it and the original property descriptor needs to be restored.
+                        if (!Reflect.defineProperty(target, propertyKey, propertyDescriptor)) {
+                            throw new Error(`Failed to restore original property descriptor "${propertyKey.toString()}" on object "${target}".`);
+                        }
+                    } else {
+                        // Otherwise, the property was inherited through the prototype chain, and so we can simply delete it from
+                        // the target object to allow it to fall back to the prototype chain as it did originally.
+                        if (!Reflect.deleteProperty(target, propertyKey)) {
+                            throw new Error(`Failed to delete transient property descriptor "${propertyKey.toString()}" on object "${target}".`);
+                        }
+                    }
+                }
+
+                isDisposed = true;
+            }
+        },
+    };
+}

packages/dev/inspector-v2/src/services/panes/properties/mesh/meshAdvancedProperties.tsx

@@ -0,0 +1,19 @@
+// eslint-disable-next-line import/no-internal-modules
+import type { AbstractMesh } from "core/index";
+
+import type { FunctionComponent } from "react";
+
+import { useInterceptObservable } from "../../../../hooks/instrumentationHooks";
+import { useObservableState } from "../../../../hooks/observableHooks";
+
+export const MeshAdvancedProperties: FunctionComponent<{ entity: AbstractMesh }> = ({ entity: mesh }) => {
+    // There is no observable for computeBonesUsingShaders, so we use an interceptor to listen for changes.
+    const computeBonesUsingShaders = useObservableState(() => mesh.computeBonesUsingShaders, useInterceptObservable("property", mesh, "computeBonesUsingShaders"));
+
+    return (
+        // TODO: Use the new Fluent property line shared components.
+        <>
+            <div key="ComputeBonesUsingShaders">Compute bones using shaders: {computeBonesUsingShaders.toString()}</div>
+        </>
+    );
+};

packages/dev/inspector-v2/src/services/panes/properties/mesh/meshGeneralProperties.tsx

@@ -3,11 +3,17 @@ import type { AbstractMesh } from "core/index";
 
 import type { FunctionComponent } from "react";
 
+import { useObservableState } from "../../../../hooks/observableHooks";
+
 export const MeshGeneralProperties: FunctionComponent<{ entity: AbstractMesh }> = ({ entity: mesh }) => {
+    // Use the observable to keep keep state up-to-date and re-render the component when it changes.
+    const material = useObservableState(() => mesh.material, mesh.onMaterialChangedObservable);
+
     return (
         // TODO: Use the new Fluent property line shared components.
         <>
             <div key="MeshIsEnabled">Is enabled: {mesh.isEnabled(false).toString()}</div>
+            {material && (!material.reservedDataStore || !material.reservedDataStore.hidden) && <div key="Material">Material: {material.name}</div>}
         </>
     );
 };

packages/dev/inspector-v2/src/services/panes/properties/mesh/meshPropertiesService.ts

@@ -5,10 +5,12 @@ import { AbstractMesh } from "core/Meshes/abstractMesh";
 
 import { GeneralPropertiesSectionIdentity } from "../common/commonPropertiesService";
 import { PropertiesServiceIdentity } from "../propertiesService";
+import { MeshAdvancedProperties } from "./meshAdvancedProperties";
 import { MeshGeneralProperties } from "./meshGeneralProperties";
 import { MeshTransformProperties } from "./meshTransformProperties";
 
 export const TransformsPropertiesSectionIdentity = Symbol("Transforms");
+export const AdvancedPropertiesSectionIdentity = Symbol("Advanced");
 
 export const MeshPropertiesServiceDefinition: ServiceDefinition<[], [IPropertiesService]> = {
     friendlyName: "Mesh Properties",
@@ -19,6 +21,11 @@ export const MeshPropertiesServiceDefinition: ServiceDefinition<[], [IProperties
             identity: TransformsPropertiesSectionIdentity,
         });
 
+        const advancedSectionRegistration = propertiesService.addSection({
+            order: 2,
+            identity: AdvancedPropertiesSectionIdentity,
+        });
+
         const contentRegistration = propertiesService.addSectionContent({
             key: "Mesh Properties",
             predicate: (entity: unknown) => entity instanceof AbstractMesh,
@@ -36,13 +43,21 @@ export const MeshPropertiesServiceDefinition: ServiceDefinition<[], [IProperties
                     order: 0,
                     component: MeshTransformProperties,
                 },
+
+                // "ADVANCED" section.
+                {
+                    section: AdvancedPropertiesSectionIdentity,
+                    order: 0,
+                    component: MeshAdvancedProperties,
+                },
             ],
         });
 
         return {
             dispose: () => {
                 contentRegistration.dispose();
                 transformsSectionRegistration.dispose();
+                advancedSectionRegistration.dispose();
             },
         };
     },