Babel plugin version mismatch issues when shipping librairies leveraging reanimated?

[mgcrea]

If I want to publish a library on NPM that ships reanimated components/hooks that uses both worklets and shared values. From my experience I must compile them (prepublish) with the reanimated babel plugin, if I omit the step, the code does not work anymore (shared values created in the library are never updated in the application code).

The obvious downside is that this only works for a single reanimated version! And as soon as you are out of sync the app errors with Mismatch between JavaScript code version and Reanimated Babel plugin version so I'm not sure what is the current best practice to address this.

I could see a way with a postinstall npm scripts running babel on the dependency but it looks a bit far-fetched.

Any ideas?

[tjzel]

Hi @mgcrea. Check out our new Troubleshooting section - you might find some more general information about this.

About transpiling with Reanimated Babel plugin before publishing - that's exactly what you shouldn't do. Our plugin is an integral part of our JavaScript code and worklets in e.g. Reanimated 3.4.2 should not be present alongside worklets from 3.4.1. Therefore, you should publish your code and list react-native-reanimated as a peer dependency and leave the transpilation process to the user that consumes your package.

What is important to check if your generated JS code (unless you publish source only) contains all the worklet directives (if you had to write any of them explicitly).

If even with those steps the code still doesn't work as expected, I'm happy to help investigate the causes of that.

[mgcrea]

Thank you for your detailed answer, I am indeed witnessing a different in behaviour when I build my library without the plugin.

Does the babel plugin is supposed to compile worklets found in node_modules/?

Not quite easy to give you a reproducible test case since it requires two projects with a build step but here is the big picture:

Issue is with my drag&drop implementation react-native-dnd:

The lib provides a DndContext that declares shared values, for instance: const droppableLayouts = useSharedValue<Layouts>({}); which is supposed to store the draggable declared children layouts (x,y,width,height) keyed by ids.

Then to use a draggable you rely on a custom hook useDraggable in your components that registers itself and its layout to the context via a useLayoutEffect:

export const useDraggable(): => {

  const layout = useSharedValue<LayoutRectangle>({
    x: 0,
    y: 0,
    width: 0,
    height: 0,
  });
  
  useLayoutEffect(() => {
    const runLayoutEffect = () => {
      "worklet";
      draggableLayouts.value[id] = layout;
    };
    runOnUI(runLayoutEffect)();
  }, [id]);
}

Then, on the library side, there is a <DndProvider /> that holds a single <GestureDetector /> that will loop over the layouts to find on gesture start if you are trying to drag an object.

I guess the nested animated value (layouts.value[id].value) combined with the direct set is probably not "orthodox" but it works great. Also I tried with a more simple update from the hook: draggableLayouts.value = { bar: "baz" }; and it does not change so I'd say nested shared values are not the culprit here.

---

The bug arises if I am not compiling the lib with the plugin as my layouts are not updated on the application UI thread (stuck on an empty object {}), the issue goes away entirely if I do pre-compile the lib with the plugin.

Thanks for your help!

[mgcrea]

@tjzel Found the time to publish a demo repo to easily reproduce the issue: https://github.com/mgcrea/react-native-dnd-demo:

Out-of-the-box everything works (with the default @mgcrea/react-native-dnd build compiled with the babel plugin). But if you uncomment the resolverMainFields line in metro.config.js it will use an alternative build (that is published alongside the default working one) that was built without the plugin, you will notice that once the alternative plugin-free build is used, the drag & drop functionality no longer works.

Hope this helps, don't hesitate to ping me if you need more info!

[tjzel]

After some digging I found what is the reason for the difference in behaviour.

Cause

Compare the following snippets:

1. Source `DndProvider.ts`

const panGesture = Gesture.Pan()
        .onBegin((event) => {
          const { state, x, y } = event;
          debug && console.log("begin", { state, x, y });
          // Gesture is globally disabled
          if (disabled) {
            return;
          }
          // console.log("begin", { state, x, y });
          // Track current state for cancellation purposes
          draggableState.value = state;
          const { value: layouts } = draggableLayouts;
          const { value: offsets } = draggableOffsets;
          const { value: options } = draggableOptions;
          const { value: lastActingId } = draggableActingId;
          // Find the active layout key under {x, y}
          const activeId = findActiveLayoutId({ x, y });
          // Update shared state
          draggableActingId.value = activeId;
          // Check if an item was actually selected
          if (activeId !== null) {
            // Update activeId directly or with an optional delay
            const { activationDelay } = options[activeId];
            activationDelay > 0
              ? runOnJS(setActiveId)(activeId, activationDelay)
              : (draggableActiveId.value = activeId);
            // Record any ongoing current offset as our initial offset for the gesture
            const activeOffset = offsets[activeId];
            draggableActiveOffset.x.value = activeOffset.x.value;
            draggableActiveOffset.y.value = activeOffset.y.value;
            // Cancel the ongoing animation if we just reactivated the same item
            if (activeId === lastActingId) {
              cancelAnimation(activeOffset.x);
              cancelAnimation(activeOffset.y);
              // If not we should reset the resting offset to the current offset value
            } else {
              draggableRestingOffset.x.value = activeOffset.x.value;
              draggableRestingOffset.y.value = activeOffset.y.value;
            }
            if (onBegin) {
              const activeLayout = layouts[activeId].value;
              onBegin(event, { activeId, activeLayout });
            }
          }
        })

2. Generated JS of `DndProvider.ts`

var panGesture = _reactNativeGestureHandler.Gesture.Pan()
        .onBegin(function (event) {
          var state = event.state,
            x = event.x,
            y = event.y;
          debug && console.log("begin", { state: state, x: x, y: y });
          if (disabled) {
            return;
          }
          draggableState.value = state;
          var layouts = draggableLayouts.value;
          var offsets = draggableOffsets.value;
          var options = draggableOptions.value;
          var lastActingId = draggableActingId.value;
          var activeId = findActiveLayoutId({ x: x, y: y });
          draggableActingId.value = activeId;
          if (activeId !== null) {
            var _activationDelay = options[activeId].activationDelay;
            _activationDelay > 0
              ? (0, _reactNativeReanimated.runOnJS)(setActiveId)(activeId, _activationDelay)
              : (draggableActiveId.value = activeId);
            var activeOffset = offsets[activeId];
            draggableActiveOffset.x.value = activeOffset.x.value;
            draggableActiveOffset.y.value = activeOffset.y.value;
            if (activeId === lastActingId) {
              (0, _reactNativeReanimated.cancelAnimation)(activeOffset.x);
              (0, _reactNativeReanimated.cancelAnimation)(activeOffset.y);
            } else {
              draggableRestingOffset.x.value = activeOffset.x.value;
              draggableRestingOffset.y.value = activeOffset.y.value;
            }
            if (onBegin) {
              var activeLayout = layouts[activeId].value;
              onBegin(event, { activeId: activeId, activeLayout: activeLayout });
            }
          }
        })

3. Generated JS of pre-transpiled `DndProvider.ts`

var panGesture = _reactNativeGestureHandler.Gesture.Pan()
        .onBegin(
          (function () {
            var _e = [new global.Error(), -16, -27];
            var _f = function _f(event) {
              var state = event.state,
                x = event.x,
                y = event.y;
              debug && console.log("begin", { state: state, x: x, y: y });
              if (disabled) {
                return;
              }
              draggableState.value = state;
              var layouts = draggableLayouts.value;
              var offsets = draggableOffsets.value;
              var options = draggableOptions.value;
              var lastActingId = draggableActingId.value;
              var activeId = findActiveLayoutId({ x: x, y: y });
              draggableActingId.value = activeId;
              if (activeId !== null) {
                var _activationDelay = options[activeId].activationDelay;
                _activationDelay > 0
                  ? (0, _reactNativeReanimated.runOnJS)(setActiveId)(activeId, _activationDelay)
                  : (draggableActiveId.value = activeId);
                var activeOffset = offsets[activeId];
                draggableActiveOffset.x.value = activeOffset.x.value;
                draggableActiveOffset.y.value = activeOffset.y.value;
                if (activeId === lastActingId) {
                  (0, _reactNativeReanimated.cancelAnimation)(activeOffset.x);
                  (0, _reactNativeReanimated.cancelAnimation)(activeOffset.y);
                } else {
                  draggableRestingOffset.x.value = activeOffset.x.value;
                  draggableRestingOffset.y.value = activeOffset.y.value;
                }
                if (onBegin) {
                  var activeLayout = layouts[activeId].value;
                  onBegin(event, { activeId: activeId, activeLayout: activeLayout });
                }
              }
            };
            _f._closure = {
              debug: debug,
              disabled: disabled,
              draggableState: draggableState,
              draggableLayouts: draggableLayouts,
              draggableOffsets: draggableOffsets,
              draggableOptions: draggableOptions,
              draggableActingId: draggableActingId,
              findActiveLayoutId: findActiveLayoutId,
              runOnJS: _reactNativeReanimated.runOnJS,
              setActiveId: setActiveId,
              draggableActiveId: draggableActiveId,
              draggableActiveOffset: draggableActiveOffset,
              cancelAnimation: _reactNativeReanimated.cancelAnimation,
              draggableRestingOffset: draggableRestingOffset,
              onBegin: onBegin,
            };
            _f.__initData = _worklet_14289401489246_init_data;
            _f.__workletHash = 14289401489246;
            _f.__stackDetails = _e;
            return _f;
          })()
        )

You can clearly see the distinction between 1. and 2. compared to 3. - 3. has the function workletized.
However, when you pay close attention to 2. You can see that

const panGesture = Gesture.Pan()

turned into

var panGesture = _reactNativeGestureHandler.Gesture.Pan()

This might seem harmless, but Reanimated Babel plugin is name sensitive. When it comes to auto-workletization of callbacks, we do it by function pattern recognition. To be more specific:

Gesture.Pan().onStart(
// Callback here will be auto-workletized.
)

but here:

someModule.Gesture.Pan().onStart(
// Callback here will not be auto-workletized.
)

because we auto-workletize Gesture Handler callbacks (and many other functions) only if Gesture is not accessed by a dot notation. Therefore generated JS code from 2. will not have some callbacks workletized and therefore the Dnd doesn't work.

We could extend this behavior to those specific functions accessed by dot notation, but it's something we don't really want to do at the moment. Going down the route of complying to various kinds of auto-generated code seems to be a bad decision. Moreover, we don't want to be too invasive with our plugin - while having something.Gesture.Pan().onStart() in your code, when it's not referring Gesture Handler is extremely unlikely, there are obviously other auto-workletized functions which could create patterns that are more common.

Solution

At the moment I see two solutions here.

1. Mark all 'potential' worklets with a 'worklet' directive - callbacks from Gesture Handler, useAnimatedStyle, useAnimatedReaction etc. This might seem burdensome at first, but you only need to do it once. This is the approach I would highly recommend.
2. Mark Reanimated a peer dependency, and its version should be the version of Reanimated Babel plugin you used. Obviously this is a problematic solution. You would probably need to update your library each time we release a new version as people might want to bump Reanimated in their project. Also, people tend to ignore peer dependencies (sadly).

Note

This is an important topic for us. For some time we have been struggling with third party libraries that incorporate various parts of Reanimated into their code, later causing errors for the users and those errors would often incorrectly point to Reanimated as the offender. Unfortunately, our library is quite complex and its parts cannot be simply separated. Therefore it's of vital importance we allow more streamlined approach to the process of creating libraries that depend on Reanimated - it obviously benefits both sides.

I'm very eager to hear back from you about this. Please tell me if the solution I provided actually fixes the problem. If yes, then we could create some guidelines on how to use Reanimated in React Native libraries.

[mgcrea]

Ok I can confirm that adding "worklet" manually to the Gesture.Pan() callbacks triggers the error. Maybe I should have tried that before tweaking the babel plugin 😅 !

[tjzel]

Worklet injecting plugin is a nice idea but then it kind-of misses the point of all auto-workletization present in the original plugin, we would simply have to split it into two parts. It's something to consider in the future, but there could be some issues that come from pre-transpilation in general, not only with our plugin. I will discuss it with my colleagues but for now I'd focus on some minimal guidelines regarding the topic.

By triggers the error I hope you mean it works correctly 😅. Also, I think that (especially when troubleshooting) it's good to publish your source code alongside the generated JavaScript - then if someone wants to find what causes some errors in your code he doesn't have to go back and forth between the npm package and your github 😃

[mgcrea]

@tjzel unfortunately no it does not work at all and I get a new error I had not seen before 😅!

I've published a reproducible test case here: https://github.com/mgcrea/react-native-dnd-demo

You can check that out of the box with the default build it works (with versions synced).

Then just uncomment // resolverMainFields: ["react-native", "browser", "alt", "main"], in metro.config.js to use the alt-build that has all methods explicitly tagged as worklets to trigger an error.

Thanks for your help!

[tjzel]

While I was looking for the root of the new problem, I stumbled upon a thing I forgot was also the case in this scenario. When you look at the published package that was pre-transpiled with the plugin, you can see things like this:

Notice your machine's file path there. This is because for development builds we have to attach relevant debug data to worklets, since they are copied to the UI thread and had they failed we wouldn't know what caused the error. When pre-transpiling, you are attaching this irrelevant debug data, also effectively blocking good debug data to come in. Luckily, we have checks for it and if you tried to run this bundle in release it would result in a crash.

I think the actual reason for the error is this:

When we copy worklets to the UI threads, we also need to capture their closures for them to properly execute. This closure is also copied to the UI thread so the function has it available in its scope. As you can see on the right (it's the pre-transpiled bundle) runOnJS is captured. This is as expected. Unluckily, on the left side (not pre-transpiled bundle) runOnJS inside the function body was changed to _reactNativeReanimated.runOnJS. Therefore our plugin tries to capture the whole object _reactNativeReanimated to the closure of the worklet - which is obviously terrible. You might ask why do we try to capture the whole object - we used to capture the properties only, but it was even more error prone than this approach.

While the situation looks dire, there is a way out. As I stated before, modifying plugin so it would work with all the possible targets and forms of generated code is a dead end. However, if you decided not to publish your code transpiled by babel, creating a commonJS target effectively and generate ESModule instead everything should be fine.

I forked your repo to generate output code with React Native Builder Bob which we use in Reanimated and it works well. Note I used yarn there, because I've never used pnpm and just wanted to make a proof of concept. All you need to do to generate code is [package manager] bob build - I tested it and it works just fine. You probably need to tweak a few things in your project with that, setting wise etc. but it's not that much of a chore I think.

[mgcrea]

Thanks for digging into this,

I think in your case, the "react-native" field in your package.json (cf. the following line) bypasses the build entirely and directly uses the unbuilt source code and that is why it works. So I don't think react-native-builder-bob is really helping for much here.

Anyway, clearly exporting ESM modules seems to be the correct decision as it seems to be nicely picked up by the final app babel plugin and everything seems to work. In my case I ditched babel and only used TypeScript with "module": "ESNext" and everything seems to work fine. I tried with different reanimated version and no more issues!

Thank you for helping peek into reanimated internals and find a good workaround!

[mgcrea]

Solution thanks to @tjzel:

exporting ESM modules seems to be the correct decision as it seems to be nicely picked up by the final app babel plugin and everything seems to work. In my case I ditched babel and only used TypeScript with "module": "ESNext" and everything seems to work fine. I tried with different reanimated version and no more issues!

You can find the relevant commit here:
mgcrea/react-native-dnd@785d519