Feature: Intercept downloads to transcode/decrypt payload

[pke]

Currently I am downloading encrypted payload and its stored on storage as a file. To display the data in the app
After the download I have to read the whole file that was just written to storage back into memory (in chunks or as a whole), decrypt it, save it back to storage and removing the encrypted version of the file.

I wonder now if the library could allow the user to supply a file writing interceptor that this lib delegates the file writing to like this:

await BlobCourier
  .onDownload({
    blockSize: 16384 * 5,
    transcode: (input: ArrayBuffer, last: boolean):Promise<ArrayBuffer> {
      last ? return aes.finish(input) : aes.decrypt(input)
    }
  }
  ).fetch("https://example.org/encrypted.enc")

Some starting point maybe?
https://medium.com/swlh/okhttp-interceptors-with-retrofit-2dcc322cc3f3

It would be crucial to operate on streams rather than having the whole response in memory.

[edeckers]

I love this idea of being able to inspect and transform the stream, I'll add it :)

It could become quite noisy on the bridge though, so I hope that doesn't thwart this plan.

As for your specific use-case and for my perspective: it would be more efficient to use an encryption library that decrypts data outside the JavaScript thread, no? Something like the snippet below:

const request0 = ...

const fetchedResult = await BlobCourier.fetchBlob(request0);

const { absoluteFilePath } = fetchedResult.data;

await NativeEncryptionLibrary.decrypt(absoluteFilePath, Method.AES);

Oddly I didn't find any library that supports this after a little googling, which I figure is your reason for processing the stream in the JavaScript thread?

[pke]

The stream would be processed natively, not in JS using react-native-simple-crypto but I'd like to not have an XX MiB encrypted file on disc temporarily when I could just decrypt the downstream and only write the decrytped bytes to disc.

Another way would be to maybe have react-native-blob-courier-crypto plugin that natively can be added to process the downstream? Then no calls over the JS bridge would have to be made.

import crypt from "react-native-blob-courier-crypt"

await BlobCourier
  .addNativeInterceptor(crypt, { key: aesKey, iv: aesIv })
  .fetch("https://example.org/encrypted.enc")

Then blob-courier could natively call the native plugin? And react-native-blob-courier-crypt would be a very thin wrapper around "react-native-simple-crypt" in that case.

Does this sound like a plan?
Performance and memory consumption is the key here.

[edeckers]

The stream would be processed natively, not in JS using react-native-simple-crypto but I'd like to not have an XX MiB encrypted file on disc temporarily when I could just decrypt the downstream and only write the decrytped bytes to disc.

Sounds very reasonable :)

The possible problem with both suggestions is congestion of the bridge; maybe it isn't even a problem, but we'll only find out when we test it. There should be a more efficient way than this, though:

1. JS sends fetch request over the bridge to native
2. Native passes chunk of received data over the bridge to JS
3. JS sends the exact same data back over the bridge to decrypt method
4. Native sends decrypted data back over the bridge to JS
5. JS sends unaltered decrypted data back over the bridge to Native where it is finally stored

This is true for both the first suggestion and the plugin solution, or maybe I don't follow correctly?

I concocted another solution: send each received package not over the JS bridge, but over an in-process (Native) bus. That way you can create a simple wrapper around a library like react-native-simple-crypto as you suggested, which subscribes to the in-process bus. This would avoid sending any unnecessary messages over the JS bridge:

1. Wrapper library subscribes to Native in-process bus
2. JS sends fetch request over the bridge to Native
3. Native passes chunk of received data to in-process bus
4. The wrapper library receives the data chunk passed to in-process bus
5. The wrapper library decrypts data and writes the result to disk

Does this make sense? What it boils down to is that separate libraries and plugins can only work together through messaging and my suggestion avoids using the JS bridge for that, which I expect to result in better performance and less memory consumption.

NB. I think I will release both solutions as separate features; I'll start with your original plan of onDownload, it probably suffices a lot of the time.

[edeckers]

I've created feature requests #139 and #140 for this; first on I'll work on will be #139.

[pke]

@edeckers I would opt for the high performance in process solution to start with. However I would give the plugin only transform capability an no file writing burdens.
I thought of a simple byte stream transform (which should work with most block ciphers).

courier	plugin
receives block
sends block ->	decipher block
	<- deciphered block
write deciphered block

[pke]

Coming back to this... how about using the new native JNI of react like react-native-vision-camera? Then no bridge is involved, if I understood that correctly?

[edeckers]

Hi @pke , apologies for the late reply I was vacationing.

As for your question, I think you meant to refer to the new JSI architecture / Turbomodules or am I wrong? I have been working on migrating BlobCourier to it #213. There's two things holding it up for now:

1. There is a bug in mockk Instrumented Android tests are all failing due to issue with mockk 1.12.4 #217, related to M1 architecture, which causes my instrumented Android tests to fail - although I could circumvent this by just reverting and hardcoding the library to its previous version
2. I need to add instrumented Android tests that check not only the NativeModules but also the TurboModules
3. I'm not 100% confident yet I migrated everything correctly, the documentation for migrating is still lacking a little, but when (1) and (2) are fixed I think I'm confident enough

That being said, I don't think react-native-vision-camera uses the TurboModules-architecture, but they do use JNI as you mentioned correctly. Although they seem to achieve it through this method: https://thebhwgroup.com/blog/react-native-jni. That fixes the problem only for specific plugins you'd write for BlobCourier though, you wouldn't be able to hook-up react-native-simple-crypto as-is.

The way I understand TurboModules at this time - as I mentioned above, I'm not 100% confident yet - I think you're right and they should solve our problem indeed. But only if both BlobCourier and react-native-simple-crypto migrate to it. There will be efforts to convince maintainers do migrate their libraries as soon as possible, so you might be in luck: https://reactnative.dev/blog/2022/03/15/an-update-on-the-new-architecture-rollout#the-3rd-party-libraries-ecosystem

If we identify a library that becomes a blocker for a number of users, we will try to reach out to the maintainer and understand why they haven’t migrated yet.