Service Workers Support

[sergiodxa]

Proposal

Add native support for Service Worker with an optional entry.worker file that Remix could compile to a SW and inject the setup into the HTML.

Background

Service Workers (SW) is not only required to build a Progressive Web App (PWA), it can also be used for a lot of other features like Push Notifications, cache assets and even provide offline support by intercepting fetch requests to the server and cache them.

Right now you need manually code the SW inside the public folder and if you want some build process to share code between the app and SW you need to manually run esbuild on the SW and put the output on the public folder.

I think Remix could help here by providing an entry point for SW, by adding an entry.worker.ts|js file inside app it could detect that and build it.

Additionally, it could either automatically register the SW or provide a function to do it in entry.client.

By doing this it would be possible to share code between SW and the rest of the app.

Something that Remix could also do is provide a magic import to has access to the built files so the developer could use that to cache or pre cache those files.

Use Case

• Include the SW in the build process
• Provide a way to register the SW
• Provide a way to share code between the SW and the rest of the app
• Allow the developer to pre cache assets
• Allow the developer to intercept requests to the server, and cache them

API/Examples

Create an entry.worker.ts file in app, together with entry.client.tsx and entry.server.tsx.

Inside this file, you can code the SW as usual, but you can also import files from the build process.

// magic export like the one used in CF/Deno for the server build
import * as build from "@remix-run/dev/client-build";

// import code from the app folder
import { someFunction } from "~/utils/some-file";
someFunction() // use the function

// cache the assets using the CacheStorage API
await caches.open("assets").then((cache) => {
  // here build.assets can include the assets from the build process
  cache.addAll(build.assets);
});

// intercept requests to the server and cache them
addEventListener("fetch", function listener(event) {
  event.respondWith(
    caches.match(event.request).then((response) => {
      return response || fetch(event.request);
    })
  );
});

// add Push Notifications
setupPushNotificationsSomehow()

To register the SW Remix could either add it to the output of <Scripts /> or give a function that the developer could call in entry.client

import { registerSW } from "@remix-run/service-worker";
// other imports

registerSW();

// render React (or move the render above the registerSW)

[ShafSpecs]

[sergiodxa]

A SW is another file you need to register on the browser, once you have the file in entry.client you need to call the function to register it.

My idea is to have entry.worker as the entry point of the SW so Remix compiler could build the file for you.

[ShafSpecs]

Got it! Separate the entry points for the SWs and use entry.client would trigger that file on the browser.

[ajhaining]

It's worth adding in @jacob-ebey's example to the discussion... https://github.com/jacob-ebey/remix-pwa

It uses the entry.worker.ts style. I think it makes sense to have it as a 'magic' entry point like the other entry points, so it doesn't need custom build scripts. Although the above example shows how it can be done with some simple package.json scripts.

If it's not too much work, it would be useful to provide a manifest of all the static assets generated as part of the build (window._remixManifest currently only has the JS files). This would be useful for pre-caching things for offline support in either a custom implementation, or via something like workbox. Workbox Webpack Plugin does this with a simple replacement when compiling.

Providing a function to register the SW seems more sensible than automatically registering it, as there might be different use-cases for how, and when to register the SW, for instance, as soon as a new one is available, or when the user clicks a button to update. Create React App provides some nice examples of this.

Excited to see this come to life.

[ShafSpecs]

Jacob's PWA example with entry.worker.ts would do as a starting point, I tweaked a bit with the code and see potential for pre-caching, that would come much later though.

Seems a bit funny to me but Remix's transition hook seems like a powerful handler for SW states and when to register one. That's the angle I am working on rn. It might be overboard but I think it's something that can be considered? Also for manually registering the SW

[ShafSpecs]

https://github.com/ShafSpecs/remix-pwa

Building something. The foundations are solid and it looks promising

[MuraraAllan]

I used Response instance + entry.client to register install and uninstall service workers.

https://remix.run/docs/en/v1/api/conventions#returning-response-instances

I'm not aware of the best solution here, as it involve security and a bunch of stuff to be considered, bnut this worked for my case :

create a route (in this case /sw)

-------- routes/sw.tsx -----------
export function loader() {

  return new Response('console.log("hello world from test")', {
    headers: {
      "Content-Type": "text/javascript",
    },
  });
}

------------ /app/entry.client.tsx -----------------

if ("serviceWorker" in navigator) {
  window.addEventListener("load", async function swLoader() {
      const registration = await navigator.serviceWorker.register('/sw');
    })
}

that is enough to be able to register an SW from the scratch, All the caching, uninstalling, etc still needs to be done.

[jamesarosen]

I've been thinking about use-cases for server-side Service Workers. These are things that would be possible to implement in application-land, but much nicer to implement on a Plugin. A Plugin can only do these things, however, if it can intercept fetch. The Service Worker API is designed for exactly that.

Shared Cache

If your server-side app makes upstream HTTP requests and runs on multiple servers/VMs, they would benefit greatly from a shared cache, say in Redis.

Cache-Control Math

You have an app that uses HTTP to connect to upstream providers. A single page connects to multiple endpoints and composes data. Each upstream response includes some combination of Cache-Control and Vary response headers (and possibly Surrogate-Key or Cache-Keys for upstream clients that come through Fastly or Cloudflare respectively). You want to combine those response headers into your response header so you get proper caching semantics.

Example: if GET /foo is cacheable for 5 minutes and GET /bar is cacheable for an hour but varies on Accept-Language, then GET /page-with-foo-and-bar is cacheable for 5 minutes and varies on Accept-Language.

[sergiodxa]

The cache-control math could be a nice and useful library.

The shared cache is possible right now by intercepting your own requests and caching the responses somewhere else.

I have suggested on Discord before to have a createCacheStorage that could be used as a unified cache interface that you could then use to model a createRedisCacheStorage or createKVCacheStorage or createMemoryCacheStorage.

And you could then create a fetch wrapper that uses that cache when interception requests.

[jamesarosen]

The shared cache is possible right now by intercepting your own requests and caching the responses somewhere else.

That's absolutely true if your application is calling fetch or even something like useAPI. But it's harder if you're using some third-party library like BazaarVoice or Pixlee where the library does the fetching and the conversion to JS objects / HTML.

Some interpreters may allow replacing globalThis.fetch. That would certainly be a viable alternative to a Service Worker.

[jdnichollsc]

I'm using this configuration with Workbox and works really great:

// Imports Workbox from the CDN. Note that "6.2.0" of the URL
// is the version of the Workbox runtime.
importScripts('https://storage.googleapis.com/workbox-cdn/releases/6.2.0/workbox-sw.js');

if (workbox) {
  console.log(`Yay! Workbox is loaded 🎉`);
  initializeWorkbox();
} else {
  console.log(`Boo! Workbox didn't load 😬`);
}

function initializeWorkbox() {
  // Automatically adds an activate event listener to service worker
  workbox.core.clientsClaim();

  const navigationRoute = new workbox.routing.NavigationRoute(new workbox.strategies.NetworkFirst({
    networkTimeoutSeconds: 3,
    cacheName: 'navigations'
  }));
  
  const staticAssetsRoute = new workbox.routing.Route(({request}) => {
    return [
      'image',
      'font',
      'style',
    ].includes(request.destination);
  }, new workbox.strategies.CacheFirst({
    cacheName: 'static-assets'
  }));

  const mediaAssetsRoute = new workbox.routing.Route(({request}) => {
    return [
      'video',
      'audio',
    ].includes(request.destination);
  }, new workbox.strategies.CacheFirst({
    cacheName: 'media-assets',
    plugins: [
      new workbox.cacheableResponse.CacheableResponsePlugin({
        statuses: [200]
      }),
      new workbox.rangeRequests.RangeRequestsPlugin(),
    ],
  }));
  
  workbox.routing.registerRoute(navigationRoute);
  workbox.routing.registerRoute(staticAssetsRoute);
  workbox.routing.registerRoute(mediaAssetsRoute);
}

// This allows the web app to trigger skipWaiting via
// registration.waiting.postMessage({type: 'SKIP_WAITING'})
self.addEventListener('message', (event) => {
  if (event.data && event.data.type === 'SKIP_WAITING') {
    self.skipWaiting();
  }
});

Are you looking to generate this public file from routes instead? 🤔

[sergiodxa]

My suggestion is to have an entry.server.ts that Remix can build together with the app and because of that you can import code from the rest of your app, and you could also import the list of built files so iyou can pre-cache them all.

[Sarabadu]

We are working trying to add some offline support to our app atm. Currently using https://github.com/remix-pwa/remix-pwa, and looks good

One issue that we found is with some routes that contain actions, our intention is if the user is offline save a draft of the from in localStorage.

the migration to this logic feels like breaking all the remix logic, moving a lot of logic to the route render of add custom loginc inside of our SW.

Currently thinking if could be posible to have this can of api:

// route file

export action = ({ context, request })=>{
   // server action code
}

export loader = ({ context, request })=>{
   // server loader code
}

export actionSW = ({ context, request, cache?})=>{
   // code to be executed on sw as interceptor of this same route action
}

export loaderSW = ({ context, request, cache?})=>{
   // code to be executed on sw as interceptor of this same route action
}

this would expect to have the same code splitting that we have now with server-client but adding the sw to the equation
would love to hear thoughts around this

[ShafSpecs]

You don't need to use the same service worker, you can create a new service worker and when loading it in your entry.client file, give it another scope so that you don't overwrite another service worker:

loadServiceWorker({
  url: '/entry.worker.js', // can't remember the property name for this one
  scope: '/'
})
loadServiceWorker({
  url: '/my-route.worker.js,
  scope: '/path/to/route'
})

At /path/to/route, the second one would be in control, and when in other routes, the first one would take control. You can have multiple service workers at various degrees if you want to isolate implementations a bit. But the rule with multiple SW is no two service worker can be active at the same time 👍. Happy building!

Check out scopes here

[ShafSpecs]

About an implementation to have a service worker action and a service worker loader, that would both execute between the client and server sounds fascinating.

So, you got:
client -> POST -> actionSW (perform some mutilation) -> POST -> action
loader -> GET -> loaderSW (more mutilations) -> GET -> client

You can still achieve this with service workers though. Would be nice to see if the Remix team got anymore in-house support for PWAs tho

[Sarabadu]

We experimented a bit having a build compilation dedicated for the service worker, we can take the workerAction and he workerLoader in a similar way as proposed here for the client #2853

const action = async ({request, params, context})=> {
   // servercode
   
   persistToDB(request)
   
   return json({
   data: "persisted"
   })
}

const workerAction = async ({request, params, context)=> {
   const {draft} = params
   
   if (draft) {
      persistToIndexDb(request)
      return json({
        data: "draft Saved"
      })
   }
   
   return new NetworkOnly({plugin:[backgroundSync]}).handle(request)

}

const loader= ({request}) =>{
   
   return json({
     data: getDatafromDB(request)
   })
} 

const workerLoader= ({request}) =>{

   const serverData = await new NetworkFirst().handle(request)
   const localData = await getXfromIndexDB()
   
   return json({
     data: merge(serverData,localData)
   })
} 

[ShafSpecs]

@Sarabadu Great! Just for clarification though, which is is in the Service Worker and which isn't? Or none are in the service worker?

This looks very interesting and I would like to see the flow of how this would work. I could then figure out something to make it actually work, remix-pwa is undergoing some inner API changes at the moment and bringing this on board would be a good idea

[Sarabadu]

workerAction and workerLoader would be function compiled and bundled inside the service worker

our current POC is having a esbuild script that manage that similar as remix does with server and client code

inside the entry.worker i imagine something like this:

// this is a virtual module that is replaced by the build process containing all the routes manifest
import * as build from '@remix/build/worker';

self.addEventListener('fetch', event => {
  const {request} = event
 
  const handler = build.getHandler(request) // the build has the entire route manifest and each route has their own "module" with the `worker*` functions
  
    if(handler){
       event.respondWith(handler(event.request.clone())) 
    }

  event.respondWith(fetch(event.request.clone()));
});

maybe we can add this into remix-pwa having a remix-pwa build command or maybe remix-sw build to run the bundler

as reference remix-pwa/remix-pwa#115