Community Experiments for v3 Adapters

[dillonkearns]

I now have the v3 release candidate ready with a simplified adapter interface, which is documented here: https://elm-pages-v3.netlify.app/docs/adapters/.

There are a lot of possible frameworks and hosting platforms that we could build adapters for. Let's use this thread to share progress, experiments, and feedback. In particular:

• If you were able to get an adapter script working, please share! Ideally a GitHub link to a public repo plus a running link, or a gist if it is provide source code would work, too.
• If you would like to try building an adapter but need help, this would be a good place to ask questions and collect ideas
• If you run into a roadblock and don't think the current v3 adapter design will support your use case, it would be helpful to hear what you've tried, what your specific use case is, and where you got stuck

A useful reference for building adapters is SvelteKit: https://kit.svelte.dev/docs/adapters. I got the idea of an adapter from SvelteKit initially, and they have some useful docs and implementations there.

[blaix]

This is awsome.

I have elm-pages-3 running on express.js on fly.io: https://elm-pages-v3-express.fly.dev/

Source code is here: https://github.com/blaix/elm-pages-v3-express

The adapter just copies the elm pages renderer and a basic express server into dist-server and node runs the server from there. The fly deployment only required a couple tweaks to their default dockerfile. Details are in the README.

[dillonkearns]

Yes, I should probably write up a section on that in the docs (maybe even a blurb about it in the adapter page).

In the Netlify adapter, what it means is that it uses Netlify's On-Demand Builders. You can see a live example here: https://elm-pages-pokedex.netlify.app/10. Notice that you'll get fresh timestamps if you visit a page that hasn't been hit within the cache window, but once you hit it it will keep the timestamp (the timestamp is just for debugging/showing the feature in action, not a real user-facing feature really).

Here's the code for that Route: https://github.com/dillonkearns/elm-pages-v3-beta/blob/8f0c334b22ab0e56dd4ea96e31a8e0c1d50244d1/examples/pokedex/app/Route/PokedexNumber_.elm#L32-L39

Conceptually, the prerender-with-fallback behavior is letting you do build-time pre-renders of some pages (eager pages), and lazily build other pages on the server and then cache them. Here are the docs for Netlify's On-Demand Builders: https://docs.netlify.com/configure-builds/on-demand-builders/. The adapter could decide what that means for different frameworks/hosts, though. Vercel has similar behavior but it is implemented through cache-control headers on the server, and with cache that is busted after a certain amount of time passes, whereas Netlify's implementation busts cache on each new deploy.

Hope that helps! Happy to answer more questions. And no pressure if you don't want to experiment with that, it's not necessarily important, just throwing that out there!

[dillonkearns]

The middleware part looks great by the way!

[blaix]

Thanks! Things got busy so I haven't been able to get back to this yet, but definitely plan to. I have a project that I'd like to move to elm-pages and plan to start dog-fooding this as soon as possible.

[dillonkearns]

That would be fantastic!

[blaix]

Since I moved away from elm-pages myself, @shahnhogan has graciously agreed to be the new owner of this repo 😊

https://github.com/shahnhogan/elm-pages-starter-express

I transferred the repo so links, forks, and clones should be redirected appropriately.

[dillonkearns]

Another adapter: @shahnhogan created a Fastify adatper with a starter repo. https://github.com/shahnhogan/elm-pages-starter-fastify.

[pyrex41]

I threw together a replit based on the fastify starter that makes getting a deployment up about as easy as it can get:
https://replit.com/@pyrex41/elm-pages-starter-fastify?v=1

I didn't remove anything, just added a script to get the right lamdera binary and a short section near the top of the Readme.

[nsadeh]

I have an adapter using Koa/Node.js. The goal of this adapter is to be able to run it anywhere Node can run, in my case I use DigitalOcean for some projects and Google Cloud Run for others. I can install Node on the former and just run node dist-server/server.mjs. On the latter, I can package it in a Node.js Docker container and publish to the GCP container registry. In both cases, I need both the dist-server and dist folders produced when npm run build is run.

you will need to install both koa and koa-static in your production environment.

This code was heavily inspired by https://github.com/blaix/elm-pages-starter-express/tree/master/adapters/express. Thanks, @blaix. It would have taken me about 3x as long to figure it out without this repo.

Under the main folder of the project, I have an adapters/koa folder with the following .mjs scripts:

adapter.mjs

import * as fs from "fs";

export default async function run({
    renderFunctionFilePath,
    // routePatterns,
    // apiRoutePatterns,
}) {
    console.log("Running elm pages express adapter");
    ensureDirSync("dist-server");
    fs.copyFileSync(renderFunctionFilePath, "./dist-server/elm-pages.mjs");
    fs.copyFileSync("./adapters/koa/server.mjs", "./dist-server/server.mjs");
    fs.copyFileSync(
        "./adapters/koa/middleware.mjs",
        "./dist-server/middleware.mjs"
    );
}

function ensureDirSync(dirpath) {
    try {
        fs.mkdirSync(dirpath, { recursive: true });
    } catch (err) {
        if (err.code !== "EEXIST") throw err;
    }
}

middlware.mjs

import * as elmPages from "./elm-pages.mjs";

export default async (ctx, next) => {
    try {
        const { headers, statusCode, body, kind } = await elmPages.render(reqToElmPagesJson(ctx.request))
        ctx.response.status = statusCode
        for (const key in headers) {
            ctx.set(key, headers[key]);
        }
        if (kind === "bytes") {
            ctx.response.body = Buffer.from(body)
        } else {
            ctx.response.body = body
        }
    } catch (error) {
        console.log("Encountered error serving request", ctx.request, error)
        ctx.response.status = 500
        // this can be styled, or taken out of a nice .html error file
        ctx.response.body = "<body><h1>Error</h1><pre>Unexpected Error</pre></body>"
    } finally {
        next()
    }
}

const reqToElmPagesJson = (req) => {
    const url = `${req.protocol}://${req.host}${req.originalUrl}`;
    return {
        requestTime: Math.round(new Date().getTime()),
        method: req.method,
        headers: req.headers,
        rawUrl: url,
        body: req.body,
        multiPartFormData: null,
    };
};

server.mjs

import Koa from 'koa';
import serve from 'koa-static';
import elmPagesMiddleware from './middleware.mjs';

const app = new Koa()
const port = 8000;

// Serve static files from 'dist' directory
app.use(serve('dist'));

// Use your custom middleware
app.use(elmPagesMiddleware);

// Start the server
app.listen(port, () => {
    console.log(`Listening on port ${port}`);
});

I edited my elm-pages.config.mjs file as follows:

import { defineConfig } from "vite";
import adapter from "./adapters/koa/adapter.mjs";

export default {
  vite: defineConfig({}),
  adapter,
  headTagsTemplate(context) {
    return `
<link rel="stylesheet" href="/style.css" />
<meta name="generator" content="elm-pages v${context.cliVersion}" />
`;
  },
  preloadTagForFile(file) {
    // add preload directives for JS assets and font assets, etc., skip for CSS files
    // this function will be called with each file that is procesed by Vite, including any files in your headTagsTemplate in your config
    return !file.endsWith(".css");
  },
};

To run this server, run npm run build to get the build output. Then, simply run node dist-server/server.mjs.

[shahnhogan]

@nsadeh I created an elm-pages-starter-koa template based on your code.

https://github.com/shahnhogan/elm-pages-starter-koa

[shahnhogan]

I've created a Hapi adapter based on the @blaix express adapter.

https://github.com/shahnhogan/elm-pages-starter-hapi

[adamdicarlo0]

Here are the important bits of the AWS Lambda + APIGatewayV2 adapter we're using at my work:

https://gist.github.com/adamdicarlo0/221e839050a3e8cef51f1849e7af71a9

I've included notes in a GitHub comment.

I don't have a starter repo put together for it, at least not yet, but feel free to ask questions here!

[adamdicarlo0]

This gist now also includes a work-around to use a static route as a CloudFront custom error response page (minus the AWS/terraform configuration needed, which is mostly to tell CloudFront to serve /404.html with response code 404 for both 403 and 404 errors).

[MattiaVerticchio]

Hi everyone,

I wanted to share an elm-pages adapter I made for Bun‘s fast HTTP server.

I created this adapter initially for my side project, which I ran on the Oracle Cloud free tier behind an Nginx reverse proxy.

Key features:

• Uses a script (compress.ts) run at compile time to pre-compress static files.
• The server (server.ts) loads these pre-compressed files on boot.
• The compression script prioritizes smaller file sizes, discarding compressed versions if they are larger than the original.
• Dynamic content is compressed on-the-fly using Bun’s built-in gzip functionality.

Compression script

// compress.ts

import File from "fs"
import Path from "path"
import { BunFile } from "bun"
import Zlib from "zlib"
import Zstd from '@mongodb-js/zstd';

// Get all the compiled files

const staticDir: string =
  process.argv[2] || 'dist'

const entries: File.Dirent[] =
  File.readdirSync(staticDir, { withFileTypes: true, recursive: true })

const files: string[] = entries
  .filter(x => x.isFile())
  .map(x => Path.join(x.parentPath, x.name))


const extensions: string[] = [
  '.html', '.css', '.js', '.json', '.xml', '.svg',
  '.txt', '.md', '.map', '.woff2', '.ttf', '.dat'
]

const compressed: string[] =
  [".br", ".gz", ".zst", ".deflate"]

const staticFiles: string[] = []
const compressedFiles: Record<string, {
  path: string,
  type: string,
  encoding: ("br" | "gzip" | "deflate" | "zstd")[]
}> = {}

for (const path of files) {

  const file: BunFile = Bun.file(path)
  const extension: string = Path.extname(path)
  const url: string = path.slice(staticDir.length)

  // Skip file if already compressed

  if (compressed.includes(extension)) {
      console.log(`${path} already compressed`)
      staticFiles.push(url)
      continue
  }

  // Skip file if unsupported

  if (!extensions.includes(extension)) {
      console.log(`${path} has unsupported extension`)
      staticFiles.push(url)
      continue
  }

  const content: ArrayBuffer = await file.arrayBuffer()
  const originalSize: number = content.byteLength

  // Priority is used to serve the smallest compressed content,
  // if supported by the client: [ smallest, ..., biggest ]

  let priority: {
      file: string,
      header: "br" | "gzip" | "deflate" | "zstd",
      size: number
  }[] = []

  // Brotli generic compression

  let brotli: Buffer = Zlib.brotliCompressSync(content, {
      params: {
          [Zlib.constants.BROTLI_PARAM_MODE]:
              Zlib.constants.BROTLI_MODE_GENERIC,
          [Zlib.constants.BROTLI_PARAM_QUALITY]:
              Zlib.constants.BROTLI_MAX_QUALITY
      },
  })

  // Brotli text compression, keep it if smaller than generic

  const brotliText: Buffer = Zlib.brotliCompressSync(content, {
      params: {
          [Zlib.constants.BROTLI_PARAM_MODE]:
              Zlib.constants.BROTLI_MODE_TEXT,
          [Zlib.constants.BROTLI_PARAM_QUALITY]:
              Zlib.constants.BROTLI_MAX_QUALITY
      },
  })

  if (brotliText.byteLength < brotli.byteLength) brotli = brotliText

  // Brotli font compression, keep it if smaller than generic and text

  const brotliFont: Buffer = Zlib.brotliCompressSync(content, {
      params: {
          [Zlib.constants.BROTLI_PARAM_MODE]:
              Zlib.constants.BROTLI_MODE_FONT,
          [Zlib.constants.BROTLI_PARAM_QUALITY]:
              Zlib.constants.BROTLI_MAX_QUALITY
      },
  })

  if (brotliFont.byteLength < brotli.byteLength) brotli = brotliFont

  // Keep the smallest file if smaller than uncompressed

  if (brotli.byteLength < originalSize) {
      const fileName = path + ".br";
      await Bun.write(fileName, brotli)
      priority.push({
          file: fileName,
          header: "br",
          size: brotli.byteLength
      })
  }

  console.log(path, brotli.byteLength)

  // Zstd compression

  const buffer: Buffer = Buffer.from(content)
  const zStandard = await Zstd.compress(buffer, 19)

  // Keep the file if smaller than uncompressed

  if (zStandard.byteLength < originalSize) {
      const fileName = path + ".zst";
      await Bun.write(fileName, zStandard.buffer)
      priority.push({
          file: fileName,
          header: "zstd",
          size: zStandard.byteLength
      })
  }

  // Gzip compression

  const gzip: Uint8Array = Bun.gzipSync(content, { level: 9 })

  // Keep the file if smaller than uncompressed

  if (gzip.byteLength < originalSize) {
      const fileName = path + ".gz";
      await Bun.write(fileName, gzip)
      priority.push({
          file: fileName,
          header: "gzip",
          size: gzip.byteLength
      })
  }

  // Deflate compression

  const deflate: Uint8Array = Bun.deflateSync(content, { level: 9 })

  // Keep the file if smaller than uncompressed

  if (deflate.byteLength < originalSize) {
      const fileName = path + ".deflate"
      await Bun.write(fileName, deflate)
      priority.push({
          file: fileName,
          header: "deflate",
          size: deflate.byteLength
      })
  }

  if (priority.length === 0) {
      staticFiles.push(url)
  } else {
      const sorted = priority
          .sort((a, b) => a.size - b.size)
          .map(({ header }) => header);

      compressedFiles[url] = {
          path: path,
          type: file.type,
          encoding: sorted
      }
  }

}

Bun.write("dist-server/static.json", JSON.stringify(staticFiles))

const urls = Object.keys(compressedFiles)

for (const url of urls) {
  if (url.endsWith("/index.html")) {
      const compressedFile = compressedFiles[url]

      const withSlash: string = url.slice(0, -10)
      compressedFiles[withSlash] = compressedFile

      const withoutSlash: string = url.slice(0, -11)
      if (!(withoutSlash in compressedFiles)) {
          compressedFiles[withoutSlash] = compressedFile
      }
  }
}

Bun.write("dist-server/compressed.json", JSON.stringify(compressedFiles))

Server

// server.ts

import * as elmPages from "./elm-pages.mjs";
import * as util from "util";
import { BunFile } from "bun";


// Load static files
// https://bun.sh/docs/api/http#static-routes

const staticDir = "dist"
const serverDir = "dist-server"

const staticFiles: string[] =
    await Bun.file(`${serverDir}/static.json`).json()

const compressedFiles: Record<string, {
    path: string,
    type: string,
    encoding: ("br" | "gzip" | "deflate" | "zstd")[]
}> = await Bun.file(`${serverDir}/compressed.json`).json()

const staticRoutes: Record<`/${string}`, Response> = {};

for (const url of staticFiles) {
    const file: BunFile =
        Bun.file(staticDir + url)

    const headers = {
        "Content-Type": file.type,
    };

    // Add cache headers for js, css, and woff2 files

    if (url.endsWith('.js') || url.endsWith('.css') || url.endsWith('.woff2')) {
        headers["Cache-Control"] = "public, max-age=31536000, immutable";
    }

    const bytes: Uint8Array =
        await file.bytes()

    staticRoutes[url] = new Response(bytes, { headers });
}

// Server
// https://bun.sh/docs/api/http#bun-serve

Bun.serve({
    static: staticRoutes,
    port: 3000,

    async fetch(req: Request): Promise<Response> {
        try {

            const { pathname }: URL = new URL(req.url);

            // Get accepted encodings            

            const encodings: string =
                req.headers.get("accept-encoding") || "";

            // Serve static file if matches

            if (pathname in compressedFiles) {

                const compressedFile = compressedFiles[pathname]

                for (const encoding of compressedFile.encoding) {
                    if (encodings.includes(encoding)) {
                        const extension: string = {
                            "br": ".br",
                            "gzip": ".gz",
                            "deflate": ".deflate",
                            "zstd": ".zst",
                        }[encoding]

                        const headers = {
                            "Content-Type": compressedFile.type,
                            "Content-Encoding": encoding
                        }

                        // Add cache forever headers for js, css, and woff2 files

                        if (pathname.endsWith('.js')
                            || pathname.endsWith('.css')
                            || pathname.endsWith('.woff2')
                        ) {
                            headers["Cache-Control"] = "public, max-age=31536000, immutable";
                        }

                        return new Response(
                            Bun.file(compressedFile.path + extension), { headers })
                    }
                }

                return new Response(
                    Bun.file(compressedFile.path), {
                    headers: { "Content-Type": compressedFile.type }
                })

            }

            // Request to elm-pages request

            let reqBody: string | null = null

            if (req.body) {

                const isFormData: boolean =
                    req.headers['content-type'] === 'application/x-www-form-urlencoded'

                if (isFormData) reqBody = toFormData(req.body)
                else reqBody = JSON.stringify(req.body)

            }

            const elmPagesReq = {
                requestTime: Math.round(new Date().getTime()),
                method: req.method,
                headers: req.headers,
                rawUrl: req.url,
                body: reqBody,
                multiPartFormData: null,
            };

            // Render in elm-pages

            const renderResult =
                await elmPages.render(elmPagesReq);

            let { body, headers, kind, statusCode } = renderResult;

            if (kind === "bytes") {
                headers["Content-Type"] = "application/octet-stream"
                body = Buffer.from(body)

            } else if (kind == "api-response") {

            } else {
                headers["Content-Type"] = "text/html"
            }

            if (encodings.includes("gzip")) {
                body = Bun.gzipSync(body)
                headers["Content-Encoding"] = "gzip"
            }

            return new Response(body, { headers, status: statusCode })

        } catch (error) {

            if (process.env.NODE_ENV === 'development') {
                console.log(util.inspect(error, { depth: null, colors: true }));
            } else {
                console.error(error);
            }

            return new Response(
                "<body><h1>Error</h1><pre>Unexpected Error</pre></body>", {
                status: 500,
                headers: { "Content-Type": "text/html", }
            })

        }
    }
})


function toFormData(body): string | null {
    if (typeof body === 'string') return body;

    const formData: URLSearchParams =
        new URLSearchParams();

    for (const [key, value] of Object.entries(body)) {
        formData.append(key, value);
    }

    const result = formData.toString()
    return result || null
}

Unfortunately, Oracle shut down my free-tier instance without notice. I migrated the site back to Netlify, but I'm not really satisfied with the performance. As you can see from the PageSpeed results, there's a significant server response time delay that pushes First Contentful Paint (FCP), Largest Contentful Paint (LCP), and Time To First Byte (TTFB) above 2 seconds, which feels too slow.

This experience made me wonder about alternatives. Has anyone in the community tried building or successfully implemented an elm-pages adapter for Cloudflare Pages + Workers?

The free tier is quite generous, static requests are unlimited, and the edge compute model seems like it could be a great fit for elm-pages performance-wise. I'd be very interested to hear if anyone has explored this path.

Thanks!