v3.20230918.0

What's Changed

• Implement simulators as Durable Objects by @mrbbot in #656.
• Allow GET requests with bodies in entry worker by @mrbbot in #677
• Remove unused handleQueue() function from entry worker by @mrbbot in #678
• Allow magic proxy to be used from Jest by @mrbbot in #683
• Only show all accessible hosts initially by @jspspike in #686
• Improved source map handling and support for VSCode's debugger by @mrbbot in #681
• Allow custom workerd binary via MINIFLARE_WORKERD_PATH variable by @mrbbot in #682
• 🎨 Pretty validation errors by @mrbbot in #674
• Bump miniflare version to 20230918 by @jspspike in #688

Full Changelog: v3.20230904.0...v3.20230918.0

---

This release changes the persistence directory layout. KV namespaces, R2 buckets and D1 databases will automatically be migrated to the new layout automatically. Caches will not be migrated.

As an example, specifying the following options...

new Miniflare({
  kvPersist: "./data/kv",
  kvNamespaces: ["NAMESPACE"],
  r2Persist: "./data/r2",
  r2Buckets: ["BUCKET"],
  d1Persist: "./data/d1",
  d1Databases: ["DATABASE"],
  cachePersist: "./data/cache"
});

...would previously persist data like...

data
├── cache
│   └── default
│       ├── blobs
│       │   └── cb439464a0136ae87dda5b16e5d5c2e25c1aadd9971defa989b5a3de8c16e7da00004bf38fc72ed2
│       └── db.sqlite
├── d1
│   └── DATABASE
│       └── db.sqlite
├── kv
│   └── NAMESPACE
│       ├── blobs
│       │   └── 6ae43bc9d5630ba36dbd152d458265eab16472e2837c6395f002cd074879ff1500004bf38ca477b4
│       └── db.sqlite
└── r2
    └── BUCKET
        ├── blobs
        │   └── c3ba545e38e76c32a9534e9406a2899d997993f0d75efc75f2507acf08735cac00004bf38da2e625
        └── db.sqlite

...but will now persist data like...

data
├── cache
│   ├── default
│   │   └── blobs
│   │       └── cb439464a0136ae87dda5b16e5d5c2e25c1aadd9971defa989b5a3de8c16e7da00004bf38fc72ed2
│   └── miniflare-CacheObject
│       └── 9f458c07675338a7426a7b81ac4fb1baf92d034efbcaaf4336379640ed744ded.sqlite
├── d1
│   └── miniflare-D1DatabaseObject
│       └── e5f2187c86795ea6326c373904215941fcc05bb3d3eb8bf40b08a5573435c797.sqlite
├── kv
│   ├── NAMESPACE
│   │   └── blobs
│   │       └── 6ae43bc9d5630ba36dbd152d458265eab16472e2837c6395f002cd074879ff1500004bf38ca477b4
│   └── miniflare-KVNamespaceObject
│       └── c5143190acbf456f17b8db87ca6e6d0c672fd61585313adee82937a956e44b8c.sqlite
└── r2
    ├── BUCKET
    │   └── blobs
    │       └── c3ba545e38e76c32a9534e9406a2899d997993f0d75efc75f2507acf08735cac00004bf38da2e625
    └── miniflare-R2BucketObject
        └── 2ba11df92c037da2e2f63b12b8924678770664954fa3deab29e63c3d63bbfb7e.sqlite

Note db.sqlites are no longer stored under the namespace directory, but flat in miniflare-*Object directories. The file names in this directory correspond to the IDs of each namespaces' Durable Object.

As of Miniflare v3, the persistence layout is no longer a public interface, and is subject to change at any time. As long as you're accessing persisted data via the Miniflare API, data will be persisted across minor versions. If you have scripts that depend on the previous persistence layout directly, we recommend using the Miniflare API to manipulate your data instead.

import { Miniflare, Response } from "miniflare";

const mf = new Miniflare({
  script: "",
  modules: true,
  kvPersist: "./data/kv",
  kvNamespaces: ["NAMESPACE"],
  r2Persist: "./data/r2",
  r2Buckets: ["BUCKET"],
  d1Persist: "./data/d1",
  d1Databases: ["DATABASE"],
  cachePersist: "./data/cache"
});

// Instance of https://workers-types.pages.dev/#KVNamespace
const kvNamespace = await mf.getKVNamespace("NAMESPACE");
await kvNamespace.put("key", "value");

// Instance of https://workers-types.pages.dev/#R2Bucket
const r2Bucket = await mf.getR2Bucket("BUCKET");
await r2Bucket.put("key", "value");

// Instance of https://workers-types.pages.dev/#D1Database
const d1Database = await mf.getD1Database("DATABASE");
await d1Database.exec("CREATE TABLE entries (key TEXT PRIMARY KEY, value TEXT);");
await d1Database.prepare("INSERT INTO entries (key, value) VALUES (?1, ?2)").bind("a", "1").run();

// Instance of https://workers-types.pages.dev/#CacheStorage
const caches = await mf.getCaches();
const response = new Response("body", { headers: { "Cache-Control": "max-age=3600" }});
await caches.default.put("http://localhost/key", response);

If you need to access databases directly, you can generate the miniflare-*Object/<id>.sqlite <id> using the following function:

import crypto from "node:crypto";

function durableObjectNamespaceIdFromName(uniqueKey, name) {
  const key = crypto.createHash("sha256").update(uniqueKey).digest();
  const nameHmac = crypto.createHmac("sha256", key).update(name).digest().subarray(0, 16);
  const hmac = crypto.createHmac("sha256", key).update(nameHmac).digest().subarray(0, 16);
  return Buffer.concat([nameHmac, hmac]).toString("hex");
}

...where uniqueKey is one of miniflare-{KVNamespace,R2Bucket,D1Database,Cache}Object, and name is the ID of your namespace/bucket/database/cache. Note this is very likely to break again in future versions, so please use the Miniflare API if you can.