Add WASMFS/OPFS filesystem backend support

[Copilot]

Adds OPFS (Origin Private File System) via Emscripten's WASMFS as a new persistent storage backend for browser environments. OPFS provides improved performance and quota management compared to IDBFS.

Build System

• Added Release-opfs and Debug-opfs targets to build.sh
• Flags: -sWASMFS -sWASM_BIGINT with asyncify enabled
• Excludes legacy FS libraries (IDBFS/NODEFS) when building WASMFS variants
• Uses async HTTP transport (same as async builds)

Implementation

• emscriptenbuild/post-opfs.js: Glue code based on post-async.js with WASMFS configuration
• test-browser-opfs/: Test suite covering clone, commit, push, and file operations
• CI: New opfs job in GitHub Actions workflow

Usage

const lgMod = await import('./lg2_opfs.js');
const lg = await lgMod.default();
const FS = lg.FS;

const workingDir = '/working';
FS.mkdir(workingDir);
FS.mount(FS.filesystems.OPFS, { root: '.' }, workingDir);
FS.chdir(workingDir);

await lg.callMain(['clone', 'https://github.com/user/repo.git', 'repo']);

Browser Support

Chrome 86+, Edge 86+, Firefox 111+, Safari 15.2+

Original prompt

Goal

Enable and test Emscripten's WASMFS (with OPFS) as a filesystem backend for wasm-git, providing the same (or better) test coverage as currently exists for MEMFS/IDBFS/NODEFS.

Closes #108

Background

• wasm-git presently supports MEMFS, IDBFS, and NODEFS, with integration in test suites for both synchronous and asynchronous modes.
• Emscripten supports WASMFS, a new filesystem backend, and the OPFS (Origin Private File System) API via WASMFS.
• OPFS provides improved performance and quota vs IDBFS. WASMFS enables using OPFS, and potentially other backends, with a unified interface.

Current Implementation Details

Build System (emscriptenbuild/build.sh)

The current build exports these filesystems:

EXPORTED_RUNTIME_METHODS=['FS','MEMFS','IDBFS','NODEFS','callMain','HEAPU8']

Build targets exist for:

• Debug / Release - synchronous versions
• Debug-async / Release-async - async versions with Asyncify

Test Structure

• test/ - Node.js tests
• test-browser/ - Browser tests (sync version with web worker)
• test-browser-async/ - Browser tests (async version)

Required Changes

1. Build Configuration Changes (emscriptenbuild/build.sh)

Add new build targets for WASMFS/OPFS:

# Add WASMFS/OPFS build targets
if [ "$1" == "Release-opfs" ]; then
    BUILD_TYPE=Release
    cp ../libgit2patchedfiles/src/transports/emscriptenhttp-async.c ../libgit2/src/libgit2/transports/emscriptenhttp.c
    
    # WASMFS requires these flags
    EXTRA_CMAKE_C_FLAGS="-O3 $ASYNCIFY_FLAGS -sWASMFS -sOPFS"
    POST_JS="--post-js $(pwd)/post-opfs.js"
    export LG2_OUTPUT_NAME=lg2_opfs
elif [ "$1" == "Debug-opfs" ]; then
    BUILD_TYPE=Debug
    cp ../libgit2patchedfiles/src/transports/emscriptenhttp-async.c ../libgit2/src/libgit2/transports/emscriptenhttp.c
    
    EXTRA_CMAKE_C_FLAGS="$ASYNCIFY_FLAGS -sWASMFS -sOPFS"
    POST_JS="--post-js $(pwd)/post-opfs.js"
    export LG2_OUTPUT_NAME=lg2_opfs
fi

Key Emscripten flags needed:

• -sWASMFS - Enable the new WASMFS implementation
• -sOPFS - Enable OPFS backend for WASMFS
• May need -pthread if synchronous file access handles are required

2. JavaScript Glue Code (emscriptenbuild/post-opfs.js)

Create a new file for OPFS-specific initialization, similar to post-async.js:

/**
 * OPFS filesystem support for wasm-git
 * Uses Emscripten's WASMFS with OPFS backend
 */

const emscriptenhttpconnections = {};
let httpConnectionNo = 0;

const chmod = FS.chmod;
    
FS.chmod = function(path, mode, dontFollow) { 
    if (mode === 0o100000 > 0) {
        return chmod(path, mode, dontFollow);
    } else {
        return 0;
    }
};

if (ENVIRONMENT_IS_WEB) {
    Module.origCallMain = Module.callMain;
    Module.callMain = async (args) => {
        await Module.origCallMain(args);
        if (typeof Asyncify === 'object' && Asyncify.currData) {            
            await Asyncify.whenDone();
        }
    };

    // OPFS mount helper
    Module.mountOPFS = async function(mountPoint) {
        try {
            FS.mkdir(mountPoint);
        } catch (e) {
            // Directory may already exist
        }
        
        // WASMFS with OPFS backend handles mounting
        FS.mount(FS.filesystems.OPFS, {}, mountPoint);
        
        return mountPoint;
    };

    // Include async HTTP transport functions (similar to post-async.js)
    Object.assign(Module, {
        emscriptenhttpconnect: async function(url, buffersize, method, headers) {
            // ... (copy from post-async.js)
        },
        // ... other HTTP methods
    });
}

3. Test Directory (test-browser-opfs/)

Create a new test directory mirroring test-browser-async/ structure:

• test-browser-opfs/test.spec.js - Test file using OPFS
• test-browser-opfs/lg2_opfs.js - Symlink to build output
• test-browser-opfs/lg2_opfs.wasm - Symlink to build output

The test should cover the same operations as existing tests:

• Create filesystem and mount OPFS
• Create working directory
• Write .gitconfig file
• Initialize a repository
• Add and commit files
• Clone from remote
• Push to remote
• Verify data persists across sessions

4. Karma/Test Runner Configuration

Add new config file karma.conf-opfs.js or extend existing configuration to support OPFS tests.

Add npm script in package.json:

"test-browser-opfs": "karma start karma.conf-opfs.js"

5. GitHub Actions CI (.github/workflows/main.yml)

Add OPFS build and test steps to the CI pipeline.

6. Documentation Updates

Update README.md to document:

• OPFS build target usage
• Browser compatibility requirements (Chrome 86+, Edge 86+, Firefox 111+, Safari 15.2+)
• CORS header requirements for SharedArrayBuffer if needed
• Example usage code

Technical Considerations

Threading/CORS Requirements

OPFS's synchronous access handles may require SharedArrayBuffer:

• Cross-Origin-Opener-Policy: same-origin header
• `Cross-Origin-Embedder-Policy: requir...

This pull request was created from Copilot chat.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.