Skip to content

How to speedup writeFile? #115

@iliakan

Description

@iliakan

I have a code to read a real folder from File Access API into lightningFS.

The repo is about 800kb, has 300 files.

The code works very slowly, because of writeFile (~1.3 sec).

I've read that indexedDB throttles writeFile, is that so?
How to speedup?

Here's the code, it recursively reads all dirs/files and uses fs.promises.writeFile to write them to lightningFS.

This writeFile call is the main reason for the delay, even though the data is very small.

  let fs = new LightningFS('fs', {wipe: true});

  let relPath = [''];

  async function handle(dirHandle) {
    console.time(dirHandle.name);

    for await (const entry of dirHandle.values()) {
      if (entry.kind === "file") {
        const file = await entry.getFile();
        let data = new Uint8Array(await file.arrayBuffer());
        let filePath = [...relPath, file.name].join('/');
        await fs.promises.writeFile(filePath, data);
      }

      if (entry.kind === "directory") {
        const newHandle = await dirHandle.getDirectoryHandle( entry.name, { create: false } );
        relPath.push(entry.name);
        let dirPath = relPath.join('/');
        await fs.promises.mkdir(dirPath);
        await handle(newHandle);
        relPath.pop();
      }
    }
  }

P.S. Is there any other backend for lightningFS? I need a simple in-memory strorage.
It's quite ironic that lightningFS is so sluggish for a tiny test repo.
Maybe I'm doing something wrong?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions