Use Origin private file system

[marcus-pousette]

What is the possibility of using Origin private file system instead of IndexedDB?

https://developer.mozilla.org/en-US/docs/Web/API/File_System_API/Origin_private_file_system

[vweevers]

Should be possible. There's an older issue here about making a wasm build of LevelDB: Level/community#63. It didn't go anywhere because of fs limitations at the time, but OPFS changes that. I think everyone would be happy to get rid of IndexedDB.

[marcus-pousette]

okey!
When I look at it, the challenging things with OPFS is that there are no prefix, query capabilities for finding all files with certain prefix, which seems to be necessary when trying to be compatible with the Level interface.

[vweevers]

Yeah, OPFS alone isn't enough, just provides raw storage. Using any filesystem as a key-value database (where keys are files) isn't gonna be performant. It needs a specialized data structure like LSM trees, which is what LevelDB provides.

[MentalGear]

OPFS would be great for stable reliable in browser usage. As far as I know browser-level uses indexdb which - unless this library has special patches - is unreliable on mobile. Mobile browsers, especially safari, more or less randomly delete indexdb stores (not to speak of the recent Apple WebApp elimination disaster).

[guest271314]

What I'm trying to do is read the levelbd created when files are written to origin private system on Chromium.

I keep getting dtabase is not open error.

import { Level } from "./node_modules/level";
const db = new Level("./000003.log");

for await (const [key, value] of db.keys()) {
 console.log(key, value);
}

I can get the file names using the code below.

  async function parseChromeDefaultFileSystem(path) {
    var set = new Set([
      32, 45, 46, 47, 48, 49, 50, 51, 52, 53,
      54, 55, 56, 57, 58, 64, 65, 66, 67, 68,
      69, 70, 71, 72, 73, 74, 75, 76, 77, 78,
      79, 80, 81, 82, 83, 84, 85, 86, 87, 88,
      89, 90, 95, 97, 98, 99, 100, 101, 102, 103,
      104, 105, 106, 107, 108, 109, 110, 111,
      112, 113, 114, 115, 116, 117, 118, 119,
      120, 121, 122,
    ]);

    return fetch(
      path,
    ).then((r) => r.text()).then((text) => {
      let str = "";
      for (const char of text) {
        const code = char.codePointAt();
        if (set.has(code)) {
          str += char;
        }
      }
      console.log(str);
      
      const matches = [
        ...new Set(
          str.replace(/./g, (s) => set.has(s.codePointAt()) ? s : " ").match(
            /00000\d+[A-Za-z-_.0-9\s]+\.crswap/g,
          ),
        ),
      ].map((s) => s.replace(/00000[\d\s]+|\.crswap/g, ""));

      return matches;
      
    }).catch(console.error);
  }

var paths = await parseChromeDefaultFileSystem("file:///home/user/.config/chromium/Default/File\ System/021/t/Paths/000003.log");
console.log(paths);

Now I need to associate the files stored in Default/File System/021/t/00 with the parent subdirectories

[guest271314]

@vweevers How would I go about parsing this
000003.log in the browser?

My best estimation from MANIFEST-000001 in Chromium configuration folder at Default/File System/021

�|¹Å"����leveldb.BytewiseComparator������

is that the contents of 000003.log are a level database.

[vweevers]

@guest271314 I'm not going to assist you in reverse engineering third-party databases. You can find such help on StackOverflow, or alternatively, you can dive into the Chromium source code. Good luck!

[guest271314]

@vweevers Thanks for your time. I'll figure it out one way or another. Good luck!

[guest271314]

@vweevers I figured it out. Thanks, again.

async function parseChromeDefaultFileSystem(path) {
  try {
    const set = new Set([
      32, 45, 46, 47, 48, 49, 50, 51, 52, 53,
      54, 55, 56, 57, 58, 64, 65, 66, 67, 68,
      69, 70, 71, 72, 73, 74, 75, 76, 77, 78,
      79, 80, 81, 82, 83, 84, 85, 86, 87, 88,
      89, 90, 95, 97, 98, 99, 100, 101, 102, 
      103, 104, 105, 106, 107, 108, 109, 110,
      111, 112, 113, 114, 115, 116, 117, 118,
      119, 120, 121, 122,
    ]);
    const request = await fetch(path);
    const text = (await request.text()).replace(/./g, (s) => set.has(s.codePointAt()) ? s : "");
    const files = [
      ...new Set(
        text.match(
          /00000\d+[A-Za-z-_.0-9\s]+\.crswap/g,
        ),
      ),
    ].map((s) => {
      const dir = [...new Set(text.slice(0, text.indexOf(s)).match(/(?<=[@\s]|CHILD_OF:0:)([\w-_])+(?=Ux)/g).map((d) =>
        d.split(/\d+|D140/)
      ))].flat().pop();
      const [key] = s.match(/00000[\d\s]+|\.crswap/g);
      return ({
        [key]: s.replace(/00000[\d\s]+|\.crswap/g, ""),
        dir
      })
    });
    return {
      name: files[0].dir,
      files
    }
  } catch (e) {
    console.error(e);
  }
}

let paths = await parseChromeDefaultFileSystem("file:///home/user/.config/chromium/Default/File\ System/021/t/Paths/000003.log");

console.log(JSON.stringify(paths,null,2));

{
  "name": "persistent-serviceworker",
  "files": [
    {
      "00000001": "README.md",
      "dir": "persistent-serviceworker"
    },
    {
      "00000003": "background.js",
      "dir": "chromium_extension_web_accessible_resources_iframe_message_event"
    },
    {
      "00000005": "index.html",
      "dir": "chromium_extension_web_accessible_resources_iframe_message_event"
    },
    {
      "00000007": "index.js",
      "dir": "chromium_extension_web_accessible_resources_iframe_message_event"
    },
    {
      "00000009": "manifest.json",
      "dir": "chromium_extension_web_accessible_resources_iframe_message_event"
    },
    {
      "00000011": "index.html",
      "dir": "docs"
    },
    {
      "00000013": "script.js",
      "dir": "docs"
    },
    {
      "00000015": "sw.js",
      "dir": "docs"
    },
    {
      "00000017": "index.html",
      "dir": "docs"
    },
    {
      "00000019": "script.js",
      "dir": "docs"
    },
    {
      "00000021": "sw.js",
      "dir": "docs"
    },
    {
      "00000023": "index.html",
      "dir": "message-event"
    },
    {
      "00000025": "script.js",
      "dir": "message-event"
    },
    {
      "00000027": "sw.js",
      "dir": "message-event"
    },
    {
      "00000029": "index.html",
      "dir": "readablestream-fetch-respondwith"
    },
    {
      "00000031": "script.js",
      "dir": "readablestream-fetch-respondwith"
    },
    {
      "00000033": "sw.js",
      "dir": "readablestream-fetch-respondwith"
    }
  ]
}