fst-util-from-fs

fst utility to create trees from file systems

Contents

• What is this?
• Install
• Use
• API
  • fromFileSystem<T>([options])
• Utilities
  • getFileSystemEntries<T>(parent[, fs])
• Types
  • Awaitable<[T]>
  • Dirent
  • Extensions
  • FileSystemEntries
  • FileSystem
  • Filter
  • Filters
  • GetFileSystemEntries<[T]>
  • Handle<[T][, Result]>
  • Handles
  • IsDirectory
  • IsFile
  • IsSymbolicLink
  • List<[T]>
  • Options
  • ReaddirDirentOptions
  • ReaddirOptions
  • Readdir
  • Realpath
  • Sort
  • Stat
  • Stats
  • ToVisitKey<[K]>
  • VisitMap
• Syntax tree
• Contribute

What is this?

This package is a utility to create file system trees.

This utility uses file system adapters to recursively read a directory, and create a tree from its contents.

Install

This package is ESM only.

In Node.js (version 20+) with yarn:

yarn add @flex-development/fst-util-from-fs

See Git - Protocols | Yarn  for details regarding installing from Git.

In Deno with esm.sh:

import { fromFileSystem } from 'https://esm.sh/@flex-development/fst-util-from-fs'

In browsers with esm.sh:

<script type="module">
  import { fromFileSystem } from 'https://esm.sh/@flex-development/fst-util-from-fs'
</script>

Use

import type {
  File,
  AnyParent as Parent,
  Root
} from '@flex-development/fst'
import {
  fromFileSystem,
  type Dirent,
  type FileSystem,
  type Stats
} from '@flex-development/fst-util-from-fs'
import pathe from '@flex-development/pathe'
import { inspect } from '@flex-development/unist-util-inspect'
import { ok } from 'devlop'
import fs from 'node:fs'
import { size } from 'unist-util-size'

declare module '@flex-development/fst' {
  interface File {
    /**
     * The size of the file.
     */
    size?: bigint | number | undefined
  }
}

declare module '@flex-development/fst-util-from-fs' {
  interface Stats {
    /**
     * The size of the entry.
     */
    size: bigint | number
  }
}

/**
 * A glob pattern matching directories to search
 * and files to include in the tree.
 *
 * @const {string} pattern
 */
const pattern: string = 'src/**/**'

/**
 * The file system tree.
 *
 * @const {Root} tree
 */
const tree: Root = await fromFileSystem({
  extensions: '.mts',
  filters: {
    /**
     * @this {void}
     *
     * @param {string} path
     *  The path to the directory, relative to `tree.path`
     * @param {number | null | undefined} depth
     *  The current search depth
     * @param {Dirent} dirent
     *  The dirent representing the file system entry
     * @param {Parent} parent
     *  The parent node
     * @param {Root} tree
     *  The file system tree
     * @return {boolean}
     *  `true` if node for `path` should be added, `false` otherwise
     */
    directory(
      this: void,
      path: string,
      depth: number | null | undefined,
      dirent: Dirent,
      parent: Parent,
      tree: Root
    ): boolean {
      return pathe.matchesGlob(path, pattern, {
        cwd: tree.path,
        dot: false,
        ignore: ['**/__mocks__/**', '**/__snapshots__/**', '**/__tests__/**'],
        noglobstar: false
      })
    },

    /**
     * @this {void}
     *
     * @param {string} path
     *  The path to the file, relative to `tree.path`
     * @param {number | null | undefined} depth
     *  The current search depth
     * @param {Dirent} dirent
     *  The dirent representing the file system entry
     * @param {Parent} parent
     *  The parent node
     * @param {Root} tree
     *  The file system tree
     * @return {boolean}
     *  `true` if node for `path` should be added, `false` otherwise
     */
    file(
      this: void,
      path: string,
      depth: number | null | undefined,
      dirent: Dirent,
      parent: Parent,
      tree: Root
    ): boolean {
      return pathe.matchesGlob(path, pattern, {
        cwd: tree.path,
        dot: true,
        ignore: ['**/.DS_Store'],
        noglobstar: false
      })
    }
  },
  fs: fs.promises,
  handles: {
    /**
     * @async
     *
     * @this {void}
     *
     * @param {File} node
     *  The node representing the file
     * @param {Dirent} dirent
     *  The dirent representing the file
     * @param {Parent} parent
     *  The parent of `node`
     * @param {Root} tree
     *  The current file system tree
     * @param {Parent[]} ancestors
     *  The ancestors of `node`, with the last node being `parent`
     * @param {FileSystem} fs
     *  The file system API
     * @return {Promise<undefined>}
     */
    async file(
      this: void,
      node: File,
      dirent: Dirent,
      parent: Parent,
      tree: Root,
      ancestors: Parent[],
      fs: FileSystem
    ): Promise<undefined> {
      /**
       * The list of relative ancestor paths.
       *
       * @const {string[]} paths
       */
      const paths: string[] = [...ancestors.slice(1), node].map(n => {
        ok(n.type !== 'root', 'did not expect tree')
        return n.name
      })

      /**
       * Info about the file.
       *
       * @const {Stats} stats
       */
      const stats: Stats = await fs.stat(pathe.join(tree.path, ...paths))

      return node.size = stats.size, void node
    }
  }
})

console.log(inspect(tree))
console.dir(size(tree))
console.dir(size(tree, node => node.type === 'directory'))
console.dir(size(tree, node => node.type === 'file'))

...yields

root[1]
│ path: "/Users/lex/Projects/flex-development/fst-util-from-fs/"
└─0 directory<src>[6]
    ├─0 directory<interfaces>[16]
    │   ├─0  file<dirent.mts> null
    │   │      size: 950
    │   ├─1  file<file-system-entries.mts> null
    │   │      size: 559
    │   ├─2  file<file-system.mts> null
    │   │      size: 643
    │   ├─3  file<filters.mts> null
    │   │      size: 583
    │   ├─4  file<handles.mts> null
    │   │      size: 640
    │   ├─5  file<index.mts> null
    │   │      size: 1102
    │   ├─6  file<is-directory.mts> null
    │   │      size: 345
    │   ├─7  file<is-file.mts> null
    │   │      size: 315
    │   ├─8  file<is-symbolic-link.mts> null
    │   │      size: 365
    │   ├─9  file<options.mts> null
    │   │      size: 2052
    │   ├─10 file<readdir-dirent-options.mts> null
    │   │      size: 744
    │   ├─11 file<readdir-options.mts> null
    │   │      size: 594
    │   ├─12 file<readdir.mts> null
    │   │      size: 798
    │   ├─13 file<realpath.mts> null
    │   │      size: 743
    │   ├─14 file<stat.mts> null
    │   │      size: 566
    │   └─15 file<stats.mts> null
    │          size: 704
    ├─1 directory<internal>[12]
    │   ├─0  file<chain-or-call.mts> null
    │   │      size: 2656
    │   ├─1  file<combine-paths.mts> null
    │   │      size: 1579
    │   ├─2  file<constant.mts> null
    │   │      size: 413
    │   ├─3  file<empty-array.mts> null
    │   │      size: 260
    │   ├─4  file<empty-file-system-entries.mts> null
    │   │      size: 524
    │   ├─5  file<fs.browser.mts> null
    │   │      size: 973
    │   ├─6  file<fs.d.mts> null
    │   │      size: 241
    │   ├─7  file<fs.node.mts> null
    │   │      size: 414
    │   ├─8  file<identity.mts> null
    │   │      size: 351
    │   ├─9  file<is-promise.mts> null
    │   │      size: 640
    │   ├─10 file<visit-directory.mts> null
    │   │      size: 10232
    │   └─11 file<with-trailing-slash.mts> null
    │          size: 1357
    ├─2 directory<types>[10]
    │   ├─0 file<awaitable.mts> null
    │   │     size: 269
    │   ├─1 file<extensions.mts> null
    │   │     size: 323
    │   ├─2 file<filter.mts> null
    │   │     size: 1006
    │   ├─3 file<get-file-system-entries.mts> null
    │   │     size: 847
    │   ├─4 file<handle.mts> null
    │   │     size: 1347
    │   ├─5 file<index.mts> null
    │   │     size: 628
    │   ├─6 file<list.mts> null
    │   │     size: 241
    │   ├─7 file<sort.mts> null
    │   │     size: 455
    │   ├─8 file<to-visit-key.mts> null
    │   │     size: 1083
    │   └─9 file<visit-map.mts> null
    │         size: 321
    ├─3 directory<utils>[2]
    │   ├─0 file<get-file-system-entries.mts> null
    │   │     size: 5805
    │   └─1 file<index.mts> null
    │         size: 157
    ├─4 file<index.mts> null
    │     size: 189
    └─5 file<util.mts> null
          size: 2055
47 # total number of nodes
5  # total number of directory nodes
42 # total number of file nodes

API

This package exports the identifiers listed below.

There is no default export.

fromFileSystem<T>([options])

Create a file system tree.

👉 Note: Returns a promise if one of the following methods returns a promise: fs.realpath, options.getFileSystemEntries, options.handles.directory, options.handlers.file.

Type Parameters

• T (Awaitable<Root>) — the tree

Parameters

• options (Options | null | undefined, optional) — options for tree creation

Returns

(T) The file system tree

Utilities

This package exports utilities from @flex-development/fst-util-from-fs/utils.

The utilities library exports the identifiers listed below.

There is no default export.

getFileSystemEntries<T>(parent[, fs])

Get a record of accessible file system entries.

Entries are relative to parent.

👉 Note: Returns a promise if fs.readdir returns a promise, or any symbolic links are encountered and fs.stat returns a promise.

Type Parameters

• T (Awaitable<FileSystemEntries>) — the entries record

Parameters

• parent (URL | string | null | undefined) — the entry id of the parent directory
• fs (FileSystem | null | undefined) — the file system api

Returns

(T) The file system entries record

Types

This package is fully typed with TypeScript.

Awaitable<T>

Create a union of T and T as a promise-like object (type).

type Awaitable<T> = PromiseLike<T> | T

Type Parameters

• T (any) — the value

Dirent

Information about a file system entry (interface).

This interface can be augmented to register custom methods and properties.

declare module '@flex-development/fst-util-from-fs' {
  interface Dirent {
    parentPath: string
  }
}

Properties

• isDirectory (IsDirectory) — check if the entry is a directory
• isFile (IsFile) — check if the entry is a file
• isSymbolicLink (IsSymbolicLink) — check if the entry is a symbolic link
• name (string) — the path to the entry, relative to its parent directory

Extensions

Union of options used to filter files by extension (type).

type Extensions = List<string> | string

FileSystemEntries

Information about directories and files (interface).

Properties

• directories (List<Dirent>) — the list of directories
• files (List<Dirent>) — the list of files

FileSystem

The file system API (interface).

Properties

• readdir (Readdir) — read the entire contents of a directory
• realpath (Realpath) — compute a canonical pathname by resolving ., .., and symbolic links
• stat (Stat) — get information about a file system entry

Filter

Determine if a node for path should be added to the tree (type).

type Filter = (
  this: void,
  path: string,
  depth: number | null | undefined,
  dirent: Dirent,
  parent: AnyParent,
  tree: Root
) => boolean

Parameters

• path (string) — the path to the file system entry, relative to its parent directory
• depth (number | null | undefined) — the current search depth
• dirent (Dirent) — the dirent representing the file system entry
• parent (AnyParent) — the current parent node
• tree (Root) — the file system tree

Returns

(boolean) true if node for path should be added, false otherwise

Filters

The filters used to determine if a node should be added to a tree (interface).

Properties

• directory (Filter | null | undefined, optional) — determine if a Directory node should be added to the tree
• file (Filter | null | undefined, optional) — determine if a File node should be added to the tree

GetFileSystemEntries<[T]>

Get a file system entries record (type).

type GetFileSystemEntries<
  T extends Awaitable<FileSystemEntries> = Awaitable<FileSystemEntries>
> = (
  this: void,
  parent: URL | string,
  fs: FileSystem
) => T

Type Parameters

• T (Awaitable<FileSystemEntries>) — the entries record

Parameters

• parent (URL | string) — the entry id of the parent directory
• fs (FileSystem) — the file system api

Returns

(T) The file system entries record

Handle<[T][, Result]>

Handle a node that has been added to the tree (type).

type Handle<
  T extends Child = Child,
  Result extends Awaitable<null | undefined | void> = Awaitable<null | undefined | void>
> = (
  this: void,
  node: T,
  dirent: Dirent,
  parent: AnyParent,
  tree: Root,
  ancestors: AnyParent[],
  fs: FileSystem
) => Result

Type Parameters

• T (Child, optional) — the fst node
• Result (Awaitable<null | undefined | void>, optional) — the result of the handle

Parameters

• node (T) — the node representing the file system entry
• dirent (Dirent) — the dirent representing the file system entry
• parent (AnyParent) — the parent of node
• tree (Root) — the current file system tree
• ancestors (AnyParent[]) — the ancestors of node, with the last node being parent
• fs (FileSystem) — the file system api

Returns

(Result) Nothing

Handles

The callbacks to fire after a node is added to a tree (interface).

Properties

• directory (Handle<Directory> | null | undefined, optional) — handle a Directory node
• file (Handle<File> | null | undefined, optional) — handle a File node

IsDirectory

Check if a file system entry is a directory (interface).

Signatures

(): boolean

Returns

(boolean) true if entry is directory, false otherwise

IsFile

Check if a file system entry is a file (interface).

Signatures

(): boolean

Returns

(boolean) true if entry is file, false otherwise

IsSymbolicLink

Check if a file system entry is a symbolic link (interface).

Signatures

(): boolean

Returns

(boolean) true if entry is symbolic link, false otherwise

List<[T]>

A list (type).

type List<T = unknown> = ReadonlySet<T> | readonly T[]

Type Parameters

• T (any, optional) — the list item type

Options

Options for creating a file system tree (interface).

Properties

• depth (number | null | undefined, optional) — the maximum search depth (inclusive)

  👉 note: a search depth less than 0 will produce an empty tree

• extensions (Extensions | null | undefined, optional) — the file extensions to filter matched files by

  👉 note: this is alternative way to exclude files from the tree

• filters (Filters | null | undefined, optional) — the filters used to determine if nodes should be added to the tree
• fs (FileSystem | null | undefined, optional) — the file system adapter
• getFileSystemEntries (GetFileSystemEntries | null | undefined, optional) — get a file system entries record
  • default: getFileSystemEntries
• handles (Handles | null | undefined, optional) — the callbacks to fire after a node is added to the tree
• root (URL | string | null | undefined, optional) — the module id of the root directory
  • default: pathe.cwd() + pathe.sep
• sort (Sort | null | undefined, optional) — the child node sorter.
  by default, nodes are sorted by type and name
• visitKey (ToVisitKey | null | undefined, optional) — generate a key for the visited directory map
  • default: path => path
• visited (VisitMap | null | undefined, optional) — the map indicating which directories have already been searched
  • default: new Map()

ReaddirDirentOptions

Options for reading the contents of a directory (interface).

Extends

• ReaddirOptions

Properties

• withFileTypes (true) — whether the result should be a content object list instead of just strings.
  if true, the result will be a list of Direct objects, which provide methods like isDirectory() and isFile() to get more information about a file system entry without additional fs.stat() calls

ReaddirOptions

Options for reading the contents of a directory (interface).

Extends

• ReaddirOptions

Properties

• withFileTypes? (boolean | null | undefined) — whether the result should be a content object list instead of just strings.
  if true, the result will be a list of Direct objects, which provide methods like isDirectory() and isFile() to get more information about a file system entry without additional fs.stat() calls

Readdir

Read the entire contents of a directory (interface).

Signatures

<T extends Awaitable<readonly Dirent[]>>(id: URL | string, options: ReaddirDirentOptions): T

Type Parameters

• T (Awaitable<readonly Dirent[]>) — the directory contents

Parameters

• id (URL | string) — the entry id
• options (ReaddirDirentOptions) — read options

Returns

(T) The directory contents

Realpath

Compute a canonical pathname by resolving ., .., and symbolic links (interface).

👉 Note: A canonical pathname is not necessarily unique. Hard links and bind mounts can expose an entity through many pathnames.

Signatures

<T extends Awaitable<string>>(id: URL | string): T

Type Parameters

• T (Awaitable<string>) — the canonical pathname

Parameters

• id (URL | string) — the entry id

Returns

(T) The canonical pathname

Sort

Decide how two nodes should be sorted (type).

type Sort = (this: void, a: Child, b: Child) => number

Parameters

• a (Child) — the current child node
• b (Child) — the next child node

Returns

(number) The comparison result

Stat

Get information about a file system entry (interface).

Signatures

<T extends Awaitable<Stats>>(id: URL | string): T

Type Parameters

• T (Awaitable<Stats>) — the entry info

Parameters

• id (URL | string) — the entry id

Returns

(T) The entry info

Stats

Information about a file system entry (interface).

This interface can be augmented to register custom methods and properties.

declare module '@flex-development/fst-util-from-fs' {
  interface Stats {
    size: bigint | number
  }
}

Properties

• isDirectory (IsDirectory) — check if the entry is a directory
• isFile (IsFile) — check if the entry is a file

ToVisitKey<[K]>

Get a visit map key for a pathname (type).

type ToVisitKey<K extends WeakKey | string | null = WeakKey | string | null> = (
  this: void,
  path: string,
  dir: string | null,
  parent: AnyParent,
  tree: Root,
  options: Options
) => K

Type Parameters

• K (WeakKey | string | null, optional) — the map key

Parameters

• path (string) — the canonical pathname (realpath) of the directory to visit
• dir (string | null) — the path to the directory to visit, relative to tree.path
• parent (AnyParent) — the current parent node
• tree (Root) — the current file system tree
• options (Options) — options for tree creation

Returns

(K) The visited map key

VisitMap

Map indicating which directories have already been searched (type).

type VisitMap = Map<string | null, boolean> | WeakMap<WeakKey, boolean>

Syntax tree

The syntax tree is fst.

Contribute

See CONTRIBUTING.md.

This project has a code of conduct. By interacting with this repository, organization, or community you agree to abide by its terms.