Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 29 additions & 57 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,71 +2,43 @@ name: CI

on:
push:
branches: [ master, main ]
branches: [main, master]
pull_request:
branches: [ master, main ]
branches: [main, master]

jobs:
test:
name: Test
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
go-version: ['1.22', '1.23']
go: ['1.21', '1.22', '1.23']

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}

- name: Download dependencies
run: go mod download

- name: Run go vet
run: go vet ./...

- name: Check formatting
if: matrix.os == 'ubuntu-latest'
run: |
if [ -n "$(gofmt -l .)" ]; then
echo "The following files are not formatted:"
gofmt -l .
exit 1
fi

- name: Run tests
run: go test -v -race -timeout=10m ./...

- name: Run tests with coverage
if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.23'
run: go test -v -race -coverprofile=coverage.out -covermode=atomic ./...

- name: Upload coverage to Codecov
if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.23'
uses: codecov/codecov-action@v4
with:
files: ./coverage.out
flags: unittests
name: codecov-umbrella
fail_ci_if_error: false

build:
name: Build
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.23'

- name: Build
run: go build -v ./...
- name: Checkout code
uses: actions/checkout@v4

- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go }}

- name: Download dependencies
run: go mod download

- name: Run tests (with coverage)
if: matrix.os == 'ubuntu-latest'
run: go test -v -race -coverprofile=coverage.out ./...

- name: Run tests
if: matrix.os != 'ubuntu-latest'
run: go test -v -race ./...

- name: Upload coverage
if: matrix.os == 'ubuntu-latest' && matrix.go == '1.23'
uses: codecov/codecov-action@v4
with:
files: coverage.out
fail_ci_if_error: false
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.claude/
CLAUDE.md
.project_meta/
100 changes: 72 additions & 28 deletions Readme.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,85 @@
# FsTools - Tools for file systems.

Work in progress.
[![Go Reference](https://pkg.go.dev/badge/github.com/absfs/fstools.svg)](https://pkg.go.dev/github.com/absfs/fstools)
[![Go Report Card](https://goreportcard.com/badge/github.com/absfs/fstools)](https://goreportcard.com/report/github.com/absfs/fstools)
[![CI](https://github.com/absfs/fstools/actions/workflows/ci.yml/badge.svg)](https://github.com/absfs/fstools/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

## Features
Utilities for traversing and manipulating abstract file systems that implement
the [absfs.Filer](https://github.com/absfs/absfs) interface.

- Walk
+ Walk walks a absfs.FileSystem similar to the filepath Walk standard
library function.
## Implemented Features

- WalkWithOptions
+ Adds the ability to specify options for how a walk is performed. Such as
if directories are sorted and in what order, normal or fast walk,
traversal order, symbolic link handling and more.
- **Walk** - Traverses an absfs.Filer similar to filepath.Walk from the standard
library, using PreOrder (depth-first) traversal by default.

- PreOrder, PostOrder, InOrder and BreadthFirst Walkers
+ Additional walking strategies for ordered traversal of file systems.
- **WalkWithOptions** - Configurable traversal with options for sorting,
custom comparison functions, and traversal order selection.

- Copy
+ Copy copies filesystem structures form one filesystem to another with
options for selecting and transforming what is copied.
- **PreOrder** - Depth-first traversal visiting parent directories before
their children (standard traversal order).

- Describe
+ Creates a data structure that describes a file system. The description
contains basic metadata as found in `os.FileInfo`, the identification
of file data and directories using a (c4 Id)[https://github.com/Avalanche-io/c4],
and can be serialized into JSON, YAML, CUE and other formats.
- **PostOrder** - Depth-first traversal visiting children before their parent
directories, useful for operations like recursive deletion.

- Diff
+ Returns the difference between two file system descriptions.
- **BreadthOrder** - Level-by-level traversal (breadth-first search), visiting
all entries at each depth before descending.

- Patch
+ Given a file system that matches the first file system in a Diff, and a
c4 data source Patch will transform the file system to match the second
file system in a diff. Requires a c4 data source.
- **KeyOrder** - Traversal that visits only files, skipping directories entirely.

- Apply
+ Apply takes a file system, c4 data source, and file system description and
transforms the file system to match the description.
- **Exists** - Simple utility to check if a path exists in a filesystem.

- **Size** - Calculate total size of a file or directory tree in bytes.

- **Count** - Count the number of files and directories under a path.

- **Equal** - Compare two files or directory trees for identical content.

- **Copy** - Copy files and directory trees between filesystems with options for:
- Parallel file copying (for thread-safe filesystems)
- Metadata preservation (permissions, timestamps)
- Filtering via callbacks (skip files/directories)
- Error handling callbacks for partial copy recovery

## Installation

```bash
go get github.com/absfs/fstools
```

## Usage

```go
import (
"fmt"
"os"
"path/filepath"

"github.com/absfs/fstools"
"github.com/absfs/memfs"
)

func main() {
fs, _ := memfs.NewFS()

// Walk with default PreOrder traversal
fstools.Walk(fs, "/", func(path string, info os.FileInfo, err error) error {
fmt.Println(path)
return nil
})

// Walk with custom options
opts := &fstools.Options{
Less: func(a, b os.FileInfo) bool {
return a.Name() < b.Name()
},
Traversal: fstools.BreadthTraversal,
}
fstools.WalkWithOptions(fs, opts, "/", func(path string, info os.FileInfo, err error) error {
fmt.Println(path)
return nil
})
}
```


Loading