-A FileSystem Abstraction System for Go
-[](https://github.com/spf13/afero/actions?query=workflow%3ACI)
-[](https://gitter.im/spf13/afero?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
-[](https://goreportcard.com/report/github.com/spf13/afero)
-
-[](https://pkg.go.dev/mod/github.com/spf13/afero)
+[](https://github.com/spf13/afero/actions?query=workflow%3ACI)
+[](https://pkg.go.dev/mod/github.com/spf13/afero)
+[](https://goreportcard.com/report/github.com/spf13/afero)
+
-# Overview
-Afero is a filesystem framework providing a simple, uniform and universal API
-interacting with any filesystem, as an abstraction layer providing interfaces,
-types and methods. Afero has an exceptionally clean interface and simple design
-without needless constructors or initialization methods.
+# Afero: The Universal Filesystem Abstraction for Go
-Afero is also a library providing a base set of interoperable backend
-filesystems that make it easy to work with, while retaining all the power
-and benefit of the os and ioutil packages.
+Afero is a powerful and extensible filesystem abstraction system for Go. It provides a single, unified API for interacting with diverse filesystems—including the local disk, memory, archives, and network storage.
-Afero provides significant improvements over using the os package alone, most
-notably the ability to create mock and testing filesystems without relying on the disk.
+Afero acts as a drop-in replacement for the standard `os` package, enabling you to write modular code that is agnostic to the underlying storage, dramatically simplifies testing, and allows for sophisticated architectural patterns through filesystem composition.
-It is suitable for use in any situation where you would consider using the OS
-package as it provides an additional abstraction that makes it easy to use a
-memory backed file system during testing. It also adds support for the http
-filesystem for full interoperability.
+## Why Afero?
+Afero elevates filesystem interaction beyond simple file reading and writing, offering solutions for testability, flexibility, and advanced architecture.
-## Afero Features
+🔑 **Key Features:**
-* A single consistent API for accessing a variety of filesystems
-* Interoperation between a variety of file system types
-* A set of interfaces to encourage and enforce interoperability between backends
-* An atomic cross platform memory backed file system
-* Support for compositional (union) file systems by combining multiple file systems acting as one
-* Specialized backends which modify existing filesystems (Read Only, Regexp filtered)
-* A set of utility functions ported from io, ioutil & hugo to be afero aware
-* Wrapper for go 1.16 filesystem abstraction `io/fs.FS`
+* **Universal API:** Write your code once. Run it against the local OS, in-memory storage, ZIP/TAR archives, or remote systems (SFTP, GCS).
+* **Ultimate Testability:** Utilize `MemMapFs`, a fully concurrent-safe, read/write in-memory filesystem. Write fast, isolated, and reliable unit tests without touching the physical disk or worrying about cleanup.
+* **Powerful Composition:** Afero's hidden superpower. Layer filesystems on top of each other to create sophisticated behaviors:
+ * **Sandboxing:** Use `CopyOnWriteFs` to create temporary scratch spaces that isolate changes from the base filesystem.
+ * **Caching:** Use `CacheOnReadFs` to automatically layer a fast cache (like memory) over a slow backend (like a network drive).
+ * **Security Jails:** Use `BasePathFs` to restrict application access to a specific subdirectory (chroot).
+* **`os` Package Compatibility:** Afero mirrors the functions in the standard `os` package, making adoption and refactoring seamless.
+* **`io/fs` Compatibility:** Fully compatible with the Go standard library's `io/fs` interfaces.
-# Using Afero
+## Installation
-Afero is easy to use and easier to adopt.
-
-A few different ways you could use Afero:
-
-* Use the interfaces alone to define your own file system.
-* Wrapper for the OS packages.
-* Define different filesystems for different parts of your application.
-* Use Afero for mock filesystems while testing
-
-## Step 1: Install Afero
-
-First use go get to install the latest version of the library.
-
- $ go get github.com/spf13/afero
+```bash
+go get github.com/spf13/afero
+```
-Next include Afero in your application.
```go
import "github.com/spf13/afero"
```
-## Step 2: Declare a backend
+## Quick Start: The Power of Abstraction
+
+The core of Afero is the `afero.Fs` interface. By designing your functions to accept this interface rather than calling `os.*` functions directly, your code instantly becomes more flexible and testable.
+
+### 1. Refactor Your Code
+
+Change functions that rely on the `os` package to accept `afero.Fs`.
-First define a package variable and set it to a pointer to a filesystem.
```go
-var AppFs = afero.NewMemMapFs()
+// Before: Coupled to the OS and difficult to test
+// func ProcessConfiguration(path string) error {
+// data, err := os.ReadFile(path)
+// ...
+// }
-or
+import "github.com/spf13/afero"
-var AppFs = afero.NewOsFs()
+// After: Decoupled, flexible, and testable
+func ProcessConfiguration(fs afero.Fs, path string) error {
+ // Use Afero utility functions which mirror os/ioutil
+ data, err := afero.ReadFile(fs, path)
+ // ... process the data
+ return err
+}
```
-It is important to note that if you repeat the composite literal you
-will be using a completely new and isolated filesystem. In the case of
-OsFs it will still use the same underlying filesystem but will reduce
-the ability to drop in other filesystems as desired.
-## Step 3: Use it like you would the OS package
+### 2. Usage in Production
-Throughout your application use any function and method like you normally
-would.
+In your production environment, inject the `OsFs` backend, which wraps the standard operating system calls.
-So if my application before had:
-```go
-os.Open("/tmp/foo")
-```
-We would replace it with:
```go
-AppFs.Open("/tmp/foo")
+func main() {
+ // Use the real OS filesystem
+ AppFs := afero.NewOsFs()
+ ProcessConfiguration(AppFs, "/etc/myapp.conf")
+}
```
-`AppFs` being the variable we defined above.
+### 3. Usage in Testing
+In your tests, inject `MemMapFs`. This provides a blazing-fast, isolated, in-memory filesystem that requires no disk I/O and no cleanup.
-## List of all available functions
-
-File System Methods Available:
```go
-Chmod(name string, mode os.FileMode) : error
-Chown(name string, uid, gid int) : error
-Chtimes(name string, atime time.Time, mtime time.Time) : error
-Create(name string) : File, error
-Mkdir(name string, perm os.FileMode) : error
-MkdirAll(path string, perm os.FileMode) : error
-Name() : string
-Open(name string) : File, error
-OpenFile(name string, flag int, perm os.FileMode) : File, error
-Remove(name string) : error
-RemoveAll(path string) : error
-Rename(oldname, newname string) : error
-Stat(name string) : os.FileInfo, error
-```
-File Interfaces and Methods Available:
-```go
-io.Closer
-io.Reader
-io.ReaderAt
-io.Seeker
-io.Writer
-io.WriterAt
-
-Name() : string
-Readdir(count int) : []os.FileInfo, error
-Readdirnames(n int) : []string, error
-Stat() : os.FileInfo, error
-Sync() : error
-Truncate(size int64) : error
-WriteString(s string) : ret int, err error
+func TestProcessConfiguration(t *testing.T) {
+ // Use the in-memory filesystem
+ AppFs := afero.NewMemMapFs()
+
+ // Pre-populate the memory filesystem for the test
+ configPath := "/test/config.json"
+ afero.WriteFile(AppFs, configPath, []byte(`{"feature": true}`), 0644)
+
+ // Run the test entirely in memory
+ err := ProcessConfiguration(AppFs, configPath)
+ if err != nil {
+ t.Fatal(err)
+ }
+}
```
-In some applications it may make sense to define a new package that
-simply exports the file system variable for easy access from anywhere.
-## Using Afero's utility functions
+## Afero's Superpower: Composition
-Afero provides a set of functions to make it easier to use the underlying file systems.
-These functions have been primarily ported from io & ioutil with some developed for Hugo.
+Afero's most unique feature is its ability to combine filesystems. This allows you to build complex behaviors out of simple components, keeping your application logic clean.
-The afero utilities support all afero compatible backends.
+### Example 1: Sandboxing with Copy-on-Write
-The list of utilities includes:
+Create a temporary environment where an application can "modify" system files without affecting the actual disk.
```go
-DirExists(path string) (bool, error)
-Exists(path string) (bool, error)
-FileContainsBytes(filename string, subslice []byte) (bool, error)
-GetTempDir(subPath string) string
-IsDir(path string) (bool, error)
-IsEmpty(path string) (bool, error)
-ReadDir(dirname string) ([]os.FileInfo, error)
-ReadFile(filename string) ([]byte, error)
-SafeWriteReader(path string, r io.Reader) (err error)
-TempDir(dir, prefix string) (name string, err error)
-TempFile(dir, prefix string) (f File, err error)
-Walk(root string, walkFn filepath.WalkFunc) error
-WriteFile(filename string, data []byte, perm os.FileMode) error
-WriteReader(path string, r io.Reader) (err error)
-```
-For a complete list see [Afero's GoDoc](https://godoc.org/github.com/spf13/afero)
+// 1. The base layer is the real OS, made read-only for safety.
+baseFs := afero.NewReadOnlyFs(afero.NewOsFs())
-They are available under two different approaches to use. You can either call
-them directly where the first parameter of each function will be the file
-system, or you can declare a new `Afero`, a custom type used to bind these
-functions as methods to a given filesystem.
+// 2. The overlay layer is a temporary in-memory filesystem for changes.
+overlayFs := afero.NewMemMapFs()
-### Calling utilities directly
+// 3. Combine them. Reads fall through to the base; writes only hit the overlay.
+sandboxFs := afero.NewCopyOnWriteFs(baseFs, overlayFs)
-```go
-fs := new(afero.MemMapFs)
-f, err := afero.TempFile(fs,"", "ioutil-test")
+// The application can now "modify" /etc/hosts, but the changes are isolated in memory.
+afero.WriteFile(sandboxFs, "/etc/hosts", []byte("127.0.0.1 sandboxed-app"), 0644)
+// The real /etc/hosts on disk is untouched.
```
-### Calling via Afero
+### Example 2: Caching a Slow Filesystem
-```go
-fs := afero.NewMemMapFs()
-afs := &afero.Afero{Fs: fs}
-f, err := afs.TempFile("", "ioutil-test")
-```
+Improve performance by layering a fast cache (like memory) over a slow backend (like a network drive or cloud storage).
-## Using Afero for Testing
+```go
+import "time"
-There is a large benefit to using a mock filesystem for testing. It has a
-completely blank state every time it is initialized and can be easily
-reproducible regardless of OS. You could create files to your heart’s content
-and the file access would be fast while also saving you from all the annoying
-issues with deleting temporary files, Windows file locking, etc. The MemMapFs
-backend is perfect for testing.
+// Assume 'remoteFs' is a slow backend (e.g., SFTP or GCS)
+var remoteFs afero.Fs
-* Much faster than performing I/O operations on disk
-* Avoid security issues and permissions
-* Far more control. 'rm -rf /' with confidence
-* Test setup is far more easier to do
-* No test cleanup needed
+// 'cacheFs' is a fast in-memory backend
+cacheFs := afero.NewMemMapFs()
-One way to accomplish this is to define a variable as mentioned above.
-In your application this will be set to afero.NewOsFs() during testing you
-can set it to afero.NewMemMapFs().
+// Create the caching layer. Cache items for 5 minutes upon first read.
+cachedFs := afero.NewCacheOnReadFs(remoteFs, cacheFs, 5*time.Minute)
-It wouldn't be uncommon to have each test initialize a blank slate memory
-backend. To do this I would define my `appFS = afero.NewOsFs()` somewhere
-appropriate in my application code. This approach ensures that Tests are order
-independent, with no test relying on the state left by an earlier test.
+// The first read is slow (fetches from remote, then caches)
+data1, _ := afero.ReadFile(cachedFs, "data.json")
-Then in my tests I would initialize a new MemMapFs for each test:
-```go
-func TestExist(t *testing.T) {
- appFS := afero.NewMemMapFs()
- // create test files and directories
- appFS.MkdirAll("src/a", 0755)
- afero.WriteFile(appFS, "src/a/b", []byte("file b"), 0644)
- afero.WriteFile(appFS, "src/c", []byte("file c"), 0644)
- name := "src/c"
- _, err := appFS.Stat(name)
- if os.IsNotExist(err) {
- t.Errorf("file \"%s\" does not exist.\n", name)
- }
-}
+// The second read is instant (serves from memory cache)
+data2, _ := afero.ReadFile(cachedFs, "data.json")
```
-# Available Backends
+### Example 3: Security Jails (chroot)
+
+Restrict an application component's access to a specific subdirectory.
-## Operating System Native
+```go
+osFs := afero.NewOsFs()
-### OsFs
+// Create a filesystem rooted at /home/user/public
+// The application cannot access anything above this directory.
+jailedFs := afero.NewBasePathFs(osFs, "/home/user/public")
-The first is simply a wrapper around the native OS calls. This makes it
-very easy to use as all of the calls are the same as the existing OS
-calls. It also makes it trivial to have your code use the OS during
-operation and a mock filesystem during testing or as needed.
+// To the application, this is reading "/"
+// In reality, it's reading "/home/user/public/"
+dirInfo, err := afero.ReadDir(jailedFs, "/")
-```go
-appfs := afero.NewOsFs()
-appfs.MkdirAll("src/a", 0755)
+// Attempts to access parent directories fail
+_, err = jailedFs.Open("../secrets.txt") // Returns an error
```
-## Memory Backed Storage
+## Real-World Use Cases
-### MemMapFs
+### Build Cloud-Agnostic Applications
-Afero also provides a fully atomic memory backed filesystem perfect for use in
-mocking and to speed up unnecessary disk io when persistence isn’t
-necessary. It is fully concurrent and will work within go routines
-safely.
+Write applications that seamlessly work with different storage backends:
```go
-mm := afero.NewMemMapFs()
-mm.MkdirAll("src/a", 0755)
-```
+type DocumentProcessor struct {
+ fs afero.Fs
+}
+
+func NewDocumentProcessor(fs afero.Fs) *DocumentProcessor {
+ return &DocumentProcessor{fs: fs}
+}
-#### InMemoryFile
+func (p *DocumentProcessor) Process(inputPath, outputPath string) error {
+ // This code works whether fs is local disk, cloud storage, or memory
+ content, err := afero.ReadFile(p.fs, inputPath)
+ if err != nil {
+ return err
+ }
+
+ processed := processContent(content)
+ return afero.WriteFile(p.fs, outputPath, processed, 0644)
+}
-As part of MemMapFs, Afero also provides an atomic, fully concurrent memory
-backed file implementation. This can be used in other memory backed file
-systems with ease. Plans are to add a radix tree memory stored file
-system using InMemoryFile.
+// Use with local filesystem
+processor := NewDocumentProcessor(afero.NewOsFs())
-## Network Interfaces
+// Use with Google Cloud Storage
+processor := NewDocumentProcessor(gcsFS)
-### SftpFs
+// Use with in-memory filesystem for testing
+processor := NewDocumentProcessor(afero.NewMemMapFs())
+```
-Afero has experimental support for secure file transfer protocol (sftp). Which can
-be used to perform file operations over a encrypted channel.
+### Treating Archives as Filesystems
-### GCSFs
+Read files directly from `.zip` or `.tar` archives without unpacking them to disk first.
-Afero has experimental support for Google Cloud Storage (GCS). You can either set the
-`GOOGLE_APPLICATION_CREDENTIALS_JSON` env variable to your JSON credentials or use `opts` in
-`NewGcsFS` to configure access to your GCS bucket.
+```go
+import (
+ "archive/zip"
+ "github.com/spf13/afero/zipfs"
+)
-Some known limitations of the existing implementation:
-* No Chmod support - The GCS ACL could probably be mapped to *nix style permissions but that would add another level of complexity and is ignored in this version.
-* No Chtimes support - Could be simulated with attributes (gcs a/m-times are set implicitly) but that's is left for another version.
-* Not thread safe - Also assumes all file operations are done through the same instance of the GcsFs. File operations between different GcsFs instances are not guaranteed to be consistent.
+// Assume 'zipReader' is a *zip.Reader initialized from a file or memory
+var zipReader *zip.Reader
+// Create a read-only ZipFs
+archiveFS := zipfs.New(zipReader)
-## Filtering Backends
+// Read a file from within the archive using the standard Afero API
+content, err := afero.ReadFile(archiveFS, "/docs/readme.md")
+```
-### BasePathFs
+### Serving Any Filesystem over HTTP
-The BasePathFs restricts all operations to a given path within an Fs.
-The given file name to the operations on this Fs will be prepended with
-the base path before calling the source Fs.
+Use `HttpFs` to expose any Afero filesystem—even one created dynamically in memory—through a standard Go web server.
```go
-bp := afero.NewBasePathFs(afero.NewOsFs(), "/base/path")
-```
+import (
+ "net/http"
+ "github.com/spf13/afero"
+)
-### ReadOnlyFs
+func main() {
+ memFS := afero.NewMemMapFs()
+ afero.WriteFile(memFS, "index.html", []byte("