yamlplus

A Go library that extends YAML with cross-file references, allowing you to split large configuration files into smaller, reusable pieces.

Features

• Reference YAML anchors across multiple files using the !xref tag
• Support for standard YAML map merges with cross-file references
• Streaming Decoder API with options like KnownFields
• Load files individually, by directory, or recursively
• Built on top of go.yaml.in/yaml/v3
• Circular dependency detection

Installation

go get github.com/supakeen/yamlplus

Quick Start

Given two YAML files:

database.yaml:

connection: &db-config
  host: localhost
  port: 5432
  timeout: 30s

app.yaml:

service:
  name: myapp
  database: !xref "database.yaml#db-config"

Load and unmarshal them:

package main

import (
    "fmt"
    "os"
    "github.com/supakeen/yamlplus"
)

func main() {
    loader := yamlplus.NewLoader(os.DirFS("config"))
    loader.RegisterFile("database.yaml")
    
    var config map[string]any
    data, _ := os.ReadFile("config/app.yaml")
    loader.Unmarshal(data, &config)
    
    // Access the cross-referenced database configuration
    service := config["service"].(map[string]any)
    db := service["database"].(map[string]any)
    fmt.Printf("Database: %s:%d\n", db["host"], db["port"])
}

Decoder

For streaming decoding or when you need options like strict field checking, use NewDecoder instead of Unmarshal:

file, _ := os.Open("config/app.yaml")
defer file.Close()

dec := loader.NewDecoder(file)
dec.KnownFields(true) // error on unknown fields

var config AppConfig
if err := dec.Decode(&config); err != nil {
    log.Fatal(err)
}

The Decoder supports all the same !xref resolution as Unmarshal. It also handles multi-document YAML streams — call Decode repeatedly until it returns io.EOF:

dec := loader.NewDecoder(file)

for {
    var doc map[string]any
    if err := dec.Decode(&doc); err == io.EOF {
        break
    } else if err != nil {
        log.Fatal(err)
    }
    fmt.Println(doc)
}

Reference Syntax

The !xref tag supports two forms:

Reference a specific anchor

config: !xref "filename.yaml#anchorname"

Reference an entire file

config: !xref "filename.yaml"

When referencing a file without an anchor, the first document in the file is used.

Map Merges

Cross-file references work with YAML's map merge syntax:

defaults.yaml:

defaults: &api-defaults
  timeout: 30s
  retries: 3
  log_level: info

service.yaml:

production:
  <<: !xref "defaults.yaml#api-defaults"
  timeout: 60s        # Override the default
  endpoint: /api/v1

Result after unmarshaling:

production:
  timeout: 60s
  retries: 3
  log_level: info
  endpoint: /api/v1

Loading Files

Load individual files

loader := yamlplus.NewLoader(os.DirFS("config"))
loader.RegisterFile("base.yaml")
loader.RegisterFile("database.yaml")

Load all YAML files in a directory

loader.RegisterDirectory("configs")

This loads all .yaml and .yml files in the directory (non-recursive).

Load files recursively

loader.RegisterRecursively("configs")

This walks the directory tree and loads all YAML files.

Path-Based Namespacing

Files are registered and referenced using their exact path relative to the filesystem root:

loader := yamlplus.NewLoader(os.DirFS("/etc"))
loader.RegisterFile("app/config.yaml")

// Must use the same path in references:
data := []byte(`settings: !xref "app/config.yaml"`)

Circular Dependency Detection

The library detects circular references and returns an error:

a.yaml:

value: !xref "b.yaml"

b.yaml:

value: !xref "a.yaml"

Attempting to unmarshal will result in an error: circular dependency detected.

Thread Safety

A Loader is safe for concurrent Unmarshal calls after all files have been registered. However, RegisterFile, RegisterDirectory, and RegisterRecursively should not be called concurrently with each other or with Unmarshal.

CLI: yamlplus-flatten

A command-line tool to resolve all !xref tags in a YAML file and output plain YAML.

Installation

go install github.com/supakeen/yamlplus/cmd/yamlplus-flatten@latest

Usage

yamlplus-flatten [flags] <input.yaml>

Flags:

Long	Short	Description
--register-file PATH	-f PATH	Register a single YAML file (repeatable)
--register-directory PATH	-d PATH	Register all YAML files in a directory (repeatable)
--register-recursively PATH	-r PATH	Register all YAML files in a directory tree (repeatable)

Example

yamlplus-flatten -f defaults.yaml -f database.yaml service.yaml

This registers defaults.yaml and database.yaml, then resolves all !xref tags in service.yaml and prints the flattened result to stdout.

Examples

See the examples in the documentation for more usage patterns.

Testing

go test ./...

License

MIT