everything-json

everything-json projects common filesystem files into structured JSON through a read-only virtual filesystem.

It mirrors source files and exposes virtual *.json siblings on read.

How it looks

Source directory (example):

/workspace/src
├── data.csv
├── page.html
└── Image 11.jpeg

Mounted virtual view:

/mnt/everything-json
├── data.csv.json
├── page.html.json
└── Image 11.jpeg.json

Supported conversions

• Text/log: .txt, .text, .log
• Tabular: .csv, .tsv
• Structured: .json, .jsonl, .ndjson, .yaml, .yml, .toml, .ini, .cfg, .conf, .xml
• Docs/web: .md, .markdown, .html, .htm, .pdf (placeholder for deeper extraction)
• Email: .eml, .mail
• Images: .png, .jpg, .jpeg, .gif, .bmp, .webp
• Archives: .zip, .tar.gz
• Gzip passthrough projection for supported inner formats, e.g. orders.csv.gz -> orders.csv.json

Output shape

Output is plain structured JSON for each file type.

CSV/TSV outputs are arrays of row objects:

[
  {
    "id": "1",
    "name": "Ada",
    "score": "98"
  },
  {
    "id": "2",
    "name": "Linus",
    "score": "95"
  }
]

Image outputs include deterministic metadata plus a normalized semantic block:

{
  "type": "image",
  "format": "jpeg",
  "width": 1440,
  "height": 1920,
  "semantic": {
    "scene_type": "indoor",
    "caption": "A black cat sleeping on a sofa",
    "objects": [
      {
        "name": "cat",
        "confidence": 0.95,
        "attributes": ["black", "sleeping", "domestic"]
      }
    ],
    "detected_text": [],
    "visual_style": "soft",
    "safety_notes": [],
    "color_palette": ["black", "gray", "beige"],
    "interesting_details": ["cat sleeping", "sofa"],
    "extraction_confidence": 0.9
  }
}

Ollama enrichment

Optional local AI enrichment:

• OLLAMA_MODEL: contextual extraction for HTML
• OLLAMA_VISION_MODEL: semantic extraction for images
• OLLAMA_ENDPOINT: defaults to http://127.0.0.1:11434/api/generate

No AI configured still returns structured deterministic JSON.

Cache

Rendered JSON is cached by source path + size + modtime + active AI profile.

• Default: $XDG_CACHE_HOME/everything-json or ~/.cache/everything-json
• Override: EVERYTHING_JSON_CACHE_DIR

CLI

Build:

go build -o bin/everything-json ./cmd/everything-json

Render one file:

./bin/everything-json render --file /path/to/page.html

List formats:

./bin/everything-json list-formats

Docker mount workflow (recommended on macOS/Windows)

Because host FUSE is often restricted on Docker Desktop/macOS/Windows, mount in a Linux container and run CLI tools via docker exec.

Start mount container:

./bin/ej-start /Users/henneberger/libs/everything-json/examples/smoke

Run commands against mounted FS:

./bin/ej-run 'ls -la /mnt/everything-json'
./bin/ej-run 'cat /mnt/everything-json/data.csv.json'
./bin/ej-run "cat '/mnt/everything-json/Image 11.jpeg.json'"

Stop:

./bin/ej-stop

What the helper scripts do

• bin/ej-start: starts everything-json-mount with FUSE privileges and Ollama env defaults
• bin/ej-run: runs any shell command inside the running mount container
• bin/ej-stop: removes the mount container

Platform notes

• Linux host with /dev/fuse: direct host mount works.
• macOS/Windows (common): host-visible FUSE mount propagation is frequently unavailable; use Docker shuttle mode (ej-start + ej-run) or render.
• Windows native: FUSE mount not supported by this implementation (render path remains available).

CI and release

GitHub Actions included:

• .github/workflows/ci.yml

  • go test ./...
  • native build + cross-build smoke test (linux, darwin, windows)
  • Docker image build smoke test

• .github/workflows/release.yml (on GitHub Release publish)

  • builds release binaries for linux/darwin/windows
  • uploads .tar.gz/.zip artifacts to the release
  • builds and publishes multi-arch container image to GHCR:
    • ghcr.io/<owner>/<repo>:<tag>
    • ghcr.io/<owner>/<repo>:latest