Skip to content

Commit 80991f0

Browse files
adeepnadeepn
authored
Merge pull request #2 from jethome-iot/feature/multi-language-header-api
Add multi-language header API (C++/Rust) with cross-language CI
2 parents 3b3c9fe + 323a766 commit 80991f0

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

67 files changed

+6917
-163
lines changed

‎.github/workflows/ci.yaml‎

Lines changed: 0 additions & 55 deletions
This file was deleted.

‎.github/workflows/ci.yml‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [master]
6+
pull_request:
7+
branches: [master]
8+
workflow_dispatch:
9+
10+
jobs:
11+
# ── C / C++ build + ctest (includes C, C++, Python, Rust verifiers) ──
12+
build-and-test:
13+
runs-on: [self-hosted, amd64]
14+
timeout-minutes: 15
15+
steps:
16+
- uses: actions/checkout@v6
17+
18+
- name: Install system dependencies
19+
run: sudo apt-get update && sudo apt-get install -y cmake zlib1g-dev python3
20+
21+
- name: Install Rust toolchain
22+
uses: dtolnay/rust-toolchain@stable
23+
24+
- name: Cache Rust dependencies
25+
uses: actions/cache@v5
26+
with:
27+
path: |
28+
rust/jeefs-header/target
29+
~/.cargo/registry
30+
~/.cargo/git
31+
key: rust-${{ runner.os }}-${{ hashFiles('rust/jeefs-header/Cargo.lock', 'rust/jeefs-header/Cargo.toml') }}
32+
restore-keys: rust-${{ runner.os }}-
33+
34+
- name: Create EEPROM test file
35+
run: |
36+
mkdir -p build/tests/common
37+
dd if=/dev/zero of=build/tests/common/eeprom.bin bs=1 count=8192
38+
39+
- name: CMake configure
40+
run: cmake -B build -DCMAKE_BUILD_TYPE=Release
41+
42+
- name: CMake build (C, C++, Rust verifiers)
43+
run: cmake --build build
44+
45+
- name: Run all ctest (C / C++ / Python / Rust cross-language)
46+
run: cd build && ctest --verbose --output-on-failure
47+
48+
- name: Show test results
49+
if: always()
50+
run: cat build/Testing/Temporary/LastTest.log >> $GITHUB_STEP_SUMMARY
51+
52+
# ── Rust unit tests ──
53+
rust-tests:
54+
runs-on: [self-hosted, amd64]
55+
timeout-minutes: 15
56+
steps:
57+
- uses: actions/checkout@v6
58+
59+
- name: Install Rust toolchain
60+
uses: dtolnay/rust-toolchain@stable
61+
62+
- name: Cache Rust dependencies
63+
uses: actions/cache@v5
64+
with:
65+
path: |
66+
rust/jeefs-header/target
67+
~/.cargo/registry
68+
~/.cargo/git
69+
key: rust-${{ runner.os }}-${{ hashFiles('rust/jeefs-header/Cargo.lock', 'rust/jeefs-header/Cargo.toml') }}
70+
restore-keys: rust-${{ runner.os }}-
71+
72+
- name: Rust unit tests
73+
run: cargo test --manifest-path rust/jeefs-header/Cargo.toml --verbose
74+
75+
# ── Python tests ──
76+
python-tests:
77+
runs-on: [self-hosted, amd64]
78+
timeout-minutes: 15
79+
steps:
80+
- uses: actions/checkout@v6
81+
82+
- name: Set up Python
83+
uses: actions/setup-python@v6
84+
with:
85+
python-version: "3.12"
86+
87+
- name: Install Python package with test deps
88+
run: pip install -e "python/[test]"
89+
90+
- name: Run pytest
91+
run: python -m pytest python/tests/ -v
92+
93+
# ── Code generation validation ──
94+
codegen-check:
95+
runs-on: [self-hosted, amd64]
96+
timeout-minutes: 15
97+
steps:
98+
- uses: actions/checkout@v6
99+
100+
- name: Set up Python
101+
uses: actions/setup-python@v6
102+
with:
103+
python-version: "3.12"
104+
105+
- name: Validate specs (no output, just check parsing)
106+
working-directory: tools
107+
run: |
108+
python -m jeefs_codegen \
109+
--specs ../docs/format/header-common.md \
110+
../docs/format/header-v1.md \
111+
../docs/format/header-v2.md \
112+
../docs/format/header-v3.md \
113+
../docs/format/filesystem-v1.md \
114+
--validate-only
115+
116+
- name: Check generated C header is up-to-date
117+
working-directory: tools
118+
run: |
119+
python -m jeefs_codegen \
120+
--specs ../docs/format/header-common.md \
121+
../docs/format/header-v1.md \
122+
../docs/format/header-v2.md \
123+
../docs/format/header-v3.md \
124+
../docs/format/filesystem-v1.md \
125+
--c-output /tmp/jeefs_generated.h
126+
diff -u ../include/jeefs_generated.h /tmp/jeefs_generated.h
127+
128+
- name: Check generated Rust module is up-to-date
129+
working-directory: tools
130+
run: |
131+
python -m jeefs_codegen \
132+
--specs ../docs/format/header-common.md \
133+
../docs/format/header-v1.md \
134+
../docs/format/header-v2.md \
135+
../docs/format/header-v3.md \
136+
../docs/format/filesystem-v1.md \
137+
--rs-output /tmp/generated.rs
138+
diff -u ../rust/jeefs-header/src/generated.rs /tmp/generated.rs

‎.gitignore‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,4 +55,13 @@ dkms.conf
5555
cmake-build-**
5656
build/**
5757

58+
# Rust
59+
rust/*/target/
60+
61+
# Python
62+
__pycache__/
63+
*.pyc
64+
*.egg-info/
65+
python/.venv/
66+
5867
.idea/**

‎CLAUDE.md‎

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
JEEFS (JetHome EEPROM File System) — a library for working with a simple linked-list filesystem on small EEPROMs (target: 64Kbit / 8KB). Dual-licensed GPL-2.0+ / MIT.
8+
9+
The long-term goal is a **universal multi-language library** (C/C++/Python/Rust/Go/TypeScript) with two areas:
10+
11+
1. **Header parsing/manipulation** (priority) — native implementations per language, `EEPROM_FORMAT.md` as source of truth, shared binary test vectors
12+
2. **File system operations** — C/C++ only, CRUD for files stored as a linked list after the header
13+
14+
### Multi-language strategy
15+
16+
Header parsing uses **native implementations per language** (not FFI). Rationale: the header is 256 bytes / ~13 fields — the parsing logic (~80 lines) is comparable in size to an FFI wrapper, and native packages are trivial to deploy (`pip install`, `cargo add`, `go get`). `EEPROM_FORMAT.md` is the canonical spec; shared binary test vectors in `test-vectors/` ensure cross-language consistency.
17+
18+
FS operations stay in C/C++ — more complex, I/O-dependent, only needed on embedded targets.
19+
20+
## Build & Test
21+
22+
### C
23+
24+
```bash
25+
# Full build
26+
mkdir -p build && cd build && cmake .. && cmake --build .
27+
28+
# Run all tests (requires eeprom.bin)
29+
mkdir -p build/tests/common
30+
dd if=/dev/zero of=build/tests/common/eeprom.bin bs=1 count=8192
31+
cd build && ctest
32+
33+
# Run a single test
34+
cd build && ctest -R test_00 # format test (v1, v2, v3)
35+
cd build && ctest -R test_01 # add files test
36+
cd build && ctest -R test_02 # read file test
37+
38+
# Verbose test output
39+
cd build && ctest --verbose
40+
```
41+
42+
**Dependencies:** zlib (for CRC32), CMake 3.14+, C11, C++17.
43+
44+
### Python
45+
46+
```bash
47+
cd python
48+
uv venv .venv && uv pip install -e ".[test]"
49+
.venv/bin/python -m pytest tests/ -v
50+
```
51+
52+
**Formatting:** `clang-format` with LLVM base style, 120-column limit, 4-space indent. Config in `.clang-format`.
53+
54+
## Architecture
55+
56+
### Layered Design
57+
58+
```
59+
┌───────────────────────────────────────────────┐
60+
│ Python package (python/jeefs/) │ Native header parsing
61+
├───────────────────────────────────────────────┤
62+
│ C++ wrapper (srcpp/jeefspp) │ std::vector-based API
63+
├───────────────────────────────────────────────┤
64+
│ Pure header API (jeefs_header.h/c) │ Byte-buffer ops, no I/O
65+
├───────────────────────────────────────────────┤
66+
│ JEEFS core C API (src/jeefs.c) │ File ops + header mgmt
67+
├───────────────────────────────────────────────┤
68+
│ eepromops interface (include/eepromops.h) │ Abstract read/write
69+
├───────────────────────────────────────────────┤
70+
│ eepromops-memory backend (eepromops-memory/) │ In-memory EEPROM sim
71+
└───────────────────────────────────────────────┘
72+
```
73+
74+
Two distinct APIs:
75+
76+
- **`jeefs_header.h`** — pure functions on byte buffers (detect version, verify/update CRC, init header). No eepromops dependency. Suitable for standalone use or integration.
77+
- **`jeefs.h`** — full FS API (open/close EEPROM, format, list/read/write/add/delete files, header get/set). Depends on eepromops backend.
78+
79+
### EEPROM Binary Layout
80+
81+
```
82+
Offset 0:
83+
┌──────────────────────────────────────────────────┐
84+
│ Header v1 (512B), v2 (256B), or v3 (256B) │
85+
│ magic "JETHOME\0" | version | boardname │
86+
│ boardversion | serial | usid | cpuid │
87+
│ mac[6] | [signature + timestamp (v3)] | crc32 │
88+
├──────────────────────────────────────────────────┤
89+
│ File1: FileHeader (24B) + data (N bytes) │
90+
│ name[16] | dataSize | crc32 | │
91+
│ nextFileAddress → points to File2 │
92+
├──────────────────────────────────────────────────┤
93+
│ ... │
94+
│ FileN: nextFileAddress = 0 (end marker) │
95+
├──────────────────────────────────────────────────┤
96+
│ Free space (0x00) │
97+
└──────────────────────────────────────────────────┘
98+
```
99+
100+
All structures use `#pragma pack(push, 1)` — no padding. Addresses are `uint16_t` (max 64KB). Empty bytes are `0x00` or `0xFF` (both treated as empty). All multi-byte fields are **little-endian**. Magic is `4A 45 54 48 4F 4D 45 00` ("JETHOME\0") for all versions. CRC32 uses IEEE 802.3 polynomial (zlib `crc32()`).
101+
102+
See `EEPROM_FORMAT.md` for full field-by-field layout tables for all versions.
103+
104+
### Key Data Structures
105+
106+
**C (include/jeefs.h):**
107+
108+
- `JEEPROMHeaderv1` (512B) / `JEEPROMHeaderv2` (256B) / `JEEPROMHeaderv3` (256B) — device identity headers
109+
- `JEEPROMHeader` — union of all header versions + `JEEPROMHeaderversion` (12-byte version-detect struct)
110+
- `JEEFSSignatureAlgorithm` — enum: `JEEFS_SIG_NONE` (0), `JEEFS_SIG_SECP192R1` (1), `JEEFS_SIG_SECP256R1` (2)
111+
- `JEEFSFileHeaderv1` (24B) — file entry: name (15 chars max), dataSize, CRC32, nextFileAddress
112+
- `EEPROMDescriptor` — opaque handle (file descriptor + EEPROM size)
113+
114+
**Python (python/jeefs/):**
115+
116+
- `EEPROMHeaderV3` — dataclass with `to_bytes()`, `from_bytes()`, `verify_crc()`, `validate()`, `to_partition_image()`
117+
- `SignatureAlgorithm` — IntEnum: `NONE` (0), `SECP192R1` (1), `SECP256R1` (2)
118+
- Constants: `EEPROM_FIELDS` (offset/size dict), `EEPROM_MAGIC`, `SIGNATURE_SIZES`
119+
120+
### Error Codes (include/eepromerr.h)
121+
122+
Negative return values are errors defined in `EEPROMError` enum: `FILEEXISTS`, `FILENAMETOOLONG`, `FILENOTFOUND`, `NOTENOUGHSPACE`, `EEPROMCORRUPTED`, `EEPROMREADERROR`, etc.
123+
124+
### Known Issues / TODOs in Code
125+
126+
- `defragEEPROM()` is declared in the header but not implemented in jeefs.c
127+
- Several `// TODO` markers for CRC validation of file data on read and write error checking
128+
- Binary test vectors (`test-vectors/`) not yet generated — planned for cross-language CI
129+
- Additional language implementations (Rust, Go, TypeScript) planned but not yet started
130+
131+
## CMake Options
132+
133+
| Option | Default | Description |
134+
|---|---|---|
135+
| `JEEFS_BUILD_TESTS` | ON | Build test executables |
136+
| `JEEFS_USEDYNAMIC_FILES` | ON | Build shared library |
137+
| `JEEFS_USE_EEPROMOPS_MEMORY` | ON | Use in-memory EEPROM backend |
138+
139+
## Repository Structure
140+
141+
```text
142+
include/
143+
jeefs.h # Core types, structs (v1/v2/v3), FS API declarations
144+
jeefs_header.h # Pure header API (no I/O dependency)
145+
eepromops.h # Hardware abstraction interface
146+
eepromerr.h # Error codes
147+
debug.h # Debug macros
148+
src/
149+
jeefs.c # FS implementation + header management (uses eepromops)
150+
jeefs_header.c # Pure header functions (uses zlib only)
151+
eepromops-memory/ # In-memory EEPROM backend
152+
srcpp/ # C++ wrapper
153+
python/
154+
jeefs/ # Python package: constants.py, header.py
155+
tests/ # pytest suite (58 tests)
156+
pyproject.toml # Package config (jeefs-header)
157+
tests/ # C test suite (ctest)
158+
EEPROM_FORMAT.md # Canonical binary format specification
159+
```
160+
161+
## CI/CD
162+
163+
- GitHub Actions workflows MUST use `runs-on: self-hosted` — never `ubuntu-latest` or other GitHub-hosted runners.
164+
165+
## Language/Style Notes
166+
167+
- C11 for core library, C++17 for wrapper
168+
- All public headers have `extern "C"` guards for C++ compatibility
169+
- Debug output controlled by `-DDEBUG` compile flag (uses `debug()` macro from `include/debug.h`)
170+
- SPDX license identifiers on all source files
171+
- Python: PEP 8, type hints, dataclasses

0 commit comments

Comments
 (0)