Skip to content

Commit 1f37eee

Browse files
committed
feat: introduce closure-based transaction API
See CHANGELOG.md for more information
1 parent f106bcc commit 1f37eee

24 files changed

+1816
-1724
lines changed

.github/workflows/ci.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@ name: Cross-platform CI
22

33
on:
44
push:
5-
branches: [ main, master ]
5+
branches: [ master ]
66
pull_request:
7-
branches: [ main, master ]
7+
branches: [ master ]
88

99
env:
1010
CARGO_TERM_COLOR: always

CHANGELOG.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Changelog
2+
3+
All notable changes to the **btree_store** project will be documented in this file.
4+
5+
## [0.1.1] - 2026-01-18
6+
7+
### Added
8+
- **Atomic Multi-Bucket Transactions**: Introduced `exec_multi` API and `MultiTxn` handle. This allows performing multiple operations across different buckets in a single atomic transaction with only one disk sync, significantly improving batch performance.
9+
- **Enhanced Data Validation**: Reinforced `Node::validate` with physical invariant checks.
10+
- **Torn Write Detection**: Enhanced `MetaNode::validate` to identify and reject zeroed-out blocks caused by power failures during I/O.
11+
- **8-Byte Memory Alignment**: Guaranteed alignment for zero-copy serialization via `AlignedPage`.
12+
13+
### Changed
14+
- **Closure-based Transaction API**: Introduced `exec` and `view` methods.
15+
- **Unified Reclamation**: Consolidated page release via `Store::free_pages`.
16+
- **Log Management**: Switched to `.pending` log truncation (`set_len(0)`).
17+
- **Auto-Refresh**: Implicit superblock sync at transaction start.
18+
- **API Simplification**: Continued the transition to closure-based APIs across all internal and external logic.
19+
- **Shared Transaction State**: Clones share `pending` containers within the same process.
20+
21+
### Fixed
22+
- **Double-Write Protocol**: Refined the superblock update sequence to guarantee zero-leak and zero-corruption recovery by splitting the commit into two distinct disk-sync phases.

Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "btree-store"
3-
version = "0.1.0"
3+
version = "0.1.1"
44
edition = "2024"
55
authors = ["AbbyCin <abbytsing@gmail.com>"]
66
readme = "README.md"

README.md

Lines changed: 49 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,57 +1,70 @@
11
# btree_store
22

3-
**btree_store** is a high-performance, persistent, embedded key-value storage engine written in Rust. It implements a robust Copy-On-Write (COW) B-Tree architecture to ensure data integrity, crash safety, and efficient concurrent access.
3+
**btree_store** is a persistent, embedded key-value storage engine written in Rust. It implements a robust Copy-On-Write (COW) B-Tree architecture to ensure data integrity, crash safety, and efficient concurrent access.
44

55
## Features
66

77
* **ACID Compliance:** Atomic commits using COW, Snapshot Isolation, and double-buffered meta pages.
8-
* **Conflict Detection:** Built-in "First-Committer-Wins" strategy ensures data integrity in multi-process environments.
9-
* **Crash Safety:** A dedicated `.pending` log recovery mechanism reclaims leaked pages from interrupted transactions.
10-
* **Buckets:** Logical namespaces for data separation within a single database file.
11-
* **Zero-Allocation Iterators:** High-performance iterators returning `(&[u8], &[u8])` with internal buffer reuse.
12-
* **Efficient Concurrency:** Shared bucket handles and Snapshot Isolation (SI) using `parking_lot` for scalable access.
13-
* **Data Integrity:** Full CRC32C checksum validation for every page (nodes, meta, and free lists).
14-
* **Zero External Dependencies:** Core engine built using native Rust standard library.
8+
* **Closure-based Transactions:** Simplified `exec` (read-write) and `view` (read-only) APIs with automatic commit and rollback.
9+
* **Auto-Refresh:** Every transaction automatically starts from the freshest disk state. No manual snapshot management required.
10+
* **Conflict Detection:** Built-in "First-Committer-Wins" strategy for concurrent handles.
11+
* **Batch Operations:** `exec_multi` for atomic updates across multiple buckets with a single disk sync, significantly reducing I/O overhead.
12+
* **Crash Safety:** Double-write superblock updates and a `.pending` log recovery mechanism ensure zero-leak recovery even from torn writes.
13+
* **Logical Namespaces:** Direct support for multiple buckets within a single database file.
14+
* **Zero-Copy Access:** 8-byte aligned memory layouts allow direct pointer-to-reference conversion for maximum performance.
15+
* **Robust Data Integrity:** Strengthened physical invariant checks and CRC32C checksum validation for every node and metadata page.
16+
* **Shared Transaction State:** `clone()` creates a new handle that shares the same transaction context, optimized for multi-threaded components.
17+
18+
> **Warning:** Multi-process concurrent access is NOT supported. Only one process should access the database file at a time.
1519
1620
## Architecture
1721

18-
* **Store (`src/store.rs`):** Low-level page management, sharded LRU caching, and positional I/O.
19-
* **Node (`src/node.rs`):** B-Tree node serialization, splitting, compacting, and checksumming.
20-
* **Tree (`src/lib.rs`):** Core B+ Tree algorithms and Snapshot Isolation logic.
21-
* **Bucket (`src/lib.rs`):** High-level API with shared handle caching.
22+
* **Store (`src/store.rs`):** Low-level page management, sharded LRU caching with mandatory invalidation on sync, and positional I/O.
23+
* **Node (`src/node.rs`):** 8-byte aligned memory management (`AlignedPage`), zero-copy serialization, and checksumming.
24+
* **Tree Logic (`src/lib.rs`):** Core B+ Tree algorithms and Transactional Snapshot Isolation logic.
2225

2326
## Usage
2427

2528
Add this to your `Cargo.toml`:
2629

2730
```toml
2831
[dependencies]
29-
btree-store = "0.1.0"
32+
btree-store = "0.1.1"
3033
```
3134

32-
### Basic Example (with Conflict Handling)
35+
### Basic Example
3336

3437
```rust
3538
use btree_store::{BTree, Error};
3639

3740
fn main() -> Result<(), Error> {
3841
let db = BTree::open("data.db")?;
39-
let bucket = db.get_bucket("users").or_else(|_| db.new_bucket("users"))?;
40-
41-
loop {
42-
// Perform operations
43-
bucket.put(b"id:100", b"Alice")?;
44-
45-
// Commit changes
46-
match db.commit() {
47-
Ok(_) => break,
48-
Err(Error::Conflict) => {
49-
// Another instance committed first, refresh and retry
50-
db.refresh()?;
51-
}
52-
Err(e) => return Err(e),
53-
}
54-
}
42+
43+
// Read-Write Transaction
44+
db.exec("users", |txn| {
45+
txn.put(b"id:100", b"Alice")?;
46+
let val = txn.get(b"id:100")?;
47+
assert_eq!(val, b"Alice");
48+
Ok(())
49+
})?;
50+
51+
// Read-Only View
52+
db.view("users", |txn| {
53+
let val = txn.get(b"id:100")?;
54+
println!("User: {:?}", String::from_utf8_lossy(&val));
55+
Ok(())
56+
})?;
57+
58+
// Multi-Bucket Atomic Transaction
59+
db.exec_multi(|multi| {
60+
multi.execute("users", |txn| {
61+
txn.put(b"id:101", b"Bob")
62+
})?;
63+
multi.execute("stats", |txn| {
64+
txn.put(b"total_users", b"2")
65+
})?;
66+
Ok(())
67+
})?;
5568

5669
Ok(())
5770
}
@@ -60,18 +73,17 @@ fn main() -> Result<(), Error> {
6073
## Performance Design
6174

6275
The engine is optimized for high-throughput scenarios:
63-
* **Snapshot Isolation (SI)** allows concurrent readers and writers to operate without blocking each other.
64-
* **Bucket Handle Caching** ensures all threads share the same metadata locks, preventing structural corruption.
65-
* **Cursor-based path traversal** avoids redundant tree searches.
66-
* **RAII CommitContext** ensures zero-leak automatic rollbacks on failure.
67-
* **Transmute-based lifetime extension** in iterators allows returning direct references to internal buffers under a read lock.
76+
* **8-Byte Alignment:** Every page is allocated with 8-byte alignment, allowing direct casting of raw bytes to internal structures without memory copies.
77+
* **Snapshot Isolation (SI):** Readers and writers operate on stable snapshots without blocking each other (non-blocking reads).
78+
* **Automatic Page Reclamation:** Failed or conflicted transactions automatically trigger page reclamation to prevent database bloat.
79+
* **Transmute-based lifetime extension:** Iterators return direct references to internal buffers under a read lock, achieving near-zero allocation.
6880

6981
## Testing
7082

7183
The project includes a comprehensive suite of integration tests:
7284
* `smo_stress_test`: Structural Modification Operations under heavy load.
7385
* `crash_safety_tests`: Verifies data integrity across simulated crashes.
74-
* `concurrency_tests`: Parallel readers and writers.
86+
* `concurrency_tests`: Parallel readers and writers with auto-refresh validation.
7587
* `leak_safety_tests`: Ensures no pages are lost during failed operations.
7688

7789
Run tests with:
@@ -81,4 +93,4 @@ cargo test
8193

8294
## License
8395

84-
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
96+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

0 commit comments

Comments
 (0)