Skip to content

Commit f10655f

Browse files
committed
Major performance tweaks, functional directory hash, parallel processing, updated readme
1 parent 010f4f2 commit f10655f

File tree

4 files changed

+596
-202
lines changed

4 files changed

+596
-202
lines changed

readme.md

Lines changed: 73 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,19 +18,30 @@ Bound Book Format (.bbf) is a high-performance binary container designed specifi
1818
### Compilation
1919
Linux
2020
```bash
21-
g++ -std=c++17 bbfenc.cpp libbbf.cpp xxhash.c -o bbfmux
21+
g++ -std=c++17 bbfenc.cpp libbbf.cpp xxhash.c -o bbfmux -pthread
2222
```
2323

2424
Windows
2525
```bash
26-
g++ -std=c++17 bbfmux.cpp libbbf.cpp xxhash.c -o bbfmux -municode
26+
g++ -std=c++17 bbfenc.cpp libbbf.cpp xxhash.c -o bbfmux -municode
2727
```
2828

2929
---
3030

3131
## Technical Details
32+
3233
BBF is designed as a Footer-indexed binary format. This allows for rapid append-only creation and immediate random access to any page without scanning the entire file.
3334

35+
### Zero-Copy Architecture
36+
The `bbfmux` reference implementation utilizes **Memory Mapping (mmap/MapViewOfFile)**. Instead of reading file data into intermediate buffers, the tool maps the container directly into the process address space. This allows the CPU to access image data at the speed of your NVMe drive's hardware limit.
37+
38+
### High-Speed Parallel Verification
39+
Integrity checks now utilize **Parallel XXH3**. On multi-core systems, the verifier splits the asset table into chunks and validates multiple pages simultaneously. This makes BBF verification up to **10x faster** than ZIP/RAR CRC checks.
40+
41+
### 4KB Alignment
42+
Every asset in a BBF file starts on a **4096-byte boundary**. This alignment is critical for modern hardware, allowing for DirectStorage transfers directly from disk to GPU memory, bypassing CPU bottlenecks entirely.
43+
44+
3445
### Binary Layout
3546
1. **Header (13 bytes)**: Magic `BBF1`, versioning, and initial padding.
3647
2. **Page Data**: The raw image payloads (AVIF, PNG, etc.), each padded to **4096-byte boundaries**.
@@ -68,9 +79,6 @@ BBF is designed as a Footer-indexed binary format. This allows for rapid append-
6879
[7] - PDF supports "Linearization" (Fast Web View), allowing the header and first pages to be read before the rest of the file is downloaded.<br/>
6980
</font>
7081

71-
### 4KB Alignment & DirectStorage
72-
Every asset in a BBF file starts on a 4KB boundary. This alignment is critical for modern NVMe-based systems. It allows developers to utilize `mmap` or **DirectStorage** to transfer image data directly from disk to GPU memory, bypassing the CPU-bottlenecked "copy and decompress" cycles found in Zip-based formats.
73-
7482
---
7583

7684
## Features
@@ -156,5 +164,65 @@ bbfmux input.bbf --info
156164

157165
---
158166

167+
## Advanced CLI Features
168+
169+
### Custom Page Ordering (`--order`)
170+
You can precisely control the reading order using a text file or inline arguments.
171+
* **Positive Integers**: Fixed 1-based index (e.g., `cover.png:1`).
172+
* **Negative Integers**: Fixed position from the end (e.g., `credits.png:-1` is always the last page).
173+
* **Unspecified**: Sorted alphabetically between the fixed pages.
174+
175+
```bash
176+
# Using an order file
177+
bbfmux ./images/ --order=pages.txt out.bbf
178+
179+
# pages.txt example:
180+
cover.png:1
181+
page1.png:2
182+
page2.png:3
183+
credits.png:-1
184+
```
185+
186+
### Batch Section Import (`--sections`)
187+
Sections define Chapters or Volumes. You can target a page by its index or filename.
188+
```bash
189+
# Target by filename
190+
bbfmux ./folder/ --section="Chapter 1":"001.png" out.bbf
191+
192+
# Using a sections file
193+
bbfmux ./folder/ --sections=sectionexample.txt out.bbf
194+
195+
# sectionexample.txt example (Name:Target[:Parent]):
196+
"Volume 1":"001.png"
197+
"Chapter 1":"001.png":"Volume 1"
198+
"Chapter 2":"050.png":"Volume 1"
199+
```
200+
201+
### Targeted Verification
202+
BBF allows for verification of data to detect bit-rot.
203+
```bash
204+
# Verify everything (All assets and Directory structure)
205+
bbfmux input.bbf --verify
206+
207+
# Verify only the directory hash (Instant)
208+
bbfmux input.bbf --verify -1
209+
210+
# Verify a specific asset by index
211+
bbfmux input.bbf --verify 42
212+
```
213+
214+
### Range-Key Extraction
215+
The `--rangekey` option allows you to extract a range of sections. The extractor starts at the specified `--section` and stops when it finds a section whose title matches the `rangekey`.
216+
217+
```bash
218+
# Extract Chapter 2 up until it hits Chapter 4
219+
bbfmux manga.bbf --extract --section="Chapter 2" --rangekey="Chapter 4" --outdir="./Ch2_to_Ch4"
220+
221+
# Extract Volume 2 until it encounters the string "Chapter 60"
222+
bbfmux manga.bbf --extract --section="Volume 2" --rangekey="Chapter 60"
223+
```
224+
225+
---
226+
159227
## License
160228
Distributed under the MIT License. See `LICENSE` for more information.

0 commit comments

Comments
 (0)