Skip to content

Commit 7cb37d9

Browse files
CieleconCielecon
authored
updated for V1.5
1 parent 18efdb2 commit 7cb37d9

File tree

1 file changed

+258
-87
lines changed

1 file changed

+258
-87
lines changed

‎README.md‎

Lines changed: 258 additions & 87 deletions
Original file line numberDiff line numberDiff line change
@@ -1,134 +1,305 @@
11
# dbcake
22

3-
`dbcake` — tiny single-file key/value database using `.dbce` files.
4-
5-
Features
6-
- Simple Python API: `dbcake.db.set("k", val)`, `dbcake.db.get("k")`
7-
- Choose storage format: `binary`, `bits01` (ASCII '0'/'1'), `dec` (3-digit per byte), `hex`
8-
- `.dbce` files include a small header to identify them
9-
- Three security levels via `db.pw`:
10-
- `low` / `normal` — plain storage
11-
- `high` — encrypted storage (AES-GCM via `cryptography` if installed; stdlib fallback if not)
12-
- Interactive CLI with `create`, `set`, `get`, `preview`, `compact`, `export`, `set-passphrase`, `set-format`, `rotate-key`, etc.
13-
- Interactive passphrase prompts (no echo) for CLI
14-
- Key rotation: re-encrypt the whole DB with a new passphrase/key
15-
- Cross-process file locking (POSIX `fcntl` and Windows `msvcrt`) for safety on Linux/macOS/Windows
16-
- Single-file module — drop `dbcake.py` into your project
17-
18-
> **Security note**: For production encryption, install `cryptography`:
19-
> ```bash
20-
> pip install cryptography
21-
> ```
22-
> The stdlib fallback provides an authenticated XOR-like stream cipher which is educational but not a substitute for vetted crypto libraries.
3+
**dbcake** — single-file, easy-to-use key/value database + secrets client for learning, quick prototypes, and small projects.
234

24-
---
25-
26-
## Quick install
5+
`dbcake.py` is a self-contained Python module that provides:
276

28-
Place `dbcake.py` next to your Python script, or `git clone` the repo and `import dbcake`.
7+
- Local key/value store in a single append-only `.dbce` file (centralized) **or** per-key files (decentralized).
8+
- Multiple on-disk formats: `binary`, `bits01`, `dec`, `hex`.
9+
- Encryption modes: `low | normal | high`. Uses AES-GCM (via `cryptography`) when available; otherwise a secure stdlib fallback.
10+
- Key rotation, file-locking for multi-process safety, compaction, export, preview, and per-key operations.
11+
- A small HTTP secrets client (`Client` / `AsyncClient`) for talking to a remote secrets API (optional).
12+
- CLI for DB + secrets client and a tiny Tkinter GUI installer for optional packages.
13+
- Single-file distribution — `dbcake.py` — drop into a project and import or call from command line.
2914

30-
Install `cryptography` (recommended):
15+
---
3116

32-
```bash
33-
pip install cryptography
34-
```
17+
## Table of contents
18+
19+
- [Quick start](#quick-start)
20+
- [Installation](#installation)
21+
- [Basic usage (Python API)](#basic-usage-python-api)
22+
- [Storage formats & modes](#storage-formats--modes)
23+
- [Encryption, passphrases & key rotation](#encryption-passphrases--key-rotation)
24+
- [CLI usage](#cli-usage)
25+
- [Secrets HTTP client](#secrets-http-client)
26+
- [Examples: local server (for testing client)](#examples-local-server-for-testing-client)
27+
- [Security notes](#security-notes)
28+
- [Troubleshooting](#troubleshooting)
29+
- [Contributing](#contributing)
30+
- [License](#license)
3531

3632
---
3733

38-
## Basic usage (Python)
34+
## Quick start
3935

40-
```py
41-
import dbcake
36+
1. Save `dbcake.py` into your project folder.
4237

43-
# set default DB file and storage format:
44-
dbcake.db.title("mydata.dbce", store_format="binary") # formats: binary, bits01, dec, hex
38+
2. Use the module-level `db` object or create your own database instance:
39+
40+
```python
41+
import dbcake
4542

46-
# set/get plain values:
43+
# simple use (module-level default DB file: data.dbce)
4744
dbcake.db.set("username", "armin")
48-
print(dbcake.db.get("username"))
45+
print(dbcake.db.get("username")) # -> "armin"
4946

50-
# change storage format:
51-
dbcake.db.set_format("bits01") # stores record payloads as ASCII '0'/'1' strings like '101000010...'
47+
# create/open a custom DB file
48+
mydb = dbcake.open_db("project.dbce", store_format="binary", dataset="centerilized")
49+
mydb.set("score", 100)
50+
print(mydb.get("score"))
51+
```
52+
# Installation
5253

53-
# enable secure mode:
54-
dbcake.db.set_passphrase("my secret") # in-memory only; or leave unset and db will use a keyfile
55-
dbcake.db.pw = "high"
56-
dbcake.db.set("secret", {"pin": 1234})
57-
print(dbcake.db.get("secret"))
54+
dbcake.py is a single-file module — no installation required beyond having Python.
5855

59-
# rotate key (programmatically)
60-
# rotate to a new passphrase (you must be in high mode and have the current passphrase set)
61-
dbcake.db.rotate_key(new_passphrase="my new passphrase")
56+
Optional (recommended) packages:
6257

63-
# compact to rewrite file and reduce size
64-
dbcake.db.compact()
58+
cryptography — provides AES-GCM & Fernet support (stronger, standard crypto).
6559

66-
# close when done
67-
dbcake.db.close()
60+
tkinter — required only if you want to run the graphical installer.
61+
62+
# Install cryptography:
63+
```bash
64+
python -m pip install cryptography
6865
```
66+
Run the GUI installer (uses tkinter) to install optional packages:
67+
```bash
68+
python dbcake.py --installer
69+
```
70+
# Basic usage (Python API)
6971

70-
---
72+
Module-level convenience DB
73+
```python
74+
import dbcake
7175

72-
## CLI usage
76+
# default DB (data.dbce)
77+
dbcake.db.set("a", 123)
78+
print(dbcake.db.get("a")) # -> 123
7379

74-
Run `python dbcake.py <command> [args]`:
80+
# change file & format
81+
dbcake.db.title("mydata.dbce", store_format="binary")
82+
dbcake.db.set("user", {"name": "alice"})
83+
print(dbcake.db.get("user"))
7584

76-
Examples:
85+
# switch to decentralized per-key files
86+
dbcake.db.decentralized()
87+
dbcake.db.set("session", {"id": 1})
7788

78-
Create file:
79-
```bash
80-
python dbcake.py create mydata.dbce --format dec
81-
```
89+
# list keys
90+
print(dbcake.db.keys())
8291

83-
Set key:
84-
```bash
85-
python dbcake.py set mydata.dbce username armin
92+
# preview a few entries
93+
print(dbcake.db.preview(limit=5))
94+
dbcake.db._backend.pretty_print_preview(limit=5) # helper that prints nice table
8695
```
87-
88-
Get key:
89-
```bash
90-
python dbcake.py get mydata.dbce username
96+
Factory style (explicit DB object)
97+
```python
98+
mydb = dbcake.open_db("project.dbce", store_format="hex", dataset="centerilized")
99+
mydb.set("k", "v")
100+
v = mydb.get("k")
91101
```
102+
# Storage formats & modes
92103

93-
Preview:
94-
```bash
95-
python dbcake.py preview mydata.dbce --limit 20
96-
```
104+
store_format options when creating or switching DB:
97105

98-
Compact:
99-
```bash
100-
python dbcake.py compact mydata.dbce
106+
binary — raw bytes (fast).
107+
108+
bits01 — ASCII '0' / '1' bit string.
109+
110+
dec — decimal digits grouped by 3 per byte.
111+
112+
hex — hex representation.
113+
114+
Switch format programmatically:
115+
```python
116+
dbcake.db.set_format("hex")
117+
```
118+
Switch dataset mode:
119+
```python
120+
dbcake.db.centerilized() # centralized append-only .dbce
121+
dbcake.db.decentralized() # per-key files in .d directory
101122
```
123+
# Encryption, passphrases & key rotation
102124

103-
Set passphrase (interactive, no echo):
104-
```bash
105-
python dbcake.py set-passphrase mydata.dbce --interactive
125+
db.pw controls on-disk security:
126+
127+
db.pw = "low" — minimal (fast).
128+
129+
db.pw = "normal" — default (no re-encryption).
130+
131+
db.pw = "high" — records encrypted before writing (AES-GCM if cryptography is installed; otherwise a fallback).
132+
133+
Set passphrase (derive key from passphrase):
134+
```python
135+
dbcake.db.pw = "high"
136+
dbcake.db.set_passphrase("my secret passphrase")
137+
dbcake.db.set("secret", "value")
106138
```
139+
Generate/store keyfile (if you do not use passphrase) — DB will generate .key next to the DB file.
107140

108-
Rotate key (interactive):
141+
Rotate keys (re-encrypt everything):
142+
143+
CLI (interactive):
109144
```bash
110-
python dbcake.py rotate-key mydata.dbce --interactive
145+
python dbcake.py db rotate-key mydata.dbce --interactive
146+
```
147+
**Programmatic:**
148+
```python
149+
dbcake.db.set_passphrase("old")
150+
dbcake.db.rotate_key(new_passphrase="new")
111151
```
152+
rotate_key rewrites the DB and re-encrypts records under the new key.
112153

113-
Switch storage format:
154+
# CLI usage
155+
156+
The single file exposes a CLI for both local DB and the secrets client.
157+
158+
# Local DB commands
114159
```bash
115-
python dbcake.py set-format mydata.dbce hex
160+
# create file
161+
python dbcake.py db create mydata.dbce --format binary
162+
163+
# set key
164+
python dbcake.py db set mydata.dbce username '"armin"'
165+
166+
# get key
167+
python dbcake.py db get mydata.dbce username
168+
169+
# list keys
170+
python dbcake.py db keys mydata.dbce
171+
172+
# preview
173+
python dbcake.py db preview mydata.dbce --limit 5
174+
175+
# compact (rewrite to keep only current items)
176+
python dbcake.py db compact mydata.dbce
177+
178+
# set passphrase (interactive)
179+
python dbcake.py db set-passphrase mydata.dbce --interactive
180+
181+
# rotate key (interactive)
182+
python dbcake.py db rotate-key mydata.dbce --interactive
183+
184+
# reveal DB file in OS file manager
185+
python dbcake.py db reveal mydata.dbce
116186
```
187+
CLI values attempt JSON parsing; unparseable input will be stored as raw string.
117188

118-
Reveal DB in file manager:
189+
# Secrets HTTP client (CLI)
119190
```bash
120-
python dbcake.py reveal mydata.dbce
191+
# set secret
192+
python dbcake.py secret set myname "value" --url https://secrets.example.com --api-key S3CR
193+
194+
# get secret (reveal)
195+
python dbcake.py secret get myname --reveal --url https://secrets.example.com --api-key S3CR
196+
197+
# list
198+
python dbcake.py secret list --url https://secrets.example.com --api-key S3CR
199+
200+
# delete
201+
python dbcake.py secret delete myname --url https://secrets.example.com --api-key S3CR
202+
```
203+
# Secrets HTTP client (Python)
204+
```python
205+
from dbcake import Client
206+
207+
client = Client("https://secrets.example.com", api_key="S3CR")
208+
meta = client.set("db_token", "S3cR3tV@lue", tags=["prod","db"])
209+
secret = client.get("db_token", reveal=True)
210+
print(secret.value)
211+
212+
# With Fernet (encrypt locally before send)
213+
from cryptography.fernet import Fernet
214+
fkey = Fernet.generate_key().decode()
215+
client2 = Client("https://secrets.example.com", api_key="S3", fernet_key=fkey)
216+
client2.set("encrypted", "very-secret")
217+
s = client2.get("encrypted", reveal=True)
218+
print(s.value)
121219
```
220+
AsyncClient is available for async code (AsyncClient.from_env() to read env vars).
221+
222+
Env vars for convenience: DBCAKE_URL, DBCAKE_API_KEY, DBCAKE_FERNET_KEY.
223+
224+
Examples: local server (for testing client)
225+
226+
Below is a tiny example (not included in dbcake.py) of a simple HTTP test server you can use to exercise the Client:
227+
```python
228+
# tiny_test_server.py (example only; not production-ready)
229+
from http.server import BaseHTTPRequestHandler, HTTPServer
230+
import json, urllib.parse
231+
232+
STORE = {}
233+
234+
class Handler(BaseHTTPRequestHandler):
235+
def _send(self, code, data):
236+
self.send_response(code)
237+
self.send_header("Content-Type", "application/json")
238+
self.end_headers()
239+
self.wfile.write(json.dumps(data).encode())
240+
241+
def do_POST(self):
242+
if self.path == "/secrets":
243+
length = int(self.headers.get("Content-Length", 0))
244+
body = self.rfile.read(length)
245+
doc = json.loads(body.decode("utf-8"))
246+
name = doc["name"]
247+
STORE[name] = doc
248+
now = "2025-10-16T00:00:00Z"
249+
self._send(201, {"name": name, "created_at": now, "updated_at": now, "tags": doc.get("tags", [])})
250+
return
251+
self.send_error(404)
252+
253+
def do_GET(self):
254+
if self.path.startswith("/secrets"):
255+
parsed = urllib.parse.urlparse(self.path)
256+
parts = parsed.path.split("/")
257+
if len(parts) == 3 and parts[2]:
258+
name = parts[2]
259+
item = STORE.get(name)
260+
if not item:
261+
self.send_error(404)
262+
return
263+
reveal = urllib.parse.parse_qs(parsed.query).get("reveal", [])
264+
doc = item.copy()
265+
self._send(200, doc)
266+
return
267+
self.send_error(404)
268+
269+
def do_DELETE(self):
270+
parts = self.path.split("/")
271+
if len(parts) == 3 and parts[2]:
272+
name = parts[2]
273+
if name in STORE:
274+
del STORE[name]
275+
self._send(204, {})
276+
return
277+
self.send_error(404)
278+
279+
if __name__ == '__main__':
280+
server = HTTPServer(("localhost", 8000), Handler)
281+
print("Listening on http://localhost:8000")
282+
server.serve_forever()
283+
```
284+
# Security notes
122285

123-
---
286+
If you use db.set_passphrase("..."), a salt file (.salt) is created and used to derive an encryption key. Keep passphrases secret.
287+
288+
If you don't set a passphrase, the DB generates a .key keyfile next to the DB. Keep that file safe.
289+
290+
Use TLS for server communication (HTTPS) and protect API keys.
291+
292+
rotate_key rewrites and re-encrypts stored data — use it regularly for long-lived data.
293+
294+
This project is intended for learning and small projects. For production secrets management, consider hardened solutions (HashiCorp Vault, AWS KMS/Secrets Manager, etc.).
295+
# Troubleshooting
296+
297+
cryptography not installed — AES-GCM and Fernet features disabled; library will use a secure fallback. Install cryptography if you need standard AES-GCM/Fernet.
124298

125-
## Notes & tips
299+
tkinter missing — GUI installer will not run. Install system package (e.g., python3-tk) or use pip-installed packages via CLI.
126300

127-
- Use `db.pw = "high"` and `db.set_passphrase(...)` to enable encryption. `set_passphrase` stores the passphrase only in memory — the CLI offers an interactive prompt so you don't have to put passphrases in shell history.
128-
- The `rotate-key` operation requires the ability to decrypt the current data (i.e., you must provide the current passphrase if the in-memory key isn't present). Rotation writes a new file and replaces the old file atomically (best-effort).
129-
- The module uses a coarse-grained file lock for multi-process safety. This works well for small-to-medium apps and scripts; for heavy concurrent workloads use a real DB server.
130-
- For portability and maximum security, install `cryptography` — AES-GCM is used for encryption when available.
131-
- If you choose `bits01` or `dec` formats, the on-disk bytes become ASCII strings (e.g. `10100010...` or `065003...`), which you requested; the API still operates with Python objects.
301+
Lock timeouts — another process may hold the DB. Wait or increase timeout; ensure only compatible writers access the DB.
132302

303+
Permission errors — ensure the process can write to DB folder and key/salt files.
133304
>[!CAUTION]
134305
>please read LICENSE and ©️ copyright by Cielecon all rights reversed.

0 commit comments

Comments
 (0)