You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: Fix minor style/grammar issues in markdown files (#1931)
The docs were somewhat inconsistent with markdown formatting which
should be mostly cleaned up now.
The cleanup was focused on the libsql specific docs (meaning all
markdown files except in the `vendored`, `libsql-sqlite3`, and
`libsql-ffi/bundled` directories)
A `.markdownlint.yaml` file was also added to help with maintenance in
the future.
Prose should be mostly identical except for fixing grammar issues.
* Follow the Git commit message guidelines for consistent commit messages.
28
+
* Review the existing issues and pull requests to avoid duplicating efforts.
29
+
30
+
### Pull Request Process
31
+
32
+
* Ensure that your pull request addresses a specific issue.
33
+
* If there isn't an issue, consider creating one first.
34
+
* Clearly describe the problem or feature in your pull request and reference the relevant issue.
35
+
* Provide steps to reproduce and test your changes if applicable.
36
+
* Ensure that your branch is up-to-date with the latest changes from the main branch.
37
+
* All checks (tests, formatting, etc.) must pass before your pull request can be merged.
38
+
* After addressing review comments, your pull request will be merged by a project maintainer.
39
+
40
+
### Documentation
37
41
38
42
Help us improve the project's documentation by fixing typos, clarifying language, or adding missing information. Documentation improvements can be submitted as pull requests.
39
43
40
-
Reporting Issues
44
+
### Reporting Issues
41
45
42
46
If you encounter a bug or have a suggestion for the project, please open an issue using the provided issue template. Include as much detail as possible to help us understand and address the problem.
43
47
44
-
Getting Help
48
+
### Getting Help
45
49
46
50
If you need assistance, have questions, or want to discuss ideas, you can:
47
51
48
52
Join our Discord server and chat with the community.
49
53
Mention @maintainer in your issue or pull request for a faster response from project maintainers.
50
54
We appreciate your time and effort in contributing to libsql!
*[D](https://github.com/pdenapo/libsql-d) (experimental, based on the C driver)
68
72
69
73
### GUI Support
74
+
70
75
*[Beekeeper Studio](https://www.beekeeperstudio.io/db/libsql-client/)— macOS, Windows, and Linux
71
76
*[Outerbase](https://www.outerbase.com)— Runs in the browser
72
77
*[TablePlus](https://tableplus.com)— macOS, Windows, and Linux
@@ -95,6 +100,7 @@ libsql>
95
100
```
96
101
97
102
### Docker
103
+
98
104
To run libSQL using docker, refer to the [Docker Docs](docs/DOCKER.md)
99
105
100
106
## Why a fork?
@@ -105,15 +111,15 @@ But despite having its code available, SQLite famously doesn't accept external c
105
111
106
112
There have been other forks in the past, but they all focus on a specific technical difference. We aim to be a community where people can contribute from many different angles and motivations.
107
113
108
-
We want to see a world where everyone can benefit from all of the great ideas and hard work that the SQLite community contributes back to the codebase. Community contributions work well, because we’ve done it before. If this was possible, what do you think SQLite could become?
114
+
We want to see a world where everyone can benefit from all the great ideas and hard work that the SQLite community contributes back to the codebase. Community contributions work well, because we’ve done it before. If this was possible, what do you think SQLite could become?
109
115
110
116
You can read more about our goals and motivation in our [product vision](https://turso.tech/libsql-manifesto).
111
117
112
118
## Compatibility with SQLite
113
119
114
120
Compatibility with SQLite is of great importance for us. But it can mean many things. So here's our stance:
115
121
116
-
***The file format**: libSQL will always be able to ingest and write the SQLite file format. We would love to add extensions like encryption, and CRC that require the file to be changed. But we commit to always doing so in a way that generates standard sqlite files if those features are not used.
122
+
***The file format**: libSQL will always be able to ingest and write the SQLite file format. We would love to add extensions like encryption, and CRC that require the file to be changed. But we commit to always doing so in a way that generates standard SQLite files if those features are not used.
117
123
***The API**: libSQL will keep 100% compatibility with the SQLite API, but we may add additional APIs.
118
124
***Embedded**: SQLite is an embedded database that can be consumed as a single .c file with its accompanying header. libSQL will always be embeddable, meaning it runs inside your process without needing a network connection. But we may change the distribution, so that object files are generated, instead of a single .c file.
Copy file name to clipboardExpand all lines: bottomless/README.md
+32-11Lines changed: 32 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,40 +1,51 @@
1
1
# Bottomless S3-compatible virtual WAL for libSQL
2
-
##### Work in heavy progress!
2
+
3
+
## Work in heavy progress
3
4
4
5
This project implements a virtual write-ahead log (WAL) which continuously backs up the data to S3-compatible storage and is able to restore it later.
5
6
6
7
## How to build
7
-
```
8
+
9
+
```shell
8
10
LIBSQL_DIR=/path/to/your/libsql/directory make
9
11
```
12
+
10
13
will produce a loadable `.so` libSQL extension with bottomless WAL implementation.
11
-
```
14
+
15
+
```shell
12
16
LIBSQL_DIR=/path/to/your/libsql/directory make release
13
17
```
18
+
14
19
will do the same, but for release mode.
15
20
16
21
## Configuration
17
-
By default, the S3 storage is expected to be available at `http://localhost:9000` (e.g. a local development [minio](https://min.io) server), and the auth information is extracted via regular S3 SDK mechanisms, i.e. environment variables and `~/.aws/credentials` file, if present. Ref: https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials_environment.html
22
+
23
+
By default, the S3 storage is expected to be available at `http://localhost:9000` (e.g. a local development [minio](https://min.io) server), and the auth information is extracted via regular S3 SDK mechanisms, i.e. environment variables and `~/.aws/credentials` file, if present. Ref: <https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials_environment.html>
18
24
19
25
Default endpoint can be overridden by an environment variable too, and in the future it will be available directly from libSQL as an URI parameter:
Bucket used for replication can be configured with:
25
-
```
32
+
33
+
```shell
26
34
export LIBSQL_BOTTOMLESS_BUCKET='custom-bucket'
27
35
```
28
36
29
37
On top of that, bottomless is implemented on top of the official [Rust SDK for S3](https://crates.io/crates/aws-sdk-s3), so all AWS-specific environment variables like `AWS_DEFAULT_REGION`, `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` also work, as well as the `~/.aws/credentials` file.
30
38
31
39
## How to use
40
+
32
41
From libSQL shell, load the extension and open a database file with `bottomless` WAL, e.g.:
42
+
33
43
```sql
34
44
.load ../target/debug/bottomless
35
45
.open file:test.db?wal=bottomless
36
46
PRAGMA journal_mode=wal;
37
47
```
48
+
38
49
Remember to set the journaling mode to `WAL`, which needs to be done at least once, before writing any content, otherwise the custom WAL implementation will not be used.
39
50
40
51
In order to customize logging, use `RUST_LOG` env variable, e.g. `RUST_LOG=info ./libsql`.
@@ -46,14 +57,18 @@ LIBSQL_DIR=/path/to/your/libsql/directory make test
46
57
```
47
58
48
59
## CLI
60
+
49
61
The command-line interface supports browsing, restoring and removing snapshot generations.
50
62
It can be installed as a standalone executable with:
All page writes committed to the database end up being asynchronously replicated to S3-compatible storage.
125
144
On boot, if the main database file is empty, it will be restored with data coming from the remote storage.
126
145
If the database file is newer, it will be uploaded to the remote location with a new generation number.
127
146
If a local WAL file is present and detected to be newer than remote data, it will be uploaded as well.
128
147
129
148
### Tests
149
+
130
150
A fully local test can be performed by using a local S3-compatible server, e.g. [Minio](https://min.io/). Assuming the server is available at HTTP port 9000,
Copy file name to clipboardExpand all lines: docs/ADMIN_API.md
+7-3Lines changed: 7 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,29 +5,33 @@ This document describes the admin API endpoints.
5
5
The admin API is used to manage namespaces on a `sqld` instance. Namespaces are isolated database within a same sqld instance.
6
6
7
7
To enable the admin API, and manage namespaces, two extra flags need to be passed to `sqld`:
8
+
8
9
-`--admin-listen-addr <addr>:<port>`: the address and port on which the admin API should listen. It must be different from the user API listen address (whi defaults to port 8080).
9
10
-`--enable-namespaces`: enable namespaces for the instance. By default namespaces are disabled.
0 commit comments