Skip to content

Commit 953158b

Browse files
committed
add: README.md
1 parent a5272d3 commit 953158b

File tree

1 file changed

+76
-0
lines changed

1 file changed

+76
-0
lines changed

README.md

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
# github-metadata-backup
2+
3+
Download issues and pull-requests from the GitHub API and store as JSON files.
4+
Supports incremental updates. The tool is written in Rust. Please
5+
6+
## Usage
7+
8+
The GitHub API is rate-limted to 60 requests per hour without authentication. This
9+
tool requires a [GitHub Personal Access Token], which increases the rate-limit to
10+
5000 requests per hour. A _classic_ token is recommended over the newer _fine-grained
11+
tokens_. Some of the events might be missing from the backup with the _fine-grained
12+
tokens_. See https://github.com/orgs/community/discussions/58534 for more information.
13+
14+
[GitHub Personal Access Token]: (https://github.com/settings/tokens)
15+
16+
The token an either be supplied to the `github-metadata-backup` binary via the
17+
`-p`/`--personal-access-token` command line option or placed in a file and read
18+
from there with the `-f`/`--personal-access-token-file` option. The tool also
19+
requires an `--owner` (a GitHub user or organization) and the `--repo` (the
20+
repository of that owner) to download the metadata from. The backup will be
21+
placed in the directory defined with `-d`/`--destination`. Make sure the tool
22+
has the required permissions to write to it.
23+
24+
```
25+
Usage: github-metadata-backup [OPTIONS] --owner <OWNER> --repo <REPO> --destination <PATH>
26+
27+
Options:
28+
-o, --owner <OWNER>
29+
Owner of the repository to backup
30+
-r, --repo <REPO>
31+
Name of the repository to backup
32+
-p, --personal-access-token <PERSONAL_ACCESS_TOKEN>
33+
Personal Access Token to the GitHub API supplied via the command line
34+
-f, --personal-access-token-file <PATH>
35+
Personal Access Token to the GitHub API read from a file
36+
-d, --destination <PATH>
37+
Destination where the backup should be written to
38+
-h, --help
39+
Print help
40+
-V, --version
41+
Print version
42+
```
43+
44+
The log-level can be controlled with the `RUST_LOG` environment variable. By
45+
default, it's `RUST_LOG=info`.
46+
47+
## Example
48+
49+
To backup the metadata from the `bitcoin/bitcoin` repository to the `bitcoin-bitcoin`
50+
directory while reading the file from the `read-only-github-access-token.sec` file the
51+
following command can be used:
52+
53+
```
54+
github-metadata-backup --owner bitcoin --repo bitcoin --destination bitcoin-bitcoin --personal-access-token-file read-only-github-access-token.sec
55+
```
56+
57+
This creates the `bitcoin-bitcoin` directory with the `issues` and `pulls`
58+
subdirectories. It requests metadata until the rate-limit is reached, waits
59+
until requests are allowed again, and then continues. Once finished with the
60+
initial backup, which can take a few hours with large repositories, it writes
61+
a `state.json` file. On subsequent runs, this file is read and only an
62+
incremental backup is made. To do a full backup again, delete the state.json
63+
file. The old backup will be overwritten. The JSON files are formatted to be
64+
easily trackable in git. It makes sense to commit each incremental backup.
65+
66+
## Nix Package and module
67+
68+
A Nix package and module for the github-metadata-backup tool are avaliable in
69+
my personal [nix-package-collection]. The package can be run with:
70+
71+
```
72+
nix run github:0xb10c/nix#github-metadata-backup
73+
```
74+
75+
[nix-package-collection]: https://github.com/0xB10C/nix
76+

0 commit comments

Comments
 (0)