|
| 1 | +# github-metadata-backup |
| 2 | + |
| 3 | +Download issues and pull-requests from the GitHub API and store as JSON files. |
| 4 | +Supports incremental updates. The tool is written in Rust. Please |
| 5 | + |
| 6 | +## Usage |
| 7 | + |
| 8 | +The GitHub API is rate-limted to 60 requests per hour without authentication. This |
| 9 | +tool requires a [GitHub Personal Access Token], which increases the rate-limit to |
| 10 | +5000 requests per hour. A _classic_ token is recommended over the newer _fine-grained |
| 11 | +tokens_. Some of the events might be missing from the backup with the _fine-grained |
| 12 | +tokens_. See https://github.com/orgs/community/discussions/58534 for more information. |
| 13 | + |
| 14 | +[GitHub Personal Access Token]: (https://github.com/settings/tokens) |
| 15 | + |
| 16 | +The token an either be supplied to the `github-metadata-backup` binary via the |
| 17 | +`-p`/`--personal-access-token` command line option or placed in a file and read |
| 18 | +from there with the `-f`/`--personal-access-token-file` option. The tool also |
| 19 | +requires an `--owner` (a GitHub user or organization) and the `--repo` (the |
| 20 | +repository of that owner) to download the metadata from. The backup will be |
| 21 | +placed in the directory defined with `-d`/`--destination`. Make sure the tool |
| 22 | +has the required permissions to write to it. |
| 23 | + |
| 24 | +``` |
| 25 | +Usage: github-metadata-backup [OPTIONS] --owner <OWNER> --repo <REPO> --destination <PATH> |
| 26 | +
|
| 27 | +Options: |
| 28 | + -o, --owner <OWNER> |
| 29 | + Owner of the repository to backup |
| 30 | + -r, --repo <REPO> |
| 31 | + Name of the repository to backup |
| 32 | + -p, --personal-access-token <PERSONAL_ACCESS_TOKEN> |
| 33 | + Personal Access Token to the GitHub API supplied via the command line |
| 34 | + -f, --personal-access-token-file <PATH> |
| 35 | + Personal Access Token to the GitHub API read from a file |
| 36 | + -d, --destination <PATH> |
| 37 | + Destination where the backup should be written to |
| 38 | + -h, --help |
| 39 | + Print help |
| 40 | + -V, --version |
| 41 | + Print version |
| 42 | +``` |
| 43 | + |
| 44 | +The log-level can be controlled with the `RUST_LOG` environment variable. By |
| 45 | +default, it's `RUST_LOG=info`. |
| 46 | + |
| 47 | +## Example |
| 48 | + |
| 49 | +To backup the metadata from the `bitcoin/bitcoin` repository to the `bitcoin-bitcoin` |
| 50 | +directory while reading the file from the `read-only-github-access-token.sec` file the |
| 51 | +following command can be used: |
| 52 | + |
| 53 | +``` |
| 54 | +github-metadata-backup --owner bitcoin --repo bitcoin --destination bitcoin-bitcoin --personal-access-token-file read-only-github-access-token.sec |
| 55 | +``` |
| 56 | + |
| 57 | +This creates the `bitcoin-bitcoin` directory with the `issues` and `pulls` |
| 58 | +subdirectories. It requests metadata until the rate-limit is reached, waits |
| 59 | +until requests are allowed again, and then continues. Once finished with the |
| 60 | +initial backup, which can take a few hours with large repositories, it writes |
| 61 | +a `state.json` file. On subsequent runs, this file is read and only an |
| 62 | +incremental backup is made. To do a full backup again, delete the state.json |
| 63 | +file. The old backup will be overwritten. The JSON files are formatted to be |
| 64 | +easily trackable in git. It makes sense to commit each incremental backup. |
| 65 | + |
| 66 | +## Nix Package and module |
| 67 | + |
| 68 | +A Nix package and module for the github-metadata-backup tool are avaliable in |
| 69 | +my personal [nix-package-collection]. The package can be run with: |
| 70 | + |
| 71 | +``` |
| 72 | +nix run github:0xb10c/nix#github-metadata-backup |
| 73 | +``` |
| 74 | + |
| 75 | +[nix-package-collection]: https://github.com/0xB10C/nix |
| 76 | + |
0 commit comments