Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Commit a63e03b

Browse files
skxskx
committed
Initial release of github2mr.
This is the initial release which is fully-functional and works for myself, self-hosted github enterprise installations, and privately hosted gitbucket installs. It has not been tested against other systems (gogs, gitea, etc), but reports of success/failure or patches would be most welcome.
0 parents  commit a63e03b

File tree

11 files changed

+708
-0
lines changed

11 files changed

+708
-0
lines changed

‎.github/build‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
#!/bin/bash
2+
3+
# The basename of our binary
4+
BASE="github2mr"
5+
6+
# Setup an output directory - creating if missing
7+
cur=$(pwd)
8+
OUTPUT="${cur}/bin"
9+
if [ ! -d "${OUTPUT}" ]; then
10+
mkdir -p "${OUTPUT}"
11+
fi
12+
13+
# We build on multiple platforms/archs
14+
BUILD_PLATFORMS="linux darwin freebsd"
15+
BUILD_ARCHS="amd64 386"
16+
17+
# For each platform.
18+
for OS in ${BUILD_PLATFORMS[@]}; do
19+
20+
# For each arch
21+
for ARCH in ${BUILD_ARCHS[@]}; do
22+
23+
# Setup a suffix for the binary
24+
SUFFIX="${OS}"
25+
26+
# i386 is better than 386
27+
if [ "$ARCH" = "386" ]; then
28+
SUFFIX="${SUFFIX}-i386"
29+
else
30+
SUFFIX="${SUFFIX}-${ARCH}"
31+
fi
32+
33+
echo "Building for ${OS} [${ARCH}] -> ${BASE}-${SUFFIX}"
34+
35+
# Run the build
36+
export GOARCH=${ARCH}
37+
export GOOS=${OS}
38+
export CGO_ENABLED=0
39+
40+
# Build the main-binary
41+
go build -ldflags "-X main.version=$(git describe --tags 2>/dev/null || echo 'master')" -o "${OUTPUT}/${BASE}-${SUFFIX}"
42+
done
43+
done

‎.github/run-tests.sh‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#!/bin/bash
2+
3+
# Install tools to test our code-quality.
4+
go get -u golang.org/x/lint/golint
5+
6+
# Failures cause aborts
7+
set -e
8+
9+
# Run the linter
10+
echo "Launching linter .."
11+
golint -set_exit_status ./...
12+
echo "Completed linter .."
13+
14+
# Run the vet-checker.
15+
echo "Launching go vet check .."
16+
go vet ./...
17+
echo "Completed go vet check .."
18+

‎.github/workflows/pull_request.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
on: pull_request
2+
name: Pull Request
3+
jobs:
4+
test:
5+
name: Run tests
6+
runs-on: ubuntu-latest
7+
steps:
8+
- uses: actions/checkout@master
9+
- name: Test
10+
uses: skx/github-action-tester@master

‎.github/workflows/push.yml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
on:
2+
push:
3+
branches:
4+
- master
5+
name: Push Event
6+
jobs:
7+
test:
8+
name: Run tests
9+
runs-on: ubuntu-latest
10+
steps:
11+
- uses: actions/checkout@master
12+
- name: Test
13+
uses: skx/github-action-tester@master

‎.github/workflows/release.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
on:
2+
release:
3+
types: [created]
4+
name: Handle Release
5+
jobs:
6+
upload:
7+
name: Upload
8+
runs-on: ubuntu-latest
9+
steps:
10+
- name: Checkout the repository
11+
uses: actions/checkout@master
12+
- name: Generate the artifacts
13+
uses: skx/github-action-build@master
14+
- name: Upload the artifacts
15+
uses: skx/github-action-publish-binaries@master
16+
env:
17+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
18+
with:
19+
args: bin/*-*

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
bin/

‎README.md‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
2+
* [github2mr](#github2mr)
3+
* [Brief mr Example](#brief-mr-example)
4+
* [Installation](#installation)
5+
* [Configuration / Usage](#configuration--usage)
6+
* [Other Git Hosts](#other-git-hosts)
7+
* [Github Setup](#github-setup)
8+
9+
10+
11+
12+
# github2mr
13+
14+
Many [Github](https://github.com/) users have a large number of repositories upon which they work. This application allows you to dump all your repository details into a configuration file for [myrepos](https://myrepos.branchable.com/).
15+
16+
The myrepos package, containing a binary named `mr`, is a _wonderful_ tool that lets you apply operations to multiple repositories at once given a suitable configuration.
17+
18+
The end result of using `mr` and `github2mr` is that you should be able to clone all your remote github repositories, and update them easily with only a couple of commands which is great for when you work/develop/live on multiple machines.
19+
20+
21+
## Brief `mr` Example
22+
23+
Let us pretend I'm moving to a new machine; first of all I export the list of all my remote repositories to a configuration file using _this_ tool:
24+
25+
github2mr > ~/Repos/.mrconfig.github
26+
27+
* **NOTE**: The first time you create a new configuration file you will need to mark it as being trusted, because it is possible for configuration files to contain arbitrary shell-commands.
28+
* Mark the configuration file as trusted by adding it's name to `~/.mrtrust`:
29+
* `echo ~/Repos/.mrconfig.github >> ~/.mrtrust`
30+
31+
Now that we've populated a configuration-file we can tell `mr` to checkout each of those repositories:
32+
33+
mr --jobs 8 --config ~/Repos/.mrconfig.github
34+
35+
Later in the week I can update all the repositories which have been cloned, pulling in any remote changes that have been made from other systems:
36+
37+
mr --jobs 8 --config ~/Repos/.mrconfig.github update
38+
39+
**NOTE**: If you prefer you can just use `update` all the time, `mr` will checkout a repository if it is missing as part of the `update` process. I'm using distinct flags here for clarity. Please read the `mr`-manpage to look at the commands it understands.
40+
41+
42+
# Installation
43+
44+
You should be able to install this application using the standard golang approach:
45+
46+
$ go get github.com/skx/github2mr
47+
48+
If you prefer you can [download the latest binary](http://github.com/skx/github2mr/releases) release, for various systems.
49+
50+
51+
52+
53+
# Configuration / Usage
54+
55+
Once installed you'll need to configure your github token, which you can generate from [withing your github settings](https://github.com/settings/tokens).
56+
57+
you can either pass the token as an argument to the tool (via `github2mr -token=xxxxx`), or store it in the environment in the variable GITHUB_TOKEN:
58+
59+
$ export GITHUB_TOKEN=xxxxx
60+
$ github2mr [options]
61+
62+
You can run `github2mr -help` to see available options, but in brief:
63+
64+
* You can choose a default prefix to clone your repositories to.
65+
* By default all repositories will be located at `~/Repos/${git_host}`.
66+
* You can exclude all-organizational repositories.
67+
* Or the reverse, ignoring all personal-repositories.
68+
* You can exclude repositories by name.
69+
* You can default to cloning repositories via HTTP, instead of SSH.
70+
71+
72+
## Other Git Hosts
73+
74+
This tool can be configured to point at other systems which use the same
75+
API as the public-facing Github site.
76+
77+
To use it against a self-hosted Github Enterprise installation, for example,
78+
simply specify the URL:
79+
80+
$ export GITHUB_TOKEN=xxxxx
81+
$ github2mr -api=https://git.example.com/ [options]
82+
83+
It has also been tested against an installation of [gitbucket](https://github.com/gitbucket/gitbucket) which can be configured a similar way - however in this case you'll find that you receive an error "401 bad credentials" unless you add the `-auth-header-token` flag:
84+
85+
$ export GITHUB_TOKEN=xxxxx
86+
$ github2mr -api=https://git.example.com/ -auth-header-token
87+
88+
This seems to be related to the OAUTH header the library I'm using sends, by default it will send a HTTP request looking like this:
89+
90+
```
91+
GET /api/v3/users/skx/repos HTTP/1.1
92+
Host: localhost:9999
93+
User-Agent: go-github
94+
Accept: application/vnd.github.mercy-preview+json
95+
Authorization: Bearer SECRET-TOKEN
96+
Accept-Encoding: gzip
97+
```
98+
99+
Notice that the value of the `Authorization`-header begins with `Bearer`? Gitbucket prefers to see `Authorization: token SECRET-VALUE-HERE`.
100+
101+
102+
103+
104+
# Github Setup
105+
106+
This repository is configured to run tests upon every commit, and when
107+
pull-requests are created/updated. The testing is carried out via
108+
[.github/run-tests.sh](.github/run-tests.sh) which is used by the
109+
[github-action-tester](https://github.com/skx/github-action-tester) action.
110+
111+
Releases are automated in a similar fashion via [.github/build](.github/build),
112+
and the [github-action-publish-binaries](https://github.com/skx/github-action-publish-binaries) action.
113+
114+
115+
Steve
116+
--

‎TODO‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
* [x] Work for github personal user - i.e. me
2+
3+
* [x] Work for organizations.
4+
5+
* [x] Work against https://git.steve.org.uk/
6+
7+
* [-] Work with user/password (+/- 2FA).
8+
9+
* [x] Allow filtering on name of repos, to cheat at excluding orgs.
10+
11+
* [x] Allow limiting "public", "private", or "all" repos.
12+
13+
* [x] Announce.
14+
15+
* [x] Squash and release.
16+
17+
* [x] Setup CI
18+
19+
* [ ] Add decent README.md
20+
21+
* [ ] Add tests (hard; since we're just injesting.

‎go.mod‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
module github.com/skx/github2mr
2+
3+
go 1.13
4+
5+
require (
6+
github.com/google/go-github/v29 v29.0.2
7+
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
8+
)

‎go.sum‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
2+
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
3+
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
4+
github.com/google/go-github v17.0.0+incompatible h1:N0LgJ1j65A7kfXrZnUDaYCs/Sf4rEjNlfyDHW9dolSY=
5+
github.com/google/go-github/v29 v29.0.2 h1:opYN6Wc7DOz7Ku3Oh4l7prmkOMwEcQxpFtxdU8N8Pts=
6+
github.com/google/go-github/v29 v29.0.2/go.mod h1:CHKiKKPHJ0REzfwc14QMklvtHwCveD0PxlMjLlzAM5E=
7+
github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
8+
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
9+
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2 h1:VklqNMn3ovrHsnt90PveolxSbWFaJdECFbxSq0Mqo2M=
10+
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
11+
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
12+
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
13+
golang.org/x/net v0.0.0-20190311183353-d8887717615a h1:oWX7TPOiFAMXLq8o0ikBYfCJVlRHBcsciT5bXOrH628=
14+
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
15+
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
16+
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw=
17+
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
18+
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
19+
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
20+
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
21+
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
22+
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=

0 commit comments

Comments
 (0)