Skip to content

Commit af408fa

Browse files
fileformatfileformat
committed
Initial release (previous work was in resolve.rs)
1 parent 1bd23f7 commit af408fa

File tree

5 files changed

+188
-0
lines changed

5 files changed

+188
-0
lines changed

‎.editorconfig‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# EditorConfig is awesome: http://EditorConfig.org
2+
3+
# top-most EditorConfig file
4+
root = true
5+
6+
[*]
7+
charset = utf-8
8+
end_of_line = lf
9+
indent_size = tab
10+
insert_final_newline = true
11+
tab_width = 4
12+
trim_trailing_whitespace = true
13+
14+
[*.json]
15+
indent_size = 2
16+
17+
[*.yaml]
18+
indent_size = 2

‎.github/workflows/release.yaml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
name: Release
2+
3+
on:
4+
schedule:
5+
- cron: '42 0 * * 1'
6+
workflow_dispatch:
7+
8+
jobs:
9+
release:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- name: Checkout
13+
uses: actions/checkout@v1
14+
15+
- name: Get and encrypt Maxmind databases
16+
run: bin/update_maxmind.sh
17+
env:
18+
MMDB_ENCRYPTION_KEY: ${{ secrets.MMDB_ENCRYPTION_KEY }}
19+
MAXMIND_LICENSE_KEY: ${{ secrets.MAXMIND_LICENSE_KEY }}
20+
21+
- name: Set release info
22+
run: |
23+
echo "RELEASE_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> $GITHUB_ENV
24+
echo "RELEASE_TAG=$(date -u +%Y%m%d-%H%M%S)" >> $GITHUB_ENV
25+
26+
- name: Release
27+
uses: softprops/action-gh-release@v1
28+
with:
29+
files: |
30+
build/GeoLite2-ASN.mmdb.enc
31+
build/GeoLite2-City.mmdb.enc
32+
build/mmdb.iv
33+
build/mmdb.md5
34+
tag_name: ${{ env.RELEASE_TAG }}
35+
name: Release ${{ env.RELEASE_DATE }}
36+
body: |
37+
Automatically created on ${{ env.RELEASE_DATE }}
38+
draft: false
39+
prerelease: false

‎.gitignore‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
build/
2+
*.crt
3+
dist/
4+
.DS_Store
5+
*.env
6+
.idea/
7+
*.key
8+
*.log
9+
*.mmdb
10+
tmp/
11+
*.tmp

‎README.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# MaxMind for Resolve.rs [<img alt="Resolve.rs Logo" src="https://resolve.rs/favicon.svg" height="96" align="right"/>](https://resolve.rs/)
2+
3+
The MaxMind "Lite" databases are free to use, but you cannot redistribute them, and cannot (reliably) download them every time a server starts. To get a server to start without hitting MaxMind directly, I use an encrypted version that automatically updates once a week.
4+
5+
## Using
6+
7+
This is just for [resolve.rs](https://resolve.rs), but you can do the same thing with your own fork. You need to set the following Github secrets:
8+
9+
* `MAXMIND_LICENSE_KEY` - from MaxMind
10+
* `MAXMIND_ACCOUNT_ID` - also from MaxMind
11+
* `MMDB_ENCRYPTION_KEY` - generate a 32-byte (64 hex digits) encryption key
12+
13+
You will use the `MMDB_ENCRYPTION_KEY` when decrypting the database files on your server.
14+
15+
## License
16+
17+
The script is licensed under the [GNU Affero General Public License v3.0](LICENSE.txt).
18+
19+
This product includes GeoLite2 data created by MaxMind, available from [www.maxmind.com](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data/)
20+
21+
## Release Files
22+
23+
- GeoLite2-ASN.mmdb.enc: encrypted ASN lookup database
24+
- GeoLite2-City.mmdb.enc: encrypted city lookup database
25+
- mmdb.iv: initialization vector used for encryption
26+
- mmdb.md5: md5 hashes of the unencrypted files (to detect updates)
27+
28+
## Credits
29+
30+
[![Bash](https://www.vectorlogo.zone/logos/gnu_bash/gnu_bash-ar21.svg)](https://www.gnu.org/software/bash/ "Scripting")
31+
[![Git](https://www.vectorlogo.zone/logos/git-scm/git-scm-ar21.svg)](https://git-scm.com/ "Version control")
32+
[![Github](https://www.vectorlogo.zone/logos/github/github-ar21.svg)](https://github.com/ "Code hosting")
33+
[![MaxMind](https://www.vectorlogo.zone/logos/maxmind/maxmind-ar21.svg)](https://www.maxmind.com/ "IP geolocation and ASN databases")
34+
[![OpenSSL](https://www.vectorlogo.zone/logos/openssl/openssl-ar21.svg)](https://www.openssl.org/ "Encryption")

‎bin/update_maxmind.sh‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
#!/usr/bin/env bash
2+
#
3+
# downloads a fresh copy of the MaxMind databases for ASN & geolocation lookups and makes an encrypted release
4+
#
5+
6+
set -o errexit
7+
set -o pipefail
8+
set -o nounset
9+
10+
echo "INFO: starting MaxMind update at $(date -u +%Y-%m-%dT%H:%M:%SZ)"
11+
12+
SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )"
13+
REPO_DIR="$(realpath "${SCRIPT_DIR}/..")"
14+
15+
ENV_FILE="$(realpath "${REPO_DIR}/.env")"
16+
if [ -f "${ENV_FILE}" ]; then
17+
echo "INFO: loading ${ENV_FILE} into environment"
18+
export $(cat ${ENV_FILE})
19+
fi
20+
21+
TAR=tar
22+
if [[ $OSTYPE == 'darwin'* ]]; then
23+
echo "INFO: running on MacOS so using gtar instead of tar"
24+
TAR=gtar
25+
fi
26+
27+
BUILD_DIR="$(realpath "${REPO_DIR}/build")"
28+
if [ ! -d "${BUILD_DIR}" ]; then
29+
echo "INFO: creating build directory ${BUILD_DIR}"
30+
mkdir -p "${BUILD_DIR}"
31+
else
32+
echo "INFO: using existing build directory ${BUILD_DIR}"
33+
fi
34+
35+
TMP_ASN_FILE=$(mktemp)
36+
echo "INFO: download MaxMind ASN database into ${TMP_ASN_FILE}"
37+
curl --silent "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-ASN&suffix=tar.gz&license_key=${MAXMIND_LICENSE_KEY}" >"${TMP_ASN_FILE}"
38+
${TAR} -xzf ${TMP_ASN_FILE} --directory="${BUILD_DIR}" --wildcards --strip-components 1 "*.mmdb"
39+
rm "${TMP_ASN_FILE}"
40+
41+
TMP_CITY_FILE=$(mktemp)
42+
echo "INFO: download MaxMind City database into ${TMP_CITY_FILE}"
43+
curl --silent "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&suffix=tar.gz&license_key=${MAXMIND_LICENSE_KEY}" >"${TMP_CITY_FILE}"
44+
${TAR} -xzf ${TMP_CITY_FILE} --directory="${BUILD_DIR}" --wildcards --strip-components 1 "*.mmdb"
45+
rm "${TMP_CITY_FILE}"
46+
47+
md5sum ${BUILD_DIR}/*.mmdb | sort >${BUILD_DIR}/mmdb.md5
48+
49+
#LATER: curl release/download/latest/mmdb.md5
50+
#DIFF=$(git diff --name-only "${BUILD_DIR}/mmdb.md5")
51+
#
52+
#if [ "${DIFF}" == "" ]; then
53+
# echo "INFO: no changes, exiting at $(date -u +%Y-%m-%dT%H:%M:%SZ)"
54+
# exit 0
55+
#fi
56+
57+
if [ "${MMDB_ENCRYPTION_KEY:-BAD}" = "BAD" ]; then
58+
echo "INFO: no encryption keys, exiting. (but app can still be run locally)"
59+
exit 1
60+
fi
61+
62+
#
63+
# generate (and save) a new IV every time
64+
#
65+
MMDB_ENCRYPTION_IV=$(head -c 16 /dev/urandom | xxd -l 16 -c 16 -p)
66+
echo -n ${MMDB_ENCRYPTION_IV} > ${BUILD_DIR}/mmdb.iv
67+
68+
ASN_FILE="${BUILD_DIR}/GeoLite2-ASN.mmdb"
69+
echo "INFO: starting encryption of ${ASN_FILE} (file size=$(du ${ASN_FILE} | cut -f 1))"
70+
gzip --stdout ${ASN_FILE} | openssl enc -aes-256-ctr \
71+
-K ${MMDB_ENCRYPTION_KEY} \
72+
-iv ${MMDB_ENCRYPTION_IV} \
73+
-out "${ASN_FILE}.enc"
74+
rm ${ASN_FILE}
75+
echo "INFO: encryption complete (file size=$(du ${ASN_FILE}.enc | cut -f 1))"
76+
77+
CITY_FILE="${BUILD_DIR}/GeoLite2-City.mmdb"
78+
echo "INFO: starting encryption of ${CITY_FILE} (file size=$(du ${CITY_FILE} | cut -f 1))"
79+
gzip --stdout ${CITY_FILE} | openssl enc -aes-256-ctr \
80+
-K ${MMDB_ENCRYPTION_KEY} \
81+
-iv ${MMDB_ENCRYPTION_IV} \
82+
-out "${CITY_FILE}.enc"
83+
rm ${CITY_FILE}
84+
echo "INFO: encryption complete (file size=$(du ${CITY_FILE}.enc | cut -f 1))"
85+
86+
echo "INFO: complete MaxMind update at $(date -u +%Y-%m-%dT%H:%M:%SZ)"

0 commit comments

Comments
 (0)