Skip to content

Commit 4efd502

Browse files
bchewbchew
authored
Merge pull request #71 from bchew/pypi
Restructure for packaging and publish to PyPI
2 parents b44ea6c + b1bc35e commit 4efd502

File tree

14 files changed

+115
-33
lines changed

14 files changed

+115
-33
lines changed

‎.github/workflows/build.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: Build and Publish (PyPI)
2+
3+
on:
4+
push:
5+
pull_request:
6+
branches: [master]
7+
8+
jobs:
9+
build:
10+
runs-on: ubuntu-latest
11+
strategy:
12+
matrix:
13+
python-version: [3.9]
14+
15+
steps:
16+
- uses: actions/checkout@v2
17+
- name: Set up Python ${{ matrix.python-version }}
18+
uses: actions/setup-python@v2
19+
with:
20+
python-version: ${{ matrix.python-version }}
21+
- name: Install build dependencies
22+
run: |
23+
pip install --upgrade build
24+
- name: Generate distribution package
25+
run: |
26+
python -m build
27+
- name: Publish package to TestPyPI
28+
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
29+
uses: pypa/gh-action-pypi-publish@release/v1
30+
with:
31+
user: __token__
32+
password: ${{ secrets.TEST_PYPI_API_TOKEN }}
33+
repository_url: https://test.pypi.org/legacy/
34+
- name: Publish package to PyPI
35+
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
36+
uses: pypa/gh-action-pypi-publish@release/v1
37+
with:
38+
user: __token__
39+
password: ${{ secrets.PYPI_API_TOKEN }}
40+

‎.github/workflows/test.yml‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -28,19 +28,19 @@ jobs:
2828
pip install -r requirements.txt
2929
- name: Test basic restore and backup
3030
run: |
31-
mkdir dump && cp -a test/testTable dump
32-
python dynamodump.py -m restore --noConfirm -r local -s testTable -d testRestoredTable --host localhost --port 8000 --accessKey a --secretKey a
33-
python dynamodump.py -m backup -r local -s testRestoredTable --host localhost --port 8000 --accessKey a --secretKey a
34-
python test/test.py
31+
mkdir dump && cp -a tests/testTable dump
32+
python dynamodump/dynamodump.py -m restore --noConfirm -r local -s testTable -d testRestoredTable --host localhost --port 8000 --accessKey a --secretKey a
33+
python dynamodump/dynamodump.py -m backup -r local -s testRestoredTable --host localhost --port 8000 --accessKey a --secretKey a
34+
python tests/test.py
3535
- name: Test wildcard restore and backup
3636
run: |
37-
python dynamodump.py -m restore --noConfirm -r local -s "*" --host localhost --port 8000 --accessKey a --secretKey a
37+
python dynamodump/dynamodump.py -m restore --noConfirm -r local -s "*" --host localhost --port 8000 --accessKey a --secretKey a
3838
rm -rf dump/test*
39-
python dynamodump.py -m backup -r local -s "*" --host localhost --port 8000 --accessKey a --secretKey a
40-
python test/test.py
39+
python dynamodump/dynamodump.py -m backup -r local -s "*" --host localhost --port 8000 --accessKey a --secretKey a
40+
python tests/test.py
4141
- name: Test prefixed wildcard restore and backup
4242
run: |
43-
python dynamodump.py -m restore --noConfirm -r local -s "test*" --host localhost --port 8000 --accessKey a --secretKey a --prefixSeparator ""
43+
python dynamodump/dynamodump.py -m restore --noConfirm -r local -s "test*" --host localhost --port 8000 --accessKey a --secretKey a --prefixSeparator ""
4444
rm -rf dump/test*
45-
python dynamodump.py -m backup -r local -s "test*" --host localhost --port 8000 --accessKey a --secretKey a --prefixSeparator ""
46-
python test/test.py
45+
python dynamodump/dynamodump.py -m backup -r local -s "test*" --host localhost --port 8000 --accessKey a --secretKey a --prefixSeparator ""
46+
python tests/test.py

‎Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
FROM python:3.9.6-alpine3.14
22

33
COPY ./requirements.txt /mnt/dynamodump/requirements.txt
4-
COPY ./dynamodump.py /usr/local/bin/dynamodump
4+
COPY ./dynamodump/dynamodump.py /usr/local/bin/dynamodump
55

66
RUN pip install -r /mnt/dynamodump/requirements.txt

‎LICENSE‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
Copyright 2013 Benny Chew
1+
Copyright 2013 Benny Chew and dynamodump contributors
22

33
Permission is hereby granted, free of charge, to any person obtaining a copy of
44
this software and associated documentation files (the "Software"), to deal in

‎README.md‎

Lines changed: 26 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,25 @@
11
# dynamodump
22

3-
![Linting Status](https://github.com/bchew/dynamodump/workflows/Linting/badge.svg) ![Test Status](https://github.com/bchew/dynamodump/workflows/Test/badge.svg) [![DockerBuildstatus](https://img.shields.io/docker/build/bchew/dynamodump.svg)](https://hub.docker.com/r/bchew/dynamodump/)
3+
[![PyPI version](https://badge.fury.io/py/dynamodump.svg)](https://badge.fury.io/py/dynamodump)
4+
![Linting Status](https://github.com/bchew/dynamodump/workflows/Linting/badge.svg)
5+
![Test Status](https://github.com/bchew/dynamodump/workflows/Test/badge.svg)
46

57
Simple backup and restore script for Amazon DynamoDB using AWS SDK for Python (boto3) to work similarly to mysqldump.
68

79
Suitable for DynamoDB usages of smaller data volume which do not warrant the usage of AWS Data Pipeline for backup/restores/empty.
810

911
dynamodump supports local DynamoDB instances as well (tested with [DynamoDB Local](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html)).
1012

13+
## Installation
14+
15+
```
16+
pip install dynamodump
17+
```
18+
1119
## Usage
1220

1321
```
14-
usage: dynamodump.py [-h] [-a {zip,tar}] [-b BUCKET]
22+
usage: dynamodump [-h] [-a {zip,tar}] [-b BUCKET]
1523
[-m {backup,restore,empty}] [-r REGION] [--host HOST]
1624
[--port PORT] [--accessKey ACCESSKEY]
1725
[--secretKey SECRETKEY] [-p PROFILE] [-s SRCTABLE]
@@ -96,73 +104,73 @@ As of v1.2.0, note that `--noConfirm` is required to perform data restores invol
96104
Single table backup/restore:
97105

98106
```
99-
python dynamodump.py -m backup -r us-west-1 -s testTable
107+
dynamodump -m backup -r us-west-1 -s testTable
100108
101-
python dynamodump.py -m restore -r us-west-1 -s testTable
109+
dynamodump -m restore -r us-west-1 -s testTable
102110
```
103111

104112
Multiple table backup/restore (assumes prefix of 'production-' of table names, use --prefixSeparator to specify a
105113
different separator):
106114

107115
```
108-
python dynamodump.py -m backup -r us-west-1 -s production*
116+
dynamodump -m backup -r us-west-1 -s production*
109117
110-
python dynamodump.py -m restore -r us-west-1 -s production*
118+
dynamodump -m restore -r us-west-1 -s production*
111119
```
112120

113121
The above, but between different environments (e.g. production-_ tables to development-_ tables):
114122

115123
```
116-
python dynamodump.py -m backup -r us-west-1 -s production*
124+
dynamodump -m backup -r us-west-1 -s production*
117125
118-
python dynamodump.py -m restore -r us-west-1 -s production* -d development*
126+
dynamodump -m restore -r us-west-1 -s production* -d development*
119127
```
120128

121129
Backup all tables and restore only data (will not delete and recreate schema):
122130

123131
```
124-
python dynamodump.py -m backup -r us-west-1 -s "*"
132+
dynamodump -m backup -r us-west-1 -s "*"
125133
126-
python dynamodump.py -m restore -r us-west-1 -s "*" --dataOnly
134+
dynamodump -m restore -r us-west-1 -s "*" --dataOnly
127135
```
128136

129137
Dump all table schemas and create the schemas (e.g. creating blank tables in a different AWS account):
130138

131139
```
132-
python dynamodump.py -m backup -r us-west-1 -p source_credentials -s "*" --schemaOnly
140+
dynamodump -m backup -r us-west-1 -p source_credentials -s "*" --schemaOnly
133141
134-
python dynamodump.py -m restore -r us-west-1 -p destination_credentials -s "*" --schemaOnly
142+
dynamodump -m restore -r us-west-1 -p destination_credentials -s "*" --schemaOnly
135143
```
136144

137145
Backup all tables based on AWS tag `key=value`
138146

139147
```
140-
python dynamodump.py -p profile -r us-east-1 -m backup -t KEY=VALUE
148+
dynamodump -p profile -r us-east-1 -m backup -t KEY=VALUE
141149
```
142150

143151
Backup all tables based on AWS tag, compress and store in specified S3 bucket.
144152

145153
```
146-
python dynamodump.py -p profile -r us-east-1 -m backup -a tar -b some_s3_bucket -t TAG_KEY=TAG_VALUE
154+
dynamodump -p profile -r us-east-1 -m backup -a tar -b some_s3_bucket -t TAG_KEY=TAG_VALUE
147155
148-
python dynamodump.py -p profile -r us-east-1 -m backup -a zip -b some_s3_bucket -t TAG_KEY=TAG_VALUE
156+
dynamodump -p profile -r us-east-1 -m backup -a zip -b some_s3_bucket -t TAG_KEY=TAG_VALUE
149157
```
150158

151159
Restore from S3 bucket to specified destination table
152160

153161
```
154162
## source_table identifies archive file in S3 bucket from which backup data is restored
155-
python dynamodump.py -a tar -b some_s3_bucket -m restore -r us-east-1 -p profile -d destination_table -s source_table
163+
dynamodump -a tar -b some_s3_bucket -m restore -r us-east-1 -p profile -d destination_table -s source_table
156164
```
157165

158166
## Local example
159167

160168
The following assumes your local DynamoDB is running on localhost:8000 and is accessible via 'a' as access/secret keys.
161169

162170
```
163-
python dynamodump.py -m backup -r local -s testTable --host localhost --port 8000 --accessKey a --secretKey a
171+
dynamodump -m backup -r local -s testTable --host localhost --port 8000 --accessKey a --secretKey a
164172
165-
python dynamodump.py -m restore -r local -s testTable --host localhost --port 8000 --accessKey a --secretKey a
173+
dynamodump -m restore -r local -s testTable --host localhost --port 8000 --accessKey a --secretKey a
166174
```
167175

168176
Multiple table backup/restore as stated in the AWS examples are also available for local.

‎test/__init__.py‎ renamed to ‎dynamodump/__init__.py‎

File renamed without changes.

‎dynamodump.py‎ renamed to ‎dynamodump/dynamodump.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
Suitable for DynamoDB usages of smaller data volume which do not warrant the usage of AWS
66
Data Pipeline for backup/restores/empty.
77
8-
dynamodump supports local DynamoDB instances as well (tested with dynalite).
8+
dynamodump supports local DynamoDB instances as well (tested with DynamoDB Local).
99
"""
1010

1111
import argparse

‎pyproject.toml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
[build-system]
2+
requires = [
3+
"setuptools>=42",
4+
"wheel"
5+
]
6+
build-backend = "setuptools.build_meta"

‎requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
boto3==1.18.26
2-
six==1.16
2+
six==1.16.0

‎setup.py‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
import setuptools
2+
3+
with open("README.md", "r", encoding="utf-8") as fh:
4+
long_description = fh.read()
5+
6+
setuptools.setup(
7+
name="dynamodump",
8+
version="1.3.0",
9+
author="Benny Chew",
10+
author_email="[email protected]",
11+
description="Simple backup and restore for Amazon DynamoDB using AWS SDK for Python (boto3)",
12+
long_description=long_description,
13+
long_description_content_type="text/markdown",
14+
url="https://github.com/bchew/dynamodump",
15+
project_urls={
16+
"Releases": "https://github.com/bchew/dynamodump/releases",
17+
},
18+
classifiers=[
19+
"Programming Language :: Python :: 3",
20+
"License :: OSI Approved :: MIT License",
21+
],
22+
packages=["dynamodump"],
23+
python_requires=">=3.6",
24+
install_requires=["boto3==1.18.26", "six==1.16.0"],
25+
entry_points={
26+
"console_scripts": ["dynamodump=dynamodump.dynamodump:main"],
27+
},
28+
)

0 commit comments

Comments
 (0)