Skip to content

Commit 30893d0

Browse files
author
Ignacio Elizaga
committed
Merge with Faisal's code
1 parent 6d2e08a commit 30893d0

30 files changed

+3197
-1581
lines changed

.gitignore

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,4 +15,3 @@
1515

1616
.vagrant/
1717
Vagrantfile
18-
pivotal_mockd_hackathon

LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2017 Ignacio Elizaga
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

README.md

Lines changed: 198 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
# MockD [![Go Version](https://img.shields.io/badge/go-v1.7.4-green.svg?style=flat-square)](https://golang.org/dl/) [![MIT License](https://img.shields.io/badge/License-MIT_License-green.svg?style=flat-square)](https://github.com/ielizaga/mockd/blob/master/LICENSE)
2+
3+
MockD is built of simple catchphrase
4+
5+
Here are my tables
6+
Load data for me
7+
I don't care how
8+
9+
with MockD you can have your own tables defined with datatypes, just supply the connection, total data needed and the tables and leave MockD to build random generated dataset on those tables.
10+
11+
MockD does try to ensure it acknowledge the constraints in the database and try to fix PK / UK / FK constraints, but it not foolproof. hence its **HIGHLY RECOMMEDED** you do take a backup of your database before you ask MockD to load data.
12+
13+
An ideal environment to make MockD work without any error's would be
14+
15+
+ No constraints
16+
+ No custom datatypes
17+
18+
Check on "Known Issues" below for things are currently not working
19+
20+
# Important Information
21+
22+
MockD program is created for generating test data and **NOT TO BE USED IN PRODUCTION DATABASE**, ensure you have the backup of the database before asking MockD to load data for you.
23+
24+
# Supported database
25+
26+
+ Postgres
27+
+ Greenplum (GPDB)
28+
+ Hawq (HDB)
29+
+ MySQL / Oracle ( coming soon <img src="https://media.giphy.com/media/MhS3BBBdYAxEc/giphy.gif" width="18" height="10"> )
30+
31+
# Supported datatypes
32+
33+
+ All datatypes that are listed on the [postgres datatype](https://www.postgresql.org/docs/9.6/static/datatype.html) website are supported
34+
+ As Greenplum / Hawq is a fork from postgres, the supported postgres datatype also applies here.
35+
36+
# Go Library Package Dependencies
37+
38+
+ [Data Faker](https://github.com/icrowley/fake) by icrowley
39+
+ [Progress bar](https://github.com/vbauerster/mpb) by vbauerster
40+
+ [Postgres Driver](https://github.com/lib/pq) by lib/pq
41+
+ [Go Logger](https://github.com/op/go-logging) by op/go-logging
42+
43+
# How it works.
44+
45+
+ Parse the argument parameters
46+
+ Check if the database connection can be established and queries work.
47+
+ If all database flag is set, then extract all the tables in the database
48+
+ if tables are specified then uses only those tables.
49+
+ First creates a backup of all constraints ( like PK (primary key), UK, CK, FK ) and also unique index ( due to cascade nature of the drop constraints )
50+
+ Also store this constraint / Unique index information on the memory
51+
+ Before loading it will remove all the constraints on the table
52+
+ Once done, it will start its loaded by picking random data based on the columns datatype.
53+
+ Once its done, it read all the constraints information from the memory
54+
+ fixes PK and UK initially
55+
+ then fixes FK
56+
+ Check constraints are ignored ( since its not easy to translate it )
57+
+ Once done it loads all the constraints that it had backed up ( MockD fails mostly at this stage if its not able to autofix the constraints violation )
58+
+ Program ends.
59+
60+
# Usage
61+
62+
```
63+
USAGE: mockd <DATABASE ENGINE> <OPTIONS>
64+
DATABASE ENGINE:
65+
postgres Postgres database
66+
greenplum Greenplum database
67+
hdb Hawq Database
68+
help Show help
69+
OPTIONS:
70+
Execute "mockd <database engine> -h" for all the database specific options
71+
```
72+
73+
# How to use it
74+
75+
### MockD User
76+
77+
+ [Download](https://github.com/ielizaga/mockd/archive/master.zip) the github repo or clone the github repo
78+
79+
```
80+
git clone https://github.com/ielizaga/mockd.git
81+
```
82+
83+
+ Navigate to the "bin" directory
84+
+ Use the appropriate binary that matches your OS
85+
86+
```
87+
XXXXX:bin XXXXX ls -ltr
88+
total 26832
89+
-rwxr-xr-x 1 XXXXX XXXXX 6878692 Jul 16 10:44 mockd-mac
90+
-rwxr-xr-x 1 XXXXX XXXXX 6853562 Jul 16 10:44 mockd-linux
91+
```
92+
93+
**NOTE:** if you have the datatype UUID defined on the table, make sure you have the execute "uuidgen" installed on the OS.
94+
95+
### Developers
96+
97+
+ [Download](https://github.com/ielizaga/mockd/archive/master.zip) the github repo or clone the github repo
98+
99+
```
100+
git clone https://github.com/ielizaga/mockd.git
101+
```
102+
103+
or use "go get" to download the source after setting the GOPATH
104+
105+
```
106+
go get github.com/ielizaga/mockd
107+
```
108+
109+
+ Download all the dependencies
110+
111+
```
112+
go get github.com/icrowley/fake
113+
go get github.com/vbauerster/mpb
114+
go get github.com/lib/pq
115+
go get github.com/op/go-logging
116+
```
117+
118+
+ You can modify the code and execute using command before creating a build
119+
120+
```
121+
go run *.go <database engine>
122+
```
123+
124+
+ To build binaries for different OS, you can use for eg.s
125+
126+
```
127+
env GOOS=linux GOARCH=amd64 go build
128+
```
129+
130+
# Command Reference
131+
132+
+ For postgres / greenplum / hawq
133+
134+
```
135+
XXXXX:bin XXXXX ./mockd-mac postgres -help
136+
2017-07-16 10:58:43.609:INFO > Parsing all the command line arguments
137+
Usage of postgres:
138+
-d string
139+
The database name where the table resides (default "postgres")
140+
-h string
141+
The hostname that can be used to connect to the database (default "localhost")
142+
-i Ignore checking and fixing constraint issues
143+
-n int
144+
The total number of mocked rows that is needed (default 1)
145+
-p int
146+
The port that is used by the database engine (default 5432)
147+
-t string
148+
The table name to be filled in with mock data
149+
-u string
150+
The username that can be used to connect to the database (default "postgres")
151+
-w string
152+
The password for the user that can be used to connect to the database
153+
-x Mock all the tables in the database
154+
```
155+
156+
# Examples
157+
158+
+ Mock one table will random data
159+
160+
```
161+
bin/mockd-mac <dbengine> -n <total rows> -u <user> -d <database> -t <table>
162+
```
163+
164+
![single table](https://github.com/ielizaga/mockd/blob/master/img/singletable.gif)
165+
166+
+ Mock multiple table with random data
167+
168+
```
169+
bin/mockd-mac <dbengine> -n <total rows> -u <user> -d <database> -t <table1>,<table2>,....
170+
```
171+
172+
![multiple table](https://github.com/ielizaga/mockd/blob/master/img/multipletable.gif)
173+
174+
+ Mock entire database
175+
176+
```
177+
bin/mockd-mac <dbengine> -n <total rows> -u <user> -d <database> -x
178+
```
179+
180+
![All Database](https://github.com/ielizaga/mockd/blob/master/img/alldb.gif)
181+
182+
# Known Issues
183+
184+
1. If you uses a small amount of rows to mock and you have defined a composite PK index, there are chances that the primary key would fail ( this is due to foreign key fix only runs after primary key fix )
185+
2. Still having issues with Check constraint, only check that works is "COLUMN > 0" & nothing else
186+
3. On Greenplum/HDB partition tables are not supported (due to check constraint issues defined above).
187+
4. Custom datatypes are not supported
188+
189+
190+
# Community
191+
192+
You can sumbit issues or pull request via [github](https://github.com/ielizaga/mockd) and we will try our level best to fix them..
193+
194+
# Authors
195+
196+
[![Ignacio](https://img.shields.io/badge/github-Ignacio_Elizaga-green.svg?style=social)](https://github.com/ielizaga) [![Aitor](https://img.shields.io/badge/github-Aitor_Cedres-green.svg?style=social)](https://github.com/Zerpet) [![Juan](https://img.shields.io/badge/github-Juan_Ramos-green.svg?style=social)](https://github.com/jujoramos) [![Faisal](https://img.shields.io/badge/github-Faisal_Ali-green.svg?style=social)](https://github.com/faisaltheparttimecoder) [![Adam](https://img.shields.io/badge/github-Adam_Clevy-green.svg?style=social)](https://github.com/adamclevy)
197+
198+

Vagrantfile_gemfire

Lines changed: 0 additions & 43 deletions
This file was deleted.

Vagrantfile_postgres

Lines changed: 0 additions & 86 deletions
This file was deleted.

0 commit comments

Comments
 (0)