Skip to content

Commit ab5e060

Browse files
authored
Merge pull request #92 from oslabs-beta/main
4.0 updates
2 parents e2b926c + 1be5fbe commit ab5e060

File tree

97 files changed

+26134
-16734
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

97 files changed

+26134
-16734
lines changed

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@
55
node_modules/
66
server/db_schemas
77

8+
coverage/
9+
810
# output from webpack
911
dist/
1012

@@ -19,3 +21,5 @@ package-lock.json
1921
.elasticbeanstalk/*
2022
!.elasticbeanstalk/*.cfg.yml
2123
!.elasticbeanstalk/*.global.yml
24+
.cert/
25+
.*/

Contributors.md

Lines changed: 9 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -2,41 +2,25 @@
22

33
Below is a list of features and improvements to which you can contribute. If you have any additional ideas, please raise an issue or implement them as well!
44

5-
- Exporting executable SQL query with every modification to your schema
6-
- After modifying/editing the database schemas, users should be able to generate executable queries isolated from their database which decreases security concerns. Users should be able to append all the query changes at the end of the new file and save this in their local machines. The codebase for query generation is in place, however not yet implemented into the current state management and front-end rendering.
7-
- Logging modeling
8-
- After connnection to the database is made the user should be able to view, modify and save the current log settings. This will assist database administrators in keeping efficient log setup in mind during the early stages of DB modeling. The codebase is in place however not yet implemented into the current state management and front-end rendering.
9-
- Add ability to log out from application, as currently log-in status will persist until app close
10-
- Save button
11-
- Allow the user to save ER diagrams into projects that can be retrieved and reloaded for future use.
12-
- Undo / Redo buttons
13-
- Allow the user to undo or redo changes made.
14-
- History
15-
- Display history of the schema state after each change.
165
- Delete Table
176
- Add a command to delete a table from the canvas, making sure to implement checks to maintain data integrity
18-
- Investigate React Flow additional functionality (there are tons!)
19-
- Take advantage of React Flow for a more dynamic interaction of the ER diagram.
20-
- Refactoring TypeScript
21-
- There are some components that are not fully typed in TypeScript.
22-
- Testing
23-
- Implementation of testing suite
247
- Logging needs a default settings list and direct reimport into live database.
258
- The SQL code to import has been placed into the DB_Schemas folder along with SQL Schema dumps. Also, log setting functionality is currently limited to Postgres databases and the code needs to be expanded to query and display log setting info from MySQL databases.
26-
- Security setting models need to be put in place for database tables as well as log files.
279
- Db-schemas
2810
- The schemas are saved under db_schemas and is saved in the server. Have a way to delete the schemas on the server from the front end. It would also be convenient to add these schemas to a list of existing schemas and be able to pull these schemas to display for the user to avoid having to make a new connection to the same database multiple times.
2911
- Expand compatibility with other SQL database such as Oracle SQL, Microsoft SQL, IBM Db2, etc.
3012
- Add additional themes and graphical options to canvas and tables
31-
- Clean up unused dependencies from pre-3.0
13+
- Clean up unused dependencies from pre-4.0
3214
- Connecting to PostgreSQL database
3315
- When connecting to PostgreSQL to grab the data from the database, it takes a really long time to load (approximately 2 minutes); further investigate pg_dump alternatives for faster loading.
16+
- Integration and End-to-End testing for the codebase.
17+
- Live query feedback so users can see what their changes to the canvas look like as queries.
3418

3519
Known bugs/issues
3620

37-
- Table connection does not automatically render after a foreign key is added and the row data is saved. Currently, there is a popup message instructing the user to perform additional steps as a workaround.
38-
- Generate a cleaner initial table position that takes into account the dimensions of the generated tables.
39-
- Generate handle positions at primary key and foreign key row that the connection references.
40-
- Create logic for only primary keys (rows) to be a target for foreign keys.
41-
- The PrimaryKeyName in the References array within the database schema object is incorrect. It is pointing to the ReferencesPropertyName causing connection label to be incorrect. Review and update the SQL parser to correct this issue.
42-
- One part of the main page updates dark/light theme visibly later than the rest upon toggle
21+
- Table connection (handle) does not automatically render after a foreign key is added and the column data is saved.
22+
- In production, remote postgres database connections do not work. We suspect this is due to container environment (nodeJS) not having access to pg_dump command.
23+
- In development, WSL/Ubuntu users may have trouble running pg_dump due to version mismatch. Run `sudo apt-get --purge remove postgresql` to clear configuration files and zombie directories before installing postgresql fresh.
24+
- Undo function will not regress the state of the canvas to the point where it is empty. Whether this is a design choice or an issue is up to you.
25+
26+

Dockerfile

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
FROM node:16.17.0
2+
WORKDIR /usr/src/app
3+
COPY . /usr/src/app
4+
RUN npm ci && npm cache clean --force && npm run build
5+
EXPOSE 3000
6+
ENTRYPOINT npm run start

README.md

Lines changed: 75 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -1,86 +1,76 @@
11
<div align="center">
22

33
<a href="https://makeapullrequest.com">![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)</a>
4-
<a href="https://img.shields.io/badge/version-1.0.1-blue">![Version](https://img.shields.io/badge/version-1.0.1-blue)</a>
4+
<a href="https://img.shields.io/badge/version-1.0.1-blue">![Version](https://img.shields.io/badge/version-4.0.0-blue)</a>
55
![GitHub Stars](https://img.shields.io/github/stars/oslabs-beta/JAKT?color=%23fb7182)
66
![MIT License](https://img.shields.io/badge/license-MIT-yellow)
77

88
</div>
99

1010
<div align="center">
1111

12-
![logo2x](https://user-images.githubusercontent.com/101940529/198836631-31a657f7-feaa-4247-8abc-cedb86540e21.png)
12+
![logo](https://user-images.githubusercontent.com/101940529/198836631-31a657f7-feaa-4247-8abc-cedb86540e21.png)
1313

1414
</div>
1515

1616
<h1 align="center">Visualize, modify, and build your database with dbSpy!</h1>
17-
<p>dbSpy is an open-source data modeling tool to facilitate relational database development.</p>
17+
<p align="center">An open-source data modeling tool to facilitate relational database development</p>
18+
<p align="center">dbSpy是一个促进关系数据库开发的开源数据建模工具。</p>
1819

19-
<!-- dbSpy is an open-source visualization tool to facilitate relational database model development using entity relationship diagrams and homogeneous database migration -->
20+
### Use Cases
21+
22+
- Connecting to an existing remote PostgreSQL and MySQL database
23+
- Renders an ER diagram of the existing database and provides an interface for users to both modify existing tables and create new tables. A log of changes is stored, and at any point, the user can execute a transaction containing the changes, such that they are reflected in the existing database.
24+
- Uploading your database's SQL file
25+
- Renders an ER diagram for the provided SQL file (db dump) and provides an interface for a user to both modify existing tables and create new tables. Changes are converted into the corresponding queries, which the user can view and execute on their own database outside of dbSpy.
26+
- Seamlessly visualize relational databases
27+
- After connecting database tables with one another, users are able to render the relationship connections of individual tables by clicking on the tables.
28+
- Creating a database + ER diagram from scratch
29+
- Provides a canvas for users to create a database by using ER diagrams, thus creating a blueprint for engineering database structures.
2030

2131
---
2232

2333
### Key Features
2434

25-
1. **Database Connection:** Connect to a SQL database
26-
27-
2. **Compatible SQL Database:** Current dbSpy is compatible with PostgreSQL and MySQL database
28-
29-
3. **Database File Upload:** Upload a database's SQL file
35+
1. **Database Uploads:** Connect to remote SQL database or upload local SQL files
3036

31-
4. **ER Diagrams:** Visualize the entity relationship diagram of a database
37+
2. **ER Diagram Visuals:** Visualize the entity relationship diagram of a database with dynamic handle placement
3238

33-
5. **Visualizing Individual Table Connections** Relationships of individual tables are now easily identified when clicking on a table.
39+
3. **Table Relationship Visuals** Relationships of individual tables are easily identified when clicking on a table.
3440

35-
6. **Schema Modification:** Modify a database's schema
41+
4. **Schema Modification:** Easily modify a database's schema through a simple UI
3642

37-
7. **Database Building:** Create a new database from scratch using entity relationship diagrams
43+
5. **Guided Database Building:** Create a new database from scratch using entity relationship diagrams to ensure the integrity of the database
3844

39-
8. **Screenshot:** Take a screenshot of the canvas with all your tables
45+
6. **Database Undo/Redo:** Provides a history of edits for easier backtracking
4046

41-
9. **Dark Mode** Additional visual setting to provide a more comfortable viewing experience in low-light environements
42-
![dbSpyTheme](https://user-images.githubusercontent.com/101940529/198836752-9727c0b4-4a8f-459b-be39-81922be15fd3.gif)
47+
7. **Query Generator:** Query generator generates executable SQL queries
4348

44-
## Future Enhancements
49+
8. **User Sessions:** Sign up/Log in securely with either Google OAuth or JWTs/Bcrypt
4550

46-
1. **SQL Query Generator:** Generate an executable SQL query with every modification to your schema
51+
9. **Save/Load:** Store and reload recent database sessions through your user account
4752

48-
2. **Time Travel:** Provides a history of edits for easier backtracking
49-
50-
3. **Exporting Queries:** Allows updated changes of queries to be saved as a SQL file for client imports to their database
51-
52-
4. **Log Modeling:** Provides a tool for admins to pull, view, modify, and export log settings from an existing database
53-
54-
5. **Database Integrity Assurance** Database Logic Check is performed as the client edits the database to ensure the integrity of the database
55-
56-
---
57-
58-
### Use Cases
59-
60-
- Connecting to an existing database
61-
- Renders an ER diagram of the existing database and provides an interface for users to both modify existing tables and create new tables. A log of changes is stored, and at any point, the user can execute a transaction containing the changes, such that they are reflected in the existing database.
62-
- Uploading your database's SQL file
63-
- Renders an ER diagram for the provided SQL file (db dump) and provides an interface for a user to both modify existing tables and create new tables. Changes are converted into the corresponding queries, which the user can view and execute on their own database outside of dbSpy.
64-
- Seamlessly visualize relational database
65-
- After connecting database tables with one another, users are able to render the relationship connections of individual tables by clicking on the tables.
66-
- Starting an ER diagram from scratch
67-
- Provides a canvas for users to create a database by using ER diagrams, thus creating a blueprint for engineering database structures.
53+
10. **Dark Mode:** Visual settings to provide a more comfortable viewing experience in low-light environements
6854

55+
<img src="images/Darkmode.png">
6956
---
7057

7158
### Getting started
72-
59+
You will need your own mySQL and redis databases for backend functions, and Google Cloud project for OAuth.
7360
- Fork and clone this repo
61+
- Add a db_schemas folder in server directory
7462
- Add a .env file to the root directory
7563
- Go to the Google Cloud Platform Console
76-
- Set up OAuth 2.0 credentials as laid out in <a href="https://support.google.com/cloud/answer/6158849?hl=en">here</a>
64+
- Set up OAuth 2.0 credentials as laid out in <a href="https://support.google.com/cloud/answer/6158849?hl=en">here</a>. Make sure
65+
your authorized redirect URI in the credentials you create matches the GOOGLE_AUTH_CALLBACK variable below.
7766
- Populate the .env file with the newly created code below:
7867

7968
```bash
8069
GOOGLE_AUTH_CALLBACK = ''
8170
DATABASE_URL = [mysql url for storing user data]
8271
DEV_SERVER_ENDPOINT = 'http://localhost:3000'
8372
DEV_CLIENT_ENDPOINT = 'http://localhost:8080'
73+
CLIENT_ENDPOINT = 'http://localhost:3000'
8474
GOOGLE_AUTH_CALLBACK = 'http://localhost:3000/api/oauth/google'
8575
GOOGLE_AUTH_CLIENT_ID = [google auth client id]
8676
GOOGLE_AUTH_CLIENT_SECRET = [google auth client secret]
@@ -92,13 +82,13 @@ ENVIRONMENT = "development"
9282
TOKEN_KEY = [any string]
9383
```
9484

95-
- Install the dependencies:
85+
- Run the following below:
9686

9787
```bash
9888
$ npm install
9989
```
10090

101-
- Execute the following command to populate your mySql database with the user table.
91+
- Execute the following command to populate your mySql database with a users table:
10292
```bash
10393
$ npm run seed
10494
```
@@ -109,92 +99,87 @@ $ npm run seed
10999
$ npm run dev
110100
```
111101

112-
- Make changes with comments
113-
- Add appropriate tests and ensure they pass
114-
- Commit your changes and make a pull request
115-
116102
---
117103

118-
### How to Use
104+
### Connecting with remote MySQL
105+
106+
- To connect with the SQL database, we must create and configure SSL certificates.
107+
108+
1. Install mkcert; you can learn more about mkcert [here](https://github.com/FiloSottile/mkcert)
109+
```bash
110+
npm install -g mkcert
111+
```
112+
2. Run the following script
113+
```bash
114+
npm run cert
115+
```
116+
117+
---
119118

120-
How To Start
119+
# How to Use
121120

122-
1. Sign up or login with Google Auth to begin.
123-
2. You will be redirected to the display page where you can connect, upload, or build a mySQL or Postgres database.
121+
### Connecting to an existing database
124122

125-
Connecting to an existing database
126-
![Screenshot 2022-10-29 093105](https://user-images.githubusercontent.com/101940529/198837154-072d2d90-3f81-4fed-ab17-8301f1387b5c.png)
123+
<img src="images/Existing%20Databse.png">
127124

128125
1. Click on the "Connect Database" button under the Action section on the left side of the page. This will open a sidebar on the right side of the page.
129126
2. Select the database type from the dropdown.
130127
3. Input your database URI information OR database connection credentials and click on "Connect".
131128
4. Once the connection to your database is established, the canvas will render and generate the tables and their relationships.
132129

133-
![dbSpy_connectToDB](https://github.com/open-source-labs/dbSpy/blob/main/src/assets/ScreenshotDemo.png?raw=true)
130+
<img src="images/Canvas%20renders.png">
134131

135-
Uploading your database's SQL file
132+
### Uploading your database's SQL file
136133

137134
1. Locate and click on the "Upload SQL File" button under the Action section on the left side of the page. This will open a dialog box for you to select the SQL file you wish to upload.
138135
2. Once the file uploads, the canvas will render and generate the tables and their relationships.
139136

140-
Starting an ER diagram from scratch
137+
### Starting an ER diagram from scratch
141138

142139
1. Locate and click on the "Build Database" button under the Action section on the left side of the page. This will render the canvas for you to create your database schema.
143140
2. Click on "Add Table" to name the first table and start building your database.
144141

145-
Adding a new row to a table
142+
## Adding column(s) to a table
146143

147-
1. To add a new row in a table, click on the add icon on the top right of the table node. This will render a new row in the table in edit mode.
148-
![Screenshot 2022-10-29 093211](https://user-images.githubusercontent.com/101940529/198837146-e81a858a-a7aa-41d8-9396-e4db38776fa5.png)
144+
1. To add a new column in a table, click on the add icon on the top right of the table node. This will render a new column in the table in edit mode.
145+
<img src="images/Create%20tables.png">
149146

150-
Editing an existing row in a table
147+
### Editing an existing column in a table
151148

152-
1. Click on the edit icon of the specific row that you want to edit.
153-
2. You can make changes to row fields based on the selections provided.
149+
1. Click on the edit icon of the specific column that you want to edit.
150+
2. You can make changes to column fields based on the selections provided.
154151
3. Click on the save icon to keep your changes.
155152
4. Click on the cancel icon to cancel your changes.
156153

157-
Adding a foreign key reference to a row
154+
### Adding a foreign key reference to a column
158155

159-
1. While editing a row, if setting Foreign Key (FK) to "true", the Foreign Key Reference sidebar will appear on the right side of the screen.
160-
2. In the Foreign Key Reference sidebar, enter the following:
161-
- Primary Key Name - the name of the Primary Key of the table that the Foreign Key is referencing
162-
- Reference Key Name - the name of this Foreign Key
163-
- Primary Table Name - the name of the Primary Key's table
164-
- Reference Table Name - the name of this Foreign Key's table
165-
- IsDestination - should be set to "false"
166-
- Constraint Name - enter constraint information OR null
156+
1. While editing a column, if setting Foreign Key (FK) to "true", the Foreign Key Reference sidebar will appear on the right side of the screen.
157+
2. In the Foreign Key Reference sidebar, use the displayed dropdowns to create references between tables.
167158
3. Click save or cancel to proceed.
168159
4. Click on the save icon to keep your changes or click on the cancel icon to cancel your changes.
169-
5. If changes are saved, an alert will appear that reads "click EDIT then SAVE on the target table row."
170-
6. Following the alert prompt will render the connection and connection label between the two tables.
171160

172-
Deleting an existing row in a table
161+
## Deleting an existing column in a table
173162

174-
1. Click on the delete icon of the specific row that you want to delete.
175-
2. Click on the confirm icon to delete the row.
163+
1. Click on the delete icon of the specific column that you want to delete.
164+
2. Click on the confirm icon to delete the column.
176165
3. Click on the cancel icon to cancel the deletion.
177166

178-
Navigating the canvas
167+
### Navigating the canvas
179168

180169
1. Located on the bottom left corner of the canvas, you will find the canvas control panel.
181170
- Zoom On [+] - this button zooms into the canvas.
182171
- Zoom Out [-] - this button zooms out of the canvas.
183172
- Fit View [[]] - this button zooms to fit all the tables in focus.
184-
- Toggle Interactivity [lock] - this button toggles on and off all table and row interactivity.
173+
- Toggle Interactivity [lock] - this button toggles on and off all table and column interactivity.
185174
- Screenshot [camera] - this button saves a .png file of the current canvas view.
186175

187176
---
188177

189-
### Request a feature
190-
191-
- Submit an appropriately tagged GitHub issue
192-
193-
---
194-
195-
### How to contribute
178+
### Future contributions and requests
196179

197-
- Click on [Contributions](Contributors.md) for more details.
180+
- If you'd like to contribute, please find a list of possible contributions [here](Contributors.md)
181+
- If you have any requests, please submit an appropriately tagged GitHub issue
182+
- PRs are welcome otherwise
198183

199184
---
200185

@@ -213,9 +198,13 @@ Navigating the canvas
213198
- Kevin Wang • [LinkedIn](https://www.linkedin.com/in/kevin-w-b841b13/)[Github](https://github.com/kwang929)
214199
- Kris Magat • [LinkedIn](https://www.linkedin.com/in/kmag/)[Github](https://github.com/KrisMagat)
215200
- Santiago Gil Maya • [LinkedIn](https://www.linkedin.com/in/santiago-gil-929721121/)[Github](https://github.com/santiago-gil)
201+
- Alexander Tu • [LinkedIn](https://www.linkedin.com/in/atu816/)[Github](http://github.com/atu816)
202+
- Michael Costello • [LinkedIn](https://www.linkedin.com/in/mcostello-swe/)[Github](https://github.com/neighbor-peace)
203+
- Steven Geiger • [LinkedIn](https://www.linkedin.com/in/sgeiger9/)[Github](https://github.com/geistnine)
204+
- Yufa Li • [LinkedIn](https://www.linkedin.com/in/yufa-li/)[Github](https://github.com/01001101CK)
216205

217206
---
218207

219208
### License
220209

221-
- dbSpy is developed under the MIT license.
210+
dbSpy is developed under the MIT license. See more about dbSpy4.0 updates [here](https://github.com/oslabs-beta/dbSpy/releases/tag/v4.0.0).

babel.config.js

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
module.exports = {
2+
presets: [
3+
['@babel/preset-env', {targets: {node: 'current'}}],
4+
'@babel/preset-typescript', '@babel/preset-react'
5+
],
6+
};

images/Add foreign key.png

77.4 KB
Loading

images/Canvas renders.png

989 KB
Loading

images/Create tables.png

332 KB
Loading

images/Darkmode.png

486 KB
Loading

images/Existing Databse.png

447 KB
Loading

0 commit comments

Comments
 (0)