Skip to content

Commit 983ca48

Browse files
authored
Merge pull request #913 from ctti-clinicaltrials/dev
Release 6.1.1
2 parents ab2be06 + 27aa4dc commit 983ca48

File tree

16 files changed

+1703
-163
lines changed

16 files changed

+1703
-163
lines changed

Gemfile.lock

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -174,7 +174,7 @@ GEM
174174
nokogiri (~> 1)
175175
rake
176176
mini_mime (1.0.2)
177-
mini_portile2 (2.5.3)
177+
mini_portile2 (2.6.1)
178178
minitest (5.14.2)
179179
multi_json (1.15.0)
180180
multipart-post (2.1.1)
@@ -183,8 +183,8 @@ GEM
183183
net-ssh (6.1.0)
184184
netrc (0.11.0)
185185
nio4r (2.5.4)
186-
nokogiri (1.11.4)
187-
mini_portile2 (~> 2.5.0)
186+
nokogiri (1.12.5)
187+
mini_portile2 (~> 2.6.1)
188188
racc (~> 1.4)
189189
normalize-rails (8.0.1)
190190
parallel (1.20.1)

README.md

Lines changed: 51 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -38,53 +38,45 @@ You can find information about the ClinicalTrials.gov beta API here: https://cli
3838

3939
## Getting Started
4040

41-
1. Install PostgreSQL 13.
41+
1. Install PostgreSQL 13. If you have a Mac you can use brew to install PostgreSQL.
42+
`brew install postgresql`
43+
4244
Here are some links that could help you setup
4345
https://www.postgresql.org/download/linux/ubuntu/
4446
https://www.postgresql.org/download/macosx/
4547
Note: do not delete the template databases, meaning template0 and template1. They are what PostgreSQL uses to create all other databases. If you remove them you will no longer be able to make new databases.
4648

47-
2. Now we will focus on the “.pgpass” file for storing your connection info. Here is a document about the “.pgpass” file
48-
https://www.postgresql.org/docs/current/libpq-pgpass.html
49-
The “.pgpass” should be at your root. You can edit it using Vim or your preferred editor that lets you edit from the terminal. I will explain editing it using Vim commands.
50-
`vim ~/.pgpass` this directs vim to open the file you want to edit. It will create that file if it does not already exist.
51-
`i` this triggers insert mode (it will say insert at the bottom) so you can edit the file.
52-
Add “hostname:port:database:username:password” to the file but fill in the words with your info.
53-
Example: `localhost:5432:*:postgres:pg_password`
54-
Press the `esc` button to leave insert mode.
55-
`:xa` will save and close the file.
49+
2. Now we will create the roles you need for running AACT.
50+
`psql postgres` this allows you to enter the postgres database that comes with PostgreSQL.
51+
`postgres=# create role <your_aact_superuser> login password '<your_superuser_password>’;`
52+
`postgres=# alter user <your_aact_superuser> with superuser;`
53+
`postgres=# create role read_only;`
54+
`postgres=# create database aact_alt;`
55+
`postgres=# \q` this exits out of psql
56+
Verify your new user can login to psql with command: `psql -U <your_aact_superuser> -d postgres`
57+
You can exit the shell once you see you can log in.
58+
59+
3. If your terminal asks for a password when logging in to psql you can give it the password automatically by adding it to the “.pgpass” file. If you haven’t been asked for a password, you can skip this step.
60+
The “.pgpass” should be at your root.
61+
`echo 'localhost:5432:aact:<superuser_name>:<superuser_password>' >> ~/.pgpass`
5662
Now check that you can login to psql without giving a password
57-
`psql -U postgres -d template1`
58-
Look here for more Vim commands https://www.linux.com/training-tutorials/vim-101-beginners-guide-vim/
59-
60-
3. Now we will create the roles you need for running AACT.
61-
`psql -U postgres -d template1` this allows you to enter the template1 database that comes with PostgreSQL.
62-
`template1=# create role <your_aact_superuser> login password '<your_superuser_password>’;`
63-
`template1=# alter user <your_aact_superuser> with superuser;`
64-
`template1=# create role read_only;`
65-
`template1=# \q` this exits out of psql
66-
Verify your new user can login to psql with command: `psql -U <your_aact_superuser> -d template1`
67-
You can exit the shell once you see you can log in.
68-
69-
4. Now we want to store the environmental variables for the superuser that you created in the previous step. That is the user you will be using within AACT to make database changes.
70-
`vim ~/.bash_profile` You can switch out bash_profile for your usual file for storing environmental variables, such as an zshrc file.
71-
`i` Enter insert mode and add the following to the file
72-
`export AACT_DB_SUPER_USERNAME=<your_aact_superuser>`
73-
`export AACT_PASSWORD=<your_superuser_password>`
74-
<br>
75-
By default, AACT saves files it creates in a directory under /aact-files. (Note, this is in the server root directory, not the root of the AACT application.) To override this, use the AACT_STATIC_FILE_DIR environment variable to define a different directory. Otherwise, you will need to create /aact-files at the server root directory and change permissions on it so that the rails application owner has permission to read/write to that directory.
76-
`export AACT_STATIC_FILE_DIR=public/static` *we are working towards removing the need to add this variable*
77-
<br>
78-
The pg gem (used by AACT) will need to know where to find your PostgreSQL installation. For Mac you can direct it to that location by adding
79-
`export PATH=$PATH:/Library/PostgreSQL/x.y/bin` *change x.y to the version number of your PostgreSQL installation, like: `export PATH=$PATH:/Library/PostgreSQL/13/bin`*
80-
https://wikimatze.de/installing-postgresql-gem-under-ubuntu-and-mac/
81-
<br>
82-
Press the `esc` button to leave insert mode, then `:xa` to save and close the file.
83-
<br>
84-
`source ~/.bash_profile` refreshes/establishes the terminals connection to the file so it will pick up the variables you added.
85-
*Note: you will need to use the source command whenever you edit the “.bash_profile” or open a new terminal.*
86-
<br>
87-
You can add `your_aact_superuser` to the “.pgpass” file to save yourself having to type the password when logging into the shell. Add `hostname:port:database:<superuser_name>:<superuser_password>` to your pgpass file. If you aren’t sure how to do it follow the guidance in step 2.
63+
`psql -U aact_pg_user -d postgres`
64+
You can exit the shell once you see you can log in without a password.
65+
66+
Here is a document about the “.pgpass” file https://www.postgresql.org/docs/current/libpq-pgpass.html.
67+
68+
4. Now we want to store the environmental variables for the superuser that you created in the previous step. That is the user you will be using within AACT to make database changes. You can store these wherever is appropriate for your system. On a Mac you can store it in your “.zshrc”. On all systems you can also store it in your “.bash_profile” or “.bashrc”.
69+
For the following commands I’m storing variables in the “.zshrc” file, change out that file in the commands for the one you use for storing variables.
70+
`echo 'export AACT_DB_SUPER_USERNAME=<your_aact_superuser>' >> ~/.zshrc`
71+
`echo 'export AACT_PASSWORD=<your_superuser_password>' >> ~/.zshrc`
72+
`echo 'export AACT_PUBLIC_DATABASE_NAME=aact' >> ~/.zshrc`
73+
`echo 'export AACT_ALT_PUBLIC_DATABASE_NAME=aact_alt' >> ~/.zshrc`
74+
`echo 'export PUBLIC_DB_USER=<your_aact_superuser>' >> ~/.zshrc`
75+
`echo 'export PUBLIC_DB_PASS=<your_superuser_password>' >> ~/.zshrc`
76+
77+
`source ~/.zshrc` to load the variables into the terminal session.
78+
79+
Depending on where you store the variables you may need to call `source` on that file each time you open a new terminal. This is not necessary for “.zshrc”.
8880

8981
5. Clone this repo: `git clone git@github.com:ctti-clinicaltrials/aact.git`
9082
Note: Cloning with a ssh url requires that your local ssh key is saved to Github. The key verifies your permission to push and pull so you won't have to log in. If you haven't saved your ssh key to Github use the html url instead when cloning.
@@ -94,6 +86,11 @@ You can find information about the ClinicalTrials.gov beta API here: https://cli
9486
7. Install a ruby version manager like rbenv, then install Ruby 2.6.2
9587

9688
8. Bundle install
89+
The pg gem (used by AACT) may have trouble finding your PostgreSQL installation. If not, skip this step.
90+
For Mac you can direct it to the right location location by adding
91+
`echo ‘export PATH=$PATH:/Library/PostgreSQL/x.y/bin’ >> ~/.zshrc` change x.y to the version number of your PostgreSQL installation.
92+
Example: `export PATH=$PATH:/Library/PostgreSQL/13/bin`
93+
https://wikimatze.de/installing-postgresql-gem-under-ubuntu-and-mac/
9794

9895
9. Use the "connections.yml.example" file and copy it to the file "connnections.yml" and just update what needs to be updated for the local environment.
9996
In the terminal, type `cp connections.yml.example connections.yml`.
@@ -105,6 +102,10 @@ You can find information about the ClinicalTrials.gov beta API here: https://cli
105102
`bin/rake db:migrate`
106103
`bin/rake db:migrate RAILS_ENV=test`
107104

105+
11. Grant read_only privileges
106+
`bin/rake grant:db_privs:run`
107+
`bin/rake grant:db_privs:run RAILS_ENV=test`
108+
108109
<br>
109110
<br>
110111

@@ -115,12 +116,16 @@ You can find information about the ClinicalTrials.gov beta API here: https://cli
115116
## Populating the Database
116117

117118
The seed files are out of date so **DO NOT** call `db:seed`. Instead use the custom rake tasks.
118-
These are your options:
119-
* `bin/rake db:restore_from_file[<path_to_file>,<database_name>]` *this method is currently not working*
120-
For this option go to https://aact.ctti-clinicaltrials.org/snapshots and download a copy of the database. Give this task the path to the file you downloaded and it will upzip it before using it to populate the database.
119+
These are your options:
120+
* `bin/rake db:restore_from_file[<path_to_file>,<database_name>]`
121+
For this option go to https://aact.ctti-clinicaltrials.org/snapshots and download a copy of the database. Unzip the snapshot folder.
122+
The file path will likely look like: `~/Downloads/<unzipped_snapshot_folder>/postgres_data.dmp`
123+
Example: `~/Downloads/20210906_clinical_trials/postgres_data.dmp`.
124+
Give this task the path to the postgres_data.dmp file and it will use it to populate the database.
125+
Example: `bin/rake "db:restore_from_file[~/Downloads/20210906_clinical_trials/postgres_data.dmp,aact]"`
121126
* `bin/rake db:restore_from_url[<url>,<database_name>]`
122-
For this option go to https://aact.ctti-clinicaltrials.org/snapshots and copy the link for one of the database copies. Give this task the url you copied and it will download the file, upzip it, and use it to populate the database.
123-
*Note: the following rake tasks take a very long time to run*
127+
For this option go to https://aact.ctti-clinicaltrials.org/snapshots and copy the link for one of the database copies. Give this task the url you copied and it will download the file, unzip it, and use it to populate the database.
128+
*Note: the rake tasks below take a very long time to run. You should not set full_featured to true if working locally.*
124129
* `bin/rake db:load[<days_back>,<event_type>,<full_featured>]`
125130
The days back is an integer, the event_type only takes "full" or "incremental", full_featured is a boolean. You do not have to give it any parameters. If you have no studies it will populate your database with all the studies.
126131

app/mailers/notifier.rb

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,9 @@ def send_msg(schema, email_addr, subject, body)
1313
@schema = schema
1414
mail(to: email_addr, subject: subject, from: 'ctti@duke.edu')
1515
end
16+
17+
def report_diff(email)
18+
subj="Differences between ClinicalTrials.gov and AACT"
19+
mail(to: email, subject: subj, from: 'ctti@duke.edu')
20+
end
1621
end

0 commit comments

Comments
 (0)