Skip to content

Commit f3ca779

Browse files
fenekkufenekku
committed
cli: document new CLI workflow
Closes inveniosoftware/invenio-cli#6 invenio-cli#13
1 parent 787b93e commit f3ca779

File tree

16 files changed

+422
-297
lines changed

16 files changed

+422
-297
lines changed

‎docs/deployment/openshit.md renamed to ‎docs/deployment/openshift.md

File renamed without changes.

‎docs/quickstart/cleanup.md renamed to ‎docs/develop/cleanup.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@
22

33
## Destroy the instance
44

5-
Finally, we want to destroy it. This will take us to a clean state. Note that it destroys images, containers and volumes (the ones defined in the `december-release/docker-compose.full.yml`. ).
5+
Finally, we want to destroy it. This will take us to a clean state. Note that it destroys images, containers and volumes (the ones defined in the `february-release-2/docker-compose.full.yml`. ).
66

77
Stop the application:
88

99
``` console
10-
(your-virtualenv)$ invenio-cli destroy --containers
11-
Destroying RDM application...
10+
$ invenio-cli destroy
11+
TODO: Revisit destroy command...
1212
```

‎docs/quickstart/config.md renamed to ‎docs/develop/config.md

Lines changed: 5 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -31,20 +31,12 @@ When we set `RECORDS_PERMISSIONS_RECORD_POLICY = MyRecordPermissionPolicy`, we a
3131

3232
**Pro tip** : You can type `can_create = []` to achieve the same effect; any empty permission list only allows super users.
3333

34-
That's it configuration-wise. If we try to create a record through the API, your instance will check if you are a super user before allowing you. The same approach to configuration holds for any other setting you would like to override. Now we must stop the current instance, rebuild the application image and re-start it to see our changes take effect.
35-
36-
You can run the same `build` command. If you are in a hurry, you can skip locking dependencies.
37-
38-
``` console
39-
(your-virtualenv)$ invenio-cli server --containers --stop
40-
(your-virtualenv)$ invenio-cli build --pre --containers [--skip-lock]
41-
(your-virtualenv)$ invenio-cli server --containers --start
42-
```
34+
That's it configuration-wise. If we try to create a record through the API, your instance will check if you are a super user before allowing you. The same approach to configuration holds for any other setting you would like to override.
4335

4436
Did the changes work? We are going to try to create a new record:
4537

4638
``` console
47-
$ curl -k -XPOST -H "Content-Type: application/json" https://localhost/api/records/ -d '
39+
$ curl -k -XPOST -H "Content-Type: application/json" https://localhost:5000/api/records/ -d '
4840
{
4941
"access": {
5042
"metadata_restricted": "false",
@@ -74,15 +66,15 @@ As you can see, the server could not know if we are authenticated/superuser and
7466
1- Create a user, for example using the web UI (*sign up* button). Alternatively, you can do so with the CLI, executing the following command (Wait until you read point number *2* before executing):
7567

7668
``` console
77-
invenio-cli users create [email protected] --password=123456
69+
pipenv run invenio users create [email protected] --password=123456
7870
```
7971

8072
Note that the user will have ID 1 if it was the first one created.
8173

8274
2- Grant admin access to it. In order to do so, we only have to add the `-a` flag to the previous command:
8375

8476
``` console
85-
invenio-cli users create [email protected] -a --password=123456
77+
pipenv run invenio users create [email protected] -a --password=123456
8678
```
8779

8880
Note that this user will have ID 2.
@@ -96,7 +88,7 @@ Note that this user will have ID 2.
9688
Afterwards we can test if the new permissions are working correctly.
9789

9890
``` console
99-
$ curl -k -XPOST -H "Authorization:Bearer sHHq1K9y7a2v5doKDRSFmSFOxa1tZDHFcbs31npaxm1sFEt27yomLMt0ynkl" -H "Content-Type: application/json" https://localhost/api/records/ -d '
91+
$ curl -k -XPOST -H "Authorization:Bearer sHHq1K9y7a2v5doKDRSFmSFOxa1tZDHFcbs31npaxm1sFEt27yomLMt0ynkl" -H "Content-Type: application/json" https://localhost:5000/api/records/ -d '
10092
{
10193
"access": {
10294
"metadata_restricted": "false",

‎docs/quickstart/customize.md renamed to ‎docs/develop/customize.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,13 +8,17 @@ We are going to change the logo, take an *svg* file and update your **local** st
88
$ cp ./path/to/new/color/logo.svg static/images/logo.svg
99
```
1010

11-
Then, use the update command:
11+
Then, use the `update` command:
1212

1313
``` console
14-
(your-virtualenv)$ invenio-cli update --containers
15-
Updating static files...
14+
$ invenio-cli update --no-install-js
15+
Collecting statics and assets...
16+
Copying project statics and assets...
17+
Building assets...
1618
```
1719

18-
Go to the browser [*https://localhost/*](https://localhost) or refresh the page. And voila! The logo has been changed!
20+
Passing the `--no-install-js` option, skips re-installation of JS dependencies.
21+
22+
Go to the browser [*https://localhost:5000/*](https://localhost:5000) or refresh the page. And voila! The logo has been changed!
1923

2024
**WARNING**: If you do not see it changing, check in an incognito window, the browser might have cached the logo.

‎docs/develop/init.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Getting Started
2+
3+
You are now ready to learn how to create, customize and iterate on your own
4+
InvenioRDM instance. The previous section showed you what is the equivalent of
5+
a deploy compilation when working with a compiled language: it gets you the final
6+
result, but you wouldn't do that when developing.
7+
8+
When working on your InvenioRDM application, you will want to use the commands
9+
and the workflow we explain in these pages.
10+
11+
## Initialize your file system
12+
13+
We start by creating a new project in a different folder. We will just follow what we
14+
did in the [Preview section](../preview/index.md). Feel free to use your own
15+
project name.
16+
17+
``` console
18+
$ invenio-cli init --flavour=RDM
19+
Initializing RDM application...
20+
project_name [My Site]: February Release 2
21+
project_shortname [february-release-2]:
22+
project_site [february-release-2.com]:
23+
github_repo [february-release-2/february-release-2]:
24+
description [Invenio RDM February Release 2 Instance]:
25+
author_name [CERN]:
26+
author_email [[email protected]]:
27+
year [2020]:
28+
Select database:
29+
1 - postgresql
30+
2 - mysql
31+
3 - sqlite
32+
Choose from 1, 2, 3 (1, 2, 3) [1]:
33+
Select elasticsearch:
34+
1 - 7
35+
2 - 6
36+
Choose from 1, 2 (1, 2) [1]:
37+
-------------------------------------------------------------------------------
38+
39+
Generating SSL certificate and private key for testing....
40+
Can't load /home/youruser/.rnd into RNG
41+
139989104693696:error:2406F079:random number generator:RAND_load_file:Cannot open file:../crypto/rand/randfile.c:88:Filename=/home/youruser/.rnd
42+
Generating a RSA private key
43+
..................++++
44+
..................................++++
45+
writing new private key to 'docker/nginx/test.key'
46+
-----
47+
-------------------------------------------------------------------------------
48+
Creating logs directory...
49+
```
50+
51+
52+
**Notes and Known Issues**
53+
54+
- For now, the only available flavour is RDM (Research Data Management). In the future, there will be others, for example ILS (Integrated Library System).
55+
56+
- You may be prompted with `You've downloaded /home/<username>/.cookiecutters/cookiecutter-invenio-rdm before. Is it okay to delete and re-download it? [yes]:`. Press `[Enter]` in that case. This will download the latest cookiecutter template.
57+
58+
- Some OpenSSL versions display an error message when obtaining random numbers, but this has no incidence (as far as we can tell) on functionality. We are investigating a possible solution to raise less eyebrows for appearance sake.

‎docs/develop/install_locally.md

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Install Locally
2+
3+
In the [Preview Section](../preview/index.md), we have been running all of the
4+
application in docker containers. This saves you the trouble of installing the
5+
instance on your host and gets something visible fast. Now, we will run the
6+
application locally and the database and other services in containers. It's the
7+
best compromise between getting up and running with relevant services fast, while
8+
allowing you to iterate on your local instance quickly.
9+
10+
Before going on, let's move into the project directory:
11+
12+
``` console
13+
$ cd february-release-2
14+
```
15+
16+
To run the application locally, we will need to install it and its dependencies
17+
first. We use the `install` command with the `--pre` flag. We need to add `--pre`
18+
in order to allow `pipenv` to install alpha releases, since many InvenioRDM packages
19+
are now in this phase. Be patient, it might take some time to build.
20+
21+
22+
``` console
23+
$ invenio-cli install --pre
24+
Installing python dependencies...
25+
Symlinking invenio.cfg...
26+
Symlinking templates/...
27+
Collecting statics and assets...
28+
Installing js dependencies...
29+
Copying project statics and assets...
30+
Symlinking assets/...
31+
Building assets...
32+
```
33+
34+
As a result, the Python dependencies for the project have been installed in
35+
a new virtualenv for the application and many of the files in your project directory
36+
have been symlinked inside it.
37+
38+
## Setup the database, Elasticsearch, Redis and RabbitMQ
39+
40+
We need to initialize the database, the indices and so on. For this, we can use
41+
the `services` command. Note that this command is only needed once. Afterwards, you
42+
can stop (not destroy) these services and start again, and your data will still be there.
43+
44+
``` console
45+
$ invenio-cli services
46+
Making sure containers are up...
47+
Creating database...
48+
Creating indexes...
49+
Creating files location...
50+
Creating admin role...
51+
Assigning superuser access to admin role...
52+
```
53+
54+
In case you want to wipe out the data that was there (say to start fresh),
55+
you can use `--force` and nuke the content!
56+
57+
``` console
58+
$ invenio-cli services --force
59+
Making sure containers are up...
60+
Flushing redis cache...
61+
Deleting database...
62+
Deleting indexes...
63+
Purging queues...
64+
Creating database...
65+
Creating indexes...
66+
Creating files location...
67+
Creating admin role...
68+
Assigning superuser access to admin role...
69+
```
70+
71+
**Known issues**:
72+
73+
The Elasticsearch container might crash due to lack of memory. One solution is to increase the maximum allowed allocation per process (See more [here](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/docker.html)). Solving this issue depends on your OS:
74+
75+
On Linux, add the following to ``/etc/sysctl.conf`` on your local machine (host machine):
76+
77+
```console
78+
# Memory mapped max size set for ElasticSearch
79+
vm.max_map_count=262144
80+
```
81+
82+
On macOS, do the following:
83+
84+
```console
85+
screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
86+
# and in the shell
87+
sysctl -w vm.max_map_count=262144
88+
```
89+
90+
## Populate DB
91+
92+
Let's add some content so you can interact a bit with the instance. For this
93+
you will generate 10 random demo records, using the `demo` command:
94+
95+
``` console
96+
$ invenio-cli demo --local
97+
Making sure containers are up...
98+
Populating instance with demo records...
99+
```
100+
101+
We are ready to run it in the next section.

‎docs/quickstart/server.md renamed to ‎docs/develop/run.md

Lines changed: 24 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,15 @@
11
# Run it!
22

3-
## Run the instance
4-
5-
Let's get it rolling!
6-
7-
Once the images have been built, they just need to run. For that, the `server` command is executed. **WARNING: It is the SERVER command, NOT the RUN command**.
3+
Once the application is installed locally and the services are running, our
4+
application just needs to run. For that, the `run` command is executed.
85

96
``` console
10-
(your-virtualenv)$ invenio-cli server --containers
11-
Booting up server...
12-
Starting docker containers. It might take up to a minute.
13-
Containers started use --stop to stop server.
14-
```
15-
16-
**Known issues**:
17-
18-
The Elasticsearch container might crash due to lack of memory. One solution is to increase the maximum allowed allocation per process (See more [here](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/docker.html)). Solving this issue depends on your OS:
19-
20-
On Linux, add the following to ``/etc/sysctl.conf`` on your local machine (host machine):
21-
22-
```console
23-
# Memory mapped max size set for ElasticSearch
24-
vm.max_map_count=262144
25-
```
26-
27-
On macOS, do the following:
28-
29-
```console
30-
screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
31-
# and in the shell
32-
sysctl -w vm.max_map_count=262144
7+
$ invenio-cli run
8+
Making sure containers are up...
9+
Starting celery worker...
10+
Starting up local (development) server...
11+
Instance running!
12+
Visit https://localhost:5000
3313
```
3414

3515
## Use your instance: have fun!
@@ -41,7 +21,7 @@ Are we done? Yes, let the fun begin...
4121
Let's see what is in the instance by querying the API. Using another terminal:
4222

4323
``` console
44-
$ curl -k -XGET https://localhost/api/records/ | python3 -m json.tool
24+
$ curl -k -XGET https://localhost:5000/api/records/ | python3 -m json.tool
4525
{
4626
"aggregations": {
4727
"access_right": {
@@ -71,8 +51,8 @@ $ curl -k -XGET https://localhost/api/records/ | python3 -m json.tool
7151
"created": "2019-12-19T20:15:29.167218+00:00",
7252
"id": "s2pwq-mzw48",
7353
"links": {
74-
"files": "https://localhost/api/records/s2pwq-mzw48/files",
75-
"self": "https://localhost/api/records/s2pwq-mzw48"
54+
"files": "https://localhost:5000/api/records/s2pwq-mzw48/files",
55+
"self": "https://localhost:5000/api/records/s2pwq-mzw48"
7656
},
7757
"metadata": {
7858
"_access": {
@@ -109,7 +89,7 @@ $ curl -k -XGET https://localhost/api/records/ | python3 -m json.tool
10989
**Pro Tip**: You can use [jq](https://github.com/stedolan/jq) for color highlighting:
11090

11191
```console
112-
$ curl -k -XGET https://localhost/api/records/ | jq .
92+
$ curl -k -XGET https://localhost:5000/api/records/ | jq .
11393
...
11494
```
11595

@@ -118,7 +98,7 @@ $ curl -k -XGET https://localhost/api/records/ | jq .
11898
You can create a new record using the API:
11999

120100
```console
121-
$ curl -k -XPOST -H "Content-Type: application/json" https://localhost/api/records/ -d '
101+
$ curl -k -XPOST -H "Content-Type: application/json" https://localhost:5000/api/records/ -d '
122102
{
123103
"access": {
124104
"metadata_restricted": "false",
@@ -140,7 +120,7 @@ $ curl -k -XPOST -H "Content-Type: application/json" https://localhost/api/recor
140120
And then search for it:
141121

142122
``` console
143-
$ curl -k -XGET https://localhost/api/records/?q=doge | python3 -m json.tool
123+
$ curl -k -XGET https://localhost:5000/api/records/?q=doge | python3 -m json.tool
144124
{
145125
"aggregations": {
146126
[...]
@@ -151,7 +131,7 @@ $ curl -k -XGET https://localhost/api/records/?q=doge | python3 -m json.tool
151131
"created": "2019-12-19T13:05:48.479895+00:00",
152132
"id": "pv1dx-rwa61",
153133
"links": {
154-
"self": "https://localhost/api/records/pv1dx-rwa61"
134+
"self": "https://localhost:5000/api/records/pv1dx-rwa61"
155135
},
156136
"metadata": {
157137
[...]
@@ -164,33 +144,32 @@ $ curl -k -XGET https://localhost/api/records/?q=doge | python3 -m json.tool
164144
"total": 1
165145
},
166146
"links": {
167-
"self": "https://localhost/api/records/?sort=bestmatch&q=doge&size=10&page=1"
147+
"self": "https://localhost:5000/api/records/?sort=bestmatch&q=doge&size=10&page=1"
168148
}
169149
}
170150
```
171151

172-
### ** NEW ** Use your browser
152+
### Use your browser
173153

174154
Alternatively, you can use the web UI.
175155

176-
Navigate to https://localhost/search . Note that you might need to accept the SSL exception since it's using a test certificate.
156+
Navigate to https://localhost:5000/search . Note that you might need to accept the SSL exception since it's using a test certificate.
177157
And visit the record page for the newly created record. You will see it has no files associated with it. Let's change that!
178158

179-
### ** NEW ** Upload a file to a record
159+
### Upload a file to a record
180160

181161
For demonstration purposes, we will attach this scientific photo:
182162

183163
![Very scientific picture of a shiba in the snow](https://images.unsplash.com/photo-1548116137-c9ac24e446c9?ixlib=rb-1.2.1&ixid=eyJhcHBfaWQiOjEyMDd9&auto=format&fit=crop&w=1350&q=80)
184164

185165
By <a href="https://unsplash.com/@matyssik" target="_blank" rel="noopener noreferrer">Ian Matyssik</a>
186166

187-
188167
Save it as `snow_doge.jpg` in your current directory. Then upload it to the record:
189168

190169
**WARNING** Change `pv1dx-rwa61` in the URLs below for the recid of your record.
191170

192171
```
193-
$ curl -k -X PUT https://localhost/api/records/pv1dx-rwa61/files/snow_doge.jpg \
172+
$ curl -k -X PUT https://localhost:5000/api/records/pv1dx-rwa61/files/snow_doge.jpg \
194173
-H "Content-Type: application/octet-stream" \
195174
--data-binary @snow_doge.jpg
196175
```
@@ -202,5 +181,7 @@ This file can then be previewed on the record page and even downloaded.
202181
We have reached the end of this journey, we are going to stop the instance. Nonetheless you can keep testing and playing around with configurations!
203182

204183
``` console
205-
(your-virtualenv)$ invenio-cli server --containers --stop
206-
```
184+
^C
185+
Stopping server and worker...
186+
Server and worker stopped...
187+
```

0 commit comments

Comments
 (0)