Skip to content

Commit c50427e

Browse files
Hagellach37Hagellach37
authored
Merge pull request #395 from mapswipe/dev
Team Manage and Projects for teams
2 parents 7104de6 + 3146806 commit c50427e

File tree

93 files changed

+591873
-1239
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

93 files changed

+591873
-1239
lines changed

‎.travis.yml‎

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -29,24 +29,18 @@ services:
2929
- docker
3030

3131
before_install:
32-
- pip install --upgrade pip setuptools
33-
- pip install flake8 black
32+
- echo "$SERVICE_ACCOUNT_KEY" > mapswipe_workers/serviceAccountKey.json
33+
- echo "MOCK_SERVICE_ACCOUNT_KEY" > postgres/serviceAccountKey.json
3434

3535
install:
36-
- pip install mapswipe_workers/
37-
- mkdir --parents ~/.config/mapswipe_workers
38-
- mkdir --parents ~/.local/share/mapswipe_workers
39-
- echo "$SERVICE_ACCOUNT_KEY" > ~/.config/mapswipe_workers/serviceAccountKey.json
40-
- docker build --tag mapswipe_postgres --file postgres/Dockerfile-dev postgres/
41-
42-
before_script:
43-
- docker run --detach --publish 5432:5432 --name mapswipe_postgres -e POSTGRES_DB="$POSTGRES_DB" -e POSTGRES_USER="$POSTGRES_USER" -e POSTGRES_PASSWORD="$POSTGRES_PASSWORD" mapswipe_postgres
44-
- docker-compose up --build --detach firebase_deploy
36+
- pip install --upgrade pip setuptools
37+
- pip install flake8 black
4538

4639
script:
40+
- docker-compose up --build --detach firebase_deploy
41+
- docker-compose run mapswipe_workers python -m unittest discover --verbose --start-directory tests/unittests/
42+
#- docker-compose run mapswipe_workers python -m unittest discover --verbose --start-directory tests/integration/
4743
# - TODO: Run black and flake8
48-
- python -m unittest -v tests.unittests.test_gdal
49-
# - TODO: python -m unittest discover -v -s mapswipe_workers/tests/unittests/ -p 'test_*.py'
5044

5145
# before_deploy:
5246
# # SSH setup to deploy to server after build.

‎docker-compose.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,8 @@ services:
5353
context: mapswipe_workers/
5454
environment:
5555
FIREBASE_DB: '${FIREBASE_DB}'
56-
FIREBASE_API_KEY: '{FIREBASE_API_KEY}'
57-
FIREBASE_TOKEN: '{FIREBASE_TOKEN}'
56+
FIREBASE_API_KEY: '${FIREBASE_API_KEY}'
57+
FIREBASE_TOKEN: '${FIREBASE_TOKEN}'
5858
GOOGLE_APPLICATION_CREDENTIALS: 'serviceAccountKey.json'
5959
POSTGRES_PASSWORD: '${POSTGRES_PASSWORD}'
6060
POSTGRES_USER: '${POSTGRES_USER}'

‎docs/source/dev_setup.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,17 @@ On how to setup development environment and how to deploy functions to the Fireb
134134
For more information refer to the official [Reference on Cloud Function for Firebase](https://firebase.google.com/docs/reference/functions/). For example function take a look at this [GitHub repository](https://github.com/firebase/functions-samples).
135135

136136

137+
## Travis Setup
138+
139+
Configuration for travis setup is utilizing environment variables.
140+
141+
One difference to production or development setup is that the service account key as json is stored in the environment variables `FIREBASE_CONFIG` as text. To make this work special characters as to be escaped first. This command will simply escape every character:
142+
143+
```bash
144+
sed -e 's/./\\&/g; 1{$s/^$/""/}; 1!s/^/"/; $!s/$/"/' serviceAccountKey.json
145+
```
146+
147+
137148
## Database Backup
138149

139150
### Firebase

‎docs/source/index.rst‎

Lines changed: 6 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,15 @@ Welcome to MapSwipe Back-End's documentation!
1313

1414
readme_link
1515
overview
16+
contributing
1617

1718
.. toctree::
1819
:maxdepth: 2
1920
:caption: Using:
2021

2122
for_mapswipe_managers
22-
cli
23+
use_cases
24+
tile_size
2325
data
2426

2527

@@ -30,33 +32,15 @@ Welcome to MapSwipe Back-End's documentation!
3032
deployment_overview
3133
configuration
3234
installation
33-
tutorials
34-
updating
35-
testing
35+
cli
36+
dev_setup
37+
testing_and_travis
3638
debugging
3739
backup
3840

3941

40-
.. toctree::
41-
:maxdepth: 2
42-
:caption: Project Types and Data Model:
43-
44-
project_type
45-
project_type-buildArea
46-
project_type-footprint
47-
project_type-changeDetection
4842

4943

50-
.. toctree::
51-
:maxdepth: 2
52-
:caption: Miscellaneous:
53-
54-
dev_setup
55-
contributing
56-
diagrams
57-
use_cases
58-
tile_size
59-
6044

6145
Indices and tables
6246
==================

‎docs/source/installation.md‎

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -146,12 +146,7 @@ To enable SSL for the API and MapSwipe Manager Dashboard use Certbot to issue st
146146

147147
### Certbot
148148

149-
Install certbot:
150-
151-
```bash
152-
apt-get install certbot
153-
```
154-
149+
To install Certbot follow instructions on https://certbot.eff.org/lets-encrypt/ubuntubionic-other
155150

156151
Create certificates:
157152

@@ -167,7 +162,7 @@ certbot certonly \
167162

168163
> Note: Certbot systemd timer for renewal of certificate will not work for standalone certificates because the service (docker nginx) which occupies port 80 has to be stopped before renewal.
169164
170-
For certificate renewal a cronjob is used:
165+
For certificate renewal a cronjob is used. This has to be run as root: `sudo crontab -e`
171166

172167
```bash
173168
SHELL=/bin/sh

‎docs/source/testing_and_travis.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Testing and Travis
2+
3+
## Tests
4+
* run tests locally during development
5+
6+
```
7+
python -m unittest discover --verbose --start-directory mapswipe_workers/tests/unittests/
8+
python -m unittest discover --verbose --start-directory mapswipe_workers/tests/integration/
9+
```
10+
11+
12+
## Travis
13+
* set environment variables in travis
14+
* travis then sets up the docker containers
15+
* test are run inside the mapswipe_workers docker container
16+
17+
```
18+
docker-compose run mapswipe_workers python -m unittest discover --verbose --start-directory tests/unittests/
19+
docker-compose run mapswipe_workers python -m unittest discover --verbose --start-directory tests/integration/
20+
```

‎firebase/database.rules.json‎

Lines changed: 84 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -5,24 +5,38 @@
55
"v2": {
66
"announcement": {
77
".write": false,
8-
".read": true
8+
".read": "auth != null"
99
},
1010
"projects": {
1111
".write": false,
12-
".read": "auth != null",
12+
".read": "
13+
// all users can query by status
14+
(auth != null && query.orderByChild == 'status' && query.equalTo == 'active')
15+
||
16+
// only team members can query by their own teamId
17+
(auth != null && query.orderByChild == 'teamId' &&
18+
query.equalTo == root.child('/v2/users/'+auth.uid+'/teamId').val())
19+
||
20+
// project managers can read all
21+
(auth.token.projectManager === true)
22+
",
1323
"$project_id": {
14-
"status": {
15-
".write": "auth.token.projectManager === true"
16-
},
17-
"isFeatured": {
18-
".write": "auth.token.projectManager === true"
19-
}
24+
// project managers can read/write all attributes
25+
".write": "auth.token.projectManager === true",
26+
".read": "
27+
// all users can read all public projects
28+
(data.child('teamId').exists() == false)
29+
||
30+
// team members can read their projects
31+
(data.child('teamId').val() == root.child('/v2/users/'+auth.uid+'/teamId').val())
32+
"
2033
},
2134
".indexOn": [
22-
"status", "isFeatured"
23-
]
35+
"status", "isFeatured", "teamId"
36+
]
2437
},
2538
"projectDrafts": {
39+
// only project managers can write projectDrafts
2640
".read": false,
2741
".write": "auth.token.projectManager === true",
2842
".indexOn": [
@@ -31,89 +45,94 @@
3145
},
3246
"groups": {
3347
".write": false,
34-
".read": true,
48+
".read": "auth.token.projectManager === true",
3549
"$project_id": {
36-
".indexOn": [
37-
"finishedCount",
38-
"requiredCount"
39-
]
40-
}
50+
".read": "
51+
// everyone can read groups for public projects
52+
(root.child('/v2/projects/'+$project_id).hasChild('teamId') === false)
53+
||
54+
(root.child('/v2/projects/'+$project_id+'/teamId').val() ==
55+
root.child('/v2/users/'+auth.uid+'/teamId').val())
56+
"
57+
},
58+
".indexOn": [
59+
"finishedCount",
60+
"requiredCount"
61+
]
4162
},
4263
"tasks": {
4364
".write": false,
44-
".read": true
65+
".read": "auth.token.projectManager === true",
66+
"$project_id": {
67+
".read": "
68+
// everyone can read groups for public projects
69+
(root.child('/v2/projects/'+$project_id).hasChild('teamId') === false)
70+
||
71+
(root.child('/v2/projects/'+$project_id+'/teamId').val() ==
72+
root.child('/v2/users/'+auth.uid+'/teamId').val())
73+
"
74+
},
75+
},
76+
"teams": {
77+
".write": false,
78+
// project managers can read all attributes
79+
".read": "auth.token.projectManager === true",
80+
"$teamId": {
81+
".write": "auth.token.projectManager === true",
82+
"teamName": {
83+
// team members read teamName
84+
".read": "(root.child('/v2/users/'+auth.uid+'/teamId').val() === $teamId)"
85+
//".read": "auth != null"
86+
},
87+
".indexOn": [
88+
"teamName"
89+
]
90+
}
4591
},
4692
"results": {
4793
".write": false,
4894
".read": false,
4995
"$project_id": {
5096
"$group_id": {
5197
"$uid": {
52-
".write": "$uid === auth.uid"
98+
".write": "
99+
// everyone can write results for public projects
100+
(root.child('/v2/projects/'+$project_id).hasChild('teamId') === false &&
101+
auth.uid == $uid
102+
)
103+
||
104+
(root.child('/v2/projects/'+$project_id+'/teamId').val() ==
105+
root.child('/v2/users/'+auth.uid+'/teamId').val() &&
106+
auth.uid == $uid
107+
)
108+
"
53109
}
54110
}
55111
}
56112
},
57113
"users": {
58114
".read": true,
115+
".write": false,
59116
"$uid": {
60-
".write": "auth != null && auth.uid == $uid"
61-
},
62-
".indexOn": [
63-
"created"
64-
]
65-
}
66-
},
67-
"groups": {
68-
".write": false,
69-
".read": true,
70-
"$project_id": {
71-
"$group_id": {
72-
".indexOn": [
73-
"distributedCount",
74-
"completedCount"
75-
],
76-
".write": "auth != null",
77-
"completedCount": {
78-
".write": "auth != null"
117+
"teamId": {
118+
".write": false
119+
},
120+
"$other": {
121+
".write": "auth != null && auth.uid == $uid"
79122
}
80123
},
81124
".indexOn": [
82-
"distributedCount",
83-
"completedCount"
125+
"created",
126+
"teamId"
84127
]
85128
}
86129
},
87-
"imports": {
88-
".read": false,
89-
".write": true,
90-
".indexOn": [
91-
"complete"
92-
]
93-
},
94-
"projects": {
95-
".write": false,
96-
".read": true
97-
},
98-
"announcement": {
99-
".write": true,
100-
".read": true
101-
},
102-
"results": {
103-
".write": false,
104-
".read": true,
105-
"$task_id": {
106-
"$user_id": {
107-
".write": "auth != null && auth.uid == $user_id"
108-
}
109-
}
110-
},
130+
// leaving this here, since version before v2 pull data from there
111131
"users": {
112132
"$uid": {
113133
".read": "auth != null && auth.uid == $uid",
114134
".write": "auth != null && auth.uid == $uid"
115-
},
116-
".read": true
135+
}
117136
}
118137
}
119138
}

‎firebase/functions/index.js‎

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,10 +18,6 @@ exports.counter = functions.database.ref('/v2/results/{projectId}/{groupId}/{use
1818
console.log('no results attribute for /v2/results/'+context.params.projectId+'/'+context.params.groupId+'/'+context.params.userId )
1919
console.log('will not update counters')
2020
return null
21-
} else if (!result.hasOwnProperty('timestamp')) {
22-
console.log('no timestamp attribute for /v2/results/'+context.params.projectId+'/'+context.params.groupId+'/'+context.params.userId )
23-
console.log('will not update counters')
24-
return null
2521
} else if (!result.hasOwnProperty('endTime')) {
2622
console.log('no endTime attribute for /v2/results/'+context.params.projectId+'/'+context.params.groupId+'/'+context.params.userId )
2723
console.log('will not update counters')
@@ -47,7 +43,6 @@ exports.counter = functions.database.ref('/v2/results/{projectId}/{groupId}/{use
4743
const groupContributionsRef = admin.database().ref('/v2/users/'+context.params.userId+'/contributions/'+context.params.projectId +'/'+context.params.groupId)
4844
const totalTimeSpentMappingRef = admin.database().ref('/v2/users/'+context.params.userId+'/timeSpentMapping')
4945

50-
const timestampRef = admin.database().ref('/v2/results/'+context.params.projectId+'/'+context.params.groupId+'/'+context.params.userId+'/timestamp')
5146
const startTimeRef = admin.database().ref('/v2/results/'+context.params.projectId+'/'+context.params.groupId+'/'+context.params.userId+'/startTime')
5247
const endTimeRef = admin.database().ref('/v2/results/'+context.params.projectId+'/'+context.params.groupId+'/'+context.params.userId+'/endTime')
5348
const timeSpentMappingRef = admin.database().ref('/v2/results/'+context.params.projectId+'/'+context.params.groupId+'/'+context.params.userId+'/timeSpentMappingRef')
@@ -115,7 +110,6 @@ exports.counter = functions.database.ref('/v2/results/{projectId}/{groupId}/{use
115110
}
116111
else {
117112
const data = {
118-
'timestamp': result['timestamp'],
119113
'startTime': result['startTime'],
120114
'endTime': result['endTime']
121115
}

0 commit comments

Comments
 (0)