Merge branch 'master' into release

Author: suricactus

.env.example

@@ -62,6 +62,7 @@ jobs:
       - name: Export the env variables file
         run: |
           cp .env.example .env
+          sed -ri 's/^ENVIRONMENT=(.*)/ENVIRONMENT=test/g' .env
           sed -ri 's/^COMPOSE_FILE=(.*)/COMPOSE_FILE=\1:docker-compose.override.test.yml/g' .env
           eval $(egrep "^[^#;]" .env | xargs -d'\n' -n1 | sed -E 's/(\w+)=(.*)/export \1='"'"'\2'"'"'/g')
 

.github/workflows/test.yml

@@ -49,13 +49,8 @@ To fetch upstream development, don't forget to update the submodules too:
 cp .env.example .env
 ```
 
-2. Change the `ENVIRONMENT` variable to `development`.
 
-```shell
-ENVIRONMENT=development
-```
-
-3. Build development images and run the containers:
+2. Build development images and run the containers:
 
 ```shell
 docker compose up -d --build
@@ -64,31 +59,31 @@ docker compose up -d --build
 The command will read the `docker-compose*.yml` files specified in the `COMPOSE_FILE` variable from the `.env` file. Then Django built-in server will be directly reachable at `http://localhost:8011` or through `nginx` at `https://localhost`.
 You should avoid using the Django's built-in server and better always develop and test QFieldCloud through the `nginx` [reverse proxy with SSL](#add-root-certificate).
 
-4. (OPTIONAL) In case you have a database dump, you can directly load some data in your development database.
+3. (OPTIONAL) In case you have a database dump, you can directly load some data in your development database.
 
 ```shell
 psql 'service=localhost.qfield.cloud' < ./qfc_dump_20220304.sql
 ```
 
-5. Run Django database migrations.
+4. Run Django database migrations.
 
 ```shell
 docker compose exec app python manage.py migrate
 ```
 
-6. And collect the static files (CSS, JS etc):
+5. And collect the static files (CSS, JS etc):
 
 ```shell
 docker compose run app python manage.py collectstatic --noinput
 ```
 
-7. Now you can get started by adding your super user that has access to the Django Admin interface:
+6. Now you can get started by adding your super user that has access to the Django Admin interface:
 
 ```shell
 docker compose run app python manage.py createsuperuser --username super_user --email super@user.com
 ```
 
-8. If QFieldCloud needs to be translated, you can compile the translations using Django's tooling:
+7. If QFieldCloud needs to be translated, you can compile the translations using Django's tooling:
 
 ```shell
 docker compose run --user root app python manage.py compilemessages
@@ -191,8 +186,6 @@ Create an <code>.env.test</code> file with the following variables that override
     WEB_HTTP_PORT=8101
     WEB_HTTPS_PORT=8102
     HOST_POSTGRES_PORT=8103
-    HOST_GEODB_PORT=8107
-    MEMCACHED_PORT=11212
     QFIELDCLOUD_DEFAULT_NETWORK=qfieldcloud_test_default
     QFIELDCLOUD_SUBSCRIPTION_MODEL=subscription.Subscription
     DJANGO_DEV_PORT=8111
@@ -219,6 +212,17 @@ Don't forget to update the `port` value in [`[test.localhost.qfield.cloud]` in y
 
 </details>
 
+
+#### Test coverage
+
+To get information about the current test coverage, run:
+
+```
+docker compose exec app coverage run manage.py test --keepdb
+docker compose exec app coverage report
+```
+
+
 ### Debugging
 
 > [!NOTE]
@@ -377,8 +381,6 @@ Based on this example
 | nginx https   | 443   | WEB_HTTPS_PORT       | :white_check_mark: | :white_check_mark: | :white_check_mark: |
 | django http   | 8011  | DJANGO_DEV_PORT      | :white_check_mark: | :x:                | :x:                |
 | postgres      | 5433  | HOST_POSTGRES_PORT   | :white_check_mark: | :white_check_mark: | :white_check_mark: |
-| memcached     | 11211 | MEMCACHED_PORT       | :white_check_mark: | :x:                | :x:                |
-| geodb         | 5432  | HOST_GEODB_PORT      | :white_check_mark: | :white_check_mark: | :x:                |
 | minio API     | 8009  | MINIO_API_PORT       | :white_check_mark: | :x:                | :x:                |
 | minio browser | 8010  | MINIO_BROWSER_PORT   | :white_check_mark: | :x:                | :x:                |
 | smtp web      | 8012  | SMTP4DEV_WEB_PORT    | :white_check_mark: | :x:                | :x:                |
@@ -398,23 +400,6 @@ For great `nginx` logs, use:
     docker compose logs nginx -f --no-log-prefix | grep ':"nginx"' | jq -r $QFC_JQ
 
 
-### Geodb
-
-The geodb (database for the users projects data) is installed on
-separated machines (db1.qfield.cloud, db2.qfield.cloud, db3&#x2026;)
-and they are load balanced and available through the
-db.qfield.cloud address.
-
-There is a template database called
-`template_postgis` that is used to create the databases for the
-users. The template db has the following extensions installed:
-
--   fuzzystrmatch
--   plpgsql
--   postgis
--   postgis<sub>tiger</sub><sub>geocoder</sub>
--   postgis<sub>topology</sub>
-
 ### Storage
 
 You can use either the integrated `minio` object storage, or use an external provider (e. g. S3) with versioning enabled. Check the corresponding `STORAGE_*` environment variables for more info.
@@ -436,10 +421,38 @@ Migration to a newer database version is a risky operation to your data, so prep
 
 Contributions welcome!
 
-Any PR including the `[WIP]` should be:
-- able to be checked-out without breaking the stack;
+
+### Before considering a new PR
+
+For the best chance of having your PR merged, you must first communicate your idea(s) and clarify the details with the QFieldCloud developers.
+
+1) Create a dedicated [issue in the QFieldCloud repository](https://github.com/opengisch/QFieldCloud/issues) detailing your suggestion.
+2) Engage in the feedback and guidance provided by the development team during their review of your issue.
+
+Discussing it first is crucial. Not every idea is accepted, and these steps will save you the time and energy of creating a PR that wouldn't be merged.
+
+Any new, follow-up discussions should be opened in a new issue that references the original.
+
+
+### Before opening a new PR
+
+- Make sure you prepare a small focused branch with your changes, properly referencing the source issue and add a detailed description to the PR opening message.
+- If the change addresses a UI/UX change, provide a before and after screenshot.
+- Your branch should be based on the latest `master` branch.
+- In the rare occurances when it is not based on `master`, please base your PR on the respective branch.
+
+
+### Pull requests
+
+If your PR is not ready to be merged, please mark it as a [draft PR](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-stage-of-a-pull-request#converting-a-pull-request-to-a-draft).
+
+A draft PR can allow QFieldCloud developers to provide you early feedback.
+However, make sure a draft PR:
+
+- can be checked-out without breaking the stack;
 - the specific feature being developed/modified should be testable locally (does not mean it should work correctly).
 
+
 ## Resources
 
 -   [QField Cloud "marketing" page](https://qfield.cloud)

README.md

@@ -3,10 +3,9 @@ branch = True
 source =
     qfieldcloud
 omit =
-    **migrations*
-    **tests*
-    **/admin.py
-    **/wsgi.py
+    */tests/*
+    */migrations/*
+    */wsgi.py
 
 [report]
 precision = 2
@@ -19,6 +18,3 @@ exclude_lines =
     raise NotImplementedError
     if 0:
     if __name__ == .__main__.:
-
-[html]
-directory = .htmlcov

docker-app/.coveragerc

@@ -4,6 +4,7 @@
 from allauth.socialaccount.models import SocialApp
 from allauth.socialaccount.providers.oauth2.provider import OAuth2Provider
 from allauth.socialaccount.providers.oauth2.views import OAuth2Adapter
+from django.conf import settings
 from django.http import HttpRequest
 from django.test import override_settings
 from rest_framework.test import APITestCase
@@ -141,7 +142,8 @@ def test_lists_configured_social_providers(self):
                 "redirect_port": 7070,
                 "redirect_url": "",
                 "client_id": "keycloak-client-id",
-                "extra_tokens": {"id_token": "X-QFC-ID-Token"},
+                "extra_tokens": {"id_token": settings.QFIELDCLOUD_ID_TOKEN_HEADER_NAME},
+                "idp_id_header": settings.QFIELDCLOUD_IDP_ID_HEADER_NAME,
                 "styles": {},
             },
         ]

docker-app/qfieldcloud/authentication/tests/test_list_auth_providers.py

@@ -189,7 +189,8 @@ def get_provider_data(self, provider: Provider, request: HttpRequest) -> dict:
             "redirect_port": self.QGIS_REDIRECT_PORT,
             "redirect_url": self.QGIS_REDIRECT_URL,
             "client_id": provider.app.client_id,
-            "extra_tokens": {"id_token": "X-QFC-ID-Token"},
+            "extra_tokens": {"id_token": settings.QFIELDCLOUD_ID_TOKEN_HEADER_NAME},
+            "idp_id_header": settings.QFIELDCLOUD_IDP_ID_HEADER_NAME,
             "styles": SSOProviderStyles(request).get(provider_id),
         }
         return provider_data

docker-app/qfieldcloud/authentication/views.py

@@ -51,7 +51,6 @@
     ApplyJobDelta,
     Delta,
     FaultyDeltaFile,
-    Geodb,
     Job,
     Organization,
     OrganizationMember,
@@ -366,18 +365,6 @@ def format_pre_json(value):
         return format_pre(value)
 
 
-class GeodbInline(admin.TabularInline):
-    model = Geodb
-    extra = 0
-    has_direct_delete_permission = False
-
-    def has_add_permission(self, request, obj):
-        return False
-
-    def has_change_permission(self, request, obj):
-        return False
-
-
 class MemberOrganizationInline(admin.TabularInline):
     model = OrganizationMember
     extra = 0
@@ -521,10 +508,7 @@ class PersonAdmin(QFieldCloudModelAdmin):
         "storage_usage__field",
     )
 
-    inlines = (
-        UserAccountInline,
-        GeodbInline,
-    )
+    inlines = (UserAccountInline,)
 
     add_form_template = "admin/change_form.html"
     change_form_template = "admin/person_change_form.html"
@@ -1416,7 +1400,6 @@ def has_change_permission(self, request, obj):
 class OrganizationAdmin(QFieldCloudModelAdmin):
     inlines = (
         UserAccountInline,
-        GeodbInline,
         ProjectInline,
         OrganizationMemberInline,
         TeamInline,

docker-app/qfieldcloud/core/admin.py

@@ -25,7 +25,7 @@ def handle(self, *args, **options):
             extra_filters["file_storage_bytes"] = 0
 
         projects_qs = Project.objects.filter(
-            the_qgis_filename__isnull=False,
+            the_qgis_file_name__isnull=False,
             **extra_filters,
         ).order_by("-updated_at")
         total_count = projects_qs.count()