Add time_zone to committee json_upload (#3438) (#3490)

* Add time_zone to committee json_upload

* Add get_valid_timezones presenter

Author: openslides-automation[bot]

docs/actions/committee.json_upload.md

@@ -17,6 +17,7 @@ The types noted below are the internal types after conversion in the backend. Se
         meeting_name: string,
         meeting_start_time: date,
         meeting_end_time: date,
+        meeting_time_zone: string,
         meeting_admins: string[],
         meeting_template: string,
     }[]
@@ -50,6 +51,10 @@ Besides the usual headers as seen in the payload (`name`, `type`, `is_list`), th
 - `meeting_template`:
   - `done`: The meeting was found in the datastore, the new meeting will be cloned from it.
   - `warning`: The meeting was not found and the new meeting will not be cloned, but freshly created.
+- `meeting_time_zone`:
+  - `done`: Valid timezone string (see [presenter](../presenters/get_valid_timezones.md)).
+  - `warning`: Field empty and there's a start/end_time.
+  - `error`: Not a valid timezone string.
 
 The fields `forward_to_committee`, `organization_tags`, `managers`, `meeting_admins` and `meeting_template` store the `id` of the related model, if it exists, in the object for the import.
 

docs/actions/meeting.create.md

@@ -9,7 +9,7 @@
 // Optional
     description: string;
     location: string;
-    timezone: string; // A valid IANA timezone string
+    time_zone: string;
     start_time: datetime;
     end_time: datetime;
     organization_tag_ids: Id[];
@@ -25,6 +25,8 @@ Creates a meeting.
 
 Checks whether the `organization.limit_of_meetings` is unlimited (=0) or lower than the amount of active meetings in `organization.active_meeting_ids`.
 
+`time_zone` must be a valid timezone string (see [presenter](../presenters/get_valid_timezones.md)).
+
 The following objects are created, too:
 - Groups: `Default`, `Admin`, `Delegates`, `Staff`, `Committees`. The first one is set as `meeting/default_group_id`, the second one as `meeting/admin_group_id`. The permissions can be found in the [initial-data.json](https://github.com/OpenSlides/openslides-backend/tree/main/data/initial-data.json)).
 - Projector: One projector named `"Default projector"` is created and set as `meeting/reference_projector_id`.

docs/actions/meeting.update.md

@@ -12,7 +12,7 @@
     name: string;
     description: string;
     location: string;
-    timezone: string; // A valid IANA timezone string
+    time_zone: string;
     start_time: timestamp;
     end_time: timestamp;
     locked_from_inside: boolean;
@@ -209,6 +209,8 @@ Updates the meeting.
 If `set_as_template` is `True`, `template_for_organization_id` has to be set to `1`. If it is `False`, `template_for_organization_id` has to be set to `None` and if there are currently no users in the meetings admin group, an exception needs to be raised.
 `reference_projector_id` can only be set to a projector, which is not internal.
 
+`time_zone` must be a valid timezone string (see [presenter](../presenters/get_valid_timezones.md)).
+
 This action doesn't allow for a meeting to be set as a template and have `locked_from_inside` set to true at the same time. if this would be the result of an action call, an exception will be thrown. Same for `enable_anonymous` and `locked_from_inside` being true at the same time.
 
 If `enable_anonymous` is set, this action will create an anonymous group for the meeting. This will have the name `Public` and otherwise differ from the other groups in the meeting due to having `anonymous_group_for_meeting_id` set. It will always have the lowest weight among all other groups in this meeting, meaning 0.

docs/actions/organization.update.md

@@ -13,7 +13,7 @@
     login_text: string;
     theme_id: Id;
     default_language: string;
-    time_zone: string; // A valid IANA timezone string
+    time_zone: string;
     users_email_sender: string;
     users_email_replyto: string;
     users_email_subject: string;
@@ -43,6 +43,8 @@
 ## Action
 Updates the organization.
 Checks if the theme_id is one of the theme_ids.
+`time_zone` must be a valid timezone string (see [presenter](../presenters/get_valid_timezones.md)).
+
 This is an example of the saml_attr_mapping, where you can see the mappable fields.
 ```json
 {

docs/actions/preface_special_imports.md

@@ -116,7 +116,7 @@ The internal types will be created by the backend service from the CSV-strings
 - **integer** Use something like "1234" without fraction
 - **number** means a `,` (comma) or `.` (point) separated value like `3123,45` or `3123.45`, but not `3.123,45` or `3,123.45`
 - **decimal** A decimal number with exactly 6 digits after the decimal seperator dot, e.g. `1.500000`
-- **date** Use a string in Isoformat "YYYY-MM-DD", for example "2023-04-26"  
+- **date** Use a string in Isoformat "YYYY-MM-DD", for example "2023-04-26"
 
 ## Import_Preview to store the data to import in database
 ```js

docs/presenters/get_valid_timezones.md

@@ -0,0 +1,27 @@
+# Payload
+```js
+{
+}
+```
+
+# Returns
+```js
+{
+    [string]: string
+    ...
+}
+```
+
+# Logic
+Gets all valid timezones from database.
+
+Returns a timezone-name to current abbreviation dict.
+(Note: Abbreviations may change over the course of the year as certain timezones switch to and from DST)
+
+In general this is going to return mostly canonic IANA timezone names.
+
+# Permissions
+The user needs to either:
+- `OML.can_manage_organization`,
+- `CML.can_manage` in any committee, or
+- `meeting.can_manage_settings` in any meeting.

openslides_backend/action/actions/committee/import_.py

@@ -182,6 +182,7 @@ def create_models(self, rows: list[ImportRow]) -> None:
                         "start_time",
                         "end_time",
                         "admin_ids",
+                        "time_zone",
                     )
                     if (meeting_field := f"meeting_{field}") in entry
                 } | {

openslides_backend/action/actions/committee/json_upload.py

@@ -2,6 +2,8 @@
 from datetime import datetime, timedelta, timezone
 from typing import Any
 
+from psycopg import sql
+
 from openslides_backend.action.actions.meeting.mixins import MeetingCheckTimesMixin
 from openslides_backend.shared.exceptions import ActionException
 from openslides_backend.shared.filters import And, Filter, FilterOperator, Or
@@ -39,7 +41,9 @@ class CommitteeJsonUpload(
                         **{
                             f"meeting_{field}": prop
                             for field, prop in Meeting()
-                            .get_properties("name", "start_time", "end_time")
+                            .get_properties(
+                                "name", "start_time", "end_time", "time_zone"
+                            )
                             .items()
                         },
                         "meeting_admins": str_list_schema,
@@ -78,6 +82,7 @@ class CommitteeJsonUpload(
         {"property": "meeting_name", "type": "string"},
         {"property": "meeting_start_time", "type": "date"},
         {"property": "meeting_end_time", "type": "date"},
+        {"property": "meeting_time_zone", "type": "string", "is_object": True},
         {
             "property": "meeting_admins",
             "type": "string",
@@ -88,6 +93,7 @@ class CommitteeJsonUpload(
         {"property": "parent", "type": "string", "is_object": True},
     ]
     import_name = "committee"
+    timezone_field_name = "meeting_time_zone"
 
     permission = OrganizationManagementLevel.CAN_MANAGE_ORGANIZATION
     skip_archived_meeting_check = True
@@ -166,12 +172,40 @@ def validate_entry(self, entry: dict[str, Any]) -> dict[str, Any]:
                 for field in (
                     "meeting_start_time",
                     "meeting_end_time",
+                    "meeting_time_zone",
                     "meeting_admins",
                     "meeting_template",
                 )
             ):
                 messages.append("No meeting will be created without meeting_name")
         else:
+            if tz := entry.get("meeting_time_zone"):
+                valid = self.datastore.execute_custom_select(
+                    sql.SQL("is_timezone(%s) AS valid"), False, [tz]
+                )[0].get("valid")
+                if not valid:
+                    row_state = ImportState.ERROR
+                    entry["meeting_time_zone"] = {
+                        "value": tz,
+                        "info": ImportState.ERROR,
+                    }
+                    messages.append(
+                        f"Error: Invalid timezone format: '{tz}' (expected valid timezone name)"
+                    )
+                else:
+                    entry["meeting_time_zone"] = {
+                        "value": tz,
+                        "info": ImportState.DONE,
+                    }
+            elif entry.get("meeting_start_time") or entry.get("meeting_end_time"):
+                entry["meeting_time_zone"] = {
+                    "info": ImportState.WARNING,
+                    "value": None,
+                }
+                zone = self.get_time_zone()
+                messages.append(
+                    f"Since no timezone was given, the dates will be interpreted as being in the '{zone}' zone."
+                )
             self.validate_with_lookup(
                 entry, "meeting_admins", self.username_lookup, messages
             )
@@ -214,9 +248,10 @@ def check_meetings(self) -> None:
                     FilterOperator("committee_id", "=", committee_id),
                     FilterOperator("name", "=", meeting_name),
                 ]
+                zone = self.get_time_zone_info(entry)
                 for field in ("start_time", "end_time"):
                     if time := entry.get(f"meeting_{field}"):
-                        start_of_day = datetime.fromtimestamp(time, timezone.utc)
+                        start_of_day = datetime.fromtimestamp(time, zone)
                         start_of_day.replace(hour=0, minute=0, second=0, microsecond=0)
                         end_of_day = start_of_day + timedelta(days=1)
                         parts.extend(

openslides_backend/action/mixins/import_mixins.py

@@ -5,7 +5,6 @@
 from datetime import datetime
 from decimal import Decimal
 from enum import Enum, StrEnum
-from time import mktime, strptime
 from typing import Any, TypedDict, Union, cast
 from zoneinfo import ZoneInfo
 
@@ -22,6 +21,7 @@
 from ...shared.interfaces.write_request import WriteRequest
 from ...shared.patterns import fqid_from_collection_and_id
 from ...shared.schema import required_id_schema
+from ...shared.util import ONE_ORGANIZATION_FQID
 from ..util.default_schema import DefaultSchema
 from ..util.typing import ActionData, ActionResultElement
 from .singular_action_mixin import SingularActionMixin
@@ -391,6 +391,7 @@ class BaseJsonUploadAction(BaseImportJsonUploadAction):
     statistics: list[StatisticEntry]
     import_state: ImportState
     meeting_id: int
+    timezone_field_name: str | None = None
 
     def base_update_instance(self, instance: dict[str, Any]) -> dict[str, Any]:
         instance = super().base_update_instance(instance)
@@ -454,6 +455,40 @@ def add_payload_index_to_action_data(self, action_data: ActionData) -> ActionDat
             entry["payload_index"] = payload_index
         return action_data
 
+    def get_time_zone(self, entry: dict[str, Any] = {}) -> str:
+        tz: str = (
+            entry.get(self.timezone_field_name or "")
+            or (
+                self.datastore.get(
+                    fqid_from_collection_and_id("meeting", self.meeting_id),
+                    ["time_zone"],
+                ).get("time_zone")
+                if hasattr(self, "meeting_id")
+                else None
+            )
+            or self.datastore.get(ONE_ORGANIZATION_FQID, ["time_zone"]).get("time_zone")
+            or "UTC"
+        )
+        return tz
+
+    def get_time_zone_info(self, entry: dict[str, Any] = {}) -> ZoneInfo:
+        tz = self.get_time_zone(entry)
+        try:
+            zone = ZoneInfo(tz)
+        except Exception:
+            if tz == entry.get(self.timezone_field_name or ""):
+                zone = ZoneInfo(
+                    self.datastore.get(ONE_ORGANIZATION_FQID, ["time_zone"]).get(
+                        "time_zone"
+                    )
+                    or "UTC"
+                )
+            else:
+                raise ActionException(
+                    f"Invalid timezone format: '{tz}' (expected valid timezone name)"
+                )
+        return zone
+
     def validate_instance(self, instance: dict[str, Any]) -> None:
         # filter extra, not needed fields before validate and parse some fields
         property_to_type = {
@@ -506,9 +541,13 @@ def validate_instance(self, instance: dict[str, Any]) -> None:
                                 f"Could not parse {entry[field]} expect boolean"
                             )
                     elif type_ == "date":
+                        zone = self.get_time_zone_info(entry)
                         try:
+                            y, m, d = entry[field].split("-")
                             entry[field] = int(
-                                mktime(strptime(entry[field], "%Y-%m-%d"))
+                                datetime(
+                                    int(y), int(m), int(d), tzinfo=zone
+                                ).timestamp()
                             )
                         except Exception:
                             raise ActionException(

openslides_backend/presenter/__init__.py

@@ -10,6 +10,7 @@
     get_user_related_models,
     get_user_scope,
     get_users,
+    get_valid_timezones,
     number_of_users,
     search_for_id_by_external_id,
     search_users,