Merge branch 'development'

Author: karel26

ARCHITECTURE.md

@@ -13,7 +13,7 @@ Discord and provides all the commands that you can use. It will always only run
 
 ### Instance
 Each `Instance` specifies a DCS executable that can be launched on that specific node. The name of the instance matches
-your Saved Games directory. The single instance that every server should have is DCS.release_server.
+your Saved Games directory. The single instance that every server should have is DCS.dcs_serverrelease.
 
 Each instance has its own UDP communication port (default 6666). It will contact the DCSServerBot via UDP usually 
 over port 10042. The whole communication is UDP-based. This has a slight risk of data loss, but is non-blocking

MULTINODE.md

@@ -146,15 +146,15 @@ installation. If not specified otherwise, the node-name will be the hostname of
 
 > [!TIP]
 > If you use multiple nodes on multiple PCs, you might get to the moment where instances are named identically. 
-> This will start with the first instance already if you keep the default name "DCS.release_server".<br>
+> This will start with the first instance already if you keep the default name "DCS.dcs_serverrelease".<br>
 > As many configuration files only use the instance name per default, you might need to add the node name as well.
 > This can be done the same as it is already in your nodes.yaml: The node can be the outer structure in each config file.
 > 
 > Single-Node-Config:
 > ```yaml
 > DEFAULT:
 >   some-param: A
-> DCS.release_server:
+> DCS.dcs_serverrelease:
 >   some-param: B
 > ```
 > 
@@ -163,15 +163,15 @@ installation. If not specified otherwise, the node-name will be the hostname of
 > DEFAULT:
 >   some-param: A
 > MyNode1:
->   DCS.release_server:
+>   DCS.dcs_serverrelease:
 >     some-param: B
 > MyNode2:
->   DCS.release_server:
+>   DCS.dcs_serverrelease:
 >     some-param: C
 > ```
 DCSServerBot will understand both versions. The DEFAULT will be used for ALL instances, independent of which node they 
 are on. If you don't provide a node in a multi-node-system, the bot will read the same parameters for all instances 
-that are named DCS.release_server on any of your nodes. This can be what you want, but it can lead to errors.<br>
+that are named DCS.dcs_serverrelease on any of your nodes. This can be what you want, but it can lead to errors.<br>
 I would always recommend creating the node-specific version (ex: "Multi-Node-Config" above) to avoid confusion. That's 
 what the bot will create during a default installation also.
 

README.md

@@ -396,8 +396,8 @@ NODENAME:                       # this will usually be your hostname
 #   Any other Extension:
 #     ...
   instances:
-    DCS.release_server:        # The name of your instance. You can have multiple instances that have to have unique names.
-      home: '%USERPROFILE%\\Saved Games\\DCS.release_server' # The path to your saved games directory.
+    DCS.dcs_serverrelease:      # The name of your instance. You can have multiple instances that have to have unique names.
+      home: '%USERPROFILE%\\Saved Games\\DCS.dcs_serverrelease' # The path to your saved games directory.
       missions_dir: '%USERPROFILE%\Documents\Missions'       # You can overwrite the default missions dir like so. Default is the Missions dir below the instance home folder.
       mission_rewrite: false    # Disable rewrite of missions by MizEdit or RealWeather. The server will be stopped for any mission change then. (default: true)
       bot_port: 6666            # The port DCSServerBot uses to communicate with your DCS server. Each instance has to have a unique port. This is NOT your DCS port (10308)!!!
@@ -408,7 +408,7 @@ NODENAME:                       # this will usually be your hostname
       priority: normal          # Optional: set the process priority (low, normal, high, realtime) for the DCS_Server.exe
       extensions:               # See the extension documentation for more detailed information on what to set here.
         SRS:
-          config: '%USERPROFILE%\Saved Games\DCS.release_server\Config\SRS.cfg'  # it is recommended to copy your SRS "server.cfg" below your instances home directory.
+          config: '%USERPROFILE%\Saved Games\DCS.dcs_serverrelease\Config\SRS.cfg'  # it is recommended to copy your SRS "server.cfg" below your instances home directory.
           host: 127.0.0.1       # SRS servers local IP (default is 127.0.0.1)
           port: 5002            # SRS servers local port (default is 5002). The bot will change this in your SRS configuration if set here!
           autostart: true       # this will autostart your DCS server with the DCS server start (default: true)

core/data/impl/nodeimpl.py

@@ -265,6 +265,7 @@ def read_locals(self) -> dict:
             if not node:
                 raise FatalException(f'No configuration found for node {self.name} in {config_file}!')
             dirty = False
+
             # check if we need to secure the database URL
             database_url = node.get('database', {}).get('url')
             if database_url:
@@ -276,6 +277,16 @@ def read_locals(self) -> dict:
                         f"{url.scheme}://{url.username}:SECRET@{url.hostname}:{port}{url.path}?sslmode=prefer"
                     dirty = True
                     self.log.info("Database password found, removing it from config.")
+            else:
+                # if we have a cluster.pkl, rename that
+                cluserdb_pkl = os.path.join(self.config_dir, '.secret', 'clusterdb.pkl')
+                if os.path.exists(cluserdb_pkl):
+                    database_pkl = os.path.join(self.config_dir, '.secret', 'database.pkl')
+                    # remove existing target to ensure a clean move
+                    if os.path.exists(database_pkl):
+                        os.remove(database_pkl)
+                    os.replace(cluserdb_pkl, database_pkl)
+
             if 'DCS' in node:
                 password = node['DCS'].pop('dcs_password', node['DCS'].pop('password', None))
                 if password:
@@ -291,6 +302,33 @@ def read_locals(self) -> dict:
             return node
         raise FatalException(f"No {config_file} found. Exiting.")
 
+    def get_database_urls(self):
+        cpool_url = self.config.get("database", self.locals.get('database'))['url']
+        lpool_url = self.locals.get("database", self.config.get('database'))['url']
+
+        try:
+            lpool_pwd = utils.get_password('database', self.config_dir)
+        except ValueError:
+            self.log.critical(
+                "Please replace the SECRET keyword in your database URL with a password!"
+            )
+            exit(SHUTDOWN)
+
+        if cpool_url != lpool_url:
+            try:
+                cpool_pwd = utils.get_password('clusterdb', self.config_dir)
+            except ValueError:
+                self.log.critical(
+                    "Please replace the SECRET keyword in your database URL in main.yaml with a password!"
+                )
+                exit(SHUTDOWN)
+        else:
+            cpool_pwd = lpool_pwd
+
+        cpool_url = cpool_url.replace('SECRET', quote(cpool_pwd) or '')
+        lpool_url = lpool_url.replace('SECRET', quote(lpool_pwd) or '')
+        return cpool_url, lpool_url
+
     async def init_db(self):
         async def check_db(url: str) -> str | None:
             max_attempts = self.locals.get("database", self.config.get('database')).get('max_retries', 10)
@@ -308,21 +346,7 @@ async def check_db(url: str) -> str | None:
             # we will never be here
             return None
 
-        cpool_url = self.config.get("database", self.locals.get('database'))['url']
-        lpool_url = self.locals.get("database", self.config.get('database'))['url']
-        try:
-            password = utils.get_password('clusterdb', self.config_dir)
-        except ValueError:
-            try:
-                password = utils.get_password('database', self.config_dir)
-                utils.set_password('clusterdb', password, self.config_dir)
-            except ValueError:
-                self.log.critical("You need to replace the SECRET keyword in your database URL with a proper password!")
-                exit(SHUTDOWN)
-
-        cpool_url = cpool_url.replace('SECRET', quote(password) or '')
-        lpool_url = lpool_url.replace('SECRET', quote(utils.get_password('database', self.config_dir)) or '')
-
+        cpool_url, lpool_url = self.get_database_urls()
         version = await check_db(lpool_url)
         if lpool_url != cpool_url:
             await check_db(cpool_url)
@@ -1026,6 +1050,7 @@ async def check_nodes():
                             return True
                         # The master is not alive, take over
                         elif not master or not await is_node_alive(master, config.get('heartbeat', 30)):
+                            self.log.warning("The master node is not responding, taking over ...")
                             await take_over()
                             return True
                         # Master is alive, but we are the preferred one

core/utils/validators.py

@@ -10,7 +10,7 @@
 from pathlib import Path
 from pykwalify import partial_schemas
 from pykwalify.core import Core
-from pykwalify.errors import SchemaError, CoreError, PyKwalifyException
+from pykwalify.errors import SchemaError, CoreError, PyKwalifyException, UnknownError
 from pykwalify.rule import Rule
 from typing import Any, Type
 
@@ -89,15 +89,15 @@ def _initialize(self):
             for node, instances in self._instances.items():
                 for instance in instances:
                     self._all_instances[instance] = self._all_instances.get(instance, 0) + 1
-        except Exception:
-            raise CoreError(msg="nodes.yaml seems to be corrupt, can't initialize the node/instance-validation!")
+        except Exception as ex:
+            raise UnknownError(error_key=ex, path='nodes.yaml')
 
         try:
             config_path = Path(os.path.join(COMMAND_LINE_ARGS.config, 'servers.yaml'))
             data = yaml.load(config_path.read_text(encoding='utf-8'))
             self._servers = list(data.keys())
-        except Exception:
-            raise CoreError(msg="servers.yaml seems to be corrupt, can't initialize the server-validation!")
+        except Exception as ex:
+            raise UnknownError(error_key=ex, path='servers.yaml')
 
     @property
     def nodes(self):
@@ -366,9 +366,13 @@ def validate(source_file: str, schema_files: list[str], *, raise_exception: bool
     try:
         c.validate(raise_exception=True)
     except PyKwalifyException as ex:
+        if ex.error_key:
+            source_file = ex.error_key
         if raise_exception:
             raise
         if isinstance(ex, SchemaError):
             logger.warning(f'Error while parsing {source_file}:\n{ex}')
+        elif isinstance(ex, UnknownError):
+            logger.error(f'Error while parsing {ex.path}:\n{ex.error_key}')
         else:
             logger.error(f'Error while parsing {source_file}:\n{ex}', exc_info=ex)

extensions/README.md

@@ -61,7 +61,7 @@ MyNode:
       param2: bb
   # [...]
   instances:
-    DCS.release_server:
+    DCS.dcs_serverrelease:
       # [...]
       extensions:
         mymodule.MyExtension:

extensions/dsmc/README.md

@@ -11,7 +11,7 @@ extension like with all others:
 MyNode:
   # [...]
   instances:
-    DCS.release_server:
+    DCS.dcs_serverrelease:
       # [...]
       extensions:
         DSMC:

extensions/github/README.md

@@ -7,7 +7,7 @@ To enable track file upload, a change in nodes.yaml is needed:
 MyNode:
   # [...]
   instances:
-    DCS.release_server:
+    DCS.dcs_serverrelease:
       # [...]
       extensions:
         GitHub:

extensions/grpc/README.md

@@ -10,7 +10,7 @@ extension:
 MyNode:
   # [...]
   instances:
-    DCS.release_server:
+    DCS.dcs_serverrelease:
       # [...]
       extensions:
         gRPC:

extensions/lardoon/README.md

@@ -17,7 +17,7 @@ MyNode:
       use_single_process: true      # Start one Lardoon process instead of one per node (default: true) 
   # [...]
   instances:
-    DCS.release_server:
+    DCS.dcs_serverrelease:
       # [...]
       extensions:
         Lardoon: