Main merge for 1.3.1 release

.gitignore

@@ -82,3 +82,5 @@ build-teraserver-Desktop_Qt_6_6_1_MSVC2019_64bit-Debug
 python-3.10
 /teraserver/python/config/certificates
 /teraserver/python/tests/*.pem
+/teraserver/python/env/python-3.12
+/teraserver/python/env/python-3.13

docs/Architecture-Overview.md

@@ -60,6 +60,11 @@ This module can also automatically launch external services as stand-alone proce
 ## System services
 Those services are features that are shared by modules and external services. Each of them serves a specific purpose and will run in a separate process. Communication between modules and services will be done using the [Main OpenTera REST API](services/teraserver/api/API) and the [internal communication protocols - subscribe-publish](developers/Internal-services-communication-module).
 
+### Email service
+The email service is a service that manages sending emails from OpenTera using a secure API.
+
+More information on this service can be found [here](services/Email-Service).
+
 ### File transfer service
 The file transfer service is a service that manages file transfers (upload and download) within OpenTera. It allows other services to upload files in a central repository. While specific services can also implements file storage and indexing, this service can be used to quickly reuse components.
 

docs/conf.py

@@ -7,9 +7,9 @@
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 
 project = 'OpenTera'
-copyright = '2024, Simon Brière, Dominic Létourneau'
+copyright = '2025, Simon Brière, Dominic Létourneau'
 author = 'Simon Brière, Dominic Létourneau'
-release = '1.3.0'
+release = '1.3.1'
 version = release
 
 html_logo = 'images/LogoOpenTera200px.png'

docs/deployment/DeployWithLinuxNative.md

@@ -25,6 +25,7 @@ This section configures the depending packages and software before installing th
 5. Create required database and assign `teraagent` user to them:
     ```
     CREATE DATABASE opentera WITH OWNER=teraagent;
+	CREATE DATABASE openteraemails WITH OWNER=teraagent;
     CREATE DATABASE openterafiles WITH OWNER=teraagent;
     CREATE DATABASE openteralogs WITH OWNER=teraagent;
     ```
@@ -101,6 +102,7 @@ There is a few config files to edit. You should edit each of them and put the co
 
 * `teraserver/python/config/TeraServerConfig.ini`: the main config file. "port" and "hostname" shouldn't be changed.
 * `teraserver/python/config/nginx.conf`: nginx config file. Unless listening to a different port and setting correct ssl certificates, nothing should be changed in that file.
+* `teraserver/python/services/EmailService/EmailService.json`: the email service configuration.
 * `teraserver/python/services/FileTransferService/FileTransferService.json`: the file transfer service configuration.
 * `teraserver/python/services/LoggingService/LoggingService.json`: the logging service configuration.
 * `teraserver/python/services/VideoRehabService/VideoRehabService.json`: make sure to set the "WebRTC - hostname" value to the external server address.
@@ -129,8 +131,8 @@ Group=**PUT THE EXECUTING GROUP HERE**
 Environment=PYTHONPATH=**(path to opentera)**/opentera/teraserver/python
 ExecStart=**(path to opentera)**/opentera/teraserver/python/env/python-3.11/bin/python3 **(path to opentera)**/opentera/teraserver/python/TeraServer.py
 WorkingDirectory=**(path to opentera)**/opentera/teraserver/python
-StandardOutput=syslog+console
-StandardError=syslog+console
+StandardOutput=journal+console
+StandardError=journal+console
 Restart=always
 RestartSec=10s
 KillMode=process

docs/developers/Developer-Setup-for-Linux.md

@@ -100,6 +100,7 @@ Each system service will requires its own database. The default database that ne
 * `opentera`, the [main OpenTera service](../services/teraserver/teraserver.rst) database
 * `openterafiles`, the database for the [file transfer](../services/FileTransfer-Service) service
 * `openteralogs`, the database for the [logging service](../services/Logging-Service)
+* `openteraemails`, the database for the [email service](../services/Email-Service)
 
 The steps to create a database are as follow. Those steps should be repeated for each database to create.
   1. In pgAdmin, under `Servers`, right-click on the correct server instance (for example `PostgreSQL 13`).
@@ -123,14 +124,14 @@ If using PyCharm, the correct Python interpreter and environment will need to be
   1. Open PyCharm, loading the project from `<base folder>/teraserver/python`
   2. In `Files -> Settings-> Project : python -> Python Interpreter`, click on the `Show all` option in the selection bar.
   3. In the `Virtualenv` tab, choose the existing environment. Change the interpreter to the following :
-`...\teraserver\python\env\python-3.8\python.exe`  
+`...\teraserver\python\env\python-3.##\python.exe`, where `##` is the current python version of the project.
 The end result should look like this :  
 ![ ](images/Windows/PyCharm.PNG)
 
 **6. Self-signed certificates generation**
 Self-signed certificates should be generated for the development server and to sign device certificates.
 
-This can simply be done by running the `CreatesCertificates.py` script (directly in PyCharm if configured)
+This can simply be done by running the `CreateCertificates.py` script (directly in PyCharm if configured)
 
 ## Starting the main OpenTera service
 If all the configuration was properly done, you should now be able to run the main script, `TeraServer.py`.

docs/developers/Developer-Setup-for-Mac.md

@@ -105,6 +105,7 @@ Each system service will requires its own database. The default database that ne
 * `opentera`, the [main OpenTera service](../services/teraserver/teraserver.rst) database
 * `openterafiles`, the database for the [file transfer](../services/FileTransfer-Service) service
 * `openteralogs`, the database for the [logging service](../services/Logging-Service)
+* `openteraemails`, the database for the [email service](../services/Email-Service)
 
 The steps to create a database are as follow. Those steps should be repeated for each database to create.
   1. In pgAdmin, under `Servers`, right-click on the correct server instance (for example `PostgreSQL 13`).
@@ -128,14 +129,14 @@ If using PyCharm, the correct Python interpreter and environment will need to be
   1. Open PyCharm, loading the project from `<base folder>/teraserver/python`
   2. In `Files -> Settings-> Project : python -> Python Interpreter`, click on the `Show all` option in the selection bar.
   3. In the `Virtualenv` tab, choose the existing environment. Change the interpreter to the following :
-`...\teraserver\python\env\python-3.8\python`  
+`...\teraserver\python\env\python-3.##\python`, where `##` is the current python version of the project.
 The end result should look like this :  
 ![ ](images/Windows/PyCharm.PNG)
 
 **6. Self-signed certificates generation**
 Self-signed certificates should be generated for the development server and to sign device certificates.
 
-This can simply be done by running the `CreatesCertificates.py` script (directly in PyCharm if configured)
+This can simply be done by running the `CreateCertificates.py` script (directly in PyCharm if configured)
 
 ## Starting the main OpenTera service
 If all the configuration was properly done, you should now be able to run the main script, `TeraServer.py`.

docs/developers/Developer-Setup-for-Windows.md

@@ -1,4 +1,4 @@
-# Getting Started for Developers - Windows 10
+# Getting Started for Developers - Windows
 ## Pre-requisites
 To get started on Windows, the following components are needed:
 * [Git](https://git-scm.com) required to get the code. Optionally, a helper software such as [TurtoiseGit](https://tortoisegit.org/) or [GitHub Desktop](https://desktop.github.com/) could be used.
@@ -104,6 +104,7 @@ Each system service will requires its own database. The default database that ne
 * `opentera`, the [main OpenTera service](../services/teraserver/teraserver.rst) database
 * `openterafiles`, the database for the [file transfer](../services/FileTransfer-Service) service
 * `openteralogs`, the database for the [logging service](../services/Logging-Service)
+* `openteraemails`, the database for the [email service](../services/Email-Service)
 
 The steps to create a database are as follow. Those steps should be repeated for each database to create.
   1. In pgAdmin, under `Servers`, right-click on the correct server instance (for example `PostgreSQL 13`).
@@ -127,18 +128,18 @@ If using PyCharm, the correct Python interpreter and environment will need to be
   1. Open PyCharm, loading the project from `<base folder>/teraserver/python`
   2. In `Files -> Settings-> Project : python -> Python Interpreter`, click on the `Show all` option in the selection bar.
   3. In the `Virtualenv` tab, choose the existing environment. Change the interpreter to the following :
-`...\teraserver\python\env\python-3.8\python.exe`  
+`...\teraserver\python\env\python-3.##\python.exe`, where `##` is the current python version of the project.
 The end result should look like this :  
 ![ ](images/Windows/PyCharm.PNG)
 
 **6. Self-signed certificates generation**
 Self-signed certificates should be generated for the development server and to sign device certificates.
 
-This can simply be done by running the `CreatesCertificates.py` script (directly in PyCharm if configured)
+This can simply be done by running the `CreateCertificates.py` script (directly in PyCharm if configured)
 
 ## Starting the main OpenTera service
 If all the configuration was properly done, you should now be able to run the main script, `TeraServer.py`.
 
 Don't forget to start the NGINX router beforehand and any other component that were not set to auto-run or as a Windows service.
 
-If all went well, you should be able to see the `About` page in your browser: [https://localhost:40075/about](https://localhost:40075/about)
+If all went well, you should be able to see the `About` page in your browser: [https://127.0.0.1:40075/about](https://127.0.0.1:40075/about)

docs/services/Email-Service.md

@@ -0,0 +1,41 @@
+# Email Service
+The email service is intented to provide email communications (sending) from the OpenTera platform. It uses an external SMTP server - the service
+doesn't implement its own server.
+
+## Main script
+The Email service can be launched by running the 
+[EmailService.py](https://github.com/introlab/opentera/blob/main/teraserver/python/services/EmailService/EmailService.py) 
+script. As a system service, it is also launched automatically when running the 
+[main OpenTera service](teraserver/teraserver.rst).
+
+## Configuration
+Configuration files for the email service are similar to the basic 
+[configuration files](../Configuration-files). They, however, add a specific section for that service.
+
+### Email config section
+`hostname`: Hostname of the SMTP email server to use.
+`port`: Port of the STMP email server
+`tls`: If true, TLS encryption will be used when communicating with the SMTP email server
+`ssl`: If true, SSL encryption will be used when communicating with the SMTP email server
+`username`: Username to use when authenticating with the SMTP email server
+`password`: Password to use when authenticating with the SMTP email server
+`default_sender`: Default email address to use as sender when none is specified
+`max_emails`: Maximum number of emails to send to the SMTP email server in one connection.
+
+## Default port and location
+By default, the service will listen to port 4043 (non-ssl) and will be at `/email` behind the NGINX router.
+
+## Web URLs and REST API
+**Doc page** - by default at https://127.0.0.1:40075/email/doc. Will display the [REST API](teraserver/api/API) 
+documentation and test system. Useful to test queries manually.
+
+## Specific notes
+Since the email server is usually associated to a domain, it's easy for emails sent that way to be classified as "junkmail" by email clients.
+
+To prevent that, ensure that users are using emails in the same domain or that you are sending emails using a default "noreply" email in the correct domain.
+
+## Web Frontend
+Currently, no web front-end is available for that service
+
+## RPC API
+None. This service uses the [asynchronous communication system](../developers/Internal-services-communication-module).

teraserver/CMakeLists.txt

@@ -10,7 +10,7 @@ endif(NOT CMAKE_BUILD_TYPE)
 # Software version
 SET(OPENTERA_VERSION_MAJOR "1")
 SET(OPENTERA_VERSION_MINOR "3")
-SET(OPENTERA_VERSION_PATCH "0")
+SET(OPENTERA_VERSION_PATCH "1")
 
 SET(OPENTERA_SERVER_VERSION OpenTera_v${OPENTERA_VERSION_MAJOR}.${OPENTERA_VERSION_MINOR}.${OPENTERA_VERSION_PATCH})
 

teraserver/README.md

@@ -1,94 +1,3 @@
 ## Getting Started for Developers
-Please follow those steps to setup your development environment. Clone this directory (change branch if not default dev) with submodules using :
-
-```
-git clone --recursive -b dev https://github.com/introlab/opentera.git
-```
-
-### Requirements
-1.  Make sure you have a valid compiler installed:
-    1.  Linux : gcc/g++. Install it with ```sudo apt-get install build-essential```
-    2.  Mac : LLVM through XCode from the App Store
-    3.  Windows: [Visual Studio C++ 2017 Community Edition](https://visualstudio.microsoft.com/fr/vs/older-downloads/)
-
-2.  Install CMake
-    1. Linux : ```sudo apt-get install cmake```
-    2. Mac & Windows: Download from [here.](https://cmake.org/download/) 
-
-3.  Install [Qt + QtCreator](https://www.qt.io/)
-    1. Use the [Qt Online Installer](https://www.qt.io/download-open-source) (will require a free Qt Account)
-    2. Linux: run the installer script, do not forget to make it executable first ```chmod +x <qt-unified-linux...>```
-    3. Install the latest Qt Open Source Edition (Qt 5.14.2 Desktop or later, will be useful for Qt client application)
-    4. Install all components except "Android" and "WebAssembly".
-    5. Use default directories
-
-4.  Install MiniConda3
-    1. Use installer from [here](https://conda.io/miniconda.html)
-    2. Install Python 3.x version for current user (in default user directory).
-    3. Use default settings.
-
-5.  Install [PyCharm Community Edition](https://www.jetbrains.com/pycharm/)
-
-6.  Install PostgreSQL with default parameters
-    1. Linux : ```sudo apt-get install postgresql```
-    2. Mac & Windows: Download and install from : https://www.postgresql.org/download/
-
-7.  Install redis server. 
-    1. Recommanded: Install redis from [Docker](https://hub.docker.com/_/redis)
-    2. Linux, Install redis with apt with `sudo apt-get install redis-server`
-    3. Windows: install [redis binaries](https://github.com/MicrosoftArchive/redis/releases)
-
-8. Install NGINX.
-    1. Windows:  http://nginx.org/en/docs/windows.html
-    2. Mac: install with [brew](https://brew.sh/index)
-    3. Linux: install with package manager : ```sudo apt-get install nginx```
-
-### Step1 : Create the database and database users (only once)
-This step needs to be done only once.
-   ```
-	sudo -u postgres psql -c "ALTER USER postgres WITH PASSWORD 'test';"
-	sudo -u postgres psql -c "create database opentera;" 
-	sudo -u postgres psql -c "create user TeraAgent with encrypted password 'test';"
-	sudo -u postgres psql -c "grant all privileges on database opentera to TeraAgent;"
-	sudo -u postgres psql -c "ALTER USER TeraAgent WITH PASSWORD 'tera';"
-	sudo -u postgres psql -c "create database openteralogs;"
-	sudo -u postgres psql -c "grant all privileges on database openteralogs to TeraAgent;"
-	sudo -u postgres psql -c "create database openterafiles;"
-	sudo -u postgres psql -c "grant all privileges on database openterafiles to TeraAgent;"
-	sudo -u postgres psql -c "create database bureauactif;"
-	sudo -u postgres psql -c "create user bureauactif with encrypted password 'bureauactif';"
-	sudo -u postgres psql -c "grant all privileges on database bureauactif to bureauactif;"
-	sudo -u postgres psql -c "ALTER USER bureauactif WITH PASSWORD 'bureauactif';"
-	sudo -u postgres psql -c "\l"
-   ```
-### Step 2 : Open the root CMakeLists.txt in QtCreator (only once)
-1.  Opening the root teraserver/CMakeLists.txt will allow to create and build the project
-    1. Build the project **using the python-all target**, it will automatically generate the Python environment in env/python-3.6
-    2. Click on the "Projects" and change Build steps target by clicking on "Details"
-    2. All python dependencies will be automatically downloaded
-    3. Once the project is built, you will not need QtCreator (for now).
-
-### Step 3 : Create a PyCharm project (only once)
-1.  Using PyCharm, opening the directory "{PROJECT_ROOT}/python"
-    1. Select the existing Python 3.6 environment in "{PROJECT_ROOT}/python/env/python-3.6" in the app menu: PyCharm->Preferences->Project:python->Project Interpreter
-
-### Step 4: Generate the TLS certicates (only once)
-1. Using PyCharm, run the CreateCretificates.py script. This will generate the TLS certificates used by nginx.
-
-### Step 5 : Run the nginx reverse proxy (every time)
-1. Go to the **{PROJECT_ROOT}/teraserver/python/config** directory.
-2. Create the logs directory (only once) : ```mkdir logs```
-3. Run the script : ```./start_nginx.sh```
-
-### Step 6 : Run the application (every time)
-1.  Run the TeraServer.py application from PyCharm
-2.  Edit the code as you would normally do in a python program.
-3.  Run tests in the tests directory
-
-### Step 7 : Try the API with swagger UI" (as needed)
-1. Navigate to : [API](https://localhost:40075/doc)
-
-### Notes
-1.  In a near future, we hope to have everything in the QtCreator IDE. Stay tuned!
-
-Enjoy!    
+Instructions to create a development setup are available here:
+https://introlab.github.io/opentera/developers/Developers.html

teraserver/easyrtc/static/js/tera_webrtc.js

@@ -1243,6 +1243,12 @@ async function shareScreen(local, start, sound_only = false){
     if (sound_only){
         streamName += "Audio";
     }
+
+    if (teraConnected){
+        if (SharedObject.setMusicSharing)
+            SharedObject.setMusicSharing(sound_only);
+    }
+
     if (start === true){
         // Start screen sharing
         let screenStream = undefined;

teraserver/python/README.md

@@ -1,41 +1,3 @@
 # OpenTera
-
-Python package for OpenTera micro-service architecture. 
-External microservices can use this package as a base.
-
-# GitHub
-* https://github.com/introlab/opentera
-
-# License
-OpenTera is licensed under [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.txt) . 
-
-# Documentation
-Please visit our [Documentation on GitHub](https://introlab.github.io/opentera/)
-
-# Dependencies
-OpenTera is based or uses the following Open Source technologies :
-* [Python 3.8+, PSFL (BSD like)](https://www.python.org)
-* [Flask, BSD](http://flask.pocoo.org)
-* [Redis, BSD](https://redislabs.com/why-redis/)
-* [txredisapi, Apache License 2.0](https://github.com/fiorix/txredisapi) 
-* [SQLAlchemy, MIT](https://www.sqlalchemy.org)
-* [Twisted, MIT](https://twistedmatrix.com)
-* [PostgreSQL, PostgreSQL License(MIT/BSD like)](https://www.postgresql.org)
-* [Node.js, Node license](https://nodejs.org/en/)
-
-# Authors
-* Dominic Létourneau, ing. M.Sc.A., IntRoLab, Université de Sherbrooke (@doumdi)
-* Simon Brière, ing. M.Sc.A., CDRV, Université de Sherbrooke (@sbriere)
-* François Michaud, ing. Ph.D., IntRoLab, Université de Sherbrooke
-* Michel Tousignant, pht, Ph.D., CDRV, Université de Sherbrooke
-
-# Contributors
-* Philippe Arsenault
-* Cédric Godin
-* Marc-Antoine Maheux
-* Cynthia Vilanova
-
-# Acknowledgements
-This project is supported by [IntRoLab - Intelligent / Interactive / Integrated / Interdisciplinary Robot Lab](https://introlab.3it.usherbrooke.ca/), Sherbrooke, Québec, Canada,
-by [Regroupement INTER](https://www.regroupementinter.com) and by the [Centre de recherche sur le vieillissement de Sherbrooke](https://www.cdrv.ca), Sherbrooke, Québec, Canada.
-
+Official documentation is available here:
+https://introlab.github.io/opentera/

teraserver/python/config/opentera.conf

@@ -39,6 +39,8 @@ location /log/ {
 
 location /file/ {
     client_max_body_size 500M;
+	proxy_buffering off;
+	proxy_request_buffering off;
     proxy_pass          http://127.0.0.1:4042/;
     proxy_redirect      http://$host/  https://$host:$server_port/;
     proxy_set_header 	X-ExternalPort      $server_port;