Merge pull request #30 from SEPIA-Framework/dev

v0.23.0 release candidate for SEPIA Home v2.5.1

Author: fquirin

README.md

@@ -8,14 +8,15 @@ Part of the [SEPIA Framework](https://sepia-framework.github.io/)
 This is a HTML-based application (SEPIA client) to communicate with SEPIA via browser, iOS and Android.  
 Features of the app are:
 * Input via voice (Android, iOS, browser) and text
-* Native (Android, iOS, Chrome) and open-source ASR supported ([see STT-Server](https://github.com/SEPIA-Framework/sepia-stt-server))
-* Text-to-Speech voice output
+* Native platform (Android, iOS, Chrome) and open-source ASR supported ([see STT-Server](https://github.com/SEPIA-Framework/sepia-stt-server))
+* Text-to-Speech voice output via client platform and open-source SEPIA server
 * WebSocket based chat with SEPIA and other users that are registered on your private or public server ([see Chat-Server](https://github.com/SEPIA-Framework/sepia-websocket-server-java))
 * Customizable start-screen with user-recommendations (created by the SEPIA server)
 * AlwaysOn mode with animated avatar (works best with OLED screens! ^_^)
 * In-chat card results (e.g. for to-do lists), audio player (e.g. for web-radio) and in-app browser for iOS and Android
 * User-account management
 * Teach-interface to add your own commands/chats/actions to SEPIA ([see Teach-Server](https://github.com/SEPIA-Framework/sepia-teach-server))
+* Remote terminal support via CLEXI server to work in 'headless' mode
 * Customizable look (skins for the app)
 * Tutorial and help screens to get started
 * Remote-actions to communicate between clients and devices (e.g. for [wake-word](https://github.com/SEPIA-Framework/sepia-wakeword-tools) integration)
@@ -28,13 +29,36 @@ Features of the app are:
 
 The latest release version is always **online** at: `https://sepia-framework.github.io/app/index.html` (requires server with SSL)  
 When you've installed **SEPIA-Home** the default link is: `http://[sepia-home-IP]:20721/app/index.html`  
-The Android app can be installed via the **Google Play Store**: [Play Store link](https://play.google.com/store/apps/details?id=de.bytemind.sepia.app.web) (currently only German)  
+The Android app can be installed via the **Google Play Store**: [Play Store link](https://play.google.com/store/apps/details?id=de.bytemind.sepia.app.web)  
 
 Note: If you don't operate your own SEPIA server you can still open the app in "demo-mode" (simply skip the log-in) and look around a bit though most of the features will not be active in this mode.  
 More languages for Android and an iOS app are in beta-test phase and will be release "when they are done" :-p  
 
 For experts only: Use the build-scripts in this repository to build your own version of the app (Android and iOS are available).
 
+### URL parameters
+
+You can modify the client configuration via several URL parameters:
+* `host` - SEPIA server host, e.g. IP address or domain + path
+* `lang` - Default language code for client (ISO code)
+* `isApp` - 'true' will change client to behave as if it was an app (not a website), e.g. login tokens will be valid for much longer etc.
+* `isTiny` - 'true' will optimize the UI and controls to support small displays (e.g. Apple Watch or 240x240px screens)
+* `isHeadless` or `autoSetup` - Load client settings from file (settings.js) and trigger setup mode if no user is logged in. 'isHeadless' will set some supported client features as well (e.g. don't open browser tabs etc.).
+* `env` - 'environment' setting for client
+* `q` - Question that will be triggered when client is connected
+* `share` - Shared data e.g. via deeplinks
+* `view` - Switch to specific view on start for example always-on mode (e.g.: aomode, teachui, "frame name")
+* `clexi` and `clexiId` - CLEXI socket URL and access ID
+* `logout` - 'true' will make sure client is logged out at start (all previous user date removed)
+* `cordova` - Set Cordova framework mode if 'true'
+* `isTest` - tbd
+
+### App deeplinks
+
+* `https://b07z.net/dl/sepia/index.html?q=[my request as text]` - e.g. `test`
+* `https://b07z.net/dl/sepia/index.html?view=[app view]` - e.g. `aomode`, `teachui`
+* `https://b07z.net/dl/sepia/index.html?share=[base64 encoded share data]` - use long-press on your user name of one of your chat entries inside the app to generate share links
+
 ### Version history ...
 
 ... can be found [here](https://github.com/SEPIA-Framework/SEPIA-Framework.github.io/blob/master/app/README.md).

config.xml

@@ -1,5 +1,5 @@
 <?xml version='1.0' encoding='utf-8'?>
-<widget id="org.example.sepia.app.web" version="0.22.0" xmlns="http://www.w3.org/ns/widgets" xmlns:cdv="http://cordova.apache.org/ns/1.0">
+<widget id="org.example.sepia.app.web" version="0.23.0" xmlns="http://www.w3.org/ns/widgets" xmlns:cdv="http://cordova.apache.org/ns/1.0">
     <name>S.E.P.I.A.</name>
     <description>
         Prototype of S.E.P.I.A. framework client
@@ -52,7 +52,7 @@
 		<!--config stuff-->
 		<preference name="AndroidLaunchMode" value="singleTask" />
 		<preference name="android-minSdkVersion" value="19" /><!-- Android 4.4+ -->
-		<preference name="android-targetSdkVersion" value="28" /><!-- Android 9.0 -->
+		<preference name="android-targetSdkVersion" value="29" /><!-- Android 10.0 -->
 		<preference name="android-windowSoftInputMode" value="adjustSize" />
 		<!-- custom manifest changes -->
 		<custom-preference name="android-manifest/application/activity/@android:theme" value="@style/SplashTheme"/> <!-- Theme mainly for splash screen or background colors -->

create-android-project.sh

@@ -24,7 +24,8 @@ cordova plugin add cordova-plugin-geolocation
 cordova plugin add cordova-plugin-inappbrowser
 cordova plugin add cordova-plugin-tts
 cordova plugin add cordova-plugin-whitelist
-cordova plugin add cordova-universal-links-plugin
+#cordova plugin add cordova-universal-links-plugin
+cordova plugin add https://github.com/sepia-assistant/cordova-universal-links-plugin.git
 cordova plugin add cordova-plugin-statusbar
 cordova plugin add cordova-plugin-splashscreen
 #cordova plugin add https://github.com/apache/cordova-plugin-splashscreen.git -- NOTE: use this in case releases are too old
@@ -34,12 +35,14 @@ cordova plugin add plugin_mods/speechrecognition/org.apache.cordova.speech.speec
 #cordova plugin add de.appplant.cordova.plugin.local-notification
 #cordova plugin add plugin_mods/localnotifications/de.appplant.cordova.plugin.local-notification
 #cordova plugin add [email protected]
-cordova plugin add cordova-plugin-local-notification
+#cordova plugin add cordova-plugin-local-notification
+cordova plugin add https://github.com/timkellypa/cordova-plugin-local-notifications
 cordova plugin add cordova-plugin-file
 cordova plugin add cordova-plugin-nativestorage
 cordova plugin add cordova-android-support-gradle-release
 cordova plugin add phonegap-plugin-media-stream
-cordova plugin add https://github.com/EddyVerbruggen/Insomnia-PhoneGap-Plugin.git
+#cordova plugin add https://github.com/EddyVerbruggen/Insomnia-PhoneGap-Plugin.git
+cordova plugin add https://github.com/tombolaltd/cordova-plugin-insomnia.git
 cordova plugin add cordova-plugin-eddystone
 cordova plugin add cordova-plugin-navigationbar-color
 #NOTE: add before intent plugin if you use: plugin add cordova-plugin-camera
@@ -56,21 +59,21 @@ cp -r -f "plugin_mods/inappbrowser/android/res/" "plugins/cordova-plugin-inappbr
 # add android platform
 sleep 2
 echo "#Adding platform ..."
-cordova platform add android@6.4.0
+cordova platform add android@8.1.0
 #
 # prepare build
 echo "#Preparing build ..."
 cordova prepare android
 # overwrite icons (this will be replaced with a proper implementation)
 sleep 2
 echo "#Adding or overwriting resources, e.g. icons, themes, xml ..."
-cp -r -f "resources/icons/android/notifications/res/" "platforms/android/res/"
-cp -r "resources/themes/android/background_splash.xml" "platforms/android/res/drawable/background_splash.xml"
-cp -r "resources/themes/android/launch_screen.png" "platforms/android/res/drawable/launch_screen.png"
-#cp -r "resources/themes/android/styles.xml" "platforms/android/res/values/styles.xml"
-cp -r "resources/themes/android/values/" "platforms/android/res/values/"
-mkdir -p "platforms/android/res/values-v21"
-cp -r "resources/themes/android/values-v21/" "platforms/android/res/values-v21/"
-cp -r "resources/config/android/xml/" "platforms/android/res/xml/"
+cp -r -f "resources/icons/android/notifications/res/" "platforms/android/app/src/main/res/"
+cp -r "resources/themes/android/background_splash.xml" "platforms/android/app/src/main/res/drawable/background_splash.xml"
+cp -r "resources/themes/android/launch_screen.png" "platforms/android/app/src/main/res/drawable/launch_screen.png"
+#cp -r "resources/themes/android/styles.xml" "platforms/android/app/src/main/res/values/styles.xml"
+cp -r "resources/themes/android/values/" "platforms/android/app/src/main/res/values/"
+mkdir -p "platforms/android/app/src/main/res/values-v21"
+cp -r "resources/themes/android/values-v21/" "platforms/android/app/src/main/res/values-v21/"
+cp -r "resources/config/android/xml/" "platforms/android/app/src/main/res/xml/"
 echo "#DONE"
 echo "If everything worked out fine the next step would be to build the app from the %APP_NAME% folder: cordova build android"

hooks/add-android-intent-filters.js

@@ -13,7 +13,11 @@ module.exports = function (context) {
 		<category android:name="android.intent.category.DEFAULT" />
 	</intent-filter>
     `;
-    const manifestPath = context.opts.projectRoot + '/platforms/android/AndroidManifest.xml';
+    let manifestPath = context.opts.projectRoot + '/platforms/android/AndroidManifest.xml';
+	if (!fs.existsSync(manifestPath)){
+		//new path
+		manifestPath = context.opts.projectRoot + '/platforms/android/app/src/main/AndroidManifest.xml';
+	}
     const androidManifest = fs.readFileSync(manifestPath).toString();
     if (!androidManifest.includes(`android:scheme="${scheme}"`)) {
         const manifestLines = androidManifest.split(/\r?\n/);

hooks/add-android-permissions.js

@@ -14,7 +14,11 @@ let permissions = [
 module.exports = function (context) {
   const parseString = xml2js.parseString;
   const builder = new xml2js.Builder();
-  const manifestPath = context.opts.projectRoot + '/platforms/android/AndroidManifest.xml';
+  let manifestPath = context.opts.projectRoot + '/platforms/android/AndroidManifest.xml';
+  if (!fs.existsSync(manifestPath)){
+    //new path
+    manifestPath = context.opts.projectRoot + '/platforms/android/app/src/main/AndroidManifest.xml';
+  }
   const androidManifest = fs.readFileSync(manifestPath).toString();
 
   let manifestRoot,

plugin_mods/manual_fix_for_splashscreen.md

@@ -0,0 +1,25 @@
+## ADD THE CODE FROM BELOW TO 'SplashScreen.java'
+
+```
+// Create and show the dialog
+splashDialog = new Dialog(context, android.R.style.Theme_Translucent_NoTitleBar);
+// check to see if the splash screen should be full screen
+if ((cordova.getActivity().getWindow().getAttributes().flags & WindowManager.LayoutParams.FLAG_FULLSCREEN)
+		== WindowManager.LayoutParams.FLAG_FULLSCREEN) {
+	splashDialog.getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN,
+			WindowManager.LayoutParams.FLAG_FULLSCREEN);
+}
+//---NEW---
+if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.LOLLIPOP) {
+	try {
+		splashDialog.getWindow().addFlags(WindowManager.LayoutParams.FLAG_DRAWS_SYSTEM_BAR_BACKGROUNDS);
+		splashDialog.getWindow().setNavigationBarColor(Color.parseColor(preferences.getString("NavigationBarBackgroundColor", "#000000")));
+	} catch (Exception e) {
+		e.printStackTrace();
+	}
+}
+//---NEW END---
+splashDialog.setContentView(splashImageView);
+splashDialog.setCancelable(false);
+splashDialog.show();
+```

plugin_mods/speechrecognition/org.apache.cordova.speech.speechrecognition/src/android/SpeechRecognition.java

@@ -99,12 +99,13 @@ public void run() {
         }
         else if (ACTION_SPEECH_RECOGNIZE_START.equals(action)) {
             // recognize speech
-            if (!recognizerPresent) {
+            if (!this.recognizerPresent) {
                 callbackContext.sendPluginResult(new PluginResult(PluginResult.Status.ERROR, NOT_PRESENT_MESSAGE));
-            }
-            this.lang = args.optString(0, "en");
-            this.speechRecognizerCallbackContext = callbackContext;
-            this.promptForMic();
+            }else{
+				this.lang = args.optString(0, "en");
+				this.speechRecognizerCallbackContext = callbackContext;
+				this.promptForMic();
+			}
         }
         else if (ACTION_SPEECH_RECOGNIZE_STOP.equals(action)) {
             stop(false);
@@ -135,7 +136,9 @@ private void startRecognition() {
 
             @Override
             public void run() {
-                recognizer.startListening(intent);
+				if (recognizer != null){
+					recognizer.startListening(intent);
+				}
             }
 
         });
@@ -152,7 +155,9 @@ private void stop(boolean abort) {
 
             @Override
             public void run() {
-                recognizer.stopListening();
+				if (recognizer != null){
+					recognizer.stopListening();
+				}
             }
 
         });

win-create-android-project.bat

@@ -23,7 +23,8 @@ cordova plugin add cordova-plugin-geolocation |more
 cordova plugin add cordova-plugin-inappbrowser |more
 cordova plugin add cordova-plugin-tts |more
 cordova plugin add cordova-plugin-whitelist |more
-cordova plugin add cordova-universal-links-plugin |more
+REM cordova plugin add cordova-universal-links-plugin |more
+cordova plugin add https://github.com/sepia-assistant/cordova-universal-links-plugin.git |more
 cordova plugin add cordova-plugin-statusbar |more
 cordova plugin add cordova-plugin-splashscreen |more
 REM cordova plugin add https://github.com/apache/cordova-plugin-splashscreen.git |more -- NOTE: use this in case releases are too old
@@ -32,12 +33,14 @@ cordova plugin add cordova-custom-config |more
 cordova plugin add plugin_mods/speechrecognition/org.apache.cordova.speech.speechrecognition |more
 REM cordova plugin add de.appplant.cordova.plugin.local-notification |more
 REM cordova plugin add plugin_mods/localnotifications/de.appplant.cordova.plugin.local-notification |more
-cordova plugin add cordova-plugin-local-notification |more
+REM cordova plugin add cordova-plugin-local-notification |more
+cordova plugin add https://github.com/timkellypa/cordova-plugin-local-notifications |more
 cordova plugin add cordova-plugin-file |more
 cordova plugin add cordova-plugin-nativestorage |more
 cordova plugin add cordova-android-support-gradle-release |more
 cordova plugin add phonegap-plugin-media-stream |more
-cordova plugin add https://github.com/EddyVerbruggen/Insomnia-PhoneGap-Plugin.git |more
+REM cordova plugin add https://github.com/EddyVerbruggen/Insomnia-PhoneGap-Plugin.git |more
+cordova plugin add https://github.com/tombolaltd/cordova-plugin-insomnia.git |more
 cordova plugin add cordova-plugin-eddystone |more
 cordova plugin add cordova-plugin-navigationbar-color |more
 REM NOTE: add before intent plugin if you use: plugin add cordova-plugin-camera
@@ -54,20 +57,20 @@ REM
 REM add android platform
 sleep 2
 echo "#Adding platform ..."
-cordova platform add android@6.4.0 |more
+cordova platform add android@8.1.0 |more
 REM
 REM prepare build
 echo "#Preparing build ..."
 cordova prepare android |more
 REM overwrite icons (this will be replaced with a proper implementation)
 sleep 2
 echo "#Adding or overwriting resources, e.g. icons, themes, xml ..."
-xcopy "resources\icons\android\notifications\res" "platforms\android\res" /e /h /y |more
-xcopy "resources\themes\android\background_splash.xml" "platforms\android\res\drawable" |more
-xcopy "resources\themes\android\launch_screen.png" "platforms\android\res\drawable" |more
-xcopy "resources\themes\android\values" "platforms\android\res\values" |more
-xcopy "resources\themes\android\values-v21" "platforms\android\res\values-v21" /i |more
-xcopy "resources\config\android\xml" "platforms\android\res\xml" /i |more
+xcopy "resources\icons\android\notifications\res" "platforms\android\app\src\main\res" /e /h /y |more
+xcopy "resources\themes\android\background_splash.xml" "platforms\android\app\src\main\res\drawable" |more
+xcopy "resources\themes\android\launch_screen.png" "platforms\android\app\src\main\res\drawable" |more
+xcopy "resources\themes\android\values" "platforms\android\app\src\main\res\values" |more
+xcopy "resources\themes\android\values-v21" "platforms\android\app\src\main\res\values-v21" /i |more
+xcopy "resources\config\android\xml" "platforms\android\app\src\main\res\xml" /i |more
 echo "#DONE"
 echo "If everything worked out fine the next step would be to build the app from the %APP_NAME% folder: cordova build android"
 pause

www/channel-manager.html

@@ -310,15 +310,15 @@ <h3>Invite User:</h3>
 					if (ids.length > 0){
 						var note = document.createElement("div");
 						var info = document.createElement("p");
-						info.innerHTML = "Users you've recently interacted with:";
+						info.textContent = "Users you've recently interacted with:";
 						var list = document.createElement("div");
 						list.className = "chooseBox";
 						ids.forEach(function(id){
 							var c = contactsFromChat[id];
 							var entry = document.createElement("div");
 							entry.className = "entry";
 							entry.dataset.id = c.id;
-							entry.innerHTML = c.name + " (" + c.id + ")";
+							entry.textContent = c.name + " (" + c.id + ")";
 							list.appendChild(entry);
 							$(entry).on('click', function(){
 								$('#sepiaFW-channel-manager-invite-user').val(this.dataset.id);

www/credits.html

@@ -13,6 +13,7 @@ <h3>Credits</h3>
 			<a href="https://github.com/mattdiamond/Recorderjs">Recorder.js</a><br>
 			<a href="https://github.com/kdavis-mozilla/vad.js">vad.js</a><br>
 			<a href="https://github.com/iconfu/svg-inject">SVGInject</a><br>
+			<a href="https://github.com/cure53/DOMPurify">DOMPurify</a><br>
 			<p><u>Icons</u></p>
 			<a href="https://material.io/tools/icons">Google Material Design</a><br>
 			<p><u>Sounds</u></p>