Introduce Logging Stack: Add fluentd, add loki (🚧🚧 DEVOPS 🚧🚧) (#1058)

* wip

* Add csi-s3 and have portainer use it

* Change request @Hrytsuk 1GB max portainer volume size

* Arch Linux Certificates Customization

* Fix pgsql exporter failure

* [Kubernetes] Introduce on-prem persistent Storage (Longhorn) 🎉  (#979)

* Introduce longhorn chart

* Further longhorn configuration

* Longhorn: further settings configuration

* Fix longhorn configuration bugs

Extra: introduce longhorn pv vales for portainer

* Add comment for deletion longhorn

* Further longhorn configuration

* Add README.md for Longhorn wit FAQ

* Update Longhorn readme

* Update readme

* Futher LH configuration

* Update LH's Readme

* Update Longhorn Readme

* Improve LH's Readme

* LH: Reduce reserved default disk space to 5%

Since we use a dedicated disk for LH, we can go ahead with 5%

* Use values to set Longhorn storage class

* Update LH's Readme

* LH Readme: add requirements reference

* PR Review: bring back portainer s3 pv

* LH: decrease portinaer volume size

* Experimental: Try to add tracing to simcore-traefik on master

* wip

* Fix accidental commit

* Fluentd fixes

* Add placement constraints

* revert arch linux changes

* Remove DNS setting

* wip

* Remove accidental commit

* Add README.md, remove loki docker volume

* Update README.md

---------

Co-authored-by: Dustin Kaiser <[email protected]>
Co-authored-by: YH <[email protected]>

Author: mrnicegyu11

scripts/deployments/deploy_everything_locally.bash

@@ -243,9 +243,9 @@ if [ "$start_opsstack" -eq 0 ]; then
     call_make "." up-"$stack_target";
     popd
 
-    # -------------------------------- GRAYLOG -------------------------------
-    log_info "starting graylog..."
-    service_dir="${repo_basedir}"/services/graylog
+    # -------------------------------- LOGGING -------------------------------
+    log_info "starting logging..."
+    service_dir="${repo_basedir}"/services/logging
     pushd "${service_dir}"
     call_make "." up-"$stack_target"
     sleep 1

services/graylog/GraylogWorkflow.png

@@ -0,0 +1,28 @@
+services:
+  mongodb:
+    deploy:
+      placement:
+        constraints:
+          - node.labels.logging==true
+  elasticsearch:
+    deploy:
+      placement:
+        constraints:
+          - node.labels.logging==true
+  graylog:
+    deploy:
+      placement:
+        constraints:
+          - node.labels.logging==true
+
+  fluentd:
+    deploy:
+      placement:
+        constraints:
+          - node.labels.logging==true
+
+  loki:
+    deploy:
+      placement:
+        constraints:
+          - node.labels.logging==true