Merge pull request #233 from f5devcentral/development

merging release_0_9_5 changes to main

Author: Vasavi-Yarramneni

COLLECTOR_VERSION

@@ -1 +1 @@
-v0.9.4
+v0.9.5

CONTRIBUTING.md

@@ -97,4 +97,4 @@ concatenate all commit messages right before you hit the "Confirm squash and mer
 button. Maintainers must make sure to edit this concatenated message to make it right before merging.
 In some cases, if the commit messages are lacking the easiest approach to have at
 least something useful is copy/pasting the PR description into the commit message box
-before merging (but see the above paragraph about writing good commit messages in the first place).
+before merging (but see the above paragraph about writing good commit messages in the first place).

README.md

@@ -11,7 +11,7 @@ configuration, troubleshooting (REST changes, HIGH CPU on control plane) info, e
 >
 > To review ideas on integrating your secrets with a vault, see the [Integrating your secrets with Hashi vault](https://community.f5.com/kb/TechnicalArticles/f5-app-study-tool-with-passwords-stored-in-vault/341155) for further information.
 
-The Application Study Tool is intended to provide enhanced insights into (classic) BIG-IP products, leveraging best in class
+The F5 Application Study Tool is intended to provide enhanced insights into (classic) BIG-IP products, leveraging best in class
 open source telemetry tools. The full installation includes:
 
 * Custom Instance of OpenTelemetry Collector with enhanced BIG-IP data receivers (data fetched via iControlRest) [Full List of Metrics Collected](pages/components/otel_collector/receiver_metrics.md).
@@ -177,6 +177,8 @@ Create a file called .env.device-secrets, and add your BIP passwords like so:
 BIGIP_PASSWORD_1=foo-bar123!
 BIGIP_PASSWORD_2=bar-foo123!
 ```
+> **Note:** Ensure that the permissions on the (_.env.device-secrets_) file are restricted to allow read access only to the user running the Docker containers.
+> This ensures that credential information remains protected from unauthorized access.
 
 The variable name (the part on the left of the equal sign) must match the configured
 value for the devices that use this password in config/ast_defaults.yaml or device specific
@@ -352,7 +354,7 @@ cp .env-example .env
 ```
 
 ### Run Application Study Tool
-Once the above configurations have been made, the tool can be started with:
+After the above configurations have been made, start the tool with:
 
 ```shell
 # `docker compose up -d` to start in background mode
@@ -361,7 +363,7 @@ docker compose up
 
 #### View The Dashboards
 The default Grafana user/pass is `admin/admin`, and can be accessed at
-`http://<hostname>:3000`.
+`http://<hostname>:3000`. If HTTPS is configured, use `https://<hostname>:3001`.
 
 
 ## Updating AST Versions
@@ -375,15 +377,14 @@ special instructions / breaking changes.
 git stash
 git fetch --tags
 git pull origin main
-git checkout tags/RELEASE_VERSION #(e.g. tags/v0.9.4)
+git checkout tags/RELEASE_VERSION #(e.g. tags/v0.9.5)
 git stash pop
 # <merge any conflicts with your local changes>
 # <re-run config scripts>
 docker compose down
 # `docker compose up -d` to start in background mode
 docker compose up
 ```
-
 ## Support
 
 For support, please open a GitHub issue.  Note, the code in this repository is community supported and is not supported by F5 Networks.  For a complete list of supported projects please reference [SUPPORT.md](SUPPORT.md).
@@ -398,7 +399,7 @@ Please refer to the [F5 DevCentral Community Code of Conduct](code_of_conduct.md
 
 ## Copyright
 
-Copyright 2014-2024 F5 Networks Inc.
+Copyright 2014-2025 F5 Networks Inc.
 
 ### F5 Networks Contributor License Agreement
 
@@ -410,4 +411,4 @@ Otherwise by submitting a CLA you represent that you are legally entitled to gra
 If your employer has rights to intellectual property that you create, such as your contributions, you represent that you have received permission to make contributions on behalf of that employer, that your employer has waived such rights for your contributions, or that your employer has executed a separate CLA with F5.
 
 If you are signing on behalf of a company, you represent that you are legally entitled to grant the license recited therein.
-You represent further that each employee of the entity that submits contributions is authorized to submit such contributions on behalf of the entity pursuant to the CLA.
+You represent further that each employee of the entity that submits contributions is authorized to submit such contributions on behalf of the entity pursuant to the CLA.

docker-compose.yaml

@@ -6,7 +6,8 @@ volumes:
 
 services:
   prometheus:
-    image: prom/prometheus:v2.54.1
+    # prom/prometheus:v2.53.5
+    image: prom/prometheus@sha256:7a34573f0b9c952286b33d537f233cd5b708e12263733aa646e50c33f598f16c
     container_name: prometheus
     restart: unless-stopped
     stop_grace_period: 5m
@@ -21,13 +22,14 @@ services:
       - '--web.enable-lifecycle'
       - '--enable-feature=otlp-write-receiver'
       - '--storage.tsdb.retention.time=1y'
-    ports:
-      - 9090:9090
+    expose:
+      - 9090
     networks:
       - 7lc_network
 
   otel-collector:
-    image: ghcr.io/f5devcentral/application-study-tool/otel_custom_collector:v0.9.4
+    # ghcr.io/f5devcentral/application-study-tool/otel_custom_collector:v0.9.5
+    image: ghcr.io/f5devcentral/application-study-tool/otel_custom_collector:v0.9.5
     restart: unless-stopped
     volumes:
       - ./services/otel_collector:/etc/otel-collector-config
@@ -40,7 +42,8 @@ services:
       - 7lc_network
 
   grafana:
-    image: grafana/grafana:11.2.0
+    # grafana/grafana:v11.6.3
+    image: grafana/grafana@sha256:6128afd8174f01e39a78341cb457588f723bbb9c3b25c4d43c4b775881767069
     container_name: grafana
     restart: unless-stopped
     ports:

https_setup.md

@@ -0,0 +1,46 @@
+### Enabling HTTPS for Grafana
+
+This section outlines the steps required to enable HTTPS for Grafana when deployed using Docker Compose.
+
+#### 1. Generate SSL Certificate and Key
+
+To enable HTTPS, you need a certificate `(cert.pem)` and a private key `(key.pem)`. For local development, you can generate self-signed certificates using OpenSSL with the following commands:
+
+```sh
+mkdir -p ./services/grafana/ssl
+openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
+  -keyout ./services/grafana/ssl/key.pem -out ./services/grafana/ssl/cert.pem \
+  -subj "/CN=localhost"
+```
+
+> **Note:** In production environments, always use certificates from a trusted Certificate Authority (CA).
+> It is recommended to rotate these certificates regularly before they expire to minimize the risk of security breaches.
+
+> **Reference:** For more detailed guidance on configuring HTTPS, refer to the [official Grafana documentation](https://grafana.com/docs/grafana/latest/setup-grafana/set-up-https/).
+
+#### 2. Modify the Docker Compose Configuration
+
+Update your `docker-compose.yaml` file with the necessary configurations to enable HTTPS for Grafana. Below is an example snippet for the Grafana service:
+
+```yaml
+grafana:
+  image: grafana/grafana:11.6.3
+  container_name: grafana
+  restart: unless-stopped
+  ports:
+    - 3000:3000
+    - 3001:3001 # HTTPS port
+  volumes:
+    - grafana:/var/lib/grafana
+    - ./services/grafana/provisioning/:/etc/grafana/provisioning
+    - ./services/grafana/ssl/cert.pem:/etc/grafana/cert.pem:ro
+    - ./services/grafana/ssl/key.pem:/etc/grafana/key.pem:ro
+  env_file: ".env"
+  environment:
+    - GF_SERVER_PROTOCOL=https
+    - GF_SERVER_CERT_FILE=/etc/grafana/cert.pem
+    - GF_SERVER_CERT_KEY=/etc/grafana/key.pem
+    - GF_SERVER_HTTP_PORT=3001
+```
+
+By following these steps, you will successfully enable HTTPS for your Grafana deployment. Ensure you test your configuration in both development and production environments to verify functionality and security compliance.

pages/Gemfile

@@ -32,8 +32,12 @@ gem "wdm", "~> 0.1", :platforms => [:mingw, :x64_mingw, :mswin]
 # do not have a Java counterpart.
 gem "http_parser.rb", "~> 0.6.0", :platforms => [:jruby]
 
-gem 'nokogiri'
-gem 'rack', '~> 2.2.4'
+gem 'nokogiri', '>= 1.18.8'
+gem 'rack', '~> 2.2.14'
 gem 'rspec'
 
+gem 'google-protobuf', '>= 4.28.2'
+gem 'webrick', '>= 1.8.2'
+gem 'rexml', '>= 3.3.9'
+
 gem "html-proofer", "~> 5.0", :group => :development

pages/Gemfile.lock

@@ -30,10 +30,10 @@ GEM
       fiber-storage
     fiber-storage (1.0.0)
     forwardable-extended (2.6.0)
-    google-protobuf (4.28.1-arm64-darwin)
+    google-protobuf (4.31.1-arm64-darwin)
       bigdecimal
       rake (>= 13)
-    google-protobuf (4.28.1-x86_64-linux)
+    google-protobuf (4.31.1-x86_64-linux-gnu)
       bigdecimal
       rake (>= 13)
     hashery (2.1.2)
@@ -89,9 +89,9 @@ GEM
       rb-fsevent (~> 0.10, >= 0.10.3)
       rb-inotify (~> 0.9, >= 0.9.10)
     mercenary (0.4.0)
-    nokogiri (1.16.7-arm64-darwin)
+    nokogiri (1.18.8-arm64-darwin)
       racc (~> 1.4)
-    nokogiri (1.16.7-x86_64-linux)
+    nokogiri (1.18.8-x86_64-linux-gnu)
       racc (~> 1.4)
     pathutil (0.16.2)
       forwardable-extended (~> 2.6)
@@ -103,13 +103,13 @@ GEM
       ttfunk
     public_suffix (6.0.1)
     racc (1.8.1)
-    rack (2.2.10)
+    rack (2.2.17)
     rainbow (3.1.1)
     rake (13.2.1)
     rb-fsevent (0.11.2)
     rb-inotify (0.11.1)
       ffi (~> 1.0)
-    rexml (3.3.7)
+    rexml (3.4.1)
     rouge (4.3.0)
     rspec (3.13.0)
       rspec-core (~> 3.13.0)
@@ -137,7 +137,7 @@ GEM
     typhoeus (1.4.1)
       ethon (>= 0.9.0)
     unicode-display_width (2.6.0)
-    webrick (1.8.1)
+    webrick (1.9.1)
     yell (2.2.2)
     zeitwerk (2.7.1)
 
@@ -146,16 +146,19 @@ PLATFORMS
   x86_64-linux-gnu
 
 DEPENDENCIES
+  google-protobuf (>= 4.28.2)
   html-proofer (~> 5.0)
   http_parser.rb (~> 0.6.0)
   jekyll (~> 4.3.4)
   just-the-docs (= 0.10.0)
-  nokogiri
-  rack (~> 2.2.4)
+  nokogiri (>= 1.18.8)
+  rack (~> 2.2.14)
+  rexml (>= 3.3.9)
   rspec
   tzinfo (>= 1, < 3)
   tzinfo-data
   wdm (~> 0.1)
+  webrick (>= 1.8.2)
 
 BUNDLED WITH
    2.5.9

pages/troubleshooting/troubleshooting.md

@@ -80,19 +80,19 @@ This output shows everything looks good:
 ```shell
 $ docker ps
 CONTAINER ID   IMAGE                                                                      COMMAND                  CREATED              STATUS          PORTS                       NAMES
-cb4cf8867390   grafana/grafana:11.2.0                                                     "/run.sh"                About a minute ago   Up 49 seconds   0.0.0.0:3000->3000/tcp      grafana
-bb8891f2cd47   prom/prometheus:v2.54.1                                                    "/bin/prometheus --c…"   About a minute ago   Up 49 seconds   0.0.0.0:9090->9090/tcp      prometheus
-df2739cd67cb   ghcr.io/f5devcentral/application-study-tool/otel_custom_collector:v0.6.0   "/otelcol-custom --c…"   About a minute ago   Up 49 seconds   4317/tcp, 55679-55680/tcp   application-study-tool-otel-collector-1
+cb4cf8867390   grafana/grafana:11.6.3                                                     "/run.sh"                About a minute ago   Up 49 seconds   0.0.0.0:3000->3000/tcp      grafana
+bb8891f2cd47   prom/prometheus:v2.53.5                                                    "/bin/prometheus --c…"   About a minute ago   Up 49 seconds   0.0.0.0:9090->9090/tcp      prometheus
+df2739cd67cb   ghcr.io/f5devcentral/application-study-tool/otel_custom_collector:v0.9.5       "/otelcol-custom --c…"   About a minute ago   Up 49 seconds   4317/tcp, 55679-55680/tcp   application-study-tool-otel-collector-1
 ```
 
 This output shows a problem (Restarting container) for the `application-study-tool-otel-collector-1`
 container (the otel collector):
 ```shell
 $ docker ps
 CONTAINER ID   IMAGE                                                                      COMMAND                  CREATED          STATUS                         PORTS                    NAMES
-fdbde8a3ee16   ghcr.io/f5devcentral/application-study-tool/otel_custom_collector:v0.6.0   "/otelcol-custom --c…"   14 seconds ago   Restarting (1) 5 seconds ago                            application-study-tool-otel-collector-1
-b7ef41accd46   grafana/grafana:11.2.0                                                     "/run.sh"                14 seconds ago   Up 13 seconds                  0.0.0.0:3000->3000/tcp   grafana
-8edff3e8666e   prom/prometheus:v2.54.1                                                    "/bin/prometheus --c…"   14 seconds ago   Up 13 seconds                  0.0.0.0:9090->9090/tcp   prometheus
+fdbde8a3ee16   ghcr.io/f5devcentral/application-study-tool/otel_custom_collector:v0.9.5       "/otelcol-custom --c…"   14 seconds ago   Restarting (1) 5 seconds ago                            application-study-tool-otel-collector-1
+b7ef41accd46   grafana/grafana:11.6.3                                                     "/run.sh"                14 seconds ago   Up 13 seconds                  0.0.0.0:3000->3000/tcp   grafana
+8edff3e8666e   prom/prometheus:v2.53.5                                                    "/bin/prometheus --c…"   14 seconds ago   Up 13 seconds                  0.0.0.0:9090->9090/tcp   prometheus
 ```
 
 ### View Docker Container Logs

services/grafana/provisioning/dashboards/bigip/device/device-overview.json

@@ -1178,7 +1178,7 @@
           "exemplar": false,
           "expr": "f5_system_memory_used_bytes{job=\"$device_name\", state=\"used\"} / ignoring(state) f5_system_memory_total_bytes{job=\"$device_name\"}",
           "instant": true,
-          "legendFormat": "__auto",
+          "legendFormat": "{{instance}}",
           "range": false,
           "refId": "A"
         }