* Create LICENSE Signed-off-by: Gerhard Dinhof g.dinhof@ibm.com * docker-compose setup to start Issuer-Verifier and Holder AcaPys as well as Postgres DBs * Clean up docker-compose setup * Add postman based load generator for testing if the local setup works as expected * choreinitialized kotlin spring boot project * chore: added copyright * added maven to git ignore * docs: removed generated help md * chore!: updated to jdk 17 * docs: added copyright * feat: added boilerplate apis * feat: added Dockerfile * docs: added README * Add submodule + script for VON-network * restructuring the files * Automatically register public DID on the ledger * code cleanup * build: Switch to Java 11 as the built target for brought compatibility * feat: Integration of an automatically generated SwaggerUI * Update README.md * Update README.md * feat(AcaPy Integration): Integration of an AcaPy Java client library for calling the AcaPy Admin API as well as receiving Webhook calls from AcaPy * Working state with permission error in tails * tails server working * added extended logging capabilities * removed gitignore * remove ignored files * chor(Postman): Restructure files * chor(.idea): Update .idea files * chor(Setup): Let user know that it will take 30 seconds to register DID * chor(Setup): Automatically load and update git submodules * build(Java Version): Switch to Java Version 11 * Added README for the setup * Fixed typo in environment files * chore: Rename WebhookController to AcaPyWebhookController and LoadGeneratorController to InfoController * refactor: Structure of controllers and Swagger UI * feat: Integrate simple webhook logging * feat(TestRunner): application.yml is used to choose and configure the test case * feat(FullProcessRunner): Write schema and credential definition to ledger during setup * feat(FullProcessRunner): Connect to Issuer/Verifier as well as Holder AcaPy * feat(FullProcessRunner): Establish connection between Issuer/Verifier and Holder * build(Restart Policy): Restart each container always * feat(FullProcessRunner): Issue credential to holder and request credential from holder via proof request * refactor(Spring Configs): Move all Spring Config files into a config package * refactor(Aries Client): Build a wrapper around the AcaPyAriesClient library to make it easier replaceable by a different AriesClient library 1. to make it easier to time the HTTP requests the AriesClient 2. to potentially be able to use this load generators to test other agents too * refactor(Logger): Move logger into their own package * refactor(Aries Event Handler): Build a wrapper around the AcaPyEventHandler to make it easier replaceable by a different AriesClient library To potentially be able to use this load generators to test other agents too. * refactor(AriesClient and AriesObserver): Move service interfaces into the domain layer * chore: Renaming of package * chore: Clean up SwaggerConfig * feat(Readme): Update the Readmes * feat(Wallet Logger): Log the size of the issuer/holder wallet database periodically * refactor(Runner): Move termination condition into own method * feat(logging): added Loki as log collector * feat(Log HTTP Requests): Log the durations and Status Codes of HTTP requests going out to the AcaPy * chore(Renaming): Rename AriesEventLogger types * chore: Remove unused volume from docker-compose * chore: Clean WalletLogger log message * refactor(Agent Implementations): Move all AcaPy agent specific code into one folder * chore: Adopt manage.sh logging * chore: Adopt manage.sh logging * feat(AcaPy Http Request Timeout): Make it possible to configure the timeout for HTTP requests to the AcaPy clients * chore(Name Containers): Name the Grafana and Loki container * chore(Logging): Remove unused Logback configuration * chore(Logging): Adopt Logback output format * feat(Provision Grafana): Add Loki as the default datasource to Grafana and deactivate authentication for Grafana * fix(Wallet Logger): remove comma from log message * feat(Dashboard): Configured a default Grafana dashboard that is automatically provisioned on startup of Grafana * feat(Dashboard): Add overviews about Connections, Credentials, and Proof Requests to the dashboard * feat(Dashboard): Visualize the HTTP Response Codes and HTTP Response Times in Grafana * feat(Dashboard): Visualize the rate in which connections are being activated, credentials are being issued, and proof requests are being verified in Grafana * fix(AcaPy Event Publisher): Fix string date to milliseconds conversion * fix(Loki Config): Add custom Loki config to avoid the "Too many outstanding requests." issue * feat(Dashboard): Add total number of connection, credential issuance, and proof request count * refactor(Dashboard): Rename the keys that are being logged * chore(.env): Remove .env file from git * chore(Dashboard): Update HTTP Response Times metric * chore(Dashboard): Fix labeling rate is not per Second but per Minute * chore(Dashboard): Provide the wallet DB size in MB instead of bytes * chore(Dashboard): Outline the number of test iterations started per minute on the dashboard to visualize the load the load generator puts on the Agents * feat(FullProcessRunner): Add a new test runner to start new full process iterations on a constant "iteration per minute" base * chore(Dashboard): Reorder dashboard panels * feat(Export PDF Report): Grafana dashboards can now be exported as a PDF * fix(Dashboard Deployment): Put dashboard and logging containers into a separate docker network to avoid error when shutting everything down * refactor(Full Process Runner): Avoid code duplication across the FullProcessRunners * feat(Full Process Runner): Schedule FullProcessConstantLoadRunner at a fixed rate leveraging a thread pool * refactor(Aries Agent): Extract connectionId from ProofRequestDo and CredentialDo in preparation for connectionless proof requests * feat(Connectionless Proof Requests): Started implementing support for connectionless proof requests - not finished yet * feat(Scale AcaPy using Nginx): Scale AcaPy horizontally by using a Nginx container as a load balancer * feat(Scale AcaPy using Nginx): Scale Issuer/Verifier as well as Holder AcaPy to 10 instances each by using a Nginx container as a load balancer * feat(AcaPy Wallet Type): Make AcaPy Wallet Type configurable via the .env file * fix(.env): Fix .env file * feat(Handle Exception on Connection Invitation Receive): Start a new iteration if the holder is not able to accept the connection invitation * feat(Dashbaord): Show number of failes "POST/connections/receive-invitation" * chore(Holder AcaPy): Increase number of Holder AcaPys to 20 * chore(Dashboard Setup): Define the containers Grafana depends on * chore(Dashboard Setup): Define the containers Grafana depends on * chore(AcaPy): Increase default AcaPy Version * fix: Circular dependency in dashboard/logging deployment * refactor(TestRunners and TestFlows): Differentiate between TestRunners and TestFlows to make it easier to implement differen TestFlows while reusing the existing TestRunners * chore(Readme and Dashboard): Adopt the Readme and Dashboard to meet the new setup * feat(Issuer Flow): Added a new test flow to simulate an issuer who is establishing connections to holders and issues credentials to them afterwards * chore(Application Config): Add new application configuration settings and rename existing ones * chore(Application Config): Add new application configuration settings and rename existing ones * chore: Readme * feat(Revocable Credentials): Configure if a credential should be revocable, the size of the revocation registry, and if a proof request should include a non revoked check * chore(OOB Credential Offer): Preparations to support OOB Credential Offers * feat(Wallet DB Connections): Configure the number of connections to the Wallet DBs * feat(Connection Flow): Test Flow that only established connections with the holder * feat(Credential Issuance Flow): Test Flow that only issues credentials to the holder * feat(Proof Request Flow): Test Flow that only requests proofs from the holder * feat(Increasing Load Test Runner): Added a new test runner that increases the peak load in predefined steps * fix: Count of iterations in the Increasing-Load-Test-Runner * feat(AcaPy): Make Storage Type used by AcaPys configurable * feat(ErrorLogger): Introduction of an error logger to log unexpected error that occurred during testing * feat: Start Load Generator as Docker Container using "./manage.sh start" - to not start it via docker use "./manage.sh startWithoutLoadGenerator" * fix: .env.example * feat(Distribute load Across many Holders): Instead of having one horizontally scaled holder agent we now start many independent holder agents and the load generator takes care of distributing the load across all the holders * feat: Increase number of stand-alone holders to 10 * feat: Increase number of issuer/verifier AcaPys to 10 * feat: Add known error to the dashboard * feat: Rework manage.sh to ensure everything starts in the correct order * feat: Offer a "restart" command via the manage.sh * fix: AcaPy provisioning * fix: AcaPy provisioning * docs: Update Readme * Prometheus set-up (#43) * #32 Initial setup (macOS only) * #32 adding cadvisor support to be docker and platform agnostic * #32 Adding node-export * #32 Adding prometheus data source provisioning config * #32 Adding prometheus / node-exporter dashboard * Remove unused redis container * Increase platform agnostic-ness * fix: remove "redis" dependency * Removed unused prometheus scraper * Added Prometheus and installation guide for node-exporter Co-authored-by: PaulWen <wenzel.pgb@googlemail.com> Signed-off-by: Gerhard Dinhof g.dinhof@ibm.com * fix(manage.sh): Rebuild load-generator docker image on every start to ensure that the latest application.yaml is loaded * feat(Docker Container Metrics): Visualize the docker container metrics in Grafana that have been collected by cAdvisor * feat(Linux Localhost): Avoid having to switch between "host.docker.internal" and "172.17.0.1" depending on the host OS * feat(Postgres Metrics): Collect Postgres metrics using postgres_exporter and visualize them in Grafana * Fixed cAdvisor volume mapping for macOS #44 Signed-off-by: Gerhard Dinhof g.dinhof@ibm.com * fix: configure postgres username and password via env-variables * feat(Configure Load Tests): The load test configuration can now be done via the .env file instead of the application.yml * feat(Configure Load Tests): The load test configuration can now be done via the .env file instead of the application.yml * chore(Dashboards): Improve description of graphics * chore(Dashboards): Add a download link to each dashboard * chore(Dashboards): Update test result dashboard * fix(Exception Handling): Catch and log exceptions thrown in the context of ScheduledThreadPools * refactor(Error Logging): Rework the error logging to differentiate between errors and provide more information for easier debugging * feat(Summary Dashboard): Introduce a new dashboard which lists the most interesting graphs in one single dashboard * fix(Grafana PDF Exporter): Build grafana-pdf-exporter docker image to have the latest bug fixes and features included * chore: Specify ID for Grafana folder to ensure a static URL * fix: Improve error logging for failed HTTP requests to AcaPy * fix(Test Results Dashboard): Improve the error logging on the test results dashboard * feat(Test Results Dashboard): Log started iterations as well as pending and failed iterations * fix(Constant and Increasing Load Runners): Previously, the scheduler used to generate the load was not running more than one task at a time. This caused the load generator not to generate the desired iterations per minute. * chore: Fix wording on test result dashboard * chore: Update default Revocation Registry Size to 3000 * refactor: Combine test result and summary dashboards into one dashboard * chore: Add CPU and memory usage of all containers to the test result dashboard * chore: Adjust tooltips * fix: Increasing Load Test Runner * chore: Improve test result dashboard * chore: Improve test result dashboard * Revert "fix(Grafana PDF Exporter): Build grafana-pdf-exporter docker image to have the latest bug fixes and features included" This reverts commit cfe7f21. * chore: Improve test result dashboard * chore: Restructure the manage.sh * fix: Cancel schedulers properly once the tests are over * chore: Update test results * docs: restructured and added architecture diagram * chore: deleted unneeded readme * docs: added nginx and small cleanups * Implement Postgres Cluster as wallet DB (#49) Issue: #48 Author: Ilie Circiumaru Reviewer: @PaulWen Description: Implement 3 instances Postgres Cluster for wallet DB for the issuer-verifier. Split docker-compose on more files to enable launch of the isolated components * chore: Update Test-Results Dashboard * chore: Remove unused grafana-pdf-exporter reference * refactor: Cleanup the setup procedure * fix: Gitmodules * fix: Gitmodules * fix: Gitmodules * fix: nginx-issuer-verifier.conf * fix: manage.sh * chore: remove unused profiles from docker-compose-agents.yml * feat(Activate Postgres Cluster via .env): Switch from many start commands to just one and configure what is started via the .env * feat(Configure Number of Issuer/Verifier Agents): Depending on the number of requested issuer/verifier AcaPy instances a different nginx.conf is loaded that matches the desired number * feat(Configure Number of Holder Agents): The number of requested holder AcaPy instances can be changed via the .env * chore: restructure .env * feat(Log Revocation Registry Metadata): Track the usage of revocation registries. In particular look out if any RevReg/Index combination has been assigned to multiple credentials. * Implement declarative deployment configuration (#58) Issue: #56 Co-authored-by: Ilie Circiumaru <ici@zurich.ibm.com> Co-authored-by: PaulWen <wenzel.pgb@googlemail.com> Description: 1. Implementation of declarative deployment configuration 2. Setup metrics for Postgres Cluster * fix(Revocation Registry Logging): Do not log any revocation registry metadata if not revocation registry is used * feat(Postgres Configuration): make the "statement_timeout" configurable * fix: network handling with docker-compose * feat(Postgres configuration): Configure "idle_in_transaction_session" and "lock_timeout" via the .env * fix(Postgres configuration): "idle_in_transaction_session_timeout" * fix: Test Results Dashboard * chore: Update Test-Results Dashboard * feat(Multitenancy): Run tests with AcaPy in multitenancy mode * chore: remove unnecessary host.docker.internal * docs: Split Readme into Developer and User Readme to avoid any confusion * docs: Improve the README.md * docs: Improve the README.md * fix: add host.docker.internal reference to load-generator so that he is able to connect to the VON Network to register a public DID * chore: Update test-results.json * docs: define test cases to verify the load generator still behaves as expected * docs: Add a troubleshooting section to the README.md * Implement monitoring of the Spring Boot app (#62) Author: Ilie Circiumaru <ici@zurich.ibm.com> Issue: //issues/45 Reviewer Paul Wen * docs: Update Troubleshooting section * docs: Fix spelling * Added configuration option to enable Postgres debug logging (#60) * Added flags to enable postgres deub logging * Removing postgres debug settings from clustered setup * added comprehensive description for POSTGRES_LOG_DEBUG * alphabetical order Signed-off-by: Gerhard Dinhof g.dinhof@ibm.com * fix(Configuration): Fix the default configuration to be compatible with for the debugging case where the Load Generator is run locally outside a docker-container * fix(Configuration): Fix the default configuration to be compatible with for the debugging case where the Load Generator is run locally outside a docker-container * #63 debug acapy (#64) feat(Debug AcaPy): Running "manage.sh debug" it is now possible to attach a debugger to the issuer-verifier-acapy. * Feat 59: credential revocation (#65) feat(Full Flow): Support for credential revocation with configurable revocation batch size * Fix 66 macos compsev2 compatibility (#67) fix: Switches Postgres from port 5000 to port 5432 as MacOS Monterey already uses the post 5000 itself and extends the README to outline that Compose V2 is not supported. * feat(Debugging): Configure via the .env if the load-generator should be started when running "manage.sh debug" * feat(Debugging): Configure via the .env if the load-generator should be started when running "manage.sh debug" * feat(Debugging): Configure via the .env if the load-generator should be started when running "manage.sh debug" * fix(Debugging): Load generator needs to be started manually after the debugger has been attached to the issuer-verifier-acapy. * docs: Improve the troubleshooting section * fix(Database Logger): Fix the configuration of the database exporter to the metrics of the wallet DB * chore(Multitenancy): Be able to run load tests with 9 tenants * feat(Problem Reports): Log problem reports and show them on the dashboard * chore(README): Add list of required dependencies * feat(Dashboards): Update the dashboards and add a dashboard to evaluate AcaPy logs * fix(Aries Client Logging): Ensure that aries client errors can be parsed by Loki * feat(manage.sh): Wait for the ledger to start until the DID was registered successfully to the ledger * feat(Apple M1 Chip): Update the dockerfile to be able to build the Load Generator Docker image on an Apple M1 Mac * feat(.env): Update the default .env values * chore(OOB): Remove out-of-band (OOB) placeholder code --------- Signed-off-by: Gerhard Dinhof g.dinhof@ibm.com Co-authored-by: PaulWen <wenzel.pgb@googlemail.com> Co-authored-by: Richard Holzeis <richard.holzeis@at.ibm.com> Co-authored-by: Lukas Willburger <lukas.willburger@fim-rc.de> Co-authored-by: Paul Wenzel <34135713+PaulWen@users.noreply.github.com> Co-authored-by: iliecirciumaru <34098446+iliecirciumaru@users.noreply.github.com>

Monitor Spring Boot/Tomcat Thread Utilization #45

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions