Adding as400 (#709)

* Updating dependencies

* Add collectors in agent manager

* Updating agent grpc proto, classes and backend interaction with agent manager.

* Add collectors to log-auth-proxy

* Update gRPC services

* Added AS_400 changeset

* Added AS_400 config keys

* Added AS_400 config keys

* Adding collector services and resources. Updating dependencies.

* Updating CollectorOpsService and UtmCollectorResource to handle the upsert configurations and error handling.

* Added AS_400 changeset

* Added AS_400 module

* Revert "Merge branch 'feature/10.5/as400' of ssh://github.com/utmstack/UTMStack into feature/10.5/as400"

This reverts commit ada4575, reversing
changes made to 2ebed07.

* Updating all changes because of an error during branch update from main.

* Updating master.xml missing.

* Updating comments and some handling error messages in UtmCollectorResource.

* Updating proto file and protobuf classes.

* Creating Agent Manager Image

* Update UTMStack Version

* Clean logs

* Updating filter for as 400

* Refactor AS_400 module

* Implemented a unique hostname validator by collector

* Refactor save collector config

* Refactor save collector config

* Added collector sources

* Added collector sources

* Added UX AS-400

* Added UX AS-400

* Added collector-group ux

* Added collector-group ux

* Added collector  group's sources

* Added collector-group ux

* Added collector-group ux

* Fixed expire token behavior

* Refactor add tenant/collector functionality

* Fixed the search by unique group name

* Fixed the search by group in collector view

* Fixed error handling in the interceptor for status code 502

* Updated unique restriction for utm_assets_group

* Fixed Some conditions doesn't need to set value when setting fields filtering for tags rules (#690)

* Feature as-400

* Disabled alerts when datasource is down

* Refactor AS400 module

* Refactor As400 module

* Refactor AS400 module

* Refactor AS400 module

* Delete collector and agent in cache after delete

* Fixing memory problems

* add aws datasource

* add sophos datasource

* Change aws and sophos images to github

* Change environment variables for aws and sophos

* skip error logging on backend not started

* Fixed conflicts after merge with main

* Updated winlogbeat uninstall command

* Updated Bitdefender guide

* initialize timegroup map

* typo

* Updated AS400 guide

* Fix configurations names in aws and sophos integrations

* Updated AS400 guide

* Change version

* Refactor synchronize collector functionality

* Please remember the following text:

Fixed validation for collector configuration

* Please remember the following text:

Fixed variable name for Linux command installation.

* Fixed variable name for Linux command installation.

* time as epoch

* better loggin on aws datasource processor

* Refactor list collector endpoint

* Fixed collector bugs.

* Fixed collector bugs.

* Fix logging in new integrations

* Update dependabot dependencies

* reduce system reserved memory and give it to elastic

* Added delete collector endpoint

* Added delete collector functionality.

* Added validation to save collector configuration.

* Trying to fix memory leak in agent manager

* make type declarations importable from outside for testing purposes

* fix function not defined

* reduce system reserved memory and increase agent-manager minimum

* Change version 10.5.0

* Disable connection key validation to prevent unrecognized instances

* Fixed uninstall Windows command

* Fixed save collector configuration param

* Add profile to agent manager

* Updating pprof ip

* Close parked goroutine

* Processing pending collector configs

* Prevent multiple connections from same client in agent manager

* Fixed guide uninstall command

* Fixed install uninstall command

* Lowercase for log type in log auth

* Added validation to delete collector for valid configuration only

* Updated AIX and FORTIWEB functions

* Added validation to delete collector for valid configuration only

* Fixed collector name on detail view

* Fixed navigation to collector view

* Fixed add group from collector/assets groups view

* Updated IBM 400 changeset

* Fixed add group from collector/assets groups view

* Added ibm-as-400 filter

* Added ibm-as-400 filter

* Added ibm-as-400 filter

---------

Co-authored-by: Yorjander Hernandez Vergara <[email protected]>
Co-authored-by: Manuel Abascal <[email protected]>
Co-authored-by: Jose Sanchez <[email protected]>

Author: 4 people

.github/workflows/principal-multi-env.yml

@@ -39,6 +39,7 @@ jobs:
         with:
           filters: |
             agent-manager: agent-manager/**
+            aws: aws/**
             backend:
               - 'backend/**'
               - 'version.yml'
@@ -48,6 +49,7 @@ jobs:
             mutate: mutate/**
             office365: office365/**
             log-auth-proxy: log-auth-proxy/**
+            sophos: sophos/**
             user-auditor: user-auditor/**
             web-pdf: web-pdf/**
   
@@ -72,7 +74,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix: 
-        service: ['agent-manager', 'backend', 'correlation', 'frontend', 'bitdefender', 'mutate', 'office365', 'log-auth-proxy', 'user-auditor', 'web-pdf'] 
+        service: ['agent-manager', 'aws', 'backend', 'correlation', 'frontend', 'bitdefender', 'mutate', 'office365', 'log-auth-proxy', 'sophos', 'user-auditor', 'web-pdf'] 
     uses: ./.github/workflows/used-runner.yml
     with:
       microservice: ${{ matrix.service }}

.github/workflows/used-runner.yml

@@ -22,7 +22,7 @@ jobs:
         id: get_tech
         run: |
           folder_changed="${{inputs.microservice}}"
-          if [[ "$folder_changed" == "agent-manager" || "$folder_changed" == "correlation" || "$folder_changed" == "bitdefender" || "$folder_changed" == "office365" || "$folder_changed" == "log-auth-proxy" ]]; then
+          if [[ "$folder_changed" == "agent-manager" || "$folder_changed" == "aws" || "$folder_changed" == "correlation" || "$folder_changed" == "bitdefender" || "$folder_changed" == "office365" || "$folder_changed" == "sophos" || "$folder_changed" == "log-auth-proxy" ]]; then
             tech="golang"
           elif [[ "$folder_changed" == "backend" ]]; then
             tech="java-11"

agent-manager/.gitignore

@@ -16,3 +16,5 @@
 # vendor/
 .idea/
 .fleet/
+
+TODO.md