Skip to content

Suseobs store #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
Draft

Suseobs store #16

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
e63fcef
feat(store): suse obs store
jvanz Mar 5, 2025
ef2e50e
feat(suse obs): send request to SUSE Obs
jvanz Mar 6, 2025
0b124a5
Allow define different type of store and add cli flag to enable SUSE obs
jvanz Mar 6, 2025
f20be44
interval configuration and cluster resource urn
jvanz Mar 7, 2025
0a1856b
add a documentation comment
jvanz Mar 7, 2025
691971c
chore(dep): update go.mod
jvanz Mar 10, 2025
458c300
chore(dep): update go version
jvanz Mar 10, 2025
7873c8a
feat: SUSE Obs stackpack
jvanz Mar 12, 2025
e3d84e0
fix typo
jvanz Mar 12, 2025
cb75f7f
clean privisioner
jvanz Mar 12, 2025
8b0c563
wip
jvanz Mar 13, 2025
0788f40
update .gitignore
jvanz Mar 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0
with:
go-version: "1.23"
go-version: "1.24"
- run: make unit-tests
- name: Upload unit-tests coverage to Codecov
uses: codecov/codecov-action@0565863a31f2c772f9f0395002a31e3f06189574 # v5.4.0
Expand All @@ -35,7 +35,7 @@ jobs:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0
with:
go-version: "1.23"
go-version: "1.24"
- name: golangci-lint
uses: golangci/golangci-lint-action@2226d7cb06a077cd73e56eedd38eecad18e5d837 # v6.5.0
with:
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,5 @@ testbin/*
*~

CRDS.tar.gz

kubewarden-stackpack/kubewarden-stackpack.sts
39 changes: 33 additions & 6 deletions cmd/root.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package cmd
import (
"context"
"fmt"
"time"

"github.com/google/uuid"
"github.com/kubewarden/audit-scanner/internal/k8s"
Expand Down Expand Up @@ -30,13 +31,24 @@ const (
//nolint:gocognit,funlen // This function is the CLI entrypoint and it's expected to be long.
func NewRootCommand() *cobra.Command {
var (
level logconfig.Level // log level.
outputScan bool // print result of scan as JSON to stdout.
skippedNs []string // list of namespaces to be skipped from scan.
insecureSSL bool // skip SSL cert validation when connecting to PolicyServers endpoints.
disableStore bool // disable storing the results in the k8s cluster.
level logconfig.Level // log level.
outputScan bool // print result of scan as JSON to stdout.
skippedNs []string // list of namespaces to be skipped from scan.
insecureSSL bool // skip SSL cert validation when connecting to PolicyServers endpoints.
disableStore bool // disable storing the results in the k8s cluster.
suseObsURL string // URL to the SUSE OBS API.
suseObsApiKey string // API key to authenticate with the SUSE OBS API.
suseObsUrn string // API key to authenticate with the SUSE OBS API.
suseObsCluster string // API key to authenticate with the SUSE OBS API.
suseObsRepeatInterval time.Duration
suseObsExpireInterval time.Duration
)

defaultInterval, err := time.ParseDuration("30m")
if err != nil {
log.Logger.Err(err).Msg("cannot parse default suseob interval value ")
}

// rootCmd represents the base command when called without any subcommands.
rootCmd := &cobra.Command{
Use: "audit-scanner",
Expand Down Expand Up @@ -112,7 +124,14 @@ There will be a ClusterPolicyReport with results for cluster-wide resources.`,
if err != nil {
return err
}
policyReportStore := report.NewPolicyReportStore(client)
var policyReportStore report.ReportStore
if len(suseObsURL) > 0 && len(suseObsApiKey) > 0 && len(suseObsUrn) > 0 && len(suseObsCluster) > 0 {
log.Debug().Msg("Using SUSE Observability as report store")
policyReportStore = report.NewSuseObsStore(suseObsApiKey, suseObsURL, suseObsUrn, suseObsCluster, suseObsRepeatInterval, suseObsExpireInterval)
} else {
log.Debug().Msg("Using Kubernetes as report store")
policyReportStore = report.NewPolicyReportStore(client)
}
Comment on lines +127 to +134
Copy link
Owner Author

@jvanz jvanz Mar 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is very ugly. And it needs to be refactored.

What do you think about having multiple stores? For example, the audit scanner could create policy reports and send the data to SUSE Obs.


scannerConfig := scanner.Config{
PoliciesClient: policiesClient,
Expand Down Expand Up @@ -162,6 +181,12 @@ There will be a ClusterPolicyReport with results for cluster-wide resources.`,
rootCmd.Flags().IntP("parallel-resources", "", defaultParallelResources, "number of resources to scan in parallel")
rootCmd.Flags().IntP("parallel-policies", "", defaultParallelPolicies, "number of policies to evaluate for a given resource in parallel")
rootCmd.Flags().IntP("page-size", "", defaultPageSize, "number of resources to fetch from the Kubernetes API server when paginating")
rootCmd.Flags().StringVar(&suseObsURL, "suseobs-url", "", "URL to the SUSE OBS API")
rootCmd.Flags().StringVar(&suseObsApiKey, "suseobs-apikey", "", "API key to authenticate with the SUSE OBS API")
rootCmd.Flags().StringVar(&suseObsUrn, "suseobs-urn", "", "SUSE Observability health check stream urn")
rootCmd.Flags().StringVar(&suseObsCluster, "suseobs-cluster", "", "SUSE Observability cluster name where audit scanner is running")
rootCmd.Flags().DurationVar(&suseObsRepeatInterval, "suseobs-repeat-interval", defaultInterval, "The frequency with which audit scanner will send health data to SUSE Observability. Max allowed value is 1800 (30 minutes)")
rootCmd.Flags().DurationVar(&suseObsExpireInterval, "suseobs-expire-interval", defaultInterval, "The time to wait after the last update before an audit scanner check is deleted by SUSE Observability if the check isn't observed again")
Comment on lines +184 to +189
Copy link
Owner Author

@jvanz jvanz Mar 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we decide to allow multiple store types. I think we should move the store configuration to a file.


return rootCmd
}
Expand All @@ -181,6 +206,8 @@ func startScanner(namespace string, clusterWide bool, scanner *scanner.Scanner)

runUID := uuid.New().String()
ctx := context.Background()
scanner.BeforeScan(ctx)
defer scanner.AfterScan(ctx)
Comment on lines +209 to +210
Copy link
Owner Author

@jvanz jvanz Mar 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the consistency mode used to send the health checks to SUSE Observabiliry (REPEAT_SNAPSHOT), it necessary to send a payload to start the snapshot and another to close it. All the request between these two request, will be the health checks send by the audit scanner.

I've add these functions here just to avoid any the work to handle the concurrency nature of the audit scanner and the need to add some control to avoid multiple start/stop snapshot payloads. If we send more then one of this payload, the health checks will not work as expected.

if clusterWide {
// only scan clusterwide
return scanner.ScanClusterWideResources(ctx, runUID)
Expand Down
3 changes: 2 additions & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/kubewarden/audit-scanner

go 1.23.0
go 1.24.0

require (
github.com/google/uuid v1.6.0
Expand Down Expand Up @@ -58,6 +58,7 @@ require (
github.com/prometheus/procfs v0.15.1 // indirect
github.com/spf13/pflag v1.0.6 // indirect
github.com/stoewer/go-strcase v1.3.0 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/x448/float16 v0.8.4 // indirect
go.opentelemetry.io/otel v1.34.0 // indirect
go.opentelemetry.io/otel/trace v1.34.0 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -423,6 +423,8 @@ github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+
github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
Expand Down
18 changes: 18 additions & 0 deletions internal/report/store.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,15 @@ import (
wgpolicy "sigs.k8s.io/wg-policy-prototypes/policy-report/pkg/api/wgpolicyk8s.io/v1alpha2"
)

type ReportStore interface {
BeforeScanning(ctx context.Context) error
AfterScanning(ctx context.Context) error
CreateOrPatchPolicyReport(ctx context.Context, policyReport *wgpolicy.PolicyReport) error
DeleteOldPolicyReports(ctx context.Context, scanRunID, namespace string) error
CreateOrPatchClusterPolicyReport(ctx context.Context, clusterPolicyReport *wgpolicy.ClusterPolicyReport) error
DeleteOldClusterPolicyReports(ctx context.Context, scanRunID string) error
}

Comment on lines +17 to +25
Copy link
Owner Author

@jvanz jvanz Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactor this for something more meaningful. As the SUSE Obs is not actually storing the reports...

// PolicyReportStore is a store for PolicyReport and ClusterPolicyReport.
type PolicyReportStore struct {
// client is a controller-runtime client that knows about PolicyReport and ClusterPolicyReport CRDs
Expand All @@ -27,6 +36,15 @@ func NewPolicyReportStore(client client.Client) *PolicyReportStore {
}
}

func (s *PolicyReportStore) BeforeScanning(ctx context.Context) error {
return nil

}

func (s *PolicyReportStore) AfterScanning(ctx context.Context) error {
return nil
}

// CreateOrPatchPolicyReport creates or patches a PolicyReport.
func (s *PolicyReportStore) CreateOrPatchPolicyReport(ctx context.Context, policyReport *wgpolicy.PolicyReport) error {
oldPolicyReport := &wgpolicy.PolicyReport{ObjectMeta: metav1.ObjectMeta{
Expand Down
Loading
Loading