Skip to content

Comments

refactor: introduce generic Resource[C] pattern for GCP resources#983

Merged
james00012 merged 1 commit intomasterfrom
feat/generic-resource-gcp
Dec 31, 2025
Merged

refactor: introduce generic Resource[C] pattern for GCP resources#983
james00012 merged 1 commit intomasterfrom
feat/generic-resource-gcp

Conversation

@james00012
Copy link
Contributor

@james00012 james00012 commented Dec 29, 2025 •
edited
Loading

Summary

This PR introduces a generic Resource[C] pattern to standardize how cloud resources are defined and managed. Starting with GCP resources as a proof-of-concept, this pattern will eventually be applied to all 100+ AWS resources.

Why this refactor?

Current state: Each resource type (EC2, S3, GCS Buckets, etc.) has its own struct with duplicated boilerplate:

  • identifiers []string field
  • nukables map[string]error field
  • ResourceIdentifiers(), IsNukable(), MaxBatchSize() methods
  • Similar Init/List/Nuke patterns repeated 100+ times

New approach: A single generic Resource[C] struct handles all common behavior. Resource implementations only need to provide:

  • Resource name and batch size
  • Client initialization function
  • Lister function (discover resources)
  • Nuker function (delete resources)

What changed

New resource/ package:

  • resource.go - Core Resource[C] generic struct with shared behavior
  • batch_deleter.go - Reusable deletion strategies (SimpleBatchDeleter, SequentialDeleter, MultiStepDeleter)
  • resource_test.go - Unit tests

GCP migration:

  • gcp/resources/adapter.go - Bridges Resource[C] to GcpResource interface
  • gcp/resources/types.go - GcpResource interface and container types
  • gcp/resources/gcs_bucket.go - Rewritten using generic pattern
  • Deleted gcs_bucket_types.go and base_resource.go (no longer needed)

Benefits

  1. Less boilerplate: GCS Bucket implementation reduced from ~150 lines across 3 files to ~80 lines in 1 file
  2. Consistent behavior: All resources share the same tested implementation for common operations
  3. Easier to add resources: New resources only implement resource-specific logic
  4. Type safety: Go generics ensure client types are correct at compile time

Backward compatibility

  • CLI commands unchanged
  • Config file format unchanged
  • gcp.GcpResource interface preserved (via type aliases)
  • All existing tests pass

Next steps

After this PR is validated, apply the same pattern to AWS resources incrementally, starting with simpler resources like AccessAnalyzer.

Test plan

  • go build ./... compiles successfully
  • go test ./resource/... passes
  • go test ./gcp/... passes
  • Manual test: cloud-nuke gcp --dry-run with real GCP project

@james00012 james00012 force-pushed the feat/generic-resource-gcp branch 2 times, most recently from 168d468 to 1c81d9d Compare December 29, 2025 20:14
@james00012 james00012 changed the title feat: introduce generic resource pattern for GCP refactor: introduce generic Resource[C] pattern for GCP resources Dec 29, 2025
@james00012 james00012 force-pushed the feat/generic-resource-gcp branch from d2ccd14 to e8a7a2a Compare December 29, 2025 21:48
@james00012 james00012 marked this pull request as ready for review December 29, 2025 21:57
@james00012 james00012 requested a review from denis256 as a code owner December 29, 2025 21:57
denis256
denis256 reviewed Dec 30, 2025
View reviewed changes
gcp/gcp.go Outdated

// Prepare context for the resource
if err := resourceType.PrepareContext(context.Background(), resourceConfig); err != nil {
if err := resourceType.PrepareContext(context.Background()); err != nil {
Copy link
Member

@denis256 denis256 Dec 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect that resource timeouts will be lost wit this since result from GetAndSetResourceConfig() is lost

Copy link
Contributor Author

@james00012 james00012 Dec 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great find. Thanks for the thorought review. It should be ready for another review.

@james00012 james00012 force-pushed the feat/generic-resource-gcp branch 5 times, most recently from 8a6b2a8 to 69252b0 Compare December 30, 2025 18:09
@james00012
feat: introduce generic Resource[C] pattern for GCP
21806af 
This PR introduces a generic Resource[C] pattern to reduce boilerplate
when implementing cloud resource types. Starting with GCP as a proof
of concept before migrating AWS resources.

## Changes

### New `resource/` package
- `Resource[C]` - Generic struct for all nukeable resources
- `Scope` - Cloud provider-specific scope (region/projectID)
- `SimpleBatchDeleter` - Concurrent deletion with semaphore
- `SequentialDeleter` - Sequential deletion for rate-limited APIs
- `MultiStepDeleter` - Multi-step deletion (e.g., empty then delete)

### GCP changes
- `GcpResourceAdapter[C]` - Minimal adapter for GcpResource interface
- `GcpResource.Nuke(ctx, identifiers)` - Context passed directly
- `NewGCSBuckets()` - Constructor using generic pattern
- Extracted `nukeResource()` helper for proper defer scope
- Added `IsNukable` filtering before nuking
- Uses `util.Split` instead of duplicate `splitIntoBatches`

### Improvements
- `report.Record` in batch deleters for consistent reporting
- `errors.Is()` and `%w` for proper error handling
- Type aliases in `gcp/types.go` to avoid import cycles
- Comprehensive tests in `resource/resource_test.go`
@james00012 james00012 force-pushed the feat/generic-resource-gcp branch from 68000b4 to 21806af Compare December 30, 2025 19:50
@james00012 james00012 merged commit 88f04bd into master Dec 31, 2025
11 checks passed
@james00012 james00012 deleted the feat/generic-resource-gcp branch December 31, 2025 00:51
james00012 added a commit that referenced this pull request Dec 31, 2025
@james00012
feat: introduce generic Resource[C] pattern for AWS (first batch)
eac9250 
Migrates AWS resources to use the same generic Resource[C] pattern
introduced for GCP in #983. This is the first batch of 10 resources.

Changes:
- Add AwsResourceAdapter[C] to wrap generic resources for AWS interface
- Update AwsResource.Nuke() signature to accept context.Context
- Migrate 10 simple resources to generic pattern:
  - AccessAnalyzer
  - CloudWatchDashboards
  - CloudWatchLogGroups
  - EC2KeyPairs
  - EC2PlacementGroups
  - EventBridgeSchedule
  - KinesisFirehose
  - KinesisStreams
  - LaunchConfigs
  - SesEmailTemplates

Benefits:
- Consolidates _types.go + .go files into single .go file
- Uses factory functions (NewXxx()) instead of struct literals
- Context passed directly to Nuke() instead of stored in struct
- Consistent pattern between AWS and GCP resources

Remaining 111 AWS resources can be migrated incrementally.
@james00012 james00012 mentioned this pull request Dec 31, 2025
Merged
3 tasks
james00012 added a commit that referenced this pull request Dec 31, 2025
@james00012
feat: introduce generic Resource[C] pattern for AWS (first batch)
6087800 
Migrates AWS resources to use the same generic Resource[C] pattern
introduced for GCP in #983. This is the first batch of 10 resources.

Changes:
- Add AwsResourceAdapter[C] to wrap generic resources for AWS interface
- Update AwsResource.Nuke() signature to accept context.Context
- Migrate 10 simple resources to generic pattern:
  - AccessAnalyzer
  - CloudWatchDashboards
  - CloudWatchLogGroups
  - EC2KeyPairs
  - EC2PlacementGroups
  - EventBridgeSchedule
  - KinesisFirehose
  - KinesisStreams
  - LaunchConfigs
  - SesEmailTemplates

Benefits:
- Consolidates _types.go + .go files into single .go file
- Uses factory functions (NewXxx()) instead of struct literals
- Context passed directly to Nuke() instead of stored in struct
- Consistent pattern between AWS and GCP resources

Remaining 111 AWS resources can be migrated incrementally.
james00012 added a commit that referenced this pull request Dec 31, 2025
@james00012
feat: introduce generic Resource[C] pattern for AWS (first batch) (#984)
c7a01b5 
* feat: introduce generic Resource[C] pattern for AWS (first batch)

Migrates AWS resources to use the same generic Resource[C] pattern
introduced for GCP in #983. This is the first batch of 10 resources.

Changes:
- Add AwsResourceAdapter[C] to wrap generic resources for AWS interface
- Update AwsResource.Nuke() signature to accept context.Context
- Migrate 10 simple resources to generic pattern:
  - AccessAnalyzer
  - CloudWatchDashboards
  - CloudWatchLogGroups
  - EC2KeyPairs
  - EC2PlacementGroups
  - EventBridgeSchedule
  - KinesisFirehose
  - KinesisStreams
  - LaunchConfigs
  - SesEmailTemplates

Benefits:
- Consolidates _types.go + .go files into single .go file
- Uses factory functions (NewXxx()) instead of struct literals
- Context passed directly to Nuke() instead of stored in struct
- Consistent pattern between AWS and GCP resources

Remaining 111 AWS resources can be migrated incrementally.

* refactor: improve generic Resource[C] pattern for testability and correctness

Changes:
- Use interface types instead of concrete clients for mockability
- Add BulkDeleter helper for APIs with batch delete support
- Fix missing pagination in KinesisFirehose, LaunchConfigs, SesEmailTemplates
- Fix logging levels: errors use Errorf, deletion start uses Infof
- Add unit tests for list, filter, and delete functions

* refactor: remove duplicate delete logs from individual functions

The batch_deleter already logs "[OK] Deleted X" for each item,
so individual delete functions don't need to log again.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@denis256 denis256 denis256 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@james00012 @denis256