Skip to content

Implement basic file fetcher #3577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 3, 2025
Merged

Implement basic file fetcher #3577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
af26ee8
Implement basic file fetcher
ciarams87 Jul 1, 2025
af020c5
Review feedback
ciarams87 Jul 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions docs/proposals/nap-waf.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,9 @@ sha256sum compiled-policy.tgz > compiled-policy.tgz.sha256
aws s3 cp compiled-policy.tgz s3://company-policies/prod-policy.tgz
aws s3 cp compiled-policy.tgz.sha256 s3://company-policies/prod-policy.tgz.sha256

# Note: In WAFPolicy, reference S3 objects using HTTPS URLs:
# fileLocation: "https://company-policies.s3.amazonaws.com/prod-policy.tgz"

# No Kubernetes resource changes needed - NGF automatically detects the update
echo "Policy updated. NGF will detect changes within polling interval."
```
Expand Down Expand Up @@ -427,6 +430,8 @@ spec:

### WAFPolicy Custom Resource with Policy Attachment

**Note**: When referencing S3 objects, use HTTPS URLs (e.g., `https://bucket.s3.amazonaws.com/path/file.tgz`) rather than S3 protocol URLs (`s3://bucket/path/file.tgz`).

```yaml
apiVersion: gateway.nginx.org/v1alpha1
kind: WAFPolicy
Expand All @@ -442,7 +447,7 @@ spec:
namespace: applications

policySource:
fileLocation: "s3://ngf-waf-policies/production/gateway-policy-v1.2.3.tgz"
fileLocation: "https://ngf-waf-policies.s3.amazonaws.com/production/gateway-policy-v1.2.3.tgz"
authSecret:
name: "policy-store-credentials"
validation:
Expand All @@ -456,7 +461,7 @@ spec:
interval: "5m" # Check every 5 minutes
# Optional: explicit checksum location
# If not specified, defaults to <fileLocation>.sha256
checksumLocation: "s3://ngf-waf-policies/production/gateway-policy-v1.2.3.tgz"
checksumLocation: "https://ngf-waf-policies.s3.amazonaws.com/production/gateway-policy-v1.2.3.tgz.sha256"

# Retry configuration for policy fetch failures
retryPolicy:
Expand All @@ -480,7 +485,7 @@ spec:
# Custom logging profile bundle (similar to policy bundle)
# logProfile and logProfileBundle are mutually exclusive per security log configuration entry
logProfileBundle:
fileLocation: "s3://ngf-waf-policies/logging/custom-log-profile.tgz"
fileLocation: "https://ngf-waf-policies.s3.amazonaws.com/logging/custom-log-profile.tgz"
authSecret:
name: "policy-store-credentials"
validation:
Expand Down Expand Up @@ -527,7 +532,7 @@ spec:

# Stricter policy for admin endpoints
policySource:
fileLocation: "s3://ngf-waf-policies/production/admin-strict-policy-v1.0.0.tgz"
fileLocation: "https://ngf-waf-policies.s3.amazonaws.com/production/admin-strict-policy-v1.0.0.tgz"
authSecret:
name: "policy-store-credentials"
polling:
Expand Down Expand Up @@ -664,7 +669,7 @@ metadata:
# NGF service account in nginx-gateway namespace provides IRSA authentication
spec:
policySource:
fileLocation: "s3://company-waf-policies/policy.tgz"
fileLocation: "https://company-waf-policies.s3.amazonaws.com/policy.tgz"
# No authSecret needed - uses IRSA automatically
```

Expand Down Expand Up @@ -1009,7 +1014,7 @@ spec:
namespace: applications

policySource:
fileLocation: "s3://company-waf-policies/production/base-policy.tgz"
fileLocation: "https://company-waf-policies.s3.amazonaws.com/production/base-policy.tgz"
# Secret referenced for fallback - NGF will use IRSA if available, secret if not
authSecret:
name: "policy-store-credentials"
Expand All @@ -1020,7 +1025,7 @@ spec:
interval: "5m"
# Optional explicit checksum location
# If not specified, defaults to base-policy.tgz.sha256
checksumLocation: "s3://company-waf-policies/production/base-policy.tgz.sha256"
checksumLocation: "https://company-waf-policies.s3.amazonaws.com/production/base-policy.tgz.sha256"

securityLogs:
- name: "gateway-logging"
Expand All @@ -1044,7 +1049,7 @@ spec:
namespace: applications

policySource:
fileLocation: "s3://company-waf-policies/production/admin-strict-policy.tgz"
fileLocation: "https://company-waf-policies.s3.amazonaws.com/production/admin-strict-policy.tgz"
polling:
enabled: true

Expand Down
23 changes: 23 additions & 0 deletions internal/framework/fetch/errors.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
package fetch

import "fmt"

// ChecksumMismatchError represents an error when the calculated checksum doesn't match the expected checksum.
// This type of error should not trigger retries as it indicates data corruption or tampering.
type ChecksumMismatchError struct {
Expected string
Actual string
}

func (e *ChecksumMismatchError) Error() string {
return fmt.Sprintf("checksum mismatch: expected %s, got %s", e.Expected, e.Actual)
}

// HTTPStatusError represents an HTTP status code error for retry logic.
type HTTPStatusError struct {
StatusCode int
}

func (e *HTTPStatusError) Error() string {
return fmt.Sprintf("unexpected status code: %d", e.StatusCode)
}
Loading