-
Notifications
You must be signed in to change notification settings - Fork 2.8k
feat(annotations): add custom annotation prefix support for split horizon DNS #5889
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
k8s-ci-robot
merged 10 commits into
kubernetes-sigs:master
from
lexfrei:feature/custom-annotation-prefix
Nov 8, 2025
+950
−394
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
e117b77
feat(annotations): add custom annotation prefix support for split hor…
lexfrei 269d1cb
docs(advanced): fix markdown formatting in split-horizon guide
lexfrei f811331
docs(advanced): fix markdown formatting in split-horizon guide
lexfrei 15cbe3c
docs(charts): regenerate Helm chart documentation
lexfrei 3693709
test: add AnnotationPrefix field to test configs
lexfrei df25e61
test(charts): update error pattern in json-schema test
lexfrei 7118adc
refactor(annotations): remove init() for explicit initialization
lexfrei 18e8b35
docs: update changelog and mkdocs to include annotationPrefix and spl…
lexfrei 5af7d51
docs(split-horizon): fix linting
lexfrei ba2ad84
refactor(annotations): replace hardcoded annotation prefix with constant
lexfrei File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,274 @@ | ||
| # Split Horizon DNS | ||
|
|
||
| Split horizon DNS allows you to serve different DNS responses based on the client's location - internal clients receive private IPs while external clients receive public IPs. External-DNS supports split horizon DNS by running multiple instances with different annotation prefixes. | ||
|
|
||
| ## Overview | ||
|
|
||
| By default, all external-dns instances use the same annotation prefix: `external-dns.alpha.kubernetes.io/`. This means all instances process the same annotations. To enable split horizon DNS, you can configure each instance to use a different annotation prefix via the `--annotation-prefix` flag. | ||
|
|
||
| ## Use Cases | ||
|
|
||
| - **Internal/External separation**: Internal DNS points to private IPs (ClusterIP), external DNS points to public Load Balancer IPs | ||
| - **Multiple DNS providers**: Route different services to different DNS providers (e.g., internal to CoreDNS, external to Route53) | ||
| - **Geographic split**: Different DNS records for different regions | ||
|
|
||
| ## Configuration | ||
|
|
||
| ### Basic Split Horizon Setup | ||
|
|
||
| **Internal DNS Instance:** | ||
|
|
||
| ```bash | ||
| external-dns \ | ||
| --annotation-prefix=internal.company.io/ \ | ||
| --source=service \ | ||
| --source=ingress \ | ||
| --provider=aws \ | ||
| --aws-zone-type=private \ | ||
| --domain-filter=internal.company.com \ | ||
| --txt-owner-id=internal-dns | ||
| ``` | ||
|
|
||
| **External DNS Instance:** | ||
|
|
||
| ```bash | ||
| external-dns \ | ||
| --annotation-prefix=external-dns.alpha.kubernetes.io/ \ # default, can be omitted | ||
| --source=service \ | ||
| --source=ingress \ | ||
| --provider=aws \ | ||
| --aws-zone-type=public \ | ||
| --domain-filter=company.com \ | ||
| --txt-owner-id=external-dns | ||
| ``` | ||
|
|
||
| ### Service with Both Annotations | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: myapp | ||
| annotations: | ||
| # Internal DNS reads this | ||
| internal.company.io/hostname: myapp.internal.company.com | ||
| internal.company.io/ttl: "300" | ||
| internal.company.io/target: 10.0.1.50 # Private IP | ||
|
|
||
| # External DNS reads this | ||
| external-dns.alpha.kubernetes.io/hostname: myapp.company.com | ||
| external-dns.alpha.kubernetes.io/ttl: "60" | ||
| # No target = uses LoadBalancer IP automatically | ||
| spec: | ||
| type: LoadBalancer | ||
| clusterIP: 10.0.1.50 | ||
| ports: | ||
| - port: 80 | ||
| targetPort: 8080 | ||
| selector: | ||
| app: myapp | ||
| ``` | ||
|
|
||
| **Result:** | ||
|
|
||
| - **Internal DNS** (Route53 Private Zone `internal.company.com`): `myapp.internal.company.com → 10.0.1.50` | ||
| - **External DNS** (Route53 Public Zone `company.com`): `myapp.company.com → 203.0.113.10` (LoadBalancer IP) | ||
|
|
||
| ### Helm Chart Configuration | ||
|
|
||
| You can use the Helm chart to deploy multiple instances: | ||
|
|
||
| **values-internal.yaml:** | ||
|
|
||
| ```yaml | ||
| annotationPrefix: "internal.company.io/" | ||
|
|
||
| provider: | ||
| name: aws | ||
|
|
||
| aws: | ||
| zoneType: private | ||
|
|
||
| domainFilters: | ||
| - internal.company.com | ||
|
|
||
| txtOwnerId: internal-dns | ||
|
|
||
| sources: | ||
| - service | ||
| - ingress | ||
| ``` | ||
|
|
||
| **values-external.yaml:** | ||
|
|
||
| ```yaml | ||
| # annotationPrefix defaults to "external-dns.alpha.kubernetes.io/" | ||
| # can be omitted or set explicitly: | ||
| # annotationPrefix: "external-dns.alpha.kubernetes.io/" | ||
|
|
||
| provider: | ||
| name: aws | ||
|
|
||
| aws: | ||
| zoneType: public | ||
|
|
||
| domainFilters: | ||
| - company.com | ||
|
|
||
| txtOwnerId: external-dns | ||
|
|
||
| sources: | ||
| - service | ||
| - ingress | ||
| ``` | ||
|
|
||
| **Deploy:** | ||
|
|
||
| ```bash | ||
| # Internal instance | ||
| helm install external-dns-internal external-dns/external-dns \ | ||
| --namespace external-dns-internal \ | ||
| --create-namespace \ | ||
| --values values-internal.yaml | ||
|
|
||
| # External instance | ||
| helm install external-dns-external external-dns/external-dns \ | ||
| --namespace external-dns-external \ | ||
| --create-namespace \ | ||
| --values values-external.yaml | ||
| ``` | ||
|
|
||
| ## Advanced Examples | ||
|
|
||
| ### Three-Way Split (Internal / DMZ / External) | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: api | ||
| annotations: | ||
| # Internal (private network only) | ||
| internal.company.io/hostname: api.internal.company.com | ||
| internal.company.io/ttl: "300" | ||
|
|
||
| # DMZ (accessible from office network) | ||
| dmz.company.io/hostname: api.dmz.company.com | ||
| dmz.company.io/ttl: "120" | ||
|
|
||
| # External (public internet) | ||
| external-dns.alpha.kubernetes.io/hostname: api.company.com | ||
| external-dns.alpha.kubernetes.io/ttl: "60" | ||
| external-dns.alpha.kubernetes.io/cloudflare-proxied: "true" | ||
| spec: | ||
| type: LoadBalancer | ||
| # ... | ||
| ``` | ||
|
|
||
| **Deploy three instances:** | ||
|
|
||
| ```bash | ||
| # Internal | ||
| --annotation-prefix=internal.company.io/ --provider=aws --aws-zone-type=private | ||
|
|
||
| # DMZ | ||
| --annotation-prefix=dmz.company.io/ --provider=aws --aws-zone-type=private | ||
|
|
||
| # External | ||
| --annotation-prefix=external-dns.alpha.kubernetes.io/ --provider=cloudflare | ||
| ``` | ||
|
|
||
| ### Different Providers Per Instance | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: webapp | ||
| annotations: | ||
| # Route53 for AWS internal | ||
| aws.company.io/hostname: webapp.aws.company.com | ||
| aws.company.io/aws-alias: "true" | ||
|
|
||
| # Cloudflare for public | ||
| cf.company.io/hostname: webapp.company.com | ||
| cf.company.io/cloudflare-proxied: "true" | ||
| spec: | ||
| type: LoadBalancer | ||
| # ... | ||
| ``` | ||
|
|
||
| **Deploy:** | ||
|
|
||
| ```bash | ||
| # AWS instance | ||
| --annotation-prefix=aws.company.io/ --provider=aws | ||
|
|
||
| # Cloudflare instance | ||
| --annotation-prefix=cf.company.io/ --provider=cloudflare | ||
| ``` | ||
|
|
||
| ## Important Notes | ||
|
|
||
| 1. **Annotation prefix must end with `/`** - The validation will fail if the prefix doesn't end with a forward slash. | ||
| 2. **Backward compatibility** - If you don't specify `--annotation-prefix`, the default `external-dns.alpha.kubernetes.io/` is used, maintaining full backward compatibility. | ||
| 3. **All annotations use the same prefix** - When you set a custom prefix, ALL external-dns annotations (hostname, ttl, target, cloudflare-proxied, etc.) must use that prefix. | ||
| 4. **TXT ownership records** - Each instance should have a unique `--txt-owner-id` to avoid conflicts in ownership tracking. | ||
| 5. **Provider-specific annotations** - Provider-specific annotations (like `cloudflare-proxied`, `aws-alias`) also use the custom prefix: | ||
|
|
||
| ```yaml | ||
| custom.io/hostname: example.com | ||
| custom.io/cloudflare-proxied: "true" # NOT external-dns.alpha.kubernetes.io/cloudflare-proxied | ||
| ``` | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Both instances processing the same resources | ||
|
|
||
| **Problem:** Both internal and external instances are creating records for the same service. | ||
|
|
||
| **Solution:** Make sure you're using different annotation prefixes and that your services have the correct annotations: | ||
|
|
||
| ```yaml | ||
| # ✅ Correct - different prefixes | ||
| internal.company.io/hostname: internal.example.com | ||
| external-dns.alpha.kubernetes.io/hostname: example.com | ||
|
|
||
| # ❌ Wrong - same prefix | ||
| external-dns.alpha.kubernetes.io/hostname: internal.example.com | ||
| external-dns.alpha.kubernetes.io/hostname: example.com # Second one overwrites first | ||
| ``` | ||
|
|
||
| ### Validation error: "annotation-prefix must end with '/'" | ||
|
|
||
| **Problem:** The annotation prefix doesn't end with a forward slash. | ||
|
|
||
| **Solution:** Always end your custom prefix with `/`: | ||
|
|
||
| ```bash | ||
| # ✅ Correct | ||
| --annotation-prefix=custom.io/ | ||
|
|
||
| # ❌ Wrong | ||
| --annotation-prefix=custom.io | ||
| ``` | ||
|
|
||
| ### Provider-specific annotations not working | ||
|
|
||
| **Problem:** Cloudflare/AWS-specific annotations are not being applied. | ||
|
|
||
| **Solution:** Provider-specific annotations must use the same prefix as the hostname: | ||
|
|
||
| ```yaml | ||
| # If using custom prefix | ||
| custom.io/hostname: example.com | ||
| custom.io/cloudflare-proxied: "true" | ||
| custom.io/ttl: "60" | ||
| ``` | ||
|
|
||
| ## See Also | ||
|
|
||
| - [Configuration Precedence](configuration-precedence.md) - Understanding how external-dns processes configuration | ||
| - [FAQ](../faq.md) - Frequently asked questions | ||
| - [AWS Provider](../tutorials/aws.md) - AWS Route53 configuration | ||
| - [Cloudflare Provider](../tutorials/cloudflare.md) - Cloudflare configuration |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.