DGS-22734 Add Accept-Version header

[rayokota]

What

• Add Accept-Version header
• Minor fix to AWS driver to account for web identity token file
• User newer subject-based DEK creation API
• Properly handle Avro bytes type

Checklist

• Contains customer facing changes? Including API/behavior changes
• Did you add sufficient unit test and/or integration test coverage for this PR?
  • If not, please explain why it is not required

References

JIRA:

Test & Review

Open questions / Follow-ups

[confluent-cla-assistant]

🎉 All Contributor License Agreements have been signed. Ready to merge.
_{Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.}

[Copilot]

Pull Request Overview

This PR implements multiple improvements to the Confluent Kafka Python client including API updates, protocol enhancements, and bug fixes. The changes focus on adding versioning support, improving AWS KMS integration, enhancing DEK (Data Encryption Key) registry functionality, and fixing Avro serialization for bytes type.

• Add Accept-Version header with value "8.0" to all Schema Registry requests
• Update DEK registry to use subject-based API with fallback to legacy API
• Fix AWS KMS driver to properly handle web identity token authentication

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/confluent_kafka/schema_registry/_sync/schema_registry_client.py	Add Accept-Version header to HTTP requests
src/confluent_kafka/schema_registry/_async/schema_registry_client.py	Add Accept-Version header to HTTP requests
src/confluent_kafka/schema_registry/rules/encryption/dek_registry/dek_registry_client.py	Implement subject-based DEK creation API with fallback
src/confluent_kafka/schema_registry/rules/encryption/awskms/aws_driver.py	Fix AWS web identity token handling
src/confluent_kafka/schema_registry/_sync/avro.py	Fix Avro bytes type serialization/deserialization
src/confluent_kafka/schema_registry/_async/avro.py	Fix Avro bytes type serialization/deserialization
tests/schema_registry/_sync/test_avro_serdes.py	Add test for Avro bytes serialization
tests/schema_registry/_async/test_avro_serdes.py	Add test for Avro bytes serialization

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sonarqube-confluent]

Quality Gate failed

Failed conditions
4 New issues
53.2% Coverage on New Code (required ≥ 80%)
38.4% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE