Skip to content

Commit d69e310

Browse files
authored
[KIP-430] Added ACL Operations to Describe Topics, Cluster and ConsumerGroup APIs (#1635)
[KIP-430] Added ACL Operations to Describe Topics, Cluster and ConsumerGroup APIs
1 parent 6673afb commit d69e310

File tree

18 files changed

+1406
-49
lines changed

18 files changed

+1406
-49
lines changed

CHANGELOG.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,20 @@
11
# Confluent's Python client for Apache Kafka
22

3+
## v2.3.0
4+
5+
v2.3.0 is a feature release with the following features, fixes and enhancements:
6+
7+
* Add support for AdminAPI `DescribeCluster()` and `DescribeTopics()`. (@jainruchir, #1635)
8+
* [KIP-430](https://cwiki.apache.org/confluence/display/KAFKA/KIP-430+-+Return+Authorized+Operations+in+Describe+Responses):
9+
Return authorized operations in Describe Responses. (@jainruchir, #1635)
10+
* Add `Rack` to the `Node` type, so AdminAPI calls can expose racks for brokers
11+
(currently, all Describe Responses) (#1635, @jainruchir).
12+
13+
confluent-kafka-python is based on librdkafka v2.3.0, see the
14+
[librdkafka release notes](https://github.com/confluentinc/librdkafka/releases/tag/v2.3.0)
15+
for a complete list of changes, enhancements, fixes and upgrade considerations.
16+
17+
318
## v2.2.0
419

520
v2.2.0 is a feature release with the following features, fixes and enhancements:

docs/index.rst

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,11 @@ Supporting classes
2525
- :ref:`Message <pythonclient_message>`
2626
- :ref:`TopicPartition <pythonclient_topicpartition>`
2727
- :ref:`ThrottleEvent <pythonclient_throttleevent>`
28+
- :ref:`TopicCollection <pythonclient_topic_collection>`
29+
- :ref:`TopicPartitionInfo <pythonclient_topic_partition_info>`
30+
- :ref:`Node <pythonclient_node>`
31+
- :ref:`ConsumerGroupTopicPartitions <pythonclient_consumer_group_topic_partition>`
32+
- :ref:`ConsumerGroupState <pythonclient_consumer_group_state>`
2833

2934
- Errors:
3035
- :ref:`KafkaError <pythonclient_kafkaerror>`
@@ -55,6 +60,19 @@ Supporting classes
5560
- :ref:`UserScramCredentialAlteration <pythonclient_user_scram_credential_alteration>`
5661
- :ref:`UserScramCredentialUpsertion <pythonclient_user_scram_credential_upsertion>`
5762
- :ref:`UserScramCredentialDeletion <pythonclient_user_scram_credential_deletion>`
63+
- :ref:`TopicDescription <pythonclient_topic_description>`
64+
- :ref:`DescribeClusterResult <pythonclient_describe_cluster_result>`
65+
- :ref:`BrokerMetadata <pythonclient_broker_metadata>`
66+
- :ref:`ClusterMetadata <pythonclient_cluster_metadata>`
67+
- :ref:`GroupMember <pythonclient_group_member>`
68+
- :ref:`GroupMetadata <pythonclient_group_metadata>`
69+
- :ref:`PartitionMetadata <pythonclient_partition_metadata>`
70+
- :ref:`TopicMetadata <pythonclient_topic_metadata>`
71+
- :ref:`ConsumerGroupListing <pythonclient_consumer_group_listing>`
72+
- :ref:`ListConsumerGroupsResult <pythonclient_list_consumer_group_result>`
73+
- :ref:`MemberAssignment <pythonclient_member_assignment>`
74+
- :ref:`MemberDescription <pythonclient_member_description>`
75+
- :ref:`ConsumerGroupDescription <pythonclient_consumer_group_description>`
5876

5977
Experimental
6078
These classes are experimental and are likely to be removed, or subject to incompatible
@@ -84,6 +102,7 @@ AdminClient
84102

85103
.. automodule:: confluent_kafka.admin
86104
:members:
105+
:noindex:
87106

88107
.. _pyclient_admin_newtopic:
89108

@@ -247,6 +266,123 @@ UserScramCredentialDeletion
247266
.. autoclass:: confluent_kafka.admin.UserScramCredentialDeletion
248267
:members:
249268

269+
.. _pythonclient_topic_description:
270+
271+
****************
272+
TopicDescription
273+
****************
274+
275+
.. autoclass:: confluent_kafka.admin.TopicDescription
276+
:members:
277+
278+
.. _pythonclient_describe_cluster_result:
279+
280+
*********************
281+
DescribeClusterResult
282+
*********************
283+
284+
.. autoclass:: confluent_kafka.admin.DescribeClusterResult
285+
:members:
286+
287+
.. _pythonclient_broker_metadata:
288+
289+
**************
290+
BrokerMetadata
291+
**************
292+
293+
.. autoclass:: confluent_kafka.admin.BrokerMetadata
294+
:members:
295+
296+
.. _pythonclient_cluster_metadata:
297+
298+
***************
299+
ClusterMetadata
300+
***************
301+
302+
.. autoclass:: confluent_kafka.admin.ClusterMetadata
303+
:members:
304+
305+
.. _pythonclient_group_member:
306+
307+
***********
308+
GroupMember
309+
***********
310+
311+
.. autoclass:: confluent_kafka.admin.GroupMember
312+
:members:
313+
314+
.. _pythonclient_group_metadata:
315+
316+
*************
317+
GroupMetadata
318+
*************
319+
320+
.. autoclass:: confluent_kafka.admin.GroupMetadata
321+
:members:
322+
323+
.. _pythonclient_partition_metadata:
324+
325+
*****************
326+
PartitionMetadata
327+
*****************
328+
329+
.. autoclass:: confluent_kafka.admin.PartitionMetadata
330+
:members:
331+
332+
.. _pythonclient_topic_metadata:
333+
334+
*************
335+
TopicMetadata
336+
*************
337+
338+
.. autoclass:: confluent_kafka.admin.TopicMetadata
339+
:members:
340+
341+
.. _pythonclient_consumer_group_listing:
342+
343+
********************
344+
ConsumerGroupListing
345+
********************
346+
347+
.. autoclass:: confluent_kafka.admin.ConsumerGroupListing
348+
:members:
349+
350+
.. _pythonclient_list_consumer_group_result:
351+
352+
************************
353+
ListConsumerGroupsResult
354+
************************
355+
356+
.. autoclass:: confluent_kafka.admin.ListConsumerGroupsResult
357+
:members:
358+
359+
.. _pythonclient_consumer_group_description:
360+
361+
************************
362+
ConsumerGroupDescription
363+
************************
364+
365+
.. autoclass:: confluent_kafka.admin.ConsumerGroupDescription
366+
:members:
367+
368+
.. _pythonclient_member_assignment:
369+
370+
****************
371+
MemberAssignment
372+
****************
373+
374+
.. autoclass:: confluent_kafka.admin.MemberAssignment
375+
:members:
376+
377+
.. _pythonclient_member_description:
378+
379+
*****************
380+
MemberDescription
381+
*****************
382+
383+
.. autoclass:: confluent_kafka.admin.MemberDescription
384+
:members:
385+
250386
.. _pythonclient_consumer:
251387

252388
********
@@ -255,6 +391,7 @@ Consumer
255391

256392
.. autoclass:: confluent_kafka.Consumer
257393
:members:
394+
:noindex:
258395

259396
.. _serde_consumer:
260397

@@ -275,6 +412,7 @@ Producer
275412

276413
.. autoclass:: confluent_kafka.Producer
277414
:members:
415+
:noindex:
278416

279417
.. _serde_producer:
280418

@@ -477,6 +615,51 @@ TopicPartition
477615
.. autoclass:: confluent_kafka.TopicPartition
478616
:members:
479617

618+
.. _pythonclient_topic_collection:
619+
620+
***************
621+
TopicCollection
622+
***************
623+
624+
.. autoclass:: confluent_kafka.TopicCollection
625+
:members:
626+
627+
.. _pythonclient_topic_partition_info:
628+
629+
******************
630+
TopicPartitionInfo
631+
******************
632+
633+
.. autoclass:: confluent_kafka.TopicPartitionInfo
634+
:members:
635+
636+
.. _pythonclient_node:
637+
638+
****
639+
Node
640+
****
641+
642+
.. autoclass:: confluent_kafka.Node
643+
:members:
644+
645+
.. _pythonclient_consumer_group_topic_partition:
646+
647+
****************************
648+
ConsumerGroupTopicPartitions
649+
****************************
650+
651+
.. autoclass:: confluent_kafka.ConsumerGroupTopicPartitions
652+
:members:
653+
654+
.. _pythonclient_consumer_group_state:
655+
656+
******************
657+
ConsumerGroupState
658+
******************
659+
660+
.. autoclass:: confluent_kafka.ConsumerGroupState
661+
:members:
662+
480663
.. _serde_field:
481664

482665
************

examples/adminapi.py

Lines changed: 92 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
# Example use of AdminClient operations.
1919

2020
from confluent_kafka import (KafkaException, ConsumerGroupTopicPartitions,
21-
TopicPartition, ConsumerGroupState)
21+
TopicPartition, ConsumerGroupState, TopicCollection)
2222
from confluent_kafka.admin import (AdminClient, NewTopic, NewPartitions, ConfigResource,
2323
ConfigEntry, ConfigSource, AclBinding,
2424
AclBindingFilter, ResourceType, ResourcePatternType,
@@ -492,8 +492,9 @@ def example_describe_consumer_groups(a, args):
492492
"""
493493
Describe Consumer Groups
494494
"""
495-
496-
futureMap = a.describe_consumer_groups(args, request_timeout=10)
495+
include_auth_ops = bool(int(args[0]))
496+
args = args[1:]
497+
futureMap = a.describe_consumer_groups(args, include_authorized_operations=include_auth_ops, request_timeout=10)
497498

498499
for group_id, future in futureMap.items():
499500
try:
@@ -502,7 +503,8 @@ def example_describe_consumer_groups(a, args):
502503
print(" Is Simple : {}".format(g.is_simple_consumer_group))
503504
print(" State : {}".format(g.state))
504505
print(" Partition Assignor : {}".format(g.partition_assignor))
505-
print(" Coordinator : ({}) {}:{}".format(g.coordinator.id, g.coordinator.host, g.coordinator.port))
506+
print(
507+
f" Coordinator : {g.coordinator}")
506508
print(" Members: ")
507509
for member in g.members:
508510
print(" Id : {}".format(member.member_id))
@@ -513,12 +515,93 @@ def example_describe_consumer_groups(a, args):
513515
print(" Assignments :")
514516
for toppar in member.assignment.topic_partitions:
515517
print(" {} [{}]".format(toppar.topic, toppar.partition))
518+
if (include_auth_ops):
519+
print(" Authorized operations: ")
520+
op_string = ""
521+
for acl_op in g.authorized_operations:
522+
op_string += acl_op.name + " "
523+
print(" {}".format(op_string))
516524
except KafkaException as e:
517525
print("Error while describing group id '{}': {}".format(group_id, e))
518526
except Exception:
519527
raise
520528

521529

530+
def example_describe_topics(a, args):
531+
"""
532+
Describe Topics
533+
"""
534+
include_auth_ops = bool(int(args[0]))
535+
args = args[1:]
536+
topics = TopicCollection(topic_names=args)
537+
futureMap = a.describe_topics(topics, request_timeout=10, include_authorized_operations=include_auth_ops)
538+
539+
for topic_name, future in futureMap.items():
540+
try:
541+
t = future.result()
542+
print("Topic name : {}".format(t.name))
543+
if (t.is_internal):
544+
print("Topic is Internal")
545+
546+
if (include_auth_ops):
547+
print("Authorized operations : ")
548+
op_string = ""
549+
for acl_op in t.authorized_operations:
550+
op_string += acl_op.name + " "
551+
print(" {}".format(op_string))
552+
553+
print("Partition Information")
554+
for partition in t.partitions:
555+
print(" Id : {}".format(partition.id))
556+
leader = partition.leader
557+
print(f" Leader : {leader}")
558+
print(" Replicas : {}".format(len(partition.replicas)))
559+
for replica in partition.replicas:
560+
print(f" Replica : {replica}")
561+
print(" In-Sync Replicas : {}".format(len(partition.isr)))
562+
for isr in partition.isr:
563+
print(f" In-Sync Replica : {isr}")
564+
print("")
565+
print("")
566+
567+
except KafkaException as e:
568+
print("Error while describing topic '{}': {}".format(topic_name, e))
569+
except Exception:
570+
raise
571+
572+
573+
def example_describe_cluster(a, args):
574+
"""
575+
Describe Cluster
576+
"""
577+
include_auth_ops = bool(int(args[0]))
578+
args = args[1:]
579+
future = a.describe_cluster(request_timeout=10, include_authorized_operations=include_auth_ops)
580+
try:
581+
c = future.result()
582+
print("Cluster_id : {}".format(c.cluster_id))
583+
584+
if (c.controller):
585+
print(f"Controller: {c.controller}")
586+
else:
587+
print("No Controller Information Available")
588+
589+
print("Nodes :")
590+
for node in c.nodes:
591+
print(f" Node: {node}")
592+
593+
if (include_auth_ops):
594+
print("Authorized operations: ")
595+
op_string = ""
596+
for acl_op in c.authorized_operations:
597+
op_string += acl_op.name + " "
598+
print(" {}".format(op_string))
599+
except KafkaException as e:
600+
print("Error while describing cluster: {}".format(e))
601+
except Exception:
602+
raise
603+
604+
522605
def example_delete_consumer_groups(a, args):
523606
"""
524607
Delete Consumer Groups
@@ -704,7 +787,9 @@ def example_alter_user_scram_credentials(a, args):
704787
'<principal1> <host1> <operation1> <permission_type1> ..\n')
705788
sys.stderr.write(' list [<all|topics|brokers|groups>]\n')
706789
sys.stderr.write(' list_consumer_groups [<state1> <state2> ..]\n')
707-
sys.stderr.write(' describe_consumer_groups <group1> <group2> ..\n')
790+
sys.stderr.write(' describe_consumer_groups <include_authorized_operations> <group1> <group2> ..\n')
791+
sys.stderr.write(' describe_topics <include_authorized_operations> <topic1> <topic2> ..\n')
792+
sys.stderr.write(' describe_cluster <include_authorized_operations>\n')
708793
sys.stderr.write(' delete_consumer_groups <group1> <group2> ..\n')
709794
sys.stderr.write(' list_consumer_group_offsets <group> [<topic1> <partition1> <topic2> <partition2> ..]\n')
710795
sys.stderr.write(
@@ -737,6 +822,8 @@ def example_alter_user_scram_credentials(a, args):
737822
'list': example_list,
738823
'list_consumer_groups': example_list_consumer_groups,
739824
'describe_consumer_groups': example_describe_consumer_groups,
825+
'describe_topics': example_describe_topics,
826+
'describe_cluster': example_describe_cluster,
740827
'delete_consumer_groups': example_delete_consumer_groups,
741828
'list_consumer_group_offsets': example_list_consumer_group_offsets,
742829
'alter_consumer_group_offsets': example_alter_consumer_group_offsets,

0 commit comments

Comments
 (0)