Strimzi Headlamp Plugin project - Linux Foundation paid mentorship project

[illume]

This issue is for a Linux Foundation mentorship project. A paid 12 week program. See https://mentorship.lfx.linuxfoundation.org/project/2f08a480-7e37-46f0-ab29-f3d18669dd17 There you can learn more and apply if interested. Please do not start this issue, as it is reserved for that project.

---

💥 Please do not ask if you can start this in this issue, see the link to apply. 💥

Please add example open source PRs to your cover letter (not necessarily headlamp plugin ones, but any that show your quality work and communication skills)

---

Strimzi Headlamp Plugin

A Kubernetes Dashboard Plugin for Apache Kafka Management

Project Summary

This project proposes the development of a Headlamp plugin for Strimzi, a leading Kubernetes operator for Apache Kafka. The plugin will provide a comprehensive UI for managing Kafka clusters, topics, users, and connectors directly within the Kubernetes dashboard, eliminating the need for separate tooling or CLI operations.

Problem Statement

Currently, Strimzi users must rely on kubectl commands and YAML manifests to manage their Kafka infrastructure. While Cruise Control UI exists for cluster rebalancing, there is no unified interface for day-to-day Kafka operations on Kubernetes. This creates several challenges:

Operational complexity

Managing topics, users, and connectors requires deep knowledge of Strimzi CRDs and kubectl commands.

Limited visibility

Understanding cluster health, consumer lag, and connector status requires multiple CLI commands or external tools.

Context switching

Platform teams must jump between Kubernetes dashboards and Kafka-specific tools.

Steep learning curve

New users struggle to understand the relationship between Strimzi resources and their Kafka clusters.

Goals and Objectives

Primary Goals

1. Provide a visual interface for Strimzi resources within Headlamp
2. Enable CRUD operations on Kafka topics, users, and connectors
3. Display cluster health and status at a glance
4. Surface operational insights without requiring CLI expertise

Stretch Goals

1. Consumer group lag visualization (via Prometheus integration)
2. Kafka Connect connector creation wizard
3. MirrorMaker2 cross-cluster replication monitoring

Proposed Features

Feature Area	Capabilities
Kafka Cluster Overview	Broker health status, listener endpoints, Kafka version, cluster conditions, readiness state
Topic Management	List/create/delete topics, partition count, replication factor, topic configs, ready status
User Management	View users, authentication type (TLS/SCRAM), ACL visualization, quota configurations
Kafka Connect	Connect cluster status, connector list, task states (running/paused/failed), restart actions
Kafka Bridge	HTTP bridge status, endpoint availability, configuration details
Entity Operator	Topic Operator and User Operator health, reconciliation status, recent events
Operational Events	Rolling updates timeline, certificate renewals, scaling events, error conditions

Technical Approach

Data Sources

The plugin will retrieve data from multiple sources via the Kubernetes API:

Strimzi CRDs

Kafka, KafkaTopic, KafkaUser, KafkaConnect, KafkaConnector, KafkaBridge, KafkaMirrorMaker2, KafkaRebalance

Kubernetes Resources

Pods, StatefulSets, Services, Secrets, ConfigMaps, Events, PVCs

Prometheus Metrics (optional)

Broker metrics, consumer lag, JVM metrics if available

Architecture

The plugin will be built using Headlamp's plugin SDK with React and TypeScript. It will leverage Headlamp's existing Kubernetes API client for all data fetching, ensuring proper RBAC enforcement and authentication handling.

Key Technical Considerations

1. No additional backend required - all data accessible via Kubernetes API
2. RBAC-aware - respects existing Kubernetes permissions
3. Consumer lag requires Prometheus or external tooling (documented limitation)
4. Plugin packaged as standard Headlamp plugin for easy distribution

Proposed Timeline

Phase	Deliverables
Week 1-2	Community bonding, environment setup, deep dive into Strimzi CRDs and Headlamp plugin SDK
Week 3-4	Plugin scaffold, Kafka cluster overview page with health status and broker information
Week 5-6	KafkaTopic management - list, create, delete, view configurations
Week 7-8	KafkaUser management - user list, ACL visualization, authentication details
Week 9	Midterm evaluation, code review, bug fixes, documentation
Week 10-11	KafkaConnect and KafkaConnector pages - connector status, task management, restart actions
Week 12-13	KafkaBridge, Entity Operator status, operational events timeline
Week 14-16	Testing, documentation, stretch goals (Prometheus integration), final polish
Week 17	Final evaluation, release preparation, community handoff

Expected Outcomes

1. Fully functional Headlamp plugin published to the Headlamp plugin repository
2. Comprehensive documentation including installation guide and user manual
3. Unit and integration tests with CI/CD pipeline
4. Demo video showcasing plugin capabilities
5. Blog post announcing the plugin to both Strimzi and Headlamp communities

Required Skills for Contributor

• Essential: TypeScript, React
• Helpful: Kubernetes fundamentals, understanding of Custom Resource Definitions, Experience with Apache Kafka, familiarity with Strimzi, Headlamp plugin development
• Nice to have: Prometheus/metrics experience, UI/UX design skills

Mentorship

We are seeking mentors from both the Headlamp and Strimzi communities to guide the contributor. Ideally, we would have one mentor familiar with Headlamp plugin development and another with deep Strimzi expertise.

Community Benefit

• For Strimzi users: Simplified Kafka management without CLI expertise, better operational visibility
• For Platform teams: Unified dashboard for Kubernetes and Kafka resources, reduced context switching
• For Headlamp: Expanded ecosystem with a high-demand integration
• For the contributor: Real-world open source experience with two active CNCF projects

Contact and Next Steps

We welcome feedback from the Strimzi community on this proposal. Please share your thoughts on the proposed features, suggest additional capabilities, or express interest in mentorship.

• Strimzi Slack: #strimzi channel on CNCF Slack
• Headlamp Slack: #headlamp channel on Kubernetes slack
• Headlamp GitHub: https://github.com/kubernetes-sig/headlamp
• Strimzi GitHub: https://github.com/strimzi/strimzi-kafka-operator

Other open Headlamp LFX mentor projects

You only get max 3 applications, but maybe consider only applying to the 1-2 you are most interested in (and state your preference of the one you are most interested in your application). There are other projects to apply to as well.

• Add Volcano to Headlamp
• Improve Cluster API Headlamp plugin
• Improve Knative Headlamp plugin
• Add Strimzi to Headlamp
• Add Kubeflow to Headlamp

[illume]

CNCF project ideas format text

Add Strimzi to Headlamp: Kubernetes Kafka Management UI

• Description:
  Strimzi is an open-source Kubernetes Operator that makes it easier to run and manage Apache Kafka clusters on Kubernetes. This project will develop a Headlamp plugin that gives users a dedicated UI for Strimzi-managed Kafka resources, directly inside Headlamp. The plugin will allow cluster operators and developers to visualize and control Kafka clusters and their associated components (like topics, users, and connectors) through the Headlamp dashboard. By integrating Strimzi's Kafka management into Headlamp, users gain a single-pane-of-glass for both standard Kubernetes objects and their Kafka ecosystem, ensuring that managing Kafka on K8s is as intuitive as managing any other resource. The plugin will be designed around Headlamp's existing UX patterns, providing list and detail views for Strimzi's custom resources and possibly linking to or embedding Kafka-specific metrics and controls.

• Expected Outcome:

  • A fully functional plugin that surfaces all key Strimzi CRDs in Headlamp's UI, including Kafka clusters, KafkaTopics, KafkaUsers, and KafkaConnect instances. Users should be able to navigate via the sidebar or homepage to a "Kafka" section where these resource types are listed (similar to how, for example, the Knative plugin lists Knative Services). Each resource type will have its own list view showing relevant summary info. For example, the Kafka cluster list might display the cluster name, namespace, the number of brokers (and Zookeepers if applicable), and high-level status (Ready or not). KafkaTopic list might show topic name, partitions, replication factor, and the cluster it belongs to.
  • Detail pages for each Strimzi resource with comprehensive information and basic management actions. For a Kafka cluster (Kafka CRD) detail, this includes the Kafka version, broker configurations (e.g., storage class, listeners, etc.), and status conditions (with any errors or warnings highlighted). It could also show sub-resources like a list of broker pods or utilization metrics if available. For a KafkaTopic detail view, show partition count, replication factor, current partition leaders (if easily fetchable), and perhaps a list of consumer groups or throughput metrics (if integrated with metrics). Additionally, allow actions like creating a new topic or editing topic configuration via a form, which would create/update the KafkaTopic CRD. For KafkaUser, detail shows the type of user (scram, tls, etc.), what ACL or permissions are granted (if part of the CRD), and link to the secret where credentials are stored. Possibly an action to reset credentials (regenerate password) could be offered. For KafkaConnect instances, the detail should show the connect cluster's status and configuration (like number of workers, image version, etc.), and ideally list the KafkaConnector resources running in that connect cluster (with their states) to give a quick overview of deployed connectors. Users might be able to quickly navigate from a KafkaConnect detail to create/view individual KafkaConnector CRDs (representing a specific connector configuration).
  • Intuitive navigation and linking between resources. For example, if viewing a Kafka cluster detail, there could be tabs or sections for "Topics" and "Users" belonging to that cluster (the plugin can filter and show KafkaTopic and KafkaUser CRDs that reference this cluster). These would be clickable to go to those resources' details. Similarly, a KafkaTopic detail might link back to the parent Kafka cluster. This relational navigation reduces context switching and helps the user see the big picture of their Kafka deployment.
  • Consistency with Headlamp UX: The plugin will use Headlamp's design system – tables, detail layouts, icons – ensuring it feels like a natural extension. Strimzi resource types will have custom icons (for instance, using the Kafka logo or a relevant symbol for Kafka cluster, maybe a database icon for topics, user icon for KafkaUser, etc., likely monochrome to fit Headlamp's style). The Map view (graphical topology) may be extended to visualize Kafka clusters (for example, show a node representing the Kafka cluster with child nodes for broker pods, and connections to ZooKeeper or Strimzi operator if relevant). At minimum, the plugin shouldn't break the Map view; at best, it will enhance it with Kafka relationships.
  • A focus on readability and usability: Large configuration sections (like the full spec of a Kafka CR) will be displayed in a structured way (possibly collapsed sections or a YAML viewer toggle), so that users can both get high-level info quickly and dig deeper when needed. Error messages or status conditions will be humanized where possible (translating "NotReady" conditions into user-friendly messages). For actions that modify resources (creating a topic, scaling a cluster's broker count, etc.), the plugin will provide forms with validation to prevent common mistakes, and confirmations for destructive actions.
  • Documentation and outreach: We'll provide a clear README for the plugin, explaining how to enable it and listing its capabilities. This will include any prerequisites (e.g., Strimzi operator must be installed on the cluster) and limitations (what it doesn't do, such as not handling KafkaBridge if we decide to omit it initially).
  • blog post on Kubernetes blog showcasing the plugin – e.g., a walkthrough of using Headlamp + Strimzi plugin to create a Kafka topic and monitor the Kafka cluster.

• Recommended Skills:

  • TypeScript & React
  • (Optional) Familiarity with Kubernetes CRDs and Operators
  • (Optional) Knowledge of Apache Kafka concepts
  • (Optional) UX design sensibilities

• Mentor(s):

  • Rene Dudfield (@illume, renedudfield@microsoft.com)
  • Jakub Scholz (@scholzj, github@scholzj.com)
  • Santhosh Nagaraj (@yolossn, sannagaraj@microsoft.com)
  • Ashu Ghildiyal (@ashu8912, ashu.ghildiyal@microsoft.com)

• Upstream Issue:
  Strimzi Headlamp Plugin project - Linux Foundation paid mentorship project #488

---

[manumathon]

@illume Is someone already working on this, or should we take it up?

[illume]

@manumathon

💥 Note: this project is not approved yet, and the issue is a draft. Please do not ask if you can start this, there will be a link if it is possible to accept applications. 💥