Python for Chaos Engineering: Observability During Failure Scenarios

[saurabhmi2212]

Talk title

Python for Chaos Engineering: Observability During Failure Scenarios

Short talk description

Chaos Engineering is a discipline that tests system resilience by injecting controlled failures into distributed systems. Python, with its rich ecosystem of libraries, plays a pivotal role in automating chaos experiments and enhancing observability during failure scenarios. This presentation explores how Python can be used to simulate failures in cloud-native applications, monitor system behavior, and collect critical observability data such as logs, metrics, and traces. By integrating Python with tools like OpenTelemetry, Prometheus, and Grafana, developers can gain insights into system weaknesses and improve reliability. The session highlights practical examples of Python scripts for chaos experiments and observability in modern infrastructures.

Long talk description

Chaos Engineering is a discipline that tests system resilience by injecting controlled failures into distributed systems. Python, with its rich ecosystem of libraries, plays a pivotal role in automating chaos experiments and enhancing observability during failure scenarios. This presentation explores how Python can be used to simulate failures in cloud-native applications, monitor system behavior, and collect critical observability data such as logs, metrics, and traces. By integrating Python with tools like OpenTelemetry, Prometheus, and Grafana, developers can gain insights into system weaknesses and improve reliability. The session highlights practical examples of Python scripts for chaos experiments and observability in modern infrastructures.

What format do you have in mind?

Talk (20-25 minutes + Q&A)

Talk outline / Agenda

Introduction (5 min)

What is Chaos Engineering?
Importance of observability during failures.
Role of Python in automating chaos experiments.
Chaos Engineering Basics (10 min)

Principles of Chaos Engineering.
Common failure scenarios in distributed systems.
Overview of Chaos Engineering tools and Python integration.
Observability Overview (10 min)

Three pillars: logs, metrics, and traces.
Tools for observability: OpenTelemetry, Prometheus, Grafana.
Why observability is critical during chaos experiments.
Python for Chaos Engineering (15 min)

Simulating failures with Python (chaoslib, Kubernetes integration).
Automating chaos experiments with Python scripts.
Python for Observability (15 min)

Collecting logs, metrics, and traces with Python.
Visualizing failure impact using Prometheus and Grafana APIs.
Tracing microservices with OpenTelemetry.
Case Study (10 min)

Real-world example: Injecting failures in Kubernetes.
Monitoring system behavior with Python.
Analyzing results to identify weaknesses.
Best Practices & Q&A (10 min)

Best practices for combining Chaos Engineering and observability.
Lessons learned and common pitfalls.
Audience Q&A and closing remarks.

Key takeaways

Python enables automated, observable chaos experiments—letting teams inject failures, collect real-time metrics/logs/traces, and quickly verify system resilience by turning failures into measurable, actionable insights rather than surprises.

What domain would you say your talk falls under?

Core Python

Duration (including Q&A)

40

Prerequisites and preparation

No response

Resources and references

No response

Link to slides/demos (if available)

No response

Twitter/X handle (optional)

No response

LinkedIn profile (optional)

https://www.linkedin.com/in/connectsaurabhmishra/

Profile picture URL (optional)

No response

Speaker bio

Saurabh Mishra is a Cloud Evangelist with a deep passion for cloud architecture, DevOps, and automation. He actively engages with the global tech community, sharing insights on cloud-native technologies, security best practices, and multi-cloud strategies.As an experienced speaker and mentor, Saurabh has delivered sessions at conferences, meetups, and workshops, helping teams accelerate their cloud adoption, modernization, and optimization journeys. His work bridges innovation and practical implementation, empowering organizations to build resilient, scalable, and secure cloud solutions.

Availability

Jan/2026

Accessibility & special requirements

No response

Speaker checklist

• I have read and understood the PyDelhi guidelines for submitting proposals and giving talks
• I will make my talk accessible to all attendees and will proactively ask for any accommodations or special requirements I might need
• I agree to share slides, code snippets, and other materials used during the talk with the community
• I will follow PyDelhi's Code of Conduct and maintain a welcoming, inclusive environment throughout my participation
• I understand that PyDelhi meetups are community-centric events focused on learning, knowledge sharing, and networking, and I will respect this ethos by not using this platform for self-promotion or hiring pitches during my presentation, unless explicitly invited to do so by means of a sponsorship or similar arrangement
• If the talk is recorded by the PyDelhi team, I grant permission to release the video on PyDelhi's YouTube channel under the CC-BY-4.0 license, or a different license of my choosing if I am specifying it in my proposal or with the materials I share

Additional comments

No response

[Daksh777]

Hey Saurabh!
Love the topic, and the agenda seems good but right now the scope feels a bit too broad for a single 30–40 minute talk. It currently tries to cover chaos engineering fundamentals, observability concepts, multiple tools, Python implementations, and a full Kubernetes case study, which risks staying at a high level rather than going deep enough for the audience to walk away with concrete learnings.

For PyDelhi, we’ve found that more focused, hands-on talks work better, for example, centering the session around one failure scenario, one Python-driven experiment, and what observability data actually surfaced during that failure. Tightening the scope would help make the talk more practical, opinionated, and engaging.

Also, could you please add the slides for your talk so we can review them? I’d also like to go over the real-world example/demo, could you share the code files or notebook you’ll be using for that?

Lastly, I’d suggest reducing the talk duration by 10 minutes. We have a cap of 30 minutes for the main talk; we can stretch slightly for Q&A if needed, but keeping the core content tighter would be better

[saurabhmi2212]

thanks for reaching out Daksh, yeah it can be reduced by 10m.
This talk is selected for PYCON MY 2025 .
Slide link:

https://docs.google.com/presentation/d/1wcW_iwGej_JC8DCUUsQIg2kGDOovtMl5/edit?slide=id.g38a63d30964_0_565#slide=id.g38a63d30964_0_565

[ghanatava]

hey @saurabhmi2212 thanks for sharing the slide. I found that the slides provide a surface level view of chaos engineering which is good for beginners but maybe we can go more in depth so intermediate folks also get something out of it.
Following are some suggestions that you can consider

1. Replace the dummy example - The /bin/true example in the quickstart is not close to reality. Swap it with a real scenario like Kubernetes pod deletion or network latency injection.
2. Add chaos metrics section - Currently missing guidance on what to actually measure. A slide covering MTTR (Mean Time To Recovery), MTTD (Mean Time To Detection), and blast radius would give people concrete metrics to track and prove the value of their chaos experiments.

You are welcome to make other changes as well, that you feel can be valuable : )

[kirtivr]

Hello @saurabhmi2212,

Dropping in to say that we need to have talks finalized by tomorrow, so if you are working on the updated slides do inform us here so we can expect to review quickly.

We are expecting to have the updated slides for review by afternoon.

If you're busy thats okay, we can work on accomodating your talk in the next PyDelhi!

[saurabhmi2212]

Thanks for reaching out! Let’s plan for the next meetup, I’m a bit tied up with a few other things right now.

[agriyakhetarpal]

Thanks for letting us know, @saurabhmi2212! I'll mark your proposal with the "on hold" label for now, and we'll reach out the next time we restart our reviews for the February edition of the meetup.