Logging Analysis Report for py-libp2p

[acul71]

Logging Analysis Report for py-libp2p

Last review: Sept 3 2025

Executive Summary

This report provides a comprehensive analysis of the logging configuration across the py-libp2p codebase. The library implements a sophisticated logging system but suffers from inconsistent adoption and configuration across modules. Key findings include proper logging in some areas, inconsistent naming patterns in core modules, and completely absent logging in critical security components.

Logging System Architecture

Core Infrastructure

The library features a well-designed logging system located in libp2p/utils/logging.py with the following capabilities:

• Environment-based Configuration: Uses LIBP2P_DEBUG for granular control
• Hierarchical Logging: Supports Python's native dot-notation hierarchy
• Queue-based Processing: Thread-safe logging with QueueHandler and QueueListener
• Flexible Output: Configurable file and console output via LIBP2P_DEBUG_FILE
• Module-specific Control: Allows fine-grained log level control per module

Configuration Format

The system expects loggers to follow the pattern: libp2p.{module}.{submodule} and supports configurations like:

• LIBP2P_DEBUG=DEBUG (global debug)
• LIBP2P_DEBUG=identity.identify:DEBUG,transport:INFO (module-specific)

Current State Analysis

✅ Well-Configured Modules

These modules correctly implement logging using logging.getLogger(__name__):

Identity & Authentication

• libp2p/identity/identify/identify.py
• libp2p/identity/identify_push/identify_push.py

Relay Components

• libp2p/relay/circuit_v2/dcutr.py

Example Code

• examples/doc-examples/multiple_connections_example.py

Test Infrastructure

• tests/core/examples/test_examples.py
• Multiple relay test modules

⚠️ Inconsistently Configured Modules

These modules use hardcoded logger names instead of __name__, breaking the hierarchical system:

Core Networking

# In libp2p/network/swarm.py
logger = logging.getLogger("libp2p.network.swarm")

# In libp2p/host/basic_host.py  
logger = logging.getLogger("libp2p.network.basic_host")

# In libp2p/host/ping.py
logger = logging.getLogger("libp2p.host.ping")

# In libp2p/host/autonat/autonat.py
logger = logging.getLogger("libp2p.host.autonat")

Transport Layer

# In libp2p/transport/tcp/tcp.py
logger = logging.getLogger("libp2p.transport.tcp")

Stream Multiplexing

# In libp2p/stream_muxer/yamux/yamux.py
logger = logging.getLogger("libp2p.stream_muxer.yamux")

# In libp2p/stream_muxer/mplex/mplex.py
logger = logging.getLogger("libp2p.stream_muxer.mplex.mplex")

Publish-Subscribe System

# In libp2p/pubsub/floodsub.py
logger = logging.getLogger("libp2p.pubsub.floodsub")

# In libp2p/pubsub/gossipsub.py
logger = logging.getLogger("libp2p.pubsub.gossipsub")

# In libp2p/pubsub/pubsub.py
logger = logging.getLogger("libp2p.pubsub")

# In libp2p/pubsub/validators.py
logger = logging.getLogger("libp2p.pubsub")

Peer Discovery

# In libp2p/discovery/bootstrap/bootstrap.py
logger = logging.getLogger("libp2p.discovery.bootstrap")

# In libp2p/discovery/mdns/mdns.py
logger = logging.getLogger("libp2p.discovery.mdns")

# Multiple other discovery modules follow similar pattern

Utility & Support Modules

# In libp2p/utils/varint.py
logger = logging.getLogger("libp2p.utils.varint")

# In libp2p/utils/version.py
logger = logging.getLogger("libp2p.utils.version")

# In libp2p/io/trio.py
logger = logging.getLogger("libp2p.io.trio")

🚨 Problematic Configurations

Non-Standard Logger Hierarchies

The Kademlia DHT implementation uses completely non-standard logger names:

# In libp2p/kad_dht/kad_dht.py
logger = logging.getLogger("kademlia-example.kad_dht")

# In libp2p/kad_dht/value_store.py
logger = logging.getLogger("kademlia-example.value_store")

# In libp2p/kad_dht/provider_store.py
logger = logging.getLogger("kademlia-example.provider_store")

# In libp2p/kad_dht/peer_routing.py
logger = logging.getLogger("kademlia-example.peer_routing")

# In libp2p/kad_dht/routing_table.py
logger = logging.getLogger("kademlia-example.routing_table")

# In libp2p/kad_dht/utils.py
logger = logging.getLogger("kademlia-example.utils")

The PubSub utilities module also uses non-standard naming:

# In libp2p/pubsub/utils.py
logger = logging.getLogger("pubsub-example.utils")

Third-party Logger Names

# In libp2p/tools/async_service/base.py
logger = logging.getLogger("async_service.Manager")

❌ Missing Logging

Critical security modules completely lack logging configuration:

• libp2p/security/noise/transport.py
• libp2p/security/secio/transport.py

Impact Assessment

Functional Impact

1. Broken Module-specific Control: Modules using hardcoded names cannot be controlled via LIBP2P_DEBUG=module:LEVEL syntax
2. Inconsistent Hierarchy: The non-standard Kademlia logger names (kademlia-example.*) are completely outside the libp2p hierarchy
3. Missing Debug Capability: Security modules provide no logging for troubleshooting encryption/authentication issues
4. Configuration Confusion: Users cannot reliably predict logger names for configuration

Development Impact

1. Debugging Difficulty: Inconsistent logging makes system-wide debugging challenging
2. Maintenance Burden: Hardcoded names require manual updates if module structure changes
3. Documentation Complexity: Multiple naming patterns make logging documentation complicated

Recommendations

1. Immediate Priority: Standardize Core Modules

Convert all modules to use logging.getLogger(__name__):

# Instead of:
logger = logging.getLogger("libp2p.network.swarm")

# Use:
logger = logging.getLogger(__name__)

High Priority Files:

• libp2p/network/swarm.py
• libp2p/host/basic_host.py
• libp2p/transport/tcp/tcp.py
• libp2p/stream_muxer/yamux/yamux.py
• libp2p/stream_muxer/mplex/mplex.py
• libp2p/pubsub/floodsub.py
• libp2p/pubsub/gossipsub.py

2. Critical: Fix Kademlia DHT Modules

Replace all kademlia-example.* logger names with proper hierarchy:

# Change from:
logger = logging.getLogger("kademlia-example.kad_dht")

# To:
logger = logging.getLogger(__name__)

Affected Files:

• libp2p/kad_dht/kad_dht.py
• libp2p/kad_dht/value_store.py
• libp2p/kad_dht/provider_store.py
• libp2p/kad_dht/peer_routing.py
• libp2p/kad_dht/routing_table.py
• libp2p/kad_dht/utils.py

3. Essential: Add Security Module Logging

Implement comprehensive logging in security modules:

import logging
logger = logging.getLogger(__name__)

# Add appropriate debug/info/error logging for:
# - Handshake processes
# - Key exchanges
# - Encryption/decryption operations
# - Authentication failures

Target Files:

• libp2p/security/noise/transport.py
• libp2p/security/secio/transport.py

4. Cleanup: Fix Remaining Inconsistencies

• Update libp2p/pubsub/utils.py from "pubsub-example.utils" to __name__
• Address test module naming inconsistencies
• Review third-party logger usage in libp2p/tools/async_service/base.py

5. Documentation Updates

Update documentation to reflect:

• Proper logger naming conventions
• Module-specific logging control examples
• Debugging best practices

Implementation Strategy

Phase 1: Core Infrastructure (Week 1)

• Fix networking, host, transport, and stream muxer modules
• Test logging functionality with LIBP2P_DEBUG

Phase 2: Application Layer (Week 2)

• Update PubSub and discovery modules
• Fix Kademlia DHT logger names
• Verify hierarchical control works correctly

Phase 3: Security & Utilities (Week 3)

• Add comprehensive security module logging
• Update utility modules
• Address remaining inconsistencies

Phase 4: Testing & Documentation (Week 4)

• Update test modules
• Verify all logging scenarios
• Update documentation and examples

Testing Verification

After implementation, verify fixes with:

# Test global logging
LIBP2P_DEBUG=DEBUG python your_app.py

# Test module-specific logging
LIBP2P_DEBUG=network:DEBUG,kad_dht:INFO python your_app.py

# Test hierarchical control
LIBP2P_DEBUG=pubsub.gossipsub:DEBUG python your_app.py

Conclusion

The py-libp2p logging system has excellent architectural design but suffers from inconsistent implementation across ~50 modules. The most critical issues are:

1. Non-standard Kademlia DHT logger names that break the entire hierarchy
2. Missing logging in security modules that hampers debugging
3. Widespread use of hardcoded logger names that prevents proper module control

Addressing these issues will provide:

• Reliable module-specific logging control
• Consistent debugging experience
• Better maintainability
• Improved developer experience

The fixes are straightforward but require systematic implementation across the codebase. Prioritizing core networking and DHT modules will provide the most immediate benefit.

[acul71]

Issue: Standardize Logger Names in Core Modules (Phase 1)

🎯 Good First Issue - Phase 1/3

This is the first phase of standardizing logging configuration across py-libp2p. Perfect for new contributors to get familiar with the codebase!

📋 Background

The py-libp2p library has an excellent logging system in libp2p/utils/logging.py that supports module-specific control via environment variables like:

LIBP2P_DEBUG=network:DEBUG,pubsub:INFO

However, many modules use hardcoded logger names instead of __name__, which breaks this hierarchical control system.

🎯 Goal

Update core networking and pubsub modules to use logging.getLogger(__name__) instead of hardcoded strings.

📂 Files to Update (Phase 1 - 7 files)

Network & Host Modules

1. libp2p/network/swarm.py

   • Current: logger = logging.getLogger("libp2p.network.swarm")
   • Change to: logger = logging.getLogger(__name__)

2. libp2p/host/basic_host.py

   • Current: logger = logging.getLogger("libp2p.network.basic_host")
   • Change to: logger = logging.getLogger(__name__)

3. libp2p/transport/tcp/tcp.py

   • Current: logger = logging.getLogger("libp2p.transport.tcp")
   • Change to: logger = logging.getLogger(__name__)

PubSub Modules

4. libp2p/pubsub/floodsub.py

   • Current: logger = logging.getLogger("libp2p.pubsub.floodsub")
   • Change to: logger = logging.getLogger(__name__)

5. libp2p/pubsub/gossipsub.py

   • Current: logger = logging.getLogger("libp2p.pubsub.gossipsub")
   • Change to: logger = logging.getLogger(__name__)

6. libp2p/pubsub/pubsub.py

   • Current: logger = logging.getLogger("libp2p.pubsub")
   • Change to: logger = logging.getLogger(__name__)

7. libp2p/pubsub/validators.py

   • Current: logger = logging.getLogger("libp2p.pubsub")
   • Change to: logger = logging.getLogger(__name__)

🔧 Step-by-Step Instructions

1. Find the Logger Declaration

In each file, look for a line like:

logger = logging.getLogger("some.hardcoded.name")

2. Replace with name

Change it to:

logger = logging.getLogger(__name__)

3. Example Change

Before:

import logging
from libp2p.abc import IHost

logger = logging.getLogger("libp2p.network.swarm")

class Swarm:
    # ... rest of the class

After:

import logging
from libp2p.abc import IHost

logger = logging.getLogger(__name__)

class Swarm:
    # ... rest of the class

✅ Testing Your Changes

1. Basic Test

Run any example to ensure no import errors:

cd examples/ping/
python ping.py

2. Test Module-Specific Logging

Test that module-specific logging control works:

# Test network module logging
LIBP2P_DEBUG=network:DEBUG python examples/ping/ping.py

# Test pubsub module logging  
LIBP2P_DEBUG=pubsub:DEBUG python examples/pubsub/pubsub.py

# Test combined logging
LIBP2P_DEBUG=network:DEBUG,pubsub:INFO python examples/pubsub/pubsub.py

You should see debug messages from the specific modules you configured.

3. Verify Logger Names

Add a temporary test to verify the logger names are correct:

import logging
from libp2p.network.swarm import logger as swarm_logger
from libp2p.pubsub.gossipsub import logger as gossip_logger

print(f"Swarm logger name: {swarm_logger.name}")  # Should be "libp2p.network.swarm"
print(f"GossipSub logger name: {gossip_logger.name}")  # Should be "libp2p.pubsub.gossipsub"

📝 Submission Guidelines

1. Commit Message Format

fix: standardize logger names in core modules (phase 1)

- Replace hardcoded logger names with __name__ in:
  - libp2p/network/swarm.py
  - libp2p/host/basic_host.py
  - libp2p/transport/tcp/tcp.py
  - libp2p/pubsub/floodsub.py
  - libp2p/pubsub/gossipsub.py
  - libp2p/pubsub/pubsub.py
  - libp2p/pubsub/validators.py

- Enables module-specific logging control via LIBP2P_DEBUG
- Part of logging standardization effort (Phase 1/3)

2. PR Description Template

## 🎯 Issue
Fixes logging standardization in core modules (Phase 1)

## 📋 Changes
- [x] Updated 7 core modules to use `logging.getLogger(__name__)`
- [x] Tested module-specific logging control works correctly
- [x] Verified no import errors or functionality changes

## 🧪 Testing
- [x] Basic functionality tests pass
- [x] Module-specific logging control verified with LIBP2P_DEBUG
- [x] No breaking changes to existing logging behavior

## 📚 Learning
Brief summary of what you learned about the codebase while working on this.

🤝 Getting Help

Questions to Ask:

• "How do I test the logging changes work correctly?"
• "Should I modify any other files I found with similar patterns?"
• "Can you help me understand what this module does?"

Resources:

• Logging System: libp2p/utils/logging.py - study this to understand how the system works
• Examples: examples/ directory - great for testing your changes
• Documentation: docs/getting_started.rst - has logging usage examples

🚀 Next Steps

If you complete Phase 1 successfully and want to continue:

• Phase 2: Discovery and relay modules (~15 files)
• Phase 3: Kademlia DHT modules (requires fixing non-standard logger names)

🎉 Why This Matters

After your changes:

• Developers can control logging per module: LIBP2P_DEBUG=network:DEBUG
• Consistent logging hierarchy across the library
• Better debugging experience for all py-libp2p users
• You'll have contributed to improving the entire library!

---

Estimated Time: 2-3 hours for a new contributor
Difficulty: ⭐⭐☆☆☆ (Beginner-friendly)
Impact: 🔥🔥🔥🔥⭐ (High impact across entire library)

[PankajJaisu]

I’d like to work on this issue.

[acul71]

Great,
Open a Issue on py-libp2p. you can write a issue with tasks (look at this #825 ) . Open maybe 4 task one for every phase ? or do as you see it fits

Then start to work on task 1
and submit a PR (you can submit a PR even if is not ready just open it as a draft PR, in this way your work will be tracked, even if the PR is not full implemented, the maintainers can see your work in progress)
I'll guide you with all requirements/good practices while reviewing the PR.

Tag me so I can follow. Ask any questions you like, also in discord as you want.
We use discussions to discuss about issues, to keep the PR cleaner, in this way the maintainers can review the PR, without having to read long discussions.

[acul71]

@PankajJaisu I'm back in github