kpd: add --dry-run-list-candidates-only option for debugging

Add a new command-line option that allows users to run KPD in dry-run mode
to list candidate patches without processing them. This is useful for
debugging issues with patch detection and understanding what patches
KPD would process.

The dry-run mode:
- Fetches relevant subjects from Patchwork using the same logic as normal operation
- Lists all candidate patches with their series information
- Exits without creating PRs or making any changes
- Provides clear output showing patch subjects, series IDs, and metadata

This helps diagnose issues like patches being outside the lookback window
or problems with the Patchwork API integration.

Generated-by: Claude AI
Signed-off-by: Luis Chamberlain <[email protected]>

Author: mcgrof

CLAUDE.md

@@ -41,6 +41,9 @@ poetry run flake8
 # Run with configuration
 poetry run python -m kernel_patches_daemon --config <config_path> --label-color configs/labels.json
 
+# Dry run mode: List candidate patches only (useful for debugging)
+poetry run python -m kernel_patches_daemon --config <config_path> --label-color configs/labels.json --dry-run-list-candidates-only
+
 # Purge all PRs and branches (destructive)
 poetry run python -m kernel_patches_daemon --config <config_path> --action purge
 ```
@@ -83,6 +86,7 @@ docker pull ghcr.io/kernel-patches/kernel-patches-daemon:latest
 - Key sections: `patchwork`, `branches`, `tag_to_branch_mapping`, `mirror_dir`
 - Authentication: GitHub Apps preferred over personal tokens
 - Mirror setup: Uses `mirror_dir` with fallback repositories for bandwidth optimization
+- Lookback period: Recommended 14-21 days for kernel projects (default 7 may be too short)
 
 ### Testing Structure
 
@@ -93,6 +97,11 @@ docker pull ghcr.io/kernel-patches/kernel-patches-daemon:latest
 
 ## Development Notes
 
+### Debugging and Troubleshooting
+- Use `--dry-run-list-candidates-only` for debugging patch detection issues
+- Check project ID type consistency (should be strings in config, auto-converted to int)
+- Verify lookback period is appropriate for the project's patch lifecycle
+
 ### Code Style
 - Uses `black` formatter (enforced)
 - Python 3.10+ required

CONFIG.md

@@ -2,7 +2,7 @@
 
 ## Overview
 
-This document provides detailed configuration options for Kernel Patches Daemon (KPD).
+This document provides detailed configuration options for Kernel Patches Daemon (KPD). For comprehensive deployment guidance including GitHub organization setup and security considerations, see the [kdevops KPD integration documentation](https://github.com/linux-kdevops/kdevops/blob/main/docs/kernel-ci/kernel-ci-kpd.md).
 
 ## Configuration File Structure
 
@@ -16,6 +16,16 @@ KPD uses a JSON configuration file with the following main sections:
 - `mirror_dir`: Optional mirror directory for bandwidth optimization
 - `email`: Optional email configuration
 
+### Patchwork Configuration
+
+The `patchwork` section includes:
+- `server`: Patchwork server hostname (e.g., "patchwork.kernel.org")
+- `project`: Project name or ID (can be string or integer)
+- `lookback`: Number of days to look back for patches (default: 7, recommended: 14-21 for kernel projects)
+- `search_patterns`: List of search criteria for filtering patches
+- `api_username`: Username for Patchwork API authentication
+- `api_token`: API token for write operations
+
 ## Branch Configuration
 
 Each branch in the `branches` section supports the following options:
@@ -74,16 +84,65 @@ For each branch, you can specify a `mirror_fallback_repo` as an alternative path
 - Ideal for corporate environments with NFS-mounted git repositories
 - Reduces storage requirements for multiple repository clones
 
-## Github personal tokens
+## Authentication
+
+### GitHub Apps (Recommended)
+
+KPD supports GitHub Apps for authentication, which provides better security and more granular permissions compared to personal access tokens. A GitHub App requires the following permissions:
+
+**Repository Permissions:**
+- **Content**: Read and write access to repository content (for creating branches and updating files)
+- **Pull requests**: Read and write access to manage pull requests
+- **Workflows**: Read access to workflow runs and write access to trigger workflows
+
+**Organization Permissions (if applicable):**
+- **Members**: Read access to organization members (for team-based access control)
+
+### GitHub Personal Tokens (Legacy)
+
+While you may use [GitHub personal tokens](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) to authenticate KPD, GitHub Apps are strongly recommended for production deployments.
+
+### Security Best Practices
+
+- **Disable webhooks** on the GitHub App for enhanced security
+- **Limit repository access** to only the repositories that need KPD integration
+- **Use environment variables** or secure secret management for tokens and keys
+- **Regularly rotate** authentication credentials
+- **Monitor access logs** and audit permissions periodically
+
+For detailed GitHub App setup instructions, including security considerations and organizational policies, refer to the [kdevops documentation](https://github.com/linux-kdevops/kdevops/blob/main/docs/kernel-ci/kernel-ci-kpd.md).
+
+## Configuration Validation
+
+KPD validates configuration on startup and will report specific errors for:
+- Missing required fields
+- Invalid JSON syntax
+- Malformed URLs or paths
+- Conflicting branch configurations
+- Authentication credential issues
+
+## Troubleshooting Common Configuration Issues
+
+**Authentication Failures:**
+- Verify GitHub App installation and permissions
+- Check token expiration dates
+- Ensure repository access is properly configured
+
+**Mirror Setup Issues:**
+- Verify mirror directory paths exist and are readable
+- Check git repository validity in mirror directories
+- Ensure proper file permissions on mirror directories
 
-You may use [Github personal token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
-to authenticate KPD to github, but using [Github App](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/about-authentication-with-a-github-app) is preferrable.
+**Branch Configuration Problems:**
+- Validate upstream and CI repository URLs
+- Verify branch names exist in respective repositories
+- Check for conflicting tag-to-branch mappings
 
-When using a GH app, it needs to have the following read and write access:
-- Content (write to repo)
-- Pull request (create PRs)
-- Workflow
+**Patch Detection Issues:**
+- Check lookback period is appropriate (7 days may be too short for kernel projects)
+- Confirm search patterns match the patches you expect to process
+- Use `--dry-run-list-candidates-only` to debug patch detection
 
 ## Example Configuration
 
-See the `configs/` directory for complete configuration examples.
+See the `configs/` directory for complete configuration examples demonstrating various deployment scenarios.

README.md

@@ -77,10 +77,19 @@ poetry run python -m kernel_patches_daemon --config <config_path> --label-color
 
 ### Debugging and Testing
 ```bash
+# Dry run mode: List candidate patches without processing them
+poetry run python -m kernel_patches_daemon --config <config_path> --label-color configs/labels.json --dry-run-list-candidates-only
+
 # Purge all existing PRs and branches (destructive operation)
 poetry run python -m kernel_patches_daemon --config <config_path> --action purge
 ```
 
+The dry-run mode is particularly useful for:
+- Debugging patch detection issues
+- Verifying configuration changes
+- Understanding what patches KPD would process
+- Testing with new patchwork instances
+
 ## Docker
 
 Kernel Patches Daemon is available as pre-build image:

kernel_patches_daemon/__main__.py

@@ -96,6 +96,11 @@ def parse_args() -> argparse.Namespace:
         choices=["start", "purge"],
         help="Purge will kill all existing PRs and delete all branches",
     )
+    parser.add_argument(
+        "--dry-run-list-candidates-only",
+        action="store_true",
+        help="Dry run mode: list candidate patches only and exit",
+    )
     args = parser.parse_args()
     return args
 
@@ -132,6 +137,19 @@ def parse_args() -> argparse.Namespace:
             logger.exception("Failed to purge")
             exit(1)
 
+    if args.dry_run_list_candidates_only:
+        try:
+            from kernel_patches_daemon.github_sync import GithubSync
+
+            github_sync = GithubSync(kpd_config=kpd_config, labels_cfg=labels_cfg)
+            import asyncio
+
+            asyncio.run(github_sync.dry_run_list_candidates())
+            exit(0)
+        except Exception:
+            logger.exception("Failed to list candidates")
+            exit(1)
+
     daemon = KernelPatchesDaemon(
         kpd_config=kpd_config,
         labels_cfg=labels_cfg,

kernel_patches_daemon/github_sync.py

@@ -373,3 +373,43 @@ async def sync_patches(self) -> None:
                 if worker._is_relevant_pr(pr):
                     self.increment_counter("prs_total")
                     processed_prs.add(1)
+
+    async def dry_run_list_candidates(self) -> None:
+        """
+        Dry run mode: List candidate patches only without processing them.
+        """
+        logger.info("Starting dry run to list candidate patches...")
+
+        # Fetch relevant subjects from Patchwork
+        self.subjects = await self.pw.get_relevant_subjects()
+
+        if not self.subjects:
+            logger.info("No candidate patches found.")
+            print("No candidate patches found.")
+            return
+
+        logger.info(f"Found {len(self.subjects)} candidate patch subjects:")
+        print(f"Found {len(self.subjects)} candidate patch subjects:")
+        print("=" * 60)
+
+        for i, subject in enumerate(self.subjects, 1):
+            print(f"{i}. Subject: {subject.subject}")
+
+            # Get the latest series for this subject
+            try:
+                latest_series = await subject.latest_series()
+                if latest_series:
+                    print(f"   Series ID: {latest_series.id}")
+                    print(f"   Series Name: {latest_series.name}")
+                    print(f"   Version: {latest_series.version}")
+                    print(f"   Date: {latest_series.date}")
+                    print(f"   URL: {latest_series.web_url}")
+                else:
+                    print("   No series found for this subject")
+            except Exception as e:
+                print(f"   Error fetching series: {e}")
+
+            print()
+
+        print("=" * 60)
+        logger.info("Dry run completed.")