|  | 
|  | 1 | +# CLAUDE.md | 
|  | 2 | + | 
|  | 3 | +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. | 
|  | 4 | + | 
|  | 5 | +## Overview | 
|  | 6 | + | 
|  | 7 | +The `nf-core/tools` repository contains a comprehensive Python package that provides command-line tools for the nf-core community. This package helps users run, create, and develop nf-core Nextflow pipelines with integrated support for modules, subworkflows, and schema management. | 
|  | 8 | + | 
|  | 9 | +## Development Commands | 
|  | 10 | + | 
|  | 11 | +### Testing | 
|  | 12 | + | 
|  | 13 | +```bash | 
|  | 14 | +# Run all tests with pytest | 
|  | 15 | +python -m pytest | 
|  | 16 | + | 
|  | 17 | +# Run specific test file | 
|  | 18 | +python -m pytest tests/test_<module>.py | 
|  | 19 | + | 
|  | 20 | +# Run tests with coverage | 
|  | 21 | +python -m pytest --cov --cov-config=.coveragerc | 
|  | 22 | + | 
|  | 23 | +# Run with verbose output | 
|  | 24 | +python -m pytest -v | 
|  | 25 | + | 
|  | 26 | +# Run tests with specific markers | 
|  | 27 | +python -m pytest -m "datafiles" | 
|  | 28 | +``` | 
|  | 29 | + | 
|  | 30 | +### Code Quality and Linting | 
|  | 31 | + | 
|  | 32 | +```bash | 
|  | 33 | +# Run pre-commit hooks (includes ruff formatting and linting) | 
|  | 34 | +pre-commit run --all-files | 
|  | 35 | + | 
|  | 36 | +# Run ruff linter | 
|  | 37 | +ruff check . | 
|  | 38 | + | 
|  | 39 | +# Run ruff formatter | 
|  | 40 | +ruff format . | 
|  | 41 | + | 
|  | 42 | +# Run mypy type checking | 
|  | 43 | +mypy nf_core/ | 
|  | 44 | +``` | 
|  | 45 | + | 
|  | 46 | +### Development Setup | 
|  | 47 | + | 
|  | 48 | +```bash | 
|  | 49 | +# Install in development mode with dev dependencies | 
|  | 50 | +pip install --upgrade -r requirements-dev.txt -e . | 
|  | 51 | + | 
|  | 52 | +# Install pre-commit hooks | 
|  | 53 | +pre-commit install | 
|  | 54 | + | 
|  | 55 | +# Build package | 
|  | 56 | +python -m build | 
|  | 57 | + | 
|  | 58 | +# Install package locally | 
|  | 59 | +pip install -e . | 
|  | 60 | +``` | 
|  | 61 | + | 
|  | 62 | +## Architecture Overview | 
|  | 63 | + | 
|  | 64 | +### Core Structure | 
|  | 65 | + | 
|  | 66 | +The codebase is organized into several key domains: | 
|  | 67 | + | 
|  | 68 | +**Command Structure**: The main CLI interface is defined in `nf_core/__main__.py` using Click with rich-click for enhanced formatting. Commands are organized hierarchically: | 
|  | 69 | + | 
|  | 70 | +- `pipelines/` - Pipeline management (create, lint, download, sync, etc.) | 
|  | 71 | +- `modules/` - Module management (install, update, create, test, etc.) | 
|  | 72 | +- `subworkflows/` - Subworkflow management (similar to modules) | 
|  | 73 | +- `test_datasets/` - Test dataset management | 
|  | 74 | + | 
|  | 75 | +**Component Architecture**: | 
|  | 76 | + | 
|  | 77 | +- `nf_core/components/` - Shared functionality for modules and subworkflows | 
|  | 78 | +- `nf_core/pipelines/` - Pipeline-specific operations including lint tests | 
|  | 79 | +- `nf_core/modules/` - Module-specific operations and lint tests | 
|  | 80 | +- `nf_core/subworkflows/` - Subworkflow-specific operations | 
|  | 81 | + | 
|  | 82 | +### Key Design Patterns | 
|  | 83 | + | 
|  | 84 | +**Lint System**: Both pipelines and modules use a comprehensive lint system: | 
|  | 85 | + | 
|  | 86 | +- `nf_core/pipelines/lint/` - Contains individual lint test files | 
|  | 87 | +- `nf_core/modules/lint/` - Module-specific lint tests | 
|  | 88 | +- Each lint test is a separate Python module with standardized interface | 
|  | 89 | + | 
|  | 90 | +**Template System**: Templates are stored in dedicated directories: | 
|  | 91 | + | 
|  | 92 | +- `nf_core/pipeline-template/` - Complete pipeline template with Jinja2 templating | 
|  | 93 | +- `nf_core/module-template/` - Module template for creating new modules | 
|  | 94 | +- `nf_core/subworkflow-template/` - Subworkflow template | 
|  | 95 | + | 
|  | 96 | +**Component Management**: Modules and subworkflows share common patterns: | 
|  | 97 | + | 
|  | 98 | +- JSON tracking files (`modules.json`) for version management | 
|  | 99 | +- Git-based remote repository integration | 
|  | 100 | +- Local vs remote component distinction | 
|  | 101 | +- Patch system for local modifications | 
|  | 102 | + | 
|  | 103 | +### Data Flow | 
|  | 104 | + | 
|  | 105 | +1. **Command Parsing**: Click commands in `__main__.py` parse arguments and delegate to command modules | 
|  | 106 | +2. **Context Management**: Click context objects carry configuration and state between commands | 
|  | 107 | +3. **Git Operations**: Remote repository operations for fetching modules/subworkflows | 
|  | 108 | +4. **Template Processing**: Jinja2 templating for generating new pipelines/modules | 
|  | 109 | +5. **Lint Execution**: Modular lint system with individual test execution and reporting | 
|  | 110 | + | 
|  | 111 | +## Testing Strategy | 
|  | 112 | + | 
|  | 113 | +### Test Organization | 
|  | 114 | + | 
|  | 115 | +- Tests mirror the source structure under `tests/` | 
|  | 116 | +- Integration tests use real pipeline/module examples | 
|  | 117 | +- Snapshot testing for CLI output with pytest-textual-snapshot | 
|  | 118 | +- Workflow testing with pytest-workflow for pipeline execution | 
|  | 119 | + | 
|  | 120 | +### Test Data | 
|  | 121 | + | 
|  | 122 | +- `tests/data/` - Mock configurations and test data | 
|  | 123 | +- `tests/fixtures/` - Reusable test fixtures | 
|  | 124 | +- Pipeline templates and module examples for integration testing | 
|  | 125 | + | 
|  | 126 | +### Key Testing Patterns | 
|  | 127 | + | 
|  | 128 | +- Use `@pytest.mark.datafiles` for tests requiring file fixtures | 
|  | 129 | +- Snapshot tests for CLI output verification | 
|  | 130 | +- Mock external dependencies (GitHub API, Docker registry) | 
|  | 131 | +- Parameterized tests for multiple scenarios | 
|  | 132 | + | 
|  | 133 | +## Important Implementation Details | 
|  | 134 | + | 
|  | 135 | +### Configuration Management | 
|  | 136 | + | 
|  | 137 | +- Global configuration in `~/.nfcore/` directory | 
|  | 138 | +- Rich console output with color support | 
|  | 139 | +- Environment variable support with `NFCORE_` prefix | 
|  | 140 | +- Logging configuration with file and console handlers | 
|  | 141 | + | 
|  | 142 | +### Error Handling | 
|  | 143 | + | 
|  | 144 | +- Custom exception hierarchy for different error types | 
|  | 145 | +- Selective traceback display for user-friendly error messages | 
|  | 146 | +- Graceful degradation for network failures | 
|  | 147 | + | 
|  | 148 | +### Performance Considerations | 
|  | 149 | + | 
|  | 150 | +- Parallel processing for download operations | 
|  | 151 | +- Caching for remote repository operations | 
|  | 152 | +- Progress bars for long-running operations | 
|  | 153 | +- Lazy loading of large data structures | 
|  | 154 | + | 
|  | 155 | +## Development Notes | 
|  | 156 | + | 
|  | 157 | +### Adding New Commands | 
|  | 158 | + | 
|  | 159 | +1. Add command function to appropriate `commands_*.py` file | 
|  | 160 | +2. Add Click decorator and options in `__main__.py` | 
|  | 161 | +3. Update command groups in `COMMAND_GROUPS` dictionary | 
|  | 162 | +4. Add comprehensive tests in `tests/` directory | 
|  | 163 | +5. Update documentation if needed | 
|  | 164 | + | 
|  | 165 | +### Lint Test Development | 
|  | 166 | + | 
|  | 167 | +1. Create new lint test module in appropriate `lint/` directory | 
|  | 168 | +2. Implement test class with `run()` method | 
|  | 169 | +3. Add test to main lint runner | 
|  | 170 | +4. Create comprehensive test cases | 
|  | 171 | +5. Update documentation with test description | 
|  | 172 | + | 
|  | 173 | +### Template Updates | 
|  | 174 | + | 
|  | 175 | +1. Modify template files in respective `-template/` directories | 
|  | 176 | +2. Test template generation with various options | 
|  | 177 | +3. Update template tests and snapshots | 
|  | 178 | +4. Consider backward compatibility for existing pipelines | 
|  | 179 | + | 
|  | 180 | +This codebase emphasizes modularity, comprehensive testing, and user experience through rich CLI interfaces and detailed error reporting. | 
0 commit comments