|
| 1 | +# GitHub Actions Workflows |
| 2 | + |
| 3 | +This directory contains the CI/CD workflows for the DSBulk project. |
| 4 | + |
| 5 | +## Workflows Overview |
| 6 | + |
| 7 | +### 1. Build and Test (`build.yml`) |
| 8 | +**Triggers**: Push to main/master, Pull Requests, Manual dispatch |
| 9 | + |
| 10 | +**Purpose**: Primary CI workflow for fast feedback on code changes |
| 11 | + |
| 12 | +**What it does**: |
| 13 | +- Builds the project with JDK 8 and 11 |
| 14 | +- Runs unit tests |
| 15 | +- Publishes test results |
| 16 | +- Caches Maven dependencies for faster builds |
| 17 | + |
| 18 | +**Duration**: ~5-10 minutes |
| 19 | + |
| 20 | +### 2. Integration Tests (`integration-tests.yml`) |
| 21 | +**Triggers**: Pull Requests, Manual dispatch |
| 22 | + |
| 23 | +**Purpose**: Comprehensive integration testing with Docker-based Cassandra/DSE |
| 24 | + |
| 25 | +**What it does**: |
| 26 | +- Tests against Cassandra 3.11, 4.0, 4.1, 5.0 |
| 27 | +- Tests against DSE 5.1.48, 6.8.61, 6.9.17 |
| 28 | +- Uses Docker service containers (no CCM required) |
| 29 | +- Supports medium and long test profiles |
| 30 | +- Publishes detailed test results |
| 31 | + |
| 32 | +**Duration**: ~15-30 minutes per matrix job |
| 33 | + |
| 34 | +**Manual Trigger Options**: |
| 35 | +- `test_profile`: Choose between `medium` (default) or `long` test profiles |
| 36 | + |
| 37 | +### 3. Release (`release.yml`) |
| 38 | +**Triggers**: Git tags (v*), Manual dispatch |
| 39 | + |
| 40 | +**Purpose**: Build and publish release artifacts |
| 41 | + |
| 42 | +**What it does**: |
| 43 | +- Builds with release profile |
| 44 | +- Runs full test suite (medium + long) |
| 45 | +- Generates distribution artifacts: |
| 46 | + - `dsbulk-{version}.tar.gz` (Linux/Mac) |
| 47 | + - `dsbulk-{version}.zip` (Windows) |
| 48 | + - `dsbulk-{version}.jar` (Standalone) |
| 49 | +- Creates SHA256 checksums |
| 50 | +- Uploads to GitHub Releases |
| 51 | +- Generates release notes |
| 52 | + |
| 53 | +**Duration**: ~30-60 minutes |
| 54 | + |
| 55 | +**Manual Trigger Options**: |
| 56 | +- `version`: Specify version number (e.g., "1.11.1") |
| 57 | + |
| 58 | +**Creating a Release**: |
| 59 | +```bash |
| 60 | +# Tag and push |
| 61 | +git tag -a v1.11.1 -m "Release 1.11.1" |
| 62 | +git push origin v1.11.1 |
| 63 | + |
| 64 | +# Or trigger manually via GitHub UI |
| 65 | +``` |
| 66 | + |
| 67 | +### 4. Nightly Build (`nightly.yml`) |
| 68 | +**Triggers**: Scheduled (Mon-Fri at 6 AM UTC), Manual dispatch |
| 69 | + |
| 70 | +**Purpose**: Comprehensive nightly testing across full matrix |
| 71 | + |
| 72 | +**What it does**: |
| 73 | +- Runs full matrix tests (all Cassandra + DSE versions) |
| 74 | +- Executes long-running test profiles |
| 75 | +- Generates distribution artifacts |
| 76 | +- Creates build summary |
| 77 | +- Optional Slack notifications |
| 78 | + |
| 79 | +**Duration**: ~1-2 hours |
| 80 | + |
| 81 | +**Manual Trigger Options**: |
| 82 | +- `generate_artifacts`: Enable/disable artifact generation (default: true) |
| 83 | + |
| 84 | +### 5. Code Quality (`code-quality.yml`) |
| 85 | +**Triggers**: Push to main/master, Pull Requests, Manual dispatch |
| 86 | + |
| 87 | +**Purpose**: Code quality checks and coverage reporting |
| 88 | + |
| 89 | +**What it does**: |
| 90 | +- Generates JaCoCo code coverage reports |
| 91 | +- Uploads coverage to Codecov |
| 92 | +- Checks code formatting (fmt-maven-plugin) |
| 93 | +- Validates license headers |
| 94 | +- Runs SpotBugs analysis |
| 95 | +- Checks for dependency vulnerabilities |
| 96 | +- Posts coverage comments on PRs |
| 97 | + |
| 98 | +**Duration**: ~10-15 minutes |
| 99 | + |
| 100 | +## Workflow Dependencies |
| 101 | + |
| 102 | +``` |
| 103 | +build.yml (fast feedback) |
| 104 | + ↓ |
| 105 | +integration-tests.yml (comprehensive testing) |
| 106 | + ↓ |
| 107 | +code-quality.yml (quality checks) |
| 108 | + ↓ |
| 109 | +release.yml (on tags) or nightly.yml (scheduled) |
| 110 | +``` |
| 111 | + |
| 112 | +## Docker Images Used |
| 113 | + |
| 114 | +### Cassandra (Public - Docker Hub) |
| 115 | +- `cassandra:3.11` - Latest 3.11.x |
| 116 | +- `cassandra:4.0` - Latest 4.0.x |
| 117 | +- `cassandra:4.1` - Latest 4.1.x |
| 118 | +- `cassandra:5.0` - Latest 5.0.x |
| 119 | + |
| 120 | +### DataStax Enterprise (Public - Docker Hub) |
| 121 | +- `datastax/dse-server:5.1.35` |
| 122 | +- `datastax/dse-server:6.8.49` |
| 123 | +- `datastax/dse-server:6.9.0` |
| 124 | + |
| 125 | +**Note**: All images are publicly available - no credentials required! |
| 126 | + |
| 127 | +## Secrets Configuration |
| 128 | + |
| 129 | +### Required Secrets |
| 130 | +None! All dependencies are available via Maven Central, and Docker images are public. |
| 131 | + |
| 132 | +### Optional Secrets |
| 133 | +- `SLACK_WEBHOOK_URL` - For Slack notifications (nightly builds) |
| 134 | +- `GPG_PRIVATE_KEY` - For signing releases (if needed) |
| 135 | +- `GPG_PASSPHRASE` - For signing releases (if needed) |
| 136 | +- `CODECOV_TOKEN` - For Codecov integration (optional, works without it for public repos) |
| 137 | + |
| 138 | +## Manual Workflow Triggers |
| 139 | + |
| 140 | +All workflows support manual triggering via GitHub UI: |
| 141 | + |
| 142 | +1. Go to **Actions** tab |
| 143 | +2. Select the workflow |
| 144 | +3. Click **Run workflow** |
| 145 | +4. Choose branch and options |
| 146 | +5. Click **Run workflow** |
| 147 | + |
| 148 | +## Caching Strategy |
| 149 | + |
| 150 | +Maven dependencies are cached using `actions/cache@v4`: |
| 151 | +- **Cache key**: Based on `pom.xml` files |
| 152 | +- **Cache path**: `~/.m2/repository` |
| 153 | +- **Restore keys**: Fallback to OS-specific Maven cache |
| 154 | + |
| 155 | +This significantly speeds up builds (2-3x faster after first run). |
| 156 | + |
| 157 | +## Test Result Reporting |
| 158 | + |
| 159 | +All workflows use `EnricoMi/publish-unit-test-result-action@v2` to: |
| 160 | +- Parse JUnit XML reports |
| 161 | +- Create check runs with test results |
| 162 | +- Show pass/fail statistics |
| 163 | +- Highlight flaky tests |
| 164 | +- Comment on PRs with results |
| 165 | + |
| 166 | +## Artifact Retention |
| 167 | + |
| 168 | +| Artifact Type | Retention Period | |
| 169 | +|---------------|------------------| |
| 170 | +| Test results | 7-14 days | |
| 171 | +| Coverage reports | 30 days | |
| 172 | +| Nightly builds | 30 days | |
| 173 | +| Release artifacts | 90 days | |
| 174 | +| GitHub Releases | Permanent | |
| 175 | + |
| 176 | +## Troubleshooting |
| 177 | + |
| 178 | +### Build Failures |
| 179 | + |
| 180 | +**Maven dependency issues**: |
| 181 | +```bash |
| 182 | +# Clear cache and retry |
| 183 | +rm -rf ~/.m2/repository |
| 184 | +``` |
| 185 | + |
| 186 | +**Docker service not ready**: |
| 187 | +- Workflows include health checks and wait loops |
| 188 | +- Increase timeout if needed (currently 30 attempts × 10s = 5 minutes) |
| 189 | + |
| 190 | +**Test failures**: |
| 191 | +- Check test reports in artifacts |
| 192 | +- Review logs for specific failure reasons |
| 193 | +- CCM-dependent tests may need adaptation |
| 194 | + |
| 195 | +### Performance Issues |
| 196 | + |
| 197 | +**Slow builds**: |
| 198 | +- Check if Maven cache is working |
| 199 | +- Consider reducing test matrix for PRs |
| 200 | +- Use `workflow_dispatch` with specific options |
| 201 | + |
| 202 | +**Timeout issues**: |
| 203 | +- Default timeout: 4 hours (matching Jenkins) |
| 204 | +- Adjust in workflow file if needed |
| 205 | + |
| 206 | +## Migration from Jenkins/Travis |
| 207 | + |
| 208 | +### Key Differences |
| 209 | + |
| 210 | +| Aspect | Jenkins/Travis | GitHub Actions | |
| 211 | +|--------|---------------|----------------| |
| 212 | +| Infrastructure | CCM | Docker containers | |
| 213 | +| Cassandra versions | 2.1-3.11 | 3.11, 4.0, 4.1, 5.0 | |
| 214 | +| DSE versions | 4.7-6.8 | 5.1, 6.8, 6.9 | |
| 215 | +| Artifacts | Jenkins storage | GitHub Releases | |
| 216 | +| Secrets | Artifactory creds | None required | |
| 217 | + |
| 218 | +### Gradual Migration |
| 219 | + |
| 220 | +1. **Phase 1**: Run GitHub Actions in parallel with Jenkins |
| 221 | +2. **Phase 2**: Validate results match |
| 222 | +3. **Phase 3**: Switch primary CI to GitHub Actions |
| 223 | +4. **Phase 4**: Deprecate Jenkins/Travis |
| 224 | + |
| 225 | +## Contributing |
| 226 | + |
| 227 | +When adding new workflows: |
| 228 | +1. Follow existing naming conventions |
| 229 | +2. Add comprehensive comments |
| 230 | +3. Include manual trigger options |
| 231 | +4. Update this README |
| 232 | +5. Test thoroughly before merging |
| 233 | + |
| 234 | +## Support |
| 235 | + |
| 236 | +For issues or questions: |
| 237 | +- Check workflow logs in Actions tab |
| 238 | +- Review [ci-modernization.md](../../ci-modernization.md) for details |
| 239 | +- Open an issue with workflow run link |
0 commit comments