diff --git a/CLAUDE.md b/CLAUDE.md index c22696b637..fa18a77374 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -48,6 +48,75 @@ Pre-commit hooks automatically run: - **⚠️ MANDATORY BEFORE GIT PUSH**: `bundle exec rubocop` and fix ALL violations + ensure trailing newlines - Never run `npm` commands, only equivalent Yarn Classic ones +### Replicating CI Failures Locally + +**CRITICAL: NEVER wait for CI to verify fixes. Always replicate failures locally first.** + +#### Switch Between CI Configurations + +The project tests against two configurations: +- **Latest**: Ruby 3.4, Node 22, Shakapacker 9.3.0, React 19 (runs on all PRs) +- **Minimum**: Ruby 3.2, Node 20, Shakapacker 8.2.0, React 18 (runs only on master) + +```bash +# Check your current configuration +bin/ci-switch-config status + +# Switch to minimum dependencies (for debugging minimum CI failures) +bin/ci-switch-config minimum + +# Switch back to latest dependencies +bin/ci-switch-config latest +``` + +**See `SWITCHING_CI_CONFIGS.md` for detailed usage and troubleshooting.** + +**See `spec/dummy/TESTING_LOCALLY.md` for local testing tips and known issues.** + +#### Re-run Failed CI Jobs + +```bash +# Automatically detects and re-runs only the failed CI jobs +bin/ci-rerun-failures + +# Search recent commits for failures (when current commit is clean/in-progress) +bin/ci-rerun-failures --previous + +# Or for a specific PR number +bin/ci-rerun-failures 1964 +``` + +This script: +- ✨ **Fetches actual CI failures** from GitHub using `gh` CLI +- 🎯 **Runs only what failed** - no wasted time on passing tests +- ⏳ **Waits for in-progress CI** - offers to poll until completion +- 🔍 **Searches previous commits** - finds failures before your latest push +- 📋 **Shows you exactly what will run** before executing +- 🚀 **Maps CI jobs to local commands** automatically + +#### Run Only Failed Examples + +When RSpec tests fail, run just those specific examples: + +```bash +# Copy failure output from GitHub Actions, then: +pbpaste | bin/ci-run-failed-specs # macOS +# xclip -o | bin/ci-run-failed-specs # Linux (requires: apt install xclip) +# wl-paste | bin/ci-run-failed-specs # Wayland (requires: apt install wl-clipboard) + +# Or pass spec paths directly: +bin/ci-run-failed-specs './spec/system/integration_spec.rb[1:1:1:1]' + +# Or from a file: +bin/ci-run-failed-specs < failures.txt +``` + +This script: +- 🎯 **Runs only failing examples** - not the entire test suite +- 📋 **Parses RSpec output** - extracts spec paths automatically +- 🔄 **Deduplicates** - removes duplicate specs +- 📁 **Auto-detects directory** - runs from spec/dummy when needed + ## Changelog - **Update CHANGELOG.md for user-visible changes only** (features, bug fixes, breaking changes, deprecations, performance improvements) diff --git a/SWITCHING_CI_CONFIGS.md b/SWITCHING_CI_CONFIGS.md new file mode 100644 index 0000000000..6bd8f4293e --- /dev/null +++ b/SWITCHING_CI_CONFIGS.md @@ -0,0 +1,299 @@ +# Switching Between CI Configurations Locally + +This guide explains how to switch between different CI test configurations locally to replicate CI failures. + +## Quick Start + +```bash +# Check your current configuration +bin/ci-switch-config status + +# Switch to minimum dependencies (Ruby 3.2, Node 20) +bin/ci-switch-config minimum + +# Switch back to latest dependencies (Ruby 3.4, Node 22) +bin/ci-switch-config latest +``` + +## CI Configurations + +The project runs tests against two configurations: + +### Latest (Default Development) + +- **Ruby**: 3.4 +- **Node**: 22 +- **Shakapacker**: 9.3.0 +- **React**: 19.0.0 +- **Dependencies**: Latest versions with `--frozen-lockfile` +- **When it runs**: Always on PRs and master + +### Minimum (Compatibility Testing) + +- **Ruby**: 3.2 +- **Node**: 20 +- **Shakapacker**: 8.2.0 +- **React**: 18.0.0 +- **Dependencies**: Minimum supported versions +- **When it runs**: Only on master branch + +## When to Switch Configurations + +**Switch to minimum when:** + +- CI fails on `dummy-app-integration-tests (3.2, 20, minimum)` but passes on latest +- You're debugging compatibility with older dependencies +- You want to verify minimum version support before releasing + +**Switch to latest when:** + +- You're done testing minimum configuration +- You want to return to normal development +- CI failures are on latest configuration + +## Prerequisites + +You must have a version manager like [mise](https://mise.jdx.dev/) (recommended) or [asdf](https://asdf-vm.com/) installed to manage Ruby and Node versions. + +```bash +# Install mise (recommended, modern alternative to asdf) +brew install mise +echo 'eval "$(mise activate zsh)"' >> ~/.zshrc +source ~/.zshrc + +# OR install asdf (legacy option) +brew install asdf +echo -e "\n. $(brew --prefix asdf)/libexec/asdf.sh" >> ~/.zshrc +source ~/.zshrc + +# Install plugins (only needed for asdf, mise reads from mise.toml) +asdf plugin add ruby +asdf plugin add nodejs +``` + +## Detailed Usage + +### 1. Check Current Configuration + +```bash +bin/ci-switch-config status +``` + +This shows: + +- Current Ruby and Node versions +- Dependency versions (Shakapacker, React) +- Which configuration you're currently on + +### 2. Switch to Minimum Configuration + +```bash +bin/ci-switch-config minimum +``` + +This will: + +1. Create `.tool-versions` with Ruby 3.2.8 and Node 20.18.1 +2. Run `script/convert` to downgrade dependencies: + - Shakapacker 9.3.0 → 8.2.0 + - React 19.0.0 → 18.0.0 + - Remove ESLint and other packages incompatible with Node 20 +3. Clean `node_modules` and `yarn.lock` +4. Reinstall dependencies without `--frozen-lockfile` +5. Clean and reinstall spec/dummy dependencies + +**After switching, run:** + +```bash +# Reload your shell to pick up new Ruby/Node versions +cd +mise current # or: asdf current + +# Build and test +rake node_package +cd spec/dummy +bin/shakapacker-precompile-hook +RAILS_ENV=test bin/shakapacker +cd ../.. +bundle exec rake run_rspec:all_dummy +``` + +### 3. Switch Back to Latest Configuration + +```bash +bin/ci-switch-config latest +``` + +This will: + +1. Create `.tool-versions` with Ruby 3.4.3 and Node 22.12.0 +2. Restore files from git (reverting changes made by `script/convert`) +3. Clean `node_modules` and `yarn.lock` +4. Reinstall dependencies with `--frozen-lockfile` +5. Clean and reinstall spec/dummy dependencies + +**After switching, run:** + +```bash +# Reload your shell to pick up new Ruby/Node versions +cd +mise current # or: asdf current + +# Build and test +rake node_package +cd spec/dummy +bin/shakapacker-precompile-hook +RAILS_ENV=test bin/shakapacker +cd ../.. +bundle exec rake run_rspec:all_dummy +``` + +## What Gets Modified + +When switching to **minimum**, these files are modified: + +- `.tool-versions` - Ruby/Node versions +- `Gemfile.development_dependencies` - Shakapacker gem version +- `package.json` - React versions, dev dependencies removed +- `spec/dummy/package.json` - React and Shakapacker versions +- `packages/react-on-rails-pro/package.json` - Test scripts modified +- `node_modules/`, `yarn.lock` - Cleaned and regenerated +- `spec/dummy/node_modules/`, `spec/dummy/yarn.lock` - Cleaned and regenerated + +When switching to **latest**, these files are restored from git. + +## Common Workflows + +### Debugging a Minimum Config CI Failure + +```bash +# 1. Check current config +bin/ci-switch-config status + +# 2. Switch to minimum +bin/ci-switch-config minimum + +# 3. Reload shell +cd + +# 4. Verify versions changed +ruby --version # Should show 3.2.x +node --version # Should show v20.x + +# 5. Build and test +rake node_package +cd spec/dummy +bin/shakapacker-precompile-hook +RAILS_ENV=test bin/shakapacker +cd ../.. + +# 6. Run the failing tests +bundle exec rake run_rspec:all_dummy + +# 7. Fix the issue + +# 8. Switch back when done +bin/ci-switch-config latest +``` + +### Quick Test in Both Configurations + +```bash +# Test in latest (current default) +bin/ci-switch-config status +bundle exec rake run_rspec:all_dummy + +# Switch and test in minimum +bin/ci-switch-config minimum +rake node_package +cd spec/dummy && bin/shakapacker-precompile-hook && RAILS_ENV=test bin/shakapacker && cd ../.. +bundle exec rake run_rspec:all_dummy + +# Switch back +bin/ci-switch-config latest +``` + +## Troubleshooting + +### "No version is set for ruby" or version didn't change + +After switching, you need to reload your shell: + +```bash +cd +# The cd command will trigger mise/asdf to load the new versions +ruby --version # Verify it changed +``` + +### Ruby/Node version didn't change + +If your version manager doesn't automatically switch: + +**For mise:** + +```bash +mise install # Install missing versions from mise.toml or .tool-versions +``` + +**For asdf:** + +```bash +asdf install # Install missing versions from .tool-versions +asdf reshim ruby +asdf reshim nodejs +``` + +### Yarn install fails + +If you get package resolution errors: + +```bash +# Clean everything and try again +rm -rf node_modules yarn.lock spec/dummy/node_modules spec/dummy/yarn.lock +yarn install +cd spec/dummy && yarn install +``` + +### Git complains about modified files + +The script will warn you if you have uncommitted changes. You can: + +- Commit or stash your changes first, OR +- Proceed (script will ask for confirmation) + +### Switching back doesn't restore everything + +If `git restore` doesn't work: + +```bash +# Manually restore from git +git restore Gemfile.development_dependencies package.json spec/dummy/package.json packages/react-on-rails-pro/package.json + +# Then run latest again +bin/ci-switch-config latest +``` + +## Integration with CI Debugging Tools + +This script works well with the other CI debugging tools: + +```bash +# 1. Check what failed in CI +bin/ci-rerun-failures + +# 2. If it's a minimum config failure, switch +bin/ci-switch-config minimum + +# 3. Run the specific failing tests +pbpaste | bin/ci-run-failed-specs + +# 4. Switch back when done +bin/ci-switch-config latest +``` + +## See Also + +- `CLAUDE.md` - Main development guide with CI debugging info +- `bin/ci-rerun-failures` - Re-run only failed CI jobs locally +- `bin/ci-run-failed-specs` - Run specific failing RSpec examples +- `bin/ci-local` - Smart test detection based on changes diff --git a/bin/ci-rerun-failures b/bin/ci-rerun-failures new file mode 100755 index 0000000000..0439e51475 --- /dev/null +++ b/bin/ci-rerun-failures @@ -0,0 +1,287 @@ +#!/usr/bin/env bash +# CI Failure Re-runner +# Automatically detects failed CI jobs and re-runs them locally +# Usage: +# bin/ci-rerun-failures # Check current commit +# bin/ci-rerun-failures --previous # Search recent commits for failures +# bin/ci-rerun-failures 1964 # Check specific PR + +set -euo pipefail + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +# Show help +show_help() { + cat << EOF +${BLUE}CI Failure Re-runner${NC} + +Automatically detects failed CI jobs from GitHub and re-runs them locally. + +${YELLOW}Usage:${NC} + bin/ci-rerun-failures [OPTIONS] [PR_NUMBER] + +${YELLOW}Options:${NC} + --previous, --prev, -p Search recent commits for failures (when current is clean) + --help, -h Show this help message + +${YELLOW}Examples:${NC} + bin/ci-rerun-failures # Check current commit + bin/ci-rerun-failures --previous # Find failures in recent commits + bin/ci-rerun-failures 1964 # Check specific PR number + +${YELLOW}Features:${NC} + - Fetches actual CI failures from GitHub using gh CLI + - Waits for in-progress CI jobs (polls every 30s) + - Maps CI job names to local rake commands + - Deduplicates commands + - Shows what will run before executing + +${YELLOW}Related Tools:${NC} + bin/ci-run-failed-specs Run specific failing RSpec examples + bin/ci-local Smart test detection based on code changes +EOF + exit 0 +} + +# Parse arguments +USE_PREVIOUS=false +PR_NUMBER="" + +while [[ $# -gt 0 ]]; do + case $1 in + --help|-h) + show_help + ;; + --previous|--prev|-p) + USE_PREVIOUS=true + shift + ;; + *) + PR_NUMBER="$1" + shift + ;; + esac +done + +echo -e "${BLUE}=== CI Failure Re-runner ===${NC}" +echo "" + +# Check required dependencies +MISSING_DEPS=() +command -v gh >/dev/null 2>&1 || MISSING_DEPS+=("gh (GitHub CLI)") +command -v jq >/dev/null 2>&1 || MISSING_DEPS+=("jq") + +if [ ${#MISSING_DEPS[@]} -gt 0 ]; then + echo -e "${RED}Error: Missing required dependencies:${NC}" + for dep in "${MISSING_DEPS[@]}"; do + echo " - $dep" + done + echo "" + echo "Install with:" + echo " brew install gh jq" + exit 1 +fi + +# Fetch PR info +if [ -z "$PR_NUMBER" ]; then + if ! gh pr view --json number,commits >/dev/null 2>&1; then + echo -e "${RED}Error: Not on a PR branch or gh CLI not authenticated${NC}" + echo "Usage: bin/ci-rerun-failures [--previous] [pr-number]" + exit 1 + fi + PR_INFO=$(gh pr view --json number,commits) + PR_NUMBER=$(echo "$PR_INFO" | jq -r '.number') +else + if ! gh pr view "$PR_NUMBER" --json number,commits >/dev/null 2>&1; then + echo -e "${RED}Error: PR #$PR_NUMBER not found${NC}" + exit 1 + fi + PR_INFO=$(gh pr view "$PR_NUMBER" --json number,commits) +fi + +# Determine which commit to check +if [ "$USE_PREVIOUS" = true ]; then + # Show recent commits and let user see which had failures + echo -e "${BLUE}Recent commits on this PR:${NC}" + echo "$PR_INFO" | jq -r '.commits[-5:] | reverse | .[] | " \(.oid[0:8]) - \(.messageHeadline)"' + echo "" + + # Check each commit from newest to oldest for failures + FOUND_FAILURES=false + for i in 2 3 4 5; do + COMMIT_SHA=$(echo "$PR_INFO" | jq -r ".commits[-$i].oid // empty") + if [ -z "$COMMIT_SHA" ]; then + continue + fi + + echo "Checking commit ${COMMIT_SHA:0:8}..." + CHECK_RUNS=$(gh api "repos/{owner}/{repo}/commits/$COMMIT_SHA/check-runs" --jq '.check_runs' 2>/dev/null || echo "[]") + FAILURE_COUNT=$(echo "$CHECK_RUNS" | jq '[.[] | select(.conclusion == "FAILURE")] | length') + + if [ "$FAILURE_COUNT" -gt 0 ]; then + echo -e "${GREEN}Found $FAILURE_COUNT failed check(s) in commit ${COMMIT_SHA:0:8}${NC}" + STATUS_JSON=$(echo "$CHECK_RUNS" | jq '{statusCheckRollup: [.[] | {name: .name, conclusion: .conclusion, status: .status}], number: '$PR_NUMBER'}') + FOUND_FAILURES=true + break + fi + done + + if [ "$FOUND_FAILURES" = false ]; then + echo -e "${YELLOW}No failures found in recent commits. Using current commit.${NC}" + STATUS_JSON=$(gh pr view "$PR_NUMBER" --json statusCheckRollup,number) + fi +else + # Use current commit (HEAD of PR) + echo "Fetching current PR status..." + STATUS_JSON=$(gh pr view "$PR_NUMBER" --json statusCheckRollup,number) +fi + +echo -e "${GREEN}Analyzing PR #$PR_NUMBER...${NC}" +echo "" + +# Check for in-progress jobs +IN_PROGRESS_COUNT=$(echo "$STATUS_JSON" | jq '[(.statusCheckRollup // [])[] | select(.status == "IN_PROGRESS")] | length') +if [ "$IN_PROGRESS_COUNT" -gt 0 ]; then + echo -e "${YELLOW}⏳ $IN_PROGRESS_COUNT CI jobs are still running...${NC}" + echo "" + read -p "Wait for CI to complete? [Y/n] " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]] || [[ -z $REPLY ]]; then + echo "Waiting for CI to complete (checking every 30 seconds)..." + while [ "$IN_PROGRESS_COUNT" -gt 0 ]; do + sleep 30 + STATUS_JSON=$(gh pr view "$PR_NUMBER" --json statusCheckRollup,number 2>/dev/null) + IN_PROGRESS_COUNT=$(echo "$STATUS_JSON" | jq '[(.statusCheckRollup // [])[] | select(.status == "IN_PROGRESS")] | length') + FAILED_COUNT=$(echo "$STATUS_JSON" | jq '[(.statusCheckRollup // [])[] | select(.conclusion == "FAILURE")] | length') + echo -e " In progress: $IN_PROGRESS_COUNT | Failed: $FAILED_COUNT" + done + echo -e "${GREEN}✓ CI completed!${NC}" + echo "" + fi +fi + +# Parse failed checks +FAILED_CHECKS=$(echo "$STATUS_JSON" | jq -r '(.statusCheckRollup // [])[] | select(.conclusion == "FAILURE") | .name') + +if [ -z "$FAILED_CHECKS" ]; then + echo -e "${GREEN}✓ No failed checks found! All CI jobs passed.${NC}" + exit 0 +fi + +echo -e "${YELLOW}Failed CI jobs:${NC}" +echo "$FAILED_CHECKS" | while read -r check; do + echo -e "${RED} ✗ $check${NC}" +done +echo "" + +# Map CI job names to local commands +declare -A JOB_MAP +JOB_MAP["lint-js-and-ruby"]="bundle exec rubocop && yarn run eslint --report-unused-disable-directives && yarn start format.listDifferent" +JOB_MAP["rspec-package-tests"]="bundle exec rake run_rspec:gem" +JOB_MAP["package-js-tests"]="yarn test" +JOB_MAP["dummy-app-integration-tests (3.4, 22, latest)"]="bundle exec rake run_rspec:all_dummy" +JOB_MAP["dummy-app-integration-tests (3.2, 20, minimum)"]="bundle exec rake run_rspec:all_dummy" +JOB_MAP["examples"]="bundle exec rake run_rspec:shakapacker_examples" + +# Track what we'll run (deduplicated) +declare -A COMMANDS_TO_RUN + +while IFS= read -r check; do + for job_name in "${!JOB_MAP[@]}"; do + if [[ "$check" == "$job_name"* ]]; then + COMMANDS_TO_RUN["${JOB_MAP[$job_name]}"]="$job_name" + break + fi + done +done <<< "$FAILED_CHECKS" + +# Check if any commands were found (handle empty array with set -u) +set +u +NUM_COMMANDS=${#COMMANDS_TO_RUN[@]} +set -u + +if [ "$NUM_COMMANDS" -eq 0 ]; then + echo -e "${YELLOW}No local equivalents found for failed jobs.${NC}" + echo "Failed jobs might be from Pro or other workflows." + echo "" + echo "You can still run common test suites:" + echo " bundle exec rake run_rspec:all_dummy # Dummy app tests" + echo " bundle exec rake run_rspec:gem # Gem tests" + echo " yarn test # JS tests" + exit 1 +fi + +echo -e "${BLUE}Will run the following commands:${NC}" +for cmd in "${!COMMANDS_TO_RUN[@]}"; do + echo -e "${BLUE} • ${COMMANDS_TO_RUN[$cmd]}:${NC} $cmd" +done +echo "" + +# Offer to show GitHub Actions logs for more specific failures +echo -e "${YELLOW}💡 Tip: For RSpec failures, you can run specific failing examples:${NC}" +echo " 1. Go to the failed job on GitHub Actions" +echo " 2. Copy the 'Failed examples:' section" +echo " 3. Run: pbpaste | bin/ci-run-failed-specs" +echo "" + +# Confirm before running +read -p "Run these tests now? [Y/n] " -n 1 -r REPLY +echo +if [[ ! "${REPLY}" =~ ^[Yy]$ ]] && [[ ! -z "${REPLY}" ]]; then + echo "Cancelled." + exit 0 +fi + +echo "" + +# Ensure dependencies +if [ ! -d "node_modules" ] || [ ! -d "vendor/bundle" ]; then + echo -e "${YELLOW}Installing dependencies...${NC}" + bundle install && yarn install + echo "" +fi + +# Run commands +FAILED_COMMANDS=() + +for cmd in "${!COMMANDS_TO_RUN[@]}"; do + job_name="${COMMANDS_TO_RUN[$cmd]}" + echo -e "${BLUE}▶ Running: $job_name${NC}" + echo -e "${BLUE}Command: $cmd${NC}" + echo "" + + # Note: Using eval here is safe because $cmd comes from predefined JOB_MAP, + # not from user input. Commands may contain shell operators like && and ||. + if eval "$cmd"; then + echo -e "${GREEN}✓ $job_name passed${NC}" + echo "" + else + echo -e "${RED}✗ $job_name failed${NC}" + echo "" + FAILED_COMMANDS+=("$job_name") + fi +done + +# Summary +echo "" +echo -e "${BLUE}=== Summary ===${NC}" + +if [ ${#FAILED_COMMANDS[@]} -eq 0 ]; then + echo -e "${GREEN}All local tests passed! ✓${NC}" + echo "" + echo "Push your changes and CI should pass." + exit 0 +else + echo -e "${RED}Some tests still failing:${NC}" + for cmd in "${FAILED_COMMANDS[@]}"; do + echo -e "${RED} ✗ $cmd${NC}" + done + echo "" + echo -e "${YELLOW}Fix these failures before pushing.${NC}" + exit 1 +fi diff --git a/bin/ci-run-failed-specs b/bin/ci-run-failed-specs new file mode 100755 index 0000000000..dfe2577f96 --- /dev/null +++ b/bin/ci-run-failed-specs @@ -0,0 +1,178 @@ +#!/usr/bin/env bash +# Run specific failed RSpec examples +# Usage: +# bin/ci-run-failed-specs './spec/system/integration_spec.rb[1:1:1:1]' './spec/system/integration_spec.rb[10:1]' +# +# Or pipe from GitHub Actions output: +# pbpaste | bin/ci-run-failed-specs +# +# Or read from rspec failure output file: +# bin/ci-run-failed-specs < failures.txt + +set -euo pipefail + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +# Check required dependencies +MISSING_DEPS=() +command -v bundle >/dev/null 2>&1 || MISSING_DEPS+=("bundle (Ruby bundler)") + +if [ ${#MISSING_DEPS[@]} -gt 0 ]; then + echo -e "${RED}Error: Missing required dependencies:${NC}" + for dep in "${MISSING_DEPS[@]}"; do + echo " - $dep" + done + echo "" + echo "Install with:" + echo " gem install bundler" + exit 1 +fi + +# Show help +show_help() { + cat << EOF +${BLUE}Failed Spec Runner${NC} + +Run only the specific RSpec examples that failed in CI. + +${YELLOW}Usage:${NC} + bin/ci-run-failed-specs [OPTIONS] [SPEC_PATHS...] + pbpaste | bin/ci-run-failed-specs + bin/ci-run-failed-specs < failures.txt + +${YELLOW}Options:${NC} + --help, -h Show this help message + +${YELLOW}Examples:${NC} + # Pass spec paths directly + bin/ci-run-failed-specs './spec/system/integration_spec.rb[1:1:1:1]' + + # Copy from GitHub Actions, paste to run + pbpaste | bin/ci-run-failed-specs + + # From file + bin/ci-run-failed-specs < failures.txt + + # Multiple specs + bin/ci-run-failed-specs './spec/system/integration_spec.rb[1:1:1:1]' './spec/system/integration_spec.rb[10:1]' + +${YELLOW}Features:${NC} + - Parses RSpec failure output automatically + - Extracts spec paths from "rspec ./spec/..." lines + - Deduplicates specs + - Auto-detects working directory (runs from spec/dummy when needed) + - Confirms before running + +${YELLOW}Workflow:${NC} + 1. Go to failed job on GitHub Actions + 2. Copy the "Failed examples:" section + 3. Run: pbpaste | bin/ci-run-failed-specs + 4. Tests run only for those specific failures + +${YELLOW}Related Tools:${NC} + bin/ci-rerun-failures Auto-detect and re-run failed CI jobs + bin/ci-local Smart test detection based on code changes +EOF + exit 0 +} + +echo -e "${BLUE}=== Failed Spec Runner ===${NC}" +echo "" + +# Check for help flag +if [[ "${1:-}" == "--help" ]] || [[ "${1:-}" == "-h" ]]; then + show_help +fi + +# Collect spec paths +SPECS=() + +# If arguments provided, use them +if [ $# -gt 0 ]; then + SPECS=("$@") +else + # Read from stdin (piped or redirected input) + echo "Reading failed specs from stdin..." + echo "Paste RSpec failure output (Ctrl+D when done):" + + while IFS= read -r line; do + # Extract rspec commands from lines like "rspec ./spec/foo.rb[1:2:3]" + if [[ "$line" =~ rspec[[:space:]]+(\./spec/[^[:space:]]+) ]]; then + spec="${BASH_REMATCH[1]}" + SPECS+=("$spec") + fi + done +fi + +if [ ${#SPECS[@]} -eq 0 ]; then + echo -e "${RED}No specs found!${NC}" + echo "" + echo "Usage examples:" + echo " bin/ci-run-failed-specs './spec/system/integration_spec.rb[1:1:1:1]'" + echo " pbpaste | bin/ci-run-failed-specs" + echo " bin/ci-run-failed-specs < failures.txt" + exit 1 +fi + +# Deduplicate specs +UNIQUE_SPECS=($(printf '%s\n' "${SPECS[@]}" | sort -u)) + +echo -e "${GREEN}Found ${#UNIQUE_SPECS[@]} unique failing spec(s):${NC}" +for spec in "${UNIQUE_SPECS[@]}"; do + echo -e " ${YELLOW}✗${NC} $spec" +done +echo "" + +# Determine the working directory (check if we need to be in spec/dummy) +WORKING_DIR="." +if [[ "${UNIQUE_SPECS[0]}" == *"spec/system"* ]] || [[ "${UNIQUE_SPECS[0]}" == *"spec/helpers"* ]]; then + if [ -d "spec/dummy" ]; then + WORKING_DIR="spec/dummy" + echo -e "${BLUE}Running from spec/dummy directory${NC}" + fi +fi + +# Build rspec command as array (safer than eval) +RSPEC_CMD=("bundle" "exec" "rspec") +for spec in "${UNIQUE_SPECS[@]}"; do + RSPEC_CMD+=("$spec") +done + +# Display command for user +DISPLAY_CMD="bundle exec rspec" +for spec in "${UNIQUE_SPECS[@]}"; do + DISPLAY_CMD="$DISPLAY_CMD '$spec'" +done +echo -e "${BLUE}Command:${NC} cd $WORKING_DIR && $DISPLAY_CMD" +echo "" + +# Confirm (read from /dev/tty to handle piped input) +if [ -t 0 ]; then + read -p "Run these specs now? [Y/n] " -n 1 -r REPLY +else + read -p "Run these specs now? [Y/n] " -n 1 -r REPLY < /dev/tty +fi +echo +if [[ ! "${REPLY}" =~ ^[Yy]$ ]] && [[ ! -z "${REPLY}" ]]; then + echo "Cancelled." + exit 0 +fi + +# Run the specs directly (no eval - safer) +cd "$WORKING_DIR" +"${RSPEC_CMD[@]}" +RESULT=$? + +echo "" +if [ $RESULT -eq 0 ]; then + echo -e "${GREEN}✓ All specs passed!${NC}" +else + echo -e "${RED}✗ Some specs failed (exit code: $RESULT)${NC}" +fi + +exit $RESULT diff --git a/bin/ci-switch-config b/bin/ci-switch-config new file mode 100755 index 0000000000..18a90d9851 --- /dev/null +++ b/bin/ci-switch-config @@ -0,0 +1,289 @@ +#!/usr/bin/env bash +# Switch between CI test configurations locally +# +# Usage: +# bin/ci-switch-config minimum # Ruby 3.2, Node 20, minimum deps +# bin/ci-switch-config latest # Ruby 3.4, Node 22, latest deps +# bin/ci-switch-config status # Show current configuration + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" + +# Color codes for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +print_header() { + echo -e "${BLUE}==>${NC} $1" +} + +print_success() { + echo -e "${GREEN}✓${NC} $1" +} + +print_warning() { + echo -e "${YELLOW}⚠${NC} $1" +} + +print_error() { + echo -e "${RED}✗${NC} $1" +} + +check_version_manager() { + if command -v mise &> /dev/null; then + echo "mise" + elif command -v asdf &> /dev/null; then + echo "asdf" + else + print_error "No version manager found. Please install mise or asdf:" + echo " mise (recommended): https://mise.jdx.dev/" + echo " asdf (legacy): https://asdf-vm.com/" + exit 1 + fi +} + +show_status() { + print_header "Current Configuration" + echo "" + + echo "Runtime versions:" + if command -v ruby &> /dev/null; then + echo " Ruby: $(ruby --version | awk '{print $2}')" + else + echo " Ruby: not found" + fi + + if command -v node &> /dev/null; then + echo " Node: $(node --version)" + else + echo " Node: not found" + fi + + echo "" + echo "Dependency versions:" + + SHAKAPACKER_GEM="" + if [ -f "$PROJECT_ROOT/Gemfile.development_dependencies" ]; then + SHAKAPACKER_GEM=$(grep 'gem "shakapacker"' "$PROJECT_ROOT/Gemfile.development_dependencies" | sed -E 's/.*"([0-9.]+)".*/\1/' || echo "") + echo " Shakapacker (gem): ${SHAKAPACKER_GEM:-not found}" + fi + + SHAKAPACKER_NPM="" + if [ -f "$PROJECT_ROOT/spec/dummy/package.json" ]; then + SHAKAPACKER_NPM=$(grep '"shakapacker"' "$PROJECT_ROOT/spec/dummy/package.json" | sed -E 's/.*"([0-9.]+)".*/\1/' || echo "") + echo " Shakapacker (npm): ${SHAKAPACKER_NPM:-not found}" + fi + + REACT_ROOT="" + if [ -f "$PROJECT_ROOT/package.json" ]; then + REACT_ROOT=$(grep '"react":' "$PROJECT_ROOT/package.json" | head -1 | sed -E 's/.*"([^"]+)".*/\1/' || echo "") + echo " React (root): ${REACT_ROOT:-not found}" + fi + + REACT_DUMMY="" + if [ -f "$PROJECT_ROOT/spec/dummy/package.json" ]; then + REACT_DUMMY=$(grep '"react":' "$PROJECT_ROOT/spec/dummy/package.json" | sed -E 's/.*"([^"]+)".*/\1/' || echo "") + echo " React (dummy): ${REACT_DUMMY:-not found}" + fi + + echo "" + if [[ "${SHAKAPACKER_GEM}" == "8.2.0" ]] && [[ "${REACT_ROOT}" == "18.0.0" ]]; then + echo -e "Current config: ${GREEN}minimum${NC} (matches CI: Ruby 3.2, Node 20, minimum deps)" + elif [[ "${SHAKAPACKER_GEM}" == "9.3.0" ]] && [[ "${REACT_ROOT}" =~ 19 ]]; then + echo -e "Current config: ${GREEN}latest${NC} (matches CI: Ruby 3.4, Node 22, latest deps)" + else + echo -e "Current config: ${YELLOW}unknown/mixed${NC}" + fi +} + +switch_to_minimum() { + local VERSION_MANAGER=$(check_version_manager) + + print_header "Switching to MINIMUM configuration" + echo " Target: Ruby 3.2, Node 20, Shakapacker 8.2.0, React 18.0.0" + echo " Using version manager: $VERSION_MANAGER" + echo "" + + # Check if we have git changes + if ! git diff --quiet || ! git diff --cached --quiet; then + print_warning "You have uncommitted changes. This script will modify files." + read -p "Continue? (y/N) " -n 1 -r REPLY + echo + if [[ ! "${REPLY}" =~ ^[Yy]$ ]]; then + echo "Aborted." + exit 1 + fi + fi + + # Set Ruby and Node versions + print_header "Setting runtime versions in .tool-versions" + cat > "$PROJECT_ROOT/.tool-versions" < "$PROJECT_ROOT/.tool-versions" </dev/null || true + print_success "Files restored from git" + + # Clean and reinstall + print_header "Cleaning node_modules and reinstalling" + rm -rf node_modules + if [ -f yarn.lock ]; then + rm yarn.lock + fi + + print_header "Installing root dependencies with --frozen-lockfile" + yarn install --frozen-lockfile --no-progress + + print_header "Cleaning spec/dummy and reinstalling" + cd spec/dummy + rm -rf node_modules vendor/bundle + if [ -f yarn.lock ]; then + rm yarn.lock + fi + + yarn install --frozen-lockfile --no-progress + bundle install --path=vendor/bundle + + print_success "Dependencies installed" + + # Reload version manager to pick up new versions + print_header "Reloading $VERSION_MANAGER to use new versions" + if [[ "$VERSION_MANAGER" == "mise" ]]; then + # mise will auto-detect .tool-versions on next cd + : + elif [ -f "$HOME/.asdf/asdf.sh" ]; then + source "$HOME/.asdf/asdf.sh" + fi + + echo "" + print_success "Restored to LATEST configuration" + print_warning "Run these commands to reload your shell and verify:" + echo " cd $PROJECT_ROOT" + if [[ "$VERSION_MANAGER" == "mise" ]]; then + echo " mise current" + else + echo " asdf current" + fi + echo "" + print_warning "Next steps to build and test:" + echo " rake node_package" + echo " cd spec/dummy && bin/shakapacker-precompile-hook && RAILS_ENV=test bin/shakapacker" + echo " bundle exec rake run_rspec:all_dummy" +} + +# Main script +case "${1:-}" in + minimum) + switch_to_minimum + ;; + latest) + restore_to_latest + ;; + status) + show_status + ;; + *) + echo "Usage: $0 {minimum|latest|status}" + echo "" + echo "Commands:" + echo " minimum - Switch to Ruby 3.2, Node 20, minimum dependencies (Shakapacker 8.2.0, React 18)" + echo " latest - Switch to Ruby 3.4, Node 22, latest dependencies (Shakapacker 9.3.0, React 19)" + echo " status - Show current configuration" + echo "" + echo "Examples:" + echo " $0 status # Check current config" + echo " $0 minimum # Switch to minimum deps for testing CI failures" + echo " $0 latest # Switch back to latest deps" + exit 1 + ;; +esac diff --git a/spec/dummy/TESTING_LOCALLY.md b/spec/dummy/TESTING_LOCALLY.md new file mode 100644 index 0000000000..c4dfb87e8e --- /dev/null +++ b/spec/dummy/TESTING_LOCALLY.md @@ -0,0 +1,73 @@ +# Testing Locally + +## Known Issues with Ruby 3.4.3 + OpenSSL 3.6 + +If you're running Ruby 3.4.3 with OpenSSL 3.6+, you may encounter SSL certificate verification errors in system tests: + +``` +SSL_connect returned=1 errno=0 peeraddr=185.199.108.153:443 state=error: +certificate verify failed (unable to get certificate CRL) +``` + +This is caused by OpenSSL 3.6's stricter CRL (Certificate Revocation List) checking when tests access external resources like GitHub Pages. + +### Workaround + +The SSL errors don't indicate issues with the code being tested - they're environment-specific. The attempted fix in `spec/rails_helper.rb` sets `OPENSSL_CONF`, but this doesn't affect all Ruby networking code. + +### Recommendation + +**Use CI as the source of truth for system tests**. These SSL issues don't occur in CI environments: + +- CI uses containerized environments with compatible OpenSSL versions +- Local environment issues (SSL, certificates, Rack 3 compat) don't affect CI + +### What You Can Test Locally + +✅ **Unit tests** - Run reliably: + +```bash +bundle exec rspec spec/react_on_rails/ +``` + +✅ **Helper tests** - Run reliably: + +```bash +bundle exec rspec spec/helpers/ +``` + +✅ **Gem-only tests** - Skip system tests: + +```bash +bundle exec rake run_rspec:gem +``` + +❌ **System tests** - May fail with SSL errors on Ruby 3.4.3 + OpenSSL 3.6 + +### Solution: Use Ruby 3.2 (Recommended) + +The easiest fix is to switch to Ruby 3.2, which CI also uses: + +```bash +# If using mise/rtx +mise use ruby@3.2 + +# Then reinstall dependencies +bundle install + +# Run system tests +cd spec/dummy +bundle exec rspec spec/system/integration_spec.rb +``` + +Ruby 3.2 doesn't have the OpenSSL 3.6 compatibility issues and matches the CI environment more closely. + +### Alternative Solutions + +If you need to run system tests locally but want to stay on Ruby 3.4: + +1. **Use Ruby 3.4 with OpenSSL 3.3** - Requires recompiling Ruby with an older OpenSSL +2. **Or rely on CI** for system test verification +3. **Focus local testing** on unit/integration tests that don't require browser automation + +This issue is tracked in: https://github.com/openssl/openssl/issues/20385