diff --git a/.github/actions/create-package-validation-issue/.gitignore b/.github/actions/create-package-validation-issue/.gitignore new file mode 100644 index 0000000000000..19c7c07fb5ed9 --- /dev/null +++ b/.github/actions/create-package-validation-issue/.gitignore @@ -0,0 +1,3 @@ +node_modules/ +*.log +.DS_Store \ No newline at end of file diff --git a/.github/actions/create-package-validation-issue/README.md b/.github/actions/create-package-validation-issue/README.md new file mode 100644 index 0000000000000..bd7adb51a767f --- /dev/null +++ b/.github/actions/create-package-validation-issue/README.md @@ -0,0 +1,152 @@ +# Create Package Validation Issue Action + +This GitHub Action creates or updates issues when package validation failures are detected in the Pipedream repository. + +## Features + +- โœ… **Smart Issue Management**: Creates new issues or updates existing ones for the same day +- ๐Ÿ“Š **Rich Reporting**: Includes detailed summaries, failure categories, and quick action commands +- ๐Ÿ”„ **Fallback Support**: Works with both JSON and text validation reports +- ๐Ÿท๏ธ **Auto-labeling**: Automatically applies appropriate labels for organization +- ๐Ÿ“ˆ **Failure Analysis**: Groups failures by category for easier debugging + +## Inputs + +| Input | Description | Required | Default | +|-------|-------------|----------|---------| +| `github-token` | GitHub token for API access | โœ… | - | +| `validation-report-json` | Path to JSON validation report | โœ… | `validation-report.json` | +| `validation-report-txt` | Path to text validation report | โŒ | `validation-report.txt` | +| `workflow-run-number` | GitHub workflow run number | โœ… | - | +| `workflow-run-id` | GitHub workflow run ID | โœ… | - | +| `server-url` | GitHub server URL | โœ… | - | +| `repository` | Repository name (owner/repo) | โœ… | - | + +## Outputs + +| Output | Description | Example | +|--------|-------------|---------| +| `issue-created` | Whether a new issue was created | `true`/`false` | +| `issue-updated` | Whether an existing issue was updated | `true`/`false` | +| `issue-url` | URL of the created/updated issue | `https://github.com/...` | +| `failed-count` | Number of failed packages | `42` | + +**Note**: All outputs are properly set for both regular and composite actions, supporting `core.setOutput()` and `$GITHUB_OUTPUT` file methods. + +## Usage + +```yaml +- name: Create Issue on Failures + if: steps.check_failures.outputs.failed_count != '0' + uses: ./.github/actions/create-package-validation-issue + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + validation-report-json: 'validation-report.json' + validation-report-txt: 'validation-report.txt' # optional + workflow-run-number: ${{ github.run_number }} + workflow-run-id: ${{ github.run_id }} + server-url: ${{ github.server_url }} + repository: ${{ github.repository }} +``` + +## Issue Format + +The action creates issues with: + +### ๐Ÿ“ฆ Title Format +``` +๐Ÿ“ฆ Package Validation Report - [Date] - [X] failures +``` + +### ๐Ÿ“‹ Content Sections +1. **Summary**: Overview statistics of validation results +2. **Links**: Direct links to workflow run and artifacts +3. **Failed Packages**: List of top failing packages with error types +4. **Failure Categories**: Grouped failures by validation type +5. **Next Steps**: Action items for developers +6. **Quick Commands**: Copy-paste commands for local testing + +### ๐Ÿท๏ธ Auto-applied Labels +- `package-validation`: Identifies validation-related issues +- `automated`: Marks as automatically created +- `bug`: Indicates packages needing fixes + +## Behavior + +### New Issues +- Creates a new issue if no open validation issue exists for the current date +- Applies appropriate labels automatically + +### Existing Issues +- Updates existing issues with new validation results as comments +- Avoids creating duplicate issues for the same day +- Maintains issue history through comments + +### No Failures +- Skips issue creation when no validation failures are detected +- Sets output flags appropriately for workflow logic + +## Error Handling + +- **JSON Parse Errors**: Falls back to text report parsing +- **Missing Files**: Gracefully handles missing report files +- **API Failures**: Provides detailed error messages for debugging +- **Network Issues**: Fails gracefully with actionable error messages + +## Development + +### Local Testing +```bash +# Install dependencies +cd .github/actions/create-package-validation-issue +npm install + +# Test with sample data +node test/test-action.js +``` + +### Dependencies +- `@actions/core`: GitHub Actions toolkit for inputs/outputs +- `@actions/github`: GitHub API client and context + +### Technical Notes +- **Composite Action**: Uses `composite` action type to handle dependency installation automatically +- **Auto-Install**: Dependencies are installed during action execution for reliability +- **Path Resolution**: File paths are resolved relative to the GitHub workspace +- **Fallback Support**: Gracefully handles missing files and parse errors + +## Integration + +This action is designed to work with: +- `scripts/generate-package-report.js`: Validation report generator +- `.github/workflows/scheduled-package-validation.yaml`: Scheduled validation workflow +- Pipedream component validation pipeline + +## Example Issue Output + +```markdown +# ๐Ÿ“ฆ Scheduled Package Validation Report - Mon Jan 15 2024 + +๐Ÿ“Š **Summary:** +- Total Components: 2932 +- โœ… Validated: 2847 +- โŒ Failed: 85 +- โญ๏ธ Skipped: 1250 +- ๐Ÿ“ˆ Publishable: 2932 +- ๐Ÿ“‰ Failure Rate: 2.90% + +## ๐Ÿ”— Links +- **Workflow Run**: [#123](https://github.com/org/repo/actions/runs/456) +- **Download Reports**: Check the workflow artifacts for detailed reports + +## โŒ Failed Packages +- **@pipedream/netlify** (netlify): import, dependencies +- **@pipedream/google-slides** (google_slides): mainFile +- ... and 83 more packages + +## Next Steps +1. Review the failed packages listed above +2. Check the full validation report artifact +3. Fix import/dependency issues in failing packages +... +``` \ No newline at end of file diff --git a/.github/actions/create-package-validation-issue/action.yml b/.github/actions/create-package-validation-issue/action.yml new file mode 100644 index 0000000000000..c94f71b63688c --- /dev/null +++ b/.github/actions/create-package-validation-issue/action.yml @@ -0,0 +1,69 @@ +name: 'Create Package Validation Issue' +description: 'Creates or updates GitHub issues when package validation failures are detected' +inputs: + github-token: + description: 'GitHub token for creating issues' + required: true + validation-report-json: + description: 'Path to the JSON validation report file' + required: true + default: 'validation-report.json' + validation-report-txt: + description: 'Path to the text validation report file' + required: false + default: 'validation-report.txt' + workflow-run-number: + description: 'GitHub workflow run number' + required: true + workflow-run-id: + description: 'GitHub workflow run ID' + required: true + server-url: + description: 'GitHub server URL' + required: true + repository: + description: 'Repository name in owner/repo format' + required: true + +outputs: + issue-created: + description: 'Whether a new issue was created' + value: ${{ steps.run_issue_creation.outputs.issue-created }} + issue-updated: + description: 'Whether an existing issue was updated' + value: ${{ steps.run_issue_creation.outputs.issue-updated }} + issue-url: + description: 'URL of the created/updated issue' + value: ${{ steps.run_issue_creation.outputs.issue-url }} + failed-count: + description: 'Number of failed packages' + value: ${{ steps.run_issue_creation.outputs.failed-count }} + +runs: + using: 'composite' + steps: + - name: Setup Node.js for action + uses: actions/setup-node@v4.0.3 + with: + node-version: '20' + + - name: Install action dependencies + shell: bash + run: | + cd ${{ github.action_path }} + npm install + + - name: Run issue creation + id: run_issue_creation + shell: bash + run: | + cd ${{ github.action_path }} + node src/index.js + env: + INPUT_GITHUB_TOKEN: ${{ inputs.github-token }} + INPUT_VALIDATION_REPORT_JSON: ${{ inputs.validation-report-json }} + INPUT_VALIDATION_REPORT_TXT: ${{ inputs.validation-report-txt }} + INPUT_WORKFLOW_RUN_NUMBER: ${{ inputs.workflow-run-number }} + INPUT_WORKFLOW_RUN_ID: ${{ inputs.workflow-run-id }} + INPUT_SERVER_URL: ${{ inputs.server-url }} + INPUT_REPOSITORY: ${{ inputs.repository }} \ No newline at end of file diff --git a/.github/actions/create-package-validation-issue/package.json b/.github/actions/create-package-validation-issue/package.json new file mode 100644 index 0000000000000..5dc4aa1b81386 --- /dev/null +++ b/.github/actions/create-package-validation-issue/package.json @@ -0,0 +1,21 @@ +{ + "name": "create-package-validation-issue", + "version": "1.0.0", + "description": "GitHub Action to create or update issues for package validation failures", + "main": "src/index.js", + "scripts": { + "test": "echo \"No tests yet\" && exit 0" + }, + "dependencies": { + "@actions/core": "^1.10.1", + "@actions/github": "^6.0.0" + }, + "keywords": [ + "github-action", + "pipedream", + "package-validation", + "issues" + ], + "author": "Pipedream", + "license": "MIT" +} \ No newline at end of file diff --git a/.github/actions/create-package-validation-issue/src/index.js b/.github/actions/create-package-validation-issue/src/index.js new file mode 100644 index 0000000000000..bae74bdd2fb34 --- /dev/null +++ b/.github/actions/create-package-validation-issue/src/index.js @@ -0,0 +1,281 @@ +const core = require('@actions/core'); +const github = require('@actions/github'); +const fs = require('fs'); + +async function run() { + try { + // Get inputs - handle both core.getInput and environment variables for composite actions + const githubToken = core.getInput('github-token') || process.env.INPUT_GITHUB_TOKEN; + const validationReportJson = core.getInput('validation-report-json') || process.env.INPUT_VALIDATION_REPORT_JSON; + const validationReportTxt = core.getInput('validation-report-txt') || process.env.INPUT_VALIDATION_REPORT_TXT; + const workflowRunNumber = core.getInput('workflow-run-number') || process.env.INPUT_WORKFLOW_RUN_NUMBER; + const workflowRunId = core.getInput('workflow-run-id') || process.env.INPUT_WORKFLOW_RUN_ID; + const serverUrl = core.getInput('server-url') || process.env.INPUT_SERVER_URL; + const repository = core.getInput('repository') || process.env.INPUT_REPOSITORY; + + // Validate required inputs + if (!githubToken) { + throw new Error('github-token is required'); + } + if (!validationReportJson) { + throw new Error('validation-report-json is required'); + } + if (!workflowRunNumber || !workflowRunId || !serverUrl || !repository) { + throw new Error('workflow metadata (run-number, run-id, server-url, repository) is required'); + } + + // Initialize GitHub client + const octokit = github.getOctokit(githubToken); + const context = github.context; + + // Set up repository context for composite actions + const [owner, repo] = repository.split('/'); + const repoContext = { + owner: owner || context.repo?.owner, + repo: repo || context.repo?.repo + }; + + // Read and parse validation reports + let reportData = null; + let failedCount = 0; + let summaryText = ''; + + // Resolve file paths relative to the workspace, not the action directory + const workspacePath = process.env.GITHUB_WORKSPACE || process.cwd(); + const jsonReportPath = validationReportJson.startsWith('/') + ? validationReportJson + : `${workspacePath}/${validationReportJson}`; + const txtReportPath = validationReportTxt && !validationReportTxt.startsWith('/') + ? `${workspacePath}/${validationReportTxt}` + : validationReportTxt; + + core.info(`Reading validation report from: ${jsonReportPath}`); + + try { + if (fs.existsSync(jsonReportPath)) { + const jsonReport = fs.readFileSync(jsonReportPath, 'utf8'); + reportData = JSON.parse(jsonReport); + failedCount = reportData.summary.failed; + + summaryText = ` +๐Ÿ“Š **Summary:** +- Total Components: ${reportData.summary.total} +- โœ… Validated: ${reportData.summary.validated} +- โŒ Failed: ${reportData.summary.failed} +- โญ๏ธ Skipped: ${reportData.summary.skipped} +- ๐Ÿ“ˆ Publishable: ${reportData.summary.publishable} +- ๐Ÿ“‰ Failure Rate: ${reportData.summary.failureRate}% + `; + + core.info(`Parsed JSON report: ${failedCount} failures found`); + } else { + core.warning(`JSON report file not found: ${jsonReportPath}`); + failedCount = 0; + } + } catch (error) { + core.warning(`Failed to parse JSON report: ${error.message}`); + + // Fallback to text report + try { + if (txtReportPath && fs.existsSync(txtReportPath)) { + const report = fs.readFileSync(txtReportPath, 'utf8'); + const failedPackages = report.match(/โŒ.*FAILED:/g) || []; + failedCount = failedPackages.length; + summaryText = `Failed to parse JSON report. Found ${failedCount} failures in text report.`; + core.info(`Fallback to text report: ${failedCount} failures found`); + } + } catch (txtError) { + core.error(`Failed to read text report: ${txtError.message}`); + failedCount = 0; + } + } + + // Exit early if no failures + if (failedCount === 0) { + core.info('No failures detected, skipping issue creation'); + core.setOutput('issue-created', 'false'); + core.setOutput('issue-url', ''); + return; + } + + core.info(`Processing ${failedCount} failures`); + + // Generate failed packages list + let failedPackagesList = ''; + if (reportData && reportData.failed) { + const topFailures = reportData.failed.slice(0, 10); + failedPackagesList = topFailures.map(pkg => { + const failureTypes = pkg.failures.map(f => f.check).join(', '); + return `- **${pkg.packageName}** (${pkg.app}): ${failureTypes}`; + }).join('\n'); + + if (reportData.failed.length > 10) { + failedPackagesList += `\n- ... and ${reportData.failed.length - 10} more packages`; + } + } else { + failedPackagesList = 'See full report for details.'; + } + + // Create issue body + const today = new Date().toDateString(); + const issueBody = ` +# ๐Ÿ“ฆ Scheduled Package Validation Report - ${today} + +${summaryText} + +## ๐Ÿ”— Links +- **Workflow Run**: [#${workflowRunNumber}](${serverUrl}/${repository}/actions/runs/${workflowRunId}) +- **Download Reports**: Check the workflow artifacts for detailed JSON and text reports + +## โŒ Failed Packages +${failedPackagesList} + +## ๐Ÿ“‹ Failure Categories +${reportData ? generateFailureCategoriesText(reportData) : 'Categories unavailable - check full report'} + +## Full Report +The complete validation report is available as an artifact in the workflow run. + +## Next Steps +1. Review the failed packages listed above +2. Check the full validation report artifact for detailed error messages +3. Fix import/dependency issues in failing packages +4. Consider updating package.json configurations +5. Ensure all main files have proper exports + +## ๐Ÿ”ง Quick Commands +To test a specific package locally: +\`\`\`bash +npm run validate:package -- +\`\`\` + +To run validation on all packages: +\`\`\`bash +npm run validate:packages:verbose +\`\`\` + +--- +*This issue was automatically created by the scheduled package validation workflow.* + `; + + // Check for existing open issues + core.info('Checking for existing open issues...'); + const { data: issues } = await octokit.rest.issues.listForRepo({ + owner: repoContext.owner, + repo: repoContext.repo, + labels: ['package-validation', 'automated'], + state: 'open' + }); + + const existingIssue = issues.find(issue => + issue.title.includes(today) && + issue.title.includes('Scheduled Package Validation Report') + ); + + let issueUrl = ''; + let issueCreated = false; + let issueUpdated = false; + + if (existingIssue) { + // Update existing issue with a comment + core.info(`Updating existing issue #${existingIssue.number}`); + + const comment = await octokit.rest.issues.createComment({ + owner: repoContext.owner, + repo: repoContext.repo, + issue_number: existingIssue.number, + body: `## ๐Ÿ”„ Updated Report - Run #${workflowRunNumber} + +${issueBody} + +--- +*Issue updated at ${new Date().toISOString()}*` + }); + + issueUrl = existingIssue.html_url; + issueCreated = false; + issueUpdated = true; + core.info(`Updated issue: ${issueUrl}`); + } else { + // Create new issue + core.info('Creating new issue...'); + + const newIssue = await octokit.rest.issues.create({ + owner: repoContext.owner, + repo: repoContext.repo, + title: `๐Ÿ“ฆ Scheduled Package Validation Report - ${today} - ${failedCount} failures`, + body: issueBody, + labels: ['package-validation', 'automated', 'bug'] + }); + + issueUrl = newIssue.data.html_url; + issueCreated = true; + issueUpdated = false; + core.info(`Created new issue: ${issueUrl}`); + } + + // Set outputs for both regular and composite actions + core.setOutput('issue-created', issueCreated.toString()); + core.setOutput('issue-updated', issueUpdated.toString()); + core.setOutput('issue-url', issueUrl); + core.setOutput('failed-count', failedCount.toString()); + + // For composite actions, also write to GITHUB_OUTPUT file + if (process.env.GITHUB_OUTPUT) { + const outputData = [ + `issue-url=${issueUrl}`, + `failed-count=${failedCount.toString()}`, + `issue-created=${issueCreated.toString()}`, + `issue-updated=${issueUpdated.toString()}` + ].join('\n') + '\n'; + + fs.appendFileSync(process.env.GITHUB_OUTPUT, outputData); + } + + } catch (error) { + core.setFailed(`Action failed: ${error.message}`); + console.error('Full error:', error); + } +} + +function generateFailureCategoriesText(reportData) { + if (!reportData.failed || reportData.failed.length === 0) { + return 'No failure categories to display.'; + } + + const failuresByCategory = {}; + + reportData.failed.forEach(({ packageName, failures }) => { + failures.forEach(failure => { + if (!failuresByCategory[failure.check]) { + failuresByCategory[failure.check] = []; + } + failuresByCategory[failure.check].push({ + packageName, + error: failure.error + }); + }); + }); + + let categoriesText = ''; + Object.entries(failuresByCategory).forEach(([category, failures]) => { + categoriesText += `\n### ๐Ÿ” ${category.toUpperCase()} Failures (${failures.length})\n`; + + failures.slice(0, 3).forEach(({ packageName, error }) => { + categoriesText += `- **${packageName}**: ${error}\n`; + }); + + if (failures.length > 3) { + categoriesText += `- ... and ${failures.length - 3} more\n`; + } + }); + + return categoriesText; +} + +// Run the action +if (require.main === module) { + run(); +} + +module.exports = { run }; \ No newline at end of file diff --git a/.github/actions/create-package-validation-issue/test/test-action.js b/.github/actions/create-package-validation-issue/test/test-action.js new file mode 100644 index 0000000000000..d85f801e5721b --- /dev/null +++ b/.github/actions/create-package-validation-issue/test/test-action.js @@ -0,0 +1,176 @@ +#!/usr/bin/env node + +/** + * Test script for the create-package-validation-issue action + * This helps test the action logic locally without GitHub Actions + */ + +const fs = require('fs'); +const path = require('path'); + +// Mock the GitHub Actions environment +process.env.GITHUB_REPOSITORY = 'pipedream/pipedream'; +process.env.GITHUB_SERVER_URL = 'https://github.com'; + +// Mock @actions/core +const mockCore = { + info: (message) => console.log(`[INFO] ${message}`), + warning: (message) => console.log(`[WARN] ${message}`), + error: (message) => console.log(`[ERROR] ${message}`), + setFailed: (message) => console.log(`[FAILED] ${message}`), + setOutput: (name, value) => console.log(`[OUTPUT] ${name}=${value}`), + getInput: (name) => { + const inputs = { + 'github-token': 'fake-token-for-testing', + 'validation-report-json': path.join(__dirname, 'sample-report.json'), + 'validation-report-txt': path.join(__dirname, 'sample-report.txt'), + 'workflow-run-number': '123', + 'workflow-run-id': '456', + 'server-url': 'https://github.com', + 'repository': 'pipedream/pipedream' + }; + return inputs[name] || ''; + } +}; + +// Mock @actions/github +const mockGithub = { + context: { + repo: { + owner: 'pipedream', + repo: 'pipedream' + } + }, + getOctokit: (token) => ({ + rest: { + issues: { + listForRepo: async () => ({ + data: [] // No existing issues for testing + }), + create: async (params) => { + console.log('[MOCK] Would create issue:', params.title); + return { + data: { + html_url: 'https://github.com/pipedream/pipedream/issues/123' + } + }; + }, + createComment: async (params) => { + console.log('[MOCK] Would create comment on issue:', params.issue_number); + return { data: {} }; + } + } + } + }) +}; + +// Create sample test data +const sampleReport = { + generatedAt: new Date().toISOString(), + summary: { + total: 100, + validated: 85, + failed: 15, + skipped: 50, + publishable: 100, + failureRate: '15.00' + }, + validated: [ + { app: 'netlify', packageName: '@pipedream/netlify' }, + { app: 'slack', packageName: '@pipedream/slack' } + ], + failed: [ + { + app: 'google_slides', + packageName: '@pipedream/google_slides', + failures: [ + { check: 'import', error: 'Cannot find module' }, + { check: 'dependencies', error: 'Missing @pipedream/platform' } + ] + }, + { + app: 'broken_app', + packageName: '@pipedream/broken_app', + failures: [ + { check: 'mainFile', error: 'Main file not found' } + ] + } + ], + skipped: [ + { app: 'private_app', packageName: 'private_app', reason: 'Not a @pipedream package' } + ] +}; + +const sampleTextReport = ` +๐Ÿ“Š PIPEDREAM PACKAGE VALIDATION REPORT +Generated: ${new Date().toISOString()} +Total Components: 100 +============================================================= + +โŒ google_slides (@pipedream/google_slides) - FAILED: Import test failed +โŒ broken_app (@pipedream/broken_app) - FAILED: Main file not found +โœ… netlify (@pipedream/netlify) - VALID +โœ… slack (@pipedream/slack) - VALID + +๐Ÿ“Š DETAILED VALIDATION SUMMARY +============================================================= +๐Ÿ“ฆ Total Components: 100 +โœ… Validated Successfully: 85 +โŒ Failed Validation: 15 +โญ๏ธ Skipped: 50 +๐Ÿ“ˆ Publishable Packages: 100 +๐Ÿ“‰ Failure Rate: 15.00% +`; + +// Write test files +const testDir = __dirname; +if (!fs.existsSync(testDir)) { + fs.mkdirSync(testDir, { recursive: true }); +} + +fs.writeFileSync( + path.join(testDir, 'sample-report.json'), + JSON.stringify(sampleReport, null, 2) +); + +fs.writeFileSync( + path.join(testDir, 'sample-report.txt'), + sampleTextReport +); + +// Mock the modules and run the action +const Module = require('module'); +const originalRequire = Module.prototype.require; + +Module.prototype.require = function(id) { + if (id === '@actions/core') { + return mockCore; + } + if (id === '@actions/github') { + return mockGithub; + } + return originalRequire.apply(this, arguments); +}; + +// Import and run the action +const { run } = require('../src/index.js'); + +console.log('๐Ÿงช Testing create-package-validation-issue action...\n'); + +run() + .then(() => { + console.log('\nโœ… Test completed successfully!'); + }) + .catch((error) => { + console.error('\nโŒ Test failed:', error); + process.exit(1); + }) + .finally(() => { + // Cleanup + try { + fs.unlinkSync(path.join(testDir, 'sample-report.json')); + fs.unlinkSync(path.join(testDir, 'sample-report.txt')); + } catch (e) { + // Ignore cleanup errors + } + }); \ No newline at end of file diff --git a/.github/workflows/scheduled-package-validation.yaml b/.github/workflows/scheduled-package-validation.yaml new file mode 100644 index 0000000000000..8a8d848aec767 --- /dev/null +++ b/.github/workflows/scheduled-package-validation.yaml @@ -0,0 +1,93 @@ +name: Scheduled Package Validation Report + +on: + schedule: + # Run every three days at midnight UTC + - cron: '0 0 */3 * *' + workflow_dispatch: # Allow manual triggering for testing + +jobs: + validate-packages: + name: Generate Package Validation Report + runs-on: ubuntu-latest + + steps: + - name: Checkout code + uses: actions/checkout@v4.1.7 + + - name: Setup pnpm + uses: pnpm/action-setup@v4.0.0 + with: + version: 9.14.2 + + - name: Setup Node.js + uses: actions/setup-node@v4.0.3 + with: + node-version: 18 + cache: 'pnpm' + + - name: Install dependencies + run: pnpm install -r --no-frozen-lockfile + + - name: Compile TypeScript + run: pnpm run build + + - name: Run Package Validation Report + id: validation + run: | + node scripts/generate-package-report.js --verbose --report-only --output=validation-report.json > validation-report.txt 2>&1 + echo "validation_exit_code=$?" >> $GITHUB_OUTPUT + continue-on-error: true + + - name: Upload Validation Report + uses: actions/upload-artifact@v4 + with: + name: package-validation-report-${{ github.run_number }} + path: | + validation-report.txt + validation-report.json + retention-days: 30 + + - name: Check for failures + id: check_failures + run: | + if [ -f "validation-report.json" ]; then + FAILED_COUNT=$(node -e " + const fs = require('fs'); + try { + const report = JSON.parse(fs.readFileSync('validation-report.json', 'utf8')); + console.log(report.summary.failed || 0); + } catch { + console.log(0); + } + ") + echo "failed_count=$FAILED_COUNT" >> $GITHUB_OUTPUT + else + echo "failed_count=0" >> $GITHUB_OUTPUT + fi + + - name: Create Issue on Failures + if: steps.check_failures.outputs.failed_count != '0' + id: create_issue + uses: ./.github/actions/create-package-validation-issue + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + validation-report-json: 'validation-report.json' + validation-report-txt: 'validation-report.txt' + workflow-run-number: ${{ github.run_number }} + workflow-run-id: ${{ github.run_id }} + server-url: ${{ github.server_url }} + repository: ${{ github.repository }} + + - name: Post Issue Summary + if: steps.create_issue.conclusion == 'success' && steps.create_issue.outputs.issue-url != '' + run: | + echo "๐Ÿ“‹ Issue created/updated: ${{ steps.create_issue.outputs.issue-url }}" + echo "โŒ Failed packages: ${{ steps.create_issue.outputs.failed-count }}" + echo "๐Ÿ”„ Issue was ${{ steps.create_issue.outputs.issue-created == 'true' && 'created' || 'updated' }}" + + - name: Post Success Summary + if: steps.check_failures.outputs.failed_count == '0' + run: | + echo "๐ŸŽ‰ All packages validated successfully!" + echo "Scheduled validation completed with no issues." \ No newline at end of file diff --git a/package.json b/package.json index 4f8f7b65569f5..b6ae9a57c6a3b 100644 --- a/package.json +++ b/package.json @@ -36,7 +36,12 @@ "platform-test": "cd platform && npm test", "types-test": "cd types && npm test", "helpers-test": "cd helpers && npm test", - "build:docs": "cd docs-v2 && pnpm install --ignore-engines && pnpm build" + "build:docs": "cd docs-v2 && pnpm install --ignore-engines && pnpm build", + "validate:packages": "node scripts/generate-package-report.js", + "validate:packages:verbose": "node scripts/generate-package-report.js --verbose", + "validate:packages:report": "node scripts/generate-package-report.js --report-only --output=package-validation-report.json", + "validate:packages:dry-run": "node scripts/generate-package-report.js --dry-run", + "validate:package": "node scripts/generate-package-report.js --package=" }, "devDependencies": { "@eslint/eslintrc": "^3.2.0", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index edefb2c3f70fb..a6e00927c63e9 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -2436,8 +2436,7 @@ importers: components/chatrace: {} - components/chatsistant: - specifiers: {} + components/chatsistant: {} components/chatsonic: {} @@ -11895,8 +11894,7 @@ importers: specifier: ^3.0.0 version: 3.0.3 - components/sap_s_4hana_cloud: - specifiers: {} + components/sap_s_4hana_cloud: {} components/sapling: {} @@ -14760,8 +14758,7 @@ importers: components/virustotal: {} - components/visibot: - specifiers: {} + components/visibot: {} components/vision6: dependencies: @@ -37394,8 +37391,6 @@ snapshots: '@putout/operator-filesystem': 5.0.0(putout@36.13.1(eslint@8.57.1)(typescript@5.6.3)) '@putout/operator-json': 2.2.0 putout: 36.13.1(eslint@8.57.1)(typescript@5.6.3) - transitivePeerDependencies: - - supports-color '@putout/operator-regexp@1.0.0(putout@36.13.1(eslint@8.57.1)(typescript@5.6.3))': dependencies: diff --git a/scripts/generate-package-report.js b/scripts/generate-package-report.js new file mode 100644 index 0000000000000..46e41ec9ff254 --- /dev/null +++ b/scripts/generate-package-report.js @@ -0,0 +1,580 @@ +const fs = require('fs'); +const path = require('path'); +const { execSync } = require('child_process'); + +// Parse command line arguments +const args = process.argv.slice(2); +const isReportOnly = args.includes('--report-only'); +const isVerbose = args.includes('--verbose'); +const outputFile = args.find(arg => arg.startsWith('--output='))?.split('=')[1]; +const singlePackage = args.find(arg => arg.startsWith('--package='))?.split('=')[1]; +const isDryRun = args.includes('--dry-run'); +const showHelp = args.includes('--help') || args.includes('-h'); + +// Show help if requested +if (showHelp) { + console.log(` +๐Ÿ“ฆ Pipedream Package Validation Tool + +Usage: node scripts/generate-package-report.js [options] + +Options: + --package= Validate a single package (e.g. --package=netlify) + --verbose Show detailed validation output + --dry-run Preview which packages would be validated + --report-only Generate report without exiting on failures + --output= Save JSON report to file (e.g. --output=report.json) + --help, -h Show this help message + +Examples: + # Validate all packages + node scripts/generate-package-report.js + + # Validate single package with details + node scripts/generate-package-report.js --package=netlify --verbose + + # Preview what would be validated + node scripts/generate-package-report.js --dry-run + + # Generate report file + node scripts/generate-package-report.js --report-only --output=report.json + +NPM Scripts: + npm run validate:packages # Validate all packages + npm run validate:packages:verbose # Validate with verbose output + npm run validate:packages:dry-run # Preview validation + npm run validate:package -- netlify # Validate single package +`); + process.exit(0); +} + +function generatePackageReport() { + const componentsDir = 'components'; + let apps = fs.readdirSync(componentsDir).filter(dir => { + const packagePath = path.join(componentsDir, dir, 'package.json'); + return fs.existsSync(packagePath); + }); + + // Filter to single package if specified + if (singlePackage) { + apps = apps.filter(app => app === singlePackage); + if (apps.length === 0) { + console.error(`โŒ Package '${singlePackage}' not found in components directory`); + + // Find similar package names + const allDirs = fs.readdirSync(componentsDir); + const similar = allDirs.filter(dir => + dir.toLowerCase().includes(singlePackage.toLowerCase()) || + singlePackage.toLowerCase().includes(dir.toLowerCase()) + ); + + if (similar.length > 0) { + console.log(`\n๐Ÿ’ก Did you mean one of these?`); + similar.slice(0, 5).forEach(name => { + console.log(` - ${name}`); + }); + } else { + console.log(`\n๐Ÿ“‚ Available packages (first 10): ${allDirs.slice(0, 10).join(', ')}...`); + console.log(` Total: ${allDirs.length} packages available`); + } + process.exit(1); + } + + console.log(`๐ŸŽฏ Focusing on single package: ${singlePackage}`); + } + + const results = { + validated: [], + failed: [], + skipped: [], + summary: {} + }; + + // Update header based on mode + if (singlePackage) { + console.log(`๐Ÿ“ฆ SINGLE PACKAGE VALIDATION: ${singlePackage}`); + console.log(`Generated: ${new Date().toISOString()}`); + console.log('='.repeat(60)); + console.log(); + } else if (isDryRun) { + console.log(`๐ŸŒต DRY RUN - PACKAGE VALIDATION PREVIEW`); + console.log(`Generated: ${new Date().toISOString()}`); + console.log(`Total Components: ${apps.length}`); + console.log('='.repeat(60)); + console.log(); + } else { + console.log(`๐Ÿ“Š PIPEDREAM PACKAGE VALIDATION REPORT`); + console.log(`Generated: ${new Date().toISOString()}`); + console.log(`Total Components: ${apps.length}`); + console.log('='.repeat(60)); + console.log(); + } + + for (const app of apps) { + const packagePath = path.join(componentsDir, app, 'package.json'); + let packageJson = null; + let packageName = app; + + try { + // Parse package.json + const packageJsonContent = fs.readFileSync(packagePath, 'utf8'); + packageJson = JSON.parse(packageJsonContent); + packageName = packageJson.name || `${app} (no name)`; + + // Only validate @pipedream/* packages with publishConfig + if (!packageName || !packageName.startsWith('@pipedream/')) { + results.skipped.push({ + app, + packageName, + reason: 'Not a @pipedream package', + category: 'scope' + }); + continue; + } + + if (!packageJson.publishConfig?.access) { + results.skipped.push({ + app, + packageName, + reason: 'No publishConfig.access', + category: 'config' + }); + continue; + } + + // In dry-run mode, just report what would be validated + if (isDryRun) { + console.log(`Would validate: ${packageName}`); + continue; + } + + if (isVerbose || singlePackage) { + console.log(`๐Ÿ“ฆ Validating ${packageName}...`); + } + + // Run all validations + const validationResults = { + packageJson: null, + mainFile: null, + dependencies: null, + relativeImports: null, + packageDependencies: null, + import: null + }; + + try { + validatePackageJson(packageJson, app); + validationResults.packageJson = 'passed'; + } catch (error) { + validationResults.packageJson = error.message; + } + + try { + validateMainFile(packageJson, app); + validationResults.mainFile = 'passed'; + } catch (error) { + validationResults.mainFile = error.message; + } + + try { + validateDependencies(packageJson, app); + validationResults.dependencies = 'passed'; + } catch (error) { + validationResults.dependencies = error.message; + } + + try { + validateRelativeImports(packageJson, app); + validationResults.relativeImports = 'passed'; + } catch (error) { + validationResults.relativeImports = error.message; + } + + try { + validatePackageDependencies(packageJson, app); + validationResults.packageDependencies = 'passed'; + } catch (error) { + validationResults.packageDependencies = error.message; + } + + try { + validateImport(packageName, app, packageJson); + validationResults.import = 'passed'; + } catch (error) { + validationResults.import = error.message; + } + + // Check if any validation failed + const failures = Object.entries(validationResults) + .filter(([key, value]) => value !== 'passed') + .map(([key, value]) => ({ check: key, error: value })); + + if (failures.length > 0) { + results.failed.push({ + app, + packageName, + failures, + validationResults + }); + console.log(`โŒ ${packageName} - FAILED (${failures.length} issues)`); + if (isVerbose || singlePackage) { + failures.forEach(failure => { + console.log(` - ${failure.check}: ${failure.error}`); + }); + } + } else { + results.validated.push({ app, packageName, validationResults }); + if (isVerbose || singlePackage) { + console.log(`โœ… ${packageName} - VALID`); + if (singlePackage) { + console.log(` โœ“ Package.json validation: passed`); + console.log(` โœ“ Main file validation: passed`); + console.log(` โœ“ Dependencies validation: passed`); + console.log(` โœ“ Relative imports validation: passed`); + console.log(` โœ“ Package dependencies validation: passed`); + console.log(` โœ“ Import validation: passed`); + } + } + } + + } catch (error) { + results.failed.push({ + app, + packageName, + error: error.message, + failures: [{ check: 'general', error: error.message }] + }); + console.log(`โŒ ${app} (${packageName}) - FAILED: ${error.message}`); + } + } + + // Handle dry-run mode - early exit + if (isDryRun) { + console.log(`\n๐ŸŒต DRY RUN COMPLETED`); + console.log(`Found ${apps.length} packages that would be validated`); + const publishableCount = apps.filter(app => { + try { + const packagePath = path.join('components', app, 'package.json'); + const packageJson = JSON.parse(fs.readFileSync(packagePath, 'utf8')); + return packageJson.name?.startsWith('@pipedream/') && packageJson.publishConfig?.access; + } catch { + return false; + } + }).length; + console.log(`${publishableCount} packages are configured for publishing`); + return { summary: { total: apps.length, publishable: publishableCount } }; + } + + // Generate summary + results.summary = { + total: apps.length, + validated: results.validated.length, + failed: results.failed.length, + skipped: results.skipped.length, + publishable: results.validated.length + results.failed.length, + failureRate: results.validated.length + results.failed.length > 0 + ? ((results.failed.length / (results.validated.length + results.failed.length)) * 100).toFixed(2) + : '0.00' + }; + + // Print detailed summary + printDetailedSummary(results); + + // Save report to file if requested + if (outputFile) { + saveReportToFile(results, outputFile); + } + + // Exit with error if any validations failed (unless report-only mode) + if (results.failed.length > 0 && !isReportOnly) { + process.exit(1); + } + + return results; +} + +function validatePackageJson(packageJson, app) { + const required = ['name', 'version', 'main']; + + for (const field of required) { + if (!packageJson[field]) { + throw new Error(`Missing required field: ${field}`); + } + } + + if (!/^\d+\.\d+\.\d+/.test(packageJson.version)) { + throw new Error(`Invalid version format: ${packageJson.version}`); + } + + if (!packageJson.publishConfig?.access) { + throw new Error('Missing publishConfig.access for public package'); + } +} + +function validateMainFile(packageJson, app) { + const mainFile = path.join('components', app, packageJson.main); + + if (!fs.existsSync(mainFile)) { + throw new Error(`Main file not found: ${packageJson.main}`); + } + + const content = fs.readFileSync(mainFile, 'utf8'); + if (!content.includes('export') && !content.includes('module.exports')) { + throw new Error(`Main file ${packageJson.main} has no exports`); + } +} + +function validateDependencies(packageJson, app) { + if (!packageJson.dependencies) return; + + const platformDep = packageJson.dependencies['@pipedream/platform']; + if (platformDep && !platformDep.match(/^[\^~]?\d+\.\d+\.\d+/)) { + throw new Error(`Invalid @pipedream/platform version: ${platformDep}`); + } +} + +function validateRelativeImports(packageJson, app) { + const mainFile = path.join('components', app, packageJson.main); + + if (!fs.existsSync(mainFile)) { + return; // Will be caught by other validations + } + + const content = fs.readFileSync(mainFile, 'utf8'); + + // Find all relative imports to other app files or components + const relativeImportRegex = /import\s+.*\s+from\s+["'](\.\.\/[^"']+\.(?:app\.)?mjs)["']/g; + const relativeImports = []; + let match; + + while ((match = relativeImportRegex.exec(content)) !== null) { + const relativePath = match[1]; + let suggestion = ''; + + // Check if it's an app import + if (relativePath.includes('.app.mjs')) { + const pathParts = relativePath.split('/'); + if (pathParts.length >= 2) { + const importedApp = pathParts[1]; + suggestion = ` Consider using '@pipedream/${importedApp}' instead.`; + } + } + + relativeImports.push({ + relativePath, + fullMatch: match[0], + suggestion + }); + } + + if (relativeImports.length > 0) { + const importList = relativeImports + .map(imp => `${imp.fullMatch}${imp.suggestion}`) + .join(', '); + throw new Error(`Relative imports to app files should be avoided. Found: ${importList}`); + } +} + +function validatePackageDependencies(packageJson, app) { + const mainFile = path.join('components', app, packageJson.main); + + if (!fs.existsSync(mainFile)) { + return; // Will be caught by other validations + } + + const content = fs.readFileSync(mainFile, 'utf8'); + + // Find all npm package imports (not relative paths) + const packageImportRegex = /import\s+.*\s+from\s+["']([^./][^"']*)["']/g; + const packageImports = []; + let match; + + while ((match = packageImportRegex.exec(content)) !== null) { + const packageName = match[1]; + // Extract the base package name (handle scoped packages and subpaths) + let basePackageName; + if (packageName.startsWith('@')) { + // Scoped package like @pipedream/platform or @aws-sdk/client-s3 + const parts = packageName.split('/'); + basePackageName = `${parts[0]}/${parts[1]}`; + } else { + // Regular package like axios or lodash (could have subpath like lodash/get) + basePackageName = packageName.split('/')[0]; + } + + packageImports.push({ + packageName: basePackageName, + fullMatch: match[0], + originalImport: packageName + }); + } + + if (packageImports.length === 0) { + return; // No npm package imports found + } + + // Check if the packages are in dependencies or devDependencies + const missingDependencies = []; + const dependencies = packageJson.dependencies || {}; + const devDependencies = packageJson.devDependencies || {}; + const allDependencies = { ...dependencies, ...devDependencies }; + + // Remove duplicates + const uniquePackages = [...new Set(packageImports.map(imp => imp.packageName))]; + + uniquePackages.forEach((packageName) => { + if (!allDependencies[packageName]) { + const exampleImport = packageImports.find(imp => imp.packageName === packageName); + missingDependencies.push({ + packageName, + importStatement: exampleImport.fullMatch + }); + } + }); + + if (missingDependencies.length > 0) { + const missingList = missingDependencies + .map(dep => `${dep.packageName} (for ${dep.importStatement})`) + .join(', '); + throw new Error(`Package imports require corresponding dependencies. Missing dependencies: ${missingList}`); + } +} + +function validateImport(packageName, app, packageJson) { + const mainFile = path.resolve('components', app, packageJson.main); + + if (!fs.existsSync(mainFile)) { + throw new Error(`Main file not found: ${packageJson.main}`); + } + + // Syntax check + try { + execSync(`node --check ${mainFile}`, { + stdio: 'pipe', + timeout: 5000 + }); + } catch (error) { + throw new Error(`Syntax error in main file: ${error.message}`); + } + + // Import test using file path + const testFile = path.join('components', app, '__import_test__.mjs'); + const testContent = ` +try { + const pkg = await import("file://${mainFile}"); + + if (!pkg.default) { + throw new Error("No default export found"); + } + + const component = pkg.default; + if (typeof component !== 'object') { + throw new Error("Default export is not an object"); + } + + console.log("โœ“ Import successful for ${packageName}"); + process.exit(0); + +} catch (error) { + console.error("Import failed for ${packageName}:", error.message); + process.exit(1); +}`; + + try { + fs.writeFileSync(testFile, testContent); + execSync(`node ${testFile}`, { + stdio: 'pipe', + cwd: process.cwd(), + timeout: 10000 + }); + } catch (error) { + throw new Error(`Import test failed: ${error.message}`); + } finally { + if (fs.existsSync(testFile)) { + fs.unlinkSync(testFile); + } + } +} + +function printDetailedSummary(results) { + console.log('\n๐Ÿ“Š DETAILED VALIDATION SUMMARY'); + console.log('='.repeat(60)); + console.log(`๐Ÿ“ฆ Total Components: ${results.summary.total}`); + console.log(`โœ… Validated Successfully: ${results.summary.validated}`); + console.log(`โŒ Failed Validation: ${results.summary.failed}`); + console.log(`โญ๏ธ Skipped: ${results.summary.skipped}`); + console.log(`๐Ÿ“ˆ Publishable Packages: ${results.summary.publishable}`); + console.log(`๐Ÿ“‰ Failure Rate: ${results.summary.failureRate}%`); + + if (results.failed.length > 0) { + console.log('\nโŒ FAILED PACKAGES BY CATEGORY:'); + + const failuresByCategory = {}; + results.failed.forEach(({ packageName, failures }) => { + failures.forEach(failure => { + if (!failuresByCategory[failure.check]) { + failuresByCategory[failure.check] = []; + } + failuresByCategory[failure.check].push({ packageName, error: failure.error }); + }); + }); + + Object.entries(failuresByCategory).forEach(([category, failures]) => { + console.log(`\n๐Ÿ” ${category.toUpperCase()} FAILURES (${failures.length}):`); + failures.slice(0, 5).forEach(({ packageName, error }) => { + console.log(` โ€ข ${packageName}: ${error}`); + }); + if (failures.length > 5) { + console.log(` ... and ${failures.length - 5} more`); + } + }); + } + + if (results.skipped.length > 0) { + console.log('\nโญ๏ธ SKIPPED PACKAGES BY REASON:'); + const skippedByReason = {}; + results.skipped.forEach(({ reason, packageName }) => { + if (!skippedByReason[reason]) { + skippedByReason[reason] = []; + } + skippedByReason[reason].push(packageName); + }); + + Object.entries(skippedByReason).forEach(([reason, packages]) => { + console.log(` โ€ข ${reason}: ${packages.length} packages`); + }); + } + + console.log('\n๐ŸŽฏ RECOMMENDATIONS:'); + if (results.failed.length > 0) { + console.log(' 1. Review failed packages and fix import/dependency issues'); + console.log(' 2. Ensure all main files have proper exports'); + console.log(' 3. Validate package.json configurations'); + } else { + console.log(' ๐ŸŽ‰ All packages are valid! Ready for publishing.'); + } +} + +function saveReportToFile(results, filename) { + const report = { + generatedAt: new Date().toISOString(), + summary: results.summary, + validated: results.validated.map(r => ({ app: r.app, packageName: r.packageName })), + failed: results.failed.map(r => ({ + app: r.app, + packageName: r.packageName, + failures: r.failures + })), + skipped: results.skipped + }; + + fs.writeFileSync(filename, JSON.stringify(report, null, 2)); + console.log(`\n๐Ÿ“„ Report saved to: ${filename}`); +} + +// Run the report generation +if (require.main === module) { + generatePackageReport(); +} + +module.exports = { generatePackageReport }; \ No newline at end of file