docs(metrics): automate usage metrics collection and publish it in the docs site #3495
Conversation
- Create Go script to query GitHub Code Search API for usage metrics - Add CSV storage for historical data (all versions from v0.13.0 to v0.39.0) - Integrate usage metrics dashboard into MkDocs documentation - Add interactive charts (trend, version comparison, distribution) - Create GitHub Actions workflow for automated weekly collection - Support manual workflow trigger with custom version queries Co-authored-by: mdelapenya <[email protected]>
✅ Deploy Preview for testcontainers-go ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings. WalkthroughAdds an automated usage-metrics pipeline: a Go collector that queries GitHub Code Search, a scheduled/manual GitHub Actions workflow to run it, CSV storage at docs/usage-metrics.csv, MkDocs dashboard assets (JS/CSS/MD), a usage-metrics submodule, and small CI/config updates. Changes
Sequence Diagram(s)sequenceDiagram
participant Scheduler as GitHub Scheduler
participant Workflow as Actions workflow
participant Collector as collect-metrics.go
participant GH as GitHub API (gh)
participant CSV as docs/usage-metrics.csv
participant Git as Git (branch/PR)
Scheduler->>Workflow: Trigger (monthly or manual)
Workflow->>Workflow: Checkout, setup Go
Workflow->>Collector: Run with resolved -version flags
Collector->>GH: Code search queries (retry/backoff)
GH-->>Collector: Return counts
Collector->>CSV: Append rows (date, version, count)
Workflow->>Git: Create branch, commit & push CSV
Workflow->>Git: Open PR to main
sequenceDiagram
participant User as Browser
participant Page as usage-metrics page
participant JS as usage-metrics.js
participant CSV as docs/usage-metrics.csv
participant Chart as Chart.js
User->>Page: Load
Page->>JS: Init
JS->>CSV: Fetch CSV
CSV-->>JS: Rows
JS->>JS: Parse & aggregate (by date/version)
JS->>Chart: Render trend, version, latest charts
JS->>Page: Show stats & update time
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes
Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (5)
usage-metrics/README.md (1)
7-7: Wrap bare URLs in markdown link syntax.Lines 7, 98, and 99 contain bare URLs that should be wrapped in markdown link format for better compliance and accessibility. For example:
- Line 7: [GitHub Code Search](https://github.com/search?q=...) - Line 98: [Production](https://golang.testcontainers.org/usage-metrics/) - Line 99: [Local](http://localhost:8000/usage-metrics/)Also applies to: 98-98, 99-99
.github/workflows/usage-metrics.yml (1)
32-60: Add defensive quoting to shell variable expansion.The shellcheck warning about SC2086 is valid here. While the current code works (word splitting is intentional), adding quotes provides better robustness against unexpected input:
- go run collect-metrics.go $VERSION_FLAGS -csv "../../docs/usage-metrics.csv" + go run collect-metrics.go $VERSION_FLAGS -csv "../../docs/usage-metrics.csv"Actually, for maximum safety with multiple flags, consider using an array:
# Earlier in script: VERSION_FLAGS=() for version in "${VERSION_ARRAY[@]}"; do version=$(echo "$version" | xargs) if [ -z "$version" ]; then continue fi VERSION_FLAGS+=("-version" "$version") done # Then invoke: go run collect-metrics.go "${VERSION_FLAGS[@]}" -csv "../../docs/usage-metrics.csv"This eliminates word-splitting concerns entirely.
docs/js/usage-metrics.js (1)
64-103: Handle empty datasets to avoid “Latest Version: undefined” UIIf the CSV is ever empty (or filtered to no rows),
versionsbecomes empty, which makeslatestVersionundefinedwhile still rendering the stats grid and latest‑usage doughnut (with zero data). Consider an upfront guard that detectsversions.length === 0, shows a “no data available” message, and skips stats + charts, so the UI degrades more cleanly.Also applies to: 262-321
usage-metrics/collect-metrics.go (2)
142-164: Avoid shell‑constructedgh apicommand; useexec.Commandarguments instead
queryGitHubUsagebuilds a single shell string and runssh -cwith interpolatedendpoint. Even though version strings are expected to be well‑formed tags, this is brittle (quoting/escaping issues) and opens the door to command injection if a malformed version ever sneaks in. Consider switching to something like:cmd := exec.Command( "gh", "api", "-H", "Accept: application/vnd.github+json", "-H", "X-GitHub-Api-Version: 2022-11-28", endpoint, ) output, err := cmd.Output()This keeps arguments properly separated and avoids shell interpretation entirely.
54-96: Optional robustness: dedupe versions before querying and consider UTC dateTwo small, non‑blocking robustness tweaks you might consider in
collectMetrics:
- If callers accidentally pass the same
-versionmultiple times, you’ll query and append duplicate metrics for that version/date. A simplemap[string]struct{}dedupe step before the loop would avoid redundant GitHub calls and duplicate CSV rows.date := time.Now().Format("2006-01-02")uses the host’s local timezone; for scheduled runs in CI you likely want a stable notion of “collection day” (e.g.,time.Now().UTC()) so dates don’t skew if the runner’s locale changes.Neither is critical, but both make behavior a bit more predictable over time.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
docs/usage-metrics.csvis excluded by!**/*.csv
📒 Files selected for processing (10)
.github/workflows/usage-metrics.yml(1 hunks).gitignore(1 hunks)docs/css/usage-metrics.css(1 hunks)docs/js/usage-metrics.js(1 hunks)docs/usage-metrics.md(1 hunks)mkdocs.yml(2 hunks)scripts/changed-modules.sh(2 hunks)usage-metrics/README.md(1 hunks)usage-metrics/collect-metrics.go(1 hunks)usage-metrics/go.mod(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-09-18T08:24:27.479Z
Learnt from: mdelapenya
Repo: testcontainers/testcontainers-go PR: 3254
File: .github/dependabot.yml:21-21
Timestamp: 2025-09-18T08:24:27.479Z
Learning: In the testcontainers-go repository, submodules like atlaslocal that are part of a parent module (e.g., mongodb) share the same go.mod file and should not have separate Dependabot entries. They are already monitored through the parent module's Dependabot configuration entry.
Applied to files:
usage-metrics/go.modusage-metrics/README.mdmkdocs.yml
🪛 actionlint (1.7.8)
.github/workflows/usage-metrics.yml
32-32: shellcheck reported issue in this script: SC2086:info:28:27: Double quote to prevent globbing and word splitting
(shellcheck)
🪛 LanguageTool
usage-metrics/README.md
[uncategorized] ~31-~31: The official name of this software platform is spelled with a capital “H”.
Context: ... mobile and desktop ### 🤖 Automation (.github/workflows/usage-metrics.yml) - Runs mo...
(GITHUB)
[style] ~180-~180: Consider using a different verb for a more formal wording.
Context: ...le ## Contributing To add features or fix issues: 1. Test changes locally with `...
(FIX_RESOLVE)
🪛 markdownlint-cli2 (0.18.1)
usage-metrics/README.md
7-7: Bare URL used
(MD034, no-bare-urls)
98-98: Bare URL used
(MD034, no-bare-urls)
99-99: Bare URL used
(MD034, no-bare-urls)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (20)
- GitHub Check: lint (modules/yugabytedb) / lint: modules/yugabytedb
- GitHub Check: lint (modules/solace) / lint: modules/solace
- GitHub Check: lint (modules/socat) / lint: modules/socat
- GitHub Check: lint (modules/scylladb) / lint: modules/scylladb
- GitHub Check: lint (modules/dockermcpgateway) / lint: modules/dockermcpgateway
- GitHub Check: lint (modules/meilisearch) / lint: modules/meilisearch
- GitHub Check: lint (modules/nats) / lint: modules/nats
- GitHub Check: lint (modules/neo4j) / lint: modules/neo4j
- GitHub Check: lint (modules/k3s) / lint: modules/k3s
- GitHub Check: lint / lint:
- GitHub Check: lint (modules/ollama) / lint: modules/ollama
- GitHub Check: lint (modules/nebulagraph) / lint: modules/nebulagraph
- GitHub Check: lint (modules/inbucket) / lint: modules/inbucket
- GitHub Check: lint (modules/memcached) / lint: modules/memcached
- GitHub Check: lint (modules/compose) / lint: modules/compose
- GitHub Check: lint (modules/milvus) / lint: modules/milvus
- GitHub Check: lint (modules/gcloud) / lint: modules/gcloud
- GitHub Check: lint (modules/localstack) / lint: modules/localstack
- GitHub Check: lint (modules/databend) / lint: modules/databend
- GitHub Check: Analyze (go)
🔇 Additional comments (11)
usage-metrics/go.mod (1)
1-3: Module definition looks correct.The usage-metrics submodule is properly scoped and Go version is current. Dependencies will be managed by go.mod when the collect-metrics.go program is compiled.
.gitignore (1)
27-28: Verify binary output path.The .gitignore entry references
usage-metrics/scripts/collect-metrics, but based on the workflow (which runs fromusage-metricsdirectory and buildscollect-metrics.go), the binary should be atusage-metrics/collect-metrics(without thescripts/subdirectory). Confirm the correct path to ignore.scripts/changed-modules.sh (1)
89-89: Exclusions properly aligned with system design.Both additions—the usage-metrics workflow and the generated CSV file—are appropriately excluded from triggering module builds, consistent with the established pattern for documentation and automation files.
Also applies to: 102-102
mkdocs.yml (1)
22-27: Configuration additions are well-structured.The externally sourced libraries are pinned to specific versions for stability, and the local assets are properly referenced. Navigation entry is correctly positioned.
Also applies to: 157-157
docs/usage-metrics.md (1)
1-30: Static structure is well-designed for dynamic population.The HTML markup provides appropriate hooks (element IDs, semantic structure) for the JavaScript runtime to populate with data. Loading and error states are properly provisioned.
.github/workflows/usage-metrics.yml (2)
36-43: Verify git tag extraction logic handles edge cases.The tag extraction pipeline uses several transformations (grep, sed, sort, awk). While the logic appears sound, test these scenarios:
- When no tags exist matching the pattern
- When v0.13.0 doesn't exist (will awk still work correctly?)
- When there are pre-release tags (e.g., v0.40.0-rc1) in the repo
The current grep pattern
v0\.[0-9]+\.[0-9]+$will correctly exclude pre-releases, which is good. However, verify that the awk line/v0.13.0/,0correctly handles the case where v0.13.0 might not exist.
62-85: PR creation flow is solid.The branch naming convention is clear, commit messages are descriptive, and the PR body provides context. The workflow correctly checks for staged changes before committing to avoid empty commits.
docs/css/usage-metrics.css (1)
1-111: CSS is well-structured and responsive.The stylesheet provides coherent styling across multiple components with appropriate responsive breakpoints. Grid layout for stats, proper shadow and radius use for depth, and mobile considerations are all in place. Color contrast and accessibility appear adequate.
docs/js/usage-metrics.js (2)
10-33: Solid data loading, processing, and init flowThe CSV fetch, Papa.parse usage, sorting/grouping, and single‑run
initguard form a clean, resilient pipeline. Error handling withshowErrorplus resettingisInitializedon failure is a good touch, and the page‑presence check viatrendChartkeeps this script from doing work on unrelated pages.Also applies to: 45-63, 323-362
105-175: Verify Chart.js time‑scale adapter is wired correctlyBoth the trend and total‑repositories charts rely on
type: 'time'for the x‑axis andx: d.datevalues. This assumes the Chart.js time adapter is loaded/registered in the page and can parse yourYYYY-MM-DDstrings. Please double‑check the docs page includes the appropriate adapter bundle/registration for the Chart.js major version you’re using so these axes render correctly.Also applies to: 177-260
usage-metrics/collect-metrics.go (1)
166-195: CSV append strategy is straightforward and safe for single-writer use
appendToCSVcorrectly resolves an absolute path, creates the file if missing, conditionally writes the header, and appends a single row per metric. For the current usage pattern (single CI job appending once per version per run) this is perfectly adequate and keeps the implementation simple.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
modulegen/internal/mkdocs/types.go (2)
25-27:ExtraJavascriptfield wiring looks correct; naming nit onlyThe YAML tag
extra_javascriptmatches MkDocs’ config key, so this will correctly round‑trip additional JS assets. If you care about Go identifier style, you might consider renaming the field toExtraJavaScriptfor conventional capitalization, but behavior is already correct as‑is.
30-41: AddingUsage Metricsnav: confirmaddModule’s hard‑coded indices still point at the right items
addModuleassumesc.Nav[3].Modulesandc.Nav[4].Examplesare the “Modules” and “Examples” entries. With a newUsage Metricsnav item added inmkdocs.yml, this will break if the new entry is inserted before either of those and shifts their indices.Two follow‑ups to consider:
- Verify that in
mkdocs.ymlthe newUsage Metricsitem is appended after the existing “Modules”/“Examples” entries so indices 3 and 4 are unchanged.- Longer term, avoid magic indices by locating the correct nav entry by key (e.g., scanning
c.Navfor the item wherelen(Modules) > 0orlen(Examples) > 0) before mutating it, so future nav additions don’t silently misplace generated links.Also applies to: 48-79
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
modulegen/internal/mkdocs/types.go(2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: lint (modules/artemis) / lint: modules/artemis
- GitHub Check: lint (modules/mariadb) / lint: modules/mariadb
- GitHub Check: lint (modules/dockermodelrunner) / lint: modules/dockermodelrunner
- GitHub Check: lint (modules/influxdb) / lint: modules/influxdb
- GitHub Check: lint (modules/vault) / lint: modules/vault
- GitHub Check: lint (modules/pinecone) / lint: modules/pinecone
- GitHub Check: lint (modules/mssql) / lint: modules/mssql
- GitHub Check: lint (modules/elasticsearch) / lint: modules/elasticsearch
- GitHub Check: Analyze (go)
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
usage-metrics/collect-metrics.go (1)
83-86: Lexicographic sort produces incorrect version orderingString comparison doesn't handle semantic versioning correctly. For example,
"v0.9.0" < "v0.10.0"isfalselexicographically, so v0.10.0 would sort before v0.9.0.Consider using a semver-aware comparison or parsing version components numerically:
sort.Slice(metrics, func(i, j int) bool { // Simple approach: pad version numbers for correct string sort // Or use a semver library like golang.org/x/mod/semver return semver.Compare(metrics[i].Version, metrics[j].Version) < 0 })Alternatively, if adding a dependency is undesirable, parse and compare major/minor/patch components as integers.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
scripts/changed-modules.sh(3 hunks)usage-metrics/Makefile(1 hunks)usage-metrics/collect-metrics.go(1 hunks)
🧰 Additional context used
🪛 checkmake (0.2.2)
usage-metrics/Makefile
[warning] 1-1: Missing required phony target "all"
(minphony)
[warning] 1-1: Missing required phony target "clean"
(minphony)
[warning] 1-1: Missing required phony target "test"
(minphony)
🪛 Shellcheck (0.11.0)
scripts/changed-modules.sh
[warning] 133-133: Quote to prevent word splitting/globbing, or split robustly with mapfile or read -a.
(SC2206)
[warning] 133-133: Quote to prevent word splitting/globbing, or split robustly with mapfile or read -a.
(SC2206)
[warning] 133-133: Quote to prevent word splitting/globbing, or split robustly with mapfile or read -a.
(SC2206)
🔇 Additional comments (2)
scripts/changed-modules.sh (2)
89-89: Exclusions for usage-metrics automation are well-placed.Adding
.github/workflows/usage-metrics.ymlanddocs/usage-metrics.csvto excluded files prevents metrics-only updates from triggering unnecessary module builds — consistent with the PR objective to suppress CI for metrics changes.Also applies to: 102-102
129-133: Usage-metrics module integrated into build logic correctly.The new
usageMetricsModulefollows the same pattern asmodulegen(manually defined, not auto-discovered), which is appropriate for a root-level module. Its inclusion inallModulesensures that changes to the Go tool itself will trigger builds.Please confirm that the
usage-metricsdirectory has a valid Go module structure (e.g.,usage-metrics/go.mod) so that the module is discoverable by downstream build logic.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
usage-metrics/collect-metrics.go (1)
89-95: Consider batching CSV writes to reduce file I/O overhead.The current implementation opens and closes the CSV file for each metric record. While functionally correct and acceptable for the expected monthly run with ~30 versions, opening the file once and writing all records would be more efficient.
- // Write all metrics to CSV - for _, metric := range metrics { - if err := appendToCSV(csvPath, metric); err != nil { - log.Printf("Warning: Failed to write metric for %s: %v", metric.Version, err) - continue - } - fmt.Printf("Successfully recorded: %s has %d usages on %s\n", metric.Version, metric.Count, metric.Date) - } + // Write all metrics to CSV in a single batch + if err := appendMetricsToCSV(csvPath, metrics); err != nil { + return fmt.Errorf("write metrics to CSV: %w", err) + } + for _, metric := range metrics { + fmt.Printf("Successfully recorded: %s has %d usages on %s\n", metric.Version, metric.Count, metric.Date) + }This would require refactoring
appendToCSVto accept a slice and write all records within a single file handle.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
usage-metrics/collect-metrics.go(1 hunks)usage-metrics/go.mod(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- usage-metrics/go.mod
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: lint (modules/yugabytedb) / lint: modules/yugabytedb
- GitHub Check: lint (examples/nginx) / lint: examples/nginx
- GitHub Check: Analyze (go)
🔇 Additional comments (2)
usage-metrics/collect-metrics.go (2)
100-140: LGTM!The retry logic with exponential backoff is now correctly implemented. All 5 backoff intervals are reachable, and the initial attempt plus retries are properly tracked.
142-168: LGTM!The command injection vulnerability has been properly addressed by using
exec.Commandwith explicit arguments instead of shell interpolation. Error handling withExitErrorstderr extraction provides useful diagnostics.
| _, err = os.Stat(absPath) | ||
| fileExists := !os.IsNotExist(err) | ||
|
|
There was a problem hiding this comment.
Edge case in file existence check.
If os.Stat returns an error other than os.IsNotExist (e.g., permission denied on parent directory), fileExists will be true, potentially skipping the header write for a new file. While unlikely in practice (the subsequent OpenFile would likely fail too), consider handling this edge case explicitly.
_, err = os.Stat(absPath)
- fileExists := !os.IsNotExist(err)
+ fileExists := err == nilThis ensures fileExists is only true when stat succeeds, and the header will be written for any stat failure scenario where the file is subsequently created.
🤖 Prompt for AI Agents
In usage-metrics/collect-metrics.go around lines 176–178, the current file
existence check treats any non-IsNotExist error as the file existing; change the
logic so fileExists is true only when os.Stat returned nil (e.g., fileExists :=
err == nil), and treat any non-nil stat error as "file does not exist" for the
purpose of writing the header (optionally log or return the stat error if you
want to surface permission/other errors separately before attempting OpenFile).
1 participant
Summary
Implements an automated system to track and visualize testcontainers-go adoption across GitHub repositories. The system queries GitHub's Code Search API for usage in go.mod
files, stores historical data, and displays interactive charts on the documentation site.
Features
Components Added
Data Collection (
usage-metrics/)collect-metrics.go: Go program that queries GitHub Code Search API-versionflags for efficient batch processingREADME.md: Complete documentation for the metrics systemVisualization (
docs/)usage-metrics.md: Dashboard page integrated into documentationjs/usage-metrics.js: Interactive charts powered by Chart.jscss/usage-metrics.css: Responsive styling for dashboardAutomation (
.github/workflows/)usage-metrics.yml: GitHub Actions workflowData Storage
docs/usage-metrics.csv: Historical usage data (758 initial records)date,version,countCI/CD Integration
scripts/changed-modules.sh:docs/usage-metrics.csvto excluded files.github/workflows/usage-metrics.ymlto excluded filesHow It Works
chore(metrics): update usage metrics (YYYY-MM-DD))Viewing the Dashboard
mkdocs serve→ http://localhost:8000/usage-metrics/Test Plan
workflow_dispatch