Improve CI Times by 70% #8530

mattKorwel · 2025-09-16T03:00:33Z

TLDR

This PR introduces several optimizations to the CI and E2E workflows to improve performance and reduce execution time.

Introduction of "merge skipper". If a pr has passing tests and is the only one in the merge queue it will skip running the tests again and merge without an additional build.
Added parallelization across all front so tets run in parallel at the package, file and os level
Moved to a much larger windows machine type (though will likely undo this) and added node and windows defender optimizations to improve speed.
Made "Slow tests" non blocking. This is the biggest change, The build will pass and PR's can merge when the linux tests pass for both ci and e2e. The tests will keep running and the build will show they failed but we will not be blocked on windows or mac. Though, mac is running pretty fast too..

Outcomes.

Before these changes CI All Jobs: 13 - 15 minutes.
After these changes CI All Jobs: 8 -9 minutes.
After These Changes CI Blocking jobs (ci & e2e) 3 - 4 minutes.

- Parallelize javascript linting in CI - Cache linting dependencies in CI - Use larger runners for CI and E2E - Centralize build step in E2E - Enable parallel testing in E2E

gemini-code-assist

Summary of Changes

Hello @mattKorwel, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on enhancing the performance of the continuous integration (CI) and end-to-end (E2E) workflows. By enabling parallel execution for both Vitest integration tests and npm workspace test commands, the changes aim to reduce overall build and test times, leading to a more efficient development cycle.

Highlights

Vitest Parallelism: Enabled "fileParallelism" in "integration-tests/vitest.config.ts" to allow Vitest to run test files in parallel, which should significantly speed up integration test execution.
Parallel Workspace Testing: Added the "--parallel" flag to the "test:ci" script in "package.json" to execute workspace tests concurrently, further reducing CI pipeline time.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This PR introduces parallel execution for tests to improve CI performance. While this is a good optimization, I've raised concerns in both changed files about the potential for race conditions and test flakiness if the tests are not properly isolated. These changes could impact CI stability if the test suites are not designed to be parallel-safe.

integration-tests/vitest.config.ts

package.json

github-actions · 2025-09-16T03:33:48Z

Size Change: +3.49 kB (+0.02%)

Total Size: 17.3 MB

Filename	Size	Change
`./bundle/gemini.js`	17.3 MB	+3.49 kB (+0.02%)

ℹ️ View Unchanged

Filename	Size
`./bundle/sandbox-macos-permissive-closed.sb`	1.03 kB
`./bundle/sandbox-macos-permissive-open.sb`	830 B
`./bundle/sandbox-macos-permissive-proxied.sb`	1.31 kB
`./bundle/sandbox-macos-restrictive-closed.sb`	3.29 kB
`./bundle/sandbox-macos-restrictive-open.sb`	3.36 kB
`./bundle/sandbox-macos-restrictive-proxied.sb`	3.56 kB

_{compressed-size-action}

github-actions · 2025-09-16T04:25:49Z

Code Coverage Summary

Package	Lines	Statements	Functions	Branches
CLI	N/A%	N/A%	N/A%	N/A%
Core	N/A%	N/A%	N/A%	N/A%

CLI Package - Full Text Report

CLI full-text-summary.txt not found at: coverage_artifact/cli/coverage/full-text-summary.txt

Core Package - Full Text Report

Core full-text-summary.txt not found at: coverage_artifact/core/coverage/full-text-summary.txt

For detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run.

mattKorwel · 2025-09-18T15:48:30Z

feat(ci): optimize ci and e2e workflows

79bcc29

- Parallelize javascript linting in CI - Cache linting dependencies in CI - Use larger runners for CI and E2E - Centralize build step in E2E - Enable parallel testing in E2E

mattKorwel requested review from a team as code owners September 16, 2025 03:00

gemini-code-assist bot reviewed Sep 16, 2025

View reviewed changes

integration-tests/vitest.config.ts Outdated Show resolved Hide resolved

package.json Outdated Show resolved Hide resolved

feat(ci): add workflow_dispatch trigger

b12b100

mattKorwel force-pushed the feature/ci-optimizations branch from 3aa5fc7 to b12b100 Compare September 16, 2025 03:03

gemini-cli bot added the status/need-issue label Sep 16, 2025

fix(ci): resolve yaml syntax errors

9a6ec5c

mattKorwel added 7 commits September 15, 2025 20:41

feat(ci): Run tests in parallel with most lint jobs

eee7e38

fix(ci): Restrict bundle_size job to pull_request events

7b374b3

feat(ci): Add final 'CI' aggregator job

2173704

refactor(ci): Remove redundant virtual test job

bc34056

fix(ci): Use correct runner for js build job

e481cba

feat(ci): Use self-hosted runners for all Linux jobs

177cf82

feat(ci): Use self-hosted runner for Windows test matrix

f22d26f

mattKorwel added 7 commits September 15, 2025 21:26

feat(ci): Enable fail-fast for test matrix

674fa1d

refactor(ci): Have coverage comment depend on final CI job

3eb9f46

refactor(ci): Make tests depend on all linters to fail faster

a69b6f6

fix(ci): Ensure coverage comment only posts on CI success

c0f3e74

feat(ci): Use self-hosted runners for E2E workflow

0f3dfc9

feat(ci): Add final 'E2E' aggregator job

4f96a1b

perf(ci): Increase parallelism for Vitest

37825cb

1042422227 approved these changes Sep 16, 2025

View reviewed changes

mattKorwel added 4 commits September 15, 2025 22:25

chore: Add debug step to CI workflow

9c079cd

fix: Correct formatting and shellcheck installation in CI

a2cc9e7

chore: Re-trigger CI

87135ab

fix(ci): Correct various linting and formatting issues in CI workflow

f5622bb

mattKorwel added 10 commits September 18, 2025 00:14

configure more windows performance

dc93018

clean up node configuration

63371dc

try and add docker directly

788238c

esbuild

8ac9cb2

linting

21ff419

try and fix macos dependency issue

8589caf

break out windows to not use shared build

45e9cd6

Merge branch 'main' into feature/ci-optimizations

d2f5dd5

liniting

ff9544b

linting

250b873

mattKorwel changed the title ~~feat(ci): optimize ci and e2e workflows~~ Improve CI Times by 70% Sep 18, 2025

mattKorwel added 4 commits September 18, 2025 08:59

break out windows in ci

dc761ac

update job names

68f7a7f

update job names pt 2

9344dcb

Merge branch 'main' into feature/ci-optimizations

2b28ea8

mattKorwel enabled auto-merge September 18, 2025 17:51

bdmorgan approved these changes Sep 18, 2025

View reviewed changes

mattKorwel added 4 commits September 18, 2025 10:57

linting

be6f9af

Merge branch 'main' into feature/ci-optimizations

d2bb9bd

Merge branch 'main' into feature/ci-optimizations

3aceb4f

Merge branch 'main' into feature/ci-optimizations

821849a

mattKorwel disabled auto-merge September 18, 2025 23:29

mattKorwel merged commit 509444d into main Sep 18, 2025
19 of 21 checks passed

mattKorwel deleted the feature/ci-optimizations branch September 18, 2025 23:29

abhipatel12 pushed a commit that referenced this pull request Sep 19, 2025

Improve CI Times by 70% (#8530)

9bb5a87

nagendrareddy10 pushed a commit to nagendrareddy10/gemini-cli that referenced this pull request Sep 22, 2025

Improve CI Times by 70% (google-gemini#8530)

bea56d8

yashv6655 added a commit to yashv6655/gemini-cli that referenced this pull request Sep 22, 2025

Improve CI Times by 70% (google-gemini#8530)

4795a47

thacio added a commit to thacio/auditaria that referenced this pull request Oct 3, 2025

Merge Commit '509444d': Improve CI Times by 70% (google-gemini#8530)

faa9788

giraffe-tree pushed a commit to giraffe-tree/gemini-cli that referenced this pull request Oct 10, 2025

Improve CI Times by 70% (google-gemini#8530)

38e287a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve CI Times by 70% #8530

Improve CI Times by 70% #8530

Uh oh!

mattKorwel commented Sep 16, 2025 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Sep 16, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Sep 16, 2025 •

edited

Loading

Uh oh!

mattKorwel commented Sep 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Improve CI Times by 70% #8530

Improve CI Times by 70% #8530

Uh oh!

Conversation

mattKorwel commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

Outcomes.

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Summary

Uh oh!

mattKorwel commented Sep 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mattKorwel commented Sep 16, 2025 •

edited

Loading

github-actions bot commented Sep 16, 2025 •

edited

Loading

github-actions bot commented Sep 16, 2025 •

edited

Loading