Add support for Git sparse checkout when `.spec.update.path` is specified #920

kane8n · 2025-06-10T11:07:51Z

Summary

If Spec.Update.Path is specified, only the manifest under this Path is required, no other files.
Therefore, SparseCheckout is valid.

Set the Path specified by Spec.Update.Path to SparseCheckoutDirectory so that only the necessary files are downloaded.

Feature Gate

To enable this feature, the controller must be started with the flag --feature-gates=GitSparseCheckout=true.

makkes

Thanks for this improvement. I would like to see this this feature aligned with the API for GitRepository resources that we implemented in fluxcd/source-controller#1774 and make it an explicit field in the API instead of a feature flag of the controller. Not sure how @stefanprodan or @matheuscscp think about this.

stefanprodan · 2025-06-11T07:36:51Z

make it an explicit field in the API instead of a feature flag

We don't have a path field in GitRepository so we had to add a dedicated field for it. For image automation we know the path that we should set as sparse checkout as it's already in the API spec. Ideally image automation should always use sparse checkout but given that this feature is experimental in Git itself and brand new in go-git, I would not enable the feature flag by default. Sparse checkout can break at any time, the Git protocol may change, then the go-git project needs to adapt to it, etc.

stefanprodan · 2025-06-13T09:17:39Z

internal/features/features.go


+	// GitSparseCheckout
+	// opt-out from v0.42
+	GitSparseCheckout: true,


Let's make this opt-in for now. We can switch this to opt-out once sparse checkout is no longer an experimental feature in Git.

stefanprodan · 2025-06-13T09:18:13Z

api/v1beta2/imageupdateautomation_types.go

+	// When a path is specified, sparse checkout will be enabled to limit the checkout scope
+	// to the specified directory, reducing clone size and time.


I would remove this, since we'll have sparse checkout as an opt-in flag.

kane8n · 2025-06-13T10:26:15Z

@stefanprodan
Thanks for the feedback!
I've addressed your comments and pushed the changes.

stefanprodan · 2025-06-13T11:12:50Z

Thanks @kane8n for this contribution! I'm putting this on hold as we need to do a patch release today. After that I'm going to ask you to rebase and then we can merge it. This feature will be included in Flux 2.7

stefanprodan · 2025-06-16T12:53:22Z

@kane8n please rebase

kane8n · 2025-06-17T01:21:02Z

@stefanprodan
Rebase done! Please check it.

stefanprodan · 2025-06-17T06:43:34Z

internal/source/source.go

 	}
 }

+// WithCheckoutOptionRecurseSubmodules is a CheckoutOption option to configure


Suggested change

// WithCheckoutOptionRecurseSubmodules is a CheckoutOption option to configure

// WithCheckoutOptionSparseCheckoutDirectories is a CheckoutOption option to configure

stefanprodan · 2025-06-17T06:44:43Z

internal/source/source.go

+// SparseCheckoutDirectories.
+func WithCheckoutOptionSparseCheckoutDirectories(updatePath string) CheckoutOption {
+	return func(cc *repository.CloneConfig) {
+		cc.SparseCheckoutDirectories = []string{filepath.Clean(updatePath)}


Maybe we should skip enabling sparse checkout when the path is the repo root e.g. path: ./

Signed-off-by: kane8n <[email protected]>

kane8n · 2025-06-17T09:03:28Z

@stefanprodan
Thanks!
I have applied the corrections and pushed them.

stefanprodan · 2025-06-17T09:21:21Z

internal/source/source.go

+func WithCheckoutOptionSparseCheckoutDirectories(updatePath string) CheckoutOption {
+	return func(cc *repository.CloneConfig) {
+		cleanedPath := filepath.Clean(updatePath)
+		if cleanedPath == "." {


We need to remove the ./ prefix as sparse checkout will fail due to a bug in go-git. See https://github.com/fluxcd/source-controller/blob/97c995b8c8e3a6c9c31cb4f1e8323428a03d07d3/internal/controller/gitrepository_controller.go#L976

@stefanprodan
Use filepath.Clean to remove the leading ./ prefix.
Should I use the same implementation as source-controller for consistency?

sample

https://go.dev/play/p/syB7j09fMii

Ok cool, let's leave it like this, SC could drop the custom code and use filepath.Clean

stefanprodan

LGTM

Thanks @kane8n 🏅

PS. Please open a PR on the website repo to add the feature gate in here: https://github.com/fluxcd/website/blob/main/content/en/flux/components/image/options.md#feature-gates

kane8n force-pushed the git-sparse-checkout-when-update-path-specify branch 2 times, most recently from 2dd1868 to 0dfd5e8 Compare June 10, 2025 11:16

makkes reviewed Jun 11, 2025

View reviewed changes

kane8n force-pushed the git-sparse-checkout-when-update-path-specify branch from 0dfd5e8 to 8be4dad Compare June 13, 2025 00:29

stefanprodan reviewed Jun 13, 2025

View reviewed changes

kane8n force-pushed the git-sparse-checkout-when-update-path-specify branch from 8be4dad to d73a337 Compare June 13, 2025 10:22

kane8n requested a review from stefanprodan June 13, 2025 10:26

stefanprodan added hold Issues and pull requests put on hold and removed hold Issues and pull requests put on hold labels Jun 13, 2025

kane8n force-pushed the git-sparse-checkout-when-update-path-specify branch from d73a337 to 5ceb013 Compare June 17, 2025 01:16

stefanprodan reviewed Jun 17, 2025

View reviewed changes

SparseCheckout when Spec.Update.Path specified

3999c65

Signed-off-by: kane8n <[email protected]>

kane8n force-pushed the git-sparse-checkout-when-update-path-specify branch from 5ceb013 to 3999c65 Compare June 17, 2025 09:00

stefanprodan reviewed Jun 17, 2025

View reviewed changes

stefanprodan changed the title ~~SparseCheckout when Spec.Update.Path specified~~ Add support for Git sparse checkout when .spec.update.path is specified Jun 17, 2025

stefanprodan added the enhancement New feature or request label Jun 17, 2025

stefanprodan approved these changes Jun 17, 2025

View reviewed changes

stefanprodan merged commit 124fd9f into fluxcd:main Jun 17, 2025
9 of 10 checks passed

kane8n mentioned this pull request Jun 17, 2025

Added description of GitSparseCheckout to feature gate fluxcd/website#2279

Merged

		// When a path is specified, sparse checkout will be enabled to limit the checkout scope
		// to the specified directory, reducing clone size and time.

	// WithCheckoutOptionRecurseSubmodules is a CheckoutOption option to configure
	// WithCheckoutOptionSparseCheckoutDirectories is a CheckoutOption option to configure

Add support for Git sparse checkout when .spec.update.path is specified #920

Add support for Git sparse checkout when .spec.update.path is specified #920

Uh oh!

Conversation

kane8n commented Jun 10, 2025 • edited by stefanprodan Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Feature Gate

Uh oh!

makkes left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stefanprodan commented Jun 11, 2025

Uh oh!

stefanprodan Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

stefanprodan Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

kane8n commented Jun 13, 2025

Uh oh!

stefanprodan commented Jun 13, 2025

Uh oh!

stefanprodan commented Jun 16, 2025

Uh oh!

kane8n commented Jun 17, 2025

Uh oh!

stefanprodan Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

stefanprodan Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

kane8n commented Jun 17, 2025

Uh oh!

stefanprodan Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

kane8n Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

stefanprodan Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

stefanprodan left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add support for Git sparse checkout when `.spec.update.path` is specified #920

Add support for Git sparse checkout when `.spec.update.path` is specified #920

kane8n commented Jun 10, 2025 •

edited by stefanprodan

Loading

makkes left a comment •

edited

Loading

stefanprodan left a comment •

edited

Loading