Skip to content

Conversation

@lunny
Copy link
Member

@lunny lunny commented May 25, 2024

Make git tree recursive iteration stoppable and reduce the memory usage when calculating language stats.

@lunny lunny added type/refactoring Existing code has been cleaned up. There should be no new functionality. performance/memory Performance issues affecting memory use labels May 25, 2024
@lunny lunny added this to the 1.23.0 milestone May 25, 2024
@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label May 25, 2024
@github-actions github-actions bot added the modifies/go Pull requests that update Go code label May 25, 2024
@lunny lunny modified the milestones: 1.23.0, 1.24.0 Sep 7, 2024
@lunny lunny force-pushed the lunny/performance_big_reference_repo branch from eaae69e to 7bdf686 Compare December 30, 2024 01:50
@lunny lunny marked this pull request as ready for review December 31, 2024 07:02
@wxiaoguang
Copy link
Contributor

It doesn't seem right, especially the different "ctx" are mixed together.

And in the end, you still read everything into memory entries = append(entries, entry)


To make it overall right, it needs to consume the outputs on the fly, and make the git command has ability to limit the read data size.

@wxiaoguang wxiaoguang removed their request for review December 31, 2024 17:34
@lunny
Copy link
Member Author

lunny commented Dec 31, 2024

It doesn't seem right, especially the different "ctx" are mixed together.

And in the end, you still read everything into memory entries = append(entries, entry)

To make it overall right, it needs to consume the outputs on the fly, and make the git command has ability to limit the read data size.

This PR introduced an iteration method and only reduces memory when calculating language stats. For tree list, they are kept as before and we can improve it in the future.

@lunny lunny marked this pull request as draft January 8, 2025 03:18
@lunny lunny modified the milestones: 1.24.0, 1.25.0 Apr 10, 2025
@lunny
Copy link
Member Author

lunny commented Jun 4, 2025

replaced by #34605

@lunny lunny closed this Jun 4, 2025
@lunny lunny deleted the lunny/performance_big_reference_repo branch June 4, 2025 21:04
@GiteaBot GiteaBot removed this from the 1.25.0 milestone Jun 5, 2025
@go-gitea go-gitea locked as resolved and limited conversation to collaborators Sep 3, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. modifies/go Pull requests that update Go code performance/memory Performance issues affecting memory use type/refactoring Existing code has been cleaned up. There should be no new functionality.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants