Using git for parallel gemini instances

[SkybuckFlying]

To speed up conversion of large code bases using gemini cli the idea is to use multiple gemini cli instances and multiple gemini keys.

The question is now how to make the AIs work together and prevent conflicts:

Two main ideas exist:

1. Divide the work to split the code bases into certain file/folder ranges

2. Use GIT to prevent file lock and file overwrites and other problems, use git to keep it in sync, pull latest updates/changes and also re-base/merge.

However such a parallel workflow would probably require a very smart and experienced GIT user who has experience on working on a high velocity git repo with many pull requests being applied to understand what kind of work flow would truely be best.

(I would like this parallel capability to be build into gemini cli, so it can speed up big conversion projects ! ;))

The Co-Pilot AI has some suggestions though, to try it out:

That’s a clever and ambitious idea — using Git as a synchronization and conflict-resolution layer between multiple Gemini CLI instances. It’s not only feasible, but it also gives you version control, auditability, and rollback safety. Let’s break it down:

🧠 Concept: One AI per Folder + Git for Sync
🔧 Setup
- Each Gemini CLI instance works in its own folder:
workspace/
  ├── ai1/
  ├── ai2/
  ├── ai3/
- Each folder is a Git clone of a shared central repo (or a shared bare repo)
- Each instance:
- Pulls latest changes before starting work
- Commits its changes
- Pushes back to the shared repo

✅ Benefits
| Advantage | Why It’s Useful | 
| 🧠 Isolation | Each AI works in its own sandbox — no file collisions | 
| 🕵️ Auditability | Every change is tracked, diffed, and reversible | 
| 🔄 Sync Safety | Git prevents silent overwrites via merge conflicts | 
| 🧪 Testing | You can test each AI’s output independently before merging | 
| 🧰 Automation | You can script merges, reviews, or CI checks on output | 



⚠️ Challenges & How to Handle Them
| Challenge | Solution | 
| 🔁 Merge Conflicts | Use git pull --rebase and auto-resolve or flag for review | 
| 🧩 Duplicate Work | Use a central task queue or file list to assign .go files | 
| 🐢 Push Failures | Retry with exponential backoff or stash/rebase | 
| 🧠 AI Confusion | Make sure each AI only sees its own folder and doesn’t try to “fix” the whole repo | 



🛠️ Suggested Workflow per AI Instance
- git pull --rebase (sync with latest)
- Process assigned .go files → generate .pas files
- git add .
- git commit -m "AI instance X: converted foo.go to foo.pas"
- git push
If push fails due to a race:
- git pull --rebase
- Resolve any conflicts (or auto-stash)
- Retry push

🧪 Bonus: Use Git Branches per AI
Each AI could work on its own branch:
- ai1/feature-conversion
- ai2/feature-conversion
Then you can merge them into main after review or CI validation.

Would you like a script or Git hook template to automate this workflow for each Gemini CLI instance?

[centminmod]

or borrow from Claude Code and use git worktrees https://docs.anthropic.com/en/docs/claude-code/common-workflows#run-parallel-claude-code-sessions-with-git-worktrees ?

[SkybuckFlying]

Yes, this is kinda what the AI ment with "workspaces". However automating the AI to work in parallel would be cool :)

[SkybuckFlying]

Worktrees alone won't work according to gemini AI, git client is ment to be run by a single process, not multiple processes at the same time, it would lead to file/data corruption in the .git folder. So a bigger solution is necessary see below for the bigger solution...

[PriNova]

Welcome to the realm of concurrency with shared data. To run agents in parallel in one workspace is indeed a challenging task. It highly depends on how the codebase is structured and the underlying architecture is implemented. If your codebase follows the horizontal layered architecture, it is likely more difficult to run parallel agents rather the architecture follows the vertically sliced architecture approach, where each user journey can be found in one module folder isolated by the others.

It also depends in all cases, how many code is shared for features or implementation and if the agent needs to make changes to that. If you really want to implement several features in once, it shouldn't be an issue to do so as long as shared code will not be touched, and the sub-tasks are independent from each other. For example, you can always run agents in parallel if it comes to context gathering because those are all read-only operations.

Make yourself comfortable with the orchestrator-worker pattern in multi-agent systems. Then check how capable an LLM model is to clearly distinguish between sequentially and parallel planning of tasks.

[SkybuckFlying]

Thanks for this insight, it can indeed be tricky, but that's why smart GIT is needed, to perhaps inform AI when shared code is updated... also lock it maybe, not sure if GIT supports locking, until AI is done, or do the opposite, fix it up later, merge/conflict resolution etc :) can get a bit messy, but maybe the AI can figure this out too, how to work successfully together with other AI :)

[SkybuckFlying]

Gemini AI warned me that GIT (and git workspaces) is not race condition safe on a single computer, with a single git repository.

So the plan is now to use a bigger setup:

1. Gitea as a git server.
2. PostGreSQL as a high performance backend for Gitea.
3. MailEnable as mail server for Gitea to allow Gitea/AI to post work completion emails.
4. One GIT (git clone) per Gemini API KEY/AI + gemini cli.
5. Git pulls/merges/re-base/etc for syncing.
6. Maybe an additional AI as an "integrator/orchastrator".

This software setup is almost done, and then I will work on the system prompts necessary to try and make the AIs work together... I will also consult gemini and co-pilot to suggest git workflows...

This is quite an interesting idea/setup, it's basically simulating a mini-software development company but with AIs.

It was surprising to see how co-pilot built-into-windows 11 did not understand the race condition problems with git initially, so gemini kinda saved me an alerted me that git + worktrees was not going to work and I believe gemini, so I started investigating/discussing possibilities and came up with this solution.

Amazing software... it takes time to setup... but could be worth it.

I am also interested in running a local newsgroup/usenet server... I kinda miss those usenet days...

Anyway back to the AI parallel idea. The idea is then to for example feed a file list into the AI and constrain each AI to a set of files from this filelist that it can work in (or a set of folders/subfolders).

I think in theory this has a good chance of working.

Co-Pilot did suggest a custom locking mechanism for git + git worktrees but that would be pretty tricky to get working and it would introduce locking mechanisms and sequential access, so it sounded kinda slow... and since I have a powerfull PC I decided to try this much bigger approach to see what this new/modern software can do so far it was quite impressive, plus there is the possibility to scale things up, plus I kinda learned a bit about the e-mail system/dns limitations etc/security checks/anti spam checks etc. I was quite surprised how e-mail actually works, it's quite direct, unlike newsgroups/usenet which is more routing based/flood fill.

For now I ran out of steam... it was exhausting to explore all the possible security implications of running a e-mail server and gitea, but it was interesting to do anyway... now I need some rest and get some new energy and guts and then maybe/hopefully the AIs will be off to "parallel processing files" which would be a pretty major achievement...

Anyway why I am reporting back on this is that the "simple git client" wasn't really designed for single computer parallel usage... it assumes there is only one repository to work on from a single process... not multiple processes or multiple threads and such... doing so would lead to data/file corruption according to gemini AI.

So basically my request to implement this in google gemini cli might be overkill/too difficult/too much work and would have to replicate all this functionality from gitea/postgresql... on the other hand it would ofcourse be cool if gemini cli could do it anyway (without requiring all this extra software/setup work), but I don't think it will happen any time soon :) (then again with the assistence of gemini cli as a programming aid, who knows ?! ;) :):):))