Git hangs on preparing worktree

[jmgirard]

Sometimes when I run quarto publish gh-pages, it hangs (i.e., gets stuck) on "Preparing worktree."

 * branch            gh-pages   -> FETCH_HEAD
Rendering for publish:

...

branch 'gh-pages' set up to track 'origin/gh-pages'.
HEAD is now at 2054a64 Built site for gh-pages
Preparing worktree (resetting branch 'gh-pages'; was at 2054a64)

Any ideas how to troubleshoot, fix, or prevent this?

[jjallaire]

We are seeing this happen sometimes on Windows -- is that the platform you are on?

@cderv Is this the same thing that you have seen?

[jmgirard]

Problem continues after updating to 1.0.36.

[cderv]

Yes exactly this is the issue I encounter and that I am still investigating.
It hangs at the exact same step.

[jmgirard]

@cderv Glad you're working on it. In the meantime, I am just rendering locally and publishing via GitHub Actions.

[cderv]

I opened #1524 to track this as an issue. It confirms that it is not just my setup. You can follow the issue @jmgirard

[paniterka]

I am struggling with this as well. Win10 and Quarto 1.0.37. It used to work, but unreliably: sometimes yes, sometimes not. Once it helped to git worktree prune, but another time not. Now it doesn't work at all. I removed the whole branch gh-pages at all and tried again, but the issue persists.

[paniterka]

For me the following worked (suggested by this tutorial and the commands quarto publish uses ).

First notice the extra directory in your git repository, called 3f90bde or similar.

Then do:

cd 3f90bde
git add -Af . 
git commit --allow-empty -m "Semi-manual deploy"
git remote -v
git push --force origin HEAD:gh-pages
cd ..
git worktree remove 3f90bde
git worktree prune

If you want to check that all went fine, run git worktree list and you should see whether the directory and parallel branch have been removed.

I made a short blog note for a future reference: https://www.pythonviz.blog/posts/tips_quarto/quarto_worktree.html

[aghaynes]

I just got

git add -Af .
fatal: Unable to create 'C:/.../.git/worktrees/63aed11a/index.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.

So I guess the other process is still active... I also cannot just delete the lock file as it's also apparently in use

[paniterka]

What happens after you restart your computer? It looks like you have got a different problem than me

[cderv]

@paniterka yes running the git commands on its own is working and can be considered a workaround. However, I still need to understand what cause an issue when those commands are run by quarto itself. Thanks for sharing this in a blog post by the way.

@aghaynes This is an error from git when something went wrong. index.lock is created by git when doing some operation so that it prevent doing another operation at the same time. Sometimes something goes wrong and this file is a leftover, as noted in the message. If you are sure of your git state, you can remove the file and start again with the command.

[paniterka]

@cderv

I did some digging today using my blog as a template (because it experiences the same problem) and isolated for you a few scenarios here:

https://github.com/paniterka/quarto_debug

This is a quarto blog for which the quarto publish gh-pages is working correctly on some branches and incorrectly (aka it hangs upon creating the worktree) for other ones. Please look at the branches starting with reproducible_xxx - some of them are working, some are not. The main works correctly.

The differences between the branches are in the posts/other_tech/empty*.qmd and posts/tips_quarto/ojs_define_series.qmd. All the other posts are either qmd or ipynb and do not contain any content - just a front matter and a markdown line saying "empty" or so.

The following scenarios are:

• reproducible_working1: 10 empty posts -> working
• reproducible_working2: 9 empty posts and 1 post containing 2kb of lorem ipsum -> working
• reproducible_working3: 7 empty posts and posts/tips_quarto/ojs_define_series.qmd containing executable code in ojs and python (0.6kb) -> working
• reproducible_notworking1: 9 empty posts and posts/tips_quarto/ojs_define_series.qmd containing executable code in ojs and python (0.6kb) -> NOT working
• reproducible_notworking2: 13 empty posts -> NOT working

Nothing else changes between those branches. Somehow the content of the posts/ influences whether quarto publish gh-pages is successful or not.

I honestly don't understand what is going on, but it has been reproducible on my setup (Win10 Educational, Quarto 1.0.37, Python 3.8.10, git 2.37.1.windows.1). It would be interesting to know if you could reproduce the behavior of those examples.

When switching branches, I always delete _freeze and _site before running quarto publish gh-pages, because I observed that sometimes the content for removed posts would still stay in the built site.

I hope it helps you figure out what is happening!

[aghaynes]

thanks for both your suggestions, @paniterka and @cderv ... I suspect you are probably right @paniterka, a restart might have solved it... i decided to render locally and use a github action to deploy (setting the render option to false). that seems to work well...

[cderv]

Update on this: I cannot reproduce anymore this issue, even using the great repo example from @paniterka

What I did: I updated Git For Windows to latest version

> git --version
git version 2.37.3.windows.

I am also using latest pre-release of Quarto for the test (v1.2.105)

This seems to be the solution for this.