[BUG] file.recurse with keep_symlinks: True not working with gitfs backend

[jcpearson]

Description
Salt master (3006.9 on Rocky 9) configured to use gitfs (via gitpython) as the file server backend - but when using a file.recurse state with keep_symlinks: True, symlinks are not retained from source to the minion (also running 3006.9) - i.e. symlinks to files in the same directory are created as files on the minion (the symlink target is resolved)

However, when using a master configured to use a local file system as the backend, then keep_symlinks: True works as expected

Setup

• on-prem machine
• onedir packaging

Using:

symlink.recurse:
  file.recurse:
    - name: /tmp/files
    - source: salt://{{ tpldir }}/files
    - makedirs: True
    - keep_symlinks: True

where salt://{{ tpldir }}/files contains:

-rw-r--r-- 1 root root 15 Sep  3 12:59 file1
lrwxrwxrwx 1 root root  5 Sep  3 12:59 symlink1 -> file1

Steps to Reproduce the behavior
Apply the file.recurse state on the minion

Expected behavior
/tmp/files on the minion should contain (this is the case when using local file system as the backend):

-rw-r--r-- 1 root root 15 Sep  3 13:41 file1
lrwxrwxrwx 1 root root  5 Sep  3 13:41 symlink1 -> file1

However, when using a master using gitfs file system as the backend, /tmp/files ends up containing:

-rw-r--r-- 1 root root 15 Sep  3 13:02 file1
-rw-r--r-- 1 root root 15 Sep  3 13:02 symlink1

Versions Report

salt --versions-report

Master and minion running the same version on Rocky 9.4

Salt Version:
          Salt: 3006.9
 
Python Version:
        Python: 3.10.14 (main, Jun 26 2024, 11:44:37) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: unknown
  cryptography: 42.0.5
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: 4.0.11
     gitpython: 3.1.43
        Jinja2: 3.1.4
       libgit2: Not Installed
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.4.8
        PyYAML: 6.0.1
         PyZMQ: 23.2.0
        relenv: 0.17.0
         smmap: 5.0.1
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: rocky 9.4 Blue Onyx
        locale: utf-8
       machine: x86_64
       release: 5.14.0-427.24.1.el9_4.x86_64
        system: Linux
       version: Rocky Linux 9.4 Blue Onyx

Additional context
Also tried pygit2 as the gitfs backend - but makes no difference

Also tried a salt-master running 3007.1 with gitfs - but makes no difference

[dmurphy18]

Wonder if this fix #66857 being worked on has any bearing on the issue

[jcpearson]

Wonder if this fix #66857 being worked on has any bearing on the issue

I don't think it does ?

That PR seems to be about an intermittent issue with symlinks being created before their target exists ?

My issue is that file.recurse with 'keep_symlinks: True' works fine when the backend file system on the master is local, but doesn't work when the backend file system on the master is via gitfs

[dmurphy18]

Waiting on PR fixes #67105 and #66954 before addressing this issue.

[dmurphy18]

@jcpearson Can you retry with the latest Salt 3006.10, a number of gitfs fixes are contained within the release

[jcpearson]

Unfortunately I'm unable to test this now - I have moved jobs and not longer use Salt ...

It should be fairly easy for someone else to test this using a gitfs back end, by using the simple examples given earlier in this ticket