Skip to content

Confusing PREFIX values printed during testing #2072

@h-vetinari

Description

@h-vetinari

I'm having to do some painful in-CI debugging, because the error doesn't reproduce locally (neither with the artefact from the PR, nor with a fresh local build)...

So I've instrumented some code with print statements and I'm very surprised to see log messages that make absolutely no sense to me. First, the code (+ my prints) roughly looks like

CONDA_PREFIX = os.environ.get('CONDA_PREFIX', None)
# turn into full path; windows has a different layout
CONDA_PREFIX = pathlib.Path(CONDA_PREFIX)/"Library" if WIN else pathlib.Path(CONDA_PREFIX)

class DLL(ctypes.CDLL):
  @staticmethod
  def findlib(nm:str, paths:list[str], extra_paths=[]):
    print(f"{nm=}, {paths=}, {extra_paths=}")
    for p in paths:
      libpaths = {"posix": [CONDA_PREFIX/"lib", "/usr/lib", "/usr/local/lib"],
                  "nt": [CONDA_PREFIX/"bin"] + os.environ['PATH'].split(os.pathsep),
                  "darwin": ["/opt/homebrew/lib", f"/System/Library/Frameworks/{p}.framework"],
                  'linux': ['/lib', f"/lib/{sysconfig.get_config_var('MULTIARCH')}", "/usr/lib/wsl/lib/"]}
      for pre in (pathlib.Path(pre) for pre in ([path] if path else []) + libpaths.get(os.name, []) + libpaths.get(sys.platform, []) + extra_paths):
        print(f"{pre=}, {p=}")
        if not pre.is_dir(): continue
        if LINUX:
          for l in (l for l in pre.iterdir() if l.is_file() and re.fullmatch(f"lib{p}\\.so\\.?[0-9]*", l.name)):
            print(f"considering file {l}")
            # filter out linker scripts
            with open(l, 'rb') as f:
              if f.read(4) == b'\x7FELF': return str(l)

However, what I'm seeing in the logs is

 │ │ Installing test environment
 │ │ ✔ Successfully updated the test environment
 │ │ nm='llvm', paths=['LLVM', 'LLVM-21', 'LLVM-20', 'LLVM-19', 'LLVM-18', 'LLVM-17', 'LLVM-16', 'LLVM-15', 'LLVM-14'], extra_paths=[]
 │ │ pre=PosixPath('$SRC_DIR_env/lib'), p='LLVM'
 │ │ considering file $SRC_DIR_env/lib/libLLVM.so

There's no point in the code that should produce anything remotely resembling the string $SRC_DIR_env, so I can imagine two scenarios. Either rattler build creates the actual prefix in a location that's named $SRC_DIR_env (though I cannot find that string in the repo here), or alternatively, that there's some "backwards replacement" (like conda-build does as well AFAIK) of instances of whatever path is chosen for the prefix back to the variable that represents it.

If it's the latter, then there's a bug. In both cases, the actual path of the environment prefix should be shown in the logs, e.g.

 ╭─ Running script test for recipe: tinygrad-0.11.1dev-cuda129_py312_hd6717e9_0.conda
 │ 
 │ Resolving test environment:
 │   Platform: linux-64 [__unix=0=0, __linux=6.11.0=0, __glibc=2.39=0, __cuda=12.9=0, __archspec=1=x86_64_v4]
 │   Channels: 
 │    - file:///tmp/.tmp5BH9WZ/
 │    - file:///home/conda/feedstock_root/build_artifacts/
 │    - conda-forge
 │   Specs:
 │    - tinygrad ==0.11.1dev cuda129_py312_hd6717e9_0
 │   Path:                                                   # <---
 │    - the_concrete_path_used_for_this_environment

not just for testing either, but for every environment created during the build. Knowing the concrete paths being used is crucial for debugging.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions