Skip to content

Projects list is always empty when initializing an S3 workspace #1848

@armandduijn

Description

@armandduijn

There seems to be a bug in the Workspace.list_projects() method when the workspace is configured with an S3 URI. The method returns an empty list, even when the workspace contains multiple projects.

ws = Workspace(path="s3://bucket/evidently")
print(ws.list_projects())
# Output: []
# Expected a list of projects

The root cause seems to lie in the code fragment below, where a dot . is passed to list the directory contents. While this works for POSIX file systems, it does not behave the same in S3. In S3, . is treated as part of the object key, meaning /foo and /foo/. are seen as distinct prefixes.

for p in self.location.listdir("."):

To confirm this behavior, I modified fslocation.py to log the path. The example below shows that, indeed, a . is being appended to the S3 path..

def listdir(self, path: str):
    try:
        fullpath = posixpath.join(self.path, path)
        print(fullpath)

# Outputs: bucket/artifacts/evidently/.
# Expected bucket/artifacts/evidently/

In another part of the codebase, directory listing is handled correct by passing an empty string instead of a ..

projects = [load_project(self.location, p) for p in self.location.listdir("") if self.location.isdir(p)]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions