Skip to content

Iterative walk for sparsely traversing large nested directories #518

@clane9

Description

@clane9

I'm using CloudPath.walk to search a large nested directory, but I'm only descending into a very small subset of subdirectories (by removing elements from dirnames). However, it seems the current walk implementation builds the entire file tree in advance before walking. Have you considered an iterative walk similar to os.walk to support sparse searches like this? I imagine this would require more network requests, but possibly save time and data requested if the search is sufficiently sparse. An iterative implementation might also be more consistent with users' expectation, coming from os.walk/Path.walk.

Thanks for this super useful library :)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions