Add `top_down` parameter to `(i)glob` and `Path.(r)glob` matching `{os,Path}.walk`

[septatrix]

Feature or enhancement

Proposal:

Recursively searching for globs is a common use case when working with the filesystem. In cases where one modifies files while working in them it is useful to specify whether one first wants to recurse into directories or first get all the files in a directory before recursing into subdirs.

One simply example is when one wants to delete all empty directories:

for d in context.root.glob("**/", top_down=False):
    if d == context.root:
        continue

    if not any(d.iterdir()):
        d.rmdir()

Currently, one has two alternatives (apart from reimplementing glob) which have their respective drawbacks:

• for d in reversed(sorted(context.root.glob("**/"))): ... This eagerly consumes and sorts the iterator. This has the drawback of requiring a lot of memory for large trees and taking additional time for the sorting.
• for d, _, _ in context.root.walk(top_down=False): ... This works for this simple case but does not allow applying search patterns e.g. only looking for empty directories somewhere under .venv (e.g. **/.venv/**/)

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

[barneygale]

• for d in reversed(sorted(context.root.glob("**/"))): ... This eagerly consumes and sorts the iterator. This has the drawback of requiring a lot of memory for large trees and taking additional time for the sorting.

FWIW, I don't think you need sorted() to visit bottom-up. reversed() should be sufficient.

• This works for this simple case but does not allow applying search patterns e.g. only looking for empty directories somewhere under .venv (e.g. **/.venv/**/)

Could you use full_match() for this? It supports almost the same language as glob(), except that it doesn't match trailing slashes (because pathlib normalises those away)

[septatrix]

• for d in reversed(sorted(context.root.glob("**/"))): ... This eagerly consumes and sorts the iterator. This has the drawback of requiring a lot of memory for large trees and taking additional time for the sorting.

FWIW, I don't think you need sorted() to visit bottom-up. reversed() should be sufficient.

The API does not guarantee this (whether subdirectories are always emitted before/after regular file entries). So dropping the sort could mean sudden breaks upon version updates.

• This works for this simple case but does not allow applying search patterns e.g. only looking for empty directories somewhere under .venv (e.g. **/.venv/**/)

Could you use full_match() for this? It supports almost the same language as glob(), except that it doesn't match trailing slashes (because pathlib normalises those away)

I think that would work, however path.walk is still a cruft as it requires several ifs and thus pretty much impossible to use in list comprehensions or when wanting to return a simple iterator. Yes, one could write a wrapper around it but at that point it's pretty much just reimplementing rglob...