Linux filenames with slashes

[dazinator]

As mentioned on #88 by @aiurovet

I need to add test coverage and potentially solve an issue where, on Linux, if a file path contains a backslash, it would be tokenized as a Path Seperator and therefore matched as a path seperator but really its actually part of the filename. I don't think this is a problem for windows file names as slashes in any direction are illegal characters, and therefore always interpreted as path separators.

This may be tricky to resolve, as dotnet.glob does not inherently know where the string it's matching is coming from. It could be a URL path, a windows registry path, a file system path, I.e it's arbitrary.

However on Linux when passing a Linux file system name that contains a backwards Slash, it would catch you out to see the glob pattern fail to match.

E.g glob pattern foo*/bar would be expected to match the linux file path foo\gh/bar however it doesn't currently match because DotNet.Glob tokenises the backslash as a path seperator, and * only matches all characters within the current paths segment. gh are therefore in their own segment and not matched by the glob pattern anywhere.

However on the flip side, in all other scenarios like where the path is not a Linux file path, it might be perfectly valid to expect this behaviour and for the match to fail.

Therefore I think there needs to be some setting that influences the parser and tells it whether to interpret \ as a path separator, or as a literal.

I could consider defaulting this setting on Linux to treat it literally, however that would be a behavioural breaking change so would need a semver major bump and appropriate notice. I think it probably better to not enable this setting automatically and instead document that this setting should be set when passing file paths from Linux to the library.

[aiurovet]

Not quite. Even on Windows, foo\gh/bar (or foo\gh\bar, or foo/gh/bar) should not match the pattern foo*/bar, but only foo**/bar. Please refer to GNU documentation I mentioned already: section globstar at https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html

I believe, the major misunderstanding arises from what majority of sites wrongly suggest that the wildcard * translates into the regexp .* In fact, it is right for filenames only, and generally, should not cross the path separator boundary. So * should translate into [^\/\\]* of Windows, and into [^\/]* on POSIX-compliant OSes

[dazinator]

Even on Windows, foo\gh/bar (or foo\gh\bar, or foo/gh/bar) should not match the pattern foo*/bar, but only foo**/bar.

Yeah I am saying the same thing - just in a long fashion, so don't worry there is no misunderstanding here. The only thing I will say is that I am not planning on removing the constraint of ** being the only content of a segment so you wouldn't use foo**/bar with this library, but foo/**/bar or foo*/**/bar

[ericnewton76]

I think potentially you add the concept of the globmatcher, which could be WindowsGlobMatcher LinuxPathGlobMatch etc which each one can give context to what the delimiters might be

So for example, FileSystemGlobber would load either WindowsGlob or LinuxGlobber and enumerate files in a directory, yield returning any that glob.IsMatch returns true

I actually have a use-case scenario of a listing of locations in a database, and a globber would help users select the ones they want to operate on. For example, all locations that start with a = [a]* or a first character, and z last character [a]*[z] realistically i'll just pass their spec string as to Sql Like function (so a% or a%z)

[fubar-coder]

Just FYI: I implemented an extension method that fixes the backslash situation:

public static class GlobbingExtensions
{
    extension(Glob glob)
    {
        /// <summary>
        /// Normalizes glob tokens by absorbing backslash path separators into adjacent
        /// <see cref="LiteralToken"/> instances.
        /// </summary>
        /// <remarks>
        /// This normalization includes:
        /// - merging backslash <see cref="PathSeparatorToken"/>s into literals,
        /// - absorbing a leading backslash from a following <see cref="WildcardDirectoryToken"/>, and
        /// - absorbing a trailing backslash from a preceding <see cref="WildcardDirectoryToken"/>.
        /// Consecutive literals are merged; all other tokens are preserved.
        /// </remarks>
        public Glob FixBackslashes()
        {
            // High-level intent:
            // - DotNet.Globbing treats path separators as dedicated tokens. On Windows, a '\\' can legally
            //   appear inside literal segments (e.g., "C:\\Users\\Public").
            // - This method re-writes the token stream, absorbing certain backslash separators into adjacent
            //   LiteralToken(s), so that downstream consumers can treat those backslashes as part of the literal
            //   rather than structural separators.
            // - Only backslashes are handled here. Forward slashes ('/') are left untouched.
            // - Single pass, O(n) over the token list, without changing the overall match semantics.

            // Work on the token sequence of this glob pattern.
            var tokens = glob.Tokens;

            // We'll build a new token list where certain backslash separators are absorbed into adjacent literals.
            // Invariants:
            // - Order of non-absorbed tokens is preserved.
            // - Any absorbed PathSeparatorToken ('\\') either becomes part of a LiteralToken
            //   or is removed when merged into a neighboring token.
            var result = new List<IGlobToken>(tokens.Length);

            IGlobToken? previousToken = null;
            foreach (var token in tokens)
            {
                switch (token)
                {
                    case PathSeparatorToken { Value: Backslash }
                        when previousToken is WildcardDirectoryToken { TrailingPathSeparator.Value: Backslash } prevWdt:
                        // Case A: "**" had a trailing backslash, followed by another backslash separator.
                        // Example: "**\\\\" → absorb the trailing '\\' from the previous WDT and emit a literal "\\\\".
                        // Rationale: Two structural backslashes in a row are not meaningful; treat the second as a literal.
                        result[^1] = new WildcardDirectoryToken(prevWdt.LeadingPathSeparator, null);
                        result.Add(previousToken = new LiteralToken($"{Backslash}{Backslash}"));
                        break;

                    case LiteralToken lit
                        when previousToken is WildcardDirectoryToken { TrailingPathSeparator.Value: Backslash } prevWdt:
                        // Case B: Previous token is a WDT with a trailing '\\', current is a literal.
                        // Action: Move the trailing '\\' into the literal so the literal starts with a backslash.
                        // Example: "**\\foo" → WDT (trailing null), Literal("\\foo").
                        result[^1] = new WildcardDirectoryToken(prevWdt.LeadingPathSeparator, null);
                        result.Add(previousToken = new LiteralToken(Backslash + lit.Value));
                        break;

                    case LiteralToken lit when previousToken is LiteralToken prevLitConcat:
                        // Case C: Merge adjacent literals to avoid fragmentation and repeated concatenations later.
                        // Example: Literal("C:"), Literal("\\Users") → Literal("C:\\Users").
                        result[^1] = previousToken = new LiteralToken(prevLitConcat.Value + lit.Value);
                        break;
                    
                    case PathSeparatorToken { Value: Backslash } when previousToken is null:
                        // Case D: Leading backslash at the very beginning of the pattern.
                        // Treat it as a literal to preserve absolute/UNC semantics in Windows paths.
                        result.Add(previousToken = new LiteralToken(Backslash.ToString()));
                        break;

                    case PathSeparatorToken { Value: Backslash } when previousToken is LiteralToken prevLitPath:
                        // Case E: A backslash right after a literal → absorb into that literal.
                        // Example: Literal("C:"), '\\' → Literal("C:\\").
                        result[^1] = previousToken = new LiteralToken(prevLitPath.Value + Backslash);
                        break;

                    case WildcardDirectoryToken { LeadingPathSeparator.Value: Backslash } wdt
                        when previousToken is LiteralToken prevLitWdt:
                        // Case F: WDT whose leading separator is a backslash and is preceded by a literal.
                        // Action: Move the leading '\\' into the preceding literal, then keep the WDT without leading sep.
                        // Example: Literal("C:"), WDT("\\**") → Literal("C:\\"), WDT("**").
                        result[^1] = new LiteralToken(prevLitWdt.Value + Backslash);
                        result.Add(previousToken = new WildcardDirectoryToken(null, wdt.TrailingPathSeparator));
                        break;

                    default:
                        // Default: Keep the token as-is. This covers forward slashes, non-backslash separators,
                        // wildcard tokens without adjacent-backslash conditions, character sets, etc.
                        result.Add(previousToken = token);
                        break;
                }
            }

            // Rebuild and return a new glob with the transformed token list.
            // Note: We construct a new Glob to keep the original immutable. The new token stream preserves
            // matching semantics while normalizing how backslashes are represented.
            return new Glob(result.ToArray());
        }
    }
}

This is the xUnit3 test case:

    [Theory]
    [InlineData(@"a\*.txt", @"a\b.txt", true)]
    [InlineData(@"a\*.txt", "a/b.txt", false)]
    public void TestMatches(string pattern, string path, bool isMatch)
    {
        var glob = Glob.Parse(pattern).FixBackslashes();
        Assert.Equal(isMatch, glob.IsMatch(path));
    }