Skip to content

Improve windows path parsing#329

Merged
EdSchouten merged 1 commit intobuildbarn:mainfrom
tomgr:chore/windows-namespace-prefixes
Mar 16, 2026
Merged

Improve windows path parsing#329
EdSchouten merged 1 commit intobuildbarn:mainfrom
tomgr:chore/windows-namespace-prefixes

Conversation

@tomgr
Copy link
Contributor

@tomgr tomgr commented Mar 9, 2026

This PR is necessary to fix some bugs with the WinFSP vfs integration in bb-remote-execution (see buildbarn/bb-remote-execution#213).

Firstly it changes the pretty printing of windows paths to append \. as a bare trailing backslash is not a valid pathname component.

Secondly this neatens up the parsing of Windows file paths and adds handling for paths starting with \\.\.

Claude helped write some of this -- particularly the tests (it has more patience than I do at converting escape sequences!).

@aspect-workflows
Copy link

aspect-workflows bot commented Mar 9, 2026

Test

3 test targets passed

Targets
//pkg/blobstore/sharding/integration:integration_test [k8-fastbuild]                 75ms
//pkg/filesystem/path:path_test [k8-fastbuild]                                       128ms
//pkg/filesystem:filesystem_test [k8-fastbuild]                                      143ms

Total test execution time was 346ms. 26 tests (89.7%) were fully cached saving 5s.

@tomgr tomgr force-pushed the chore/windows-namespace-prefixes branch from acd8c86 to 1e36e61 Compare March 9, 2026 17:10
// directory. This is needed when constructing symlink
// substitute names for Windows reparse points, where a
// trailing backslash can produce consecutive backslashes
// after path substitution.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just want to get clarified: is this a limitation/feature of Windows, or is it really just WinFSP that freaks out in case symlink targets have trailing slashes? What happens if you create a symlink on NTFS that has a trailing slash?

In the PR description I see you mention that the issue is in FspFileSystemResolveReparsePointsInternal. Maybe instead of fixing this on the Buildbarn side, we should report an issue against WinFSP that they fix this?

Copy link
Member

@EdSchouten EdSchouten Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, what if instead of a trailing slash we append \. ? Is that valid?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the Windows standards require symlink reparse buffers not to have a trailing slash. The symbolic link reparse data buffer docs state that the substitute name has to be a pathname. However, the pathname documentation does not allow an empty final component: each component of the path has to have at least one character.

\. is valid; WinFSP would form \.\ after substitution which does seem to get handled correctly. However I thought that wouldn't be as elegant as that would mean that whenever directory paths were printed with WindowsPathFormatStandard (which is the default) they would have a trailing \.\. There's also a minor concern which is that Windows has a fairly low limit on path sizes, so keeping them short is always a good idea.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still have a preference for appending \. instead of removing the suffix entirely. One intentional design goal of the pathname library was that it does not relax paths in any way.

That MS-FSCC link that you shared is interesting. It raises the question: is it ever valid for us to generate Windows pathnames that end with slashes? If not, maybe we should just make a change like this?

diff --git pkg/filesystem/path/builder.go pkg/filesystem/path/builder.go
index 80be90a..a4ff01d 100644
--- pkg/filesystem/path/builder.go
+++ pkg/filesystem/path/builder.go
@@ -116,7 +116,7 @@ func (b *Builder) GetWindowsString(format WindowsPathFormat) (string, error) {
        // backslash.
        suffix := b.suffix
        if suffix == "/" {
-               suffix = "\\"
+               suffix = "\\."
        }
        out.WriteString(suffix)
        return out.String(), nil

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I've done this. I think it should be ok given the definition of pathname. And the winfsp tests all pass in bb-remote-execution with this.

@tomgr tomgr force-pushed the chore/windows-namespace-prefixes branch from 1e36e61 to 80f5bd7 Compare March 10, 2026 14:34
@tomgr tomgr marked this pull request as draft March 16, 2026 08:59
@tomgr tomgr force-pushed the chore/windows-namespace-prefixes branch 2 times, most recently from dc929ad to 5d53541 Compare March 16, 2026 17:09
@tomgr tomgr changed the title Propagate Windows namespace prefixes Improve windows file parsing Mar 16, 2026
@tomgr tomgr changed the title Improve windows file parsing Improve windows path parsing Mar 16, 2026
@tomgr tomgr marked this pull request as ready for review March 16, 2026 18:32
// We append "\." because a bare trailing backslash is not a valid pathname component:
// https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/ffb795f3-027d-4a3c-997d-3085f2332f6f
if format == WindowsPathFormatDevicePath {
// But device paths bypass win32 path normalization, so we can't use `\\.`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please remove these two comments, and update/augment the larger comment above it to fully describe why the code is structured the way it is?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure done!

Also consolidate namespace prefix stripping in the Windows path
parser and add support for the \\.\ device namespace prefix.
@tomgr tomgr force-pushed the chore/windows-namespace-prefixes branch from 5d53541 to d7bfd38 Compare March 16, 2026 19:01
@EdSchouten EdSchouten merged commit 6d68947 into buildbarn:main Mar 16, 2026
3 checks passed
@tomgr tomgr deleted the chore/windows-namespace-prefixes branch March 16, 2026 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants