fix(printf): redesign %q implementation for gnu compatibility by naoNao89 · Pull Request #9640 · uutils/coreutils

naoNao89 · 2025-12-12T01:06:24Z

Redesigned printf %q implementation to match bash behavior. Previous approach incorrectly used SHELL_ESCAPE (designed for ls). Created dedicated PrintfQuoter with proper algorithm: empty→'', simple→unchanged, metacharacters→backslash, control→$'...'. Includes 18 tests and related apostrophe bug fix.

codspeed-hq · 2025-12-12T01:21:08Z

Merging this PR will improve performance by 3.7%

⚡ 1 improved benchmark
✅ 287 untouched benchmarks
⏩ 38 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	Simulation	`cp_large_file[16]`	393.3 µs	379.3 µs	+3.7%

_{Comparing naoNao89:fix-printf-q-9638 (afb3f3b) with main (bd58575)}

38 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

github-actions · 2025-12-12T01:24:19Z

GNU testsuite comparison:

GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?

github-actions · 2025-12-12T01:58:22Z

GNU testsuite comparison:

GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

github-actions · 2025-12-12T03:43:26Z

GNU testsuite comparison:

GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

oech3 · 2025-12-12T05:32:19Z

Please ask GNU before opening/merging this. bash is not coreutils.

$ bash-builtin-printf "%q\n" $'\001'\'$'\001'
$'\001\'\001'
$ uu-printf "%q\n" $'\001'\'$'\001'
"'$'\001''''$'\001"
$ archlinux-gnu-printf "%q\n" $'\001'\'$'\001'
''$'\001'\'''$'\001'

@collinfunk

naoNao89 · 2025-12-12T05:35:30Z

💀

naoNao89 · 2025-12-12T05:36:32Z

# bash 5.2.37's builtin printf
prompt$ printf "%q\n" $'\001'\'$'\001'
$'\001\'\001'
prompt$ printf "%s" $'\001\'\001' | od -t x1
0000000 01 27 01
0000003

# GNU Coreutils current main HEAD's printf
prompt$ ./src/printf "%q\n" $'\001'\'$'\001'
''$'\001'\'''$'\001'
prompt$ printf "%s" ''$'\001'\'''$'\001' | od -t x1
0000000 01 27 01
0000003

./target/release/coreutils printf "%q\n" $'\001'\'$'\001'
$'\001\'\001'

github-actions · 2025-12-12T05:57:33Z

GNU testsuite comparison:

Skip an intermittent issue tests/printf/printf-quote (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

egmontkob · 2025-12-12T08:52:49Z

Thanks for working on this!

Previous approach incorrectly used SHELL_ESCAPE (designed for ls). Created dedicated PrintfQuoter [...]

I don't think it's the right approach. The naming of the cmdline option ls --quoting-style=shell-escape suggests to me that its exact purpose is to provide an output format that's properly escaped to be copy-pasteable into a shell command line. That is, escaped identically to printf %q.

I think reusing SHELL_ESCAPE (designed for ls) was the right thing to do, it's just that its implementation is incorrect. You should have a single code for handling these two cases, not two different ones.

I believe GNU Coreutils also fixed these two hand-in-hand, even though the NEWS entry doesn't specify the exact cmdline option of ls:

  ls and printf fix shell quoted output in the edge case of escaped
  first and last characters, and single quotes in the string.

The story is more complicated than this since there are 4 different relevant formats of ls: shell, shell-escape, shell-always, shell-escape-always; I'm not sure what they all do exactly, nor which ones were exactly affected by that recent fix in GNU Coreutils, but I'm fairly certain that printf %q shouldn't have yet another quoting style, it should share one (shell-escape I think) with ls. Or rather, semantically the other way around: printf %q's goal is clear, and I believe that one (or some?) of ls's shell-related quoting styles should match that.

naoNao89 · 2025-12-12T13:06:22Z

tks, found it

pub fn printf_quote(name: &OsStr) -> OsString {
    escape_name(name, QuotingStyle::SHELL_ESCAPE,

naoNao89 · 2025-12-12T14:14:45Z

use shared EscapedShellQuoter instead of separate PrintfQuoter,result: 12/21 tests pass. EscapedShellQuoter has bugs that affect both tools.

pixelb · 2025-12-12T15:16:11Z

Some notes from GNU ...

The difference between shell and shell-escape is that "shell-escape" uses POSIX $'\xxx' syntax to output non-printing characters, whereas the "shell" just outputs ? for such characters. I.e. shell-escape is the most general in that its output should always be copy and pasteable back to the shell to specify any file name. "shell-escape" is the default mode in ls when outputting to a tty. Also this is what printf %q uses since it's the most general. I'll expand on this a bit in the gnu docs.

So, yes ideally there would not be a separate implementation of ls --quoting-style and printf %q.

As for conciseness of output, the GNU output can have redundant leading single quotes, though that is for subtle alignment reasons. BTW folks can be very sensitive about this. I got threatening personal voicemails when I changed this alignment output slightly 10 years ago now. For example:

$ mkdir ta
$ touch ta/foo$'\001'bar ta/barfoo ta/foobar
$ ls -l ta
-rw-r--r--. 1 padraig padraig 0 Dec 12 15:00  barfoo
-rw-r--r--. 1 padraig padraig 0 Dec 12 15:00 'foo'$'\001''bar'
-rw-r--r--. 1 padraig padraig 0 Dec 12 15:00  foobar

Notice how the file names are aligned, whereas if we used the more concise foo$'\001'bar or even bash's $'foo\001bar' the alignment / visual indication of quoting would be different. But yes we probably could make the quoting more concise in some cases, like removing redundant leading '' for example, but saying that it's an edge case, so we didn't address that.

egmontkob · 2025-12-12T15:43:58Z

I got threatening personal voicemails

What the f.ck... I'm so sorry to hear this!

Thanks for the additional info!

There are multiple correct behaviors, and for ls some subjectcive visual beauty is surely preferable (which doesn't only influence the exact quoting method but also the indentation). For printf %q I'd argue that it doesn't matter since it's supposed to be used in scripts, its output hardly ever appears on the UI, so it should probably just stick to whatever format was picked for ls.

I don't think it's a problem if the two coreutilses (how to say this correctly? :)) use different format, as long as they are both technically correct (the shell resolves them to the same original string), but surely following GNU Coreutils is one possible reasonable choice.

github-actions · 2025-12-12T18:53:38Z

GNU testsuite comparison:

GNU test failed: tests/ls/quote-align. tests/ls/quote-align is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/ls/symlink-quote. tests/ls/symlink-quote is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/wc/wc-files0. tests/wc/wc-files0 is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/wc/wc-files0-from. tests/wc/wc-files0-from is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/printf/printf-quote (fails in this run but passes in the 'main' branch)

github-actions · 2025-12-13T08:20:21Z

GNU testsuite comparison:

Skip an intermittent issue tests/printf/printf-quote (fails in this run but passes in the 'main' branch)

github-actions · 2025-12-13T10:33:31Z

GNU testsuite comparison:

Skip an intermittent issue tests/printf/printf-quote (fails in this run but passes in the 'main' branch)

github-actions · 2025-12-13T11:49:26Z

GNU testsuite comparison:

Skip an intermittent issue tests/printf/printf-quote (fails in this run but passes in the 'main' branch)

github-actions · 2025-12-15T14:51:04Z

GNU testsuite comparison:

GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

github-actions · 2025-12-15T16:08:21Z

GNU testsuite comparison:

GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

github-actions · 2025-12-15T21:02:02Z

GNU testsuite comparison:

GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

github-actions · 2025-12-15T21:20:46Z

GNU testsuite comparison:

GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

github-actions · 2025-12-16T00:36:03Z

GNU testsuite comparison:

GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

github-actions · 2025-12-16T20:39:39Z

GNU testsuite comparison:

GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Congrats! The gnu test tests/tail/inotify-dir-recreate is now passing!

Fixes CI failure from PR uutils#9640 where GNU test suite update (commit ba3442f) exposed fundamental design flaws in printf %q shell-quoting implementation. Problem: Original code pre-scanned for control characters and wrapped ENTIRE strings in $'...' if ANY control char was present (e.g., "a\r" → $'a\r'). Solution: Implemented selective quoting that only wraps control characters themselves (e.g., "a\r" → a$'\r'), matching GNU coreutils behavior. Key Changes: - Removed has_control_chars() pre-scanning logic - Never start in dollar mode - enter/exit dynamically - Exit dollar mode when encountering regular chars (selective quoting) - Keep consecutive control chars in single dollar quote - Handle apostrophes by exiting dollar mode and using \' escape - Updated test expectations to match selective quoting behavior Examples: - 'a\tb' → a$'\t'b (not $'a\tb') - '\x01\x02\x03' → $'\001\002\003' (not $'\001'$'\002'$'\003') - 'hello\x01world' → hello$'\001'world (not $'hello\001world')

github-actions · 2025-12-17T18:51:21Z

GNU testsuite comparison:

GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/tail/inotify-dir-recreate is now passing!

sylvestre · 2025-12-30T21:43:39Z

@naoNao89 is it still a draft ? thanks

naoNao89 · 2026-01-03T02:57:31Z

readyy

Fixes CI failure from PR uutils#9640 where GNU test suite update (commit ba3442f) exposed fundamental design flaws in printf %q shell-quoting implementation. Problem: Original code pre-scanned for control characters and wrapped ENTIRE strings in $'...' if ANY control char was present (e.g., "a\r" → $'a\r'). Solution: Implemented selective quoting that only wraps control characters themselves (e.g., "a\r" → a$'\r'), matching GNU coreutils behavior. Key Changes: - Removed has_control_chars() pre-scanning logic - Never start in dollar mode - enter/exit dynamically - Exit dollar mode when encountering regular chars (selective quoting) - Keep consecutive control chars in single dollar quote - Handle apostrophes by exiting dollar mode and using \' escape - Updated test expectations to match selective quoting behavior Examples: - 'a\tb' → a$'\t'b (not $'a\tb') - '\x01\x02\x03' → $'\001\002\003' (not $'\001'$'\002'$'\003') - 'hello\x01world' → hello$'\001'world (not $'hello\001world')

github-actions · 2026-02-17T01:32:13Z

GNU testsuite comparison:

GNU test failed: tests/date/date-locale-hour. tests/date/date-locale-hour is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Congrats! The gnu test tests/tail/retry is no longer failing!
Note: The gnu test tests/rm/many-dir-entries-vs-OOM is now being skipped but was previously passing.

github-actions · 2026-02-17T03:54:27Z

GNU testsuite comparison:

GNU test failed: tests/cut/bounded-memory. tests/cut/bounded-memory is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Congrats! The gnu test tests/misc/io-errors is no longer failing!
Congrats! The gnu test tests/tail/retry is no longer failing!
Note: The gnu test tests/tail/tail-n0f is now being skipped but was previously passing.

github-actions · 2026-02-17T04:24:31Z

GNU testsuite comparison:

GNU test failed: tests/date/date-locale-hour. tests/date/date-locale-hour is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/misc/sleep. tests/misc/sleep is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/pr/bounded-memory (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/misc/io-errors is no longer failing!
Congrats! The gnu test tests/tail/retry is no longer failing!
Note: The gnu test tests/cp/link-heap is now being skipped but was previously passing.

Implement printf %q format specifier to match bash behavior for shell-escaping strings. Changes: - Fix integer overflow panic in extreme field width parsing - Update quoting logic for control characters with single quotes - Adjust test expectations for GNU compatibility Fixes uutils#9638

github-actions · 2026-02-17T04:41:28Z

GNU testsuite comparison:

GNU test failed: tests/printf/printf-quote. tests/printf/printf-quote is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/pr/bounded-memory (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/misc/io-errors is no longer failing!
Congrats! The gnu test tests/tail/retry is no longer failing!

naoNao89 force-pushed the fix-printf-q-9638 branch from 647e8cb to 9f3a0f9 Compare December 12, 2025 01:08

naoNao89 force-pushed the fix-printf-q-9638 branch 2 times, most recently from 04050fa to ac19251 Compare December 12, 2025 01:42

naoNao89 force-pushed the fix-printf-q-9638 branch from 9312501 to ac19251 Compare December 12, 2025 03:27

naoNao89 force-pushed the fix-printf-q-9638 branch from ac19251 to ee02b5e Compare December 12, 2025 05:27

naoNao89 force-pushed the fix-printf-q-9638 branch from a701296 to 78d0e23 Compare December 12, 2025 05:41

naoNao89 force-pushed the fix-printf-q-9638 branch 2 times, most recently from 92a9d4b to dd82476 Compare December 12, 2025 18:38

naoNao89 marked this pull request as draft December 12, 2025 20:49

naoNao89 changed the title ~~fix(printf): redesign %q implementation for bash compatibility~~ fix(printf): redesign %q implementation for gnu compatibility Dec 13, 2025

naoNao89 force-pushed the fix-printf-q-9638 branch from dd82476 to e449066 Compare December 13, 2025 08:04

naoNao89 force-pushed the fix-printf-q-9638 branch from e449066 to f751ca1 Compare December 13, 2025 10:17

naoNao89 force-pushed the fix-printf-q-9638 branch from f751ca1 to b987db6 Compare December 13, 2025 11:33

naoNao89 force-pushed the fix-printf-q-9638 branch 3 times, most recently from 1e6ef3e to aec867c Compare December 15, 2025 20:42

naoNao89 force-pushed the fix-printf-q-9638 branch from aec867c to 49c25d8 Compare December 15, 2025 21:04

naoNao89 force-pushed the fix-printf-q-9638 branch from 49c25d8 to 65373e4 Compare December 16, 2025 00:21

naoNao89 force-pushed the fix-printf-q-9638 branch from 9c29e2b to f54f118 Compare December 16, 2025 20:23

naoNao89 force-pushed the fix-printf-q-9638 branch from f54f118 to 8743bea Compare December 17, 2025 18:30

naoNao89 marked this pull request as ready for review January 1, 2026 23:31

naoNao89 force-pushed the fix-printf-q-9638 branch 2 times, most recently from f9b78e2 to 76194d6 Compare February 17, 2026 01:24

naoNao89 force-pushed the fix-printf-q-9638 branch from 76194d6 to 9cb8de0 Compare February 17, 2026 03:45

naoNao89 force-pushed the fix-printf-q-9638 branch from 352e1af to 9cb8de0 Compare February 17, 2026 04:15

naoNao89 force-pushed the fix-printf-q-9638 branch from 9cb8de0 to 51161b0 Compare February 17, 2026 04:30

naoNao89 force-pushed the fix-printf-q-9638 branch from 51161b0 to afb3f3b Compare February 17, 2026 04:32

Uh oh!

Conversation

naoNao89 commented Dec 12, 2025

Uh oh!

codspeed-hq bot commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by 3.7%

Performance Changes

Footnotes

Uh oh!

github-actions bot commented Dec 12, 2025

Uh oh!

github-actions bot commented Dec 12, 2025

Uh oh!

github-actions bot commented Dec 12, 2025

Uh oh!

oech3 commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

naoNao89 commented Dec 12, 2025

Uh oh!

naoNao89 commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 12, 2025

Uh oh!

egmontkob commented Dec 12, 2025

Uh oh!

naoNao89 commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

naoNao89 commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pixelb commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

egmontkob commented Dec 12, 2025

Uh oh!

github-actions bot commented Dec 12, 2025

Uh oh!

github-actions bot commented Dec 13, 2025

Uh oh!

github-actions bot commented Dec 13, 2025

Uh oh!

github-actions bot commented Dec 13, 2025

Uh oh!

github-actions bot commented Dec 15, 2025

Uh oh!

github-actions bot commented Dec 15, 2025

Uh oh!

github-actions bot commented Dec 15, 2025

Uh oh!

github-actions bot commented Dec 15, 2025

Uh oh!

github-actions bot commented Dec 16, 2025

Uh oh!

github-actions bot commented Dec 16, 2025

Uh oh!

github-actions bot commented Dec 17, 2025

Uh oh!

sylvestre commented Dec 30, 2025

Uh oh!

naoNao89 commented Jan 3, 2026

Uh oh!

github-actions bot commented Feb 17, 2026

Uh oh!

github-actions bot commented Feb 17, 2026

Uh oh!

github-actions bot commented Feb 17, 2026

Uh oh!

github-actions bot commented Feb 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

codspeed-hq bot commented Dec 12, 2025 •

edited

Loading

oech3 commented Dec 12, 2025 •

edited

Loading

naoNao89 commented Dec 12, 2025 •

edited

Loading

naoNao89 commented Dec 12, 2025 •

edited

Loading

naoNao89 commented Dec 12, 2025 •

edited

Loading

pixelb commented Dec 12, 2025 •

edited

Loading