gh-136156: Allow using linkat() with TemporaryFile #136281

vstinner · 2025-07-04T10:55:06Z

tempfile.TemporaryFile() no longer uses os.O_EXCL with os.O_TMPFILE, so it's possible to use linkat() on the file descriptor.

Issue: Don't use O_EXCL when using O_TMPFILE #136156

tempfile.TemporaryFile() no longer uses os.O_EXCL with os.O_TMPFILE, so it's possible to use linkat() on the file descriptor.

serhiy-storchaka

LGTM.

serhiy-storchaka · 2025-07-04T19:07:25Z

Could you add a test using the proc filesystem? I tried to implement AT_EMPTY_PATH support (see #89341), but it only works for root.

serhiy-storchaka

LGTM. 👍

serhiy-storchaka · 2025-07-08T15:54:39Z

Lib/test/test_tempfile.py

            self.assertEqual(os.listdir(dir), [])

+        @unittest.skipUnless(tempfile._O_TMPFILE_WORKS, 'need os.O_TMPFILE')
+        @unittest.skipUnless(os.path.exists('/proc/self/fd'),


Maybe even isdir()?

serhiy-storchaka · 2025-07-08T15:55:36Z

!buildbot BSD

bedevere-bot · 2025-07-08T15:55:38Z

🤖 New build scheduled with the buildbot fleet by @serhiy-storchaka for commit c912963 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136281%2Fmerge

The command will test the builders whose names match following regular expression: BSD

The builders matched are:

AMD64 FreeBSD PR
AMD64 FreeBSD14 PR
AMD64 FreeBSD Refleaks PR
AMD64 FreeBSD15 PR

vstinner · 2025-07-08T15:59:02Z

!buildbot BSD

I tested manually: the test is skipped since os.O_TMPFILE is missing.

vstinner · 2025-07-08T16:40:11Z

Merged, thanks for review @serhiy-storchaka.

bedevere-bot · 2025-07-08T17:00:04Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Android 3.x (tier-3) has failed when building commit 6c81e8c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1594/builds/2820) and take a look at the build logs.
Check if the failure is related to this commit (6c81e8c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1594/builds/2820

Failed tests:

test_tempfile

Failed subtests:

test_link_tmpfile - test.test_tempfile.TestTemporaryFile.test_link_tmpfile

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/data/user/0/org.python.testbed/files/python/lib/python3.15/test/test_tempfile.py", line 1614, in test_link_tmpfile
    os.link(f'/proc/self/fd/{fd}',
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
            filename,
            ^^^^^^^^^
            follow_symlinks=True)
            ^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/proc/self/fd/82' -> '/data/user/0/org.python.testbed/cache/tmpd0x9o66s/link'


Traceback (most recent call last):
  File "/data/user/0/org.python.testbed/files/python/lib/python3.15/test/test_tempfile.py", line 1614, in test_link_tmpfile
    os.link(f'/proc/self/fd/{fd}',
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
            filename,
            ^^^^^^^^^
            follow_symlinks=True)
            ^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/proc/self/fd/82' -> '/data/user/0/org.python.testbed/cache/tmpmjq3eb8x/link'

vstinner · 2025-07-08T17:12:55Z

Oh, the test fails with PermissionError on Android:

ERROR: test_link_tmpfile (test.test_tempfile.TestTemporaryFile.test_link_tmpfile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/user/0/org.python.testbed/files/python/lib/python3.15/test/test_tempfile.py", line 1614, in test_link_tmpfile
    os.link(f'/proc/self/fd/{fd}',
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
            filename,
            ^^^^^^^^^
            follow_symlinks=True)
            ^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/proc/self/fd/82' -> '/data/user/0/org.python.testbed/cache/tmpmjq3eb8x/link'

vstinner · 2025-07-08T17:17:11Z

I wrote #136430 to fix the test on Android.

bedevere-bot · 2025-07-08T17:34:48Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Android 3.x (tier-3) has failed when building commit 6c81e8c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1591/builds/2548) and take a look at the build logs.
Check if the failure is related to this commit (6c81e8c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1591/builds/2548

Failed tests:

test_tempfile

Failed subtests:

test_link_tmpfile - test.test_tempfile.TestTemporaryFile.test_link_tmpfile

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/data/user/0/org.python.testbed/files/python/lib/python3.15/test/test_tempfile.py", line 1614, in test_link_tmpfile
    os.link(f'/proc/self/fd/{fd}',
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
            filename,
            ^^^^^^^^^
            follow_symlinks=True)
            ^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/proc/self/fd/83' -> '/data/user/0/org.python.testbed/cache/tmprqz0o6wv/link'


Traceback (most recent call last):
  File "/data/user/0/org.python.testbed/files/python/lib/python3.15/test/test_tempfile.py", line 1614, in test_link_tmpfile
    os.link(f'/proc/self/fd/{fd}',
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
            filename,
            ^^^^^^^^^
            follow_symlinks=True)
            ^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/proc/self/fd/83' -> '/data/user/0/org.python.testbed/cache/tmpocwzcriy/link'

vstinner · 2025-07-10T15:18:29Z

Oh. The test fails on Arch Linux buildbots: https://buildbot.python.org/#/builders/582/builds/6857

ERROR: test_link_tmpfile (test.test_tempfile.TestTemporaryFile.test_link_tmpfile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.asan/build/Lib/test/test_tempfile.py", line 1614, in test_link_tmpfile
    os.link(f'/proc/self/fd/{fd}',
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
            filename,
            ^^^^^^^^^
            follow_symlinks=True)
            ^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/proc/self/fd/3' -> '/tmp/test_python_712__99i/tmpgkq51fdc/link'

tempfile.TemporaryFile() no longer uses os.O_EXCL with os.O_TMPFILE, so it's possible to use linkat() on the file descriptor.

vstinner · 2025-08-05T13:26:56Z

Oh. The test fails on Arch Linux buildbots: https://buildbot.python.org/#/builders/582/builds/6857

The buildbot is back to green.

tempfile.TemporaryFile() no longer uses os.O_EXCL with os.O_TMPFILE, so it's possible to use linkat() on the file descriptor.

pythongh-136156: Allow using linkat() with TemporaryFile

97c3d25

tempfile.TemporaryFile() no longer uses os.O_EXCL with os.O_TMPFILE, so it's possible to use linkat() on the file descriptor.

bedevere-app bot added the awaiting core review label Jul 4, 2025

bedevere-app bot mentioned this pull request Jul 4, 2025

Don't use O_EXCL when using O_TMPFILE #136156

Open

serhiy-storchaka approved these changes Jul 4, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Jul 4, 2025

Add test

c912963

serhiy-storchaka approved these changes Jul 8, 2025

View reviewed changes

vstinner merged commit 6c81e8c into python:main Jul 8, 2025
43 of 44 checks passed

vstinner deleted the tempfile branch July 8, 2025 16:39

bedevere-app bot removed the awaiting merge label Jul 8, 2025

vstinner mentioned this pull request Jul 8, 2025

gh-136156: Skip test_tempfile.test_link_tmpfile() on Android #136430

Merged

vstinner mentioned this pull request Jul 11, 2025

gh-136156: Remove tempfile test_link_tmpfile() #136534

Merged

Uh oh!

gh-136156: Allow using linkat() with TemporaryFile #136281

gh-136156: Allow using linkat() with TemporaryFile #136281

Uh oh!

Conversation

vstinner commented Jul 4, 2025 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

serhiy-storchaka commented Jul 4, 2025

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

serhiy-storchaka Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

serhiy-storchaka commented Jul 8, 2025

Uh oh!

bedevere-bot commented Jul 8, 2025

Uh oh!

vstinner commented Jul 8, 2025

Uh oh!

Uh oh!

vstinner commented Jul 8, 2025

Uh oh!

bedevere-bot commented Jul 8, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

vstinner commented Jul 8, 2025

Uh oh!

vstinner commented Jul 8, 2025

Uh oh!

bedevere-bot commented Jul 8, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

vstinner commented Jul 10, 2025

Uh oh!

vstinner commented Aug 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vstinner commented Jul 4, 2025 •

edited by bedevere-app bot

Loading