Update test environments by giordano · Pull Request #80 · JuliaPackaging/Preferences.jl

giordano · 2025-08-10T12:02:30Z

This fixes some test failures on nightly, but there are still other errors I don't know what to do about.

codecov · 2025-08-10T12:03:51Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.37%. Comparing base (b9bb86e) to head (5b2d806).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #80      +/-   ##
==========================================
- Coverage   93.85%   93.37%   -0.49%     
==========================================
  Files           2        2              
  Lines         179      166      -13     
==========================================
- Hits          168      155      -13     
  Misses         11       11

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

giordano · 2025-08-10T12:05:27Z

Preferences: Test Failed at /home/runner/work/Preferences.jl/Preferences.jl/test/runtests.jl:97
  Expression: did_precompile(output)
   Evaluated: did_precompile("\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mRejecting cache file /tmp/jl_NKUBuh/compiled/v1.13/UsesPreferences/B1CNm_qZdrf.ji because preferences hash does not match 0x0 != 0x7d24e6f85a1be55d\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:4120\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mRejecting cache file /tmp/jl_NKUBuh/compiled/v1.13/UsesPreferences/B1CNm_qZdrf.ji because preferences hash does not match 0x0 != 0x7d24e6f85a1be55d\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:4120\e[39m\n\e[32m\e[1mPrecompiling\e[22m\e[39m packages...\n\e[36m\e[1mInfo\e[22m\e[39m Given UsesPreferences was explicitly requested, output will be shown live \e[0K\n\e[0K\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Printf/3FQLY_qZdrf.so for Printf [de0858da-6303-5e67-8744-51eddeeeb8d7]\n\e[0K\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[0K\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Dates/p8See_qZdrf.so for Dates [ade2ca70-3891-5945-98fb-dc099432e06a]\n\e[0K\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[0K\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/TOML/mjrwE_qZdrf.so for TOML [fa267f1f-6049-4f14-aa54-33bafae1ed76]\n\e[0K\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[0K\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Preferences/pWSk8_qZdrf.so for Preferences [21216c6a-2e73-6563-6e65-726566657250]\n\e[0K\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n    508.5 ms\e[32m  ✓ \e[39mUsesPreferences\n  1 dependency successfully precompiled in 1 seconds. 5 already precompiled.\n  \e[33m1\e[39m dependency had output during precompilation:\e[33m\n┌ \e[39mUsesPreferences\e[33m\n│  \e[39m[Output was shown above]\e[33m\n└  \e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Printf/3FQLY_qZdrf.so for Printf [de0858da-6303-5e67-8744-51eddeeeb8d7]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Dates/p8See_qZdrf.so for Dates [ade2ca70-3891-5945-98fb-dc099432e06a]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/TOML/mjrwE_qZdrf.so for TOML [fa267f1f-6049-4f14-aa54-33bafae1ed76]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Preferences/pWSk8_qZdrf.so for Preferences [21216c6a-2e73-6563-6e65-726566657250]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/UsesPreferences/B1CNm_Mvual.so for UsesPreferences [056c4eb5-4491-6b91-3d28-8fffe3ee2af9]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Base64/D7K0n_qZdrf.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/StyledStrings/UcVoM_qZdrf.so for StyledStrings [f489334b-da3d-4c2e-b8f0-e476e12c162b]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/JuliaSyntaxHighlighting/8OCEv_qZdrf.so for JuliaSyntaxHighlighting [ac6e5ff7-fb65-4e79-a425-ec3bc9c03011]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Markdown/AREjX_qZdrf.so for Markdown [d6f4376e-aef5-505a-96c1-9c027394607a]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/InteractiveUtils/0TrXF_qZdrf.so for InteractiveUtils [b77e0a4c-d291-57a0-90e8-8db25a27a240]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Serialization/zGad9_qZdrf.so for Serialization [9e88b42a-f829-5b0c-bbe9-9e923198166b]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Logging/PWFjL_qZdrf.so for Logging [56ddb016-857b-54e1-b83d-db4d58db5568]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n\e[34m\e[1m┌ \e[22m\e[39m\e[34m\e[1mDebug: \e[22m\e[39mLoading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Test/JfdTE_qZdrf.so for Test [8dfed614-e22c-5e08-85e1-65c5234f0b40]\n\e[34m\e[1m└ \e[22m\e[39m\e[90m@ Base loading.jl:1270\e[39m\n")
  Stacktrace:
     [1] macro expansion
       @ /opt/hostedtoolcache/julia/nightly/x64/share/julia/stdlib/v1.13/Test/src/Test.jl:745 [inlined]
     [2] (::var"#10#11"{String})()
       @ Main ~/work/Preferences.jl/Preferences.jl/test/runtests.jl:97
     [3] (::var"#with_temp_depot##0#with_temp_depot##1"{var"#10#11"{String}})(dir::String)
       @ Main ~/work/Preferences.jl/Preferences.jl/test/runtests.jl:20
     [4] mktempdir(fn::var"#with_temp_depot##0#with_temp_depot##1"{var"#10#11"{String}}, parent::String; prefix::String)
       @ Base.Filesystem ./file.jl:899
   ┌ [5] mktempdir
   │   @ ./file.jl:895 [inlined]
   ╰──── repeated 2 times
     [7] with_temp_depot(f::var"#10#11"{String})
       @ Main ~/work/Preferences.jl/Preferences.jl/test/runtests.jl:15
     [8] top-level scope
       @ ~/work/Preferences.jl/Preferences.jl/test/runtests.jl:55
     [9] macro expansion
       @ /opt/hostedtoolcache/julia/nightly/x64/share/julia/stdlib/v1.13/Test/src/Test.jl:1929 [inlined]
    [10] macro expansion
       @ ~/work/Preferences.jl/Preferences.jl/test/runtests.jl:57 [inlined]
Test Summary: | Pass  Fail  Total   Time
Preferences   |   37     1     38  55.8s
RNG of the outermost testset: Random.Xoshiro(0x2ce972da4d4cb996, 0xd0b54bc7f9f6bde4, 0x8813135dc879bcd6, 0xa4512e5d588ea9cf, 0xc5a0fb9491784819)
ERROR: LoadError: Some tests did not pass: 37 passed, 1 failed, 0 errored, 0 broken.
in expression starting at /home/runner/work/Preferences.jl/Preferences.jl/test/runtests.jl:53
Package Preferences errored during testing
Error: Process completed with exit code 1.

giordano · 2025-08-10T12:07:53Z

Actual output is:

Info Given UsesPreferences was explicitly requested, output will be shown live
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Printf/3FQLY_qZdrf.so for Printf [de0858da-6303-5e67-8744-51eddeeeb8d7]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Dates/p8See_qZdrf.so for Dates [ade2ca70-3891-5945-98fb-dc099432e06a]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/TOML/mjrwE_qZdrf.so for TOML [fa267f1f-6049-4f14-aa54-33bafae1ed76]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Preferences/pWSk8_qZdrf.so for Preferences [21216c6a-2e73-6563-6e65-726566657250]
└ @ Base loading.jl:1270
    508.5 ms  ✓ UsesPreferences
  1 dependency successfully precompiled in 1 seconds. 5 already precompiled.
  1 dependency had output during precompilation:
┌ UsesPreferences
│  [Output was shown above]
└
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Printf/3FQLY_qZdrf.so for Printf [de0858da-6303-5e67-8744-51eddeeeb8d7]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Dates/p8See_qZdrf.so for Dates [ade2ca70-3891-5945-98fb-dc099432e06a]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/TOML/mjrwE_qZdrf.so for TOML [fa267f1f-6049-4f14-aa54-33bafae1ed76]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Preferences/pWSk8_qZdrf.so for Preferences [21216c6a-2e73-6563-6e65-726566657250]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/UsesPreferences/B1CNm_Mvual.so for UsesPreferences [056c4eb5-4491-6b91-3d28-8fffe3ee2af9]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Base64/D7K0n_qZdrf.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/StyledStrings/UcVoM_qZdrf.so for StyledStrings [f489334b-da3d-4c2e-b8f0-e476e12c162b]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/JuliaSyntaxHighlighting/8OCEv_qZdrf.so for JuliaSyntaxHighlighting [ac6e5ff7-fb65-4e79-a425-ec3bc9c03011]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Markdown/AREjX_qZdrf.so for Markdown [d6f4376e-aef5-505a-96c1-9c027394607a]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/InteractiveUtils/0TrXF_qZdrf.so for InteractiveUtils [b77e0a4c-d291-57a0-90e8-8db25a27a240]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Serialization/zGad9_qZdrf.so for Serialization [9e88b42a-f829-5b0c-bbe9-9e923198166b]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Logging/PWFjL_qZdrf.so for Logging [56ddb016-857b-54e1-b83d-db4d58db5568]
└ @ Base loading.jl:1270
┌ Debug: Loading object cache file /tmp/jl_NKUBuh/compiled/v1.13/Test/JfdTE_qZdrf.so for Test [8dfed614-e22c-5e08-85e1-65c5234f0b40]
└ @ Base loading.jl:1270

Is the test at

Preferences.jl/test/runtests.jl

Lines 88 to 91 in 3ffa542

    
           function did_precompile(output) 
        
               occursin("Precompiling UsesPreferences [$(string(up_uuid))]", output) || 
        
               occursin("Precompiling\e[22m\e[39m UsesPreferences", output) 
        
           end

simply not right in newer versions of julia? CC @IanButterworth.

IanButterworth · 2025-08-10T12:32:27Z

Some of the messaging around precompiling has changed and been backported. Ideally we wouldn't have such strict messaging tests here.

giordano · 2025-08-10T12:34:25Z

Any other way to detect that a package was precompiled? Now I added

Preferences.jl/test/runtests.jl

Line 91 in 044c236

occursin(r"\e\[32m +✓ +\e\[39mUsesPreferences", output)

which I believe should work also in previous versions (but still relying on some printing).

giordano · 2025-08-10T12:41:23Z

Ok, I'm going to merge this so as to have CI working again with minimal changes, but if there are ways to improve testing of the precompilation condition they can be implemented in a follow up PR.

IanButterworth · 2025-08-10T12:46:43Z

Base.isprecompiled(pkgid)

giordano mentioned this pull request Aug 10, 2025

Tag v1.5.0 #79

Merged

giordano force-pushed the mg/fix-test-envs branch from 5b2d806 to 4b5c249 Compare August 10, 2025 12:19

giordano added 2 commits August 10, 2025 14:24

Update test environments

0a6aebd

Support yet another way to detect precompilation in tests

044c236

giordano force-pushed the mg/fix-test-envs branch from 4b5c249 to 044c236 Compare August 10, 2025 12:24

[CI] Add concurrency group

2e6da86

giordano merged commit a9e0962 into master Aug 10, 2025
17 checks passed

giordano deleted the mg/fix-test-envs branch August 10, 2025 12:41

giordano mentioned this pull request Aug 10, 2025

Bump julia-actions/setup-julia from 1 to 2 #69

Merged

giordano mentioned this pull request Aug 10, 2025

Use Base.isprecompiled to test package is already precompiled #83

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update test environments#80

Update test environments#80
giordano merged 3 commits intomasterfrom
mg/fix-test-envs

giordano commented Aug 10, 2025

Uh oh!

codecov bot commented Aug 10, 2025

Uh oh!

giordano commented Aug 10, 2025

Uh oh!

giordano commented Aug 10, 2025

Uh oh!

IanButterworth commented Aug 10, 2025

Uh oh!

giordano commented Aug 10, 2025

Uh oh!

giordano commented Aug 10, 2025

Uh oh!

Uh oh!

IanButterworth commented Aug 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

giordano commented Aug 10, 2025

Uh oh!

codecov bot commented Aug 10, 2025

Codecov Report

Uh oh!

giordano commented Aug 10, 2025

Uh oh!

giordano commented Aug 10, 2025

Uh oh!

IanButterworth commented Aug 10, 2025

Uh oh!

giordano commented Aug 10, 2025

Uh oh!

giordano commented Aug 10, 2025

Uh oh!

Uh oh!

IanButterworth commented Aug 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants