Skip to content

Commit fbaec79

Browse files
author
Release Manager
committed
gh-37391: Make installation of "wheel" packages less noisy <!-- ^^^^^ Please provide a concise, informative and self-explanatory title. Don't put issue numbers in there, do this in the PR body below. For example, instead of "Fixes #1234" use "Introduce new method to calculate 1+1" --> <!-- Describe your changes here in detail --> Some care for `build/bin/sage-spkg`, reducing unhelpful verbosity. In particular, for "wheel" packages (https://deploy-livedoc-- sagemath.netlify.app/html/en/developer/packaging#package-source-types), there is no build step, so there is no point in talking about what C compiler is in use. Also saving a few lines of output (and making it clearer who does what) by prefixing the output from the various `spkg-...` scripts. Example: ``` $ make packaging-no-deps [packaging-23.2] Using cached file /Users/mkoeppe/s/sage/sage- rebasing/worktree-pristine/upstream/packaging-23.2-py3-none-any.whl [packaging-23.2] Setting up build directory /Users/mkoeppe/s/sage/sage- rebasing/worktree-pristine/local/var/lib/sage/venv- python3.11/var/tmp/sage/build/packaging-23.2 [packaging-23.2] [spkg-piprm] Found existing installation: packaging 23.2 [packaging-23.2] [spkg-piprm] Uninstalling packaging-23.2: [packaging-23.2] [spkg-piprm] Successfully uninstalled packaging-23.2 [packaging-23.2] Removing stamp file /Users/mkoeppe/s/sage/sage- rebasing/worktree-pristine/local/var/lib/sage/venv- python3.11/var/lib/sage/installed/packaging-23.2 [packaging-23.2] [spkg-install] Staged wheel file, staged /Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/local/var/lib/sage/venv- python3.11/var/lib/sage/scripts/packaging/spkg-requirements.txt [packaging-23.2] Moving package files from temporary location /Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/local/var/lib/sage/venv- python3.11/var/tmp/sage/build/packaging-23.2/inst to /Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/local/var/lib/sage/venv-python3.11 [packaging-23.2] [spkg-pipinst] Using pip 23.3.1 from /Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/local/var/lib/sage/venv-python3.11/lib/python3.11/site- packages/pip (python 3.11) [packaging-23.2] [spkg-pipinst] Looking in links: /Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/local/var/lib/sage/venv-python3.11/var/lib/sage/wheels [packaging-23.2] [spkg-pipinst] Processing /Users/mkoeppe/s/sage/sage- rebasing/worktree-pristine/local/var/lib/sage/venv- python3.11/var/lib/sage/wheels/packaging-23.2-py3-none-any.whl (from -r /Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/local/var/lib/sage/venv- python3.11/var/lib/sage/scripts/packaging/spkg-requirements.txt (line 1)) [packaging-23.2] [spkg-pipinst] Installing collected packages: packaging [packaging-23.2] [spkg-pipinst] ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. [packaging-23.2] [spkg-pipinst] meson-python 0.15.0 requires meson>=0.63.3; python_version < "3.12", which is not installed. [packaging-23.2] [spkg-pipinst] tox 4.11.1 requires cachetools>=5.3.1, which is not installed. [packaging-23.2] [spkg-pipinst] tox 4.11.1 requires chardet>=5.2, which is not installed. [packaging-23.2] [spkg-pipinst] Successfully installed packaging-23.2 [packaging-23.2] Deleting build directory /Users/mkoeppe/s/sage/sage- rebasing/worktree-pristine/local/var/lib/sage/venv- python3.11/var/tmp/sage/build/packaging-23.2 [packaging-23.2] Finished installing packaging-23.2 ``` Also displaying build times for each package, even when using `make V=0`, but suppressing very short build times: ``` $ make V=0 ... [setuptools_scm-8.0.4] installing. Log file: /Users/mkoeppe/s/sage/sage- rebasing/worktree-pristine/logs/pkgs/setuptools_scm-8.0.4.log [hatchling-1.20.0] installing. Log file: /Users/mkoeppe/s/sage/sage- rebasing/worktree-pristine/logs/pkgs/hatchling-1.20.0.log [setuptools_scm-8.0.4] successfully installed. [hatchling-1.20.0] successfully installed. ... [sage_conf-10.3.beta8] successfully installed (real 11.36 user 8.32 sys 2.67). [pyproject_metadata-0.7.1] successfully installed (real 16.57 user 10.70 sys 2.97). [ipykernel-6.27.1] successfully installed (real 19.31 user 11.35 sys 3.71). ``` <!-- Why is this change required? What problem does it solve? --> <!-- If this PR resolves an open issue, please link to it here. For example "Fixes #12345". --> <!-- If your change requires a documentation PR, please link it appropriately. --> ### 📝 Checklist <!-- Put an `x` in all the boxes that apply. --> <!-- If your change requires a documentation PR, please link it appropriately --> <!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! --> <!-- Feel free to remove irrelevant items. --> - [x] The title is concise, informative, and self-explanatory. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies <!-- List all open PRs that this PR logically depends on - #12345: short description why this is a dependency - #34567: ... --> <!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! --> URL: #37391 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee
2 parents 7855be0 + cb4e718 commit fbaec79

File tree

7 files changed

+111
-77
lines changed

7 files changed

+111
-77
lines changed

‎build/bin/sage-dist-helpers

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -323,8 +323,13 @@ sdh_store_wheel() {
323323
wheel_basename="${wheel##*/}"
324324
distname="${wheel_basename%%-*}"
325325
# Record name and wheel file location
326+
if [ -n "$SAGE_DESTDIR" ]; then
327+
echo "Staged wheel file, staged ${SAGE_SPKG_SCRIPTS}/${PKG_BASE}/spkg-requirements.txt"
328+
else
329+
echo "Copied wheel file, wrote ${SAGE_SPKG_SCRIPTS}/${PKG_BASE}/spkg-requirements.txt"
330+
fi
326331
mkdir -p ${SAGE_DESTDIR}${SAGE_SPKG_SCRIPTS}/${PKG_BASE}
327-
echo "${distname} @ file://${wheel}" >> ${SAGE_DESTDIR}${SAGE_SPKG_SCRIPTS}/${PKG_BASE}/spkg-requirements.txt
332+
echo "${distname} @ file://${wheel}" >> "${SAGE_DESTDIR}${SAGE_SPKG_SCRIPTS}/${PKG_BASE}/spkg-requirements.txt"
328333
fi
329334
wheel="${SAGE_DESTDIR}${wheel}"
330335
}
@@ -392,7 +397,7 @@ sdh_actually_pip_install_wheel() {
392397
# of sage-conf, sage-setup, etc.) bump the version number, we need to
393398
# override this behavior. The pip install option --force-reinstall does too
394399
# much -- it also reinstalls all dependencies, which we do not want.
395-
$sudo sage-pip-uninstall "$distname"
400+
$sudo sage-pip-uninstall "$distname" 2>&1 | sed '/^WARNING: Skipping .* as it is not installed./d'
396401
if [ $? -ne 0 ]; then
397402
echo "(ignoring error)" >&2
398403
fi

‎build/bin/sage-logger

Lines changed: 30 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,25 @@ else
6161
SED=cat
6262
fi
6363

64+
timefile="$logdir/$logname.time"
65+
rm -f "$timefile"
66+
if /usr/bin/time -h -o /dev/null true; then
67+
TIME="/usr/bin/time -h -o $timefile"
68+
else
69+
TIME=""
70+
fi
71+
72+
report_time ()
73+
{
74+
time=$(echo $(cat $timefile))
75+
case "$time" in
76+
*m*real*|*h*real*|*[1-9][0-9].*real*|*[1-9][0-9],*real*)
77+
# at least 10 seconds wall time
78+
echo "$time"
79+
;;
80+
esac
81+
}
82+
6483
mkdir -p "$logdir"
6584

6685
# Do all logging of child processes with V=1 to ensure that no
@@ -75,22 +94,30 @@ if [ -n "$SAGE_SILENT_BUILD" -a ${use_prefix} = true ]; then
7594
# Silent build.
7695
# Similar to https://www.gnu.org/software/automake/manual/html_node/Automake-Silent-Rules.html#Automake-Silent-Rules
7796
echo "[$logname] installing. Log file: $logfile"
78-
( exec>> $logfile 2>&1 ; eval "$cmd" )
97+
( exec>> $logfile 2>&1 ; $TIME sh -c "$cmd"; status=$?;
98+
[ -r $timefile ] && cat $timefile; exit $status )
7999
status=$?
80100
if [[ $status != 0 ]]; then
81101
echo " [$logname] error installing, exit status $status. End of log file:"
82102
tail -n 120 "$logfile" | sed "/Please email sage-devel/,$ d;s;^; [$logname] ;" >&2
83103
echo " [$logname] Full log file: $logfile"
84104
else
85-
echo " [$logname] successfully installed."
105+
time=$(report_time)
106+
if [ -n "$time" ]; then
107+
echo " [$logname] successfully installed ($time)."
108+
else
109+
echo " [$logname] successfully installed."
110+
fi
86111
fi
87112
exit $status
88113
else
89114
# Redirect stdout and stderr to a subprocess running tee.
90115
# We trap SIGINT such that SIGINT interrupts the main process being
91116
# run, not the logging.
92117

93-
( exec 2>&1; eval "$cmd" ) | \
118+
( exec 2>&1;
119+
$TIME sh -c "$cmd"; status=$?; report_time;
120+
exit $status ) | \
94121
( trap '' SIGINT; if [ -n "$GITHUB_ACTIONS" -a -n "$prefix" ]; then echo "::group::${logname}"; fi; tee -a "$logfile" | $SED; if [ -n "$GITHUB_ACTIONS" -a -n "$prefix" ]; then echo "::endgroup::"; fi )
95122

96123
pipestatus=(${PIPESTATUS[*]})

‎build/bin/sage-spkg

Lines changed: 26 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -84,8 +84,8 @@ Modes of operation (provide at most one):
8484
-x, --check-only: exclusively run the test suite;
8585
this may assume that:
8686
* the package has been installed already and/or that
87-
* the temporary build directory has not been deleted
88-
-e, --erase-build-directory-only: erase (delete) the temporary
87+
* the build directory has not been deleted
88+
-e, --erase-build-directory-only: erase (delete) the
8989
build directory only
9090
--info: print information on the package only
9191
--help: print this help only
@@ -103,7 +103,7 @@ Other options:
103103
exit with an error on test suite failures
104104
-w: after installing, run the test suite for the package;
105105
print a warning on test suite failures
106-
-s: save (do not delete) the temporary build directory,
106+
-s: save (do not delete) the build directory,
107107
even when the installation was successful
108108
EOF
109109
}
@@ -451,9 +451,6 @@ if [ ! -w "$SAGE_INST_LOCAL" ]; then
451451
exit 1
452452
fi
453453

454-
echo "$PKG_NAME"
455-
echo "===================================================="
456-
457454
# Make absolutely sure that we are in the build directory before doing
458455
# a scary "rm -rf" below.
459456
cd "$SAGE_BUILD_DIR" || exit $?
@@ -483,7 +480,7 @@ fi
483480
extract_the_package() { ##########################################
484481

485482
cd "$SAGE_BUILD_DIR" || exit $?
486-
echo "Setting up build directory for $PKG_NAME"
483+
echo "Setting up build directory $SAGE_BUILD_DIR/$PKG_NAME"
487484
if [ -z "$PKG_NAME_UPSTREAM" ]; then
488485
# Not a normal or wheel package.
489486
# Transplant the 'src' symlink, copy scripts.
@@ -521,7 +518,6 @@ else
521518
error_msg "Error: failed to extract $PKG_SRC"
522519
exit 1
523520
fi
524-
echo "Finished extraction"
525521
cd src
526522
if ! sage-apply-patches; then
527523
error_msg "Error applying patches"
@@ -646,14 +642,15 @@ done
646642

647643
actually_build_and_install() { ###################################
648644

649-
echo "****************************************************"
650-
echo "Host system:"
651-
uname -a
652-
echo "****************************************************"
653-
echo "C compiler: $CC"
654-
echo "C compiler version:"
655-
$CC -v
656-
echo "****************************************************"
645+
case "$PKG_SRC" in
646+
*.whl)
647+
# Silence is golden.
648+
;;
649+
*)
650+
echo "Host system: $(uname -a)"
651+
echo "C compiler: $CC$($CC -v 2>&1 | while read -r line; do echo -n ", $line"; done)"
652+
;;
653+
esac
657654
658655
# Poison the proxy variable to forbid downloads in spkg-install
659656
# for normal packages
@@ -665,19 +662,21 @@ if [ -n "$PKG_NAME_UPSTREAM" ]; then
665662
export rsync_proxy=$http_proxy
666663
fi
667664
665+
# Make sage-logger show the full logs
666+
unset SAGE_SILENT_BUILD
667+
unset V
668668
669669
# First uninstall the previous version of this package, if any
670670
if [ "$KEEP_EXISTING" != "yes" ]; then
671-
sage-spkg-uninstall "$PKG_BASE" "$SAGE_INST_LOCAL"
671+
sage-spkg-uninstall "$PKG_BASE" "$SAGE_INST_LOCAL" --log-directory .
672672
fi
673673
674674
# To work around #26996: Create lib and set a symlink so that writes into lib64/ end up in lib/
675675
(mkdir -p "$SAGE_DESTDIR_LOCAL/lib" && cd "$SAGE_DESTDIR_LOCAL" && ln -sf lib lib64)
676676
677677
# Run the pre-install script, if any
678678
if [ -f spkg-preinst ]; then
679-
echo "Running pre-install script for $PKG_NAME."
680-
time $SAGE_SUDO ./spkg-preinst
679+
sage-logger -p "$SAGE_SUDO ./spkg-preinst" spkg-preinst.log
681680
if [ $? -ne 0 ]; then
682681
error_msg "Error running the preinst script for $PKG_NAME."
683682
exit 1
@@ -686,19 +685,19 @@ fi
686685
687686
if [ -f spkg-build ]; then
688687
# Package has both spkg-build and spkg-install; execute the latter with SAGE_SUDO
689-
time ./spkg-build
688+
sage-logger -p "./spkg-build" spkg-build.log
690689
if [ $? -ne 0 ]; then
691690
error_msg "Error building package $PKG_NAME" "make"
692691
exit 1
693692
fi
694-
time $SAGE_SUDO ./spkg-install
693+
sage-logger -p "$SAGE_SUDO ./spkg-install" spkg-install.log
695694
if [ $? -ne 0 ]; then
696695
error_msg "Error installing package $PKG_NAME" "make"
697696
exit 1
698697
fi
699698
else
700699
# Package only has spkg-install
701-
time ./spkg-install
700+
sage-logger -p "./spkg-install" spkg-install.log
702701
if [ $? -ne 0 ]; then
703702
error_msg "Error installing package $PKG_NAME" "make"
704703
exit 1
@@ -784,16 +783,14 @@ post_install() { #################################################
784783
# Run the post-install script, if any
785784
# But first complete the delayed installation of wheels.
786785
if [ -f spkg-pipinst ]; then
787-
echo "Running pip-install script for $PKG_NAME."
788-
$SAGE_SUDO ./spkg-pipinst
786+
sage-logger -p "$SAGE_SUDO ./spkg-pipinst" spkg-pipinst.log
789787
if [ $? -ne 0 ]; then
790788
error_msg "Error running the pipinst script for $PKG_NAME."
791789
exit 1
792790
fi
793791
fi
794792
if [ -f spkg-postinst ]; then
795-
echo "Running post-install script for $PKG_NAME."
796-
time $SAGE_SUDO ./spkg-postinst
793+
sage-logger -p "$SAGE_SUDO ./spkg-postinst" spkg-postinst.log
797794
if [ $? -ne 0 ]; then
798795
error_msg "Error running the postinst script for $PKG_NAME."
799796
exit 1
@@ -813,7 +810,7 @@ run_test_suite() { ###############################################
813810
if [ -f "$INSTALLED_SCRIPTS_DEST"/spkg-check ]; then
814811
echo "Running the test suite for $PKG_NAME..."
815812
export PKG_BASE
816-
time "$INSTALLED_SCRIPTS_DEST"/spkg-check
813+
sage-logger -p "$INSTALLED_SCRIPTS_DEST"/spkg-check spkg-check.log
817814
if [ $? -ne 0 ]; then
818815
TEST_SUITE_RESULT="failed"
819816
if [ "$SAGE_CHECK" = "warn" ]; then
@@ -869,8 +866,7 @@ __EOF__
869866
} ###################################### write_installation_record
870867
871868
delete_the_temporary_build_directory() { #########################
872-
echo "Deleting temporary build directory"
873-
echo "$SAGE_BUILD_DIR/$PKG_NAME"
869+
echo "Deleting build directory $SAGE_BUILD_DIR/$PKG_NAME"
874870
# On Solaris, the current working directory cannot be deleted,
875871
# so we "cd" out of $SAGE_BUILD_DIR/$PKG_NAME. See #12637.
876872
cd "$SAGE_BUILD_DIR"
@@ -881,8 +877,7 @@ delete_the_temporary_build_directory_if_required() { #############
881877
if [ ! -e "$SAGE_BUILD_DIR/$PKG_NAME/.keep" ]; then
882878
delete_the_temporary_build_directory
883879
else
884-
echo "You can safely delete the temporary build directory"
885-
echo "$SAGE_BUILD_DIR/$PKG_NAME"
880+
echo "You can safely delete the build directory $SAGE_BUILD_DIR/$PKG_NAME"
886881
fi
887882
} ############### delete_the_temporary_build_directory_if_required
888883
@@ -929,7 +924,6 @@ fi
929924
930925
if [ $POST_INSTALL = 1 ]; then
931926
write_installation_record
932-
echo "Successfully installed $PKG_NAME"
933927
delete_the_temporary_build_directory_if_required
934928
echo "Finished installing $PKG_NAME"
935929
fi

‎build/make/install

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,7 @@ EOF
103103
###############################################################################
104104
# NOW do the actual build:
105105
###############################################################################
106-
time $MAKE "$@"
106+
$MAKE "$@"
107107
if [ $? -ne 0 ]; then
108108
cat >&2 <<EOF
109109
***************************************************************

‎build/pkgs/sagelib/spkg-install.in

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ if [ "$SAGE_EDITABLE" = yes ]; then
5353
(cd "$SITEPACKAGESDIR" && rm -rf sage sage-[1-9]*.egg-info sage-[1-9]*.dist-info)
5454
# Until https://github.com/sagemath/sage/issues/34209 switches us to PEP 660 editable wheels
5555
export SETUPTOOLS_ENABLE_FEATURES=legacy-editable
56-
time sdh_pip_editable_install .
56+
sdh_pip_editable_install .
5757

5858
if [ "$SAGE_WHEELS" = yes ]; then
5959
# Additionally build a wheel (for use in other venvs)
@@ -69,10 +69,10 @@ else
6969
if [ "$SAGE_WHEELS" = yes ]; then
7070
# Use --no-build-isolation to avoid rebuilds because of dependencies:
7171
# Compiling sage/interfaces/sagespawn.pyx because it depends on /private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pip-build-env-609n5985/overlay/lib/python3.10/site-packages/Cython/Includes/posix/unistd.pxd
72-
time sdh_pip_install --no-build-isolation .
72+
sdh_pip_install --no-build-isolation .
7373
else
7474
SAGE_OPTIONAL_PACKAGES_WITH_EXTENSIONS+="mcqd,tdlib,coxeter3,sirocco,meataxe,bliss"
75-
time python3 -u setup.py --no-user-cfg build install || exit 1
75+
python3 -u setup.py --no-user-cfg build install || exit 1
7676
fi
7777
fi
7878

0 commit comments

Comments
 (0)