Merge bitcoin/bitcoin#27445: Update src/secp256k1 subtree to release v0.3.1

621c178 Respect and update FILES_ARGS in test/lint/lint-python.py (Pieter Wuille)
719a749 Disable Python lint in src/secp256k1 (Pieter Wuille)
c981671 Squashed 'src/secp256k1/' changes from bdf39000b9..4258c54f4e (Pieter Wuille)

Pull request description:

  There is no strict need for any of the changes in v0.3.1 (compared to the v0.3.0 that's currently subtreed) for Bitcoin Core release builds, but if anyone may compile Bitcoin Core from source using Clang v14+, this will prevent known timing leaks in the signing/keygen logic.

  This also includes a CI fix from libsecp256k1 master (on top of 0.3.1) which fixes Wycheproof test vector generation.

  I also had to amend some of the linters to avoid enforcing their rules on the .py files in the secp256k1 subtree.

ACKs for top commit:
  real-or-random:
    utACK 621c178 subtree matches. diff to linter script looks good
  fanquake:
    ACK 621c178

Tree-SHA512: 059722540a4fd387d9e231036e59685db373c085a346c7a9d2b87eac3ffe538099356b5f06fc2112a1df80e3818d80fe380f27a47901496e8092c836ea3ee14d

Author: fanquake

src/secp256k1/CHANGELOG.md

@@ -7,6 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.3.1] - 2023-04-10
+We strongly recommend updating to 0.3.1 if you use or plan to use Clang >=14 to compile libsecp256k1, e.g., Xcode >=14 on macOS has Clang >=14. When in doubt, check the Clang version using `clang -v`.
+
+#### Security
+ - Fix "constant-timeness" issue with Clang >=14 that could leave applications using libsecp256k1 vulnerable to a timing side-channel attack. The fix avoids secret-dependent control flow and secret-dependent memory accesses in conditional moves of memory objects when libsecp256k1 is compiled with Clang >=14.
+
+#### Added
+  - Added tests against [Project Wycheproof's](https://github.com/google/wycheproof/) set of ECDSA test vectors (Bitcoin "low-S" variant), a fixed set of test cases designed to trigger various edge cases.
+
+#### Changed
+ - Increased minimum required CMake version to 3.13. CMake builds remain experimental.
+
+#### ABI Compatibility
+The ABI is compatible with version 0.3.0.
+
 ## [0.3.0] - 2023-03-08
 
 #### Added
@@ -25,7 +40,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
  - Removed the configuration header `src/libsecp256k1-config.h`. We recommend passing flags to `./configure` or `cmake` to set configuration options (see `./configure --help` or `cmake -LH`). If you cannot or do not want to use one of the supported build systems, pass configuration flags such as `-DSECP256K1_ENABLE_MODULE_SCHNORRSIG` manually to the compiler (see the file `configure.ac` for supported flags).
 
 #### ABI Compatibility
-
 Due to changes in the API regarding `secp256k1_context_static` described above, the ABI is *not* compatible with previous versions.
 
 ## [0.2.0] - 2022-12-12
@@ -45,7 +59,6 @@ Due to changes in the API regarding `secp256k1_context_static` described above,
  - Module `schnorrsig`: renamed `secp256k1_schnorrsig_sign` to `secp256k1_schnorrsig_sign32`.
 
 #### ABI Compatibility
-
 Since this is the first release, we do not compare application binary interfaces.
 However, there are earlier unreleased versions of libsecp256k1 that are *not* ABI compatible with this version.
 
@@ -55,7 +68,8 @@ This version was in fact never released.
 The number was given by the build system since the introduction of autotools in Jan 2014 (ea0fe5a5bf0c04f9cc955b2966b614f5f378c6f6).
 Therefore, this version number does not uniquely identify a set of source files.
 
-[unreleased]: https://github.com/bitcoin-core/secp256k1/compare/v0.3.0...HEAD
+[unreleased]: https://github.com/bitcoin-core/secp256k1/compare/v0.3.1...HEAD
+[0.3.1]: https://github.com/bitcoin-core/secp256k1/compare/v0.3.0...v0.3.1
 [0.3.0]: https://github.com/bitcoin-core/secp256k1/compare/v0.2.0...v0.3.0
 [0.2.0]: https://github.com/bitcoin-core/secp256k1/compare/423b6d19d373f1224fd671a982584d7e7900bc93..v0.2.0
 [0.1.0]: https://github.com/bitcoin-core/secp256k1/commit/423b6d19d373f1224fd671a982584d7e7900bc93

src/secp256k1/CMakeLists.txt

@@ -1,4 +1,4 @@
-cmake_minimum_required(VERSION 3.1)
+cmake_minimum_required(VERSION 3.13)
 
 if(CMAKE_VERSION VERSION_GREATER 3.14)
   # MSVC runtime library flags are selected by the CMAKE_MSVC_RUNTIME_LIBRARY abstraction.
@@ -10,15 +10,15 @@ endif()
 # The package (a.k.a. release) version is based on semantic versioning 2.0.0 of
 # the API. All changes in experimental modules are treated as
 # backwards-compatible and therefore at most increase the minor version.
-project(libsecp256k1 VERSION 0.3.0 LANGUAGES C)
+project(libsecp256k1 VERSION 0.3.2 LANGUAGES C)
 
 # The library version is based on libtool versioning of the ABI. The set of
 # rules for updating the version can be found here:
 # https://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
 # All changes in experimental modules are treated as if they don't affect the
 # interface and therefore only increase the revision.
 set(${PROJECT_NAME}_LIB_VERSION_CURRENT 2)
-set(${PROJECT_NAME}_LIB_VERSION_REVISION 0)
+set(${PROJECT_NAME}_LIB_VERSION_REVISION 2)
 set(${PROJECT_NAME}_LIB_VERSION_AGE 0)
 
 set(CMAKE_C_STANDARD 90)
@@ -147,7 +147,7 @@ else()
 endif()
 
 # Define custom "Coverage" build type.
-set(CMAKE_C_FLAGS_COVERAGE "${CMAKE_C_FLAGS_RELWITHDEBINFO} -O0 -DCOVERAGE=1 --coverage -Wno-unused-parameter" CACHE STRING
+set(CMAKE_C_FLAGS_COVERAGE "${CMAKE_C_FLAGS_RELWITHDEBINFO} -O0 -DCOVERAGE=1 --coverage" CACHE STRING
   "Flags used by the C compiler during \"Coverage\" builds."
   FORCE
 )
@@ -203,11 +203,6 @@ else()
   try_add_compile_option(-Wundef)
 endif()
 
-if(CMAKE_VERSION VERSION_GREATER 3.2)
-  # Honor visibility properties for all target types.
-  # See: https://cmake.org/cmake/help/latest/policy/CMP0063.html
-  cmake_policy(SET CMP0063 NEW)
-endif()
 set(CMAKE_C_VISIBILITY_PRESET hidden)
 
 # Ask CTest to create a "check" target (e.g., make check) as alias for the "test" target.

src/secp256k1/Makefile.am

@@ -247,3 +247,20 @@ endif
 if ENABLE_MODULE_SCHNORRSIG
 include src/modules/schnorrsig/Makefile.am.include
 endif
+
+EXTRA_DIST += src/wycheproof/WYCHEPROOF_COPYING
+EXTRA_DIST += src/wycheproof/ecdsa_secp256k1_sha256_bitcoin_test.h
+EXTRA_DIST += src/wycheproof/ecdsa_secp256k1_sha256_bitcoin_test.json
+EXTRA_DIST += tools/tests_wycheproof_generate.py
+
+TESTVECTORS = src/wycheproof/ecdsa_secp256k1_sha256_bitcoin_test.h
+
+src/wycheproof/ecdsa_secp256k1_sha256_bitcoin_test.h:
+	python3 tools/tests_wycheproof_generate.py src/wycheproof/ecdsa_secp256k1_sha256_bitcoin_test.json > $@
+
+testvectors: $(TESTVECTORS)
+
+maintainer-clean-testvectors: clean-testvectors
+
+clean-testvectors:
+	rm -f $(TESTVECTORS)

src/secp256k1/ci/cirrus.sh

@@ -109,8 +109,8 @@ fi
 # Rebuild precomputed files (if not cross-compiling).
 if [ -z "$HOST" ]
 then
-    make clean-precomp
-    make precomp
+    make clean-precomp clean-testvectors
+    make precomp testvectors
 fi
 
 # Check that no repo files have been modified by the build.

src/secp256k1/configure.ac

@@ -5,16 +5,16 @@ AC_PREREQ([2.60])
 # backwards-compatible and therefore at most increase the minor version.
 define(_PKG_VERSION_MAJOR, 0)
 define(_PKG_VERSION_MINOR, 3)
-define(_PKG_VERSION_PATCH, 0)
-define(_PKG_VERSION_IS_RELEASE, true)
+define(_PKG_VERSION_PATCH, 2)
+define(_PKG_VERSION_IS_RELEASE, false)
 
 # The library version is based on libtool versioning of the ABI. The set of
 # rules for updating the version can be found here:
 # https://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
 # All changes in experimental modules are treated as if they don't affect the
 # interface and therefore only increase the revision.
 define(_LIB_VERSION_CURRENT, 2)
-define(_LIB_VERSION_REVISION, 0)
+define(_LIB_VERSION_REVISION, 2)
 define(_LIB_VERSION_AGE, 0)
 
 AC_INIT([libsecp256k1],m4_join([.], _PKG_VERSION_MAJOR, _PKG_VERSION_MINOR, _PKG_VERSION_PATCH)m4_if(_PKG_VERSION_IS_RELEASE, [true], [], [-dev]),[https://github.com/bitcoin-core/secp256k1/issues],[libsecp256k1],[https://github.com/bitcoin-core/secp256k1])
@@ -29,6 +29,11 @@ AM_INIT_AUTOMAKE([1.11.2 foreign subdir-objects])
 # Make the compilation flags quiet unless V=1 is used.
 m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
 
+if test "${CFLAGS+set}" = "set"; then
+  CFLAGS_overridden=yes
+else
+  CFLAGS_overridden=no
+fi
 AC_PROG_CC
 AM_PROG_AS
 AM_PROG_AR
@@ -88,11 +93,14 @@ esac
 AC_DEFUN([SECP_TRY_APPEND_DEFAULT_CFLAGS], [
     # GCC and compatible (incl. clang)
     if test "x$GCC" = "xyes"; then
-      # Try to append -Werror=unknown-warning-option to CFLAGS temporarily. Otherwise clang will
-      # not error out if it gets unknown warning flags and the checks here will always succeed
-      # no matter if clang knows the flag or not.
+      # Try to append -Werror to CFLAGS temporarily. Otherwise checks for some unsupported
+      # flags will succeed.
+      # Note that failure to append -Werror does not necessarily mean that -Werror is not
+      # supported. The compiler may already be warning about something unrelated, for example
+      # about some path issue. If that is the case, -Werror cannot be used because all
+      # of those warnings would be turned into errors.
       SECP_TRY_APPEND_DEFAULT_CFLAGS_saved_CFLAGS="$CFLAGS"
-      SECP_TRY_APPEND_CFLAGS([-Werror=unknown-warning-option], CFLAGS)
+      SECP_TRY_APPEND_CFLAGS([-Werror], CFLAGS)
 
       SECP_TRY_APPEND_CFLAGS([-std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef], $1) # GCC >= 3.0, -Wlong-long is implied by -pedantic.
       SECP_TRY_APPEND_CFLAGS([-Wno-overlength-strings], $1) # GCC >= 4.2, -Woverlength-strings is implied by -pedantic.
@@ -241,6 +249,12 @@ fi
 if test x"$enable_coverage" = x"yes"; then
     SECP_CONFIG_DEFINES="$SECP_CONFIG_DEFINES -DCOVERAGE=1"
     SECP_CFLAGS="-O0 --coverage $SECP_CFLAGS"
+    # If coverage is enabled, and the user has not overridden CFLAGS,
+    # override Autoconf's value "-g -O2" with "-g". Otherwise we'd end up
+    # with "-O0 --coverage -g -O2".
+    if test "$CFLAGS_overridden" = "no"; then
+      CFLAGS="-g"
+    fi
     LDFLAGS="--coverage $LDFLAGS"
 else
     # Most likely the CFLAGS already contain -O2 because that is autoconf's default.

src/secp256k1/doc/release-process.md

@@ -15,15 +15,20 @@ This process also assumes that there will be no minor releases for old major rel
 ## Regular release
 
 1. Open a PR to the master branch with a commit (using message `"release: prepare for $MAJOR.$MINOR.$PATCH"`, for example) that
-   * finalizes the release notes in [CHANGELOG.md](../CHANGELOG.md) (make sure to include an entry for `### ABI Compatibility`) and
-   * updates `_PKG_VERSION_*`, `_LIB_VERSION_*`, and sets `_PKG_VERSION_IS_RELEASE` to `true` in `configure.ac`.
+   * finalizes the release notes in [CHANGELOG.md](../CHANGELOG.md) (make sure to include an entry for `### ABI Compatibility`),
+   * updates `_PKG_VERSION_*` and `_LIB_VERSION_*` and sets `_PKG_VERSION_IS_RELEASE` to `true` in `configure.ac`, and
+   * updates `project(libsecp256k1 VERSION ...)` and `${PROJECT_NAME}_LIB_VERSION_*` in `CMakeLists.txt`.
 2. After the PR is merged, tag the commit and push it:
    ```
    RELEASE_COMMIT=<merge commit of step 1>
    git tag -s v$MAJOR.$MINOR.$PATCH -m "libsecp256k1 $MAJOR.$MINOR.$PATCH" $RELEASE_COMMIT
    git push [email protected]:bitcoin-core/secp256k1.git v$MAJOR.$MINOR.$PATCH
    ```
-3. Open a PR to the master branch with a commit (using message `"release cleanup: bump version after $MAJOR.$MINOR.$PATCH"`, for example) that sets `_PKG_VERSION_IS_RELEASE` to `false` and `_PKG_VERSION_PATCH` to `$PATCH + 1` and increases `_LIB_VERSION_REVISION`. If other maintainers are not present to approve the PR, it can be merged without ACKs.
+3. Open a PR to the master branch with a commit (using message `"release cleanup: bump version after $MAJOR.$MINOR.$PATCH"`, for example) that
+   * sets `_PKG_VERSION_IS_RELEASE` to `false` and increments `_PKG_VERSION_PATCH` and `_LIB_VERSION_REVISION` in `configure.ac`, and
+   * increments the `$PATCH` component of `project(libsecp256k1 VERSION ...)` and `${PROJECT_NAME}_LIB_VERSION_REVISION` in `CMakeLists.txt`.
+
+   If other maintainers are not present to approve the PR, it can be merged without ACKs.
 4. Create a new GitHub release with a link to the corresponding entry in [CHANGELOG.md](../CHANGELOG.md).
 
 ## Maintenance release
@@ -38,7 +43,9 @@ Note that bugfixes only need to be backported to releases for which no compatibl
 2. Open a pull request to the `$MAJOR.$MINOR` branch that
    * includes the bugfixes,
    * finalizes the release notes,
-   * bumps `_PKG_VERSION_PATCH` and `_LIB_VERSION_REVISION` in `configure.ac` (with commit message `"release: update PKG_ and LIB_VERSION for $MAJOR.$MINOR.$PATCH"`, for example).
+   * increments `_PKG_VERSION_PATCH` and `_LIB_VERSION_REVISION` in `configure.ac`
+     and the `$PATCH` component of `project(libsecp256k1 VERSION ...)` and `${PROJECT_NAME}_LIB_VERSION_REVISION` in `CMakeLists.txt`
+     (with commit message `"release: bump versions for $MAJOR.$MINOR.$PATCH"`, for example).
 3. After the PRs are merged, update the release branch and tag the commit:
    ```
    git checkout $MAJOR.$MINOR && git pull