Merge bitcoin/bitcoin#32219: test: enabling wallet migration functional test on windows

941b8f5 ci: run get_previous_releases as part of test cross win job (Max Edwards)
5e21821 test: increment mocked time for migrating wallet backups (Max Edwards)
5174565 ci: disable feature_unsupported_utxo_db functional test (Max Edwards)
3dc90d6 test: remove mempool.dat before copying (Max Edwards)
67a6b20 test: add windows support to get previous releases script (Max Edwards)
1a1b478 scripted-diff: rename tarball to archive (Max Edwards)
4f06dc8 test: remove building from source from get prev releases script (Max Edwards)

Pull request description:

  This PR updates the `test/get_previous_releases.py` script to also work on Windows by changing to be pure python rather than using unix tools such as `curl` and `tar`.

  This enables additional functional tests to run such as `wallet_migration.py`, `mempool_compatability.py` and `wallet_backwards_compatibility.py`.

  Unfortunately `feature_unsupported_utxo_db.py` _could_ run but this test requires Bitcoin `v0.14.3` which will not run under windows with emojis in the data directory (as the functional test runner has by default) . This test could be run as it's own step in the ci workflow file and would pass but as it's quite an old version / feature I have assumed it's not worth worrying about and best just to exclude.

  Two tests needed to be slightly modified to run under windows. Both were issues with trying to overwrite a file that already exists which windows seems to be more strict on than the unix based systems.

  Finally, building from source has been dropped from the `get_previous_releases.py` script. This had not been updated after the move to cmake and so it was assumed that nobody could have been using that feature.

ACKs for top commit:
  maflcko:
    re-ACK 941b8f5 🍪
  achow101:
    ACK 941b8f5
  hodlinator:
    re-ACK 941b8f5

Tree-SHA512: 22933d0ec278b9b0ffcd2a8e90026e1a3631b00186e7f78bd65be925049021e319367d488c36a82ab526a07b264bac18c2777f87ca1174b231ed49fed56d11cb

Author: achow101

.github/workflows/ci.yml

@@ -393,10 +393,19 @@ jobs:
       - name: Run rpcauth test
         run: py -3 test/util/rpcauth-test.py
 
+      - name: Set previous release directory
+        run: |
+          echo "PREVIOUS_RELEASES_DIR=${{ runner.temp }}/previous_releases" >> "$GITHUB_ENV"
+
+      - name: Get previous releases
+        working-directory: test
+        run: ./get_previous_releases.py --target-dir $PREVIOUS_RELEASES_DIR
+
       - name: Run functional tests
         env:
           # TODO: Fix the excluded test and re-enable it.
-          EXCLUDE: '--exclude wallet_multiwallet.py'
+          # feature_unsupported_utxo_db.py fails on windows because of emojis in the test data directory
+          EXCLUDE: '--exclude wallet_multiwallet.py,feature_unsupported_utxo_db.py'
           TEST_RUNNER_EXTRA: ${{ github.event_name != 'pull_request' && '--extended' || '' }}
         run: py -3 test/functional/test_runner.py --jobs $NUMBER_OF_PROCESSORS --ci --quiet --tmpdirprefix="$RUNNER_TEMP" --combinedlogslen=99999999 --timeout-factor=$TEST_RUNNER_TIMEOUT_FACTOR $EXCLUDE $TEST_RUNNER_EXTRA
 

ci/test/03_test_script.sh

@@ -99,7 +99,7 @@ if [ -z "$NO_DEPENDS" ]; then
   bash -c "$SHELL_OPTS make $MAKEJOBS -C depends HOST=$HOST $DEP_OPTS LOG=1"
 fi
 if [ "$DOWNLOAD_PREVIOUS_RELEASES" = "true" ]; then
-  test/get_previous_releases.py -b -t "$PREVIOUS_RELEASES_DIR"
+  test/get_previous_releases.py --target-dir "$PREVIOUS_RELEASES_DIR"
 fi
 
 BITCOIN_CONFIG_ALL="-DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON"

test/README.md

@@ -99,7 +99,7 @@ build/test/functional/test_runner.py --extended
 In order to run backwards compatibility tests, first run:
 
 ```
-test/get_previous_releases.py -b
+test/get_previous_releases.py
 ```
 
 to download the necessary previous release binaries.

test/functional/mempool_compatibility.py

@@ -56,6 +56,7 @@ def run_test(self):
         self.log.info("Move mempool.dat from old to new node")
         old_node_mempool = old_node.chain_path / "mempool.dat"
         new_node_mempool = new_node.chain_path / "mempool.dat"
+        new_node_mempool.unlink()
         old_node_mempool.rename(new_node_mempool)
 
         self.log.info("Start new node and verify mempool contains the tx")

test/functional/test_framework/test_framework.py

@@ -555,7 +555,7 @@ def bin_dir_from_version(version):
                 bins_missing = True
         if bins_missing:
             raise AssertionError("At least one release binary is missing. "
-                                 "Previous releases binaries can be downloaded via `test/get_previous_releases.py -b`.")
+                                 "Previous releases binaries can be downloaded via `test/get_previous_releases.py`.")
         assert_equal(len(extra_confs), num_nodes)
         assert_equal(len(extra_args), num_nodes)
         assert_equal(len(versions), num_nodes)

test/functional/wallet_migration.py

@@ -490,11 +490,16 @@ def test_encrypted(self):
 
         # Use self.migrate_and_get_rpc to test this error to get everything copied over to the master node
         assert_raises_rpc_error(-4, "Error: Wallet decryption failed, the wallet passphrase was not provided or was incorrect", self.migrate_and_get_rpc, "encrypted")
+
         # Use the RPC directly on the master node for the rest of these checks
+        self.master_node.bumpmocktime(1) # Prevents filename duplication on wallet backups which is a problem on Windows
         assert_raises_rpc_error(-4, "Error: Wallet decryption failed, the wallet passphrase was not provided or was incorrect", self.master_node.migratewallet, "encrypted", "badpass")
+
+        self.master_node.bumpmocktime(1) # Prevents filename duplication on wallet backups which is a problem on Windows
         assert_raises_rpc_error(-4, "The passphrase contains a null character", self.master_node.migratewallet, "encrypted", "pass\0with\0null")
 
         # Verify we can properly migrate the encrypted wallet
+        self.master_node.bumpmocktime(1) # Prevents filename duplication on wallet backups which is a problem on Windows
         self.master_node.migratewallet("encrypted", passphrase="pass")
         wallet = self.master_node.get_wallet_rpc("encrypted")