Merge #157233 #157845

157233: Add fingerprint checking to import roachtests r=mw5h a=mw5h

#### roachtest/import: refactor existing roachtests for import

Previously, the roachtests for import were distressingly absent solid
verification of the imported data. This patch changes those tests to use
the newly revamed SHOW FINGERPRINTS to validate the imported data. Along
the way, we attempt to standardize and rationalize the test design so
that it's more easily extensible.

As part of this work, we're moving away from using TPC-C data even though
it most closely resembles OLTP workloads because it's much easier to
validate the import of the static TPC-H data files. We keep the TPC-C
benchmark because that's potentially interesting, but it's a separate
thing for now.

We also now standardize how we deal with local clusters by picking a
scale factor during data set initialization. Local uses scale factor 1
and our four node tests use scale factor 100. The data set initializer
has access to the cluster configuration, so we can select even larger
scale factors for larger clusters in the future, if desired.

Informs: #156816
Release note: None

#### roachtest/import: add cancellation test to import

Previously, import_cancellation was a separate test that imported the
entire TPC-H dataset and then ran the TPC-H workload to verify that
there were no errors. Unfortunately, the TPC-H workload does not verify
the correctness of results (only the number of results), which means
there could be import errors that go undetected.

This patch brings the core import & cancel logic from the
import-cancellation test into the import test as a custom test runner,
which lets us use the validation logic already in the import test. For
now we only import a single, random file but a future patch will select
a random set of files to import to maintain the same coverage in the
face of concurrent imports.

Fixes: #156816
Release note: None

157845: build: reduce `shard_count` of `allccl_test` to 4 r=rail a=rickystewart

This package only has 4 tests.

Release note: none
Epic: none

Co-authored-by: Matt White <[email protected]>
Co-authored-by: Ricky Stewart <[email protected]>

Author: 3 people

pkg/ccl/workloadccl/allccl/BUILD.bazel

@@ -46,7 +46,7 @@ go_test(
         "main_test.go",
     ],
     embed = [":allccl"],
-    shard_count = 16,
+    shard_count = 4,
     deps = [
         "//pkg/base",
         "//pkg/ccl",

pkg/cmd/roachtest/tests/BUILD.bazel

@@ -88,7 +88,6 @@ go_library(
         "hibernate_blocklist.go",
         "hotspotsplits.go",
         "import.go",
-        "import_cancellation.go",
         "inconsistency.go",
         "indexes.go",
         "inspect_throughput.go",

pkg/cmd/roachtest/tests/backup.go

@@ -626,7 +626,7 @@ func runBackupMVCCRangeTombstones(
 	require.NoError(t, err)
 	_, err = conn.Exec(`USE tpch`)
 	require.NoError(t, err)
-	createStmt, err := readCreateTableFromFixture(
+	createStmt, err := readFileFromFixture(
 		"gs://cockroach-fixtures-us-east1/tpch-csv/schema/orders.sql?AUTH=implicit", conn)
 	require.NoError(t, err)
 	_, err = conn.ExecContext(ctx, createStmt)