Remove benchmark feature (#4330)

## Motivation

This code doesn't need to be behind a feature anymore

## Proposal

Remove benchmark feature

## Test Plan

CI

## Release Plan

- Nothing to do / These changes follow the usual release cycle.

Author: ndr-ds

.github/workflows/rust.yml

@@ -195,8 +195,8 @@ jobs:
         cargo run --release -p linera-storage-service -- memory --endpoint $LINERA_STORAGE_SERVICE &
     - name: Run the benchmark test
       run: |
-        cargo build --locked -p linera-service --bin linera-benchmark --features benchmark,storage-service
-        cargo test --locked -p linera-service --features benchmark,storage-service benchmark
+        cargo build --locked -p linera-service --bin linera-benchmark --features storage-service
+        cargo test --locked -p linera-service --features storage-service benchmark
 
   ethereum-tests:
     needs: changed-files

CLI.md

@@ -24,6 +24,8 @@ This document contains the help content for the `linera` command-line program.
 * [`linera remove-validator`↴](#linera-remove-validator)
 * [`linera revoke-epochs`↴](#linera-revoke-epochs)
 * [`linera resource-control-policy`↴](#linera-resource-control-policy)
+* [`linera benchmark`↴](#linera-benchmark)
+* [`linera multi-benchmark`↴](#linera-multi-benchmark)
 * [`linera create-genesis-config`↴](#linera-create-genesis-config)
 * [`linera watch`↴](#linera-watch)
 * [`linera service`↴](#linera-service)
@@ -88,6 +90,8 @@ A Byzantine-fault tolerant sidechain with low-latency finality and high throughp
 * `remove-validator` — Remove a validator (admin only)
 * `revoke-epochs` — Deprecates all committees up to and including the specified one
 * `resource-control-policy` — View or update the resource control policy
+* `benchmark` — Start a benchmark, maintaining a given TPS or just sending one transfer per chain in bulk mode
+* `multi-benchmark` — Runs multiple `linera benchmark` processes in parallel
 * `create-genesis-config` — Create genesis configuration for a Linera deployment. Create initial user chains and print information to be used for initialization of validator setup. This will also create an initial wallet for the owner of the initial "root" chains
 * `watch` — Watch the network for notifications
 * `service` — Run a GraphQL service to explore and extend the chains of the wallet
@@ -207,6 +211,7 @@ Open (i.e. activate) a new chain deriving the UID from an existing one
 * `--initial-balance <BALANCE>` — The initial balance of the new chain. This is subtracted from the parent chain's balance
 
   Default value: `0`
+* `--super-owner` — Whether to create a super owner for the new chain
 
 
 
@@ -520,6 +525,81 @@ View or update the resource control policy
 
 
 
+## `linera benchmark`
+
+Start a benchmark, maintaining a given TPS or just sending one transfer per chain in bulk mode
+
+**Usage:** `linera benchmark [OPTIONS]`
+
+###### **Options:**
+
+* `--num-chains <NUM_CHAINS>` — How many chains to use
+
+  Default value: `10`
+* `--tokens-per-chain <TOKENS_PER_CHAIN>` — How many tokens to assign to each newly created chain. These need to cover the transaction fees per chain for the benchmark
+
+  Default value: `0.1`
+* `--transactions-per-block <TRANSACTIONS_PER_BLOCK>` — How many transactions to put in each block
+
+  Default value: `1`
+* `--fungible-application-id <FUNGIBLE_APPLICATION_ID>` — The application ID of a fungible token on the wallet's default chain. If none is specified, the benchmark uses the native token
+* `--bps <BPS>` — The fixed BPS (Blocks Per Second) rate that block proposals will be sent at
+
+  Default value: `10`
+* `--close-chains` — If provided, will close the chains after the benchmark is finished. Keep in mind that closing the chains might take a while, and will increase the validator latency while they're being closed
+* `--health-check-endpoints <HEALTH_CHECK_ENDPOINTS>` — A comma-separated list of host:port pairs to query for health metrics. If provided, the benchmark will check these endpoints for validator health and terminate if any validator is unhealthy. Example: "127.0.0.1:21100,validator-1.some-network.linera.net:21100"
+* `--wrap-up-max-in-flight <WRAP_UP_MAX_IN_FLIGHT>` — The maximum number of in-flight requests to validators when wrapping up the benchmark. While wrapping up, this controls the concurrency level when processing inboxes and closing chains
+
+  Default value: `5`
+* `--confirm-before-start` — Confirm before starting the benchmark
+* `--runtime-in-seconds <RUNTIME_IN_SECONDS>` — How long to run the benchmark for. If not provided, the benchmark will run until it is interrupted
+* `--delay-between-chains-ms <DELAY_BETWEEN_CHAINS_MS>` — The delay between chains, in milliseconds. For example, if set to 200ms, the first chain will start, then the second will start 200 ms after the first one, the third 200 ms after the second one, and so on. This is used for slowly ramping up the TPS, so we don't pound the validators with the full TPS all at once
+* `--config-path <CONFIG_PATH>` — Path to YAML file containing chain IDs to send transfers to. If not provided, only transfers between chains in the same wallet
+
+
+
+## `linera multi-benchmark`
+
+Runs multiple `linera benchmark` processes in parallel
+
+**Usage:** `linera multi-benchmark [OPTIONS] --faucet <FAUCET>`
+
+###### **Options:**
+
+* `--processes <PROCESSES>` — The number of `linera benchmark` processes to run in parallel
+
+  Default value: `1`
+* `--faucet <FAUCET>` — The faucet (which implicitly defines the network)
+* `--client-state-dir <CLIENT_STATE_DIR>` — If specified, a directory with a random name will be created in this directory, and the client state will be stored there. If not specified, a temporary directory will be used for each client
+* `--num-chains <NUM_CHAINS>` — How many chains to use
+
+  Default value: `10`
+* `--tokens-per-chain <TOKENS_PER_CHAIN>` — How many tokens to assign to each newly created chain. These need to cover the transaction fees per chain for the benchmark
+
+  Default value: `0.1`
+* `--transactions-per-block <TRANSACTIONS_PER_BLOCK>` — How many transactions to put in each block
+
+  Default value: `1`
+* `--fungible-application-id <FUNGIBLE_APPLICATION_ID>` — The application ID of a fungible token on the wallet's default chain. If none is specified, the benchmark uses the native token
+* `--bps <BPS>` — The fixed BPS (Blocks Per Second) rate that block proposals will be sent at
+
+  Default value: `10`
+* `--close-chains` — If provided, will close the chains after the benchmark is finished. Keep in mind that closing the chains might take a while, and will increase the validator latency while they're being closed
+* `--health-check-endpoints <HEALTH_CHECK_ENDPOINTS>` — A comma-separated list of host:port pairs to query for health metrics. If provided, the benchmark will check these endpoints for validator health and terminate if any validator is unhealthy. Example: "127.0.0.1:21100,validator-1.some-network.linera.net:21100"
+* `--wrap-up-max-in-flight <WRAP_UP_MAX_IN_FLIGHT>` — The maximum number of in-flight requests to validators when wrapping up the benchmark. While wrapping up, this controls the concurrency level when processing inboxes and closing chains
+
+  Default value: `5`
+* `--confirm-before-start` — Confirm before starting the benchmark
+* `--runtime-in-seconds <RUNTIME_IN_SECONDS>` — How long to run the benchmark for. If not provided, the benchmark will run until it is interrupted
+* `--delay-between-chains-ms <DELAY_BETWEEN_CHAINS_MS>` — The delay between chains, in milliseconds. For example, if set to 200ms, the first chain will start, then the second will start 200 ms after the first one, the third 200 ms after the second one, and so on. This is used for slowly ramping up the TPS, so we don't pound the validators with the full TPS all at once
+* `--config-path <CONFIG_PATH>` — Path to YAML file containing chain IDs to send transfers to. If not provided, only transfers between chains in the same wallet
+* `--delay-between-processes <DELAY_BETWEEN_PROCESSES>` — The delay between starting the benchmark processes, in seconds. If --cross-wallet-transfers is true, this will be ignored
+
+  Default value: `10`
+* `--cross-wallet-transfers` — Whether to send transfers between chains in different wallets
+
+
+
 ## `linera create-genesis-config`
 
 Create genesis configuration for a Linera deployment. Create initial user chains and print information to be used for initialization of validator setup. This will also create an initial wallet for the owner of the initial "root" chains

linera-chain/Cargo.toml

@@ -12,7 +12,6 @@ repository.workspace = true
 version.workspace = true
 
 [features]
-benchmark = ["linera-base/test"]
 metrics = ["prometheus", "linera-views/metrics", "linera-execution/metrics"]
 test = [
     "dep:anyhow",

linera-chain/src/data_types.rs

@@ -366,7 +366,7 @@ pub struct LiteVote {
 
 impl LiteVote {
     /// Returns the full vote, with the value, if it matches.
-    #[cfg(any(feature = "benchmark", with_testing))]
+    #[cfg(with_testing)]
     pub fn with_value<T: CertificateValue>(self, value: T) -> Option<Vote<T>> {
         if self.value.value_hash != value.hash() {
             return None;

linera-client/Cargo.toml

@@ -13,16 +13,6 @@ version.workspace = true
 
 [features]
 test = ["linera-views/test", "linera-execution/test"]
-benchmark = [
-    "linera-base/test",
-    "dep:linera-sdk",
-    "dep:num-format",
-    "dep:reqwest",
-    "dep:anyhow",
-    "dep:prometheus-parse",
-    "dep:rand",
-    "dep:serde_yaml",
-]
 wasmer = [
     "linera-core/wasmer",
     "linera-execution/wasmer",
@@ -55,7 +45,7 @@ indexed-db = ["web", "indexed_db_futures", "serde-wasm-bindgen", "gloo-utils"]
 web-default = ["web", "wasmer", "indexed-db"]
 
 [dependencies]
-anyhow = { workspace = true, optional = true }
+anyhow.workspace = true
 bcs.workspace = true
 clap.workspace = true
 futures.workspace = true
@@ -66,16 +56,16 @@ linera-core.workspace = true
 linera-execution.workspace = true
 linera-persistent = { version = "0.15.0", path = "../linera-persistent" }
 linera-rpc.workspace = true
-linera-sdk = { workspace = true, optional = true }
+linera-sdk.workspace = true
 linera-storage.workspace = true
 linera-version = { workspace = true }
 linera-views.workspace = true
-num-format = { workspace = true, optional = true }
-prometheus-parse = { workspace = true, optional = true }
-rand = { workspace = true, optional = true }
-reqwest = { workspace = true, optional = true }
+num-format.workspace = true
+prometheus-parse.workspace = true
+rand.workspace = true
+reqwest.workspace = true
 serde.workspace = true
-serde_yaml = { workspace = true, optional = true }
+serde_yaml.workspace = true
 thiserror.workspace = true
 thiserror-context.workspace = true
 tokio.workspace = true

linera-client/src/client_context.rs

@@ -24,7 +24,7 @@ use linera_rpc::node_provider::{NodeOptions, NodeProvider};
 use linera_version::VersionInfo;
 use thiserror_context::Context;
 use tracing::{debug, info};
-#[cfg(feature = "benchmark")]
+#[cfg(not(web))]
 use {
     crate::benchmark::{Benchmark, BenchmarkError},
     futures::{stream, StreamExt, TryStreamExt},
@@ -663,7 +663,7 @@ where
     }
 }
 
-#[cfg(feature = "benchmark")]
+#[cfg(not(web))]
 impl<Env: Environment, W> ClientContext<Env, W>
 where
     W: Persist<Target = Wallet>,

linera-client/src/error.rs

@@ -9,7 +9,7 @@ use linera_persistent as persistent;
 use linera_version::VersionInfo;
 use thiserror_context::Context;
 
-#[cfg(feature = "benchmark")]
+#[cfg(not(web))]
 use crate::benchmark::BenchmarkError;
 use crate::util;
 
@@ -40,7 +40,7 @@ pub(crate) enum Inner {
     Arithmetic(#[from] linera_base::data_types::ArithmeticError),
     #[error("incorrect chain ownership")]
     ChainOwnership,
-    #[cfg(feature = "benchmark")]
+    #[cfg(not(web))]
     #[error("Benchmark error: {0}")]
     Benchmark(#[from] BenchmarkError),
     #[error("Validator version {remote} is not compatible with local version {local}.")]

linera-client/src/lib.rs

@@ -17,7 +17,7 @@ mod error;
 pub mod util;
 pub mod wallet;
 
-#[cfg(feature = "benchmark")]
+#[cfg(not(web))]
 pub mod benchmark;
 
 #[cfg(test)]

linera-indexer/example/Cargo.toml

@@ -13,7 +13,6 @@ version.workspace = true
 
 [features]
 default = ["rocksdb"]
-benchmark = ["linera-base/test", "linera-indexer/benchmark"]
 rocksdb = ["linera-indexer/rocksdb", "linera-indexer-plugins/rocksdb"]
 dynamodb = ["linera-indexer/dynamodb", "linera-indexer-plugins/dynamodb"]
 scylladb = ["linera-indexer/scylladb", "linera-indexer-plugins/scylladb"]

linera-indexer/lib/Cargo.toml

@@ -13,7 +13,6 @@ version.workspace = true
 
 [features]
 default = ["rocksdb"]
-benchmark = ["linera-base/test"]
 rocksdb = ["linera-views/rocksdb", "linera-core/rocksdb"]
 dynamodb = ["linera-views/dynamodb", "linera-core/dynamodb"]
 scylladb = ["linera-views/scylladb", "linera-core/scylladb"]