Add a "How-To" example for performing HTTP requests (#3509)

## Motivation

Performing HTTP requests from applications is an important feature for
Linera. There are a few different ways for applications to perform them,
and there are tradeoffs between these ways. Therefore, this feature
should be documented and a "How-To" example demonstrating how to use and
test the feature would be useful.

## Proposal

Add an example that performs HTTP requests in all different ways, and
test them.

## Test Plan

Unit and integration tests were added to test not just the application,
but also to serve as testing examples and to also test the HTTP
allow-list.

## Release Plan

- Nothing to do, because this is just a new example being added to the
repository.

## Links

- [reviewer
checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)

Author: jvff

examples/Cargo.lock

@@ -8,6 +8,7 @@ members = [
     "ethereum-tracker",
     "fungible",
     "gen-nft",
+    "how-to/perform-http-requests",
     "hex-game",
     "llm",
     "matching-engine",
@@ -20,8 +21,10 @@ members = [
 
 [workspace.dependencies]
 alloy = { version = "0.9.2", default-features = false }
+anyhow = "1.0.80"
 assert_matches = "1.5.0"
 async-graphql = { version = "=7.0.2", default-features = false }
+axum = "0.7.4"
 base64 = "0.22.0"
 bcs = "0.1.3"
 candle-core = "0.4.1"

examples/Cargo.toml

@@ -0,0 +1,36 @@
+[package]
+name = "how-to-perform-http-requests"
+version = "0.1.0"
+authors = ["Linera <[email protected]>"]
+edition = "2021"
+
+[dependencies]
+async-graphql.workspace = true
+linera-sdk.workspace = true
+serde.workspace = true
+
+[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
+anyhow.workspace = true
+axum.workspace = true
+tokio.workspace = true
+
+[dev-dependencies]
+assert_matches.workspace = true
+linera-sdk = { workspace = true, features = ["test"] }
+
+[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]
+futures.workspace = true
+linera-sdk = { workspace = true, features = ["test", "wasmer"] }
+test-log.workspace = true
+
+[[bin]]
+name = "how_to_perform_http_requests_contract"
+path = "src/contract.rs"
+
+[[bin]]
+name = "how_to_perform_http_requests_service"
+path = "src/service.rs"
+
+[[bin]]
+name = "test_http_server"
+path = "src/test_http_server.rs"

examples/how-to/perform-http-requests/Cargo.toml

@@ -0,0 +1,140 @@
+# How to perform HTTP requests
+
+This example application demonstrates how to perform HTTP requests from the service and from the
+contract, in a few different ways:
+
+- From the service while handling a mutation.
+- From the contract directly.
+- From the service when it is being used as an oracle by the contract.
+
+## HTTP requests from the service
+
+The service is executed either on the client when requested by the user or on validators when the
+service is queried as an oracle by the contract. In this first usage scenario, the HTTP request is
+executed only in the client.
+
+The HTTP response can then be used by the service to either prepare a query response to the caller
+or to prepare operations to be executed by the contract in a block proposal.
+
+## HTTP requests from the contract
+
+The contract can perform HTTP requests as well, but the responses must always be the same. The
+requests are executed on the client and on all the validators. That means that the client and each
+validator perform the HTTP request independently. The responses must all match (or at least match
+in a quorum of validators) for the block the be confirmed.
+
+If the response varies per request (as a simple example, due to the presence of a "Date" timestamp
+header in the response), the block proposal may end up being rejected by the validators. If there's
+a risk of that happening, the contract should instead call the service as an oracle, and let the
+service perform the HTTP request and return only the deterministic parts of the response.
+
+## HTTP requests using the service as an oracle
+
+The contract may call the service as an oracle. That means that that contracts sends a query to the
+service and waits for its response. The execution of the contract is metered by executed
+instruction, while the service executing as an oracle is metered by a coarse-grained timer. That
+allows the service to execute non-deterministically, and as long as it always returns a
+deterministic response back to the contract, the validators will agree on its execution and reach
+consensus.
+
+In this scenario, the contract requests the service to perform the HTTP request. The HTTP request
+is also executed in each validator.
+
+## Recommendation
+
+It is recommended to minimize the number of HTTP requests performed in total, in order to reduce
+costs. Whenever possible, it's best to perform the request in the client using the service, and
+forward only the HTTP response to the contract. The contract should then verify that the response
+can be trusted.
+
+If there's no way to verify an off-chain HTTP response in the contract, then the request should be
+made in the contract. However, if there's a risk of receiving different HTTP responses among the
+validators, the contract should use the service as oracle to perform the HTTP request and return to
+the contract only the data that is deterministic. Using the service as an oracle is more expensive,
+so it should be avoided if possible.
+
+## Usage
+
+### Setting Up
+
+Before getting started, make sure that the binary tools `linera*` corresponding to
+your version of `linera-sdk` are in your PATH.
+
+For the test, a simple HTTP server will be executed in the background.
+
+```bash
+HTTP_PORT=9090
+cd examples
+cargo run --bin test_http_server -- "$HTTP_PORT" &
+cd ..
+```
+
+From the root of Linera repository, the environment can be configured to provide a `linera_spawn`
+helper function useful for scripting, as follows:
+
+```bash
+export PATH="$PWD/target/debug:$PATH"
+source /dev/stdin <<<"$(linera net helper 2>/dev/null)"
+```
+
+To start the local Linera network:
+
+```bash
+linera_spawn linera net up --with-faucet --faucet-port 8081
+
+# Remember the URL of the faucet.
+FAUCET_URL=http://localhost:8081
+```
+
+We then create a wallet and obtain a chain to use with the application.
+
+```bash
+export LINERA_WALLET="$LINERA_TMP_DIR/wallet.json"
+export LINERA_STORAGE="rocksdb:$LINERA_TMP_DIR/client.db"
+
+linera wallet init --faucet $FAUCET_URL
+
+INFO=($(linera wallet request-chain --faucet $FAUCET_URL))
+CHAIN="${INFO[0]}"
+```
+
+Now, compile the application WebAssembly binaries, publish and create an application instance.
+
+```bash
+(cd examples/how-to/perform-http-requests && cargo build --release --target wasm32-unknown-unknown)
+
+APPLICATION_ID=$(linera publish-and-create \
+  examples/target/wasm32-unknown-unknown/release/how_to_perform_http_requests_{contract,service}.wasm \
+  --json-parameters "\"http://localhost:$HTTP_PORT\"")
+```
+
+The `APPLICATION_ID` is saved so that it can be used in the GraphQL URL later. But first the
+service that handles the GraphQL requests must be started.
+
+```bash
+PORT=8080
+linera service --port $PORT &
+```
+
+#### Using GraphiQL
+
+Type each of these in the GraphiQL interface and substitute the env variables with their actual
+values that we've defined above.
+
+- Navigate to the URL you get by running `echo "http://localhost:8080/chains/$CHAIN/applications/$APPLICATION_ID"`.
+- To query the service to perform an HTTP query locally:
+```gql,uri=http://localhost:8080/chains/$CHAIN/applications/$APPLICATION_ID
+query { performHttpRequest }
+```
+- To make the service perform an HTTP query locally and use the response to propose a block:
+```gql,uri=http://localhost:8080/chains/$CHAIN/applications/$APPLICATION_ID
+mutation { performHttpRequest }
+```
+- To make the contract perform an HTTP request:
+```gql,uri=http://localhost:8080/chains/$CHAIN/applications/$APPLICATION_ID
+mutation { performHttpRequestInContract }
+```
+- To make the contract use the service as an oracle to perform an HTTP request:
+```gql,uri=http://localhost:8080/chains/$CHAIN/applications/$APPLICATION_ID
+mutation { performHttpRequestAsOracle }
+```

examples/how-to/perform-http-requests/README.md

@@ -0,0 +1,120 @@
+// Copyright (c) Zefchain Labs, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+#![cfg_attr(target_arch = "wasm32", no_main)]
+
+use how_to_perform_http_requests::{Abi, Operation};
+use linera_sdk::{http, linera_base_types::WithContractAbi, Contract as _, ContractRuntime};
+
+pub struct Contract {
+    runtime: ContractRuntime<Self>,
+}
+
+linera_sdk::contract!(Contract);
+
+impl WithContractAbi for Contract {
+    type Abi = Abi;
+}
+
+impl linera_sdk::Contract for Contract {
+    type Message = ();
+    type InstantiationArgument = ();
+    type Parameters = String;
+
+    async fn load(runtime: ContractRuntime<Self>) -> Self {
+        Contract { runtime }
+    }
+
+    async fn instantiate(&mut self, (): Self::InstantiationArgument) {
+        // Check that the global parameters can be deserialized correctly.
+        self.runtime.application_parameters();
+    }
+
+    async fn execute_operation(&mut self, operation: Self::Operation) -> Self::Response {
+        match operation {
+            Operation::HandleHttpResponse(response_body) => {
+                self.handle_http_response(response_body)
+            }
+            Operation::PerformHttpRequest => self.perform_http_request(),
+            Operation::UseServiceAsOracle => self.use_service_as_oracle(),
+        }
+    }
+
+    async fn execute_message(&mut self, (): Self::Message) {
+        panic!("This application doesn't support any cross-chain messages");
+    }
+
+    async fn store(self) {}
+}
+
+impl Contract {
+    /// Handles an HTTP response, ensuring it is valid.
+    ///
+    /// Because the `response_body` can come from outside the contract in an
+    /// [`Operation::HandleHttpResponse`], it could be forged. Therefore, the contract should
+    /// assume that the `response_body` is untrusted, and should perform validation and
+    /// verification steps to ensure that the `response_body` is real and can be trusted.
+    ///
+    /// Usually this is done by verifying that the response is signed by the trusted HTTP server.
+    /// In this example, the verification is simulated by checking that the `response_body` is
+    /// exactly an expected value.
+    fn handle_http_response(&self, response_body: Vec<u8>) {
+        assert_eq!(response_body, b"Hello, world!");
+    }
+
+    /// Performs an HTTP request directly in the contract.
+    ///
+    /// This only works if the HTTP response (including any HTTP headers the response contains) is
+    /// the same in a quorum of validators. Otherwise, the contract should call the service as an
+    /// oracle to perform the HTTP request and the service should only return the data that will be
+    /// the same in a quorum of validators.
+    fn perform_http_request(&mut self) {
+        let url = self.runtime.application_parameters();
+        let response = self.runtime.http_request(http::Request::get(url));
+
+        self.handle_http_response(response.body);
+    }
+
+    /// Uses the service as an oracle to perform the HTTP request.
+    ///
+    /// The service can then receive a non-deterministic response and return to the contract a
+    /// deterministic response.
+    fn use_service_as_oracle(&mut self) {
+        let application_id = self.runtime.application_id();
+        let request = async_graphql::Request::new("query { performHttpRequest }");
+
+        let graphql_response = self.runtime.query_service(application_id, request);
+
+        let async_graphql::Value::Object(graphql_response_data) = graphql_response.data else {
+            panic!("Unexpected response from service: {graphql_response:#?}");
+        };
+        let async_graphql::Value::List(ref http_response_list) =
+            graphql_response_data["performHttpRequest"]
+        else {
+            panic!(
+                "Unexpected response for service's `performHttpRequest` query: {:#?}",
+                graphql_response_data
+            );
+        };
+        let http_response = http_response_list
+            .iter()
+            .map(|value| {
+                let async_graphql::Value::Number(number) = value else {
+                    panic!("Unexpected type in HTTP request body's bytes: {value:#?}");
+                };
+
+                number
+                    .as_i64()
+                    .and_then(|integer| u8::try_from(integer).ok())
+                    .unwrap_or_else(|| {
+                        panic!("Unexpected value in HTTP request body's bytes: {number:#?}")
+                    })
+            })
+            .collect();
+
+        self.handle_http_response(http_response);
+    }
+}
+
+#[path = "unit_tests/contract.rs"]
+mod unit_tests;

examples/how-to/perform-http-requests/src/contract.rs

@@ -0,0 +1,35 @@
+// Copyright (c) Zefchain Labs, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+//! ABI of the Counter Example Application
+
+use async_graphql::{Request, Response};
+use linera_sdk::{
+    abi::{ContractAbi, ServiceAbi},
+    graphql::GraphQLMutationRoot,
+};
+use serde::{Deserialize, Serialize};
+
+/// The marker type that connects the types used to interface with the application.
+pub struct Abi;
+
+impl ContractAbi for Abi {
+    type Operation = Operation;
+    type Response = ();
+}
+
+impl ServiceAbi for Abi {
+    type Query = Request;
+    type QueryResponse = Response;
+}
+
+/// Operations that the contract can handle.
+#[derive(Debug, Deserialize, Eq, PartialEq, Serialize, GraphQLMutationRoot)]
+pub enum Operation {
+    /// Handles the HTTP response of a request made outside the contract.
+    HandleHttpResponse(Vec<u8>),
+    /// Performs an HTTP request inside the contract.
+    PerformHttpRequest,
+    /// Requests the service to perform the HTTP request as an oracle.
+    UseServiceAsOracle,
+}