Full ReferenceDA implementation (#3873)

* Add proof enhancer system with customda enhancers

This adds infrastructure to enhance one-step proofs with additional
data required by the arbitrator, particularly for custom DA systems.

The proof enhancer system intercepts one-step proofs that have an
enhancement flag set by the arbitrator. When the arbitrator needs
additional data that it cannot access directly (like DA certificates
or preimage data), it sets this flag along with a marker byte indicating
what type of enhancement is needed.

The system includes:
- ProofEnhancementManager: Routes proofs to appropriate enhancers based on marker bytes
- ReadPreimageProofEnhancer: Handles DA preimage read requests (marker 0xDA)
- ValidateCertificateProofEnhancer: Handles certificate validation requests (marker 0xDB)

Both enhancers retrieve the certificate from the sequencer message stored
in the inbox, then use the daprovider.Validator interface to generate the
appropriate proofs. This design allows the arbitrator to request DA operations
without needing to store large certificates in its limited WASM memory.

The enhanced proofs are then sent to the OSP (on-chain prover) which can
verify them against the actual DA system's validation logic.

* Add more comments explaining proof enhancement

* ProofMarker byte type alias

* Remove hardcoded values

* Move proof enhancer to its own package

* Convenience method for creating custom DA proof enhancers

* add comments about enhancement flags

* Full ReferenceDA implementation

This commit moves the ReferenceDAProofValidator contract and tests from
nitro-contracts to contracts-local, as this is a reference
implementation that doesn't need to be part of the core nitro-contracts
package. The solidity contract was already reviewed in
OffchainLabs/nitro-contracts#357

Since the Reference DA contract is now available, this commit
activates contract-based certificate validation by uncommenting the
ValidateWithContract calls in certificate.go, reference_reader.go, and
reference_validator.go. These were previously disabled with TODO
comments waiting for contract merge.

This commit also includes some changes required for nitro-testnode to
work in CustomDA mode with Reference DA. It Ensures contracts are
available in Docker builds by copying both contracts/ and
contracts-local/ directories. It also adds ReferenceDA signing key to
config dump exclusion list to prevent accidental exposure of private
keys.  This change was merged into the custom-da branch in:
#3803

Other changes required that were needed for the standalone daprovider to
work with nitro-testnode were:
   - New parent-chain-node-url and parent-chain-connection-attempts
     config
   - L1 client creation in daprovider startup for ReferenceDA mode
This change was merged into the custom-da branch in:
#3819

* Add ProviderType byte to ReferenceDA certificate

This shows how different custom DA providers can distinguish themselves
by using a byte after the DACertificateMessageHeaderFlag which
identifies the certificate as coming from some custom DA system.

Author: Tristan-Wilson

Dockerfile

@@ -331,6 +331,8 @@ COPY --from=node-builder  /workspace/target/bin/bidder-client  /usr/local/bin/
 COPY --from=node-builder  /workspace/target/bin/el-proxy  /usr/local/bin/
 COPY --from=node-builder  /workspace/target/bin/datool    /usr/local/bin/
 COPY --from=node-builder  /workspace/target/bin/genesis-generator  /usr/local/bin/
+COPY --from=contracts-builder  /workspace/contracts/  /contracts/
+COPY --from=contracts-builder  /workspace/contracts-local/  /contracts-local/
 COPY --from=nitro-legacy /home/user/target/machines /home/user/nitro-legacy/machines
 RUN rm -rf /workspace/target/legacy-machines/latest
 RUN export DEBIAN_FRONTEND=noninteractive && \

cmd/daprovider/daprovider.go

@@ -112,7 +112,8 @@ func parseDAProvider(args []string) (*Config, error) {
 
 	if config.Conf.Dump {
 		err = confighelpers.DumpConfig(k, map[string]interface{}{
-			"anytrust.key.priv-key": "",
+			"anytrust.key.priv-key":               "",
+			"referenceda.signing-key.private-key": "",
 		})
 		if err != nil {
 			return nil, fmt.Errorf("error removing extra parameters before dump: %w", err)
@@ -232,6 +233,10 @@ func startup() error {
 		if !config.ReferenceDA.Enable {
 			return errors.New("--referenceda.enable is required to start a ReferenceDA provider server")
 		}
+		l1Client, err = das.GetL1Client(ctx, config.ReferenceDA.ParentChainConnectionAttempts, config.ReferenceDA.ParentChainNodeURL)
+		if err != nil {
+			return err
+		}
 	}
 
 	// Create DA provider factory based on mode

contracts-local/foundry.toml

@@ -7,7 +7,8 @@ cache_path = 'forge-cache/sol'
 via_ir = false
 remappings = ['ds-test/=lib/forge-std/lib/ds-test/src/',
               'forge-std/=lib/forge-std/src/',
-              'openzeppelin-contracts/=lib/openzeppelin-contracts/contracts/']
+              'openzeppelin-contracts/=lib/openzeppelin-contracts/contracts/',
+              '@nitro-contracts/=../contracts/src/']
 fs_permissions = [{ access = "read", path = "./"}]
 solc = '0.8.17'
 optimizer = true
@@ -23,7 +24,8 @@ cache_path = 'forge-cache/sol'
 via_ir = false
 remappings = ['ds-test/=lib/forge-std/lib/ds-test/src/',
               'forge-std/=lib/forge-std/src/',
-              'openzeppelin-contracts/=lib/openzeppelin-contracts/contracts/']
+              'openzeppelin-contracts/=lib/openzeppelin-contracts/contracts/',
+              '@nitro-contracts/=../contracts/src/']
 fs_permissions = [{ access = "read", path = "./"}]
 solc = '0.8.24'
 optimizer = true

contracts-local/lib/forge-std

@@ -0,0 +1 @@
+../../contracts/lib/forge-std

contracts-local/src/osp/ReferenceDAProofValidator.sol

@@ -0,0 +1,207 @@
+// Copyright 2021-2024, Offchain Labs, Inc.
+// For license information, see https://github.com/OffchainLabs/nitro-contracts/blob/main/LICENSE
+// SPDX-License-Identifier: BUSL-1.1
+
+pragma solidity ^0.8.0;
+
+/* TODO Once the Custom DA contracts branch is merged we can uncomment this
+   and remove the interface definition below.
+import "@nitro-contracts/osp/ICustomDAProofValidator.sol";
+*/
+interface ICustomDAProofValidator {
+    function validateReadPreimage(
+        bytes32 certHash,
+        uint256 offset,
+        bytes calldata proof
+    ) external view returns (bytes memory preimageChunk);
+
+    function validateCertificate(
+        bytes calldata proof
+    ) external view returns (bool isValid);
+}
+
+/**
+ * @title ReferenceDAProofValidator
+ * @notice Reference implementation of a CustomDA proof validator
+ */
+contract ReferenceDAProofValidator is ICustomDAProofValidator {
+    uint256 private constant CERT_SIZE_LEN = 8;
+    uint256 private constant CLAIMED_VALID_LEN = 1;
+    uint256 private constant VERSION_LEN = 1;
+    uint256 private constant PREIMAGE_SIZE_LEN = 8;
+    uint256 private constant CERT_HEADER = 0x01;
+    uint256 private constant PROVIDER_TYPE = 0xFF;
+    uint256 private constant CERT_TOTAL_LEN = 99;
+    uint256 private constant PROOF_VERSION = 0x01;
+
+    mapping(address => bool) public trustedSigners;
+
+    constructor(
+        address[] memory _trustedSigners
+    ) {
+        for (uint256 i = 0; i < _trustedSigners.length; i++) {
+            trustedSigners[_trustedSigners[i]] = true;
+        }
+    }
+    /**
+     * @notice Validates a ReferenceDA proof and returns the preimage chunk
+     * @param certHash The keccak256 hash of the certificate (from machine's proven state)
+     * @param offset The offset into the preimage to read from (from machine's proven state)
+     * @param proof The proof data: [certSize(8), certificate, version(1), preimageSize(8), preimageData]
+     * @return preimageChunk The up to 32-byte chunk at the specified offset
+     */
+
+    function validateReadPreimage(
+        bytes32 certHash,
+        uint256 offset,
+        bytes calldata proof
+    ) external pure override returns (bytes memory preimageChunk) {
+        // Extract certificate size from proof
+        uint256 certSize = uint256(uint64(bytes8(proof[0:CERT_SIZE_LEN])));
+
+        require(proof.length >= CERT_SIZE_LEN + certSize, "Proof too short for certificate");
+        bytes calldata certificate = proof[CERT_SIZE_LEN:CERT_SIZE_LEN + certSize];
+
+        // Verify certificate hash matches what OSP validated
+        require(keccak256(certificate) == certHash, "Certificate hash mismatch");
+
+        // Validate certificate format: [header(1), providerType(1), dataHash(32), v(1), r(32), s(32)] = 99 bytes
+        // First byte must be 0x01 (CustomDA message header flag)
+        // Second byte must be 0xFF (ReferenceDA provider type)
+        require(certificate.length == CERT_TOTAL_LEN, "Invalid certificate length");
+        require(certificate[0] == bytes1(uint8(CERT_HEADER)), "Invalid certificate header");
+        require(certificate[1] == bytes1(uint8(PROVIDER_TYPE)), "Invalid provider type");
+
+        // Custom data starts after certificate
+        uint256 customDataStart = CERT_SIZE_LEN + certSize;
+        require(
+            proof.length >= customDataStart + VERSION_LEN + PREIMAGE_SIZE_LEN,
+            "Proof too short for custom data"
+        );
+
+        // Verify version
+        require(proof[customDataStart] == bytes1(uint8(PROOF_VERSION)), "Unsupported proof version");
+
+        // Extract preimage size
+        uint256 preimageSize = uint256(
+            uint64(
+                bytes8(
+                    proof[
+                        customDataStart + VERSION_LEN:
+                            customDataStart + VERSION_LEN + PREIMAGE_SIZE_LEN
+                    ]
+                )
+            )
+        );
+
+        require(
+            proof.length >= customDataStart + VERSION_LEN + PREIMAGE_SIZE_LEN + preimageSize,
+            "Invalid proof length"
+        );
+
+        // Extract and verify preimage against sha256sum in the certificate
+        bytes calldata preimage = proof[
+            customDataStart + VERSION_LEN + PREIMAGE_SIZE_LEN:
+                customDataStart + VERSION_LEN + PREIMAGE_SIZE_LEN + preimageSize
+        ];
+        bytes32 dataHashFromCert = bytes32(certificate[2:34]);
+        require(sha256(preimage) == dataHashFromCert, "Invalid preimage hash");
+
+        // Extract chunk at offset, matching the behavior of other preimage types
+        // Returns up to 32 bytes from the specified offset
+        uint256 preimageEnd = offset + 32;
+        if (preimageEnd > preimage.length) {
+            preimageEnd = preimage.length;
+        }
+
+        if (offset >= preimage.length) {
+            return new bytes(0);
+        }
+
+        return preimage[offset:preimageEnd];
+    }
+
+    /**
+     * @notice Validates whether a certificate is well-formed and legitimate
+     * @dev The proof format is: [certSize(8), certificate, claimedValid(1), validityProof...]
+     *      For ReferenceDA, the validityProof is simply a version byte (0x01).
+     *      Other DA providers can include more complex validity proofs after the claimedValid byte,
+     *      such as cryptographic signatures, merkle proofs, or other verification data.
+     *
+     *      Return vs Revert behavior:
+     *      - Reverts when:
+     *        - Proof is malformed (checked in this function)
+     *        - Provided cert matches proven hash in the instruction (checked in hostio)
+     *        - Claimed valid but is invalid and vice versa (checked in hostio)
+     *      - Returns false when:
+     *        - Certificate is malformed, including wrong length
+     *        - Signature is malformed
+     *        - Signer is not a trustedSigner
+     *      - Returns true when:
+     *        - Signer is a trustedSigner and certificate is valid
+     *
+     * @param proof The proof data starting with [certSize(8), certificate, claimedValid(1), validityProof...]
+     * @return isValid True if the certificate is valid, false otherwise
+     */
+    function validateCertificate(
+        bytes calldata proof
+    ) external view override returns (bool isValid) {
+        // Extract certificate size
+        require(proof.length >= CERT_SIZE_LEN, "Proof too short");
+
+        uint256 certSize = uint256(uint64(bytes8(proof[0:CERT_SIZE_LEN])));
+
+        // Check we have enough data for certificate and validity proof
+        require(
+            proof.length >= CERT_SIZE_LEN + certSize + CLAIMED_VALID_LEN + VERSION_LEN,
+            "Proof too short for cert and validity"
+        );
+
+        bytes calldata certificate = proof[CERT_SIZE_LEN:CERT_SIZE_LEN + certSize];
+
+        // Certificate format is: [header(1), providerType(1), dataHash(32), v(1), r(32), s(32)] = 99 bytes total
+        // First byte must be 0x01 (CustomDA message header flag)
+        // Second byte must be 0xFF (ReferenceDA provider type)
+        // Note: We return false for invalid certificates instead of reverting
+        // because the certificate is already onchain. An honest validator must be able
+        // to win a challenge to prove that ValidatePreImage should return false
+        // so that an invalid cert can be skipped.
+        if (certificate.length != CERT_TOTAL_LEN) {
+            return false; // Invalid certificate length
+        }
+        if (certificate[0] != bytes1(uint8(CERT_HEADER))) {
+            return false; // Invalid certificate header
+        }
+        if (certificate[1] != bytes1(uint8(PROVIDER_TYPE))) {
+            return false; // Invalid provider type
+        }
+
+        // Extract data hash and signature components
+        bytes32 dataHash = bytes32(certificate[2:34]);
+        uint8 v = uint8(certificate[34]);
+        bytes32 r = bytes32(certificate[35:67]);
+        bytes32 s = bytes32(certificate[67:99]);
+
+        // Recover signer from signature
+        address signer = ecrecover(dataHash, v, r, s);
+
+        // Check if signature is valid (ecrecover returns 0 on invalid signature)
+        if (signer == address(0)) {
+            return false;
+        }
+
+        // Check if signer is trusted
+        if (!trustedSigners[signer]) {
+            return false;
+        }
+
+        // Check version byte at the end of the proof
+        // Note: This is a deliberately simple example. A good rule of thumb is that
+        // anything added to the proof beyond the isValid byte must not be able to cause both
+        // true and false to be returned from this function, given the same certificate.
+        uint8 version = uint8(proof[proof.length - VERSION_LEN]);
+        require(version == PROOF_VERSION, "Invalid proof version");
+
+        return true;
+    }
+}