NativeTokenTransfer additional payload (#522)

* docs: NativeTokenTransfer additional payload

* evm: NativeTokenTransfer additional payload

* solana: NativeTokenTransfer additional payload

* sdk: NativeTokenTransfer additional payload

* evm: additional and custom payload hooks

* docs: integrator notes for NativeTokenTransfer additional payload

* docs: fixup comments

* evm: more flexible additional payload generation

* solana: fix anchor test

* evm: update _handleAdditionalPayload params

* solana: NativeTokenTransfer written_size logic mirrors write

* docs: more additional payload info

* evm: update _prepareNativeTokenTransfer and _handleAdditionalPayload for integrator flexibility

* evm: MockNttManagerAdditionalPayload and TestAdditionalPayload

* evm: _prepareNativeTokenTransfer docstring update

Author: evan-gray

docs/NttManager.md

@@ -17,13 +17,24 @@ uint16   payload_len // length of the payload
 
 ### Payloads
 
+> Note: Integrators who need to send different types of payloads should also use a unique 4-byte prefix to distinguish them from `NativeTokenTransfer` and one another.
+
 #### NativeTokenTransfer
 
 ```go
-[4]byte  prefix = 0x994E5454 // 0x99'N''T''T'
-uint8    decimals            // number of decimals for the amount
-uint64   amount              // amount being transferred
-[32]byte source_token        // source chain token address
-[32]byte recipient_address   // the address of the recipient
-uint16   recipient_chain     // the Wormhole Chain ID of the recipient
+[4]byte   prefix = 0x994E5454 // 0x99'N''T''T'
+uint8     decimals            // number of decimals for the amount
+uint64    amount              // amount being transferred
+[32]byte  source_token        // source chain token address
+[32]byte  recipient_address   // the address of the recipient
+uint16    recipient_chain     // the Wormhole Chain ID of the recipient
+```
+
+To support integrators who may want to send additional, custom data with their transfers, this format may be extended to also include these additional, optional fields. Customizing transfers in this way ensures compatibility of the canonical portion of the payload across the ecosystem (Connect, explorers, NTT Global Accountant, etc).
+
+In order to aid parsers in identifying your additional payload, it is recommended to start it with a unique 4-byte prefix.
+
+```go
+uint16 additional_payload_len // length of the custom payload
+[]byte additional_payload     // custom payload - recommended that the first 4 bytes are a unique prefix
 ```

docs/Transceiver.md

@@ -27,7 +27,7 @@ uint16   transceiver_payload_length
 #### TransceiverMessage
 
 ```go
-prefix = 0x9945FF10 // 0x99'E''W''H'
+prefix = 0x9945FF10
 ```
 
 #### Initialize Transceiver

evm/README.md

@@ -1,4 +1,6 @@
-# Message Lifecycle (EVM)
+# EVM
+
+## Message Lifecycle (EVM)
 
 1. **Transfer**
 
@@ -205,9 +207,37 @@ $ anvil --help
 $ cast --help
 ```
 
-### Deploy Wormhole NTT
+## Message Customization
+
+See the [NttManager](../docs/NttManager.md) doc for wire format details.
+
+> Note: The size of `NttManager` is significantly higher (and very close to the contract size limit) than `NttManagerNoRateLimiting`. See [No-Rate-Limiting](#no-rate-limiting) for more detail.
+
+### NativeTokenTransfer Additional Payload
+
+Your contract can extend `NttManagerNoRateLimiting` to provide an additional payload on the `NativeTokenTransfer` message. Override the following:
+
+- `_prepareNativeTokenTransfer`
+- `_handleAdditionalPayload`
+
+Be sure to review the code to ensure that they are called at an appropriate place in the flow for your use case.
+
+> Note: This is _not_ supported when RateLimiting is enabled, as this implementation does not propagate the additional payloads on queued messages. Use either with `NttManager` when `_rateLimitDuration` is set to `0` and `_skipRateLimiting` is set to `true` in the constructor or with `NttManagerNoRateLimiting`.
+
+### Custom Manager Payloads
+
+`NttManager` (or `NttManagerNoRateLimiting`) can also be extended to exchange other kinds of custom messages. To send custom messages, create a new public function and roughly follow the steps in `_transfer`. What exactly you need to do will vary on your use case, but in order to leverage the existing transceivers, you will roughly need to
+
+- call `_prepareForTransfer`
+- construct and encode your payload
+- construct the `NttManagerMessage` payload
+- call `_sendMessageToTransceivers`
+
+On the receiving side, override `_handleMsg` and adjust its code based on your custom payload prefixes, defaulting to `_handleTransfer`.
+
+## Deploy Wormhole NTT
 
-#### Environment Setup
+### Environment Setup
 
 Note: **All Chain IDs set in the deployment environment files and configuration files should be the Wormhole Chain ID**
 
@@ -222,7 +252,7 @@ Do this for each blockchain network that the `NTTManager` and `WormholeTransceiv
 
 Currently the `MAX_OUTBOUND_LIMIT` is set to zero in the sample `.env` file. This means that all outbound transfers will be queued by the rate limiter.
 
-#### Config Setup
+### Config Setup
 
 Before deploying the contracts, navigate to the `evm/cfg` directory and copy the sample file. Make sure to preserve the existing name:
 
@@ -242,7 +272,7 @@ Configure each network to your liking (including adding/removing networks). We w
 
 Currently the per-chain `inBoundLimit` is set to zero by default. This means all inbound transfers will be queued by the rate limiter. Set this value accordingly.
 
-#### Deploy
+### Deploy
 
 Deploy the `NttManager` and `WormholeTransceiver` contracts by running the following command for each target network:
 
@@ -258,7 +288,7 @@ bash sh/deploy_wormhole_ntt.sh -n NETWORK_TYPE -c CHAIN_NAME -k PRIVATE_KEY
 
 Save the deployed proxy contract addresses (see the forge script output) in the `WormholeNttConfig.json` file.
 
-#### Configuration
+### Configuration
 
 Once all of the contracts have been deployed and the addresses have been saved, run the following command for each target network:
 
@@ -272,7 +302,7 @@ bash sh/configure_wormhole_ntt.sh -n NETWORK_TYPE -c CHAIN_NAME -k PRIVATE_KEY
 -c avalanche, ethereum, sepolia
 ```
 
-#### Additional Notes
+### Additional Notes
 
 Tokens powered by NTT in **burn** mode require the `burn` method to be present. This method is not present in the standard ERC20 interface, but is found in the `ERC20Burnable` interface.
 

evm/src/NttManager/NttManager.sol

@@ -210,6 +210,27 @@ contract NttManager is INttManager, RateLimiter, ManagerBase {
             return;
         }
 
+        _handleMsg(sourceChainId, sourceNttManagerAddress, message, digest);
+    }
+
+    /// @dev Override this function to handle custom NttManager payloads.
+    /// This can also be used to customize transfer logic by using your own
+    /// _handleTransfer implementation.
+    function _handleMsg(
+        uint16 sourceChainId,
+        bytes32 sourceNttManagerAddress,
+        TransceiverStructs.NttManagerMessage memory message,
+        bytes32 digest
+    ) internal virtual {
+        _handleTransfer(sourceChainId, sourceNttManagerAddress, message, digest);
+    }
+
+    function _handleTransfer(
+        uint16 sourceChainId,
+        bytes32 sourceNttManagerAddress,
+        TransceiverStructs.NttManagerMessage memory message,
+        bytes32 digest
+    ) internal {
         TransceiverStructs.NativeTokenTransfer memory nativeTokenTransfer =
             TransceiverStructs.parseNativeTokenTransfer(message.payload);
 
@@ -231,9 +252,28 @@ contract NttManager is INttManager, RateLimiter, ManagerBase {
             return;
         }
 
+        _handleAdditionalPayload(
+            sourceChainId, sourceNttManagerAddress, message.id, message.sender, nativeTokenTransfer
+        );
+
         _mintOrUnlockToRecipient(digest, transferRecipient, nativeTransferAmount, false);
     }
 
+    /// @dev Override this function to process an additional payload on the NativeTokenTransfer
+    /// For integrator flexibility, this function is *not* marked pure or view
+    /// @param - The Wormhole chain id of the sender
+    /// @param - The address of the sender's NTT Manager contract.
+    /// @param - The message id from the NttManagerMessage.
+    /// @param - The original message sender address from the NttManagerMessage.
+    /// @param - The parsed NativeTokenTransfer, which includes the additionalPayload field
+    function _handleAdditionalPayload(
+        uint16, // sourceChainId
+        bytes32, // sourceNttManagerAddress
+        bytes32, // id
+        bytes32, // sender
+        TransceiverStructs.NativeTokenTransfer memory // nativeTokenTransfer
+    ) internal virtual {}
+
     function _enqueueOrConsumeInboundRateLimit(
         bytes32 digest,
         uint16 sourceChainId,
@@ -500,9 +540,8 @@ contract NttManager is INttManager, RateLimiter, ManagerBase {
         // push it on the stack again to avoid a stack too deep error
         uint64 seq = sequence;
 
-        TransceiverStructs.NativeTokenTransfer memory ntt = TransceiverStructs.NativeTokenTransfer(
-            amount, toWormholeFormat(token), recipient, recipientChain
-        );
+        TransceiverStructs.NativeTokenTransfer memory ntt =
+            _prepareNativeTokenTransfer(amount, token, recipient, recipientChain, seq, sender);
 
         // construct the NttManagerMessage payload
         bytes memory encodedNttManagerPayload = TransceiverStructs.encodeNttManagerMessage(
@@ -547,6 +586,29 @@ contract NttManager is INttManager, RateLimiter, ManagerBase {
         return seq;
     }
 
+    /// @dev Override this function to provide an additional payload on the NativeTokenTransfer
+    /// For integrator flexibility, this function is *not* marked pure or view
+    /// @param amount TrimmedAmount of the transfer
+    /// @param token Address of the token that this NTT Manager is tied to
+    /// @param recipient The recipient address
+    /// @param recipientChain The Wormhole chain ID for the destination
+    /// @param - The sequence number for the manager message (unused, provided for overriding integrators)
+    /// @param - The sender of the funds (unused, provided for overriding integrators). If releasing
+    /// queued transfers, when rate limiting is used, then this value could be different from msg.sender.
+    /// @return - The TransceiverStructs.NativeTokenTransfer struct
+    function _prepareNativeTokenTransfer(
+        TrimmedAmount amount,
+        address token,
+        bytes32 recipient,
+        uint16 recipientChain,
+        uint64, // sequence
+        address // sender
+    ) internal virtual returns (TransceiverStructs.NativeTokenTransfer memory) {
+        return TransceiverStructs.NativeTokenTransfer(
+            amount, toWormholeFormat(token), recipient, recipientChain, ""
+        );
+    }
+
     function _mintOrUnlockToRecipient(
         bytes32 digest,
         address recipient,

evm/src/Transceiver/WormholeTransceiver/WormholeTransceiverState.sol

@@ -36,7 +36,6 @@ abstract contract WormholeTransceiverState is IWormholeTransceiverState, Transce
     // ==================== Constants ================================================
 
     /// @dev Prefix for all TransceiverMessage payloads
-    ///      This is 0x99'E''W''H'
     /// @notice Magic string (constant value set by messaging provider) that idenfies the payload as an transceiver-emitted payload.
     ///         Note that this is not a security critical field. It's meant to be used by messaging providers to identify which messages are Transceiver-related.
     bytes4 constant WH_TRANSCEIVER_PAYLOAD_PREFIX = 0x9945FF10;

evm/src/libraries/TransceiverStructs.sol

@@ -102,6 +102,8 @@ library TransceiverStructs {
     ///      - sourceToken - 32 bytes
     ///      - to - 32 bytes
     ///      - toChain - 2 bytes
+    ///      - additionalPayloadLength - 2 bytes, optional
+    ///      - additionalPayload - `additionalPayloadLength` bytes
     struct NativeTokenTransfer {
         /// @notice Amount being transferred (big-endian u64 and u8 for decimals)
         TrimmedAmount amount;
@@ -111,6 +113,9 @@ library TransceiverStructs {
         bytes32 to;
         /// @notice Chain ID of the recipient
         uint16 toChain;
+        /// @notice Custom payload
+        /// @dev Recommended that the first 4 bytes are a unique prefix
+        bytes additionalPayload;
     }
 
     function encodeNativeTokenTransfer(
@@ -119,6 +124,22 @@ library TransceiverStructs {
         // The `amount` and `decimals` fields are encoded in reverse order compared to how they are declared in the
         // `TrimmedAmount` type. This is consistent with the Rust NTT implementation.
         TrimmedAmount transferAmount = m.amount;
+        if (m.additionalPayload.length > 0) {
+            if (m.additionalPayload.length > type(uint16).max) {
+                revert PayloadTooLong(m.additionalPayload.length);
+            }
+            uint16 additionalPayloadLength = uint16(m.additionalPayload.length);
+            return abi.encodePacked(
+                NTT_PREFIX,
+                transferAmount.getDecimals(),
+                transferAmount.getAmount(),
+                m.sourceToken,
+                m.to,
+                m.toChain,
+                additionalPayloadLength,
+                m.additionalPayload
+            );
+        }
         return abi.encodePacked(
             NTT_PREFIX,
             transferAmount.getDecimals(),
@@ -153,6 +174,14 @@ library TransceiverStructs {
         (nativeTokenTransfer.sourceToken, offset) = encoded.asBytes32Unchecked(offset);
         (nativeTokenTransfer.to, offset) = encoded.asBytes32Unchecked(offset);
         (nativeTokenTransfer.toChain, offset) = encoded.asUint16Unchecked(offset);
+        // The additional payload may be omitted, but if it is included, it is prefixed by a u16 for its length.
+        // If there are at least 2 bytes remaining, attempt to parse the additional payload.
+        if (encoded.length >= offset + 2) {
+            uint256 payloadLength;
+            (payloadLength, offset) = encoded.asUint16Unchecked(offset);
+            (nativeTokenTransfer.additionalPayload, offset) =
+                encoded.sliceUnchecked(offset, payloadLength);
+        }
         encoded.checkLength(offset);
     }