Shelley mempool per-tx size check: do not include perTxOverhead

See the PR description for details

Author: amesgen

ouroboros-consensus-cardano/changelog.d/20250107_124744_alexander.esgen_shelley_mempool_txsize_check.md

@@ -0,0 +1,4 @@
+### Patch
+
+- Fixed a bug where a valid tx with less than `4` bytes less than the max tx
+  size would be incorrectly rejected by the mempool.

ouroboros-consensus-cardano/src/shelley/Ouroboros/Consensus/Shelley/Ledger/Mempool.hs

@@ -328,9 +328,9 @@ txInBlockSize ::
 txInBlockSize st (ShelleyTx _txid tx') =
     validateMaybe (maxTxSizeUTxO txsz limit) $ do
       guard $ txsz <= limit
-      Just $ IgnoringOverflow $ ByteSize32 $ fromIntegral txsz
+      Just $ IgnoringOverflow $ ByteSize32 $ fromIntegral txsz + perTxOverhead
   where
-    txsz = perTxOverhead + (tx' ^. sizeTxF)
+    txsz = tx' ^. sizeTxF
 
     pparams = getPParams $ tickedShelleyLedgerState st
     limit   = fromIntegral (pparams ^. L.ppMaxTxSizeL) :: Integer