feat: merge-train/barretenberg (#19653)

BEGIN_COMMIT_OVERRIDE
chore: add graph_description_keccak test (#19575)
chore: update commit hash in blake files in blake audit scope (#19593)
feat: enable asserts in WASM builds (#19632)
chore: remove unnecessary "inputs" structs (#19660)
fix: overflow in wasm assert (#19690)
chore!: sha audit 2 (#19436)
chore: add logic audit scope (and add old bigfield scope) (#19680)
END_COMMIT_OVERRIDE

Author: AztecBot

barretenberg/cpp/scripts/audit/audit_scopes/bigfield_audit_scope.md

@@ -0,0 +1,58 @@
+# External Audit Scope: Bigfield
+
+Repository: https://github.com/AztecProtocol/aztec-packages
+
+Commit hash: [d0ee94134b6cf290cf93cccf30354278d2bdff59](https://github.com/AztecProtocol/aztec-packages/tree/d0ee94134b6cf290cf93cccf30354278d2bdff59)
+
+## Files to Audit
+
+Note: Paths relative to `aztec-packages/barretenberg/cpp/src/barretenberg`
+
+1. `stdlib/primitives/bigfield/bigfield.hpp`
+2. `stdlib/primitives/bigfield/bigfield_impl.hpp`
+3. `stdlib/primitives/bigfield/constants.hpp`
+
+Relations: (wasn't explicitly in the SoW but was still audited)
+
+4. `relations/non_native_field_relations.hpp`
+
+## Summary of Module
+
+The `bigfield` module implements non-native field arithmetic inside a circuit. It enables arithmetic operations on field elements from a different (larger) field than the native circuit field, which is essential for operations like
+
+- Recursive verification of BN254-based proofs inside BN254 circuits, and
+- ECDSA verification where we need to work with secp256k1/r1 field elements inside BN254-based circuits.
+
+**Representation**: Each `bigfield` element is represented using:
+
+- 4 binary basis limbs of 68 bits each (total 272 bits)
+- A prime basis limb (the value mod native field modulus)
+- Maximum value tracking for each limb to enable lazy reduction
+
+The value is: `limb[0] + limb[1] * 2^68 + limb[2] * 2^136 + limb[3] * 2^204`
+
+**Operations**: Implements full field arithmetic (+, -, \*, /) with:
+
+- Lazy reduction to minimize expensive range checks
+- Chinese Remainder Theorem (CRT) for efficient multiplication verification
+- Optimized gate usage (4 gates for addition, custom gates for multiplication)
+
+**CRT-based Multiplication**: To verify `a * b = r mod p`:
+
+- Checks equation holds mod 2^272 (binary basis) via schoolbook multiplication
+- Checks equation holds mod native field (prime basis) via single multiplication gate
+- Ensures both sides are less than CRT modulus `M = 2^272 * n`
+
+**Range Tracking**: The module tracks maximum values of limbs to:
+
+- Determine when reduction is needed before overflow
+- Compute appropriate range constraints for quotient/carry values
+- Enable batching multiple operations before reduction
+
+Please refer to the [bigfield README](https://github.com/AztecProtocol/aztec-packages/blob/d0ee94134b6cf290cf93cccf30354278d2bdff59/barretenberg/cpp/src/barretenberg/stdlib/primitives/bigfield/README.md) for detailed specification of the multiplication, addition, subtraction, and division algorithms.
+
+> Note: The README uses LaTeX notation which doesn't render well on GitHub; you might need to use Markdown preview in VS Code to render the file.
+
+## Test Files
+
+1. `stdlib/primitives/bigfield/bigfield.test.cpp`

barretenberg/cpp/scripts/audit/audit_scopes/biggroup_audit_scope.md

@@ -1,6 +1,7 @@
 # External Audit Scope: Biggroup
 
 Repository: https://github.com/AztecProtocol/aztec-packages
+
 Commit hash: [553c5eb82901955c638b943065acd3e47fc918c0](https://github.com/AztecProtocol/aztec-packages/tree/553c5eb82901955c638b943065acd3e47fc918c0)
 
 ## Files to Audit
@@ -13,24 +14,20 @@ The following files are to be audited, located in the `stdlib/primitives/biggrou
 4. `stdlib/primitives/biggroup/biggroup_tables.hpp`
 5. `stdlib/primitives/biggroup/biggroup_secp256k1.hpp`
 6. `stdlib/primitives/biggroup/biggroup_edgecase_handling.hpp`
-
-Update: Fixed lookup tables are implemented in `stdlib_circuit_builders/plookup_tables/non_native_group_generator.cpp` which must be added to the scope.
-
 7. `stdlib_circuit_builders/plookup_tables/non_native_group_generator.cpp`
 8. `stdlib_circuit_builders/plookup_tables/non_native_group_generator.hpp`
 
 ## Brief Summary of Module
 
-The biggroup module implements elliptic-curve operations using UltraHonk arithmetisation in barretenberg. This is specifically implemented to work for three curves[^1]: bn254, secp256k1 and secp256r1.
+The biggroup module implements elliptic-curve operations using UltraHonk arithmetisation in barretenberg. This is specifically implemented to work for three curves: bn254, secp256k1 and secp256r1 (see Note 1).
 
-Please refer to the [biggroup README](https://github.com/AztecProtocol/aztec-packages/blob/553c5eb82901955c638b943065acd3e47fc918c0/barretenberg/cpp/src/barretenberg/stdlib/primitives/biggroup/README.md) for details on the specification and implementation details.[^2]
+Please refer to the [biggroup README](https://github.com/AztecProtocol/aztec-packages/blob/553c5eb82901955c638b943065acd3e47fc918c0/barretenberg/cpp/src/barretenberg/stdlib/primitives/biggroup/README.md) for details on the specification and implementation details (see Note 2).
+
+> Note 1: Our implementation can _technically_ work for other curves as well (so long as the base and scalar fields of the curve can be represented with our bigfield module) but we have not tested it for other curves.
+>
+> Note 2: The README uses Latex notation which doesn't render well on Github, you might need to use Markdown preview in VS code to render the file.
 
 ## Test Files
+
 1. `stdlib/primitives/biggroup/biggroup.test.cpp`
 2. `stdlib/primitives/biggroup/biggroup_secp256k1.test.cpp`
-
-
-
-[^1]: Our implementation can _technically_ work for other curves as well (so long as the base and scalar fields of the curve can be represented with our bigfield module) but we have not tested it for other curves.
-
-[^2]: The README uses Latex notation which doesn't render well on Github, you might need to use Markdown preview in VS code to render the file.

barretenberg/cpp/scripts/audit/audit_scopes/eccvm_builder_prover_audit_scope.md

@@ -0,0 +1,40 @@
+# External Audit Scope: Logic
+
+Repository: https://github.com/AztecProtocol/aztec-packages
+
+Commit hash: TBD
+
+## Files to Audit
+
+Note: Paths relative to `aztec-packages/barretenberg/cpp/src/barretenberg`
+
+1. `stdlib/primitives/logic/logic.hpp`
+2. `stdlib/primitives/logic/logic.cpp`
+3. `stdlib_circuit_builders/plookup_tables/uint.hpp` (lookup tables)
+
+## Summary of Module
+
+The `logic` module provides circuit-friendly implementations of bitwise logical operations (XOR and AND) over variable-length unsigned integers using plookup tables.
+
+Main function: `create_logic_constraint(a, b, num_bits, is_xor_gate)`
+
+- Computes `a XOR b` or `a AND b` for inputs up to `num_bits` in length
+- Supports inputs up to 252 bits (grumpkin::MAX_NO_WRAP_INTEGER_BIT_LENGTH)
+
+The implementation:
+
+- Decomposes inputs into 32-bit chunks
+- Performs lookups against `UINT32_XOR` or `UINT32_AND` multi-tables for each chunk
+- The lookup operation implicitly enforces 32-bit range constraints on each chunk
+- For non-32-bit-aligned inputs, the final chunk is explicitly range-constrained to the remaining bits
+- Input values are reconstructed from chunks and verified via `assert_equal`
+- If both inputs are constants, the operation is computed natively without circuit constraints
+- If one input is constant, it is converted to a witness before processing
+
+## Test Files
+
+1. `stdlib/primitives/logic/logic.test.cpp`
+
+## Dependencies
+
+- Plookup read: `stdlib/primitives/plookup/plookup.hpp`

barretenberg/cpp/scripts/audit/audit_scopes/logic_scope_doc.md

@@ -1,16 +1,17 @@
 # External Audit Scope: Ultra + MegaCircuitBuilder
 
 Repository: https://github.com/AztecProtocol/aztec-packages
-Commit hash: To be added in January
+
+Commit hash: [6d14241271ad07c72937498b66f28df630662c2c](https://github.com/AztecProtocol/aztec-packages/tree/6d14241271ad07c72937498b66f28df630662c2c)
+
 Status: Planned, [Luke, Raju]
 
 ## Files to Audit
 
 Note: Paths relative to `aztec-packages/barretenberg/cpp/src/barretenberg`
 
-
-
 ### Circuit Builders
+
 1. `stdlib_circuit_builders/circuit_builder_base.hpp`
 2. `stdlib_circuit_builders/circuit_builder_base_impl.hpp`
 3. `stdlib_circuit_builders/ultra_circuit_builder.hpp`
@@ -23,6 +24,7 @@ Note: Paths relative to `aztec-packages/barretenberg/cpp/src/barretenberg`
 10. `honk/execution_trace/gate_data.hpp`
 
 ### Relations (Ultra)
+
 11. `relations/ultra_arithmetic_relation.hpp`
 12. `relations/permutation_relation.hpp`
 13. `relations/logderiv_lookup_relation.hpp`
@@ -34,10 +36,12 @@ Note: Paths relative to `aztec-packages/barretenberg/cpp/src/barretenberg`
 19. `relations/poseidon2_internal_relation.hpp`
 
 ### Relations (Mega-only)
+
 20. `relations/ecc_op_queue_relation.hpp`
 21. `relations/databus_lookup_relation.hpp`
 
 ### Lookup infrastructure
+
 22. `stdlib_circuit_builders/plookup_tables/plookup_tables.hpp`
 23. `stdlib_circuit_builders/plookup_tables/plookup_tables.cpp`
 24. `stdlib_circuit_builders/plookup_tables/types.hpp`
@@ -48,21 +52,34 @@ Note: Paths relative to `aztec-packages/barretenberg/cpp/src/barretenberg`
 ### ECC Op Queue
 
 The following is "joint" functionality for the ECCVM and the Mega circuit builder (called `UltraOps`. In this audit, we only care about how things are represented in the Mega circuit builder.
+
 28. `op_queue/ecc_op_queue.hpp`
 29. `op_queue/ecc_ops_table.hpp` (especially the `UltraEccOpsTable` class and its methods)
 
+### Stdlib Goblin Components
+
+We represent bn254 group elements in the Mega circuit builder using "Goblinized" representations. Particularly, the `goblin_field` represents bn254 base field elements (x, y coordinates), and the `biggroup_goblin` represents bn254 group elements.
+
+30. `stdlib/primitives/bigfield/goblin_field.hpp`
+31. `stdlib/primitives/biggroup/biggroup_goblin.hpp`
+32. `stdlib/primitives/biggroup/biggroup_goblin_impl.hpp`
+
 ### Databus
+
 Within this audit, it is important to make sure that the databus "correctly links up" with the Mega circuit builder. Therefore, the following file is also in the scope of the audit.
-30. `stdlib_circuit_builders/databus.hpp`
+
+33. `stdlib_circuit_builders/databus.hpp`
 
 ### ACIR Format
-31. `dsl/acir_format/range_constraint.hpp`
+
+34. `dsl/acir_format/range_constraint.hpp`
 
 ## Brief Summary of Module
 
 The Ultra/MegaCircuitBuilder module implements the core circuit construction infrastructure for Barretenberg's proving system.
 
 **Class Hierarchy:**
+
 ```
 CircuitBuilderBase<FF>
     └── UltraCircuitBuilder_<ExecutionTrace>
@@ -78,6 +95,7 @@ CircuitBuilderBase<FF>
 ## Test Files
 
 ### Circuit Builder Tests
+
 1. `circuit_checker/ultra_circuit_builder_basic.test.cpp`
 2. `circuit_checker/ultra_circuit_builder_arithmetic.test.cpp`
 3. `circuit_checker/ultra_circuit_builder_elliptic.test.cpp`
@@ -89,20 +107,25 @@ CircuitBuilderBase<FF>
 9. `circuit_checker/mega_circuit_builder.test.cpp`
 
 ### Relation Tests
+
 10. `relations/ultra_relation_consistency.test.cpp`
 
 ### Test Utilities
+
 1. `circuit_checker/circuit_checker.hpp`
 2. `circuit_checker/ultra_circuit_checker.hpp`
 3. `circuit_checker/ultra_circuit_checker.cpp`
 
 ## Security Mechanisms
 
 ### SMT (Satisfiability Modulo Theories) Verification
+
 1. `smt_verification/circuit/ultra_circuit.test.cpp`
 
 ## Misc. Tests (NOT part of the audit, but might be helpful to situation)
+
 The full prove-verify testing package is more extensive than the mere `circuit_checker` tests. Therefore, the following tests might be helpful as reference points.
+
 1. `ultra_honk/lookup.test.cpp`
 2. `ultra_honk/permutation.test.cpp`
 3. `ultra_honk/rom_ram.test.cpp`

barretenberg/cpp/scripts/audit/audit_scopes/ultra_mega_builder_audit_scope.md

@@ -13,7 +13,7 @@ cd ..
 # - Generate a hash for versioning: sha256sum bb-chonk-inputs.tar.gz
 # - Upload the compressed results: aws s3 cp bb-chonk-inputs.tar.gz s3://aztec-ci-artifacts/protocol/bb-chonk-inputs-[hash(0:8)].tar.gz
 # Note: In case of the "Test suite failed to run ... Unexpected token 'with' " error, need to run: docker pull aztecprotocol/build:3.0
-pinned_short_hash="db8f42e3"
+pinned_short_hash="0d7388db"
 pinned_chonk_inputs_url="https://aztec-ci-artifacts.s3.us-east-2.amazonaws.com/protocol/bb-chonk-inputs-${pinned_short_hash}.tar.gz"
 
 script_path="$(cd "$(dirname "${BASH_SOURCE[0]}")/scripts" && pwd)/$(basename "${BASH_SOURCE[0]}")"

barretenberg/cpp/scripts/test_chonk_standalone_vks_havent_changed.sh

@@ -1266,6 +1266,47 @@ inline void StaticAnalyzer_<FF, CircuitBuilder>::remove_unnecessary_sha256_plook
     }
 }
 
+/**
+ * @brief This method removes false positive cases from keccak lookup tables.
+ * Tables which are enumerated in keccak_plookup_tables are used by keccak lookup constraints. Some lookup-gate outputs
+ * are auxiliary (e.g. MSB) and may appear in only one gate but this is not dangerous. So we remove these variables.
+ * @tparam FF
+ * @tparam CircuitBuilder
+ * @param table_id
+ * @param gate_index
+ */
+template <typename FF, typename CircuitBuilder>
+inline void StaticAnalyzer_<FF, CircuitBuilder>::remove_unnecessary_keccak_plookup_variables(BasicTableId& table_id,
+                                                                                             size_t gate_index)
+{
+    auto find_position = [&](uint32_t real_variable_index) {
+        return variables_in_one_gate.contains(real_variable_index);
+    };
+
+    std::unordered_set<BasicTableId> keccak_plookup_tables{
+        BasicTableId::KECCAK_INPUT, BasicTableId::KECCAK_OUTPUT, BasicTableId::KECCAK_CHI,   BasicTableId::KECCAK_THETA,
+        BasicTableId::KECCAK_RHO,   BasicTableId::KECCAK_RHO_1,  BasicTableId::KECCAK_RHO_2, BasicTableId::KECCAK_RHO_3,
+        BasicTableId::KECCAK_RHO_4, BasicTableId::KECCAK_RHO_5,  BasicTableId::KECCAK_RHO_6, BasicTableId::KECCAK_RHO_7,
+        BasicTableId::KECCAK_RHO_8, BasicTableId::KECCAK_RHO_9
+    };
+
+    auto& lookup_block = circuit_builder.blocks.lookup;
+
+    if (keccak_plookup_tables.contains(table_id)) {
+        uint32_t real_out_idx = this->to_real(lookup_block.w_o()[gate_index]);
+        uint32_t real_right_idx = this->to_real(lookup_block.w_r()[gate_index]);
+        if (variables_gate_counts[real_out_idx] != 1 || variables_gate_counts[real_right_idx] != 1) {
+            bool find_out = find_position(real_out_idx);
+            auto q_c = lookup_block.q_c()[gate_index];
+            if (q_c.is_zero()) {
+                if (find_out) {
+                    variables_in_one_gate.erase(real_out_idx);
+                }
+            }
+        }
+    }
+}
+
 /**
  * @brief this method removes false cases in lookup table for a given gate.
  * it uses all functions above for lookup tables to remove all variables that appear in one gate,
@@ -1294,6 +1335,8 @@ inline void StaticAnalyzer_<FF, CircuitBuilder>::process_current_plookup_gate(si
             this->remove_unnecessary_aes_plookup_variables(table_id, gate_index);
             // false cases for sha256
             this->remove_unnecessary_sha256_plookup_variables(table_id, gate_index);
+            // false cases for keccak
+            this->remove_unnecessary_keccak_plookup_variables(table_id, gate_index);
             // if the amount of unique elements from columns of plookup tables = 1, it means that
             // variable from this column aren't used and we can remove it.
             if (column_1.size() == 1) {

barretenberg/cpp/src/barretenberg/boomerang_value_detection/graph.cpp

@@ -142,6 +142,7 @@ template <typename FF, typename CircuitBuilder> class StaticAnalyzer_ {
     void remove_unnecessary_range_constrains_variables();
     void remove_unnecessary_aes_plookup_variables(bb::plookup::BasicTableId& table_id, size_t gate_index);
     void remove_unnecessary_sha256_plookup_variables(bb::plookup::BasicTableId& table_id, size_t gate_index);
+    void remove_unnecessary_keccak_plookup_variables(bb::plookup::BasicTableId& table_id, size_t gate_index);
     void remove_record_witness_variables();
 
     std::unordered_set<uint32_t> get_variables_in_one_gate();