Merging 3.1.0 release branch to master

Author: amc-ie

clog.txt

@@ -0,0 +1,253 @@
+PR #16774: Granular nix: support dune files with only `public_name`
+PR #16771: [Sync] Port compatibility script fix to comp
+PR #16764: [Clean up] Remove obsolete mina-bots docker
+PR #16766: Remove some old files
+PR #16761: Remove unused sexp derivations from masking ledger
+PR #16762: Decoupling Daemon External Port from healthcheck script
+PR #16760: Fix rosetta sanity success message
+PR #16755: [CI] Prune dockers before build
+PR #16754: Fix internal logs for itn 
+PR #16751: Remove User_command.Valid.Stable
+PR #16739: Remove Transaction.Valid.Stable
+PR #16748: [compatible] fix work vk tracking issue
+PR #16743: Buildkite: fix shellcheck on setup-db-archive-node
+PR #16738: Remove Zkapp_command.Verifiable.Stable
+PR #16737: Remove Mina_wire_types.Mina_transaction.Valid
+PR #16736: Remove Stable from Transaction_hash.User_command_with_valid_signature
+PR #16733: [CI][Publish] skip debian channel check for unstable channel
+PR #16698: [CI] Unify buildkite cache access
+PR #16727: [CI] Fix running CI directly from buildkite
+PR #16726: [Repo Cleaning] Remove every trace of itn_orchestrator from repo
+PR #16665: Fix pipeline run from forked repo 
+PR #16701: [CI] remove helm ci jobs
+PR #16710: [Repo Cleaning]  Remove not used k8s folder 
+PR #16711: [Repo Cleaning] Remove unused helm charts 
+PR #16719: [Sync] flake.nix 
+PR #16720: [Sync] Bump watchdog docker from version
+PR #16717: [Sync] Port docs fixes from develop 
+PR #16706: Reformat dune files in compatible
+PR #16708: Updated Rosetta code-owners
+PR #16695: An attempt to fix the random Rosetta processing stalled issue.
+PR #16653: Small refactoring of `mina_block.ml` and `block_sink.ml`
+PR #16703: remove obsoleted terraform script
+PR #16694: [COMP] Port caqti revert to compatible
+PR #16613: 
+PR #16392: Skip publishing dockers on PR 
+PR #16630: [CI] remove integration tests cloud leftovers
+PR #16681: remove root key on split of the root -> root_(hash, common)
+PR #16651: Stubs/wasm: rename config into config.toml
+PR #16648: Kimchi-bindings: use UnsafeCell
+PR #16633: [Release]  Fix promotion  jobs race condition
+PR #16674: [Caching] use zkapp_vk_cache_tag project for caching zkapp verification keys
+PR #16683: [CI][Promotion] Do not assign env var to debs3 command
+PR #16664: [Promotion] Split reversion script
+PR #16679: [Promotion][CI] Remove buildkite/debian/publish.sh script
+PR #16678: [Promotion] Introduce quiet mode for docker promoting
+PR #16647: [CI][Refactoring] Use variant in place of plain string 
+PR #16662: [Promotion] Usability improvements for promotion scripts
+PR #16671: [Zkapp VK Caching] Cache Tag Project
+PR #16668: Version linter missing ref files check
+PR #16657: Introduce identity cache db
+PR #16658: Add functions in Ledger_proof.Cached
+PR #16654: Remove dead code
+PR #16656: replace Root with Root_(hash, common) in persistent database
+PR #16637: [Int tests] remove boilerplate code for fetching node and genesis keypair
+PR #16645: Add proof caching test
+PR #16644: [single node tests] use float comparision instead of flaky string comparision
+PR #16632: [CI] remove unused rosetta builder image scripts
+PR #16642: [CI][Comp] Disable publishing debians to debian repo in nightly
+PR #16638: [Promotion] fix two small nits
+PR #16634: [CI] do not use deb-s3 for when the version is the same
+PR #16623: [COMP] Port fixing promotion job
+PR #16604: Comprehensive json roundtrip test for precomputed blocks
+PR #16614: Upgrade proof systems to master
+PR #16620: Fix granular nix compilation
+PR #16612: remove validation service from nix
+PR #16618: Fixup archive unit tests by matching port to toolchain image
+PR #16617: Fixup some typos
+PR #16615: Small tidy-ups of Transaction_snark and Pickles
+PR #16475: [compatible] Remove apt-get update from aptly script
+PR #16610: Merge back to compatible from master
+PR #16607: [CI] Remove unused DeployTestnet job from CI
+PR #16606: [CI] Remove ValidationService from CI
+PR #16547: Unwind all config loading changes since last release
+PR #15881: Apply caqti patch 
+PR #16598: Simplify interface of persistent frontier worker
+PR #16571: Use Proof_cache_tag.t in scan state
+PR #16592: Log WAITING spam less frequently, only log when we are waiting
+PR #16591: Stop sending data on libp2p streams after the first error
+PR #16583: Manually implement JSON array printing to avoid stack overflow
+PR #16587: Check validity of memos from GraphQL before printing them
+PR #16585: Feature/fix more readmes
+PR #16584: Fixup broken link in README
+PR #16497: Decouple Transaction_snark_scan_state types
+PR #16564: add additional quick check for network
+PR #16484: Remove derivations from Transaction_snark_work.t
+PR #16574: Remove unnecessary use of virtual modules
+PR #16494: Decouple Mina_block.Validated.{Stable.Latest.t, t}
+PR #16569: Do not compute hash for logging in RPC
+PR #16568: Don't compute staged ledger hash on every RPC call
+PR #16570: [Proof cache] lmdb implementation
+PR #16312: Use devnet packages by default when promoting
+PR #16372: allow instrumentation sigterm in archive node
+PR #16528: [Proof cache] Disk cache: LMDB implementation
+PR #16560: [Proof cache] disk cache - introduce test lib
+PR #16561: [Benchmarks] fix for moving average. Add category for jane benchmarks
+PR #16554: soft fail for benches comp
+PR #16555: 
+PR #16315: Use devnet packages in connectivity tests. Do not connect to berkeley network
+PR #16552: Fixup typos
+PR #16488: Remove sexp derivation from Snark_pool
+PR #16483: Remove sexp derivation from `validated_block`
+PR #16479: Use Proof_carrying_data in Transaction_snark.t
+PR #16537: Allow override of slot times for lightnet
+PR #16538: Remove useless log messages
+PR #16521: disk cache lib
+PR #16477: Mirror Proof_carrying_data in Mina_wire_types
+PR #16500: Remove Stable from Transaction_snark_work.Checked
+PR #16491: Simplify types of Database.{move_root, find_arcs}
+PR #16490: Change type of Best_tip_lru
+PR #16456: Bring mina/compatible up to date
+PR #16524: Benchmarks - fix upload on compatible branch
+PR #16527: Fix src/lib/mina_lmdb_storage
+PR #16220: Update impure-shell.nix: use go 1.19
+PR #16509: [Trivial] lib/crypto: run dune fmt --auto-promote
+PR #16361: mina_lmdb_storage
+PR #16512: Do not load config twice in mina_cli_entrypoint.ml
+PR #15945: Run mina benchmarks in CI - First step
+PR #16478: Add basic proof_cache_tag
+PR #16515: Merge master back to compatible
+PR #16441: [COMP] reset fork config (in case there is magic config file with fork defined)
+PR #16508: [Trivial] kimchi-bindings/stubs: enforce 80 characters limit
+PR #16507: [Trivial] Reorg Rust dependencies alphabetically in Cargo.toml files
+PR #16506: [Trivial] kimchi-bindings/stubs: make clippy happier
+PR #16505: [Trivial] Kimchi-bindings/wasm: make clippy a bit happier
+PR #16492: Remove equal from Mina_block.t
+PR #16486: Hide fields of `Transaction_snark_work.Checked.t`
+PR #16485: Make Transaction_snark_work.{Checked.t != t}
+PR #16493: Persistent_frontier: use get_root_hash instead of get_root
+PR #16487: Move to_yojson outside of Stable in Mina_block.t
+PR #16482: Remove [%sexp] from Available_job.t
+PR #16474: Send command with insufficient replace fee earlier to reduce flakiness
+PR #16280: Reformat src/app/dump_blocks
+PR #16415: Increase test coverage for transaction pool module
+PR #16466: Merge `release/3.0.3.1` back to compatible
+PR #16453: Generalize pickles and snarky utils over field variable type
+PR #16402: add e2e tests for mina cli (unexpected log output)
+PR #16447: Fix indexer test
+PR #16461: Revert transaction pool reorganization
+PR #16455: Pickles_base: remove ignored warning 40-42
+PR #16452: Specialize pickles types instead of using snarky's `Cvar.t`s
+PR #16403: removing compile time config load logging from cli
+PR #16454: refactor network tests
+PR #16450: Tidy up some pickles interfaces
+PR #16449: Update snarky for backend-specific constraints
+PR #16451: CI: snarky - only checking against master
+PR #16448: Update snarky for generalized `Run_state`
+PR #16437: Print key sizes in disk_caching_stats
+PR #16433: Simplify snarky `Typ.t`, removing unnecessary type parameters
+PR #16399: Remove constraint_constants from currency_consumed
+PR #16446: Remove now-unnecessary `Pickles.compile` parameter `~constraint_constants`
+PR #16029: add option to use new debian repo
+PR #16425: [trivial] Pickles/scalar_challenge: remove old comment
+PR #16424: [trivial] Pickles: document to_field_checked'
+PR #16423: [trivial] Transaction_snark: remove old comment reg. type constraints
+PR #16432: Remove inline_test_quiet_logs package declaration
+PR #16412: Update code for simplified snarky interface
+PR #16216: using snark work rpc in bootstrap to initialize completed snark pool
+PR #16409: publish standalone snark worker
+PR #16410: [Trivial] Bump snarky to remove unused code
+PR #16406: Scripts: add a script to customize snark work production
+PR #16408: Do not derive sexp for the scan state
+PR #16367: Standalone snarkwork process to submit proofs via graphql
+PR #16366: Submit snark work using graphql
+PR #16244: GraphQL: add a query to fetch specific pending snark work
+PR #16346: use mina missing block guardian script in rosetta
+PR #16398: plonkish prelude
+PR #16212: fetch snark work on start
+PR #16370: allow to sign apt debians
+PR #16236: Introduce LedgerTestApplyTx to Ci
+PR #16379: Remove `Inline_test_quiet_logs` library
+PR #16382: naming sync lag variable
+PR #16093: Wide merkle query
+PR #16373: update sample_db data for archive test
+PR #16381: fix compile config logging
+PR #16374: Differentiate snarky `Typ.t`s between instances
+PR #16375: [Rosetta] use mina missing block guardian script instead of download missing blocks
+PR #16371: Concretize gadgets for optional custom gates
+PR #16377: Clean up noisy logging, remove `Inline_test_quiet_logs` from `Bootstrap_controller`
+PR #16368: move bitswap_tag from staged_ledger_diff to mina_net2
+PR #16364: [CMP] remove terraform , cloud engine and update README.md
+PR #16356: provide full runtime config to graphql
+PR #16360: [Trivial] Stop using `Snarky.As_prover0` and `Snarky.Types.Typ` everywhere
+PR #16359: Split `Pickles.Composition_types.Spec.ETyp.t` according to backend
+PR #16358: Specialize `Pickles.Composition_types.Branch_data`
+PR #16357: Use concrete snarky implementations for `Vector.typ` and friends
+PR #16355: Specialize `Typ.t`s in `Pickles_types`
+PR #16354: Specialize more `Typ.t`s in `Pickles.Composition_types`
+PR #16353: Rework `Pickles.Composition_types.Spec` to handle distinct snarky instances
+PR #16352: Specialize `Typ.t`s for `Pickles_types.Composition_types` to the specific step/wrap snarky implementations
+PR #16340: Update proof systems to latest `develop`
+PR #16351: [Trival] Remove spurious generalized `Typ.t`s
+PR #16350: [Trivial] Remove unused interfaces from pickles
+PR #16349: [Trivial] Fixup messy interface file
+PR #16348: [Trivial] Remove `Pickles_types` dependency from `Kimchi_pasta_snarky_backend`
+PR #16343: [COMP] Publish debians only on nightly and stable
+PR #16274: LMDB storage 
+PR #16344: merge master to compatible 15 11 24
+PR #16336: Extract snarky implementations from kimchi_backend
+PR #16285: Fix generator for transaction_pool tests
+PR #16304: Remove redundant fields from compile config type
+PR #16334: [COMP] Use new version of deb-s3 for validating job publishing
+PR #16242: consolidated benchmarks execution into python app
+PR #16201: optimize memory consumption for verifier subprocess
+PR #16327: [COMP] Fail the build on error match in output
+PR #16322: [compt] port remove fuzzy zkapp test from CI
+PR #16268: Memory monitor script for local network
+PR #16319: add coverage gather for archive related tests
+PR #16307: Fill-in TODO for runtime table feature flag
+PR #15891: [Compatible] Update badges and logo
+PR #16043: [Comp] Patch missing type shapes in a fly 
+PR #16055: Optimize the non-present zkapp uri hashing
+PR #16298: Merge back to compatible
+PR #16284: Extract Transaction_applied from Transaction_logic
+PR #16281: Do not over-expose Transaction_applied
+PR #16283: Extract Call_forest from Zkapp_command
+PR #16282: Remove unused Mina_block.payments function
+PR #16297: Javascript bindings refactor
+PR #16231: Stop using `Inline_test_quiet_logs` for `Network_pool` tests
+PR #16272: Improve lagrange retrieval performance & remove unsafes
+PR #16261: Fix Lagrange basis performance regression
+PR #16265: Re-introduce {Transaction,Blockchain}_snark.verify
+PR #16263: Remove some unused code in snark-related modules
+PR #16264: Reduce repetitions in TestBitswapSmall
+PR #16196: Remove unconfigurable_constants module
+PR #16186: [Part 1 of 2] Constants loader and Config loader
+PR #14712: Revive extract dump archive test
+PR #16233: Remove unused block window duration arguments
+PR #16232: Remove dead block storage library
+PR #16073: Add referencing for safe removal of unused Bitswap blocks
+PR #16072: Bitswap: rename `deleteXX` to `removeXX`, add tag to IPC
+PR #16252: [Comp] port mina-sidecar removal
+PR #16243: Support for uploading benchmark metrics to influx db
+PR #16222: Block_producer: move code outside of run
+PR #16229: [compatible] Remove build of libp2p_helper that we never use
+PR #16202: Update toolchain with influx db cli
+PR #15997: [Trivial] Fix innaccuracy in snarky label
+PR #16228: Remove the arbitrary 30s wait in the zkapps integration test
+PR #16226: Rosetta integration test: make block producer produce more blocks
+PR #16207: Update proof systems commit to include safe lagrange basis
+PR #16164: Use headers for v1 pubsub block topic
+PR #16206: Work_selector: small cleanup
+PR #16156: Build rosetta mainnet/devnet docker
+PR #16200: Pickles/test : moved content from tests to test
+PR #16167: unify json config loading
+PR #16163: Explicitly declare missing daemon runtime configuration
+PR #16057: Small fix of zk rows for chunking
+PR #16194: Remove Fuzzy Test run on PR
+PR #16184: [comp] split Fuzzy zkapp tests into two: quick and long
+PR #16166: Fail the build on error match in output
+PR #15898: Fix tooling to generate a test ledger
+PR #16162: Rename constructor Proof_level.{None -> No_check}
+PR #16145: Refactor transition router
+PR #16139: Update proof-systems: external prover randomness & regression tests