Skip to content

Releases: onflow/atree

v0.13.0

02 Mar 15:18
@fxamacker fxamacker
v0.13.0
b264ff6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.13.0 Latest
Latest

This release adds two functions that can speedup array conversions by 13.7x with 8x less memory use. For example, cadence.ByteArrayValueToByteSlice() using the new atree.ByteArrayToByteSlice() is:

  • 13.7 times faster 🚀
  • 8 times less memory use (bytes/op)
  • 8 times fewer memory allocations (allocs/op)

Array conversions currently done in onflow/cadence can be optimized by using these new functions from atree:

  • atree.ByteArrayToByteSlice()
  • atree.ByteSliceToByteArray()

What's Changed

  • Add two new functions to optimize atree array and Go []byte conversions by @fxamacker in #629

Atree Security Policy

The guidelines in Atree Security Policy is updated based on experience with the bug bounties since October 2025.

  • Add more guidelines for security disclosures by @fxamacker in #592
  • Add "Flow Rewards" section to Atree Security Policy by @fxamacker in #620

CI / GitHub Actions and Chores

🔎 Details...

Thanks @turbolent for porting v0.12.1 to main branch!

Full Changelog: v0.12.0...v0.13.0

Contributors

turbolent, dependabot, and fxamacker
Assets 2
Loading

v0.12.1

29 Jan 16:05
@turbolent turbolent
v0.12.1
7b1806d

Choose a tag to compare

View all tags
v0.12.1

This release adds a defensive check to prevent inserting a new element if the array already has MaxUInt32 (4.29+ billion) number of elements. In practice, arrays that need to reach 4 billion elements would require minimum of 4+ GB storage, so they would have run into some limits already enforced by client software (e.g., onflow/cadence and onflow/flow-go).

Please refer to Atree's Security Policy for more details:

Security reports should not evaluate Atree as a standalone component, because Atree relies on some limits and security guarantees provided by other components, i.e. in the Cadence programming language and the Flow node software.

Changes

Full Changelog: v0.12.0...v0.12.1

Contributors

fxamacker
Loading

v0.12.0

07 Nov 14:12
@fxamacker fxamacker
v0.12.0
e354664
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.12.0

What's Changed

There are no changes that affect behavior on testnet or mainnet.

  • Change slab size related variables from uint64 to uint32 by @fxamacker in #586
  • Bump Go and linter versions and update the code by @fxamacker in #582
  • Bump fxamacker/cbor to feature/stream-mode based on v2.9.0 by @fxamacker in #590
  • Update "Guidelines for Responsible Disclosure" in SECURITY.md by @fxamacker in #589
  • Remove unused storage function FixLoadedBrokenReferences by @fxamacker in #588
  • Add parallelism to tests to reduce atree test duration (64m -> 31m) by @fxamacker in #584

CI / GitHub Actions and Docs

🔎 Details...
  • Bump github/codeql-action from 3.30.6 to 4.30.7 by @dependabot[bot] in #581
  • Bump github/codeql-action from 4.30.7 to 4.30.8 by @dependabot[bot] in #585
  • Bump github/codeql-action from 4.30.8 to 4.30.9 by @dependabot[bot] in #591
  • Bump github/codeql-action from 4.30.9 to 4.31.0 by @dependabot[bot] in #593
  • Bump github/codeql-action from 4.31.0 to 4.31.2 by @dependabot[bot] in #594

Full Changelog: v0.11.0...v0.12.0

Contributors

dependabot and fxamacker
Loading

v0.11.0

07 Oct 17:07
@fxamacker fxamacker
v0.11.0
4041265
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.11.0

What's Changed

There are no changes that affect behavior on testnet or mainnet.

This release focused on two features that are currently only used for debugging client software:

  • Add support for iterating over map data slab elements by @turbolent in #577
  • Return more data from String() for debugging by @fxamacker in #560

CI / GitHub Actions and Docs

🔎 Details...

New Contributors

Full Changelog: v0.10.1...v0.11.0

Contributors

turbolent, dependabot, and 2 other contributors
Loading

v0.10.1

22 Aug 18:40
@fxamacker fxamacker
v0.10.1
3bbc228
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.10.1

What's Changed

This release is to bump lukechampine.com/blake3 from 1.4.0 to 1.4.1 because 1.4.0 was retraced by the author. However, lukechampine.com/blake3 is only by unit tests so this doesn't change anything.

The only other change outside of CI is effectively a lint (no change in behavior of the code):

CI / GitHub Actions

🔎 Details...
  • Bump github/codeql-action from 3.28.13 to 3.28.15 by @dependabot[bot] in #545
  • Bump codecov/codecov-action from 5.4.0 to 5.4.2 by @dependabot[bot] in #546
  • Bump github/codeql-action from 3.28.15 to 3.28.16 by @dependabot[bot] in #547
  • Bump github/codeql-action from 3.28.16 to 3.28.17 by @dependabot[bot] in #548
  • Bump lukechampine.com/blake3 from 1.4.0 to 1.4.1 by @dependabot[bot] in #549
  • Bump actions/setup-go from 5.4.0 to 5.5.0 by @dependabot[bot] in #550
  • Bump codecov/codecov-action from 5.4.2 to 5.4.3 by @dependabot[bot] in #551
  • Bump github/codeql-action from 3.28.17 to 3.28.18 by @dependabot[bot] in #552
  • Bump github/codeql-action from 3.28.18 to 3.28.19 by @dependabot[bot] in #553
  • Bump github/codeql-action from 3.28.19 to 3.29.0 by @dependabot[bot] in #554
  • Bump github/codeql-action from 3.29.0 to 3.29.1 by @dependabot[bot] in #555
  • Bump github/codeql-action from 3.29.1 to 3.29.2 by @dependabot[bot] in #556
  • Bump github/codeql-action from 3.29.2 to 3.29.4 by @dependabot[bot] in #559
  • Bump github/codeql-action from 3.29.4 to 3.29.5 by @dependabot[bot] in #561
  • Bump github/codeql-action from 3.29.7 to 3.29.8 by @dependabot[bot] in #562
  • Bump actions/checkout from 4.2.2 to 5.0.0 by @dependabot[bot] in #563
  • Bump github/codeql-action from 3.29.8 to 3.29.9 by @dependabot[bot] in #564
  • Bump github/codeql-action from 3.29.9 to 3.29.10 by @dependabot[bot] in #565
  • Bump codecov/codecov-action from 5.4.3 to 5.5.0 by @dependabot[bot] in #566
  • Bump github/codeql-action from 3.29.10 to 3.29.11 by @dependabot[bot] in #567

Full Changelog: v0.10.0...v0.10.1

Contributors

dependabot and fxamacker
Loading

v0.10.0

07 Apr 22:07
@fxamacker fxamacker
v0.10.0
c53605f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.10.0

Atree v0.10.0 was refactored to reduce tech debt and it uses the updated features/stream-mode branch of fxamacker/cbor v2.8.

Notable issues resolved:

  • #464 Refactor and reduce technical debt. This helps us add new features, maintain, and optimize (e.g., Lazy Decoding).
  • #539 and flow-go#7221 Update to newer feature/stream-mode branch in fxamacker/cbor from 2.4 to 2.8. This resolves build issue for external projects that depend on fxamacker/cbor 2.5 or newer versions.

Other changes needed by this release are at fxamacker/cbor:

What's Changed

Upgrade to fxamacker/cbor v2.8

  • Bump fxamacker/cbor to latest feature/stream-mode by @fxamacker in #542

We ran extensive tests to check backward compatibility, etc. #539 (comment) before merging the version bump to main branches.

Refactor to reduce tech debt

  • Remove old test files by @fxamacker in #475
  • Rename and export SlabIDLength and related constants by @fxamacker in #478
  • Replace SlabID{...} with NewSlabID() in tests by @fxamacker in #479
  • Use NewDefaultDigesterBuilder() in tests by @fxamacker in #480
  • Export PersistentSlabStorage funcs in export_test.go by @fxamacker in #481
  • Avoid using unexported SlabID fields in tests by @fxamacker in #484
  • Avoid creating ArrayDataSlab directly in tests by @fxamacker in #490
  • Avoid using unexported OrderedMap fields in tests by @fxamacker in #489
  • Avoid using unexported Array fields in tests by @fxamacker in #488
  • Avoid using unexported constants in tests by @fxamacker in #485
  • Avoid creating ArrayMetaDataSlab directly in tests by @fxamacker in #495
  • Avoid using MapDataSlab & MapMetaDataSlab in tests by @fxamacker in #497
  • Rename vars to reduce confusion (sizes vs counts) by @fxamacker in #498
  • Avoid using unexported mutableValueNotifier in tests by @fxamacker in #499
  • Use helper functions to compute array sizes in tests by @fxamacker in #500
  • Use helper funcs to compute map byte sizes in tests by @fxamacker in #501
  • Decouple non-test and test code by @fxamacker in #503
  • Move test util funcs to test_utils package for reuse by smoke tests by @fxamacker in #504
  • Rename cmd/stress to cmd/smoke (it only contains smoke tests) by @fxamacker in #505
  • Refactor smoke test to reduce duplicate code by @fxamacker in #506
  • Split array.go into smaller files and group related functions by @fxamacker in #508
  • Split map.go into smaller files and group related functions by @fxamacker in #509
  • Split storable.go & typeinfo.go into smaller files by @fxamacker in #510
  • Split storage.go into smaller files by @fxamacker in #511
  • Replace panic("not reachable") with panic(NewUnreachableError()) by @turbolent in #512
  • Split array_debug.go and map_debug.go into smaller files by @fxamacker in #513
  • Replace empty interface with any by @fxamacker in #514
  • Refactor array test & validation code to use range loops by @fxamacker in #515
  • Improve consistency of variable names in array tests by @fxamacker in #516
  • Refactor map tests & validation to use range loops by @fxamacker in #519
  • Improve consistency of variable names in map tests by @fxamacker in #520
  • Refactor non-test code to use range loops by @fxamacker in #521
  • Fix some misspelled words in docs/comments by @leopardracer in #529
  • Simplify and lint test code to improve maintainability and fix flakey test by @fxamacker in #522
  • Use go1.21 slices package functions in tests by @fxamacker in #524
  • Refactor to simplify merging child slabs in array by @fxamacker in #525
  • Refactor to simplify rebalancing child slabs in array by @fxamacker in #526
  • Refactor to simplify merging child slabs in map by @fxamacker in #527
  • Refactor to simplify rebalancing child slabs in map by @fxamacker in #528
  • Refactor ArrayMetaDataSlab.SplitChildSlab() to improve readability by @fxamacker in #530
  • Use go1.21 slices package in array by @fxamacker in #531
  • Use go1.21 slices package in map by @fxamacker in #532
  • Use go1.21 clear() instead of loops to clear elements by @fxamacker in #533
  • Simplify slab operations and reduce risks such as memory leaks, etc. by @fxamacker in #534
  • Fix smoke test memory usage by @fxamacker in #536

Other Changes

New Contributors

Full Changelog: v0.9.0...v0.10.0

Contributors

turbolent, dependabot, and 2 other contributors
Loading

v0.9.0

27 Jan 17:41
@fxamacker fxamacker
v0.9.0
7c1be9a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.9.0

NOTE: These release notes were updated to add the list of changes and new features AFTER all the related client software updates were deployed to mainnet.

@turbolent in #491 ported these 5 PRs created by @fxamacker from onflow/atree-internal:

New Features

This release adds 2 new interfaces that can be used by client software:

  • atree.WrapperValue
  • atree.WrapperStorable

Some client software might use containers wrapped by an opaque object, which prevents external packages (e.g. Atree) from accessing the wrapped container.

To allow Atree to access such wrapped containers, client software can implement these new Atree interfaces in their opaque object to unwrap the container if needed.

Other changes are nits that don't affect mainnet (fix or improve some tests, cleanup an unreachable scenario, add more tests, etc.)

Contributors

turbolent and fxamacker
Loading

v0.8.1

27 Nov 18:08
@fxamacker fxamacker
v0.8.1
3b39aec
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.8.1

What's Changed

This release adds the PersistentSlabStorage.HasUnsavedChanges() function.

HasUnsaveChanges() will be used by Cadence and Flow to optimize storage of domain registers to reduce payload count and mtrie count by 28.8% each (using zero downtime migration). Most of that effort will be at onflow/cadence and onflow/flow-go repos.

  • Add function to check if address has unsaved changes in storage by @fxamacker in #450
Changes to CI and version bumps...

Full Changelog: v0.8.0...v0.8.1

Contributors

dependabot and fxamacker
Loading

v0.8.0

04 Sep 15:43
@fxamacker fxamacker
v0.8.0
2a525f5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.8.0

Atree v0.8.0 introduces payload inlining and metadata deduplication to improve memory use, storage size, and speed of various servers that use payloads (not limited to EN). This improves network stability and performance as we handle future data growth.

Inlining and deduplication reduces payload count and mtrie nodes. This reduces size and growth rate of execution state.

  • Inlining stores small arrays and maps (previously stored in their own payload) into an existing payload.
  • Deduplication removes redundant metadata (e.g. Cadence struct field names and type information) to reduce payload size.

This release passed multiple types of tests, including 1034 hours of smoke tests that concluded on August 19, 2024.

Results exceeded goals mentioned in Scaling Execution Node (forum announcement).

Note

Sept. 4, 2024: 🏆 Atree inlined 500 million payloads (-61%) and eliminated over 1 billion mtrie nodes on mainnet!

Impact of these payload and mtrie node reductions include:

  • Reduced RAM use by hundreds of GB on each mainnet execution node.
  • Reduced SSD storage use (e.g. checkpoint file sizes).
  • Reduced network bandwidth for chunk data packs (mentioned by Peter).
  • Speedup transactions by 5-8% (mentioned by Jan).
  • Speedup network upgrades (e.g. Cadence 1.0 migrations, future migrations).
  • Future efficiency of other servers, components, databases (files/cache/index), etc. that benefit from fewer payloads.

Sept. 30, 2024: 🏆 Beyond execution nodes, bluesign shared a 5x speedup in tinyAN bootstrap time!
"btw amazing work this atree inlining, my tinyAN bootstrap time improved like 5x"

Future: We can replace nonatree domain payloads with atree payloads to reduce mainnet payload count by another 20-30%.

While devnet (shown below) had better percentage improvements, mainnet eliminated more mtrie nodes (over 1 billion)!

Memory Reduction Before/After Atree Inlining & Deduplication (Devnet Aug 14, 2024)

image

Mtrie Nodes and Atree Payloads (Aug 14, 2024 Devnet)Atree Payload Sizes (bytes)

Total Counts

Before After Change
mtrie nodes 1,182,195,945 343,949,052 -838,246,893
atree payloads 455,399,623 102,283,931 -353,115,692

Total Sizes

Before After Change
mtrie nodes 113.49 GB 33.02 GB -80.47 GB
atree payloads 87.74 GB 69.28 GB -18.46 GB
combined 201.23 GB 102.30 GB -98.93 GB
Before After
count 455,399,623 102,283,931
mean 192.7 677.4
std 996.4 2,065.7
min 13 22
25% 88 199
50% 139 665
75% 206 1,043
99% 876 1,267
99.5% 1,078 1,453
max 5,853,892 5,853,892

What's Changed (since v0.6.0)

  • Change OrderedMap.Get() to return Value instead of Storable by @fxamacker in #318
  • Change Array.Get() to return Value instead of Storable by @fxamacker in #316
  • Optimize max map value size to reduce number of registers by @fxamacker in #314
  • Refactor to remove lint warnings by @fxamacker in #319
  • Add Array.ID() and OrderedMap.ID() by @fxamacker in #321
  • Rename ID to ValueID by @fxamacker in #325
  • Refactor creating new StorableSlab by @fxamacker in #324
  • Unexport SlabID fields to prevent misuse by @fxamacker in #323
  • Rename StorageID related types, vars, and funcs by @fxamacker in #322
  • Add support for iterating only loaded values by @fxamacker in #311
  • Refactor hasPointer for register inlining by @fxamacker in #327
  • Reduce size of encoded array slab and bump version by @fxamacker in #330
  • Add ignore filters to dependabot.yml by @fxamacker in #302
  • Reduce encoded size of map and bump version by @fxamacker in #331
  • Update README.md to describe Atree by @fxamacker in #332
  • Update responsible disclosure link by @jribbink in #333
  • Fix slab size when resetting mutable storable in OrderedMap by @fxamacker in #337
  • Omit empty next slab ID in encoded map data slab by @fxamacker in #340
  • Omit empty next slab ID in encoded array data slab by @fxamacker in #339
  • Fix slab size when resetting mutable storable in Array by @fxamacker in #336
  • Refactor encoding version and flag to add more flags by @fxamacker in #338
  • Add readonly iterators and support value mutations only from non-readonly iterators by @fxamacker in #345
  • Atree Register Inlining and Data Deduplication by @fxamacker in #342
  • Update smoke test for atree inlining by @fxamacker in #348
  • Update for Cadence integration for atree inlining and deduplication by @fxamacker in #352
  • Make smoke tests check recently added data deduplication feature by @fxamacker in #350
  • Remove ContainerStorable.EncodeAsElement by @fxamacker in #354
  • Add feature to support mutation for array and map iterators by @fxamacker in #359
  • Add support for changing type info of atree arrays (atree inlining branch) by @fxamacker in #376
  • Add support for changing type info of atree maps (atree inlining branch) by @fxamacker in #377
  • Reduce RAM and persistent storage by deduplicating inlined dict type info by @fxamacker in #369
  • Use encoded type info to deduplicate extra data by @fxamacker in #381
  • Fix error type for external errors during serialization by @fxamacker in #382
  • Add PersistentSlabStorage.GetAllChildReferences() for atree inlining by @fxamacker in #392
  • Add feature to enable atree inlining migration to fix references to non-existent registers by @fxamacker in #388
  • Fix migration filter for old unreferenced slabs (atree inlining feature branch) by @fxamacker in #396
  • Add NonderterministicFastCommit to speed up migrations when ordering isn't required by @fxamacker in #403
  • Add BatchPreload to decode slabs in parallel and cache by @fxamacker in #404
  • Check mutation of elements from readonly map iterator by @fxamacker in #410
  • Check mutation of elements from readonly array iterator by @fxamacker in #411
  • Update comment for NondeterministicFastCommit by @fxamacker in #412
  • Update copyright notice to Flow Foundation by @fxamacker in #415
  • Update copyright notice to Flow Foundation (in main branch) by @fxamacker in #417
  • Update feature/array-map-inlining (atree inlining feature branch) by @fxamacker in #428
  • Merge feature/array-map-inlining (atree inlining feature branch) to main by @fxamacker in #429
  • Add functions to check availablility of CBOR tag numbers by @fxamacker in #434
  • Add SlabID.Address() by @fxamacker in #435
  • Make smoke tests check if CBOR tag nums are reserved by @fxamacker in #436
Changes to CI and version bumps...

Read more

Contributors

jribbink, dependabot, and fxamacker
Loading

v0.8.0-rc.6

19 Aug 14:31
@fxamacker fxamacker
v0.8.0-rc.6
141ec5c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.8.0-rc.6 Pre-release
Pre-release

Release 0.8.x (since v0.8.0-rc.1) introduces atree inlining and deduplication, so the data format is a breaking change from 0.7.x.

What's Changed Since 0.8.0-rc.5

To reduce risks, new functions were added to help projects using Atree avoid producing undecodable data in storage.

  • Add IsCBORTagNumberRangeAvailable() and ReservedCBORTagNumberRange() to check availablility of CBOR tag numbers by @fxamacker in #434

  • Add SlabID.Address() by @fxamacker in #435

Changes to CI and version bumps...

Full Changelog: v0.8.0-rc.5...v0.8.0-rc.6

Contributors

dependabot and fxamacker
Loading