feat(gnolang): add SaveTypes function to save types in cache when node starts #5008

omarsy · 2025-12-22T19:06:51Z

alternative #4984

Gno2D2 · 2025-12-22T19:07:20Z

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):

IGNORE the bot requirements for this PR (force green CI check)

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)
🟢 Pending initial approval by a review team member, or review from tech-staff

☑️ Contributor Actions:

Fix any issues flagged by automated checks.
Follow the Contributor Checklist to ensure your PR is ready for review.
- Add new tests, or document why they are unnecessary.
- Provide clear examples/screenshots, if necessary.
- Update documentation, if required.
- Ensure no breaking changes, or include BREAKING CHANGE notes.
- Link related issues/PRs, where applicable.

☑️ Reviewer Actions:

Complete manual checks for the PR, including the guidelines and additional checks if applicable.

📚 Resources:

Debug

Automated Checks

Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 The pull request was created from a fork (head branch repo: omarsy/gno)

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

Pending initial approval by a review team member, or review from tech-staff

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 Not (🔴 Pull request author is a member of the team: tech-staff)

Then

🟢 Requirement satisfied
└── 🟢 If
    ├── 🟢 Condition
    │   └── 🟢 Or
    │       ├── 🟢 User jefft0 already reviewed PR 5008 with state APPROVED
    │       ├── 🟢 At least 1 user(s) of the team tech-staff reviewed pull request
    │       └── 🔴 This pull request is a draft
    └── 🟢 Then
        └── 🟢 Not (🔴 This label is applied to pull request: review/triage-pending)

Manual Checks

**IGNORE** the bot requirements for this PR (force green CI check)
If
🟢 Condition met
└── 🟢 On every pull request
Can be checked by

Any user with comment edit permission

codecov · 2025-12-22T19:09:01Z

Codecov Report

❌ Patch coverage is 81.25000% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
gnovm/pkg/gnolang/store.go	81.25%	4 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

…e start

jaekwon · 2026-01-08T15:53:32Z

i'll take a look by saturday

jefft0

@Kouteki moved this to In Review and requested a review from core devs in a comment.

thehowl · 2026-01-19T23:36:43Z

gnovm/pkg/gnolang/machine.go

 			// Save BlockNodes to m.Store.
 			SaveBlockNodes(m.Store, fn)
+
+			SaveTypes(m.Store, fn)


This shouldn't be necessary. Types are properly cached:

gno/gnovm/pkg/gnolang/machine.go

Lines 298 to 300 in e254b48

if save {

// store new package values and types

throwaway = m.saveNewPackageValuesAndTypes()

saveNewPackageValuesAndTypes calls SetType, and so the types should be stored.

(I noticed something weird that it may be worth investigating... and that is that only top-level types seem to be saved. If you have time, could you check whether types declared ie. inside of function bodies don't work across restarts especially?)

It seems to me that the cause is that a "cold" run would incur in the gas for GetType, while a "hot" run wouldn't (because it's already in the cache).

The solution is probably to lower the gas cost of GetType, but make it always paid regardless of whether it's cached or not.

This fix seems to work for the wrong reasons (it's actually doing unnecessary work on the database).

This shouldn't be necessary. Types are properly cached:

But this part of the code is never called when the vm start, I am wrong ?

SetCacheType? I'll see if that works instead.

@thehowl: and that is that only top-level types seem to be saved. If you have time, could you check whether types declared ie. inside of function bodies don't work across restarts especially?)

I haven't checked empirically but pretty sure these are derived from preprocess and cached as AST node constTypeExpr so don't need to be in store.Cache. The only weird part I see is func (fv *FuncValue) GetType(store Store) has a case for RefType -> *FuncType but I'm checking to see if that ever happens.

This is because the preprocessor is run for all code and the types are already derived; but it's slower for saving because non-declared types need to be serialized for every TypedValue; and for loading because of the same reason. When we upgrade the chain in the future we can work on this so that the persisted types are appropriately RefTypes even for non-declared types.

I'm going to make a diff that does something slightly different than this PR. It will keep types in store.cacheTypes for efficiency but not just Type but TypeAndSize, and a second map for cacheTypesEphemeral (TypeID -> struct{}) that gets cleared at the end of the tx. Then later when we replace the cacheTypes map with a proper LRU type cache the logic that checks cacheTypesEphemeral is deterministic for every tx regardless of how cacheTypes is implemented, and regardless of restarts.

But this part of the code is never called when the vm start, I am wrong ?

No, but the types are stored in the database, so they can be loaded from there (and that's why this PR has only an impact on gas rather than actual behaviour; because if data is in the store cache, loading it doesn't cost anything).

ltzmaxwell · 2026-01-21T02:50:53Z

does the original issue still exist after #4925?

omarsy · 2026-01-21T21:01:32Z

does the original issue still exist after #4925?

I will say more this pr https://github.com/gnolang/gno/pull/2319/changes#diff-152fa445ed649a7f12eb067c6a67f7c6cb98c3242e23da200744b851aaeb3b20R173. The type cache for non uverse packages not being populated when node start. Consequently, a restarted node will ask more gas than one that hasn't been restarted.

jaekwon · 2026-01-22T17:00:07Z

looking at this now

jaekwon · 2026-01-22T20:02:44Z

@omarsy please check this diff.

diff --git a/gnovm/pkg/gnolang/machine.go b/gnovm/pkg/gnolang/machine.go
index e0676337c..244a6fca8 100644
--- a/gnovm/pkg/gnolang/machine.go
+++ b/gnovm/pkg/gnolang/machine.go
@@ -201,12 +201,10 @@ func (m *Machine) PreprocessAllFilesAndSaveBlockNodes() {
 		m.Store.SetBlockNode(pn)
 		PredefineFileSet(m.Store, pn, fset)
 		for _, fn := range fset.Files {
-			// Save Types to m.Store (while preprocessing).
+			// Save Types to constTypeExpr (while preprocessing).
 			fn = Preprocess(m.Store, pn, fn).(*FileNode)
 			// Save BlockNodes to m.Store.
 			SaveBlockNodes(m.Store, fn)
-
-			SaveTypes(m.Store, fn)
 		}
 		// Normally, the fileset would be added onto the
 		// package node only after runFiles(), but we cannot
diff --git a/gnovm/pkg/gnolang/store.go b/gnovm/pkg/gnolang/store.go
index 6cfa8f442..62fa16dc7 100644
--- a/gnovm/pkg/gnolang/store.go
+++ b/gnovm/pkg/gnolang/store.go
@@ -53,7 +53,7 @@ type Store interface {
 	DelObject(Object) int64 // returns size difference of the object
 	GetType(tid TypeID) Type
 	GetTypeSafe(tid TypeID) Type
-	SetCacheType(Type)
+	SetCacheType(Type) // for builtin types only
 	SetType(Type)
 	GetPackageNode(pkgPath string) *PackageNode
 	GetBlockNode(Location) BlockNode
@@ -134,16 +134,22 @@ func DefaultGasConfig() GasConfig {
 	}
 }
 
+type TypeAndSize struct {
+	Type Type
+	Size int
+}
+
 type defaultStore struct {
 	// underlying stores used to keep data
 	baseStore store.Store // for objects, types, nodes
 	iavlStore store.Store // for escaped object hashes
 
 	// transaction-scoped
-	cacheObjects map[ObjectID]Object            // this is a real cache, reset with every transaction.
-	cacheTypes   txlog.Map[TypeID, Type]        // this re-uses the parent store's.
-	cacheNodes   txlog.Map[Location, BlockNode] // until BlockNode persistence is implemented, this is an actual store.
-	alloc        *Allocator                     // for accounting for cached items
+	cacheObjects  map[ObjectID]Object            // this is a real cache, reset with every transaction.
+	cacheTypes    txlog.Map[TypeID, TypeAndSize] // this re-uses the parent store's.
+	cacheTypesEph map[TypeID]Type                // also gets cleared on ClearObjectCache.
+	cacheNodes    txlog.Map[Location, BlockNode] // until BlockNode persistence is implemented, this is an actual store.
+	alloc         *Allocator                     // for accounting for cached items
 
 	// Partially restored package; occupies memory and tracked for GC,
 	// this is more efficient than iterating over cacheObjects.
@@ -172,9 +178,10 @@ func NewStore(alloc *Allocator, baseStore, iavlStore store.Store) *defaultStore
 		alloc:     alloc,
 
 		// cacheObjects is set; objects in the store will be copied over for any transaction.
-		cacheObjects: make(map[ObjectID]Object),
-		cacheTypes:   txlog.GoMap[TypeID, Type](map[TypeID]Type{}),
-		cacheNodes:   txlog.GoMap[Location, BlockNode](map[Location]BlockNode{}),
+		cacheObjects:  make(map[ObjectID]Object),
+		cacheTypes:    txlog.GoMap[TypeID, TypeAndSize](map[TypeID]TypeAndSize{}),
+		cacheTypesEph: make(map[TypeID]Type),
+		cacheNodes:    txlog.GoMap[Location, BlockNode](map[Location]BlockNode{}),
 
 		// reset at the message level
 		realmStorageDiffs: make(map[string]int64),
@@ -202,10 +209,11 @@ func (ds *defaultStore) BeginTransaction(baseStore, iavlStore store.Store, gasMe
 		iavlStore: iavlStore,
 
 		// transaction-scoped
-		cacheObjects: make(map[ObjectID]Object),
-		cacheTypes:   txlog.Wrap(ds.cacheTypes),
-		cacheNodes:   txlog.Wrap(ds.cacheNodes),
-		alloc:        ds.alloc.Fork().Reset(),
+		cacheObjects:  make(map[ObjectID]Object),
+		cacheTypes:    txlog.Wrap(ds.cacheTypes),
+		cacheTypesEph: make(map[TypeID]Type),
+		cacheNodes:    txlog.Wrap(ds.cacheNodes),
+		alloc:         ds.alloc.Fork().Reset(),
 
 		// store configuration
 		pkgGetter:      ds.pkgGetter,
@@ -231,7 +239,7 @@ type transactionStore struct {
 }
 
 func (t transactionStore) Write() {
-	t.cacheTypes.(txlog.MapCommitter[TypeID, Type]).Commit()
+	t.cacheTypes.(txlog.MapCommitter[TypeID, TypeAndSize]).Commit()
 	t.cacheNodes.(txlog.MapCommitter[Location, BlockNode]).Commit()
 }
 
@@ -764,11 +772,24 @@ func (ds *defaultStore) GetTypeSafe(tid TypeID) Type {
 		defer bm.ResumeOpCode()
 	}
 
-	// check cache.
-	if tt, exists := ds.cacheTypes.Get(tid); exists {
+	// check ephemeral cache.
+	tt, ephExists := ds.cacheTypesEph[tid]
+	if ephExists {
 		return tt
 	}
 
+	// check cache.
+	if tts, exists := ds.cacheTypes.Get(tid); exists {
+		// if not yet it ds.cacheTypesEph, consume gas.
+		if !ephExists {
+			gas := overflow.Mulp(ds.gasConfig.GasGetType, store.Gas(tts.Size))
+			ds.consumeGas(gas, GasGetTypeDesc)
+			ds.cacheTypesEph[tid] = tts.Type
+		}
+		// return type.
+		return tts.Type
+	}
+
 	// check backend.
 	if ds.baseStore != nil {
 		key := backendTypeKey(tid)
@@ -785,7 +806,8 @@ func (ds *defaultStore) GetTypeSafe(tid TypeID) Type {
 				}
 			}
 			// set in cache.
-			ds.cacheTypes.Set(tid, tt)
+			ds.cacheTypes.Set(tid, TypeAndSize{tt, len(bz)})
+			ds.cacheTypesEph[tid] = tt
 			// after setting in cache, fill tt.
 			fillType(ds, tt)
 			return tt
@@ -794,15 +816,20 @@ func (ds *defaultStore) GetTypeSafe(tid TypeID) Type {
 	return nil
 }
 
+// For builtin types only.
 func (ds *defaultStore) SetCacheType(tt Type) {
 	tid := tt.TypeID()
-	if tt2, exists := ds.cacheTypes.Get(tid); exists {
-		if tt != tt2 {
+	// We do not set cacheTypesEph such that txs pay gas for these.
+	// // ds.cacheTypesEph[tid] = struct{}{}
+	if tts2, exists := ds.cacheTypes.Get(tid); exists {
+		if tt != tts2.Type {
 			panic(fmt.Sprintf("cannot re-register %q with different type", tid))
 		}
 		// else, already set.
 	} else {
-		ds.cacheTypes.Set(tid, tt)
+		tcopy := copyTypeWithRefs(tt)
+		bz := amino.MustMarshalAny(tcopy)
+		ds.cacheTypes.Set(tid, TypeAndSize{tt, len(bz)})
 	}
 }
 
@@ -820,9 +847,11 @@ func (ds *defaultStore) SetType(tt Type) {
 		}()
 	}
 	tid := tt.TypeID()
+	// first set ephemeral cache.
+	ds.cacheTypesEph[tid] = tt
 	// return if tid already known.
-	if tt2, exists := ds.cacheTypes.Get(tid); exists {
-		if tt != tt2 {
+	if tts2, exists := ds.cacheTypes.Get(tid); exists {
+		if tt != tts2.Type {
 			// this can happen for a variety of reasons.
 			// TODO classify them and optimize.
 			return
@@ -839,7 +868,7 @@ func (ds *defaultStore) SetType(tt Type) {
 		size = len(bz)
 	}
 	// save type to cache.
-	ds.cacheTypes.Set(tid, tt)
+	ds.cacheTypes.Set(tid, TypeAndSize{tt, size})
 }
 
 // Convenience
@@ -1112,6 +1141,7 @@ func (ds *defaultStore) RealmStorageDiffs() map[string]int64 {
 func (ds *defaultStore) ClearObjectCache() {
 	ds.alloc.Reset()
 	ds.cacheObjects = make(map[ObjectID]Object) // new cache.
+	ds.cacheTypesEph = make(map[TypeID]Type)    // new cache.
 	ds.realmStorageDiffs = make(map[string]int64)
 	ds.opslog = nil // new ops log.
 	ds.SetCachePackage(Uverse())
@@ -1165,9 +1195,9 @@ func (ds *defaultStore) Print() {
 	utils.Print(ds.iavlStore)
 	fmt.Println(colors.Yellow("//----------------------------------------"))
 	fmt.Println(colors.Green("defaultStore:cacheTypes..."))
-	ds.cacheTypes.Iterate()(func(tid TypeID, typ Type) bool {
+	ds.cacheTypes.Iterate()(func(tid TypeID, ts TypeAndSize) bool {
 		fmt.Printf("- %v: %v\n", tid,
-			stringz.TrimN(fmt.Sprintf("%v", typ), 50))
+			stringz.TrimN(fmt.Sprintf("%v", ts.Type), 50))
 		return true
 	})
 	fmt.Println(colors.Yellow("//----------------------------------------"))
diff --git a/gnovm/pkg/gnolang/values.go b/gnovm/pkg/gnolang/values.go
index c328e40a6..2a1343c02 100644
--- a/gnovm/pkg/gnolang/values.go
+++ b/gnovm/pkg/gnolang/values.go
@@ -531,6 +531,9 @@ func (fv *FuncValue) GetType(store Store) *FuncType {
 	case nil:
 		return nil
 	case RefType:
+		// NOTE: This doesn't happen yet:
+		// only *DeclaredTypes become RefTypes.
+		// In the future we may change this.
 		typ := store.GetType(ct.ID).(*FuncType)
 		fv.Type = typ
 		return typ

and new gnovm/tests/files/zrealm_func.gno:

// PKGPATH: gno.land/r/test
package test

var a any

func init() {
	a = func(a int) {
	}
}

func main(cur realm) {
	a = func(b string) {
		println("dontcare")
	}
	println("done")
}

// Output:
// done

// Realm:
// finalizerealm["gno.land/r/test"]
// c[a8ada09dee16d791fd406d629fe29bb0ed084a30:8](309)={
//     "Crossing": false,
//     "FileName": "",
//     "IsClosure": false,
//     "IsMethod": false,
//     "Name": "",
//     "NativeName": "",
//     "NativePkg": "",
//     "ObjectInfo": {
//         "ID": "a8ada09dee16d791fd406d629fe29bb0ed084a30:8",
//         "LastObjectSize": "309",
//         "ModTime": "0",
//         "OwnerID": "a8ada09dee16d791fd406d629fe29bb0ed084a30:3",
//         "RefCount": "1"
//     },
//     "Parent": null,
//     "PkgPath": "gno.land/r/test",
//     "Source": {
//         "@type": "/gno.RefNode",
//         "BlockNode": null,
//         "Location": {
//             "File": "zrealm_func.gno",
//             "PkgPath": "gno.land/r/test",
//             "Span": {
//                 "End": {
//                     "Column": "3",
//                     "Line": "14"
//                 },
//                 "Num": "0",
//                 "Pos": {
//                     "Column": "6",
//                     "Line": "12"
//                 }
//             }
//         }
//     },
//     "Type": {
//         "@type": "/gno.FuncType",
//         "Params": [
//             {
//                 "Embedded": false,
//                 "Name": "b",
//                 "Tag": "",
//                 "Type": {
//                     "@type": "/gno.PrimitiveType",
//                     "value": "16"
//                 }
//             }
//         ],
//         "Results": []
//     }
// }
// u[a8ada09dee16d791fd406d629fe29bb0ed084a30:3](0)=
//     @@ -2,7 +2,7 @@
//          "ObjectInfo": {
//              "ID": "a8ada09dee16d791fd406d629fe29bb0ed084a30:3",
//              "LastObjectSize": "351",
//     -        "ModTime": "6",
//     +        "ModTime": "7",
//              "OwnerID": "a8ada09dee16d791fd406d629fe29bb0ed084a30:2",
//              "RefCount": "1"
//          },
//     @@ -12,11 +12,11 @@
//                  "Params": [
//                      {
//                          "Embedded": false,
//     -                    "Name": "a",
//     +                    "Name": "b",
//                          "Tag": "",
//                          "Type": {
//                              "@type": "/gno.PrimitiveType",
//     -                        "value": "32"
//     +                        "value": "16"
//                          }
//                      }
//                  ],
//     @@ -24,8 +24,8 @@
//              },
//              "V": {
//                  "@type": "/gno.RefValue",
//     -            "Hash": "8806871058538d9ab7d4c168c6788e300f7e72fa",
//     -            "ObjectID": "a8ada09dee16d791fd406d629fe29bb0ed084a30:7"
//     +            "Hash": "7b5d94f791350d9cbeab878b065cdf86f258c507",
//     +            "ObjectID": "a8ada09dee16d791fd406d629fe29bb0ed084a30:8"
//              }
//          }
//      }
// d[a8ada09dee16d791fd406d629fe29bb0ed084a30:7](-309)

And can you please make a txtar for gno.land/pkgs/integration/testdata where, in one file, both from #4983 are tested, and the gas used are checked to be identical? If you cannot figure out a way to do this easily maybe we can keep them in one file with two outputs, with prominent comment in the file that the two gas outputs should be the same.

+              func SaveTypes(store Store, fn *FileNode) {
+              	Transcribe(fn, func(ns []Node, ftype TransField, index int, n Node, stage TransStage) (Node, TransCtrl) {
+              		if stage != TRANS_ENTER {
+              			return n, TRANS_CONTINUE
+              		}
+              		if typeDecl, ok := n.(*TypeDecl); ok {
+              			store.SetType(getType(typeDecl.Type))
+              		}
+              		return n, TRANS_CONTINUE
+              	})
+              }

	if save {
	// store new package values and types
	throwaway = m.saveNewPackageValuesAndTypes()

feat(gnolang): add SaveTypes function to save types in cache when node starts #5008

Are you sure you want to change the base?

feat(gnolang): add SaveTypes function to save types in cache when node starts #5008

Conversation

omarsy commented Dec 22, 2025

Uh oh!

Gno2D2 commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛠 PR Checks Summary

Manual Checks (for Reviewers):

✅ Automated Checks (for Contributors):

☑️ Contributor Actions:

☑️ Reviewer Actions:

📚 Resources:

If

Then

If

Then

If

Can be checked by

Uh oh!

codecov bot commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jaekwon commented Jan 8, 2026

Uh oh!

jefft0 left a comment

Choose a reason for hiding this comment

Uh oh!

thehowl Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

omarsy Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

jaekwon Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

jaekwon Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

thehowl Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

ltzmaxwell commented Jan 21, 2026

Uh oh!

omarsy commented Jan 21, 2026

Uh oh!

jaekwon commented Jan 22, 2026

Uh oh!

jaekwon commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

omarsy commented Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thehowl commented Feb 6, 2026

Uh oh!

ltzmaxwell left a comment

Choose a reason for hiding this comment

Uh oh!

ltzmaxwell Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

thehowl commented Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Gno2D2 commented Dec 22, 2025 •

edited

Loading

codecov bot commented Dec 22, 2025 •

edited

Loading

thehowl Jan 19, 2026 •

edited

Loading

jaekwon commented Jan 22, 2026 •

edited

Loading

omarsy commented Jan 25, 2026 •

edited

Loading