Skip to content

Commit 0b814d3

Browse files
protolambdagballet
protolambda
authored and
gballet
committed
accounts/abi: Abi binding support for nested arrays, fixes #15648, including nested array unpack fix (#15676)
* accounts/abi/bind: support for multi-dim arrays Also: - reduce usage of regexes a bit. - fix minor Java syntax problems Fixes #15648 * accounts/abi/bind: Add some more documentation * accounts/abi/bind: Improve code readability * accounts/abi: bugfix for unpacking nested arrays The code previously assumed the arrays/slices were always 1 level deep. While the packing supports nested arrays (!!!). The current code for unpacking doesn't return the "consumed" length, so this fix had to work around that by calculating it (i.e. packing and getting resulting length) after the unpacking of the array element. It's far from ideal, but unpacking behaviour is fixed now. * accounts/abi: Fix unpacking of nested arrays Removed the temporary workaround of packing to calculate size, which was incorrect for slice-like types anyway. Full size of nested arrays is used now. * accounts/abi: deeply nested array unpack test Test unpacking of an array nested more than one level. * accounts/abi: Add deeply nested array pack test Same as the deep nested array unpack test, but the other way around. * accounts/abi/bind: deeply nested arrays bind test Test the usage of bindings that were generated for methods with multi-dimensional (and not just a single extra dimension, like foo[2][3]) array arguments and returns. edit: trigger rebuild, CI failed to fetch linter module. * accounts/abi/bind: improve array binding wrapArray uses a regex now, and arrayBindingJava is improved. * accounts/abi: Improve naming of element size func The full step size for unpacking an array is now retrieved with "getFullElemSize". * accounts/abi: support nested nested array args Previously, the code only considered the outer-size of the array, ignoring the size of the contents. This was fine for most types, but nested arrays are packed directly into it, and count towards the total size. This resulted in arguments following a nested array to replicate some of the binary contents of the array. The fix: for arrays, calculate their complete contents size: count the arg.Type.Elem.Size when Elem is an Array, and repeat when their child is an array too, etc. The count is the number of 32 byte elements, similar to how it previously counted, but nested. * accounts/abi: Test deep nested arr multi-arguments Arguments with a deeply nested array should not cause the next arguments to be read from the wrong position.
1 parent 7b1d637 commit 0b814d3

File tree

6 files changed

+259
-79
lines changed

6 files changed

+259
-79
lines changed

‎accounts/abi/argument.go‎

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,21 @@ func (arguments Arguments) unpackAtomic(v interface{}, marshalledValues []interf
169169
return set(elem, reflectValue, arguments.NonIndexed()[0])
170170
}
171171

172+
// Computes the full size of an array;
173+
// i.e. counting nested arrays, which count towards size for unpacking.
174+
func getArraySize(arr *Type) int {
175+
size := arr.Size
176+
// Arrays can be nested, with each element being the same size
177+
arr = arr.Elem
178+
for arr.T == ArrayTy {
179+
// Keep multiplying by elem.Size while the elem is an array.
180+
size *= arr.Size
181+
arr = arr.Elem
182+
}
183+
// Now we have the full array size, including its children.
184+
return size
185+
}
186+
172187
// UnpackValues can be used to unpack ABI-encoded hexdata according to the ABI-specification,
173188
// without supplying a struct to unpack into. Instead, this method returns a list containing the
174189
// values. An atomic argument will be a list with one element.
@@ -181,9 +196,14 @@ func (arguments Arguments) UnpackValues(data []byte) ([]interface{}, error) {
181196
// If we have a static array, like [3]uint256, these are coded as
182197
// just like uint256,uint256,uint256.
183198
// This means that we need to add two 'virtual' arguments when
184-
// we count the index from now on
185-
186-
virtualArgs += arg.Type.Size - 1
199+
// we count the index from now on.
200+
//
201+
// Array values nested multiple levels deep are also encoded inline:
202+
// [2][3]uint256: uint256,uint256,uint256,uint256,uint256,uint256
203+
//
204+
// Calculate the full array size to get the correct offset for the next argument.
205+
// Decrement it by 1, as the normal index increment is still applied.
206+
virtualArgs += getArraySize(&arg.Type) - 1
187207
}
188208
if err != nil {
189209
return nil, err

‎accounts/abi/bind/bind.go‎

Lines changed: 99 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -164,118 +164,147 @@ var bindType = map[Lang]func(kind abi.Type) string{
164164
LangJava: bindTypeJava,
165165
}
166166

167+
// Helper function for the binding generators.
168+
// It reads the unmatched characters after the inner type-match,
169+
// (since the inner type is a prefix of the total type declaration),
170+
// looks for valid arrays (possibly a dynamic one) wrapping the inner type,
171+
// and returns the sizes of these arrays.
172+
//
173+
// Returned array sizes are in the same order as solidity signatures; inner array size first.
174+
// Array sizes may also be "", indicating a dynamic array.
175+
func wrapArray(stringKind string, innerLen int, innerMapping string) (string, []string) {
176+
remainder := stringKind[innerLen:]
177+
//find all the sizes
178+
matches := regexp.MustCompile(`\[(\d*)\]`).FindAllStringSubmatch(remainder, -1)
179+
parts := make([]string, 0, len(matches))
180+
for _, match := range matches {
181+
//get group 1 from the regex match
182+
parts = append(parts, match[1])
183+
}
184+
return innerMapping, parts
185+
}
186+
187+
// Translates the array sizes to a Go-lang declaration of a (nested) array of the inner type.
188+
// Simply returns the inner type if arraySizes is empty.
189+
func arrayBindingGo(inner string, arraySizes []string) string {
190+
out := ""
191+
//prepend all array sizes, from outer (end arraySizes) to inner (start arraySizes)
192+
for i := len(arraySizes) - 1; i >= 0; i-- {
193+
out += "[" + arraySizes[i] + "]"
194+
}
195+
out += inner
196+
return out
197+
}
198+
167199
// bindTypeGo converts a Solidity type to a Go one. Since there is no clear mapping
168200
// from all Solidity types to Go ones (e.g. uint17), those that cannot be exactly
169201
// mapped will use an upscaled type (e.g. *big.Int).
170202
func bindTypeGo(kind abi.Type) string {
171203
stringKind := kind.String()
204+
innerLen, innerMapping := bindUnnestedTypeGo(stringKind)
205+
return arrayBindingGo(wrapArray(stringKind, innerLen, innerMapping))
206+
}
207+
208+
// The inner function of bindTypeGo, this finds the inner type of stringKind.
209+
// (Or just the type itself if it is not an array or slice)
210+
// The length of the matched part is returned, with the the translated type.
211+
func bindUnnestedTypeGo(stringKind string) (int, string) {
172212

173213
switch {
174214
case strings.HasPrefix(stringKind, "address"):
175-
parts := regexp.MustCompile(`address(\[[0-9]*\])?`).FindStringSubmatch(stringKind)
176-
if len(parts) != 2 {
177-
return stringKind
178-
}
179-
return fmt.Sprintf("%scommon.Address", parts[1])
215+
return len("address"), "common.Address"
180216

181217
case strings.HasPrefix(stringKind, "bytes"):
182-
parts := regexp.MustCompile(`bytes([0-9]*)(\[[0-9]*\])?`).FindStringSubmatch(stringKind)
183-
if len(parts) != 3 {
184-
return stringKind
185-
}
186-
return fmt.Sprintf("%s[%s]byte", parts[2], parts[1])
218+
parts := regexp.MustCompile(`bytes([0-9]*)`).FindStringSubmatch(stringKind)
219+
return len(parts[0]), fmt.Sprintf("[%s]byte", parts[1])
187220

188221
case strings.HasPrefix(stringKind, "int") || strings.HasPrefix(stringKind, "uint"):
189-
parts := regexp.MustCompile(`(u)?int([0-9]*)(\[[0-9]*\])?`).FindStringSubmatch(stringKind)
190-
if len(parts) != 4 {
191-
return stringKind
192-
}
222+
parts := regexp.MustCompile(`(u)?int([0-9]*)`).FindStringSubmatch(stringKind)
193223
switch parts[2] {
194224
case "8", "16", "32", "64":
195-
return fmt.Sprintf("%s%sint%s", parts[3], parts[1], parts[2])
225+
return len(parts[0]), fmt.Sprintf("%sint%s", parts[1], parts[2])
196226
}
197-
return fmt.Sprintf("%s*big.Int", parts[3])
227+
return len(parts[0]), "*big.Int"
198228

199-
case strings.HasPrefix(stringKind, "bool") || strings.HasPrefix(stringKind, "string"):
200-
parts := regexp.MustCompile(`([a-z]+)(\[[0-9]*\])?`).FindStringSubmatch(stringKind)
201-
if len(parts) != 3 {
202-
return stringKind
203-
}
204-
return fmt.Sprintf("%s%s", parts[2], parts[1])
229+
case strings.HasPrefix(stringKind, "bool"):
230+
return len("bool"), "bool"
231+
232+
case strings.HasPrefix(stringKind, "string"):
233+
return len("string"), "string"
205234

206235
default:
207-
return stringKind
236+
return len(stringKind), stringKind
208237
}
209238
}
210239

240+
// Translates the array sizes to a Java declaration of a (nested) array of the inner type.
241+
// Simply returns the inner type if arraySizes is empty.
242+
func arrayBindingJava(inner string, arraySizes []string) string {
243+
// Java array type declarations do not include the length.
244+
return inner + strings.Repeat("[]", len(arraySizes))
245+
}
246+
211247
// bindTypeJava converts a Solidity type to a Java one. Since there is no clear mapping
212248
// from all Solidity types to Java ones (e.g. uint17), those that cannot be exactly
213249
// mapped will use an upscaled type (e.g. BigDecimal).
214250
func bindTypeJava(kind abi.Type) string {
215251
stringKind := kind.String()
252+
innerLen, innerMapping := bindUnnestedTypeJava(stringKind)
253+
return arrayBindingJava(wrapArray(stringKind, innerLen, innerMapping))
254+
}
255+
256+
// The inner function of bindTypeJava, this finds the inner type of stringKind.
257+
// (Or just the type itself if it is not an array or slice)
258+
// The length of the matched part is returned, with the the translated type.
259+
func bindUnnestedTypeJava(stringKind string) (int, string) {
216260

217261
switch {
218262
case strings.HasPrefix(stringKind, "address"):
219263
parts := regexp.MustCompile(`address(\[[0-9]*\])?`).FindStringSubmatch(stringKind)
220264
if len(parts) != 2 {
221-
return stringKind
265+
return len(stringKind), stringKind
222266
}
223267
if parts[1] == "" {
224-
return fmt.Sprintf("Address")
268+
return len("address"), "Address"
225269
}
226-
return fmt.Sprintf("Addresses")
270+
return len(parts[0]), "Addresses"
227271

228272
case strings.HasPrefix(stringKind, "bytes"):
229-
parts := regexp.MustCompile(`bytes([0-9]*)(\[[0-9]*\])?`).FindStringSubmatch(stringKind)
230-
if len(parts) != 3 {
231-
return stringKind
232-
}
233-
if parts[2] != "" {
234-
return "byte[][]"
273+
parts := regexp.MustCompile(`bytes([0-9]*)`).FindStringSubmatch(stringKind)
274+
if len(parts) != 2 {
275+
return len(stringKind), stringKind
235276
}
236-
return "byte[]"
277+
return len(parts[0]), "byte[]"
237278

238279
case strings.HasPrefix(stringKind, "int") || strings.HasPrefix(stringKind, "uint"):
239-
parts := regexp.MustCompile(`(u)?int([0-9]*)(\[[0-9]*\])?`).FindStringSubmatch(stringKind)
240-
if len(parts) != 4 {
241-
return stringKind
242-
}
243-
switch parts[2] {
244-
case "8", "16", "32", "64":
245-
if parts[1] == "" {
246-
if parts[3] == "" {
247-
return fmt.Sprintf("int%s", parts[2])
248-
}
249-
return fmt.Sprintf("int%s[]", parts[2])
250-
}
280+
//Note that uint and int (without digits) are also matched,
281+
// these are size 256, and will translate to BigInt (the default).
282+
parts := regexp.MustCompile(`(u)?int([0-9]*)`).FindStringSubmatch(stringKind)
283+
if len(parts) != 3 {
284+
return len(stringKind), stringKind
251285
}
252-
if parts[3] == "" {
253-
return fmt.Sprintf("BigInt")
286+
287+
namedSize := map[string]string{
288+
"8": "byte",
289+
"16": "short",
290+
"32": "int",
291+
"64": "long",
292+
}[parts[2]]
293+
294+
//default to BigInt
295+
if namedSize == "" {
296+
namedSize = "BigInt"
254297
}
255-
return fmt.Sprintf("BigInts")
298+
return len(parts[0]), namedSize
256299

257300
case strings.HasPrefix(stringKind, "bool"):
258-
parts := regexp.MustCompile(`bool(\[[0-9]*\])?`).FindStringSubmatch(stringKind)
259-
if len(parts) != 2 {
260-
return stringKind
261-
}
262-
if parts[1] == "" {
263-
return fmt.Sprintf("bool")
264-
}
265-
return fmt.Sprintf("bool[]")
301+
return len("bool"), "boolean"
266302

267303
case strings.HasPrefix(stringKind, "string"):
268-
parts := regexp.MustCompile(`string(\[[0-9]*\])?`).FindStringSubmatch(stringKind)
269-
if len(parts) != 2 {
270-
return stringKind
271-
}
272-
if parts[1] == "" {
273-
return fmt.Sprintf("String")
274-
}
275-
return fmt.Sprintf("String[]")
304+
return len("string"), "String"
276305

277306
default:
278-
return stringKind
307+
return len(stringKind), stringKind
279308
}
280309
}
281310

@@ -325,11 +354,13 @@ func namedTypeJava(javaKind string, solKind abi.Type) string {
325354
return "String"
326355
case "string[]":
327356
return "Strings"
328-
case "bool":
357+
case "boolean":
329358
return "Bool"
330-
case "bool[]":
359+
case "boolean[]":
331360
return "Bools"
332-
case "BigInt":
361+
case "BigInt[]":
362+
return "BigInts"
363+
default:
333364
parts := regexp.MustCompile(`(u)?int([0-9]*)(\[[0-9]*\])?`).FindStringSubmatch(solKind.String())
334365
if len(parts) != 4 {
335366
return javaKind
@@ -344,8 +375,6 @@ func namedTypeJava(javaKind string, solKind abi.Type) string {
344375
default:
345376
return javaKind
346377
}
347-
default:
348-
return javaKind
349378
}
350379
}
351380

‎accounts/abi/bind/bind_test.go‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -737,6 +737,72 @@ var bindTests = []struct {
737737
}
738738
`,
739739
},
740+
{
741+
`DeeplyNestedArray`,
742+
`
743+
contract DeeplyNestedArray {
744+
uint64[3][4][5] public deepUint64Array;
745+
function storeDeepUintArray(uint64[3][4][5] arr) public {
746+
deepUint64Array = arr;
747+
}
748+
function retrieveDeepArray() public view returns (uint64[3][4][5]) {
749+
return deepUint64Array;
750+
}
751+
}
752+
`,
753+
`6060604052341561000f57600080fd5b6106438061001e6000396000f300606060405260043610610057576000357c0100000000000000000000000000000000000000000000000000000000900463ffffffff168063344248551461005c5780638ed4573a1461011457806398ed1856146101ab575b600080fd5b341561006757600080fd5b610112600480806107800190600580602002604051908101604052809291906000905b828210156101055783826101800201600480602002604051908101604052809291906000905b828210156100f25783826060020160038060200260405190810160405280929190826003602002808284378201915050505050815260200190600101906100b0565b505050508152602001906001019061008a565b5050505091905050610208565b005b341561011f57600080fd5b61012761021d565b604051808260056000925b8184101561019b578284602002015160046000925b8184101561018d5782846020020151600360200280838360005b8381101561017c578082015181840152602081019050610161565b505050509050019260010192610147565b925050509260010192610132565b9250505091505060405180910390f35b34156101b657600080fd5b6101de6004808035906020019091908035906020019091908035906020019091905050610309565b604051808267ffffffffffffffff1667ffffffffffffffff16815260200191505060405180910390f35b80600090600561021992919061035f565b5050565b6102256103b0565b6000600580602002604051908101604052809291906000905b8282101561030057838260040201600480602002604051908101604052809291906000905b828210156102ed578382016003806020026040519081016040528092919082600380156102d9576020028201916000905b82829054906101000a900467ffffffffffffffff1667ffffffffffffffff16815260200190600801906020826007010492830192600103820291508084116102945790505b505050505081526020019060010190610263565b505050508152602001906001019061023e565b50505050905090565b60008360058110151561031857fe5b600402018260048110151561032957fe5b018160038110151561033757fe5b6004918282040191900660080292509250509054906101000a900467ffffffffffffffff1681565b826005600402810192821561039f579160200282015b8281111561039e5782518290600461038e9291906103df565b5091602001919060040190610375565b5b5090506103ac919061042d565b5090565b610780604051908101604052806005905b6103c9610459565b8152602001906001900390816103c15790505090565b826004810192821561041c579160200282015b8281111561041b5782518290600361040b929190610488565b50916020019190600101906103f2565b5b5090506104299190610536565b5090565b61045691905b8082111561045257600081816104499190610562565b50600401610433565b5090565b90565b610180604051908101604052806004905b6104726105a7565b81526020019060019003908161046a5790505090565b82600380016004900481019282156105255791602002820160005b838211156104ef57835183826101000a81548167ffffffffffffffff021916908367ffffffffffffffff16021790555092602001926008016020816007010492830192600103026104a3565b80156105235782816101000a81549067ffffffffffffffff02191690556008016020816007010492830192600103026104ef565b505b50905061053291906105d9565b5090565b61055f91905b8082111561055b57600081816105529190610610565b5060010161053c565b5090565b90565b50600081816105719190610610565b50600101600081816105839190610610565b50600101600081816105959190610610565b5060010160006105a59190610610565b565b6060604051908101604052806003905b600067ffffffffffffffff168152602001906001900390816105b75790505090565b61060d91905b8082111561060957600081816101000a81549067ffffffffffffffff0219169055506001016105df565b5090565b90565b50600090555600a165627a7a7230582087e5a43f6965ab6ef7a4ff056ab80ed78fd8c15cff57715a1bf34ec76a93661c0029`,
754+
`[{"constant":false,"inputs":[{"name":"arr","type":"uint64[3][4][5]"}],"name":"storeDeepUintArray","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[],"name":"retrieveDeepArray","outputs":[{"name":"","type":"uint64[3][4][5]"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[{"name":"","type":"uint256"},{"name":"","type":"uint256"},{"name":"","type":"uint256"}],"name":"deepUint64Array","outputs":[{"name":"","type":"uint64"}],"payable":false,"stateMutability":"view","type":"function"}]`,
755+
`
756+
// Generate a new random account and a funded simulator
757+
key, _ := crypto.GenerateKey()
758+
auth := bind.NewKeyedTransactor(key)
759+
sim := backends.NewSimulatedBackend(core.GenesisAlloc{auth.From: {Balance: big.NewInt(10000000000)}})
760+
761+
//deploy the test contract
762+
_, _, testContract, err := DeployDeeplyNestedArray(auth, sim)
763+
if err != nil {
764+
t.Fatalf("Failed to deploy test contract: %v", err)
765+
}
766+
767+
// Finish deploy.
768+
sim.Commit()
769+
770+
//Create coordinate-filled array, for testing purposes.
771+
testArr := [5][4][3]uint64{}
772+
for i := 0; i < 5; i++ {
773+
testArr[i] = [4][3]uint64{}
774+
for j := 0; j < 4; j++ {
775+
testArr[i][j] = [3]uint64{}
776+
for k := 0; k < 3; k++ {
777+
//pack the coordinates, each array value will be unique, and can be validated easily.
778+
testArr[i][j][k] = uint64(i) << 16 | uint64(j) << 8 | uint64(k)
779+
}
780+
}
781+
}
782+
783+
if _, err := testContract.StoreDeepUintArray(&bind.TransactOpts{
784+
From: auth.From,
785+
Signer: auth.Signer,
786+
}, testArr); err != nil {
787+
t.Fatalf("Failed to store nested array in test contract: %v", err)
788+
}
789+
790+
sim.Commit()
791+
792+
retrievedArr, err := testContract.RetrieveDeepArray(&bind.CallOpts{
793+
From: auth.From,
794+
Pending: false,
795+
})
796+
if err != nil {
797+
t.Fatalf("Failed to retrieve nested array from test contract: %v", err)
798+
}
799+
800+
//quick check to see if contents were copied
801+
// (See accounts/abi/unpack_test.go for more extensive testing)
802+
if retrievedArr[4][3][2] != testArr[4][3][2] {
803+
t.Fatalf("Retrieved value does not match expected value! got: %d, expected: %d. %v", retrievedArr[4][3][2], testArr[4][3][2], err)
804+
}`,
805+
},
740806
}
741807

742808
// Tests that packages generated by the binder can be successfully compiled and

‎accounts/abi/pack_test.go‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -299,6 +299,11 @@ func TestPack(t *testing.T) {
299299
[32]byte{1},
300300
common.Hex2Bytes("0100000000000000000000000000000000000000000000000000000000000000"),
301301
},
302+
{
303+
"uint32[2][3][4]",
304+
[4][3][2]uint32{{{1, 2}, {3, 4}, {5, 6}}, {{7, 8}, {9, 10}, {11, 12}}, {{13, 14}, {15, 16}, {17, 18}}, {{19, 20}, {21, 22}, {23, 24}}},
305+
common.Hex2Bytes("000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000003000000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000050000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000700000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000009000000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000b000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000d000000000000000000000000000000000000000000000000000000000000000e000000000000000000000000000000000000000000000000000000000000000f000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000110000000000000000000000000000000000000000000000000000000000000012000000000000000000000000000000000000000000000000000000000000001300000000000000000000000000000000000000000000000000000000000000140000000000000000000000000000000000000000000000000000000000000015000000000000000000000000000000000000000000000000000000000000001600000000000000000000000000000000000000000000000000000000000000170000000000000000000000000000000000000000000000000000000000000018"),
306+
},
302307
{
303308
"address[]",
304309
[]common.Address{{1}, {2}},

0 commit comments

Comments
 (0)