What is the preferrable count for batch write ledger?

[bh4rtp2023]

I am testing batch write ledger using StartWriteBatch/FinishWriteBatch. The batch data count is tested with 100 and 50. When count is 100, write data can fail with errors reporting deadline context. While it is ok for 50.
So what is the consideration for the selection of batch data count?

[pfi79]

Check what is the reason for your deadline context?

[bh4rtp2023]

I do the test using10 workers, every one batch writes 100 data, 7000 in toal. There are two kinds of error:

1)
Error: endorse error for transaction 2eb9b6e4531421b0cf6a637a9bbf86299be0cc34e5221ccabac01096d2e23cd1 with gRPC status Aborted: rpc error: code = Aborted desc = failed to collect enough transaction endorsements, see attached details for more info
Error details:
- address: market:13051, mspId: MarketOrg, message: ProposalResponsePayloads do not match
- address: market:12051, mspId: MarketOrg, message: ProposalResponsePayloads do not match

2)
Error: endorse error for transaction 404f4970d400c420e72cba121474318065530b48239430389b9ac126f2b1d314 with gRPC status DeadlineExceeded: rpc error: code = DeadlineExceeded desc = context deadline exceeded

[pfi79]

I'll answer again: you have a problem with the client and the configured timeouts.

If you disable batching, it will be the same, but earlier.

[bh4rtp2023]

@pfi79 Here is part of log content:

2025-04-11 10:48:38.737 CST 0502 ERRO [grpc] Errorf -> [transport] Client received GoAway with error code ENHANCE_YOUR_CALM and debug data equal to ASCII "too_many_pings".
2025-04-11 10:48:38.737 CST 0504 INFO [grpc] Infof -> [transport] [client-transport 0xc000161b08] loopyWriter exiting with error: received GOAWAY with no active streams
2025-04-11 10:48:38.738 CST 0505 INFO [grpc] Infof -> [transport] [client-transport 0xc000161b08] Closing: connection error: desc = "error reading from server: read tcp 172.19.0.11:45382->192.168.122.110:12051: use of closed network connection"
2025-04-11 10:48:38.739 CST 0506 INFO [grpc] Infof -> [pick-first-lb] [pick-first-lb 0xc0067c83c0] Received SubConn state update: 0xc0042f5540, {ConnectivityState:IDLE ConnectionError:<nil> connectedAddress:{Addr: ServerName: Attributes:<nil> BalancerAttributes:<nil> Metadata:<nil>}}
2025-04-11 10:48:38.740 CST 0507 INFO [grpc] AddTraceEvent -> [core] [Channel #112]Channel Connectivity change to IDLE
2025-04-11 10:48:39.077 CST 0508 INFO [endorser] callChaincode -> finished chaincode: mycc duration: 5511ms channel=mych txID=4d4be992
2025-04-11 10:48:39.768 CST 0509 INFO [kvledger] commit -> [mych] Committed block [19] with 1 transaction(s) in 6195ms (state_validation=622ms block_and_pvtdata_commit=34ms state_commit=5531ms) commitHash=[b5a0e870b13b431933cb6135b9fd4097f335a281cf4c6269fbc287403855b6b1]
2025-04-11 10:48:39.768 CST 050a INFO [gossip.privdata] StoreBlock -> Received block [20] from buffer channel=mych
2025-04-11 10:48:39.771 CST 050b INFO [comm.grpc.server] func1 -> unary call completed grpc.service=gateway.Gateway grpc.method=CommitStatus grpc.request_deadline=2025-04-11T10:49:20.404+08:00 grpc.peer_address=192.168.122.109:60322 grpc.code=OK grpc.call_duration=19.367145497s
2025-04-11 10:48:39.937 CST 050c INFO [committer.txvalidator] Validate -> [mych] Validated block [20] in 168ms
2025-04-11 10:48:39.940 CST 050d INFO [gossip.privdata] RetrievePvtdata -> Successfully fetched (or marked to reconcile later) all 3 eligible collection private write sets for block [20] (0 from local cache, 3 from transient store, 0 from other peers) channel=mych
2025-04-11 10:48:39.962 CST 0519 INFO [gateway] func1 -> Successful response from orderer txID=4d4be9922ff4c28b2d4f9699fd884d9da7f1ba7921f9dd04f5788961b1c78c36 endpoint=ctrlcenter:10050
2025-04-11 10:48:43.058 CST 051a WARN [gateway] Endorse -> Endorse call timed out while collecting endorsements channel=mych chaincode=mycc txID=b1a5da9b2dbdb83c82db39bfdb6ddcf8bd08316589df4881d9427a5dcac51e4a numEndorsers=1
2025-04-11 10:48:43.058 CST 051b INFO [comm.grpc.server] func1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.request_deadline=2025-04-11T10:48:43.055+08:00 grpc.peer_address=192.168.122.109:60322 error="rpc error: code = DeadlineExceeded desc = endorsement timeout expired while collecting endorsements" grpc.code=DeadlineExceeded grpc.call_duration=15.002890327s
2025-04-11 10:48:44.160 CST 051c INFO [endorser] callChaincode -> finished chaincode: mycc duration: 4351ms channel=mych txID=a1631f51
2025-04-11 10:48:44.335 CST 051d INFO [comm.grpc.server] func1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.request_deadline=2025-04-11T10:48:46.006+08:00 grpc.peer_address=192.168.122.109:60322 grpc.code=OK grpc.call_duration=13.329189263s

Do you think it is caused by batching write data timeout? It also occurs for count 50.

[bh4rtp2023]

@pfi79 What is the reason for Errorf -> [transport] Client received GoAway with error code ENHANCE_YOUR_CALM and debug data equal to ASCII "too_many_pings". I never find this error before.

[pfi79]

I've seen all these errors before. They are not related to batching. They all have to do with your clients. Deal with them. For experiment, remove the write by batches and repeat your actions.