Skip to content

Commit f4e5d70

Browse files
author
MarcoFalke
committed
Merge bitcoin/bitcoin#24118: Add 'sendall' RPC née sweep
bb84b71 add tests for no recipient and using send_max while inputs are specified (ishaanam) 49090ec Add sendall RPC née sweep (Murch) 902793c Extract FinishTransaction from send() (Murch) 6d2208a Extract interpretation of fee estimation arguments (Murch) a31d75e Elaborate error messages for outdated options (Murch) 35ed094 Extract prevention of outdated option names (Murch) Pull request description: Add sendall RPC née sweep _Motivation_ Currently, the wallet uses a fSubtractFeeAmount (SFFO) flag on the recipients objects for all forms of sending calls. According to the commit discussion, this flag was chiefly introduced to permit sweeping without manually calculating the fees of transactions. However, the flag leads to unintuitive behavior and makes it more complicated to test many wallet RPCs exhaustively. We proposed to introduce a dedicated `sendall` RPC with the intention to cover this functionality. Since the proposal, it was discovered in further discussion that our proposed `sendall` rpc and SFFO have subtly different scopes of operation. • sendall: Use _given UTXOs_ to pay a destination the remainder after fees. • SFFO: Use a _given budget_ to pay an address the remainder after fees. While `sendall` will simplify cases of spending a given set of UTXOs such as paying the value from one or more specific UTXOs, emptying a wallet, or burning dust, we realized that there are some cases in which SFFO is used to pay other parties from a limited budget, which can often lead to the creation of change outputs. This cannot be easily replicated using `sendall` as it would require manual computation of the appropriate change amount. As such, sendall cannot replace all uses of SFFO, but it still has a different use case and will aid in simplifying some wallet calls and numerous wallet tests. _Sendall call details_ The proposed sendall call builds a transaction from a specific subset of the wallet's UTXO pool (by default all of them) and assigns the funds to one or more receivers. Receivers can either be specified with a given amount or receive an equal share of the remaining unassigned funds. At least one recipient must be provided without assigned amount to collect the remainder. The `sendall` call will never create change. The call has a `send_max` option that changes the default behavior of spending all UTXOs ("no UTXO left behind"), to maximizing the output amount of the transaction by skipping uneconomic UTXOs. The `send_max` option is incompatible with providing a specific set of inputs. --- Edit: Replaced OP with latest commit message to reflect my updated motivation of the proposal. ACKs for top commit: achow101: re-ACK bb84b71 Tree-SHA512: 20aaf75d268cb4b144f5d6437d33ec7b5f989256b3daeeb768ae1e7f39dc6b962af8223c5cb42ecc72dc38cecd921c53c077bc0ec300b994e902412213dd2cc3
2 parents 171f6f2 + bb84b71 commit f4e5d70

File tree

7 files changed

+681
-92
lines changed

7 files changed

+681
-92
lines changed

doc/release-notes-24118.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
New RPCs
2+
--------
3+
4+
- The `sendall` RPC spends specific UTXOs to one or more recipients
5+
without creating change. By default, the `sendall` RPC will spend
6+
every UTXO in the wallet. `sendall` is useful to empty wallets or to
7+
create a changeless payment from select UTXOs. When creating a payment
8+
from a specific amount for which the recipient incurs the transaction
9+
fee, continue to use the `subtractfeefromamount` option via the
10+
`send`, `sendtoaddress`, or `sendmany` RPCs. (#24118)

src/rpc/client.cpp

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,10 @@ static const CRPCConvertParam vRPCConvertParams[] =
142142
{ "send", 1, "conf_target" },
143143
{ "send", 3, "fee_rate"},
144144
{ "send", 4, "options" },
145+
{ "sendall", 0, "recipients" },
146+
{ "sendall", 1, "conf_target" },
147+
{ "sendall", 3, "fee_rate"},
148+
{ "sendall", 4, "options" },
145149
{ "importprivkey", 2, "rescan" },
146150
{ "importaddress", 2, "rescan" },
147151
{ "importaddress", 3, "p2sh" },

src/wallet/rpc/spend.cpp

Lines changed: 341 additions & 92 deletions
Large diffs are not rendered by default.

src/wallet/rpc/wallet.cpp

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -644,6 +644,7 @@ RPCHelpMan fundrawtransaction();
644644
RPCHelpMan bumpfee();
645645
RPCHelpMan psbtbumpfee();
646646
RPCHelpMan send();
647+
RPCHelpMan sendall();
647648
RPCHelpMan walletprocesspsbt();
648649
RPCHelpMan walletcreatefundedpsbt();
649650
RPCHelpMan signrawtransactionwithwallet();
@@ -723,6 +724,7 @@ static const CRPCCommand commands[] =
723724
{ "wallet", &setwalletflag, },
724725
{ "wallet", &signmessage, },
725726
{ "wallet", &signrawtransactionwithwallet, },
727+
{ "wallet", &sendall, },
726728
{ "wallet", &unloadwallet, },
727729
{ "wallet", &upgradewallet, },
728730
{ "wallet", &walletcreatefundedpsbt, },

test/functional/test_runner.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -277,6 +277,8 @@
277277
'wallet_create_tx.py --legacy-wallet',
278278
'wallet_send.py --legacy-wallet',
279279
'wallet_send.py --descriptors',
280+
'wallet_sendall.py --legacy-wallet',
281+
'wallet_sendall.py --descriptors',
280282
'wallet_create_tx.py --descriptors',
281283
'wallet_taproot.py',
282284
'wallet_inactive_hdchains.py',

test/functional/wallet_sendall.py

Lines changed: 316 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,316 @@
1+
#!/usr/bin/env python3
2+
# Copyright (c) 2022 The Bitcoin Core developers
3+
# Distributed under the MIT software license, see the accompanying
4+
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
5+
"""Test the sendall RPC command."""
6+
7+
from decimal import Decimal, getcontext
8+
9+
from test_framework.test_framework import BitcoinTestFramework
10+
from test_framework.util import (
11+
assert_equal,
12+
assert_greater_than,
13+
assert_raises_rpc_error,
14+
)
15+
16+
# Decorator to reset activewallet to zero utxos
17+
def cleanup(func):
18+
def wrapper(self):
19+
try:
20+
func(self)
21+
finally:
22+
if 0 < self.wallet.getbalances()["mine"]["trusted"]:
23+
self.wallet.sendall([self.remainder_target])
24+
assert_equal(0, self.wallet.getbalances()["mine"]["trusted"]) # wallet is empty
25+
return wrapper
26+
27+
class SendallTest(BitcoinTestFramework):
28+
# Setup and helpers
29+
def skip_test_if_missing_module(self):
30+
self.skip_if_no_wallet()
31+
32+
def set_test_params(self):
33+
getcontext().prec=10
34+
self.num_nodes = 1
35+
self.setup_clean_chain = True
36+
37+
def assert_balance_swept_completely(self, tx, balance):
38+
output_sum = sum([o["value"] for o in tx["decoded"]["vout"]])
39+
assert_equal(output_sum, balance + tx["fee"])
40+
assert_equal(0, self.wallet.getbalances()["mine"]["trusted"]) # wallet is empty
41+
42+
def assert_tx_has_output(self, tx, addr, value=None):
43+
for output in tx["decoded"]["vout"]:
44+
if addr == output["scriptPubKey"]["address"] and value is None or value == output["value"]:
45+
return
46+
raise AssertionError("Output to {} not present or wrong amount".format(addr))
47+
48+
def assert_tx_has_outputs(self, tx, expected_outputs):
49+
assert_equal(len(expected_outputs), len(tx["decoded"]["vout"]))
50+
for eo in expected_outputs:
51+
self.assert_tx_has_output(tx, eo["address"], eo["value"])
52+
53+
def add_utxos(self, amounts):
54+
for a in amounts:
55+
self.def_wallet.sendtoaddress(self.wallet.getnewaddress(), a)
56+
self.generate(self.nodes[0], 1)
57+
assert_greater_than(self.wallet.getbalances()["mine"]["trusted"], 0)
58+
return self.wallet.getbalances()["mine"]["trusted"]
59+
60+
# Helper schema for success cases
61+
def test_sendall_success(self, sendall_args, remaining_balance = 0):
62+
sendall_tx_receipt = self.wallet.sendall(sendall_args)
63+
self.generate(self.nodes[0], 1)
64+
# wallet has remaining balance (usually empty)
65+
assert_equal(remaining_balance, self.wallet.getbalances()["mine"]["trusted"])
66+
67+
assert_equal(sendall_tx_receipt["complete"], True)
68+
return self.wallet.gettransaction(txid = sendall_tx_receipt["txid"], verbose = True)
69+
70+
@cleanup
71+
def gen_and_clean(self):
72+
self.add_utxos([15, 2, 4])
73+
74+
def test_cleanup(self):
75+
self.log.info("Test that cleanup wrapper empties wallet")
76+
self.gen_and_clean()
77+
assert_equal(0, self.wallet.getbalances()["mine"]["trusted"]) # wallet is empty
78+
79+
# Actual tests
80+
@cleanup
81+
def sendall_two_utxos(self):
82+
self.log.info("Testing basic sendall case without specific amounts")
83+
pre_sendall_balance = self.add_utxos([10,11])
84+
tx_from_wallet = self.test_sendall_success(sendall_args = [self.remainder_target])
85+
86+
self.assert_tx_has_outputs(tx = tx_from_wallet,
87+
expected_outputs = [
88+
{ "address": self.remainder_target, "value": pre_sendall_balance + tx_from_wallet["fee"] } # fee is neg
89+
]
90+
)
91+
self.assert_balance_swept_completely(tx_from_wallet, pre_sendall_balance)
92+
93+
@cleanup
94+
def sendall_split(self):
95+
self.log.info("Testing sendall where two recipients have unspecified amount")
96+
pre_sendall_balance = self.add_utxos([1, 2, 3, 15])
97+
tx_from_wallet = self.test_sendall_success([self.remainder_target, self.split_target])
98+
99+
half = (pre_sendall_balance + tx_from_wallet["fee"]) / 2
100+
self.assert_tx_has_outputs(tx_from_wallet,
101+
expected_outputs = [
102+
{ "address": self.split_target, "value": half },
103+
{ "address": self.remainder_target, "value": half }
104+
]
105+
)
106+
self.assert_balance_swept_completely(tx_from_wallet, pre_sendall_balance)
107+
108+
@cleanup
109+
def sendall_and_spend(self):
110+
self.log.info("Testing sendall in combination with paying specified amount to recipient")
111+
pre_sendall_balance = self.add_utxos([8, 13])
112+
tx_from_wallet = self.test_sendall_success([{self.recipient: 5}, self.remainder_target])
113+
114+
self.assert_tx_has_outputs(tx_from_wallet,
115+
expected_outputs = [
116+
{ "address": self.recipient, "value": 5 },
117+
{ "address": self.remainder_target, "value": pre_sendall_balance - 5 + tx_from_wallet["fee"] }
118+
]
119+
)
120+
self.assert_balance_swept_completely(tx_from_wallet, pre_sendall_balance)
121+
122+
@cleanup
123+
def sendall_invalid_recipient_addresses(self):
124+
self.log.info("Test having only recipient with specified amount, missing recipient with unspecified amount")
125+
self.add_utxos([12, 9])
126+
127+
assert_raises_rpc_error(
128+
-8,
129+
"Must provide at least one address without a specified amount" ,
130+
self.wallet.sendall,
131+
[{self.recipient: 5}]
132+
)
133+
134+
@cleanup
135+
def sendall_duplicate_recipient(self):
136+
self.log.info("Test duplicate destination")
137+
self.add_utxos([1, 8, 3, 9])
138+
139+
assert_raises_rpc_error(
140+
-8,
141+
"Invalid parameter, duplicated address: {}".format(self.remainder_target),
142+
self.wallet.sendall,
143+
[self.remainder_target, self.remainder_target]
144+
)
145+
146+
@cleanup
147+
def sendall_invalid_amounts(self):
148+
self.log.info("Test sending more than balance")
149+
pre_sendall_balance = self.add_utxos([7, 14])
150+
151+
expected_tx = self.wallet.sendall(recipients=[{self.recipient: 5}, self.remainder_target], options={"add_to_wallet": False})
152+
tx = self.wallet.decoderawtransaction(expected_tx['hex'])
153+
fee = 21 - sum([o["value"] for o in tx["vout"]])
154+
155+
assert_raises_rpc_error(-6, "Assigned more value to outputs than available funds.", self.wallet.sendall,
156+
[{self.recipient: pre_sendall_balance + 1}, self.remainder_target])
157+
assert_raises_rpc_error(-6, "Insufficient funds for fees after creating specified outputs.", self.wallet.sendall,
158+
[{self.recipient: pre_sendall_balance}, self.remainder_target])
159+
assert_raises_rpc_error(-8, "Specified output amount to {} is below dust threshold".format(self.recipient),
160+
self.wallet.sendall, [{self.recipient: 0.00000001}, self.remainder_target])
161+
assert_raises_rpc_error(-6, "Dynamically assigned remainder results in dust output.", self.wallet.sendall,
162+
[{self.recipient: pre_sendall_balance - fee}, self.remainder_target])
163+
assert_raises_rpc_error(-6, "Dynamically assigned remainder results in dust output.", self.wallet.sendall,
164+
[{self.recipient: pre_sendall_balance - fee - Decimal(0.00000010)}, self.remainder_target])
165+
166+
# @cleanup not needed because different wallet used
167+
def sendall_negative_effective_value(self):
168+
self.log.info("Test that sendall fails if all UTXOs have negative effective value")
169+
# Use dedicated wallet for dust amounts and unload wallet at end
170+
self.nodes[0].createwallet("dustwallet")
171+
dust_wallet = self.nodes[0].get_wallet_rpc("dustwallet")
172+
173+
self.def_wallet.sendtoaddress(dust_wallet.getnewaddress(), 0.00000400)
174+
self.def_wallet.sendtoaddress(dust_wallet.getnewaddress(), 0.00000300)
175+
self.generate(self.nodes[0], 1)
176+
assert_greater_than(dust_wallet.getbalances()["mine"]["trusted"], 0)
177+
178+
assert_raises_rpc_error(-6, "Total value of UTXO pool too low to pay for transaction."
179+
+ " Try using lower feerate or excluding uneconomic UTXOs with 'send_max' option.",
180+
dust_wallet.sendall, recipients=[self.remainder_target], fee_rate=300)
181+
182+
dust_wallet.unloadwallet()
183+
184+
@cleanup
185+
def sendall_with_send_max(self):
186+
self.log.info("Check that `send_max` option causes negative value UTXOs to be left behind")
187+
self.add_utxos([0.00000400, 0.00000300, 1])
188+
189+
# sendall with send_max
190+
sendall_tx_receipt = self.wallet.sendall(recipients=[self.remainder_target], fee_rate=300, options={"send_max": True})
191+
tx_from_wallet = self.wallet.gettransaction(txid = sendall_tx_receipt["txid"], verbose = True)
192+
193+
assert_equal(len(tx_from_wallet["decoded"]["vin"]), 1)
194+
self.assert_tx_has_outputs(tx_from_wallet, [{"address": self.remainder_target, "value": 1 + tx_from_wallet["fee"]}])
195+
assert_equal(self.wallet.getbalances()["mine"]["trusted"], Decimal("0.00000700"))
196+
197+
self.def_wallet.sendtoaddress(self.wallet.getnewaddress(), 1)
198+
self.generate(self.nodes[0], 1)
199+
200+
@cleanup
201+
def sendall_specific_inputs(self):
202+
self.log.info("Test sendall with a subset of UTXO pool")
203+
self.add_utxos([17, 4])
204+
utxo = self.wallet.listunspent()[0]
205+
206+
sendall_tx_receipt = self.wallet.sendall(recipients=[self.remainder_target], options={"inputs": [utxo]})
207+
tx_from_wallet = self.wallet.gettransaction(txid = sendall_tx_receipt["txid"], verbose = True)
208+
assert_equal(len(tx_from_wallet["decoded"]["vin"]), 1)
209+
assert_equal(len(tx_from_wallet["decoded"]["vout"]), 1)
210+
assert_equal(tx_from_wallet["decoded"]["vin"][0]["txid"], utxo["txid"])
211+
assert_equal(tx_from_wallet["decoded"]["vin"][0]["vout"], utxo["vout"])
212+
self.assert_tx_has_output(tx_from_wallet, self.remainder_target)
213+
214+
self.generate(self.nodes[0], 1)
215+
assert_greater_than(self.wallet.getbalances()["mine"]["trusted"], 0)
216+
217+
@cleanup
218+
def sendall_fails_on_missing_input(self):
219+
# fails because UTXO was previously spent, and wallet is empty
220+
self.log.info("Test sendall fails because specified UTXO is not available")
221+
self.add_utxos([16, 5])
222+
spent_utxo = self.wallet.listunspent()[0]
223+
224+
# fails on unconfirmed spent UTXO
225+
self.wallet.sendall(recipients=[self.remainder_target])
226+
assert_raises_rpc_error(-8,
227+
"Input not available. UTXO ({}:{}) was already spent.".format(spent_utxo["txid"], spent_utxo["vout"]),
228+
self.wallet.sendall, recipients=[self.remainder_target], options={"inputs": [spent_utxo]})
229+
230+
# fails on specific previously spent UTXO, while other UTXOs exist
231+
self.generate(self.nodes[0], 1)
232+
self.add_utxos([19, 2])
233+
assert_raises_rpc_error(-8,
234+
"Input not available. UTXO ({}:{}) was already spent.".format(spent_utxo["txid"], spent_utxo["vout"]),
235+
self.wallet.sendall, recipients=[self.remainder_target], options={"inputs": [spent_utxo]})
236+
237+
# fails because UTXO is unknown, while other UTXOs exist
238+
foreign_utxo = self.def_wallet.listunspent()[0]
239+
assert_raises_rpc_error(-8, "Input not found. UTXO ({}:{}) is not part of wallet.".format(foreign_utxo["txid"],
240+
foreign_utxo["vout"]), self.wallet.sendall, recipients=[self.remainder_target],
241+
options={"inputs": [foreign_utxo]})
242+
243+
@cleanup
244+
def sendall_fails_on_no_address(self):
245+
self.log.info("Test sendall fails because no address is provided")
246+
self.add_utxos([19, 2])
247+
248+
assert_raises_rpc_error(
249+
-8,
250+
"Must provide at least one address without a specified amount" ,
251+
self.wallet.sendall,
252+
[]
253+
)
254+
255+
@cleanup
256+
def sendall_fails_on_specific_inputs_with_send_max(self):
257+
self.log.info("Test sendall fails because send_max is used while specific inputs are provided")
258+
self.add_utxos([15, 6])
259+
utxo = self.wallet.listunspent()[0]
260+
261+
assert_raises_rpc_error(-8,
262+
"Cannot combine send_max with specific inputs.",
263+
self.wallet.sendall,
264+
recipients=[self.remainder_target],
265+
options={"inputs": [utxo], "send_max": True})
266+
267+
def run_test(self):
268+
self.nodes[0].createwallet("activewallet")
269+
self.wallet = self.nodes[0].get_wallet_rpc("activewallet")
270+
self.def_wallet = self.nodes[0].get_wallet_rpc(self.default_wallet_name)
271+
self.generate(self.nodes[0], 101)
272+
self.recipient = self.def_wallet.getnewaddress() # payee for a specific amount
273+
self.remainder_target = self.def_wallet.getnewaddress() # address that receives everything left after payments and fees
274+
self.split_target = self.def_wallet.getnewaddress() # 2nd target when splitting rest
275+
276+
# Test cleanup
277+
self.test_cleanup()
278+
279+
# Basic sweep: everything to one address
280+
self.sendall_two_utxos()
281+
282+
# Split remainder to two addresses with equal amounts
283+
self.sendall_split()
284+
285+
# Pay recipient and sweep remainder
286+
self.sendall_and_spend()
287+
288+
# sendall fails if no recipient has unspecified amount
289+
self.sendall_invalid_recipient_addresses()
290+
291+
# Sendall fails if same destination is provided twice
292+
self.sendall_duplicate_recipient()
293+
294+
# Sendall fails when trying to spend more than the balance
295+
self.sendall_invalid_amounts()
296+
297+
# Sendall fails when wallet has no economically spendable UTXOs
298+
self.sendall_negative_effective_value()
299+
300+
# Leave dust behind if using send_max
301+
self.sendall_with_send_max()
302+
303+
# Sendall succeeds with specific inputs
304+
self.sendall_specific_inputs()
305+
306+
# Fails for the right reasons on missing or previously spent UTXOs
307+
self.sendall_fails_on_missing_input()
308+
309+
# Sendall fails when no address is provided
310+
self.sendall_fails_on_no_address()
311+
312+
# Sendall fails when using send_max while specifying inputs
313+
self.sendall_fails_on_specific_inputs_with_send_max()
314+
315+
if __name__ == '__main__':
316+
SendallTest().main()

test/functional/wallet_signer.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,12 @@ def test_valid_signer(self):
194194
assert(res["complete"])
195195
assert_equal(res["hex"], mock_tx)
196196

197+
self.log.info('Test sendall using hww1')
198+
199+
res = hww.sendall(recipients=[{dest:0.5}, hww.getrawchangeaddress()],options={"add_to_wallet": False})
200+
assert(res["complete"])
201+
assert_equal(res["hex"], mock_tx)
202+
197203
# # Handle error thrown by script
198204
# self.set_mock_result(self.nodes[4], "2")
199205
# assert_raises_rpc_error(-1, 'Unable to parse JSON',

0 commit comments

Comments
 (0)