Add the ability to query for the transaction that spends a specified transparent outpoint.

[nuttycom]

What is your feature request?
Add a GRPC method:

message OutPoint {
    bytes txid = 1;           // transaction identifier
    uint32 output_index = 2;  // index of the output in the transactions `vout` array
}

rpc GetSpendingTx(OutPoint) returns (RawTransaction) {}

where GetSpendingTx returns the transaction that spends the provided output, or NotFound if the outpoint is not recognized or the outpoint is unspent.

How would this feature help you?
In restoring transparent history, it is possible to traverse the transaction graph from spending tx to source of funds using available methods of the GRPC interface, but there is no direct way to find the transaction that spends a given output. As a consequence, it's currently necessary to use the GetTaddressTxids method to attempt to find spends related to a given outpoint, but this is highly imprecise - it will return all of the transactions that involve that address, and queries to this method must specify a block range to query. In highly active wallets, this could result in numerous calls that each return hundreds or thousands of results, rather than just the one result that is wanted.

Adding this RPC method will make it possible to fully restore a wallet's transparent transaction graph in an efficient manner.

[zancas]

@idky137 let's coordinate on this

[nuttycom]

Instead of OutPoint, this method could take a sum type that is OutPoint | SaplingNullifier | OrchardNullifier. This API leaks the same amount of information to the lightwalletd server as does GetTransaction (though, in the case of investigating transparent OutPoints it's possible that a client may be tracing the history of someone else's transactions).

The indexer should be considered to provide all the functionality that might be desired of it in a trusted context - i.e. it should provide all the functionality that could be desired of it if the end user of a wallet is also running their own indexer, such as is the case for an exchange wallet. Then, subsets of that functionality may remain unused or disabled (or only accessible via TOR/Nym?) if it's running in an untrusted context such as as lightwalletd server.

[Oscar-Pepper]

I don't quite understand the benefit of this proposed gRPC method over calling GetTransaction with the txid provided in the Outpoint

[nuttycom]

@Oscar-Pepper You're misunderstanding it then. The txid component of an OutPoint returns the transaction in which an output was created. What we want is, given an OutPoint, determine the transaction (if any) in which that output is spent.

When we discover a transaction where one of its outputs belongs to us, we then want to also find out if that output is already spent, and if so, where.

This requires a separate index that is constructed by inspecting each transaction, and recording a mapping between each OutPoint in its vin array and the txid of the transaction being inspected.

[LarryRuane]

This can use zcashd's getspentinfo rpc, which requires the insightexplorer configuration option.

getspentinfo help

getspentinfo {"txid": "txidhex", "index": n}

Returns the txid and index where an output is spent.

WARNING: getspentinfo is disabled.
To enable it, restart zcashd with the following command line options:
-experimentalfeatures and -insightexplorer

Alternatively add these two lines to the zcash.conf file:

experimentalfeatures=1
insightexplorer=1

Arguments:
{
  "txid"   (string) The hex string of the txid
  "index"  (number) The vout (output) index
}

Result:
{
  "txid"   (string) The transaction id
  "index"  (number) The spending (vin, input) index
  ,...
}

Examples:
> zcash-cli getspentinfo '{"txid": "33990288fb116981260be1de10b8c764f997674545ab14f9240f00346333b780", "index": 4}'
> curl --user myusername --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getspentinfo", "params": [{"txid": "33990288fb116981260be1de10b8c764f997674545ab14f9240f00346333b780", "index": 4}] }' -H 'content-type: text/plain;' http://127.0.0.1:8232/

Some zcashd nodes may be configured this way already, but many likely have only the lightwalletd=1 option, and those would need to reindex, which takes a significant amount of time. I'll time that reindex on mainnet and update this issue when it completes so we have an idea of how long that will take.

An alternative might be for lightwalletd to create this index itself (not depend on zcashd), but that would make its storage requirement (either memory, disk, or both) quite a bit higher.

Also, I don't understand about the nullifier extension; is that required? Or should we implement only the transparent part, at least initially?

[LarryRuane]

Some zcashd nodes may be configured this way already, but many likely have only the lightwalletd=1 option

lightwalletd could gracefully fail this new GetSpendingTx gRPC if zcashd doesn't support it (doesn't have insightexplorer configured), rather than just halting. We will have to document in lightwalletd's release notes that zcashd must reindex with insightexplorer enabled. Also, zcashd's documentation should say that for use with lightwalletd, insightexplorer should be configured, not just lightwalletd as in the past. Or just change zcashd's lightwalletd config option to enable this extra index, but care must be taken so that if zcashd is upgraded, it doesn't require reindexing, because that takes a long time (for me, it's days). Since zcashd isn't going to be the main client for much longer, it's probably better to simply require insightexplorer (even if that's slightly confusing) if the user wants this GetSpendingTx gRPC.

Just to confirm, is it still worth making this change? If so, I could start on it soon.

[LarryRuane]

@nuttycom, I got sidetracked on other things, would this still be useful? My previous two comments (above) are still accurate, except I hadn't thought about zebrad. Unfortunately, zebrad doesn't implement zcashd's getspentinfo RPC, so this would need to be added. (That RPC is not supported by zebrad because, previous to this PR, that RPC was needed only to support insightexplorer, not lightwalletd.)

One additional thought this comment of mine (above):

An alternative might be for lightwalletd to create this index itself (not depend on zcashd), but that would make its storage requirement (either memory, disk, or both) quite a bit higher.

An additional problem with having lightwalletd maintain this index is that getting it right in the presence of chain reorgs is notoriously difficult. It would certainly be better to let zebrad (or zcashd) alone deal with that complexity, and have lightwalletd simply query for this spent information on demand.

[LarryRuane]

@nuttycom - still would like to know if this is needed. I think I'll close it soon if it seems not to be needed soon.