[bug]: base64 parsing fails at /v2/invoices/lookup?payment_hash=X (base64 typesafe + urlencoded, still)

[martinoshub]

Background

I am trying to use the REST endpoint at /v2/invoices/lookup?payment_hash=X to get information about a payment/invoice.
If the payment_hash contains a "+" or a "/" character (which it often does), the request fails citing a base64 decode error. Is there perhaps an alphanumeric check on it by mistake?

Your environment

• lncli version 0.18.4-beta commit=v0.18.4-beta
• Debian

Steps to reproduce

GET /v2/invoices/lookup?payment_hash=ZNC%2BM5yd%2BS%2FFiu1cJEG94j9qdBo5Z7oc7TGsq4xVoIc%3D HTTP/1.1
Host: X
Grpc-Metadata-macaroon: X

Expected behaviour

Json response with invoice/payment data

Actual behaviour

{"code":3,"message":"parsing field "payment_hash": illegal base64 data at input byte 3","details":[]}

The above is just an example random 32-byte-base-64-urlencoded for privacy.

If I do the exact same thing but pick an r_hash that doesn't contain a + or /, there is no base64 error thrown, it returns the invoice data. If I pick one with + or / and then replace the naughty characters in the base64 with something else (say "A" or %41) the complaining about invalid base64 stops, but of course then the invoice is non existent as expected.

GET /v2/invoices/lookup?payment_hash=f2cu0kZA7IrlAt4F8vmDo4CIGwYVVs1L1ImkqXuBrW8= HTTP/1.1
Host: X
Grpc-Metadata-macaroon: X

{"code":5,"message":"unable to locate invoice","details":[]}

If I'm missing something, apologies and please do let me know. For now, it seems like a bug.

[ziggie1984]

You need to use typesafe base64 encoding, meaning the base64 encoding should not have a + or /, how did you convert the bytes of the hash to a base64 string ?

[ziggie1984]

try replacing + and / with the following:

echo -n "This is some text to encode" | base64 | tr '+/' '-_'

[martinoshub]

I've tried that. I've also tried encoding with https://github.com/RGBboy/urlsafe-base64
It's not rocket science though, this "safe" base64 is just eliminate the trailing =, replacing + with -, / with _, so I get the same base64 whether I use tr, that javascript library or just a notepad. My payment_hash contains only alphanumeric, _ and - now. And I still get this:

{"code":3,"message":"parsing field "payment_hash": illegal base64 data at input byte 40","details":[]}

[martinoshub]

Update: very interesting! I've read contradictory statements over the trailing =, but apparently not only is it not removed, my test failed because it was missing it.

So the solution is: keep the = at the end, replace +/ with -_

Thanks for your help!

[ziggie1984]

because the underlying go lib uses typ-safe encoding/decoding