Passphrase protected key store spec

Summary

This page details the crypto & encodings in the passphrase protected key store in Go. To implement this in another Ethereum implementation the steps below can be followed. The example JSON also acts as a test vector.

TODO: update github links to mainline branch once the latest key store PR is merged.

References

• Go source code & cryptography notes: https://github.com/Gustav-Simonsson/go-ethereum/blob/improve_key_store_crypto/crypto/key_store_passphrase.go#L29

The source code comments there have a list of further references.

• Discussion with Alex about the cryptography used: https://docs.google.com/document/d/179oqUuyePYp0xedF3JurpAPfrCEJJYWwMw-Q3NuL0s0/edit

From JSON to privkey

We start with the JSON in the key file on disk and list operations to get a privkey for signing.

JSON in key file on disk (password is "foo") :

{
    "Address": "d4584b5f6229b7be90727b0fc8c6b91bb427821f",
    "Crypto": {
        "CipherText": "07533e172414bfa50e99dba4a0ce603f654ebfa1ff46277c3e0c577fdc87f6bb4e4fe16c5a94ce6ce14cfa069821ef9b",
        "IV": "16d67ba0ce5a339ff2f07951253e6ba8",
        "KeyHeader": {
            "Kdf": "scrypt",
            "KdfParams": {
                "DkLen": 32,
                "N": 262144,
                "P": 1,
                "R": 8,
                "SaltLen": 32
            },
            "Version": "1"
        },
        "MAC": "8ccded24da2e99a11d48cda146f9cc8213eb423e2ea0d8427f41c3be414424dd",
        "Salt": "06870e5e6a24e183a5c807bd1c43afd86d573f7db303ff4853d135cd0fd3fe91"
    },
    "Id": "0498f19a-59db-4d54-ac95-33901b4f1870",
    "Version": "1"
}

Lots wrong.

• Capitalisation is unjustified and inconsistent (scrypt lowercase, Kdf mixed-case, MAC uppercase).
• Address unnecessary and compromises privacy - remove.
• Salt is a param of the KDF, thus belong in KdfParams.
• SaltLen unnecessary (just derive it from Salt).
• KDF algorithm given, but not crypto algorithm.
• Version is intrinsically numeric yet is a string.
• KDF and cipher are notionally sibling concepts yet are organised differently.

Gustav's reply to above suggestions:

ACK on capitalisation, removal of address and saltlen fields and addition of encryption cipher field.

NACK on bundling salt with KDF parameters; for both scrypt and PBKDF2 both specs and implementations usually separate between the KDF "configuration" and the input for a single call of the KDF.

Morever, the "parameters" for scrypt / PBKDF2 can stay the same for multiple keys, whereas the salt is unique for each key. Bundling it together with "configuration" KDF parameters can give the wrong impression that it's part of the KDF config, when it's a per-key value.

NACK on having version as integer; a version field should only be compared for equality, so a string makes more sense than an integer. Morever, we haven't defined what version semantic to use here, so using string prepares for any semantic we decide on (and also changes to it in the future!).

Go code unmarshaling and decrypting this: https://github.com/Gustav-Simonsson/go-ethereum/blob/improve_key_store_crypto/crypto/key_store_passphrase.go#L199

1. Read in the JSON and parse out the KeyHeader field. Get the JSON representation of the KeyHeader as a string / byte array. We need this for MAC calculation.

2. Get the derived key by calling scrypt with the scrypt params, the salt and the passphrase.

3. Calculate MAC by taking SHA3 of concatentation of KeyHeader JSON, last 16 bytes of derived key and cipher text:

SHA3( KeyHeaderJSON || derivedKey[16:32] || cipherText )

4. Compare calculated MAC with MAC from the JSON object. If not equal, abort and return an error.

5. Decrypt the cipherText using AES-128 with CBC mode and the IV from the JSON. The key used to decrypt is the last 16 bytes of SHA3 of the last 16 bytes of the scrypt derived key:

AES_128_CBC_Decrypt( SHA3(derivedKey[:16])[:16], cipherText, iv )

6. The decrypted plaintext is the 32 bytes EC privkey.

7. For encryption: reverse the above steps. Ensure both salt and iv are generated by CSPRNG.

Further notes:

• The "Address" is OPTIONAL. It is the ethereum address of the key.
• The Id is OPTIONAL. It is a UUID Version 4. It is used to identify a key not using key-derived data such as the ethereum address, for e.g. key file names.