Protocol notes

[uzlonewolf]

I wanted to collect my notes about the new 6699/v3.5 protocol in one place without burying it in back-and-forth troubleshooting discussion, so here it is.

---

Packet formats

As a recap, the older 55AA/v3.2-v3.4 packet format looks like:

000055aaSSSSSSSSMMMMMMMMLLLLLLLL[RRRRRRRR]DD..DDCC..CC0000aa55 where:
000055aa - prefix
SSSSSSSS - 32-bit sequence number
MMMMMMMM - 32-bit Command ID
LLLLLLLL - 32-bit packet length - count every byte from the return code through (and including) the footer
[RRRRRRRR] - packets from devices have a 32-bit return code. Packets from the app/client do not have this field
DD..DD - variable length encrypted payload data
CC..CC - checksum, either 32-bit (4-byte) CRC32 for v3.2-v3.3, or 256-bit (32-byte) HMAC-SHA256 for v3.4
0000aa55 - footer

Everything except the payload data is sent unencrypted/in the clear. Payload data is padded to have a multiple-of-16 length and is AES128 encrypted in ECB mode. All multi-byte integers are big-endian.

The newer v3.5 format looks like:

00006699UUUUSSSSSSSSMMMMMMMMLLLLLLLL(II*12)DD..DD(TT*16)00009966
00006699 - prefix
UUUU - Unknown 16-bit or 2*8-bit field. Current devices (as of June 2024) always transmit as 0x0000 and completely ignore on reception
SSSSSSSS - 32-bit sequence number
MMMMMMMM - 32-bit Command ID
LLLLLLLL - 32-bit packet length - count every byte from the IV/nonce through (and including) the Tag, but not the footer
(II*12) - 96-bit (12-byte) per-packet AES-GCM IV/nonce
DD..DD - variable length encrypted payload data
(TT*16) - 128-bit (16-byte) signature Tag value from AES-GCM encrypt-and-sign
00009966 - footer

Everything except the payload data is sent unencrypted/in the clear. Payload data is not padded and is AES128 encrypted in GCM mode. The GCM Tag value is computed while encrypting/decrypting; the UUUU through LLLLLLLL fields are signed but not encrypted while the payload is both signed and encrypted. The GCM IV/nonce changes for every packet and is in the II*12 field.

Packets from devices still have a 32-bit return code, however it is encrypted and prepended to the payload and needs to be stripped after decryption but before payload processing.

---

"Version Protocol" header

For some commands (such as STATUS), v3.3+ devices prepend a data source header to the payload data that needs to be stripped off. The format of this header is 3.xCCCCCCCCSSSSSSSSUUUUUUUU:

3.x - The protocol version as an ASCII string, i.e. "3.3"
CCCCCCCC - When the message comes from the MQTT server, this is a crc32 hash of the message; when it comes from the LAN it is set to 0.
SSSSSSSS - Source-specific message sequence number that increments on every message and also persists across connections. If a device receives a 2nd message from the same source with the same or decreasing sequence number, that message is discarded. This allows a client to send a message via both the cloud and the local network simultaneously and the device will process whichever arrives first and ignore the other. The client can also use it to figure out if it has missed any messages by looking for jumps in the sequence number.
UUUUUUUU - Unique source identifier. The device is source 1 and clients should generate a unique (random?) number to identify that specific client - this identifier should persist across connections.

If the unique source identifier is "0" then the sequence number is ignored and all messages are processes.

The device I looked at only "remembers" the last 5 unique source identifiers; the oldest is overwritten when a new source shows up.

---

Session keys

Both v3.4 and v3.5 use a 3-way session key negotiation scheme to generate unique per-connection session keys to encrypt the payload data. All payloads during this exchange are encrypted like normal using the real key.

First, the client sends the device a 16-byte nonce. Next, the device responds with its own 16-byte nonce plus a 32-byte HMAC-SHA256 of the client nonce (again using the real key as the key). Finally, the client sends the device the 32-byte HMAC-SHA256 of the device nonce.

The session key is then calculated by XORing the nonces and encrypting the result. The encryption routine used for this is the exact same as for payloads; the v3.5 GCM IV is the first 12 bytes of the client nonce. I.e.:

tmp_key = bytes( [ a^b for (a,b) in zip(device_nonce, client_nonce) ] )
cipher = tinytuya.AESCipher( real_key )
session_key = cipher.encrypt( tmp_key, use_base64=False, pad=False, iv=client_nonce[:12] )[12:28]

---

Device discovery broadcast packets

v3.5 devices also use the new 6699 packet format for their discovery broadcasts. For these, the encryption key is the MD5 digest of yGAdlopoPVldABfn. AES-GCM decryption and Tag verification is then performed as usual. A small stand-alone script to receive and decrypt these broadcasts is:

from Crypto.Cipher import AES
from hashlib import md5
import socket

clients = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
clients.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
clients.bind(("", 6667))
clients.settimeout(7)

key = md5( b'yGAdlopoPVldABfn' ).digest()

for i in range( 10 ):
    data, addr = clients.recvfrom(4048)
    prefix = data[:4]

    if prefix == b'\x00\x00\x55\xAA':
        print( 'Ignoring 55AA packet' )
    elif prefix == b'\x00\x00\x66\x99':
        header = data[4:18]
        iv = data[18:30]
        payload = data[30:-20]
        tag = data[-20:-4]

        print( addr, '=', payload )

        cipher = AES.new(key, AES.MODE_GCM, nonce=iv)
        cipher.update( header ) # needed for tag verification
        decrypted = cipher.decrypt_and_verify( payload, tag )

        print( 'Decrypted:', decrypted )
        print( 'Return Code:', decrypted[3]) # really decrypted[:3]
        print( 'Payload:', decrypted[4:])
    else:
        print( 'Unknown packet format!', prefix )

    print('')

Once decrypted these new devices are reporting protocol v3.5.

---

Device discovery packets redux

Newer v3.5 devices do not send out unsolicited discovery broadcasts. Instead, they listen for broadcasts from a client app and send their discovery packet directly to that client.

The client broadcast contains the payload {"from":"app","ip":"192.168.1.42"} and is sent to the subnet broadcast address (i.e. 192.168.1.255) on port 7000. It is GCM encrypted the same way as broadcasts above.

When a device receives a client broadcast, it responds by sending a device discovery packet directly to the IP address specified in the client broadcast. This again uses port 7000 and is GCM encrypted the same way as broadcasts above. The device I used in my testing will send the device discovery packet out as a broadcast if the client app specifies "255.255.255.255" as the IP address, however I suspect this is an accident and is not something they intended to be used.

[uzlonewolf]

Added "Device discovery broadcast packets" section.

[uzlonewolf]

Reworked discovery example to be a bit more complete.

[Ahmad-Agbaria]

shouldn't you change clients.bind(("", 6667)) to clients.bind(("", 6699)) as you mentioned in notes that v3.5 uses this port

[uzlonewolf]

No, 00006699 is the header prefix (replacing 000055AA), it still uses ports 6667/6668.

[uzlonewolf]

Added:

"Version Protocol" header

The 3.xCCCCCCCCSSSSSSSSUUUUUUUU header found in v3.3+ commands breaks down as follows:

3.x - The protocol version as a string, i.e. "3.3"
CCCCCCCC - When the message comes from the MQTT server, this is a crc32 hash of the message; when it comes from the LAN it is set to 0.
UUUUUUUU - Unique source identifier. The device is source 1 and clients should generate a unique (random?) number to identify that specific client - this identifier should persist across connections.
SSSSSSSS - Source-specific message sequence number that increments on every message and also persists across connections. If a device receives a 2nd message from the same source with the same or decreasing sequence number, that message is discarded. This allows a client to send a message via both the cloud and the local network simultaneously and the device will process whichever arrives first and ignore the other. The client can also use it to figure out if it has missed any messages by looking for jumps in the sequence number.

If the unique source identifier is "0" then the sequence number is ignored and all messages are processes.

The device I looked at only "remembers" the last 5 unique source identifiers; the oldest is overwritten when a new source shows up.

[uzlonewolf]

Added:

Device discovery packets redux

Newer v3.5 devices do not send out unsolicited discovery broadcasts. Instead, they listen for broadcasts from a client app and send their discovery packet directly to that client.

The client broadcast contains the payload {"from":"app","ip":"192.168.1.42"} and is sent to the subnet broadcast address (i.e. 192.168.1.255) on port 7000. It is GCM encrypted the same way as broadcasts above.

When a device receives a client broadcast, it responds by sending a device discovery packet directly to the IP address specified in the client broadcast. This again uses port 7000 and is GCM encrypted the same way as broadcasts above. The device I used in my testing will send the device discovery packet out as a broadcast if the client app specifies "255.255.255.255" as the IP address, however I suspect this is an accident and is not something they intended to be used.

[jasonacox]

Great discovery, @uzlonewolf ! Is this exclusive to 3.5 or do other versions also see this 7000 broadcast and respond back?

In any case, it would be good to add this to our scanner.

[uzlonewolf]

The broadcast from the app uses GCM and the 6699 packet format, so I suspect we'll only see it in v3.5 devices. In theory they could add a 55AA packet as well, but I find that unlikely as it would require a firmware update to older devices and that's not something they really do (I think I've seen maybe 3 devices get firmware updates).

[uzlonewolf]

And yes, I do intend to add it to the scanner as well as find_device() (which is used by address='Auto'; I'll probably just rewrite this to use the scanner so we don't have to implement this in multiple places). As I mentioned in 507 the tricky part is going to be getting the list of local IP addresses/interfaces in python on multi-interface machines.

[ayushsharma82]

Added support for 3.5 protocol in my fork of homebridge plugin thanks to these veryy helpful notes by @uzlonewolf .

I would like to add to the notes that, command 0x00000008 sent from a 3.5 protocol tuya device to any client has 19 bytes prepended in decrypted payload which have to be stripped before parsing it as JSON. Command 0x00000008 is for notifying connected clients of state updates.

As of now, I don't know exactly what those 19 bytes are but they don't seem to be of any use. JSON payload is what we really need.

[uzlonewolf]

That's great!

As for those extra bytes, are you talking about the "Version Protocol" header?

[ayushsharma82]

I don't think so.. It was weirdly 19 bytes on cmd 8 only. For comparison, client receives cmd 16 from the device with only 4 bytes prepended to payload which I believe is the return code.

I'll add a debug line to get the raw payload and paste here soon. Till then, here's the scenario in which this happens:

1. Client sends command 13 (Update DP) to Tuya 3.5 Device
2. Device sends back command 8 to the client with state update data.

[uzlonewolf]

I edited the post to move the "Version Protocol" header section higher and reworded the into sentence to make it more clear that it needs to be stripped off. On v3.5 devices the cmd 8 encrypted payload data breaks down as:

4 bytes: return code (0 =success, 1 = error)
3 bytes: device version as a string ("3.5")
4 bytes: MQTT CRC32, or all 0's when from LAN.
4 bytes: Source-specific message sequence number, or all 0's if not used.
4 bytes: Unique source identifier, or all 0's when not used. The device is source 1.
Remainder of message: JSON data

[ayushsharma82]

Makes sense! Here's the decrypted payload from cmd 8 if you can spot any oddity or confirm the findings:

00000000332e350000000000003561000000017b2270726f746f636f6c223a342c2274223a313735343835353232362c2264617461223a7b22647073223a7b223230223a66616c73657d7d7d

[uzlonewolf]

Yep, looks like a normal retcode+header.

00000000332e350000000000003561000000017b2270726f746f636f6c223a342c2274223a313735343835353232362c2264617461223a7b22647073223a7b223230223a66616c73657d7d7d

00000000 = retcode 0
332e35 = ASCII "3.5"
00000000 = From LAN, so all 0's
00003561 = Source-specific message sequence number
00000001 = Source #1 (device)
7b2270726f746f636f6c223a342c2274223a313735343835353232362c2264617461223a7b22647073223a7b223230223a66616c73657d7d7d = JSON {"protocol":4,"t":1754855226,"data":{"dps":{"20":false}}}

I just noticed I had the source/sequence fields swapped in my previous message because I swapped the description in my original post to make the text flow better. Fixed.