Abbreviate some terms in the decoder files.

[h2zero]

Seems the decoders are getting larger and larger, one way we could compress these a bit would be to abbreviate certain terms.

For instance we could abbreviate the word "service data" to "svcd" and "manufacturerdata" to "mfrd" or something similar.

The functions could be abbreviated also "value_from_hex_data" could just be "vfhd".

Interested to hear thoughts on this. I think it would be a large size improvement and help performance.

[DigiH]

I'm definitely for reducing the decoders' sizes in any way possible. There will only ever be more decoders in the future, not less ;)

While I find abbreviating as much as possible, especially with reucrring "servicedata", "manufacturerdata", "value_from_hex_data" and the likes, I also think it very important to clearly state this in the docs, so as to avoid confusing people or even making the start into getting into creating new decoders more of a hurdle.

For me everything being completely written out definitely helped in getting started.

Other than that I also very much like to push the binary-bit decoder option, as this will make such a big impact on compressing decoder sizes. I started thinking and testing it while working more on the SwicthBot decoders, with, you wouldn't have guessed it, the simple sounding motion sensor being a real beast. Not even open for discussion in submitting it anywhere close to what it is now.

https://github.com/DigiH/decoder/blob/switchbot-motion/src/devices/SBMS_json.h

Even with some, possibly not so necessary, properties removed it would still be huge. This is how I then started testing a binary-bit implementation.

I think all that is needed is for a single hex-byte digit to be tested for values of any of its corresponding 4-bit binary. This would allow for nearly all the current multi-OR conditions to only have a single such condition.

Even when most of the part of 2 hex bytes are used, as in @1technophile's example of the SwitchBots' battery 7 binary bits, instead of creating an additional hex-to-binary-to-decimal decoding routine this can also easily, and quite briefly be substituted as currently with an underscore variant and subtraction (-128) by checking bit[3] of digit[1] of the 2 byte hex, so

"batt":{
         "condition":["servicedata", 4, "0", "|", "servicedata", 4, "1", "|", "servicedata", 4, "2", "|", "servicedata", 4, "3", "|", "servicedata", 4, "4", "|", "servicedata", 4, "5", "|", "servicedata", 4, "6"],
         "decoder":["value_from_hex_data", "servicedata", 4, 2, false, false]
      },
      "_batt":{
         "condition":["servicedata", 4, "8", "|", "servicedata", 4, "9", "|", "servicedata", 4, "a", "|", "servicedata", 4, "b", "|", "servicedata", 4, "c", "|", "servicedata", 4, "d", "|", "servicedata", 4, "e"],
         "decoder":["value_from_hex_data", "servicedata", 4, 2, false, false],
         "post_proc":['-', 128]
      }

would turn into

"batt":{
        "condition":["servicedata", 4, "binbit”, 3, "0"],
         "decoder":["value_from_hex_data", "servicedata", 4, 2, false, false]
      },
      "_batt":{
         "condition":["servicedata", 4, "binbit”, 3, "1"],
         "decoder":["value_from_hex_data", "servicedata", 4, 2, false, false],
         "post_proc":['-', 128]
      }

and much the same with all other multi-OR conditions.

Just a thought in also getting the decoder sizes down.

[h2zero]

Yes, that is another addition to the overall efficiency improvement going forward. We should focus on the low hanging fruit first and tackle the harder to reach stuff once that's done.

Any suggestions for the abbreviations? Other ideas?

[DigiH]

I like your suggestions of

"servicedata" - "svcd"
"manufacturerdata" - "mfrd"

also

"properties"- "props"
"condition" - "cond"
"decoder" - "dec"

with the functions I would drop the d(ata) at the end, and maybe use the saved letter for an extra written out hex at the end for clarity?!?

"value_from_hex_data" - "vfhex"
"bf_value_from_hex_data" - "bfvfhex"
"string_from_hex_data" - "sfhex"
"static_value" - "static"

"post_proc" - "pproc"

[DigiH]

Apologies for coming back to this, but please have a look at my 4-bit binary condition testing test, and the included changed SwitchBot S1 decoder with quite a bit of saving through the direct 4-bit binary evaluations.

DigiH@fb38fcf

Existing S1 tests running fine.

Still a bit messy though, with cond_len unnecessarily evaluated, even if not needed for binbit evaluation, and the actual code could probably also be cleaned further ... especially with #include bumping the size up 148 kB :(

I'm just worried that without such direct binary bit condition possibility any further SwitchBot decoders won't really be feasible.

But as per the topic, I think we should start with shortening definitions as much as possible :)

Thanks

[h2zero]

Yes, std::stingstream is a big no-no on the esp32. That's why the size increased so much. Better to use a simple conversion from the sting byte to byte and test with a bitmask.

[DigiH]

Better to use a simple conversion from the sting byte to byte and test with a bitmask.

Sounds good, but remember I'm a total c++ noob ;) if this is easy to get svc_data[prop_condition[i + 1] into current unsigned n that would be great, as including <bitset> hardly has any impact size wise. Otherwise, I suppose even a simple lookup table would have less of an impact than using std::stingstream. Still, it was a good exercise for me, but let's put this and the remaining SwitchBot decoders aside for now and come back to the issue at hand here.

What do you think about the suggestions above for abbreviating remaining decoder denotations?

"value_from_hex_data" - "vfhex" or "vfhd"

as long as it's short and well explained in the documentation, should all be fine.

[DigiH]

@h2zero - what do you think about my current change for a binary 4-bit evaluation

DigiH@d9797e3

even shorter than before, and the only additional include is for , which doesn't really impact size.

[h2zero]

Looks better but you can remove the std::bitset and std::string.

Give stroul the const char input from the array directly instead of the std::string.

Replace the bit set with some shifting and masking.

(Val >> binbit) & 0x01 == bitval

Also instead of "binbit" we should probably just call it "bit".

[h2zero]

What do you think about the suggestions above for abbreviating remaining decoder denotations?

4 letter abbreviations are preferable IMO, not always practical though. I'm going to try a few things today.

[h2zero]

For interest I started abbreviating these and figured I'd put a bit of performance into it as well. I pushed the branch here, there's more that can be done but this is a start.

OMG code size before:

RAM:   [==        ]  17.8% (used 58344 bytes from 327680 bytes)
Flash: [=======   ]  73.3% (used 1440664 bytes from 1966080 bytes)

After:

RAM:   [==        ]  17.8% (used 58376 bytes from 327680 bytes)
Flash: [=======   ]  73.2% (used 1439532 bytes from 1966080 bytes)

So far it's saved ~1100 bytes, not as much as hoped but it's something 😄

[DigiH]

So far it's saved ~1100 bytes, not as much as hoped but it's something 😄

Still "string_from_hex_data" for a bit more squeezing ;) Great way to go, especially with the related faster comparison checking!

[h2zero]

Yeah, there's a couple little things like that to do. Not sure how much of a performance boost it is, depends on the compiler, but at least it would be consistent.