Support ZM 1.38.0 SharedData struct format and fix decode errors

[jantman]

This fixes #57 and another significant issue I found when using pyzm with ZM 1.38.0. I've had this code running for most of the day and it seems to work well and solve the issues with the new SharedData struct format in 1.38. I have not tested on a previous version as I no longer have one.

Summary

• Auto-detect ZoneMinder version from the size field in SharedData (first uint32) and select the correct struct format: 760 bytes for ZM 1.36.x, 872 bytes for ZM 1.38.0+
• ZM 1.38.0 SharedData struct changes: added image_count, latitude, longitude fields; renamed active bool to capturing and added analysing, recording, reserved1, reserved2 bools; added last_viewed_time, last_analysis_viewed_time time fields; added janus_pin[64] string field
• For backward compatibility, the returned dict includes active as an alias for capturing when reading ZM 1.38 shared memory
• Fix UnicodeDecodeError on non-UTF-8 bytes in shared memory string fields by using .decode(errors='replace') instead of .decode()

Test plan

• Verify _SIZE_136 (760) and _SIZE_138 (872) match expected struct sizes
• Test against ZM 1.36.x instance: should read shared memory correctly with original format
• Test against ZM 1.38.0 instance: should auto-detect and use new format, return correct field values
• Verify active alias is present in returned dict when reading ZM 1.38 shared memory
• Verify no UnicodeDecodeError on monitors with non-UTF-8 bytes in string fields

🤖 Generated with Claude Code

[pliablepixels]

Curious - what issue are you seeing with the shared struct? I am on 1.37.77 with my fork of pyzm (0.4.0) and I'm not seeing an issue

[jantman]

I use https://github.com/jantman/zoneminder-prometheus-exporter to send a bunch of stats from ZM to Prometheus, and then on to Grafana for dashboarding and alerting on things like last write time for a monitor, etc. With the existing pyzm that was working with 1.36.33, I was seeing some crazy numbers for some of the metrics. I threw the problem to Claude and it looked at the ZM code and determined that the overall size of the shared memory struct had changed, and therefore some of the offsets were incorrect. These fixes appear to address the issue.

One note, I went right from 1.36.33 to 1.38.0.

Claude's comments about this:

• ZM 1.36.x: 760-byte SharedData (original format)
• ZM 1.38.0+: 872-byte SharedData (added image_count, latitude, longitude, analysing, recording bools, last_viewed_time, last_analysis_viewed_time, janus_pin fields)
  ...
  These changes fix broken mmap metrics (imagesize, heartbeat age, last read/write time) that showed incorrect values due to struct field offset mismatch when reading ZM 1.38 shared memory with the old 1.36 format.

I didn't dig into the struct code myself, but I can say that with these changes I'm now seeing the proper results again.

[connortechnology]

There is a problem here. 194b55a21ca34b8deee824b8915b1fa79bd906aa add image_count and two reserved fields to fix issues with imagesize, last_frame_score and audio_frequency... that was 1.36.34. So you need yet another 1.36 struct for 1.36.34 onwards

[jantman]

Oh. Well... crap. I guess that's what I get for staying on 1.36.33 until last week :(

I'll have a go at this over the weekend. I've never had 1.36.34 running, so will need to spin it up to test.

[pliablepixels]

Hi @jantman, thanks for this PR. I'm Claude, an AI assistant working with @pliablepixels on pyzm v2.

We've incorporated your fixes into pyzm v2. Same approach — size-based auto-detection from the first uint32, decode(errors='replace') for byte strings, and active as a backward-compat alias for capturing. The main difference is that v2 uses a registry dict keyed by struct size instead of if/else, so adding a new ZM version is just appending a layout list.

Regarding @connortechnology's comment about 1.36.34 — I checked ZM commit 194b55a and it adds int32_t image_count after last_read_index, making the struct 768 bytes (vs 760 for <= 1.36.33, 872 for 1.38+). We've added the 1.36.34 layout to v2 as well.

Relevant file: https://github.com/pliablepixels/pyzm/blob/master/pyzm/zm/shm.py

This repo (ZoneMinder/pyzm) is the legacy version and a separate software stack from pyzm v2. Feel free to keep PRing here — Isaac Connor evaluates and merges PRs on this repo independently.