Build a driver from source for an existing installation: apc_modbus + USB

[KartmanCanada]

I'm looking for a bit of help here... similar to the previous issue, I've got an APC UPS working correctly under USBHID-UPS driver but it's not showing some of the key info that I'd like. I was hoping to keep the NUT install as is but add the APC_MODBUS driver. I've been following the below info:

https://github.com/networkupstools/nut/wiki/APC-UPS-with-Modbus-protocol
https://www.kreaweb.be/diy-home-server-2021-software-proxmox-ups/
#2348

I'm running on Proxmox host (v8.3.5). I've gotten the patched LIBMODBUS (prefix = /usr/local) and NUT (config from wiki) working. The APC_MODBUS driver is compiling but I can't just copy to the "/lib/nut" directory as I need the NUT config to be following the system install paths. I understand the goals but I'm not sure about all of the correct NUT config options. Below is what I've got so far...

./configure --with-drivers=apc_modbus --with-usb --with-modbus \
    --with-modbus-includes=-I/usr/local/include/modbus \
    --with-modbus-libs="-L/usr/local/lib -lmodbus" \
    --with-user=root --with-group=nut \
    --datadir=/usr/share/nut \
    --sysconfdir=/etc/nut \
    --with-statepath=/var/run/nut \
    --with-pidpath=/var/run/nut \
    --with-altpidpath=/var/run/nut \
    --with-drvpath=/lib/nut \
    --with-systemdsystemunitdir=/usr/lib/systemd/system \
    --with-pkgconfig-dir=/usr/lib/pkgconfig \
    --with-udev-dir=/usr/lib/udev \
    --libdir=/usr/lib/

The goal is to build against the patched LIBMODBUS and, at the end, copy the APC_MODBUS driver file direct from the build directory into /lib/nut and have it live alongside the USBHID-UPS driver installed by the basic NUT install.

[KartmanCanada]

Update: above config and test:

/lib/nut/apc_modbus -DDDDD -s apc -x vendorid=051D -x productid=0003 -x port=auto
   0.000002     [D5] send_to_all: SETINFO driver.state "init.starting"
Network UPS Tools 2.8.2.2806-2806+gef66c3c8f (development iteration after 2.8.2) - NUT APC Modbus driver with USB support 0.12
   0.000053     [D5] send_to_all: SETINFO driver.version.usb "libusb-1.0.26 (API: 0x01000109)"
   0.000056     [D1] Using USB implementation: libusb-1.0.26 (API: 0x01000109)
   0.000059     [D1] This build of the driver is USB-capable; also Serial and TCP Modbus RTU are supported
   0.000066     [D3] main_arg: var='vendorid' val='051D'
   0.000070     [D5] send_to_all: SETINFO driver.parameter.vendorid "051D"
   0.000075     [D3] main_arg: var='productid' val='0003'
   0.000078     [D5] send_to_all: SETINFO driver.parameter.productid "0003"
   0.000080     [D3] main_arg: var='port' val='auto'
   0.000085     [D5] send_to_all: SETINFO driver.parameter.port "auto"
   0.000087     [D1] Network UPS Tools version 2.8.2.2806-2806+gef66c3c8f (development iteration after 2.8.2) built with gcc (Debian 12.2.0-14) 12.2.0 and configured with flags: --with-drivers=apc_modbus --with-usb --with-modbus --with-modbus-includes=-I/usr/local/include/modbus --with-modbus-libs='-L/usr/local/lib -lmodbus' --with-user=root --with-group=nut --datadir=/usr/share/nut --sysconfdir=/etc/nut --with-statepath=/var/run/nut --with-pidpath=/var/run/nut --with-altpidpath=/var/run/nut --with-drvpath=/lib/nut --with-systemdsystemunitdir=/usr/lib/systemd/system --with-pkgconfig-dir=/usr/lib/pkgconfig --with-udev-dir=/usr/lib/udev --libdir=/usr/lib/
   0.000093     [D1] debug level is '5'
   0.000096     [D5] send_to_all: SETINFO driver.debug "5"
   0.000099     [D5] send_to_all: SETFLAGS driver.debug RW NUMBER
   0.000463     [D1] Succeeded to become_user(root): now UID=0 GID=0
   0.000476     [D1] Signalling UPS [apc]: driver.exit (quietly, no fuss if no driver is running or responding)
   0.000484     Can't open /var/run/nut/apc_modbus-apc: No such file or directory
   0.000487     [D1] Request for other driver to exit returned code -1
   0.000489     [D1] Socket dialog with the other driver instance (may be absent) failed: No such file or directory
   0.000493     [D5] send_to_all: SETINFO device.type "ups"
   0.000496     [D5] send_to_all: SETINFO driver.state "init.device"
   0.024789     [D3] match_function_regex: matching a device...
   0.024816     [D2] _apc_modbus_usb_callback: Matched American Power Conversion  Smart-UPS_2200 FW:UPS 18.0 / ID=1015 (USB VID/PID 051d:0003)
   0.024820     [D2] _apc_modbus_usb_callback: Checking American Power Conversion  Smart-UPS_2200 FW:UPS 18.0 / ID=1015 (USB VID/PID 051d:0003) report descriptors
   0.024871     [D1] _apc_modbus_usb_callback: Found report ids RX=0x90 TX=0x89
   0.024877     [D5] send_to_all: SETINFO ups.vendorid "051d"
   0.024879     [D5] send_to_all: SETINFO ups.productid "0003"
   0.035003     [D5] send_to_all: SETINFO driver.state "init.quiet"
   0.035013     [D5] send_to_all: SETINFO driver.version "2.8.2.2806-2806+gef66c3c8f"
   0.035017     [D5] send_to_all: SETINFO driver.version.internal "0.12"
   0.035020     [D5] send_to_all: SETINFO driver.name "apc_modbus"
   0.035023     [D5] send_to_all: SETINFO driver.state "init.info"
   0.539698     _apc_modbus_read_registers: Read of 516:636 failed: Connection timed out (auto)
   1.550002     Can't read inventory information from the UPS
   1.550023     [D5] send_to_all: SETINFO driver.state "cleanup.upsdrv"
   1.550161     [D5] send_to_all: SETINFO driver.state "cleanup.exit"
   1.550166     upsnotify: failed to notify about state NOTIFY_STATE_STOPPING: no notification tech defined, will not spam more about it
   1.550168     [D1] On systems without service units, consider `export NUT_QUIET_INIT_UPSNOTIFY=true`

Seems the driver fails to read from the UPS but, with several attempts, sometimes it gets a response... grrr... I seem to be on the right track. Any help would be appreciated.

I've been trying USB but, if necessary, I can try modbus over TCP too...

[jimklimov]

That seems related to #2609 which is so far unresolved, sadly. I've investigated that for some time previously, but still can't point a finger at which code base is at fault - libmodbus core (I did have a fun time trying to get its own test suite to pass reliably, though mostly did so by increasing timeouts and retrying in a few cases, so maybe some timing issues are in there already), or additions for RTU USB contributed by @EchterAgo, or the NUT apc_modbus driver using those.

[jimklimov]

Actually, if you can run tests with Modbus over TCP or Serial, and can do so with the stock packaged libmodbus (and an apc_modbus rebuilt to not require USB ability), that can in fact be helpful to differentiate which code base is the first in the stack to have flaws.

In case of USB devices in #2609 and other issues linked from it, seems that sometimes it sort-of worked for significant time, but then after a NUT restart or system reboot it no longer connected. I suppose there may be also "survivor bias" of people for whom it all works reliably so they never posted.

Then there's always a possibility that the devices are flawed (their controllers), at least APC-branded boxes of other lines did have peculiar behaviors over the years. They've bought up too many other vendors it seems, so have a wild mix of protocols and qualities good and bad published under same brand. But that seems to be the story with many long-term names on the market.

MAYBE if the UPS controller(s) got wedged, some sort of controller restart can help; but this might range from USB bus reset (maybe even via software or by re-plugging the cable, depending on USB hub chipset capabilities), to UPS power-off, delay or even battery unplug, and restart.

[KartmanCanada]

I need a but of guidance.

./configure --with-drivers=apc_modbus  --with-modbus \
    --with-user=root --with-group=nut \
    --datadir=/usr/share/nut \
    --sysconfdir=/etc/nut \
    --with-statepath=/var/run/nut \
    --with-pidpath=/var/run/nut \
    --with-altpidpath=/var/run/nut \
    --with-drvpath=/lib/nut \
    --with-systemdsystemunitdir=/usr/lib/systemd/system \
    --with-pkgconfig-dir=/usr/lib/pkgconfig \
    --with-udev-dir=/usr/lib/udev \
    --libdir=/usr/lib/

Configured using above then "make". I copied the new apc_modbus driver to /lib/nut. Then used the following in ups.conf file:

pollonly = true
pollinterval = 15
maxretry = 3

offdelay = 120
ondelay = 240

[apc]
# APC Smart-UPS SMT2200RM2
driver = apc_modbus
porttype = tcp
port = 192.168.0.249:502
slaveid = 1
desc = "APC Smart-UPS SMT2200RM2"
vendorid = 051D
productid = 0003
serial = XXXXXXXXX

Below is the output from the syslog:

Apr 15 19:22:09 pve01 nut-monitor[2485]: Poll UPS [apc@localhost] failed - Driver not connected
Apr 15 19:22:10 pve01 nut-driver@apc[232792]: Network UPS Tools 2.8.2.2806-2806+gef66c3c8f (development iteration after 2.8.2) - NUT APC Modbus driver with USB support 0.12
Apr 15 19:22:10 pve01 nut-driver@apc[232792]: modbus_connect: unable to connect: Connection refused
Apr 15 19:22:10 pve01 nut-driver@apc[232792]: upsnotify: failed to notify about state NOTIFY_STATE_STOPPING: no notification tech defined, will not spam more about it
Apr 15 19:22:10 pve01 nut-driver@apc[232683]: Driver failed to start (exit status=1)

[jimklimov]

NUT APC Modbus driver with USB support 0.12

Apparently it did use the custom libmodbus on the build system (with usb), so not a clean experiment in terms of ruling out if those changes add the instability.

The driver program did load on the new system though, so dynamic linker did not complain about missing symbols: either same custom libmodbus.so is available there, or the program has linked statically with libmodbus.a during build and directly carries a copy (has no requirement that the library is installed on final system nor conflicts with a non-usb stock one if present) which is in line with one of build approaches documented on wiki.

modbus_connect: unable to connect: Connection refused

Is the TCP server enabled on UPS, are IP and port numbers correct?

[jimklimov]

The vendorid, productid (and likely serial) options are only used for usb matching. Probably harmless (ignored) in TCP mode, but may be better to comment them away.

What would higher debug level show for driver startup? (e.g. using debug_min=6 in ups.conf while investigating)?

[KartmanCanada]

Sorry for the delay... I had an issue with my AP9630. It's been replaced and fully updated.

So... I tried the following and the driver seems to start successfully, connects to my UPS, and I can see info/status updates from the UPS on the console! This is good, me thinks.

/lib/nut/apc_modbus -DDDDD -s apc -x porttype=tcp -x port=192.168.0.192

I then edited /etc/nut/ups.conf to have only the following:

pollinterval = 15
offdelay = 120
ondelay = 240

debug_min = 6

[apc]
# APC Smart-UPS SMT2200RM2
driver = apc_modbus
porttype = tcp
port = 192.168.0.192
desc = "APC Smart-UPS SMT2200RM2"

logs look good except:

May 02 10:01:49 pve01 nut-monitor[2482]: Poll UPS [apc@localhost] failed - Driver not connected
May 02 10:01:51 pve01 nut-server[2345251]: Can't connect to UPS [apc] (apc_modbus-apc): Permission denied
May 02 10:01:51 pve01 upsd[2345251]: Can't connect to UPS [apc] (apc_modbus-apc): Permission denied

Other than the ups.conf changes, I've not changed anything and USD-HID driver didn't have this issue... UPSD seems to be owned by "nut" but APC_MODBUS and UPSMON seem appear to be owned by "root".

[jimklimov]

@KartmanCanada : the permission part means you should check upsd.users vs. upsmon.conf for the login/pass with an upsmon role.

[KartmanCanada]

Nope... I switched back to USB connection and check driver ownership. It was "nut"... reconfigured APC_MODBUS for WITH-USER= "nut" and recompiled. Seems to be functioning now using TCP connection. This is OK and I'm not looking to make trouble but the goal was to swap to modbus to get access to additional UPS parameters that were not visible via USB_HID.

success but i'm still interested in getting modbus working over USB... not a panic...