Feature request: bcmxcp_usb driver to support more than one UPS

[kyuferev]

Hi.
Right now according to official documentation bcmxcp_usb driver supports only one UPS at a time. Unfortunately I have lots of Eaton PW9120 6000i in my disposal and a task to monitor all of them.

According to this comment there is (or was?) a plan/idea to implement a serial number differentiantion between multiple UPSes. At the same time usbhid-ups driver already has this functionality.

I've been poking around this issue for quite some time now and haven't found any working solution (except for obvious 1 UPS = 1 Raspberry Pi). Is there a way to implement this feature for bcmxcp_usb driver or maybe modify usbhid-ups driver to work with Eaton PW9120 6000i? Or maybe I'm missing something and there is some kind of a quick workaround for this issue?

FYI:

# lsusb -v -s 001:009

Bus 001 Device 009: ID 06da:0002 Phoenixtec Power Co., Ltd UPS
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               1.10
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0         8
  idVendor           0x06da Phoenixtec Power Co., Ltd
  idProduct          0x0002 UPS
  bcdDevice            0.06
  iManufacturer           4 Phoenixtec Power
  iProduct               38 UPS USB Interface (V2.00)
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           34
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xa0
      (Bus Powered)
      Remote Wakeup
    MaxPower               20mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              0 
      ** UNRECOGNIZED:  09 21 00 01 21 01 22 51 00
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval              20
Device Status:     0x0000
  (Bus Powered)

# upsc <ups-name>
Init SSL without certificate database
ambient.temperature: 23
ambient.temperature.high: 40
battery.charge: 100
battery.charger.status: floating
battery.runtime: 1344
battery.runtime.low: 2
battery.voltage: 284.0
device.mfr: Eaton
device.model: PW9120 6000i
device.part: 1018290         
device.serial: RV442A0674      
device.type: ups
driver.name: bcmxcp_usb
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.parameter.synchronous: no
driver.version: 2.7.4
driver.version.internal: 0.31
input.bypass.voltage: 227
input.frequency: 50.0
input.frequency.high: 52
input.frequency.low: 48
input.frequency.nominal: 50
input.transfer.boost.high: 176
input.transfer.high: 276
input.transfer.low: 160
input.transfer.trim.low: 253
input.voltage: 225
input.voltage.nominal: 220
output.current:  6.7
output.current.nominal: 27.0
output.frequency: 50.0
output.frequency.nominal: 50
output.phases: 1
output.voltage: 222
output.voltage.nominal: 220
ups.alarm: UPS_CABINET_OVER_TEMP TRANSFORMER_OVER_TEMP CABINET_DOOR_OPEN_VOLT_PRESENT AUTOMATIC_STARTUP_PENDING MODEM_CONNECTION_ESTABLISHED
ups.beeper.status: enabled
ups.firmware: Cont:05.20 Inve:05.20
ups.load:  23.8
ups.mfr: Eaton
ups.model: PW9120 6000i
ups.power: 1430
ups.power.nominal: 6000
ups.realpower: 1354
ups.serial: RV442A0674      
ups.status: ALARM OL
ups.test.result: Done and passed

# upsdrvctl start <second bcmxcp_usb ups-name>
Network UPS Tools - UPS driver controller 2.7.4
Network UPS Tools - BCMXCP UPS driver 0.31 (2.7.4)
USB communication subdriver 0.22
Communications with UPS lost: Error executing command
Could not communicate with the ups: Device or resource busy
CLOSING

Driver failed to start (exit status=1)

# /lib/nut/bcmxcp_usb -a <second bcmxcp_usb ups-name> -DDD
Network UPS Tools - BCMXCP UPS driver 0.31 (2.7.4)
USB communication subdriver 0.22
   0.000000     debug level is '3'
   0.003664     entering nutusb_open()
   0.005690     device 010 opened successfully
   0.005974     Can't claim POWERWARE USB interface: could not claim interface 0: Device or resource busy
   0.006213     Can't reset POWERWARE USB endpoint: could not clear/halt ep 129: Device or resource busy

[clepple]

The major difference is that usbhid-ups is matching serial numbers at the time that the USB code scans the bus (before the device is opened). In the lsusb output above, your UPS has iSerial: 0, which means that the ups.serial value is coming from somewhere other than the descriptors visible during the bus scan. I am not very familiar with the bcmxcp* code, but it looks like the serial number is retrieved from the UPS after it is opened/claimed. We have a similar issue with the tripplite_usb driver.

It may be possible to change the driver bus scan logic to treat the Can't claim POWERWARE USB interface error as a soft failure, and continue scanning for the next available UPS. Then, once the device is opened, the serial number could be compared. However, this isn't a quick fix, and we would want to resolve some of the decisions being made in issue #300, which involves a rewrite of the USB API code, before modifying this driver.

A potentially easier method would be for you to add in some of the USB bus matching code, if each UPS can be made to appear on a different USB bus (possibly by adding a USB hub upstream of the UPS). This is not guaranteed to work, and may be fragile if the bus numbers are different after reboot, but it may be sufficient for some purposes. The tripplite_usb driver uses a regex matcher, though it was intended to distinguish between different UPS models (and it sounds like all of your units are the same model).

Another hardware-based method might be to switch to using the serial port on each UPS, and find a set of USB-to-serial adapters that have unique serial numbers as shown in lsusb. Then, you can set up a udev rule to make a symlink for each converter that includes the serial number (e.g. /dev/ttyUSB.SN12345) and pass that to the port option for the serial bcmxcp driver. Let us know if you need more information on this idea. (If you have more than a handful of UPSes, you may also be able to use a single multi-port "USB serial port multiplexer" or a network-based terminal server to accomplish the same task for a lower cost.)

[kyuferev]

Thanks for the quick and detailed response.
I've tried to connect my UPSes via USB-to-serial adapter and came to some kind of a success. At first I've not been able to recieve information from connected Eaton PW9120 6000i unit because of a permission problem:

upsdrvctl[7119]: Fatal error: unusable configuration
upsdrvctl[7119]: Network UPS Tools - BCMXCP UPS driver 0.31 (2.7.4)
upsdrvctl[7119]: RS-232 communication subdriver 0.20
upsdrvctl[7119]: Unable to open /dev/ttyUSB0: Permission denied
upsdrvctl[7119]:   Current user id: nut (109)
upsdrvctl[7119]: Serial port owner: root (0)
upsdrvctl[7119]: Serial port group: dialout (20)
upsdrvctl[7119]:      Mode of port: 0660
upsdrvctl[7119]: Things to try:
upsdrvctl[7119]:  - Use another port (with the right permissions)
upsdrvctl[7119]:  - Fix the port owner/group or permissions on this port
upsdrvctl[7119]:  - Run this driver as another user (upsdrvctl -u or 'user=...' in ups.conf).
upsdrvctl[7119]:    See upsdrvctl(8) and ups.conf(5).
upsdrvctl[7119]: Driver failed to start (exit status=1)

I've tried to fix it by adding user = root directive into ups.conf and nut-driver service has been able to start but then nut-server service complained about permission problem: upsd[6170]: Can't connect to UPS <ups-name> (bcmxcp-<ups-name>): Permission denied. I've also tried to chown root:nut /dev/ttyUSB0 but this fix didn't survive between reboots. Then I've come to a quick-n-dirty solution usermod -aG dialout nut and it helped for a while.

Then I've connected second Eaton PW9120 6000i unit and I've been able to recieve some information from it but after nut-driver service restart I've caught new error:

bcmxcp[9145]: Communications with UPS lost: Error executing command
bcmxcp[9145]: Short read from UPS

And from nut-server service:

upsd[10113]: Connected to UPS <second-ups-name>: bcmxcp-<second-ups-name>
upsd[10114]: Startup successful
systemd[1]: Started Network UPS Tools - power devices information server.
upsd[10114]: Data for UPS <second-ups-name> is stale - check driver

I've seen this error before and tried to fix it:

# upsdrvctl start <second-ups-name>
Network UPS Tools - UPS driver controller 2.7.4
Network UPS Tools - BCMXCP UPS driver 0.31 (2.7.4)
RS-232 communication subdriver 0.20
Duplicate driver instance detected! Terminating other driver!
Connected to UPS on /dev/ttyUSB1 with baudrate 9600
# upsc <second-ups-name>
Init SSL without certificate database
ups.status: WAIT

And after some time:

# upsc <second-ups-name>
Init SSL without certificate database
Error: Data stale

I've googled a bit and found this thread and tried to apply suggested fix by adding pollinterval = 5 directive into ups.conf but no success. From nut-driver service:

bcmxcp[10869]: Warning: excessive comm failures, limiting error reporting
bcmxcp[10869]: Communications with UPS lost: Error executing command
bcmxcp[10869]: Short read from UPS

And from nut-server service:

upsd[10942]: UPS <second-ups-name> data is no longer stale
upsd[10942]: Data for UPS <second-ups-name> is stale - check driver
upsd[10942]: UPS <second-ups-name> data is no longer stale
upsd[10942]: Data for UPS <second-ups-name> is stale - check driver

And at the same time the first connected Eaton PW9120 6000i unit works well and I'm able to receive all the information needed. I'm a bit puzzled by this situation and pretty sure that there is more simple and elegant solution. Any advices/guidelines are appreciated.

FYI1: I've also set baud_rate parameter to 9600 for both units.
FYI2: lsusb -v -s still doesn't return iSerial parameter:

# lsusb -v -s 001:010

Bus 001 Device 010: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               1.10
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x067b Prolific Technology, Inc.
  idProduct          0x2303 PL2303 Serial Port
  bcdDevice            4.00
  iManufacturer           1 Prolific Technology Inc. 
  iProduct                2 USB-Serial Controller D
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           39
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           3
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x000a  1x 10 bytes
        bInterval               1
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval               0
Device Status:     0x0000
  (Bus Powered)

[clepple]

And at the same time the first connected Eaton PW9120 6000i unit works well and I'm able to receive all the information needed. I'm a bit puzzled by this situation and pretty sure that there is more simple and elegant solution. Any advices/guidelines are appreciated.

I would definitely check the cables and adapters, and see whether the problem follows the cables, or if it might be with the second UPS itself. If the cables and adapters that worked with the first UPS still cause problems with the second UPS (with the first disconnected, just to be sure), I think that's worth taking up with Eaton's service department.

There is a small chance that there is a negative interaction between the two UPSes when they are both connected, but with a baud rate of 9600, I can't imagine the USB bandwidth is saturated.

FYI2: lsusb -v -s still doesn't return iSerial parameter:

Right, not all USB-to-serial converters have iSerial strings. It's disappointing, because they usually have a configuration EEPROM to store the numeric idVendor:idProduct combination, and there is frequently room for strings at the end (as seen after the iProduct and iManufacturer fields). (As you may have guessed, iSerial: 0 means that there is no string there, since String Descriptor #0 contains a language ID rather than an actual string.)

[kyuferev]

I would definitely check the cables and adapters

I've got the same idea and went to double-check cables and connections. It turns out that second UPS cable was not properly connected. I've fixed it in place and then restarted nut-driver and nut-server services. Right now everything works fine but just to be sure I'll keep pollinterval = 5 in ups.conf.

Sidenote: I've possibly found a better solution for "permission denied" problem in this article:

Create the file /etc/udev/rules.d/99_nut-serialups.rules:
# cat /etc/udev/rules.d/99_nut-serialups.rules
KERNEL=="<device name>", GROUP="nut"
# udevadm control --reload-rules
# udevadm trigger

I hope this workaround/solution will help someone with the similar problem. I'll polish it a bit before deploying into production. I'm also looking forward to a solid software solution (i.e. bcmxcp_usb driver to support more than one UPS). Thanks alot for the help, keep up the good job! This issue can be closed, I guess.

[RoganDawes]

Just as a suggestion, you could specify the particular bus/port combination to bind the USB driver to, if there are multiple devices present. You'd still need to deal with allowing multiple usb driver instances to coexist, though.

[jimklimov]

Note: #1763 addressed a similar issue in another driver, with #1764 logged to follow up for bcmxcp drivers which are structured differently.