PowerWalker UPS, NUT, and Home Assistant

I’m using a Raspberry Pi to communicate with the power equipment in my homelab rack; the UPS, ATS and PDU. For the UPS, I’m using NUT (Network UPS Tools), and for the ATS and PDU; my own PowerWalker Python library.

In this post I’ll be going through how I’ve set up the NUT server to communicate with my PowerWalker VI 1500 RT HID UPS, and NUT clients running on all other physical machines in the rack.

Raspberry Pi

I wanted to run the NUT server on a dedicated machine, preferably one that doesn’t use a lot of power. The NUT server notifies all other machines that the UPS is running low on battery, which causes them to shutdown. It then waits for all NUT clients to disconnect, before initiating its own shutdown.

This means that the NUT server is typically the last machine to shutdown, and with the UPS low on battery — it makes sense to not use too much power 🙂

UPS

I’m communicating with my UPS in two ways: via USB, and Ethernet using an NMC card. The NMC card adds an Ethernet port and some logic directly to the UPS; like the ability to send e-mails, SNMP, Wake-on-LAN, etc. All managed through a web interface.

The NMC card exposes a few more UPS parameters; like battery temperature, so I’m using this to provide data to Home Assistant. As the USB connection doesn’t rely on a working network to talk to the UPS; I’m using this as input to upsmon.

Not sure I would buy the NMC card again — it’s pretty expensive, and the features aren’t amazing. I think the USB connection is enough.

The NUT server can also issue a shutdown command to the UPS itself, after all clients have disconnected. But I haven’t been able to get that to work on my UPS.

Looking at the supported commands — there are none on the USB interface:

$ upscmd -l ups
Instant commands supported on UPS [ups]:

It does support a few commands on the NMC card interface:

$ upscmd -l pw
Instant commands supported on UPS [pw]:

beeper.disable - Disable the UPS beeper
beeper.enable - Enable the UPS beeper
beeper.mute - Temporarily mute the UPS beeper
load.off - Turn off the load immediately
load.on - Turn on the load immediately
test.battery.start - Start a battery test
test.battery.start.deep - Start a deep battery test
test.battery.start.quick - Start a quick battery test
test.battery.stop - Stop the battery test

A testing mode shutdown shows that the kill command is send on both the USB and NMC interface:

$ sudo upsdrvctl -t shutdown
Network UPS Tools - UPS driver controller 2.7.4
*** Testing mode: not calling exec/kill
   0.000000
If you're not a NUT core developer, chances are that you're told to enable debugging
to see why a driver isn't working for you. We're sorry for the confusion, but this is
the 'upsdrvctl' wrapper, not the driver you're interested in.

Below you'll find one or more lines starting with 'exec:' followed by an absolute
path to the driver binary and some command line option. This is what the driver
starts and you need to copy and paste that line and append the debug flags to that
line (less the 'exec:' prefix).

   0.001999	Shutdown UPS: ups
   0.002250	exec:  /lib/nut/usbhid-ups -a ups -k
   0.002495	Shutdown UPS: pw
   0.002548	exec:  /lib/nut/snmp-ups -a pw -k

I don’t know why my UPS isn’t shutting down. Testing UPS and whole rack shutdown is a bit painful, and not something I do very often…

I typically find that I care a lot about the shutdown process right after a power outage, and then less and less as time passes — until I forget it completely 😛

Anyway — the most important thing for me is that all machines turn off properly should a power outage drain the UPS of battery, and that does work 👍

NUT shutdown design

To better understand what happens during a shutdown, here is the procedure from the NUT user manual:

1. The UPS goes on battery
2. The UPS reaches low battery (a “critical” UPS), that is to say upsc displays: ups.status: OB LB
   • The exact behavior depends on the specific device, and is related to:
     • battery.charge and battery.charge.low
     • battery.runtime and battery.runtime.low
3. The upsmon master notices and sets “FSD” - the “forced shutdown” flag to tell all slave systems that it will soon power down the load.
4. upsmon slave systems see “FSD” and:
   • generate a NOTIFY_SHUTDOWN event
   • wait FINALDELAY seconds - typically 5
   • call their SHUTDOWNCMD
   • disconnect from upsd
5. The upsmon master system waits up to HOSTSYNC seconds (typically 15) for the slaves to disconnect from upsd. If any are connected after this time, upsmon stops waiting and proceeds with the shutdown process.
6. The upsmon master:
   • generates a NOTIFY_SHUTDOWN event
   • waits FINALDELAY seconds - typically 5
   • creates the POWERDOWNFLAG file - usually /etc/killpower
   • calls the SHUTDOWNCMD
7. On most systems, init takes over, kills your processes, syncs and unmounts some file systems, and remounts some read-only.
8. init then runs your shutdown script. This checks for the POWERDOWNFLAG, finds it, and tells the UPS driver(s) to power off the load.
9. The system loses power.
10. Time passes. The power returns, and the UPS switches back on.
11. All systems reboot and go back to work.

NUT server

The role of the NUT server is to communicate directly with the UPS, and share this information with the connected clients.

Packages installed:

• nut
• nut-client
• nut-server
• nut-snmp (for talking with the NMC card)

Configuration files

The configuration files are located in /etc/nut. All files are explained in the NUT manual pages.

nut.conf

Make the server — a server.

MODE=netserver

ups.conf

I initially had problems with communication lost when using the USB interface on the PowerWalker UPS, but adding the properties below fixed that.

maxretry = 3
retrydelay = 5
pollinterval = 5
synchronous = yes

[ups]
  driver = usbhid-ups
  port = auto
  vendorid = 06da

[pw]
  driver = snmp-ups
  port = 192.168.1.x
  community = private
  snmp_version = v2c
  synchronous = no
  pollfreq = 15

upsd.conf

upsmon should listen both on localhost and LAN IP.

MAXAGE 30
LISTEN 127.0.0.1
LISTEN 192.168.1.x

upsd.users

Define two users; the master user can issue all commands.

[upsmaster]
  password = secret
  upsmon master
  instcmds = ALL

[upsremote]
  password = secret
  upsmon slave

upsmon.conf

Monitor ups on localhost, log in with the master user.

All events should log to syslog and broadcast to the terminal of all logged-in users. For all events, except NOPARENT; the shell script /etc/nut/notify.sh should be executed (see below).

See the NUT user manual for a description of all properties.

MONITOR ups@localhost 1 upsmaster secret master
MINSUPPLIES 1
SHUTDOWNCMD "/sbin/shutdown -h +0"
POLLFREQ 5
POLLFREQALERT 5
HOSTSYNC 15
DEADTIME 30
POWERDOWNFLAG /etc/killpower

NOTIFYCMD /etc/nut/notify.sh
NOTIFYFLAG ONLINE   SYSLOG+WALL+EXEC
NOTIFYFLAG ONBATT   SYSLOG+WALL+EXEC
NOTIFYFLAG LOWBATT  SYSLOG+WALL+EXEC
NOTIFYFLAG FSD      SYSLOG+WALL+EXEC
NOTIFYFLAG COMMOK   SYSLOG+WALL+EXEC
NOTIFYFLAG COMMBAD  SYSLOG+WALL+EXEC
NOTIFYFLAG SHUTDOWN SYSLOG+WALL+EXEC
NOTIFYFLAG REPLBATT SYSLOG+WALL+EXEC
NOTIFYFLAG NOCOMM   SYSLOG+WALL+EXEC
NOTIFYFLAG NOPARENT SYSLOG+WALL

RBWARNTIME 43200
NOCOMMWARNTIME 900
FINALDELAY 5

The notify.sh script sends the UPS event via e-mail.

#!/bin/bash

EMAIL='admin@awesome.net'
printf "UPS: $UPSNAME\r\nAlert type: $NOTIFYTYPE" | mail -s "NUT ALERT: $NOTIFYTYPE" $EMAIL

Rebooting issues

When the Raspberry Pi is rebooted, the UPS driver doesn’t always reconnect. Not sure why, but since I don’t reboot it very often — it’s not something I’ve spend time looking into.

Restarting the driver and NUT server fixes the issue:

sudo upsdrvctl stop
sudo upsdrvctl start
sudo systemctl restart nut-server

To verify that the driver is working you can query the UPS for its status:

upsc ups
upsc pw

NUT clients

The clients connect to the server over the network, and receives shutdown notifications should the UPS run low on battery.

Packages installed:

• nut-client

Configuration files

The configuration files are located in /etc/nut. All files are explained in the NUT manual pages.

nut.conf

This time we define it as a client.

MODE=netclient

upsmon.conf

Monitor ups on the Raspberry Pi, log in with the remote user.

No notifications are configured here, only on the server.

See the NUT user manual for a description of all properties.

MONITOR ups@powerpi.lan.uctrl.net 1 upsremote secret slave
MINSUPPLIES 1
SHUTDOWNCMD "/sbin/shutdown -h +0"
POLLFREQ 5
POLLFREQALERT 5
HOSTSYNC 15
DEADTIME 30
POWERDOWNFLAG /etc/killpower

RBWARNTIME 43200
NOCOMMWARNTIME 300
FINALDELAY 5

Ansible playbook

To make it easy to install and configure NUT client on new machines; I’ve made a simple Ansible playbook.

Playbook file structure:

nut-client.yml
roles/nut-client/tasks/main.yml
roles/nut-client/templates/nut.conf
roles/nut-client/templates/upsmon.conf

The template files are the ones listed above — under NUT clients.

nut-client.yml

- hosts:
    - servers
  vars:
    ansible_python_interpreter: /usr/bin/python3
  roles:
    - nut-client

roles/nut-client/tasks/main.yml

---
- name: Install nut-client
  become: true
  package: name=nut-client state=latest

- name: Copy nut.conf
  become: true
  template:
    src: "nut.conf"
    dest: "/etc/nut/nut.conf"

- name: Copy upsmon.conf
  become: true
  template:
    src: "upsmon.conf"
    dest: "/etc/nut/upsmon.conf"

- name: Restart nut-client
  become: true
  service: name=nut-client state=restarted

Home Assistant

Implementing NUT in Home Assistant is very easy, it even has its own integration.

You need to enter the host, port, username, and password to the NUT server. As we defined it in upsd.users.

The UPS is then added as a device, showing UPS stats 🙂

The runtime of the UPS is reported in seconds. To make it a bit more readable, I added a template sensor to convert it to minutes:

template:
  - sensor:
    - name: pw_battery_runtime_min
      unit_of_measurement: 'min'
      state: "{{ (states.sensor.pw_battery_runtime.state | int / 60) | round(1) }}"

UPSes don’t (typically) report power usage in watts or amps, but they do report the load in percentage. You can make a template sensor to approximate the power usage, using the specified capacity and load. Something like this, where 1350 is the rated capacity of the UPS:

template:
  - sensor:
    - name: pw_approx_power
      unit_of_measurement: 'W'
      state: "{{ ((states.sensor.pw_load.state | int / 100) * 1350) | round(0) }}"
      device_class: power

Ending thoughts

I started writing this post to help a friend to interface his PowerWalker UPS with NUT and Home Assistant. While writing I realized that I don’t fully understand the shutdown sequence, or capability, of the UPS itself.

I used to have an APC UPS, and it shut itself down without any special configuration. I think the more “advanced” UPS commands is something you get with more expensive UPS devices.

It would be interesting to learn and understand more about how the UPS is suppose to shut itself down and power back up. I may further explore this in the future 🙂