WJCarpenter's pages

In the Spring of 2023, a mallard duck hen made a nest in our yard. We knew that meant ducklings in a few weeks, so I scrambled to figure out how to stream it for the enjoyment of friends, neighbors, and strangers. Of course, I don't mean the kind of stream where you might expect to find a duck. I mean a video stream. It's all over now, and the ducks have gone. Here is my Youtube playlist of the highlights.

I already had a few video cameras scattered about watching the outdoors. Even though we live in a suburb, and even though we have a fenced yard, we do get occasional visits from wildlife: cats, raccoons, cats, a possum, cats, squirrels, cats, and birds. None of those cameras cover the area where the duck nested (I'm pretty sure she didn't think about that when she picked the spot, but who knows). None of my current cameras were suitable for exposure to the amount of rain we're getting this Spring, and none of them have a big friendly button with a label liike "click here to stream to the universe".

This is a write-up of the ultra-fast learning I did to get my live stream going. 

I had to work fast because we didn't know how long the duck had already been nesting when we found out about it. A duck will lay eggs for about 10 days before starting incubation. Then all of the eggs incubate for about 28 days. All of the ducklings will hatch within a few hours of each other. Ducklings are "precocial". The mother duck does not actually feed them directly. Instead, within a day or so of hatching, she leads them to some place where they can feed themselves.

The first thing I did was order an outdoor surveillance camera. My criteria were:

• at least full HD (1080p) resolution
• fast delivery
• cheap, cheap, cheap
• video stream accessible by 3rd party tools

Fast delivery and cheap requirements funnelled me to the many, many no-name Chinese cameras on Amazon that could arrive next day or the day after. Without time to really study the matter, 3rd party access to me meant a camera supporting Real Time Streaming Protocol  (RTSP). I later discovered that ONVIF is a sort of profile of conventions and discovery that uses RTSP. I only know a little bit about video codecs, but I had a hunch that H.264 was a good thing to have, and that H.265 was OK but not great to have (for reasons not completely clear to me). Most of the no-name Chinese cameras don't say anything about any of that stuff in their descriptions, but a few do. Most are locked into some proprietary access scheme developed by a chipset maker and need a dedicated mobile app to view the videos.

I'm not going to bother mentioning the particular model of camera that I got because they come and go anyhow, and it might not even be available by the time you read this. I'll just say that the Amazon description did explicitly mention RTSP support, and it met my other requirements. I'll also mention that the mobile app to configure it was Tuya Smart, which I was already using for other reasons.

OK, now I had the camera, and I'm skipping the step about running a long orange extension cord and mounting the camera to my fence with an amazing variety of boards and clamps. I was able to get the camera configured in the Tuya Smart app so that I could access the video stream over WiFi. How to get from there to live streaming?

One of the first things I did, after some Google searching, was install the Agent DVR application from ISpyConnect. This application is pretty cool and supports a huge variety of IP cameras. Since my camera supported ONVIF and RTSP, Agent DVR was able to work out the connection particulars with no trouble at all. Once that was working, it was also willing to tell me what it had discovered about RTSP streams. I found that my camera had a main stream of 1080p and a secondary stream of VGA (640x480). Agent DVR does have the ability to read an RTSP stream and relay it out to a server for live streaming, but that feature is not available in the free, unlicensed version. (That...

I'm surely not the first person to figure this out, but I didn't come across this solution while searching around. They are probably out there and I just didn't find them.

The problem, in a nutshell, is how to reliably map USB devices into a docker container. The more specific problem, for me, is that I run the Home Assistant docker container (with docker-compose) and want to map in the USB ports for my Zigbee and Z-Wave interfaces. (I use the popular HUSBZB-1 combo dongle, which slightly complicates things, but that's incidental.) Things were pretty easy when I could always count on them being /dev/ttyUSB0`and /dev/ttyUSB1 on the host machine. When I started using an unrelated USB device on the same host machine, things turned into a little dance of plugging, unplugging, restarting containers, pulling out hair, etc.

It's easy to find solutions for creating reliable, persistent names for USB devices on Linux. The answer is to use something in udev rules. udev even has a really easy way to create a symlink with whatever fixed name you choose. It's very simple, and you will find zillions of internet articles describing how to do it. It doesn't take long to discover, however, that docker can't cope with mapping those host machine symlink names to a container device name. When you start looking for solutions to that problem, you find that it's been a docker sore spot for years and years.

The solution is to use hard links instead of symlinks. It's not particularly difficult, but it's also not particularly obvious if your only exposure to udev is from reading canned recipes on various web pages.

Here's how I did it. I have 3 USB devices of interest: the Zigbee and Z-Wave devices mentioned, and a 3D printer that I use with Octoprint in an unrelated docker container. There are many articles around to tell you how to find the internal details (vendor, serial number), etc of USB devices for use with udev, so I won't go into that here. However, I will direct your attention to the very useful pseudo-directory /dev/serial/by-id/. (There is also /dev/serial/by-path/. You don't want that one.) Without you doing anything, it will contain symlinks to your serial USB devices. Here's part of mine:

lrwxrwxrwx 1 root root 13 Sep  1 18:18 usb-Silicon_Labs_HubZ_Smart_Home_Controller_813004BE-if00-port0 -> ../../ttyUSB2
lrwxrwxrwx 1 root root 13 Sep  1 18:18 usb-Silicon_Labs_HubZ_Smart_Home_Controller_813004BE-if01-port0 -> ../../ttyUSB3

For a given device, the name of the symlink is always going to be the same (at least as far as I can tell). If docker could map symlinks to devices, that would be all you would need for reliable names. But, alas. There are various Linux utilities for dereferencing symlinks, and the one we want to use here is realpath. It provides an absolute path to the real file. (A commonly-suggested alternative, readlink, would only give you the relative path mentioned in the symlink, which is not directly helpful.)

$ realpath /dev/serial/by-id/usb-Silicon_Labs_HubZ_Smart_Home_Controller_813004BE-if00-port0
/dev/ttyUSB2

Armed with that, we can now stitch it into udev rules. If you want to know more about udev rules, there are plenty of explanations available, so this is going to be just a distilled description of how to apply them. The persistent names I use are /dev/ttyUSB-ender3v2, /dev/ttyUSB-zigbee, and /dev/ttyUSB-zwave. Those are the hard links I want to end up with. Here is a tiny shell script that maps one of the above symlinks to the applicable hard link, depending on its single command line argument:

#!/bin/bash

p=""
if [ "$1" == "ender3v2" ]
then
  p=`realpath /dev/serial/by-id/usb-1a86_USB_Serial-if00-port0`
elif [ "$1" == "zwave" ]
then
  p=`realpath /dev/serial/by-id/usb-Silicon_Labs_HubZ_Smart_Home_Controller_813004BE-if00-port0`
elif [ "$1" == "zigbee" ]
then
  p=`realpath /dev/serial/by-id/usb-Silicon_Labs_HubZ_Smart_Home_Controller_813004BE-if01-port0`
fi

if  [...