Home Automation

I've been adding lots of additional devices to my home network. In addition to the wireless wall switches and LED bulbs, I've also added an LED strip light (with 1m extension) which runs along the inside edge of the staircase between the two floors of the house. It's really neat to be able to have those lights illuminate as soon as I hit the top of the stairs and stay on for five minutes (programmed.) Enough time to don my shoes, tie my laces and leave the house, locking up after myself. And I'm able to control all the lights (grouped into "rooms") from the Hue app running on my iPhone and iPad. Voice control is possible via Siri on the iPhone as well my Amazon Echo Dots.

History

A little historical perspective is required at this juncture. Arguably the first widespread home automation technology was known as X10. Introduced in the mid-seventies, it enjoyed somewhat limited popularity due to the nature of communications and the wiring in the standard North-American detached home. There you have a 230V feed at 60 Hz supplied on three wires: neutral, leg 1 and leg 2. The potential between each of the legs and neutral is 115V. The potential between the legs is the full 230V feed. Because X10 sends signals over the power lines, signals sent on leg 1 might not be received on leg 2 and vice versa. There are solutions available, including this coupler, but it obviously led to some frustrations for early adopters: unless all your devices were on the same leg then the commands might or might not arrive at the intended destination. Plus it was incredibly slow. Would you believe 120 bits per second?

But technology continued its relentless progress and wireless products became prevelant as the cost of hardware continued to drop. In the same way that your cellular telephone has more power than the mainframe computers of the '70s, wireless devices have shrunk in size. It's now possible to manufacture LED bulbs in the standard form factors which incorporate a wireless transceiver. They're not exactly cheap, but with some governments effectively outlawing standard (Edison) incandescent bulbs, along with significantly lower current draw, switching can still be cost-effective.

But as is often the case, companies want to distinguish themselves and "protect their turf." Philips has a marvelous system for lighting control known as Hue. Under the covers it uses the ZigBee protocol, but only a subset, namely the Light Link. If you want to control things like appliances or electrical outlets (it can also control light switches) then you have to use a different set of products, ones which use Z-Wave. So you might end up using both a Philips Hue hub and a Z-wave controller, as I have done, in order to arrive at a complete solution. I also had a number of devices left over from my previous experimentation with X10. Here's a schematic diagram of my network:

I have a number of infrared sensors installed (MS16A) in almost every room. They communicate wirelessly with the CM15A X10 control module. That's connected via USB to the Raspberry Pi 3. The Pi also has a Razberry Z-wave board installed. That communicates wirelessly with a wall switch for the basement lights and an outlet for my reading lights. The Pi sits on the gigabit LAN which is also connected to the wireless router. That communicates with a repeater which plugs into the Philips Hue hub with a short CAT-5 Ethernet cable. The Amazon Dots connect to either the router or the repeater, depending on where in the house they're located.

These are the devices in my office. I built an acrylic box to house everything.
On the left is a wireless modem which I plan to use as a backup if the power and cable go out.
On the right is the Raspberry Pi with the Razberry board mounted on top.

I installed a shelf in the kitchen so that I would have a place for (left to right) an Amazon Dot, Philips Hue Hub and my wireless repeater.

Raspberry Pi

The Raspberry Pi 3 is a single-board computer which is only about 85x56x16mm in size but packs a big punch. It has four USB ports, an RJ-45 Ethernet port and supports WiFi (802.11n) and Bluetooth. The operating system I use is called Raspbian and is a port of Debian Linux. It's stored on a 64 GB MicroSD card so no traditional hard drive is required. If you're at all familiar with Linux (and you really should be) then you'll have no difficulty navigating this system. There's also a 40-pin GPIO (General-Purpose Input/Output) header. The required 5 volts at 2.5 amperes is provided by an external power supply, but that's the only separate part needed to get up and running.

There are some commonalities and differences in the various Linux distributions but the requisite functionality can usually be found if you root around. For example, rather than using yum to install and update packages, on Raspbian you use apt-get. Raspbian does include systemd so you can use standard service files or even use this to convert SysV init scripts to service files.

Amazon Echo Dot

If you're anything like me then you've probably been intrigued by the Amazon Echo. At close to US$200, it's an interesting device but has some capabilities which don't match my needs. The audio performance might be entirely capable but you're not going to get true stereo rendering from the built-in speaker system, impressive as it may be. So when they introduced the Echo Dot it struck me as the perfect platform for someone with more "modest" requirements. And unlike the newer Tap, it's constantly listening for the command word or phrase. It has a basic speaker to allow Alexa to communicate. Experience confirms that it is able to discern commands spoken at a conversational volume, no shouting required. In short, it's pretty amazing technology.

I'll make some admissions here. I had cancer surgery late last year which involved the excising (cutting out) of almost 2/3 of my tongue. They reconstructed it with some flesh from my forearm but that can't be expected to behave like the original muscle. It's improving over time but my speech at times, with the inability to pronounce sibilants (like the letter S,) can make me sound like a blithering idiot. So I was extremely impressed by Alexa's ability to understand me. In fact, to date there have only been a few instances of misunderstanding what I've said. The comprehension of fluid speech has, entirely honestly, "blown me away!" And I obviously take note of the details, given my decades of experience in Information Technology.

So I ordered a couple of Dots, as mentioned above, from B&H Photo Video in New York. It's a smart company which offers free shipping for orders over $99. Also smart enough to know about the "quirks" of cross-border shipping to Canada, sensibly including HST and customs brokerage fees in the total price. An excellent company to do business with, BTW, whether you're in the United States or Canada. A most impressive product catalog! I didn't mention their name before out of concern for their arrangement with Amazon. But since Amazon is now touting that they sold some nine times as many units in their Echo line in 2016 as they did the year before, I don't think that they're going to be overly concerned about a few units being shipped up to Canada, even though it's not "officially" supported here, i.e. you can't order them through amazon.ca. UPDATE: Amazon now sells the Echo in Canada.

So after working around some of the issues, I was able to load the Alexa app on my iPhone and iPad. Configuration after that was fairly straightforward, although weather reports come from Niagara Falls, New York, the closest American city. As mentioned previously, comprehension is impressive! And it integrates seamlessly (well relatively, once you get it configured) with the Philips Hue hub. I originally ordered two units so that I could take advantage of the free shipping and set up one in my office and another in my kitchen. But here's another admission: I subsequently ordered four more Dots! I've been so impressed that I now want one in almost every room in the house, and it's not as though it's a big house.

But I would be remiss in my obligations to the community if I didn't address some of the criticisms readily found on-line. I don't doubt for a moment that some people have had less-than-stellar experiences. That said, I haven't been able to find fault in the product execution. Realistically, I could have expected problems with speech recognition, for reasons outlined above. But that didn't happen. Granted, it can take some time, research and effort to make everything work seamlessly, but far less than some might have you believe. Full and fair disclosure: my knowledge and extensive experience doesn't qualify me as perhaps your typical user. That said, the investment is more than justified by the rewards.

Philips Hue

The Philips Hue system is a bit of a niche product line, concerned only with illumination. You need a hub which can control up to 63 lights. Various controls are also available, including wall switches which can replace existing ones. But they have a wide range of products, from simply white bulbs to coloured light-strips. I have a colour bulb in my dining room which I can adjust to provide the appropriate ambience. As previously mentioned, I run a light-strip down the staircase to the landing and basement. They even have a Bloom which can display in full colour. I use mine for purposes detailed below.

The Hue has a REST (REpresentational State Transfer) interface which facilitates programmatic control. There's also an app available on the App Store so that you can control the lights from your iPhone and iPad. One of the quirks of the system is that when you experience a power failure you'll return home to find all of your lights on. I'm planning on writing a daemon for the Raspberry to address this limitation. The use of the ZigBee protocol is sensible as the devices create a dynamic mesh network so commands can be relayed from source to destination.

X10

The CM15A is a control module (hence the designation) and has an antenna as well as a USB port. There was software available for Windows XP (which gives you some idea of the heritage) called Active Home and it permitted configuring schedules for controlling devices as well as using sensors to trigger events. You can still find on-line documentation which describes how to interface with the device. Although the company behind X10 in America filed for Chapter 7 bankruptcy back in 2013, new devices are still available and can be found on ebay.

Although I might still have a lamp module hanging around somewhere, my primary motivation for re-using the devices I had on-hand was the wireless infrared sensors. They work just fine and are a darn sight cheaper than the models which communicate on the other wireless systems. A Philips Hue sensor will run you about US$40 while you can pick up brand new X10 MS16As for as little as US$10 each. I even got a little creative and used black electrical tape to limit the spread of the sensor illumination beams.

The infrared sensors are placed in suitable locations such as just inside the doors to my office, bedroom and bathroom, as well as at the top of the stairs. A daemon on the Raspberry Pi listens on the USB port for the sensor reports then pushes a notification onto the message queue on Glassfish 5. Another daemon listens on the queue and executes various commands in response to the received messages.

XBee Cellular Modem

I've written another article about the Dig Xbee which you can find here so no need to repeat in here.

Integration

The core of my system is the Raspberry Pi. I'm using JMS on Glassfish 5 but could just as easily port the software to use MQTT or really any other message queue provider such as Amazon's Simple Queue Sysem (SQS) or even leverage the capabilities of Lambda. But there are a couple of other considerations when you're talking home automation. You have to worry about Internet connectivity for one thing. Even though I plan to use the XBee as a backup for that, there's also the issue of immediacy. I don't want to wait a half second for the lights to turn on when entering a room because of technological impediments. Plus I can keep my devices protected with multiple UPS systems in case the power goes out.

So we end up with a system like this:

Whoa, Nelly! You might be saying to yourself "that's an awfully complicated set-up for something which should be quite simple!" and you'd be right. But there's actually a good reason for each of the elements, derived both from my personal philosophy and experience gained from many years of working in the IT industry. Allow me to elucidate.

Specialization

One of the defining philosophies of the UNIX™ system was the concept of specialized, single-purpose utilities which could then be arranged in a variety of ways. So you have commands like wc for counting words and characters. You have sed for straight-forward stream editing. There's also a plethora of other utlities such as tr (translate,) od (octal dump) and grep (global regular-expression print, from the ed command.) Most read from standard input and write to standard output, which also means that you can chain them together easily, a mechanism facilitated by the shell. Of course you can also create shell scripts for functions you might need to perform more than once. And isn't this the essence of programming in general? You assemble individual instructions into methods and classes (or protocols or interfaces, depending on your language of choice) in order to implement an algorithm and achieve a defined goal.

I have three daemons in my implementation. Each reads from a single input and writes to a single output, even if that output is shared. One daemon reads from the X10 controller and writes the room sensor name to a JMS message queue. Another listens for SMS messages and writes the authenticated commands to the same JMS queue. The third listens on the queue and then relays commands to the Philips Hue hub. Of course I could have collaped the first two into a single daemon with a multiplexed input and have it send directly to the lighting hub. But here come a couple of important questions. What if I wanted to add another input? What if I need to add a second lighting hub?

Some support systems send e-mail alerts to an account I set up just for this purpose. I have another daemon which polls the mailbox and sends commands directly to the Philips Hue hub to control a Philips Bloom which visually indicates system status. That is, when an alert e-mail is received, the light turns on and the colour is set to red. When the "all clear" is received then the colour gets set to green. But it would be straight-forward to re-use the functionlity utilized by the other daemons, particularly since it's encapsulated in a stand-alone Java class, and simply extend the existing paradigm. That's a lot faster, easier and cheaper than extending a class which has to support multiple inputs.

And that's where the philosophy comes into play. If I was to have a monolithic application then it would necessarily take more time to debug and prove correct. Having each daemon concern itself with only a single input and output allows me to debug them separately, without impacting the rest of the system. I can even add cronjobs which control lights according to a schedule, and the daemon invoked would only need to push a message onto the JMS queue, making it simple and light-weight. I'll mention cronjobs again in the database section.

De-coupling

Many processes don't need to be performed synchronously; a prime example is sending an e-mail. The application doesn't need to wait for the SMTP process to complete and can be de-coupled from the sending process. Loosely-coupled systems are inherently more flexible. You can readily add elements to the processing pipeline or tee off the flow for logging and/or auditing purposes. Judicious component selection can also add a level of robustness to your applications. Ome of the reasons that I'm keen on message queues, whether IBM's MQ, Amazon's SQS or other variants, is that messages are typically persisted. If there's a connectivity or even a hardware issue then messages are not lost. My application is hardly "mission critical" but having Glassfish JMS provides for future flexibility.

I should mention that I originally considered named pipes but the problem there is that you're restricted to a single instance. You could then argue for a Remote Procedure Call (RPC) interface or even REST but now you're talking about taking things to an entirely different level of complexity. You'd have to configure things at the system layer or add some form of HTTP server, perhaps not Apache but maybe something simpler like JRun. But if you're going to add complexity then isn't is just as easy to install a complete J2EE solution like Glassfish? That way you can avail yourself of not just an HTTP server and servlet container but also JNDI, JavaMail and JMS functionality.

So how is this different from the monolithic solutions I was disparaging earlier? I don't have to use all the capabilities of Glassfish. In fact, my primary webserver runs Apache and Tomcat. Apache is a balazing fast HTTP server written in C, rather than Java. Tomcat is the "reference" version of a servlet container. Both services are provided by Glassfish but I prefer the use of specialized tools (see above.) But a J2EE server can also host more sophisticated web and enterprise applications. And while I'm personally fond of Glassfish, the same functionality is available on other platforms such as IBM WebSphere and BEA (now Oracle) WebLogic. If I wasn't concerned about latentcy, I could even rewrite the messaging interface module to use 'net-based solutions such as the aforementioned Amazon SQS.

Persistence

This is somewhat related the the previous discussion. While I like the staircase lights to come on when I go downstairs to take a shower in the morning, it doesn't make sense for them to be always active. If the sun is up then I don't need the staircase lights, paticularly since I have to turn them off manually. I wanted a mechanism whereby it would be possible to activate/deactivate sensors programatically. In keeping with the loose-coupling mindset, it wouldn't make sense for the X10 daemon to determine whether or not to forward commands based on the time of day. But it would make sense to perform some simple external lookup to determine whether or not the sensor was considered to be active. Note that the functionality could also have been implemented in the queue listener daemon but it makes more sense to me philosophically to implement control on the feed side of the equation.

I originally wanted to use the Berkeley db package since it's so prevelant on *NIX systems, including Raspbian, the Debian Linux port which runs on the Raspberry Pi. But my application is written exclusively in Java and I couldn't find any implementations of the simple, basic functionality available to C developers. Oracle has something they call Berkeley DB but it doesn't read/write the single files used by the original implementation. There are various other options available, including using Java Native Interface (JNI) but then you've got a combination of Java and C code to deal with and (possibly) debug when you run into issues.

It turns out that the cleanest implementation is to use MySQL along with the corresponding JDBC driver. I have a number of servers here at the house so my situation is obviously not the same as others. I run MySQL along with Oracle, DB/2, and PostgreSQL so had an instance immediately available to me. A very simple schema comprising a single table was all that was required. The beauty is that I can have a cronjob directly invoke the mysql command-line utility to enable/disable rooms, and it's location-agnostic, i.e. it doesn't matter where it runs. The hostname, schema and SQL can all be specified on the command-line, making the functionality truly portable. I could even extend the SMS functionality in future to manage enabling/disabling rooms.

Some of you are likely thinking that I'm "using a sledgehammer to kill a fly" but it's a little more complex than that. You always have to be cognizant of how requirements tend to be "fluid" and will often "mutate" over time. While a RDBMS is obviously over-kill in this scenario, there could come a time when taking an initially simpler solution would come back to bite you. And for the truly curious among you, I've performed timings of my solution and it takes about 28ms for a notification to go from an infrared sensor all the way to the Philips Hue hub, turning on the light in the appropriate room. There is no discernable delay.

Conclusion

Obviously not all of this came together "overnight" but it didn't actually take an excessive amount of either time or effort. Having said that, I have much more familiarity with the components than most. And it's a remarkably robust system, reliably controlling my lights without manual intervention, except to turn the lights off and I can use Amazon Alexa (voice-control on the Echo Dots) or the Philips app on my iPhone or iPad for that. I could even (egads!) hit the light switch! I could obviously go a lot further, controlling how long the lights stay on, as I do for the staircase lights in the morning. But I don't know in advance how long I want the lights to stay on in most rooms. I don't want the den lights turning off while I'm watching television, for example, or my reading lights to decide when to turn off. Even with automation there's still a need for human control.

Things also change over time. I started off running Glassfish on the original Pi but there were performance issues and a couple of times it completely ran out of memory, including swap. I picked up another Pi for the living room in order to drive the HDMI LED television there with graphical displays of near-real-time system status. So now I run Glassfish on that server, which is wirelessly connected to the repeater in the kitchen. It took some configuration on the Belkin wireless router to permit traffic on port 3700 (Internet Inter-ORB Protocol) to get to the new Pi from the internal network but nothing onerous. I've also built a new server so might shift Glassfish there. Everything is configurable so no code changes would be required. And shouldn't that always be the goal for systems large or small?

NOTE: I'll bundle up the source and add it here when I get time.