Digi XBee Cellular Modem

It's sometimes painful being on the "bleeding edge" but it can also be the way to get ahead. As part of my home automation project I wanted to include a cellular modem for a couple of uses: backup for Internet access when the cable modem is out, and being able to receive commands via SMS from my cellular telephone. A company called Digi makes some interesting products. If you're into compact networking, IoT or cellular modems then they're definitely worth a look.

What piqued my interest were their cellular solutions so I ordered their LTE CAT-1 Cellular Development Kit. It comes with a small motherboard which has a USB port, some status LEDs and other interface electronics, the modem itself (which includes MicroPython,) a couple of antennas and a power supply. While a Verizon micro-SIM is included, that doesn't work in Canada. So I had to seek out a provider which operated on the LTE bands supported by the modem. It took a lot of digging to find a product called the Tablet Plan from Bell Canada which only costs $10/month and includes 1 GB of data. The SIM card is extra and I couldn't wait to plug it into the modem and get down to work.

There are some serious limitations on the LTE CAT-1 modem, such as the fact that it's only certified on the Verizon network in the United States. Even after swapping the SIM card, I simply couldn't get the modem to register on the Bell network. So I had to purchase the Digi XBee3 Cellular LTE-M/NB-IoT module. Fortunately it plugs into the board supplied with the development kit. It cost me about $120 all-in, with shipping, customs clearance, HST, etc. and I purchased it from an excellent company by the name of Mouser Electronics out of Kitchener, Ontario. I ordered the part and they were able to get it delivered all the way from Texas the very next business day! Most impressive.

For a variety of reasons, Java is still my development language of choice for certain applications. Digi provides a Java library and their XCTU, XBee Configuration and Test Utility. XCTU runs on various platforms and is used for low-level configuration, such as setting the interface speed, parity, operation mode, etc. Interestingly enough, it's built on top of Eclipse. The MicroPython is neat and I could see it being useful in certain situations but I require only basic functionality. They also have a Python library for communicating with the modem but I also need to communicate with JMS on Glassfish and I've already got code written in Java for that purpose. And there are probably people out there saying "well that's easy enough to do in Python" and that's as may be but it's quicker and easier to re-purpose existing code than rewrite eveything in a different language.

Building off the code examples provided by Digi, I was soon able to make outgoing TCP connections to their echo server as well as my own servers. The problem was that the SMS examples didn't work at all: I could neither send nor receive SMS messages. I got on the telephone will Bell support and explained my problem. Now I'm not about to claim that I'm either the first or only customer trying to get their XBee working on the Bell network. It's just very difficult to describe the type of hardware you're using. These people typically keep thinking in terms of cellular telephones. I was lucky enough to speak with someone (thanks, Mike!) who was able to grok the situation and provide a solution. It turns out that SMS isn't enabled by default on the Tablet Plan. It's going to add 30 cents per text to the monthly bill but it now works a treat.

I'm a big proponent of "loosely-coupled" systems. While it would be easy enough to simply have the SMS daemon relay commands directly to the Philips Hue hub via their REST interface, I always look to the bigger picture. There might come a time in future where I want to have other sources for commands, not just SMS. By keeping the SMS daemon separate from the one which consumes messages from the queue and carries out the request, I'm keeping my options option. There could come a time when I want to be able to generate commands from my AWS cloud instance, for example. If sources only need to push to a message queue then they become simple, stand-alone and can be leveraged in future. I could even have cron jobs push messages to the queue at specific times or an e-mail reader which could do the same. This approach just opens up a world of possibilities.

Speaking of e-mail, I already have a daemon which monitors my GMail account, looking for alerts generated by Cronitor. That's configured to send e-mails when certain conditions are detected as well as when they clear. The current implementation turns on a Philips Hue Bloom in my den and sets the colour to red when an alert is received. It turns the light green when the "all clear" message is received. The Bloom still needs to be manually turned off either via Alexa (voice control) on the Amazon Echo Dots in the house or the Hue application on my iPhone and iPad.

What follows are some instructions and code detailing how I was able to accomplish my goal. Where appropriate, I've also included some Bourne-again shell scripts and directory listings. The reality is that, while it's fairly straight-forward if you've ever done stuff like this before, it can be quite daunting if it's all new to you. I'll have to assume some basic familiarity with how things work under *NIX; while details may differ, the underlying concepts are all the same.

Raspberry Pi

The Rasperry Pi is the core of my home automation system. I've purchased a couple of the Canakit packages. They sell everything from the bare board to kits including power supply, case, heat sinks, even a Micro-SD card with NOOBS pre-installed. Even though I burned Raspbian Stretch onto a 64 GB Micro-SD card, giving me quite a bit of free disk space, applications like Glassfish take up a lot of room. I had an external enclsoure sitting around so I picked up a Samsung 860 EVO M.2 250GB SSD drive (from Amazon) and added an external USB drive. In addition to the USB keyboard/mouse and HDMI cables which run to my HMDI KVM switch, I've also plugged-in the Digi XBee and an old X10 control module. The Digi Java library and latest version of Java 8 (1.8.0_181) were installed along with Glassfish 5.

Glassfish

I need the JMS functionality included in Glassfish. It's important to note that JMS is part of the J2EE (now JEE) standard so you can also find this in WebLogic, WebSphere and even JBoss. I've standardized on the "reference" version which also has the added attraction of being open-source, hence free. I won't bore you with installation details as examples galore can be found on the 'net. Here's a screen shot of the JMS factory configuration:

Here's a screen shot of the queue configuration:

Other than that, it's a vanilla install with all the standard default settings.

SMS Receiver

As mentioned previously, the listener has two jobs: receiving incoming SMS messages and pushing them onto the message queue. Here's the code:

package net.sudsy.homecontrol; import java.util.Hashtable; import javax.naming.Binding; import javax.naming.Context; import javax.naming.InitialContext; import javax.naming.NameClassPair; import javax.naming.NamingEnumeration; import javax.jms.QueueConnectionFactory; import javax.jms.QueueConnection; import javax.jms.Queue; import javax.jms.ConnectionMetaData; import javax.jms.JMSContext; import javax.jms.JMSProducer; import javax.jms.JMSConsumer; import javax.jms.Message; import org.apache.commons.cli.BasicParser; import org.apache.commons.cli.CommandLine; import org.apache.commons.cli.CommandLineParser; import org.apache.commons.cli.Option; import org.apache.commons.cli.Options; import org.apache.commons.cli.ParseException; import com.digi.xbee.api.CellularDevice; import com.digi.xbee.api.exceptions.XBeeException; import com.digi.xbee.api.listeners.ISMSReceiveListener; import com.digi.xbee.api.models.SMSMessage; /** * SMS receiver which pushes incoming texts to a JMS queue. */ public class SMSReceiver implements ISMSReceiveListener { private static final String PORT = "/dev/XBee"; private static final int BAUD_RATE = 115200; private static JMSProducer sender = null; private static Queue destination = null; public static void main( String args[] ) { CellularDevice dev = new CellularDevice( PORT, BAUD_RATE ); Hashtable props = new Hashtable(); props.put( "javax.naming.factory.initial", "com.sun.enterprise.naming.SerialInitContextFactory" ); props.put( "org.omg.CORBA.ORBInitialHost", "localhost" ); props.put( "org.omg.CORBA.ORBInitialPort", "3700" ); try { Context ictx = new InitialContext( props ); Queue queue = null; QueueConnectionFactory factory = (QueueConnectionFactory) ictx.lookup( "jms/homeFactory" ); destination = (Queue) ictx.lookup( "jms/home" ); JMSContext qctx = factory.createContext(); sender = qctx.createProducer(); dev.open(); dev.addSMSListener( new SMSReceiver() ); } catch( Exception e) { e.printStackTrace(); System.exit( 12 ); } } @Override public void smsReceived( SMSMessage msg ) { sender.send( destination, msg.getData() ); } }

Not much to it, right? But things quickly get more complicated. First of all you have to build the the jar from the Digi Java library source. That uses maven for the build so some of the dependencies are downloaded to your $HOME/.m2/repository directory tree. I'm not a big fan of maven for reasons I don't need to explain here. I've been using ant as my build tool for a number of years and have many functional scripts sitting around on various servers. But I have my own standard project layout and, unlike the unecessarily complex and convoluted maven structure, I have just three sub-directories: src, lib and build. After a number of iterations I was able to determine which jars I needed and copied them into the lib directory. Here's my build.xml:

<project name="Tools" default="jar"> <property name="srcdir" value="${basedir}/src" /> <property name="dstdir" value="${basedir}/build" /> <property name="libdir" value="${basedir}/lib" /> <property name="glassfish" value="/opt2/workspace/glassfish5/glassfish" /> <property name="configdir" value="${basedir}/config" /> <property name="nativelib" value="/usr/lib/arm-linux-gnueabihf/libusb4java.so" /> <property name="nativedir" value="/org/usb4java/linux-arm" /> <mkdir dir="${dstdir}" /> <path id="compile.classpath"> <fileset dir="${libdir}"> <include name="**/*.jar" /> </fileset> <fileset dir="${glassfish}/lib"> <include name="javaee.jar" /> </fileset> <fileset dir="${glassfish}/modules"> <include name="javax.jms-api.jar" /> </fileset> </path> <target name="compile"> <javac srcdir="${basedir}/src" destdir="${dstdir}" includeantruntime="false" classpathref="compile.classpath" debug="true" > <compilerarg value="-Xlint" /> </javac> </target> <target name="jar" depends="compile"> <copy file="${configdir}/javax.usb.properties" toDir="${dstdir}" /> <mkdir dir="${dstdir}/${nativedir}" /> <copy file="${nativelib}" toDir="${dstdir}/${nativedir}" /> <jar destfile="${dstdir}/${ant.project.name}.jar" baseDir="${dstdir}"> <include name="**/*.class" /> <include name="**/*.properties" /> <include name="**/*.so" /> <zipfileset src="${libdir}/commons-cli-1.2.jar" excludes="META-INF/*" /> <zipfileset src="${libdir}/jsoup-1.8.3.jar" excludes="META-INF/*" /> <zipfileset src="${libdir}/mysql-connector-java-5.1.31-bin.jar" excludes="META-INF/*" /> <zipfileset src="${libdir}/javax.json-1.0.4.jar" excludes="META-INF/*" /> <zipfileset src="${libdir}/javax.json-api-1.0.jar" excludes="META-INF/*" /> <zipfileset src="${libdir}/usb4java-1.2.1.jar" excludes="META-INF/*" /> <zipfileset src="${libdir}/usb4java-javax-1.2.0.jar" excludes="META-INF/*" /> <zipfileset src="${libdir}/usb-api-1.0.2.jar" excludes="META-INF/*" /> <zipfileset src="${libdir}/commons-lang3-3.8.jar" excludes="META-INF/*" /> </jar> </target> <target name="clean"> <delete dir="${dstdir}"> <include name="**/*.class" /> <include name="*.jar" /> </delete> </target> <target name="cleanAll"> <delete dir="${dstdir}" /> </target> </project>

Please accept that this isn't a "minimal" build script: it contains some references which aren't used in this project. But note that I don't include the jars from the glassfish installation directory: those I leave in place. Here's a list of files in the lib directory:

android-sdk-5.1.1.jar commons-cli-1.2.jar commons-lang3-3.8.jar javax.json-1.0.4.jar javax.json-api-1.0.jar jsoup-1.8.3.jar mail.jar mysql-connector-java-5.1.31-bin.jar RXTXcomm.jar slf4j-api-1.7.12.jar slf4j-jdk14-1.7.12.jar usb4java-1.2.0 usb4java-1.2.0.tar usb4java-1.2.1.jar usb4java-javax-1.2.0 usb4java-javax-1.2.0.jar usb4java-javax-1.2.0.tar usb-api-1.0.2.jar xbee-java-library-1.2.1.jar

Again, not all of these libraries are required for this project. The project contains some other code used for different purposes and some of these jars are dependencies.
There are some other wrinkles as well. You'll note that the communication port specified in the code is /dev/XBee. Where does that come from? There's a file on Raspbian named /etc/udev/rules.d/99-com.rules and I had to add a couple of lines to the end:

SUBSYSTEM=="tty", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", SYMLINK="XBee" SUBSYSTEM=="tty", ATTRS{idVendor}=="0bc7", ATTRS{idProduct}=="0001", SYMLINK="X10"

This maps USB devices to links in the /dev directory. But we also need to tell the RXTX package about the port. Again, multiple iterations produced the recipe for all the jars which need to be included in the classpath and the additional flags which need to be set in the java invocation. It was easier just to create a Bourne-again script to make life easier so here's what that looks like:

#!/bin/sh BASEDIR=$HOME/svn/Tools CLASSPATH="/opt2/workspace/glassfish5/glassfish/lib/javaee.jar:\ /opt2/workspace/glassfish5/glassfish/modules/glassfish-naming.jar:\ /opt2/workspace/glassfish5/glassfish/modules/bean-validator.jar:\ /opt2/workspace/glassfish5/glassfish/modules/gf-client-module.jar:\ /opt2/workspace/glassfish5/glassfish/modules/appserv-rt.jar:\ /opt2/workspace/glassfish5/glassfish/modules/javax.jms-api.jar:\ /opt2/workspace/glassfish5/glassfish/lib/install/applications/jmsra/imqjmsra.jar:\ $BASEDIR/lib/xbee-java-library-1.2.1.jar:\ $BASEDIR/lib/RXTXcomm.jar:\ $BASEDIR/lib/slf4j-jdk14-1.7.12.jar:\ $BASEDIR/lib/slf4j-api-1.7.12.jar:\ $BASEDIR/build/Tools.jar" java -cp $CLASSPATH \ -Dgnu.io.rxtx.SerialPorts=/dev/XBee \ -Djava.library.path=/usr/lib/jni \ net.sudsy.homecontrol.SMSReceiver $*

Queue Listener

The queue listener is similarly straight-forward. It's easily extensible and most values of interest are defined as static final variables.

package net.sudsy.homecontrol; import java.util.Hashtable; import java.net.URL; import java.net.URL; import java.net.HttpURLConnection; import java.io.OutputStream; import java.util.HashMap; import java.util.Map; import javax.naming.Binding; import javax.naming.Context; import javax.naming.InitialContext; import javax.naming.NameClassPair; import javax.naming.NamingEnumeration; import javax.jms.QueueConnectionFactory; import javax.jms.QueueConnection; import javax.jms.Queue; import javax.jms.ConnectionMetaData; import javax.jms.JMSContext; import javax.jms.JMSConsumer; import javax.jms.Message; import org.apache.commons.cli.BasicParser; import org.apache.commons.cli.CommandLine; import org.apache.commons.cli.CommandLineParser; import org.apache.commons.cli.Option; import org.apache.commons.cli.Options; import org.apache.commons.cli.ParseException; import com.digi.xbee.api.CellularDevice; import com.digi.xbee.api.exceptions.XBeeException; import com.digi.xbee.api.listeners.ISMSReceiveListener; import com.digi.xbee.api.models.SMSMessage; import javax.json.Json; import javax.json.JsonObject; import javax.json.JsonReader; /** * JMS listener which reads from a queue and sends commands to lights. */ public class QueueListener { private static final String USERID = "SECRET"; private static final String URL_GROUPS = "http://belkin/api/" + USERID + "/groups"; private static final Map deviceMap = new HashMap(); private static final String CMD_HOME = "HOME"; private static final String CMD_AWAY = "AWAY"; private static final int lights[] = { 1, 7, 11 }; public static void main( String args[] ) { Hashtable props = new Hashtable(); props.put( "javax.naming.factory.initial", "com.sun.enterprise.naming.SerialInitContextFactory" ); props.put( "org.omg.CORBA.ORBInitialHost", "localhost" ); props.put( "org.omg.CORBA.ORBInitialPort", "3700" ); try { loadScenes(); Context ictx = new InitialContext( props ); QueueConnectionFactory factory = (QueueConnectionFactory) ictx.lookup( "jms/homeFactory" ) Queue queue = (Queue) ictx.lookup( "jms/home" ); JMSContext ctx = factory.createContext(); JMSConsumer receiver = ctx.createConsumer( queue ); for(;;) { Message msg = receiver.receive(); String body = msg.getBody( String.class ); String newState = null; if( CMD_HOME.equals( body ) ) newState = "true"; if( CMD_AWAY.equals( body ) ) newState = "false"; if( newState == null ) continue; for( int light : lights ) sendCommand( light, newState ); } } catch( Exception e ) { e.printStackTrace(); System.exit( 12 ); } } /** * Load the scenes from the Philips Hue hub and populate device information. */ private static void loadScenes() throws Exception { QueueListener app = new QueueListener(); URL url = null; HttpURLConnection conn = null; int rc = -1; url = new URL( URL_GROUPS ); conn = (HttpURLConnection) url.openConnection(); conn.setDoOutput( true ); conn.setRequestMethod( "GET" ); conn.connect(); rc = conn.getResponseCode(); if( rc != HttpURLConnection.HTTP_OK ) { System.err.println( "Bad HTTP response: aborting" ); System.exit( 12 ); } JsonReader rdr = Json.createReader( url.openStream() ); JsonObject obj = rdr.readObject(); for( int idx = 1;; idx++ ) { JsonObject child = obj.getJsonObject( idx + "" ); if( child == null ) break; String name = child.getString( "name" ); JsonObject action = child.getJsonObject( "action" ); deviceMap.put( name, app.new DeviceInfo( name, idx, action.getBoolean( "on" ) ) ); } } /** * Send the command to the Philips Hue hub to turn on/off a light. */ public static void sendCommand( int lightNumber, String state ) throws Exception { URL url = null; String cmd = "{ \"on\": " + state + " }"; url = new URL( URL_GROUPS + "/" + lightNumber + "/action" ); HttpURLConnection connection = (HttpURLConnection) url.openConnection(); connection.setRequestMethod( "PUT" ); connection.setDoOutput( true ); connection.connect(); OutputStream os = connection.getOutputStream(); os.write( cmd.getBytes() ); os.flush(); System.out.println( connection.getResponseMessage() ); } /** * Wrapper for device information. */ class DeviceInfo { String name; int groupID; int debounce = 0; int delay = -1; boolean state = false; // constructors public DeviceInfo( String name, int groupID ) { this.name = name; this.groupID = groupID; } public DeviceInfo( String name, int groupID, boolean state ) { this.name = name; this.groupID = groupID; this.state = state; } // accessors and mutators public String getName() { return name; } public int getGroupID() { return groupID; } public void setDebounce( int debounce ) { this.debounce = debounce; } public int getDebounce() { return debounce; } public void setDelay( int delay ) { this.delay = delay; } public int getDelay() { return delay; } public void setState( boolean state ) { this.state = state; } public boolean getState() { return state; } } }

I should mention that I'm usually a stickler for only including imports which are referenced in the code. There's a wonderful mechanism in Eclipse where you just select Source -> Organize Imports from the pop-up menu but you have to realize that I'm writing the code on a Raspberry Pi. It uses an ARM V7 processor and I haven't been able to find a binary version of Eclipse for the platform. It might exist, just not on the Eclipse website. I might have been able to build it from source but that's a lot of work and given the limited memory of the Pi...

Conclusion

So there you hve it, warts and all. Was it easy? Hardly, but working everything out for yourself can be so much more rewarding than using a canned, "point and shoot" solution. And, as mentioned previously, it's readily extensible. I'm looking forward to being able to send a message from my iPhone, as the bus is approching my stop, and having the requisite lights burning before I get to my door. Very useful in winter when the short days mean that I'm leaving the house and arriving home in the dark. I hope that this information is useful to others.