Freeboard has been on my "list of dashboards to test" for a while now, and I finally got around to it because I recently discovered that AL S-M had created an MQTT plugin for it.

Freeboard runs in your Web browser and has a number of data sources from which you can chose, including a generic JSON plugin which reads an HTTP URL for its data. After installing the Freeboard MQTT plugin as per instructions (make sure you get the development version of mqttws31.js!) we're ready to go. For posterity, this is how I patched index.html:

--- index.html.orig   2014-11-12 17:01:37.000000000 +0100
+++ index.html 2014-11-12 17:02:22.000000000 +0100
@@ -11,6 +11,7 @@
     <script src="js/freeboard.thirdparty.min.js"></script>
     <script type="text/javascript">
         head.js("js/freeboard+plugins.min.js",
+      "plugins/mqtt/paho.mqtt.plugin.js",
                 // *** Load more plugins here ***
                 function(){
                     $(function()

I also placed the plugin and the Paho MQTT JavaScript client here:

plugins/mqtt/mqttws31.js
plugins/mqtt/paho.mqtt.plugin.js

and modified paho.mqtt.plugin.js as follows:

--- plugins/mqtt/paho.mqtt.plugin.js.orig 2014-11-12 17:02:31.000000000 +0100
+++ plugins/mqtt/paho.mqtt.plugin.js   2014-11-12 17:33:12.000000000 +0100
@@ -10,7 +10,7 @@
        "display_name": "Paho MQTT",
         "description" : "Receive data from an MQTT server.",
        "external_scripts" : [
-          "<full address of the paho mqtt javascript client>"
+          "plugins/mqtt/mqttws31.js",
        ],
        "settings"    : [
            {

For test purposes, I added three MQTT sources (called Paho MQTT in the plugin). The first looks like this:

OwnTracks data source

You'll note that the port isn't your typical MQTT TCP port but rather a Websocket port such as that provided by Mosquitto's Websocket code. It's also here, whilst configuring the data source, that you specify the topic the plugin will subscribe to, and you specify JSON messages if the messages contain JSON.

source

When adding a pane, you specify the data source and data element you're interested in. As an example, I'm using the OwnTracks JSON format to show the value of altitude in a sparkline, so I configure the value as shown here. (I will not confess as to how long it took me to find out how that works.)

datasources["OwnTracks"]["msg"]["alt"]

In other words, ["OwnTracks"]["msg"] is the JSON payload received in the message, and ["alt"] is the JSON element therein. If you specified your payload as not being JSON, the value is in

datasources["DataSourceName"].msg

dashboard

I rather like the resulting dashboard. I can drag the widgets around, and I can save the dashboard (as a JSON) file to later reload that. (I can't get the Save to work in Safari.)

Related:

View Comments :: Dashboard, MQTT, and JSON :: 12 Nov 2014 :: e-mail

Slack calls itself a platform for team communication and it's all of that, and if you forgive my enthusiasm, a shit-load more. It has changed the way our tiny team works quite radically; for one, we've abandoned e-mail. I won't go into all that Slack offers (their documentation is extensive and very good, and there are tons of introductory articles), but I will tell you what we do with it, other than just simple "chat".

BTW, Slack is free of charge until you get addicted to it. If you're like me, you'll actually want to pay for it. :-)

Aside from simple searchable chats, as I mentioned earlier, Slack allows us to create any number of so-called "channels", a little bit like an IRC channel. Team members can be invited to channels to talk about particular topics. A two-person team doesn't need many of those, but larger organizations will want to separate topics.

Where Slack gets really interesting is in the integrations it offers; a seemingly endless list of services which allow us to get notifications, information, etc. into Slack. If you like a wee bit of hacking, there's an API and what they term incoming and outgoing "WebHooks" with which we can send data into and get data out of Slack respectively.

For example, for our Greenwich devices we've created a WebHook which allows us to query a database of information which is kept updated via MQTT from the devices.

In other words, I can just enter something like versions or howis XX into the particular Slack channel we created for it, and Slack goes out to a service we created which returns the information into that channel.

We also get notifications in specific channels from external processes. For example, a Nagios integration brings alerts into one channel. Another channel displays warnings we emit via MQTT and get into Slack with the help of mqttwarn (for which I created a slack plugin).

I can't tell you how incredibly cool this is, particularly since it also works mobile with the respective iOS and Android apps. Trust me on this: you want to use Slack.

View Comments :: Productivity :: 23 Oct 2014 :: e-mail

Lunchtime humor

(A lunchtime inspiration by a party of 4)

A 10-minute session with a gin/tonic at a bar in Berlin earlier this year seems to have sufficed for me to convince Lennart Koopmann to add an MQTT input to Graylog2. (Zoom into the picture in that tweet :-)) The damage a single drink can do ...

A new version of Graylog2 was released yesterday. One of its new features is support for loadable plugins, so I built the graylog2-mqtt-input plugin, and dropped it into Graylog2's plugin/ directory before starting graylog2-server.

The first thing I then did was to add the MQTT plugin as an input using Graylog2's Web interface:

Add MQTT as input

plugin config

I like that the developers have thought of allowing me to specify multiple topics Graylog2 will subscribe to. As soon as that was complete, the plugin was launched and I saw the connection in my broker's log.

running input

The MQTT input plugin expects messages to be in GELF. In particular, the mandatory host and short_message elements must be present or the message isn't processed.

Using this simple program as an example (without the sleep() in it), I obtained a throughput of just over 950 messages/second in Graylog2 on my portable datacenter. To be quite explicit: the container in which I tested this is running on my laptop and has ElasticSearch, Graylog2-server, and Graylog2-Web, all in a meager 768 MB RAM!

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import paho.mqtt.client as paho   # pip install paho-mqtt
import time
import socket
import random
import json
import os

hostname = socket.gethostname()

clientid = 'jp-%s-%s' % (hostname, os.getpid())

mqttc = paho.Client(clientid, clean_session=False, userdata=None)
mqttc.connect("localhost", 1883, 60)

while True:
    temp = random.randint(10, 33)
    text = "Temperature %s" % temp
    data = {
        'host'      : hostname,
        'short_message' : text,
        'temperature'   : temp,
    }

    mqttc.publish('gl/temp', json.dumps(data))
    time.sleep(1)

The program publishes messages to the topic gl/temp which Graylog2 subscribes to, and these messages are added to the index. Selecting one of them demonstrates how Graylog2 adds a timestamp as well as the MQTT topic the message was published to.

temp details

Because of the GELF requirements, without modifying the modus operandi (i.e. the Java code) of the plugin, we can't just publish MQTT payloads which contain, say, an integer temperature or something similar. We'll typically have to create a republisher which subscribes to the topics you want logged, transforms the payload into GELF and republishes messages to the topic Graylog2 is subscribed to. One possible candidate for doing this may be mqttwarn, if you want something quick and dirty. Here is an example taking OwnTracks data and splitting it up for the MQTT plugin.

OwnTracks pubs

All in all, this is looking very good. I have only had the combination running for a couple of hours, but it's working well so far.

Thank you very much, Lennart and the Graylog2 team for doing this.

View Comments :: MQTT and logging :: 24 Sep 2014 :: e-mail

My hands are almost bleeding from plugging and unplugging Molex connectors from the back of 26 Choral Greenwich devices, but it's been fun. (Those damn things are almost impossible to take out, which is good; they don't fall out by themselves!) I've told you about the Choral Greenwich before, but a lot has happened in the last three months.

Greenwich

First and foremost, my unstoppable friend Christoph, who also made OwnTracks for iOS and MQTT Inspector for iOS, has completely revamped the software onboard this vehicle and asset tracking device. With some advice from our friends at Choral, and a boatload of testing by both of us, we've transformed the software into something which is ready to roll and, in fact, has already been running live on six vehicles during the last couple of months without a hitch.

In particular, what we lovingly call the OwnTracks Edition of the software has the following features:

  • Publishing of location updates over MQTT with TLS
  • Device configuration via MQTT or SMS
  • Message payloads compatible with OwnTracks' JSON which means you can use our OwnTracks apps to display location data on a smartphone
  • Watchdog resets the device if it becomes unresponsive
  • Over the Air (OTA) upgrades
  • GPIO (input and output) controllable over MQTT
  • Alarming if device is moved

and many more features which make this a very versatile and affordable device. In particular, I'm proud of being able to trigger an OTA upgrade directly over MQTT, which, a) saves on the cost of an SMS (relatively expensive with M2M SIM data plans), and b) allows us to seamlessly push software updates out into devices installed in the field. While OTA upgrades may sound expensive, the M2M data plan we're using translates a full upgrade into 0.0416 €, which is almost not worth speaking of.

As for the JSON payloads, we've added a few elements. First and foremost, a so-called tid which we use also in the new iOS app to display what we call an "automatic face", if you don't want to associate a photo from your address book with an OwnTracks MQTT topic. Next up are cog (course over ground), vel (speed in KM/h), dist (distance covered in meters from the previous location published), and trip which shows the total distance covered since reset (and yes: you can zero this counter via MQTT).

{
    "_type": "location", 
    "alt": 497, 
    "cog": 223, 
    "dist": 1572, 
    "lat": "47.453453", 
    "lon": "8.703986", 
    "t": "t", 
    "tid": "AP", 
    "trip": 119779, 
    "tst": "1411120145", 
    "vel": 98
}

In fact we've already got our first "customer", namely my brother in law, who's ordered said 26 devices to put on tour buses they operate. He's been so impressed with the first four, that he's decided to put one in each of their vehicles.

Initial provisioning of naked Greenwich devices has been a bit of a pain, but we've now solved that rather nicely: a small program creates and sends the PDU SMS for the OTA upgrade, and off we go. Here's a photo of my workspace during the last 20 hours or so.

Provisioning Greenwich

We're also hard at work on an appropriate back-end which will collect and visualize data. This is almost ready and will be contributed to the OwnTracks project in due course. In addition to being able to display historic routes using GeoJSON we generate from the database, we're making heavy use of "live" updates, which are provided by Mosquitto with Websockets.

table view

So, if you're a fan of MQTT, and you're looking for a very neat device for tracking the position of your vehicles, talk to me.

View Comments :: MQTT, GPS, and vehicle :: 18 Sep 2014 :: e-mail

Other recent entries