I met Holger Koch at a conference in April this year. Holger works for DB Systel, the IT company of the Deutsche Bahn, and he has had the guts to convince the organization to host an internal "Open Source Workshop". Knowing a bit of how large organizations "operate", and believe me: the Deutsch Bahn is large, I'm sure that task was gargantuan.

I had the honour to be invited to speak there, and I gladly accepted the invitation. (I spoke for an hour of what Ansible is and how it has evolved in the course of the last year or so.) And this was an invitation: travel (by first-class train :-) ) and very comfortable accomodation were provided for.

To be honest I wouldn't typically report this; there are many conferences which focus on Open Source topics, and some of my readers go to many more of those than I do. What makes this event noteworthy, is that this was a company-internal show: all participants were employees of the Deutsche Bahn or its assets. Promoting Open Source within an organization in that form is quite special, at least I've yet to hear about that being done at that kind of scale.

During his ending keynote, Holger spoke of why people and particularly companies should promote and contribute to Open Source. Contributions can take the form of donations or code contributions, and he gave several examples. One of the examples Holger brought back to memory was how the OpenSSL project, one of the pillars of security on the Internet, had received just a few thousand dollars in donations before the HeartBleed bug. (That has changed since then.) A few voices in the audience suggested promoting Open Source within organizations, as Holger has done, is also a great way to contribute, and I fully agree. It would be great to see much more of this done, and it isn't difficult to do. What's also very important: it doesn't cost a lot; certainly a few work hours must be sacrificed, but on the other hand, the added value for companies is certainly higher motivation of employees. I think this model would work on a larger scale.

That got me thinking about whether I, as an individual, actually contribute to Open Source, and if so, how. I suddenly felt like a bit of a leach because I use a lot of Open Source compontents (utilities, operating systems, programming languages, etc.)

I have no doubt you'll correct me if you think I'm wrong, but I believe I do contribute, at least to projects which interest me:

  • I do quite a bit of testing and try to report bugs, to the best of my ability. (If you've ever done a bit of this, you know it can be very time consuming.)
  • I contribute little bits and pieces of code and/or documentation.
  • I take the time (and typically pay for my own travel and accomodation expenses) to speak at conferences about particular Open Source topics.

In the course of the last few years, I've spoken maybe two dozen times about DNS, DNSSEC, Ansible, MQTT, and a few other topics I can't recall off the top of my head.

Does that count as contributing to the Open Source community? I hope it does.

When Peter asked me the other day whether I could recommend some stack/platform/whatever for storing/viewing metrics, my first answer was Graphite, but it then quickly occurred to me that I myself never really got around to finishing that project. A Graphite installation is a bit messy, and I don't find it trivial to support.

I had previously stumbled over InfluxDB, and I liked the look of it because

  • it's a single binary
  • it has okay-ish documentation
  • it worked for me out of the box

Reasons enough for me to have a closer look.

I could write whole books about what I don't know about InfluxDB, and I have no idea whatsoever how it'll fare up to really heavy 10,000-server-type of usage, but I don't really care: my use-case is to get some decent time-series database set up for a SOHO environment (storing a few sensor datapoints, electricty consumption, and things like that), in addition to being able to easily render some nice-looking graphs, and all that without spending hours setting it up and maintaining it. (Also, my experience with Xiveley, the artist previously known as Cosm, the artist previously known as Pachube was only okay-ish (sometimes), and it's in a cloud, but I digress.)

Whatever it is I use should be easy. Period. It must have a decent-enough API, be supported by Python, and I want to be up and running (and hopefully not look back) within an hour or two. Maximum three.

InfluxDB seems to be a good match. It is based on LevelDB (I know somebody I admire who'd tear my head off for even using that word...), and it runs on all relevant platforms I can think of. It has a built-in administration panel which is good enough, API support with HTTP Basic authentication (or parametrized auth), and it's extensible: in addition to an HTTP API, I can configure InfluxDB for Graphite carbon support, which means I can submit data via a number different methods, and adding further listeners is possible and looks pretty straight forward for somebody in the know of Go. (I can immediately think of a listener I'd like to see; talk to me, do it, and make me happy. :-)

InfluxDB can also query Graphite data if that's your cuppa and/or if you already have a big dataset you want to use for it.

First off, the Carbon support made me add a carbon plugin to mqttwarn, and I used that to feed in a set of test data to InfluxDB to then glance at it via the built-in Web thing that InfluxDB has.

Using the InfluxDB API, I can then, say, retrieve values via HTTP:

        "columns": [
        "name": "c.youless", 
        "points": [

InfluxDB's Web console is useful but a bit rudimentary. Enter Grafana which is based on or at least heavily inspired by ElasicSearch's Kibana.

I set that up, and the bit that took longest in this whole exercise was to find the right button to click in order to have Grafana read from InfluxDB. (I had already configured it to do so, I just couldn't find the data source button, so to speak.)

The result is rather nice, I find, even though I haven't been able to switch to a white background...

Whilst possibly not as flexible as Graphite, I'm certainly going to install this pair of utilities at home, and let it run for a while; it appears to be just what this doctor ordered for himself.

View Comments :: MQTT, metrics, graphite, and carbon :: 10 Jul 2014 :: e-mail

When I wrote about the HiveMQ MQTT broker I said it had a killer-feature: support for Websockets. I'll admit to have been pestering Roger Light, creator of Mosquitto, so much that to shut me up (I think) he's been adding support for Websockets into the Mosquitto broker!

This makes using MQTT even more fun.

Mosquitto and Websockets

Websocket support must be explicitly enabled when building Mosquitto, and building requires libwebsockets. Once enabled, we can add listeners with the websocket protocol. For example, this mosquitto.conf provides a default MQTT listener on port 1883, and two Websocket listeners, one with TLS, one without:

autosave_interval 1800
persistence true
persistence_file m2.db
persistence_location /var/mosquitto/
connection_messages true
log_timestamp true

acl_file /etc/mosquitto/jp.acl
password_file /etc/mosquitto/jp.pw

listener 1883

listener 9001
protocol websockets

listener 9002
protocol websockets
cafile /etc/mosquitto/tls/ca.crt
certfile /etc/mosquitto/tls/tiggr.ww.mens.de.crt
keyfile /etc/mosquitto/tls/tiggr.ww.mens.de.key

(In case you're interested, I create the X.509 certificates for TLS using generate-CA.sh from the OwnTracks project.)

Mosquitto ACLs work for Websockets just as they do for MQTT publishes & subscribes; when accessing the broker via Websockets a different TCP transport channel is used -- everthing else remains the same.

Using the Paho MQTT JavaScript client, we can create something like this which shows messages published to the Mosquitto broker as soon as they're received.

<html xmlns="http://www.w3.org/1999/xhtml">
    <title>Mosquitto Websockets</title>
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <script src="mqttws31.js" type="text/javascript"></script>
    <script src="jquery.min.js" type="text/javascript"></script>
    <script src="config.js" type="text/javascript"></script>

    <script type="text/javascript">
    var mqtt;
    var reconnectTimeout = 2000;

    function MQTTconnect() {
        mqtt = new Messaging.Client(
                        "web_" + parseInt(Math.random() * 100,
        var options = {
            timeout: 3,
            useSSL: useTLS,
            cleanSession: cleansession,
            onSuccess: onConnect,
            onFailure: function (message) {
                $('#status').val("Connection failed: " + message.errorMessage + "Retrying");
                setTimeout(MQTTconnect, reconnectTimeout);

        mqtt.onConnectionLost = onConnectionLost;
        mqtt.onMessageArrived = onMessageArrived;

        if (username != null) {
            options.userName = username;
            options.password = password;
        console.log("Host="+ host + ", port=" + port + " TLS = " + useTLS + " username=" + username + " password=" + password);

    function onConnect() {
        $('#status').val('Connected to ' + host + ':' + port);
        // Connection succeeded; subscribe to our topic
        mqtt.subscribe(topic, {qos: 0});

    function onConnectionLost(response) {
        setTimeout(MQTTconnect, reconnectTimeout);
        $('#status').val("connection lost: " + responseObject.errorMessage + ". Reconnecting");


    function onMessageArrived(message) {

        var topic = message.destinationName;
        var payload = message.payloadString;

        $('#ws').prepend('<li>' + topic + ' = ' + payload + '</li>');

    $(document).ready(function() {

    <h1>Mosquitto Websockets</h1>
        <div>Subscribed to <input type='text' id='topic' disabled />
        Status: <input type='text' id='status' size="80" disabled /></div>

        <ul id='ws' style="font-family: 'Courier New', Courier, monospace;"></ul>

I've put the source to this simple example on Github, and I've verified it works with Safari, Chrome, and FireFox.


Instead of writing your own Web application, you can easily use the mqtt.io or the HiveMQ Websocket clients; configure them to use the host name and TCP port number (from the listener directive) of your Mosquitto broker.

Mosquitto with Websocket support should be with you very soon, and I hope you'll agree with me, that this is very good news. :-)

The idea for an application on a smartphone which publishes your location via MQTT was born a few weeks prior to the first commit to the Android app. My first notes are dated June 16th, and shortly thereafter I spoke to Alexander about the idea of the app which would turn into what we now call OwnTracks. Just after first writing about it, Christoph created the iOS app.

The principle is simple enough: an app on an iOS or Android device transmits the device's location in a JSON payload via MQTT to a broker I operate and from there, I can do whatever I wish with the data.

Andy's tweet

We started this project under what we thought was a very techy-sounding and very sexy name MQTTitude, but it was unpronounceable; friends thought we were being rude when we tried to say the word, and since we couldn't agree on giving the name a few more t_s to make it really impossible, we decided to re-brand and chose OwnTracks. The name should associate tracks as in "location tracking", and own_ as in "you own the data" or "your own data".

Initially, the JSON payload contained a time-stamp, and decimal latitude and longitude. These were augmented with the device's battery level (for monitoring, you know) and positional accuracy. JSON makes it easy to extract the data you want from the payload in almost any programming or scripting language, and it enables us to add elements to the object without breaking your application. (You are not using regular expressions to access payload data, are you?!)

    "_type": "location", 
    "acc": "141", 
    "batt": "27", 
    "lat": "48.858334", 
    "lon": "2.295134", 
    "tst": "1402507486"

Early on, we had a lot of ideas pour in, and people started doing fun things with OwnTracks. Marshall T. Rose added support for OwnTracks into The Thing System which greatly pleased us, and Ben Jones wanted us to add support for waypoints and geo-fences which he used to implemented a binding for openHAB to control "stuff" upon entering or leaving the premises.

If I recall correctly, we then added support for what we call "Friends and Family": people who share an MQTT broker can 'see' each other, and the user-interface displays icons on a list or on a map. Friends optionally get to see me entering/leaving waypoints, if I share them.

Screenshots iOS/Android

Bryan Boyd literally took OwnTracks for a run when he live-tweeted a marathon using a bunch of technology together with our app as a GPS capture device, and others have put OwnTracks to good use in other ways, using it to determine colleagues' whereabouts, track and display delayed trains, and all sorts of other things.

While we do try and keep the apps' capabilities in sync with eachother (iOS / Android), Christoph tends to implement features on iOS as soon as we breathe an idea to him, which is why the iOS app is slightly ahead feature-wise. For example, on iOS we now have iBeacon support as well as support for the iPhone 5 pedometer function, and I'll tell you something: testing that has literally been exhausting for me: all that walking ... The way it works is, OwnTracks keeps track of the number of steps and I can request it reports the steps taken since midnight:

{"_type":"steps", "tst":1401966882, "steps":3269}

OwnTracks has really become international, and I think the user farthest away from us here in Germany is in Tasmania. (The only reason I know is because I've seen an OwnTracks JSON payload which was handed over for debugging purposes; remember: it's your data; we don't see it if you don't show us, and that's good! :-) Update: Matt reminds me that I'm quite wrong: Ben's furthest away, in New Zealand. :-)

OwnTracks in Tasmania

Many thanks to everybody who has contributed code, ideas, time for testing, documentation, and everything else we've needed to have over one thousand people now use our Open Source apps!

View Comments :: OwnTracks, location, and MQTT :: 16 Jun 2014 :: e-mail

Stefano Costa sent me a Choral Greenwich for evaluation. I've had my eyes on that or something similar for a while now because my brother in law has been fascinated by OwnTracks, and he's interested in tracking a number of tour buses they operate.

Choral Greenwich

The Greenwich is a lovely little device which comes preloaded with a demo-application which Choral calls OrionMQTT -- an application which periodically reports the device's location over MQTT. (The too-shiny coin on the photo above is a 1 Euro piece.)

Some notable features of the Greenwich:

  • Powered by 8-32 V DC
  • Li-Ion battery with internal charger
  • GSM/GPRS chip
  • Accelerometer
  • Digital inputs and outputs
  • UART serial
  • Optional CAN bus
  • Operating temperature: -20 +60 °C
  • Java programmable

That last bit is what held me off a little because of my ignorance of all things Java; I was hoping to be able to easily modify the code (which will soon be open sourced by Choral). Instead, and for the time being, I'm just re-publishing the data in a format suitable for OwnTracks. Choral provides an SDK for programming the device, but I haven't looked at that yet.

I was given the advanced Greenwich which has all the trimmings on it, including a D-SUB9 through which I configured the device. After booting, I can use #commands to configure it; authenticate, set up GPRS connectivity, tracking interval, and start tracking:

#PWD 1234567890

#CPRSCFG internet.apn,TCP,,1883

#TRKTIM 120s


Other commands such as #PUBTOPIC allow me to configure the MQTT topic to which Greenwich publishes, and #SETID (that's my 41 below) configures a device identifier which can be made to be meaningful for your application (e.g. vehicle number).

After that, it's basically just a matter of carrying the light little box to the vehicle and installing it somewhere (e.g. in the glove box or boot). Mine is provisionally installed, and I hooked it up to an always-on cigarette-lighter in the back of the car.

The Greenwich publishes a CSV-like string which contains a header ($CHX) and a configurable ID (here: 41). The device also reports date (YY) and time obtained via GPS, position as well as a battery level, travelled distance in metres, and a few other interesting values such as altitude and speed over ground.



The temporary republisher I have extracts the elements from the CSV and produces OwnTracks-compatible JSON payloads.

    "alt": "240.4", 
    "batt": "4.4V E", 
    "distance": "102780", 
    "lat": "50.961008", 
    "lon": "8.716252", 
    "sat": "09", 
    "setid": "41", 
    "tst": "1402666628", 
    "vel": "24.89"

In addition to periodic publishing over MQTT, I can send an SMS to the device with the string gps in it, and it responds via SMS, which is rather neat. The software can also be rebooted via SMS, and we can send it a park message which will alert (via SMS) if the vehicle is moved after it receives the park.

The Greenwich I have has been behaving very well; a watchdog on it automatically restarts it if something goes wrong. For example, I drove through an area with zero GPRS coverage yesterday afternoon for about 10 kilometers, and I noticed the LEDs flashing to indicate a reboot; after a few minutes (my logs tell me later), the device continued reporting. (A reboot resets the travelled distance, of course.)

This is a track I recorded yesterday (via the OwnTracks m2s back-end), with the device reporting every 30 seconds.

Driving with Greenwich

and if I zoom in a bit we still see the quite fine granularity of the reports:

Zoom in

Before we can take this to production, there are definitely features missing in the software, and I'm talking to Choral about this as we speak (the people at Choral have been extremely helpful, and they're very responsive):

  • TLS is a must-have; currently the connection to the MQTT broker is in plain text.
  • Authentication is also a must-have, though this should be no great issue as Greenwich uses the Paho MQTT client for Java
  • I'd very much like to see a #TRKDIST command with which I can define a distance (in meters) after which a location is published; this would allow lowering the required GPRS data volume.

The Choral Greenwich is a really neat little box, and it's performing very well.

View Comments :: MQTT, OwnTracks, and Location :: 14 Jun 2014 :: e-mail

Other recent entries