Having a few temperature sensors at my disposal, I made a big mistake: I compared the values they're emitting.

Sensor values

From left to right we have here, in an area of 20 cm2, the following sensors:

Humidity is even more off than temperature:

humidity

The TI SensorTag CC2650 currently doesn't feel like reporting humidity, so there!

Who actually calibrates these things? But much more importantly: how high is the temperature in my office?

View Comments :: sensors :: 02 Feb 2016 :: e-mail

The back-end program we've created for the OwnTracks apps is finding a modest amount of appreciation, but in spite of relatively simple building instructions, I've been wanting to provide ready-to-run packages for a while now. (Not to say that many people have requested such packages.) I bit the bullet, and I'm happy to say I think I'm doing this like all the cool kids do it. Sort of. I hope.

A while back I created a Homebrew formula for the Recorder. A client suggested a Docker image, so I did that too. Whenever I create a release, I update both and push them into their appropriate locations. About a week ago, I read that Docker hub allows automatic building, so I switched to that and had a builder kicked by a Github notification thing. And now packages. Sigh.

Without Jordan Sissel's fpm I wouldn't have done this. My copy of Maximum RPM (I think that was the name) collects dust somewhere in the basement; I read it many many years ago and can't be bothered to study that again. Furthermore, I knew I'd have to dig into the Debian way of packaging and flatly refuse to learn how to build a .deb the way it probably should be done. fpm to the rescue, wrapped in a little shell script. Life is good.

Then of course I need hosts on which to build said packages. I wanted to offer Debian 8, Raspbian, and Centos 7. A short while later, I could ssh into these systems, launch make and, ta-da!, had three package files. Which need to be signed. Sigh. Another few shell scripts wrapped rpm --sign and reprepro, pushed the files here and there, and an rsync then finally dropped them in our repository. It worked, and I took copious notes on which machine to do what after which other step. You know how it is. (Ansible wouldn't have helped me much I decided.)

rundeck

After several years of hearing how people automate things, I thought: no, this can't be it. Let me try and do this properly; it'll hurt, but the effort should be worth it. I re-read the notes I took about Jenkins and read most carefully what Mark Maas had to say about Rundeck. In for a penny, in for a pound, why not? Moments later, (java -jar rundeck-launcher.jar) I had Rundeck's Web interface up and running and looked around a bit. It appeared simple, but how to add hosts (nodes)?

<?xml version="1.0" encoding="UTF-8"?>
<project>
  <node name="ot-make-cen7"
    description="Centos7 builder"
    tags=""
    hostname="ot-make-cen7"
    osArch="x86_64"
    osFamily="unix"
    osName="centos"
    osVersion="7.3"
    username="jpm"/>
</project>

Aha: an editor on the enterprisey-looking XML file resources.xml fixed that, and few minutes after downloading Rundeck I was launching remote tasks. Very nice indeed. The Web interface is good and snappy, and it has an API. Of course it has an API.

Web

So, I created a job to pull our code, run make and fpm, and drop the resulting package file on shared storage. Hey, this is very productive! Rundeck talks to my servers over SSH (and I recommend you allow its variables to pass through. I then quickly write another job to sign the packages, but this job never stops running ... What's the problem? Ah, the Rundeck debug log shows "Enter passphrase:". Signing means GnuPG keys, and these have a passphrase. Duh. OK, I'm quite sure the world has solved this problem before I have to. It seems that lots (most?) people "solve" this by running expect as a wrapper around the signing program to have it "key in" the clear-text passphrase. I'm aghast, and it seems rpm has trouble with gpg-agent... So, if the cool kids do it, I must also:

#!/usr/bin/expect

## Usage: ./rpmsign.exp *.rpm
#  requires: $GPGPASSPHRASE in environment
#  hacked together by JPM from http://aaronhawley.livejournal.com/10615.html and
#  from @roidela_____ ;-)

global env

spawn rpmsign "--addsign" {*}$argv
expect "Enter pass phrase: "
send $env(GPGPASSPHRASE)
send \r
expect eof

Anyway, where to put the passphrase? Rundeck uses something they call "key storage".

key storage

Basically these are files on the Rundeck server's file system which contain my clear-text secrets which Rundeck can pass into a job via, say, an environment variable. This is not what I'd call "safe storage", but I'll let it pass for the moment. (The reason I'll let is pass is that all this is happening locally on my Mac and not in the cloud, so I feel relatively safe -- famous last words...)

I now have two Rundeck jobs: one to get and compile code, and the other to sign packages. (I typically learn things in small steps to get to know how things work.) What I could now do is to merge those two jobs into one, but Rundeck can do that for me with a "workflow" job, in which I tell it to run this job, then that. Perfect. Rundeck also exports and imports job descriptions in XML or YAML:

- description: Pull git repo and make
  executionEnabled: true
  id: d5c0f354-5c85-4181-9791-ab170f1a6612
  loglevel: INFO
  name: make Recorder packages
  nodefilters:
    dispatch:
      excludePrecedence: true
      keepgoing: true
      rankOrder: ascending
      threadcount: 1
    filter: ot-make-cen7 ot-make-deb8 ot-raspi
  nodesSelectedByDefault: true
  scheduleEnabled: true
  sequence:
    commands:
    - description: pullmake
      script: |
        rm -rf recorder.git
        git clone https://github.com/owntracks/recorder.git recorder.git
        cd recorder.git
        cp etc/@node.os-name@/config.mk.in config.mk
        make
        sh etc/@node.os-name@/fpm-make.sh
    keepgoing: false
    strategy: node-first
  uuid: d5c0f354-5c85-4181-9791-ab170f1a6612

This is looking very good, even though I'm still unhappy with the GPG passphrase thing. What I'm probably going to do is to have a passphrase-less key: whether I have that or the passphrase in clear text is probably six of one or half-a-dozen of the other.

Hey, can I have Rundeck tell my colleagues when a new package is ready? Yes, with notifications directly into our Slack channel.

notify slack

So, my workflow now goes like this: when I want to build packages, I kick my local Rundeck into action via its API:

#!/bin/sh

token=QeAJ32IWc1qYb0uuMzuEdOErCarm7Mqo
jobid=ec5bcb05-fe01-43bd-9bf9-240f15d18875

curl --header "X-Rundeck-Auth-Token:$token" http://localhost:4440/api/1/job/$jobid/run

Rundeck then launches the build on the (currently) three machines, takes the packages, signs them, and pushes the result out to our public repository.

job started via API

As soon as that's done, the last Rundeck job is to kick the Docker hub builder into action so that it builds our OwnTracks Recorder Docker container. (I'm aware a git push to Github could trigger Rundeck, but recall my installation is not publically reachable which is why I kick it into life manually.)

I knew it would be a lot of work, and I guessed the investment would amortize itself quickly; I was right. Automation is always a good investment.

May I now, finally, consider myself grown up?

View Comments :: automation :: 30 Jan 2016 :: e-mail

Roger Light has just committed not one, but two lovely new functions called mosquitto_subscribe_simple() and mosquitto_subscribe_callback() to the develop branch of libmosquitto.

This first, mosquitto_subscribe_simple(), allows us to wait for a broker to give us a specific number of messages in just one function.

#include <stdio.h>
#include <mosquitto.h>

#define mqtt_host "127.0.0.1"
#define mqtt_port 1883
#define COUNT 1

int main(int argc, char *argv[])
{
    struct mosquitto_message *msg, *mp;
    int n, rc;

    mosquitto_lib_init();

    rc = mosquitto_subscribe_simple(
        &msg,            /* struct mosquitto_message ** */
        COUNT,           /* desired message count */
        1,               /* want retained */
        "config/param",  /* topic */
        0,               /* QoS */
        mqtt_host,       /* host */
        mqtt_port,       /* port */
        "jp-one",        /* clientID */
        60,              /* keepalive */
        1,               /* clean session */
        NULL,            /* username */
        NULL,            /* password */
        NULL,            /* libmosquitto_will */
        NULL             /* libmosquitto_tls */
        );

    if (rc != MOSQ_ERR_SUCCESS) {
        printf("rc = %d %s\n", rc, mosquitto_strerror(rc));
        goto out;
    }

    for (n = 0, mp = msg; n < COUNT; n++, mp++) {
        printf("%s %s\n", mp->topic, mp->payload);
    }

   out:
    mosquitto_lib_cleanup();
    return (rc);
}

Assume I have a topic with a retained value which I want in a C program. By configuring COUNT to be 1 and setting want retain to true, mosquitto_subscribe_simple() will connect to the specified broker, subscribe to the topic, wait for the message, and immediately return, making the message available in msg.

If we ask for more than one message, mosquitto_subscribe_simple() will wait accordingly.

The second new function is called mosquitto_subscribe_callback() and expects a callback function you provide. This callback function is invoked for each and every message you've subscribed to. You return 0 from your function to have libmosquitto continue to listen for messages, or 1 to have it stop.

#include <stdio.h>
#include <mosquitto.h>

#define mqtt_host "127.0.0.1"
#define mqtt_port 1883

int func(struct mosquitto *mosq, void *userdata, const struct mosquitto_message *m)
{
    char *ud = (char *)userdata;

    printf("%s -> %s %s\n", ud, m->topic, m->payload);
    return (0);    /* True to break out of loop */
}

int main(int argc, char *argv[])
{
    int rc;

    mosquitto_lib_init();

    rc = mosquitto_subscribe_callback(
        func,            /* callback function */
        "hello",         /* user data for callback */
        "test",          /* topic */
        0,               /* QoS */
        mqtt_host,       /* host */
        mqtt_port,       /* port */
        "jp-cb",         /* clientID */
        60,              /* keepalive */
        1,               /* clean session */
        NULL,            /* username */
        NULL,            /* password */
        NULL,            /* libmosquitto_will */
        NULL             /* libmosquitto_tls */
        );

    if (rc != MOSQ_ERR_SUCCESS) {
        printf("rc = %d %s\n", rc, mosquitto_strerror(rc));
    }

    mosquitto_lib_cleanup();
    return (rc);
}

The user data you specify on invocation is passed directly into our func().

Roger has added a similar routines to Paho Python, which he also maintains. These are currently in the develop branch.

One of the e-mails I've been pending to work on for (gasp!) a year now is from Karanbir; we discussed a simple, cloudless, and inexpensive solution to driving a few lights for monitoring purposes, and I think I've found a solution which fits the bill, even if this can probably be done less expensively.

The Particle Photon I wrote about recently, can be hooked up to a strip of LEDs. I chose to use an Adafruit Dotstar digital LED strip which has 30 RGB LEDs on it.

monitoring strip

Instead of using the Particle cloud functions, I decided to drive the LEDs using MQTT, which makes things easy in my environment. The code on the Photon sets up the dotstar strip and expects an MQTT payload with the LED number (0 through 29) and the state (0 == OK, etc.).

/*
 * warner.ino (C) December 2015 by Jan-Piet Mens <jpmens@gmail.com>
 * Subscribe to an MQTT topic and read a string with two integers from
 * each message. The first integer specifies an LED to set (0 -- NUMPIXELS)
 * on a Dotstar strip, whereas the second integer is a Nagios-type 
 * number from Unknown (-1) through ERROR (2).
 *
 * To set LED 7 to OK:      mosquitto_pub -t led -m "7 0"
 * To set LED 9 to WARNING: mosquitto_pub -t led -m "9 1"
 *
 *
 * The Dotstar is Adafruit's https://www.adafruit.com/products/2238
 *
 *                            +-----+
 *                 +----------| USB |----------+
 *                 |          +-----+       *  |
 *                 | [ ] VIN           3V3 [ ] |
 *                 | [ ] GND           RST [ ] |
 *                 | [ ] TX           VBAT [ ] |
 *                 | [ ] RX  [S]   [R] GND [ ] |
 *                 | [ ] WKP            D7 [ ] |
 *                 | [ ] DAC +-------+  D6 [ ] |
 * Dotstar CLCK -->| [*] A5  |   *   |  D5 [ ] |
 *                 | [ ] A4  |Photon |  D4 [ ] |
 * Dotstar DATA -->| [*] A3  |       |  D3 [ ] |
 *                 | [ ] A2  +-------+  D2 [ ] |
 *                 | [ ] A1             D1 [ ] |
 *                 | [ ] A0             D0 [ ] |
 *                 |                           |
 *                  \    []         [______]  /
 *                   \_______________________/
 *
 *
 */
 
#include "MQTT/MQTT.h"
#include "dotstar/dotstar.h"
#include <stdlib.h>


#define NUMPIXELS   30 // Number of LEDs in strip
#define SUBTOPIC    "led"
#define PUBTOPIC    "led/status"

//-------------------------------------------------------------------
// Here's how to control the LEDs from any two pins (Software SPI):
//-------------------------------------------------------------------
// #define DATAPIN   D4
// #define CLOCKPIN  D5
// Adafruit_DotStar strip = Adafruit_DotStar(NUMPIXELS, DATAPIN, CLOCKPIN /* JPM */, DOTSTAR_GBR);

// Hardware SPI
Adafruit_DotStar strip = Adafruit_DotStar(NUMPIXELS, A5, A3); 

// char *host = "mqtt.example.org"
byte host[] = { 192, 168, 1, 130 };
MQTT client(host, 1888, callback);

static int colors[] = { 0x000000,   // Off    Unknown (-1)
                        0x00000B,   // Green  OK (0)
                        0x040004,   // Yello  WARNING (1)
                        0x0B0000 }; // Red    ERROR (2)

void ledon(int led, int status) {
    char payload[128];
    
    if (status >= -1 && status <= 2) {
        if (led >= 0 && led <= NUMPIXELS) {
            strip.setPixelColor(led, colors[status + 1]);
            strip.show();
            
            snprintf(payload, sizeof(payload), "LED %d set to %d at %ld", led, status, Time.now());
            client.publish(PUBTOPIC, payload);
        }
    }
}

void callback(char* topic, byte* payload, unsigned int length) {
    char p[length + 1];
    memcpy(p, payload, length);
    p[length] = 0;
    int led, status;
    
    if (sscanf(p, "%d %d", &led, &status) == 2) {
        ledon(led, status);
    } else {
        client.publish(PUBTOPIC, "can't parse two integers from payload");
    }
}

void mqtt_connect() {

    client.connect("photon-warner", PUBTOPIC, MQTT::QOS2, 0, "Oops: warner going down!");

    if (client.isConnected()) {
        client.publish(PUBTOPIC, "online");
        client.subscribe(SUBTOPIC);
    }
}

void setup() {
    strip.begin(); // Initialize pins for output
    strip.show();  // Turn all LEDs off ASAP
  
    mqtt_connect();
}

void loop() {
    if (client.isConnected()) {
        client.loop();
    } else {
        client.disconnect();
        mqtt_connect();
    }
}

For EUR 40.00 and a bit of LEGO which my Assistant In Charge Of LEGO Enclosures provided, I have a blinkenlights thing which shows me the state of 30 services. Add a few labels to the LEDs, and Bob's your uncle.

View Comments :: monitoring, blinkenlights, and photon :: 02 Jan 2016 :: e-mail

I don't recall ever having lost track of where I parked my car, but I know people who occasionally do, particularly when they visit unfamiliar locations. Christoph had a cool idea on how to use OwnTracks to pinpoint the location where you last exited your vehicle.

OwnTracks for iOS (and very soon also OwnTracks for Android) has support for iBeacons which let it quite precisely locate an iBeacon and, in particular, fire a transition event when approaching or leaving a beacon.

I configure OwnTracks to track a beacon by setting up a region on the device. Attila is the name I associate with the beacon, and the funny number behind it is the UUID of the iBeacon.

{
  "desc": "*Attila:DEADBEEF-ABBA-CDEF-1001-000000000001:1:2",
  "rad": -1,
  "lon": 7.47,
  "lat": 42.034,
  "tst": 1450453843,
  "_type": "waypoint"
}

The iBeacon itself I just chuck into the glove box of the car. When I get into the car or exit it, OwnTracks fires a transition event, and the app places the beacon icon on the map to indicate its current position.

iBeacon on OwnTracks app

The transition event itself is published to your MQTT broker and our Recorder, say, will store it accordingly. (It is a leave event as I am leaving the car; upon returning to the iBeacon in the car an appropriate enter event is fired.)

{
  "desc": "*Attila",
  "tid": "jp",
  "t": "b",
  "tst": 1451380551,
  "acc": 163.4991504437675,
  "_type": "transition",
  "event": "leave",
  "lon": 13.38506413201261,
  "lat": 52.51784009344821,
  "wtst": 1450453843
}

This way I always have the position of my car with me in case I once do forget where I've parked it.

View Comments :: GPS, iBeacon, and OwnTracks :: 02 Jan 2016 :: e-mail

Other recent entries