Ben and Alexander recently twisted my arm until I promised to create something which would consume Mosquitto’s $SYS/broker/# topic branch and write those values to collectd. You will know that that topic branch emits broker statistics via MQTT:

$SYS/broker/bytes/received 79665672
$SYS/broker/bytes/sent 27887950
$SYS/broker/load/messages/received/1min 70.10

My first thought was: let me add a service to mqttwarn for that (if you don’t know mqttwarn you may be interested in these two articles introducing it), but for the systems it was to run on, it was to be a standalone thing in C. The first cut of the program was operational within a couple of hours: it is hooked into collectd via collectd’s exec plugin which launches a long-lived process and reads metrics it issues from stdin.

It then occurred to me I could also handle JSON payloads easily enough, extracting an element’s value from the JSON to use as a number. One thing led to another, and I then wanted elements from the JSON payload to be interpolated into the metric names collectd is given, so I added that as well. The result is a rather fast minimal mqttwarn which hands metrics to collectd.

An ini-type configuration file provides the settings required to run mqttcollect, instead of a dozen command-line options. Hostname, port, CA certificate file for TLS, TLS-PSK are all supported. The nodename given to collectd is configurable (it defaults to the short uname) as is an optional prefix which is, well, prefixed to a metric to differentiate instances of a plugin.

[defaults]
host = localhost
port = 1883
username = jane
password = s1c#ret
; psk_key =
; psk_identity =
; ca_file =
; nodename = foob
; progname = mqttcollect
; prefix   = PREFIX

You configure any number of topics which mqttcollect will subscribe to, and you specify the type of metric as well as whether or not the MQTT topic should be translated before being handed off. There are three possibilities:

1. No translation. E.g a topic temperature/arduino is passed through unchanged.
2. Rewrite a topic (e.g. temperature/arduino) to hot/in/here.
3. Rewrite with JSON interpolation.

The first two mechanisms are described in the mqttcollect.ini.example file we provide. The third is a bit more difficult to grasp, so let me elaborate.

Assume for instance I subscribe to the wildcarded topic branch arduino/temp/+ and that the payload of messages which are published on that branch contains JSON with a celsius and fahrenheit temperatures as well as the name of the room in which the temperature was measured.

{"fahrenheit": 53.26, "celsius": 11.81, "room": "kitchen"}

I can have mqttcollect use each of the elements in the JSON (e.g. celsius, room, etc.) to construct the name of a metric. So, if I configure, say, a metric name of heat.{room} and, as shown above, the JSON payload has a { "room" : "kitchen" } in it, the metric name will be rewritten to heat.kitchen.

You’ll notice the < character followed by a word; this indicates that mqttcollect should retrieve the value of the metric from said JSON element. So, for example, <celsius means: the value of the metric should be taken from the payload’s { "celsius" : 11.81 } element. Likewise, <fahrenheit would consume that value, but who’d want that? ;-)

Putting this together, if I configure mqttcollect with this section:

[arduino/temp/+]
gauge = heat.{room}<celsius

mqttcollect will rewrite the metric name from the JSON payload and obtain the value for the metric from the same JSON payload, handing collectd the following line:

PUTVAL tiggr/mqttcollect/gauge-heat.kitchen 1431548550:11.81

A practical example we’re using this for is for our OwnTracks Greenwich devices which publish JSON payloads like this example (shortened for clarity):

{
    "_type": "location",
    "alt": 53,
    "tid": "BB",
    "vel": 62
}

mqttcollect will, when configured as below, produce three metrics per message it receives.

[owntracks/+/+]
gauge = vehicle/{tid}/speed<vel
gauge = vehicle/{tid}/altitude<alt
counter = vehicle/{tid}/odometer<trip

PUTVAL tiggr/mqttcollect/gauge-vehicle/BB/speed 1431543655:62.00
PUTVAL tiggr/mqttcollect/gauge-vehicle/BB/altitude 1431543655:53.00
PUTVAL tiggr/mqttcollect/counter-vehicle/BB/odometer 1431543655:672798.00

This lets us produce nice graphs with all sorts of useless information, such as at which altitude vehicles are driving at using InfluxDB and Grafana.

I’ve put the source code to mqttcollect up on this repository.