I’ve never used Yahoo! Pipes, but I understand it’s a construction kit with which I can build things which take their input from “other things” (i.e. Internet resources). It’s probably grand, but as I just said: I’ve never used it.
For some time now, I’ve been very enthusiastic with all things related to MQTT as faithful readers know. Most of the time I have programs which need to produce messages to be sent over MQTT or, conversely, I have consumers of MQTT messages with payloads that need transforming or need to be stored somewhere. The outcome of this is a pretty large collection of little programs, each of which do a dedicated task. One example, is OwnTracks’s m2s program. It subscribes to a topic branch, and for each message, decodes the payload, does a couple of (optional) HTTP lookups, and writes the result into a relational database.
All in all, I’m pleased with the way it works, but I’ve always thought it must be possible to do this kind of message manipulation / transformation somewhat easier. In other words, without having to write custom programs, ensure they’re running, etc.
There is a way, using Node-RED.
Node-RED is a visual tool for wiring the Internet of Things. Node-RED, created by Nick O’Leary, is built upon Node.JS. It has a Web-based rapid-prototyping panel (see screen shot above), and it offers a number of building blocks I click and wire together. The building blocks are arranged into groups such as:
- Inputs (e.g.
- Outputs (e.g.
- Functions (e.g.
- Storage (e.g.
What I then do, is to drag the building blocks I want onto the work area and wire those
together. If necessary, and it will sometimes be, I add
function blocks which contain
function blocks can have more than one output (see my
transform payload function
It is fascinating! (I don’t use the a-word.)
The flow shown in the screen shot above works as follows:
- It subscribes to an MQTT topic branch of
- Arriving messages flow into the
topicswitch, which checks the topic name. Messages published on topic
nt/notifyare passed to
notif(Growl, in green on the top right), messages published on topic
nt/dataare passed to the
httprequest node. It does an HTTP POST of the payload in the message, and reads the returned HTTP body (here in JSON), which is handed off to
transform payload, a function with two outputs: 1)
mysql, and 2)
When everything is wired up, I hit the big red “Deploy” button on the UI, and off Node-RED goes.
Let me show you what happens when I publish the following message with
A running subscription to the same broker will show me:
The first message (
nt/data) comes from the “manual” publish with
mosquitto_pub. The second (
nt/NL) is the result of the Node-RED “piping” to an HTTP POST, obtaining that result and re-publishing to a different topic (and you’ll believe me when I say the output was also stored in MySQL thanks to it being piped to
This really is brilliant.