I created msoak as a utility to subscribe to multiple MQTT brokers simultaneously and consume any number of topics; it soaks up what they have to offer and prints the topic/payload to stdout. I pipe that output to tinylog for storage, compression, and file rotation. I’ve been using this setup to log from a bunch of brokers ever since msoak was born two years ago.

Today, while looking at previously-logged data in preparation for consumption of future similar data, I discovered to my horror, that tinylog(8) truncates lines at what I first assumed was 1024 but actually is 1000 bytes.

I simply hadn’t noticed because most of the payloads we save are less than 1000 bytes. It’s my fault, as it’s documented, but I’d overlooked that sentence.

tinylog ignores empty lines, truncates lines longer than 1000 characters

Within minutes I found the definition and I’m now good. (80k is really sufficient for the moment.)

--- perp-2.07/perp/tinylog.c	2013-01-11 16:10:32.000000000 +0100
+++ perp-2.07-jp/perp/tinylog.c	2022-05-25 17:53:35.000000000 +0200
@@ -96,7 +96,7 @@
 /* default maximum size for log file (bytes): */
 #define CURRENT_MAX  100000
 /* maximum size for line in log file (bytes): */
-#define LOGLINE_MAX    1000
+#define LOGLINE_MAX    80000
 /* pause on exceptional error (billionths of second): */
 #define EPAUSE  555444321UL

They say a backup is only as good as its recovery. Very true.