Yet another database, you ask? This one may be a bit different: etcd calls itself a highly-available key value store for shared configuration and service discovery.
The program is written in the Go language, but you can download binaries for multiple platforms, which makes it trivial to get started. In fact launching
./etcd suffices on a single node, whereupon I can then use the included etcdctl program to set my first configuration value:
Not really a big deal so far, eh? OK, let’s retrieve that value, first with etcdctl and then with curl:
etcd’s REST API allows me to get and set values, wait for a particular value to change, list keys, delete keys, and atomically test and set a value. etcd provides support for TLS as well as client authentication with X.509 certificates, and interestingly, keys in etcd can be set to expire after a specified number of seconds
One particular interesting point is clustering which offers consistency and persistence across etcd instances, and the cluster can talk over a different interface from “normal” client connections. The cluster automatically chooses a new leader when a cluster member goes down, and cluster members can also use TLS for protecting their communication.
Kelsey Hightower created confd which can watch for changes in values in etcd and create configuration files from templates. This program, also written in the Go language, uses a set of configuration files in which I specify etcd cluster members I want to speak to and files I want to create with it.
Although documentation of confd is sufficient, I had a beast of a time understanding where which file went. (The documentation says template files should go in
confdir/conf.d and I misunderstood the basename to be
confdir.) I’ll show you what I did:
The template source itself uses a Go templating engine which I haven’t looked at further (I don’t want to mix things up with Jinja2 …)
If I now launch confd, it will contact one of the configured etcd services, ask for the requested keys, process the template into its _dest_ination file and sit back to wait for a configurable interval before checking again:
I was introduced to etcd and confd by a customer who has made use of both these utilities. These people want to use Ansible for configuration management, but they also want to leverage existing settings from etcd, so I thought I’d create a simple lookup plugin for Ansible which queries etcd. I submitted a pull request for that today. Basically, usage will be like this:
Who knows: maybe this will be useful for some people.
Update: Kelsey Hightower took the suggestion I made in a comment below seriously, and he’s added DNS SRV record support to confd. This allows us to use the DNS to find the etcd servers, without having to specify a list of HTTP(S) URLs.