I tell people that my first contribution to the Ansible project was killing off the cows, but while researching for this post, I see memory fails me: the first written record of me contributing to Ansible is dated a bit earlier, on July 20th, 2012, but that doesn’t matter. By the way, it was quite a bit later that the cow thing became configurable. Be that as it may, it’s been almost seven years – a long time and a good time.

I’ve contributed a number of modules to Ansible, quite a few lookup plugins, documentation patches, and the piece de resistance was the documentation (a.k.a. ansible-doc) system which is still used today. I recall sitting on the floor in a conference center in Boston getting ready to push what I lovingly called a “jumbo patch” which would target each and every module: I had Michael’s ok that he wouldn’t commit anything for 24h so that my patch would apply cleanly. It was very exciting for me, having to wield a new utility (git) and hoping I wouldn’t mess anything up. A lot of adrenalin flowed. Good times.

Ansible’s grown enormously since then, and I’ve not always been able to pay attention to it as much as I’d have liked to, but I’ve used it and have also had the pleasure of training a large number of people in using Ansible. In the course of the last two or three years, I’ve met two hundred people interested in how Ansible works. That is a lot.

During that time I’ve occasionally commented on a pull request, responded to an issue, and I’ve even submitted at least one new lookup plugin for consideration: it’s the lmdb lookup plugin.

I’m telling you this story to set a bit of background for what follows and to demonstrate that Ansible is a project I’ve been very fond of.

If you look at that last link, you’ll see I submitted it two years ago, in May 2017. Since then, I’ve had to rebase the plugin once because of changes which made it unmergeable. Judging by labels added (by a human?) the plugin affects Ansible version 2.4 (current is 2.8.1), and it needs all sorts of things like a maintainer (huh?), a revision, tests, and whatnot. Does that mean I should leave it as is? Does that mean I should just close the PR and forget about it? I don’t know. I do know one thing: I will not rebase it again, and the powers that be must just as well close the PR as I’ve completely lost interest in it.

Ansible is a hugely popular project; it’s approaching 38K stars at the time of this writing, and it has 16K forks. I don’t know how many people Red Hat has working for Ansible, but I guess there to be several committers, and I know some of them by name and a couple personally.

In my opinion they’re being inundated.

At this time Ansible has 3900 open issues, and 1900 open pull requests. The oldest open PR is dated March 2015, and the oldest open issue is two months older. Never ever will it be possible to process all that, if only because the code has changed so much meanwhile that most of the PRs can likely no longer be applied cleanly, and I would think a lot of the issues are no longer even valid. What I also expect is that many (possibly first-time) contributors have meanwhile lost interest in their contribution just as I’ve done for the lmdb lookup plugin.

I was at a customer site recently a fortnight ago, and we had a devil of a time because it appeared that the archive module was omitting files from an archive. Hard to believe, I know, but an empty file gintonic in a directory-to-be archived, never appeared on the target hosts. What?!

After a lot of testing, and upon beginning the tedious process of submitting a bug report, I discovered this bug was already reported in January 2018. 1.5 years ago. This module error is not just some cosmetic thing of a feature request. It is a bug which causes data loss, and somebody added a waiting_on_contributor label to it, and I am starting to wonder how long the wait is going to be, particularly since a kind newcomer provided a fix in February of this year. (It also saddens me that this contributor may have been frightened off by being requested to provide integration tests for the fix.) To be fair, the archive module is a module maintained by the community which its documentation clearly states; not being a core module, the Ansible project proper is not responsible for it. So Ansible includes batteries but some batteries are not be as powerful as others?

This is but one example which I use as it’s the one which happened most recently to me.

In the course of the past year I’ve spoken to a small handful of Red Hat people about this. Unofficially I see a lot of nodding, and I see some organization which has taken place in the project (repositories, communities, etc.), but I don’t see PRs and issues diminishing in great numbers which could, of course, be due to an ever increasing number of submissions.

I proposed a dramatic and drastic solution: close the 4000 issues, close the 1900 PRs, and start over. Explain why this is being done. (It’ll be less painful to have a week of bad press than permanent bad press.) Have people begin to raise issues and PRs again. Brutally close whatever makes no sense. Merge the PRs quickly after a group of peers have OK’d them. (e.g. 3 peers apply an OK, the PR gets merged, somewhat how the OpenBSD project does it, I think). If an issue remains unchanged for, say, 3 months, or a PR remains without activity for that time, it’s marked as stale and will be closed after a short grace period. Ansible is an Open Source project backed by a strong company. I for one would much prefer to be told “no, we don’t want that” than to be left hanging and guessing.

Ansible users are telling me their trust in Ansible is diminishing: every release brings breakage (e.g. for Solaris, OpenBSD, a lot of non-Linux). PRs are not being merged in a timely fashion, and issues are not being solved. (Here’s an example of an issue addressed to me, and I have clearly ignored it.)

This makes me very sad.

Ansible Communities was an attempt by Dag Wieers to give people a trusted process and some privileges to work independently on sets of.modules, but it’s not got a lot of traction, unfortunately.

The new Ansible Collections, proposed in Ansible 2.8, could alleviate the situation with modules by moving the modules to Galaxy and to people’s repositories. This would mean that Ansible in future might just be a core with a minimal set of modules. My worry is that the number of disparate versions of a particular module will explode: contributors will go away, there’ll be no central module repository and that can mean dozens of versions of a particular module, each will different features and/or patches. For example, searching Galaxy for apache server filtered by Role produces 869 results, not all of which really are an Apache server, and there are 10 roles to install an Mosquitto broker; that’s a lot of choice… And many users I speak to in Europe are quite uninterested in Galaxy; are you really willing to pull configuration code you didn’t write to blindly deploy onto your servers? I smell lots of cake ;-)

On the plus side, moving most of the modules out of Ansible will make core developers happier, as they don’t feel responsible for the dozens of contributed modules anyway. All the more reason to clear out the large number of issues and PRs.

Let’s see what happens.

Update: Ansible responds to my blog post: Thoughts on restructuring the Ansible project.