Applying a Puppet configuration to a machine or group of machines (called nodes) entails defining classes which are applied to said nodes. Classes are defined in manifest files and nodes are often defined as a collection of classes to be applied to a particular machine. These definitions are typically also contained in files.
Instead of classifying nodes in files I can use an external node classifier or ENC, basically a program that outputs a YAML document when passed a node name. (Puppet can also retrieve classification data directly from an LDAP directory, but even then I’d use the ENC method as it is more flexible.) This is good because I can obtain a list of classes for a machine from any source I desire, e.g. a database, and Puppet will, as long as the classes exist of course, happily apply the configuration I create on the fly to the node.
I’ll also shortly discuss the generate and extlookup functions because they also relate to obtaining information dynamically within a class.
Here’s the YAML document my ENC outputs. I’m applying two classes called “dyn” and
“ssh”. The former has three parameters (“ports”, “smtpmx”, and “users”), and the
latter none. Note that the values “parameters” in the element are passed into
the Puppet class as variables, and that these are not associated with a particular
class. (I trust this will become clear in a moment, when we see the class definition for dyn
.)
---
classes:
dyn:
ports:
- 25
- 587
smtpmx: gw.mens.de
users:
alex:
home: /nfs/alex
uid: 502
jane:
home: /home/jane
uid: 201
ssh: ''
parameters:
location: Chicago
manager: Jane Doe
my_memory: 216
nodename: cp09.mens.de
The ENC can, as shown in the above YAML example, provide classes with parameters (a.k.a. parametrized classes). The “dyn” class is passed the specified parameters. Class parameters defined in the ENC missing in the class definition result in an error when the agent attempts to apply the class. This error 400 (“Invalid parameter … on node …”) is very useful in tracking down errors.
Here is the class definition:
class dyn($smtpmx, $ports, $users) {
file { "/tmp/d2":
owner => 'puppet',
group => 'root',
mode => 0400,
content => template("${module_name}/tm.erb"),
}
# Use inline_template() to build an /etc/passwd-like list of
# users "username:home:uid;". Lines are semi-colon-terminated
$res = inline_template("<% users.each do |key,val| -%>
<%= key %>:<%= users[key]['home'] %>:<%= val['uid'] %>;<% end -%>")
# Now create an array from that result
$users_array = split($res, '[ ;]')
define one_user() {
$u = split($title, '[:]')
$username = $u[0]
$userhome = $u[1]
$useruid = $u[2]
$msg = sprintf('user=%s, home=%s, uid=%d', $username, $userhome, $useruid)
file { "/tmp/d2.$username":
# Two examples:
# content => "Hello: your home is in $userhome",
# content => generate('/tmp/jpgen')
content => inline_template(generate('/bin/jpgen'))
}
}
one_user{ $users_array: }
$myname = extlookup('myname', 'NONAME')
notice("*************************** $myname ")
}
This is the template called tm.erb
:
- The
location
andmy_memory
variables are from theparameters
section of the YAML produced by the ENC. - The rest are parameters from the parametrized class.
This is a template for [<%= location %>]
Memory: <%= has_variable?("my_memory") ? my_memory : '64' %>
Server passed == [<%= smtpmx %>]
Userlist:
<% users.each do |key,val| -%>
Home for <%= key %> is <%= users[key]['home'] %>
userid == <%= val['uid'] %>
<% end -%>
And here is the resulting file d2
as produced by Puppet applying the template:
This is a template for [Chicago]
Memory: 216
Server passed == [gw.mens.de]
Userlist:
Home for alex is /nfs/alex
userid == 502
Home for jane is /home/jane
userid == 201
My intention in experimenting with parametized classes was to create a class
with which I can define a list of users as parameter. I don’t seem to be
able to use control structures (i.e. loops) within a class definition (or at least
I haven’t found a way to do that) which is why I have to resort to inline_templates()
and arrays to create an array of users and then manually process that array. This is
done by the defined one_user()
resource:
- With an
inline_template()
I create a string containing a colon-separated list of values, with each “user” separated from the next with a semi-colon. - I then
split()
this into an array. - The
one_user()
resource is invoked with this array (note the trailing colon on the variable name) which appears to call myone_user()
“function” once for each array element, setting$title
to the content of the array element.
More data from server: generate
Puppet’s generate()
function invokes an executable program on the server and
collects/returns that program’s output. Unfortunately, generate()
doesn’t
accept arguments so it’s a bit limited. (I can’t say “generate for this user” or
“generate for this node”.) About the only thing the program can do is check its
environment to determine for which node it is being invoked
($SSL_CLIENT_S_DN_CN
). Although I can’t pass parameters to the program in generate()
I can use its result as a string I pass to inline_template()
as shown above.
The jpgen
program in this example is trivial:
#!/bin/sh
cat <<!
Hello, this is <%= nodename %>: how are
you, <%= username %>?
!
One of files produced by the file
resource in one_user()
then contains:
Hello, this is cp09.mens.de: how are
you, jane?
External data: extlookup
External lookups from a database into a class are possible with extlookup, a function that extracts data from a CSV file. (There seem to be multiple iterations of extlookup floating around with differing capabilities including a pluggable version with JSON and YAML lookups.) I’m limiting this discussion to the extlookup function as I have access to in a 2.7 Puppet release. (Puppet’s quite idiotic release numbering (0.25, 0.26, 2.7) makes searching for solid information difficult at best. Tip: remove all leading zeroes and decimal points and read what’s left as the release number. But I digress.)
Because it wasn’t obvious to me when I first used this, I’d like to explicitly point
out that the CSV files
extlookup consults are on the Puppet master (i.e. the server) and not on the
client node. In my site.pp
I configure the order in which extlookup
searches
for CSV files:
$extlookup_datadir = "/etc/db/"
$extlookup_precedence = ["%{fqdn}", "common"]
Puppet will first try the fully qualified domain name (i.e. /etc/db/cp09.mens.de.csv
)
and will fall back to /etc/db/common.csv
. Suppose I have the following line
in common.csv
myname,John Doe <support@my.de>
The notice()
in my class (see above) will print the contact and e-mail address. If
extlookup()
doesn’t find myname
anywhere, extlookup()
will return the default
value I specified as “NONAME”.
And here, for good measure, is the small Python program I used to create above
ENC. Note how I read the content of the the node’s facts from a file populated
by Puppet before my ENC is invoked. The file contains
all the node’s facts, including special facts.
They have been collected and deposited on the puppet master in a file called
/var/lib/puppet/yaml/facts/
nodename.yaml
. The ENC classifier can read
that YAML to find facts about the node it is creating classes for. (The YAML
in this file contains a few Ruby objects which I remove with a text substitution instead
of doing it properly.)
#!/usr/bin/env python
import sys
import re
from yaml import load, dump
n = sys.argv[1]
# Too lazy to do it properly
text = re.sub('.*!ruby\/.*','', open("/var/lib/puppet/yaml/facts/" + n + '.yaml').read())
facts = load(text)['values']
classes = [ 'ssh', 'dyn' ] # obtain from DB
# Convert list to dict with empty values
cdict = {}
cdict = dict((x, "") for x in classes)
userlist = {
'jane': {
'uid': 201,
'home': '/home/jane'
},
'alex': {
'uid': 502,
'home': '/nfs/alex'
}
}
cdict['dyn'] = { "smtpmx":"gw.mens.de", "ports" : [ 25, 587], "users" : userlist }
node = {
'classes' : cdict,
'parameters' : {
"my_memory" : 216,
"location" : facts['location'], # "Hamburg",
"manager" : "Jane Doe",
"nodename" : facts['fqdn']
}
}
dump(node, sys.stdout,
default_flow_style=False,
explicit_start=True,
indent=10 )
All in all, these features make for very powerful combinations when designing a Puppet infrastructure. And as I’m a beginner, I hope I haven’t made any grave mistakes here.