This site is now 100% read-only, and retired.

cfengine [3/3] : Using cfengine in a client/server setup

Posted by Steve on Tue 23 Aug 2005 at 09:44

After the previous coverage of cfengine we'll now look at actually installing it and using it for real on a number of different hosts. The rules will come from one central host and be automatically pushed to a collection of managed servers where they will be executed.

The rules file cfagent.conf which controls the jobs which are actually executed we've previously discussed.

For the purpose of this example we will be using three hosts:

  • mystery.my.flat
    • The central server from which all rules are downloaded, and which manages the clients.
  • scratchy.my.flat
    • A client we wish to remotely manage.
  • lappy.my.flat
    • Another client system we wish to remotely manage.

As previously mentioned installing cfengine is as simple as running :

apt-get install cfengine2

(Here cfengine2 is the most recent version of the software; newer and incompatible with the previous package cfengine).

After installing the package upon each host you'll be presented with a debconf dialog asking if you wish to start several processes at boot time, along with an explanation of what each process is used for.

I chose to enable all services at boot time for the moment, although you'll likely not need all services running upon each host.

Now comes the hardest part actually configuring the software to do something useful. I strongly recommend that you also install the cfengine2-doc package upon at least one host and read the information it installs. This will be located beneath the directory:

/usr/share/doc/cfengine2-doc/
|-- examples
`-- html

To do someting useful you'll need two things:

  • A client-server setup which we'll explore in this article.
  • A collection of useful rules, which you'll largely have to create by yourself (or find from elsewhere!).
Overview

We are going to setup is a centralised server which will host a file cfagent.conf containing the actions to be conducted upon each of the hosts we manage.

Each managed client will retrieve this rule file and then execute the rules locally.

The server itself will be able to force "pushes" of this file, and thus execute the rules upon any of the managed clients, either individually or en masse.

A Note On Security

If you setup cfengine in anger one of the biggest difficulties you'll encounter is getting the keyfile negotiation to work properly.

As discussed in the introductory piece cfengine uses a two-way trust. This means that the server's public key must be copied to the client, and the client's key must also be known to the server.

The fine documentation will explain the process of generating the keys with the command cfkey, and copying/renaming the keys appropriately. However due partly to misleading error messages and a lack of good discussion this is a common stumbling point.

As the communication is key to getting something working if you don't manage to get the keys setup correctly you'll not get anything working if you have problems.

So instead of using keys I'm going to ignore keyfiles completely - and trust all machines on the LAN.

This simplifies the description of the setup enormously; but it does weaken security.

(With a bit of firewalling it is possible to setup each client such that it only accepts connections from the central host. If you choose to ignore keyfiles, as I do, then this is something you should investigate.)

Setting Up The Server

The role of the server is going to be twofold:

  • It contains the central cfagent.conf file which has the rules to be applied to hosts.
  • It maintains a list of clients which it manages.

What is less obvious is that the clients have to fetch the most current version of the cfagent.conf file on every run (if it has changed), so to make that explicit I will create a new location for this file to exist in:

mkdir -p /var/lib/cfengine2/masterfiles/inputs

Once you've done that you can install the cfagent.conf file there. A simple example file is:

#
# /var/lib/cfengine2/masterfiles/inputs/cfagent.conf
#
# Master cfagent file on the server, 
#
control:
  domain = ( my.flat )
  access = ( root )
  cfrunCommand = ( "/usr/sbin/cfagent" )
  actionsequence = ( files directories tidy resolve )
  maxage = ( 7 )


#
# Fix some basic file permissions.
#
files:
  /etc/sudoers mode=440 owner=root group=root   action=fixall
  /etc/passwd mode=644  owner=root group=root   action=fixall
  /etc/shadow mode=640  owner=root group=shadow action=fixall

#
# Clean out *ALL* files older than $(maxage) days from /tmp.
#
# Clean out files older than $(maxage) which match the pattern *~
# inside user home directories.
#
tidy:
  /tmp pattern=* age=$(maxage) recurse=inf
  /home pattern=*~ age=$(maxage) recurse=inf

directories:
  /tmp mode=1777 owner=root group=root

resolve:
        "search my.flat"
        192.168.1.1
        "# Edit with cfengine"

(This file is available for download.)

As you can see there is a "domain" setting right at the top, this is important as most of the cfengine files will use a "domain" when testing if accesses are allowed, as we'll see in the server configuration file.

You should change all domain settings to match your local network setup.

The server's configuration file is /etc/cfengine/cfservd.conf and ours will look like this:

#
# /etc/cfengine/cfservd.conf - for the server


control:
  domain = ( my.flat )
  TrustKeysFrom = ( 192.168.1.0/24 )
  AllowUsers = ( root )

any::

  IfElapsed = ( 0 )
  ExpireAfter = ( 15 )
  MaxConnections = ( 50 )
  MultipleConnections = ( true )


grant:

   # Grant access to all hosts in my.flat.
   /var/lib/cfengine2/masterfiles/inputs   *.my.flat

You can download this file if you wish.

For your setup you will need to modify the two mentions of my.flat, along with the local IP address range you're using.

The last job for the server is to build up a list of hosts you're going to manage. Do that by creating the file /etc/cfengine/cfrun.hosts:

domain = my.flat

#
# Clients
#
scratchy.my.flat
lappy.my.flat

(Again this file is available for download.)

Now that you've created the two file cfrun.hosts, and cfservd.conf inside /etc/cfengine you can restart the server:

/etc/init.d/cfengine2 restart
Setting Up The Clients

The client setup really consists of three things:

  • Creating an update.conf file.
  • Creating a minimal cfservd.conf file.
  • Connecting to the server once manually, required to get the keys setup.

As we've covered the server we know that the cfagent.conf rule file will be stored on the host in the directory /var/lib/cfagent/masterfiles/input - the update file contains the magic which will copy it from there into /etc/cfengine on our host - where it can actually be used.

This means that our rulefile will always be current.

The /etc/cfengine/update.conf file will look like this:

#
# /etc/cfengine/update.conf - for the clients
#

control:
  actionsequence  = ( copy )
  domain          = ( my.flat )
  policyhost      = ( mystery.my.flat  )

# smtpserver      = ( smtp.domain.com )
# sysadm          = ( address@bogus.example.com )

  master_cfinput  = ( /var/lib/cfengine2/masterfiles/inputs )
  repository      = ( /var/lib/cfengine2/outputs )


#
# Download the most recent 'cfagent.conf' file from the
# server, and install it to /etc/cfengine
#

copy:
     $(master_cfinput)/cfagent.conf    dest=/etc/cfengine/cfagent.conf
                                       mode=600
                                       server=$(policyhost)
                                       force=true
                                       trustkey=true



(This file is available for download.)

You will need to change "my.flat" to your own domain, and "mystery.my.flat" to the name of your server if you wish to use it. No other changes should be required.

Once you've created the update file you're ready to create the cfservd.conf file for this client.

The following is a good sample:

#
# /etc/cfengine/cfservd.conf for the clients
#

control:
  domain = ( my.flat )
  AllowConnectionsFrom = ( 192.168.1.0/24 )
  TrustKeysFrom = ( 192.168.1.0/24 )
  cfrunCommand = ( "/usr/sbin/cfagent" )
  AllowUsers = ( root )
  LogAllConnections = ( true )
  IfElapsed = ( 1 )
  ExpireAfter = ( 15 )
  MaxConnections = ( 50 )
  MultipleConnections = ( true )



grant:
  /usr/sbin/cfagent  *.my.flat

(This file is available for download.)

Again you will need to modify the domain, grant, and the various IP addresses if you wish to use this yourself.

Now that you've copied everything onto the client properly you should be ready to restart the cfengine processes and run the initial agent test.

To restart run:

/etc/init.d/cfengine2 restart

Once this is done become root upon the client and run:

root@scratchy:~# cfagent -q
cfengine:: Trusting server identity and willing to accept key from mystery.my.flat=192.168.1.80

This should give you a message indicating successful trust, as we see, and you should discover the key of the server has now appeared within /var/lib/cfengine2/ppkeys:

root@scratchy:~# ls -l /var/lib/cfengine2/ppkeys/
total 12
-rw-------  1 root root 1743 2005-08-22 06:44 localhost.priv
-rw-------  1 root root  426 2005-08-22 06:44 localhost.pub
-rw-r--r--  1 root root  426 2005-08-22 17:22 root-192.168.1.80.pub
Using the installation

Now that you've setup the basic connection with the server and the clients you're ready to have fun!

To get started you should experiment with adding rules to the master cfagent.conf you've installed in /var/lib/cfengine2/masterfiles/inputs on the server.

To execute these rules against a host run:

cfrun hostname.as.in.cfrun.hosts

(e.g. "cfrun scratchy.my.flat").

Or against all managed hosts with just cfrun.

Note that the rules dont' all get applied in one run, sometimes it will take several executions.

As a sample I've added a rule to make sure the file permissions on /etc/gshadow are correct:

files:
  /etc/gshadow mode=440 owner=root group=root   action=fixall

Running this against a host gives me this :

root@mystery:~# cfrun scratchy.my.flat
cfrun(0):         .......... [ Hailing scratchy.my.flat ] ..........

cfengine::
         Update of image /etc/cfengine/cfagent.conf from master
        /var/lib/cfengine2/masterfiles/inputs/cfagent.conf on mystery.my.flat
cfengine:: Moved /etc/cfengine/cfagent.conf.cfsaved to repository location
        /var/lib/cfengine2/outputs/_etc_cfengine_cfagent.conf.cfsaved
cfengine:scratchy: Object /etc/gshadow had permission 0, changed it to 640
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(As you can see I ran "chmod 0 /etc/gshadow" on scratchy first to see that the change worked)

What Now?

If you're using this in production you'll want to do three things, most likely:

  • Learn more about the rules you can include in cfagent.conf.
  • Learn all about the key-based authentication.
  • Examining "classes" so you can have different rules for different OS types.

 

 


Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by oxtan (80.126.xx.xx) on Wed 24 Aug 2005 at 21:10
[ View Weblogs ]
great series of articles! Thaks. I am definately going to get to learn this stuff for my work.

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by Steve (82.41.xx.xx) on Thu 25 Aug 2005 at 00:27
[ View Weblogs ]

Thanks for the feedback!

If you have a good success story to report afterwards it'd be great to hear about it ..

Steve
-- Steve.org.uk

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by Anonymous (203.41.xx.xx) on Wed 7 Sep 2005 at 05:06
I finally got cfengine going after reading the article, thanks.

You might want to check the follow

mkdir -p /var/lib/cfengine/masterfiles/inputs

probably meant to be

mkdir -p /var/lib/cfengine2/masterfiles/inputs

as that's what's in the update.conf, and

Once you've created the update file you're ready to create the cfserved.conf file for this client.

probably meant to be cfservd.conf not cfserved.conf

Thanks again for the articles, they hit the spot.

chesty

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by Steve (82.41.xx.xx) on Wed 7 Sep 2005 at 11:12
[ View Weblogs ]

Thanks for the corrections. I've fixed the mistakes you spotted, and updated the associated downloadable files to match.

Steve
--

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by Anonymous (66.60.xx.xx) on Wed 21 Dec 2005 at 21:03
This tut rocks. Just enough to get you going but not too much info to kill the learning process.

Thanks

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by diesis (83.211.xx.xx) on Fri 23 Feb 2007 at 14:44
Another fix:
/etc/init.d/cfengine2 restart
not cfengine restart

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by Steve (80.68.xx.xx) on Fri 23 Feb 2007 at 15:00
[ View Weblogs ]

Thanks a lot - I'm not sure how I could have missed that.I've updated the text now.

Steve

[ Parent ]

Scheduling
Posted by diesis (83.211.xx.xx) on Thu 22 Mar 2007 at 17:15
It's not clear to me which is the parameter that schedules updates pushed to the clients (or they are pulled to ?). Where I tune this ? Another strange thing is the "antispam" feature that prevents frequents updates: how can I disable this for testing purposes ?

[ Parent ]

Re: Scheduling
Posted by Steve (80.68.xx.xx) on Thu 22 Mar 2007 at 17:32
[ View Weblogs ]

The scheduling mostly comes from the clients - if you enable the service it will "know" when to run.

Add something like this to cfagent and push it out manually to kick off the process:

control:
  domain          = ( my.flat )
  ...
  ...
  Schedule        = ( Min30_35 )

This says to run each hour sometime between 30 and 35 minutes past the hour. This is how you want to do it - not fixed at a given time. N hosts each with NTP-synchronised clocks all going off immediately will kill your server/network!

To run more times per hour you could add something like this:

  Schedule        = ( Min00_05 Min30_35 Min Min45_50)

You can also schedule to run only on certain days, or only during the week days for example. See the cfengine documentation for more details.

From the server side you can force an update with "cfrun [hostname1] [hostname2]" - but generally you shouldn't need to.

As for the "antispam" do you just mean splay-time ? You can add --no-splay to disable that..

Steve

[ Parent ]

Re: Scheduling
Posted by Anonymous (134.158.xx.xx) on Wed 21 May 2008 at 13:19
I'm trying to make cfexecd run on week days, during work time. Without success.
Has anyone done anything similar ?

Here's my cfagent.conf schedule parameter :

# schedule = ( Monday.(Hr09|Hr10|Hr11|Hr12|Hr13|Hr14|Hr15|Hr16|Hr17)
# Tuesday.(Hr09|Hr10|Hr11|Hr12|Hr13|Hr14|Hr15|Hr16|Hr17)
# Wednesday.(Hr09|Hr10|Hr11|Hr12|Hr13|Hr14|Hr15|Hr16|Hr17)
# Thursday.(Hr09|Hr10|Hr11|Hr12|Hr13|Hr14|Hr15|Hr16|Hr17)
# Friday.(Hr09|Hr10|Hr11|Hr12|Hr13|Hr14|Hr15|Hr16|Hr17) )
schedule = ( Monday.Hr09 Monday.Hr10 Monday.Hr11 Monday.Hr12
Monday.Hr13 Monday.Hr14 Monday.Hr15 Monday.Hr16 Monday.Hr17
Tuesday.Hr09 Tuesday.Hr10 Tuesday.Hr11 Tuesday.Hr12
Tuesday.Hr13 Tuesday.Hr14 Tuesday.Hr15 Tuesday.Hr16 Tuesday.Hr17
Wednesday.Hr09 Wednesday.Hr10 Wednesday.Hr11 Wednesday.Hr12
Wednesday.Hr13 Wednesday.Hr14 Wednesday.Hr15 Wednesday.Hr16 Wednesday.Hr17
Thursday.Hr09 Thursday.Hr10 Thursday.Hr11 Thursday.Hr12
Thursday.Hr13 Thursday.Hr14 Thursday.Hr15 Thursday.Hr16 Thursday.Hr17
Friday.Hr09 Friday.Hr10 Friday.Hr11 Friday.Hr12
Friday.Hr13 Friday.Hr14 Friday.Hr15 Friday.Hr16 Friday.Hr17 )

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by Anonymous (216.31.xx.xx) on Tue 12 Jun 2007 at 23:17
The 'Note on Security' is very misleading, if not just wrong. You cannot 'ignore keyfiles completely', even if you want to. And in fact, the given configuration in no way ignores them, it just makes the initial exchange easy. If you are ok with accepting an ssh key the first time you connect to a remote host, then 'TrustKeysFrom' (in cfservd.conf on a server) and the 'trustkeys=true' parameter in the first copy action (update.conf on a client) accomplishes the same thing. Thereafter, there is NO trust, keys have to match on both ends or no communication takes place. Ssh will let you ignore a key mismatch, cfengine will not.

Some folks will manually distribute cfengine keys with scp thinking they are being more secure than letting cfengine handle the exchange, when in fact the ssh keys were themselves initially exchanged 'on trust'. Silly.

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by rpetre (83.166.xx.xx) on Tue 3 Jul 2007 at 15:57
SSH key trust can be enforced via DNS or manual updated known_hosts files, one can hold on to his paranoia (read "trust chain") continuously since OS install.

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by etptupaf (158.227.xx.xx) on Fri 20 Jul 2007 at 09:15
I bookmarked this series of articles, which set me going with cfengine. Thank you very much. I am slowly building a cfagent.conf which already does quite a few things useful for the machines I administer. However, there is something that I cannot get done, and I wonder if someone could provide a working example. I would like to be able to install Debian packages via cfengine. Reading the docs I thought it would be enough something like:


DefaultPktMgr = (dpkg )

in the control: section and then something like


packname action=install pkgmgr=dpkg

in the packages: section. Doing that gives a message about the installer command undefined. I added to the control: section:


DPKGInstallCommand = ( "/usr/bin/apt-get install %s" )

but still doesn't work. Could someone tell me what is the right way of doing this?

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by Anonymous (62.121.xx.xx) on Tue 26 Feb 2008 at 23:11
I know it's probably massively too late, but the answer is probably that apt-get locked up whilst answering a question (do you really really want to install that package or were maybe you just kidding when you asked for it? for sure? absolutely and for the last time for certain sure?)

Try

DPKGInstallCommand = ( "/usr/bin/apt-get -y install %s" )

instead.

P.S. at leat your post easily let me work out what I wanted to do.. Thanks.

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by Anonymous (90.58.xx.xx) on Tue 12 Aug 2008 at 23:20
Still have a problem here. I did exactly the same and have the error :

cfengine:myhost:apt-get -y install %s: Reading package lists...
cfengine:myhost:/usr/bin/apt-get -y install %s: Building dependency tree...
cfengine:myhost:/usr/bin/apt-get -y install %s: E: Couldn't find package %s
Finished with cfagent.conf

Any idea ?

thanks

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by Anonymous (90.58.xx.xx) on Wed 13 Aug 2008 at 09:43
Ok, found the issue :

I had insert a "classes:" line just before the dpkg declarations, so they were not in "control:" any more....

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by 270377471 (58.246.xx.xx) on Fri 20 Mar 2009 at 05:25
i operated as this article ,but i got errors "Server returned error: Unspecified server refusal (see verbose server output)
cfengine:dd.debian.com: Can't stat /var/lib/cfengine2/masterfiles/inputs/cfagent.conf in copy" i checked the version both of clients and server they are different,maybe it caused the errors? i dont know,who can help me ? :(

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by Anonymous (87.209.xx.xx) on Wed 1 Apr 2009 at 20:34
I have the same problem. Did you find an answer?

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by Anonymous (87.209.xx.xx) on Wed 1 Apr 2009 at 21:04
What might help to debug the problem is running 'cfservd -F -v' on the server. It looks like a problem with granting access to the files to a specific host in cfservd.conf

[ Parent ]

Re: cfengine [3/3] : Using cfengine in a client/server setup
Posted by 270377471 (58.246.xx.xx) on Tue 7 Apr 2009 at 04:12
i checked my configuration files with this article and there was no problem inside my files

[ Parent ]