The new 0.3 release of cfe-profiler adds support for CFEngine-3.6.1, and a nicer output (bundles names are now correctly aligned).

Once you have started to manage tens of servers with CFEngine (great!), if you stick to mail reports you will quickly become overwhelmed. To classify them more easily, here is a trick with CFEngine-3.6.

Want to quickly canonify a string for testing ? A shell one-liner has been given by the CFEngine folks :

Example with the mac address 00:1a:d1:48:ff:d0

$ echo -n "00:1a:d1:48:ff:d0"| perl -p -e 's/\W/_/g'
00_1a_d1_48_ff_d0loic@iron[0]: ~

The mac address is correctly canonified to 00_1a_d1_48_ff_d0, but my shell prompt is appended. For extra clarity a newline can be added with the -l flag:

$ echo -n "00:1a:d1:48:ff:d0"| perl -ple 's/\W/_/g'
00_1a_d1_48_ff_d0

Neil H. Watson has also transformed this one-liner to an independent script

• 2014-01-19 update: A memory leak has been corrected in the development version
• 2013-12-22 update: Added a graph on results with classes defined inside classes: promise type
• 2013-12-16 update: Quick reaction from the CFEngine developers: a bug has been opened to investigate the Git memory consumption

This article is based on the same principle as my latest post CFEngine: maximum strings length : using CFEngine under extreme situations to reveal where some limits are, or getting confident on CFEngine’s scalability.

Update 2013-12-04: If you would like to participate, there is a dedicated bug report and a thread on the mailing-list

As far as I know, the maximum length of strings variables is not outlined in the official documentation.

You will hit internal limitations through warnings and errors during cf-agent execution, some examples:

Buffer exceeded 8192 bytes (...)
Expansion overflow constructing string. Increase CF_EXPANDSIZE macro. Tried to add (...)
error: Fatal CFEngine error: Can't expand varstring

Because I prefer to know beforehand such limits, I have tried to empirically quantify them, by reading files of different sizes using 2 methods:

Introduction

A first glance, reading files located in /proc or /sys directories on GNU/Linux systems is not very different from reading plain files.

But you may encounter surprises while doing so through CFEngine and its readfile() function. I will try to explain why, the current state with stable CFEngine versions, and a workaround.

TL;DR Workaround

Using CFEngine’s readfile()

Because of the “everything is a file” approach of all Unix derivatives, /proc or /sys files allow an usual I/O interface. In CFEngine, the readfile() function lets you reading an external file into a variable:

The next 3.6 release of CFEngine will offer some text manipulation functions. Their names are explicit enough, but the examples below will make their purposes crystal clear.

• strlen()
• upcase()
• downcase()
• head()
• tail()

Wondering which bundle takes the most time to execute? Here is an attempt to get a more precise idea, with a bundle granularity:

cfe-profiler

Recently (still in master branch, and certainly in CFEngine-3.5), great new functions have been added to make powerful manipulations on slists. You will find below for each function its documentation (extracted from master branch) and a downloadable example:

filter

Extracts a sublist of elements matching arg1 as a regular expression (if arg3 is true) or as an exact string (if arg3 is false) from a list variable specified in arg2. In regular expression mode, the regex is anchored. If arg4 is true, the matches are inverted (you get elements that don’t match arg1). arg5 specifies a maximum number of elements to return.