CFEngine: maximum strings length
  2013-11-20

Update 2013-12-04: If you would like to participate, there is a dedicated bug report and a thread on the mailing-list

As far as I know, the maximum length of strings variables is not outlined in the official documentation.

You will hit internal limitations through warnings and errors during cf-agent execution, some examples:

Buffer exceeded 8192 bytes (...)
Expansion overflow constructing string. Increase CF_EXPANDSIZE macro. Tried to add (...)
error: Fatal CFEngine error: Can't expand varstring

Because I prefer to know beforehand such limits, I have tried to empirically quantify them, by reading files of different sizes using 2 methods:

This way, I’m more confident that I will not encounter a bug specific to readfile() or execresult().

Here are my findings for each stable versions: 3.3.9, 3.4.5 and 3.5.2

Official documentation

  • 3.3.9: Nothing ? The most accurate information I had found is ~~~ Each scalar may have one of three types: string, int or real ~~~
  • 3.4.5: same as 3.3.9
  • 3.5.2: unfortunately, the newest documentation is not more explicit:

    Allowed input range: (arbitrary string)
    

    Empirical tests

The following bundle will be used to find the limits. The protocol is:

  • Create plain files with a precise size: 1000, 2000, 10 000 bytes (teaser: going further is not necessary)
  • Read the file’s content into a string with both readfile() and execresult() functions.
  • Display the string with a reports: promise, and measure its length with awk.
body common control {
    bundlesequence => { "string_limits" };
}

bundle agent string_limits {

  vars:
    "sizes" ilist => { 1000, 5000, 10000};

    "size"  int   => "$(sizes)", ifvarclass => "s$(sizes)";

    "file_$(size)" string => "/tmp/$(size}";

    "content_cat_$(size)"      string => execresult("/bin/cat $(file_$(size))", "useshell");
    "content_readfile_$(size)" string => readfile("$(file_$(size))", "20000");

  classes:
    "file_$(size)_exists"  expression  =>  fileexists("$(file_$(size))");

  commands:
    "/bin/perl -e 'print \"A\"x$(size)' > $(file_$(size))"
      contain     => ushell,
      ifvarclass  => "!file_$(size)_exists";

  reports:
    cfengine::
      "cfe-$(sys.cf_version) - $(size) bytes - execresult()$(const.endl)$(content_cat_$(size))";
      "cfe-$(sys.cf_version) - $(size) bytes - readfile()$(const.endl)$(content_readfile_$(size))";
}

body contain ushell {
  useshell => "true";
}

CFEngine-3.3.9

  • 1000 bytes: OK
$ cf-agent -f string_limits.cf -D s1000|awk '/R:/ {print} /AAA/ {print length($0)}'
R: cfe-3.3.9 - 1000 bytes - execresult()
1000
R: cfe-3.3.9 - 1000 bytes - readfile()
1000
  • 5000 bytes: NOK

    $ cf-agent -f string_limits.cf -D s5000|awk '/R:/ {print} /AAA/ {print length($0)}'
    R: cfe-3.3.9 - 5000 bytes - execresult()
    4053
    R: cfe-3.3.9 - 5000 bytes - readfile()
    4055
    
  • 10 000 bytes: NOK

    $ cf-agent -f string_limits.cf -D s10000|awk '/R:/ {print} /AAA/ {print length($0)}'
    4094
    $ cf-agent -f string_limits.cf -D s10000
    Buffer exceeded 8192 bytes in exec /bin/cat /tmp/10000
    Expansion overflow constructing string. Increase CF_EXPANDSIZE macro. Tried to add AAA(...)AAAA
    Fatal CFEngine error: Can't expand varstring
    
  • The limit is between 1000 and 5000 characters

CFEngine-3.4.5

  • 1000 bytes: OK

    $ cf-agent -f string_limits.cf -D s1000|awk '/R:/ {print} /AAA/ {print length($0)}'
    R: cfe-3.4.5 - 1000 bytes - execresult()
    1000
    R: cfe-3.4.5 - 1000 bytes - readfile()
    1000
    
  • 5000 bytes: NOK

    $ cf-agent -f string_limits.cf -D s5000|awk '/R:/ {print} /AAA/ {print length($0)}'
    $ cf-agent -f string_limits.cf -D s5000
    Segmentation fault (core dumped)
    
  • 10 000 bytes: NOK

    $ cf-agent -f string_limits.cf -D s10000
    Buffer exceeded 8192 bytes in exec /bin/cat /tmp/10000
    Segmentation fault (core dumped)
    
  • The limit is also between 1000 and 5000 characters

CFEngine-3.5.2

  • 1000 bytes: OK

    $ cf-agent -f string_limits.cf -D s1000|awk '/R:/ {print} /AAA/ {print length($0)}'
    2013-11-19T20:48:08+0100   notice: R: cfe-3.5.2 - 1000 bytes - execresult()
    1000
    2013-11-19T20:48:08+0100   notice: R: cfe-3.5.2 - 1000 bytes - readfile()
    1000
    
  • 5000 bytes: OK

    $ cf-agent -f string_limits.cf -D s5000|awk '/R:/ {print} /AAA/ {print length($0)}'
    2013-11-19T20:48:20+0100   notice: R: cfe-3.5.2 - 5000 bytes - execresult()
    5000
    2013-11-19T20:48:20+0100   notice: R: cfe-3.5.2 - 5000 bytes - readfile()
    5000
    
  • 10 000 bytes: NOK

    $ cf-agent -f string_limits.cf -D s10000|awk '/R:/ {print} /AAA/ {print length($0)}'
    8317
    $ cf-agent -f string_limits.cf -D s10000
    2013-11-19T20:48:39+0100    error: Buffer exceeded 8192 bytes in exec '/bin/cat /tmp/10000'
    2013-11-19T20:48:39+0100    error: Expansion overflow constructing string. Increase CF_EXPANDSIZE macro. Tried to add 'AA(...)AA' to ''
    2013-11-19T20:48:39+0100    error: Fatal CFEngine error: Can't expand varstring
    
  • The limit is between 5000 and 10 000 characters

    Summary

More tests have been done to get the exact number of characters a string can hold before truncation/unexpected behavior results are on the following table:

readfile() execresult()
CFEngine-3.3.9 4055 4053
CFEngine-3.4.5 4055 4053
CFEngine-3.5.2 8028 8026
  • CFEngine-3.4.5 starts to segfault once the string length is 5000+ ~~~ cf-agent -f string_limits.cf -D s5000 Segmentation fault (core dumped) ~~~

Questions:

  • Is something wrong regarding my “protocol” to find limits ?
  • Does it worth adding these limits to the documentation ? What do you think ?