CFEngine: maximum strings length

Update 2013-12-04: If you would like to participate, there is a dedicated bug report and a thread on the mailing-list

As far as I know, the maximum length of strings variables is not outlined in the official documentation.

You will hit internal limitations through warnings and errors during cf-agent execution, some examples:

1
2
3

Buffer exceeded 8192 bytes (...)
Expansion overflow constructing string. Increase CF_EXPANDSIZE macro. Tried to add (...)
error: Fatal CFEngine error: Can't expand varstring

Because I prefer to know beforehand such limits, I have tried to empirically quantify them, by reading files of different sizes using 2 methods:

This way, I’m more confident that I will not encounter a bug specific to readfile() or execresult().

Here are my findings for each stable versions: 3.3.9, 3.4.5 and 3.5.2

Official documentation

1

Each scalar may have one of three types: string, int or real
  • 3.4.5: same as 3.3.9
  • 3.5.2: unfortunately, the newest documentation is not more explicit:
1

Allowed input range: (arbitrary string)

Empirical tests

The following bundle will be used to find the limits. The protocol is:

  • Create plain files with a precise size: 1000, 2000, 10 000 bytes (teaser: going further is not necessary)
  • Read the file’s content into a string with both readfile() and execresult() functions.
  • Display the string with a reports: promise, and measure its length with awk.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

body common control {
    bundlesequence => { "string_limits" };
}

bundle agent string_limits {

  vars:
    "sizes" ilist => { 1000, 5000, 10000};

    "size"  int   => "$(sizes)", ifvarclass => "s$(sizes)";

    "file_$(size)" string => "/tmp/$(size}";

    "content_cat_$(size)"      string => execresult("/bin/cat $(file_$(size))", "useshell");
    "content_readfile_$(size)" string => readfile("$(file_$(size))", "20000");

  classes:
    "file_$(size)_exists"  expression  =>  fileexists("$(file_$(size))");

  commands:
    "/bin/perl -e 'print \"A\"x$(size)' > $(file_$(size))"
      contain     => ushell,
      ifvarclass  => "!file_$(size)_exists";

  reports:
    cfengine::
      "cfe-$(sys.cf_version) - $(size) bytes - execresult()$(const.endl)$(content_cat_$(size))";
      "cfe-$(sys.cf_version) - $(size) bytes - readfile()$(const.endl)$(content_readfile_$(size))";
}

body contain ushell {
  useshell => "true";
}

CFEngine-3.3.9

  • 1000 bytes: OK
1
2
3
4
5

$ cf-agent -f string_limits.cf -D s1000|awk '/R:/ {print} /AAA/ {print length($0)}'
R: cfe-3.3.9 - 1000 bytes - execresult()
1000
R: cfe-3.3.9 - 1000 bytes - readfile()
1000
  • 5000 bytes: NOK
1
2
3
4
5

$ cf-agent -f string_limits.cf -D s5000|awk '/R:/ {print} /AAA/ {print length($0)}'
R: cfe-3.3.9 - 5000 bytes - execresult()
4053
R: cfe-3.3.9 - 5000 bytes - readfile()
4055
  • 10 000 bytes: NOK
1
2
3
4
5
6

$ cf-agent -f string_limits.cf -D s10000|awk '/R:/ {print} /AAA/ {print length($0)}'
4094
$ cf-agent -f string_limits.cf -D s10000
Buffer exceeded 8192 bytes in exec /bin/cat /tmp/10000
Expansion overflow constructing string. Increase CF_EXPANDSIZE macro. Tried to add AAA(...)AAAA
Fatal CFEngine error: Can't expand varstring
  • The limit is between 1000 and 5000 characters

CFEngine-3.4.5

  • 1000 bytes: OK
1
2
3
4
5

$ cf-agent -f string_limits.cf -D s1000|awk '/R:/ {print} /AAA/ {print length($0)}'
R: cfe-3.4.5 - 1000 bytes - execresult()
1000
R: cfe-3.4.5 - 1000 bytes - readfile()
1000
  • 5000 bytes: NOK
1
2
3

$ cf-agent -f string_limits.cf -D s5000|awk '/R:/ {print} /AAA/ {print length($0)}'
$ cf-agent -f string_limits.cf -D s5000
Segmentation fault (core dumped)
  • 10 000 bytes: NOK
1
2
3

$ cf-agent -f string_limits.cf -D s10000
Buffer exceeded 8192 bytes in exec /bin/cat /tmp/10000
Segmentation fault (core dumped)
  • The limit is also between 1000 and 5000 characters

CFEngine-3.5.2

  • 1000 bytes: OK
1
2
3
4
5

$ cf-agent -f string_limits.cf -D s1000|awk '/R:/ {print} /AAA/ {print length($0)}'
2013-11-19T20:48:08+0100   notice: R: cfe-3.5.2 - 1000 bytes - execresult()
1000
2013-11-19T20:48:08+0100   notice: R: cfe-3.5.2 - 1000 bytes - readfile()
1000
  • 5000 bytes: OK
1
2
3
4
5

$ cf-agent -f string_limits.cf -D s5000|awk '/R:/ {print} /AAA/ {print length($0)}'
2013-11-19T20:48:20+0100   notice: R: cfe-3.5.2 - 5000 bytes - execresult()
5000
2013-11-19T20:48:20+0100   notice: R: cfe-3.5.2 - 5000 bytes - readfile()
5000
  • 10 000 bytes: NOK
1
2
3
4
5
6

$ cf-agent -f string_limits.cf -D s10000|awk '/R:/ {print} /AAA/ {print length($0)}'
8317
$ cf-agent -f string_limits.cf -D s10000
2013-11-19T20:48:39+0100    error: Buffer exceeded 8192 bytes in exec '/bin/cat /tmp/10000'
2013-11-19T20:48:39+0100    error: Expansion overflow constructing string. Increase CF_EXPANDSIZE macro. Tried to add 'AA(...)AA' to ''
2013-11-19T20:48:39+0100    error: Fatal CFEngine error: Can't expand varstring
  • The limit is between 5000 and 10 000 characters

Summary

More tests have been done to get the exact number of characters a string can hold before truncation/unexpected behavior results are on the following table:

readfile() execresult()
CFEngine-3.3.9 4055 4053
CFEngine-3.4.5 4055 4053
CFEngine-3.5.2 8028 8026
  • CFEngine-3.4.5 starts to segfault once the string length is 5000+
1
2

cf-agent -f string_limits.cf -D s5000
Segmentation fault (core dumped)

Questions:

  • Is something wrong regarding my “protocol” to find limits ?
  • Does it worth adding these limits to the documentation ? What do you think ?