# Exporting Custom Output

### Introduction

As with --import, the --export option allows one to build custom modules, but in this case for generating output. Unlike --import, which allows multiple modules to be specified, you only --export using a single module, which in turn overrides all of collectl's output formatting routines.

These modules all end in the extension .ph and are searched for first in the directory collectl is being executed from and then /usr/share/collectl. If the module name is prepended with a directory name, collectl will search only there.

The reason you might care about this is that now if you want to produce your own exportable form of output and be able to print it locally, make it available to another program over a socket or even write to a local file while still being able to log to raw and/or plot formats, you get that all that functionality for free.

### How It Works

The interface to all this is really quite simple. At the command line the user types collectl --export name[,options] where:

- name both names a file to be required by collectl as well as the name of the entry points to both an initialization routine as well as one that produces the output
- options specifies an optional list of arguments that are passed to both routines to do with whatever they wish

There are currently 5 different custom exports that are part of a standard collectl distribution:

- gexpr allows one to send collectl data to ganglia
- graphite allows one to send collectl data to graphite
- lexpr generates output in an easy to parse format
- proctree provides another way to look at process data
- vmstat is the standard unix vmstat command most people know and love, so why reinvent it? It's both a useful example of how to write an export as well as provides the benefit of allowing one to play back collectl data in vmstat format!

The first four are the most interesting because they have been built to take advantage of collectl's capability of sending their output over a socket and are therefore the ideal vehicle for interfacing with other tools and environments. All four share a common set of options as described in the following table:

align | Used in conjunction with i= and when specified data samples will be aligned to whole minute boundaries. In other words if used with i=15, data will be reported at the top of the minute, 15 seconds past, etc. The first sample may therefore be partial.
avg|max|min|tot | used in conjunction with i=, send the average, maximum, minimum or total of the data over the associated set of intervals specified by i=. If none of these are specified, the values from the most recent monitoring interval will be reported.
co | this does not take a value and indicates changes only such that only data elements that have changed since they were last sampled are reported in an attempt to minimize processing and network bandwidth. If not specified, samples for all reporting intervals will be sent.
d=mask | debugging mask, see beginning of the actual export file for details
f=file | names the output snapshot file, which applies to only lexpr. If this option is not used, -f must be and the snapshot file name which is set to the single character L filename and written into the directory associated with -f caution If you are writing your own export module that doesn't use a snapshot file, you must explicity include error checking to assure it is not run without at least -P or --rawtoo in combination with -f
h | shows help/usage
i=secs | specifies the reporting interval in seconds. In other words, if you specify i=60, a sample will be reported every 60 seconds independent of collectl's monitoring interval. The default is to report every sample. This interval must be a multiple of the base collectl interval. note that while collectl always rounds rates to the next whole value, when multiple intervals are added together only the totals are rounded.
s | specifies a subset of those subsystems specified with -s in the collectl command line and only data collected for that subset will be reported. The default to report everything. In the case where you only want report data collected via --import, use s= with no args.
ttl | is the time to live in intervals for each piece of performance data. If more than this number of intervals passes data will be sent regardless of whether it changed or not and the ttl countdown timer reset. The default is 5. This actually has a second use for gexpr and that is to set the gmond ttl to double this number multiplied by the interval.

Logging
Collectl can actually create up to 3 different type of log files and it's worth spending a little more time enumerating how collectl decides where and when to create them.

- In its simplest form, when one runs collectl with --export and no options, all output is sent to the terminal, noting gexpr and graphite always require an address.
- If collectl is run with -f, and one is exporting with lexpr a snapshot file is created in the directory associated with -f.
- If a socket is to be used by specifying collectl -A and the export supports socket I/O:

- all output goes out over the socket and no snapshot file is involved
- if one specifies -f, a raw file is created in the directory specified by -f. If -P is also used, a plot file is created instead and if -P and --rawtoo are used, both types of files are created. caution: this is different behavior than one sees when running without sockets in which case a snapshot file is written to the directory named by -f
- Since gexpr and graphite do not use -A for specifying socket I/O and do their own communications, collectl's rules for non-socket logging apply, in which case -f by itself implies a snapshot file is involved and so requires -P or --rawtoo or both to do any logging.

- To create the snapshot file in a directory other then the one specified with -f, use the f= option with --export
- If one tries an invalid combination of switches or something not supported, it is up to the export itself to determine that since by design collectl has no knowledge of those module's inner workings.

### Example

Perhaps the best way to see how all this works is with a simple example and it turns out that vmstat.ph is small enough to meet that need. You may also wish to refer to the others as well to see how some of the more exotic capabilities are implemented.

This first section gets called almost immediately by collectl after reading in the various user switches. This is the place to catch switch errors and since this routine always requires -scm we'll just hardcode it to that and reject any user entered ones. This initialization subroutine must be named for our module followed by Init.

    sub vmstatInit
    {
      error("-s not allowed with 'vmstat'")          if $userSubsys ne '';
      error("-f requires either --rawtoo or -P")     if $filename ne '' && !$rawtooFlag && !$plotFlag;
      error("-P or --rawtoo require -f")             if $filename eq '' && ($rawtooFlag || $plotFlag);
      $subsys=$userSubsys='cm';
    }

Next we define the output routine, with the same base name as that of our included file.

The if statement uses collectl's standard idiom for printing headers based on the number of lines printed and whether or not the user wants only a single header, no header or even to clear the screen between headers.

    sub vmstat
    {
      my $line;
      if (printHeader())
      {
        $line= "${cls}#${miniBlanks}procs ---------------memory (KB)--------------- --swaps-- -----io---- --system-- ----cpu-----\n";
        $line.="#$miniDateTime r  b   swpd   free   buff  cache  inact active   si   so    bi    bo   in    cs us sy  id wa\n";
      }

Next comes the handling of optional date/time prefixes that I stole from printTerm() in formatit.ph and which can be controlled by various switch options. Again, if you have no intent of supporting these you can even put in error handling in your initialization routine or simply ignore the switches.

      my $datetime='';
      if ($options=~/[dDTm]/)
      {
        ($ss, $mm, $hh, $mday, $mon, $year)=localtime($lastSecs);
        $datetime=sprintf("%02d:%02d:%02d", $hh, $mm, $ss);
        $datetime=sprintf("%02d/%02d %s", $mon+1, $mday, $datetime)                  if $options=~/d/;
        $datetime=sprintf("%04d%02d%02d %s", $year+1900, $mon+1, $mday, $datetime)   if $options=~/D/;
        $datetime.=".$usecs"                                                         if ($options=~/m/);
        $datetime.=" ";
      }

Here we build the actual output, noting that we're not really printing anything yet, but rather building up a string (which may contain the header) that we will print in one shot.

      my $i=$NumCpus;
      my $usr=$userP[$i]+$niceP[$i];
      my $sys=$sysP[$i]+$irqP[$i]+$softP[$i]+$stealP[$i];
      $line.=sprintf("%s %2d %2d %6s %6s %6s %6s %6s %6s %4d %4d %5d %5d %4d %5d %2d %2d %3d %2d\n",
                    $datetime, $procsRun, $procsBlock,
                    cvt($swapUsed,6,1,1),  cvt($memFree,6,1,1),  cvt($memBuf,6,1,1),
                    cvt($memCached,6,1,1), cvt($inactive,6,1,1), cvt($active,6,1,1),
                    $swapin/$intSecs, $swapout/$intSecs, $pagein/$intSecs, $pageout/$intSecs,
                    $intrpt/$intSecs, $ctxt/$intSecs,
                    $usr, $sys, $idleP[$i], $waitP[$i]);

Finally comes the output. There is actually a lot of latitude here and in this case we're calling printText() which will send the output to the terminal or over a socket. It will not write to a local file as does lexpr, but if you want to see how to do that, refer to its source. As with all perl require files, they must return true and therefore the final line is the digit 1.

      printText($line);
    }
    1;

Try running it and you'll see all the pagination and time formats work just as they do with standard output formats.

updated November 9, 2012