PCAP::Threaded(3)

NAME
PCAP::Threaded - Run threaded processing easily.

  Constructor

PCAP::Threaded->new(max_threads)
    Generate a threaded processing object with the defined number of threads.

  Methods

add_function
    Register a named coderef to be run using threads.

      $threads->add_function($function_name, $coderef [, $divisor]);

      function_name - Text to address function by in L<run()|PCAP::Threaded/run>.
      coderef       - Reference to subroutine.
      divisor       - See L<_suitable_threads()|PCAP::Threaded/_suitable_threads>.

run Run the named function for the stated number of iterations.

    NOTE: If your process needs to honor selected index processing this needs to be handled in the callback registered in add_function.

      $threads->run($iterations, $function_name, @params);

      iterations    - Number of times the function should be started iteration
                      number (origin 1) is passed as first argument to the code-ref
                      defined in L<add_function()|PCAP::Threaded/add_function>.
      function_name - Name of function as defined in L<add_function()|PCAP::Threaded/add_function>.
      @params       - Any additional params for the coderef.

  Utility Methods
These are non-object methods which are useful to related code

Configuration

use_out_err
    Determines if stdout/stderr are redirected to file, by default 1/true.

disable_out_err
    Prevent calls to external_process_handler from redirecting stdout/stderr to files.

enable_out_err
    Enable redirect of stdout/stderr to files when calling external_process_handler.

thread_join_interval
    Set/get the number of seconds to wait between thread joins. Default 1.

Resume Helpers

These are useful methods to help you program resume from the last successfully completed step.

It is recommended that you use these in any callbacks used in the run() function to prevent unnecessary processing after restarting a job.

These can also be used for other processes in your program flow that aren't using threads. Where these don't require an incrementing index value please use '0' (zero);

success_exists
    Before compute step add this code:

      return if PCAP::Threaded::success_exists($tmpdir, $index[, $index_2...]);

    Requires implementation of touch_success().

    ($index_2... may be useful for some other implementation, see bwa_aln()).

touch_success
    After compute step add this code:

      return if PCAP::Threaded::touch_success($tmpdir, $index[, $index_2...]);

    Requires implementation of success_exists().

    ($index_2... may be useful for some other implementation, see bwa_aln()).

external_process_handler
      PCAP::Threaded::external_process_handler($logdir, $commands, $index[, $index_2...]);

      @params logdir   - Path to pre-existing log directory
      @params commands - Scalar command or arr-ref of commands
      @params index    - Which index this is, specifically for log/err files.

    Wraps up command with stdout and stderr catchalls to keep the output of each threaded process separated from the script itself. Added to simplify interpretation of any issues that may occur.

    If you don't want to capture stdout/stderr see <disable_out_err>.

    ($index_2... may be useful for some other implementation, see bwa_aln()).

  Internal functions

_suitable_threads
      $self->_suitable_threads($divisor);

    If the code-ref to be executed uses multiple threads (or piped processes) the total number of parallel jobs will be divided by this number to prevent over subscription of resources. Take this simple example:

      grep -wF 'wibble' | cut -f 5 | sort | uniq -c

    This 'pipeline' can theoretically use 4 CPU's at 100% each. When executing multiple parallel executions of this the total number of threads defined on the command line is divided by the provided value:

      max_parallel = int(total_threads / divisor);
      max_parallel++ if(max_parallel == 0);

    If total_threads is < divisor a single process will run, any prevention of oversubscription then falls to the OS and any CPU affinity settings.