# fevalDistr

## PURPOSE

Wrapper for embarrassingly parallel function evaluation.

## SYNOPSIS

function [out,res] = fevalDistr( funNm, jobs, varargin )

## DESCRIPTION

     Wrapper for embarrassingly parallel function evaluation.

     Runs "r=feval(funNm,jobs{i}{:})" for each job in a parallel manner. jobs
     should be a cell array of length nJob and each job should be a cell array
     of parameters to pass to funNm. funNm must be a function in the path and
     must return a single value (which may be a dummy value if funNm writes
     results to disk). Different forms of parallelization are supported
     depending on the hardware and Matlab toolboxes available. The type of
     parallelization is determined by the parameter 'type' described below.

     type='LOCAL': jobs are executed using a simple "for" loop. This implies
     no parallelization and is the default fallback option.

     type='PARFOR': jobs are executed using a "parfor" loop. This option is
     only available if the Matlab *Parallel Computing Toolbox* is installed.
     Make sure to setup Matlab workers first using "matlabpool open".

     type='DISTR': jobs are executed on the Caltech cluster. Distributed
     queuing system must be installed separately. Currently this option is
     only supported on the Caltech cluster but could easily be installed on
     any Linux cluster as it requires only SSH and a shared filesystem.
     Parameter pLaunch is used for controller('launchQueue',pLaunch{:}) and
     determines cluster machines used (e.g. pLaunch={48,401:408}).

     type='COMPILED': jobs are executed locally in parallel by first compiling
     an executable and then running it in background. This option requires the
     *Matlab Compiler* to be installed (but does NOT require the Parallel
     Computing Toolbox). Compiling can take 1-10 minutes, so use this option
     only for large jobs. (On Linux alter startup.m by calling addpath() only
     if ~isdeployed, otherwise will get error about "CTF" after compiling).
     Note that relative paths will not work after compiling so all paths used
     by funNm must be absolute paths.

     type='WINHPC': jobs are executed on a Windows HPC Server 2008 cluster.
     Similar to type='COMPILED', except after compiling, the executable is
     queued to the HPC cluster where all computation occurs. This option
     likewise requires the *Matlab Compiler*. Paths to data, etc., must be
     absolute paths and available from HPC cluster. Parameter pLaunch should
     have two fields 'scheduler' and 'shareDir' that define the HPC Server.
     For example, at MSR one possible cluster is defined by scheduler =
     'MSR-L25-DEV21' and shareDir = '\\msr-arrays\scratch\msr-pool\L25-dev21'.
     Note call to 'job submit' from Matlab will hang unless pwd is saved
     (simply call 'job submit' from cmd prompt and enter pwd).

     USAGE
      [out,res] = fevalDistr( funNm, jobs, [varargin] )

     INPUTS
      funNm      - name of function that will process jobs
      jobs       - [1xnJob] cell array of parameters for each job
      varargin   - additional params (struct or name/value pairs)
       .type       - ['local'], 'parfor', 'distr', 'compiled', 'winhpc'
       .pLaunch    - [] extra params for type='distr' or type='winhpc'
       .group      - [1] send jobs in batches (only relevant if type='distr')

     OUTPUTS
      out        - 1 if jobs completed successfully
      res        - [1xnJob] cell array containing results of each job

     EXAMPLE
      % Note: in this case parallel versions are slower since conv2 is so fast
      n=16; jobs=cell(1,n); for i=1:n, jobs{i}={rand(500),ones(25)}; end
      tic, [out,J1] = fevalDistr('conv2',jobs,'type','local'); toc,
      tic, [out,J2] = fevalDistr('conv2',jobs,'type','parfor'); toc,
      tic, [out,J3] = fevalDistr('conv2',jobs,'type','compiled'); toc
      [isequal(J1,J2), isequal(J1,J3)], figure(1); montage2(cell2array(J1))

     See also matlabpool mcc

     Piotr's Image&Video Toolbox      Version 3.02
     Copyright 2012 Piotr Dollar.  [pdollar-at-caltech.edu]
     Please email me if you find bugs, or have suggestions or questions!
     Licensed under the Simplified BSD License [see external/bsd.txt]