# DaskLSFBackend

class `propertyestimator.backends.``DaskLSFBackend`(minimum_number_of_workers=1, maximum_number_of_workers=1, resources_per_worker=<propertyestimator.backends.backends.QueueWorkerResources object>, queue_name='default', setup_script_commands=None, extra_script_options=None, adaptive_interval='10000ms', disable_nanny_process=False)

  A property estimator backend which uses a dask_jobqueue LSFCluster object to run calculations within an existing LSF queue.

  See also

  `dask_jobqueue.LSFCluster`

  `__init__`(minimum_number_of_workers=1, maximum_number_of_workers=1, resources_per_worker=<propertyestimator.backends.backends.QueueWorkerResources object>, queue_name='default', setup_script_commands=None, extra_script_options=None, adaptive_interval='10000ms', disable_nanny_process=False)

    Constructs a new DaskLSFBackend object

    Parameters

      - minimum_number_of_workers (int) – The minimum number of workers to request from the queue system.

      - maximum_number_of_workers (int) – The maximum number of workers to request from the queue system.

      - resources_per_worker (QueueWorkerResources) – The resources to request per worker.

      - queue_name (str) – The name of the queue which the workers will be requested from.

      - setup_script_commands (list of str) –

        A list of bash script commands to call within the queue submission script before the call to launch the dask worker.

        This may include activating a python environment, or loading an environment module

      - extra_script_options (list of str) –

        A list of extra job specific options to include in the queue submission script. These will get added to the script header in the form

        #BSUB <extra_script_options[x]>

      - adaptive_interval (str) – The interval between attempting to either scale up or down the cluster, of of the from ‘XXXms’.

      - disable_nanny_process (bool) –

        If true, dask workers will be started in –no-nanny mode. This is required if using multiprocessing code within submitted tasks.

        This has not been fully tested yet and my lead to stability issues with the workers.

    Examples

    To create an LSF queueing compute backend which will attempt to spin up workers which have access to a single GPU.

        >>> # Create a resource object which will request a worker with
        >>> # one gpu which will stay alive for five hours.
        >>> from propertyestimator.backends import QueueWorkerResources
        >>>
        >>> resources = QueueWorkerResources(number_of_threads=1,
        >>>                                  number_of_gpus=1,
        >>>                                  preferred_gpu_toolkit=QueueWorkerResources.GPUToolkit.CUDA,
        >>>                                  wallclock_time_limit='05:00')
        >>>
        >>> # Define the set of commands which will set up the correct environment
        >>> # for each of the workers.
        >>> setup_script_commands = [
        >>>     'module load cuda/9.2',
        >>> ]
        >>>
        >>> # Define extra options to only run on certain node groups
        >>> extra_script_options = [
        >>>     '-m "ls-gpu lt-gpu"'
        >>> ]
        >>>
        >>>
        >>> # Create the backend which will adaptively try to spin up between one and
        >>> # ten workers with the requested resources depending on the calculation load.
        >>> from propertyestimator.backends import DaskLSFBackend
        >>>
        >>> lsf_backend = DaskLSFBackend(minimum_number_of_workers=1,
        >>>                              maximum_number_of_workers=10,
        >>>                              resources_per_worker=resources,
        >>>                              queue_name='gpuqueue',
        >>>                              setup_script_commands=setup_script_commands,
        >>>                              extra_script_options=extra_script_options)

  Methods

  `__init__`([minimum_number_of_workers, …]) | Constructs a new DaskLSFBackend object
  `start`() | Start the calculation backend.
  `stop`() | Stop the calculation backend.
  `submit_task`(function, *args, **kwargs) | Submit a task to the compute resources managed by this backend.

  `start`()

    Start the calculation backend.

  `submit_task`(function, *args, **kwargs)

    Submit a task to the compute resources managed by this backend.

    Parameters

      function (function) – The function to run.

    Returns

      Returns a future object which will eventually point to the results of the submitted task.

    Return type

      Future

  `stop`()

    Stop the calculation backend.