Continuum Analytics: Documentation

# Profile settings

The following profile settings can be used to define cluster configurations for cloud-based or bare-metal clusters. Each named profile file should contain information for a single cluster definition.

- Profile settings
  - anaconda_url (optional)
  - aws (optional)
  - conda_channels (optional)
  - default_channels (optional)
  - channel_alias (optional)
  - name
  - node_id
  - node_type
  - num_nodes
  - machines (optional)
  - plugins (optional)
  - provider
  - root_size (optional)
  - security (optional)
  - user
- Sample cloud-based profile file
- Sample bare-metal profile file

## Profile settings

### anaconda_url (optional)

Set a custom location to download Anaconda/Miniconda from (default: Miniconda with Python 2).

    anaconda_url: http://localhost/miniconda/Miniconda-latest-Linux-x86_64.sh

### aws (optional)

Set AWS options for all instances:

> - `tags` are optional key/value metadata that are assigned to each instance.
> - `termination_protection` prevents your instances from being accidentally terminated (default: true).
> - `use_private_ip` uses the private IP addresses of the AWS nodes instead of the public IP addresses. This is useful if you are using a custom security group (default: false).

    aws:
      tags:
        - billingProject: anaconda-cluster
        - cluster_version: production
        - ...
      termination_protection: true
      use_private_ip: true

### conda_channels (optional)

Set custom channels to download conda packages from on the cluster nodes. This setting will overwrite all of the configured conda channels, including the default channels. If you are using this setting, you will need to explicitly include the two default channels, `defaults` and `anaconda-cluster`, followed by your custom channels.

    conda_channels:
      - defaults
      - anaconda-cluster
      - blaze
      - pypi
      - username
      - https://conda.anaconda.org/username/

### default_channels (optional)

Set default_channels to download the default conda packages from the URL defined. This setting will overwrite the default conda channels: http://repo.continuum.io.

    default_channels: http://localhost/conda/anaconda

### channel_alias (optional)

Set channel_alias to instruct conda to look for channels in a local repository This setting will overwrite the default: http://conda.anaconda.org.

    channel_alias: http://localhost/conda/

### name

Name of the profile.

    name: profile_name

### node_id

Image to configure on each node. For bare-metal clusters, a dummy value such as `bare-metal` can be used.

    node_id: ami-d05e75b8  # Ubuntu 14.04, us-east-1 region

### node_type

The type of node to launch. For bare-metal clusters, a dummy value such as `bare-metal` can be used.

    node_type: m3.large

### num_nodes

Number of nodes to launch (cloud-based) or manage (bare-metal).

    num_nodes: 4

### machines (optional)

IP addresses of head and compute nodes in the cluster. Note that only IP addresses should be used with this setting, not hostnames or FQDNs.

This is used along with the Bare-metal Cluster Setup and can refer to bare-metal nodes, cloud-based nodes that were manually launched outside of Anaconda for cluster management, or a collection of virtual machines. If SSH is running on a port other than 22, you can optionally include the SSH port number (e.g., `192.168.1.1:2222`).

    machines:
      head:
        - 192.168.1.1
      compute:
        - 192.168.1.2
        - 192.168.1.3
        - 192.168.1.4

### plugins (optional)

Install plugins upon cluster creation or provisioning. Some plugins also have configruable settings. Refer to the Plugin Settings documentation for more information.

    plugins:
      - notebook
      - dask

### provider

Name of the provider to use to launch instances for a cloud-based or bare-metal cluster. For more information, refer to the Provider settings documentation.

    provider: aws_east

### root_size (optional)

Size of the root volume (GB). Currently only used for Amazon EBS volumes.

    root_size: 200

### security (optional)

Security settings to enable/disable SELinux or flush iptables rules (default: true).

    security:
      disable_selinux: true
      flush_iptables: true

### user

User to SSH to the cluster nodes as. This user must have passwordless sudo access.

    user: ubuntu

## Sample cloud-based profile file

Below is a sample cloud-based profile named `profile_name` located in the `~/.acluster/profiles/profile_name.yaml` file that is configured with all required and optional settings.

    name: profile_name
    node_id: ami-d05e75b8  # Ubuntu 14.04, us-east-1 region
    node_type: m3.large
    num_nodes: 4
    provider: aws_east
    root_size: 50
    user: ubuntu
    anaconda_url: http://localhost/miniconda/Miniconda-latest-Linux-x86_64.sh

    aws:
      tags:
        - billingProject: anaconda-cluster

    plugins:
      - notebook
      - dask

    default_channels: http://localhost/conda/anaconda

    channel_alias: http://localhost/conda/

    conda_channels:
      - defaults
      - anaconda-cluster
      - blaze
      - pypi
      - username
      - https://conda.anaconda.org/username/

    security:
      disable_selinux: true
      flush_iptables: true

## Sample bare-metal profile file

Below is a sample bare-metal profile named `profile_name` located in the `~/.acluster/profiles/profile_name.yaml` file that is configured with all required and optional settings.

    name: profile_name
    node_id: bare_metal
    node_type: bare_metal
    num_nodes: 4
    provider: bare_metal
    user: ubuntu
    anaconda_url: http://localhost/miniconda/Miniconda-latest-Linux-x86_64.sh

    machines:
      head:
        - 192.168.1.1
      compute:
        - 192.168.1.2
        - 192.168.1.3
        - 192.168.1.4

    plugins:
      - notebook
      - dask

    default_channels: http://localhost/conda/anaconda

    channel_alias: http://localhost/conda/

    conda_channels:
      - defaults
      - anaconda-cluster
      - blaze
      - pypi
      - username
      - https://conda.anaconda.org/username/

    security:
      disable_selinux: false
      flush_iptables: false