Continuum Analytics: Documentation

# Installation - Advanced Configuration

- Overview
- Cluster Profile - Reference
- Cluster Name
- SSH Authentication
- Node Specification
- Security and Network Settings
- Proxy Configuration
- Conda Settings
- Salt Settings
- System Settings

## Overview

The installation process described on the Installation documentation is appropriate for most environments and use cases. However, you might want to customize Anaconda platform settings related to security, network, installation paths, or other settings using the process described on this page.

Cluster profiles are used to define information about a cluster before it is created, including the cluster name, nodes in the cluster, and other platform component options. Cluster files are located in the `~/.continuum/adam/profile.d/` directory.

Cluster definitions are used to identify information about a running cluster, including the cluster name, nodes in the cluster, and other platform component options. Cluster definitions are located in the `~/.continuum/adam/cluster.d/` directory.

## Cluster Profile - Reference

A cluster profile located at `~/.continuum/adam/profile.d/cluster.yaml` with all configurable settings is shown below:

    name: cluster
    provider: bare

    bare:
      username: centos
      # password: anaconda  # Optional
      port: 22
      keypair: ~/.ssh/my-private-key
      nodes:
      - host: node1.company.com
      - host: node2.company.com
      - host: node3.company.com
      - host: node4.company.com

    security:
      flush_iptables: false
      selinux_context: false
      selinux_permissive: false

    network:
      http_proxy: http://server:port/
      # https_proxy: http://server:port/

    system:
      tmp_dir: /tmp

    plugins:
      conda:
        anaconda_hash: md5=d72add23bc937ccdfc7de4f47deff843
        anaconda_url: http://repo.continuum.io/archive/Anaconda2-4.4.0-Linux-x86_64.sh
        channel_alias: https://conda.anaconda.org/
        channels:
        - defaults
        - anaconda-adam
        conda_canary: false
        enabled: true
        miniconda_hash: md5=7097150146dd3b83c805223663ebffcc
        miniconda_url: http://repo.continuum.io/miniconda/Miniconda2-4.3.21-Linux-x86_64.sh
        rootdir: /opt/continuum
        ssl_verify: null

      dask:
        bokeh_port: 8787
        bokeh_whitelist: '''*'''
        enabled: false
        host: 0.0.0.0
        http_port: 9786
        nprocs: 1
        port: 8786

      enterprise-notebooks:
        admin_email: admin@yourdomain.com
        admin_password: anaconda
        admin_user: wakari
        directory: /opt/wakari
        elasticsearch_fn: elasticsearch-1.7.2.noarch.rpm
        enabled: false
        enterprise_notebooks_version: 4.0.0
        gateway_port: 8089
        java_fn: jre-8u65-linux-x64.rpm
        mongodb_fn: mongodb-org-2.6.8-1.x86_64.rpm
        mongodb_mongos_fn: mongodb-org-mongos-2.6.8-1.x86_64.rpm
        mongodb_server_fn: mongodb-org-server-2.6.8-1.x86_64.rpm
        mongodb_shell_fn: mongodb-org-shell-2.6.8-1.x86_64.rpm
        mongodb_tools_fn: mongodb-org-tools-2.6.8-1.x86_64.rpm
        mongodb_version: 2.6.8
        nginx_fn: nginx-1.6.2-1.el6.ngx.x86_64.rpm
        root_download_url: https://820451f3d8380952ce65-4cc6343b423784e82fd202bb87cf87cf.ssl.cf1.rackcdn.com/
        user: wakari

      jupyter:
        dashboards_server:
          enabled: false
          ip: 0.0.0.0
          port: 3000
          user: anaconda
        enabled: false
        kernel_gateway:
          enabled: false
          ip: 0.0.0.0
          port: 7000
          user: anaconda
        notebook:
          directory: ~/notebooks
          enabled: false
          ip: 0.0.0.0
          password: 'anaconda'
          port: 8888
          user: anaconda

      repository:
        binstar_server_version: 2.21.0
        channel: main
        email: youremail@continuum.io
        enabled: false
        mongodb_fn: mongodb-org-2.6.8-1.x86_64.rpm
        mongodb_mongos_fn: mongodb-org-mongos-2.6.8-1.x86_64.rpm
        mongodb_server_fn: mongodb-org-server-2.6.8-1.x86_64.rpm
        mongodb_shell_fn: mongodb-org-shell-2.6.8-1.x86_64.rpm
        mongodb_tools_fn: mongodb-org-tools-2.6.8-1.x86_64.rpm
        mongodb_version: 2.6.9
        password: anaconda
        port: 8080
        root_download_url: https://820451f3d8380952ce65-4cc6343b423784e82fd202bb87cf87cf.ssl.cf1.rackcdn.com/
        server_user: anaconda-server
        superuser: superuser
        token: qu-a49e2e69-1047-4eab-a879-a2ee9c198381

      salt_settings:
        acl:
        - anaconda
        job_pub_port: 14505
        minion_pub_port: 14510
        minion_pull_port: 14511
        minion_ret_port: 14506
        rest_port: 18000
        salt_groupname: anaconda
        salt_password: anaconda
        salt_username: anaconda
        service_scripts: false

## Cluster Name

This setting specifies the name of the cluster and is used by the `-n` option in `adam` commands. For example, to create a new cluster named `cluster` from a profile called `profile`, you would use the command:

    $ adam up -n cluster profile

## SSH Authentication

To connect to remote machines via SSH, Anaconda Adam requires a valid username, port (default: `22`), and an authentication method. The following SSH authentication methods are supported in Anaconda Adam:

- Password
- Key pair (full path to your private key)
- Encrypted key pair (full path to your public key and ssh-agent)

Refer to the Requirements page for more information about system and account requirements.

For example, to use SSH with a key pair:

    bare:
      username: centos
      keypair: /full/path/to/key.rsa

Or, to use SSH with a username and password:

    bare:
      username: centos
      password: anaconda

Or, to use SSH with an encrypted key pair, the private key must first be added to a running ssh-agent. Use the `agent_pubkey` setting in the cluster profile to specify the full path to the associated public key. For example:

    bare:
      username: centos
      agent_pubkey: /full/path/to/key.pub

## Node Specification

This setting defines the nodes that exist within the cluster. By default, the first node in the list of `nodes` will be defined as the head node, and the remaining nodes will be defined as compute nodes.

Each host should be able to connect to the other hosts via the specified FQDN or IP address. It is recommended that you use the FQDN of the nodes that matches the hostname on each machine.

    bare:
      nodes:
      - host: node1.company.com
      - host: node2.company.com
      - host: node3.company.com
      - host: node4.company.com

## Security and Network Settings

These settings allow you to configure iptabes and SELinux.

To flush iptables (default: `false`):

    security:
      flush_iptables: false

To set SELinux to permissive (default: `false`):

    security:
      selinux_permissive: false

To set SELinux contexts (default: `false`):

    security:
      selinux_context: false

## Proxy Configuration

You can install Anaconda Adam to nodes that access the Internet via an HTTP or HTTPS proxy by using the `network` setting. To specify an HTTP proxy:

    network:
      http_proxy: http://server:port/

or, to specify an HTTPS proxy:

    network:
      https_proxy: http://server:port/

## Conda Settings

These settings are related to the installation of Anaconda/Miniconda across the cluster using Anaconda Scale.

    conda:
        anaconda_hash: md5=d72add23bc937ccdfc7de4f47deff843
        anaconda_url: http://repo.continuum.io/archive/Anaconda2-4.4.0-Linux-x86_64.sh
        channel_alias: https://conda.anaconda.org/
        channels:
        - defaults
        - anaconda-adam
        enabled: true
        miniconda_hash: md5=7097150146dd3b83c805223663ebffcc
        miniconda_url: http://repo.continuum.io/miniconda/Miniconda2-4.3.21-Linux-x86_64.sh
        rootdir: /opt/continuum
        ssl_verify: False

Above are the default settings for configuring conda. The conda setting for ssl_verify can be set to one of three options:

- False: no SSL verification (default),
- True: SSL verification is used and conda verifies certificates for SSL connections,
- Cert Path: string path to a certificate to be used to verify SSL connections

## Salt Settings

These settings allow you to configure the network options, access control list, and API user credentials used by Salt, which is the configuration management system used by Anaconda Adam. To specify the ports used by the Salt master, minions, and REST API:

    plugins:
      salt_settings:
        job_pub_port: 14505
        minion_pub_port: 14510
        minion_pull_port: 14511
        minion_ret_port: 14506
        rest_port: 18000

To specify non-root system users that can execute Salt commands via an access control list:

    plugins:
      salt_settings:
        acl:
        - anaconda

To specify the username, password, and group that gets created across the cluster and owns the directory and files located in the root installation directory (default `/opt/continuum`):

    plugins:
      salt_settings:
        salt_username: anaconda
        salt_password: anaconda
        salt_groupname: anaconda

If you do not want to store a clear text password in the cluster profile or cluster definition file, you can set the `salt_password` to an empty string:

    plugins:
      salt_settings:
        salt_username: anaconda
        salt_password: ''
        salt_groupname: anaconda

In this case, Anaconda Adam will prompt for a password in the CLI and only store the password in memory for the duration of the command/job.

## System Settings

These settings allow you to configure system options.

To specify the download directory where temporary installers are downloaded to (default: `/tmp`):

    system:
      tmp_dir: /tmp