Continuum Analytics: Documentation

# User Guide

- Overview
- Manage Anaconda on the Cluster
- Install and Manage Platform Components
- Remote Commands
- SSH to Nodes
- Version and License Information
- List Cluster Information
- Creating, Provisioning, and Removing Clusters
- Adding Nodes to an Existing Cluster
- Additional Provision Commands (Advanced)
- Execute Salt Commands (Advanced)

## Overview

Typing the `adam` command shows the different subcommands that can be used:

    $ adam
    Usage: adam [OPTIONS] COMMAND [ARGS]...

    Options:
      --version                       Show the version and exit.
      -l, --log-level [info|debug|error]
                                      Logging level  [default: debug]
      -h, --help                      Show this message and exit.

    Commands:
      cmd                   Execute a command on the nodes
      describe              Describe an Adam cluster or profile
      ec2                   EC2 options
      enterprise-notebooks  Anaconda Enterprise Notebooks options
      info                  Display Adam version, system, and license information
      list                  List Adam clusters and profiles
      provision             Provision options
      remove                Remove a cluster
      repository            Anaconda Repository options
      salt                  Execute a Salt module
      scale                 Anaconda Scale options
      ssh                   SSH to one of the nodes (0-based index)
      up                    Create a cluster from a profile

You can view more information about a subcommand using the `-h` or `--help` option, for example, `adam up --help`.

More detail on the available subcommands is shown in the sections below.

## Manage Anaconda on the Cluster

Using Anaconda Scale, you can create and manage conda packages, including Python and R, in multiple conda environments across your nodes. Refer to the Anaconda Scale documentation for more information about installing and managing Anaconda across a cluster.

## Install and Manage Platform Components

- `adam enterprise-notebooks`
- `adam repository`
- `adam scale`

For more information on installing and managing platform components, refer to the Getting Started documentation.

## Remote Commands

You can execute remote commands on all of the cluster nodes using `adam cmd`:

    $ adam cmd --help
    Usage: adam cmd [OPTIONS] COMMAND

    Options:
      -n, --cluster TEXT  Cluster name  [required]
      -t, --target TEXT   Target nodes  [default: *]
      -h, --help          Show this message and exit.

For example, to run the `date` command on all nodes:

    $ adam cmd -n cluster date
    192.168.1.1: Wed Jun 29 03:08:10 UTC 2016
    192.168.1.2: Wed Jun 29 03:08:10 UTC 2016
    192.168.1.3: Wed Jun 29 03:08:10 UTC 2016
    192.168.1.4: Wed Jun 29 03:08:10 UTC 2016

## SSH to Nodes

You can SSH to the cluster nodes using `adam ssh`:

    $ adam ssh -n cluster
    Last login: Wed Jun 29 03:08:49 2016 from 172.18.0.1
    [root@192.168.1.1 ~]#

By default, you will connect to the head node. You can optionally include the node to SSH to, where node 0 is the head node, node 1 is the first compute node, and so on:

    $ adam ssh -n cluster 2
    Last login: Wed Jun 29 03:10:14 2016 from 172.18.0.1
    [root@192.168.1.3 ~]#

## Version and License Information

You can view version and license information using `adam info`:

    $ adam info
    Adam version: 4.1.0
    Plugins:
      adam_enterprise_notebooks: 4.1.0
      adam_repository: 4.1.0
      adam_notebook: 4.1.0
      adam_dask: 4.1.0

    Platform: Darwin-15.6.0-x86_64
    Darwin Kernel Version 15.6.0: Thu Jun 23 18:25:34 PDT 2016; root:xnu-3248.60.10~1/RELEASE_X86_64
    Python version: 3.5.1 |Continuum Analytics, Inc.| (default, Dec  7 2015, 11:24:55)
    [GCC 4.2.1 (Apple Inc. build 5577)]
    Processor: i386
    Byte-ordering: little

    License information:
    Number of nodes currently in use: 14
    Number of licensed nodes: 16
    Number of managed clusters: 2

    Valid platform component licenses:
    1: Anaconda Repository Enterprise. End date: 2017-05-24
    2: Anaconda Cluster. End date: 2017-05-24
    3: Wakari Enterprise. End date: 2017-05-24
    4: mkl-optimizations. End date: 2017-05-24
    5: iopro. End date: 2017-05-24
    6: accelerate. End date: 2017-05-24

## List Cluster Information

You can view a list of running clusters using `adam list`:

    $ adam list
    cluster
    second-cluster
    third-cluster

You can view a list of cluster profiles using `adam list profiles`:

    $ adam list profiles
    profile
    second-profile
    third-profile

You can view more detailed information about a cluster or profile using `adam describe`. For example, to view information about the cluster named `cluster`:

    $ adam describe -n cluster
    conda:
      anaconda_hash: md5=7097150146dd3b83c805223663ebffcc
      anaconda_url: https://repo.continuum.io/miniconda/Miniconda2-4.3.21-Linux-x86_64.sh#adam-ci
      enabled: true
      rootdir: /opt/continuum
    name: cluster
    nodes:
    - host: ci_head_1
      keypair: ~/.ssh/my_private_key
      port: 22
      username: root
    - host: ci_compute_1
      keypair: ~/.ssh/my_private_key
      port: 22
      username: root

## Creating, Provisioning, and Removing Clusters

You can create, provision, and remove clusters using `adam up`, `adam provision`, and `adam remove`.

To create a new cluster named `cluster` from a profile named `profile`:

    $ adam up -n cluster profile

To (re)provision an existing cluster named `cluster`:

    $ adam provision -n cluster

To remove the cluster definition file for an existing bare-metal cluster named `cluster`:

    $ adam remove -n cluster
    Are you sure you want to remove the cluster cluster? [y/N]: y

To destroy an existing cloud-based cluster named `cluster` and terminate the corresponding instances:

    $ adam ec2 -n cluster destory
    Are you sure you want to destroy the cluster cluster? [y/N]: y

## Adding Nodes to an Existing Cluster

The following steps describe how to add additional nodes to an existing cluster in Anaconda Adam.

To add additional nodes to a cluster managed by Anaconda Adam, you can modify an existing cluster profile by specifying additional node(s). For example, if you want to add the following nodes:

- `node10.company.com`
- `node11.company.com`
- `node12.company.com`

to a cluster profile located at `~/.continuum/adam/profile.d/profile.yaml` with the following settings:

    name: profile
    provider: bare

    bare:
      username: centos
      # password: password  # Optional
      port: 22
      keypair: ~/.ssh/my-private-key
      nodes:
      - host: node1.company.com
      - host: node2.company.com
      - host: node3.company.com
      - host: node4.company.com

Assuming the same authentication mode and credentials for the additional nodes, you can add a `host` line for each node to the cluster profile:

    name: profile
    provider: bare

    bare:
      username: centos
      # password: password  # Optional
      port: 22
      keypair: ~/.ssh/my-private-key
      nodes:
      - host: node1.company.com
      - host: node2.company.com
      - host: node3.company.com
      - host: node4.company.com
      - host: node10.company.com
      - host: node11.company.com
      - host: node12.company.com

Then, you can re-run the `adam up` command to reprovision the cluster:

    $ adam up -n cluster profile

The `adam up` command will prompt to overwrite the cluster definition file located at `~/.continuum/adam/cluster.d/cluster.yaml` to include the newly defined nodes. During this reprovision process, Anaconda Adam will not reinstall on the previously defined nodes.

## Additional Provision Commands (Advanced)

By default, `adam provision` will install check SSH connectivity, create a default user, install Salt, sync formulas, and provision enabled plugins and security settings that are defined in the cluster profile.

If you only need to run a portion of the `adam provision` process, you can view the available subcommands:

    $ adam provision --help
    Usage: adam provision [OPTIONS] COMMAND [ARGS]...

      Execute all the provisioning steps for a cluster

      See subcommands help for more info. Usage: adam provision -n

    Options:
      -n, --cluster TEXT    Cluster name  [required]
      --flush-iptables      Flush iptables  [default: False]
      --selinux-permissive  Set SELinux to permissive  [default: False]
      --selinux-context     Set SELinux contexts  [default: False]
      --plugins             Install enabled plugins on provision  [default: False]
      -h, --help            Show this message and exit.

    Commands:
      anaconda-user       Create the default anaconda user in all the nodes
      check-ssh           Check SSH connection to the nodes
      flush-iptables      Flush iptables
      plugins             Provision enabled plugins
      roles               Set roles on the cluster nodes
      salt                Provision Salt
      security-settings   Execute the security settings
      selinux-context     Set SELinux contexts
      selinux-permissive  Set SELinux to permissive
      sync                Sync formulas and settings

For example, to check SSH connectivity to all of the nodes in an existing cluster:

    $ adam provision -n cluster check-ssh
    Checking SSH connection to nodes
    2016-10-03 22:10:49,327 - adam.models - DEBUG - Checking ssh connection for 54.88.20.164
    2016-10-03 22:10:50,905 - adam.models - DEBUG - SSH connection to 54.88.20.164: OK
    2016-10-03 22:10:51,008 - adam.models - DEBUG - Checking ssh connection for 52.90.168.130
    2016-10-03 22:10:51,996 - adam.models - DEBUG - SSH connection to 52.90.168.130: OK
    +---------------+---------------+
    | Node IP       | SSH Available |
    +---------------+---------------+
    | 54.88.20.164  | True          |
    | 52.90.168.130 | True          |
    +---------------+---------------+
    | All nodes     | True          |
    +---------------+---------------+

## Execute Salt Commands (Advanced)

You can execute Salt modules using `adam salt`.

For example, to execute the `test.ping` Salt module:

    $ adam salt -n cluster '*' test.ping
    192.168.1.1: true
    192.168.1.2: true
    192.168.1.3: true
    192.168.1.4: true