Continuum Analytics: Documentation

# User Guide

- Overview
- Manage Anaconda on the Cluster
- Install and Manage Platform Components
- Remote Commands
- SSH to Nodes
- Version and License Information
- List Cluster Information
- Creating, Provisioning, and Removing Clusters
- Additional Provision Commands (Advanced)
- Execute Salt Commands (Advanced)

## Overview

Typing the `adam` command shows the different subcommands that can be used:

    $ adam
    Usage: adam [OPTIONS] COMMAND [ARGS]...

    Options:
      --version                       Show the version and exit.
      -l, --log-level [info|debug|error]
                                      Logging level  [default: debug]
      -h, --help                      Show this message and exit.

    Commands:
      accelerate            Anaconda Accelerate options
      cmd                   Execute a command on the nodes
      describe              Describe an Adam cluster or profile
      ec2                   EC2 options
      enterprise-notebooks  Anaconda Enterprise Notebooks options
      info                  Display Adam version, system, and license information
      list                  List Adam clusters and profiles
      mosaic                Anaconda Mosaic options
      provision             Provision options
      remove                Remove a cluster
      repository            Anaconda Repository options
      salt                  Execute a Salt module
      scale                 Anaconda Scale options
      ssh                   SSH to one of the nodes (0-based index)
      up                    Create a cluster from a profile

You can view more information about a subcommand using the `-h` or `--help` option, for example, `adam up --help`.

More detail on the available subcommands is shown in the sections below.

## Manage Anaconda on the Cluster

Using Anaconda Scale, you can create and manage conda packages, including Python and R, in multiple conda environments across your nodes. Refer to the Anaconda Scale documentation for more information about installing and managing Anaconda across a cluster.

## Install and Manage Platform Components

- `adam accelerate`
- `adam enterprise-notebooks`
- `adam mosaic`
- `adam repository`
- `adam scale`

For more information on installing and managing platform components, refer to the Getting Started documentation.

## Remote Commands

You can execute remote commands on all of the cluster nodes using `adam cmd`:

    $ adam cmd --help
    Usage: adam cmd [OPTIONS] COMMAND

    Options:
      -n, --cluster TEXT  Cluster name  [required]
      -t, --target TEXT   Target nodes  [default: *]
      -h, --help          Show this message and exit.

For example, to run the `date` command on all nodes:

    $ adam cmd -n my-cluster date
    192.168.1.1: Wed Jun 29 03:08:10 UTC 2016
    192.168.1.2: Wed Jun 29 03:08:10 UTC 2016
    192.168.1.3: Wed Jun 29 03:08:10 UTC 2016
    192.168.1.4: Wed Jun 29 03:08:10 UTC 2016

## SSH to Nodes

You can SSH to the cluster nodes using `adam ssh`:

    $ adam ssh -n my-cluster
    Last login: Wed Jun 29 03:08:49 2016 from 172.18.0.1
    [root@192.168.1.1 ~]#

By default, you will connect to the head node. You can optionally include the node to SSH to, where node 0 is the head node, node 1 is the first compute node, and so on:

    $ adam ssh -n my-cluster 2
    Last login: Wed Jun 29 03:10:14 2016 from 172.18.0.1
    [root@192.168.1.3 ~]#

## Version and License Information

You can view version and license information using `adam info`:

    $ adam info
    Adam version: 4.1.0
    Plugins:
      adam_enterprise_notebooks: 4.1.0
      adam_repository: 4.1.0
      adam_accelerate: 4.1.0
      adam_mosaic: 4.1.0
      adam_notebook: 4.1.0
      adam_dask: 4.1.0

    Platform: Darwin-15.6.0-x86_64
    Darwin Kernel Version 15.6.0: Thu Jun 23 18:25:34 PDT 2016; root:xnu-3248.60.10~1/RELEASE_X86_64
    Python version: 3.5.1 |Continuum Analytics, Inc.| (default, Dec  7 2015, 11:24:55)
    [GCC 4.2.1 (Apple Inc. build 5577)]
    Processor: i386
    Byte-ordering: little

    License information:
    Number of nodes currently in use: 14
    Number of licensed nodes: 16
    Number of managed clusters: 2

    Valid platform component licenses:
    1: Anaconda Repository Enterprise. End date: 2017-05-24
    2: Anaconda Cluster. End date: 2017-05-24
    3: Wakari Enterprise. End date: 2017-05-24
    4: mkl-optimizations. End date: 2017-05-24
    5: iopro. End date: 2017-05-24
    6: accelerate. End date: 2017-05-24

## List Cluster Information

You can view a list of running clusters using `adam list`:

    $ adam list
    my-cluster
    second-cluster
    third-cluster

You can view a list of cluster profiles using `adam list profiles`:

    $ adam list profiles
    my-cluster-profile
    second-cluster-profile
    third-cluster-profile

You can view more detailed information about a cluster or profile using `adam describe`. For example, to view information about the cluster named `my-cluster`:

    $ adam describe -n my-cluster
    conda:
      anaconda_hash: md5=42dac45eee5e58f05f37399adda45e85
      anaconda_url: https://repo.continuum.io/miniconda/Miniconda2-4.0.5-Linux-x86_64.sh#adam-ci
      enabled: true
      rootdir: /opt/continuum
    name: my-cluster
    nodes:
    - host: ci_head_1
      keypair: ~/.ssh/my_private_key
      port: 22
      username: root
    - host: ci_compute_1
      keypair: ~/.ssh/my_private_key
      port: 22
      username: root

## Creating, Provisioning, and Removing Clusters

You can create, provision, and remove clusters using `adam up`, `adam provision`, and `adam remove`.

To create a new cluster from a profile:

    $ adam up -n my-cluster my-cluster

To (re)provision an existing cluster:

    $ adam provision -n my-cluster

To remove an existing bare-metal cluster:

    $ adam remove -n my-cluster
    Are you sure you want to remove the cluster my-cluster? [y/N]: y

To destroy an existing cloud-based cluster:

    $ adam ec2 -n my-cluster destory
    Are you sure you want to destroy the cluster my-cluster? [y/N]: y

## Additional Provision Commands (Advanced)

By default, `adam provision` will install check SSH connectivity, create a default user, install Salt, sync formulas, and provision enabled plugins and security settings that are defined in the cluster profile.

If you only need to run a portion of the `adam provision` process, you can view the available subcommands:

    $ adam provision --help
    Usage: adam provision [OPTIONS] COMMAND [ARGS]...

      Execute all the provisioning steps for a cluster

      See subcommands help for more info. Usage: adam provision -n

    Options:
      -n, --cluster TEXT    Cluster name  [required]
      --flush-iptables      Flush iptables  [default: False]
      --selinux-permissive  Set SELinux to permissive  [default: False]
      --selinux-context     Set SELinux contexts  [default: False]
      --plugins             Install enabled plugins on provision  [default: False]
      -h, --help            Show this message and exit.

    Commands:
      anaconda-user       Create the default anaconda user in all the nodes
      check-ssh           Check SSH connection to the nodes
      flush-iptables      Flush iptables
      plugins             Provision enabled plugins
      roles               Set roles on the cluster nodes
      salt                Provision Salt
      security-settings   Execute the security settings
      selinux-context     Set SELinux contexts
      selinux-permissive  Set SELinux to permissive
      sync                Sync formulas and settings

For example, to check SSH connectivity to all of the nodes in an existing cluster:

    $ adam provision -n cluster check-ssh
    Checking SSH connection to nodes
    2016-10-03 22:10:49,327 - adam.models - DEBUG - Checking ssh connection for 54.88.20.164
    2016-10-03 22:10:50,905 - adam.models - DEBUG - SSH connection to 54.88.20.164: OK
    2016-10-03 22:10:51,008 - adam.models - DEBUG - Checking ssh connection for 52.90.168.130
    2016-10-03 22:10:51,996 - adam.models - DEBUG - SSH connection to 52.90.168.130: OK
    +---------------+---------------+
    | Node IP       | SSH Available |
    +---------------+---------------+
    | 54.88.20.164  | True          |
    | 52.90.168.130 | True          |
    +---------------+---------------+
    | All nodes     | True          |
    +---------------+---------------+

## Execute Salt Commands (Advanced)

You can execute Salt modules using `adam salt`.

For example, to execute the `test.ping` Salt module:

    $ adam salt -n my-cluster '*' test.ping
    192.168.1.1: true
    192.168.1.2: true
    192.168.1.3: true
    192.168.1.4: true