======================================
Updating Anaconda Enterprise Notebooks
======================================

Summary
=======

These instructions are for upgrading Anaconda Enterprise Notebooks
Platform 0.10.0 or later. For help updating earlier versions, please
contact your enterprise support representative.

NOTE: The Named Service Account functionality of Anaconda Enterprise
Notebooks 4.0 is available only via a full installation of Anaconda
Enterprise Notebooks. The upgrade process does not add this new
functionality. For full installation support, please contact your
enterprise support representative.

An update to the Anaconda Enterprise Notebooks Platform requires all
instances of the three service categories to be updated individually:

-  *AEN Server*
-  *AEN Gateway*
-  *AEN Compute*

A typical installation of Anaconda Enterprise Notebooks likely consists
of several *Compute* instances running on independent *Compute Nodes*,
and possibly multiple *Gateways* if there is more than one *Data Center*
tied into Anaconda Enterprise Notebooks. More details can be found in
the :doc:`Overview <index>`.

The update process requires that all of Anaconda Enterprise Notebooks
service instances be stopped, updated, and then restarted. These
instructions describe how to perform this process.

Note: Any commands that call for the ``root`` user can also be done
using ``sudo``.

If you encounter any difficulty during the process, please refer to the
:doc:`Troubleshooting Guide <../admin/troubleshooting>` which
provides guidance on:

-  processes
-  configuration files
-  log files
-  ports

In the event you are unable to resolve an installation or upgrade
problem please contact your enterprise support representative.

.. _aen-installation-update-preflight:

Pre-Flight Check
================

IMPORTANT: The upgrade process to v4.0.0 is not reversible. You must
have a tested backup of your installation before starting the upgrade.

Prior to updating each service on each host, you should perform the
following operations:

#. As the ``root`` user, check for pre-existing ``.condarc``
   configuration files:

   ::

       ls ~root/.condarc
       ls ~wakari/.condarc

   If they exist, inspect these files for any special configuration
   **not** related to Anaconda Enterprise Notebooks. These settings need
   to be transferred to ``.condarc`` in the *root* environment
   (``/opt/wakari/miniconda/.condarc`` or
   ``/opt/wakari/anaconda/.condarc``)

   If there is an *anaconda.org token string* (similar to a UUID) in one
   of *channel* entries embedded in the path or URL, make a note of it.

   Once this has been completed, remove the following files:

   -  ``~root/.condarc``
   -  ``~wakari/.condarc``

#. The Anaconda (or Miniconda) install's conda configuration file
   ``.condarc`` needs to be verified. This file is in the top level
   directory of the Anaconda or Miniconda installation. It should exist
   and have at least the following entries (and possibly others):

   ::

       channels:
         - https://conda.anaconda.org/wakari
         - defaults

   The ``.condarc`` file **should not have any other** references to
   specific versions of Anaconda Enterprise Notebooks. If found, remove
   such entries from the ``.condarc`` file.

#. Suspend the services on each of the nodes with the appropriate
   command. For Anaconda Enterprise Notebooks versions 0.10.0 and later:

   ::

       sudo service wakari-server stop
       sudo service wakari-gateway stop
       sudo service wakari-compute stop

#. Versions of Anaconda Enterprise Notebooks prior to 0.10.0 used
   ``crontab`` entries to restart the services after a reboot. These
   have been replaced by init scripts. Remove the ``crontab`` entries
   from all nodes by running ``crontab -e`` and removing the ``@reboot``
   lines.

AEN Server
==========

**NOTE:** If you are using LDAP-based authentication, you need to backup
the configuration file:

``/opt/wakari/wakari-server/etc/wakari/wk-server-config.json``

And then after the *Server* has been updated, copy it back in place. The
current update process will overwrite this file. This is a known issue
scheduled to be resolved in an upcoming release.

The following operations are all done on the *Server* host.

#. To take advantage of the new search capabilities, install
   ElasticSearch and a Java Runtime Environment (JRE 8).

   -  `ElasticSearch <https://www.elastic.co/downloads/elasticsearch>`_
   -  `Oracle JRE 8 for
      Linux-64 <http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html>`_

   Stop the ElasticSearch service:

   ::

       service elasticsearch stop

   and remove any previous index (if exist) with:

   ::

       rm -rf /var/lib/elasticsearch/*

#. Ensure that Anaconda Enterprise Notebooks administrative account
   ``wakari`` owns all files in the *Server* installation, by running
   the following as ``root``

   ::

       chown -R wakari:wakari /opt/wakari/miniconda /opt/wakari/wakari-server

#. Update the *Server*:

   ::

       pushd /tmp
       wget http://j.mp/aen-server-update-latest

       sudo -u wakari /opt/wakari/miniconda/bin/conda install \
                        -p /opt/wakari/wakari-server          \
                        --file aen-server-update-latest

       sudo -u wakari /opt/wakari/miniconda/bin/conda install \
                        -p /opt/wakari/wakari-server          \
                        --no-deps                             \
                        wakari-enterprise-server-conf-update
       popd

#. Fix file ownership and permissions, by running the following as
   ``root``

   ::

       chown -R wakari:wakari /opt/wakari/miniconda /opt/wakari/wakari-server
       chmod -R a+rX  /opt/wakari/miniconda
       chmod -R o-w   /opt/wakari/miniconda
       chmod -R go+rX /opt/wakari/wakari-server
       chmod -R o-rwx /opt/wakari/wakari-server
       chmod    o+rX  /opt/wakari/wakari-server
       chmod -R o+rX  /opt/wakari/wakari-server/share

#. Start ElasticSearch:

   ::

       service elasticsearch start

   Or, if you do *not* want to use the search features, edit your
   server's ``config.json``:

   ::

       /opt/wakari/wakari-server/etc/wakari/config.json

   adding the line item: ``"SEARCH_ENABLED": false``.

#. Start the Server:

   ::

       service wakari-server start

#. Check that the Server is running properly:

   ::

       service wakari-server status

#. Finally, use your web browser to connect to AEN Server, using the
   correct protocol (*http* or *https*), hostname, and port number.

AEN Gateway
===========

The following operations are all done on **each** *Gateway* host.

#. Ensure that Anaconda Enterprise Notebooks administrative account
   ``wakari`` owns all files in the *Gateway* installation, by running
   the following as ``root``

   ::

       chown -R wakari:wakari /opt/wakari/miniconda /opt/wakari/wakari-gateway

#. Update the *Gateway*:

   ::

       pushd /tmp
       wget http://j.mp/aen-gateway-update-latest

       sudo -u wakari /opt/wakari/miniconda/bin/conda install \
                        -p /opt/wakari/wakari-gateway         \
                        --file aen-gateway-update-latest

       sudo -u wakari /opt/wakari/miniconda/bin/conda install \
                        -p /opt/wakari/wakari-gateway         \
                        --no-deps                             \
                        wakari-enterprise-gateway-conf-update
       popd

#. Fix file ownership and permissions, by running the following as
   ``root``

   ::

       chown -R wakari:wakari /opt/wakari/miniconda /opt/wakari/wakari-gateway
       chmod -R a+rX /opt/wakari/miniconda
       chmod -R o-w  /opt/wakari/miniconda
       chmod -R go+rX /opt/wakari/wakari-gateway
       chmod -R o-rwx  /opt/wakari/wakari-gateway

#. Start the *Gateway*:

   ::

       service wakari-gateway start

#. Finally, use your web browser to connect to the *Gateway*, using the
   correct protocol (*http* or *https*), hostname, and port number.

AEN Compute
===========

The following operations are all done on **each** host where an AEN
Compute service is running.

#. Check if there is any ``wakari-indexer`` processes running with:

   ::

       ps aux | grep indexer

   And terminate any remaining ``wakari-indexer`` processes, if you
   find them, with:

   ::

       killall wakari-indexer

#. Ensure that the AEN administrative account ``wakari`` owns all files
   in the compute installation, by running the following as ``root``:

   ::

       chown -R wakari:wakari /opt/wakari/anaconda
       chown -R wakari:wakari /opt/wakari/wakari-compute/{bin,conda-meta,config,etc}
       chown -R wakari:wakari /opt/wakari/wakari-compute/{include,lib,lib64,pkgs,share,ssl}

#. Verify the contents of ``/opt/wakari/anaconda/.condarc``, as
   described :ref:`above <aen-installation-update-preflight>`. In addition, it should contain the
   following entries (and possibly others):

   ::

       channels:
         - https://conda.anaconda.org/wakari
         - https://conda.anaconda.org/t/<TOKEN>/anaconda-nb-extensions
         - r
         - defaults

       create_default_packages:
         - python
         - ipython-we
         - pip

**NOTE:** Contact our sales representatives to get your ``<TOKEN>`` for
the Anaconda channel referenced above.

#. Install the ``R`` dependencies:

   ::

       yum groupinstall "X Window System" -y

   Note: The ``groupinstall`` should be enough to cover all the R needs
   but in case you have problems with R-based plotting in the notebook,
   please also run:

   ::

       yum install libXrender libXext libSM -y

#. Update the *Compute* service:

   ::

       sudo -u wakari /opt/wakari/anaconda/bin/conda install \
                        -p /opt/wakari/anaconda/             \
                        conda=3.19.3 conda-env=2.4.5

       pushd /tmp
       wget http://j.mp/aen-compute-update-latest
       wget http://j.mp/aen-anaconda-update-latest

       sudo -u wakari /opt/wakari/anaconda/bin/conda install \
                        -p /opt/wakari/wakari-compute        \
                        --file aen-compute-update-latest

       sudo -u wakari /opt/wakari/anaconda/bin/conda install \
                        --no-deps                            \
                        -p /opt/wakari/wakari-compute        \
                        wakari-enterprise-compute-conf-update 

       sudo -u wakari /opt/wakari/anaconda/bin/conda install \
                        -p /opt/wakari/anaconda              \
                        --file aen-anaconda-update-latest
       popd

#. Fix file ownership and permissions, by running the following as
   ``root``

   ::

       chown -R wakari:wakari /opt/wakari/anaconda
       chown -R wakari:wakari /opt/wakari/wakari-compute/{bin,conda-meta,config,etc}
       chown -R wakari:wakari /opt/wakari/wakari-compute/{include,lib,lib64,pkgs,share,ssl}
       chmod -R a+rX /opt/wakari/anaconda
       chmod -R o-w  /opt/wakari/anaconda

#. Initialize the root environment to prime the package cache

   ::

       sudo -u wakari /opt/wakari/anaconda/bin/conda create \
                        -p /opt/wakari/testenv \
                        --clone root

       rm -rf /opt/wakari/testenv

#. Pin conda to version 3.19.3 (and compatible conda-env) to avoid
   automatic self-updates on the anaconda environment

   ::

       sudo echo -e "conda ==3.19.3\nconda-env ==2.4.5" > \
                 /opt/wakari/anaconda/conda-meta/pinned

#. With the latest Anaconda Enterprise Notebooks release (4.0.0) we have
   introduced new features such as Functional IDs and no root access
   during normal operations. You **must** execute the
   ``upgrade\_400.sh`` script to make your projects compatible with the new codebase:

   ::

       chmod a+x upgrade_400.sh
       ./upgrade_400.sh

#. Start the *Compute* service:

   ::

       service wakari-compute start

#. Verify the *Compute* service is running properly:

   ::

       ps auxef | grep bin/wk-compute
       service wakari-compute status

   If you need ``R`` in a pre-existing project, use the following
   command to install the needed ``R`` packages:

   ::

       source activate /projects/$USERNAME/$PROJECTNAME/envs/default
       sudo -u $USERNAME conda install r-base r-essentials -c r
       source deactivate

   **NOTE:** In order to start using the new jupyter extensions you'll
   need to remove the ``.jupyter_$USERNAME`` and ``.ipython_$USERNAME``
   folders.

#. If you have a :doc:`customized default
   environment <../user/projects>`, you could update it
   depending of your users needs.

**NOTE:** Updating the ``default`` environment at
``/opt/wakari/anaconda/envs/default`` do not automatically update the
``default`` environment in the users **pre-existing** projects. For
pre-existing projects the update (if it is rquested) should be done on a
per-user basis.

.. toctree::
   :hidden:

   scripts/project_permissions_py
   scripts/repair_team_members_py
   scripts/upgrade_400_sh
