=============================
Mirroring a PyPI repository
=============================

Before you begin
================

You need to have already completed `installing and configuring your Anaconda Repository instance <install.html>`_.  Due to the size of Anaconda Repository, it is important that you have configured a file storage location with sufficient disk space.  If necessary please see the `instructions for setting the file storage location <install.html#configure-the-server>`_.

The full PyPI mirror requires approximately 120GB.

The mirror command
==================

To create a PyPI mirror run::

    anaconda-server-sync-pypi

This will load all of the packages on pypi.python.org into the `~pypi` binstar user account.

Verify that this is working by opening your browser to ``http://your.anaconda.repository/pypi/~pypi``

NOTE: Replace your.anaconda.repository with the actual URL to your Anaconda Repository.

Customizing the mirror
======================

It is possible to customize the mirror behavior by creating a configuration file such as ``$PREFIX/etc/anaconda-server/mirror/pypi.yaml`` and using the ``--mirror-config`` option::

    anaconda-server-sync-pypi --mirror-config /etc/binstar/mirrors/pypi.yaml

The following configuration options are available:

+-----------------------+--------------------------------------------------------+
| Name                  | Description                                            |
+=======================+========================================================+
| ``user``              | The local user under which the PyPI packages are       |
|                       | imported. **Default:** ``pypi``                        |
+-----------------------+--------------------------------------------------------+
| ``whitelist``         | A list of packages to mirror. Only packages listed are |
|                       | mirrored. If the list is empty, all packages are       |
|                       | checked. **Default:** ``[]``                           |
+-----------------------+--------------------------------------------------------+
| ``blacklist``         | A list of packages to skip. The packages listed are    |
|                       | ignored. **Default:** ``[]``                           |
+-----------------------+--------------------------------------------------------+
| ``latest_only``       | Only download the latest versions of the packages.     |
|                       | **Default:** ``false``.                                |
+-----------------------+--------------------------------------------------------+
| ``remote_url``        | The URL of the PyPI mirror. ``/pypi`` is appended to   |
|                       | build the XML RPC API URL, ``/simple`` for the simple  |
|                       | index, and ``/pypi/{package}/{version}/json`` for the  |
|                       | JSON API. **Default:** ``https://pypi.python.org/``    |
+-----------------------+--------------------------------------------------------+
| ``xml_rpc_api_url``   | A custom value for XML RPC URL. If this value is       |
|                       | present, it takes precedence over the URL built using  |
|                       | ``remote_url``. **Default:** ``null``                  |
+-----------------------+--------------------------------------------------------+
| ``simple_index_url``  | A custom value for the simple index URL. If this value |
|                       | is present, it takes precedence over the URL built     |
|                       | using ``remote_url``. **Default:** ``null``            |
+-----------------------+--------------------------------------------------------+
| ``use_xml_rpc``       | Whether to use the XML RPC API as specified by         |
|                       | `PEP381 <https://www.python.org/dev/peps/pep-0381/>`_. |
|                       | If this is set to ``true``, the simple index is used   |
|                       | to determine which packages to check. The simple index |
|                       | is also used if the XML RPC API fails for any reason.  |
|                       | **Default:** ``true``                                  |
+-----------------------+--------------------------------------------------------+
| ``use_serial``        | Whether to use the serial number provided by the XML   |
|                       | RPC API. Only packages updated since the last serial   |
|                       | saved will be checked. If this is set to false **all   |
|                       | PyPI packages are going to be checked for updates.**   |
|                       | **Default:** ``true``                                  |
+-----------------------+--------------------------------------------------------+
| ``create_org``        | Create the mirror user as an Organization instead of a |
|                       | regular user account. All superusers are added to the  |
|                       | "Owners" group of the organization.                    |
|                       | **Default:** ``false``                                 |
+-----------------------+--------------------------------------------------------+

Example:
--------

.. code-block:: yaml

    whitelist:
      - requests
      - six
      - numpy
      - simplejson
    latest_only: true
    remote_url: http://pypimirror.local/
    use_xml_rpc: false

Configure pip
=============

To configure pip to use this new mirror you must edit your `~/.pip/pip.conf` file::

    [global]
    index-url = http://your.anaconda.repo:<port>/pypi/~pypi/simple

NOTE: Replace ``your.anaconda.repo`` with the actual URL to your Anaconda Repository.
