{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "lgtY-H9A--wi"
   },
   "source": [
    "# Tutorial 04 - Optimizing Force Fields\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openforcefield/openff-evaluator/blob/master/docs/tutorials/tutorial04.ipynb)\n",
    "\n",
    "In this tutorial we will be using the OpenFF Evaluator framework in combination with the fantastic [*ForceBalance*](\n",
    "https://github.com/leeping/forcebalance) software to optimize a molecular force field against the physical property data \n",
    "set we created in the [first tutorial](tutorial01.ipynb).\n",
    "\n",
    "*ForceBalance* offers a suite of tools for optimizing molecular force fields against a set of target data. Perhaps one of \n",
    "the most fundamental targets to fit against is experimental physical property data. Physical property data has been used \n",
    "extensively for decades to inform the values of non-bonded Van der Waals (VdW) interaction parameters (often referred to \n",
    "as Lennard-Jones parameters).\n",
    "\n",
    "*ForceBalance* is seamlessly integrated with the evaluator framework, using it to evaluate the deviations between\n",
    "target experimentally measured data points and those evaluated using the force field being optimized (as well as the \n",
    "gradient of those deviations with respect to the force field parameters being optimized).\n",
    "\n",
    "The tutorial will cover:\n",
    "\n",
    "- setting up the input files and directory structure required by ForceBalace.\n",
    "- setting up an `EvaluatorServer` for *ForceBalance* to connect to.\n",
    "- running *ForceBalance* using those input files.\n",
    "- extracting and plotting a number of statistics output during the optimization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "cPgYOGJG--wj"
   },
   "source": [
    "*Note: If you are running this tutorial in google colab you will need to run a setup script instead of following the \n",
    "installation instructions:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OS7HuXMj--wn",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# !wget https://raw.githubusercontent.com/openforcefield/openff-evaluator/master/docs/tutorials/colab_setup.ipynb\n",
    "# %run colab_setup.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "4JOt9Apq--wt"
   },
   "source": [
    "*For this tutorial make sure that you are using a GPU accelerated runtime.*\n",
    "\n",
    "For the sake of clarity all warnings will be disabled in this tutorial:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "k7tFiECf--wu",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "import logging\n",
    "logging.getLogger(\"openforcefield\").setLevel(logging.ERROR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "sFwGxGQA--w0"
   },
   "source": [
    "We will also enable time-stamped logging to help track the progress of our calculations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "nqp2BKbJ--w1",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from openff.evaluator.utils import setup_timestamp_logging\n",
    "setup_timestamp_logging()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "cdnCJ18b--w6"
   },
   "source": [
    "## Setting up the ForceBalance Inputs\n",
    "\n",
    "In this section we will be creating the directory structure required by *ForceBalance*, and populating it with the \n",
    "required input files.\n",
    "\n",
    "### Creating the Directory Structure\n",
    "\n",
    "To begin with, we will create a directory to store the starting force field parameters in:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Fp-lsXMt--w7",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "!mkdir forcefield"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "J7sd8tN8--w-"
   },
   "source": [
    "and one to store the input parameters for our 'fitting target' - in this case a data set of physical properties:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_ZX45ta6--xA",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "!mkdir -p targets/pure_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "dYYPy60I--xE"
   },
   "source": [
    "### Defining the Training Data Set\n",
    "\n",
    "With the directories created, we will next specify the data set of physical properties which we will be training the \n",
    "force field against:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "qlLsxPQm--xG",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# For convenience we will use the copy shipped with the framework\n",
    "from openff.evaluator.utils import get_data_filename\n",
    "data_set_path = get_data_filename(\"tutorials/tutorial01/filtered_data_set.json\")\n",
    "\n",
    "# Load the data set.\n",
    "from openff.evaluator.datasets import PhysicalPropertyDataSet\n",
    "data_set = PhysicalPropertyDataSet.from_json(data_set_path)\n",
    "\n",
    "# Due to a small bug in ForceBalance we need to zero out any uncertainties\n",
    "# which are undefined. This will be fixed in future versions.\n",
    "from openff.evaluator.attributes import UNDEFINED\n",
    "\n",
    "for physical_property in data_set:\n",
    "    \n",
    "    if physical_property.uncertainty != UNDEFINED:\n",
    "        continue\n",
    "        \n",
    "    physical_property.uncertainty = 0.0 * physical_property.default_unit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "GaPGpS4d--xL",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "To speed up the runtime of this tutorial, we will only train the force field against measurements made for ethanol"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UD7Vim_u--xM",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "data_set.filter_by_smiles(\"CCO\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "hDpgPpB4--xQ",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "in real optimizations however the data set should be **much** larger than two data points!\n",
    "\n",
    "With those changes made, we can save the data set in our targets directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MMsDtL9y--xR",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Store the data set in the `pure_data` targets folder:\n",
    "data_set.json(\"targets/pure_data/training_set.json\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "5Zy5rehy--xV"
   },
   "source": [
    "### Defining the Starting Force Field Parameters\n",
    "\n",
    "We will use the OpenFF Parsley 1.0.0 force field as the starting parameters for the optimization. These can be loaded\n",
    "directly into an OpenFF `ForceField` object using the OpenFF toolkit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "EaHKkQ61--xW",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from openforcefield.typing.engines.smirnoff import ForceField\n",
    "force_field = ForceField('openff-1.0.0.offxml')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "fCSYfufg--xa",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "In order to use these parameters in *ForceBalance*, we need to 'tag' the individual parameters in the force field that\n",
    "we wish to optimize. The toolkit easily enables us to add these tags using cosmetic attributes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "fFzLhLKE--xb",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Extract the smiles of all unique components in our data set.\n",
    "from openforcefield.topology import Molecule, Topology\n",
    "\n",
    "all_smiles = set(\n",
    "    component.smiles\n",
    "    for substance in data_set.substances\n",
    "    for component in substance.components\n",
    ")\n",
    "\n",
    "for smiles in all_smiles:\n",
    "\n",
    "    # Find those VdW parameters which would be applied to those components.\n",
    "    molecule = Molecule.from_smiles(smiles)\n",
    "    topology = Topology.from_molecules([molecule])\n",
    "\n",
    "    labels = force_field.label_molecules(topology)[0]\n",
    "\n",
    "    # Tag the exercised parameters as to be optimized.\n",
    "    for parameter in labels[\"vdW\"].values():\n",
    "        parameter.add_cosmetic_attribute(\"parameterize\", \"epsilon, rmin_half\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "cIs15IZl--xe",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Here we have made use of the toolkit's handy `label_molecules` function to see which VdW parameters will be assigned\n",
    "to the molecules in our data set, and tagged them to be parameterized.\n",
    "\n",
    "With those tags added, we can save the parameters in the `forcefield` directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tOs0bcBo--xf",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Save the annotated force field file.\n",
    "force_field.to_file('forcefield/openff-1.0.0-tagged.offxml')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "Zu8JXeni--xj"
   },
   "source": [
    "*Note: The force field parameters are stored in the [OpenFF SMIRNOFF XML format](https://open-forcefield-toolkit.readthedocs.io/en/0.6.0/smirnoff.html).*\n",
    "\n",
    "### Creating the Main Input File\n",
    "\n",
    "Next, we will create the main *ForceBalance* input file. For the sake of brevity a default input file which ships with \n",
    "this framework will be used:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "OnaiZmGW--xl",
    "outputId": "bffbe59b-b819-4307-d5fd-4f01fd2422e6",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'optimize.in'"
      ]
     },
     "execution_count": 12,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_file_path = get_data_filename(\"tutorials/tutorial04/optimize.in\")\n",
    "\n",
    "# Copy the input file into our directory structure\n",
    "import shutil\n",
    "shutil.copyfile(input_file_path, \"optimize.in\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "Oxvx-yIo--xq"
   },
   "source": [
    "While there are many options that can be set within this file, the main options of interest for our purposes appear\n",
    "at the bottom of the file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 119
    },
    "colab_type": "code",
    "id": "vDgpFQUq--xr",
    "outputId": "bc95c1d2-991b-422d-e503-2b68fdd5e250",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "$target\n",
      "name pure_data\n",
      "type Evaluator_SMIRNOFF\n",
      "weight 1.0\n",
      "openff.evaluator_input options.json\n",
      "$end"
     ]
    }
   ],
   "source": [
    "!tail -n 6 optimize.in"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "SleJoA3i--xu"
   },
   "source": [
    "Here we have specified that we wish to create a new *ForceBalance* `Evaluator_SMIRNOFF` target called `pure_data` \n",
    "(corresponding to the name of the directory we created in the earlier step).\n",
    " \n",
    "The main input to this target is the file path to an `options.json` file - it is this file which will specify all the\n",
    "options which should be used when *ForceBalance* requests that our target data set be estimated using the current sets\n",
    "of force field parameters.\n",
    "\n",
    "We will create this file in the `targets/pure_data` directory later in this section."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "DIUi886c--xv",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "The data set is the JSON serialized representation of the `PhysicalPropertyDataSet` we created during the [first \n",
    "tutorial](tutorial01.ipynb).\n",
    "\n",
    "### Defining the Estimation Options\n",
    "\n",
    "The final step before we can start the optimization is to create the set of options which will govern how our data set\n",
    "is estimated using the Evaluator framework.\n",
    " \n",
    "These options will be stored in an `Evaluator_SMIRNOFF` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Tyloz19F--xw",
    "pycharm": {
     "name": "#%% \n"
    }
   },
   "outputs": [],
   "source": [
    "from forcebalance.evaluator_io import Evaluator_SMIRNOFF\n",
    "\n",
    "# Create the ForceBalance options object\n",
    "target_options = Evaluator_SMIRNOFF.OptionsFile()\n",
    "# Set the path to the data set\n",
    "target_options.data_set_path = \"training_set.json\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "mOy6-hgf--xz",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "This object exposes both a set of *ForceBalance* specific options, as well as the set of Evaluator options. \n",
    "\n",
    "The *ForceBalance* specific options allow us to define how each type of property will contribute to the optimization\n",
    "objective function (the value which we are trying to minimize):\n",
    "\n",
    "$$\\Delta(\\theta) = \\sum^N_n \\dfrac{weight_n}{M_n} \\sum^{M_n}_m \\left(\\dfrac{y_m^{ref} - y_m(\\theta)}{denominator_{n}}\\right)^2$$ \n",
    "\n",
    "where $N$ is the number of types of properties (e.g. density, enthalpy of vaporization, etc.), $M_n$ is the number of\n",
    "data points of type $n$, $y_m^{ref}$ is the experimental value of data point $m$ and $y_m(\\theta)$ is the estimated \n",
    "value of data point $m$ using the current force field parameters\n",
    "\n",
    "In particular, the options object allows us to specify both an amount to scale each type of properties contribution to \n",
    "the objective function by ($weight_n$), and the amount to scale the difference between the experimental and estimated\n",
    "properties ($denominator_n$):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yLM9plwn--xz",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from openff.evaluator import unit\n",
    "\n",
    "target_options.weights = {\n",
    "    \"Density\": 1.0,\n",
    "    \"EnthalpyOfVaporization\": 1.0\n",
    "}\n",
    "target_options.denominators = {\n",
    "    \"Density\": 30.0 * unit.kilogram / unit.meter ** 3,\n",
    "    \"EnthalpyOfVaporization\": 3.0 * unit.kilojoule / unit.mole\n",
    "}\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "JfdFCewd--x3",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    " \n",
    "where here we have chosen values that ensure that both types of properties contribute roughly equally to the total\n",
    "objective function.\n",
    "\n",
    "The Evaluator specific options correspond to a standard `RequestOptions` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "70fNiyjg--x4",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from openff.evaluator.client import RequestOptions\n",
    "\n",
    "# Create the options which evaluator should use.\n",
    "evaluator_options = RequestOptions()\n",
    "# Choose which calculation layers to make available.\n",
    "evaluator_options.calculation_layers = [\"SimulationLayer\"]\n",
    "\n",
    "# Reduce the default number of molecules\n",
    "from evaluator.properties import Density, EnthalpyOfVaporization\n",
    "\n",
    "density_schema = Density.default_simulation_schema(n_molecules=256)\n",
    "h_vap_schema = EnthalpyOfVaporization.default_simulation_schema(n_molecules=256)\n",
    "\n",
    "evaluator_options.add_schema(\"SimulationLayer\", \"Density\", density_schema)\n",
    "evaluator_options.add_schema(\"SimulationLayer\", \"EnthalpyOfVaporization\", h_vap_schema)\n",
    "\n",
    "target_options.estimation_options = evaluator_options"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "O4-QdP-E--x7",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "These options allow us to control exactly how each type of property should be estimated, which calculation approaches\n",
    "should be used and more. Here we use the same options are were used in the [second tutorial](tutorial02.ipynb)\n",
    "\n",
    "*Note: more information about the different estimation options can be [found here](../gettingstarted/client.rst)*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "0qXKzNY7--x8",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "And that's the options created! We will finish off by serializing the options into our target directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7TNFFE75--x9",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Save the options to file.\n",
    "with open(\"targets/pure_data/options.json\", \"w\") as file:\n",
    "    file.write(target_options.to_json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "vZkeoQTV--yA",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Launching an Evaluator Server\n",
    "\n",
    "With the *ForceBalance* options created, we can now move onto launching the `EvaluatorServer` which *ForceBalance* will \n",
    "call out to when it needs the data set to be evaluated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "Jdr55Aez--yA",
    "outputId": "b56f258e-55c3-4e21-ae0c-fb939c37a95e",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "01:30:20.505 INFO     Server listening at port 8000\n"
     ]
    }
   ],
   "source": [
    "# Launch the calculation backend which will distribute any calculations.\n",
    "from openff.evaluator.backends import ComputeResources\n",
    "from openff.evaluator.backends.dask import DaskLocalCluster\n",
    "\n",
    "calculation_backend = DaskLocalCluster(\n",
    "    number_of_workers=1,\n",
    "    resources_per_worker=ComputeResources(\n",
    "        number_of_threads=1, \n",
    "        number_of_gpus=1, \n",
    "        preferred_gpu_toolkit=ComputeResources.GPUToolkit.CUDA\n",
    "    ),\n",
    ")\n",
    "calculation_backend.start()\n",
    "\n",
    "# Launch the server object which will listen for estimation requests and schedule any \n",
    "# required calculations.\n",
    "from openff.evaluator.server import EvaluatorServer\n",
    "\n",
    "evaluator_server = EvaluatorServer(calculation_backend=calculation_backend)\n",
    "evaluator_server.start(asynchronous=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "R2QFxF68--yF"
   },
   "source": [
    "We will not go into the details of this here as this was already covered in the [second tutorial](tutorial02.ipynb)\n",
    "\n",
    "## Running ForceBalance\n",
    "\n",
    "With the inputs created and an Evaluator server spun up, we are finally ready to run the optimization! This can be \n",
    "accomplished with a single command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "mKt1jGxI--yG",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "!ForceBalance optimize.in"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "xSuicGKR--yI",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "If everything went well *ForceBalance* should exit cleanly, and will have stored out newly optimized force field in the\n",
    "`results` directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "9XwxXlBX--yK",
    "outputId": "c0640494-9b26-4f7e-8e79-406158c3ab3d",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "openff-1.0.0-tagged_1.offxml  openff-1.0.0-tagged.offxml\n"
     ]
    }
   ],
   "source": [
    "!ls result/optimize"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "THb-Zxd4--yN",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Plotting the results\n",
    "\n",
    "As a last step in this tutorial, we will extract the objective function at each iteration from the *ForceBalance* output\n",
    "files and plot this using `matplotlib`.\n",
    "\n",
    "First, we will extract the objective function from the `pickle` serialized output files which can be found in the \n",
    "`optimize.tmp/pure_data/iter_****/` directories:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "o5Hhm6XB--yP",
    "outputId": "8fc45775-b549-425f-f49e-2e0c8fec1b78",
    "pycharm": {
     "name": "#%% \n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.9270359101845124, 0.011497456194198362]\n"
     ]
    }
   ],
   "source": [
    "from forcebalance.nifty import lp_load\n",
    "\n",
    "# Determine how many iterations ForceBalance has completed.\n",
    "from glob import glob\n",
    "n_iterations = len(glob(\"optimize.tmp/pure_data/iter*\"))\n",
    "\n",
    "# Extract the objective function at each iteration.\n",
    "objective_function = []\n",
    "\n",
    "for iteration in range(n_iterations):\n",
    "\n",
    "    folder_name = \"iter_\" + str(iteration).zfill(4)\n",
    "    file_path = f\"optimize.tmp/pure_data/{folder_name}/objective.p\"\n",
    "    \n",
    "    statistics = lp_load(file_path)\n",
    "    objective_function.append(statistics[\"X\"])\n",
    "\n",
    "print(objective_function)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "p0VXrv-3--yT",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "The objective function is then easily plotted:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 297
    },
    "colab_type": "code",
    "id": "DzOieX2B--yU",
    "outputId": "51331014-27cc-4829-84b6-bf64f0a550f4",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEYCAYAAACHjumMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxU9b3/8ddnlhC2sIZFQBZZBDFA\nHBjU1qXaQt2RihKQPVzXq72tba/2V71WrYr26s+r/kpCRNmUturF1sqtVOpSCQmrCCIIUjYhsm8h\nk5nP748ZvCFCMgk5OTOTz/PxmEdmTk7OvDPAm7N+j6gqxhjjBI/bAYwxqcsKxhjjGCsYY4xjrGCM\nMY6xgjHGOMbndoCaatu2rXbr1s3tGMaYCpYtW/a1qmZWnp50BdOtWzeKi4vdjmGMqUBEtpxqum0i\nGWMcYwVjjHGMFYwxxjFWMMYYx1jBGGMcYwVjjHFM0h2mjsebK7YzbeF6duw/xlktG3PfsD7cMKiT\n27GMaXBSrmDeXLGdf3/9E46FwgBs33+Mf3/9EwArGWPqWcptIk1buP6bcjnhWCjMtIXrXUpkTMOV\ncgWzY/+xGk03xjgn5QrmrJaNazTdGOOclCuY+4b1obHfe9I0Ae647Bx3AhnTgKVcwdwwqBO/ufF8\nOrVsjABtm6XhEZi79J8cOBpyO54xDYok26DfgUBAa3o19eL1u5n6yjL6dmzOrClBMtL9DqUzpmES\nkWWqGqg8PeXWYE7lsj7teHFsNmt3HmTiS0UcPl7udiRjGoQGUTAAV/Rtz3Ojs1m5dT+TZhZxtMxK\nxhinNZiCARjevwPP3DyQ4i/3MuXlYkornS9jjKlbDapgAK4dcBZPjxrAx5v2MHXWMisZYxzU4AoG\nYMSgzjxxYxbvf17CnXOWU1YecTuSMSmpQRYMwKjBXXh0RH8Wfbabu+ctJxS2kjGmrjXYggEYE+zK\nQ9f2Y+Gnu/jxayspt5Ixpk6l3NXUNTXh4u6Ewsqjb6/D7/Xw1E0D8HrE7VjGpIQGXzAAuZf0oCwc\nYdrC9fi9wuM3ZuGxkjHmjFnBxNx5eU/KyiM8u2gDPq+HR2/oj4iVjDFnwgqmgnuv7EUoHOGFxV+Q\n5vXw4LX9rGSMOQNWMBWICPcN60NZeYT8Dzfj9wr3X9XXSsaYWrKCqUREeODqvoTCEfI+2Eyaz8NP\nf9DHSsaYWrCCOQUR4aHrziMUUZ5/7wvSvF7uubKX27GMSTpWMKchIjxyfX9C5RH+893P8XmFOy/v\n6XYsY5KKFUwVPB7h8ZFZhGKHsBv5PEz5bg+3YxmTNKxgquH1CE/dNIBQWHnkz9GT8cZf1M3tWMYk\nBSuYOPi8Hp65ZSChcIQHF3yK3+shJ3i227GMSXgN+lqkmvB7PfxXTjbfO7cd97/xCfOLt7odyZiE\nZwVTA2k+Dy+Myea7vdry8z+u5o0V29yOZExCs4KpoXS/l7xxAS7s0YafzF/Fn1bvcDuSMQnLCqYW\n0v1e8scHCHRtzT2vruSdNV+5HcmYhORowYjIcBFZLyIbReQXp/j+2SLynoisEJHVInKVk3nqUpM0\nHwUTBzOgcwvunrecRet2uR3JmITjWMGIiBd4Hvgh0A8YLSL9Ks32S2C+qg4CbgFecCqPE5o18jFz\n0hD6dszg9tnL+fvnJW5HMiahOLkGMwTYqKqbVLUMeBW4vtI8CmTEnrcAkm6HRka6n1mTgvRs14yp\nrxTzj41fux3JmIThZMF0Aioey90Wm1bRQ8BYEdkGvA3cfaoFichUESkWkeKSksRbS2jRxM/sKUG6\ntWnK5JeLWbp5r9uRjEkIbu/kHQ3MVNXOwFXALBH5ViZVna6qAVUNZGZm1nvIeLRumsac3CBntUxn\n4ktLWbZln9uRjHGdkwWzHehS4XXn2LSKJgPzAVT1YyAdaOtgJke1bdaIeblDaZeRzoSCpazaut/t\nSMa4ysmCKQJ6iUh3EUkjuhN3QaV5/glcASAifYkWTOJtA9VAu4x05uYGadU0jVtnFLJm+wG3Ixnj\nGscKRlXLgbuAhcA6okeLPhWRh0XkuthsPwFyRWQVMA+YoKrqVKb60rFFY+bmBmme7ufWGYV89tVB\ntyMZ4wpJtn/PgUBAi4uL3Y4Rly17jnDz75YQCkd47V+G0rNdc7cjGeMIEVmmqoHK093eyZvSurZp\nytzcIB6PMDqvkE0lh92OZEy9soJxWI/MZsydEiQSUXLyCtmy54jbkYypN1Yw9aBX++bMyQ1yvDxM\nTl4h2/YddTuSMfXCCqaenNshg1mTgxwqDZGTV8jOA8fcjmSM46xg6lH/Ti2YNTnIviNl5OQVsvtg\nqduRjHGUFUw9G9ClJTMnDWb3wVJy8gv5+vBxtyMZ4xgrGBdc0LU1BRMGs33fMcbmF7L3SJnbkYxx\nhBWMS4I92pA/PsDmr48wNr+QA0dDbkcyps5Zwbjo4p5tmT4uwMbdhxlXUMjBUisZk1qsYFx2ae9M\nXhybzdqdB5lQsJTDx8vdjmRMnbGCSQBX9G3Pc6OzWbXtAJNmFnG0zErGpAYrmAQxvH8Hnr1lIMVf\n7mXKy8WUhsJuRzLmjFnBJJBrss7i6VED+HjTHqbOWmYlY5KeFUyCGTGoM0/cmMX7n5dw55zllJVH\n3I5kTK1ZwSSgUYO78OiI/iz6bDd3z1tOKGwlY5KTFUyCGhPsykPX9mPhp7u497WVlFvJmCTkczuA\nOb0JF3cnFFYefXsdaV4PT900AK9H3I5lTNysYBJc7iU9KAtHmLZwPT6P8MTILDxWMiZJWMEkgTsv\n70lZeYRnF23A7/Pw6A39EbGSMYnPCiZJ3HtlL0LhCC8s/oI0r4cHr+1nJWMSnhVMkhAR7hvWh1A4\nQt4Hm/F7hfuv6mslYxKaFUwSEYmWSiissZLxcN+wPlYyJmFZwSQZEeHBa/tRdmJzyefh3it7ux3L\nmFOygklCIsIj1/cnVB7hmXc34Pd6uPPynm7HMuZbrGCSlMcjPD4yi1DsEHaa10PuJT3cjmXMSaxg\nkpjXIzx10wBCkejJeH6vMOHi7m7HMuYbVjBJzuf18MzNAykPR3jorbX4fR7GBLu6HcsYII6CEZFM\nIBfoVnF+VZ3kXCxTE36vh+dGZ3Pb7GU88MYa/F4PowJd3I5lTFxrMP8NfAC8C9gAJQkqzefhhTHZ\n5L5SzM//uBq/VxgxqLPbsUwDF0/BNFHVnzuexJyxdL+XvHEBJs0s4ifzV+H3ergm6yy3Y5kGLJ7h\nGv4kIlc5nsTUiXS/l/zxAQJdW3PPqyt5Z81XbkcyDVg8BXMP0ZIpFZFDscdBp4OZ2muS5qNg4mAG\ndG7B3fOWs2jdLrcjmQaq2oJR1eaq6lHV9Njz5qqaUR/hTO01a+Rj5qQh9O2Ywe2zl/P3z0vcjmQa\noLhGtBOR60TkqdjjGqdDmbqRke5n1qQgPds1Y+orxXy08Wu3I5kGptqCEZHHiW4mrY097hGR3zgd\nzNSNFk38zJ4SpHvbpkx5uZjCTXvcjmQakHjWYK4Cvq+qBapaAAwHrnY2lqlLrZumMXtKkE6tGjNp\nZhHLtuxzO5JpIOId9LtlhectnAhinNW2WSPmTgnSLiOdCQVLWbV1v9uRTAMQT8H8BlghIjNF5GVg\nGfBoPAsXkeEisl5ENorIL04zzygRWSsin4rI3Pijm5pql5HO3NwgrZqmceuMQtZsP+B2JJPiRFWr\nn0mkIzA49nKpqlZ7coWIeIHPge8D24AiYLSqrq0wTy9gPvA9Vd0nIu1UdXdVyw0EAlpcXFxtZnN6\n2/Yd5ebfLeFIWTmvTh3KuR3soKA5MyKyTFUDlaefdg1GRM6Nfc0GOhItiW3AWbFp1RkCbFTVTapa\nBrwKXF9pnlzgeVXdB1BduZi60blVE+blDiXd52VMXiEbdh1yO5JJUVVtIv1b7OvTp3g8FceyOwFb\nK7zeFptWUW+gt4h8JCJLRGR4XKnNGTu7TRPm5gbxeISc/EI2lRx2O5JJQactGFWdGnv6Q1W9vOKD\n6JGluuADegGXAaOBPBFpWXkmEZkqIsUiUlxSYieM1ZUemc2YOyVIJKLk5BWyZc8RtyOZFBPPTt5/\nxDmtsu1AxTEDOsemVbQNWKCqIVXdTHSfTa/KC1LV6aoaUNVAZmZmHG9t4tWrfXPm5AY5Xh4mJ6+Q\nbfuOuh3JpJCq9sF0EJELgMYiMkhEsmOPy4AmcSy7COglIt1FJA24BVhQaZ43ia69ICJtiW4ybar5\nr2HOxLkdMpg1Ocih0hCj85aw88AxtyOZFFHVGswwovtaOnPy/pcfA/dXt2BVLQfuAhYC64D5qvqp\niDwsItfFZlsI7BGRtcB7wH2qaqeauqB/pxbMmhxk/5EQOXmF7D5Y6nYkkwKqPUwtIiNV9Y/1lKda\ndpjaWcu27GXcjKV0aJHOq1MvJLN5I7cjmSRQ48PUFVxQcceriLQSkUfqNJ1JGBd0bU3BhMHs2F/K\n2PxC9h4pczuSSWLxFMwPVfWb88pj56zYAFQpLNijDfnjA3y55whj8wvZf9RKxtROPAXjFZFv1pNF\npDFg680p7uKebZk+LsDG3YcZV7CUg6UhtyOZJBRPwcwBFonIZBGZDPwVeNnZWCYRXNo7kxfHZrNu\n50EmFCzl8PFytyOZJBPPiHZPEL24sW/s8WtVfdLpYCYxXNG3Pc+NzmbVtgNMeqmIo2VWMiZ+cQ3X\noKp/UdWfxh4LnQ5lEsvw/h149paBFG/Zy5SXiykN2d1rTHziGdHuRhHZICIHROSgDfrdMF2TdRZP\njxrAx5v2kPuKlYyJTzxrME8C16lqC1XNsEG/G64RgzrzxMgsPtjwNXfMWU5ZecTtSCbBxVMwu1R1\nneNJTFIYFejCoyP687fPdnP3vOWEwlYy5vTiubNjsYi8RvS6oeMnJqrq646lMgltTLArofIID721\nlntfW8mzNw/E54139FXTkMRTMBnAUeAHFaYpYAXTgE24uDuhsPLo2+vwe4SnRw3E6xG3Y5kEU23B\nqOrE+ghikk/uJT0oC0eYtnA9fq+HJ0Zm4bGSMRVUWzAi8hLRNZaTqOokRxKZpHLn5T0JhSM88+4G\n/D4Pj97QHxErGRMVzybSnyo8TwdGADuciWOS0T1X9KKsPMILi78gzevhwWv7WckYIL5NpJOGahCR\necCHjiUySUdEuG9YH0LhCHkfbMbnER64uq+VjIlrDaayXkC7ug5ikpuIcP9VfQmFlfwPN5Pm83Df\nsD5WMg1cPPtgDnHyPpivgJ87lsgkLRHhwWv7URaObS75PNx7ZW+3YxkXxbOJ1Lw+gpjUICI8cn1/\nQuWxHb9eD3de3tPtWMYlVQ36fVeF5+fVTxyTCjwe4fGRWYwY1IlpC9eT976N495QVXX6ZcXD0LOc\nDmJSi9cjTPtRFldndeTRt9cx86PNbkcyLoh3J6/tqTM15vN6eObmgZSHo5cV+H0exgS7uh3L1KOq\nCqaliIwgupaTISI3VvymXYtk4uH3enhudDa3z17GA2+swe/xMGpwl+p/0KSEqgrm78CJ+xe9D1xb\n4Xt2LZKJW5rPw/Njspk6axk/f301fp8wYlBnt2OZenDagrFrkExdSvd7mX7rBUyaWcRP5q/C5/Fw\n7YCz3I5lHGbX2Jt6k+73kj8+QKBra+59bSXvrPnK7UjGYVYwpl41SfNRMHEwAzq34O55y1m0bpfb\nkYyDrGBMvWvWyMfMSUPo1zGD22cv5++fl7gdyTgknkG/m4jI/xGRvNjrXiJyjfPRTCrLSPfzyqQg\nPds1Y+orxXy08Wu3IxkHxLMG8xLRoTIvjL3eDti9qc0Za9HEz+wpQbq3bcrkl4so3LTH7UimjsVT\nMOfEbrQWAlDVo9iJd6aOtG6axuwpQTq3asLEmUUs27LX7UimDsVTMGWx+1ErgIicQ4XBv405U22b\nNWLulCDtM9KZUFDEqq373Y5k6kg8BfMQ8A7QRUTmAIuAnzkZyjQ87TLSmZsbpFXTNG6dUcia7Qfc\njmTqQDz3pv4f4EZgAjAPCKjqYmdjmYaoY4vGzM0N0jzdz9gZhazbaTcQTXbxHEV6i+gtSxar6p9U\n1Xb3G8d0btWEeblDSfd5GZtfyIZdh9yOZM5APJtITwHfBdaKyB9E5Eciku5wLtOAnd2mCXNzg3g8\nQk5+IZtKDrsdydRSPJtIf1fVO4AewO+AUcBup4OZhq1HZjPm5QaJRJScvEK27DnidiRTC3GdyRs7\nijQSuA0YDLzsZChjAHq2a86c3CDHy8Pk5BWybd9RtyOZGopnH8x8YB3wPeC/iJ4Xc7fTwYwBOLdD\nBrMmBzlUGmJ03hJ2HjjmdiRTA/GswcwgWiq3qep7qhqJd+EiMlxE1ovIRhH5RRXzjRQRFZFAvMs2\nDUf/Ti2YNTnI/iMhcvIK2X2w1O1IJk5VDfr9vdjTpsD1InJjxUd1CxYRL/A88EOgHzBaRPqdYr7m\nwD1AYW1+AdMwDOjSkpmTBrP7YCmj85ZQcsjO9UwGVa3BXBr7eu0pHvFc7DgE2Kiqm1S1DHgVuP4U\n8/0aeAKw/5ZMlS7o2pqCCYPZsb+UsfmF7D1S5nYkU43TFoyqPhh7+rCqTqz4IFoK1ekEbK3welts\n2jdEJBvooqp/rmpBIjJVRIpFpLikxC7tb8iCPdowY3yAL/ccYWx+IfuPWskksnj2wfzxFNP+cKZv\nLCIe4LfAT6qbV1Wnq2pAVQOZmZln+tYmyV3Usy3TxwXYuPsw4wqWcrA05HYkcxpV7YM5V0RGAi0q\n7X+ZAMRzot12oOLw8Z1j005oDvQHFovIl8BQYIHt6DXxuLR3Ji+OzWbdzoOML1jK4ePlbkcyp1DV\nGkwfovtaWnLy/pdsIDeOZRcBvUSku4ikAbcAC058U1UPqGpbVe2mqt2AJcB1qlpcq9/ENDhX9G3P\nc6OzWb3tAJNeKuJomZVMoqnqrgL/Dfy3iFyoqh/XdMGqWh67/exCwAsUqOqnIvIwUKyqC6pegjHV\nG96/A8/eMpB/nbeCyTOLKZgwmMZpXrdjmZh49sHcJiItT7wQkVYiUhDPwlX1bVXtrarnqOqjsWm/\nOlW5qOpltvZiauOarLP47aiBLNm8h6mziikNhd2OZGLiKZgsVf1mBCBV3QcMci6SMTV3w6BOPDEy\niw82fM0dc5ZTVh73+aDGQfEUjEdEWp14ISKtif+e1sbUm1GBLjw24nz+9tlu7pq7nFDYSsZt8RTF\n08DHIvL72OubgEedi2RM7eUEzyYUjvDggk+597WVPHvzQHxeuzuPW6otGFV9RUSKiV7sCHCjqq51\nNpYxtTf+om6EwhEe+fM6/B7h6VED8XpsnHo3xLup0xo4oqoviUimiHRX1c1OBjPmTEz5bg+Ol0eY\ntnA9Pq+HJ0dm4bGSqXfVFoyIPAgEiJ4X8xLgB2YDFzsbzZgzc+flPQmFIzzz7gb8Xg+PjeiPiJVM\nfYpnDWYE0aNGywFUdUfsCmhjEt49V/QiFI7w/HtfkOYVHrruPCuZehRPwZSpqorIifsiNXU4kzF1\nRkT46Q/6UFYeIe+Dzfi9Hh64uq+VTD2Jp2Dmi8jvgJYikgtMAvKcjWVM3RER7r+qL6Gwkv/hZvw+\nDz8b1sdKph7EcxTpKRH5PnCQ6H6YX6nqXx1PZkwdEhEevLYfZeEILy7+gjSvhx9/v7fbsVJeXEeR\nYoVipWKSmojwyPX9CZVHeHbRBtJ8Hu68vKfbsVLaaQtGRD5U1e+IyCFi96WuZA8wTVVfcCydMXXM\n4xEeH5lFeUSZtnA9fq8w9ZJz3I6Vsqq6mvo7sa+nPGIkIm2AfwBWMCapeD3CtB9lURaO8Njbn+H3\neph4cXe3Y6WkuDaRYkNbfofomsyHqrpCVfeIyGVOhjPGKT6vh2duHkh5OMJ/vLUWv9fD2KFd3Y6V\ncuK5L9KviN5orQ3QFpgpIr8EUNWdzsYzxjl+r4fnRmdzxbnt+OWba5hftLX6HzI1Es9VYGOAwar6\nYGwg8KHArc7GMqZ+pPk8PD8mm0t6Z/Lz11fzxoptbkdKKfEUzA5OHoO3ESePrWtMUkv3e5l+6wVc\n2KMNP5m/irdW7XA7Usqo6ijSc0T3uRwAPhWRv8Zefx9YWj/xjKkf6X4v+eMDTCgo4t7XVuL3CsP7\nd3Q7VtKraifvieErlwFvVJi+2LE0xrioSZqPgomDGTejkLvnreDFMR6u7Nfe7VhJTVRPdYpLhRlE\n0oETZyNtVFVX78AYCAS0uNiG7jXOOVga4tb8QtbtPMT0cRdwWZ92bkdKeCKyTFW/dcuhqu6L5BOR\nJ4nekfFl4BVgq4g8KSJ+56Ia466MdD+vTArSs10z/mXWMj7a+LXbkZJWVTt5pxEdaKq7ql6gqtnA\nOUTvk/RUfYQzxi0tmviZPSVI97ZNmfxyEUs27XE7UlKqqmCuAXJV9dCJCap6ELgduMrpYMa4rXXT\nNGZPCdK5VRMmzSxi2Za9bkdKOlUVjOopdtCoaphTX5tkTMpp26wRc6cEaZ+RzoSCIlZu3V/9D5lv\nVFUwa0VkXOWJIjIW+My5SMYklnYZ6czNDdKqaRrjZhSyZvsBtyMljdMeRRKRTsDrwDGih6ohOjZv\nY2CEqrpysp0dRTJu2bbvKDf/bglHysqZlzuUvh0z3I6UMGp8FElVt6tqEHgY+DL2eFhVh7hVLsa4\nqXOrJszLHUq6z8vY/EI27DpU/Q81cNVeKqCqf1PV52KPRfURyphEdXabJsybOhSPR8jJL2RTyWG3\nIyU0u+WdMTXUvW1T5uUGiUSUnLxCtuw54nakhGUFY0wt9GzXnDm5QY6Xh8nJK2Tr3qNuR0pIVjDG\n1NK5HTKYNTnIodIQOflL2LH/mNuREo4VjDFnoH+nFsyaHGT/kRBj8gvZddDVS/USjhWMMWdoQJeW\nzJw0hN0HS8nJW0LJoeNuR0oYVjDG1IELuraiYMJgduwvZWx+IXuPlLkdKSFYwRhTR4I92jBjfIAv\n9xxhbH4h+49ayVjBGFOHLurZlunjAmzcfZhxBUs5WBpyO5KrrGCMqWOX9s7kxbHZrNt5kPEFSzl8\nvNztSK6xgjHGAVf0bc9zo7NZve0AE19aytGyhlkyjhaMiAwXkfUislFEfnGK7/+biKwVkdUiskhE\n7M5XJmUM79+BZ28ZyLIt+5g8s5hjZWG3I9U7xwpGRLzA88APgX7AaBHpV2m2FUBAVbOAPwBPOpXH\nGDdck3UWvx01kCWb9zB1VjGloYZVMk6uwQwhOkj4JlUtA14Frq84g6q+p6onzrFeAnR2MI8xrrhh\nUCeeGJnFBxu+5o45yykrj7gdqd44WTCdgIr34twWm3Y6k4G/nOobIjJVRIpFpLikpKQOIxpTP0YF\nuvDYiPP522e7uWvuckLhhlEyCbGTNzZKXoDoQOPfoqrTVTWgqoHMzMz6DWdMHckJns1/XHce/7N2\nF/e+upLyBlAyVd147UxtB7pUeN2ZU9xyVkSuBB4ALlVVO8fapLTxF3UjFI7wyJ/X4fMKvx01EK9H\n3I7lGCcLpgjoJSLdiRbLLUBOxRlEZBDwO2C4qu52MIsxCWPKd3tQFo7w5Dvr8Xs9PDkyC0+Kloxj\nBaOq5SJyF7AQ8AIFqvqpiDwMFKvqAqKbRM2A34sIwD9V9TqnMhmTKO64rCdl5RGeeXcDfq+Hx0b0\nJ/ZvIKU4uQaDqr4NvF1p2q8qPL/Syfc3JpHdc0UvQuEIz7/3BWle4aHrzku5knG0YIwxpyci/PQH\nfSgrj5D3wWZ8Xg+/vLpvSpWMFYwxLhIR7r+qL6GwMuPDzaT5PPxsWJ+UKRkrGGNcJiI8eG0/QuEI\nLy7+gjSvhx9/v7fbseqEFYwxCUBE+PX1/QmFIzy7aANpPg93Xt7T7VhnzArGmATh8Qi/uTGLUFiZ\ntnA9fq8w9ZJz3I51RqxgjEkgXo8w7UdZlIUjPPb2Z/i9HiZe3N3tWLVmBWNMgvF5PTxz80DKwxH+\n4621+L0exg5NzpFMEuJaJGPMyfxeD8+NzuaKc9vxyzfXML9oa/U/lICsYIxJUGk+Dy+MzeaS3pn8\n/PXVvL58m9uRaswKxpgE1sjnZfqtF3Bhjzb89PereGvVDrcj1YgVjDEJLt3vJX98gEC31tz72kre\nWbPT7Uhxs4IxJgk0SfNRMGEwAzq34K65K3h37S63I8XFCsaYJNGskY+Zk4Zw3lkZ3DFnOYvXJ/4I\nJ1YwxiSRjHQ/r0wK0qt9M6bOWsaHG752O1KVrGCMSTItmviZNTlIj7ZNmfJKEUs27XE70mlZwRiT\nhFo3TWP2lCCdWzVh0swiir/c63akU7KCMSZJtW3WiLlTgrTPSGfCS0Ws3Lrf7UjfYgVjTBJrl5HO\n3NwgrZumMW5GIWu2H3A70kmsYIxJch1bNGZubpDm6X7Gzihk3c6Dbkf6hhWMMSmgc6smzMsdSrrP\ny5j8QjbsOuR2JMAKxpiUcXabJsybOhSfRxidV8gXJYfdjmQFY0wq6d62KXNzg4CSk7eELXuOuJrH\nCsaYFNOzXXNmTwlSVh4hJ6+QrXuPupbFCsaYFHRuhwxmTQ5yqDRETv4Sduw/5koOKxhjUlT/Ti2Y\nNTnI/iMhcvKWsOtgab1nsIIxJoUN6NKSmZOGUHLoODl5Syg5dLxe398KxpgUd0HXVrw0cQg79pcy\nNr+QvUfK6u29rWCMaQCGdG/NjPEBvtxzhLH5hew/Wj8lYwVjTANxUc+2TB8XYOPuw9w6YykHjoUc\nf08rGGMakEt7Z/Li2Gw+++ogE15ayuHj5Y6+nxWMMQ3MFX3b81852Xyy7QATX1rK0TLnSsYKxpgG\naNh5HXj2lkEs27KPyTOLOVYWduR9rGCMaaCuzurIb0cNZMnmPUydVUxpqO5LxgrGmAbshkGdeGJk\nFh9s+JrbZy/jeHndlowVjDEN3KhAFx4bcT7vrS/hrrkrCIUjdbZsX50tyRiTtHKCZxMKR3hwwafc\n9OI/2H34ODv3l3JWy8bcN6wPNwzqVKvlWsEYYwAYf1E3VvxzH2+u/N/b027ff4x/f/0TgFqVjG0i\nGWO+UfTlvm9NOxYKM23h+lotz9GCEZHhIrJeRDaKyC9O8f1GIvJa7PuFItLNyTzGmKqdbliH2g73\n4FjBiIgXeB74IdAPGC0i/TL1Y08AAAY/SURBVCrNNhnYp6o9gf8EnnAqjzGmeme1bFyj6dVxcg1m\nCLBRVTepahnwKnB9pXmuB16OPf8DcIWIiIOZjDFVuG9YHxr7vSdNa+z3ct+wPrVanpMF0wnYWuH1\ntti0U86jquXAAaBN5QWJyFQRKRaR4pKSEofiGmNuGNSJ39x4Pp1aNkaATi0b85sbz0/to0iqOh2Y\nDhAIBNTlOMaktBsGdap1oVTm5BrMdqBLhdedY9NOOY+I+IAWQOLeydsYUyNOFkwR0EtEuotIGnAL\nsKDSPAuA8bHnPwL+pqq2hmJMinBsE0lVy0XkLmAh4AUKVPVTEXkYKFbVBcAMYJaIbAT2Ei0hY0yK\ncHQfjKq+DbxdadqvKjwvBW5yMoMxxj12Jq8xxjFWMMYYx0iy7VMVkRJgS5yztwW+djBOXUiGjGA5\n61IyZISa5eyqqpmVJyZdwdSEiBSrasDtHFVJhoxgOetSMmSEuslpm0jGGMdYwRhjHJPqBTPd7QBx\nSIaMYDnrUjJkhDrImdL7YIwx7kr1NRhjjIusYIwxjkn6gkmWYTnjyPlvIrJWRFaLyCIR6ZqIOSvM\nN1JEVETq/XBrPBlFZFTs8/xURObWd8ZYhur+zM8WkfdEZEXsz/0qFzIWiMhuEVlzmu+LiPzf2O+w\nWkSya/QGqpq0D6IXUX4B9ADSgFVAv0rz3AH8v9jzW4DXEjTn5UCT2PPbEzVnbL7mwPvAEiCQaBmB\nXsAKoFXsdbtE/CyJ7kS9Pfa8H/ClCzkvAbKBNaf5/lXAXwABhgKFNVl+sq/BJMuwnNXmVNX3VPVo\n7OUSouPn1Ld4Pk+AXxMdP7m0PsPFxJMxF3heVfcBqOrues4I8eVUICP2vAWwg3qmqu8THcngdK4H\nXtGoJUBLEekY7/KTvWDqbFhOh8WTs6LJRP/XqG/V5oytIndR1T/XZ7AK4vksewO9ReQjEVkiIsPr\nLd3/iifnQ8BYEdlGdNSBu+snWo3U9O/uSZJiyMyGRETGAgHgUrezVCYiHuC3wASXo1THR3Qz6TKi\na4Lvi8j5qrrf1VTfNhqYqapPi8iFRMdG6q+qdXfvVpcl+xpMsgzLGU9ORORK4AHgOlU9Xk/ZKqou\nZ3OgP7BYRL4kuk2+oJ539MbzWW4DFqhqSFU3A58TLZz6FE/OycB8AFX9GEgneoFhIonr7+5p1fdO\npTreQeUDNgHd+d8daedVmudOTt7JOz9Bcw4iulOwVyJ/npXmX0z97+SN57McDrwce96W6Cp+mwTM\n+RdgQux5X6L7YMSFP/dunH4n79WcvJN3aY2WXd+/jAMfzlVE/4f6AnggNu1homsBEP1f4ffARmAp\n0CNBc74L7AJWxh4LEjFnpXnrvWDi/CyF6KbcWuAT4JZE/CyJHjn6KFY+K4EfuJBxHrATCBFd85sM\n3AbcVuGzfD72O3xS0z9vu1TAGOOYZN8HY4xJYFYwxhjHWMEYYxxjBWOMcYwVjDHGMVYwJm4icjj2\ntZuI5NTxsu+v9Pofdbl84w4rGFMb3YAaFUzsLOqqnFQwqnpRDTOZBGQFY2rjceC7IrJSRH4sIl4R\nmSYiRbExQ/4FQEQuE5EPRGQB0ZPeEJE3RWRZbJyWqbFpjwONY8ubE5t2Ym1JYsteIyKfiMjNFZa9\nWET+ICKficgcF66SN9Vx4wxHeyTnAzgc+3oZ8KcK06cCv4w9bwQUEz1F/jLgCNC9wrytY18bA2uI\nncJ/YtmneK+RwF+Jjq/SHvgn0DG27ANEr43xAB8D33H7M7LHyQ9bgzF14QfAOBFZCRQSHQ7jxMWF\nSzV6weEJ/yoiq4iOedOF6i9C/A4wT1XDqroL+DswuMKyt2n06uOVRDfdTAKx4RpMXRDgblVdeNJE\nkcuIrsFUfH0lcKGqHhWRxUSvFautilech7G/zwnH1mBMbRwiOnTDCQuB20XEDyAivUWk6Sl+rgWw\nL1Yu5xK9OveE0Imfr+QD4ObYfp5MokM8Lq2T38I4zhrf1MZqIBzb1JkJPEt082R5bEdrCXDDKX7u\nHeA2EVkHrCe6mXTCdGC1iCxX1TEVpr8BXEj0imMFfqaqX8UKyiQ4u5raGOMY20QyxjjGCsYY4xgr\nGGOMY6xgjDGOsYIxxjjGCsYY4xgrGGOMY/4/tQ7TOK4oOwUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot\n",
    "figure, axis = pyplot.subplots(1, 1, figsize=(4, 4))\n",
    "\n",
    "axis.set_xlabel(\"Iteration\")\n",
    "axis.set_ylabel(\"Objective Function\")\n",
    "\n",
    "axis.plot(range(n_iterations), objective_function, marker=\"o\")\n",
    "\n",
    "figure.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "collapsed": false,
    "id": "PbzleU137H9m",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Conclusion\n",
    "\n",
    "And that concludes the fourth tutorial! \n",
    "\n",
    "If you have any questions and / or feedback, please open an issue on the\n",
    "[GitHub issue tracker](https://github.com/openforcefield/openff-evaluator/issues)."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "tutorial04.ipynb",
   "provenance": [
    {
     "file_id": "https://github.com/openforcefield/openff-evaluator/blob/master/docs/tutorials/tutorial04.ipynb",
     "timestamp": 1583264626727
    }
   ]
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
