SHARE
TWEET

Untitled

a guest Jan 11th, 2017 63 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. {
  2.  "cells": [
  3.   {
  4.    "cell_type": "markdown",
  5.    "metadata": {},
  6.    "source": [
  7.     "We need to test how pickling behaves in MDAnalysis. This notebook tests how it works with the old anchor system. \n",
  8.     "\n",
  9.     "Full disclosure: I personally dislike this scheme for its use of globals, likelihood for non-deterministic behavior, etc., but here we go."
  10.    ]
  11.   },
  12.   {
  13.    "cell_type": "code",
  14.    "execution_count": 1,
  15.    "metadata": {
  16.     "collapsed": true
  17.    },
  18.    "outputs": [],
  19.    "source": [
  20.     "import MDAnalysis as mda\n",
  21.     "from MDAnalysisTests.datafiles import GRO, XTC"
  22.    ]
  23.   },
  24.   {
  25.    "cell_type": "code",
  26.    "execution_count": 2,
  27.    "metadata": {
  28.     "collapsed": false
  29.    },
  30.    "outputs": [
  31.     {
  32.      "name": "stderr",
  33.      "output_type": "stream",
  34.      "text": [
  35.       "/home/alter/Library/mdanalysis/MDAnalysis/package/MDAnalysis/topology/guessers.py:54: UserWarning: Failed to guess the mass for the following atom types: M\n",
  36.       "  \"\".format(', '.join(misses)))\n"
  37.      ]
  38.     }
  39.    ],
  40.    "source": [
  41.     "u = mda.Universe(GRO, XTC)"
  42.    ]
  43.   },
  44.   {
  45.    "cell_type": "code",
  46.    "execution_count": 3,
  47.    "metadata": {
  48.     "collapsed": false
  49.    },
  50.    "outputs": [],
  51.    "source": [
  52.     "CA = u.atoms.select_atoms('name CA')"
  53.    ]
  54.   },
  55.   {
  56.    "cell_type": "code",
  57.    "execution_count": 4,
  58.    "metadata": {
  59.     "collapsed": false
  60.    },
  61.    "outputs": [
  62.     {
  63.      "data": {
  64.       "text/plain": [
  65.        "<AtomGroup with 214 atoms>"
  66.       ]
  67.      },
  68.      "execution_count": 4,
  69.      "metadata": {},
  70.      "output_type": "execute_result"
  71.     }
  72.    ],
  73.    "source": [
  74.     "CA"
  75.    ]
  76.   },
  77.   {
  78.    "cell_type": "markdown",
  79.    "metadata": {},
  80.    "source": [
  81.     "Alright, so now we'll play with using other processes. Easiest way to do this in a Jupyter notebook is to use `dask.multiprocessing`, since I think the `multiprocessing` module last I checked doesn't play well with the Jupyter kernels this is a quick way to check things that's pretty commonly used by us in production."
  82.    ]
  83.   },
  84.   {
  85.    "cell_type": "markdown",
  86.    "metadata": {},
  87.    "source": [
  88.     "Using `dask.delayed` lets us build dask graphs from function calls, then we can process these using a multiprocessing scheduler and get the results back."
  89.    ]
  90.   },
  91.   {
  92.    "cell_type": "code",
  93.    "execution_count": 6,
  94.    "metadata": {
  95.     "collapsed": false
  96.    },
  97.    "outputs": [],
  98.    "source": [
  99.     "import dask\n",
  100.     "import dask.multiprocessing\n",
  101.     "from dask.delayed import delayed"
  102.    ]
  103.   },
  104.   {
  105.    "cell_type": "code",
  106.    "execution_count": 7,
  107.    "metadata": {
  108.     "collapsed": true
  109.    },
  110.    "outputs": [],
  111.    "source": [
  112.     "def get_names(atomgroup):\n",
  113.     "    return atomgroup.names"
  114.    ]
  115.   },
  116.   {
  117.    "cell_type": "markdown",
  118.    "metadata": {},
  119.    "source": [
  120.     "A normal call in this process:"
  121.    ]
  122.   },
  123.   {
  124.    "cell_type": "code",
  125.    "execution_count": 8,
  126.    "metadata": {
  127.     "collapsed": false
  128.    },
  129.    "outputs": [
  130.     {
  131.      "data": {
  132.       "text/plain": [
  133.        "array(['CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  134.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  135.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  136.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  137.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  138.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  139.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  140.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  141.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  142.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  143.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  144.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  145.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  146.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  147.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  148.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  149.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  150.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  151.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  152.        "       'CA', 'CA', 'CA', 'CA', 'CA'], dtype=object)"
  153.       ]
  154.      },
  155.      "execution_count": 8,
  156.      "metadata": {},
  157.      "output_type": "execute_result"
  158.     }
  159.    ],
  160.    "source": [
  161.     "get_names(CA)"
  162.    ]
  163.   },
  164.   {
  165.    "cell_type": "markdown",
  166.    "metadata": {},
  167.    "source": [
  168.     "Building a `dask` graph for the same..."
  169.    ]
  170.   },
  171.   {
  172.    "cell_type": "code",
  173.    "execution_count": 9,
  174.    "metadata": {
  175.     "collapsed": true
  176.    },
  177.    "outputs": [],
  178.    "source": [
  179.     "names = delayed(get_names)(CA)"
  180.    ]
  181.   },
  182.   {
  183.    "cell_type": "code",
  184.    "execution_count": 10,
  185.    "metadata": {
  186.     "collapsed": false
  187.    },
  188.    "outputs": [
  189.     {
  190.      "data": {
  191.       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJAAAADwCAIAAABg0Hq1AAAABmJLR0QA/wD/AP+gvaeTAAAgAElE\nQVR4nO2deUATV/7A30xuwSRAQECuiFZtORSlFqJFQF3xWlfqUXWr1Gq3rgu1WmU90K3adm2l+qt2\nPWux2grWC6sgCFoEVBQQAwgRSBQQww2BQJKZvN8fESoh4QhJIDifv+DNm/e+k8/Mmzczb+YhEEJA\nYDqg/R0AQe8ghJkYhDATg9w5SSwW//HHH8YPhUANb29vLperngo7cf369f4Ij0Cdo0ePdraj4QhT\nUVdXx2azjRkfwauwWCyN6cQ5zMQghJkYhDATgxBmYhDCTAxCmIlBCDMxCGEmBiHMxCCEmRiEMBOD\nEGZiEMJMDEKYiUEI6zuw9tR7cw8/hwAAoCz/39yFP9cZbmCT8YUpnx77+kzlYBqqhfGz6t29bBAA\nAJDnZMvGjWcibcsUZQlfvT/JiW3G5vJW/ZDRd5NGF9aUuv/72y3GrtWQKJ9lFtp5vUkGAAD8SaaQ\n6zWSpFoCq+NCA/92FF91qahS8Mu80v/MnPNVZh+33bjClOLfP1v9gwA3aqWGRvrwIRw3bggAAMCG\n7BzG+HE01YKW1C9Dj4unf3FwtRfHzMZnQ+Q6l/u7Qw8V9m3ztY3pqKur67yoI015Zz6b5W5nzmA7\nv700Mr0a77C0uSBm0zzP4UNpDKvRMzace9KqFMd+PGZIe2NBm3WyRqm9cKyuIP5/n783wWnW/0qK\nLm+b62ZtZmY/6ZMYIdaWQ1Z88d/zxg1nUsk0tovPih8yG5UQbyxKOrbl/Xdc/L8peH77uw98nNjm\n1u7v7c9owGvufh/i48xkMF38P/+9/M9YO8UJIYRQmnfyo3eGDx3qPG3zhfs/H4gqxTtF2Hp+iRmi\n8TeljN/Fx2Dz1Q+HoVT/g+2rKu7/ewyZ7B7xsLsfFkIImUymxjEdOgtTVkQvsady3z+ZUy2pyjq+\nmEtGAAAI7d0DQhxCrOTH+W7TIq4J6poqM48vHUmhjv70pgRCrODrtym0vxwTd6EKQgghlrc3wJ5N\nRQB5VFDonl8yXzQ38PcFDCUN+/Bas2rrc74YT7eYeYBf3yopil7pSqbx9hUphIfmOFjSEUB2nR/+\n3S/3yhurs7+fMwyleizauGV/nKBeWp2xe4oZyWFtssqLljixR7u83VbFCOqlDSVJe2Y4+H//rLOw\nl7/Di6Pz/vZjlRJCCPHiyFkrLjSpFigyt71FRq1Xxcnas1adCKIhtKAT3Ww7hFD/wpS1UXMZJO76\nVDmEEEJcdOBdGivo2Mu9qTnxH9xJXz5+eTDgRft8qQjzvV/rei4MQgjldz4fRaLyIotVheIlkTwq\ndcp+IQ4hhPK7W9ysp+xT1SGLW8UhO/wzWQ4hxHJ2epIpE/bkv6y99coHFiTOiitS1b9Y3u4JFNr0\nIxVK7XHWVh2faeYefk+1a+DCwzuOaxUmS1g7LSJLASGEsOnCillt0ULZ1RArlDx6c4aiPa80+j06\noEzY04Nt1ypM13MYQmWxGEBSU6sAAACAWllbIZKHDwowAADA8pNvld7bMpaMIAiCIKSRG9LlUJrP\nL8Z6VQfKsbFCETqDrmp4EMYQBgIVCgwCAABl0h5+ZcpnIyrvnd75d/81v9QoFXIFbFsLUKiUtkjN\nzIcggER+2REAVBoFQEyBQe1xljC4Iyzy987wXrDlp7QyucvHO1c5ooqUMJeXGREEIXPXpyoAALgw\nq8jBazQZAACwwuyKUeMdO/ykf8YBAMAwDABAIoE+oHOnw2zm5p2B5Mu7Iq4WNzaL7x2Ouo1Z+wV6\nUgAAQFldWUOefuRFh8NIkbdngtZBdRpBEATRfJIAAAAgf3o1YrbbpH/Go0Ff7lo0rH1LUBTtYq1X\n0Rqnmf/OqG0BQ4svfRUyZeSowPDfSzXsarIL75uj5DHhCT/NN0MQBEEoE3YlHPCnIgjVa3cujtrY\n2qBQ0tDY3pNXNtTVKxHyMPue/wSd0b2XSHNbvfezaYwHW3m2liPmnVQuPhJ/5D1rBAAAECabiWVc\nuPTMcN1BWH89LDA4ir0r9c7piKU+wxk9dNQB7XEi1gE7Ewvy4/d/zLOqurl30aLIx+i7B0TtHR6I\nCb8LXPxrkyx9U8D6P1SnBdmNdbP25GIQQnnWNjcSeYyPtwUUC57UtxnDRUUinPzmOxP7suG6C5Om\nbg9J8otKfPhCKmsW5yUc/HBc2/Uixe1dX0tJwoZZK/7vhqBWhsnqRWk/hq6PKlUCBAEAKJU9un6E\nMpkcQhzDX+ZWKiEAmEIBAACKtLMxIouZK//m1N7kKFUvTqkaTUzRfkwolUqI422FqEpRKLCu4qyL\n+XJ/AWY2YkbY4ZS85I0e2MP0rGYNESrF2XlWXh4UAADARVlCx/Guf7Z35oEhS1ywu9cSa1RVY/k3\nksvpvh8sHt2TjdeGzsIUGdHRj66HcqltrTpKMnOa+vnV50oAADPo31v82C35Z8Kmj7aiU+gWIwL+\niwfNc0BRFpuNYiU5Oc8Lz/5w4Zmyqxrk5el3ijFMkJb6XA4BUIjv3C3E8JK76WUyCFCODQepvnk2\ntkRSm3/5wMl0iVJaWZ57OSa5MDWtAMOFGellMggA3pCTklmrbOKnP6zHAVBKRal3hTj2JP12aSvU\nHqc8Y/eSj4/eFja0Njc0yiDDa/IEM00h5mTLx40fCgAAoPlhDqX9EgwAAMAQv22Rf7eI+8/mc4JG\nybPEXWEHn078977Vrn279u3cD+lht74hNeIdtnrdqMPaJFU3VlmfefQf/iMt6VQzO8+/bosVqpLx\nitgwn+EWwyZ+dOaJvIvi8bJDAdS2Uqk+/72+l/fnv/4HS3H8edwmf2cW08599uZzjwujgu2ZDn4b\nonYH0duykT13Zpxf3n6XCDFbEn1vp2f7aZQ+N6pOW5wNvx0+ce/KF++Nt2WyHCYs3J1QqtAUpCLn\nP9M+jlNdIMjTN8/acq9Tttbi85uD3rQ2M7MeO2Pdsfu1PekdQwgNcB2mEEV/GhpV0hahEmupf54X\nG+b7fnRTT2Mi6AJtwnrXb2sHy/tueXjT1sfctvUREp1lN8Z9/PymyQzdiiToEToKkxRkFzxL3L1p\nEnl1kPco26GklirB3auxfLulofbEExtDouOva7Hg6O2YUFf+/hU8Vw6TbTt2xr9OPR+3JizIqadX\nhXjubq/2DksnGAtjBtUdff2h4xEGEPMxC7ZHLdiuc8Ukt21Z8m06r/7aQrRfJgYhzMQghJkYhDAT\ngxBmYhDCTAxCmIlBCDMxCGEmBiHMxCCEmRiEMBNj0ApTKrscf2CyaL1bf+nSpSFDhhgzFD1SXl4u\nFou9vLz6OxDdUagGG3VCq7CQkBCDBUOgOwgcdF/VhhC6uLhUVVVVV1ebbiOhjUF4DsvIyHj27FlL\nS8uVK1f6Oxb9MwiF/fLLL1QqlUwmnz59ur9j0T+DrUnEcXzYsGE1NTUAADKZLBaLLS0t+zsofTLY\njrDk5GSVLQAAhPDChQv9G4/eGWzCVO2h6m8I4alTp/o3Hr0zqJpEmUzG4XCampraUxAEKS0tHT58\neD9GpV8G1RF29erV5uYOb5mQSKRz5871VzyGYFAJO3PmDJnc4VYAjuNRUVH9FY8hGDxNokQi4XA4\ncrm88yKBQDBq1Cjjh2QIBs8RduHCBdU7xGpQKJSzZ88aPx4DMXiE/fzzzxpfiVYoFD/99JPRwzEU\ng6RJrKqqsrOzw3GtL1VnZ2ePGzfOmCEZCF1fhhhgJCcnDx8+vF1Yc3OzUqkcOlT1MitAUTQ5OXlw\nCBskR5gaa9asEYlECQkJ/R2I/hk857DXBEKYiUEIMzEIYSYGIczEIISZGIQwE4MQZmIQwkwMQpiJ\nQQgzMQhhJgYhzMQghJkYhDATgxBmYhDCTAxCmIlBCDMxCGEmBiHMxCCEmRiEMBODEGZiEMJMDEKY\niUEIMzEIYSYGIczEIISZGIQwE2MwvNBXXV1d00ZrayuEsKCgoLq6+uzZsyQSiUqlWrXB4XBQ1LT3\nURN7oa+5uTkvLy8nJ4fP5xcVFYlEIqFQ2Nra+moeBEFYLBaEsKmpSe0lWiqV6uzs7OLiMmLECE9P\nT3d3d3d3dxaLZdyN6BMmIKyysjIpKSk9PT0tLe3Ro0c4jpubm7/11lujR4/mcrkuLi4ODg42Njaq\nY4hOp6utLpfLa2pqVEdhaWmpSCQSiURPnjzh8/n19fUIgrz55ps8Hs/X1zcwMNDBwaE/NrEXDFxh\nmZmZly9fjouLy8rKIpFIXl5ePj4+vr6+Xl5eXC5XLy3b06dPc3JyVLvCgwcPWltb3d3dZ86cOW/e\nPF9f3wHaeBpjxtTekJeXt2XLlpEjRwIAXFxcPvnkk8uXLzc1GXzq2paWluvXr3/66aejR48GADg4\nOKxfv/7+/fuGrre3DBRhUqn0p59+4vF4AABnZ+eNGzdmZGT0VzC5ubnbt29XmRs3btyhQ4fq6+v7\nKxg1+l+YWCzesWMHh8OhUqkLFy6MjY3FMKy/g3rJgwcP1qxZY25ubm5uvmbNmsLCwv6OqF+FlZaW\nfvLJJzQazdraeseOHWKxuB+D6YL6+vpvv/3W0dGRTCYvW7asf7X1jzCxWBwaGkqn052dnQ8fPtzS\n0tIvYfQKuVx++vTpsWPHksnkkJAQkUjUL2EYW5hMJtu7dy+TybS3tz948KBMJjNyAH0Ew7BTp06N\nHDmSwWBs3bpVIpEYOQCjCktISFBt6vbt243Q8TMccrl83759bDbb3t4+OjramFUbSVhNTc2KFSsA\nAAsWLHj69KlxKjU0lZWVq1atQhBk7ty5paWlxqnUGMKuXr06bNgwe3v7ixcvGqE6I3Pz5s1Ro0Yx\nmcxTp04ZoTrDCpNKpevWrUMQZPny5QPnUkbvSKXSsLAwBEHef/99Q2+mAYU9ffp04sSJTCbz559/\nNlwtA4fExER7e3snJyeDXvIbSti1a9csLCzGjx9fUlJioCoGIGKxeOrUqQwGw3DNo0GE7d+/H0XR\nZcuWNTc3G6L8gQyGYZs3b0YQJDQ0FMdxvZevZ2EYhv3rX/9CEGTHjh36Ldm0iI6OptPpwcHBUqlU\nvyXrU5hMJgsODmYwGOfOndNjsSZKcnKyhYWFn59fY2OjHovVm7CWlpZZs2YxmcyUlBR9lWnq8Pl8\nW1vbt99+u6amRl9l6kdYS0vLtGnTLC0t+/GZyMBEIBA4OTl5enrqy5kehMlkslmzZllaWmZnZ/e9\ntMGHUCh0cnLy9vZuaGjoe2l9FYZhWHBwMJPJJI6tLhAIBHZ2dlOmTOl7H6SvwtatW8dgMIjzVrfk\n5uZaWloGBwf3sa/fJ2GRkZEIgpw5c6Yvhbw+pKSk0Gi0jRs39qUQ3YVdu3YNRdG9e/f2pfrXjdOn\nTyMIcuLECZ1L0FGYUCi0srJatmyZzhW/toSHh9PpdJ1P+bqMS2xpafHx8UFRNC0tjcFg6Hnc3WAH\nx/GgoCCBQJCZmWllZdXr9XWQvG7dOjab/Vrd1dUvVVVVjo6OCxYs0GHdXguLj48nOhp9JyUlhUQi\nHTt2rLcr9k5YdXX1sGHDli9f3ttqCDqzefNmMzOz4uLiXq3Vu3PYypUrExMT8/PzTeuNj4GJXC6f\nMGGCra1tQkKCxrkFNdNzt0lJSQiCXLhwoXc7EoF27t27RyKRTp482fNVeipMJpONHDkyODhYl7gI\ntBMaGsrhcGpra3uYv6fC9u7dy2Aw+mu46yCmsbHR1tY2LCysh/l7JEwsFrNYrIiIiD4ERqCVI0eO\nUCiUx48f9yRzj4SFhoba29ub9FjdgQyGYR4eHj083XQvrLy8nMFgHDp0qM+BEWjl0qVLCIJkZWV1\nm7P7bv0nn3wSFxdXWFhIo9H62pMl0AKE8O2337azs4uNje0+axe8ePGCRqMdPnxYL/sRQRdcuXIF\nQRA+n991tm6Ebdu2zdraWu9jtQg6o1Qq33rrrZUrV3adrSthUqmUw+G85iMMjcnx48epVGp5eXkX\nebr6skFMTExjY+PatWv12VoTaGf58uUsFuvHH3/sIk9Xwo4dOzZ//nwbGxt9B0agGRqN9ve///3E\niRNKpVJrJm2HXl5eHgAgMTHRAIc+gVby8/MBANevX9eWQesRdubMGWdn54CAAAPsSQRaGTt2rI+P\nz5kzZ7Rl0CosJiZm0aJFA/TzPYOaxYsXX7p0Se2LZ+1o9vHgwYOioqJFixYZMjACzSxatKipqen6\n9esal2oWFhsb6+LiMmHCBEMGRqAZOzs7Hx+fy5cva1yqWdi1a9eCgoJ68RiUQK8EBQXFx8dDTXcN\nNQirqqrKzs4OCgoyfGAEmpk5c2ZFRcWjR486L9IgLCkpiUQi+fv7Gz4wAs14eXnZ2NjcuHGj8yIN\nwlJTU728vMzNzQ0fGIFmEATh8XipqamdF2kQlpaW5uvra/ioCLpCJazzaUxdWHNzM5/P9/HxMVZg\nBJrh8XjV1dXFxcVq6erC+Hw+juNeXl7GCoxAM56eniQSKScnRy1dXdijR4/MzMy4XK6xAiPQDIPB\ncHV15fP5aunqwnJzc93d3Yk7UgMBDw+Pzj17dTFPnjxRfZyYoN954403ioqK1BLVhYlEIhcXFyNF\nRNAlLi4uIpFILbGDMAjhs2fPCGEDBC6XK5FIqqurX03sIKy2tlYqlTo6Oho3MI0onx77+kzlAJ21\nwjg4OTkBAMrKyl5N7CBMJdPa2tqYYWmmKXX/97db+juK/oXD4QAAampqXk3UIEyVrz9Rin//bPUP\nArz7nIMaNptNIpG6ahLr6uoAABYWFnqttzn/lw2zPeyHDrFwmbTsuzs1HceXSAvPbf7rOAcmfQhn\nzF82/lYkg5VX1k5dfFwgl11fPQxF6LN/qu2iYcTrC68f3rRwovPsw8Li2O3z3G3MzYe/s/acqN22\nvOTSlr+Od2DRKHQLru/K/2VJIFBKipOPb13qww34trAidf8KX2eLoTYeCw/cb1TW3jv4oa8LawiL\nG7Dp6vM/Y+0UJwAAgJb8n1b7ODCZLtPDLz44/X+nyrQPnuk9KIpaWFjU1tZ2SH11gEdMTAyCIEql\nUn+DSpQV0Uvsqdz3T+ZUS6qyji/mkhEAAEJ794AQhxAr+XG+27SIa4K6psrM40tHUqijP70pgRAr\n+PptCu0vx8TdRYLl7Q2wZ1MRQB4VFLrnl8wXzQ38fQFDScM+vKb6tKYi54vxdIuZB/j1rZKi6JWu\nZBpvX5FCeGiOgyUdAWTX+eHf/XKvvLE6+/s5w1Cqx6KNW/bHCeql1Rm7p5iRHNYmt6qq0Rwn9miX\nt9uqGEG9tKEkac8MB//vn+n5i5aOjo6RkZGvpnQQdvr0aSqVqs8KlbVRcxkk7vpUOYQQQlx04F0a\nK+hYqWq7mhP/wZ305eOXM63gRft8qQjzvV/rei4MQgjldz4fRaLyIotVheIlkTwqdcp+IQ4hhPK7\nW9ysp+xT1SGLW8UhO/wzWQ4hxHJ2epIpE/bkv6y99coHFiTOiisvxzhjebsnUGjTj1QotcdZW3V8\nppl7+D3VroELD+84rm9hrq6uX3311aspHaZUlMlkVCpVjwc1QKgsFgNIamoVAFAAQK2srRDJ3QcF\n2EcOVIDlJ98qvVcwlrzllTXI+fxibGFvHu2gHBsrFKEz6Krn4whjCAOBTQoMAgAAZdIefiUA8op7\np48c/N+P52uUTLkCtq0FpFRKW6Rm5kMQQCKTXv5PpVEAxBQYBADXEmcJI3CERf7eGd6Faz/fsHYJ\n7+Odq3T8mbRCo9FkMlmHrdV3FWqYzdy8M5B8eVfE1eLGZvG9w1G3MWu/QE8KAAAoqytryNOPvOhw\nGCny9kzo3cScCNLlWAb506sRs90m/TMeDfpy16Jh7RuMomgPR0BojdPMf2fUtoChxZe+CpkyclRg\n+O+lWK8i7x4EUX+9qIMwKpWqUCj0WyXNbfXez6YxHmzl2VqOmHdSufhI/JH3rBEAAECYbCaWceHS\nM8N1B2H99bDA4Cj2rtQ7pyOW+gxn6DJKRXuciHXAzsSC/Pj9H/Osqm7uXbQo8rEegn4FmUym9paX\nujC5XK7fKqWp20OS/KISH76QyprFeQkHPxzHfPmrUdze9bWUJGyYteL/bghqZZisXpT2Y+j6qFIl\nQBAAgFLZo+tmKJPJIcQx/GVupRICgKl2PEXa2RiRxcyVf3Nqa/qAUrXHQoUCgwBTtB8TSqUS4nhb\nIapSFAqsqzjrYr7cX4CZjZgRdjglL3mjB/YwPavPP1gHuhHGYDAghGqNZt9QZERHP7oeyqUiL0FJ\nZk5TP1d1mJlB/97ix27JPxM2fbQVnUK3GBHwXzxongOKsthsFCvJyXleePaHC8+67CvLy9PvFGOY\nIC31uRwCoBDfuVuI4SV308tkEKAcGw5SffNsbImkNv/ygZPpEqW0sjz3ckxyYWpaAYYLM9LLZBAA\nvCEnJbNW2cRPf1iPA6CUilLvCnHsSfrt0laoPU55xu4lHx+9LWxobW5olEGG12Q9DwyUyWTq07C+\n2i7fvn0bAPD8+XP9dXOUDakR77DVz5Sow9ok1TxUyvrMo//wH2lJp5rZef51W6xQlYxXxIb5DLcY\nNvGjM0/kXRSPlx0KaO8lUX3+e30v789//Q+W4vjzuE3+ziymnfvszeceF0YF2zMd/DZE7Q5q/xXI\nnjszzi9vO+oBYrYk+t5Oz/bTKH1uVJ22OBt+O3zi3pUv3htvy2Q5TFi4O6FUob9fDkKlUkkmk3/9\n9ddXEzsIU43Ef/Tokf4qVYiiPw2NKmnbECXWUv88LzbM9/1o4g337lBdMickJLya2KFDpvoanNq9\nkL6A5X23PLxp62NuWzUIic6yG+M+fn7TZOKzfd2huotoaWn5amIHYRwOh0ajlZeX66tKSUF2wbPE\n3ZsmkVcHeY+yHUpqqRLcvRrLt1saak880+4O1X16tbnAO/xsKIo6OjoKhUJ9VWmx4OjtmFBX/v4V\nPFcOk207dsa/Tj0ftyYsyInU/coAAADw3N1e7R2WTjAWxgziO/pCoZDBYKi9UKl+jcrlcp8+faq3\nOhHzMQu2Ry3YrnMBJLdtWfJteovHpFA9/Ve7K6DeMLm6ugoEAiNGRaCVJ0+euLq6qiWqC3N3d1f1\nEo0VFYFWcnJyPD091RLVhXl4eDQ0NJSWlhorKgLNyGSyJ0+euLu7q6VrOMJQFM3OzjZWYASaycvL\nUygU3QtjsVhjx469c+eOsQIj0ExaWhqbzR4zZoxauoarIR6Pl5aWZpSoCLSSlpbG4/E6D8HWIMzX\n1/fBgwfa3mInMA7aXvrSICwwMFAmk6luBBP0C3l5eWVlZdOmTeu8SIMwBweHt956Ky4uzvCBEWgm\nLi6Ow+FMnDix8yLNd/SCgoKuXbtm4KgItBIfHz9jxgyN7xBpFjZv3rzCwkLV56YIjEx1dfUff/wx\nd+5cjUs1C+PxeE5OTtHR0YYMjEAz58+fp1Kpc+bM0bhUszAEQYKDg2NiYgwZGIFmzp07N3v2bK1f\ncdD2uPP+/fsAgLS0NIM/WCV4BZFIhKLoxYsXtWXQ+hhx4sSJ48ePP378uIH2IwKNnDhxwtraevbs\n2doydPXc96OPPoqOjm5oaDBAYAQawHH85MmTISEhFApFW56uhC1btoxEIhEHmdE4f/58RUXF6tWr\nu8jTzUQD69ev/+2330pKSrpwTqAv3nnnHScnp276el2fA4VCIZlMPn36tJ7PrQSduHnzJgDgzp07\nXWfrfiqP5cuXZ2Zm5ubmkkg9HTlDoAMBAQHt2rqiW/MCgYBMJp86dUov+xGBRpKSkgAAt27d6jZn\nj+bADAkJuX37dn5+vp7fHiMAAAAAIZw8ebKZmVlCQkKPcneLUCik0+n79u3r435EoJGzZ8+iKNrD\nudN7OqXi1q1bmUxmRUVFHwIj0IBUKnV2dg4JCelh/p4Kk0gk9vb2q1at0jUwAs1ERESYm5v3/I2h\nXkwLHB0djSDIzZs3dYmLQBN5eXk0Gk3tOwFd07uJt+fPn5+fn5+Tk8NgEC+f9BWlUunn5yeTye7c\nudOLS6Ze7RGlpaUsFuvTTz/t3Y5EoIlvvvmGQqE8fPiwV2v1ThiE8NSpUwiC/P77771dkeBVsrKy\nqFTq119/3dsVe9ckqli6dOmtW7cePnxITC2mG01NTd7e3nZ2djdu3Oj1x1912Dvq6upGjBgxdepU\nhUKv7/S+NixevNja2rq0tFSHdXURBiHMyckZMmTIhg0bdFv9dWbfvn0oiqqmVtEBHYXBtpNZVFSU\nziW8hsTHx5PJZLXPR/UK3YVBCDdt2kShUG7cuNGXQl4fcnNz2Wz2kiVL+vK9vD4Jw3F80aJFFhYW\n3U4XTfDs2TNHR8epU6fKZLK+lKNLL/FVWltbZ86cWVhYmJKSMmrUqL4UNYgRi8V+fn4UCiUlJaWP\n3w/tqzAAgEQimTZt2osXL/744w/ii9ydqa2t9ff3b2lpSUlJsbW17WNpevhaxtChQ+Pi4iwsLPz8\n/Dp/GP81RywW+/v7NzQ03Lhxo++2gL6+l2hpaXnr1i17e/vJkyd3ni7ktaWioiIwMFAikSQnJ6u+\naa4H9HNKhRBC2NTUFBAQYGFhQdzRhxDy+XwnJ6exY8eWl5frsVh9CoMQtra2LlmyhEqlvuZjQBIT\nE1ksFo/Hq6qq0m/JehYGIcRxfMOGDQiChIeHYxim9/IHPpGRkWQyefny5X3swWtE/8JUnDhxgk6n\nT58+Xe+72EBGIpEsXryYRCJ99dVXev2a/J8YShiE8MGDB87Ozo6OjsnJyYarZeCQlZU1ZswYa2tr\ng976MaAwCGF1dfWCBQtQFA0PDzdE+zBAwHH8m2++oVKp/v7+ut2D7zmGFaYiKirK3Nzczc2t23HI\npohAIPD39yeTyTt27DDCOdsYwiCExcXF06ZNQ1E0NDS0vr7eOJUaGqlUGhERQaVSvb29e/ukX2eM\nJAxCqFQqT548yeFwrK2tjxw5YuodyOjoaGdnZ3Nz88jISGNui/GEqaitrZyappoAAAJUSURBVA0L\nC6NQKO7u7pcuXTJQV8qgJCcn+/r6oii6cuVKvX6BvEcYW5iKx48fL1iwAEGQiRMnXrlyxVS03bx5\n09/fHwAwffr0+/fv90sM/SNMRVZW1ty5cxEEefPNN48fP97a2tqPwXQBhmHR0dHe3t4AgKlTp6ak\npPRjMP0pTIVAIAgNDaXT6Ww2e82aNTk5Of0d0Z+UlZV9/fXXLi4uKIrOmTNnIHxTof+FqXj+/Pmu\nXbtUj9N8fX33799fVlbWX8FUVVUdOXIkICAARVFbW9vw8PDi4uL+CkaNgSJMBY7jCQkJH3zwAYvF\nQlGUx+Pt2bMnMzPTOCe53Nzcb7/9NjAwkEwmDxkyZOHChRcvXpTLu5pKxPjo4YmzIZDJZPHx8Zcv\nX46Pj6+oqLCxseHxeJMnT/b19fX09NTXyH65XJ6bm5uenp6Wlnb79u3y8nIOhzN9+vR58+bNnTvX\nzMxML7XolwEqrB0IYU5OTlJSUmpqanp6emVlJYlEcnV19fDweOONN7hcrouLi6OjI4fDUc1Doo2G\nhobKysry8nKRSCQUCgUCAZ/PFwgECoWCzWb7+vryeLzAwEBvb+9eD8U1LgNdmBpFRUUPHz7k8/m5\nubkCgUAkEjU1NakWqeZkZbFYAAA6nY6iqFQqBQBIJJK6ujoMezlRGJ1O53K5I0eOdHNz8/Dw8PDw\nGDNmzACX9ComJqwzVVVV5eXlNTU1VVVVjY2NqpmNW1palEqlqk1jsVhMJtPKysra2tre3l4vAyv6\nEZMX9rphMk0BgQpCmIlBCDMx/h8QQ5ua4DDU4AAAAABJRU5ErkJggg==\n",
  192.       "text/plain": [
  193.        "<IPython.core.display.Image object>"
  194.       ]
  195.      },
  196.      "execution_count": 10,
  197.      "metadata": {},
  198.      "output_type": "execute_result"
  199.     }
  200.    ],
  201.    "source": [
  202.     "names.visualize()"
  203.    ]
  204.   },
  205.   {
  206.    "cell_type": "markdown",
  207.    "metadata": {},
  208.    "source": [
  209.     "...and executing with a child process:"
  210.    ]
  211.   },
  212.   {
  213.    "cell_type": "code",
  214.    "execution_count": 11,
  215.    "metadata": {
  216.     "collapsed": false
  217.    },
  218.    "outputs": [
  219.     {
  220.      "data": {
  221.       "text/plain": [
  222.        "array(['CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  223.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  224.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  225.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  226.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  227.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  228.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  229.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  230.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  231.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  232.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  233.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  234.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  235.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  236.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  237.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  238.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  239.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  240.        "       'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA', 'CA',\n",
  241.        "       'CA', 'CA', 'CA', 'CA', 'CA'], dtype=object)"
  242.       ]
  243.      },
  244.      "execution_count": 11,
  245.      "metadata": {},
  246.      "output_type": "execute_result"
  247.     }
  248.    ],
  249.    "source": [
  250.     "names.compute(get=dask.multiprocessing.get)"
  251.    ]
  252.   },
  253.   {
  254.    "cell_type": "markdown",
  255.    "metadata": {},
  256.    "source": [
  257.     "Okay, so doing something basic works fine. Good."
  258.    ]
  259.   },
  260.   {
  261.    "cell_type": "markdown",
  262.    "metadata": {},
  263.    "source": [
  264.     "## What if we have two universes that are built from the same files, but they have different topology attributes?"
  265.    ]
  266.   },
  267.   {
  268.    "cell_type": "markdown",
  269.    "metadata": {},
  270.    "source": [
  271.     "We'll make two universes, and an `AtomGroup` for each. The universes are generated from the same files:"
  272.    ]
  273.   },
  274.   {
  275.    "cell_type": "code",
  276.    "execution_count": 12,
  277.    "metadata": {
  278.     "collapsed": true
  279.    },
  280.    "outputs": [],
  281.    "source": [
  282.     "u1 = mda.Universe(GRO, XTC)\n",
  283.     "u2 = mda.Universe(GRO, XTC)"
  284.    ]
  285.   },
  286.   {
  287.    "cell_type": "code",
  288.    "execution_count": 13,
  289.    "metadata": {
  290.     "collapsed": true
  291.    },
  292.    "outputs": [],
  293.    "source": [
  294.     "CA1 = u1.select_atoms('name CA')\n",
  295.     "CA2 = u2.select_atoms('name CA')"
  296.    ]
  297.   },
  298.   {
  299.    "cell_type": "markdown",
  300.    "metadata": {},
  301.    "source": [
  302.     "Let's change the atom names of each universe to new values:"
  303.    ]
  304.   },
  305.   {
  306.    "cell_type": "code",
  307.    "execution_count": 14,
  308.    "metadata": {
  309.     "collapsed": true
  310.    },
  311.    "outputs": [],
  312.    "source": [
  313.     "u1.atoms.names = 'lol'"
  314.    ]
  315.   },
  316.   {
  317.    "cell_type": "code",
  318.    "execution_count": 15,
  319.    "metadata": {
  320.     "collapsed": true
  321.    },
  322.    "outputs": [],
  323.    "source": [
  324.     "u2.atoms.names = 'cats'"
  325.    ]
  326.   },
  327.   {
  328.    "cell_type": "markdown",
  329.    "metadata": {},
  330.    "source": [
  331.     "Now, if we get names for each, what do we get?"
  332.    ]
  333.   },
  334.   {
  335.    "cell_type": "code",
  336.    "execution_count": 16,
  337.    "metadata": {
  338.     "collapsed": false
  339.    },
  340.    "outputs": [
  341.     {
  342.      "data": {
  343.       "text/plain": [
  344.        "array(['lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  345.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  346.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  347.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  348.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  349.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  350.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  351.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  352.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  353.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  354.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  355.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  356.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  357.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  358.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  359.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  360.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  361.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  362.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  363.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  364.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  365.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  366.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol',\n",
  367.        "       'lol', 'lol', 'lol', 'lol', 'lol', 'lol', 'lol'], dtype=object)"
  368.       ]
  369.      },
  370.      "execution_count": 16,
  371.      "metadata": {},
  372.      "output_type": "execute_result"
  373.     }
  374.    ],
  375.    "source": [
  376.     "get_names(CA1)"
  377.    ]
  378.   },
  379.   {
  380.    "cell_type": "code",
  381.    "execution_count": 17,
  382.    "metadata": {
  383.     "collapsed": false
  384.    },
  385.    "outputs": [
  386.     {
  387.      "data": {
  388.       "text/plain": [
  389.        "array(['cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  390.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  391.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  392.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  393.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  394.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  395.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  396.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  397.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  398.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  399.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  400.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  401.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  402.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  403.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  404.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  405.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  406.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  407.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  408.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  409.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  410.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  411.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  412.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  413.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  414.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  415.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats'], dtype=object)"
  416.       ]
  417.      },
  418.      "execution_count": 17,
  419.      "metadata": {},
  420.      "output_type": "execute_result"
  421.     }
  422.    ],
  423.    "source": [
  424.     "get_names(CA2)"
  425.    ]
  426.   },
  427.   {
  428.    "cell_type": "markdown",
  429.    "metadata": {},
  430.    "source": [
  431.     "Now what if we do multiprocessing, which requires serializing an `AtomGroup` and the function name, deserializing them in a new process, and executing the function on the `AtomGroup`?"
  432.    ]
  433.   },
  434.   {
  435.    "cell_type": "code",
  436.    "execution_count": 18,
  437.    "metadata": {
  438.     "collapsed": false
  439.    },
  440.    "outputs": [
  441.     {
  442.      "data": {
  443.       "text/plain": [
  444.        "array(['cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  445.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  446.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  447.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  448.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  449.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  450.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  451.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  452.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  453.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  454.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  455.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  456.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  457.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  458.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  459.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  460.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  461.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  462.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  463.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  464.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  465.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  466.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  467.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  468.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  469.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  470.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats'], dtype=object)"
  471.       ]
  472.      },
  473.      "execution_count": 18,
  474.      "metadata": {},
  475.      "output_type": "execute_result"
  476.     }
  477.    ],
  478.    "source": [
  479.     "delayed(get_names)(CA1).compute(get=dask.multiprocessing.get)"
  480.    ]
  481.   },
  482.   {
  483.    "cell_type": "markdown",
  484.    "metadata": {},
  485.    "source": [
  486.     "What's happening here? It looks like getting the names for the first `AtomGroup` yields names you'd expect only to see for the second, since it comes from a different universe. What if we do the same for the second `AtomGroup`?"
  487.    ]
  488.   },
  489.   {
  490.    "cell_type": "code",
  491.    "execution_count": 19,
  492.    "metadata": {
  493.     "collapsed": false
  494.    },
  495.    "outputs": [
  496.     {
  497.      "data": {
  498.       "text/plain": [
  499.        "array(['cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  500.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  501.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  502.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  503.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  504.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  505.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  506.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  507.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  508.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  509.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  510.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  511.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  512.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  513.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  514.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  515.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  516.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  517.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  518.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  519.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  520.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  521.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  522.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  523.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  524.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats', 'cats',\n",
  525.        "       'cats', 'cats', 'cats', 'cats', 'cats', 'cats'], dtype=object)"
  526.       ]
  527.      },
  528.      "execution_count": 19,
  529.      "metadata": {},
  530.      "output_type": "execute_result"
  531.     }
  532.    ],
  533.    "source": [
  534.     "delayed(get_names)(CA2).compute(get=dask.multiprocessing.get)"
  535.    ]
  536.   },
  537.   {
  538.    "cell_type": "markdown",
  539.    "metadata": {},
  540.    "source": [
  541.     "This is because the universes for both these `AtomGroup` were defined with the same topology/trajectory, so the only one that got held on to with a global weakref was the second one we defined, and this is the one that got used by both `AtomGroup`s upon being deserialized:"
  542.    ]
  543.   },
  544.   {
  545.    "cell_type": "code",
  546.    "execution_count": 21,
  547.    "metadata": {
  548.     "collapsed": false
  549.    },
  550.    "outputs": [
  551.     {
  552.      "data": {
  553.       "text/plain": [
  554.        "(47681,\n",
  555.        " '/home/alter/Library/mdanalysis/MDAnalysis/testsuite/MDAnalysisTests/data/adk_oplsaa.gro',\n",
  556.        " '/home/alter/Library/mdanalysis/MDAnalysis/testsuite/MDAnalysisTests/data/adk_oplsaa.xtc')"
  557.       ]
  558.      },
  559.      "execution_count": 21,
  560.      "metadata": {},
  561.      "output_type": "execute_result"
  562.     }
  563.    ],
  564.    "source": [
  565.     "u1.anchor_name"
  566.    ]
  567.   },
  568.   {
  569.    "cell_type": "code",
  570.    "execution_count": 22,
  571.    "metadata": {
  572.     "collapsed": false
  573.    },
  574.    "outputs": [
  575.     {
  576.      "data": {
  577.       "text/plain": [
  578.        "(47681,\n",
  579.        " '/home/alter/Library/mdanalysis/MDAnalysis/testsuite/MDAnalysisTests/data/adk_oplsaa.gro',\n",
  580.        " '/home/alter/Library/mdanalysis/MDAnalysis/testsuite/MDAnalysisTests/data/adk_oplsaa.xtc')"
  581.       ]
  582.      },
  583.      "execution_count": 22,
  584.      "metadata": {},
  585.      "output_type": "execute_result"
  586.     }
  587.    ],
  588.    "source": [
  589.     "u2.anchor_name"
  590.    ]
  591.   },
  592.   {
  593.    "cell_type": "markdown",
  594.    "metadata": {},
  595.    "source": [
  596.     "As an aside: no `Universe` objects have to be serialized here, because the processes spawned by `dask.multiprocessing` have access to the memory space of the parent, and therefore to the global store of weakrefs."
  597.    ]
  598.   },
  599.   {
  600.    "cell_type": "markdown",
  601.    "metadata": {},
  602.    "source": [
  603.     "### Does it work with the distributed scheduler?"
  604.    ]
  605.   },
  606.   {
  607.    "cell_type": "markdown",
  608.    "metadata": {},
  609.    "source": [
  610.     "Given how this scheme works, I don't think this works at all with the distributed scheduler, which can't rely on shared memory between parent and child processes (there is no parent-child relationship; everything must be serialized). I want to test this, though:"
  611.    ]
  612.   },
  613.   {
  614.    "cell_type": "code",
  615.    "execution_count": 20,
  616.    "metadata": {
  617.     "collapsed": true
  618.    },
  619.    "outputs": [],
  620.    "source": [
  621.     "from dask.distributed import Client"
  622.    ]
  623.   },
  624.   {
  625.    "cell_type": "code",
  626.    "execution_count": 21,
  627.    "metadata": {
  628.     "collapsed": true
  629.    },
  630.    "outputs": [],
  631.    "source": [
  632.     "cl = Client('127.0.0.1:8786')"
  633.    ]
  634.   },
  635.   {
  636.    "cell_type": "code",
  637.    "execution_count": 22,
  638.    "metadata": {
  639.     "collapsed": false
  640.    },
  641.    "outputs": [],
  642.    "source": [
  643.     "names_d = delayed(get_names)(CA1)"
  644.    ]
  645.   },
  646.   {
  647.    "cell_type": "code",
  648.    "execution_count": 23,
  649.    "metadata": {
  650.     "collapsed": true
  651.    },
  652.    "outputs": [],
  653.    "source": [
  654.     "L_names_d = cl.compute(names_d)"
  655.    ]
  656.   },
  657.   {
  658.    "cell_type": "code",
  659.    "execution_count": 26,
  660.    "metadata": {
  661.     "collapsed": false
  662.    },
  663.    "outputs": [
  664.     {
  665.      "data": {
  666.       "text/plain": [
  667.        "RuntimeError('Long error message',\n",
  668.        "             '(RuntimeError(\"Couldn\\'t find a suitable Universe to unpickle AtomGroup onto with Universe hash \\'(47681, \\'/home/alter/Library/mdanalysis/MDAnalysis/testsuite/MDAnalysisTests/data/adk_oplsaa.gro\\', \\'/home/alter/Library/mdanalysis/MDAnalysis/testsuite/MDAnalysisTests/data/adk_oplsaa.xtc\\')\\'.  Available hashes: \",), <function _unpickle at 0x7fd1841ab758>, ((47681, \\'/home/alter/Library/mdanalysis/MDAnalysis/testsuite/MDAnalysisTests/data/adk_oplsaa.gro\\', \\'/home/alter/Library/mdanalysis/MDAnalysis/testsuite/MDAnalysisTests/data/adk_oplsaa.xtc\\'), array([   4,   21,   45,   64,   83,  102,  121,  128,  137,  152,  159,\\n        169,  176,  198,  205,  219,  236,  246,  263,  283,  302,  319,\\n        334,  356,  377,  384,  402,  417,  434,  453,  464,  478,  485,\\n        497,  514,  533,  557,  567,  577,  593,  615,  626,  633,  644,\\n        659,  678,  685,  707,  724,  734,  756,  768,  787,  804,  816,\\n        826,  833,  855,  874,  890,  904,  916,  931,  950,  966,  985,\\n        995, 1014,')"
  669.       ]
  670.      },
  671.      "execution_count": 26,
  672.      "metadata": {},
  673.      "output_type": "execute_result"
  674.     }
  675.    ],
  676.    "source": [
  677.     "L_names_d.exception()"
  678.    ]
  679.   },
  680.   {
  681.    "cell_type": "markdown",
  682.    "metadata": {},
  683.    "source": [
  684.     "Yeah, there is no universe to connect the `AtomGroup` to, as one might expect."
  685.    ]
  686.   },
  687.   {
  688.    "cell_type": "markdown",
  689.    "metadata": {},
  690.    "source": [
  691.     "## Iteration through trajectories for e.g. fitting"
  692.    ]
  693.   },
  694.   {
  695.    "cell_type": "markdown",
  696.    "metadata": {},
  697.    "source": [
  698.     "What happens if we write a function that takes two `AtomGroup`s and iterates through one of their trajectories, when each `AtomGroup` comes from a different Universe made from the same files? This is very often the case when doing, say, fitting to a reference frame."
  699.    ]
  700.   },
  701.   {
  702.    "cell_type": "code",
  703.    "execution_count": 27,
  704.    "metadata": {
  705.     "collapsed": true
  706.    },
  707.    "outputs": [],
  708.    "source": [
  709.     "from MDAnalysis.analysis.align import alignto\n",
  710.     "\n",
  711.     "def fit_to_reference(ag, reference):\n",
  712.     "    \n",
  713.     "    rmsd = list()\n",
  714.     "    \n",
  715.     "    for ts in ag.universe.trajectory:\n",
  716.     "        rmsd.append(alignto(ag, reference)[1])\n",
  717.     "        \n",
  718.     "    return rmsd"
  719.    ]
  720.   },
  721.   {
  722.    "cell_type": "code",
  723.    "execution_count": 28,
  724.    "metadata": {
  725.     "collapsed": true
  726.    },
  727.    "outputs": [],
  728.    "source": [
  729.     "u3 = mda.Universe(GRO, XTC)\n",
  730.     "u4 = mda.Universe(GRO, XTC)"
  731.    ]
  732.   },
  733.   {
  734.    "cell_type": "code",
  735.    "execution_count": 29,
  736.    "metadata": {
  737.     "collapsed": true
  738.    },
  739.    "outputs": [],
  740.    "source": [
  741.     "CA3 = u3.select_atoms('name CA')\n",
  742.     "CA4 = u4.select_atoms('name CA')"
  743.    ]
  744.   },
  745.   {
  746.    "cell_type": "markdown",
  747.    "metadata": {},
  748.    "source": [
  749.     "The right answer:"
  750.    ]
  751.   },
  752.   {
  753.    "cell_type": "code",
  754.    "execution_count": 30,
  755.    "metadata": {
  756.     "collapsed": false
  757.    },
  758.    "outputs": [
  759.     {
  760.      "data": {
  761.       "text/plain": [
  762.        "[1.2774938328230612e-06,\n",
  763.        " 9.81877665004486,\n",
  764.        " 8.223452081220534,\n",
  765.        " 6.44980413011577,\n",
  766.        " 8.28539036019982,\n",
  767.        " 19.38629287358165,\n",
  768.        " 16.88768126690707,\n",
  769.        " 16.88520038716091,\n",
  770.        " 18.211024048293112,\n",
  771.        " 21.305871377997253]"
  772.       ]
  773.      },
  774.      "execution_count": 30,
  775.      "metadata": {},
  776.      "output_type": "execute_result"
  777.     }
  778.    ],
  779.    "source": [
  780.     "fit_to_reference(CA3, CA4)"
  781.    ]
  782.   },
  783.   {
  784.    "cell_type": "markdown",
  785.    "metadata": {},
  786.    "source": [
  787.     "What do we get with `dask.multiprocessing`?"
  788.    ]
  789.   },
  790.   {
  791.    "cell_type": "code",
  792.    "execution_count": 31,
  793.    "metadata": {
  794.     "collapsed": false
  795.    },
  796.    "outputs": [
  797.     {
  798.      "data": {
  799.       "text/plain": [
  800.        "[1.2774938328230612e-06,\n",
  801.        " 1.2231072255363424e-06,\n",
  802.        " 1.2231072255363424e-06,\n",
  803.        " 9.033245521131802e-07,\n",
  804.        " 1.0430693466562854e-06,\n",
  805.        " 9.033245521131802e-07,\n",
  806.        " 1.0430693466562854e-06,\n",
  807.        " 9.757020314584897e-07,\n",
  808.        " 1.0430693466562854e-06,\n",
  809.        " 8.246187232343996e-07]"
  810.       ]
  811.      },
  812.      "execution_count": 31,
  813.      "metadata": {},
  814.      "output_type": "execute_result"
  815.     }
  816.    ],
  817.    "source": [
  818.     "delayed(fit_to_reference)(CA3, CA4).compute(get=dask.multiprocessing.get)"
  819.    ]
  820.   },
  821.   {
  822.    "cell_type": "markdown",
  823.    "metadata": {},
  824.    "source": [
  825.     "This is the wrong answer. It turns out that both `AtomGroup`s get linked into the same universe in the child process, so they both step through the same frames, giving approximately 0 for all rmsds following a fit. This is a big problem."
  826.    ]
  827.   }
  828.  ],
  829.  "metadata": {
  830.   "kernelspec": {
  831.    "display_name": "Python 2",
  832.    "language": "python2",
  833.    "name": "python2"
  834.   },
  835.   "language_info": {
  836.    "codemirror_mode": {
  837.     "name": "ipython",
  838.     "version": 2
  839.    },
  840.    "file_extension": ".py",
  841.    "mimetype": "text/x-python",
  842.    "name": "python",
  843.    "nbconvert_exporter": "python",
  844.    "pygments_lexer": "ipython2",
  845.    "version": "2.7.13"
  846.   }
  847.  },
  848.  "nbformat": 4,
  849.  "nbformat_minor": 2
  850. }
RAW Paste Data
Pastebin PRO Summer Special!
Get 60% OFF on Pastebin PRO accounts!
Top