rocm_jax/docs/notebooks/thinking_in_jax.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "aPUwOm-eCSFD",
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "# Configure ipython to hide long tracebacks.\n",
    "import sys\n",
    "ipython = get_ipython()\n",
    "\n",
    "def minimal_traceback(*args, **kwargs):\n",
    "  etype, value, tb = sys.exc_info()\n",
    "  value.__cause__ = None  # suppress chained exceptions\n",
    "  stb = ipython.InteractiveTB.structured_traceback(etype, value, tb)\n",
    "  del stb[3:-1]\n",
    "  return ipython._showtraceback(etype, value, stb)\n",
    "\n",
    "ipython.showtraceback = minimal_traceback"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LQHmwePqryRU"
   },
   "source": [
    "# How to Think in JAX\n",
    "\n",
    "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/google/jax/blob/main/docs/notebooks/thinking_in_jax.ipynb)\n",
    "\n",
    "JAX provides a simple and powerful API for writing accelerated numerical code, but working effectively in JAX sometimes requires extra consideration. This document is meant to help build a ground-up understanding of how JAX operates, so that you can use it more effectively."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nayIExVUtsVD"
   },
   "source": [
    "## JAX vs. NumPy\n",
    "\n",
    "**Key Concepts:**\n",
    "\n",
    "- JAX provides a NumPy-inspired interface for convenience.\n",
    "- Through duck-typing, JAX arrays can often be used as drop-in replacements of NumPy arrays.\n",
    "- Unlike NumPy arrays, JAX arrays are always immutable.\n",
    "\n",
    "NumPy provides a well-known, powerful API for working with numerical data. For convenience, JAX provides `jax.numpy` which closely mirrors the numpy API and provides easy entry into JAX. Almost anything that can be done with `numpy` can be done with `jax.numpy`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "kZaOXL7-uvUP",
    "outputId": "17a9ee0a-8719-44bb-a9fe-4c9f24649fef"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2deXxk1XXnv6eqtJd2lZaW1K3W0ns33SC6MWBwMKvtALEdBzye4MQOySTO5iQTnHw+dmInGTyZzziTGScTYjvBE2LsYCcQG4MxAbwATTfQq7pbWy/aVdqrtKvqzh9VT10tJLWk2t5yv5+PPqp69eq9o6f37u+ce889V5RSaDQajca5uNJtgEaj0WjSixYCjUajcThaCDQajcbhaCHQaDQah6OFQKPRaByOJ90GbISysjJVV1eXbjM0Go3GUrz55ptDSinf0u2WFIK6ujqOHj2abjM0Go3GUojIxeW2664hjUajcThaCDQajcbhaCHQaDQah6OFQKPRaByOFgKNRqNxOAkRAhH5mogMisipFT4XEflrEWkXkRMicm3MZw+JSFv056FE2KPRaDSatZOoiOAfgbtX+fweoCn68zDwtwAiUgJ8DjgEHAQ+JyLFCbJJo9FoNGsgIfMIlFI/EpG6VXa5D/i6itS8fl1EikSkCngP8IJSagRARF4gIijfSIRdqeTi8CQ/OD2A2yW8b28VlYXZ6TbJMYTCip+0D3Gia4ytvjzu2l1Jhlv3eqaK0ck5njvdz9jUPO9uKmNPdWG6TdKsk1RNKKsGumLed0e3rbT9HYjIw0SiCTZv3pwcKzeAUoq/faWD//H8OcLRpR2++NxZvnD/Hj7SXJte4xzAUHCW33jiLQ6fH1nc1lTu5Wsfv57aktw0WuYMXjo7yO988xjj0/MAfPE5+MV3beFzP7sbt0vSbJ1mrVjGbVJKPaaUalZKNft875ghnTb+/sed/PfnzvG+vVW89pnbePn338P1dSX816dO8G9v96TbPFszNbfAL//jEY53j/HfPriX0396F3/3n69jMDDLR/7uNYaCs+k20da83jnMr/6/N6kuyuG7v3kzxz57Bx+/sY6vv3aRzz697HChxqSkSgh6gFj3uCa6baXtluBY1xiPfv8s799bxV8/cICqwhzqyvL42sev5+DWEj7znZOcH5pMt5m25dHvn+Vkzzhf/ui1PHhwM3lZHu7aXckTnzzE8OQcn/7WcfQKfMlhfGqe337ybWpKcvjnXznEnupCinIz+ZN7d/Ort9TzxOFLfPdEb7rN1KyRVAnBM8AvRrOHbgDGlVJ9wPPAnSJSHB0kvjO6zfSEwopHvn2CioJsHv3QXlwxYXCmx8X/fvAAbpfwp/9+WjdGSeDtS6N8/bWL/NKNW3nvzoorPttTXcgf3bODH7X6+f6p/jRZaG/+8gdnGQrO8b9+4QBFuZlXfPYHd21nX00hn//3FoKzC2myULMeEpU++g3gNWC7iHSLyCdE5NdE5NeiuzwLdALtwN8Dvw4QHST+AnAk+vN5Y+DY7Hz3RC9n+wP80ft2kp+d8Y7PKwqy+Z3bm3j5nJ/XOofTYKG9+Z8vtFKal8nv3blt2c8/dsMWdlTm8+j3z7IQCqfYOnvTNTLFk2908eDBWvbWvHNg2ON28af37mYwMMvf/6gzDRZq1ktChEAp9aBSqkoplaGUqlFKfVUp9X+VUv83+rlSSv2GUqpBKbVXKXU05rtfU0o1Rn/+IRH2JJtQWPHXL7axvSKf9++tWnG/j92whTJvFn/7ckcKrbM/Ry6M8OO2IX7t1gbyspbPd/C4XfzuHdu4NDLF9072pdhCe/PXL7bhdgm/eVvTivsc2FzM7TvLefy1C0zN6ajA7FhmsNhMvHxukA7/JJ+6rfGKLqGlZGe4+eS7t/LjtiFOdo+n0EJ789Ufn6c4N4OP3bBl1f3u2FlBgy+Pv325Q3fPJYjh4CxPH+vlF66vpaJg9RTpX7u1gbGpeZ58o2vV/TTpRwvBBnji8CV8+Vncvafyqvv+p0Obyclw889vLFsGXLNO+sdneOHMAB9priUn073qvi6X8Ml313O2P8Bbl8ZSZKG9+dbRbuZCYX7xXauLMEBzXQnXbi7in16/qIU4AYTCilA4OddRC8E66Rmb5qVzgzxwfe2aJi3lZ2fw/n1V/PvxPh0iJ4BvHukiFFZ89NDa5pL87DWbyMlw8y9HtVcaL+Gw4onDF7mhvoTG8vw1feeB6zfTOTTJW5dGk2yd/flp+xCH/uJFzvRNJPzYWgjWyb+93YNSrGuy2EeaawnOLvC9E7qvOh6UUvzr293c2FDKltK8NX3Hm+WJCnGvFuI4eevSKN2j0/zC9Wu/99+3ryoqxN1JtMwZfPdELzPzIbaWre3eXw9aCNbJ9070ce3monXNWr2+rpgtpbk8c1znVcdDS98EF4an+NlrNq3rez9/XQ2TcyF+eGYwSZY5g++e6CPT4+KOXVfvEjXwZnm4Z28l3zvRx+xCKInW2Zu5hTDPnx7gjl0VZGes3iW6EbQQrIPzQ5O09E3wvlUyhZZDRLhnTxWvdQwzNjWXJOvsz/dO9OF2CXftXntDBJG+6jJvFs+d0hHZRgmFFc+e7ONntvvwrpCptRIf2FdFYHaBV9t1GvVG+Wn7EOPT83xg3/ranrWihWAdPBtNQ1yvEES+U8lCWPFCy0CizXIESim+d7KPGxtKKcnLvPoXYoiIRwUvnfUzPae90o1w5MIIg4FZPrBvfdEYwE2NZeRneXhOT+7bMP9+opeCbA/vbkpOeR0tBOvg2ZORbqFNRTnr/u7e6kKqi3L0w7BBWvomuDg8teq8jdW4Z08V0/MhXmn1J9gyZ/DcqX6yPC5u21G+7u9medzctrOcH7T068l9G2AhFOaHLQPcubuSTE9ymmwtBGtkYGKG070T6+ofjUVEuGNXBa92DDMzr73S9fLyuUgDftvO9TdEADfUl5Cf7eGls3qcYCO8fG6QGxtKV5zAdzXu2l3J6NQ8x7p0Gu96ebtrjImZhQ2J8FrRQrBGfhT1JG/dtvHQ7NZtPqbnQxy9oFPp1ssrrX52byqgPH9j6zx43C5ubizjlVa/zmlfJxeGJrkwPMV7tm+8IbqpsQy3SxafI83aefncIG6XcFNjWdLOoYVgjbzS6seXn8XOqrXlTy/HofoSMj0uXmnVXul6mJiZ582Lo3GJMESEuH9ihrbBYIIscwavJMAJKszJYH9tEa+0DSXKLMfw8jk/120upjDnnTXNEoUWgjUQCit+3DbErdt8iGx8sY3cTA+Htpbofup18mr7EKGwissjBbgl2pC9ck5f//Xw8rlB6kpzqYszf/2WJh8nuscYmdSZc2tlMBDpkr51e3LXYNFCsAaOd48xPj0ft0cKkYehdSBI79h0AixzBq+0+snP8nBgc1Fcx9lUlENjuZcftWkhWCuzCyFe6xyOW4QBbtlWhlLwk3YdFayVH7dGrlUi2p7V0EKwBl6N3rg3J6CPzvBK9cOwdl7tGOaGhtKErEN8S5OPw+dH9ID9GjneNc7MfJgbG0rjPta+miKKcjP4sY6I18zrncMU5Wawq6ogqefRQrAGDp8fYUdlPsXrzF9fjqZyL8W5Gbxx3hLLLqSd/vEZLg5PcWhrSUKO966GUuYWwhzX2Str4nDnMCJwMAHX3+0SDm0tuWJ9ac3qHD4/wsG6klWrHCcCLQRXYSEU5s2Lowl5ECBSEfPg1hIOn9ezLNfCGxcijcahrfF7pBAp9yGCFuI18saFEbZX5L9jFbKNcnBrKZdGpugb112jV6NvfJpLI1Mcqk/Mvb8aiVqh7G4ROSci7SLyyDKff0lEjkV/WkVkLOazUMxnzyTCnkRyuneCqblQwoQAIo1a18i0HidYA4c7h/FmeeLK1oqlKDeT7RX52itdA/NRJyhR0RiweCwtxFfHuEaJvP4rEbcQiIgb+DJwD7ALeFBEdsXuo5T6XaXUfqXUfuB/A9+J+Xja+EwpdW+89iQa459xsC5x/4yD+mFYM2+cH+G6LcV4EjA+YHBoawlvXhxlXs9yXZVTPeNMzYUS6pHurCrAm+XR9/4aeL1zhPxsDzuTPD4AiYkIDgLtSqlOpdQc8CRw3yr7Pwh8IwHnTQmHz4+wtSyP8qusxrQedlYVkJ/t0d1DV2E4OEvbYJBD9Yn1iA7VlzI9H+JUj141bjWMxvr6BDpBbpfQXFeshWANHD4/zPV1JbiTPD4AiRGCaiB21Y/u6LZ3ICJbgK3Af8RszhaRoyLyuojcv9JJROTh6H5H/f7UZB2Ew4ojF0YSGg1A5GG4vk4Pml2NI9EZ2IkOjY2GTTdGq3P4/Aj1vjx8+VkJPe7BrSW0DQYZDs4m9Lh2wh+YpdM/mZJuIUj9YPEDwFNKqdjcvS1KqWbgo8BfiUjDcl9USj2mlGpWSjX7fMnNqTVoGwwyPj3P9Un4Z1y7uYhO/yTjU/MJP7ZdOHJhhCyPi73V8c0fWIovP4stpbm8rZevXBGlVCRJIsFOEEDzlsgx9fVfmTcvRpyg5iRc/+VIhBD0ALFLFtVEty3HAyzpFlJK9UR/dwIvAwcSYFNCONYV+Wdct6U44cfeXxs55vFu/TCsxLGuMfZWFyal4uL+2iLe7tI1n1biwvAU49PzcU/iW4691YW4XaIL0K3C8e4xMtzC7k3JHx+AxAjBEaBJRLaKSCaRxv4d2T8isgMoBl6L2VYsIlnR12XATUBLAmxKCMe6xinI9lBXuvbVyNbKvtpCRLRXtBLzoTCnesa5pjbxDRHAgdoiBiZmdRrjChhOUDKuf06mm+0V+VoIVuF41xg7qwqSshrZcsQtBEqpBeBTwPPAGeBbSqnTIvJ5EYnNAnoAeFJdWfpxJ3BURI4DLwGPKqVMIwTHu8a4prYorvpCK1GQnUGjz7v4wGmu5Fx/gNmFcNKEYP/mSER2TAvxshzvGic3003TGhepXy/7NxdxvGuMcFhXgl1KKKw40T3ONTXJufeXY2PFxZeglHoWeHbJts8uef8ny3zvVWBvImxINNNzIc4NBPj1ncsOWSSEA5uLeKFlAKVUUsTGyhhdZvuT9DDsrMon0+3i7a4x7tngYjd2xuiWS1bGyv7aIv758CU6h4I0JklsrEqnP0hwdiFpTtBy6JnFK3C6d5xQWCVVlffXFjM6Nc/F4amkncOqHO8aozg3g9qS9a8GtxayPG52VxfoiGAZ5hbCtPROsD+JDdGB6LHf0tf/Hbwd7TJL5vVfihaCFTD6L/fVFibtHMY/Wg9avpPjXeNJ65Yz2F9bxMmecb184hLO9E0wF0petxxAg89LfpZHjxMsw/GuMfKzPNTHWfZ7PWghWIHj3eNUF+VseEWstbCtwktuplt7pUsIzi7QOhhIeh/p/toipucjXYCayyx2yyVRCFwuYV9tob73l+FY1xj7aguTXmguFi0EKxAZKE5eNACR5RP3bCrkpJ7hegUnu8dRKjKgmEyMhu5Et77+sRzrGsOXn0VVYfKcIIhc/3MDAV0SPIaZ+RBn+wMp7RYCLQTLMjI5x6WRqZSM2u+uLuBMX4CQzp5YxPBIk339N5fkkp/t4XSvFoJYjnWNcU1NcrvlAPZsKiQUVpzt1xGZQSrGJpdDC8EyLDZEKVDl3ZsKmZ4PcX5Ir6NrcKJ7jNqSHEoSsP7DaogIu6oKONUzkdTzWInAzDyd/kmuqUluNAywpzpyDi3ElzGi031aCNJPS2+kYUjFrL491ZFz6MboMqd7J9hbnfyGCCKN0dn+CT1gHOVMX8Q735OC619TnENBtofTvfreNzjdO0GZN5OKgsTWd7oaWgiWoaV3gi2lueRnZyT9XA0+L5kel/aKogRmIum0yV6az2D3pgJm5sN0Dk2m5HxmpyV6H6bCCRIRdm0q4LQeI1ukpXeCnVUFKZ9XpIVgGVr6JlLWEGW4XeyszNcRQRTDI92Vohoruzfp7olYDI800RVHV2L3pkLO9gd0REZk/kbbYGDxnkwlWgiWEJxd4MLwZEoWgzDYXV3I6d5xrqy+4Uwue6SpeRgafHlkeVxaiKO09KXWI91TXcDsQpgOv47I2gYDzIdUypygWLQQLOFc/wRKkbKIACJh+MTMAt2jugBaS98EpXmZlKfII/W4XeyoKtARAVGPdCCYUo9UR2SXSeXY5FK0ECzB+GekUpX36IdhkdO9E+zalNo+0j2bCjjdO+H4iKx9MMhcKJzSe7++LBKR6QHjyL2fk+GmrjR1M4oNtBAsoaUvQFFuRtIn08SyvTIft0sc3z1heKSpDo13byokMLNA14izI7KWvqgTlMJo2IjI9LKhRrdcfkqWplyKFoIlGAPFqfRIszPcNJV7HR8RdPijHmkKGyK4HIo7/fqf7h0nJ8PN1hTWuIFIRNbS5+yILBxWnIlGw+lAC0EMC6EwZ6ODZalmV1XBYsaMU0lXH6kRkRkesVNp6Z1gRxo8Uh2RQffoNIHZhbRkDIEWgiu4MDzJ7ELqPVKAbZX59E/MOHoN49O9E2RnuNha5k3pebMz3NSV5jq61IFSKqVp07Fsr4ysR+Dk4n9GNJqO6w8JEgIRuVtEzolIu4g8ssznHxcRv4gci/58Muazh0SkLfrzUCLs2Sin0zBQbGA8DGf7neuVtvSNs6OyIC19pDsqCzjnYCHoHp0mMLOQ1nv/nIPv/TN9E7jk8rVINXELgYi4gS8D9wC7gAdFZNcyu35TKbU/+vOV6HdLgM8Bh4CDwOdEJPErxa+Rlr4JMt0uGnyp9UgBdjjcK1JKca4/wM6q9DwI2yvzuTQyxeTsQlrOn24MEdxRmXoh8GZ5qCnOcXREdm4gwNayvJStUbyUREQEB4F2pVSnUmoOeBK4b43fvQt4QSk1opQaBV4A7k6ATRuipXeCpopIyYdUU1mQTUG2x7EPgz84y+jUPNsq0icEAG2Dziz+Zzgg2ypS7wRBxBFyckTWOhBMWzQAiRGCaqAr5n13dNtSPiQiJ0TkKRGpXed3EZGHReSoiBz1+/0JMPudtA4E0vbPEBFHd0+09kca4O1pEoIdDu+eaB0IUF2Uk5L6WsuxvTKfzqFJZhectzbBzHyIC8OTaXOCIHWDxf8O1Cml9hHx+h9f7wGUUo8ppZqVUs0+ny/hBo5PzTMwMZu2hggiD0Nrf8CRaXSLHmmahLi2OJecDLdjI7Jz/YG0RQMA2ysLCIUVHYPOKzXRPhhEqfQ5QZAYIegBamPe10S3LaKUGlZKzUbffgW4bq3fTRWtg0ZonF4hCMwu0DPmvDS61v4ApXmZlHlTW37XwOUStlV4HRmRzYfCdPon0ybCcLkRbHXgGJlxz6Xz+idCCI4ATSKyVUQygQeAZ2J3EJGqmLf3Ameir58H7hSR4ugg8Z3RbSnHDP+My90TDnwYBgJpFWGICLETr/3F4UnmQuG0eqT1vjwy3OLIiKx1IECmx8WWkty02RC3ECilFoBPEWnAzwDfUkqdFpHPi8i90d1+S0ROi8hx4LeAj0e/OwJ8gYiYHAE+H92WctoGAnizPGxKYWmJpWxbTCF11sOglKItjeMzBtsrCxienMMfmL36zjbiXHR8Jp1CnBHN1nPiGM25gQCNPi8ed/qmdXkScRCl1LPAs0u2fTbm9WeAz6zw3a8BX0uEHfFwbiBAU4U35QtCxFKQnUF1UY7jvNKesWkm50JpjwiMiKx1IJCyevxm4NxAAJdAY3n6xgggEpEdOZ8WPzCttPYHOFRfmlYb9MziKG0DQbaVp7chAmd2Txj9wtsr098QgfMistb+AHWl6cthN9hemU/v+Azj086ZXT8xM0/v+EzanSAtBMBQcJbhybm0jg8YbKvIjxRfW3DOik1G10RjmoW4zJtFmTfTcd0TrSYYn4HLA8ZtDhowbjOJE6SFgMseaTrT5wx2VOazEFZ0DjlnYlPrQICqwmwKc9KTwx6L0yKyxRx2EzhBTozIzDA+A1oIgEhoDOnN4zXYtugVOUcIIjns6b/2ELn+rQNBwmFnzOVoHwwSTnMOu0F1UQ75WR5HCXHrQIC8TDfVRTlptUMLAdA6GKQwJ8MUA4T1vjxcEnlAncBCKEy7P73T62NpKs9nej5E77gz5nKYZXwGIrPrG8q9jrn3IeoEVeanNUkFtBAAkYhge0X6/xkQKYlcW5LrmIfh4sgUcwth00QERuaMU67/uYEAmW4XW9KwPOJyNJV7afc749pDtKyNCe59xwuBUorWaOqoWWhykFdkpm45iFx7cI4QtPYHopO5zNEUNJZ78QdmHbEux2KSignufXP899PIwMQsEzMLpumaAGgo99I5FGQhZP/ModaBIGKCHHaD4rxMSvMynSMEA0FTNEQGhkPW7rf/OIHhBJnh+jteCIw+0iYTzCEwaCrPZz6kuDgylW5Tkk7bYICa4hxyMtObwx5LY7nXEeWop+Yida2aTCLCAI0+5yRLdPiNtOn0X38tBCZKHTVwUj91h3+SxjQsBLQajdGuObtXge30Ryp9mqEhMqguziE7w+WYez8v001FQfqTVBwvBO2DQUryMilNU9XL5XCKEITDik5/MC0rwq1GU7mX8el5/EF71xwyPNIGEwmB2yXUlzkjIuvwB2koT29ZGwPHC0GHP2g6j9Qofmd3IegZm2Z2IWyqhgguz3C2+/XvGAziEthSmr6ql8vRVOGMZImOQfM4QVoI/JOma4gg4qW1Ddp7wMxMfaSxLA5Y2rwx6vBPsrkklyyPecZnABp93kghQhuvHz05u0Dv+Ixp7n1HC8HI5Bwjk3M0+MyRQx1LU3k+HYOTtp7h2hHtozaLV2RQnp9FfpbHAUJgHo80FkOIjTEMO9K5eO+bo+1xtBCYsY/UoLHcy/R8yNarlbUPBinOzaAkLzPdplyBiNBY4bV15koorOgcMmc0bHjJdo6IF9sekwhxQoRARO4WkXMi0i4ijyzz+adFpCW6eP2LIrIl5rOQiByL/jyz9LvJpCPq8ZltjACc0T1hVo8UIveEnWe49oxOM7cQNo1HGsuW0jw8LrH9ve92CZtNMj4TtxCIiBv4MnAPsAt4UER2LdntbaA5unj9U8B/j/lsWim1P/pzLymkwx8ky+NiU5oLPi2HIU52fhjMmDFk0FRh7xmuZvNIY8lwu6gry7N15lCHP2iq8ZlERAQHgXalVKdSag54Ergvdgel1EtKKWN21OtEFqlPOx3+SbaW5eF2pT99aynFeZmUeTNtGx6PTc0xFJyjodx8HinEpPDadIar4WCYUQggksLbYWchGJw0VTSWCCGoBrpi3ndHt63EJ4Dvx7zPFpGjIvK6iNy/0pdE5OHofkf9fn98Fkfp8AdNM2q/HI02rjlk1oFiA2OmuV3HCTr8QUrzMik22fiMQWO5lwvDk8wuhNJtSsIJhRXnTTY+k9LBYhH5GNAM/GXM5i1KqWbgo8BfiUjDct9VSj2mlGpWSjX7fL64bZmZD9E1MmXahggulzqw4wxXM3dNQKQ2vp1nuJp5fAYi935YwYUh+5VZ6R6dYi4UNtX1T4QQ9AC1Me9rotuuQERuB/4YuFcptThlUynVE/3dCbwMHEiATVflwvAkYWXOjCGDpvJ8AjML+AP2m+Ha4Q+S6XZRW2KOwbKluFxCg8++M1wj82fM0zWxFDtnDpmxWy4RQnAEaBKRrSKSCTwAXJH9IyIHgL8jIgKDMduLRSQr+roMuAloSYBNV6Vj0Fx5vMtx+WGwX2PUMWje8RkDu3bNXZ4/Y56GaCkNPi9i0wWaLkfD5ml74hYCpdQC8CngeeAM8C2l1GkR+byIGFlAfwl4gX9Zkia6EzgqIseBl4BHlVKpEYLoP6O+zLwPg51rDkXqrJjnQVgOY4br1Jy9Zrh2mrxbDqILNBXn2tYJKvNmUpRrnvEZTyIOopR6Fnh2ybbPxry+fYXvvQrsTYQN66XDH6S6yFzlj5dizHDtsFk+++xCiEsjU3xgX1W6TVkVQ4g7/ZPsqS5MszWJw+zjMwaNNs0c6vAHqTfZtXfszGKzZwyBfddwvTQ8RSisTN8QNdg0ImsfDJLpcVFdbL75M7E0+PLoHJokZLMyK2ZsexwpBOGwiubxmuufsRwNPvsJgVU80rrSyBiG3SKyDv8k9SYfn4FIRDC3EKZn1D5lVkYm5xidmjfdve9IIeibmGF6PmT6PmqIPAyDgVkmZuwzw9WYQ1BvosGy5cj0uNhSkmtLITZztpyBHSf1Xc4YMte970gh6DBh+tZKGA+DnfpKOwaDVBVmk5eVkCGqpFLv89oqIrDC/BkDw0Yjw88OmDUadqYQmPSfsRyG52Anr9SMfaQr0Vju5fzQJAuhcLpNSQgXh6ci82dM5pEuR1FupMyKre79wUh9s2qT1TdzrBAUZHso85onfWslNpfkkul22aYSplIqMpnJAiIMkQZzPqS4NGKPGa5WcoIgEpHZ5d6HyxlDLpONzzhTCAYnaTTJWqFXw+N2UVeWa5vweDAwS3B2wRIeKcR0zdlkkRSji9Hs4zMGxqQ+u5RZ6fBPmjIadqYQmLzOylIay+3TT23G6fWrYbcUUmP+TG6m+cdnIDKpb3x6nuHJuXSbEjcz8yG6RqdM6QQ5TggmZuYZDMxaImvCoMHn5aJNKjGaeVW45SjIzqA8P8s2QtDuD1omGoDL94kdkiXOD02ilDmdIMcJgZUyhgyMSowXh63fT90xGMSb5aE8PyvdpqwZu0RkxvwZM3ZNrMTlFFLrX38zj884TwhMtmj0WjBuHDt4pZGB4jxLjM8YNPgipQ6s3k/db8yfMWFDtBJVBdnkZLjtce8PTiICW8vM1/Y4UAiCZLiFzSYtf7wc9TZKIbXa+AxEvNLArPXLgZvZI10Jl0toKM+zxWC9meubOU8IBoPUlebhcVvnT8/N9FBdlGP57ong7AJ94zOWGR8wsEsV2MVuUQvMqI+l0WeP4nNmnj9jndYwQVjRIwV71MY/b/LlKVdisWvO4kLc4Z8kP9uDz2ud8RmIXP+esWkmZ61bDjwcVnSaeP6Mo4RgPhTm4vCU5TwiiPZT+4OELVyJ0YhoGi12/SsKsvBmeSzvlRpOkJXGZ+ByRHZ+yLrdQ30mH59xlBBcHJ5iwQLlj5ejsdzLzHyY3ur+lRIAACAASURBVHHrVmJsHwzidgmbS6wlBCJCgy/P8hFB+6B1o2GwdtecWYvNGSRECETkbhE5JyLtIvLIMp9nicg3o58fFpG6mM8+E91+TkTuSoQ9K2HFwTIDOzwMHf4gW0pyyfRYz/9oKPdaenb35fkz5myIVmOLDcqBXx6fMWfbE/cTKSJu4MvAPcAu4EER2bVkt08Ao0qpRuBLwBej391FZI3j3cDdwN9Ej5cUFpenNKkqr4Ydis+ZcWWmtdJY7qV/YoaARcuBd1p0fAbsUQ68wx+kMCeD0jxz1jdLhGt2EGhXSnUqpeaAJ4H7luxzH/B49PVTwHsl0lF5H/CkUmpWKXUeaI8eLyl0DE5SWZBNfnZGsk6RNEq9WRTnZlg2jW4hFObCkDXHZyCmJLJFr7/hkZo1a+Vq1Ft8gSYjY8is4zOJEIJqoCvmfXd027L7RBe7HwdK1/hdAETkYRE5KiJH/X7/hgzNyXRxXV3xhr5rBqy8hmv36DRzobAlPVKw/roQHf4gHpe15s/E0lju5cKwdcuBGxMpzYo1Kk8BSqnHgMcAmpubN5Q682f3702oTammsdzL86cH0m3GhricMWRNIdhckovHJZYdMO7wB9lSmkuGhebPxNJY7l0sB2617sXx6Xn8gVlTO0GJuCt6gNqY9zXRbcvuIyIeoBAYXuN3NVEafF5GJucYsWAlxsWB+jLzPgyrkeF2UVeWZ+GIwLw57GvB8Kat2DVnhSSVRAjBEaBJRLaKSCaRwd9nluzzDPBQ9PWHgf9QkcItzwAPRLOKtgJNwBsJsMmWLFZitKBX2jE4SZk3i8Jc643PGDRadJGU+VCYC0OTps1YWQtWLgdu9owhSIAQRPv8PwU8D5wBvqWUOi0inxeRe6O7fRUoFZF24NPAI9Hvnga+BbQAzwG/oZSyfq3lJNFo4eJz7f6gqftI10JDeR4Xh6eYW7BWP/WlEevOnzEoyM6gosCa5cA7/JNkul3UFptrecpYEjJGoJR6Fnh2ybbPxryeAX5+he/+OfDnibDD7lQX5ZCd4bLcw6CUon0wyPv3VaXblLhoLPcSCisujUzSWJ6fbnPWTIfJJzOtFWN2vdXo8AepK8s1dX0z81qmeQcul1BfZr2HYWRyjvHpeUt7pGDdcuCLpddN3DWxFoysOauVA7dCfTMtBBbDisXnrLgGxHJYVwiClOdnUWDB+TOxNPisVw58PhTm0vCUFgJNYjEqMU7PWWcoxQpZE2shL8vDpsJsy2WuWMEjXQtWLLOyWN/M5BMptRBYjMZyL0pB55B1HoaOwSDZGS6qi8w7WLZWGiwWkRnjM1Ysq7IUKy5bebnYnLmFWAuBxbCiV9TuD1Jf5sXlMuf0+vVgDFhapZ/aH5wlMLNg2Yl8sZTnW68cuFWiYS0EFqOuLBeXWKvUQYc/aPmBSoOGci9TcyH6xmfSbcqaMCqm2kEIRCQSkVkoIugYDFJVmE1elrmLOGghsBhZHjebS3It0089Mx+ie3R6cQ6E1bHaXI52i3ika6XBl2eZaw/WGZ/RQmBBrJQ51OmfRCnrrZO7Eo0Wm93dMRgkN9NNVWF2uk1JCI3lXgYmZi1RDlwpRYd/0hLRmBYCC9Lg83J+yBqVGNstXmxuKWXeTAqyPZYRYqsuT7kSjRYqBz4wMUtwdsESadNaCCxIQ7mXuVCY7lHzL1vZMRjEJVBXav6HYS2ISGRik4UiAis0RGulwULlwBcHii3gBGkhsCBWyhxq9wepLcklOyNpC8+lnAafl3YLLFs5ObtA7/iMbaIxgC0luWS4rVEO3Hg+rTA+poXAgizOcLXAw9Bh0QXTV6Ox3MtQcJbxKXP3U1t5ecqV8Lhd1JVaY8C4wx8kP8uDLz8r3aZcFS0EFqQwJwNffpbpw+NQWNE5ZO6VmTaCVSY2WX0xoJWwSvG59sFI2rQVxme0EFgUK9TG7xmdZm4hbMuGCMzfT90+GMTtEjaXWnN5ypVoLPdaohy4VVJHQQuBZTFSSM08w7XdHwDs1TUBUFuSS6bbZXqvtMMfZHNJLlke+4zPQCQV2SgHblYmZuYZmJi1jBOkhcCiNPjyCMws4A+atxKjMavVbkLgdglby8zfT20lj3Q9NPoia0GY+fp3WqziblxCICIlIvKCiLRFfxcvs89+EXlNRE6LyAkR+YWYz/5RRM6LyLHoz/547HESxsIoZn4YOvxBSvMyKc7LTLcpCafR5KUOFkJhzg9N2mYiXyxGAT1T3/sWWJ4ylngjgkeAF5VSTcCL0fdLmQJ+USm1G7gb+CsRKYr5/A+UUvujP8fitMcxNFogn9oYLLMjDeVeukammJk3ZznwrtFp5kPKEqmL68UK5cDb/UEy3MLmEmuMz8QrBPcBj0dfPw7cv3QHpVSrUqot+roXGAR8cZ7X8VQURCsxmvhhsGvXBERC/rCCC8PmvP7tFvNI14vZy4F3DAbZUppHhomXp4wlXisrlFJ90df9QMVqO4vIQSAT6IjZ/OfRLqMviciKCbci8rCIHBWRo36/P06zrY+ImLoA13BwltGpecsMlq0Xs0/qs0r5441izO42a7JEuz9oqWjsqkIgIj8UkVPL/NwXu5+K/EdW/K+ISBXw/4BfUkoZeV+fAXYA1wMlwB+u9H2l1GNKqWalVLPPpwMKMLdXZJflKVeivsyLyOUBcbPRMRjEl59FYY61l6dciQafecuBLy5PaaHxmasWyVZK3b7SZyIyICJVSqm+aEM/uMJ+BcD3gD9WSr0ec2wjmpgVkX8Afn9d1jucBp+X77zVQ2BmnnyTrUe7OL3ephFBTqab6qIc0w4YW80jXS+xEdkmk618t7g8pYWuf7xdQ88AD0VfPwQ8vXQHEckE/hX4ulLqqSWfVUV/C5HxhVNx2uMojIeh04TjBB3+yPKUmwrN9ZAmksZyrykH65VSkdIeFvJI18vipD4TCrEVnaB4heBR4A4RaQNuj75HRJpF5CvRfT4C3AJ8fJk00SdE5CRwEigD/ixOexyFmfup2wftszzlSjT4vHQOBQmHzdVP7Q/OMjGzYCmPdL2UeTMpzMkw5b1viFO9ha5/XOunKaWGgfcus/0o8Mno638C/mmF798Wz/mdzuaSXDwuc1Zi7PAHuXbzO6aV2IrGci8z82F6xqapNVGaoJ2Wp1wJM5cDN5an9Jp8ecpYrJHbpFmWDLeLurI803VPTM+F6BmbtrVHCuatAmu35SlXIpI1Z85uUatdey0EFseMxec6h4K2Wp5yJcw6qc9uy1OuhBnLgSulaB8MWi4a00JgccxYibFtINIwbqvIT7MlyaUkL5OSvEzTdU+0DgRoqsi3RPnjeDBjRNYzNs3kXIimCi0EmhRixkqM5wYCeFxim+UpV8OMk/paB4Jss5hHuhHMGJFZ1QnSQmBxzFiJsW0gQL0vj0yP/W+vRpNN6huZnGMoOGu5hmgj1BTnkulxmSoiODcQKb2+rdxa19/+T6rNMWMlxtaBIE0OaIgg0j0xOjXPyORcuk0BIt1CANsq7X/93S6h3mTJEq0DASoKsijMNdcEz6uhhcDimK0S49TcApdGpiznEW2UBpPN5WgzhMBifdQbpcFk5cDbBoKWjMa0ENgAM9UcMuzYXumMhqjRZDNcWwciC6ZXFtg7Y8ig0WeecuDhsKJtMECTBZ0gLQQ2wJhYY4YZrq3RwTKndA1VF+WQneEyjRCfGwiwrdL+GUMGDeVe05QD7xqdYmY+bMloTAuBDTAqMfZPpL8SY+tAgEy3iy0mmmmbTFwuob7MHBGZUoq2gYAlG6KNYkRkZrj+VnaCtBDYADPVHGodCNBQ7sVjkQU5EkGDSUodDAXnGJ2at2TXxEap9+WZphx4q4XHZ5zztNoYMwlBZLDMeg9CPDT6vPSMTTM9l95+aqMh2u6AjCGD7Aw3NcXmKAfeOhBgU2G26UrCrwUtBDagNC9SiTHdXmlgZp6esWlLZk3EQ2O5F6XSP2BsCIHVZrXGS6PPHOXArZw2rYXABhiVGNMdEbQNWnNWZbwYNZXSLwRBinIz8HlXXPHVlpihHHgorOjwWzca1kJgExp96e+ndloOu0FdaR4uSX+pg9aBANscUGNoKbHlwNPFxeFJ5hbClnWC4hICESkRkRdEpC36e9kC9CISilmU5pmY7VtF5LCItIvIN6OrmWk2QKQS4xxjU+mb4XquP7IqWW2xMzKGDLIz3GwpzVssL5AOlFJRIXCWCMPlMbK2wfRd/8sDxQ4UAuAR4EWlVBPwYvT9ckwrpfZHf+6N2f5F4EtKqUZgFPhEnPY4FqN7Ip3dQ8ZkGjuvSrYSOyrzOdufvoZoYGKWwMyCZRuieDAGx8/0pVMIrLc8ZSzxCsF9wOPR148TWXd4TUTXKb4NMNYxXtf3NVeyo7IAgDNpbIwi5Y+t+SDEy86qAi4OTxGcXUjL+Y1oxEmpowb52RnUluRwpm8ibTacGwhQU5xDnoVWJYslXiGoUEr1RV/3AxUr7JctIkdF5HURMRr7UmBMKWU8Od1A9UonEpGHo8c46vf74zTbflQVZlOYk0FLb3oehpHJOQYmZtnhoNTFWHZWRYT4XJqE2GgEd1Y58/rvqCxIqxCc6ZtYvAesyFWFQER+KCKnlvm5L3Y/pZQCVhq236KUagY+CvyViDSs11Cl1GNKqWalVLPP51vv122PiLCzKj9tD4Nx3l1VhWk5f7oxGuB0Xv9NhdkU5TpzmG1nVQHnhybTUnNoam6B80OTlhaCq8YxSqnbV/pMRAZEpEop1SciVcDgCsfoif7uFJGXgQPAt4EiEfFEo4IaoGcDf4Mmys6qAp58o4tQWOFOcT+90z3S6qIc8rM9aRUCKzdE8bKrKp+wikRk19QWpfTc5/oDKAW7LHz94+0aegZ4KPr6IeDppTuISLGIZEVflwE3AS3RCOIl4MOrfV+zdnZVFTA9H0pLAa6W3gkqCrIodVgOu4GIsLOyIC0DxjPzITr81vZI48X429MhxMYgtZOF4FHgDhFpA26PvkdEmkXkK9F9dgJHReQ4kYb/UaVUS/SzPwQ+LSLtRMYMvhqnPY4mnQ9Di8M9UohEQ2f7JlI+saltIEgorNi1ybnXv7Y4l7xMd5qEYAJvloea4pyUnztRxDXErZQaBt67zPajwCejr18F9q7w/U7gYDw2aC7TVOHF4xLO9E3wgX2bUnbe2YUQ7YNBfmZHecrOaUZ2VhUwOReia3SKLSlcr/lyt5xzhcDlEnZUFaQlhfRM3wQ7Kq2dNq1nFtuILI+bBp835ZlD7YNBFsLK0qFxItiRpoispW+C3Ey3Y0p/r8TOqnzO9E8Q6XVODeGw4mx/wPLRmBYCm7FrU+q9IuN8TvZIAbZX5OOS1E9saumbYLvFPdJEsKOygMDMQkpLTXSPThOcXbD8va+FwGbsrMqnf2KG0RQupt7SO0F2houtZanrDjEjOZlu6sryUhoRKKU40zfh+GgMYsfIUifELX3jV5zbqmghsBnpGDA+0zfB9sqClKesmpGdVQWc6U/dte8ZmyYwY32PNBHsqMxHJLX3fktfAJdEokEro4XAZhgNQkuKHgalFC19E+xy6PyBpeyqKqBrZJqJmfmUnE93y10mL8vDlpLclI6RnembYGtZHjmZ7pSdMxloIbAZZd4syvOzUvYw9I7PMD49r7smouypjsysPtUznpLzne4dxyU4trTHUnZXF3IyRdce4HTPOLs2WX82vRYCG7InhQ/Dye4xAPbWpHY2p1nZGxWCk92puf4nusdpLPdatthZormmppCesWmGg7NJP5c/MEvv+AzX1Ggh0JiQfTWFtPuDKamEebx7HI9LtEcapSQvk9qSHE6kQAiUUpzoHmdvtRZhA+NapMIRMqI+Q/ytjBYCG3JNTRFKpaZ74mT3ODuq8snOsHYfaSLZV13EiZ6xpJ+nf2KGoeAs+2zgkSaKPdWRLspUCPGJ7nFEIt1RVkcLgQ0xGoYT3cltjCIe6Zj2SJewt6aQrpHppKfwGo3dXi0Ei+RnZ1Dvy0uJEJzsGaPB58Vrg245LQQ2pNSbRXVRDseT/DBcHJ5iYmZBe6RL2GeMEyQ5IjsZ7ZbTA/VXck1NESdTEJGd6B5f/F9bHS0ENuWa2sKkRwQnog2dFoIr2ZOiiOxEzzjbKnS33FL2VhcyMDHLwMRM0s4xMDHDYMA+3XJaCGzKvpoiukamGUli98TJ7jEyPS5HrpO7GgXZGdSXJbd7wuiWs0tDlEgud40m7/pf7pazR7eoFgKbkopxguPd4+yqKiDDrW+jpeytSW4Kb/foNGNT83p8YBl2byrEJcm99090j+G2UbecfoJtyt7qQkSS5xUthMKc6hm3RQ51MthbXUjf+AyDgeR0T7zdFWnkrrGJR5pIcjLdbKvIT2pE8PalMbZV5Ft+RrGBFgKbkh/tnjjelRyv6Gx/gKm5ENduKU7K8a3Ogc2RBvqti8m5/m9eGCE3063nb6zANTVFHOsaS8oiQQuhMG9fGqXZRvd+XEIgIiUi8oKItEV/v+PKiMjPiMixmJ8ZEbk/+tk/isj5mM/2x2OP5koObC7m7a6xpNRnP3phBIDmupKEH9sO7KkuJNPj4s2LI0k5/tGLo+yvLcKju+WW5bq6Ysan5+nwBxN+7HMDASbnQjTXaSEweAR4USnVBLwYfX8FSqmXlFL7lVL7gduAKeAHMbv8gfG5UupYnPZoYri+rpiRyTk6/Ilfw/joxVGqCrOpLrLu8nzJJMvj5pqaQo5cGE34sYOzC5zpm7CVR5poro86KMm4/m9ejBzz2s32uf7xCsF9wOPR148D919l/w8D31dKTcV5Xs0aMLx1w3tPJG9eHOU63RCtynVbSjjdO870XCihxz3eNUZYwXU6GluRutJcyryZSbv3KwqyLL1G8VLiFYIKpVRf9HU/UHGV/R8AvrFk25+LyAkR+ZKIZK30RRF5WESOishRv98fh8nOob4sj9K8TN5I8MPQMzZN3/iM9kivwvV1xcyHFMcTnL1y9MIoIpfHITTvRERo3lLC0YuJjwiOXhileUsJIvZZf+OqQiAiPxSRU8v83Be7n4p0RK/YGS0iVUQWsX8+ZvNngB3A9UAJ8IcrfV8p9ZhSqlkp1ezz+a5mtobow1BXzNEEh8dGaHzdFu2RroYRMb2Z4Mbo6MURtlfkU5CdkdDj2o3mumIujUwxmMCJZf3jM/SMTdsuSeKqQqCUul0ptWeZn6eBgWgDbzT0g6sc6iPAvyqlFlfsUEr1qQizwD8AB+P7czRLub6uhEsjUwmdZWlkrOzUi9GsSlFuJk3lXo4kMCILhRXHLo3pbrk1sNg1mkAhNkTdbtFwvF1DzwAPRV8/BDy9yr4PsqRbKEZEhMj4wqk47dEs4eDWyMPwxvnENUaHz49wYLPOWFkLzXXFvHlxlFCC0hhbeicIzC4sDoZqVmb3pgKyM1wJvfdf6xwiL9PNrk32mEhmEO+T/Chwh4i0AbdH3yMizSLyFWMnEakDaoFXlnz/CRE5CZwEyoA/i9MezRJ2VRWQm+lOmFc6FJzlbH+AGxvKEnI8u3NDfSmBmQVO9yZmctNPO4YAuLGhNCHHszMZbhfXbSnm9c7hhB3z1fZhDm4tsd1s+rjqpyqlhoH3LrP9KPDJmPcXgOpl9rstnvNrro7H7eL6uhJ+2j6UkOO92hF5qG5q1EKwFgzB/En7EPsSMAv4p+1DbKvwUl6QHfexnMDNjT6++NxZBgMzlOfHd836xqfpHJrko4c2J8g682AvWdMsy7ubyujwT9IzNh33sX7aNkR+tscWqzKlAl9+Fjsq8/lJW/xCPLsQ4siFER2NrYObow7Lq+3xRwXGMex4/bUQOIBbtkWyrH7SFl/arVKKn7QPcWNDKW6XfVLnks3NjWUcvTAa93yCty6OMTMf1tHYOti9qYDi3Ax+koCI+KcdQ5TkZdqyrIcWAgfQVO6loiCLH8XplV4amaJnbFo3ROvkpqYy5kJhjsZZbuLVjiHcLuFQvR4oXisul3BjYxk/aRuKq9SKUopX24d5V0MpLhs6QVoIHICI8O4mHz9tH4ore+Wls5Hs4Hc36Xkc6+HQ1hIy3S5eORdfRPbimUEO1Bbp+QPr5ObGMvonZmgf3HjdodO9E/RPzHDrNnve+1oIHMIt23yMTc1zrGvjOdUvnBmgwZfH1rK8BFpmf3IzPbyroZQXzgxs2CvtGZumpW+CO3ZdbfK+Zik/s70cgB+0DGz4GD88M4AIvHdHeaLMMhVaCBzCe7b7yHALz53q39D3x6fnOdw5wh27KhNsmTO4c3cFF4enaNugV/rDaCOmhWD9VBZmc01tUdxCcN3mYkq9K1bBsTRaCBxCQXYGNzWW8dzp/g15pS+fG2QhrHRDtEHu2Bm5bj84vTEhfqElEo3V+7yJNMsx3LmrguNdY/SPr3+Gfd/4NKd6Jrjdxve+FgIHcc+eSrpGpjndO7Hu7/6gZYAybxYHanWhs41QXpDNgc0b80rHp+d5vXPY1g1RsrlrdySSfaFl/UL8QvR/dvtO+15/LQQO4vadFbiEdXcPBWcX+I8zg9y5u8KWGROp4q7dlZzoHufS8PqqsH//ZB8LYcX79lQlyTL701jupd6Xx/dO9l195yX829s9bKvw0uCz79iYFgIHUerN4saGMv7tWM+6lvB77lQ/0/MhPnTtOyaHa9bBffs3IQLffqt7Xd/7zts91Pvy2KfXh46Ln9tfzeudI3SNrF2ILwxN8talMX7uQI2tyk4vRQuBw/j55hq6R6d5bR31V77zVjdbSnNttSJTOqgqzOHmxjK+/Vb3moW4a2SKN86P8KFr7d0QpYIPXlezbiH+17d7EIH7D2xKomXpRwuBw7hrdyWFORl880jXmvbv9Ad5rXOYD9rcI0oVH74uIsRGzaar8Y03LuESuP+Ajsbipbooh5saynjqze41zaeZD4X55pEubm4so6rQPquRLYcWAoeRneHmg9dW8/1TffSNX7320D/89AIZLpctC22lg7t2V1LmzeLvf9x51X2n50L88xuXuHNXpV4bOkF87IbNdI9O8/wasreePdlH/8QMv3RTXfINSzNaCBzIJ27eilLw9z86v+p+o5Nz/MubXdx/YBO+fHvmT6ea7Aw3H79xC6+0+jnXH1h136fe6mZsap5PvHtriqyzP3fsqqSuNJe/e6Vj1TRqpRRf/cl56svyeM82e04ii0ULgQOpKc7lvv3V/PMbF1ddxu/LL7UzuxDmk++uT6F19udjN2whL9PNXz5/bsV9pudC/J//aOPazUW2Ww0rnbhdwq/cUs/x7nF+eGblBRWfPz3Aie5xHr6l3hGZcloIHMpvvbeRsII/f/bMsp+3Dwb4+msX+fnrathWYb9qi+mkKDeTT93WxA/PDPBK6/L1h/7m5XYGJmb5zPt26rGZBPOR5lqayr18/runmZl/Z0XYqbkF/tv3z9Dgy+PD19WkwcLUE5cQiMjPi8hpEQmLSPMq+90tIudEpF1EHonZvlVEDke3f1NEMuOxR7N2tpTm8V9ubeDpY708faznis+m50L8zjeP4c328Pt3bU+Thfbml2+uo74sj0e+fYKh4OwVn71xfoS/ebmDD15brZekTAIZbhd/et9uukam+cJ3W674TCnFF77bwsXhKb5w/x7HLMca7195Cvgg8KOVdhARN/Bl4B5gF/CgiOyKfvxF4EtKqUZgFPhEnPZo1sGnbmvk4NYS/uBfTvCdt7pRSjEUnOVXvn6U070TfPFD++Je1UmzPFkeN3/94AFGJuf42FcOc3F4EoCftA3xicePsLkklz+5d3earbQvNzaU8au31vPE4Uv8xbNnmJkPMbcQ5s++d4ZvvNHFr93aYMsFaFZC4qnRvXgQkZeB348uUbn0s3cBf6KUuiv6/jPRjx4F/EClUmph6X6r0dzcrI4efcepNBtgbGqOX/n6UY5cGMWXn8XE9DxKwV98cK9jwuJ08qNWP7/+xFtMzS1QWZBN7/gM9b48vv7LB6kpzk23ebYmFFZ87plT/NPrlyjIjqzaOzGzwC++awt/8rO7bTk2ICJvKqXe0XsT15rFa6QaiE1a7wYOAaXAmFJqIWb7isnSIvIw8DDA5s06lTFRFOVm8o1fuYGnj/VGVmDKzeSBg5tpLNfFzVLBLdt8vPh7t/LE6xe5NDLF3poi/tOhzWRnuNNtmu1xu4Q/u38v79tTxTPHexGB9+/dxM1NzokEDK4qBCLyQ2C52sN/rJR6OvEmLY9S6jHgMYhEBKk6rxPwuF186LoaPqQjgLRQUZDNp+/UYzHp4sbGMm50+Kp7VxUCpdTtcZ6jB6iNeV8T3TYMFImIJxoVGNs1Go1Gk0JSMSR+BGiKZghlAg8Az6jI4MRLwIej+z0EpCzC0Gg0Gk2EeNNHf05EuoF3Ad8Tkeej2zeJyLMAUW//U8DzwBngW0qp09FD/CHwaRFpJzJm8NV47NFoNBrN+klI1lCq0VlDGo1Gs35WyhpyxmwJjUaj0ayIFgKNRqNxOFoINBqNxuFoIdBoNBqHY8nBYhHxAxc3+PUyYCiB5lgB/Tc7A/032594/94tSinf0o2WFIJ4EJGjy42a2xn9NzsD/Tfbn2T9vbprSKPRaByOFgKNRqNxOE4UgsfSbUAa0H+zM9B/s/1Jyt/ruDECjUaj0VyJEyMCjUaj0cSghUCj0WgcjqOEQETuFpFzItIuIo+k255kIiK1IvKSiLSIyGkR+e1025QqRMQtIm+LyHfTbUsqEJEiEXlKRM6KyJnosq+2RkR+N3pfnxKRb4iI7RbXFpGvicigiJyK2VYiIi+ISFv0d3EizuUYIRARN/Bl4B5gF/CgiOxKr1VJZQH4PaXULuAG4Dds/vfG8ttESp47hf8FPKeU2gFcg83/dhGpBn4LaFZK7QHcRNY5sRv/CNy9ZNsjwItKqSbgxej7uHGMEAAHgXalVKdSnIbpAQAAAkNJREFUag54ErgvzTYlDaVUn1LqrejrAJHGYcU1oe2CiNQA7we+km5bUoGIFAK3EF3LQyk1p5QaS69VKcED5IiIB8gFetNsT8JRSv0IGFmy+T7g8ejrx4H7E3EuJwlBNdAV874bBzSMACJSBxwADqfXkpTwV8B/BcLpNiRFbAX8wD9Eu8O+IiJ56TYqmSileoD/AVwC+oBxpdQP0mtVyqhQSvVFX/cDFYk4qJOEwJGIiBf4NvA7SqmJdNuTTETkA8CgUurNdNuSQjzAtcDfKqUOAJMkqLvArET7xe8jIoKbgDwR+Vh6rUo90eV+E5L/7yQh6AFqY97XRLfZFhHJICICTyilvpNue1LATcC9InKBSNffbSLyT+k1Kel0A91KKSPae4qIMNiZ24HzSim/Umoe+A5wY5ptShUDIlIFEP09mIiDOkkIjgBNIrJVRDKJDC49k2abkoaICJF+4zNKqf+ZbntSgVLqM0qpGqVUHZH/738opWztKSql+oEuEdke3fReoCWNJqWCS8ANIpIbvc/fi80HyGN4Bngo+voh4OlEHNSTiINYAaXUgoh8CnieSJbB15RSp9NsVjK5CfjPwEkRORbd9kdKqWfTaJMmOfwm8ETUwekEfinN9iQVpdRhEXkKeItIdtzb2LDUhIh8A3gPUCYi3cDngEeBb4nIJ4iU4v9IQs6lS0xoNBqNs3FS15BGo9FolkELgUaj0TgcLQQajUbjcLQQaDQajcPRQqDRaDQORwuBRqPROBwtBBqNRuNw/j90YWwgtgIpsgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "x_np = np.linspace(0, 10, 1000)\n",
    "y_np = 2 * np.sin(x_np) * np.cos(x_np)\n",
    "plt.plot(x_np, y_np);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "18XbGpRLuZlr",
    "outputId": "9e98d928-1925-45b1-d886-37956ca95e7c"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO29eXhk1Xnn/3mrSntpV2lpSd1qLb130w2iwYCxg8GA7RhiOw54PMGJHZJf4mxOMsGZ54kTO8ngyTzjTGacTIjtBE+Il9hOIDYGYwJ4AZpuoBda3a2tF+0q7VXaVXV+f9S93YWQ1JJqu8v5PI8eVd26de/R1b3n+77nfc97RCmFRqPRaNyLJ9MN0Gg0Gk1m0UKg0Wg0LkcLgUaj0bgcLQQajUbjcrQQaDQajcvxZboBm6GiokI1NDRkuhkajUZjK1599dURpVRg+XZbCkFDQwPHjh3LdDM0Go3GVojIxZW266EhjUajcTlaCDQajcblaCHQaDQal6OFQKPRaFyOFgKNRqNxOUkRAhH5iogMi8gbq3wuIvLXItIpIidF5Nq4zx4QkQ7j54FktEej0Wg06ydZHsE/Anet8fndQIvx8yDwtwAiUgZ8BrgBOAx8RkRKk9QmjUaj0ayDpMwjUEr9SEQa1tjlHuCrKlbz+mURKRGRGuCdwDNKqTEAEXmGmKB8LRntSicXR6f5wekhvB7hPftrqC7OzXSTXEMkqvhJ5wgneybYHijgzr3VZHn1qGe6GJ9e4KnTg0zMLPL2lgr21RZnukmaDZKuCWW1QE/c+15j22rb34KIPEjMm2Dr1q2paeUmUErxty908T+ePkfUWNrh80+d5XP37uPDrfWZbZwLGAnP8xuPvcaR82OXt7VU+vnKx66nviw/gy1zB8+dHeZ3vnGcydlFAD7/FPzi27bxmZ/di9cjGW6dZr3YxmxSSj2ilGpVSrUGAm+ZIZ0x/v7H3fz3p87xnv01vPTp23j+99/J9Q1l/JdvneTfXu/LdPMczczCEr/8j0c50TvBf/vAfk7/6Z383X++juHQPB/+u5cYCc9nuomO5uXuUX71/71KbUke3/3NWzj+x3fwsZsa+OpLF/njx1cMF2osSrqEoA+IN4/rjG2rbbcFx3smePj7Z3nv/hr++r5D1BTn0VBRwFc+dj2Ht5fx6e+c4vzIdKab6Vge/v5ZTvVN8sWPXMv9h7dSkOPjzr3VPPaJGxidXuBT3zyBXoEvNUzOLPLbX3+durI8/vlXbmBfbTEl+dn8yfv38qu3NvLYkUt892R/ppupWSfpEoIngF80soduBCaVUgPA08C7RaTUCBK/29hmeSJRxUPfPklVUS4Pf3A/njg3ONvn4X/ffwivR/jTfz+tO6MU8Pqlcb760kV+6abtvGt31Zs+21dbzB/dvYsftQf5/huDGWqhs/nLH5xlJLzA//qFQ5TkZ7/psz+4cycH6or57L+3EZ5fylALNRshWemjXwNeAnaKSK+IfFxEfk1Efs3Y5UmgG+gE/h74dQAjSPw54Kjx81kzcGx1vnuyn7ODIf7oPbspzM16y+dVRbn8zu0tPH8uyEvdoxloobP5n8+0U16Qze+9e8eKn3/0xm3sqi7k4e+fZSkSTXPrnE3P2Axff6WH+w/Xs7/urYFhn9fDn75/L8Ohef7+R90ZaKFmoyRFCJRS9yulapRSWUqpOqXUl5VS/1cp9X+Nz5VS6jeUUk1Kqf1KqWNx3/2KUqrZ+PmHZLQn1USiir9+toOdVYW8d3/Nqvt99MZtVPhz+Nvnu9LYOudz9MIYP+4Y4dfe0URBzsr5Dj6vh9+9YweXxmb43qmBNLfQ2fz1sx14PcJv3tay6j6HtpZy++5KHn3pAjML2iuwOrYJFluJ588N0xWc5pO3Nb9pSGg5uVlePvH27fy4Y4RTvZNpbKGz+fKPz1Oan8VHb9y25n537K6iKVDA3z7fpYfnksRoeJ7Hj/fzC9fXU1W0dor0r72jiYmZRb7+Ss+a+2kyjxaCTfDYkUsECnO4a1/1Vff9TzdsJS/Lyz+/smIZcM0GGZyc45kzQ3y4tZ68bO+a+3o8wife3sjZwRCvXZpIUwudzTeP9bIQifKLb1tbhAFaG8q4dmsJ//TyRS3ESSASVUSiqbmOWgg2SN/ELM+dG+a+6+vXNWmpMDeL9x6o4d9PDGgXOQl842gPkajiIzesby7Jz16zhbwsL/9yTFuliRKNKh47cpEbG8torixc13fuu34r3SPTvHZpPMWtcz4/7Rzhhr94ljMDU0k/thaCDfJvr/ehFBuaLPbh1nrC80t876Qeq04EpRT/+novNzWVs628YF3f8ef4DCHu10KcIK9dGqd3fJZfuH799/57DtQYQtybwpa5g++e7GduMcL2ivXd+xtBC8EG+d7JAa7dWrKhWavXN5SyrTyfJ07ovOpEaBuY4sLoDD97zZYNfe/nr6tjeiHCD88Mp6hl7uC7JwfI9nm4Y8/Vh0RN/Dk+7t5fzfdODjC/FElh65zNwlKUp08PcceeKnKz1h4S3QxaCDbA+ZFp2gameM8amUIrISLcva+Gl7pGmZhZSFHrnM/3Tg7g9Qh37l1/RwSxseoKfw5PvaE9ss0SiSqePDXAz+wM4F8lU2s13neghtD8Ei926jTqzfLTzhEmZxd534GN9T3rRQvBBnjSSEPcqBDEvlPNUlTxTNtQspvlCpRSfO/UADc1lVNWkH31L8QRE48qnjsbZHZBW6Wb4eiFMYZD87zvwMa8MYCbmysozPHxlJ7ct2n+/WQ/Rbk+3t6SmvI6Wgg2wJOnYsNCW0ryNvzd/bXF1Jbk6Ydhk7QNTHFxdGbNeRtrcfe+GmYXI7zQHkxyy9zBU28Mkpvl4V27Kzf83Ryfl9t2V/KDtkE9uW8TLEWi/LBtiHfvrSbbl5ouWwvBOhmamuN0/9SGxkfjERHu2FPFi12jzC1qq3SjPH8u1oHftomOCODGxjIKc308d1bHCTbD8+eGeVtjOfnZmytYfOfeasZnFjneo9N4N8rrPRNMzS1x267N3fvrQQvBOvmRYUm+Y8fmXbN37Agwuxjh2AWdSrdRXmgPsndLEZWFm1vnwef1cEtzBS+0B3VO+wa5MDLNhdEZ3rlz8x3Rzc0VeD1y+TnSrJ/nzw3j9Qg3N1ek7BxaCNbJC+1BAoU57K5ZX/70StzQWEa2z8ML7doq3QhTc4u8enE8IRGGmBAPTs3RMRxOUsvcgTmc9s6dm7/+xXlZHKwv4YWOkWQ1yzU8fy7IdVtLKc57a02zZKGFYB1Eooofd4zwjh0BRDa/2EZ+to8btpfpceoN8mLnCJGoSsgiBbjVEJIXzunrvxGePzdMQ3n+uudurMatLQFO9k4wNq0z59bLcCg2JP2OBER4PWghWAcneieYnF1M2CKF2MPQPhSmf2I2CS1zBy+0BynM8XFoa0lCx9lSkkdzpZ8fdWghWC/zSxFe6h5NWIQBbt1RgVLwk07tFayXH7fHrlUy+p610EKwDl40btxbkjBGZ1ql+mFYPy92jXJjU3lS1iG+tSXAkfNjOmC/Tk70TDK3GOWmpvKEj3WgroSS/Cx+rD3idfNy9ygl+VnsqSlK6Xm0EKyDI+fH2FVdSOkG89dXoqXST2l+Fq+ct8WyCxlncHKOi6Mz3LC9LCnHe1tTOQtLUU7o7JV1caR7FBE4nITr7/UIN2wve9P60pq1OXJ+jMMNZWtWOU4GWgiuwlIkyqsXx5PyIECsIubh7WUcOa9nWa6HVy7EOo0btidukUKs3IcIWojXySsXxthZVfiWVcg2y+Ht5Vwam2FgUg+NXo2ByVkujc1wQ2Ny7v21SNYKZXeJyDkR6RSRh1b4/Asictz4aReRibjPInGfPZGM9iST0/1TzCxEkiYEEOvUesZmdZxgHRzpHsWf40soWyuekvxsdlYVaqt0HSwaRlCyvDHg8rG0EF8d8xol8/qvRsJCICJe4IvA3cAe4H4R2RO/j1Lqd5VSB5VSB4H/DXwn7uNZ8zOl1PsTbU+yMf8ZhxuS9884rB+GdfPK+TGu21aKLwnxAZMbtpfx6sVxFvUs1zV5o2+SmYVIUi3S3TVF+HN8+t5fBy93j1GY62N3iuMDkByP4DDQqZTqVkotAF8H7llj//uBryXhvGnhyPkxtlcUUHmV1Zg2wu6aIgpzfXp46CqMhufpGA5zQ2NyLaIbGsuZXYzwRp9eNW4tzM76+iQaQV6P0NpQqoVgHRw5P8r1DWV4UxwfgOQIQS0Qv+pHr7HtLYjINmA78B9xm3NF5JiIvCwi9652EhF50NjvWDCYnqyDaFRx9MJYUr0BiD0M1zfooNnVOGrMwE62a2x2bLozWpsj58doDBQQKMxJ6nEPby+jYzjMaHg+qcd1EsHQPN3B6bQMC0H6g8X3Ad9SSsXn7m1TSrUCHwH+SkSaVvqiUuoRpVSrUqo1EEhtTq1Jx3CYydlFrk/BP+ParSV0B6eZnFlM+rGdwtELY+T4POyvTWz+wHIChTlsK8/ndb185aoopWJJEkk2ggBat8WOqa//6rx6MWYEtabg+q9EMoSgD4hfsqjO2LYS97FsWEgp1Wf87gaeBw4loU1J4XhP7J9x3bbSpB/7YH3smCd69cOwGsd7JthfW5ySiosH60t4vUfXfFqNC6MzTM4uJjyJbyX21xbj9YguQLcGJ3onyPIKe7ekPj4AyRGCo0CLiGwXkWxinf1bsn9EZBdQCrwUt61URHKM1xXAzUBbEtqUFI73TFKU66OhfP2rka2XA/XFiGiraDUWI1He6Jvkmvrkd0QAh+pLGJqa12mMq2AaQam4/nnZXnZWFWohWIMTPRPsrilKyWpkK5GwECilloBPAk8DZ4BvKqVOi8hnRSQ+C+g+4OvqzaUfdwPHROQE8BzwsFLKMkJwomeCa+pLEqovtBpFuVk0B/yXHzjNmzk3GGJ+KZoyITi4NeaRHddCvCIneibJz/bSss5F6jfKwa0lnOiZIBrVlWCXE4kqTvZOck1dau79ldhccfFlKKWeBJ5ctu2Pl73/kxW+9yKwPxltSDazCxHODYX49d0rhiySwqGtJTzTNoRSKiViY2fMIbODKXoYdtcUku318HrPBHdvcrEbJ2MOy6UqY+VgfQn/fOQS3SNhmlMkNnalOxgmPL+UMiNoJfTM4lU43T9JJKpSqsoH60sZn1nk4uhMys5hV070TFCan0V92cZXg1sPOT4ve2uLtEewAgtLUdr6pziYwo7okHHs1/T1fwuvG0Nmqbz+y9FCsArm+OWB+uKUncP8R+ug5Vs50TOZsmE5k4P1JZzqm9TLJy7jzMAUC5HUDcsBNAX8FOb4dJxgBU70TFCY46OxIrGy3xtBC8EqnOidpLYkb9MrYq2HHVV+8rO92ipdRnh+ifbhUMrHSA/WlzC7GBsC1Fzh8rBcCoXA4xEO1Bfre38FjvdMcKC+OOWF5uLRQrAKsUBx6rwBiC2fuG9LMaf0DNc3cap3EqViAcVUYnZ0J3v19Y/neM8EgcIcaopTZwRB7PqfGwrpkuBxzC1GODsYSuuwEGghWJGx6QUujc2kJWq/t7aIMwMhIjp74jKmRZrq67+1LJ/CXB+n+7UQxHO8Z4Jr6lI7LAewb0sxkaji7KD2yEzSEZtcCS0EK3C5I0qDKu/dUszsYoTzI3odXZOTvRPUl+VRloT1H9ZCRNhTU8QbfVMpPY+dCM0t0h2c5pq61HrDAPtqY+fQQnwF0zs9oIUg87T1xzqGdMzq21cbO4fujK5wun+K/bWp74gg1hmdHZzSAWODMwMx63xfGq5/XWkeRbk+Tvfre9/kdP8UFf5sqoqSW9/pamghWIG2/im2ledTmJuV8nM1Bfxk+zzaKjIIzcXSaVO9NJ/J3i1FzC1G6R6ZTsv5rE6bcR+mwwgSEfZsKeK0jpFdpq1/it01RWmfV6SFYAXaBqbS1hFleT3sri7UHoGBaZHuSVONlb1b9PBEPKZFmuyKo6uxd0sxZwdD2iMjNn+jYzh0+Z5MJ1oIlhGeX+LC6HRaFoMw2VtbzOn+Sd5cfcOdXLFI0/MwNAUKyPF5tBAbtA2k1yLdV1vE/FKUrqD2yDqGQyxGVNqMoHi0ECzj3OAUSpE2jwBibvjU3BK947oAWtvAFOUF2VSmySL1eT3sqinSHgGGRToUTqtFqj2yK6QzNrkcLQTLMP8Z6VTlffphuMzp/in2bEnvGOm+LUWc7p9yvUfWORxmIRJN673fWBHzyHTAOHbv52V5aShP34xiEy0Ey2gbCFGSn5XyyTTx7KwuxOsR1w9PmBZpul3jvVuKCc0t0TPmbo+sbcAwgtLoDZsemV421ByWK0zL0pTL0UKwDDNQnE6LNDfLS0ul3/UeQVfQsEjT2BHBFVfc7df/dP8keVletqexxg3EPLK2AXd7ZNGo4ozhDWcCLQRxLEWinDWCZelmT03R5YwZt5KpMVLTIzMtYrfS1j/FrgxYpNojg97xWULzSxnJGAItBG/iwug080vpt0gBdlQXMjg15+o1jE/3T5Gb5WF7hT+t583N8tJQnu/qUgdKqbSmTcezszq2HoGbi/+Z3mgmrj8kSQhE5C4ROScinSLy0Aqff0xEgiJy3Pj5RNxnD4hIh/HzQDLas1lOZyBQbGI+DGcH3WuVtg1Msqu6KCNjpLuqizjnYiHoHZ8lNLeU0Xv/nIvv/TMDU3jkyrVINwkLgYh4gS8CdwN7gPtFZM8Ku35DKXXQ+PmS8d0y4DPADcBh4DMikvyV4tdJ28AU2V4PTYH0WqQAu1xuFSmlODcYYndNZh6EndWFXBqbYXp+KSPnzzSmCO6qTr8Q+HN81JXmudojOzcUYntFQdrWKF5OMjyCw0CnUqpbKbUAfB24Z53fvRN4Rik1ppQaB54B7kpCmzZFW/8ULVWxkg/pprool6Jcn2sfhmB4nvGZRXZUZU4IADqG3Vn8zzRAdlSl3wiCmCHkZo+sfSicMW8AkiMEtUBP3PteY9tyPigiJ0XkWyJSv8HvIiIPisgxETkWDAaT0Oy30j4Uytg/Q0RcPTzRPhjrgHdmSAh2uXx4on0oRG1JXlrqa63EzupCukemmV9y39oEc4sRLoxOZ8wIgvQFi/8daFBKHSBm9T+60QMopR5RSrUqpVoDgUDSGzg5s8jQ1HzGOiKIPQztgyFXptFdtkgzJMT1pfnkZXld65GdGwxlzBsA2FldRCSq6Bp2X6mJzuEwSmXOCILkCEEfUB/3vs7Ydhml1KhSat54+yXguvV+N120D5uucWaFIDS/RN+E+9Lo2gdDlBdkU+FPb/ldE49H2FHld6VHthiJ0h2czpgIw5VOsN2FMTLznsvk9U+GEBwFWkRku4hkA/cBT8TvICI1cW/fD5wxXj8NvFtESo0g8buNbWnHCv+MK8MTLnwYhkIZFWGICbEbr/3F0WkWItGMWqSNgQKyvOJKj6x9KES2z8O2svyMtSFhIVBKLQGfJNaBnwG+qZQ6LSKfFZH3G7v9loicFpETwG8BHzO+OwZ8jpiYHAU+a2xLOx1DIfw5PraksbTEcnZcTiF118OglKIjg/EZk53VRYxOLxAMzV99ZwdxzojPZFKIs4xsPTfGaM4NhWgO+PF5Mzety5eMgyilngSeXLbtj+Nefxr49Crf/QrwlWS0IxHODYVoqfKnfUGIeIpys6gtyXOdVdo3Mcv0QiTjHoHpkbUPhdJWj98KnBsK4RForsxcjABiHtnR8xmxAzNK+2CIGxrLM9oGPbPYoGMozI7KzHZE4M7hCXNceGd15jsicJ9H1j4YoqE8cznsJjurC+mfnGNy1j2z66fmFumfnMu4EaSFABgJzzM6vZDR+IDJjqrCWPG1Jfes2GQOTTRnWIgr/DlU+LNdNzzRboH4DFwJGHe4KGDcYREjSAsBVyzSTKbPmeyqLmQpqugecc/EpvahEDXFuRTnZSaHPR63eWSXc9gtYAS50SOzQnwGtBAAMdcYMpvHa7LjslXkHiGI5bBn/tpD7Pq3D4WJRt0xl6NzOEw0wznsJrUleRTm+FwlxO1DIQqyvdSW5GW0HVoIgPbhMMV5WZYIEDYGCvBI7AF1A0uRKJ3BzE6vj6elspDZxQj9k+6Yy2GV+AzEZtc3Vfpdc++DYQRVF2Y0SQW0EAAxj2BnVeb/GRAriVxflu+ah+Hi2AwLS1HLeARm5oxbrv+5oRDZXg/bMrA84kq0VPrpDLrj2oNR1sYC977rhUApRbuROmoVWlxkFVlpWA5i1x7cIwTtgyFjMpc1uoLmSj/B0Lwr1uW4nKRigXvfGv/9DDI0Nc/U3JJlhiYAmir9dI+EWYo4P3OofSiMWCCH3aS0IJvygmz3CMFQ2BIdkYlpkHUGnR8nMI0gK1x/1wuBOUbaYoE5BCYtlYUsRhQXx2Yy3ZSU0zEcoq40j7zszOawx9Nc6XdFOeqZhVhdqxaLiDBAc8A9yRJdQTNtOvPXXwuBhVJHTdw0Tt0VnKY5AwsBrUWzMTTn9Cqw3cFYpU8rdEQmtaV55GZ5XHPvF2R7qSrKfJKK64WgczhMWUE25RmqerkSbhGCaFTRHQxnZEW4tWip9DM5u0gw7OyaQ6ZF2mQhIfB6hMYKd3hkXcEwTZWZLWtj4noh6AqGLWeRmsXvnC4EfROzzC9FLdURwZUZzk6//l3DYTwC28ozV/VyJVqq3JEs0TVsHSNIC0Fw2nIdEcSstI5hZwfMrDRGGs/lgKXDO6Ou4DRby/LJ8VknPgPQHPDHChE6eP3o6fkl+ifnLHPvu1oIxqYXGJteoClgjRzqeFoqC+kannb0DNcuY4zaKlaRSWVhDoU5PhcIgXUs0nhMITZjGE6k+/K9b42+x9VCYMUxUpPmSj+zixFHr1bWORymND+LsoLsTDflTYgIzVV+R2euRKKK7hFresOmlexkj/hy32MRIU6KEIjIXSJyTkQ6ReShFT7/lIi0GYvXPysi2+I+i4jIcePnieXfTSVdhsVntRgBuGN4wqoWKcTuCSfPcO0bn2VhKWoZizSebeUF+Dzi+Hvf6xG2WiQ+k7AQiIgX+CJwN7AHuF9E9izb7XWg1Vi8/lvAf4/7bFYpddD4eT9ppCsYJsfnYUuGCz6thClOTn4YrJgxZNJS5ewZrlazSOPJ8npoqChwdOZQVzBsqfhMMjyCw0CnUqpbKbUAfB24J34HpdRzSilzdtTLxBapzzhdwWm2VxTg9WQ+fWs5pQXZVPizHeseT8wsMBJeoKnSehYpxKXwOnSGq2lgWFEIIJbC2+VkIRietpQ3lgwhqAV64t73GttW4+PA9+Pe54rIMRF5WUTuXe1LIvKgsd+xYDCYWIsNuoJhy0TtV6LZwTWHrBooNjFnmjs1TtAVDFNekE2pxeIzJs2Vfi6MTjO/FMl0U5JOJKo4b7H4TFqDxSLyUaAV+Mu4zduUUq3AR4C/EpGmlb6rlHpEKdWqlGoNBAIJt2VuMULP2IxlOyK4UurAiTNcrTw0AbHa+E6e4Wrl+AzE7v2oggsjziuz0js+w0Ikaqnrnwwh6APq497XGdvehIjcDvxX4P1KqctTNpVSfcbvbuB54FAS2nRVLoxOE1XWzBgyaaksJDS3RDDkvBmuXcEw2V4P9WXWCJYtx+MRmgLOneEamz9jnaGJ5Tg5c8iKw3LJEIKjQIuIbBeRbOA+4E3ZPyJyCPg7YiIwHLe9VERyjNcVwM1AWxLadFW6hq2Vx7sSVx4G53VGXcPWjc+YOHVo7sr8Get0RMtpCvgRhy7QdMUbtk7fk7AQKKWWgE8CTwNngG8qpU6LyGdFxMwC+kvAD/zLsjTR3cAxETkBPAc8rJRKjxAY/4zGCus+DE6uORSrs2KdB2ElzBmuMwvOmuHabfFhOTAWaCrNd6wRVOHPpiTfOvEZXzIOopR6Enhy2bY/jnt9+yrfexHYn4w2bJSuYJjaEmuVP16OOcO1y2H57PNLES6NzfC+AzWZbsqamELcHZxmX21xhluTPKwenzFpdmjmUFcwTKPFrr1rZxZbPWMInLuG66XRGSJRZfmOqMmhHlnncJhsn4faUuvNn4mnKVBA98g0EYeVWbFi3+NKIYhGlZHHa61/xko0BZwnBHaxSBvKYzEMp3lkXcFpGi0en4GYR7CwFKVv3DllVsamFxifWbTcve9KIRiYmmN2MWL5MWqIPQzDoXmm5pwzw9WcQ9BooWDZSmT7PGwry3ekEFs5W87EiZP6rmQMWeved6UQdFkwfWs1zIfBSWOlXcNhaopzKchJSogqpTQG/I7yCOwwf8bEbKOZ4ecErOoNu1MILPrPWAnTcnCSVWrFMdLVaK70c35kmqVINNNNSQoXR2di82csZpGuREl+rMyKo+794Vh9s1qL1TdzrRAU5fqo8FsnfWs1tpblk+31OKYSplIqNpnJBiIMsQ5zMaK4NOaMGa52MoIg5pE55d6HKxlDHovFZ9wpBMPTNFtkrdCr4fN6aKjId4x7PByaJzy/ZAuLFOKG5hyySIo5xGj1+IyJOanPKWVWuoLTlvSG3SkEFq+zspzmSueMU1txev1aOC2F1Jw/k59t/fgMxCb1Tc4uMjq9kOmmJMzcYoSe8RlLGkGuE4KpuUWGQ/O2yJowaQr4ueiQSoxWXhVuJYpys6gszHGMEHQGw7bxBuDKfeKEZInzI9MoZU0jyHVCYKeMIROzEuPFUfuPU3cNh/Hn+KgszMl0U9aNUzwyc/6MFYcmVuNKCqn9r7+V4zPuEwKLLRq9HswbxwlWaSxQXGCL+IxJUyBW6sDu49SD5vwZC3ZEq1FTlEteltcZ9/7wNCKwvcJ6fY8LhSBMllfYatHyxyvR6KAUUrvFZyBmlYbm7V8O3MoW6Wp4PEJTZYEjgvVWrm/mPiEYDtNQXoDPa58/PT/bR21Jnu2HJ8LzSwxMztkmPmDilCqwl4dFbTCjPp7mgDOKz1l5/ox9esMkYUeLFJxRG/+8xZenXI3LQ3M2F+Ku4DSFuT4CfvvEZyB2/fsmZpmet2858GhU0W3h+TOuEoLFSJSLozO2s4jAGKcOhonauBKj6dE02+z6VxXl4M/x2d4qNY0gO8Vn4IpHdn7EvsNDAxaPz7hKCC6OzrBkg4sbhhMAACAASURBVPLHK9Fc6WduMUr/pH0rMXYOh/F6hK1l9hICEaEpUGB7j6Bz2L7eMNh7aM6qxeZMkiIEInKXiJwTkU4ReWiFz3NE5BvG50dEpCHus08b28+JyJ3JaM9q2DFYZuKEh6ErGGZbWT7ZPvvZH02VflvP7r4yf8aaHdFabHNAOfAr8Rlr9j0JP5Ei4gW+CNwN7AHuF5E9y3b7ODCulGoGvgB83vjuHmJrHO8F7gL+xjheSri8PKVFVXktnFB8zoorM62X5ko/g1NzhGxaDrzbpvEZcEY58K5gmOK8LMoLrFnfLBmm2WGgUynVrZRaAL4O3LNsn3uAR43X3wLeJbGBynuAryul5pVS54FO43gpoWt4muqiXApzs1J1ipRR7s+hND/Ltml0S5EoF0bsGZ+BuJLINr3+pkVq1ayVq9Fo8wWazIwhq8ZnkiEEtUBP3PteY9uK+xiL3U8C5ev8LgAi8qCIHBORY8FgcFMNzcv2cF1D6aa+awXsvIZr7/gsC5GoLS1SsP+6EF3BMD6PvebPxNNc6efCqH3LgZsTKa2KPSpPAUqpR4BHAFpbWzeVOvNn9+5PapvSTXOln6dPD2W6GZviSsaQPYVga1k+Po/YNmDcFQyzrTyfLBvNn4mnudJ/uRy43YYXJ2cXCYbmLW0EJeOu6APq497XGdtW3EdEfEAxMLrO72oMmgJ+xqYXGLNhJcbLgfoK6z4Ma5Hl9dBQUWBjj8C6OezrwbSm7Tg0Z4cklWQIwVGgRUS2i0g2seDvE8v2eQJ4wHj9IeA/VKxwyxPAfUZW0XagBXglCW1yJJcrMdrQKu0anqbCn0Nxvv3iMybNNl0kZTES5cLItGUzVtaDncuBWz1jCJIgBMaY/yeBp4EzwDeVUqdF5LMi8n5jty8D5SLSCXwKeMj47mngm0Ab8BTwG0op+9daThHNNi4+1xkMW3qMdD00VRZwcXSGhSV7jVNfGrPv/BmTotwsqorsWQ68KzhNttdDfam1lqeMJykxAqXUk8CTy7b9cdzrOeDnV/nunwN/nox2OJ3akjxyszy2exiUUnQOh3nvgZpMNyUhmiv9RKKKS2PTNFcWZro566bL4pOZ1os5u95udAXDNFTkW7q+mXVbpnkLHo/QWGG/h2FseoHJ2UVbW6Rg33Lgl0uvW3hoYj2YWXN2Kwduh/pmWghshh2Lz9lxDYiVsK8QhKkszKHIhvNn4mkKxMqBD9uoHPhiJMql0RktBJrkYlZinF2wTyjFDlkT66Egx8eW4lzbZa7YwSJdD3acy3G5vpnFJ1JqIbAZzZV+lILuEfs8DF3DYXKzPNSWWDdYtl6abOaRmfEZO5ZVWY4dl628UmzO2kKshcBm2LH4XGcwTGOFH4/HmtPrN4IZsLTLOHUwPE9obsm2E/niqSy0Xzlwu3jDWghsRkNFPh6xl3vcFQzbPlBp0lTpZ2YhwsDkXKabsi7MiqlOEAIRiXlkNvIIuobD1BTnUpBj7SIOWghsRo7Py9ayfNuMU88tRugdn708B8Lu2G0uR6dNLNL10hQosM21B/vEZ7QQ2BA7ZQ51B6dRyn7r5K5Gs81md3cNh8nP9lJTnJvppiSF5ko/Q1PztigHrpSiKzhtC29MC4ENaQr4OT9ij0qMnTYvNrecCn82Rbk+2wixXZenXI1mG5UDH5qaJzy/ZIu0aS0ENqSp0s9CJErvuPWXrewaDuMRaCi3/sOwHkQkNrHJRh6BHTqi9dJkoxTSy4FiGxhBWghsiJ0yhzqDYerL8snNStnCc2mnKeCn0wbLVk7PL9E/OecYbwxgW1k+WV57lAM3n087xMe0ENiQyzNcbfAwdNl0wfS1aK70MxKeZ3LG2uPUdl6ecjV8Xg8N5fYIGHcFwxTm+AgU5mS6KVdFC4ENKc7LIlCYY3n3OBJVdI9Ye2WmzWCXiU12XwxoNexSfK5zOJY2bYf4jBYCm2KH2vh947MsLEUd2RGB9cepO4fDeD3C1nJ7Lk+5Gs2VfluUA7dL6ihoIbAtZgqplWe4dgZDgLOGJgDqy/LJ9nosb5V2BcNsLcsnx+ec+AzEUpHNcuBWZWpukaGpedsYQVoIbEpToIDQ3BLBsHUrMZqzWp0mBF6PsL3C+uPUdrJIN0JzILYWhJWvf7fNKu4mJAQiUiYiz4hIh/G7dIV9DorISyJyWkROisgvxH32jyJyXkSOGz8HE2mPmzAXRrHyw9AVDFNekE1pQXamm5J0mi1e6mApEuX8yLRjJvLFYxbQs/S9b4PlKeNJ1CN4CHhWKdUCPGu8X84M8ItKqb3AXcBfiUhJ3Od/oJQ6aPwcT7A9rsEOJXnNYJkTaar00zM2w9yiNcuB94zPshhRtkhd3Ch2KAfeGQyT5RW2ltkjPpOoENwDPGq8fhS4d/kOSql2pVSH8bofGAYCCZ7X9VQVGZUYLfwwOHVoAmIuf1TBhVFrXv9Om1mkG8Xq5cC7hsNsKy8gy8LLU8aTaCurlFIDxutBoGqtnUXkMJANdMVt/nNjyOgLIrJqwq2IPCgix0TkWDAYTLDZ9kdELF2AazQ8z/jMom2CZRvF6pP67FL+eLOYs7ujUWsmS3QGw7byxq4qBCLyQxF5Y4Wfe+L3U7H0lVX/KyJSA/w/4JeUUmbe16eBXcD1QBnwh6t9Xyn1iFKqVSnVGghohwKsbRU5ZXnK1Wis8CNyJSBuNbqGwwQKcyjOs/fylKvRFIiVAx+csl458MvLU9ooPnPVItlKqdtX+0xEhkSkRik1YHT0w6vsVwR8D/ivSqmX445tehPzIvIPwO9vqPUupyng5zuv9RGaW6TQYuvRXp5e71CPIC/bS21JnmUDxnazSDdKvEe2xWIr311entJG1z/RoaEngAeM1w8Ajy/fQUSygX8FvqqU+tayz2qM30IsvvBGgu1xFebD0G3BOEFXMLY85ZZiaz2kyaS50m/JYL1SKlbaw0YW6Ua5PKnPgkJsRyMoUSF4GLhDRDqA2433iEiriHzJ2OfDwK3Ax1ZIE31MRE4Bp4AK4M8SbI+rsPI4deewc5anXI2mgJ/uEeuNUwfD80zNLdnKIt0oFf5sivOyLHnvm+LUaKPrn9D6aUqpUeBdK2w/BnzCeP1PwD+t8v3bEjm/29lalo/PY81KjF3BMNdufcu0EkfRXOlnbjFK38Qs9RZKE3TS8pSrYZYDt6QQGMtT+i2+PGU89sht0qxIltdDQ0WB5YYnZhci9E3MOtoiBetWgXXa8pSr0RQosGT6tB3TprUQ2BwrFp/rHgk7annK1bDqpD6nLU+5GlYsB66UonM4bDtvTAuBzbFiJcaOoVjHuKOqMMMtSS1lBdmUFWRbLmDZPhSiparQFuWPE8GKHlnfxCzTCxFaqrQQaNKIFSsxnhsK4fOIY5anXAsrTuprHwqzw2YW6WawokdmVyNIC4HNsWIlxo6hEI2BArJ9zr+9rBawHJteYCQ8b7uOaDPUleaT7fNYyiM4NxQrvb6j0l7X3/lPqsOxYiXG9qEwLS7oiCA2PDE+s8jY9EKmmwLEhoUAdlQ7//p7PUKjxZIl2odCVBXlUJxvrQmeV0MLgc2xWiXGmYUlLo3N2M4i2ixNFpvL0WEKgc3GqDdLk8XKgXcMhW3pjWkhcABWqjlktmNntTs6omaLzXBtH4otmF5d5OyMIZPmgHXKgUejio7hEC02NIK0EDgAK1VibDeCZW4ZGqotySM3y2MZIT43FGJHtfMzhkyaKv2WKQfeMz7D3GLUlt6YFgIHYKVKjO1DIbK9HrZZaKZtKvF4hMYKa3hkSik6hkK27Ig2i+mRWeH629kI0kLgAKxUc6h9KERTpR+fTRbkSAZNhkeWaUbCC4zPLNpyaGKzNAYKLFMOvN3G8Rn3PK0OxkpCEAuW2e9BSITmgJ++iVlmFzI7Tm12RDtdkDFkkpvlpa7UGuXA24dCbCnOtVxJ+PWghcABlBfEKjFm2ioNzS3SNzFry6yJRGiu9KNU5gPGphDYbVZrojQHrDE0Z+e0aS0EDsAqlRg7hu05qzJRzJpKmReCMCX5WQT8q6746kiaAn66M5wsEYkquoL29Ya1EDiE5kDmx6ndlsNu0lBegEcyX+qgfSjEDhfUGFpOc6Wf+aVYOfBMcXF0moWlqG2NoISEQETKROQZEekwfq9YgF5EInGL0jwRt327iBwRkU4R+YaxmplmE8QqMS4wMZO5Ga7nBmOrktWXuiNjyCQ3y8u28oLL5QUygVLKEAJ3iTBciZF1DGfu+l8JFLtQCICHgGeVUi3As8b7lZhVSh00ft4ft/3zwBeUUs3AOPDxBNvjWszhiUwOD5mTaZy8Ktlq7Kou5Oxg5jqioal5QnNLtu2IEsEMjp8ZyKQQ2G95yngSFYJ7gEeN148SW3d4XRjrFN8GmOsYb+j7mjezq7oIgDMZ7Ixi5Y/t+SAkyu6aIi6OzhCeX8rI+U1vxE2poyaFuVnUl+VxZmAqY204NxSirjSPAhutShZPokJQpZQaMF4PAlWr7JcrIsdE5GURMTv7cmBCKWU+Ob1A7WonEpEHjWMcCwaDCTbbedQU51Kcl0Vbf2YehrHpBYam5tnlotTFeHbXxIT4XIaE2OwEd9e48/rvqi7KqBCcGZi6fA/YkasKgYj8UETeWOHnnvj9lFIKWC1sv00p1Qp8BPgrEWnaaEOVUo8opVqVUq2BQGCjX3c8IsLumsKMPQzmeffUFGfk/JnG7IAzef23FOdSku/OMNvumiLOj0xnpObQzMIS50embS0EV/VjlFK3r/aZiAyJSI1SakBEaoDhVY7RZ/zuFpHngUPAt4ESEfEZXkEd0LeJv0FjsLumiK+/0kMkqvCmeZze7RZpbUkehbm+jAqBnTuiRNlTU0hUxTyya+pL0nruc4MhlII9Nr7+iQ4NPQE8YLx+AHh8+Q4iUioiOcbrCuBmoM3wIJ4DPrTW9zXrZ09NEbOLkYwU4Grrn6KqKIdyl+Wwm4gIu6uLMhIwnluM0BW0t0WaKObfngkhNoPUbhaCh4E7RKQDuN14j4i0isiXjH12A8dE5ASxjv9hpVSb8dkfAp8SkU5iMYMvJ9geV5PJh6HN5RYpxLyhswNTaZ/Y1DEUJhJV7Nni3utfX5pPQbY3Q0IwhT/HR11pXtrPnSwSCnErpUaBd62w/RjwCeP1i8D+Vb7fDRxOpA2aK7RU+fF5hDMDU7zvwJa0nXd+KULncJif2VWZtnNakd01RUwvROgZn2FbGtdrvjIs514h8HiEXTVFGUkhPTMwxa5qe6dN65nFDiLH56Up4E975lDncJilqLK1a5wMdmXII2sbmCI/2+ua0t+rsbumkDODU8RGndNDNKo4OxiyvTemhcBh7NmSfqvIPJ+bLVKAnVWFeCT9E5vaBqbYaXOLNBnsqi4iNLeU1lITveOzhOeXbH/vayFwGLtrChmcmmM8jYupt/VPkZvlYXtF+oZDrEhetpeGioK0egRKKc4MTLneG4P4GFn6hLhtYPJN57YrWggcRiYCxmcGpthZXZT2lFUrsrumiDOD6bv2fROzhObsb5Emg13VhYik995vGwjhkZg3aGe0EDgMs0NoS9PDoJSibWCKPS6dP7CcPTVF9IzNMjW3mJbz6WG5KxTk+NhWlp/WGNmZgSm2VxSQl+1N2zlTgRYCh1Hhz6GyMCdtD0P/5ByTs4t6aMJgX21sZvUbfZNpOd/p/kk8gmtLeyxnb20xp9J07QFO902yZ4v9Z9NrIXAg+9L4MJzqnQBgf116Z3Nalf2GEJzqTc/1P9k7SXOl37bFzpLNNXXF9E3MMhqeT/m5gqF5+ifnuKZOC4HGghyoK6YzGE5LJcwTvZP4PKItUoOygmzqy/I4mQYhUEpxsneS/bVahE3Ma5EOQ8j0+kzxtzNaCBzINXUlKJWe4YlTvZPsqikkN8veY6TJ5EBtCSf7JlJ+nsGpOUbC8xxwgEWaLPbVxoYo0yHEJ3snEYkNR9kdLQQOxOwYTvamtjOKWaQT2iJdxv66YnrGZlOewmt2dvu1EFymMDeLxkBBWoTgVN8ETQE/fgcMy2khcCDl/hxqS/I4keKH4eLoDFNzS9oiXcYBM06QYo/slDEspwP1b+aauhJOpcEjO9k7efl/bXe0EDiUa+qLU+4RnDQ6Oi0Eb2Zfmjyyk32T7KjSw3LL2V9bzNDUPENTcyk7x9DUHMMh5wzLaSFwKAfqSugZm2UshcMTp3onyPZ5XLlO7loU5WbRWJHa4QlzWM4pHVEyuTI0mrrrf2VYzhnDoloIHEo64gQneifZU1NEllffRsvZX5faFN7e8VkmZhZ1fGAF9m4pxiOpvfdP9k7gddCwnH6CHcr+2mJEUmcVLUWivNE36Ygc6lSwv7aYgck5hkOpGZ54vSfWyV3jEIs0meRle9lRVZhSj+D1SxPsqCq0/YxiEy0EDqXQGJ440ZMaq+jsYIiZhQjXbitNyfHtzqGtsQ76tYupuf6vXhgjP9ur52+swjV1JRzvmUjJIkFLkSivXxqn1UH3fkJCICJlIvKMiHQYv99yZUTkZ0TkeNzPnIjca3z2jyJyPu6zg4m0R/NmDm0t5fWeiZTUZz92YQyA1oaypB/bCeyrLSbb5+HVi2MpOf6xi+McrC/Bp4flVuS6hlImZxfpCoaTfuxzQyGmFyK0NmghMHkIeFYp1QI8a7x/E0qp55RSB5VSB4HbgBngB3G7/IH5uVLqeILt0cRxfUMpY9MLdAWTv4bxsYvj1BTnUlti3+X5UkmOz8s1dcUcvTCe9GOH55c4MzDlKIs02VxvGCipuP6vXowd89qtzrn+iQrBPcCjxutHgXuvsv+HgO8rpWYSPK9mHZjWumm9J5NXL45zne6I1uS6bWWc7p9kdiGS1OOe6JkgquA67Y2tSkN5PhX+7JTd+1VFObZeo3g5iQpBlVJqwHg9CFRdZf/7gK8t2/bnInJSRL4gIjmrfVFEHhSRYyJyLBgMJtBk99BYUUB5QTavJPlh6JuYZWByTlukV+H6hlIWI4oTSc5eOXZhHJErcQjNWxERWreVcexi8j2CYxfGad1Whohz1t+4qhCIyA9F5I0Vfu6J30/FBqJXHYwWkRpii9g/Hbf508Au4HqgDPjD1b6vlHpEKdWqlGoNBAJXa7YG42FoKOVYkt1j0zW+bpu2SNfC9JheTXJndOziGDurCinKzUrqcZ1Ga0Mpl8ZmGE7ixLLByTn6JmYdlyRxVSFQSt2ulNq3ws/jwJDRwZsd/fAah/ow8K9KqcsrdiilBlSMeeAfgMOJ/Tma5VzfUMalsZmkzrI0M1Z268Vo1qQkP5uWSj9Hk+iRRaKK45cm9LDcOrg8NJpEITZF3WnecKJDQ08ADxivHwAeX2Pf+1k2LBQnIkIsvvBGgu3RLOPw9tjD8Mr55HVGR86PcWirzlhZD60Npbx6cZxIktIY2/qnCM0vXQ6GalZn75YicrM8Sb33X+oeoSDby54tzphIZpLok/wwcIeIdAC3G+8RkVYR+ZK5k4g0APXAC8u+/5iInAJOARXAnyXYHs0y9tQUkZ/tTZpVOhKe5+xgiJuaKpJyPKdzY2M5obklTvcnZ3LTT7tGALipqTwpx3MyWV4P120r5eXu0aQd88XOUQ5vL3PcbPqE6qcqpUaBd62w/Rjwibj3F4DaFfa7LZHza66Oz+vh+oYyfto5kpTjvdgVe6hubtZCsB5MwfxJ5wgHkjAL+KedI+yo8lNZlJvwsdzALc0BPv/UWYZDc1QWJnbNBiZn6R6Z5iM3bE1S66yDs2RNsyJvb6mgKzhN38Rswsf6accIhbk+R6zKlA4ChTnsqi7kJx2JC/H8UoSjF8a0N7YBbjEMlhc7E/cKzGM48fprIXABt+6IZVn9pCOxtFulFD/pHOGmpnK8HuekzqWaW5orOHZhPOH5BK9dnGBuMaq9sQ2wd0sRpflZ/CQJHvFPu0YoK8h2ZFkPLQQuoKXST1VRDj9K0Cq9NDZD38Ss7og2yM0tFSxEohxLsNzEi10jeD3CDY06ULxePB7hpuYKftIxklCpFaUUL3aO8ramcjwONIK0ELgAEeHtLQF+2jmSUPbKc2dj2cFvb9HzODbCDdvLyPZ6eOFcYh7Zs2eGOVRfoucPbJBbmisYnJqjc3jzdYdO908xODXHO3Y4897XQuASbt0RYGJmkeM9m8+pfubMEE2BArZXFCSxZc4nP9vH25rKeebM0Kat0r6JWdoGprhjz9Um72uW8zM7KwH4QdvQpo/xTNsQIvCuXZXJapal0ELgEt65M0CWV3jqjcFNfX9ydpEj3WPcsac6yS1zB+/eW8XF0Rk6NmmV/tDoxLQQbJzq4lyuqS9JSAh+eGaI67aWUu5ftQqOrdFC4BKKcrO4ubmCp04Pbsoqff7cMEtRpTuiTXLH7th1+8HpzQnxM20xb6wx4E9ms1zDu/dUcaJngsHJjc+w75+Y5XT/FLc7+N7XQuAi7t5XTc9Y7KbeKD9oG6LCn8Ohel3obDNUFuVyaOvmrNLJ2UVe7h51dEeUau7cG/Nkn2nbuBA/Y/zPbt/t3OuvhcBF3L67Co+w4eGh8PwS/3FmmHfvrXJkxkS6uHNvNSd7J7k0urEq7N8/NcBSVPGefTUpapnzaa700xgo4HunBq6+8zIeP97Hjio/TQHnxsa0ELiIcn8ONzVV8G/H+za0hN9Tbwwyuxjhg9e+ZXK4ZgPcc3ALIvDt13o39L3vvN5HY6CAA3p96IT4uYO1vNw9Rs/Y+oX4wsg0r12a4OcO1Tmq7PRytBC4jJ9vraN3fJaXNlB/5Tuv9bKtPN9RKzJlgpriPG5pruDbr/WuW4h7xmZ45fwYH7zW2R1ROvjAdXUbFuJ/fb0PEbj30JYUtizzaCFwGXfuraY4L4tvHO1Z1/7dwTAvdY/yAYdbROniQ9fFhNis2XQ1vvbKJTwC9x7S3lii1JbkcXNTBd96tXdd82kWI1G+cbSHW5orqCl2zmpkK6GFwGXkZnn5wLW1fP+NAQYmr1576B9+eoEsj8eRhbYywZ17q6nw5/D3P+6+6r6zCxH++ZVLvHtPtV4bOkl89Mat9I7P8vQ6sreePDXA4NQcv3RzQ+oblmG0ELiQj9+yHaXg7390fs39xqcX+JdXe7j30BYChc7Mn043uVlePnbTNl5oD3JuMLTmvt96rZeJmUU+/vbtaWqd87ljTzUN5fn83Qtda6ZRK6X48k/O01hRwDt3OHMSWTxaCFxIXWk+9xys5Z9fubjmMn5ffK6T+aUon3h7Yxpb53w+euM2CrK9/OXT51bdZ3Yhwv/5jw6u3VriuNWwMonXI/zKrY2c6J3kh2dWX1Dx6dNDnOyd5MFbG12RKaeFwKX81ruaiSr48yfPrPh553CIr750kZ+/ro4dVc6rtphJSvKz+eRtLfzwzBAvtK9cf+hvnu9kaGqeT79nt47NJJkPt9bTUunns989zdziWyvCziws8d++f4amQAEfuq4uAy1MPwkJgYj8vIicFpGoiLSusd9dInJORDpF5KG47dtF5Iix/Rsikp1IezTrZ1t5Af/fO5p4/Hg/jx/ve9NnswsRfucbx/Hn+vj9O3dmqIXO5pdvaaCxooCHvn2SkfD8mz575fwYf/N8Fx+4tlYvSZkCsrwe/vSevfSMzfK577a96TOlFJ/7bhsXR2f43L37XLMca6J/5RvAB4AfrbaDiHiBLwJ3A3uA+0Vkj/Hx54EvKKWagXHg4wm2R7MBPnlbM4e3l/EH/3KS77zWi1KKkfA8v/LVY5zun+LzHzyQ8KpOmpXJ8Xn56/sPMTa9wEe/dISLo9MA/KRjhI8/epStZfn8yfv3ZriVzuWmpgp+9R2NPHbkEn/x5BnmFiMsLEX5s++d4Wuv9PBr72hy5AI0qyGJ1Oi+fBCR54HfN5aoXP7Z24A/UUrdabz/tPHRw0AQqFZKLS3fby1aW1vVsWNvOZVmE0zMLPArXz3G0QvjBApzmJpdRCn4iw/sd41bnEl+1B7k1x97jZmFJaqLcumfnKMxUMBXf/kwdaX5mW6eo4lEFZ954g3+6eVLFOXGVu2dmlviF9+2jT/52b2OjA2IyKtKqbeM3iS0ZvE6qQXik9Z7gRuAcmBCKbUUt33VZGkReRB4EGDrVp3KmCxK8rP52q/cyOPH+2MrMOVnc9/hrTRX6uJm6eDWHQGe/b138NjLF7k0NsP+uhL+0w1byc3yZrppjsfrEf7s3v28Z18NT5zoRwTeu38Lt7S4xxMwuaoQiMgPgZVqD/9XpdTjyW/SyiilHgEegZhHkK7zugGf18MHr6vjg9oDyAhVRbl86t06FpMpbmqu4CaXr7p3VSFQSt2e4Dn6gPq493XGtlGgRER8hldgbtdoNBpNGklHSPwo0GJkCGUD9wFPqFhw4jngQ8Z+DwBp8zA0Go1GEyPR9NGfE5Fe4G3A90TkaWP7FhF5EsCw9j8JPA2cAb6plDptHOIPgU+JSCexmMGXE2mPRqPRaDZOUrKG0o3OGtJoNJqNs1rWkDtmS2g0Go1mVbQQaDQajcvRQqDRaDQuRwuBRqPRuBxbBotFJAhc3OTXK4CRJDbHDui/2R3ov9n5JPr3blNKBZZvtKUQJIKIHFspau5k9N/sDvTf7HxS9ffqoSGNRqNxOVoINBqNxuW4UQgeyXQDMoD+m92B/pudT0r+XtfFCDQajUbzZtzoEWg0Go0mDi0EGo1G43JcJQQicpeInBORThF5KNPtSSUiUi8iz4lIm4icFpHfznSb0oWIeEXkdRH5bqbbkg5EpEREviUiZ0XkjLHsq6MRkd817us3RORrIuK4xbVF5CsiMiwib8RtKxORZ0Skw/hdmoxzuUYIRMQLfBG4G9gD3C8iezLbqpSyBPyeUmoPcCPwGw7/e+P5bWIlz93C/wKeUkrtAq7B4X+7iNQCvwW0KqX2AV5i65w4jX8E7lq27SHgWaVUd/ssHQAAAlVJREFUC/Cs8T5hXCMEwGGgUynVrZRaAL4O3JPhNqUMpdSAUuo143WIWOew6prQTkFE6oD3Al/KdFvSgYgUA7dirOWhlFpQSk1ktlVpwQfkiYgPyAf6M9yepKOU+hEwtmzzPcCjxutHgXuTcS43CUEt0BP3vhcXdIwAItIAHAKOZLYlaeGvgP8CRDPdkDSxHQgC/2AMh31JRAoy3ahUopTqA/4HcAkYACaVUj/IbKvSRpVSasB4PQhUJeOgbhICVyIifuDbwO8opaYy3Z5UIiLvA4aVUq9mui1pxAdcC/ytUuoQME2ShgusijEufg8xEdwCFIjIRzPbqvRjLPeblPx/NwlBH1Af977O2OZYRCSLmAg8ppT6TqbbkwZuBt4vIheIDf3dJiL/lNkmpZxeoFcpZXp73yImDE7mduC8UiqolFoEvgPclOE2pYshEakBMH4PJ+OgbhKCo0CLiGwXkWxiwaUnMtymlCEiQmzc+IxS6n9muj3pQCn1aaVUnVKqgdj/9z+UUo62FJVSg0CPiOw0Nr0LaMtgk9LBJeBGEck37vN34fAAeRxPAA8Yrx8AHk/GQX3JOIgdUEoticgngaeJZRl8RSl1OsPNSiU3A/8ZOCUix41tf6SUejKDbdKkht8EHjMMnG7glzLcnpSilDoiIt8CXiOWHfc6Diw1ISJfA94JVIhIL/AZ4GHgmyLycWKl+D+clHPpEhMajUbjbtw0NKTRaDSaFdBCoNFoNC5HC4FGo9G4HC0EGo1G43K0EGg0Go3L0UKg0Wg0LkcLgUaj0bic/x84B2wg4jy+EQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import jax.numpy as jnp\n",
    "\n",
    "x_jnp = jnp.linspace(0, 10, 1000)\n",
    "y_jnp = 2 * jnp.sin(x_jnp) * jnp.cos(x_jnp)\n",
    "plt.plot(x_jnp, y_jnp);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kTZcsCJiuPG8"
   },
   "source": [
    "The code blocks are identical aside from replacing `np` with `jnp`, and the results are the same. As we can see, JAX arrays can often be used directly in place of NumPy arrays for things like plotting.\n",
    "\n",
    "The arrays themselves are implemented as different Python types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "PjFFunI7xNe8",
    "outputId": "e1706c61-2821-437a-efcd-d8082f913c1f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 4,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(x_np)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "kpv5K7QYxQnX",
    "outputId": "8a3f1cb6-c6d6-494c-8efe-24a8217a9d55"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "jax.interpreters.xla._DeviceArray"
      ]
     },
     "execution_count": 5,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(x_jnp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Mx94Ri7euEZm"
   },
   "source": [
    "Python's [duck-typing](https://en.wikipedia.org/wiki/Duck_typing) allows JAX arrays and NumPy arrays to be used interchangeably in many places.\n",
    "\n",
    "However, there is one important difference between JAX and NumPy arrays: JAX arrays are immutable, meaning that once created their contents cannot be changed.\n",
    "\n",
    "Here is an example of mutating an array in NumPy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "fzp-y1ZVyGD4",
    "outputId": "300a44cc-1ccd-4fb2-f0ee-2179763f7690"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[10  1  2  3  4  5  6  7  8  9]\n"
     ]
    }
   ],
   "source": [
    "# NumPy: mutable arrays\n",
    "x = np.arange(10)\n",
    "x[0] = 10\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nQ-De0xcJ1lT"
   },
   "source": [
    "The equivalent in JAX results in an error, as JAX arrays are immutable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "pCPX0JR-yM4i",
    "outputId": "02a442bc-8f23-4dce-9500-81cd28c0b21f",
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "ignored",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-7-6b90817377fe>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# JAX: immutable arrays\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mjnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mx\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m10\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: '<class 'jax.interpreters.xla._DeviceArray'>' object does not support item assignment. JAX arrays are immutable; perhaps you want jax.ops.index_update or jax.ops.index_add instead?"
     ]
    }
   ],
   "source": [
    "# JAX: immutable arrays\n",
    "x = jnp.arange(10)\n",
    "x[0] = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yRYF0YgO3F4H"
   },
   "source": [
    "For updating individual elements, JAX provides an [indexed update syntax](https://jax.readthedocs.io/en/latest/jax.ops.html#indexed-update-operators) that returns an updated copy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "8zqPEAeP3UK5",
    "outputId": "7e6c996d-d0b0-4d52-e722-410ba78eb3b1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 1 2 3 4 5 6 7 8 9]\n",
      "[10  1  2  3  4  5  6  7  8  9]\n"
     ]
    }
   ],
   "source": [
    "y = x.at[0].set(10)\n",
    "print(x)\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "886BGDPeyXCu"
   },
   "source": [
    "## NumPy, lax & XLA: JAX API layering\n",
    "\n",
    "**Key Concepts:**\n",
    "\n",
    "- `jax.numpy` is a high-level wrapper that provides a familiar interface.\n",
    "- `jax.lax` is a lower-level API that is stricter and often more powerful.\n",
    "- All JAX operations are implemented in terms of operations in [XLA](https://www.tensorflow.org/xla/) – the Accelerated Linear Algebra compiler."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BjE4m2sZy4hh"
   },
   "source": [
    "If you look at the source of `jax.numpy`, you'll see that all the operations are eventually expressed in terms of functions defined in `jax.lax`. You can think of `jax.lax` as a stricter, but often more powerful, API for working with multi-dimensional arrays.\n",
    "\n",
    "For example, while `jax.numpy` will implicitly promote arguments to allow operations between mixed data types, `jax.lax` will not:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "c6EFPcj12mw0",
    "outputId": "730e2ca4-30a5-45bc-923c-c3a5143496e2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DeviceArray(2., dtype=float32)"
      ]
     },
     "execution_count": 9,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import jax.numpy as jnp\n",
    "jnp.add(1, 1.0)  # jax.numpy API implicitly promotes mixed types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "0VkqlcXL2qSp",
    "outputId": "601b0562-3e6a-402d-f83b-3afdd1e7e7c4",
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "ignored",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-10-63245925fccf>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mjax\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mlax\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mlax\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1.0\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# jax.lax API requires explicit type promotion.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: add requires arguments to have the same dtypes, got int32, float32."
     ]
    }
   ],
   "source": [
    "from jax import lax\n",
    "lax.add(1, 1.0)  # jax.lax API requires explicit type promotion."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aC9TkXaTEu7A"
   },
   "source": [
    "If using `jax.lax` directly, you'll have to do type promotion explicitly in such cases:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "3PNQlieT81mi",
    "outputId": "cb3ed074-f410-456f-c086-23107eae2634"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DeviceArray(2., dtype=float32)"
      ]
     },
     "execution_count": 11,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lax.add(jnp.float32(1), 1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M3HDuM4x2eTL"
   },
   "source": [
    "Along with this strictness, `jax.lax` also provides efficient APIs for some more general operations than are supported by NumPy.\n",
    "\n",
    "For example, consider a 1D convolution, which can be expressed in NumPy this way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "Bv-7XexyzVCN",
    "outputId": "f5d38cd8-e7fc-49e2-bff3-a0eee306cb54"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DeviceArray([1., 3., 4., 4., 4., 4., 4., 4., 4., 4., 3., 1.], dtype=float32)"
      ]
     },
     "execution_count": 12,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = jnp.array([1, 2, 1])\n",
    "y = jnp.ones(10)\n",
    "jnp.convolve(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0GPqgT7S0q8r"
   },
   "source": [
    "Under the hood, this NumPy operation is translated to a much more general convolution implemented by [`lax.conv_general_dilated`](https://jax.readthedocs.io/en/latest/_autosummary/jax.lax.conv_general_dilated.html):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "pi4f6ikjzc3l",
    "outputId": "b9b37edc-b911-4010-aaf8-ee8f500111d7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DeviceArray([1., 3., 4., 4., 4., 4., 4., 4., 4., 4., 3., 1.], dtype=float32)"
      ]
     },
     "execution_count": 13,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from jax import lax\n",
    "result = lax.conv_general_dilated(\n",
    "    x.reshape(1, 1, 3).astype(float),  # note: explicit promotion\n",
    "    y.reshape(1, 1, 10),\n",
    "    window_strides=(1,),\n",
    "    padding=[(len(y) - 1, len(y) - 1)])  # equivalent of padding='full' in NumPy\n",
    "result[0, 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7mdo6ycczlbd"
   },
   "source": [
    "This is a batched convolution operation designed to be efficient for the types of convolutions often used in deep neural nets. It requires much more boilerplate, but is far more flexible and scalable than the convolution provided by NumPy (See [Convolutions in JAX](https://jax.readthedocs.io/en/latest/notebooks/convolutions.html) for more detail on JAX convolutions).\n",
    "\n",
    "At their heart, all `jax.lax` operations are Python wrappers for operations in XLA; here, for example, the convolution implementation is provided by [XLA:ConvWithGeneralPadding](https://www.tensorflow.org/xla/operation_semantics#convwithgeneralpadding_convolution).\n",
    "Every JAX operation is eventually expressed in terms of these fundamental XLA operations, which is what enables just-in-time (JIT) compilation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NJfWa2PktD5_"
   },
   "source": [
    "## To JIT or not to JIT\n",
    "\n",
    "**Key Concepts:**\n",
    "\n",
    "- By default JAX executes operations one at a time, in sequence.\n",
    "- Using a just-in-time (JIT) compilation decorator, sequences of operations can be optimized together and run at once.\n",
    "- Not all JAX code can be JIT compiled, as it requires array shapes to be static & known at compile time.\n",
    "\n",
    "The fact that all JAX operations are expressed in terms of XLA allows JAX to use the XLA compiler to execute blocks of code very efficiently.\n",
    "\n",
    "For example, consider this function that normalizes the rows of a 2D matrix, expressed in terms of `jax.numpy` operations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "SQj_UKGc-7kQ"
   },
   "outputs": [],
   "source": [
    "import jax.numpy as jnp\n",
    "\n",
    "def norm(X):\n",
    "  X = X - X.mean(0)\n",
    "  return X / X.std(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0yVo_OKSAolW"
   },
   "source": [
    "A just-in-time compiled version of the function can be created using the `jax.jit` transform:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "oHLwGmhZAnCY"
   },
   "outputs": [],
   "source": [
    "from jax import jit\n",
    "norm_compiled = jit(norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q3H9ig5GA2Ms"
   },
   "source": [
    "This function returns the same results as the original, up to standard floating-point accuracy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "oz7zzyS3AwMc",
    "outputId": "914f9242-82c4-4365-abb2-77843a704e03"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 16,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.seed(1701)\n",
    "X = jnp.array(np.random.rand(10000, 10))\n",
    "np.allclose(norm(X), norm_compiled(X), atol=1E-6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3GvisB-CA9M8"
   },
   "source": [
    "But due to the compilation (which includes fusing of operations, avoidance of allocating temporary arrays, and a host of other tricks), execution times can be orders of magnitude faster in the JIT-compiled case (note the use of `block_until_ready()` to account for JAX's [asynchronous dispatch](https://jax.readthedocs.io/en/latest/async_dispatch.html)):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "6mUB6VdDAEIY",
    "outputId": "5d7e1bbd-4064-4fe3-f3d9-5435b5283199"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100 loops, best of 3: 4.3 ms per loop\n",
      "1000 loops, best of 3: 452 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%timeit norm(X).block_until_ready()\n",
    "%timeit norm_compiled(X).block_until_ready()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B1eGBGn0tMba"
   },
   "source": [
    "That said, `jax.jit` does have limitations: in particular, it requires all arrays to have static shapes. That means that some JAX operations are incompatible with JIT compilation.\n",
    "\n",
    "For example, this operation can be executed in op-by-op mode:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "YfZd9mW7CSKM",
    "outputId": "899fedcc-0857-4381-8f57-bb653e0aa2f1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DeviceArray([-0.10570311, -0.59403396, -0.8680282 , -0.23489487], dtype=float32)"
      ]
     },
     "execution_count": 18,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def get_negatives(x):\n",
    "  return x[x < 0]\n",
    "\n",
    "x = jnp.array(np.random.randn(10))\n",
    "get_negatives(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "g6niKxoQC2mZ"
   },
   "source": [
    "But it returns an error if you attempt to execute it in jit mode:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "id": "yYWvE4rxCjPK",
    "outputId": "765b46d3-49cd-41b7-9815-e8bb7cd80175",
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [
    {
     "ename": "IndexError",
     "evalue": "ignored",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mIndexError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-19-ec8799cf80d7>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mjit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mget_negatives\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mIndexError\u001b[0m: Array boolean indices must be concrete."
     ]
    }
   ],
   "source": [
    "jit(get_negatives)(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vFL6DNpECfVz"
   },
   "source": [
    "This is because the function generates an array whose shape is not known at compile time: the size of the output depends on the values of the input array, and so it is not compatible with JIT."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BzBnKbXwXjLV"
   },
   "source": [
    "## JIT mechanics: tracing and static variables\n",
    "\n",
    "**Key Concepts:**\n",
    "\n",
    "- JIT and other JAX transforms work by *tracing* a function to determine its effect on inputs of a specific shape and type.\n",
    "\n",
    "- Variables that you don't want to be traced can be marked as *static*\n",
    "\n",
    "To use `jax.jit` effectively, it is useful to understand how it works. Let's put a few `print()` statements within a JIT-compiled function and then call the function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "TfjVIVuD4gnc",
    "outputId": "df6ad898-b047-4ad1-eb18-2fbcb3fd2ab3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running f():\n",
      "  x = Traced<ShapedArray(float32[3,4])>with<DynamicJaxprTrace(level=0/1)>\n",
      "  y = Traced<ShapedArray(float32[4])>with<DynamicJaxprTrace(level=0/1)>\n",
      "  result = Traced<ShapedArray(float32[3])>with<DynamicJaxprTrace(level=0/1)>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "DeviceArray([0.25773212, 5.3623195 , 5.4032435 ], dtype=float32)"
      ]
     },
     "execution_count": 20,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@jit\n",
    "def f(x, y):\n",
    "  print(\"Running f():\")\n",
    "  print(f\"  x = {x}\")\n",
    "  print(f\"  y = {y}\")\n",
    "  result = jnp.dot(x + 1, y + 1)\n",
    "  print(f\"  result = {result}\")\n",
    "  return result\n",
    "\n",
    "x = np.random.randn(3, 4)\n",
    "y = np.random.randn(4)\n",
    "f(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ts1fP45A40QV"
   },
   "source": [
    "Notice that the print statements execute, but rather than printing the data we passed to the function, though, it prints *tracer* objects that stand-in for them.\n",
    "\n",
    "These tracer objects are what `jax.jit` uses to extract the sequence of operations specified by the function. Basic tracers are stand-ins that encode the **shape** and **dtype** of the arrays, but are agnostic to the values. This recorded sequence of computations can then be efficiently applied within XLA to new inputs with the same shape and dtype, without having to re-execute the Python code.\n",
    "\n",
    "When we call the compiled function again on matching inputs, no re-compilation is required and nothing is printed because the result is computed in compiled XLA rather than in Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "id": "xGntvzNH7skE",
    "outputId": "66694b8b-181f-4635-a8e2-1fc7f244d94b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DeviceArray([1.4344584, 4.3004413, 7.9897013], dtype=float32)"
      ]
     },
     "execution_count": 21,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x2 = np.random.randn(3, 4)\n",
    "y2 = np.random.randn(4)\n",
    "f(x2, y2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9EB9WkRX7fm0"
   },
   "source": [
    "The extracted sequence of operations is encoded in a JAX expression, or *jaxpr* for short. You can view the jaxpr using the `jax.make_jaxpr` transformation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "89TMp_Op5-JZ",
    "outputId": "151210e2-af6f-4950-ac1e-9fdb81d4aae1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{ lambda  ; a b.\n",
       "  let c = add a 1.0\n",
       "      d = add b 1.0\n",
       "      e = dot_general[ dimension_numbers=(((1,), (0,)), ((), ()))\n",
       "                       precision=None ] c d\n",
       "  in (e,) }"
      ]
     },
     "execution_count": 22,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from jax import make_jaxpr\n",
    "\n",
    "def f(x, y):\n",
    "  return jnp.dot(x + 1, y + 1)\n",
    "\n",
    "make_jaxpr(f)(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0Oq9S4MZ90TL"
   },
   "source": [
    "Note one consequence of this: because JIT compilation is done *without* information on the content of the array, control flow statements in the function cannot depend on traced values. For example, this fails:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "id": "A0rFdM95-Ix_",
    "outputId": "d7ffa367-b241-488e-df96-ad0576536605",
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [
    {
     "ename": "ConcretizationTypeError",
     "evalue": "ignored",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mConcretizationTypeError\u001b[0m                   Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-23-acbedba5ce66>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m   \u001b[0;32mreturn\u001b[0m \u001b[0;34m-\u001b[0m\u001b[0mx\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mneg\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mConcretizationTypeError\u001b[0m: Abstract tracer value encountered where concrete value is expected.\n\nThe problem arose with the bool function. \n\nWhile tracing the function f at <ipython-input-23-acbedba5ce66>:1, this concrete value was not available in Python because it depends on the value of the arguments to f at <ipython-input-23-acbedba5ce66>:1 at flattened positions [1], and the computation of these values is being staged out (that is, delayed rather than executed eagerly).\n\nYou can use transformation parameters such as static_argnums for jit to avoid tracing particular arguments of transformed functions, though at the cost of more recompiles.\n\nSee https://jax.readthedocs.io/en/latest/faq.html#abstract-tracer-value-encountered-where-concrete-value-is-expected-error for more information.\n\nEncountered tracer value: Traced<ShapedArray(bool[], weak_type=True)>with<DynamicJaxprTrace(level=0/1)>"
     ]
    }
   ],
   "source": [
    "@jit\n",
    "def f(x, neg):\n",
    "  return -x if neg else x\n",
    "\n",
    "f(1, True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DkTO9m8j-TYI"
   },
   "source": [
    "If there are variables that you would not like to be traced, they can be marked as static for the purposes of JIT compilation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "K1C7ZnVv-lbv",
    "outputId": "cdbdf152-30fd-4ecb-c9ec-1d1124f337f7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DeviceArray(-1, dtype=int32)"
      ]
     },
     "execution_count": 24,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from functools import partial\n",
    "\n",
    "@partial(jit, static_argnums=(1,))\n",
    "def f(x, neg):\n",
    "  return -x if neg else x\n",
    "\n",
    "f(1, True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dD7p4LRsGzhx"
   },
   "source": [
    "Note that calling a JIT-compiled function with a different static argument results in re-compilation, so the function still works as expected:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "id": "sXqczBOrG7-w",
    "outputId": "3a3f50e6-d1fc-42bb-d6df-eb3d206e4b67"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DeviceArray(1, dtype=int32)"
      ]
     },
     "execution_count": 25,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f(1, False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZESlrDngGVb1"
   },
   "source": [
    "Understanding which values and operations will be static and which will be traced is a key part of using `jax.jit` effectively."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r-RCl_wD5lI7"
   },
   "source": [
    "## Static vs Traced Operations\n",
    "\n",
    "**Key Concepts:**\n",
    "\n",
    "- Just as values can be either static or traced, operations can be static or traced.\n",
    "\n",
    "- Static operations are evaluated at compile-time in Python; traced operations are compiled & evaluated at run-time in XLA.\n",
    "\n",
    "- Use `numpy` for operations that you want to be static; use `jax.numpy` for operations that you want to be traced.\n",
    "\n",
    "This distinction between static and traced values makes it important to think about how to keep a static value static. Consider this function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "XJCQ7slcD4iU",
    "outputId": "a89a5614-7359-4dc7-c165-03e7d0fc6610",
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [
    {
     "ename": "ConcretizationTypeError",
     "evalue": "ignored",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mConcretizationTypeError\u001b[0m                   Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-26-5fa933a68063>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mjnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mones\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m3\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 9\u001b[0;31m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mConcretizationTypeError\u001b[0m: Abstract tracer value encountered where concrete value is expected.\n\nThe error arose in jax.numpy.reshape.\n\nWhile tracing the function f at <ipython-input-26-5fa933a68063>:4, this value became a tracer due to JAX operations on these lines:\n\n  operation c:int32[] = reduce_prod[ axes=(0,) ] b:int32[2]\n    from line <ipython-input-26-5fa933a68063>:6 (f)\n\nSee https://jax.readthedocs.io/en/latest/faq.html#abstract-tracer-value-encountered-where-concrete-value-is-expected-error for more information.\n\nEncountered tracer value: Traced<ShapedArray(int32[])>with<DynamicJaxprTrace(level=0/1)>"
     ]
    }
   ],
   "source": [
    "import jax.numpy as jnp\n",
    "from jax import jit\n",
    "\n",
    "@jit\n",
    "def f(x):\n",
    "  return x.reshape(jnp.array(x.shape).prod())\n",
    "\n",
    "x = jnp.ones((2, 3))\n",
    "f(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZO3GMGrHBZDS"
   },
   "source": [
    "This fails with an error specifying that a tracer was found instead of a 1D sequence of concrete values of integer type. Let's add some print statements to the function to understand why this is happening:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "id": "Cb4mbeVZEi_q",
    "outputId": "f72c1ce3-950c-400f-bfea-10c0d0118911"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = Traced<ShapedArray(float32[2,3])>with<DynamicJaxprTrace(level=0/1)>\n",
      "x.shape = (2, 3)\n",
      "jnp.array(x.shape).prod() = Traced<ShapedArray(int32[])>with<DynamicJaxprTrace(level=0/1)>\n"
     ]
    }
   ],
   "source": [
    "@jit\n",
    "def f(x):\n",
    "  print(f\"x = {x}\")\n",
    "  print(f\"x.shape = {x.shape}\")\n",
    "  print(f\"jnp.array(x.shape).prod() = {jnp.array(x.shape).prod()}\")\n",
    "  # comment this out to avoid the error:\n",
    "  # return x.reshape(jnp.array(x.shape).prod())\n",
    "\n",
    "f(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "viSQPc3jEwJr"
   },
   "source": [
    "Notice that although `x` is traced, `x.shape` is a static value. However, when we use `jnp.array` and `jnp.prod` on this static value, it becomes a traced value, at which point it cannot be used in a function like `reshape()` that requires a static input (recall: array shapes must be static).\n",
    "\n",
    "A useful pattern is to use `numpy` for operations that should be static (i.e. done at compile-time), and use `jax.numpy` for operations that should be traced (i.e. compiled and executed at run-time). For this function, it might look like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "GiovOOPcGJhg",
    "outputId": "399ee059-1807-4866-9beb-1c5131e38e15"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DeviceArray([1., 1., 1., 1., 1., 1.], dtype=float32)"
      ]
     },
     "execution_count": 28,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from jax import jit\n",
    "import jax.numpy as jnp\n",
    "import numpy as np\n",
    "\n",
    "@jit\n",
    "def f(x):\n",
    "  return x.reshape((np.prod(x.shape),))\n",
    "\n",
    "f(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "C-QZ5d1DG-dv"
   },
   "source": [
    "For this reason, a standard convention in JAX programs is to `import numpy as np` and `import jax.numpy as jnp` so that both interfaces are available for finer control over whether operations are performed in a static matter (with `numpy`, once at compile-time) or a traced manner (with `jax.numpy`, optimized at run-time)."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "thinking_in_jax.ipynb",
   "provenance": []
  },
  "jupytext": {
   "formats": "ipynb,md:myst"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}