{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7290ff22",
   "metadata": {},
   "source": [
    "# Differentiability\n",
    "\n",
    "**What you'll learn / who it's for (training).** The *simulator → trainable* half of\n",
    "the bridge. Spiking models in ``brainpy.state`` are differentiable: a surrogate\n",
    "gradient replaces the non-differentiable spike threshold, and\n",
    "backpropagation-through-time runs straight over the ``transform`` loops you already\n",
    "use to simulate. You'll plug in a surrogate, train a tiny SNN end to end, and learn\n",
    "when to switch to checkpointed loops for long rollouts."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdf08c41",
   "metadata": {},
   "source": [
    "## 1. Why spikes need surrogate gradients\n",
    "\n",
    "A spike is a step function of the membrane potential: it is $0$ below threshold and\n",
    "$1$ at threshold. Its derivative is zero almost everywhere and undefined at the\n",
    "threshold itself — useless for gradient descent, which would see no signal to follow.\n",
    "\n",
    "The **surrogate gradient** trick keeps the hard spike on the *forward* pass (so the\n",
    "dynamics are exactly the spiking dynamics) but substitutes a smooth, well-behaved\n",
    "function on the *backward* pass — a narrow bump centered at threshold. You select one\n",
    "by passing ``spk_fun=`` to any neuron; ``braintools.surrogate`` provides the\n",
    "standard family."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d5a91264",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-17T09:10:46.208346Z",
     "iopub.status.busy": "2026-06-17T09:10:46.208188Z",
     "iopub.status.idle": "2026-06-17T09:10:53.279808Z",
     "shell.execute_reply": "2026-06-17T09:10:53.278856Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "An NVIDIA GPU may be present on this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "surrogate: ReluGrad(alpha=0.3, width=1.0)\n"
     ]
    }
   ],
   "source": [
    "import brainpy\n",
    "import brainstate\n",
    "import braintools\n",
    "import brainunit as u\n",
    "import jax.numpy as jnp\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# A LIF neuron whose backward pass uses a ReLU-shaped surrogate gradient.\n",
    "neuron = brainpy.state.LIF(\n",
    "    100,\n",
    "    tau=20. * u.ms, V_rest=0. * u.mV, V_reset=0. * u.mV, V_th=1. * u.mV,\n",
    "    spk_fun=braintools.surrogate.ReluGrad(),\n",
    ")\n",
    "print('surrogate:', neuron.spk_fun)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d1896a4",
   "metadata": {},
   "source": [
    "Other choices include ``braintools.surrogate.SigmoidGrad()``, ``GaussianGrad()``,\n",
    "and ``SuperSpike()`` — they differ only in the shape of the backward-pass bump.\n",
    "``ReluGrad`` is a robust default."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f65d07ce",
   "metadata": {},
   "source": [
    "## 2. Backprop through time with `transform`\n",
    "\n",
    "Two facts make training work without any special machinery:\n",
    "\n",
    "1. The ``brainstate.transform`` loops — ``for_loop`` and ``scan`` — are\n",
    "   **differentiable**. Wrapping a simulation in ``for_loop`` and asking\n",
    "   ``brainstate.transform.grad`` for gradients gives you\n",
    "   backpropagation-through-time (BPTT) over the unrolled trajectory.\n",
    "2. ``brainstate.transform.grad`` differentiates with respect to exactly the\n",
    "   ``ParamState`` variables you select (``net.states(brainstate.ParamState)``),\n",
    "   leaving dynamical state alone.\n",
    "\n",
    "The recipe is therefore: build the network, define a loss that runs it with\n",
    "``for_loop`` and reduces over time, then take its gradient with respect to the\n",
    "parameters — all inside a single ``jit``-compiled train step.\n",
    "\n",
    "> **Memory warning.** Plain BPTT stores the activations of *every* unrolled step for\n",
    "> the backward pass, so peak memory grows with rollout length. For long rollouts use\n",
    "> ``brainstate.transform.checkpointed_for_loop`` / ``checkpointed_scan`` (§4)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36696ed2",
   "metadata": {},
   "source": [
    "## 3. A small runnable trainable example\n",
    "\n",
    "A three-layer SNN — input → recurrent LIF → readout — trained to classify random\n",
    "spike patterns. This is the canonical training pattern: a ``loss_fn`` that simulates\n",
    "with ``for_loop`` and averages the readout over time, a ``jit``-compiled\n",
    "``train_step`` that resets state, takes gradients, and steps the optimizer, and a\n",
    "plain-Python **outer epoch loop** (which is allowed — the rule against bare loops is\n",
    "about time-stepping the model, not about optimization)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "4a1bbc64",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-17T09:10:53.282042Z",
     "iopub.status.busy": "2026-06-17T09:10:53.281653Z",
     "iopub.status.idle": "2026-06-17T09:10:55.992814Z",
     "shell.execute_reply": "2026-06-17T09:10:55.991811Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAEiCAYAAABkykQ1AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAYulJREFUeJzt3Xd4VFX6B/DvzCSZSe+9J0DoCQQIoYVqQKQoSJGVAK4oVZoiuhTBFYVVUXEB+aFgWaWosApEIBQpoYViaAkJ6ZDeezJzfn9kGRmTQMIkmYR8P89zHzLnnnvnvXcuyX3nnHuORAghQEREREREpAWprgMgIiIiIqKWj4kFERERERFpjYkFERERERFpjYkFERERERFpjYkFERERERFpjYkFERERERFpjYkFERERERFpjYkFERERERFpjYkFERERERFpjYkFEVErdfz4cUgkEhw/flxdNm3aNHh4eOgsJmpeBg4ciIEDB+o6DCJqIZhYEBHV4saNG1i1ahXi4+N1HcoTpb7nddWqVZBIJOpFKpXC0dERzzzzDM6ePatRNz4+XqOuTCaDm5sbnn32WVy5cgVAVfL0YJ3alvrUa8ma63V+6tQpjBgxAs7OzlAoFHBzc8OoUaPwn//8R6Pe/c/hww8/rLaP7du3QyKR4OLFi+qy+9eTvb09iouLq23j4eGBZ555puEPiKgV0NN1AEREzdWNGzfwzjvvYODAga3mW/ytW7dCpVI16ns87nndtGkTTExMoFKpkJSUhK1bt2LAgAE4f/48/Pz8NOpOnjwZTz/9NJRKJW7evIlNmzbh4MGDOHv2LF555RUMHTpUXTcuLg4rVqzAzJkz0b9/f3W5t7d3neu1ZA/7PA4dOqSTmHbv3o2JEyfCz88Pr732GiwtLREXF4fff/8dW7duxQsvvFBtm/Xr12PWrFkwMjKq03ukp6dj06ZNWLx4cUOHT9RqMbEgIp0qLS2FgYEBpNLqDahFRUUwNjbWQVTN28POmbb09fUbfJ8NZfz48bCxsVG/Hjt2LDp37ozdu3dXSyy6d++Ov/3tb+rXffv2xejRo7Fp0yZs2bIFgYGB6nUXL17EihUrEBgYqLENgDrXe1IZGBjo5H1XrVqFjh074uzZs9ViSE9Pr1bfz88PV65cwebNm7Fo0aI6vYefnx/Wr1+P2bNnw9DQsEHiJmrt2BWKiB6qoKAACxYsgIeHB+RyOezs7DBs2DBcunRJXcfDw6PG7iB/7Z99v0//Dz/8gH/84x9wdnaGkZER8vPzMW3aNJiYmCA2NhZPP/00TE1NMWXKFABVCcbixYvh6uoKuVwOHx8f/Otf/4IQQuP9SkpKMH/+fNjY2MDU1BSjR49GSkoKJBIJVq1apa6XkJCA2bNnw8fHB4aGhrC2tsbzzz+v0RVk+/bteP755wEAgwYNUne3ePB5hIMHD6J///4wNjaGqakpRo4cievXr9f53H7++efw8vKCoaEhevXqhZMnT9brnGVnZ2PJkiXo0qULTExMYGZmhhEjRuDq1avV3is5ORljx46FsbEx7OzssHDhQpSVlVWrV9MzFiqVChs2bECnTp2gUChgb2+PV155BTk5ORr17nchOXXqFHr16gWFQgEvLy98/fXX9TqvdeXg4AAA0NN79HdkgwcPBlDV6tDUjh49qr5OLCwsMGbMGNy8eVOjzv3uOTExMZg2bRosLCxgbm6O6dOn19hd569OnjyJ559/Hm5ubpDL5XB1dcXChQtRUlLy0O0e9XnUdj3u2rUL//znP+Hi4gKFQoEhQ4YgJiZGXW/lypXQ19dHRkZGtfecOXMmLCwsUFpaWmtcsbGx6NmzZ42JjZ2dXbWyvn37YvDgwVi3bt0jj/m+FStWIC0tDZs2bapTfSJ6NLZYENFDvfrqq9izZw/mzp2Ljh07IisrC6dOncLNmzfRvXv3x9rnmjVrYGBggCVLlqCsrEx981BZWYng4GD069cP//rXv2BkZAQhBEaPHo1jx47hpZdegp+fH3777Te8/vrrSElJwccff6ze77Rp07Br1y68+OKL6N27N06cOIGRI0dWe/8LFy7gzJkzmDRpElxcXBAfH49NmzZh4MCBuHHjBoyMjDBgwADMnz8fn376Kd566y106NABANT/fvPNNwgJCUFwcDA++OADFBcXY9OmTejXrx8uX778yC4+mzZtwty5c9G/f38sXLgQ8fHxGDt2LCwtLeHi4lKnc3bjxg3s3bsXzz//PDw9PZGWloYtW7YgKCgIN27cgJOTE4CqhGvIkCFITEzE/Pnz4eTkhG+++QZHjx6t0+f1yiuvYPv27Zg+fTrmz5+PuLg4bNy4EZcvX8bp06c1WjliYmIwfvx4vPTSSwgJCcGXX36JadOmwd/fH506dXrkeX2Y7OxsAFWJTkpKCtasWQOFQoEJEyY8ctvY2FgAgLW1dZ2OuaEcOXIEI0aMgJeXF1atWoWSkhJ89tln6Nu3Ly5dulTtOpkwYQI8PT2xdu1aXLp0Cf/3f/8HOzs7fPDBBw99n927d6O4uBizZs2CtbU1zp8/j88++wzJycnYvXt3rds97ufx/vvvQyqVYsmSJcjLy8O6deswZcoUnDt3DgDw4osvYvXq1di5cyfmzp2r3q68vBx79uzBuHHjoFAoat2/u7s7wsLCkJycXOP/h5qsWrUKAwYMwKZNm+rUatG/f391MjJr1iy2WhA1BEFE9BDm5uZizpw5D63j7u4uQkJCqpUHBQWJoKAg9etjx44JAMLLy0sUFxdr1A0JCREAxJtvvqlRvnfvXgFAvPvuuxrl48ePFxKJRMTExAghhIiIiBAAxIIFCzTqTZs2TQAQK1euVJf99b2FECI8PFwAEF9//bW6bPfu3QKAOHbsmEbdgoICYWFhIV5++WWN8tTUVGFubl6t/K/KysqEtbW16Nmzp6ioqFCXb9++XQCo8zkrLS0VSqVSoywuLk7I5XKxevVqddmGDRsEALFr1y51WVFRkWjTpk214wsJCRHu7u7q1ydPnhQAxHfffafxPqGhodXK3d3dBQDx+++/q8vS09OFXC4XixcvVpfVdl5rs3LlSgGg2mJhYSFCQ0OrHT8A8c4774iMjAyRmpoqjh8/Lrp16yYAiB9//LHa/i9cuCAAiK+++uqhcdS13oP8/PyEnZ2dyMrKUpddvXpVSKVSMXXq1GrHOGPGDI3tn332WWFtbf3I96npml67dq2QSCQiISHhods+7POo7f9whw4dRFlZmbr8k08+EQBEZGSkuiwwMFAEBARo7O+nn36q02e/bds2AUAYGBiIQYMGieXLl4uTJ09Wu96FEAKA+nfUoEGDhIODg/p8fPXVVwKAuHDhgrr+/XOdkZEhTpw4IQCIjz76SL3e3d1djBw58qHxEVHN2BWKiB7KwsIC586dw927dxtsnyEhIbV+Ozhr1iyN1wcOHIBMJsP8+fM1yhcvXgwhBA4ePAgACA0NBQDMnj1bo968efOqvceD711RUYGsrCy0adMGFhYWGl28anP48GHk5uZi8uTJyMzMVC8ymQwBAQE4duzYQ7e/ePEisrKy8PLLL2t045kyZQosLS1r3KamcyaXy9XPWSiVSmRlZcHExAQ+Pj4ax3HgwAE4Ojpi/Pjx6jIjIyPMnDnzkce6e/dumJubY9iwYRrH6u/vDxMTk2rH2rFjR40Hm21tbeHj44M7d+488r0e5ccff8Thw4dx6NAhfPXVV2jXrh3GjRuHM2fOVKu7cuVK2NrawsHBAQMHDkRsbCw++OADPPfcc1rHUVf37t3DlStXMG3aNFhZWanLu3btimHDhuHAgQPVtnn11Vc1Xvfv3x9ZWVnIz89/6Hs9eG0UFRUhMzMTffr0gRACly9f1vJIqps+fbpGN6X7n/mDn/PUqVNx7tw5dWsRAHz33XdwdXVFUFDQQ/c/Y8YMhIaGYuDAgTh16hTWrFmD/v37o23btjV+3vetWrUKqamp2Lx5c52OY8CAARg0aFC9ulARUe2YWBDRQ61btw7Xrl2Dq6srevXqhVWrVml9k+jp6VljuZ6eXrVuDwkJCXBycoKpqalG+f2uGgkJCep/pVJptX23adOm2vuUlJRgxYoV6mc2bGxsYGtri9zcXOTl5T0y/tu3bwOo6rdva2ursRw6dEj9cGlJSQlSU1M1lgdj/mtsenp6tXahqumcqVQqfPzxx2jbtq3Gcfzxxx8ax5GQkIA2bdpAIpFobO/j41OnY83Ly4OdnV21Yy0sLKz2IK2bm1u1fVhaWlZ7HuNxDBgwAEOHDsWwYcMwbdo0hIWFwdTUtMbkcebMmTh8+DDCwsIQERGB9PR0vPHGG1rHUB/3P+eaznOHDh2QmZmJoqIijfK/nr/7ieajzl9iYqI6gTExMYGtra365r0u13R91SXOiRMnQi6X47vvvlPH8euvv2LKlCnVrsWaBAcH47fffkNubi5+//13zJkzBwkJCXjmmWdqfIAbeLxEob7JCBHVjs9YENFDTZgwAf3798fPP/+MQ4cOYf369fjggw/w008/YcSIEQBQ602CUqmETCarVl5ba8WD38A3pnnz5uGrr77CggULEBgYCHNzc0gkEkyaNKlOQ63er/PNN9+oHyB+0P1WiJ07d2L69Oka68RfHjivq5rO2XvvvYfly5djxowZWLNmDaysrCCVSrFgwYIGGzJWpVLBzs5OfXP4V7a2thqva/q8gcc/7ocxMTFBQEAA9u3bV20EsbZt22oMFdtSPM75UyqVGDZsGLKzs7F06VK0b98exsbGSElJwbRp0xpl+OC6xGlpaYlnnnkG3333HVasWIE9e/agrKys3iNqGRkZoX///ujfvz9sbGzwzjvv4ODBgwgJCamx/sqVKzFw4EBs2bIFFhYWj9z/gAEDMHDgQKxbt65aixER1Q8TCyJ6JEdHR8yePRuzZ89Geno6unfvjn/+85/qxMLS0hK5ubnVtktISICXl5dW7+3u7o4jR46goKBAo9Xi1q1b6vX3/1WpVIiLi0Pbtm3V9R4cqea+PXv2ICQkRGNCrdLS0mrHUFvCdH/eAjs7u4fevAYHB+Pw4cM1HtP92AYNGqQur6ysRHx8PLp27VrrPv96HIMGDcK2bds0ynNzczWGZXV3d8e1a9cghNA4pqioqEe+h7e3N44cOYK+ffs22MOtdfm2uq4qKysBAIWFhc1uaOL7n3NN5/nWrVuwsbFpkJgjIyMRHR2NHTt2YOrUqerymq69mjTk5/FXU6dOxZgxY3DhwgV899136NatGzp16vTY++vRoweAqm5mtQkKCsLAgQPxwQcfYMWKFXXa76pVq9TJCBE9PnaFIqJaKZXKat0o7Ozs4OTkpDFUqbe3N86ePYvy8nJ12a+//oqkpCStY7g/ydnGjRs1yj/++GNIJBJ1chMcHAwA+Pe//61R77PPPqu2T5lMVu0b4M8++wxKpVKj7P5N318TjuDgYJiZmeG9995DRUVFtf3fH2LT0dERQ4cO1ViAqpsja2trbN26VX1jDFT1P69Pl6GajmP37t1ISUnRKHv66adx9+5d7NmzR11WXFyML7744pHvMWHCBCiVSqxZs6bausrKyhoTykep7bzWV3Z2Ns6cOQMHB4cahyDVNUdHR/j5+WHHjh0ax3rt2jUcOnQITz/9dIO8z/3WgwevBSEEPvnkkzpt31CfR01GjBgBGxsbfPDBBzhx4kSdWyvCwsJqLL//XMqjuvHd795Ul2sc0ExGHjYMLhE9HFssiKhWBQUFcHFxwfjx4+Hr6wsTExMcOXIEFy5c0Pi2/+9//zv27NmD4cOHY8KECYiNjcW3337bIDMSjxo1CoMGDcLbb7+N+Ph4+Pr64tChQ9i3bx8WLFigfg9/f3+MGzcOGzZsQFZWlnq42ejoaACa38o+88wz+Oabb2Bubo6OHTsiPDwcR44cqTYUqZ+fH2QyGT744APk5eVBLpdj8ODBsLOzw6ZNm/Diiy+ie/fumDRpEmxtbZGYmIj9+/ejb9++1RKhBxkYGGDVqlWYN28eBg8ejAkTJiA+Ph7bt2+Ht7d3nb9BfuaZZ7B69WpMnz4dffr0QWRkJL777rtqrUQvv/wyNm7ciKlTpyIiIgKOjo745ptv6jRDcVBQEF555RWsXbsWV65cwVNPPQV9fX3cvn0bu3fvxieffKLxUHhdPOy8PsyePXtgYmICIQTu3r2Lbdu2IScnB5s3b27Ub921sX79eowYMQKBgYF46aWX1MPNmpuba8ytoo327dvD29sbS5YsQUpKCszMzPDjjz/WOUl93M+jLvT19TFp0iRs3LgRMpkMkydPrtN2Y8aMgaenJ0aNGgVvb28UFRXhyJEj+OWXX9CzZ0+MGjXqodsHBQUhKCgIJ06cqHOsK1eu1GhBJKLHoJvBqIioJSgrKxOvv/668PX1FaampsLY2Fj4+vqKf//739Xqfvjhh8LZ2VnI5XLRt29fcfHixVqHqty9e3e17UNCQoSxsXGNcRQUFIiFCxcKJycnoa+vL9q2bSvWr18vVCqVRr2ioiIxZ84cYWVlJUxMTMTYsWNFVFSUACDef/99db2cnBwxffp0YWNjI0xMTERwcLC4detWjcPmbt26VXh5eQmZTFZtmMxjx46J4OBgYW5uLhQKhfD29hbTpk0TFy9erMPZFeLTTz8V7u7uQi6Xi169eonTp08Lf39/MXz48Dqds9LSUrF48WLh6OgoDA0NRd++fUV4eHi18y6EEAkJCWL06NHCyMhI2NjYiNdee009ZOzDhpu974svvhD+/v7C0NBQmJqaii5duog33nhD3L17V12ntmE6a4rnYef1r2oabtbY2FgEBgZqDKErxJ/Dza5fv77W/f1VYw43K4QQR44cEX379hWGhobCzMxMjBo1Sty4cUOjzoNDoD7o/nCpcXFxD32PGzduiKFDhwoTExNhY2MjXn75ZXH16tU6x1vb51HX/8P3z3tN73X+/HkBQDz11FOPjOO+77//XkyaNEl4e3sLQ0NDoVAoRMeOHcXbb78t8vPzNerigeFmH3Q/VjxkuNm/CgoKEgA43CzRY5II0QhP1BERNRNXrlxBt27d8O2336pn8m6uVCoVbG1t8dxzz2Hr1q26DoeoQVy9ehV+fn74+uuv8eKLL+o6HCJqRHzGgoieGDUNL7lhwwZIpVIMGDBABxHVrrS0tNrzEV9//TWys7MxcOBA3QRF1Ai2bt0KExOTJp1DhIh0g89YENETY926dYiIiMCgQYOgp6eHgwcP4uDBg5g5cyZcXV11HZ6Gs2fPYuHChXj++edhbW2NS5cuYdu2bejcuTOef/55XYdHpLVffvkFN27cwBdffIG5c+c2u1G7iKjhsSsUET0xDh8+jHfeeQc3btxAYWEh3Nzc8OKLL+Ltt9/WmOG6OYiPj8f8+fNx/vx5ZGdnw8rKCk8//TTef//9ZjnCEVF9eXh4IC0tDcHBwfjmm2+qTXJJRE8eJhZERERERKQ1PmNBRERERERaY2JBRERERERaa16djpsJlUqFu3fvwtTUtNlOukRERERE1NiEECgoKICTkxOk0ke0SehuCo0/bdy4UWOSqHPnztVa9/7kNX9dnn76aXUdlUolli9fLhwcHIRCoRBDhgwR0dHRdY4nKSmpxvfgwoULFy5cuHDhwqU1LklJSY+8h9Z5i8XOnTuxaNEibN68GQEBAdiwYQOCg4MRFRVV48goP/30E8rLy9Wvs7Ky4OvrqzE847p16/Dpp59ix44d8PT0xPLlyxEcHIwbN25AoVA8Mqb7I1ckJSXBzMysAY6SiIiIiKjlyc/Ph6ura51GdtP5qFABAQHo2bMnNm7cCKCqG5KrqyvmzZuHN99885Hbb9iwAStWrMC9e/dgbGwMIQScnJywePFiLFmyBACQl5cHe3t7bN++HZMmTXrkPvPz82Fubo68vDwmFkRERETUatXnvlinD2+Xl5cjIiICQ4cOVZdJpVIMHToU4eHhddrHtm3bMGnSJPXEO3FxcUhNTdXYp7m5OQICAuq8TyIiIiIiqh+ddoXKzMyEUqmEvb29Rrm9vT1u3br1yO3Pnz+Pa9euYdu2beqy1NRU9T7+us/76/6qrKwMZWVl6tf5+fl1PgYiIiIiImrhw81u27YNXbp0Qa9evbTaz9q1a2Fubq5eXF1dGyhCIiIiIqLWQaeJhY2NDWQyGdLS0jTK09LS4ODg8NBti4qK8MMPP+Cll17SKL+/XX32uWzZMuTl5amXpKSk+h4KEREREVGrptPEwsDAAP7+/ggLC1OXqVQqhIWFITAw8KHb7t69G2VlZfjb3/6mUe7p6QkHBweNfebn5+PcuXO17lMul8PMzExjISIiIiKiutP5cLOLFi1CSEgIevTogV69emHDhg0oKirC9OnTAQBTp06Fs7Mz1q5dq7Hdtm3bMHbsWFhbW2uUSyQSLFiwAO+++y7atm2rHm7WyckJY8eObarD0srByHvILi6Hn6sFfOxNoSdr0T3WiIiIiKgV0HliMXHiRGRkZGDFihVITU2Fn58fQkND1Q9fJyYmVpvlLyoqCqdOncKhQ4dq3Ocbb7yBoqIizJw5E7m5uejXrx9CQ0PrNIdFc/B1eALC72QBAAz1ZejoZAZLI33I9WWQy6SoUAkUlVWiqKwSxeVKFJVXorhMibJKJWRSKfSkEsikEujLqv7Vk0qhJ5Ooy/Vkf9bRe+C1VCqBEAKVSgGVEFCqBCpVVT8LAUgkgASSqn8lEkhwvwyQSqrK8b/1Uo26tW8H/FlW9aKqLh6oc3/y8wdq6cSjJmF/1Hp9mRSmCj2YKvRhLNeD/l8+i6rPSwpDfRlMFHowketBrlf12ckkEhjJq8qIiIiImiOdz2PRHOl6HovPj8UgPDYLV5NyUVBW2eTvT82Xt60xurtZooOjGVLzSxGVWoA7mYVwNDNEv7Y26NvGBr4u5mzlIiIiogZRn/tiJhY10HVicZ9KJXAnsxA37hWguKwSpRVKlFWqoC+Twlgug5FB1TfYRgYyGMv1YKAnhVL1Z0tDpVKFynq8VipVkP7v23OZpKoFQyapei2RAEIAAlWtF1U/A6r//aAu/1/Z/Z8hRNU88EKz/MHL7v6P9/dR9fOf7/dgneaqLv+NyipVKCitRH5pBYrKKqFUCVQoxf/+VVX9qxIoKa9EUZkShf/7zO+3HqnqeA4M9KRoa2cCH3tTdHExx3h/F5gq9LU8QiIiImqNmFhoqbkkFkQPyi4qx+XEHFxKzEF0WiGcLQzRzt4UXrbGuJNRhFMxGTgdk4W8kgqN7SyM9PFyfy9M6+MBY3alIiIionpgYqElJhbUUqlUAsk5JbiVmo+o1ALsvZKC2IwiAICVsQFGdHbA4PZ26ONtA0MDmY6jJSIiouaOiYWWmFjQk0KpEvjl6l18EnYbcZlF6nK5nhRzBrXB/CFtdRgdERERNXf1uS9mvwiiJ5hMKsHYbs54pqsjTt7ORNitNBy7lYGU3BJ8dDgaZgo9TOvrqeswiYiI6AnAxIKoFdCTSTGovR0GtbeDEAKfH4vBvw5F451fb8DJwhBPdXr4TPdEREREj8IxKYlaGYlEgjmD2mByLzcIAcz/4TIuJ+boOiwiIiJq4ZhYELVCEokEa8Z0wiAfW5RWqDBj+wVcS8nTdVhERETUgjGxIGql9GRSbHyhO3xdLZBTXIHJW88iIoEtF0RERPR4mFgQtWLGcj18+1Iv9PSwREFpJV7cdg5nYjN1HRYRERG1QEwsiFo5U4U+dszohf5tbVBcrsT0ry7g2K10XYdFRERELQwTCyKCkYEetk7tgaEd7FBWqcLMby7iYOQ9XYdFRERELQgTCyICACj0Zdj0N38809URFUqBud9fxs+Xk3UdFhEREbUQnMeCiNT0ZVJ8MqkbFPoy7IlIxqJdV7H7YjIGtLNF/7Y26OhoBolEouswiYiIqBmSCCGEroNobuozdTnRk0ilElj96w1sPxOvUe5ta4xpfTzwXHcXGMv5vQQREdGTrj73xUwsasDEgqhKXGYRTkSl4+TtTJyJzUJJhRIAYCrXw6RernipnxcczBU6jpKIiIgaCxMLLTGxIKquoLQCP0YkY0d4AuIyiwAABjIpxvm74NUgL7hbG+s4QiIiImpoTCy0xMSCqHYqlcCJ6AxsOh6L8/HZAAADPSm+ntELvb2sdRwdERERNaT63BdzVCgiqhepVIJB7e2w69VA7H41EL08rVBeqcLs7y4hOadY1+ERERGRjjCxIKLH1tPDCjum90JnZzNkF5Vj5tcRKC6v1HVYREREpANMLIhIK4YGMmx5sQdsTAxw414+Xt/zB9jDkoiIqPVhYkFEWnO2MMSmv/lDXybB/j/uYfm+a6hUqnQdFhERETUhJhZE1CB6eljhn892gUQCfHs2ETO/iUBRGbtFERERtRZMLIiowUzo4YpNU7pDrifF0VvpmLAlHJHJeVCq/uwalZpXim/PJuCr03Fs1SAiInqCcLjZGnC4WSLtXE7Mwd93XERWUTmAqgn1/D0skVVYjsiUPHW95/1d8MG4rpBKJboKlYiIiB6Cw80SkU51c7PEz7P7YmgHe5jI9VBQVonjURmITMmDRAJ0dTGHVALsjkjGmv03+LA3ERHRE0DnicXnn38ODw8PKBQKBAQE4Pz58w+tn5ubizlz5sDR0RFyuRzt2rXDgQMH1OtXrVoFiUSisbRv376xD4OI/sLN2gj/F9IDV1YMw6/z+mHVqI5YN74rzr81FP+d2w/rxvsCAL46HY+Pj9zWcbRERESkLT1dvvnOnTuxaNEibN68GQEBAdiwYQOCg4MRFRUFOzu7avXLy8sxbNgw2NnZYc+ePXB2dkZCQgIsLCw06nXq1AlHjhxRv9bT0+lhErVqejIpOjubo7OzuUb5eH8XFJVVYuV/r+PTsNsoKqvEmyPaQ1+m8+87iIiI6DHo9I77o48+wssvv4zp06cDADZv3oz9+/fjyy+/xJtvvlmt/pdffons7GycOXMG+vr6AAAPD49q9fT09ODg4NCosROR9kL6eKC4XIkPQm9h26k4RCbnYeOUbrAzVeg6NCIiIqonnX01WF5ejoiICAwdOvTPYKRSDB06FOHh4TVu89///heBgYGYM2cO7O3t0blzZ7z33ntQKpUa9W7fvg0nJyd4eXlhypQpSExMbNRjIaLHN2ugNzb/rTtM5Ho4H5+NkZ+ewumYTF2HRURERPWks8QiMzMTSqUS9vb2GuX29vZITU2tcZs7d+5gz549UCqVOHDgAJYvX44PP/wQ7777rrpOQEAAtm/fjtDQUGzatAlxcXHo378/CgoKao2lrKwM+fn5GgsRNZ3hnR2xb25ftLM3QUZBGab83zks3HkFmYVlug6NiIiI6qhFdWZWqVSws7PDF198AX9/f0ycOBFvv/02Nm/erK4zYsQIPP/88+jatSuCg4Nx4MAB5ObmYteuXbXud+3atTA3N1cvrq6uTXE4RPQAb1sT/Dy7L6YGukMiAX6+nIIhH57A7otJug6NiIiI6kBniYWNjQ1kMhnS0tI0ytPS0mp9PsLR0RHt2rWDTCZTl3Xo0AGpqakoLy+vcRsLCwu0a9cOMTExtcaybNky5OXlqZekJN7IEOmCsVwPq8d0xs+z+6KjoxnySirw+p4/8MaeqyitUD56B0RERKQzOkssDAwM4O/vj7CwMHWZSqVCWFgYAgMDa9ymb9++iImJgUr152y90dHRcHR0hIGBQY3bFBYWIjY2Fo6OjrXGIpfLYWZmprEQke74uVrgv3P7YslT7SCVALsuJmPcpjNIyi7WdWhERERUC512hVq0aBG2bt2KHTt24ObNm5g1axaKiorUo0RNnToVy5YtU9efNWsWsrOz8dprryE6Ohr79+/He++9hzlz5qjrLFmyBCdOnEB8fDzOnDmDZ599FjKZDJMnT27y4yOix6cnk2Lu4Lb4ekYArIwNcP1uPkZtPIUL8dm6Do2IiIhqoNPhZidOnIiMjAysWLECqamp8PPzQ2hoqPqB7sTEREilf+Y+rq6u+O2337Bw4UJ07doVzs7OeO2117B06VJ1neTkZEyePBlZWVmwtbVFv379cPbsWdja2jb58RGR9vq1tcEv8/ph9rcRuJqchyn/dw6fTPTDiC61t0ISERFR05MIIYSug2hu8vPzYW5ujry8PHaLImomSsqVmPf9ZRy5mQaJBFjxTEdM6+MBiUSi69CIiIieWPW5L2ZiUQMmFkTNU6VShZX/vY7vzlXNTeNhbYSRXR3xTFcntHcwZZJBRETUwJhYaImJBVHzJYTAF7/fwcdHolFa8edADkHtbPHu2M5wtTLSYXRERERPFiYWWmJiQdT8FZVVIuxWOn69ehfHotJRoRQw1Jdh4bC2mNHXE3qyFjVNDxERUbPExEJLTCyIWpbYjEK89VMkzsVVjRil0JfCydwQDuYKWJvIodCTQq4vhbFcD142xmhrb4q2diYwVejrOHIiIqLmjYmFlphYELU8QgjsvpiM9w7eRG5xxSPrSyTAtD4e+MfIjpBJ+WwGERFRTZhYaImJBVHLVV6pwr28EtzNLcW9vBJkF5WjXKlCWYUKeSUViEkvRHRaAdILygAAwzra49NJ3WBoINNx5ERERM0PEwstMbEgevLt/+MeFu66gvJKFfxcLbB1ag/Ymsp1HRYREVGzwsRCS0wsiFqHC/HZ+PuOi8grqYBEAvjYm8Lf3RKDfOwwpIMdh68lIqJWj4mFlphYELUeMemFmPf9Zdy8l69R/mw3Z7w7tjOM5Xo6ioyIiEj3mFhoiYkFUeuTnl+KS4k5OBObhe/OJUKpEvC2NcbnU7qjvQN/DxARUevExEJLTCyIWrfzcdmY//1lpOaXQq4nxeoxnTChhyu7RhERUatTn/tiziBFRPQXvTytsH9+PwS1s0VZpQpLf4zEwp1XUFRWqevQiIiImi22WNSALRZEBAAqlcCW3+/gX4eioFQJeNkaY2pvd/g4mKG9gyksjQ10HSIREVGjatKuUPn5+Th69Ch8fHzQoUMHbXbVbDCxIKIHXYjPxrz/VHWNelBnZzO82Nsdo32dOQ8GERE9kRo1sZgwYQIGDBiAuXPnoqSkBL6+voiPj4cQAj/88APGjRunVfDNARMLIvqr7KJyfB0ej2sp+YhKy0dSdol6nbmhPkb5OqKnhxW6u1nCxdKQz2MQEdEToVETCwcHB/z222/w9fXFf/7zH6xcuRJXr17Fjh078MUXX+Dy5ctaBd8cMLEgokfJLirH7otJ+OZsApJzSjTW2ZjI0c3NAt3cLODvZomeHlaQSploEBFRy9OoiYWhoSGio6Ph6uqKqVOnwsnJCe+//z4SExPRsWNHFBYWahV8c8DEgojqSqkSOBGdjt+jM3E5MQfX7+ajUqX5azXQyxqfTPaDnalCR1ESERE9nvrcF9d75idXV1eEh4fDysoKoaGh+OGHHwAAOTk5UCj4R5OIWheZVILB7e0xuL09AKC0QolrKXm4nJiLS4k5OB6VgfA7WRj56Sl8NrkbentZ6zhiIiKixlHvxGLBggWYMmUKTExM4O7ujoEDBwIAfv/9d3Tp0qWh4yMialEU+jL08LBCDw8rAEBMegFmf3cJ0WmFeGHrWTzX3QW9vazR08MSblZGfBaDiIieGI81KtTFixeRlJSEYcOGwcTEBACwf/9+WFhYoG/fvg0eZFNjVygiakjF5ZX4x95r+OlSika5i6UhxnV3wXh/F7haGekoOiIioto16XCzSqUSkZGRcHd3h6WlpTa7ajaYWBBRQxNC4HRMFk7ezsDFhBz8kZyLCuWfv357eVihq4s5fBxM0cHRDB0czSDjA99ERKRjjZpYLFiwAF26dMFLL70EpVKJoKAgnDlzBkZGRvj111/VXaNaMiYWRNTYSsqVOHQjFbsvJuN0bCb++pvY3FAfA9rZYmA7WwxoZwtbU7luAiUiolatURMLFxcX7N27Fz169MDevXsxZ84cHDt2DN988w2OHj2K06dPaxV8c8DEgoiaUnJOMU7ezkRUagGiUgtw7W4eCkorNep0cTbHQB9bDPSxhZ+rJVsziIioSTRqYqFQKBATEwMXFxfMnDkTRkZG2LBhA+Li4uDr64v8/Hytgm8OmFgQkS5VKlW4kpSL41EZOB6djmspmr9X2ZpBRERNpVGHm7W3t8eNGzfg6OiI0NBQbNq0CQBQXFwMmUz2eBETEZGankyqHllqSbAP0gtK8Xt0Jo5FpeNkdAbySirwy9W7+OXqXQBVrRmDfGzxXHcXeNgY6zh6IiJqreqdWEyfPh0TJkyAo6MjJBIJhg4dCgA4d+4c2rdv3+ABEhG1dnamCoz3rxo96sHWjGNR6bh+Nx+RKXmITMnDp0djMKCdLV7s7Y6BPrbQl0l1HToREbUijzUq1J49e5CUlITnn38eLi4uAIAdO3bAwsICY8aMafAgmxq7QhFRS5FeUIoTURnYH3kPJ6Iz1A+BG8ikaOdggo6OZghqZ4cRnR0g5XMZRERUT/W5L36sr7PGjx+PhQsXqpMKAAgJCXmspOLzzz+Hh4cHFAoFAgICcP78+YfWz83NxZw5c+Do6Ai5XI527drhwIEDWu2TiKilsjNV4Pkertg+vReOLxmIVwZ4wcrYAOVKFa6l5GPXxWTM+c8ljNt8BleScnUdLhERPcEeq8XixIkT+Ne//oWbN28CADp27IjXX38d/fv3r9d+du7cialTp2Lz5s0ICAjAhg0bsHv3bkRFRcHOzq5a/fLycvTt2xd2dnZ466234OzsjISEBFhYWMDX1/ex9lkTtlgQUUumUgkk55Tg+t08XE7KxXdnE1BUrgQAjPZ1wrCO9gjwtIKdmULHkRIRUXPXqKNCffvtt5g+fTqee+459Szbp0+fxs8//4zt27fjhRdeqPO+AgIC0LNnT2zcuBEAoFKp4Orqinnz5uHNN9+sVn/z5s1Yv349bt26BX19/QbZZ02YWBDRkyQtvxQfhN6qNvO3u7URXC2NYGcmh72ZAl42xmjvYIa29iZQ6HMwDiIiauTEokOHDpg5cyYWLlyoUf7RRx9h69at6laMRykvL4eRkRH27NmDsWPHqstDQkKQm5uLffv2Vdvm6aefhpWVFYyMjLBv3z7Y2trihRdewNKlSyGTyR5rnwBQVlaGsrIy9ev8/Hy4uroysSCiJ8rVpFzsvZKCc3eycTM1v9qkfPdJJUAXFws8180Zo3ydYGVs0LSBEhFRs9Gow83euXMHo0aNqlY+evRovPXWW3XeT2ZmJpRKJezt7TXK7e3tcevWrVrf++jRo5gyZQoOHDiAmJgYzJ49GxUVFVi5cuVj7RMA1q5di3feeafOsRMRtUS+rhbwdbUAAOSVVOBaSh5S80qRVlCK1LxS3E4rxK3UfOQUV+BqUi6uJuXi3f03ENTODk93ccCQ9vYwN/qztVgIAYmED4QTEVGVeicWrq6uCAsLQ5s2bTTKjxw5AldX1wYLrCYqlQp2dnb44osvIJPJ4O/vj5SUFKxfvx4rV6587P0uW7YMixYtUr++32JBRPSkMjfUR982NtXKhRBIzS/FwchU/HQ5GddS8nHkZhqO3EyDnlQCX1cLlFYokZZfhuyiMrhaGaGHuxV6eljCz80CXjYmMNDjMLdERK1RvROLxYsXY/78+bhy5Qr69OkDoOoZi+3bt+OTTz6p835sbGwgk8mQlpamUZ6WlgYHB4cat3F0dIS+vr7GRHwdOnRAamoqysvLH2ufACCXyyGXc+ZaIiKJRAJHc0PM6OeJGf08EZVagP1/3MVv19MQlVaAiIQcjfoJWcVIyCrGj5eSAQD6Mgm8bU3gbWsCcyN9mCn0YWGkDw9rI7SxM4GblTETDyKiJ1S9E4tZs2bBwcEBH374IXbt2gWg6uZ+586d9Rpu1sDAAP7+/ggLC1M/D6FSqRAWFoa5c+fWuE3fvn3xn//8ByqVClJp1R+m6OhoODo6wsCgqg9wffdJRES183EwhY+DDxY95YP4zCJcTsqBhaEB7MzksDQyQHRaAS7G5+BCfDZu3M1HQVklbqUW4FZqQY3705NKMLSDPeYOboPOzuZNfDRERNSYHmu42Yayc+dOhISEYMuWLejVqxc2bNiAXbt24datW7C3t8fUqVPh7OyMtWvXAgCSkpLQqVMnhISEYN68ebh9+zZmzJiB+fPn4+23367TPuuCo0IREdWfEFXD3N68l4/knBIUlFYiv7QCWYVluJNZhNj0QvWwtwAQ1M4Wwzs7QK4nhYGeFK6WRupnQIiIqHlo1Ie3G9LEiRORkZGBFStWIDU1FX5+fggNDVUnAImJieqWCaDq+Y7ffvsNCxcuRNeuXeHs7IzXXnsNS5curfM+iYiocUgkErhaGcHVyqjG9UII3EotwJYTsfjv1bs4EZ2BE9EZGnWGdbTH8pEd4WZd8z6IiKj5qlOLhaWlZZ1H/sjOztY6KF1jiwURUeNKyCrC9jPxSMouRlmlCmUVKkQk5kCpEjDQk+KVAV6YPbANDA04nwYRkS41+DwWO3bsqPObh4SE1Lluc8XEgoio6d1OK8CqX67jdEwWAMDJXIG3R3bE010cOKwtEZGONOoEea0BEwsiIt0QQiD0Wire3X8TKbklAIBAL2u8PMATvb2sYWSg0x68REStDhMLLTGxICLSrZJyJTafiMXmE7Eoq1QBAAxkUvTytEJQO1sE+diirZ0JWzKIiBoZEwstMbEgImoekrKL8cXvd3D0Vrq6BeM+R3MFhnSww4u9PeDjYKqjCImInmxMLLTExIKIqHkRQuBOZhFORGXg99sZCI/NUrdkAEAfb2tM7+uJwe3tIJOyFYOIqKEwsdASEwsiouattEKJs3eysOtiEkKvpUL1v79kblZGCOnjged7uMBMoa/bIImIngBMLLTExIKIqOVIyS3B1+Hx+OF8EvJKKgAAxgYyLB3RHlMDPXQbHBFRC9eoicWzzz5b48NyEokECoUCbdq0wQsvvAAfH5/6Rd2MMLEgImp5issr8fPlFGw/HY/b6YUAgJf7e2LZiA6QsnsUEdFjqc99sfSha2tgbm6Oo0eP4tKlS5BIJJBIJLh8+TKOHj2KyspK7Ny5E76+vjh9+vRjHwAREVF9GRnoYUqAOw4tHIDXg6u+3Np6Mg7zfriM9IJSxKQX4tydLFy/mwc21hMRNbx6t1i8+eabyM/Px8aNGyGVVuUlKpUKr732GkxNTfHPf/4Tr776Kq5fv45Tp041StCNjS0WREQt38+Xk/HGnj9Qoaz+Z87D2gijfJ0w2tcJbe05ohQRUW0atSuUra0tTp8+jXbt2mmUR0dHo0+fPsjMzERkZCT69++P3NzcegffHDCxICJ6MpyJycT8H64gs7AMZgo9WJvIkZpXipIKpbpOewdTjPZzwqiuTnC1MtJhtEREzU997ovrPYVpZWUlbt26VS2xuHXrFpTKql/UCoWCkxYREZHO9Wljg3NvDUGlSgW5ngxA1bMYh2+k4Zerd3EiOgO3UgtwKzQK60KjsOSpdpg7uK2OoyYiapnqnVi8+OKLeOmll/DWW2+hZ8+eAIALFy7gvffew9SpUwEAJ06cQKdOnRo2UiIioscgk0ogk8rUr40M9DDGzxlj/JyRW1yO0Gup+O/VuzgTm4V/HYqGm7UxRvs66TBiIqKWqd5doZRKJd5//31s3LgRaWlpAAB7e3vMmzcPS5cuhUwmQ2JiIqRSKVxcXBol6MbGrlBERK3P2gM3seX3O5DrSbH71UB0dbHQdUhERDrXZPNY5OfnA8ATd/PNxIKIqPVRqgRe/voijt5Kh72ZHP+d2w/2Zgpdh0VEpFOcIE9LTCyIiFqngtIKPPfvM7idXghDfRm6u1sgwNMavb2s0c3NAvqyeo/STkTUojVqYpGWloYlS5YgLCwM6enp1cYCv/8Ad0vGxIKIqPVKyCrC1C/PIyGrWKPcRK6HPt7W6ONtDRdLI9ibKWBvLoedKVs1iOjJ1aiJxYgRI5CYmIi5c+fC0dGx2uhPY8aMqX/EzQwTCyKi1k2lEojJKMS5uGycu5OFM7FZyC4qr7Gum5URgtrZIqidLfq2sYGhgazGekRELVGjJhampqY4efIk/Pz8tImxWWNiQURED1KpBK7dzcOJqAxcTc5FWn4Z0vJLkVlYBtUDf0UtjfQR0scDIYEesDQ20F3AREQNpFHnsXB1da3W/YmIiOhJJpVK0NXFotpIUUVllQiPzcKJ6AyE3UzD3bxSbDhyG1tO3MHkXm6YNdAbtqZy3QRNRNTE6t1icejQIXz44YfYsmULPDw8Giks3WKLBRER1VelUoWD11Kx+UQsrt+tGjXRUF+GGf08MKOvJ4rLlUjOKUFWURk8rI3Rzt4UBnp8GJyImrdG7QplaWmJ4uJiVFZWwsjICPr6+hrrs7Oz6x9xM8PEgoiIHpcQAidvZ+LDw9G4mpRbaz0DmRQ+Dqbo4WGJfm1sEOBlDRN5vTsSEBE1qkZNLHbs2PHQ9SEhIfXZXbPExIKIiLQlhMDhG2n48FA0otIKYKAnhbOFISyM9BGbXoj80kqN+npSCfzdLTGiswOGd3aEgzlHmyIi3eM8FlpiYkFERA1FCIH8kkqYGeqpR1IUQiApuwRXk3MRficLp2Myqw1v28vTCiue6YjOzua6CJuICEAjJBb5+fnqHd2fbbs2T8KNOBMLIiJqaknZxTh0Iw0HI+/hYkIOgKpWjDmD2mDOoDZ8HoOIdKLBEwuZTIZ79+7Bzs4OUqm02twVQNW3LxKJhBPkERERaeleXgnW/HoDByJTAQDt7E1gbSxHSm4J0vJL4etigZkDvDC4vR2k0up/k4mIGkqDDzd79OhRWFlZAQCOHTumfYR/8fnnn2P9+vVITU2Fr68vPvvsM/Tq1avGutu3b8f06dM1yuRyOUpLS9Wvp02bVu1ZkODgYISGhjZ47ERERA3N0dwQ/57ij1+u3sXyfdcQnVYIoFC9/nx8Ns7HZ6ONnQnmDW6D0b5ONX7pR0TUlOqUWAQFBdX4c0PYuXMnFi1ahM2bNyMgIAAbNmxAcHAwoqKiYGdnV+M2ZmZmiIqKUr+u6Zfp8OHD8dVXX6lfy+UcR5yIiFqWUb5O6O1ljdDrqTCRy+BsYQRzQ338fDkF351NQEx6IV774Qr2Xk7B2ue68oFvItKpx3p4Ozc3F+fPn0d6ejpUKpXGuqlTp9ZrXwEBAejZsyc2btwIAFCpVHB1dcW8efPw5ptvVqu/fft2LFiwALm5ubXuc9q0acjNzcXevXvrFct97ApFRETNXUFpBbafjsdnx2JQXqmCqUIPK57piPH+Lmy9IKIG06gzb//yyy+YMmUKCgsLYWZmpvHLSyKR1CuxKC8vR0REBJYtW6Yuk0qlGDp0KMLDw2vdrrCwEO7u7lCpVOjevTvee+89dOrUSaPO8ePHYWdnB0tLSwwePBjvvvsurK2ta9xfWVkZysrK1K8f9YA6ERGRrpkq9DFvSFuM6OKAxbv/wNWkXLy+5w9kFZXj1SBvXYdHRK1QvYeYWLx4MWbMmIHCwkLk5uYiJydHvdR3crzMzEwolUrY29trlNvb2yM1NbXGbXx8fPDll19i3759+Pbbb6FSqdCnTx8kJyer6wwfPhxff/01wsLC8MEHH+DEiRMYMWJErQ+Wr127Fubm5urF1dW1XsdBRESkK23sTPHjq4GYP7gNAOD9g7fwY0TyI7YiImp49e4KZWxsjMjISHh5eWn95nfv3oWzszPOnDmDwMBAdfkbb7yBEydO4Ny5c4/cR0VFBTp06IDJkydjzZo1Nda5c+cOvL29ceTIEQwZMqTa+ppaLFxdXdkVioiIWpR/7r+BrSfjIJNK8H8hPTDIp+ZnFYmI6qo+XaHq3WIRHByMixcvPnZwD7KxsYFMJkNaWppGeVpaGhwcHOq0D319fXTr1g0xMTG11vHy8oKNjU2tdeRyOczMzDQWIiKilmbZiA4Y6+cEpUpg9reXsPHobcSkF+g6LCJqJer9jMXIkSPx+uuv48aNG+jSpQv09fU11o8ePbrO+zIwMIC/vz/CwsIwduxYAFUPb4eFhWHu3Ll12odSqURkZCSefvrpWuskJycjKysLjo6OdY6NiIiopZFKJVg33hfZxRX4PToD/zoUjX8dioa3rTFeDfLmg91E1Kjq3RVKKq29keNxJsjbuXMnQkJCsGXLFvTq1QsbNmzArl27cOvWLdjb22Pq1KlwdnbG2rVrAQCrV69G79690aZNG+Tm5mL9+vXYu3cvIiIi0LFjRxQWFuKdd97BuHHj4ODggNjYWLzxxhsoKChAZGRknYad5ahQRETUkpVVKvHzpRSEXk/F6ZhMVCir/tSP93fBmjGdYWgg03GERNRSNOqoUH8dXlZbEydOREZGBlasWIHU1FT4+fkhNDRU/UB3YmKiRjKTk5ODl19+GampqbC0tIS/vz/OnDmDjh07AqiaJfyPP/7Ajh07kJubCycnJzz11FNYs2YN57IgIqJWQa4nw6RebpjUyw35pRX4+kw8PjocjT0RybiWkoe1z3VBB0czKPSZYBBRw3mseSyedGyxICKiJ014bBbmfX8ZmYV/DlbiaK5AJydzLBjaFp2dzXUYHRE1V/W5L65TYvHpp59i5syZUCgU+PTTTx9ad/78+fWLthliYkFERE+i9PxSvL33Gs7eyUJBaaW6XCIBJvZwxeKnfGBrytZ9IvpTgycWnp6euHjxIqytreHp6Vn7ziQS3Llzp/4RNzNMLIiI6EkmhEBucQXuZBbhm/B47L1yFwBgbCBDoLcN/N0t0d3NAt3dLaEvq/cAkkT0BGnwxKK1YWJBREStSURCNt755Qb+SM7TKLczlWNyLze8EOAGezOFjqIjIl1iYqElJhZERNTaqFQCV5JzcSkhBxEJOTgXl43sonIAgJ5Ugh4elvC0MYGHtRE6O5ujj7c1h64lagUaPbFITk7Gf//7XyQmJqK8vFxj3UcffVTf3TU7TCyIiKi1K69UIfR6Kr4+E4+LCTnV1vu6WmDpcB/08bbRQXRE1FQaNbEICwvD6NGj4eXlhVu3bqFz586Ij4+HEALdu3fH0aNHtQq+OWBiQURE9KfotAJEJuchIasIdzKLcPRWOorLq+at6uNtDX93SzhbGMLJwhDt7E1hbyZnawbRE6JRE4tevXphxIgReOedd2BqaoqrV6/Czs4OU6ZMwfDhwzFr1iytgm8OmFgQERHVLqOgDJ8fi8F35xLUk+89yNrYAB2dzNDJyRydnMzQyckMHtbGkEqZbBC1NI2aWJiamuLKlSvw9vaGpaUlTp06hU6dOuHq1asYM2YM4uPjtYm9WWBiQURE9GhJ2cXYH3kPyTnFSMkpQVJOCeIyi6BUVb+1MDaQoYNjVZLR1cUCI7s6coI+ohagUWfeNjY2Vj9X4ejoiNjYWHTq1AkAkJmZ+RjhEhERUUvkamWEV4O8NcpKK5S4lVqA63fzcP1uPq7fzcete/koKlfiYkLO/57XSMC/DkVhyVM+eLabM1syiJ4Q9U4sevfujVOnTqFDhw54+umnsXjxYkRGRuKnn35C7969GyNGIiIiaiEU+jL4uVrAz9VCXVapVOFOZlFVspGSj4PXUpGSW4LFu69i26k4/L2/J4Z0sIe5ob7uAicirdW7K9SdO3dQWFiIrl27oqioCIsXL8aZM2fQtm1bfPTRR3B3d2+sWJsMu0IRERE1ntIKJbaficfnR2NQUFY1A7i+TIK+bWwworMDhnV0gJWxgY6jJCKgEZ+xUCqVOH36NLp27QoLCwtt42y2mFgQERE1vuyicuw4E4+D1+4hOq1QXS6TShDgaYVenlaQSiQQAjBV6GGcvwtbNYiaWKM+vK1QKHDz5k14enpqFWRzxsSCiIioacWkFyL02j0cvJaK63fza6zjaK7AuvFd0b+tbRNHR9R6NWpi0aNHD3zwwQcYMmSIVkE2Z0wsiIiIdCcxqxih1+8hLrMIgAQSCXA6JhMJWcUAgL/1dsO8wW1hb6bQbaBErUCjJhahoaFYtmwZ1qxZA39/fxgbG2usfxJuxJlYEBERNS/F5ZX44OAt7AhPUJd52Rijt7c1hndyQP+2NpyUj6gRNGpiIZVK/9z4gf/AQghIJBIolcp6htv8MLEgIiJqnk7dzsT6327hj5Q8PHgH08HRDK8GeWFkF0foyaS174CI6qVRE4sTJ048dH1QUFB9dtcsMbEgIiJq3vKKK3AuLgu/387AT5dSUFxe9cWmg5kCT3dxxMiujujmasE5Moi01KiJRWJiIlxdXas1NwohkJSUBDc3t/pH3MwwsSAiImo5covL8U14ArafiUdWUbm63MbEAE4WhrAyNoCdqRwvBLhrzK9BRI/WqImFTCbDvXv3YGdnp1GelZUFOzs7doUiIiIinSitUOL36Azsj7yHIzfSUFSueU+iJ5Vg8VM+eGWAF1syiOqoPvfF9Z55+/6zFH9VWFgIhYKjMxAREZFuKPRleKqTA57q5IDSCiVu3stHVmE5sovKcSwqHQevpeKD0Fs4E5uJFc90hKeNMZ/HIGpAdU4sFi1aBKDqge3ly5fDyMhIvU6pVOLcuXPw8/Nr8ACJiIiI6kuhL0M3N0v16+d7uGDXxSSs/O91nLydiWEf/w4DmRSeNsbo5maBqYEe6OjEXgpE2qhzYnH58mUAVS0WkZGRMDAwUK8zMDCAr68vlixZ0vAREhEREWlJIpFgYk83dHezxIp913E5KQelFSpEpRUgKq0AP1xIQt821nipnyeC2tlBxq5SRPVW72cspk+fjk8++eSJfvaAz1gQERE92VQqgZTcEkSnFeDnyyk4eC0VSlXVLZGdqRyjfZ3wbHdndHQ04/wY1Ko16sPbrQETCyIiotYlOacY20/HY8+lZOQWV6jL3ayMMLi9HYZ2sIerlSEqVQJKlYCVsQFsTOQ6jJioaTCx0BITCyIiotapvFKFE9EZ2Hs5BYdvpqG8UlVr3c7OZhjkU5V0+HIYW3pCMbHQEhMLIiIiKi6vxMnbmQi7mYYT0RkoLK2ETCqBnkyK7AfmywCAAE8rLBzWDr29rHUULVHjaHGJxeeff47169cjNTUVvr6++Oyzz9CrV68a627fvh3Tp0/XKJPL5SgtLVW/FkJg5cqV2Lp1K3Jzc9G3b19s2rQJbdu2rVM8TCyIiIjoYTIKynAiOgPHbqXj8I00lCurWjb6eFtj6fD2bMGgJ0Z97ot1Pnjzzp07sWjRIqxcuRKXLl2Cr68vgoODkZ6eXus2ZmZmuHfvnnpJSEjQWL9u3Tp8+umn2Lx5M86dOwdjY2MEBwdrJB9EREREj8vWVI7x/i74fEp3HH99IP7W2w36MgnOxGZhzOensWjXFaTl876DWhedt1gEBASgZ8+e2LhxIwBApVLB1dUV8+bNw5tvvlmt/vbt27FgwQLk5ubWuD8hBJycnLB48WL18Ld5eXmwt7fH9u3bMWnSpEfGxBYLIiIiqq+U3BJ8eCgKP11KAQAYGcgwe6A3/t7fCwp9mY6jI3o8LabFory8HBERERg6dKi6TCqVYujQoQgPD691u8LCQri7u8PV1RVjxozB9evX1evi4uKQmpqqsU9zc3MEBAQ8dJ9ERERE2nC2MMRHE/ywd05fdHOzQHG5Ev86FI0hH57Agch7aAa9z4kalU4Ti8zMTCiVStjb22uU29vbIzU1tcZtfHx88OWXX2Lfvn349ttvoVKp0KdPHyQnJwOAerv67LOsrAz5+fkaCxEREdHj8HO1wE+z+mDDRD84mCmQkluC2d9dwvjN4fjt+p/zZRA9aXT+jEV9BQYGYurUqfDz80NQUBB++ukn2NraYsuWLY+9z7Vr18Lc3Fy9uLq6NmDERERE1NpIJBKM7eaMo0uCMH9IW8j1pIhIyMEr30Rg8IfHsf10HIrKKnUdJlGD0mliYWNjA5lMhrS0NI3ytLQ0ODg41Gkf+vr66NatG2JiYgBAvV199rls2TLk5eWpl6SkpPoeChEREVE1RgZ6WDSsHX5/YxBmD/SGuaE+ErKKseqXG+i9NgxrD9zE3dwSXYdJ1CB0mlgYGBjA398fYWFh6jKVSoWwsDAEBgbWaR9KpRKRkZFwdHQEAHh6esLBwUFjn/n5+Th37lyt+5TL5TAzM9NYiIiIiBqKvZkCbwxvj/Blg7FmbGd42hijoLQSW36/g/7rjmHMxlN499cb+O16arU5MohaCj1dB7Bo0SKEhISgR48e6NWrFzZs2ICioiL1XBVTp06Fs7Mz1q5dCwBYvXo1evfujTZt2iA3Nxfr169HQkIC/v73vwOoanpcsGAB3n33XbRt2xaenp5Yvnw5nJycMHbsWF0dJhERERGMDPTwYm93TOnlhmNR6dh2Kg5nYrNwNTkPV5Pz8H+n4gAAbexM0NPDCgN9bDGkvR30ZC2u9zq1QjpPLCZOnIiMjAysWLECqamp8PPzQ2hoqPrh68TEREilf/5nysnJwcsvv4zU1FRYWlrC398fZ86cQceOHdV13njjDRQVFWHmzJnIzc1Fv379EBoaCoVC0eTHR0RERPRXUqkEQzrYY0gHe6TkluBCXDbOx2fjQlw2bqcXIuZ/y/fnE+FkrsDfAt0xqacbrIwNdB06Ua10Po9Fc8R5LIiIiEhXsovKcTE+G2fvZGPflRRk/a9rlFxPijF+Tgjp44FOTua4cTcf355LwK9X7yK4kwPeH9cVMqlEx9HTk6Y+98VMLGrAxIKIiIiag9IKJX794x6+Oh2H63f/HA7f1coQSdmaD32P6+6C9eO7QsrkghoQEwstMbEgIiKi5kQIgYiEHHx1Jh6h16rmwtCTShDcyQFdXcyx7rcoKFUCUwLc8O7YzkjOKcHJ25koLq/EGD9n2JrKdX0I1EIxsdASEwsiIiJqru7lleBSQi56eFjC3qzq+dF9V1KwYOcVCAHYmcqRXlCmri/Xk2JST1fMDPKGs4WhrsKmFoqJhZaYWBAREVFLs+tiEt7Y8wcAQCaVoLubBcqVAleTctV1TOR6MJHrwcxQD729rDElwB0+DqY6iphaAiYWWmJiQURERC3RuTtZyCupQG9va5gp9CGEQHhsFj4/HoPTMVk1btPD3RIv9fPE8M4OkEj4fAZpYmKhJSYWRERE9KTJLS5HbnEFCkorcS+vBD9fTsGhG2lQqqpuBfu3tcHqMVWT9xHdx8RCS0wsiIiIqDVIyy/FN+EJ+OLkHZRXqmCgJ8WsIG/MGugNhb5M1+FRM8DEQktMLIiIiKg1ic8swvJ913DydiYAwMPaCKvHdMaAdrY6jox0jYmFlphYEBERUWsjhMD+yHtY/csN9ahST3dxwMguTujkZAY3KyPOkdEKMbHQEhMLIiIiaq0KSivw0eFo7DgTD9UDd4mmCj1M7uWGRcPasZtUK8LEQktMLIiIiKi1u5aSh/+cT8S1lDzcSi1AeaUKAOBlY4z1z3eFv7uVjiOkpsDEQktMLIiIiIj+VKFU4XhUBv6xNxJp+WWQSICnOzvCz9UCnZzM0NHJDBZGBroOkxoBEwstMbEgIiIiqi6vuAJr9t/AnojkauucLQzR0ckMXZ3NMbGnK+z+Nys4tWxMLLTExIKIiIiodhEJ2TgTk4Xrd/Nx/V4ekrJLNNbL9aSYGuiOV4O8YW0i11GU1BCYWGiJiQURERFR3eWVVODmvXxcv5uPA5H3EJGQAwAwMpDh6S6OGNbRHv3b2sDIQE/HkVJ9MbHQEhMLIiIioscjhMCJ6Ax8dDgafyTnqcvlelK0szeFhZE+zA314WCmQBcXc3R1sYA7h7JttupzX8y0kYiIiIgajEQiwUAfOwS1s8XZO9k4dCMVh2+kITmnBJEpeTVuY6gvg7FcDwp9KYwMZHCzMkZ7B1O0czBFV2dzuFsbQSJh4tHcscWiBmyxICIiImo4QgjcTi9Eck4xcosrkFNcgcSsIvyRkofrd/PVQ9nWxsZEjp4elujmZoG29qZoa2cCZwtDJhtNgF2htMTEgoiIiKhpVChVSMkpQXG5EqWVShSVVSI2vRBRaYW4lZqP6yn5KFdWTzzkelKYKvRhIpfBVKEPezMFXCwN4WJpiEBva3RyMtfB0Tx52BWKiIiIiFoEfZkUHjbGGmX929qqfy6tUCIyJQ8X4rNxPSUft9MLcCejCGWVKpQVliGzsKreX7tZBbWzxZxBbdDLkxP5NRW2WNSALRZEREREzVeFUoXUvFIUlFaiqLwSBaUVuJtbiuScEtxOK8CxqHSo/neHG+hljY8m+sLR3FC3QbdQ7AqlJSYWRERERC1XQlYRtvx+B3suJqNcqYK1sQE+m9wNfdrY6Dq0FoeJhZaYWBARERG1fIlZxXj12wjcuJcPqQRYMLQdRnR2gJu1EeR6Ml2H1yIwsdASEwsiIiKiJ0NphRLL917D7ohkdZlUAnXXqLJKFcorlejkZI6pge4Y1tEeejKprsJtdphYaImJBREREdGTQwiB3RHJ+PZsAu5kFKGwrLLWuo7mCvyttzsm9nSFjYm8CaNsnphYaImJBREREdGTSQiBjMIyJGWXQCaVQK4nhRDAgch7+P58IrKKygEABjIpRnZ1xJAOdohKLUBEQg5iMwoxK8gb0/p66vgomg4TCy0xsSAiIiJqfcoqldj/xz3sCE/A1aTcWuvNHuiN14N9WsUEffW5L24WHcg+//xzeHh4QKFQICAgAOfPn6/Tdj/88AMkEgnGjh2rUT5t2jRIJBKNZfjw4Y0QORERERE9KeR6MjzX3QX75vTFvjl9Ma67Czo4mmFcdxe892wXzB/cBgDw7+OxePPHSFTWMHFfa6bzCfJ27tyJRYsWYfPmzQgICMCGDRsQHByMqKgo2NnZ1bpdfHw8lixZgv79+9e4fvjw4fjqq6/Ur+Vy9pEjIiIiorrxdbXAh64W1cqdLAzx1s+R2HkxCb/fzkBHRzP4OJiis7M5entZw8rYAACQUVCGA5H3EJGQg87OZniqo0O1iQCfNDrvChUQEICePXti48aNAACVSgVXV1fMmzcPb775Zo3bKJVKDBgwADNmzMDJkyeRm5uLvXv3qtdPmzatWll9sCsUEREREdUm9FoqFu68gpIKZbV17R1MYWlkgHNxWepJ+u7zsTfFyK6OGOvnDDdroyaKVjv1uS/WaYtFeXk5IiIisGzZMnWZVCrF0KFDER4eXut2q1evhp2dHV566SWcPHmyxjrHjx+HnZ0dLC0tMXjwYLz77ruwtrausW5ZWRnKysrUr/Pz8x/ziIiIiIjoSTe8swP6tBmCm3fzEZVWgJv3CnApIQdRaQW4lVqgrufnaoH+bW1wOTEXZ+9kISqtAFGHC/DR4Wh0d7PAOH8XjPFzholc552IGoROjyIzMxNKpRL29vYa5fb29rh161aN25w6dQrbtm3DlStXat3v8OHD8dxzz8HT0xOxsbF46623MGLECISHh0Mmqz4Zytq1a/HOO+9odSxERERE1HqYKfQR4GWNAK8/v7jOLCzD2TtZyCwow6D2dnC3/rPrU15xBQ7fTMO+Kyk4HZOJS4m5uJSYi/f238Sz3Z0x3t8VXrbGMFPo6+JwGkSLSo8KCgrw4osvYuvWrbCxqX1K9kmTJql/7tKlC7p27Qpvb28cP34cQ4YMqVZ/2bJlWLRokfp1fn4+XF1dGzZ4IiIiInqi2ZjI8UxXpxrXmRvpY7y/C8b7uyA9vxT7rtzF9xcScSejCN+eTcS3ZxMBAKZyPbhZG+GVIG+M6urYokae0mliYWNjA5lMhrS0NI3ytLQ0ODg4VKsfGxuL+Ph4jBo1Sl2mUlU9ja+np4eoqCh4e3tX287Lyws2NjaIiYmpMbGQy+V8uJuIiIiImoSdmQIvD/DC3/t7Ijw2C9+eS0B4bBZyiitQUFaJ63fzMf/7y/jtWirWjO2sfiC8udNpYmFgYAB/f3+EhYWph4xVqVQICwvD3Llzq9Vv3749IiMjNcr+8Y9/oKCgAJ988kmtrQzJycnIysqCo6Njgx8DEREREdHjkEgk6NPGBn3aVPXEKS6vxN3cUvxy9S4+PxaD/ZH3cC4uC2uf64phHe0fsTfd03lXqEWLFiEkJAQ9evRAr169sGHDBhQVFWH69OkAgKlTp8LZ2Rlr166FQqFA586dNba3sLAAAHV5YWEh3nnnHYwbNw4ODg6IjY3FG2+8gTZt2iA4OLhJj42IiIiIqK6MDPTQxs4EC4e1w9AO9li8+wqi0wqxct819G9rA4V+9WeFmxOdJxYTJ05ERkYGVqxYgdTUVPj5+SE0NFT9QHdiYiKk0rrP4yeTyfDHH39gx44dyM3NhZOTE5566imsWbOG3Z2IiIiIqEXo4mKO/87th4+PRKNfm+afVADNYB6L5ojzWBARERER1e++uO5NAURERERERLVgYkFERERERFpjYkFERERERFpjYkFERERERFpjYkFERERERFpjYkFERERERFrT+TwWzdH9EXjz8/N1HAkRERERke7cvx+uywwVTCxqUFBQAABwdXXVcSRERERERLpXUFAAc3Pzh9bhBHk1UKlUuHv3LkxNTSGRSJr8/fPz8+Hq6oqkpCRO0PeYeA61x3PYMHgetcdzqD2eQ+3xHGqP51B7ujiHQggUFBTAyckJUunDn6Jgi0UNpFIpXFxcdB0GzMzM+B9PSzyH2uM5bBg8j9rjOdQez6H2eA61x3OovaY+h49qqbiPD28TEREREZHWmFgQEREREZHWmFg0Q3K5HCtXroRcLtd1KC0Wz6H2eA4bBs+j9ngOtcdzqD2eQ+3xHGqvuZ9DPrxNRERERERaY4sFERERERFpjYkFERERERFpjYkFERERERFpjYlFM/T555/Dw8MDCoUCAQEBOH/+vK5DarbWrl2Lnj17wtTUFHZ2dhg7diyioqI06gwcOBASiURjefXVV3UUcfOzatWqauenffv26vWlpaWYM2cOrK2tYWJignHjxiEtLU2HETc/Hh4e1c6hRCLBnDlzAPAarMnvv/+OUaNGwcnJCRKJBHv37tVYL4TAihUr4OjoCENDQwwdOhS3b9/WqJOdnY0pU6bAzMwMFhYWeOmll1BYWNiER6FbDzuHFRUVWLp0Kbp06QJjY2M4OTlh6tSpuHv3rsY+arp233///SY+Et151HU4bdq0audn+PDhGnV4HT78HNb0u1EikWD9+vXqOq39OqzLvUxd/hYnJiZi5MiRMDIygp2dHV5//XVUVlY25aEwsWhudu7ciUWLFmHlypW4dOkSfH19ERwcjPT0dF2H1iydOHECc+bMwdmzZ3H48GFUVFTgqaeeQlFRkUa9l19+Gffu3VMv69at01HEzVOnTp00zs+pU6fU6xYuXIhffvkFu3fvxokTJ3D37l0899xzOoy2+blw4YLG+Tt8+DAA4Pnnn1fX4TWoqaioCL6+vvj8889rXL9u3Tp8+umn2Lx5M86dOwdjY2MEBwejtLRUXWfKlCm4fv06Dh8+jF9//RW///47Zs6c2VSHoHMPO4fFxcW4dOkSli9fjkuXLuGnn35CVFQURo8eXa3u6tWrNa7NefPmNUX4zcKjrkMAGD58uMb5+f777zXW8zp8+Dl88Nzdu3cPX375JSQSCcaNG6dRrzVfh3W5l3nU32KlUomRI0eivLwcZ86cwY4dO7B9+3asWLGiaQ9GULPSq1cvMWfOHPVrpVIpnJycxNq1a3UYVcuRnp4uAIgTJ06oy4KCgsRrr72mu6CauZUrVwpfX98a1+Xm5gp9fX2xe/duddnNmzcFABEeHt5EEbY8r732mvD29hYqlUoIwWvwUQCIn3/+Wf1apVIJBwcHsX79enVZbm6ukMvl4vvvvxdCCHHjxg0BQFy4cEFd5+DBg0IikYiUlJQmi725+Os5rMn58+cFAJGQkKAuc3d3Fx9//HHjBtdC1HQOQ0JCxJgxY2rdhtehprpch2PGjBGDBw/WKON1qOmv9zJ1+Vt84MABIZVKRWpqqrrOpk2bhJmZmSgrK2uy2Nli0YyUl5cjIiICQ4cOVZdJpVIMHToU4eHhOoys5cjLywMAWFlZaZR/9913sLGxQefOnbFs2TIUFxfrIrxm6/bt23BycoKXlxemTJmCxMREAEBERAQqKio0rsn27dvDzc2N12QtysvL8e2332LGjBmQSCTqcl6DdRcXF4fU1FSN687c3BwBAQHq6y48PBwWFhbo0aOHus7QoUMhlUpx7ty5Jo+5JcjLy4NEIoGFhYVG+fvvvw9ra2t069YN69evb/KuE83d8ePHYWdnBx8fH8yaNQtZWVnqdbwO6yctLQ379+/HSy+9VG0dr8M//fVepi5/i8PDw9GlSxfY29ur6wQHByM/Px/Xr19vstj1muyd6JEyMzOhVCo1LgoAsLe3x61bt3QUVcuhUqmwYMEC9O3bF507d1aXv/DCC3B3d4eTkxP++OMPLF26FFFRUfjpp590GG3zERAQgO3bt8PHxwf37t3DO++8g/79++PatWtITU2FgYFBtRsRe3t7pKam6ibgZm7v3r3Izc3FtGnT1GW8Buvn/rVV0+/C++tSU1NhZ2ensV5PTw9WVla8NmtQWlqKpUuXYvLkyTAzM1OXz58/H927d4eVlRXOnDmDZcuW4d69e/joo490GG3zMXz4cDz33HPw9PREbGws3nrrLYwYMQLh4eGQyWS8Dutpx44dMDU1rdadltfhn2q6l6nL3+LU1NQaf2feX9dUmFjQE2POnDm4du2axvMBADT6unbp0gWOjo4YMmQIYmNj4e3t3dRhNjsjRoxQ/9y1a1cEBATA3d0du3btgqGhoQ4ja5m2bduGESNGwMnJSV3Ga5B0qaKiAhMmTIAQAps2bdJYt2jRIvXPXbt2hYGBAV555RWsXbu22c7s25QmTZqk/rlLly7o2rUrvL29cfz4cQwZMkSHkbVMX375JaZMmQKFQqFRzuvwT7Xdy7QU7ArVjNjY2EAmk1V7yj8tLQ0ODg46iqplmDt3Ln799VccO3YMLi4uD60bEBAAAIiJiWmK0FocCwsLtGvXDjExMXBwcEB5eTlyc3M16vCarFlCQgKOHDmCv//97w+tx2vw4e5fWw/7Xejg4FBtUIvKykpkZ2fz2nzA/aQiISEBhw8f1mitqElAQAAqKysRHx/fNAG2MF5eXrCxsVH/3+V1WHcnT55EVFTUI38/Aq33OqztXqYuf4sdHBxq/J15f11TYWLRjBgYGMDf3x9hYWHqMpVKhbCwMAQGBuowsuZLCIG5c+fi559/xtGjR+Hp6fnIba5cuQIAcHR0bOToWqbCwkLExsbC0dER/v7+0NfX17gmo6KikJiYyGuyBl999RXs7OwwcuTIh9bjNfhwnp6ecHBw0Lju8vPzce7cOfV1FxgYiNzcXERERKjrHD16FCqVSp24tXb3k4rbt2/jyJEjsLa2fuQ2V65cgVQqrda9h6okJycjKytL/X+X12Hdbdu2Df7+/vD19X1k3dZ2HT7qXqYuf4sDAwMRGRmpkeje/zKhY8eOTXMgAEeFam5++OEHIZfLxfbt28WNGzfEzJkzhYWFhcZT/vSnWbNmCXNzc3H8+HFx79499VJcXCyEECImJkasXr1aXLx4UcTFxYl9+/YJLy8vMWDAAB1H3nwsXrxYHD9+XMTFxYnTp0+LoUOHChsbG5Geni6EEOLVV18Vbm5u4ujRo+LixYsiMDBQBAYG6jjq5kepVAo3NzexdOlSjXJegzUrKCgQly9fFpcvXxYAxEcffSQuX76sHrHo/fffFxYWFmLfvn3ijz/+EGPGjBGenp6ipKREvY/hw4eLbt26iXPnzolTp06Jtm3bismTJ+vqkJrcw85heXm5GD16tHBxcRFXrlzR+P14f4SYM2fOiI8//lhcuXJFxMbGim+//VbY2tqKqVOn6vjIms7DzmFBQYFYsmSJCA8PF3FxceLIkSOie/fuom3btqK0tFS9D16HD/+/LIQQeXl5wsjISGzatKna9rwOH30vI8Sj/xZXVlaKzp07i6eeekpcuXJFhIaGCltbW7Fs2bImPRYmFs3QZ599Jtzc3ISBgYHo1auXOHv2rK5DarYA1Lh89dVXQgghEhMTxYABA4SVlZWQy+WiTZs24vXXXxd5eXm6DbwZmThxonB0dBQGBgbC2dlZTJw4UcTExKjXl5SUiNmzZwtLS0thZGQknn32WXHv3j0dRtw8/fbbbwKAiIqK0ijnNVizY8eO1fh/NyQkRAhRNeTs8uXLhb29vZDL5WLIkCHVzm1WVpaYPHmyMDExEWZmZmL69OmioKBAB0ejGw87h3FxcbX+fjx27JgQQoiIiAgREBAgzM3NhUKhEB06dBDvvfeexk3zk+5h57C4uFg89dRTwtbWVujr6wt3d3fx8ssvV/uij9fhw/8vCyHEli1bhKGhocjNza22Pa/DR9/LCFG3v8Xx8fFixIgRwtDQUNjY2IjFixeLioqKJj0Wyf8OiIiIiIiI6LHxGQsiIiIiItIaEwsiIiIiItIaEwsiIiIiItIaEwsiIiIiItIaEwsiIiIiItIaEwsiIiIiItIaEwsiIiIiItIaEwsiIiIiItIaEwsiInqiHD9+HBKJBLm5uboOhYioVWFiQUREREREWmNiQUREREREWmNiQUREDUqlUmHt2rXw9PSEoaEhfH19sWfPHgB/dlPav38/unbtCoVCgd69e+PatWsa+/jxxx/RqVMnyOVyeHh44MMPP9RYX1ZWhqVLl8LV1RVyuRxt2rTBtm3bNOpERESgR48eMDIyQp8+fRAVFdW4B05E1MoxsSAioga1du1afP3119i8eTOuX7+OhQsX4m9/+xtOnDihrvP666/jww8/xIULF2Bra4tRo0ahoqICQFVCMGHCBEyaNAmRkZFYtWoVli9fju3bt6u3nzp1Kr7//nt8+umnuHnzJrZs2QITExONON5++218+OGHuHjxIvT09DBjxowmOX4iotZKIoQQug6CiIieDGVlZbCyssKRI0cQGBioLv/73/+O4uJizJw5E4MGDcIPP/yAiRMnAgCys7Ph4uKC7du3Y8KECZgyZQoyMjJw6NAh9fZvvPEG9u/fj+vXryM6Oho+Pj44fPgwhg4dWi2G48ePY9CgQThy5AiGDBkCADhw4ABGjhyJkpISKBSKRj4LREStE1ssiIiowcTExKC4uBjDhg2DiYmJevn6668RGxurrvdg0mFlZQUfHx/cvHkTAHDz5k307dtXY799+/bF7du3oVQqceXKFchkMgQFBT00lq5du6p/dnR0BACkp6drfYxERFQzPV0HQERET47CwkIAwP79++Hs7KyxTi6XayQXj8vQ0LBO9fT19dU/SyQSAFXPfxARUeNgiwURETWYjh07Qi6XIzExEW3atNFYXF1d1fXOnj2r/jknJwfR0dHo0KEDAKBDhw44ffq0xn5Pnz6Ndu3aQSaToUuXLlCpVBrPbBARke6xxYKIiBqMqakplixZgoULF0KlUqFfv37Iy8vD6dOnYWZmBnd3dwDA6tWrYW1tDXt7e7z99tuwsbHB2LFjAQCLFy9Gz549sWbNGkycOBHh4eHYuHEj/v3vfwMAPDw8EBISghkzZuDTTz+Fr68vEhISkJ6ejgkTJujq0ImIWj0mFkRE1KDWrFkDW1tbrF27Fnfu3IGFhQW6d++Ot956S90V6f3338drr72G27dvw8/PD7/88gsMDAwAAN27d8euXbuwYsUKrFmzBo6Ojli9ejWmTZumfo9NmzbhrbfewuzZs5GVlQU3Nze89dZbujhcIiL6H44KRURETeb+iE05OTmwsLDQdThERNSA+IwFERERERFpjYkFERERERFpjV2hiIiIiIhIa2yxICIiIiIirTGxICIiIiIirTGxICIiIiIirTGxICIiIiIirTGxICIiIiIirTGxICIiIiIirTGxICIiIiIirTGxICIiIiIirTGxICIiIiIirf0/oUzwnZ2JGtUAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 800x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "with brainstate.environ.context(dt=1.0 * u.ms):\n",
    "\n",
    "    class SNN(brainstate.nn.Module):\n",
    "        def __init__(self, n_in, n_rec, n_out):\n",
    "            super().__init__()\n",
    "            # input -> recurrent: a trainable Linear (weights in mA) feeding an\n",
    "            # exponential synapse\n",
    "            self.i2r = brainstate.nn.Sequential(\n",
    "                brainstate.nn.Linear(\n",
    "                    n_in, n_rec,\n",
    "                    w_init=braintools.init.KaimingNormal(unit=u.mA),\n",
    "                    b_init=braintools.init.ZeroInit(unit=u.mA)),\n",
    "                brainpy.state.Expon(n_rec, tau=5. * u.ms,\n",
    "                                    g_initializer=braintools.init.Constant(0. * u.mA)),\n",
    "            )\n",
    "            # recurrent LIF with a surrogate gradient -> this is what makes it trainable\n",
    "            self.r = brainpy.state.LIF(\n",
    "                n_rec, tau=20. * u.ms, V_reset=0. * u.mV, V_rest=0. * u.mV,\n",
    "                V_th=1. * u.mV, spk_fun=braintools.surrogate.ReluGrad())\n",
    "            # recurrent -> output readout\n",
    "            self.r2o = brainstate.nn.Linear(n_rec, n_out,\n",
    "                                            w_init=braintools.init.KaimingNormal())\n",
    "            self.o = brainpy.state.Expon(n_out, tau=10. * u.ms,\n",
    "                                         g_initializer=braintools.init.Constant(0.))\n",
    "\n",
    "        def update(self, spike):\n",
    "            return self.o(self.r2o(self.r(self.i2r(spike))))\n",
    "\n",
    "    # tiny synthetic dataset: random Poisson-like input spikes, binary labels\n",
    "    n_in = 100\n",
    "    net = SNN(n_in=n_in, n_rec=4, n_out=2)\n",
    "    num_step, num_sample = 100, 128\n",
    "    freq = 5 * u.Hz\n",
    "    x_data = brainstate.random.rand(num_step, num_sample, n_in) \\\n",
    "        < freq * brainstate.environ.get_dt()\n",
    "    y_data = u.math.asarray(brainstate.random.rand(num_sample) < 0.5, dtype=int)\n",
    "\n",
    "    # optimizer registered on the trainable parameters only\n",
    "    optimizer = braintools.optim.Adam(lr=3e-3)\n",
    "    optimizer.register_trainable_weights(net.states(brainstate.ParamState))\n",
    "\n",
    "    def loss_fn():\n",
    "        # for_loop over time -> BPTT-able trajectory of readouts [T, B, C]\n",
    "        preds = brainstate.transform.for_loop(net.update, x_data)\n",
    "        preds = u.math.mean(preds, axis=0)   # average readout over time -> [B, C]\n",
    "        return braintools.metric.softmax_cross_entropy_with_integer_labels(\n",
    "            preds, y_data).mean()\n",
    "\n",
    "    @brainstate.transform.jit\n",
    "    def train_step():\n",
    "        brainstate.nn.init_all_states(net, batch_size=num_sample)  # reset each step\n",
    "        grads, loss = brainstate.transform.grad(\n",
    "            loss_fn, net.states(brainstate.ParamState), return_value=True)()\n",
    "        optimizer.update(grads)\n",
    "        return loss\n",
    "\n",
    "    # outer optimization loop in plain Python -- this is fine\n",
    "    losses = []\n",
    "    for epoch in range(1, 201):\n",
    "        losses.append(float(train_step()))\n",
    "\n",
    "    plt.figure(figsize=(8, 3))\n",
    "    plt.plot(losses)\n",
    "    plt.xlabel('epoch'); plt.ylabel('training loss')\n",
    "    plt.title('surrogate-gradient BPTT on a tiny SNN')\n",
    "    plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0f9b7e2",
   "metadata": {},
   "source": [
    "The full version of this example (with a Fashion-MNIST-scale dataset and accuracy\n",
    "reporting) is in the training how-to track — see\n",
    "{doc}`/brainpy-style/howto/train-surrogate-gradients`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45bd52a9",
   "metadata": {},
   "source": [
    "## 4. Long rollouts: checkpoint to bound memory\n",
    "\n",
    "Because BPTT stores every step's activations, a long simulation can exhaust memory\n",
    "on the backward pass. The **checkpointed** loop variants trade compute for memory:\n",
    "they keep only a sparse set of checkpoints on the forward pass and *recompute* the\n",
    "intervening activations during backprop.\n",
    "\n",
    "Swap the loop and tune the ``base`` parameter (larger ``base`` → fewer checkpoints →\n",
    "less memory, more recomputation); the semantics are otherwise identical to\n",
    "``for_loop`` / ``scan``:\n",
    "\n",
    "```python\n",
    "preds = brainstate.transform.checkpointed_for_loop(net.update, x_data)\n",
    "# or, with an explicit carry:\n",
    "# carry, ys = brainstate.transform.checkpointed_scan(step, carry0, xs)\n",
    "```\n",
    "\n",
    "Reach for these *only* when reverse-mode gradients through a long rollout would\n",
    "otherwise run out of memory — for ordinary-length rollouts, plain ``for_loop`` /\n",
    "``scan`` is simpler and faster. The dedicated how-to walks through the trade-off:\n",
    "{doc}`/brainpy-style/howto/train-long-rollouts-checkpoint`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bc6c5fb",
   "metadata": {},
   "source": [
    "## Recap\n",
    "\n",
    "- A spike's gradient is unusable; a **surrogate** (``spk_fun=...``) supplies a smooth\n",
    "  backward pass while the forward pass stays exactly spiking.\n",
    "- ``transform.for_loop`` / ``scan`` are **differentiable**, so wrapping a simulation\n",
    "  in them and calling ``transform.grad`` *is* backpropagation-through-time.\n",
    "- Select trainable variables with ``net.states(brainstate.ParamState)``; reset state\n",
    "  each step with ``init_all_states``.\n",
    "- For long rollouts, switch to ``checkpointed_for_loop`` / ``checkpointed_scan`` to\n",
    "  bound memory.\n",
    "\n",
    "## See also\n",
    "\n",
    "- {doc}`/concepts/alignpre-alignpost` — why neuron-aligned state keeps BPTT\n",
    "  activations small, and the keystone the next page builds on.\n",
    "- {doc}`/concepts/online-learning` — the *trainable → scalable* half: linear-memory\n",
    "  RTRL.\n",
    "- {doc}`/concepts/state-paradigm` — the ``transform`` primitives used here.\n",
    "- {doc}`/brainpy-style/howto/train-surrogate-gradients` — the full training how-to."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}