{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8ca5efef",
   "metadata": {},
   "source": [
    "# D-RTRL: parameter-dimensional online traces\n",
    "\n",
    "D-RTRL propagates an eligibility trace forward with the model rather than\n",
    "retaining a sequence-wide reverse-mode graph. This chapter identifies the\n",
    "approximation, trains a compact recurrent model, and limits each conclusion to\n",
    "what the example actually tests.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "569defd1",
   "metadata": {},
   "source": [
    "## 1. Principle and retained terms\n",
    "\n",
    "For parameter $\\theta$ and hidden state $h_t$, RTRL propagates\n",
    "\n",
    "$$G_t = \\frac{\\partial h_t}{\\partial \\theta}\n",
    "      = \\frac{\\partial h_t}{\\partial h_{t-1}}G_{t-1}\n",
    "      + \\frac{\\partial h_t}{\\partial \\theta}.$$\n",
    "\n",
    "D-RTRL stores a parameter-shaped trace but retains only the per-position\n",
    "diagonal/block-diagonal part of the hidden-to-hidden Jacobian. The instantaneous\n",
    "parameter term remains. The online gradient estimate contracts the current\n",
    "learning signal with that trace. It therefore avoids BPTT activation storage,\n",
    "but it is not generally element-wise equal to BPTT when hidden positions mix.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e236734",
   "metadata": {},
   "source": [
    "## 2. Applicability\n",
    "\n",
    "**Use D-RTRL when** a recurrent ETP model needs forward-only updates and its\n",
    "per-parameter trace is affordable, especially when cross-position hidden\n",
    "coupling is limited or an explicitly diagonal approximation is acceptable.\n",
    "\n",
    "**Do not use this example to claim** universal BPTT equivalence, biological\n",
    "plausibility, or favorable scaling for very wide recurrent layers. A decreasing\n",
    "loss is an optimization smoke check, not a gradient oracle.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dac25eb7",
   "metadata": {},
   "source": [
    "## 3. Setup\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "drtrl-imports",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-08-07T14:13:47.303265Z",
     "iopub.status.busy": "2026-08-07T14:13:47.303265Z",
     "iopub.status.idle": "2026-08-07T14:13:49.950749Z",
     "shell.execute_reply": "2026-08-07T14:13:49.950749Z"
    }
   },
   "outputs": [],
   "source": [
    "import brainstate\n",
    "import braintools\n",
    "import braintrace\n",
    "import jax.numpy as jnp\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "brainstate.random.seed(7)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8569603",
   "metadata": {},
   "source": [
    "## 4. A compact recurrent task\n",
    "\n",
    "The MiniGRU supplies genuine recurrent state and ETP-aware parameterized\n",
    "operations. The fixed sequence keeps the optimization check reproducible; it\n",
    "does not remove the estimator's approximation error.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "drtrl-model",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-08-07T14:13:49.952802Z",
     "iopub.status.busy": "2026-08-07T14:13:49.952802Z",
     "iopub.status.idle": "2026-08-07T14:13:52.187759Z",
     "shell.execute_reply": "2026-08-07T14:13:52.187759Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\BrainTrace\\braintrace\\_compiler\\hid_param_op.py:969: UserWarning: ETP primitive etp_mm (weight=('readout', 'weight')) has no connected hidden states. It will be treated as a non-temporal parameter.\n",
      "  _emit_no_relation_diag(\n"
     ]
    }
   ],
   "source": [
    "class SequenceModel(brainstate.nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.rnn = braintrace.nn.MiniGRU(in_size=1, out_size=6)\n",
    "        self.readout = braintrace.nn.Linear(6, 1)\n",
    "\n",
    "    def update(self, x):\n",
    "        return self.readout(self.rnn(x))\n",
    "\n",
    "\n",
    "model = SequenceModel()\n",
    "inputs = jnp.linspace(-1.0, 1.0, 12).reshape(12, 1, 1)\n",
    "targets = 0.7 * inputs + 0.2\n",
    "\n",
    "# The Linear readout is intentionally non-temporal; Section 5 explains\n",
    "# the compiler diagnostic emitted for it.\n",
    "learner = braintrace.compile(\n",
    "    model,\n",
    "    braintrace.D_RTRL,\n",
    "    inputs[0],\n",
    "    batch_size=1,\n",
    ")\n",
    "weights = model.states(brainstate.ParamState)\n",
    "optimizer = braintools.optim.SGD(lr=0.08)\n",
    "optimizer.register_trainable_weights(weights);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f15e5dc",
   "metadata": {},
   "source": [
    "## 5. Online update and evidence\n",
    "\n",
    "`etrace_grad` owns the temporal loop, advances the model exactly once per time\n",
    "step, and accumulates the estimated gradient. The final assertion is deliberately\n",
    "narrow: the fixed-seed objective should decrease.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "drtrl-training",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-08-07T14:13:52.190122Z",
     "iopub.status.busy": "2026-08-07T14:13:52.190122Z",
     "iopub.status.idle": "2026-08-07T14:13:54.115838Z",
     "shell.execute_reply": "2026-08-07T14:13:54.115838Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "initial loss: 0.0802\n",
      "epoch  0 loss: 0.0802\n",
      "epoch 12 loss: 0.0100\n",
      "epoch 24 loss: 0.0086\n",
      "final loss: 0.0086\n",
      "loss decreased: True\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAArEAAAFyCAYAAADiToKqAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlcelbwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAVwNJREFUeJzt3Qd4U2X7BvA7Sdt00QKl7LKnTNlLRFHZiAofoIILHJ+i/AFBHCjCJ/o5UNyfgiKoiCAiKCIqIkP2lo3svbrobvK/njdNSDogTdom5+T+XVdIcnLOyWneht5587zvMVitViuIiIiIiDTE6OsDICIiIiIqLIZYIiIiItIchlgiIiIi0hyGWCIiIiLSHIZYIiIiItIchlgiIiIi0hyGWCIiIiLSHIZYIiIiItIchlgiIiIi0hyGWCKiArz99tswGAw4f/68T7YPVI0bN0a/fv18tr23fP38RIGCIZZIZ/bs2aOCk/0SFBSEsmXLokWLFnj88cexceNGj/dlNBpRunRp3HTTTfjxxx/VOqdPn3ZZ52qXTz/9FJ9//rnLMpPJhAoVKuCuu+7Cjh078hyDff3FixcjUCxcuBC33347KleuDLPZrF6fli1bYuTIkdi6davLup06dXJ5PUNDQ9GgQQNMmDABqampefYdGRmJ7t27X7W9n3nmmaseX506ddC/f38vf0oiIu8wxBLp1HPPPQer1YrMzEwcPHgQb731Fi5duoQ2bdpg1KhRHu0rKysL69evR0hICPr06YMlS5agYsWK6jHnS69evVQ4zb182LBhjn0uWrRILZOgtWDBAhVgJZAdPXoU/kJCoxxjuXLlSmR7aat//etfGDhwoPrQ8fvvvyMxMRG7d+/GCy+8gHXr1uH6669X7eBMQq79NT558qT6sDJp0iQMHToUWrRz5058//33PtueiLSBIZZI56RnrUyZMujSpQu++uorTJw4EVOnTsWHH35Y6H1JT2y9evXw2WefqcD03nvveX18Eog7dOiAV199VQW2mTNnIlCNHz8e8+bNUz2xL774oupRlZ5Y6UmXr6fXrFmDcePGqTYtiKw7YsQI9OjRQ+3r1KlTJfozEBGVFIZYogAjXxVL7+lrr73m8T7ka+7w8HAcP368yI6rbt266rqo9vnSSy+psCe9z48++qgKd9Ij+vzzz6sAnpKSgkceeUQti46OVrczMjKuWdNq368Ebulple3lK/o777wTZ86cueb2BZF15EOB9GJ369Yt33VkXxL2pZe7pF9POylPkZ79+fPnO0oYpAbUrmrVqrj33nuxefNm3Hjjjer3ZMyYMeqxW265xaWMRH4PBw0ahEOHDl2zptS+X+mxl/2GhYWhWrVqeOONN/Ico7fbS4/4s88+iypVqqjjl/KZXbt2qTKM5s2be/S6ye+c/D40atRIlXzExMSo3xnpZXf2888/o3Pnzur3VT58yrcT3333XaHXIQoEDLFEASY4OFj1yh45cgSHDx/2aB8SjCQESggoKnv37lXXRblP8fTTT6vwIT/rJ598gv/+9794//33VbC97bbb8M8//2DWrFmq9vb1118v1H6lB1m2/+WXX1QvqQRhT/3xxx9IT08vMMB68npKWIyLi0NRklKG2rVrqxpmewmDfH3vTEoapPxBQrkEXnmdxK+//urYRn5/pK76xIkT6NmzZ771u7lJ/bV8iJD2kx7mhx9+WLWDu/XS7m4/fPhwvPPOO+p3RdaTby5kvfj4eHhKSjykF/2JJ55Qr8/KlSvVB5d27dqpWmQhr2Pfvn3RsWNHtUzeZxKyv/76a8cHJHfWIQoUDLFEAUh6pYT8MS0Mi8WiwtEDDzygwnBha2vzI72fEgClhzg2NlbtuyhJDan0ykVFReGOO+5QX7PLc8lAKQlislxCgfSATp8+3e39Sm+f1K/K9hLSnnzySVUG4GmQsNcCexs6L168qHr8li5dqmpipbezpK1evVoN4mvSpAkqVaqkehxzkzIJaYOPPvpIhbE///zzmvtdu3atWl9eexlgKLXatWrVwv/+9z+3jsud7eVYpKRFAuc999yjeuml91Ve0w0bNsAT8p6R55UA+9hjj6ke1Ouuu07VgmdnZ6tvB+yvm/QC/9///R/Kly+PiIgIFXK//fZbVffs7jpEgYIhligASU+YsNdW5p5FQL6ydvaf//zH8RWw1GnKH1LpvZKvWT0lA8NknxJmpFdJ/jD/9ddfqlShKElodSbHf/ny5TzLGzZsqIJk7kFTBZHQ68z+lXrur8Zzh6jcr7X0wDq3SW7S+5d7mzlz5risI8HZ/ph8TS0B58EHH1RB0hdat26twmtu+/fvV8FQvqaXD0HOpQgHDhy45n5lUKJ80LGT7eXreekNd4c72y9fvtzx+5m7PEN+dzwh+5T2zR3mpa3kW5HffvtN3W/atKm6vu+++1SvdX690+6sQxQoGGKJApC9TtLdwGifnUC+7rYHTel5lHpTT9lnJ5BAKb1I0oM4evRo1dtblHKHqVKlShW4XHrFkpOTPdqv9MgKT79yrl69uro+duyYy3LpMbR/BS89d/mxz04gr520rXwdLr2JP/zwQ76D8wp6je3L3am5vRoJqbnJ74p8WJGQLzMHyOskx2wvaZEPMdeSXzCW193d19yd7S9cuKCupZczt/yWucO+z/x6xWWZPL/87rVv316VBZw9e1aVukgvsIRcGaBn5846RIGCIZYowEhYkN6/GjVqOIJT7qmwCponVGYSkK8u5Q+m9JzJwCZvycAZmXNUpgCTr+M//vhjFKWCRvJfbYS/N/u9Gnntcr/WEkCEXMvrK2UA3hyTBEh5DaU39KGHHlJhx5kMRCtooJl9XU+nFLOTXtbcZDq2c+fOqa/l5djka/Br9Vz7os2kd1Tkft0KWuYOKR8Q+ZWayDL5oGL/4CAD3TZt2qTaSN5n8k3FgAEDXKYMc2cdokDAEEsUYKZMmaL+cMp0Tp6SGkH5OlMGROWefN9TErjkq2WZWsrd3lA9keAog39++ukn9TWxt2FNPhRID9/kyZNdHpOwLCP085t6yx6g7cG6IBJApVfeExK4nH3xxRfwJ/YSGfvJPOzkQ5t9AFZh3Xzzzeo6d0+6fPsgHyi7du2ab/CVWm2ZBULkVzPszjpEesYQS6Rz0tsnYWbFihWqHlFGZ8uURzIy2xsSNqXHTUoNioJ8zS0T9EtvnYwGD0Qy7ZkMQpNgIq/Fvn371MA3aT8ZVCRfI7vboyhfO0vdrvTKykwUdjKISL5Cl5pPqW2WDwwyQ4D0kMrrLh9OZDDc1ciHjS1bthTqxBQSjKVkQz48ScmD9Gq+8sorqpzEn0jdq7wG0hbyestUatu2bVN1xtKD7In69eur99u0adPUIDIprZCptWRgoXj55ZfVtcwyMHbsWDU9mbSLhFzZxvmDhTvrEAUKhlginbIPxpKgWbNmTfXVv3xtKWGoMFNJFUSmwpI/zNJzuGrVqiI5ZglwEhTkD7W9jjD3QLDcFz3VAkpbSa/a7Nmz1ZnRZC5QmYNWTvMqPdVSbiA9hPbwcy3SCyvlI/KBw05G48uph5s1a4a7775b9eZJyPryyy/x5ptvYsaMGW7tVwY6yQj73PPEFkTqqGUwoIQueT4JyklJSeo5/Y1MxSYzCUiNttQbP/XUU2qwo8wtm7sn2V1ychEJ7fJhQepgZUYLqWeVGnN5HYXUMsvzyVRtso6cWESmb5MeXPlg4+46RIHCYC1oSCwRERE52IO71G4Tke+xJ5aIiOgapKRATtwgZ/siIv/AnlgiIiInUtIhc8dK2YZMqyUlOHKSApmTdfv27aoMgIh8jz2xRERETuTUv1K7K6fDlVkjZEorqdWW2m8GWCL/wZ5YIiIiItIc9sQSERERkeYwxBIRERGR5gT5+gD8lZxD/OTJk2pybm9PdUhEREREeclMrzJntMwlLSe9KQyG2AJIgI2LiyvUi0lEREREhXfs2DFUrVq1UNswxBZAemDtL6qcorG4e33lVJuxsbGF/hRC/oVtqQ9sR/1gW+oD21G/bZmYmKg6De25qzAYYgtgLyGQAFsSITYtLU09D0OstrEt9YHtqB9sS31gO+q/LQ0elG6y24+IiIiINIchloiIiIg0hyGWiIiIiDSHIZaIiIiINIchloiIiIg0hyGWiIiIiDTH5yH2xx9/RLdu3dC8eXMMGTIE//zzj9fbpKen44033sDNN9+M66+/Hrfffjt++umnYvwpiIiIiChgQqwEy379+uHWW2/F+++/j9TUVNxwww24dOmSV9s89dRTmDp1KkaMGIEZM2agXbt26NOnD37++ecS+smIiIiISLch9qWXXsLdd9+NMWPGoGPHjvjqq69UKP3oo4+82mbp0qUYNmwY7rjjDtUTO378eDRq1Ai//PJLCf1kRERERKTLEJucnIyNGzeie/fujmUhISG45ZZbsHz5cq+26dy5M1asWIGUlBR1f/fu3Th06JBa7m+SUix4enoyhrxjQmaW1deHQ0RERKQJPjvt7PHjx2G1WlGpUiWX5RUrVsTOnTu92uaTTz7Bfffdh/Lly6NcuXLqHL3vvvuuKkMoiNTRysVOzuVrPz2aXIpLWIgVK3ZkIjHFgL+PZKJ57cKfdo38h/yuyO9ocf7OUPFjO+oH21If2I76bUuLF38vfRZis7OzHT2pzsxmM7Kysrza5uWXX8bKlSvx2WefoXbt2qqMYOTIkahXrx46deqU776nTJmCiRMn5lkuAVjO8VucGscZsWavEX9siUflUsX6VFTM5M2YkJCg3qDO54QmbWE76gfbUh/Yjvpty6SkJO2F2JiYGHV9/vx5l+Vy3/6YJ9vIC/Pqq6+qAV0DBgxQy1q0aIH169ercFtQXazUzY4aNcqlJzYuLg6xsbGIiopCcerQKAVr9qZh7+lQlC9fvM9Fxf/mNBgM6veGIVa72I76wbbUB7ajftsyNDRUeyFWSgCqVq2KtWvXom/fvo7lf/31l5p5wNNtpNdUemxzB+GyZcvi9OnTBR6P9ObKJTd5gYs7jLSuJz3Ladh8UHqaDTAaWVKgZfLmLInfGypebEf9YFvqA9tRn21p9OJvpU//yj7yyCP49NNPsX//fnVfek8PHDiA4cOHO9Z54YUXXALrtbapUKECGjdujHfeecfRRb1r1y5899136Nq1K/zRddVMCA22IuGyFQdO2UomiIiIiMgPe2LFM888g6NHj6rQGR0drepaP//8czRt2tSxzqlTp1xOZuDONnPnzlVTbEmglYFdZ8+eVSdFeP755+GPgoMMaBRnxaZ/DNi4LxP1qvi0WYiIiIj8nsEqlbU+JvWnFy5cUKUCwcHBLo9JCYDMA1uzZk23t3GekkvWqVy5coHrXO2YJCRLjW1x18RKfchrc87hs99N6N3GjLce5ugurZK2lA9NMjMGywm0i+2oH2xLfWA76rctE73IW37R5ScHXdCBSx1sYbexi4yMVBctaFrddr1hf6YasSf1IkRERESUP4488RNSThBkBM5csuD4ec4xSkRERHQ1DLF+IjQEaFTDpG5v3J/p68MhIiIi8msMsX6kVR1b3e6GfQyxRERERFfDEOtHWtWzlShv2p//GcuIiIiIyIYh1o+0qB0EGc916Ew2ziewLpaIiIioIAyxfiQ6woh6lVkXS0RERHQtDLF+plU9W10sB3cRERERFYwh1s+0qssQS0RERHQtDLF+Orhr97FsJKWwLpaIiIgoPwyxfqZCaROqxRohJwPefJCzFBARERHlhyHWD7GkgIiIiOjqGGL9eHAXT3pARERElD+GWD/UOmdw147DWUjPtPr6cIiIiIj8DkOsH6pW3ojYaAMys4Dth1gXS0RERJQbQ6wfMhgMjrpYlhQQERER5cUQ66d40gMiIiKigjHE+il7T+yWA1nIymZdLBEREZEzhlg/Va+KCaXCDLicbsXuY6yLJSIiInLGEOunTEYDWta1nb1r4z6GWCIiIiJnDLF+jCc9ICIiIsofQ6xGQqxVzkNLRERERApDrB9rXCMI5mDgUrIVB09l+/pwiIiIiPwGQ6wfCwkyoHkte28s62KJiIiI7Bhi/Vwrx+CuTF8fChEREZHfsCUkHzp69ChmzpyJM2fOoEmTJrj//vthNpu92uaJJ55AWlpanu1uuOEG3HfffdDeSQ9SVV0sEREREflBT+yuXbvQrFkzbN68GXFxcZg2bRpuvvlmZGZmerVN69at0a5dO8elatWqmD59OjIyMqA1Uk5gMgInL1pw4gLrYomIiIh83hM7btw4tGjRAt999x0MBgOGDh2KmjVrYtasWXjwwQc93iZ3b+vLL7+MyMhIDBo0CFoTEWrAddWCsONwliopqNLe5OtDIiIiIgrcnljpFV26dKkKlhJGRaVKlXDTTTdh0aJFRbaNTE312WefqW1KlSoFLWqtSgo4uIuIiIjI5z2xUtcqJQA1atRwWS69qitXriyybX777TccPnwYw4cPv+rxpKenq4tdYmKiurZYLOpSnGT/ErYLep6WdUyY8YuE2IxiPxYq3rYkbWA76gfbUh/YjvptS4sXfy99FmJTU1PVde7eUblvf6wotpFa2KZNm6JNmzZXPZ4pU6Zg4sSJeZafO3cu30FiRUkaMCEhQTWq0Zi3c7xaafk3CAdPWbDv0FmUjijWw6FibEvSBrajfrAt9YHtqN+2TEpK0l6IjYqKUteXLl1yWX7x4kXHY95uI8sXLFiAN95445rHM378eIwaNcqlJ1YGjsXGxhZ4PEXZoFIeIc+VX/ApD6Bu5QTsP5mNI/GlUa9mSLEeDxVfW5I2sB31g22pD2xH/bZlaGio9kKsBEQJhzLbQI8ePRzL//77bzRu3LhItpk9e7Z6oe69995rHo9M0ZXf1F7yApdEGJHjvNpzySloJcRuOpCFbi09b3CCz9uStIHtqB9sS31gO+qzLY1e/K302V9ZOeiBAwdixowZuHz5slq2YcMGrF27FoMHD3asJ/PBvvjii4Xaxk7WGzBgAEqXVt/Ha1qrevaTHvDMXUREREQ+7Sp65ZVXEBQUpOZ9vfPOO3HLLbfgsccec+lllQFb8+fPL9Q2YuPGjdi2bds1B3RphfTEil1Hs5CcxkFDREREFNh8Ok9suXLlVNhcvny5OvvWhAkT0Lx5c5d15GxcPXv2LNQ2Ijg4WE2tJWfp0oNKZU2oWs6I4+ct2HowC50asS6WiIiIApfPTzsrYfO2224r8PFOnToVehshPbVy0ZOWdYNx/Hy6OgUtQywREREFMo480ZDWOSUFG/YVfFpeIiIiokDAEKvBwV3b/slCRqbV14dDRERE5DMMsRpSs4IJMaUMyMgCdhzmLAVEREQUuBhiNTavWqt6OSUF+1lSQERERIGLIVajU21tZF0sERERBTCGWI2G2M0Hs5BtYV0sERERBSaGWI1pEGdCRKgByalW7D2e7evDISIiIvIJhliNMRkNaFHHfgpa1sUSERFRYGKI1XJdLAd3ERERUYBiiNWg1vYZCvZlwmplXSwREREFHoZYDWpaIwghQcCFJCsOn7H4+nCIiIiIShxDrAaFBBvQtGZOXSxLCoiIiCgAMcTqoKSAiIiIKNAwxGoUB3cRERFRIGOI1ajrawfBaACOn7fg9EXOF0tERESBhSFWoyLDjLiumr0uNsvXh0NERERUohhiNaxVXQ7uIiIiosDEEKthLXnSAyIiIgpQDLE6mKFg34lsXErmfLFEREQUOBhiNaxsKSNqVTSp25sPcKotIiIiChwMsRrXqh4HdxEREVHgYYjVuNY5dbE86QEREREFEoZYjWuVUxe762gWUtKtvj4cIiIiohLBEKtxVWJMqFTWiKxsYOs/rIslIiKiwODzEJudnY2VK1di3rx52Lt3b5Fuc+HCBSxevBi///47MjIyoFcsKSAiIqJA49MQe+nSJbRv3x733HMPpk+fjlatWmHMmDFFss0777yD6tWrY+rUqeoi2xw/fhx6Hty1iWfuIiIiogBhSz8+8txzzyExMRF///03SpUqhTVr1qBTp07o1q0bbr31Vo+3+f777zFq1Cj8/PPPjmXSY5ueng49apUzuEvKCTKyrAgJMvj6kIiIiIj8rydWvppfvny54/7atWtx//33Y9KkScjMdK8u02q14uuvv8ZDDz2kwqjo0KED2rRpgy+//NKrbaZMmYI77rjDJQjXr18ftWvXhh7VrmRCmUgD0jKAv49k+fpwiIiIiPyzJ3by5MkIDQ3FTTfdhOTkZPTu3RutW7dWdapy/7XXXrvmPo4dO4b4+Hg0btzYZXmTJk2wefNmj7dJTU3Fxo0bVdA9cOAAtm3bhsqVK6vjCwoq+MeVXlrnnlrp7RUWi0VdipPsXwK6N8/Tsk4Qft2aiQ17M9Cspu0ECFTyiqItyffYjvrBttQHtqN+29Lixd9Lj0LsrFmzsGrVKnX7l19+UT2cS5Yswa5du9C9e3e3QmxCQoK6LlOmjMvysmXLOh7zZBsZzCUviByPHEezZs2wZcsWhIeH46efflJ1svmR3tuJEyfmWX7u3DmkpaWhOMnxyvFLoxqNnpUp169kwK9bTVjz92X0bZFU5MdIJdeW5HtsR/1gW+oD21G/bZmUlFSyIfbs2bOIjo5Wt6WsoGfPnup2jRo1VIh0R1hYmLqWnltn8sPYH/NkG+khFgcPHlR1s3Jfyh+k7GD06NFqRoP8jB8/XtXROvfExsXFITY2FlFRUSjuBjUYDOq5PA0+Xa7PwvtLErHzmAnlysXAaGRdrC8URVuS77Ed9YNtqQ9sR/22ZWhObiuxECtf58uI/169emHOnDlqGishPbGNGjVyax8SEIODg3HkyBGX5XK/Vq1aHm9Trlw5lC5dWg30sr8wISEh6NGjB2bPnl3g8ZjNZnXJTV7gkggj0qDePFej6sGIMBuQmGLFgVNWNIhjSYGveNuW5B/YjvrBttQHtqM+29Loxd9Kj7Z8/fXX1RRWLVu2VEG2bdu2arkE25EjR7q1DwmMXbt2xdy5c12+upc5XWWfzoPG7CHZ3W369OmjArUzuV9QKYEeBJkMaF7b9plk436e9ICIiIj0zWCVogQPu4Pla33nr9r37duHunXrqoTtju3bt6Njx45qYJjM4zpjxgzV07p69WrVeyqGDRumguzOnTvd3kZ6ZiVYS2+srLtu3To1e8GyZctwww03uHVsUk4gJRNSt1ES5QRSolG+fHmvPpF8sDgFb3+fgh6tQvDOo8V7zFS8bUm+xXbUD7alPrAd9duWiV7kLY+n2FqxYoXjyexTbH3zzTfIynJ/iqemTZti69atqFmzpppFQPYh+7WHUSFBVXpWC7ON9LjKOnXq1MH69evV/d27d7sdYLU+X6yc9MDDzyZERERE+u2JnTBhgqo3ffbZZ1VvrAzokimspCe2f//+bs1O4O+02BOblmFFyxEXkJkN/PpKGVQrz7rYksbeAn1gO+oH21If2I76YfF1T6xMsXXfffflmWJr0aJF6mQE5BuhIQY0qWmri92wj3WxREREpF9GX02xRcVbUsDBXURERKRnRm+m2JKzZMkUW3KCg8JOsUXFgyGWiIiIAoHPptii4iGnn5XJIY6cteBsPE99SkRERPrk0ckOOnfurEoKck+x9eKLL6optsh3SoUb0aCqCbuPZauSgp6t857AgYiIiEjrPB4KLyPKco8iq1evnttzxFLxYUkBERER6Z3HIVbmYb333ntVSUGLFi3UbVlGvte6nm1wF2coICIiIr3yKMQuXLhQhdfTp0+jR48eqi5WbsuyH374oeiPkjzqid13IhuJKayLJSIiIv3xqCb2hRdewFtvvYWnnnrKZbkM9nr++efRt2/fojo+8kC5aCNqVDDi8BkLNh3Iwk1Nr5zNjIiIiChge2L37NmjTveamyzbu3dvURwXFVVdLE96QERERDrkUYiVU4Vt2bIlz/JNmzYhNja2KI6LvMTBXURERKRnHpUTDB8+HAMHDsT48ePRpk0btWzdunWYMmUKHn/88aI+RvJAq5zBXTsPZyE13YowM2eNICIiIv3wuCY2LCwMkydPdpxmNiYmBmPHjsWYMWOK+hjJA3HljKhQxogzlyzYdigT7RqwLpaIiIgCvJxA5oiVwHr+/Hk1K8GZM2fUbVkmj5HvyXy9reraPqNs3J/l68MhIiIiKlJeJ84KFSqoGlnyPxzcRURERAj0coJBgwa5vdM5c+Z4ejxUDCc92PpPJtIyrAgNYV0sERERBViIjYyMLN4joSJXp5IJVWKMOHHBgmVb0tGnbShfZSIiIgqsEPvpp58W75FQkTMaDbizYyje/SEF365kiCUiIiL94CgsnburoxkGA7B2TyaOns329eEQERERFQmGWJ2rHGNCp+tstbHzVqX5+nCIiIiIigRDbAAYcIOtFva7NenIyrb6+nCIiIiIvMYQGwBubh6CMpEGnI23YOXOTF8fDhEREZHXGGIDQEiQAf3am9XtuStZUkBEREQBGmItFgvee+89XH/99YiOjnYsHzduHI4fP16Ux0dFXFLwx/YMnEuw8HUlIiKiwAuxb7/9Nt544w0MHz4ciYmJjuUNGjTApEmTCr2/tLQ0dfpaCcdFsc2FCxdw+PBhl8vJkycRyOpUDkKL2kHItgAL1rA3loiIiAIwxH700UeYO3cu/v3vf7ssv/XWW7FgwQK39yMB9KmnnkLp0qVRv359VK5cGfPmzfN6m+eeew6NGjVCly5dHJcHHngAga5/Tm+szFJgtXKAFxEREQVYiD1y5AiaNGmibhtkEtIc4eHhLj2z1/L666/jyy+/xMaNGxEfH48JEyZg8ODB+Pvvv73epkePHi49sUuXLkWg69HKjAizAYfPWLBxf5avD4eIiIioZENsjRo1sGnTpjwhdv78+aqkwF0ffPABhg0bhsaNG6v9SM+u7PuTTz4pkm3Onj2L1NTUQv98ehURakDPNiHqNgd4ERERUcCF2FGjRmHo0KGYM2eOur9ixQqMHz8eTz75pHrMHRIwjx49ig4dOrgs79ixIzZs2OD1Nt99950K1DLwrE2bNgXuM9D8K6ekYOmmdCSmcIAXERERaVOQJxs98sgjyMjIwMiRI1WNqtSclitXTn3VL+HWHefOnVPXMTExLstlP2vWrPFqm7Zt2+Lxxx9XJQ/Jycnqdrdu3VTJQaVKlfLdd3p6urrY2csi5OcrzIAzT8j+pUa1uJ9HNK5uRN3KJuw/mY1Fa9MwuIst1JL22pKKD9tRP9iW+sB21G9bWrz4e+lRiBUjRozAE088oabUkgOIi4uD0eh+x6593aws19rMzMxMmEwmr7ZxHsQVGRmJjz/+WJU6yEWOOT9TpkzBxIkT8w3OMhNCcZLXLyEhQTVqYV5DT3VrbsD+kyZ8/cdldL3O/Rpm8r+2pOLBdtQPtqU+sB3125ZJSUklH2KF1KRKePVE1apV1bVMk+VM7lepUqXIthGhoaGqB1YGeBVEyiGcSyGkJ1Z+ttjYWERFRaG4G1ReS3mukgg+93S14ONf4rHvpAHn08riumpe/RqQD9uSigfbUT/YlvrAdtRvW4aGev6NsEfpRQZLTZs2DatXr8bFixfzPL5q1apr7qNUqVJo0aKFmjVg4MCBjh7V3377TU2h5Tznq3zNL1NpubuNvEDOAULmiJUZFWrXrl3g8ZjNZnXJTfZTEmFEGrSknism2ohbrw/BTxsyMH91BhrXsA32Iu21JRUftqN+sC31ge2oz7Y0evG30qMQ+9hjj2HZsmXo37+/CpWekumxZB8tW7ZE+/bt8eabb6qyANm/81nA1q5di507d7q1jQTezp07Y/To0WquWBkIJvPGVqtWDffee6/Hx6o3AzqFqhD7w9p0jBsQgdCQK7NMEBEREfk7j0LswoULVW+rhERv3H777WqGg3feeUf17MpArD///FMN1LKT286lAtfaRnpTP/30UzXITM4eVqZMGTVn7NixY1VPLtm0bxiMKjFGnLhgwS+b09G3HQd4ERERkXYYrB6cuklC5e7du4u9VtSXpCZWpueS4uOSqImV6cPKly9fol9Bv7coBdMWpqBt/WDMejq6xJ5Xz3zVllS02I76wbbUB7ajftsy0Yu85dFf2d69e2PmzJmebEp+5K6OZsi5KtbtzcSRM9m+PhwiIiKi4i0nuHTpkjqxwbx581CnTh2Xs3YJ+Tqf/F+lsibc0CgYf+7MxLzVaRh9Z4SvD4mIiIio+HpipftXZgeQaasuX76sTijgfCHtGJBzBq8Fq9ORlV3oyhIiIiIi7fTE2k83S9p3U7MQlC1lwNkEC/7ckYGbm+edZoyIiIjI33DkSYALCTLgjg623thvV1057S4RERGRLnpiBw0a5OiFtd8uCHtqtaV/JzOmL03FH9szcDbegvKl+dmGiIiIdBJiIyMj871N2le7UhBa1AnC5gNZWLAmDY/0DPf1IREREREVTYh1nnGAsw/o8wxemw8k49uVaXi4R1ieGSeIiIiI/Am/NyaleyszIkINOHrOgvV7M/mqEBERkb5qYt3BmljtkQDbu40Z3/yZpgZ4tW0Q4utDIiIiIvK+J1bqYN29kHYHeImlm9KRmGLx9eEQERERFW1NLOlT05pBqF/FhL0nsrFoXTruuSnM14dERERElC/WxJKDDOayn8FLBngRERER6eqMXampqZg2bRpWr16Nixcv5nl81apVRXFs5AN92pnx2rzL2HU0G38fyUKj6h79ihAREREVK48SymOPPYZly5ahf//+aNGiRdEfFflMmUgjbmsRgh/XZ+DbVWloVJ01zkRERKSTELtw4ULV29qoUaOiPyLyuf6dQlWIXbQ2HeP6RyDMzDljiYiISAc1seHh4YiLiyv6oyG/0L5BMKqWMyIp1Yqlm9N9fThERERERRNie/fujZkzZ3qyKWmA0WhQvbFiHgd4ERERkV7KCS5duoQnn3wS8+bNQ506dfKcopTTcWnfnR3MmLYwBev3ZeHQ6WzUrGjy9SERERERedcTazQaMXDgQFSqVAmXL19GcnKyy4W0r2JZE25oHKxuz1vF6baIiIhIBz2xPK1sYJA5Y1fsyMT3f6VhZL9wBAdxgBcRERH5B57sgAp0U9MQxJQy4FyCFSt2ZPCVIiIiIu2H2K1bt+Lee+9Fy5Yt1VyxcluWkX5Iz+sdHXIGeK3iLAVERESk8RAr88RKeD19+jR69OiBXr16qduy7Icffij6oySfuauTWV3/sT0Dpy9lsyWIiIhIuzWxL7zwAt566y089dRTLsvfeecdPP/88+jbt29RHR/5WO1KQWhZNwib9mdhwZp0PNYr3NeHRERERORZT+yePXtw//3351kuy/bu3Vuofc2dOxcdO3ZUU3Xdcccd2L17d5FuI2G7atWqeOmllwp1XHTFgJw5Y+evSoPFYuVLQ0RERNoMseXLl8eWLVvyLN+0aRNiY2Pd3s+CBQtwzz33YMiQIZg/fz5KlSqFG2+8EefOnSuSbdavX493330XwcHBiI+PL8RPSM66tzIjItSAo+csWL8vky8OERERaTPEDh8+XM0T+/bbb2PNmjXqMnXqVAwaNEg95q5Jkybhvvvuw6OPPopmzZphxowZsFqt+PDDD73eJjExEXfffbc68UJ0dLQnPyblCDcb0KetrTb225Uc4EVEREQarokNCwvD5MmTceHCBbUsJiYGY8eOxZgxY9zah4RM6c0dN27clYMJCkLXrl3x559/er3Nww8/jH79+qnHyHv9O5kxZ0Ualm5Kx4S7IxAdwdnZiIiISGMhVs7YJYFVLmfOnFGnnZUSg8I4ceKEuq5YsaLL8goVKmDbtm1ebSO9r1K3O3PmTLePJz09XV2cA7OwWCzqUpxk/9KbXNzP441G1YyoX9WEvcezsfCvNNx7s61OlrTXlnRtbEf9YFvqA9tRv21p8eLvpUchNneA9IT9oKUn1ZnUr2ZnZ3u8jQzyeuaZZ7BixQqYzbavwN0xZcoUTJw4Mc9yqbVNSyve067Kz5WQkKAaVT4g+KvuzQzYe9yEOX9cxq2NEmHgCbw025Z0dWxH/WBb6gPbUb9tmZSU5LsQ6yn7ALDz58+7LJf7BQ0Oc2eb3377DcnJyejWrZvj8bNnz+LgwYOYN28ejhw5ApPJlGff48ePx6hRo1x6YuPi4tR+o6KiUNwNKr3Z8lz+HHwG32LBh0vjceC0AefSyqJxdZ/9+vgtrbQlXR3bUT/YlvrAdtRvW4aGev7Nrs9SiJQfVK9eHatXr8btt9/uWL5q1Sr06dPH420eeOABVQvrTAJt+/bt1TRb+QVYIb22+fXcygtcEmFEGrSknstTZUsZcVsLMxavT8f8VRloWjPE14fkl7TQlnRtbEf9YFvqA9tRn21p9OJvpU//yj7++OOYPn26qmeVZD5t2jQcPXpUDcqyk7rbW265xe1tIiIi1LywzhcpN4iMjFS3yTsDbrAF/UXr05GazjljiYiIyDd8+n3w6NGjcerUKbRr1071kMrX9nPmzEHDhg0d61y8eFGd0rYw21DxaVs/GFXLGXH8vAU/b0rHHR04wIuIiIhKnsEqlbWFlJqaqnpA5Wt9CZm5ydf7hZGRkaGKfMuVK6e6mJ1dunRJPZ57ANnVtslNamKlVKAw88VKTaysL89REjWxcoxSLqGFr6A/XJyCqd+noFXdIHw1rrSvD8evaK0tKX9sR/1gW+oD21G/bZnoRd7yqCf2sccew7Jly9C/f3+0aNEC3goJCSlwMFeZMmUKvU1uhZ3+i67ujo5mvLMwBRv3Z+HQ6WzUrJh/nTERERFRcfEoxC5cuFD1tjZq1Kjoj4j8XsUyJnRuEow/tmdi3qo0PN0/wteHRERERAHGo+87w8PD1fRTFLgG3GCrhf12ZRouJXNifyIiItJAiO3du3ehzoZF+nNT0xDUq2JC/GUr/vvtZV8fDhEREQUYj8oJZLDVk08+qU4eUKdOnTwDq+S0r6RvQSYDJg2NxKBXEzB/dTr6tTejbQPOG0tERER+3BMro8kGDhyISpUq4fLly+oMWc4XCgzX1w7G4BttZQUTZl1GeibnjSUiIiI/7omVeVmJxOg7w7FsSzoOncnGRz+l4KnbOciLiIiIih8nsiSvlAo34vnBker2/35KxYGTWXxFiYiIyH/P2JWSkoK1a9eqU75mZbkGl2HDhhXFsZFGdG8Zgi5NbVNuTZiVjNlPR8NovPoJKIiIiIhKPMRu27ZNzVAg9a/x8fHqbFpnzpxRj1WrVo0hNsDIwL4X74nEuj2X1AkQ5q1Kx78683S0RERE5GflBKNGjcKQIUPULAXi9OnTOHz4MDp16sQAG6CqxJjwVD9bPex/513G+QTOHUtERER+FmI3bdqEMWPGOHrhMjIyUL16dUyfPh2ffPJJUR8jacTQrqFoVM2ExBQrXvmGc8cSERGRn4XYhIQElC1bVt2OjY3FiRMn1G2Zcuvs2bNFe4SksbljS0HKYRevT8efOzN8fUhERESkU17PTiAlBC+88IIa5DV27Fg0atSoaI6MNKlxjSDVIytemp2M1HTOHUtERER+EmKfe+45x+3XXnsNf//9N9q3b48lS5bg/fffL8rjIw2S2thKZY04ft6C9xal+PpwiIiISIc8CrGTJ0923JbTzm7ZsgWpqalqcFe7du2K8vhIgyJCDXjxbtsgrxm/pGL3Mc4dS0RERH56soPQUE6pRFfc3NyMbi1DkG0BXvgiGdkWlhUQERGRj0OsxWLBe++9h+uvvx7R0dGO5ePGjcPx48eL8PBIy54fHIHIMAO2H8rCV3+k+fpwiIiIKNBD7Ntvv4033ngDw4cPR2JiomN5gwYNMGnSpKI8PtKwCqVNGH1nuLr91ncpOH0p29eHRERERIEcYj/66CPMnTsX//73v12W33rrrViwYEFRHRvpwOAbQ9G8VhAup1kx6SvOHUtEREQ+DLFHjhxBkyZNHCc7sAsPD3fpmSUyGmXu2EgEmYBlWzLw65Z0vihERETkmxBbo0YNddau3CF2/vz5qqSAyFn9qkF48LYwdfvlry4jOY2npCUiIiIfhNhRo0Zh6NChmDNnjrq/YsUKjB8/Hk8++aR6jCi3J/qEIy7WiNOXLHh7AeeOJSIiIu8EebLRI488goyMDIwcOVLNVNClSxeUK1cOr7/+ugq3RLmFhhjw8r2ReGBqImb9noa+7cxoWjOYLxQRERGV7DyxI0aMwKlTp3D06FF1koMzZ87giSee8HR3FAA6NgpB37ZmWK22uWOzsjl3LBEREZVgT6yd1MPGxcV5swvs378fn376qQrBMljs0UcfRUREhNfb/PLLL1i6dCmSk5PRqFEj3HfffS5z2pJvjB8YgT93ZmD3sWzM/DUVD3WzTcFFREREVGw9sa1atXLr4q6tW7eqEyacPHkSrVu3xpdffonOnTurUgVvthk2bBg++OADNQCtefPm+Oqrr9Q258+fL8yPS8UgJsqIsQNsHzimLUzB8fOcO5aIiIgKz2C1ype7bq5sMKB69ero1asXgoODr3oyBHd0795d7XPJkiXqvoTMatWqYerUqaru1tNtpMyhUqVKjm2kN7Z06dL47LPPMGTIELeOTaYKk57bhIQEREVFoThJXfHZs2dRvnx5GI1FdiZgvyW/ckNeT8D6fVno3DgYnzwV5TLLhZYFWlvqFdtRP9iW+sB21G9bJnqRtwr1V/a1116D2WzGvHnzEBQUpEKjBNbcF3ekp6fjt99+w4ABAxzLZHDYzTffjB9//NGrbZwDrDh27Biys7NV2CXfk8D68tBIBAcBf+7MxE8bCu55JyIiIvK6Jnbs2LHq8ueff6qaVCkdaNasGR566CEMHDgQkZGRbu9LBoRlZWXlCZbS0ytTdnm7zbZt2zBlyhSV8Ldv347p06fjxhtvLPB4JCDLxc5+0gb5xCCX4iT7l97J4n4ef1KjvBGP9AjDe4tS8Z85yejQ0IToCO33XAZiW+oR21E/2Jb6wHbUb1tavPh76dHALqlBlcu7776r6k3lq/ynnnpKfW3vrrS0NHWdO/jKfftj3mxToUIF9OvXT5UbyOXDDz/E7bffjpiYmHz3LYF34sSJeZafO3euwOMpKtKA0o0ujRpIX0H3awn8sNaEo+eAyV9exNP9tB/8ArUt9YbtqB9sS31gO+q3LZOSkkp+dgJ5chlktXbtWhw6dEj1yBaGfaaAixcvuiy/cOGCql/1dpuKFSti0KBB6rb0FNerV0+F7cmTJ+e7bzlZg/OJGqQnVmZeiI2NLZGaWPmKXZ4r0ILPf+7LxJA3krBooxGDb45Gizranjs2kNtST9iO+sG21Ae2o37bMjQ0tORC7IkTJzBz5kw1SEqCnpzcQE5BW9jTzUpAlOC5c+dO9OzZ07F8x44datqsotpGhIWFoWbNmjhy5EiB60itr1xykxe4JMKINGhJPZc/advAjP6dMjBvVTomzE7B9xNKIyRI24O8ArUt9YbtqB9sS31gO+qzLY1e/K0s1JYyK0GtWrWwZs0a/Pe//8Xx48fVWboKG2DtP8Ddd9+talXt9aerVq3C+vXrce+99zrW+9///oenn37a7W2krnX+/Pkuz7VlyxZs3LgRnTp1KvRxUvEb2z8CMaUMOHAyG9N/TuVLTkREREU/xVaVKlXUV/VXI4HRHfHx8WrKLAnD1113nQrHjz/+uJoFwXnOVylZkN5Xd7aRgV/333+/CrYSrmV9OR7Zj5QTmEwmt46NU2yVrEXr0jD6k2SEBAGLJ5ZBjQrutZO/4TQw+sB21A+2pT6wHfXDUoRTbBWqnGD06NEoSlIaICFUQqqcfUtOUFCnTh2XdWQar/79+7u9jUz9NXv2bBVyZYYCOZOXnLFLai/If/VuY8aCNelY9XcmJsxKxszR+pk7loiIiHzcExtI2BNb8o6ezUavFy8hPRN47cFI3NHB82JvX2FvgT6wHfWDbakPbEf9sPjqZAdExalaeROe6BOubk+ZexkXk7Q/5RYREREVD4ZY8isP3haG+lVMiE+24sXZyci28IsCIiIiyoshlvxKcJABk++LhMkILN2UgXHTGWSJiIgoL4ZY8jvNagXj7UdKIcgE/LAuHWOnJyMrmz2yREREdAVDLPmlbi3NjiC7aF06np6exCBLREREDgyx5Ldua2HGtEdLIdgE/Lg+A6M/YZAlIiIiG4ZY8mu3XG/GtMdsQXbJRluQzcxiaQEREVGgY4glv9e1uRnv/TsKwUG2IDuKQZaIiCjgMcSSJtzULMQRZGXWgpEfJyGDPbJEREQBiyGWNOOmpiH44PEohAQBy7YwyBIREQUyhljSlBubhOCDJ2xB9tctGXjqI/bIEhERBSKGWNKczo1D8NGIKJiDgd+2ZuDJD5OQkcnBXkRERIGEIZY0qVOjEHz0hC3I/r4tA098mMggS0REFEAYYkmzOjYKwcdPRiE0BPhjeyYe/yAR6eyRJSIiCggMsaRpHRqG4OMRtiC7Ykcm/v0+gywREVEgYIglzWvfMASfPBmFsBBg5c5MPPZeItIyWCNLRESkZwyxpAttG4Tgk6eiVZBd9TeDLBERkd4xxJJutKkfjE9HRiPcDKzelYlH301Eajp7ZImIiPSIIZZ0pXW9YNUjK0F2ze5MPPoegywREZEeMcSSLoOs9MhGmA34a3cmHnk3ESnskSUiItIVhljSpVZ1gzH9/6IQEWrA2j2ZeGQagywREZGeMMSSbrWoE4wZOUF23d5MDH8nAZfTWCNLRESkBwyxpGvX1w7GZ/8XhcgwAzbsy2KQJSIi0gmGWNK95jlBtlSYARv3Z2HYOwlITrP4+rCIiIhIyyE2IyMDS5Ysweeff45NmzYV2TZHjx7Ft99+iwULFuDEiRNFfNSkNc1qBeOzUbYgu2l/Fu6cFI+/dmf4+rCIiIhIiyH23LlzaNmyJUaOHIkff/wRt9xyCx599FGvtrFarRg8eDC6dOmiQuz06dNRt25dvP322yXwE5E/a1ozGJ+PjkL50kYcPmPBfW8mYuz0JFxMYq8sERGR1gT58snHjx+vrrds2YLw8HDVq9q6dWv06dMHvXr18mgbCbG33347vvzySxiNtow+a9Ys3H///Wqd2rVrl+BPSP6mSY1gLHm5NKZ+n4Ivl6fh+7/S8ceODDwzIAJ3dDDDYDD4+hCJiIjIn3tiLRYL5s6diwceeECFUSE9rB06dMCcOXM83kaC66BBgxwBVnTr1k1tu3fv3hL52ci/lQo3YsLdkZg7Phr1q5oQn2zFM58lY8jrCTh4KsvXh0dERET+3BN77NgxJCUl4brrrnNZLvc3btxYZNuIxYsXw2QyoWnTpgWuk56eri52iYmJ6lrCr1yKk+xfepCL+3nIVZMaJsx7Ngpf/J6G935Ixfp9Weg7MR4Pdw/Fwz3CYA4ufK8s21If2I76wbbUB7ajftvS4kX28VmItYfE0qVLuywvU6aM47Gi2Gb37t0YNWqUulStWrXA45kyZQomTpyYbw1uWloaipM0YEJCgmpU5x5kKhl9mgOtqgNTFxmxdp8R7y9Oww9rUzG6rwUtahVuXlm2pT6wHfWDbakPbEf9tmVSUpL2QmxYWJi6zn3wEkbtpQLebvPPP//gtttuQ8+ePfHqq69e9Xik1laCrvM+4+LiEBsbi6ioKBR3g0otpjwXQ6xvlC8PfFbPiqWbMvGfOZdx7DwwcoYJ/dqHYFz/cJQp5d6HC7alPrAd9YNtqQ9sR/22ZWhoqPZCbPXq1RESEoJDhw65LJf7derU8XobWSYzFEi9rAzsulY4NJvN6pKbbFcSwVIatKSeiwrWs40JNzQOwZsLUvD1HzLwKwN/7Mgs1MAvtqU+sB31g22pD2xHfbal0Yvc47PEFBwcjO7du6sBWdKlLE6ePInly5ejb9++jvVWrFihBnMVZpvDhw+rANu+fXs1S4HUwxIVZuDXS/dE4ptnolG/ypWBX0PfSMQ/pznwi4iIyB8YrPY06AN79uxRPaWdOnVCu3bt8MUXX6B8+fL4/fffERRk6yQeNmwY1q5di507d7q1jdSvNmzYECkpKZgwYYJLgJVg26BBA7eOTcoJoqOjVd1GSZQTnD17Vv0c7In1L5lZVny+LBXvLkpBWgYQHAQ81iscD3cPQ0g+A7/YlvrAdtQPtqU+sB3125aJXuQtn84TK4Fy+/btKoieOXMGY8aMwdChQx0B1h48pYzA3W2ys7PVlFpix44dLs/XpEmTEvvZSB+CgwwY3iMc3VuZ8dKXyVi5MxPTFqZg0bp0TBoSiTb1g319iERERAHJpz2x/ow9sZSbvFV+2pCB/8xJxvlE29vmro5mjB0QgTKRtsoc9hboA9tRP9iW+sB21A9LEfbEchQRUSEK0Xu1MePnyWUw+EbbaMr5q9PR/flLWLAmzVGnTURERMWPIZaokKLCjZg4JBJznolGvSomXEq2YtyMZNz3ZiIOncnm60lERFQCGGKJPNSiTjAWvFAao+8MhzkYWLsnE7dPTMCbPxix6yhnMSAiIipODLFEXg78eqRnOH58uQw6NQpGRhawcL0Rd05OxF2T4/HtyjRcTmOZARERUVHz6ewERHpRLdaE6SOj8NfuDHyxLAErdxmx43AWdhxOxivfXMbt7cz4V+dQXFeNbzkiIqKiwL+oREU48Ktdg2DUKmuBKbQsFq7NwDd/puHIWQu++iNNXZrWDMLAzqFqgFi4+dpn/yIiIqL8McQSFYOYKCOGdQ/Hg7eFYe3eTHyzIg2/bsnA9kNZ2H4oGVPmXkbfdmYVaBvG8W1IRERUWPzrSVSMjEYDOjQMUZcLiRZ8tzpN9c4ePWfBV8vT1KVZLVvvbM/W7J0lIiJyF0MsUQn2zsrZvx7qdqV3dtmWDGz7Jwvb/rHVzkrv7KDOoWjA3lkiIqKrYogl8mHv7PkEC75bY+udPebUO9vcqXc2jLWzREREeTDEEvlQuWgjHu4RjmHdwvDXnpza2a0Z2PpPFrY6987eGIr6Vfl2JSIisuNfRSI/6Z3teF2IupxLuFI7e/y8BV8uT1MXqZ3t2jwE7RsEo3GNIJiMnN2AiIgCF0MskZ+JjTaqEygM7x6Gv3ZnYs6fafhtq7121nYmsFJhBrSpH4z2DYNVqK1T2aSm+CIiIgoUDLFE/tw72yhEXaR39udN6SrUrtuTiaRUqwq2chGx0TJHbYgt1DYMRpUYk68Pn4iIqFgxxBJppHd2yM1h6pJtseLvI1kq0Mpl04FMnEuwYtG6dHUR1WKNaN/QFmrlBAxlS/EM00REpC8MsUQaI7WwTWsGq8sjPYH0TCu2HLQF2rV7MtUJFWQe2qPnbHW1okGcSZUdSLBtXS8YEaEsPSAiIm1jiCXSOHOwrZRALiI51YIN+6SnNkMF270nsrHnmO3y2bI0BJmgTn8rvbQyzZcMGAsJYqglIiJtYYgl0pnIMCNuahaiLkLmopWTK9hDrcx4sPlAlrq8vygVYSFAizrB6vS39aqaUK9KEOpUMiEkmMGWiIj8F0MsUQDMRdu7jVldxLFz2bZ62j0ZWLs7ExeSrFi9K1Nd7ExGoHp5E+rnhFoVbisHIS7WqAacERER+RpDLFGAiYs1qcu/OofCarVi/8lsbD6Qib3Hs7HvRBb2Hc9GQooV/5zOVpclG20zIAjpta1TOUiF27qVbeG2fpUgFZSJiIhKEkMsUQCTuWVVT2uVK/8VSLA9E2/B/hO2UGsPtwdOZiM1A9hxOEtdANtMCKJsKft+cnpuq5hQt0oQB5AREVGxYYglojzBtmIZk7rc0NhWVytkaq8jZ7NVT+3eE1mOkHvkrAUXk6xqZgS5OKtazqhOxCDz1laJMaJyznWlsiaUizKwNIGIiDzGEEtEbk/tVatikLp0b2WrrxWp6VYcOGULtXuPZ2FfTriVuWtlEJlcANdwK0KCgEplrwRbua6s7ttuVyxj5KwJRERUIIZYIvJKmNmAJjWC1cXZxSQpSchSdbUnL1pw8oIFJy5kq+uz8RZkZEH14solP3IWXTnJgy3gGlG5rFPYzVkWGcpaXCKiQOUXITY+Ph7nz59HtWrVEBISUmTbJCQk4NixY6hZsyYiIiKK+KiJ6GrkLGFtG4SgbYO8j2Vm2epunYPtyYs51zm30zOhwq5cthzM/zlKhRkQE2VQzxUjlyi5NqBslBFlI+W+wbE8OsKgepOJiEgffBpis7Ky8Mgjj2D27NmIiYlBSkoKpk2bhqFDh3q1za5du/DWW2/h+++/x4ULF7B8+XJ06dKlhH4qIrqW4CADqpYzqQvg2oNrH1wmdbb2gHsl6ErItd2WGRSSUm2Xw2fy7811Jvm1TKSEXqMKvTIYzR5wVQjOCbxyu0yEFVYr25GIyJ/5NMS+8sorWLx4sQqdtWvXxhdffIEHHngATZs2RfPmzT3eZsWKFWjbti2efvppNGiQTzcQEfn94DIVKqOMaFoz/3WS0yw4c8k2qExKFy4kWtSctxeSLLiYKMuv3I9PtsJiRc79bBmmds1jMBlNiAq/pHpwS4UZERVuUJdSci33I+Ralhlty+SxMAOiI2zrypnUiIio+Bis0uXhI1WrVsX999+PyZMnO5Y1bNgQXbt2xXvvvef1NsePH0dcXJxHPbGJiYmIjo5WJQlRUVEoThaLBWfPnkX58uVhNLLGT8vYlv5JyhfiL1tzgm5OwE3MCb/qvtNjiVZcTvf+v0UZuGYLvVcCsLqEGREZZkC42aCmIJPr8FDb7Qiz/T4c96XmmGUQnuN7Uh/Yjvpty0Qv8pbPemJPnz6NEydOqB5TZ+3bt8emTZuKbBsiIilfiI2Wi3sfEi+nZuPg0XMICS+L5DQDklKsSEzJKWHIuS1lDInqtiy78pgsl15fGbh2PtGK84nX7vW9FjnJhD3s2sOvCrxO12p5TugNDTGobeTadlt6hm2D8NSy4CuPy2tDRKRFPguxUqsqpK7VmdyXAVtFtY270tPT1cVOPhnYPzHIpTjJ/qVDvLifh4of21IfzMFWxEZZEatOs1u4b0csFitS0pEn6Kqwm2pFwmULLqcDKWm2Hl+5Tkm34nLOtWxrv5+d81+CnGQiNUNKIYr+i7MgI2AOkaB8JeCGugTgnNvBBrWeOcgWiEOCDaq3WcombBfpfbZdm3Mek3Xs99XFaVmQqWTCM9+T+sB21G9bWrzIPj4LsUFBtqfOyLhySkshQTI4OLjItnHXlClTMHHixDzLz507h7S0NBQnaUDpRpdGZTmBtrEt9aEo2lH+tyoTYrugdOG3t+b05kqAlWBrC7Jy2+B0277c4LgvszqkySVDtjfYbmcCGTnX9setVluIzLIAWWlQoRkoueoyo8FqC7o5l2C5mJyuTRJ0besEmXLWMV25HeSyrtVx2/FYzuNBRgvS01IQXeqc6nW2LbOtY1/PZESe5XKb1V3+g/+36rctk5KStBdiq1SpogZvnDp1ymW53Jc61qLaxl3jx4/HqFGjXHpiZZ+xsbElUhMrP5c8F0OstrEt9UHv7Sh/PDJVQLaqsJuWKddWFYTlOi3Pctv99EyrCsEZWVYVim33rUjPkpB85TG5luVqHXXfdjvTqbLCYr0SsAtWFL21MgNGtGdbSrhVoTYn/KpgbMgJzoYrgTef27ZwfGW5rG9yWscenq+1vm0d23rqthEw2W/n2oft8SvbuewjZzvbtW22Dvkd1wq9vycDiSVXW4aGhmovxEZGRqJ169b46aefMHjwYEeP6q+//opx48Y51jt58qSaRqtOnTpub+MJs9msLrnJC1wSbxhp0JJ6LipebEt90Hs7SpAJzftfXrGSUgvpXbaHYXsIlmUy+C4jJwxnZNuuM52vs2zB2xagC34s97Wsk5qeBSuCVK+zPE9WttN1tm0fsn5uUs6hSjoynXuo9TP3WlBOmLUFYFswNjmFZHtQNhquhGS1nuP2leBt24+8Z1wfU/cNtt+3fO/LbefnNdh6wB3PldMjLj33l5ONKB2d6XR8V57Pto5c5zzm2I/TOs7Lcp7XeR37Msn29ue23b6y7yv70N4HAX/9/9Wb/2N9OsWWfH3fu3dvNG7cWA3Omjp1qjopwaOPPupYZ8KECVi7di127tzp9jZyIgSZmeDMmTPq/qFDh1CuXDk1Ek4uRERU8iQE2OttS34kdMxV/1hK77SEWnuwlesMR+CV5fawe+XaeX0Ju2odta7ttirVsN92uc77uOwz2+m2XEupoG0dq+0xCdU526mArZblPOZ8Wz1uu+3c+52bbCPkw4NrOPfXoC696pfhTyTD2oPtlWB8JexeCcYGp+BrD8EGl5BsyAnfBqMb6zo9l8F+DDnLbR8UDGp/uZfLfhz3nY7RfszOy+tUDkLvNiX8SVdLIbZ79+5qzlc5WcHcuXPRpEkTrFq1CqVLl3YpIahbt26htpF5Yp977jl1u1GjRnjzzTfV7X//+9/qQkRE5ExChqqrDQLCiqSMwX9ID7gtAF8Jz7aAbAu69h5nFZyttnArIVit43jcHtSd9ucI07bHnNez79didX7syn7duW8/Rtt9K1JS0xEUbFbHeGUdK7KtgFXuW52W5WzvfAxSZ25/Hpd1HNvbnkv2b1//WtQ+c9Z3WprfmtCabi1D/D7E+nSeWH/GeWLJE5zLUB/YjvrBttQHX7SjGkGfE5gdwdkedHMtcw7RuQOx67XTcudlTuvZ9mF1CdP5rZvt9Jgcqy2w5yzLWdf+uOt+rix3PT7X5Q2qBuFfnT2vV9X1PLFERERE/kqVAOR8be+01HcHRHnoc8QCEREREekaQywRERERaQ5DLBERERFpDkMsEREREWkOQywRERERaQ5DLBERERFpDkMsEREREWkOQywRERERaQ5DLBERERFpDs/YVQD72XjldGjFTU7BlpSUhNDQ0BI7nR4VD7alPrAd9YNtqQ9sR/22ZWJOzrLnrsJgiC2AvMAiLi7Om7YiIiIiIjdyV3R0NArDYPUk+gbIJ4WTJ0+iVKlS6vzJxUk+hUhYPnbsGKKioor1uah4sS31ge2oH2xLfWA76rctrVarCrCVK1cu9LfR7IktgLyQVatWRUmSxmSI1Qe2pT6wHfWDbakPbEd9tmV0IXtg7ViASURERESawxBLRERERJrDEOsHzGYzXnzxRXVN2sa21Ae2o36wLfWB7agf5iLMPBzYRURERESaw55YIiIiItIchlgiIiIi0hyGWCIiIiLSHM4T62Myye+uXbuQlZWFxo0bw2Qy+fqQqJBOnz6NAwcO5FneqVMnvpYakJCQgJ07d6JWrVqoVKlSvuucO3cOhw8fRvXq1VG+fPkSP0Zyz44dO5CSkoK2bdvmeWzdunXIzMx0WVatWjV1If/6m3jo0CGkpqaidu3a6tSk+Tl79iyOHDnC96QfkxMYyN9G+X+1YsWKeR7/66+/kJ2d7bKsRo0ahZujX87YRb6xe/dua7169awVKlSwVq1aVV3WrVvH5tCYDz/80BoaGmrt2LGjyyUrK8vXh0ZXcejQIevw4cOtFStWtJpMJuu7776b73pjxoyxms1m63XXXaeuR4wYYbVYLHxt/ciMGTOszZs3t5YpU8YaExOT7zqyvH79+i7v0enTp5f4sdLV27FmzZrq0rBhQ2vp0qXV/6+5jRw50uU9KffJfxw+fNg6cOBA1X7yvoyKirJ27drVeurUKZf1IiIiVDs7vydnz55dqOdiiPUR+SPYtGlTa79+/azZ2dlq2UMPPWSNi4uzpqWl+eqwyAPyn2zt2rX52mnMzz//bP3444+tSUlJ1ujo6HxDrPyHGhYWZt20aZO6v23bNmt4eDjDj5955plnVBtNnTr1qiH266+/LvFjI/f95z//UQHITtrLYDBY//rrL8eyzz//XL0Ht27dqu5v3rxZvUdnzpzJl9pPLF++3PrNN984ss2lS5esLVu2tPbp0ydPiF2wYIFXz8WaWB/ZuHEjtm/fjueff95xruAXXnhBnUt42bJlvjos8pDFYlFfZUppSEZGBl9HDejWrRsefvhhREZGFrjOjBkz0KtXL7Ro0ULdb9q0Kfr27auWk/+YMmWKo42uRr6Clv97pTyE/M+zzz6rygPsBg0apE5HumbNGscyee/Je7BZs2bq/vXXX6/eo3xP+o8uXbrgX//6lyPblC5dGoMHD8aqVavyrHvmzBn1njx//rxHz8UQ6yNbtmxRDSxvQDt7vZ08Rtoi9ZIDBgxAz549Ua5cObz//vu+PiQqAvJebNmypcuyNm3a8D2qURMmTMCwYcPU/7W33nqr6jQg/7V//35Vs16nTh3HMr4ntWnDhg0u7Wj3zDPP4KGHHkJcXBx69OiBkydPFmq/DLE+cvHiRfXpxP5JxS4mJkY9RtrRsGFD1QO7Z88eFWYlwD7xxBNYsmSJrw+NvCDlVvHx8eo96Uzuy+Ch9PR0vr4a8tprr+HChQvYunWrep9KOJLeIfJP8o3W/fffrzp6pHNAyABoGSyU33syMTExzyAh8g8LFizA3Llz1bfNzt5++231nty2bRv++ecfnDp1CkOGDCnUvhlifSQ4OBhpaWl5lsuIzJCQEJ8cE3nmxhtvRIMGDRz35U3YsWNHzJkzhy+phhkMBgQFBeV5n8p71P4eJu2Q3h777C/yjdfLL7+M1atXszfWD0lYlVIC6ZX7/vvv1ftQSPtJx09+70lZztl9/M/y5ctxzz334JVXXkGfPn1cHpNvRewdeTKDwUsvvYTff/+9UOU+DLE+Il9nSW+O9PQ4v3GlZotTvmhfhQoVcOLECV8fBnlJ3ou521HuyxQwub9FIe29RwXfp/5F/g5KD/nmzZtVAJKvmZ0/WMr9/N6T/Lvpf1asWKGCq9Q6S9lAcbwn+b+wDwufpSfnhx9+cCyTAV0SbG+55RZfHRZ54PLlyy73k5OT1fx3Mu8vaZvUTS5evFiVFgi5lvesLCftvkfFL7/8ov4Prl+/vk+OifKScoC7775b1U/+8ccfas5Qd96TixYt4nvSz6xcuVINuBs3bpwawO7ue1LmBc6vdrYgPNmBj8jgnzFjxmDkyJHqk6fZbFaN/eCDD/I/VY3p168fOnTooAb8SL2W1PlIj4G0L/kv+bAh9ZH2P55SkyWjZ+W9aS8PkffkN998o0pEBg4ciPnz56uJ2OfNm+fjoydnUpMuYwmkbeT/U/soaJmxIDw8HL/++is++OAD1cMnX1vKH9g33nhD/XEtU6YMX0w/8cADD6hA+sknn+D48ePqIuSbD3ugHT9+vBpsKfWy/fv3x7fffqvWc6enj0rGpk2bVB3zzTffjJtuusllVgIptZO/j/JB5IsvvlCzGEgPrHxomTp1KiZNmnTVGWNyM8g8W8X0c9A1yEs/ffp0VfMj//F2795dDQiy1/+QdsLQhx9+qP4wSs+O/OEcMWIEoqKifH1odBV79+5VdZK5yX+68h+p3b59+/D666/j4MGDqFmzpvpwIoP5yH/Ihw2pb81t1qxZqs2E/CH9/PPPVQ2slHNJnZ7Us5P/6N27t0uJnZ18+Hj88ccd92UQrXwIkQ+ecqY9eU86j0sg35IPFu+8806+j0nNq33cjwRXeY/KhxD5kDJ06FAVcguDIZaIiIiINIc1sURERESkOQyxRERERKQ5DLFEREREpDkMsURERESkOQyxRERERKQ5DLFEREREpDkMsURERESkOQyxRETF4OjRo1iwYEGxb6N1MmG9nKWJiKiweLIDIgpYf/31F44cOVLg40ajUZ0W0RNz5sxRZ+A7f/58sW6jdZ9++ikmT56Mw4cP+/pQiEhjeH5TIgpYGzZswJo1a9Rti8WiTpfYrl07dVpSYTKZPA6xso8777yz2LchIgpUDLFEFLCefPJJdRFpaWkqxMo52u+9916Xr7p79OiBtWvX4tSpU+jTpw8SExPVOcBFWFgY6tWrh4YNG7rsu0qVKmo7O9nX33//jZ49e2LHjh04ceIEmjZtiri4OK+2EVarVR3fpUuX1OPSgyy9zHfddddVf/5jx45h8+bNKFu2LFq0aIGIiIg8zy3Hs23bNnV+8zZt2qBSpUp59iPr7du3D+XLl1cfAiT85ybHLs9Vrlw5tG7dGkFBrn9+srOzr/ozEhHlxhBLRFQACarPPPMM6tatq4JZ1apVcdttt6mv+7///nu1TkpKiurN7dKlC+bOnasCpJBlUhpwxx13uOyrQYMGap3g4GCsXr0as2fPRv/+/T3eJiMjA71798amTZvQtm1b7Ny5E9dddx1WrVp11RA7duxY9VW+hE4Jv1KPKyG+Q4cOLs9du3Zt9bPL88tzzJo1y/Hc0nt999134+eff0b79u1VmC1TpgyWLl2KihUrOp7r6aefxvvvv69CcGZmpgrdP/74o1pXJCcn48YbbyzwZyQiypeViIisqampVvkvcdasWY5X45NPPlHLPvzww6u+QhcvXrRWq1bNOnv2bMeyr7/+2hoTE5NnX87rPP/889Z69ep5tc0HH3ygtjl27Ji6Hx8frx6PiIgo8Hi/+OILdbynT592LHvjjTesNWvWtGZnZ7s896RJkxzrTJkyxVquXDlrYmKiuv/pp59ao6KirAcPHlT3k5OTrS1btrQOGTLEsc306dOtZrPZunHjRsey9evXW0+cOOH2z0hElB/OTkBEdBWhoaF4+OGH8yzPysrCunXr8N1336mex2rVqmH9+vVXfS2joqJwzz33OO5L7+3BgwfVV+mebjNv3jwMHjxY9RKL6OjofI/X2Weffaa+sl+5cqXqfZUe5PDwcBw6dMhloJv0wI4aNcpxf+TIkS6lFDIQbdCgQahVq5a6L+UI//d//6f2J720YubMmer4WrZs6diPlBNUrlzZq9eFiIjlBEREVyF1nvYSAbs9e/age/fuqq5TvuqPjIzEyZMnHUGyIPavz+3MZrMKavIVe351pO5sI3WtUjPrrEaNGlc9DpkJQMKuBGBnAwcOdAmOsbGxKtw6B3qpibUHXbm+5ZZbXPYh5Qfp6ek4c+aMWlfKFOS1KurXhYiIIZaI6CoMBkOeZRMmTFD1p998841jWb9+/VStZ0mTQVnx8fEuy6TG9Wqk5/OGG27AtGnTrrqe7Fd+JufXQPYtg7OEXF+8eNFlG7kvoV+OS5QuXRoXLlwo9M9FRHQtLCcgIiqk06dPo379+o77MtBrxYoVPnkdO3bsqGZQcA7QP/zww1W3kZ5RKQXIHX5lZgBnMmODlErY/fbbb7h8+bIaDCY6deqEhQsXuvTeSnlCq1atVG+qkIFwsiw1NdWxjtyWwVxERN5gTywRUSFJr+ukSZNQqlQp9RX7Bx984LPXcMyYMfjiiy/UDAVyXH/++aeaXiu/HmS7Z599Fr/88osKm48++qgqh5B6XpldQOp87WT6sOHDh2PEiBGqd/W1115T69trYGX2gq+//loFVSlFkHl35f6vv/7q2Mf48eOxePFiFXwffPBBVSLw1VdfqTOTyfMSEXmKIZaIKGcQkwQx53pSqe+UeWFzk8FLFSpUwPLly9WUUBLupHdWei4LOnFBfvuSmlN5TnvdpyfbSN2phEcJ0nItU13JdFVS8nC1cgIJul9++aW6DgkJUdt8/PHHeeqBlyxZotaTXtpXX30VDzzwgONxKRmQuV//97//qSm95DXZuHEjGjdu7FhHygkkIMtgMpmiS6bekl5g+wkl3PkZiYjyw9POEhFpnNSh2mtQhcwYIF/7S5mBp3g6WCLyd+yJJSLSuF69eqFbt25qdgT5Kl++vl+2bJmvD4uIqFhxYBcRkcbJ2cOkJEBKA+RsXXL6Vikr8EZBpRRERP6C5QREREREpDnsiSUiIiIizWGIJSIiIiLNYYglIiIiIs1hiCUiIiIizWGIJSIiIiLNYYglIiIiIs1hiCUiIiIizWGIJSIiIiLNYYglIiIiImjN/wOq0qI6/ZMnHQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 700x380 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def reset_sequence():\n",
    "    brainstate.nn.reset_all_states(model, batch_size=1)\n",
    "    learner.reset_state(batch_size=1)\n",
    "\n",
    "\n",
    "def evaluate():\n",
    "    reset_sequence()\n",
    "    predictions = learner.etrace_evolve(inputs, return_outputs=True)\n",
    "    return jnp.mean((predictions - targets) ** 2)\n",
    "\n",
    "\n",
    "def local_loss(x, target):\n",
    "    prediction = learner(x)\n",
    "    return jnp.mean((prediction - target) ** 2)\n",
    "\n",
    "\n",
    "def train_epoch(_):\n",
    "    reset_sequence()\n",
    "    # etrace_grad owns the loop, the accumulation and the reduction; local_loss\n",
    "    # owns the model call. 'mean' divides by the total mask weight -- here T --\n",
    "    # which is exactly the hand-written `grads / inputs.shape[0]` it replaces.\n",
    "    grads, step_losses = learner.etrace_grad(\n",
    "        inputs, targets, step_fn=local_loss,\n",
    "        reduction='mean', return_value=True,\n",
    "    )\n",
    "    optimizer.update(grads)\n",
    "    return step_losses.mean()\n",
    "\n",
    "\n",
    "initial_loss = evaluate()\n",
    "training_losses = brainstate.transform.for_loop(\n",
    "    train_epoch, jnp.arange(25)\n",
    ")\n",
    "final_loss = evaluate()\n",
    "\n",
    "print(f\"initial loss: {float(initial_loss):.4f}\")\n",
    "for epoch in (0, 12, 24):\n",
    "    print(f\"epoch {epoch:2d} loss: {float(training_losses[epoch]):.4f}\")\n",
    "print(f\"final loss: {float(final_loss):.4f}\")\n",
    "print(\"loss decreased:\", bool(final_loss < initial_loss))\n",
    "\n",
    "with plt.style.context(\"default\"), plt.rc_context({\n",
    "    \"figure.facecolor\": \"white\",\n",
    "    \"axes.facecolor\": \"white\",\n",
    "    \"savefig.facecolor\": \"white\",\n",
    "}):\n",
    "    fig, ax = plt.subplots(figsize=(7, 3.8))\n",
    "    ax.plot(training_losses, color=\"#2563eb\")\n",
    "    ax.set(xlabel=\"Training epoch\", ylabel=\"Mean online loss\")\n",
    "    ax.set_title(\"D-RTRL mini-GRU training loss\")\n",
    "    ax.grid(True, alpha=0.3)\n",
    "    fig.tight_layout()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68bf1a60",
   "metadata": {},
   "source": [
    "## 6. Interpretation and limits\n",
    "\n",
    "- The loss curve supports that this estimator can provide a descent direction\n",
    "  on this task; it does not establish equality with BPTT.\n",
    "- A non-temporal readout may appear in compiler diagnostics because only paths\n",
    "  that influence recurrent hidden state carry temporal eligibility traces.\n",
    "- Memory follows participating parameter and hidden dimensions. pp-prop is the\n",
    "  relevant next comparison when a factorized trace is required.\n",
    "\n",
    "## References and API\n",
    "\n",
    "- Wang et al., \"Model-agnostic linear-memory online learning in spiking neural\n",
    "  networks,\" *Nature Communications* (2026),\n",
    "  [doi:10.1038/s41467-026-68453-w](https://doi.org/10.1038/s41467-026-68453-w).\n",
    "- Williams and Zipser, \"A Learning Algorithm for Continually Running Fully\n",
    "  Recurrent Neural Networks,\" *Neural Computation* (1989),\n",
    "  [doi:10.1162/neco.1989.1.2.270](https://doi.org/10.1162/neco.1989.1.2.270).\n",
    "- API: {class}`braintrace.D_RTRL`, {func}`braintrace.compile`,\n",
    "  {meth}`braintrace.SequenceDriverMixin.etrace_grad`.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}