{ "cells": [ { "cell_type": "markdown", "id": "intro", "metadata": {}, "source": [ "# ``Module`` System Protocol\n", "\n", "The module system is the foundation for building neural networks in BrainState. It provides a clean, object-oriented interface for organizing stateful computations.\n", "\n", "In this tutorial, you will learn:\n", "\n", "- 🏗️ The `Module` base class and its role\n", "- 🔨 How to create custom modules\n", "- 🧩 Module composition and nesting\n", "- 🎯 Parameter management and initialization\n", "- 📦 Working with module hierarchies\n", "\n", "## Why Modules?\n", "\n", "Modules (via `brainstate.nn.Module`) provide:\n", "\n", "✅ **Automatic state management** - States are tracked automatically \n", "✅ **Clean abstractions** - Encapsulate related computations \n", "✅ **Reusability** - Build once, use everywhere \n", "✅ **Composability** - Combine simple modules into complex systems" ] }, { "cell_type": "code", "execution_count": 1, "id": "imports", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:10.100523Z", "start_time": "2025-10-11T08:24:08.256525Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:23.674127Z", "iopub.status.busy": "2026-05-30T16:19:23.673955Z", "iopub.status.idle": "2026-05-30T16:19:25.875108Z", "shell.execute_reply": "2026-05-30T16:19:25.873913Z" } }, "outputs": [], "source": [ "import brainstate\n", "import jax.numpy as jnp\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "id": "node_intro", "metadata": {}, "source": [ "## 1. The Module Base Class\n", "\n", "`brainstate.nn.Module` is the base class for all modules in BrainState. It provides:\n", "\n", "- Automatic registration of child modules\n", "- State collection and management\n", "- Pretty printing and inspection\n", "- Integration with JAX transformations\n", "\n", "### Creating Your First Module\n", "\n", "The simplest module inherits from `Module` and implements `update()`:" ] }, { "cell_type": "code", "execution_count": 2, "id": "first_module", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:10.223290Z", "start_time": "2025-10-11T08:24:10.109530Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:25.879111Z", "iopub.status.busy": "2026-05-30T16:19:25.878491Z", "iopub.status.idle": "2026-05-30T16:19:25.984680Z", "shell.execute_reply": "2026-05-30T16:19:25.983815Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "An NVIDIA GPU may be present on this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Input: [1. 2. 3.]\n", "Output: [6. 7. 8.]\n", "\n", "Module:\n", "SimpleModule(\n", " constant=5.0\n", ")\n" ] } ], "source": [ "class SimpleModule(brainstate.nn.Module):\n", " \"\"\"A minimal module that adds a constant.\"\"\"\n", " \n", " def __init__(self, constant=1.0):\n", " super().__init__() # Always call parent __init__\n", " self.constant = constant\n", " \n", " def update(self, x):\n", " return x + self.constant\n", "\n", "# Create and use the module\n", "module = SimpleModule(constant=5.0)\n", "result = module(jnp.array([1.0, 2.0, 3.0]))\n", "\n", "print(\"Input:\", jnp.array([1.0, 2.0, 3.0]))\n", "print(\"Output:\", result)\n", "print(\"\\nModule:\")\n", "print(module)" ] }, { "cell_type": "markdown", "id": "stateful_module", "metadata": {}, "source": [ "### Adding States to Modules\n", "\n", "Modules become powerful when they contain states:" ] }, { "cell_type": "code", "execution_count": 3, "id": "stateful_example", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:10.291549Z", "start_time": "2025-10-11T08:24:10.232214Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:25.986700Z", "iopub.status.busy": "2026-05-30T16:19:25.986438Z", "iopub.status.idle": "2026-05-30T16:19:26.081331Z", "shell.execute_reply": "2026-05-30T16:19:26.080238Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Initial count: 0\n", "Call 1: count=1, result=10.0\n", "Call 2: count=2, result=20.0\n", "Call 3: count=3, result=30.0\n", "Call 4: count=4, result=40.0\n", "Call 5: count=5, result=50.0\n" ] } ], "source": [ "class Counter(brainstate.nn.Module):\n", " \"\"\"A module that counts how many times it's called.\"\"\"\n", " \n", " def __init__(self):\n", " super().__init__()\n", " # Create a state to track the count\n", " self.count = brainstate.ShortTermState(jnp.array(0))\n", " \n", " def update(self, x):\n", " # Increment counter\n", " self.count.value = self.count.value + 1\n", " # Return input with count\n", " return x * self.count.value\n", "\n", "# Test the counter\n", "counter = Counter()\n", "print(\"Initial count:\", counter.count.value)\n", "\n", "for i in range(5):\n", " result = counter(jnp.array(10.0))\n", " print(f\"Call {i+1}: count={counter.count.value}, result={result}\")" ] }, { "cell_type": "markdown", "id": "linear_module", "metadata": {}, "source": [ "## 2. Creating Custom Modules\n", "\n", "Let's build a complete linear layer from scratch to understand module design:\n", "\n", "### Example: Custom Linear Layer" ] }, { "cell_type": "code", "execution_count": 4, "id": "custom_linear", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:10.684872Z", "start_time": "2025-10-11T08:24:10.297350Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:26.083933Z", "iopub.status.busy": "2026-05-30T16:19:26.083677Z", "iopub.status.idle": "2026-05-30T16:19:26.552338Z", "shell.execute_reply": "2026-05-30T16:19:26.551582Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Module:\n", "Linear(in_features=5, out_features=3, use_bias=True)\n", "\n", "Weight shape: (5, 3)\n", "Bias shape: (3,)\n", "\n", "Input shape: (5,)\n", "Output shape: (3,)\n", "Output: [ 0.3793956 -0.9351347 -0.94997764]\n" ] } ], "source": [ "class Linear(brainstate.nn.Module):\n", " \"\"\"A linear transformation: y = W @ x + b\"\"\"\n", " \n", " def __init__(self, in_features, out_features, use_bias=True):\n", " super().__init__()\n", " \n", " self.in_features = in_features\n", " self.out_features = out_features\n", " self.use_bias = use_bias\n", " \n", " # Initialize weight with Xavier/Glorot initialization\n", " std = jnp.sqrt(2.0 / (in_features + out_features))\n", " self.weight = brainstate.ParamState(\n", " brainstate.random.randn(in_features, out_features) * std\n", " )\n", " \n", " # Initialize bias to zero\n", " if use_bias:\n", " self.bias = brainstate.ParamState(jnp.zeros(out_features))\n", " \n", " def update(self, x):\n", " \"\"\"Forward pass.\n", " \n", " Args:\n", " x: Input tensor of shape (..., in_features)\n", " \n", " Returns:\n", " Output tensor of shape (..., out_features)\n", " \"\"\"\n", " out = x @ self.weight.value\n", " if self.use_bias:\n", " out = out + self.bias.value\n", " return out\n", " \n", " def __repr__(self):\n", " return f\"Linear(in_features={self.in_features}, out_features={self.out_features}, use_bias={self.use_bias})\"\n", "\n", "# Create and test the linear layer\n", "brainstate.random.seed(42)\n", "linear = Linear(in_features=5, out_features=3)\n", "\n", "# Forward pass\n", "x = jnp.ones(5)\n", "y = linear(x)\n", "\n", "print(\"Module:\")\n", "print(linear)\n", "print(f\"\\nWeight shape: {linear.weight.value.shape}\")\n", "print(f\"Bias shape: {linear.bias.value.shape}\")\n", "print(f\"\\nInput shape: {x.shape}\")\n", "print(f\"Output shape: {y.shape}\")\n", "print(f\"Output: {y}\")" ] }, { "cell_type": "markdown", "id": "activation_module", "metadata": {}, "source": [ "### Example: Custom Activation Module" ] }, { "cell_type": "code", "execution_count": 5, "id": "custom_activation", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:11.047274Z", "start_time": "2025-10-11T08:24:10.692880Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:26.554332Z", "iopub.status.busy": "2026-05-30T16:19:26.554190Z", "iopub.status.idle": "2026-05-30T16:19:26.859103Z", "shell.execute_reply": "2026-05-30T16:19:26.858384Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Activation: LeakyReLU(negative_slope=0.1)\n", "Input: [-2. -1. 0. 1. 2.]\n", "Output: [-0.2 -0.1 0. 1. 2. ]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAArMAAAHWCAYAAABkNgFvAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAcqNJREFUeJzt3Xl4E9X+P/D3JG2T7gt0pYWWsu9by+JVQFBERAsKiCilLG4gIve6cL0KqIiiXPGrAm5QRFkUBf0JIoiAF1lapMgme6FQutI26ZakzZzfH7WxsQttaTNN8349Tx6YkzMzn5xMJp9OzpwjCSEEiIiIiIjskErpAIiIiIiI6ovJLBERERHZLSazRERERGS3mMwSERERkd1iMktEREREdovJLBERERHZLSazRERERGS3mMwSERERkd1iMktEREREdovJLBHV26VLlyBJEt5++22lQ3F4kiRhwYIFiuw7PDwcU6ZMUWTf9qb8MxMfH690KETNBpNZomYiPj4ekiTh8OHDSofSoIYMGQJJkiwPV1dX9OjRA8uWLYMsy/Xa5pQpU+Dh4VHt8zdqy3vuuQfh4eF12md0dDQkScKKFSvqtF5F27ZtUyxh3b9/PxYsWIC8vDxF9l+V8vepqscLL7ygaGzr1q3DsmXLFI2ByFE4KR0AEdGNhIaGYvHixQCA7OxsrFu3Ds888wyysrKwaNEihaO7sXPnziExMRHh4eH44osv8MQTT9RrO9u2bcMHH3xQZUJbXFwMJ6fGO6Xv378fCxcuxJQpU+Dj42P13JkzZ6BSKXdt5JVXXkFERIRVWbdu3RSKpsy6detw4sQJzJkzx6q8TZs2KC4uhrOzszKBETVDTGaJqMnz9vbGww8/bFl+/PHH0alTJ7z33nt45ZVXoFarFYzuxj7//HMEBARg6dKleOCBB3Dp0qU6X9m9Ea1W26DbqwuNRqPYvgFg5MiR6Nevn6Ix1JYkSYq+V0TNEbsZEDmY1NRUTJ06FYGBgdBoNOjatStWrVplVcdkMuHll19G37594e3tDXd3d9x6663YvXv3DbcvhMCjjz4KFxcXfPPNNxg8eDB69uxZZd2OHTtixIgRdX4NWq0WUVFRyM/PR2ZmptVzn3/+Ofr27QtXV1f4+fnhwQcfxJUrV+q8j4a0bt06PPDAA7jnnnvg7e2NdevWVVnv0KFDuPvuu+Hr6wt3d3f06NED7777LoCyrhEffPABAFj9nF6uYp/ZTZs2QZIk7N27t9I+PvzwQ0iShBMnTgAAjh07hilTpqBt27bQarUICgrC1KlTcf36dcs6CxYswLPPPgsAiIiIsOz70qVLAKruM3vx4kWMGzcOfn5+cHNzw4ABA7B161arOnv27IEkSfjyyy+xaNEihIaGQqvVYtiwYTh//nwtW7dm1fUl/nvM5V0Wfv31V8ydOxf+/v5wd3fHmDFjkJWVVWn9H374AYMHD4anpye8vLwQFRVleV+HDBmCrVu34vLly5a2Kv/jpbo+sz///DNuvfVWuLu7w8fHB/fddx/++OMPqzoLFiyAJEk4f/685Qq5t7c34uLiUFRUdFPtRGTPeGWWyIFkZGRgwIABkCQJs2bNgr+/P3744QdMmzYNer3e8pOoXq/HJ598gokTJ2LGjBnIz8/Hp59+ihEjRiAhIQG9evWqcvtmsxlTp07Fxo0bsXnzZowaNQo5OTmYMWMGTpw4YfXTb2JiIs6ePYv//Oc/9Xot5UlBxZ+8Fy1ahJdeegnjx4/H9OnTkZWVhffeew+33XYbkpKSKv08bguHDh3C+fPnsXr1ari4uGDs2LH44osv8O9//9uq3s6dO3HPPfcgODgYTz/9NIKCgvDHH3/g+++/x9NPP43HHnsM165dw86dO7F27doa9zlq1Ch4eHjgyy+/xODBg62e27hxI7p27Wp5L3bu3ImLFy8iLi4OQUFBOHnyJD766COcPHkSBw8ehCRJGDt2LM6ePYv169fjnXfeQcuWLQEA/v7+Ve4/IyMDgwYNQlFREWbPno0WLVpgzZo1uPfee7Fp0yaMGTPGqv4bb7wBlUqFf/3rX9DpdFiyZAkmTZqEQ4cO1aqNdTodsrOzrcrKY6yrp556Cr6+vpg/fz4uXbqEZcuWYdasWdi4caOlTnx8PKZOnYquXbti3rx58PHxQVJSErZv346HHnoIL774InQ6Ha5evYp33nkHAGrso/3TTz9h5MiRaNu2LRYsWIDi4mK89957uOWWW3DkyJFKV/HHjx+PiIgILF68GEeOHMEnn3yCgIAAvPnmm/V6zUR2TxBRs7B69WoBQCQmJlZbZ9q0aSI4OFhkZ2dblT/44IPC29tbFBUVCSGEKC0tFUaj0apObm6uCAwMFFOnTrWUJScnCwDirbfeEiUlJWLChAnC1dVV/Pjjj5Y6eXl5QqvViueff95qe7Nnzxbu7u6ioKCgxtc1ePBg0alTJ5GVlSWysrLE6dOnxbPPPisAiFGjRlnqXbp0SajVarFo0SKr9Y8fPy6cnJysymNjY4W7u3u1+7xRW44aNUq0adOmxrjLzZo1S4SFhQlZloUQQuzYsUMAEElJSZY6paWlIiIiQrRp00bk5uZarV++nhBCzJw5U1R32gYg5s+fb1meOHGiCAgIEKWlpZaytLQ0oVKpxCuvvGIpK3/PK1q/fr0AIH755RdL2VtvvSUAiOTk5Er127RpI2JjYy3Lc+bMEQDE//73P0tZfn6+iIiIEOHh4cJsNgshhNi9e7cAIDp37mx1vL377rsCgDh+/HiVr7Vc+ftU1aO6dqku5vJtDR8+3KrNn3nmGaFWq0VeXp4Qoux49vT0FP379xfFxcVW26y4XnXHSPlnZvXq1ZayXr16iYCAAHH9+nVL2e+//y5UKpWYPHmypWz+/PkCgNVnUAghxowZI1q0aFF1IxE5AHYzIHIQQgh8/fXXGD16NIQQyM7OtjxGjBgBnU6HI0eOAADUajVcXFwAALIsIycnB6WlpejXr5+lTkUmkwnjxo3D999/j23btuHOO++0POft7Y377rsP69evhxACQNkV3I0bNyImJgbu7u43jP306dPw9/eHv78/OnXqhLfeegv33nuv1U+133zzDWRZxvjx461eW1BQENq3b1+rLhINrbS0FBs3bsSECRMsXQJuv/12BAQE4IsvvrDUS0pKQnJyMubMmVPp6nHFrgR1MWHCBGRmZmLPnj2Wsk2bNkGWZUyYMMFS5urqavm/wWBAdnY2BgwYAABVvte1sW3bNkRHR+Mf//iHpczDwwOPPvooLl26hFOnTlnVj4uLsxxvAHDrrbcCKOuqUBsffPABdu7cafWor0cffdSqzW+99VaYzWZcvnwZQNmV7Pz8fLzwwguV+r7W571KS0vD0aNHMWXKFPj5+VnKe/TogTvuuAPbtm2rtM7jjz9utXzrrbfi+vXr0Ov1dd4/UXPAbgZEDiIrKwt5eXn46KOP8NFHH1VZp2L/0zVr1mDp0qU4ffo0SkpKLOV/v2scABYvXoyCggL88MMPGDJkSKXnJ0+ejI0bN+J///sfbrvtNvz000/IyMjAI488UqvYw8PD8fHHH0OWZVy4cAGLFi1CVlaWVTJx7tw5CCHQvn37KrfR0HeP1yZx2bFjB7KyshAdHW3VB3To0KFYv3493nzzTahUKly4cAFAw96Bf9ddd8Hb2xsbN27EsGHDAJR1MejVqxc6dOhgqZeTk4OFCxdiw4YNlfof63S6eu378uXL6N+/f6Xyzp07W56v+Fpbt25tVc/X1xcAkJubW6v9RUdHN9gNYDeKpaHfq/IkuWPHjpWe69y5M3788UcUFhZa/dFXU4xeXl4NEheRPWEyS+QgysdkffjhhxEbG1tlnR49egAou4lqypQpiImJwbPPPouAgACo1WosXrzY8mVe0YgRI7B9+3YsWbIEQ4YMqXTFasSIEQgMDMTnn3+O2267DZ9//jmCgoIwfPjwWsXu7u5uVfeWW25Bnz598O9//xv/93//Z3l9kiThhx9+qHJ0g5r6LP5defzFxcVVPl9UVFSrO9LLr76OHz++yuf37t2LoUOH1jquutBoNIiJicHmzZuxfPlyZGRk4Ndff8Xrr79uVW/8+PHYv38/nn32WfTq1QseHh6QZRl33XVXvcfxravqRqMov5LfGMxmc5OJpa7sIUYiW2IyS+Qg/P394enpCbPZfMMkctOmTWjbti2++eYbqyuQ8+fPr7L+gAED8Pjjj+Oee+7BuHHjsHnzZqsxT9VqNR566CHEx8fjzTffxJYtWzBjxox6D6nVo0cPPPzww/jwww/xr3/9C61bt0ZkZCSEEIiIiLC68lgfbdq0AVA2fmr5T94VnT179oZX5goLC/Htt99iwoQJeOCBByo9P3v2bHzxxRcYOnQoIiMjAQAnTpyo8b2p68/YEyZMwJo1a7Br1y788ccfEEJYdTHIzc3Frl27sHDhQrz88suW8nPnzt3Uvtu0aYMzZ85UKj99+rTleVvx9fWtNNGDyWRCWlpavbZX8b1q165dtfVq214Vj7W/O336NFq2bFmrrjhEjox9ZokchFqtxv3334+vv/7aMixTRRWHHypPMite6Tl06BAOHDhQ7faHDx+ODRs2YPv27XjkkUcqXdV75JFHkJubi8ceewwFBQVW48bWx3PPPYeSkhL897//BQCMHTsWarUaCxcurHSFSghhNdTUjfTt2xcBAQH45JNPYDQarZ7bsmULUlNTMXLkyBq3sXnzZhQWFmLmzJl44IEHKj3uuecefP311zAajejTpw8iIiKwbNmySolXxddSntTUdhau4cOHw8/PDxs3bsTGjRsRHR1t1U2kqvcZQJUzV9Vl33fffTcSEhKsjpfCwkJ89NFHCA8PR5cuXWoVf0OIjIzEL7/8YlX20UcfVXtl9kbuvPNOeHp6YvHixTAYDFbP/f29qk03jeDgYPTq1Qtr1qyxatsTJ05gx44duPvuu+sVJ5Ej4ZVZomZm1apV2L59e6Xyp59+Gm+88QZ2796N/v37Y8aMGejSpQtycnJw5MgR/PTTT8jJyQFQNl3rN998gzFjxmDUqFFITk7GypUr0aVLFxQUFFS775iYGKxevRqTJ0+Gl5cXPvzwQ8tzvXv3Rrdu3fDVV1+hc+fO6NOnz029zi5duuDuu+/GJ598gpdeegmRkZF47bXXMG/ePFy6dAkxMTHw9PREcnIyNm/ejEcffRT/+te/LOuXlJTgtddeq7RdPz8/PPnkk3j77bcRGxuLqKgoTJgwAS1atEBSUhJWrVqFHj164NFHH60xvi+++AItWrTAoEGDqnz+3nvvxccff4ytW7di7NixWLFiBUaPHo1evXohLi4OwcHBOH36NE6ePIkff/wRQFmSDZRd1R0xYgTUajUefPDBamNwdnbG2LFjsWHDBhQWFuLtt9+2et7Lywu33XYblixZgpKSErRq1Qo7duxAcnJypW2V7/vFF1/Egw8+CGdnZ4wePbrKq4YvvPAC1q9fj5EjR2L27Nnw8/PDmjVrkJycjK+//tqms4VNnz4djz/+OO6//37ccccd+P333/Hjjz/We+guLy8vvPPOO5g+fTqioqLw0EMPwdfXF7///juKioqwZs0aAGXttXHjRsydOxdRUVHw8PDA6NGjq9zmW2+9hZEjR2LgwIGYNm2aZWgub29vxaYvJrIrCoygQESNoKZhigCIK1euCCGEyMjIEDNnzhRhYWHC2dlZBAUFiWHDhomPPvrIsi1ZlsXrr78u2rRpIzQajejdu7f4/vvvRWxsrNVwQxWH5qpo+fLlAoD417/+ZVW+ZMkSAUC8/vrrtX5dgwcPFl27dq3yuT179lQaeunrr78W//jHP4S7u7twd3cXnTp1EjNnzhRnzpyx1ImNja22nSIjIy31fvjhBzF06FDh5eUlnJ2dRUREhJg7d26l4bP+LiMjQzg5OYlHHnmk2jpFRUXCzc1NjBkzxlK2b98+cccddwhPT0/h7u4uevToId577z3L86WlpeKpp54S/v7+QpKkWg1BtXPnTgFASJJkOQYqunr1qhgzZozw8fER3t7eYty4ceLatWtVbu/VV18VrVq1EiqVymqYrr8PcyWEEBcuXBAPPPCA8PHxEVqtVkRHR4vvv//eqk750FxfffWVVXlVw1dVpTbD0ZnNZvH888+Lli1bCjc3NzFixAhx/vz5aofm+vu2ymPcvXu3Vfl3330nBg0aJFxdXYWXl5eIjo4W69evtzxfUFAgHnroIeHj4yMAWD431b22n376Sdxyyy2W7Y0ePVqcOnXKqk750FxZWVlVtkNVw6YROQJJCPYYJyLbePfdd/HMM8/g0qVLle7IJiIiqg8ms0RkE0II9OzZEy1atFBkzFciImqe2GeWiBpVYWEhvvvuO+zevRvHjx/Ht99+q3RIRETUjPDKLBE1qkuXLiEiIgI+Pj548sknsWjRIqVDIiKiZoTJLBERERHZLY4zS0RERER2i8ksEREREdkth7sBTJZlXLt2DZ6ennWeGpKIiIiIGp8QAvn5+QgJCbnhRCsOl8xeu3YNYWFhSodBRERERDdw5coVhIaG1ljH4ZJZT09PAGWN4+Xl1ej7k2UZWVlZ8Pf3t+kUjo6O7a4MtrvtCSFgMpmQlZWF4OBgqNVqpUNyGDzelcF2V4at212v1yMsLMySt9XE4ZLZ8q4FXl5eNktmDQYDvLy8+KGzIba7MtjutifLMs6ePQu9Xo8OHTrAycnhTuuK4fGuDLa7MpRq99p0CeVRQERERER2i8ksEREREdktJrNEREREZLfYuaoKQgiUlpbCbDbf9LZkWUZJSQkMBgP79tiQvbS7Wq2Gk5MTh4kjIiKqJyazf2MymZCWloaioqIG2Z4QArIsIz8/nwmLDdlTu7u5uSE4OBguLi5Kh0JERGR3mMxWIMsykpOToVarERISAhcXl5tOhMqv8vLqm23ZQ7tXHFIpOTkZ7du3b9JXkYmIiJoiJrMVmEwmyLKMsLAwuLm5Ncg27SGpao7spd1dXV3h7OyMy5cvw2QyQavVKh0S2RlJkuDt7Q2z2dykj3UiosbCZLYKvDpGtsTjjW6GJEkIDAyEJElMZonIIfFblIiIiIjsFpNZIiI7ZzabG2T0FSIie6RoMrtixQr06NHDMrXswIED8cMPP9S4zldffYVOnTpBq9Wie/fu2LZtm42idVyXLl2CJEk4evSo0qE0CJPJhHbt2mH//v2Nup/s7GwEBATg6tWrjbofcmyyLOPChQu4cuUKZFlWOhwiIptTNJkNDQ3FG2+8gd9++w2HDx/G7bffjvvuuw8nT56ssv7+/fsxceJETJs2DUlJSYiJiUFMTAxOnDhh48ibnilTpiAmJkbpMKpUngyXP/z8/DB48GD873//q9N2anqNQ4YMwZw5cyqVx8fHw8fHx6ps5cqViIiIwKBBgyxlOTk5mDRpEry8vODj44Np06ahoKCgxng++ugjDBkyBF5eXpAkCXl5eVbPt2zZEpMnT8b8+fNr8/KIiIioHhRNZkePHo27774b7du3R4cOHbBo0SJ4eHjg4MGDVdZ/9913cdddd+HZZ59F586d8eqrr6JPnz54//33bRw51cdPP/2EtLQ0/PLLLwgJCcE999yDjIwMm8YghMD777+PadOmWZVPmjQJJ0+exM6dO/H999/jl19+waOPPlrjtoqKinDXXXfh3//+d7V14uLi8MUXXyAnJ6dB4iciIlLCr+ez8WuyTukwqtRkRjMwm8346quvUFhYiIEDB1ZZ58CBA5g7d65V2YgRI7Bly5Zqt2s0GmE0Gi3Ler0eQNlPc3//SU6WZQghLI+GUr6thtzmjfZV0YkTJ/Dcc8/hf//7H9zd3XHnnXfiv//9L1q2bAkA2L59OxYtWoQTJ05ArVZj4MCBWLZsGSIjIyvFL4SA2WzGjBkzcODAAXz44YcYOnQoDh06hH79+ln2uWzZMixbtgwXL160rO/n54fAwEAEBgZi3rx52LBhAw4ePIh77723VnHW9BorxldVvfL/Hz58GBcuXMDdd99tKfvjjz+wfft2JCQkWF7D//3f/2HUqFF46623EBISUuX+nn76aQDAnj17qtw/AHTp0gUhISH45ptvKiXQf4+7qmPS3pR/huz9ddiTisdNcziG7AmPd2Ww3W3vjzQ9Hv/iCAwmMwySBhOjWzf6Puvy/iqezB4/fhwDBw6EwWCAh4cHNm/ejC5dulRZNz09HYGBgVZlgYGBSE9Pr3b7ixcvxsKFCyuVZ2VlwWAwWJWVlJRAlmWUlpaitLTUUj5mxUFkFRj/vonaEwDqMWKOv4cGm58YUKu65V9iFeMGgLy8PAwbNgxxcXFYsmQJiouL8eKLL2L8+PHYsWMHgLIEf/bs2ejevTsKCgqwcOFCjBkzBocPH4ZKpbJss7S0FIWFhXj44Ydx+fJl/Pzzz/D398ewYcOwatUq9OrVy7Lf1atX45FHHrGKqbxdi4uLsWbNGgBl07mWlpbWKs7qXiPwV0JY/lx50l1+U0x5+d69e9G+fXu4urpayn799Vf4+PigV69elrIhQ4ZApVJh//79N+y+UXEfVcXWr18//PLLL4iNja1y/dLSUsiyjOvXr8PZ2bnGfTV1sixDp9NBCMEhx2ykvM2LioqQmZkJJyfFT+sOg8e7MtjutpWuN2HGxtMoNJZ9120/dhW3t9E0+lCA+fn5ta6r+FmvY8eOOHr0KHQ6HTZt2oTY2Fjs3bu32oS2rubNm2d1NVev1yMsLAz+/v7w8vKyqmswGJCfnw8nJyerL4TsAhMy9DeRzNaTBKnWX0wqlQoqlapS/ZUrV6J379544403LGWrVq1C69atcfHiRXTo0AHjx4+3Wmf16tUICAjA2bNn0a1bN8s2DQYDYmJiYDQasXv3bnh7ewMApk+fjieeeALvvPMONBoNjhw5ghMnTuDbb7+1asvBgwdDpVKhqKgIQgj07dsXd955J5ycnGoVZ3WvEYClP+7fn1Or1QBgKb9y5QpatWplVS8zMxMBAQFWZU5OTvDz80NWVtYN34OK+6iqbqtWrXD06NFqt+Pk5ASVSoUWLVrY/aQJsixDkiT4+/vzS8ZGZFlGXl4eJEmqdBxT4+Lxrgy2u+3oikvw7LqDyCosAQB0C3LH8oej4K5t/Asvdfk+VPys5+Lignbt2gEA+vbti8TERLz77rv48MMPK9UNCgqq1McyIyMDQUFB1W5fo9FAo9FUKi9PjP5eVvFGpXL+npXXrwsBAakel2b9Pev+l8/f6x87dgy7d++Gp6dnpboXL15Ex44dce7cObz88ss4dOgQsrOzLZf2r1y5gu7du1u2+dBDDyE0NBQ///wzXF1dLdsZM2YMZs2ahS1btuDBBx/EmjVrMHToUERERFjFtHHjRnTq1MnSnSA+Ph4uLi61jrO611ixvPw5IYRVvfL/GwwGaLXaKp+raru1GYi+4vpV1XVzc0NRUdEN467qmLRHzem12Ivytma72x6Pd2Ww3RufocSMxz4/gnOZZTdDh7dww9v3tYO71tkm7V6XfSiezP6dLMtWfVwrGjhwIHbt2mV11/rOnTur7WPbUP7fU/+o97pKT6taUFCA0aNH480336z0XHBwMICyG/HatGmDjz/+GCEhIZBlGd26dYPJZLKqf/fdd+Pzzz/HgQMHcPvtt1vKXVxcMHnyZKxevRpjx47FunXr8O6771baX1hYGNq3b4/27dujtLQUY8aMwYkTJ6DRaGoVZ028vLyg01XumJ6Xl2e5ggyUjTBw/PhxqzpBQUHIzMy0KistLUVOTk6NfyjVVk5ODvz9/W96O0RVkSQJXl5eKC0t5QxgRNQgZFngn1/9joTkspuXW3q4YPWUfnA1FyocWdUU/ZNm3rx5+OWXX3Dp0iUcP34c8+bNw549ezBp0iQAwOTJkzFv3jxL/aeffhrbt2/H0qVLcfr0aSxYsACHDx/GrFmzlHoJTV6fPn1w8uRJhIeHo127dlYPd3d3XL9+HWfOnMF//vMfDBs2DJ07d0Zubm6V23riiSfwxhtv4N5778XevXutnps+fTp++uknLF++HKWlpRg7dmyNcT3wwANwcnLC8uXLaxXnjXTs2BFHjhypVJ6UlIQOHTpYlnv37o3Tp09b3ag1cOBA5OXl4bfffrOU/fzzz5BlGf3797/hvm/kxIkT6N27901vh6gqkiQhKCgILVu2ZDJLRA3i9W1/YOuxNACAq7Man8ZGoU2LG38XK0XRZDYzMxOTJ09Gx44dMWzYMCQmJuLHH3/EHXfcAQBISUlBWlqapf6gQYOwbt06fPTRR+jZsyc2bdqELVu2oFu3bkq9hCZFp9Ph6NGjVo9HH30UOTk5mDhxIhITE3HhwgX8+OOPiIuLg9lshq+vL1q0aIGPPvoI58+fx88//1xpxIiKnnrqKbz22mu45557sG/fPkt5586dMWDAADz//POYOHGiVTeEqkiShNmzZ+ONN95AUVERZs6cWWOcNb3GK1eu4IknnsDZs2cxe/ZsHDt2DGfOnMGyZcuwfv16/POf/7SsP3ToUBQUFFiNZdy5c2fcddddmDFjBhISEvDrr79i1qxZePDBBy0jGaSmpqJTp05ISEiwrJeeno6jR4/i/PnzAMpuZjx69KjVMFxFRUX47bffcOedd97o7SMiIlLcp/uS8cm+ZACAWiVh+aQ+6Bnmo2xQNyIcjE6nEwCETqer9FxxcbE4deqUKC4ubrD9ybIsTCaTkGW5wbZZldjYWIGycROsHtOmTRNnz54VY8aMET4+PsLV1VV06tRJzJkzxxLTzp07RefOnYVGoxE9evQQe/bsEQDE5s2bhRBCJCcnCwAiKSnJsr+lS5cKT09P8euvv1rKPv30UwFAJCQkWMVW1fpCCFFYWCh8fX3Fm2++KYQQN4yzptcohBAJCQnijjvuEP7+/sLb21tER0eLb775plJbjR8/XrzwwgtWZdevXxcTJ04UHh4ewsvLS8TFxYn8/PxKr2H37t2Wsvnz51cZz+rVqy111q1bJzp27FjDO9c4x51SzGazSEtLE2azWelQHEpJSYlITU1lu9sYj3dlsN0bz/e/XxPhL3wv2jxf9lh/6LLlOVu3e0352t9JQthg8NMmRK/Xw9vbGzqdrsrRDJKTkxEREdFgd5ULhfvM2tKrr76Kr776CseOHVM6lBrb/dixY7jjjjtw4cIFeHh4NGocAwYMwOzZs/HQQw9VW6cxjjulyLJsGR2CN2bYhizLOHv2LPLy8tCvXz+OZmBDPN6VwXZvHIcuXscjnybAZC67CXz2sPaYe8df3fRs3e415Wt/x6OAblpBQQFOnDiB999/H0899ZTS4dxQjx498OabbyI5OblR95OdnY2xY8di4sSJjbofIiKim3E2Ix8zPjtsSWTH9Q3FM8PbKxxV7TGZpZs2a9Ys9O3bF0OGDMHUqVOVDqdWpkyZgu7duzfqPlq2bInnnnuu2V+RJyIi+5WhN2DKqgToDWWT/gzu4I/Xx3a3q+8u/h5FNy0+Ph7x8fFKh0FERER1kG8oQeyqBFzTlc2I2q2VF5ZP6gNntX1d67SvaImIiIjopplKZTzx+RGcTi+bNjbU1xWrpkTBXWN/1zmZzFbBwe6JI4XxeCMiIlsSQuD5r49h3/lsAICPmzPWTI1GgKd93oTMZLYCZ+eyuYaLiooUjoQcSfnxVn78ERERNaa3fjyDzUmpAACNkwqfxvZDpH/jju7TmOzvWnIjUqvV8PHxsUxt6ubmdtMdoB1paK6mxB7aXQiBoqIiZGZmwsfHB2q1WumQyA5JkgQPDw+UlJQ02WOdiJqOtQcvY/meCwAASQLefbA3+rbxUziqm8Nk9m+CgoIAwJLQ3iwhBGRZhkql4heNDdlTu/v4+FiOO6K6kiQJISEhTfoPNyJqGnacTMf8b09YlheM7oq7utn/9w+T2b+RJAnBwcEICAhASUnJTW9PlmVcv34dLVq04ODONmQv7e7s7MwrskRE1OiOpORi9oYkyH/epvHY4LaIHRSuaEwNhclsNdRqdYMkGbIsw9nZGVqttkknVc0N252IiKjMxawCTItPhKGkbFKEmF4heH5EJ4Wjajj8licismPl09leunQJsiwrHQ4RNTFZ+UZMWZ2I3KKyX5sHRbbAkgd6QqVqPt2SmMwSERERNUOFxlJMW5OIlJyyUXM6BXli5SN94eLUvNK/5vVqiIiIiAilZhmz1h3Bsas6AECItxbxcdHw0ja/YSCZzBIRERE1I0IIvLj5BHafyQIAeGqdED81GkHe9jkpwo0wmSUiIiJqRt7ddQ4bD18BALioVfh4cj90CPRUOKrGw2SWiIiIqJnYmJiCZT+dsywvHd8TA9q2UDCixsdkloiIiKgZ2H06E//e/NekCC/e3Rmje4YoGJFtcJxZIiI7JkkS3N3dYTQaOQMYkQM7djUPM9cdgfnPWRHibgnH9FsjFI7KNpjMEhHZMUmS0KpVKzg7OzOZJXJQKdeLMDU+EUUmMwBgZLcgvDSqi8OcE9jNgIiIiMhO5RSaELs6AdkFJgBAVLgv3pnQq1lNinAjTGaJiIiI7FCxyYxpaxKRnF0IAGgX4IGPJ/eD1lmtcGS2xWSWiMiOybKMc+fOISUlhdPZEjkQsyzw9IYkJKXkAQACPDWIj4uCj5uLsoEpgMksEZGdE0IwkSVyIEIILPjuJHacygAAeGicsDouCqG+bgpHpgwms0RERER2ZOXei1h78DIAwEklYcXDfdA1xFvhqJTDZJaIiIjITmxJSsWb209blpc80AO3tvdXMCLlMZklIiIisgO/ns/Gs5t+tyw/O6IjxvYJVTCipoHJLBEREVETd+qaHo+t/Q0l5rJJESb1b40nh0QqHFXTwGSWiIiIqAlLzStGXHwCCoylAIDhnQPxyn3dHGZShBthMktEZOdcXV2h1WqVDoOIGoGuqASxqxKQoTcCAHqF+eC9ib2hdqBJEW6E09kSEdkxlUqFsLAwaDQaqFS8PkHUnBhKzJix9jDOZxYAAMJbuOHT2H5wdXGsSRFuhGc+IiIioiZGlgX++eXvSEjOAQC0cHfBmqnRaOGhUTiypofJLBEREVETs2jbH9h6PA0A4OqsxqopUWjTwl3hqJomJrNERHZMlmVcuHABV65c4SxgRM3EJ/+7iE/3JQMA1CoJH0zqjZ5hPsoG1YQxmSUisnNmsxlms1npMIioAWw9loZF2/6wLC+K6YbbOwUqGFHTx2SWiIiIqAk4dPE6ntl4FKJsKFnMHtYeD0a3VjYoO8BkloiIiEhh5zLyMeOzwzCZy7oLje8XimeGt1c4KvvAZJaIiIhIQRl6A6asToTeUDYpwuAO/lg0pjsnRaglJrNERERECsk3lGDK6kSk5hUDALq18sLySX3grGaKVltsKSIiIiIFmEplPPH5EfyRpgcAhPq6YtWUKLhrOKdVXTCZJSKyc1qtFhoNB1InsidCCLzw9THsO58NAPB1c8aaqdEI8OTU1HXF1J+IyI6pVCq0bt0aWq2W09kS2ZG3d5zBN0mpAACNkwqfxPZDpL+HwlHZJ0XPfIsXL0ZUVBQ8PT0REBCAmJgYnDlzpsZ14uPjIUmS1UOr5V8xREREZB8+P3gZH+y+AACQJODdB3ujbxs/haOyX4oms3v37sXMmTNx8OBB7Ny5EyUlJbjzzjtRWFhY43peXl5IS0uzPC5fvmyjiImIiIjqb+epDLz87QnL8sJ7u+KubkEKRmT/FO1msH37dqvl+Ph4BAQE4LfffsNtt91W7XqSJCEoiG88EZEsy7h48SLy8vLQsmVLdjUgasKOpOTiqfVHIP85KcLjgyMxeWC4ojE1B02qz6xOpwMA+PnVfKm9oKAAbdq0gSzL6NOnD15//XV07dq1yrpGoxFGo9GyrNeX3TEoy7JN5jGXZRlCCM6ZbmNsd2Ww3W1PlmWYTCaUlpba7LxGZXi8K8Ne2z05uxDT4xNhKCmL+96ewfjXHe3t5nXYut3rsp8mk8zKsow5c+bglltuQbdu3aqt17FjR6xatQo9evSATqfD22+/jUGDBuHkyZMIDQ2tVH/x4sVYuHBhpfKsrCwYDIYGfQ1VkWUZOp0OQgheMbEhtrsy2O62V97mRUVFyMzMhJNTkzmtN3s83pVhj+1+vbAEj355GjlFJQCAfmGeePa2IGRnZykcWe3Zut3z8/NrXVcSonwGYGU98cQT+OGHH7Bv374qk9LqlJSUoHPnzpg4cSJeffXVSs9XdWU2LCwMubm58PLyapDYayLLMrKysuDv7283H7rmgO2uDLa77cmyjLNnz0Kv16NPnz5MZm2Ix7sy7K3dC42leOiTBBxPLfv1uWOQJzY+2h9eWmeFI6sbW7e7Xq+Hr68vdDrdDfO1JnHWmzVrFr7//nv88ssvdUpkAcDZ2Rm9e/fG+fPnq3xeo9FUOf6iSqWy2YdAkiSb7o/KsN2VwXa3vfK2ZrvbHo93ZdhLu5eaZczecNSSyAZ7axEfFwUfN/scF9qW7V6XfSh6FAghMGvWLGzevBk///wzIiIi6rwNs9mM48ePIzg4uBEiJCIiIqo7IQRe3HwCu8+UdSXw1DphzdRoBHu7KhxZ86PoldmZM2di3bp1+Pbbb+Hp6Yn09HQAgLe3N1xdy97syZMno1WrVli8eDEA4JVXXsGAAQPQrl075OXl4a233sLly5cxffp0xV4HERERUUXv7jqHjYevAABc1Cp8PLkfOgR6KhxV86RoMrtixQoAwJAhQ6zKV69ejSlTpgAAUlJSrC415+bmYsaMGUhPT4evry/69u2L/fv3o0uXLrYKm4ioSXFxcYGLi4vSYRDRn75MvIJlP52zLC8d3xMD2rZQMKLmTdFktjb3nu3Zs8dq+Z133sE777zTSBEREdkXlUqF8PBwZGZmNvn+g0SOYPeZTMzbfNyy/OLdnTG6Z4iCETV/PPMRERERNYDjV3WY+cURmP+cFSHulnBMv7Xu9wNR3TCZJSIiIrpJV3KKEBefiCKTGQBwd/cgvDSqCyRJUjiy5o/JLBGRHZNlGZcuXcK1a9fsZiYhouYmp9CE2FUJyC4oG9c+KtwX/x3fCyoVE1lbYDJLRGTnTCYTTCaT0mEQOSRDiRnT1yTiYnYhAKBdgAc+ntwPWme1wpE5DiazRERERPVglgVmr0/CkZQ8AECAp+bPSRE4uogtMZklIiIiqiMhBBb+v5PYcSoDAOChccLquCiE+ropHJnjYTJLREREVEcr917EZwcuAwCcVBJWPNwHXUO8FY7KMTGZJSIiIqqDLUmpeHP7acvym/f3wK3t/RWMyLExmSUiIiKqpV/PZ+PZTb9blp8d0RH39w1VMCJiMktEZOecnJzg5KTohI5EDuHUNT0eW/sbSsxlkyJM6t8aTw6JVDgq4tmPiMiOqVQqtG3bltPZEjWy1LxixMUnoMBYCgAY3jkQr9zXjZMiNAE88xERERHVQFdUgimrEpChL5sUoVeYD96b2BtqTorQJDCZJSIiIqqGocSMGWsP41xmAQAgvIUbPo3tB1cXTorQVDCZJSKyY7IsIyUlBWlpaZzOlqiBybLAP7/6HQnJOQCAlh4uWDM1Gi08NApHRhUxmSUisnMGgwFGo1HpMIiande3/YGtx9IAAK7OanwaG4U2LdwVjor+jsksERER0d98ui8Zn+xLBgCoVRI+mNQbPcN8lA2KqsRkloiIiKiCbcfT8NrWU5blRTHdcHunQAUjopowmSUiIiL6U0JyDuZsPApRNpQsZg9rjwejWysbFNWIySwRERERgHMZ+Zi+JhGm0rKbKcf1DcUzw9srHBXdCJNZIiIicngZegOmrE6E3lA2KcLgDv54fWx3TopgB5jMEhHZObVaDbWaY14S1Ve+oQSxqxKQmlcMAOjWygvLJ/WBs5ppkj3gdLZERHZMpVIhMjKS09kS1ZOpVMbjn/+G0+n5AIBQX1esmhIFdw1TJHvBMx8RERE5JCEEnv/6GH49fx0A4OPmjDVToxHgqVU4MqoLJrNERETkkN768Qw2J6UCADROKnwa2w+R/h4KR0V1xWSWiMiOybKMK1euID09ndPZEtXB2oOXsXzPBQCAJAHvPtgbfdv4KRwV1QeTWSIiO1dcXAyDwaB0GER2Y8fJdMz/9oRleeG9XXFXtyAFI6KbwWSWiIiIHMaRlFzM3pAE+c9JER4fHInJA8MVjYluDpNZIiIicggXswowLT4RhpKyLjkxvULw3IiOCkdFN4vJLBERETV7WflGxK5OQG5RCQBgUGQLLHmgJ1QqTopg75jMEhERUbNWaCzFtDWJuJJTNilCpyBPrHykL1ycmAY1B3wXiYiIqNkqNcuYte4Ijl3VAQCCvbWIj4uGl9ZZ4ciooTCZJSKyc5IkcfYvoioIIfDi5hPYfSYLAOCpdUJ8XDSCvDkpQnPCudqIiOyYSqVC+/btOZ0tURXe3XUOGw9fAQC4qFX4eHI/dAzyVDgqamg88xEREVGzszExBct+OmdZXjq+Jwa0baFgRNRYmMwSERFRs7L7TCb+vfmvSRH+M6ozRvcMUTAiakxMZomI7JgQAqmpqcjIyIAQQulwiBR3/KoOM784AvOfsyLE3RKOaf+IUDgqakxMZomI7JgQAoWFhSguLmYySw4v5XoR4uITUGQyAwDu7h6El0Z1gSRxLNnmjMksERER2b2cQhNiVycgu8AEAIgK98V/x/fipAgOgMksERER2bVikxnT1yQiObsQANAuwAMfT+4HrbNa4cjIFpjMEhERkd0yywJPb0jCkZQ8AECApwbxcVHwcXNRNjCyGSazREREZJeEEFjw3UnsOJUBAPDQOGF1XBRCfd0UjoxsicksERER2aWVey9i7cHLAAAnlYQVD/dB1xBvhaMiW1M0mV28eDGioqLg6emJgIAAxMTE4MyZMzdc76uvvkKnTp2g1WrRvXt3bNu2zQbREhERUVOx5Wgq3tx+2rK85IEeuLW9v4IRkVIUTWb37t2LmTNn4uDBg9i5cydKSkpw5513orCwsNp19u/fj4kTJ2LatGlISkpCTEwMYmJicOLEiWrXISJqrlQqFTp06IDw8HBOZ0sOIzFFj+e/Pm5ZfnZER4ztE6pgRKQkSTShgQmzsrIQEBCAvXv34rbbbquyzoQJE1BYWIjvv//eUjZgwAD06tULK1euvOE+9Ho9vL29odPp4OXl1WCxV0eWZWRmZiIgIIBfNDbEdlcG210ZbHdlsN2VcTI1D+M/PIBCkwwAmNS/NV6L6caxZBuZrY/3uuRrTo0eTR3odDoAgJ+fX7V1Dhw4gLlz51qVjRgxAlu2bKmyvtFohNFotCzr9XoAZW+KLMs3GfGNybIMIYRN9kV/Ybsrg+2uDLa7MtjutpeaV4yp8YctieywTgGYf09nCCE4aUgjs/XxXpf9NJlkVpZlzJkzB7fccgu6detWbb309HQEBgZalQUGBiI9Pb3K+osXL8bChQsrlWdlZcFgMNxc0LUgyzJ0Oh2EEPzL3YbY7spgu9ueEAKZmZkoKCiALMtQqzmupq3weLctvaEUj315Bhn5ZReouga64aVhrZBzPVvhyByDrY/3/Pz8WtdtMsnszJkzceLECezbt69Btztv3jyrK7l6vR5hYWHw9/e3WTcDSZLg7+/Pk50Nsd2VwXa3PVmWkZubC7VaDX9/fzg5NZnTerPH4912jKVmzF6ViOScsotQod4arJraH/6eWoUjcxy2Pt612tq/t03irDdr1ix8//33+OWXXxAaWnMH7qCgIGRkZFiVZWRkICgoqMr6Go0GGo2mUrlKpbLZyUeSJJvuj8qw3ZXBdre98rZmu9sej/fGJ8sC/9p0HAmXcgEALdxdsGxMe/h7atnuNmbL470u+1D0KBBCYNasWdi8eTN+/vlnRERE3HCdgQMHYteuXVZlO3fuxMCBAxsrTCIiIlLI69v+wNZjaQAAV2c1Ponti1CfyhepyHEpmszOnDkTn3/+OdatWwdPT0+kp6cjPT0dxcXFljqTJ0/GvHnzLMtPP/00tm/fjqVLl+L06dNYsGABDh8+jFmzZinxEoiIiKiRfLovGZ/sSwYAqCTgg0m90TPUR9mgqMlRNJldsWIFdDodhgwZguDgYMtj48aNljopKSlIS0uzLA8aNAjr1q3DRx99hJ49e2LTpk3YsmVLjTeNERERkX3ZeiwNr209ZVleNKY7bu8UWMMa5KgU7TNbm2E09uzZU6ls3LhxGDduXCNEREREREo7dPE6ntl4FOVpwuzb22FidGtlg6Imiz2niYiIqMk4l5GPGZ8dhslcNs7ouL6heOaODgpHRU1ZkxjNgIiI6kelUqFdu3bIzMzknd1k9zL0BkxZnQi9oRQAcFsHf7w+tjtn96Ia8cxHRGTnODQUNQf5hhLErkpAal7ZTeDdWnlh+aQ+cFbz2Kaa8QghIiIiRZlKZTzx+RGcTi+b9SnU1xWrpkTBQ8MfkOnGmMwSEdkxIQTS09ORnZ3NuenJLgkh8MLXx7DvfNm0tD5uzlgzNRoBnN2LaonJLBGRHRNCQK/Xo6CggMks2aW3fjyDb5JSAQAaJxU+je2HSH8PhaMie8JkloiIiBSx9uBlLN9zAQAgScC7D/ZG3zZ+CkdF9obJLBEREdnczlMZmP/tCcvygtFdcVe3IAUjInvFZJaIiIhs6khKLp5afwTynz1jHhvcFrGDwhWNiewXk1kiIiKymYtZBZgWnwhDSdmkCPf1CsHzIzopHBXZMyazREREZBNZ+UbErk5AblEJAGBg2xZY8kAPqFScFIHqj8ksERERNbpCYymmrUnElZyySRE6BXniw8l9oXFSKxwZ2TuORkxEZMdUKhUiIyM5nS01aaVmGbPWHcGxqzoAQLC3FqvjouCldVY4MmoOeOYjIrJzarUaajWvblHTJITAi5tPYPeZLACAp9YJa6ZGI9jbVeHIqLlgMktERESN5v92ncfGw1cAAC5qFT56pB86BHoqHBU1J0xmiYjsmBACGRkZuH79OmcAoybny8QreOens5blt8f3xMDIFgpGRM0Rk1kiIjsmhIBOp0N+fj6TWWpSdp/JxLzNxy3LL97dGff2DFEwImqumMwSERFRgzp2NQ8zvzgC85+zIsTdEo7pt0YoHBU1V0xmiYiIqMGkXC/C1PhEFJnMAIC7uwfhpVFdIEkcS5YaB5NZIiIiahA5hSZMWZ2A7AITACAq3Bf/Hd+LkyJQo2IyS0RERDet2GTG9DWJuJhdCABoF+CBjyf3g9aZw8ZR42IyS0RERDfFLAs8vSEJR1LyAAABnhrEx0XBx81F2cDIITCZJSIionoTQmDBdyex41QGAMDdRY3VcVEI9XVTODJyFJzOlojIjkmShIiICGRmZvIGG1LEyr0XsfbgZQCAk0rCykf6omuIt8JRkSPhlVkiIjsmSRKcnZ3h7OzMZJZsbktSKt7cftqy/Ob9PXBre38FIyJHxGSWiIiI6uzX89l4dtPvluVnR3TE/X1DFYyIHBWTWSIiOyaEQFZWFnJzczkDGNnMqWt6PLb2N5SYy465Sf1b48khkQpHRY6KySwRkR0TQiA3Nxc6nY7JLNlEal4x4uITUGAsBQAM7xyAhfd2ZTcXUgyTWSIiIqoVXVEJpqxKQIbeCADoFeaD9yb2gZOa6QQph0cfERER3ZChxIwZaw/jXGYBACC8hRs+je0HVxdOikDKYjJLRERENZJlgX9+9TsSknMAAC3cXbBmajRaeGgUjoyIySwRERHdwOvb/sDWY2kAAFdnNVZNiUKbFu4KR0VUhsksERERVevTfcn4ZF8yAECtkvDBpN7oGeajbFBEFTCZJSIioiptPZaG17aesiwviumG2zsFKhgRUWWczpaIyI5JkoQ2bdrAzc2NQyNRg0pIzsEzXx5F+Yhvs4e1x4PRrZUNiqgKvDJLRGTHJEmCRqOBi4sLk1lqMOcy8jF9TSJMpTIA4IG+oXhmeHuFoyKqGpNZIiIissjQGzBldSL0hrJJEQZ38Mfisd35xxI1WUxmiYjsmBAC2dnZyMvL4wxgdNPyDSWIXZWA1LxiAEC3Vl5YPqkPnDkpAjVhPDqJiOyYEAI5OTlMZummmUplPPH5EZxOzwcAhPq6YtWUKLhreHsNNW1MZomIiBycEAIvfH0M+85nAwB83JyxZmo0Ajy1CkdGdGNMZomIiBzcWz+ewTdJqQAAjZMKn8b2Q6S/h8JREdVOvZLZtm3b4vr165XK8/Ly0LZt25sOioiIiGxj7cHLWL7nAgBAkoB3H+yNvm38FI6KqPbqlcxeunQJZrO5UrnRaERqamqtt/PLL79g9OjRCAkJgSRJ2LJlS4319+zZA0mSKj3S09Pr+hKIiIgc3o6T6Zj/7QnL8oLRXXFXtyAFIyKquzr16v7uu+8s///xxx/h7e1tWTabzdi1axfCw8Nrvb3CwkL07NkTU6dOxdixY2u93pkzZ+Dl5WVZDggIqPW6REREBBxJycXsDUmQ/7xv8LHBbRE7KFzRmIjqo07JbExMDICyQbpjY2OtnnN2dkZ4eDiWLl1a6+2NHDkSI0eOrEsIAMqSVx8fnzqvR0RERMDFrAJMi0+EoaRsUoSYXiF4fkQnhaMiqp86JbOyXHbQR0REIDExES1btmyUoG6kV69eMBqN6NatGxYsWIBbbrml2rpGoxFGo9GyrNfrAZS9lvLX05hkWYYQwib7or+w3ZXBdrc9IQRCQ0Oh1WrZ9jZmr8d7Vr4RsasSkFtUAgAY2LYF3hjbHYCALDf94d3std3tna3bvS77qdfgccnJyfVZ7aYFBwdj5cqV6NevH4xGIz755BMMGTIEhw4dQp8+fapcZ/HixVi4cGGl8qysLBgMhsYOGbIsQ6fTQQgBlYqDR9gK210ZbHdlyLKM4uJiZGVlsd1tyB6P9yKTGU9uOosruWWTIkS2dMWrI8KQl5OtcGS1Z4/t3hzYut3z8/NrXVcS9Rhl+5VXXqnx+Zdffrmum4QkSdi8ebOlK0NtDR48GK1bt8batWurfL6qK7NhYWHIzc216nfbWGRZRlZWFvz9/fmhsyG2uzLY7spguyvD3tq91Czj0c+PYM+ZLABAsLcWmx4fgGBvV4Ujqxt7a/fmwtbtrtfr4evrC51Od8N8rV5XZjdv3my1XFJSguTkZDg5OSEyMrJeyWx9RUdHY9++fdU+r9FooNFoKpWrVCqbfQgkSbLp/qgM210ZbHfbEkIgLy8Per0eAQEBbHcbs5fjXQiBl749ZUlkPbVOiI+LRitfd4Ujqx97affmxpbtXpd91CuZTUpKqlSm1+sxZcoUjBkzpj6brLejR48iODjYpvskImoqhBDIzs7mdLZUo3d3ncPGw1cAAC5qFT6e3A8dgzwVjoqoYTTYhMteXl5YuHAhRo8ejUceeaRW6xQUFOD8+fOW5eTkZBw9ehR+fn5o3bo15s2bh9TUVHz22WcAgGXLliEiIgJdu3aFwWDAJ598gp9//hk7duxoqJdBRETUrHyZeAXLfjpnWV46vicGtG2hYEREDavBklkA0Ol00Ol0ta5/+PBhDB061LI8d+5cAEBsbCzi4+ORlpaGlJQUy/Mmkwn//Oc/kZqaCjc3N/To0QM//fST1TaIiIiozO4zmZi3+bhl+cW7O2N0zxAFIyJqePVKZv/v//7PalkIgbS0NKxdu7ZO48YOGTKkxp/F4uPjrZafe+45PPfcc3WKlYiIyBEdu5qHmV8cgfnP4bbibgnH9FsjFI6KqOHVK5l95513rJZVKhX8/f0RGxuLefPmNUhgREREVD8p14swNT4RRaayqefv7h6El0Z1gSRJCkdG1PDsapxZIiIiqllOoQmxqxOQXWACAESF++K/43tBpWIiS83TTY+tcOXKFVy5cqUhYiEiIqKbUGwyY/qaRCRnFwIA2gV44OPJ/aB1ViscGVHjqVcyW1paipdeegne3t4IDw9HeHg4vL298Z///AclJSUNHSMREVVDkiSEhoYiKCiIPyE7OLMs8PSGJBxJyQMABHhqEB8XBR83F2UDI2pk9epm8NRTT+Gbb77BkiVLMHDgQADAgQMHsGDBAly/fh0rVqxo0CCJiKhqkiTBzc0NWq2WyawDE0JgwXcnseNUBgDAQ+OE1XFRCPV1UzgyosZXr2R23bp12LBhg9XIBT169EBYWBgmTpzIZJaIiMiGVuy9gLUHLwMAnFQSVjzcB11DvBWOisg26tXNQKPRIDw8vFJ5REQEXFz4cwYRka2UT2ebn5/PGcAc1Oakq1iy/Yxl+c37e+DW9v4KRkRkW/VKZmfNmoVXX30VRqPRUmY0GrFo0SLMmjWrwYIjIqKaCSGQmZmJ69evM5l1QL+ez8Zzm45Zlp8d0RH39w1VMCIi26tXN4OkpCTs2rULoaGh6NmzJwDg999/h8lkwrBhwzB27FhL3W+++aZhIiUiIiKLU9f0eGztbygxl/0RM6l/azw5JFLhqIhsr17JrI+PD+6//36rsrCwsAYJiIiIiGqWmleMuPgEFBhLAQDDOwfilfu68SZAckj1SmZXr17d0HEQERFRLeiKSjBlVQIy9GVd/XqF+eC9ib2h5qQI5KDq1Wf29ttvR15eXqVyvV6P22+//WZjIiIioioYSsyYsfYwzmUWAADCW7jh09h+cHXhpAjkuOqVzO7Zswcmk6lSucFgwP/+97+bDoqIiIisybLAP7/6HQnJOQCAlh4uWDM1Gi08NApHRqSsOnUzOHbsrzsmT506hfT0dMuy2WzG9u3b0apVq4aLjoiIiAAAr2/7A1uPpQEAXJ3V+DQ2Cm1auCscFZHy6pTM9urVC5IkQZKkKrsTuLq64r333muw4IiIqGaSJCEkJAQuLi68+acZ+3RfMj7ZlwwAUKskfDCpN3qG+SgbFFETUadkNjk5GUIItG3bFgkJCfD3/2tQZhcXFwQEBECtZr8dIiJbkSQJHh4eKCoqYjLbTG09lobXtp6yLL8W0w23dwpUMCKipqVOyWybNm0AALIsN0owRERE9JdDF6/jmY1HUT4fxuxh7TExurWyQRE1MfUamuuzzz6r8fnJkyfXKxgiIqobIQR0Oh0KCgqsfi0j+3c2Ix8zPjsMk7nsAtK4vqF4Znh7haMianrqlcw+/fTTVsslJSUoKiqCi4sL3NzcmMwSEdmIEAIZGRnIy8tDeHi40uFQA8nQGzBlVQL0hrJJEQZ38MfrY7uzKwlRFeo1NFdubq7Vo6CgAGfOnME//vEPrF+/vqFjJCIichj5hhJMWZ2IazoDAKBbKy8sn9QHzup6fWUTNXsN9slo37493njjjUpXbYmIiKh2TKUyHv/8N/yRpgcAhPq6YtWUKLhr6vVDKpFDaNA/85ycnHDt2rWG3CQREZFDEELg+a+P4dfz1wEAPm7OWDM1GgGeWoUjI2ra6vWn3nfffWe1LIRAWloa3n//fdxyyy0NEhgREZEjeevHM9iclAoA0Dip8GlsP0T6eygcFVHTV69kNiYmxmpZkiT4+/vj9ttvx9KlSxsiLiIiIoex9uBlLN9zAQAgScC7D/ZG3zZ+CkdFZB/qlcyWjzOblZUFABwOhoiIqJ52nEzH/G9PWJYX3tsVd3ULUjAiIvtS5z6zeXl5mDlzJlq2bImgoCAEBQWhZcuWmDVrFvLy8hohRCIiqo4kSQgODoa/vz+HbbJDR1JyMXtDEuQ/J0V4fHAkJg8MVzQmIntTpyuzOTk5GDhwIFJTUzFp0iR07twZAHDq1CnEx8dj165d2L9/P3x9fRslWCIisiZJEjw9PVFcXMxk1s4kZxdi+prDMJSU/doZ0ysEz43oqHBURPanTsnsK6+8AhcXF1y4cAGBgYGVnrvzzjvxyiuv4J133mnQIImIiJqTrHwjYlclIKfQBAAYFNkCSx7oCZWKf5AQ1VWduhls2bIFb7/9dqVEFgCCgoKwZMkSbN68ucGCIyKimgkhkJ+fj8LCQgghlA6HaqHQWIppaxKRklMEAOgU5ImVj/SFixMnRSCqjzp9ctLS0tC1a9dqn+/WrRvS09NvOigiIqqd8qERs7KymMzagVKzjFnrjuDYVR0AINhbi/i4aHhpnRWOjMh+1SmZbdmyJS5dulTt88nJyfDz41AiREREfyeEwIubT2D3mbKRgDy1TlgzNRpB3pwUgehm1CmZHTFiBF588UWYTKZKzxmNRrz00ku46667Giw4IiKi5uLdXeew8fAVAICLWoWPHumHDoGeCkdFZP/qfANYv3790L59e8ycOROdOnWCEAJ//PEHli9fDqPRiLVr1zZWrERERHbpy8QrWPbTOcvy2+N7YmBkCwUjImo+6pTMhoaG4sCBA3jyyScxb948S/8sSZJwxx134P3330dYWFijBEpERGSPdp/JxLzNxy3LL97dGff2DFEwIqLmpc4zgEVEROCHH35Abm4uzp0r+yuzXbt27CtLRET0N8eu5mHmF0dg/nNWhCmDwjH91giFoyJqXuo1nS0A+Pr6Ijo6uiFjISIiajZSrhdhanwiikxmAMDIbkF46Z4unNyCqIHVO5klIiLlSZKEwMBAODk5MUlqQnIKTYhdnYDsgrIbpqPCffHOhF5Qc1IEogbHZJaIyI5JkgRvb28YjUYms01EscmM6WsSkZxdCABoF+CBjyf3g9ZZrXBkRM0TpxshIiJqIGZZ4OkNSTiSkgcACPDUID4uCj5uLsoGRtSMMZklIrJjQggUFBSgqKiIM4ApTAiBhf/vJHacygAAeGicsDouCqG+bgpHRtS8MZklIrJjQghcu3YNmZmZTGYVtnLvRXx24DIAwEklYcXDfdA1xFvhqIiaPyazREREN2lLUire3H7asrzkgR64tb2/ghEROQ5Fk9lffvkFo0ePRkhICCRJwpYtW264zp49e9CnTx9oNBq0a9cO8fHxjR4nERFRdX49n41nN/1uWX52REeM7ROqYEREjkXRZLawsBA9e/bEBx98UKv6ycnJGDVqFIYOHYqjR49izpw5mD59On788cdGjpSIiKiyc1lFeOKLIygxl3XxmNS/NZ4cEqlwVESORdGhuUaOHImRI0fWuv7KlSsRERGBpUuXAgA6d+6Mffv24Z133sGIESMaK0wiIqJKUvOKMXfLeRQYyyZFGN45AAvv7coh0ohszK7GmT1w4ACGDx9uVTZixAjMmTOn2nWMRiOMRqNlWa/XAwBkWYYsy40SZ0WyLEMIYZN90V/Y7spgu9texXOZrc5rBOiKSxAXn4iswhIAQK8wb7w7oRdUEvgeNDKeZ5Rh63avy37sKplNT09HYGCgVVlgYCD0ej2Ki4vh6upaaZ3Fixdj4cKFlcqzsrJgMBgaLdZysixDp9NBCAGVivfb2QrbXRlsd9srb/OioiJkZmbCycmuTut2yVgqY87mczifWTYpQqi3BotHtkF+3nXkKxybI+B5Rhm2bvf8/Np/mpr9WW/evHmYO3euZVmv1yMsLAz+/v7w8vJq9P3LsgxJkuDv788PnQ2x3ZXBdrc9IQRcXFyQk5ODwMBAqNWcZaoxybLA0xuPIim1AADg6+qE+KlRaOvvqXBkjoPnGWXYut21Wm2t69pVMhsUFISMjAyrsoyMDHh5eVV5VRYANBoNNBpNpXKVSmWzD4EkSTbdH5VhuyuD7W57fn5+KC0thVqtZrs3ste3ncLW4+kAAFdnNZbe1w5t/T3Z7jbG84wybNnuddmHXR0FAwcOxK5du6zKdu7ciYEDByoUEREROYpP9yXjk33JAAC1SsJ7E3uhS5C7wlERkaLJbEFBAY4ePYqjR48CKBt66+jRo0hJSQFQ1kVg8uTJlvqPP/44Ll68iOeeew6nT5/G8uXL8eWXX+KZZ55RInwiIsUJIVBUVASDwcAZwBrR1mNpeG3rKcvyophuuL1TgIIREVE5RZPZw4cPo3fv3ujduzcAYO7cuejduzdefvllAEBaWpolsQWAiIgIbN26FTt37kTPnj2xdOlSfPLJJxyWi4gclhACV69eRXp6OpPZRpKQnINnNh5FefPOHtYeD0a3VjYoIrJQtM/skCFDajz5VjW715AhQ5CUlNSIUREREZU5l5GP6WsSYTKXDRM0vl8onhneXuGoiKgiu+ozS0REZCsZegOmrE6E3lAKABjcwR+LxnTnpAhETQyTWSIior/JN5QgdlUCUvOKAQDdWnlh+aQ+cFbza5OoqeGnkoiIqAJTqYzHP/8Np9PLBm0P9XXFqilRcNfY1WiWRA6DySwREdGfhBB4/utj+PX8dQCAj5sz1kyNRoBn7QdwJyLbYjJLRET0pyU/nsHmpFQAgMZJhU9j+yHS30PhqIioJvzNhIjIjkmShJYtW0KSJN6YdJPWHryMFXsuAAAkCXj3wd7o28ZP4aiI6EaYzBIR2TFJkizT2TKZrb8dJ9Mx/9sTluUFo7virm5BCkZERLXFbgZEROTQfruci6fWJ0H+c9jzxwa3ReygcEVjIqLaYzJLRGTHhBAwGAwwGo2cAaweLmYVYPqaRBhLyyZFiOkVgudHdFI4KiKqCyazRER2TAiBlJQUpKWlMZmto6x8I2JXJyC3qAQAMCiyBZY80BMqFbtrENkTJrNERORwCo2lmBqfiCs5ZZMidAryxMpH+sLFiV+LRPaGn1oiInIopWYZs9YdwfFUHQAgxFuL+LhoeGmdFY6MiOqDySwRETkMIQRe3HwCu89kAQA8tU6InxqNIG9OikBkr5jMEhGRw3h31zlsPHwFAOCiVuHjyf3QIdBT4aiI6GYwmSUiIoewMTEFy346Z1leOr4nBrRtoWBERNQQmMwSEVGzt/t0Jv69+a9JEf4zqjNG9wxRMCIiaiicAYyIyI6VzwBW/n+q7NjVPDz5xRGY/5wVIe6WcEz7R4TCURFRQ2EyS0RkxyRJQsuWLSHLMpPZKqRcL8LU+EQUl5gBAHd3D8JLo7qwrYiaEXYzICKiZimn0IQpqxOQXWACAESF++K/43txUgSiZobJLBGRHRNCwGg0wmQycQawCopNZkxbk4iL2YUAgEh/d3w8uR+0zmqFIyOihsZklojIjgkhcPnyZVy7do3J7J/MssDsDUlISskDAAR4arBmajR83FyUDYyIGgWTWSIiajaEEFjw3UnsPJUBAPDQOGF1XBRCfd0UjoyIGguTWSIiajZW7r2ItQcvAwCcVBJWPNwHXUO8FY6KiBoTk1kiImoWtiSl4s3tpy3LSx7ogVvb+ysYERHZApNZIiKye7+ez8azm363LD87oiPG9glVMCIishUms0REZNdOXdPjsbW/ocRcdgPcpP6t8eSQSIWjIiJbYTJLRER2KzWvGHHxCSgwlgIAhncOxCv3deOkCEQOhDOAERHZMUmS4OvrCyGEwyVwuqISTFmVgAy9EQDQu7UP3pvYG2pOikDkUJjMEhHZMUmS4O/v73DJrKHEjBlrD+NcZgEAIKKlOz6NjYKrCydFIHI07GZARER2RZYF/vnV70hIzgEAtPRwQXxcFPzcOSkCkSNiMktEZMeEECgpKUFJSYnDzAD2+rY/sPVYGgDA1VmNT2Oj0KaFu8JREZFSmMwSEdkxIQSSk5ORmprqEMnsp/uS8cm+ZACAWiXhg0m90TPMR9mgiEhRTGaJiMgubD2Whte2nrIsvxbTDbd3ClQwIiJqCpjMEhFRk3fo4nU8s/Eoyi8+zx7WHhOjWysbFBE1CUxmiYioSTuXkY8Znx2GySwDAMb1DcUzw9srHBURNRVMZomIqMnK0BswZXUi9IaySREGd/DH62O7O9QwZERUMyazRETUJOUbShC7KgGpecUAgG6tvLB8Uh84q/nVRUR/4RmBiIiaHFOpjCc+P4LT6fkAgFBfV6yaEgV3Def6ISJrPCsQEdkxSZLg7e0Ns9ncbH56F0Lgha+PYd/5bACAr5sz1kyNRoCnVuHIiKgpYjJLRGTHJElCYGAgJElqNsnsWz+ewTdJqQAAjZMKn8T2Q6S/h8JREVFTxW4GRETUZKw9eBnL91wAAEgS8H8Te6NvGz+FoyKipozJLBGRnTObzTCbzUqHcdN2nEzH/G9PWJYX3tsVI7oGKRgREdmDJpHMfvDBBwgPD4dWq0X//v2RkJBQbd34+HjLz2nlD62W/aiIyDHJsowLFy7gypUrkGVZ6XDq7UhKLmZvSIL856QIjw+OxOSB4YrGRET2QfFkduPGjZg7dy7mz5+PI0eOoGfPnhgxYgQyMzOrXcfLywtpaWmWx+XLl20YMRERNaTk7EJMX3MYhpKyZDymVwieG9FR4aiIyF4onsz+97//xYwZMxAXF4cuXbpg5cqVcHNzw6pVq6pdR5IkBAUFWR6BgZybm4jIHmXlGxG7KgE5hSYAwKDIFljyQE+oVM3jZjYianyKjmZgMpnw22+/Yd68eZYylUqF4cOH48CBA9WuV1BQgDZt2kCWZfTp0wevv/46unbtWmVdo9EIo9FoWdbr9QDKfpqzxU9ysixDCGHXP//ZI7a7MtjutlfxXGar81pDKTSWYmp8IlJyigAAHYM8sXxSbzipYBevg8e7MtjuyrB1u9dlP4oms9nZ2TCbzZWurAYGBuL06dNVrtOxY0esWrUKPXr0gE6nw9tvv41Bgwbh5MmTCA0NrVR/8eLFWLhwYaXyrKwsGAyGhnkhNZBlGTqdDkIIqFSKXwh3GGx3ZbDdba+8zYuKipCZmQknJ/sYcbFUFnjuu/M4nlp2gSHAwxlvjQqHQZ8Lg17h4GqJx7sy2O7KsHW75+fn17qufZz1Khg4cCAGDhxoWR40aBA6d+6MDz/8EK+++mql+vPmzcPcuXMty3q9HmFhYfD394eXl1ejxyvLMiRJgr+/Pz90NsR2Vwbb3fZkWUZeXh4kSUJAQIBdJLNCCMzbfAL7L5VlrZ5aJ3w2rT86BHoqHFnd8HhXBttdGbZu97rc3K/oWa9ly5ZQq9XIyMiwKs/IyEBQUO2GY3F2dkbv3r1x/vz5Kp/XaDTQaDSVylUqlc0+BJIk2XR/VIbtrgy2u+2Vt7W9tPuyn87iy8NXAQAuahU+ntwPnYK9FY6qfni8K4Ptrgxbtntd9qHoUeDi4oK+ffti165dljJZlrFr1y6rq681MZvNOH78OIKDgxsrTCKiJkuSJHh5ecHDw8MuZgD7MvEKlv10zrK8dHxPDGjbQsGIiMjeKf571Ny5cxEbG4t+/fohOjoay5YtQ2FhIeLi4gAAkydPRqtWrbB48WIAwCuvvIIBAwagXbt2yMvLw1tvvYXLly9j+vTpSr4MIiJFlI/uolKpmnwyu/tMJuZtPm5ZfvHuzhjdM0TBiIioOVA8mZ0wYQKysrLw8ssvIz09Hb169cL27dstN4WlpKRYXWrOzc3FjBkzkJ6eDl9fX/Tt2xf79+9Hly5dlHoJRER0A8eu5mHmF0dg/nNWhLhbwjH91giFoyKi5kASQgilg7AlvV4Pb29v6HQ6m90AlpmZiYCAAPbtsSG2uzLY7sooLS1FZmam5QptU5NyvQhjV/yK7IKysWTv7h6E9yb2gdrOx5Ll8a4MtrsybN3udcnXeBQQEdkxWZZx/vx5pKSkNMlxN3MKTZiyOsGSyEaF++K/43vZfSJLRE0Hk1kiImoUxSYzpq9JxMXsQgBAuwAPfDy5H7TOaoUjI6LmhMksERE1OLMs8PSGJBxJyQMABHhqEB8XBR83F2UDI6Jmh8ksERE1KCEEFv6/k9hxqmwMcQ+NE1bHRSHU103hyIioOWIyS0REDWrl3ov47MBlAICTSsKKh/uga4h9TopARE0fk1kiImowW5JS8eb205blN+/vgVvb+ysYERE1d0xmiYioQfx6PhvPbvrdsvzsiI64v2+oghERkSNQfNIEIiKqP0mS4OHhgZKSEkVnADt1TY/H1v6GEnPZ0OWT+rfGk0MiFYuHiBwHk1kiIjsmSRJCQkLg5OSkWDKbmleMuPgEFBhLAQDDOwfilfu6NfnpdYmoeWA3AyIiqjddUQmmrEpAht4IAOgV5oP3JvbmpAhEZDNMZomIqF4MJWbMWHsY5zILAADhLdzwaWw/uLpwUgQish0ms0REdkyWZZw9exaXLl2y6XS2sizwz69+R0JyDgCghbsL1kyNRgsPjc1iICICmMwSEVE9vL7tD2w9lgYAcHVWY9WUKLRp4a5wVETkiJjMEhFRnXy6Lxmf7EsGAKhVEj6Y1Bs9w3yUDYqIHBaTWSIiqrWtx9Lw2tZTluVFMd1we6dABSMiIkfHZJaIiGolITkHz3x5FKJsKFnMHtYeD0a3VjYoInJ4TGaJiOiGzmXkY/qaRJhKy24yG9c3FM8Mb69wVERETGaJiOgGMvQGTFmdCL2hbFKEwR388frY7pwUgYiaBM4ARkRkxyRJgru7O4xGY6Mkl/mGEkxZnYjUvGIAQLdWXlg+qQ+c1bwWQkRNA5NZIiI7JkkSWrVqBWdn5wZPZk2lMp74/Aj+SNMDAEJ9XbFqShTcNfzqIKKmg39aExFRJUIIvPD1Mew7nw0A8HFzxpqp0Qjw1CocGRGRNSazRERUyVs/nsE3SakAAI2TCp/G9kOkv4fCURERVcZklojIjsmyjHPnziElJaXBprNde/Aylu+5AACQJODdB3ujbxu/Btk2EVFDYzJLRGTnhBANlsjuOJmO+d+esCwvvLcr7uoW1CDbJiJqDExmiYgIAPDb5Vw8tT4J8p+TIjw2uC0mDwxXNCYiohthMktERLiYVYDpaxJh/HNShJheIXh+RCeFoyIiujEms0REDi4r34jY1QnILSoBAAyKbIElD/SESsVJEYio6WMyS0TkwAqNpZi2JhFXcsomRegU5ImVj/SFixO/HojIPvBsRUTkoErNMmatO4JjV3UAgGBvLeLjouGldVY4MiKi2mMyS0Rk51xdXaHV1m0yAyEE/rPlBHafyQIAeGqdsGZqNIK8OSkCEdkXzklIRGTHVCoVwsLCoNFooFLV/vrE/+06jw2JVwAALmoVPp7cDx0CPRsrTCKiRsMrs0REDubLxCt456ezluWl43tiQNsWCkZERFR/TGaJiBzI7jOZmLf5uGX5P6M6Y3TPEAUjIiK6OUxmiYjsmCzLuHDhAq5cuXLDWcCOXc3DzC+OwPznrAhxt4Rj2j8ibBEmEVGjYTJLRGTnzGYzzGZzjXVSrhdhanwiikxl9e7uHoSXRnWBJHEsWSKyb0xmiYiauZxCE2JXJyC7wAQAiAr3xX/H9+KkCETULDCZJSJqxopNZkxbk4jk7EIAQLsAD3w8uR+0zmqFIyMiahhMZomImimzLPD0hiQkpeQBAPw9NYiPi4KPm4uygRERNSAms0REzZAQAgu+O4kdpzIAAO4uasTHRSHU103hyIiIGhaTWSKiZmjF3gtYe/AyAMBJJWHlI33RNcRb4aiIiBoek1kiIjun1Wqh0Wgsy5uTrmLJ9jOW5Tfv74Fb2/srERoRUaPjdLZERHZMpVKhdevW0Gq1UKlU+PV8Np7bdMzy/LMjOuL+vqEKRkhE1LiYzBIRNQOlssCn+5Lx353nUGIumxRhUv/WeHJIpMKRERE1ribRzeCDDz5AeHg4tFot+vfvj4SEhBrrf/XVV+jUqRO0Wi26d++Obdu22ShSIqKm5/creZi6/g8s2nYaxSVlkyIM7xyIV+7rxkkRiKjZU/zK7MaNGzF37lysXLkS/fv3x7JlyzBixAicOXMGAQEBlerv378fEydOxOLFi3HPPfdg3bp1iImJwZEjR9CtW7da71eW5SqnfpQkyerkf6PpIVWqv/4eqKpu+X5kWb5h3bps19Z1hRAQQjTpuhXfOyFEte9xVXXrsl3Wrb5ueZtXfK6xYijfX1OuCzTu51NvKMHb209jV+JxCAGo4AlIEiZGtcaLozpDgoAsC54j/mTPxzvrVj633+x3eFOoCzS97/vq6lbV5o15jrjRtiuSRE1btoH+/fsjKioK77//PoCyhgkLC8NTTz2FF154oVL9CRMmoLCwEN9//72lbMCAAejVqxdWrlxZqb7RaITRaLQs6/V6hIWFITExER4eHpXqu7u7o1WrVpblc+fOVdv4rq6uCAsLsyxfuHCh0pSSsixDr9cjICAA4eHhlvKLFy+itLS0yu26uLhY1b106RJMJlOVdZ2cnNC2bVvLckpKCgwGQ5V11Wo1IiP/+snxypUrKC4urrKuJElo3769ZTk1NRWFhYVV1gWADh06WP5/7do1FBQUVFu3Xbt2loM2PT0der2+2rqRkZFQq8sGd8/IyIBOp6u2bkREBJydnS11L1++DC8vL6sPSLk2bdpYbpjJzs5GTk5Otdst748IADk5OcjOzq62bmhoKNzcyoY+ysvLQ2ZmZrV1Q0JCLMegTqdDRkZGtXWDg4Ph6ekJAMjPz0daWlq1dQMDA+HtXXbXekFBAa5du1Zt3YCAAPj4+AAAioqKcPXq1WrrtmzZEn5+fgAAg8GAlJSUSnXKj/eIiAj4+5fdcGQ0GnH58uVqt+vr62upW1JSguTk5Grrent7IzAwEEDZFK4XLlyotq6XlxeCgoIscZ0/f77auh4eHggJCbEsnz17ttq6DX2OKKfVatG6dWvLcsVzRKksQ1dUipxCI3KKSpBnFLgOb2w/mY6sfAO6O6VBDQGtlx8eGRiOtv5/ndt4jvhLxXNEVlYWcnNzq61bm3NE+fHerVs3y+ee54iazxHl/Pz80LJlSwB1P0dcuHABer2+yvO7o54j/q4x8ghZllFQUIA+ffpY2r0xzxHp6emIioqCTqeDl5dXtesBCl+ZNZlM+O233zBv3jxLmUqlwvDhw3HgwIEq1zlw4ADmzp1rVTZixAhs2bKlyvqLFy/GwoULK5XrdLoqDwKj0Wg52ZXXq+6vA4PBYHUHcV5eXqWDUAiBoqIi5ObmWk5g5XVrOggrnuDy8vJqPAgr1s3NzbVK3itSq9WV6lb3paZSqazq5uTkVHvAAqi03ZoO2MzMTMsHIScnp8YvtczMTMsXVU5ODvLz82usW/7e5eTkoLCwEEKIKn9mdXNzg4tL2cDxeXl5yMvLq3a7rq6ulvdZp9PVWFer1VoS3/z8/Brruri4oKioCEDZF0pNdZ2dnS3tX1hYWGNdJycnyzFQVFRUY121Wm05tgwGQ411JUmyHLNGo7HKuuXH+/Xr1y0nb5PJVON2K/61XlJSUmNds9lseT/NZnONdUtLSy3HmSzLNdYtKSmBk9Nfp8Oa6jb0OUIWQIGxFPmlauy/UoyMfBMy8k3QZ6dBV2RCXnEp8o1mVPwuLIEK6XLZyV1C2dBbPQM1uL1nMJydSq3i5znCum75e5ebm1tj4lubc0T58Z6VlQVXV1cAPEfc6BxRUfnnpq7nCJ1Oh6KioirP783xHFFOo9FYjp3yurbMI4QQMBgMVp/PxjxH1PT5/DtFr8xeu3YNrVq1wv79+zFw4EBL+XPPPYe9e/fi0KFDldZxcXHBmjVrMHHiREvZ8uXLsXDhwir/aq3uyuz169erzPQbo5tBVlYW/P39rT4ITfmnhKrq2ttPiGazGZmZmfD396/yymxT/OmsOdQtP94DAgIsCYYjdzMQQiC3uBRpOgPSdAZcyy1Ems5oWU7XGZCZb0CpXPaaBf7aroSaT83ldYd19EdsF2c4mYvRp08fq/NMOZ4jytjz8c66f9U1m82W79W/n9+bwue+OXczyMrKQmBgoNUfAbXdbl0/yzqdDi1atGj6V2ZtQaPRWP3VU87JyanKk/7fVZUI1aWuLMtQq9VwcnKyev5mt2vruvaoqnanxlV+vKvVapu0u5KfDSEEdMUluJZnQJquGNd0BqTlFf+ZqBZbElZTaW36fVX+9aA8WW3poUGIjxbB3loEe7v++f+yf8N83dDSwwVnz55FXp6xVuc1niMajq2PdyojSVKtz+9N4fuzudQtP95VKpWlTmMe97XJ0Sx1Gy2KWmjZsiXUanWlK6oZGRmWfix/FxQUVKf6RET1kW8oKbuaWp6g5v2VoF7TFSMtz2AZOaC+fN2crRLUYB8tQrxdLYlroLcGGid1jduoy00SRETNkaLJrIuLC/r27Ytdu3YhJiYGQNmJedeuXZg1a1aV6wwcOBC7du3CnDlzLGU7d+606qZARFSTYpPZkpD+dRW12HKVNS3PgHxj1X3RastT61SWmJZfSfXWIshbixCfv5JVV5eaE1UiIroxxbsZzJ07F7GxsejXrx+io6OxbNkyFBYWIi4uDgAwefJktGrVCosXLwYAPP300xg8eDCWLl2KUaNGYcOGDTh8+DA++ugjJV8GETURxlIz0nWGvxJT3V8J6rU//59XVHJT+3BzUSO4QmIa9GeyGuzz178eGtudXl1cXCw3KxERORrFk9kJEyYgKysLL7/8MtLT09GrVy9s377dMrRGSkqKVZ+MQYMGYd26dfjPf/6Df//732jfvj22bNlSpzFmicg+lZhlZOYbkZZn3Uf1WoW+qtkFVd+xW1suTqqyhLT8536fv/VV9XaFl6tTk5mMQKVSITw83OoOYyIiR6J4MgsAs2bNqrZbwZ49eyqVjRs3DuPGjWvkqIjIlsyyQHaB0ZKYVkxQy6+yZuUbId/E+CtOKgmBXtoq+6iWX2X1c3dpMokqERHdWJNIZomoeRNC4Hqh6c+f+ov/uqJa4epqhv6vIarqQyUBAZ5a65uofKx//m/poYFKxUSViKg5YTJLRDel4hBVqXlFOHslCwVyLtL1xnoMUVW9moaoCvZ2RYCnBk5qx/uZXZZlXLp0CXl5eWjZsiW7GhCRw2EyS0Q1spchqhyZyWSqdnYfIqLmjskskQOrOERVxaGqrukMSG/EIaqCLcNTcYgqIiK6OUxmiZqpqoaoulbhqmpjDFEV6KWFh6oEHUP90crXzeZDVBERkePhtwyRHSo1y8iwwRBV5VdPra6s1jBElSzLyMzMREBA5TnTiYiIGgOTWaIm5kZDVKXrDMjMNzTKEFVBFRLXFhyiioiI7ACTWSIb4hBVREREDYvJLFEDqThEVflNVBV//k/Xc4gqahxOTk5wcuLpnIgcE89+RLWk9BBVIT6uCPTSwsWJiSr9RaVSoW3btpzOlogcFpNZItQ8RFV50lrAIaqIiIiaHCaz1OxVHKIqXV/811BVeX/2VW2AIapcndU19lHlEFVERESNg9+uZNdKzDIyqxmiKiU7H1mFx3G98OaHqArx1lrd6W/VFcBbC29XZ975T4qQZRkpKSnIzc3ldLZE5JCYzFKTdaMhqtJ0xcjKN970EFVBFX7m/3sf1WBvLfw4RBU1cQaDAUajUekwiIgUwWSWFGG7Iao0f/7UzyGqiIiImiMms9TghBDIKyrBNV1xWV9VBYaoCvTUAAY9QoIC+bMrERFRM8ZkluqsqiGqym+kKhsFoHGHqAr2dkWgtwYap+rv/C+bVjX/pmIgIiKipo/JLFn5+xBV6X/ro5qWZ0D+zQ5RpXGq8iaqEB9XDlFFREREdcJk1oHYYogqNxe1VWIa5M0hqoiIiKjxMKtoJkrMMjL0hir7qJaPAJBd0DBDVAVbbqb6W19Vb1d4uTrxzn8iG1Or1VCr+WsGETkmJrN2wBZDVDmrJQR6lfVLDfozUeUQVURNn0qlQmRkJKezJSKHxWRWYUIIZBeYyn7ur3hFtcGHqNLWOEMVh6giIiIie8RkthEJIZBbZMK5rCIcz8lEut5Y6ef/dJ0BJnPjDVEV7O2KAE8NnNS8YkNERETND5PZRmQyy+j72q6b2kb5EFXV9VG90RBVRNS8ybKMK1eucDpbInJYTGYbkcZJjRbuLrheWPWNV55ap7Kf/StcVeUQVURUV8XFxTAYDEqHQUSkCCazjWxktyDoCgoRHuiLVj6uHKKKiIiIqAExm2pkr9zXFZmZmQgICODPf0REREQNjNkVEREREdktJrNEREREZLeYzBIRERGR3WIyS0Rk5yRJYp98InJYvAGMiMiOqVQqtG/fntPZEpHD4pmPiIiIiOwWk1kiIiIisltMZomI7JgQAqmpqcjIyIAQQulwiIhsjsksEZEdE0KgsLAQxcXFTGaJyCExmSUiIiIiu8VkloiIiIjsFpNZIiIiIrJbTGaJiIiIyG4xmSUiIiIiu+VwM4CV3+2r1+ttsj9ZlpGfnw+tVsvZeWyI7a4MtrvtybKMgoICFBYWQq/Xw8nJ4U7riuHxrgy2uzJs3e7leVptRmlxuLNefn4+ACAsLEzhSIiIiIioJvn5+fD29q6xjiQcbGBCWZZx7do1eHp6QpKkRt+fXq9HWFgYrly5Ai8vr0bfH5VhuyuD7a4Mtrsy2O7KYLsrw9btLoRAfn4+QkJCbngl2OGuzKpUKoSGhtp8v15eXvzQKYDtrgy2uzLY7spguyuD7a4MW7b7ja7IlmNnEyIiIiKyW0xmiYiIiMhuMZltZBqNBvPnz4dGo1E6FIfCdlcG210ZbHdlsN2VwXZXRlNud4e7AYyIiIiImg9emSUiIiIiu8VkloiIiIjsFpNZIiIiIrJbTGaJiIiIyG4xmbWxe++9F61bt4ZWq0VwcDAeeeQRXLt2TemwmrVLly5h2rRpiIiIgKurKyIjIzF//nyYTCalQ2v2Fi1ahEGDBsHNzQ0+Pj5Kh9NsffDBBwgPD4dWq0X//v2RkJCgdEjN2i+//ILRo0cjJCQEkiRhy5YtSofkEBYvXoyoqCh4enoiICAAMTExOHPmjNJhNXsrVqxAjx49LJMlDBw4ED/88IPSYVlhMmtjQ4cOxZdffokzZ87g66+/xoULF/DAAw8oHVazdvr0aciyjA8//BAnT57EO++8g5UrV+Lf//630qE1eyaTCePGjcMTTzyhdCjN1saNGzF37lzMnz8fR44cQc+ePTFixAhkZmYqHVqzVVhYiJ49e+KDDz5QOhSHsnfvXsycORMHDx7Ezp07UVJSgjvvvBOFhYVKh9ashYaG4o033sBvv/2Gw4cP4/bbb8d9992HkydPKh2aBYfmUth3332HmJgYGI1GODs7Kx2Ow3jrrbewYsUKXLx4UelQHEJ8fDzmzJmDvLw8pUNpdvr374+oqCi8//77AABZlhEWFoannnoKL7zwgsLRNX+SJGHz5s2IiYlROhSHk5WVhYCAAOzduxe33Xab0uE4FD8/P7z11luYNm2a0qEA4JVZReXk5OCLL77AoEGDmMjamE6ng5+fn9JhEN0Uk8mE3377DcOHD7eUqVQqDB8+HAcOHFAwMqLGp9PpAIDnchsym83YsGEDCgsLMXDgQKXDsWAyq4Dnn38e7u7uaNGiBVJSUvDtt98qHZJDOX/+PN577z089thjSodCdFOys7NhNpsRGBhoVR4YGIj09HSFoiJqfLIsY86cObjlllvQrVs3pcNp9o4fPw4PDw9oNBo8/vjj2Lx5M7p06aJ0WBZMZhvACy+8AEmSanycPn3aUv/ZZ59FUlISduzYAbVajcmTJ4O9Pequru0OAKmpqbjrrrswbtw4zJgxQ6HI7Vt92p2IqCHNnDkTJ06cwIYNG5QOxSF07NgRR48exaFDh/DEE08gNjYWp06dUjosC/aZbQBZWVm4fv16jXXatm0LFxeXSuVXr15FWFgY9u/f36Qu2duDurb7tWvXMGTIEAwYMADx8fFQqfi3XH3U53hnn9nGYTKZ4Obmhk2bNln12YyNjUVeXh5/9bEB9pm1vVmzZuHbb7/FL7/8goiICKXDcUjDhw9HZGQkPvzwQ6VDAQA4KR1Ac+Dv7w9/f/96rSvLMgDAaDQ2ZEgOoS7tnpqaiqFDh6Jv375YvXo1E9mbcDPHOzUsFxcX9O3bF7t27bIkU7IsY9euXZg1a5aywRE1MCEEnnrqKWzevBl79uxhIqsgWZabVN7CZNaGDh06hMTERPzjH/+Ar68vLly4gJdeegmRkZG8KtuIUlNTMWTIELRp0wZvv/02srKyLM8FBQUpGFnzl5KSgpycHKSkpMBsNuPo0aMAgHbt2sHDw0PZ4JqJuXPnIjY2Fv369UN0dDSWLVuGwsJCxMXFKR1as1VQUIDz589blpOTk3H06FH4+fmhdevWCkbWvM2cORPr1q3Dt99+C09PT0u/cG9vb7i6uiocXfM1b948jBw5Eq1bt0Z+fj7WrVuHPXv24Mcff1Q6tL8Ispljx46JoUOHCj8/P6HRaER4eLh4/PHHxdWrV5UOrVlbvXq1AFDlgxpXbGxsle2+e/dupUNrVt577z3RunVr4eLiIqKjo8XBgweVDqlZ2717d5XHdWxsrNKhNWvVncdXr16tdGjN2tSpU0WbNm2Ei4uL8Pf3F8OGDRM7duxQOiwr7DNLRERERHaLHQeJiIiIyG4xmSUiIiIiu8VkloiIiIjsFpNZIiIiIrJbTGaJiIiIyG4xmSUiIiIiu8VkloiIiIjsFpNZIiIiIrJbTGaJiIiIyG4xmSUiagKmTJmCmJgYm+4zPj4ePj4+Nt0nEVFDYzJLRERERHaLySwRURMzZMgQzJ49G8899xz8/PwQFBSEBQsWWNWRJAkrVqzAyJEj4erqirZt22LTpk2W5/fs2QNJkpCXl2cpO3r0KCRJwqVLl7Bnzx7ExcVBp9NBkiRIklRpH0RE9oDJLBFRE7RmzRq4u7vj0KFDWLJkCV555RXs3LnTqs5LL72E+++/H7///jsmTZqEBx98EH/88Uettj9o0CAsW7YMXl5eSEtLQ1paGv71r381xkshImpUTGaJiJqgHj16YP78+Wjfvj0mT56Mfv36YdeuXVZ1xo0bh+nTp6NDhw549dVX0a9fP7z33nu12r6Liwu8vb0hSRKCgoIQFBQEDw+PxngpRESNisksEVET1KNHD6vl4OBgZGZmWpUNHDiw0nJtr8wSETUXTGaJiJogZ2dnq2VJkiDLcq3XV6nKTu9CCEtZSUlJwwRHRNSEMJklIrJTBw8erLTcuXNnAIC/vz8AIC0tzfL80aNHreq7uLjAbDY3bpBERI2MySwRkZ366quvsGrVKpw9exbz589HQkICZs2aBQBo164dwsLCsGDBApw7dw5bt27F0qVLrdYPDw9HQUEBdu3ahezsbBQVFSnxMoiIbgqTWSIiO7Vw4UJs2LABPXr0wGeffYb169ejS5cuAMq6Kaxfvx6nT59Gjx498Oabb+K1116zWn/QoEF4/PHHMWHCBPj7+2PJkiVKvAwiopsiiYodqoiIyC5IkoTNmzfbfNYwIqKmhldmiYiIiMhuMZklIiIiIrvlpHQARERUd+whRkRUhldmiYiIiMhuMZklIiIiIrvFZJaIiIiI7BaTWSIiIiKyW0xmiYiIiMhuMZklIiIiIrvFZJaIiIiI7BaTWSIiIiKyW/8fEwi2IVXPlHAAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "class LeakyReLU(brainstate.nn.Module):\n", " \"\"\"Leaky ReLU activation: y = max(alpha * x, x)\"\"\"\n", " \n", " def __init__(self, negative_slope=0.01):\n", " super().__init__()\n", " self.negative_slope = negative_slope\n", " \n", " def update(self, x):\n", " return jnp.where(x > 0, x, self.negative_slope * x)\n", " \n", " def __repr__(self):\n", " return f\"LeakyReLU(negative_slope={self.negative_slope})\"\n", "\n", "# Test the activation\n", "activation = LeakyReLU(negative_slope=0.1)\n", "x = jnp.array([-2.0, -1.0, 0.0, 1.0, 2.0])\n", "y = activation(x)\n", "\n", "print(\"Activation:\", activation)\n", "print(f\"Input: {x}\")\n", "print(f\"Output: {y}\")\n", "\n", "# Visualize\n", "x_plot = jnp.linspace(-3, 3, 100)\n", "y_plot = activation(x_plot)\n", "\n", "plt.figure(figsize=(8, 5))\n", "plt.plot(x_plot, y_plot, linewidth=2, label='LeakyReLU(0.1)')\n", "plt.axhline(0, color='gray', linestyle='--', alpha=0.3)\n", "plt.axvline(0, color='gray', linestyle='--', alpha=0.3)\n", "plt.grid(alpha=0.3)\n", "plt.xlabel('Input')\n", "plt.ylabel('Output')\n", "plt.title('Leaky ReLU Activation Function')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "composition", "metadata": {}, "source": [ "## 3. Module Composition and Nesting\n", "\n", "The real power of modules comes from composing them into larger networks.\n", "\n", "### Sequential Composition\n", "\n", "Build a network by stacking layers sequentially:" ] }, { "cell_type": "code", "execution_count": 6, "id": "sequential", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:11.901818Z", "start_time": "2025-10-11T08:24:11.078282Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:26.861518Z", "iopub.status.busy": "2026-05-30T16:19:26.861276Z", "iopub.status.idle": "2026-05-30T16:19:27.887499Z", "shell.execute_reply": "2026-05-30T16:19:27.886615Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MLP Architecture:\n", "MLP(\n", " layers=[\n", " Linear(in_features=10, out_features=64, use_bias=True),\n", " LeakyReLU(negative_slope=0.0),\n", " Linear(in_features=64, out_features=32, use_bias=True),\n", " LeakyReLU(negative_slope=0.0),\n", " Linear(in_features=32, out_features=5, use_bias=True)\n", " ],\n", " layer_0=Linear(in_features=10, out_features=64, use_bias=True),\n", " activation_0=LeakyReLU(negative_slope=0.0),\n", " layer_1=Linear(in_features=64, out_features=32, use_bias=True),\n", " activation_1=LeakyReLU(negative_slope=0.0),\n", " layer_2=Linear(in_features=32, out_features=5, use_bias=True)\n", ")\n", "\n", "Input shape: (10,)\n", "Output shape: (5,)\n", "Output: [-0.49218878 0.5558434 -0.6296929 0.252957 0.37388653]\n" ] } ], "source": [ "class MLP(brainstate.nn.Module):\n", " \"\"\"Multi-layer perceptron with customizable architecture.\"\"\"\n", " \n", " def __init__(self, layer_sizes, activation='relu'):\n", " super().__init__()\n", " \n", " self.layers = []\n", " \n", " # Create layers\n", " for i in range(len(layer_sizes) - 1):\n", " # Add linear layer\n", " layer = Linear(layer_sizes[i], layer_sizes[i+1])\n", " setattr(self, f'layer_{i}', layer) # Register as attribute\n", " self.layers.append(layer)\n", " \n", " # Add activation (except for last layer)\n", " if i < len(layer_sizes) - 2:\n", " if activation == 'relu':\n", " act = LeakyReLU(negative_slope=0.0) # Standard ReLU\n", " else:\n", " act = LeakyReLU(negative_slope=0.01)\n", " setattr(self, f'activation_{i}', act)\n", " self.layers.append(act)\n", " \n", " def update(self, x):\n", " \"\"\"Forward pass through all layers.\"\"\"\n", " for layer in self.layers:\n", " x = layer(x)\n", " return x\n", "\n", "# Create a 3-layer MLP\n", "brainstate.random.seed(0)\n", "mlp = MLP(layer_sizes=[10, 64, 32, 5])\n", "\n", "# Forward pass\n", "x = brainstate.random.randn(10)\n", "y = mlp(x)\n", "\n", "print(\"MLP Architecture:\")\n", "print(mlp)\n", "print(f\"\\nInput shape: {x.shape}\")\n", "print(f\"Output shape: {y.shape}\")\n", "print(f\"Output: {y}\")" ] }, { "cell_type": "markdown", "id": "residual", "metadata": {}, "source": [ "### Residual Connections\n", "\n", "Implement skip connections for deeper networks:" ] }, { "cell_type": "code", "execution_count": 7, "id": "residual_block", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:14.082398Z", "start_time": "2025-10-11T08:24:13.803132Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:27.890012Z", "iopub.status.busy": "2026-05-30T16:19:27.889770Z", "iopub.status.idle": "2026-05-30T16:19:28.282829Z", "shell.execute_reply": "2026-05-30T16:19:28.282026Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ResNet:\n", "ResNet(\n", " input_proj=Linear(in_features=10, out_features=32, use_bias=True),\n", " blocks=[\n", " ResidualBlock(\n", " linear1=Linear(in_features=32, out_features=32, use_bias=True),\n", " activation=LeakyReLU(negative_slope=0.0),\n", " linear2=Linear(in_features=32, out_features=32, use_bias=True)\n", " ),\n", " ResidualBlock(\n", " linear1=Linear(in_features=32, out_features=32, use_bias=True),\n", " activation=LeakyReLU(negative_slope=0.0),\n", " linear2=Linear(in_features=32, out_features=32, use_bias=True)\n", " ),\n", " ResidualBlock(\n", " linear1=Linear(in_features=32, out_features=32, use_bias=True),\n", " activation=LeakyReLU(negative_slope=0.0),\n", " linear2=Linear(in_features=32, out_features=32, use_bias=True)\n", " )\n", " ],\n", " block_0=ResidualBlock(...),\n", " block_1=ResidualBlock(...),\n", " block_2=ResidualBlock(...),\n", " output_proj=Linear(in_features=32, out_features=5, use_bias=True)\n", ")\n", "\n", "Output shape: (5,)\n" ] } ], "source": [ "class ResidualBlock(brainstate.nn.Module):\n", " \"\"\"Residual block: y = F(x) + x\"\"\"\n", " \n", " def __init__(self, dim):\n", " super().__init__()\n", " \n", " # Two linear layers with activation in between\n", " self.linear1 = Linear(dim, dim)\n", " self.activation = LeakyReLU(0.0)\n", " self.linear2 = Linear(dim, dim)\n", " \n", " def update(self, x):\n", " # Compute residual\n", " residual = x\n", " \n", " # Forward through layers\n", " out = self.linear1(x)\n", " out = self.activation(out)\n", " out = self.linear2(out)\n", " \n", " # Add residual\n", " return out + residual\n", "\n", "class ResNet(brainstate.nn.Module):\n", " \"\"\"Simple ResNet with multiple residual blocks.\"\"\"\n", " \n", " def __init__(self, input_dim, hidden_dim, output_dim, n_blocks=3):\n", " super().__init__()\n", " \n", " # Input projection\n", " self.input_proj = Linear(input_dim, hidden_dim)\n", " \n", " # Residual blocks\n", " self.blocks = []\n", " for i in range(n_blocks):\n", " block = ResidualBlock(hidden_dim)\n", " setattr(self, f'block_{i}', block)\n", " self.blocks.append(block)\n", " \n", " # Output projection\n", " self.output_proj = Linear(hidden_dim, output_dim)\n", " \n", " def update(self, x):\n", " # Project to hidden dimension\n", " x = self.input_proj(x)\n", " \n", " # Pass through residual blocks\n", " for block in self.blocks:\n", " x = block(x)\n", " \n", " # Project to output\n", " x = self.output_proj(x)\n", " return x\n", "\n", "# Create ResNet\n", "brainstate.random.seed(0)\n", "resnet = ResNet(input_dim=10, hidden_dim=32, output_dim=5, n_blocks=3)\n", "\n", "# Forward pass\n", "x = brainstate.random.randn(10)\n", "y = resnet(x)\n", "\n", "print(\"ResNet:\")\n", "print(resnet)\n", "print(f\"\\nOutput shape: {y.shape}\")" ] }, { "cell_type": "markdown", "id": "size_inference", "metadata": {}, "source": [ "## 4. Automatic Input/Output Size Inference\n", "\n", "One of BrainState's most powerful features is **automatic input/output size inference**. Every `brainstate.nn.Module` instance has `in_size` and `out_size` properties that track the shape of data flowing through the module (excluding the batch dimension).\n", "\n", "### Key Concepts\n", "\n", "✅ **`in_size`**: Input shape without batch dimension \n", "✅ **`out_size`**: Output shape without batch dimension (automatically inferred) \n", "✅ **Automatic propagation**: When `in_size` is known, `out_size` is computed automatically \n", "✅ **Sequential composition**: Output size of one layer becomes input size of next layer\n", "\n", "This mechanism eliminates the need to manually calculate dimensions through network layers, making it much easier to build complex architectures." ] }, { "cell_type": "markdown", "id": "size_basic", "metadata": {}, "source": [ "### Example 1: Basic Size Inference" ] }, { "cell_type": "code", "execution_count": 8, "id": "size_basic_code", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:14.681746Z", "start_time": "2025-10-11T08:24:14.087343Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:28.284967Z", "iopub.status.busy": "2026-05-30T16:19:28.284778Z", "iopub.status.idle": "2026-05-30T16:19:31.759707Z", "shell.execute_reply": "2026-05-30T16:19:31.758700Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Layer: Linear(\n", " in_size=(10,),\n", " out_size=(5,),\n", " weight=ParamState(\n", " value={\n", " 'bias': ShapedArray(float32[5]),\n", " 'weight': ShapedArray(float32[10,5])\n", " }\n", " )\n", ")\n", "Input size: (10,)\n", "Output size: (5,)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Input shape: (32, 10) (batch_size=32, in_features=10)\n", "Output shape: (32, 5) (batch_size=32, out_features=5)\n", "\n", "Note: in_size and out_size DO NOT include the batch dimension!\n" ] } ], "source": [ "# Create a linear layer with explicit in_size and out_size\n", "layer = brainstate.nn.Linear(in_size=(10,), out_size=(5,))\n", "\n", "print(\"Layer:\", layer)\n", "print(f\"Input size: {layer.in_size}\")\n", "print(f\"Output size: {layer.out_size}\")\n", "\n", "# Forward pass with batch dimension\n", "x = brainstate.random.randn(32, 10) # (batch_size, in_features)\n", "y = layer(x)\n", "\n", "print(f\"\\nInput shape: {x.shape} (batch_size=32, in_features=10)\")\n", "print(f\"Output shape: {y.shape} (batch_size=32, out_features=5)\")\n", "print(\"\\nNote: in_size and out_size DO NOT include the batch dimension!\")" ] }, { "cell_type": "markdown", "id": "size_conv", "metadata": {}, "source": [ "### Example 2: Size Inference with Convolution\n", "\n", "Convolution layers automatically compute output spatial dimensions based on:\n", "- Input spatial size\n", "- Kernel size\n", "- Stride\n", "- Padding mode" ] }, { "cell_type": "code", "execution_count": 9, "id": "size_conv_code", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:15.016234Z", "start_time": "2025-10-11T08:24:14.686828Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:31.762752Z", "iopub.status.busy": "2026-05-30T16:19:31.762203Z", "iopub.status.idle": "2026-05-30T16:19:32.467688Z", "shell.execute_reply": "2026-05-30T16:19:32.466768Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Conv2d Layer:\n", " in_size: (28, 28, 3)\n", " out_size: (28, 28, 32)\n", "\n", " Input: (H, W, C) = (28, 28, 3)\n", " Output: (H', W', C') = (28, 28, 32)\n", "\n", "With 'SAME' padding and stride=1, spatial dimensions are preserved!\n", "\n", "With 'VALID' padding and stride=2:\n", " in_size: (28, 28, 3)\n", " out_size: (13, 13, 32)\n", " Spatial dimensions are reduced!\n" ] } ], "source": [ "# Create a 2D convolution layer\n", "conv = brainstate.nn.Conv2d(\n", " in_size=(28, 28, 3), # (height, width, channels)\n", " out_channels=32,\n", " kernel_size=3,\n", " stride=1,\n", " padding='SAME'\n", ")\n", "\n", "print(\"Conv2d Layer:\")\n", "print(f\" in_size: {conv.in_size}\")\n", "print(f\" out_size: {conv.out_size}\")\n", "print(f\"\\n Input: (H, W, C) = {conv.in_size}\")\n", "print(f\" Output: (H', W', C') = {conv.out_size}\")\n", "print(\"\\nWith 'SAME' padding and stride=1, spatial dimensions are preserved!\")\n", "\n", "# Test with different padding\n", "conv_valid = brainstate.nn.Conv2d(\n", " in_size=(28, 28, 3),\n", " out_channels=32,\n", " kernel_size=3,\n", " stride=2,\n", " padding='VALID'\n", ")\n", "\n", "print(f\"\\nWith 'VALID' padding and stride=2:\")\n", "print(f\" in_size: {conv_valid.in_size}\")\n", "print(f\" out_size: {conv_valid.out_size}\")\n", "print(\" Spatial dimensions are reduced!\")" ] }, { "cell_type": "markdown", "id": "size_pooling", "metadata": {}, "source": [ "### Example 3: Size Inference with Pooling and Flatten\n", "\n", "Pooling layers reduce spatial dimensions, and Flatten layers convert multi-dimensional tensors to 1D vectors. BrainState tracks all these transformations automatically." ] }, { "cell_type": "code", "execution_count": 10, "id": "size_pooling_code", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:15.028083Z", "start_time": "2025-10-11T08:24:15.021240Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:32.470234Z", "iopub.status.busy": "2026-05-30T16:19:32.469909Z", "iopub.status.idle": "2026-05-30T16:19:32.480268Z", "shell.execute_reply": "2026-05-30T16:19:32.479520Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MaxPool2d Layer:\n", " in_size: (28, 28, 32) (H=28, W=28, C=32)\n", " out_size: (14, 14, 32) (H=14, W=14, C=32)\n", " Spatial dimensions reduced by 2x!\n", "\n", "Flatten Layer:\n", " in_size: (14, 14, 32) (3D tensor)\n", " out_size: (6272,) (1D vector)\n", " Total elements: 6272 = 6272\n" ] } ], "source": [ "# MaxPool reduces spatial dimensions\n", "pool = brainstate.nn.MaxPool2d(\n", " in_size=(28, 28, 32),\n", " kernel_size=(2, 2),\n", " stride=(2, 2),\n", " channel_axis=-1\n", ")\n", "\n", "print(\"MaxPool2d Layer:\")\n", "print(f\" in_size: {pool.in_size} (H=28, W=28, C=32)\")\n", "print(f\" out_size: {pool.out_size} (H=14, W=14, C=32)\")\n", "print(\" Spatial dimensions reduced by 2x!\")\n", "\n", "# Flatten converts to 1D\n", "flatten = brainstate.nn.Flatten(in_size=(14, 14, 32))\n", "\n", "print(f\"\\nFlatten Layer:\")\n", "print(f\" in_size: {flatten.in_size} (3D tensor)\")\n", "print(f\" out_size: {flatten.out_size} (1D vector)\")\n", "print(f\" Total elements: {14 * 14 * 32} = {flatten.out_size[0]}\")" ] }, { "cell_type": "markdown", "id": "sequential_intro", "metadata": {}, "source": [ "## 5. Sequential Composition and Deep Networks\n", "\n", "`brainstate.nn.Sequential` is a powerful container that chains multiple modules together. It automatically propagates `out_size` from one layer to the `in_size` of the next layer, enabling effortless construction of deep networks.\n", "\n", "### The `.desc()` Pattern\n", "\n", "For layers that need to infer their `in_size` from the previous layer, BrainState provides the `.desc()` method, which creates a **layer descriptor** that will be instantiated when the input size becomes available.\n", "\n", "```python\n", "# Instead of:\n", "brainstate.nn.Linear(in_size=(10,), out_size=(5,))\n", "\n", "# Use descriptor in Sequential:\n", "brainstate.nn.Linear.desc(out_size=5) # in_size will be inferred!\n", "```" ] }, { "cell_type": "markdown", "id": "sequential_basic", "metadata": {}, "source": [ "### Example 1: Simple Sequential Network" ] }, { "cell_type": "code", "execution_count": 11, "id": "sequential_basic_code", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:16.027483Z", "start_time": "2025-10-11T08:24:15.053887Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:32.482177Z", "iopub.status.busy": "2026-05-30T16:19:32.482001Z", "iopub.status.idle": "2026-05-30T16:19:34.021659Z", "shell.execute_reply": "2026-05-30T16:19:34.020938Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sequential MLP:\n", "Sequential(\n", " in_size=(10,),\n", " out_size=(5,),\n", " layers=[\n", " Linear(\n", " in_size=(10,),\n", " out_size=(64,),\n", " weight=ParamState(\n", " value={\n", " 'bias': ShapedArray(float32[64]),\n", " 'weight': ShapedArray(float32[10,64])\n", " }\n", " )\n", " ),\n", " ReLU(),\n", " Linear(\n", " in_size=(64,),\n", " out_size=(32,),\n", " weight=ParamState(\n", " value={\n", " 'bias': ShapedArray(float32[32]),\n", " 'weight': ShapedArray(float32[64,32])\n", " }\n", " )\n", " ),\n", " ReLU(),\n", " Linear(\n", " in_size=(32,),\n", " out_size=(5,),\n", " weight=ParamState(\n", " value={\n", " 'bias': ShapedArray(float32[5]),\n", " 'weight': ShapedArray(float32[32,5])\n", " }\n", " )\n", " )\n", " ]\n", ")\n", "\n", "Input size: (10,)\n", "Output size: (5,)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Forward pass:\n", " Input: (8, 10)\n", " Output: (8, 5)\n" ] } ], "source": [ "# Build a simple MLP with Sequential\n", "brainstate.random.seed(42)\n", "\n", "mlp = brainstate.nn.Sequential(\n", " brainstate.nn.Linear((10,), (64,)), # First layer needs explicit in_size\n", " brainstate.nn.ReLU(), # Element-wise, preserves shape\n", " brainstate.nn.Linear.desc(out_size=32), # in_size inferred from previous layer\n", " brainstate.nn.ReLU(),\n", " brainstate.nn.Linear.desc(out_size=5) # Final output layer\n", ")\n", "\n", "print(\"Sequential MLP:\")\n", "print(mlp)\n", "print(f\"\\nInput size: {mlp.in_size}\")\n", "print(f\"Output size: {mlp.out_size}\")\n", "\n", "# Test forward pass\n", "x = brainstate.random.randn(8, 10) # batch of 8 samples\n", "y = mlp(x)\n", "print(f\"\\nForward pass:\")\n", "print(f\" Input: {x.shape}\")\n", "print(f\" Output: {y.shape}\")" ] }, { "cell_type": "markdown", "id": "cnn_example", "metadata": {}, "source": [ "### Example 2: CNN Network with Automatic Size Propagation\n", "\n", "Let's build a complete CNN for image classification, demonstrating how `in_size` and `out_size` propagate through convolutional, pooling, flattening, and fully-connected layers." ] }, { "cell_type": "code", "execution_count": 12, "id": "cnn_example_code", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:17.445590Z", "start_time": "2025-10-11T08:24:16.065619Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:34.023958Z", "iopub.status.busy": "2026-05-30T16:19:34.023697Z", "iopub.status.idle": "2026-05-30T16:19:36.692104Z", "shell.execute_reply": "2026-05-30T16:19:36.691092Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CNN Network Architecture:\n", "CNNNet(\n", " layer=Sequential(\n", " in_size=(28, 28, 3),\n", " out_size=(10,),\n", " layers=[\n", " Conv2d(\n", " in_size=(28, 28, 3),\n", " out_size=(28, 28, 32),\n", " channel_first=False,\n", " channels_last=True,\n", " in_channels=3,\n", " out_channels=32,\n", " stride=(1, 1),\n", " kernel_size=(3, 3),\n", " lhs_dilation=(1, 1),\n", " rhs_dilation=(1, 1),\n", " groups=1,\n", " dimension_numbers=ConvDimensionNumbers(lhs_spec=(0, 3, 1, 2), rhs_spec=(3, 2, 0, 1), out_spec=(0, 3, 1, 2)),\n", " padding=SAME,\n", " kernel_shape=(3, 3, 3, 32),\n", " w_initializer=XavierNormal(\n", " scale=1.0,\n", " mode='fan_avg',\n", " in_axis=-2,\n", " out_axis=-1,\n", " distribution='truncated_normal',\n", " rng=RandomState(Array((), dtype=key) overlaying:\n", " [1825841970 3512247751]),\n", " unit=Unit(\"1\")\n", " ),\n", " weight=ParamState(\n", " value={\n", " 'weight': ShapedArray(float32[3,3,3,32])\n", " }\n", " )\n", " ),\n", " ReLU(),\n", " MaxPool2d(\n", " in_size=(28, 28, 32),\n", " out_size=(14, 14, 32),\n", " init_value=-inf,\n", " computation=,\n", " pool_dim=2,\n", " return_indices=False,\n", " kernel_size=(2, 2),\n", " stride=(2, 2),\n", " padding=VALID,\n", " channel_axis=-1\n", " ),\n", " Conv2d(\n", " in_size=(14, 14, 32),\n", " out_size=(14, 14, 64),\n", " channel_first=False,\n", " channels_last=True,\n", " in_channels=32,\n", " out_channels=64,\n", " stride=(1, 1),\n", " kernel_size=(3, 3),\n", " lhs_dilation=(1, 1),\n", " rhs_dilation=(1, 1),\n", " groups=1,\n", " dimension_numbers=ConvDimensionNumbers(lhs_spec=(0, 3, 1, 2), rhs_spec=(3, 2, 0, 1), out_spec=(0, 3, 1, 2)),\n", " padding=SAME,\n", " kernel_shape=(3, 3, 32, 64),\n", " w_initializer=XavierNormal(\n", " scale=1.0,\n", " mode='fan_avg',\n", " in_axis=-2,\n", " out_axis=-1,\n", " distribution='truncated_normal',\n", " rng=RandomState(Array((), dtype=key) overlaying:\n", " [1825841970 3512247751]),\n", " unit=Unit(\"1\")\n", " ),\n", " weight=ParamState(\n", " value={\n", " 'weight': ShapedArray(float32[3,3,32,64])\n", " }\n", " )\n", " ),\n", " ReLU(),\n", " MaxPool2d(\n", " in_size=(14, 14, 64),\n", " out_size=(7, 7, 64),\n", " init_value=-inf,\n", " computation=,\n", " pool_dim=2,\n", " return_indices=False,\n", " kernel_size=(2, 2),\n", " stride=(2, 2),\n", " padding=VALID,\n", " channel_axis=-1\n", " ),\n", " Flatten(\n", " in_size=(7, 7, 64),\n", " out_size=(3136,),\n", " start_axis=0,\n", " end_axis=-1\n", " ),\n", " Linear(\n", " in_size=(3136,),\n", " out_size=(1024,),\n", " weight=ParamState(\n", " value={\n", " 'bias': ShapedArray(float32[1024]),\n", " 'weight': ShapedArray(float32[3136,1024])\n", " }\n", " )\n", " ),\n", " ReLU(),\n", " Linear(\n", " in_size=(1024,),\n", " out_size=(512,),\n", " weight=ParamState(\n", " value={\n", " 'bias': ShapedArray(float32[512]),\n", " 'weight': ShapedArray(float32[1024,512])\n", " }\n", " )\n", " ),\n", " ReLU(),\n", " Linear(\n", " in_size=(512,),\n", " out_size=(10,),\n", " weight=ParamState(\n", " value={\n", " 'bias': ShapedArray(float32[10]),\n", " 'weight': ShapedArray(float32[512,10])\n", " }\n", " )\n", " )\n", " ]\n", " )\n", ")\n", "\n", "Network input size: None\n", "Network output size: None\n", "\n", "============================================================\n", "Size transformations through the network:\n", "============================================================\n", "Layer 0 (Conv2d ): (28, 28, 3) -> (28, 28, 32) \n", "Layer 1 (ReLU ): None -> None \n", "Layer 2 (MaxPool2d ): (28, 28, 32) -> (14, 14, 32) \n", "Layer 3 (Conv2d ): (14, 14, 32) -> (14, 14, 64) \n", "Layer 4 (ReLU ): None -> None \n", "Layer 5 (MaxPool2d ): (14, 14, 64) -> (7, 7, 64) \n", "Layer 6 (Flatten ): (7, 7, 64) -> (3136,) \n", "Layer 7 (Linear ): (3136,) -> (1024,) \n", "Layer 8 (ReLU ): None -> None \n", "Layer 9 (Linear ): (1024,) -> (512,) \n", "Layer 10 (ReLU ): None -> None \n", "Layer 11 (Linear ): (512,) -> (10,) \n" ] } ], "source": [ "class CNNNet(brainstate.nn.Module):\n", " \"\"\"Convolutional Neural Network for image classification.\"\"\"\n", " \n", " def __init__(self, in_size):\n", " super().__init__()\n", " self.layer = brainstate.nn.Sequential(\n", " # Convolutional block 1\n", " brainstate.nn.Conv2d(in_size, out_channels=32, kernel_size=(3, 3), \n", " stride=(1, 1), padding='SAME'),\n", " brainstate.nn.ReLU(),\n", " brainstate.nn.MaxPool2d.desc(kernel_size=(2, 2), stride=(2, 2), channel_axis=-1),\n", " \n", " # Convolutional block 2\n", " brainstate.nn.Conv2d.desc(out_channels=64, kernel_size=(3, 3), \n", " stride=(1, 1), padding='SAME'),\n", " brainstate.nn.ReLU(),\n", " brainstate.nn.MaxPool2d.desc(kernel_size=(2, 2), stride=(2, 2), channel_axis=-1),\n", " \n", " # Flatten and fully-connected layers\n", " brainstate.nn.Flatten.desc(),\n", " brainstate.nn.Linear.desc(out_size=1024),\n", " brainstate.nn.ReLU(),\n", " brainstate.nn.Linear.desc(out_size=512),\n", " brainstate.nn.ReLU(),\n", " brainstate.nn.Linear.desc(out_size=10)\n", " )\n", "\n", " def update(self, x):\n", " return self.layer(x)\n", "\n", "# Create CNN with image size (28, 28, 3)\n", "example_image = brainstate.random.normal(size=(28, 28, 3))\n", "cnn = CNNNet(example_image.shape)\n", "\n", "print(\"CNN Network Architecture:\")\n", "print(cnn)\n", "print(f\"\\nNetwork input size: {cnn.in_size}\")\n", "print(f\"Network output size: {cnn.out_size}\")\n", "\n", "# Trace size transformations through the network\n", "print(\"\\n\" + \"=\"*60)\n", "print(\"Size transformations through the network:\")\n", "print(\"=\"*60)\n", "for i, layer in enumerate(cnn.layer.layers):\n", " if hasattr(layer, 'in_size') and hasattr(layer, 'out_size'):\n", " print(f\"Layer {i:2d} ({layer.__class__.__name__:15s}): \"\n", " f\"{str(layer.in_size):20s} -> {str(layer.out_size):20s}\")" ] }, { "cell_type": "markdown", "id": "cnn_forward", "metadata": {}, "source": [ "### Example 3: Forward Pass Through CNN\n", "\n", "Let's actually run a forward pass and see how data flows through the network." ] }, { "cell_type": "code", "execution_count": 13, "id": "cnn_forward_code", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:18.048845Z", "start_time": "2025-10-11T08:24:17.508357Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:36.694288Z", "iopub.status.busy": "2026-05-30T16:19:36.694112Z", "iopub.status.idle": "2026-05-30T16:19:37.410922Z", "shell.execute_reply": "2026-05-30T16:19:37.409866Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Input batch shape: (4, 28, 28, 3)\n", " (batch_size, height, width, channels) = (4, 28, 28, 3)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Output shape: (4, 10)\n", " (batch_size, num_classes) = (4, 10)\n", "\n", "Output logits for first sample:\n", "[ 0.13889295 0.49220052 -0.63533866 0.36826408 -0.5574133 -0.22296715\n", " 1.5445011 0.7295154 0.04205689 -0.02874881]\n" ] } ], "source": [ "# Create a batch of images\n", "batch_size = 4\n", "batch_images = brainstate.random.normal(size=(batch_size, 28, 28, 3))\n", "\n", "print(f\"Input batch shape: {batch_images.shape}\")\n", "print(f\" (batch_size, height, width, channels) = ({batch_size}, 28, 28, 3)\")\n", "\n", "# Forward pass\n", "output = cnn(batch_images)\n", "\n", "print(f\"\\nOutput shape: {output.shape}\")\n", "print(f\" (batch_size, num_classes) = ({batch_size}, 10)\")\n", "print(f\"\\nOutput logits for first sample:\")\n", "print(output[0])" ] }, { "cell_type": "markdown", "id": "sequential_benefits", "metadata": {}, "source": [ "### Benefits of Automatic Size Inference\n", "\n", "The automatic `in_size`/`out_size` inference system provides several key advantages:\n", "\n", "1. **🎯 No manual dimension calculations**: You don't need to compute output sizes after each layer\n", "2. **🔧 Easy architecture modifications**: Change one layer without updating all subsequent layers\n", "3. **🐛 Early error detection**: Shape mismatches are caught at construction time\n", "4. **📊 Built-in documentation**: Network architecture is self-documenting with size information\n", "5. **🚀 Rapid prototyping**: Quickly experiment with different architectures\n", "\n", "### Key Pattern: `.desc()` for Layer Descriptors\n", "\n", "When building networks with `Sequential`, use the `.desc()` pattern for all layers except the first:\n", "\n", "```python\n", "brainstate.nn.Sequential(\n", " FirstLayer(in_size, ...), # Explicit in_size\n", " SecondLayer.desc(...), # in_size inferred\n", " ThirdLayer.desc(...), # in_size inferred\n", " # ...\n", ")\n", "```\n", "\n", "This pattern ensures that:\n", "- The first layer knows the input size\n", "- All subsequent layers automatically infer their input sizes\n", "- The network construction is clean and maintainable" ] }, { "cell_type": "markdown", "id": "advanced_sequential", "metadata": {}, "source": [ "### Example 4: Complex Architecture with Mixed Layer Types" ] }, { "cell_type": "code", "execution_count": 14, "id": "advanced_sequential_code", "metadata": { "ExecuteTime": { "end_time": "2025-10-11T08:24:19.906254Z", "start_time": "2025-10-11T08:24:18.049433Z" }, "execution": { "iopub.execute_input": "2026-05-30T16:19:37.414287Z", "iopub.status.busy": "2026-05-30T16:19:37.413988Z", "iopub.status.idle": "2026-05-30T16:19:40.771855Z", "shell.execute_reply": "2026-05-30T16:19:40.771186Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Complex Network:\n", "Input size: (32, 32, 3)\n", "After features: (8, 8, 64)\n", "After flatten: (4096,)\n", "Final output: (10,)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Forward pass: (2, 32, 32, 3) -> (2, 10)\n" ] } ], "source": [ "# Build a more complex network with different layer types\n", "class ComplexNet(brainstate.nn.Module):\n", " \"\"\"Complex network demonstrating various layer types.\"\"\"\n", " \n", " def __init__(self, in_size):\n", " super().__init__()\n", " \n", " self.features = brainstate.nn.Sequential(\n", " # Initial conv block\n", " brainstate.nn.Conv2d(in_size, out_channels=16, kernel_size=3, padding='SAME'),\n", " brainstate.nn.ReLU(),\n", " \n", " # Strided conv (reduces spatial size)\n", " brainstate.nn.Conv2d.desc(out_channels=32, kernel_size=3, stride=2, padding='SAME'),\n", " brainstate.nn.ReLU(),\n", " \n", " # Another conv + pool\n", " brainstate.nn.Conv2d.desc(out_channels=64, kernel_size=3, padding='SAME'),\n", " brainstate.nn.ReLU(),\n", " brainstate.nn.MaxPool2d.desc(kernel_size=(2, 2), stride=(2, 2), channel_axis=-1),\n", " )\n", " \n", " self.classifier = brainstate.nn.Sequential(\n", " brainstate.nn.Flatten(in_size=self.features.out_size),\n", " brainstate.nn.Linear.desc(out_size=256),\n", " brainstate.nn.ReLU(),\n", " brainstate.nn.Linear.desc(out_size=10),\n", " )\n", " \n", " def update(self, x):\n", " x = self.features(x)\n", " x = self.classifier(x)\n", " return x\n", "\n", "# Create network\n", "net = ComplexNet(in_size=(32, 32, 3))\n", "\n", "print(\"Complex Network:\")\n", "print(f\"Input size: {net.features.in_size}\")\n", "print(f\"After features: {net.features.out_size}\")\n", "print(f\"After flatten: {net.classifier.layers[0].out_size}\")\n", "print(f\"Final output: {net.classifier.out_size}\")\n", "\n", "# Test\n", "x = brainstate.random.randn(2, 32, 32, 3)\n", "y = net(x)\n", "print(f\"\\nForward pass: {x.shape} -> {y.shape}\")" ] } ], "metadata": { "kernelspec": { "display_name": "Ecosystem-py", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.11" } }, "nbformat": 4, "nbformat_minor": 5 }