{ "cells": [ { "cell_type": "markdown", "id": "intro", "metadata": {}, "source": [ "# Basic Neural Network Layers\n", "\n", "BrainState provides a comprehensive set of pre-built layers for building neural networks. This tutorial covers the essential building blocks.\n", "\n", "You will learn about:\n", "\n", "- 📏 **Linear layers** - Fully connected transformations\n", "- 🔲 **Convolutional layers** - Spatial feature extraction (1D, 2D, 3D)\n", "- 🏊 **Pooling layers** - Downsampling operations\n", "- 💧 **Dropout layers** - Regularization techniques\n", "- 🔧 **Utility layers** - Flatten, reshape, and more\n", "\n", "These layers are optimized, well-tested, and ready to use in your models!" ] }, { "cell_type": "code", "execution_count": 1, "id": "imports", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:45.678298Z", "start_time": "2025-10-10T15:36:43.413355Z" } }, "outputs": [], "source": [ "import brainstate\n", "from brainstate import environ\n", "import brainunit as u\n", "import jax.numpy as jnp\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n" ] }, { "cell_type": "markdown", "id": "linear_layers", "metadata": {}, "source": [ "\n", "## 1. Linear (Fully Connected) Layers\n", "\n", "Linear layers perform the transformation: **y = Wx + b**.\n", "\n", "BrainState linear modules expect feature shapes as tuples so they can validate tensor shapes. For example, use `Linear(in_size=(10,), out_size=(5,))`.\n", "\n", "### Basic Usage\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "linear_basic", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:46.738416Z", "start_time": "2025-10-10T15:36:45.693664Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Linear Layer:\n", "Linear(\n", " in_size=(10,),\n", " out_size=(5,),\n", " w_mask=None,\n", " weight=ParamState(\n", " value={\n", " 'bias': ShapedArray(float32[5]),\n", " 'weight': ShapedArray(float32[10,5])\n", " }\n", " )\n", ")\n", "Weight shape: (10, 5)\n", "Bias shape: (5,)\n", "Input shape: (10,)\n", "Output shape: (5,)\n", "Output: [ 0.24681929 1.2860886 -1.6367221 0.29457197 -0.9486235 ]\n" ] } ], "source": [ "\n", "# Create a linear layer\n", "brainstate.random.seed(42)\n", "linear = brainstate.nn.Linear(in_size=(10,), out_size=(5,))\n", "\n", "print(\"Linear Layer:\")\n", "print(linear)\n", "print(f\"Weight shape: {linear.weight.value['weight'].shape}\")\n", "print(f\"Bias shape: {linear.weight.value['bias'].shape}\")\n", "\n", "# Forward pass\n", "x = brainstate.random.randn(10)\n", "y = linear(x)\n", "\n", "print(f\"Input shape: {x.shape}\")\n", "print(f\"Output shape: {y.shape}\")\n", "print(f\"Output: {y}\")\n" ] }, { "cell_type": "markdown", "id": "linear_batch", "metadata": {}, "source": [ "### Batch Processing\n", "\n", "Linear layers automatically handle batched inputs:" ] }, { "cell_type": "code", "execution_count": 3, "id": "linear_batched", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:49.447570Z", "start_time": "2025-10-10T15:36:49.071992Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Batched input shape: (32, 10)\n", "Batched output shape: (32, 5)\n", "\n", "Multi-batch input: (8, 4, 10)\n", "Multi-batch output: (8, 4, 5)\n" ] } ], "source": [ "# Batched input: (batch_size, features)\n", "x_batch = brainstate.random.randn(32, 10) # 32 samples, 10 features each\n", "y_batch = linear(x_batch)\n", "\n", "print(f\"Batched input shape: {x_batch.shape}\")\n", "print(f\"Batched output shape: {y_batch.shape}\")\n", "\n", "# Works with arbitrary batch dimensions\n", "x_multi = brainstate.random.randn(8, 4, 10) # (batch1, batch2, features)\n", "y_multi = linear(x_multi)\n", "\n", "print(f\"\\nMulti-batch input: {x_multi.shape}\")\n", "print(f\"Multi-batch output: {y_multi.shape}\")" ] }, { "cell_type": "markdown", "id": "linear_variants", "metadata": {}, "source": [ "\n", "### Linear Layer Variants\n", "\n", "BrainState provides specialized linear layers:\n", "\n", "- `ScaledWSLinear` applies weight standardization for stable training.\n", "- `LoRA` adds low-rank adapters to an existing projection.\n", "- `SparseLinear` consumes a `brainunit.sparse` matrix so you can encode custom connectivity patterns.\n" ] }, { "cell_type": "code", "execution_count": 4, "id": "linear_sparse", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:49.995627Z", "start_time": "2025-10-10T15:36:49.454578Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sparse Linear Layer:\n", "SparseLinear(\n", " in_size=(100,),\n", " out_size=(50,),\n", " spar_mat=CSR(float32[100, 50], nse=485),\n", " weight=ParamState(\n", " value={\n", " 'weight': ShapedArray(float32[485])\n", " }\n", " )\n", ")\n", "Input: (100,) → Output: (50,)\n" ] } ], "source": [ "\n", "# SparseLinear: For sparse connectivity\n", "brainstate.random.seed(0)\n", "dense_weight = brainstate.random.rand(100, 50)\n", "sparsity_mask = brainstate.random.rand(100, 50) < 0.1\n", "sparse_matrix = u.sparse.CSR.fromdense(dense_weight * sparsity_mask)\n", "sparse_linear = brainstate.nn.SparseLinear(sparse_matrix, in_size=(100,))\n", "\n", "print(\"Sparse Linear Layer:\")\n", "print(sparse_linear)\n", "\n", "x = brainstate.random.rand(100)\n", "y = sparse_linear(x)\n", "print(f\"Input: {x.shape} → Output: {y.shape}\")\n" ] }, { "cell_type": "markdown", "id": "conv_layers", "metadata": {}, "source": [ "\n", "## 2. Convolutional Layers\n", "\n", "Convolutional layers extract spatial features using learnable filters. Supply the expected input shape (without the batch dimension) via `in_size` so each layer can initialize weights and validate its inputs.\n", "\n", "### Conv1d - 1D Convolution\n", "\n", "Used for sequential data (time series, audio, text):\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "conv1d", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:50.397808Z", "start_time": "2025-10-10T15:36:50.002634Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Conv1d Layer:\n", "Conv1d(\n", " in_size=(100, 3),\n", " out_size=(100, 16),\n", " channel_first=False,\n", " channels_last=True,\n", " in_channels=3,\n", " out_channels=16,\n", " stride=(1,),\n", " kernel_size=(3,),\n", " lhs_dilation=(1,),\n", " rhs_dilation=(1,),\n", " groups=1,\n", " dimension_numbers=ConvDimensionNumbers(lhs_spec=(0, 2, 1), rhs_spec=(2, 1, 0), out_spec=(0, 2, 1)),\n", " padding=SAME,\n", " kernel_shape=(3, 3, 16),\n", " w_mask=None,\n", " w_initializer=XavierNormal(\n", " scale=1.0,\n", " mode='fan_avg',\n", " in_axis=-2,\n", " out_axis=-1,\n", " distribution='truncated_normal',\n", " rng=RandomState([1797259609 2579123966]),\n", " unit=Unit(10.0^0)\n", " ),\n", " b_initializer=None,\n", " weight=ParamState(\n", " value={\n", " 'weight': ShapedArray(float32[3,3,16])\n", " }\n", " )\n", ")\n", "Input shape: (4, 100, 3)\n", "Output shape: (4, 100, 16)\n", "Kernel shape: (3, 3, 16)\n" ] } ], "source": [ "\n", "# Conv1d: (batch, length, in_channels) → (batch, length, out_channels)\n", "brainstate.random.seed(0)\n", "conv1d = brainstate.nn.Conv1d(\n", " in_size=(100, 3),\n", " out_channels=16,\n", " kernel_size=3,\n", " padding='SAME'\n", ")\n", "\n", "print(\"Conv1d Layer:\")\n", "print(conv1d)\n", "\n", "# Input: (batch=4, length=100, channels=3)\n", "x = brainstate.random.randn(4, 100, 3)\n", "y = conv1d(x)\n", "\n", "print(f\"Input shape: {x.shape}\")\n", "print(f\"Output shape: {y.shape}\")\n", "print(f\"Kernel shape: {conv1d.weight.value['weight'].shape}\")\n" ] }, { "cell_type": "markdown", "id": "conv2d", "metadata": {}, "source": [ "### Conv2d - 2D Convolution\n", "\n", "The workhorse for image processing:" ] }, { "cell_type": "code", "execution_count": 6, "id": "conv2d_basic", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:50.831519Z", "start_time": "2025-10-10T15:36:50.404817Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Conv2d Layer:\n", "Conv2d(\n", " in_size=(28, 28, 3),\n", " out_size=(28, 28, 32),\n", " channel_first=False,\n", " channels_last=True,\n", " in_channels=3,\n", " out_channels=32,\n", " stride=(1, 1),\n", " kernel_size=(3, 3),\n", " lhs_dilation=(1, 1),\n", " rhs_dilation=(1, 1),\n", " groups=1,\n", " dimension_numbers=ConvDimensionNumbers(lhs_spec=(0, 3, 1, 2), rhs_spec=(3, 2, 0, 1), out_spec=(0, 3, 1, 2)),\n", " padding=SAME,\n", " kernel_shape=(3, 3, 3, 32),\n", " w_mask=None,\n", " w_initializer=XavierNormal(\n", " scale=1.0,\n", " mode='fan_avg',\n", " in_axis=-2,\n", " out_axis=-1,\n", " distribution='truncated_normal',\n", " rng=RandomState([ 683029726 1624662641]),\n", " unit=Unit(10.0^0)\n", " ),\n", " b_initializer=None,\n", " weight=ParamState(\n", " value={\n", " 'weight': ShapedArray(float32[3,3,3,32])\n", " }\n", " )\n", ")\n", "Input shape: (8, 28, 28, 3)\n", "Output shape: (8, 28, 28, 32)\n", "Kernel shape: (3, 3, 3, 32)\n" ] } ], "source": [ "\n", "# Conv2d: (batch, height, width, in_channels) → (batch, height, width, out_channels)\n", "conv2d = brainstate.nn.Conv2d(\n", " in_size=(28, 28, 3), # (height, width, channels)\n", " out_channels=32, # 32 feature maps\n", " kernel_size=(3, 3), # 3x3 kernel\n", " stride=(1, 1), # Stride of 1\n", " padding='SAME'\n", ")\n", "\n", "print(\"Conv2d Layer:\")\n", "print(conv2d)\n", "\n", "# Input: (batch=8, height=28, width=28, channels=3)\n", "x_image = brainstate.random.randn(8, 28, 28, 3)\n", "y_image = conv2d(x_image)\n", "\n", "print(f\"Input shape: {x_image.shape}\")\n", "print(f\"Output shape: {y_image.shape}\")\n", "print(f\"Kernel shape: {conv2d.weight.value['weight'].shape}\")\n" ] }, { "cell_type": "markdown", "id": "conv_features", "metadata": {}, "source": [ "### Visualizing Convolutional Features" ] }, { "cell_type": "code", "execution_count": 7, "id": "visualize_conv", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:51.500349Z", "start_time": "2025-10-10T15:36:50.841525Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABb4AAAExCAYAAACzsrRmAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjYsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvq6yFwwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAI+RJREFUeJzt3QuYVWW9P/A1MDAgDiiIl1DxLnhF7IKmaVZqaZ6s9JRPimlxKu1id/OUplmW18xullmnzMqs5KgdO500L6RJmaaWpoGKiiKCIneY9X9+65yZZobBd9N/mNn7nc/neUZk85v1vnvzrB9rvmutdzWVZVkWAAAAAACQiUH9PQEAAAAAAOhNgm8AAAAAALIi+AYAAAAAICuCbwAAAAAAsiL4BgAAAAAgK4JvAAAAAACyIvgGAAAAACArgm8AAAAAALIi+AYAAAAAICuCbwAAAAAAsiL47mPf/e53i6ampmLmzJlFPViyZElxxhlnFDfddFNN9VEX8//pT3+63ucGrJ/+09PXJz/5yfUy5owZM6oes3DhwqKeP49bb711jT8vy7LYaqutqj8//PDDi/5y2WWXFRMnTiyGDRtW7LjjjsVXvvKVfpsLrE96VOP1qK9//evFUUcdVWy99dbVPI4//vh+mQf0BT2qsXrUY489Vnz2s58tXv7ylxcbb7xxsckmmxQHHnhg8etf/7rP5wJ9QY9qrB61dOnS4sQTTyx22223YtSoUcWGG25Y7LnnnsWXv/zlYuXKlX0+n5w19/cE6P/gOw4IQhwIAPk788wzi2233bbLa/EP7vo6GIoeE2HIRhttVNSjCJR/+MMfFvvtt1+X13/7298Wc+bMKVpaWvptbt/85jeL97znPcVb3vKW4sMf/nBxyy23FB/4wAeq3v2JT3yi3+YF65Me1Tg96otf/GKxaNGiKlh68skn+20e0Jf0qMboUddcc03Vo970pjcVU6dOLVatWlX8x3/8R/G6172u+M53vlO8853v7Jd5wfqmRzVGj4rg+7777ive8IY3FNtss00xaNCg6vM85ZRTijvuuKOaM71D8A0wwLz+9a8vXvrSlxaNbPHixcWIESN6ZVtxsHHVVVcVF198cdHc/I9/FuNgY++99y6eeeaZor8Ohk477bTisMMO67jL5t3vfnfR1tZWnHXWWcW0adOqK5ggN3pUY/So9h8a26/2jiuVYCDQoxqjR7361a8uHn300epK73ZxMcGkSZOKz3zmM4JvsqVHNUaPGj16dHH77bd3eS16VFz9fckllxQXXHBBsfnmm/fL3HJjqZM6EGfH4oeFxx9/vDojHf8/duzY4qMf/WixevXqjrrZs2dXP1icd955xYUXXliMHz++GD58eHHAAQcU9957b5dtxtXbPV3BHWPF2aT27cU4Ic7Std8GEreqrIuoj+978MEHi3e84x3Vjhrb/fSnP13dPhK3mf3Lv/xLMXLkyGrHPf/887t8/4oVK6qDj2g68b3R4Pbff//ixhtvXGOs+fPnF8cee2y1rTijGGfv77777mr8uJWls7/+9a/FW9/61qqhxFm+aP7Tp09fp/cGA9Evf/nLah+MfbG1tbUKXuNsdGf33HNP1U+22267av+KffuEE06o9tHOveFjH/tY9f9x1UF7j4ne097Puu+3oXsfau8x999/f3HMMcdUYW/nM/Y/+MEPqv4R/TD297e97W1V36nV29/+9mre//3f/92lL0XYHOP1JPrwvvvuW4wZM6YaN8bvaQmomPfJJ59cXHHFFcXOO+9cfVZRe/PNNyfnFT0w5vW+972vy+snnXRSdTB43XXX1fweISd6VH30qBDHorEN4B/0qProUbvuumuX0DvElZ0RgsVVnnG3CgxEelR99Ki1ac/r6nH5mEYl+K4TEXAfcsgh1Y4VO1mE2REQX3rppWvUxi1acbYqwo9TTz21Cr0POuig4qmnnlqnMSOcjrUZw5FHHll8//vfr77e/OY3/1Pv4V//9V+rKxHPOeec4hWveEXxuc99rrjooouq28nGjRtX3Wq2ww47VIF+50bw/PPPF9/+9reroD5qovHNmzev+jz+9Kc/ddTFtt/4xjcWV155ZRV4n3322dVttfH/3UXjnjJlSvGXv/ylWs8qPsto7HFi4ec///k/9f4gF88991x1ZrvzV7voAXHwEyfgYn+ME1hxEBIHH3EA0y4OHP7+979XV8vEmtNxAPKjH/2o+mEiTniF6CVxoBHiZF17j2k/4bauYh3ZWOLj85//fHXlc4g+cNxxx1VrX8dZ8Q996EPF//zP/xSvetWraj5YiIOLffbZp+otnQ8I43OK99WTWHttr732qm4ljPnE1QMxv57C6LgiMuYVJwajPg68Dj300DVOWHZ31113Vb92v2IjDqbiVrj2P4fc6FGN0aNgoNKjGrtHzZ07t9hggw2qL8iRHtVYPSpC+Pg7ijA/sqrIA+PCgsjO6CUlferyyy+PLlHeeeedHa9NnTq1eu3MM8/sUrvXXnuVe++9d8fvZ82aVdUNHz68nDNnTsfrd9xxR/X6Kaec0vHaAQccUH11F2ONHz++4/fz5s2rvvf000+vaf433nhjVX/VVVd1vBbfG69Nmzat47VVq1aVW265ZdnU1FSec845Ha8vWLCgmn/Mo3Pt8uXLu4wTdZtttll5wgkndLx29dVXV+NcdNFFHa+tXr26POigg6rX47Nt95rXvKbcfffdy2XLlnW81tbWVu67777ljjvuWNN7hVz7T09fYdGiReVGG21Uvvvd7+7yfXPnzi1HjRrV5fUlS5assf0rr7yy2tbNN9/c8dq5555bvRb9q7P2ftZ5v23XvSe195i3v/3tXepmz55dDh48uDz77LO7vP7nP/+5bG5uXuP1F+vHl1xySdna2trxvo466qjy1a9+dfX/0TMPO+ywLt/b/f2vWLGi3G233ap+1P29xNfMmTM7XnvkkUfKYcOGlUceeeSLzu+kk06q3l9Pxo4dW77tbW970e+HRqNHNVaP6m7EiBFdju8gN3pUY/eo8Le//a363mOPPXadvxfqnR7VmD2q/XNt/3rpS19a3nPPPTV9L7VxxXcdifV8OovbT+IsW3dx1XJcQd0uHigUV1hff/31RX9617ve1fH/gwcPrq5SjF4QT6ptF8uTxC0gnd9X1A4dOrTjqu5nn322evhIfP8f//jHjrr/+q//KoYMGdJx9i/EVY9x5Xtn8f2/+c1viqOPPrq6ha39LGeceYuryP/2t79Vy8rAQPXVr361Oovf+SvEr3HmPM7cd75CIPbR6DGdlx+KW77aLVu2rKqLuyxC5/12ffbIn/3sZ1XPiH2983zjVry4KqCn5ZLWJrYRa2pfe+21Vd+IX9d261v3979gwYLqioHo2T2997jCIK7Sbhfr4cbyTzfccEOX5ay6i/m098bu4ja6+HPIkR7VGD0KBio9qjF7VFxJGldsxthxhzLkSo9qrB4VzyOIv5tYhzw+g8i8YllLeo+HW9aJCDG63xISaxvFTtZd7OTd7bTTTsVPfvKToj/FDt5ZrNcd76v72mrxeue1ocL3vve9ajmSWJd75cqVHa93fhrxI488UmyxxRZr3JbW/RaQhx56qArc47ad+OrJ008/3eXkAQwkcbKspweexEmhEEsn9STW1u98gimeDRC3vMX+1FkcGKwP3Z9OHvONfb2nnhjioKFW0X9f+9rXVg85iR+M4iAlnhGwNnGwFMs5xXJMy5cv73i9p7Vu19azY5xY1mltDy2JA6649a0ncQDa+YAMcqJHNUaPgoFKj2q8HhXziSUNYkmHWOLgJS95Sc3vDRqNHtVYPWqzzTarvkLMKZZWieWC4/07Busdgu86EWfZelPskO1rL3W2Pq/c6ek9rO19dZ5bPKwgHpwQV7LHwxE23XTT6vu+8IUvFA8//PA6zyPOCoZYSzyu8O6J9ZJg7ftOrM3W0z+ynZ+CHWfNZ8yYUe2zkyZNqtaJi++P9czat/Ni1vYwtBfrUd2D3hgnthM/wPTUa2JO6yLO+scdJbH2YzwNPe5Q6cktt9xSHHHEEdXacl/72teqE3Jx4HX55ZdXB1O9JbYbn0ccbEZfbBdheJw89EMbA40eVV89CuhKj6rfHhXziiArHkC3ttAPcqdH1W+P6izC79NOO6245pprin/7t39b7+MNBILvBtR+pq6zBx98sOPpr+1Xi/e0TEpcNV1LQ+pL8XTceFpw3MrSeT6nn356l7pY4D9uZ4kzZ52v+o4rvDuLbYVoTnFWD6jN9ttvX/0aIeuL7TtxJ0o8VCSuAvjMZz7zor1pbT0melTo/lCS7j0qNd84iRZXB8RZ9f9f8ZDfOLi4/fbbix//+Mdrrbv66quru1ni9rWWlpaO1+NgaF16dvSxF3v4SxxkhpkzZ1YPkmkXv48DwfY/h4FCj6qvHgV0pUfVZ4+K4C62fdFFF3U8iA8GIj2qPntUd+3LWa6vK+sHImt8N6Bf/OIXXdao/v3vf1/ccccd1Vmrzk0ilg2JWyva3X333cVtt93WZVvtAXKtT8RdH9rP3nW+Cjzez+9+97sudXH1diyD8q1vfavjtQh/Yg2rzqKRH3jggcU3v/nN4sknn1xjvM6fCdB1H4tb3OL2qs5LDnXfd3raZ0P8QNHdiBEjeuwxMU4sg3TzzTd3eT3OqtcqniQec4mDsu5zid93X1IpJa4a+PrXv16cccYZxRvf+Ma11sWYcZDX+YqFeAp69OaeRC/rvB5cPLE7zuAffPDBL3q3T1yRNHr06GpOncXvo3fHE9lhINGj6qtHAV3pUfXXo84999zivPPOKz71qU8VH/zgB9fp/UBu9Kj66lGxXnlPqzR8+9vfrn7tabka/jmu+G5AsUzHfvvtV7z3ve+t1huKBjRmzJji4x//eEfNCSecUFxwwQVVc4uHS8at8t/4xjeKXXfdtXj++ee73E6yyy67VGe84ixahCy77bZb9dVXDj/88Opq7zgDF0HOrFmzqrnGvF544YWOulgKJdar+shHPlJd5T1hwoRi+vTp1fpT3c82Rhgen9Huu+9e3c4SV4E/9dRTVVOaM2dOdRIAKNY4QImDgWOPPbaYPHlytRZinKV+9NFHi+uuu6545StfWVxyySVVXdz69aUvfak6aIr18n/1q19V+2537Q/6iNu1YntxJ0YcaMRBUjwQNx4uFL/GP+xxYBRnx2sVJ/hi7bVTTz21OhiJHtHa2lrN4+c//3kxbdq0asmjdTF16tRkTfSp6K9xq1/cMhf9NXpO9OZ77rlnjfrop9GLP/CBD1RXDbQf8MVB3IuJ/nzWWWdVD/CNhzHFNuLWu1ge6uyzz676NQwkelR99ajwn//5nx3HVPFZx/bjPYe4TXiPPfZYp/cHjUyPqq8eFe8hfj6O9XcnTpxYHT91Fmvotq+rCwOBHlVfPSp6UuRe8b4ir4qHbsZV5vGgy/gMLcvUi0r61OWXXx6ndMo777yz47WpU6eWI0aMWKP29NNPr2rbzZo1q/r9ueeeW55//vnlVlttVba0tJT7779/effdd6/x/T/4wQ/K7bbbrhw6dGg5adKk8oYbbqjGGj9+fJe6GTNmlHvvvXdVF9uPcdfmxhtvrGquuuqqNeY5b968LrVre18HHHBAueuuu3b8vq2trfz85z9fzSvez1577VVee+21Pc41xjjmmGPK1tbWctSoUeXxxx9f3nbbbdX4P/rRj7rUPvzww+Vxxx1Xbr755uWQIUPKcePGlYcffnj505/+dK3vDwZa/1nbfn7IIYdU+9iwYcPK7bffvtrXZs6c2VEzZ86c8sgjjyw32mijqu6oo44qn3jiiR57yFlnnVXtf4MGDar+PHpZWLJkSXniiSdW3x/79NFHH10+/fTTa2xjbT2m3dVXX13ut99+Vb+JrwkTJpQnnXRS+cADD/TK5xF96LDDDuvy2mWXXVbuuOOOVc+K8WJb3Xt2iN/HXKIft9dHj4vPuFaXXnppufPOO1c9Ov4uLrzwwqpvQm70qMbrUXGsFtvo6SvGhJzoUY3Vo9q3t7avdTkWg0agRzVWj4p5xee69dZbV98X723y5MnlBRdcUK5cuTL5/dSuKf7Tm0E660+c5Yr1jeKWrXU9s5WzuOUkrha/9dZbq7OUAPUi7kSJK7bj6gmAeqNHAfVMjwLqmR7VGKzxTUNpX+i/Xay79JWvfKW6HSdu1wEAAAAAsMY3DeX9739/FX7vs88+1frmsTb4jBkzqgc0xHq4AAAAAACCbxpKLPB//vnnF9dee22xbNmy6gEDccX3ySef3N9TAwAAAADqhDW+AQAAAADIijW+AQAAAADIiuAbAAAAAICsCL4BAAAAABiYD7dsampavzMBGlK9PCZg/KXn9vcUgDr0yLSPFfVg949c2N9TAOrQn88/pagHH/nT0f09BaAOnT/pJ0U9+NpfD+zvKQB16H0TbkrWuOIbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsNPf3BACoT80LBydrNpiwMFmzbPmQmsbbZNQLyZonH9g0WVO2tNU0HtDYFu2wOlkz4eL5yZq2DVpqGq+8675kzZxT903WNC+raTigwV0/a5dkzbJHW5M1I7Z9rqbxjtjm3mTNW0fNTNZcsWBKTeMBje1LMw9J1mx497BkzaKdVtY03vFTbkvWnD72/mTNNxaOq2k8aOeKbwAAAAAAsiL4BgAAAAAgK4JvAAAAAACyIvgGAAAAACArgm8AAAAAALIi+AYAAAAAICuCbwAAAAAAsiL4BgAAAAAgK839PQFeXFmWfTpeU1NTn44H1K8hi9L9oLxl42TNoCmLahpvfOuCZM1e+z6erLnuD3vWNB7Q2I6Y8odkzTWDJydrRo9bWNN4Iy57ebJm+Zi2ZE3z4647gYFgybwRyZox96ePteYPb61pvLbx6W1NamlJ1lxR02hAoxs1akmyZvDz6Z4xdF5tseIvZu+RrHnv6Dtr2hasC0feAAAAAABkRfANAAAAAEBWBN8AAAAAAGRF8A0AAAAAQFYE3wAAAAAAZEXwDQAAAABAVgTfAAAAAABkRfANAAAAAEBWmvt7AgDUp+al6Zotpz+ZrHlozOY1jXdXsWWy5pLJP0zWXFfsWdN4QGNbunpIsmbbn7Ula+ZOG1rTeBuuStccedAdyZpffX+fmsYDGtvQ+YOTNWP+vDhZM/KR2nrUlcWUZM0W+z9X07aA/C1aPCxZs9OtzyRrNvlTejthzgujkzW37zK2pm3BunDFNwAAAAAAWRF8AwAAAACQFcE3AAAAAABZEXwDAAAAAJAVwTcAAAAAAFkRfAMAAAAAkBXBNwAAAAAAWRF8AwAAAACQFcE3AAAAAABZEXwDAAAAAJAVwTcAAAAAAFkRfAMAAAAAkBXBNwAAAAAAWRF8AwAAAACQFcE3AAAAAABZEXwDAAAAAJAVwTcAAAAAAFkRfAMAAAAAkBXBNwAAAAAAWRF8AwAAAACQFcE3AAAAAABZEXwDAAAAAJAVwTcAAAAAAFkRfAMAAAAAkBXBNwAAAAAAWRF8AwAAAACQFcE3AAAAAABZEXwDAAAAAJCV5v6eAEDOmhcOTtZsMGFhsmbZ8iFFX3th2+HJmqbFS3ttvGXPtSRrXjN8dbJm6MbLemlGA8Mmo15I1jz5wKbJmrKlrZdmRF9atEN6n5pw8fxkTdsG6f23t/1u0F7Jmo2Gr+q18RbsnD5svvp3L0/W7HTTc700o4GhvOu+ZM2cU/dN1jT7p6EhXT9rl2TNskdbkzUjtu37/a5MHwIWC3cakawZMXdlTeONfCDdoy4YdHCypnXT9HEB/3DENvcma946amay5ooFU3ppRvSlL808JFmz4d3DkjWLdqptP+9VzWWyZOGeY5I1Ix9eXNNwW96Q7sOfbD4+WbN0q374rBrY8VNuS9acPvb+ZM03Fo4rGpUrvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArDQXA0BZlv09hYbhs+ofTU1N/T0F1pMhi9J/t+UtGydrBk1ZVPS1po1WJGvmHrFtr4036Pn0P0nvmH1gejuD9LF1Mb51QbJmr30fT9Zc94c9e2lG9KUjpvwhWXPN4MnJmtHjFhZ9rVyVHnPJc6N6bbym/dP7yk2Tv5WsOXLrE3ppRgPDiMtenqxZPqYtWdP8uOt9GtGSeSOSNWPuTx9rzR/eWvS1ITUcjiwbk577iLm1jbfBU+n9oCiGJCueb+v7z6qRtY1P/x1OamlJ1lzRS/Ohb40atSRZM/j59N//0HnNdflzavPSVb023oqxw5M1Q2r4kXd1P3xWjewXs/dI1rx39J1FzhwBAgAAAACQFcE3AAAAAABZEXwDAAAAAJAVwTcAAAAAAFkRfAMAAAAAkBXBNwAAAAAAWRF8AwAAAACQFcE3AAAAAABZae7vCQDkrHlpumbL6U8max4as3nR1wbXULNofC+Ot6wpWfP7myb23oBU7iq2TNZcMvmHyZrrij17aUb0paWrhyRrtv1ZW7Jm7rShRT1acuiiXtvWylXprnjo79/Ta+PxvzZcla458qA7kjW/+v4+vTMh+tTQ+en9bsyfFydrRj7SHz0q3Tt70/Bn0jvL8GfS2xlzX+/MZ6C4spiSrNli/+f6ZC70vUWLhyVrdro1veNt8qf0dhrd0AXLkzVbzEjXsG7mvDA6WXP7LmOLnLniGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsNPf3BABytmJUumb1Q7OSNS3PblH0tZaFZbJm4YR0zaAVTTWNt2pUW7Jm1F8Gp7czrKbh+D/LZm2YrBn7ssV9Mhf63m9n75CsWX3QkGTNabtNL/ra8SOfTtbscOM7kzVDW1bWNN7Kh1qTNRe99fJkzbxVI2saj/91znGHJGueXTGiT+ZC3xu886Jkzey29L45/OnajkV605j7lidrhj3wZLJm6a4vqWm8+bsMTda8sHX6WKt1lmvj1knbqmTJ9Ll7JGsmbTynlyZEX/r3ydcna8785OHJmiGPtRR9batfp3vU4Jv+mKxpetnuNY336MHpXt22Z7rnlw+mf3bhHwalW1TxnSf2T9YcOvbeolH5Vw0AAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyEpzMQA0NTUVjaosyz4dr5E/K6hHK0e2JWuWvPkVyZqm9GZ63bAF6UG3/E265on9avynpnVlsmTpq5Yka4be3lrbeFSGLkz3/Xfdd2yfzIW+N/zm9P6yxRV/SdZc/Mhbir7226n3JmvanmlJb2hcuveELW9alaz58Op3JmtGzKlpOP7PsCHpHvXXN2zaJ3Oh7223yfxkzd8nprezdFA/HBvcny5Z9fgT6aJdX1LTcEs3Tf/cuOnEecmaZ5fan9ZJDT8+P75gVLJm0sb+cWhE9y0Zl6zZoHV5smbx2MFFX5u/67BkzeZ/SPfOWhOrloXpmmGt6Z/15o5Nz5t/aK7hZ+wxLYuLnLniGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADISnN/TwAgZ23D25I1cw6uZUurinr8J2LL6U+nN7Pf5jWN1jJ8ZbLmksk/TNa869kTaxqP2i15bOP+ngLrSduQdM3jx08s6tHQQem+uM116Zq502obr625KVlzxOtvT9b86vv71DYgNXvhhtr+naHxTBw5t1dqiu2KPje9mJKs2WbxHr023srRq5M1x2w9M1nz6Gaje2lGkL/th6V/Fjp5Qg0/L00o+tx5ra9L1mzxm/S/r2WN461sTdd8dodfJmueGO/nEtaNK74BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyEpzf08AgPq0YlS6ZvVDs5I1Lc9uUdN4y2ZtmKwZ+7LFNW0LyN9vZ++QrFl90JBkzWm7Ta9pvHOOOyRZ8+yKETVtC8jf4J0XJWtmt7Uma4Y/3VTbgG2rkiXT5+6RrJm08ZzaxgMa2r9Pvj5Zc+YnD0/WDHmspabxBqVbVPGdJ/ZP1hw69t6axoN2rvgGAAAAACArgm8AAAAAALIi+AYAAAAAICuCbwAAAAAAsiL4BgAAAAAgK4JvAAAAAACyIvgGAAAAACArgm8AAAAAALLS3N8T4MU1NTX19xSAAWrlyLZkzZI3vyJZ05TeTGXownS/e9d9x9a2MSB7w29uTdZsccVfkjUXP/KWmsYbNiTdo/76hk1r2haQv+02mZ+s+fvE9HaWDkr3ukoNPzY+vmBUsmbSxnNqGw9oaPctGZes2aB1ebJm8djBNY3X3LoyWTOmZXFN24J14YpvAAAAAACyIvgGAAAAACArgm8AAAAAALIi+AYAAAAAICuCbwAAAAAAsiL4BgAAAAAgK4JvAAAAAACyIvgGAAAAACArzf09AQDqU9vwtmTNnINr2dKqorcseWzjXtsW0NjahqRrHj9+YtGXXrhh8z4dD6hfE0fO7ZWaYrvemQ9AZ9sPezpZc/KEdE0xoXfmA+uLK74BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMhKU1mWZX9PAgAAAAAAeosrvgEAAAAAyIrgGwAAAACArAi+AQAAAADIiuAbAAAAAICsCL4BAAAAAMiK4BsAAAAAgKwIvgEAAAAAyIrgGwAAAACArAi+AQAAAAAocvL/AE48A+/2zahiAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "\n", "# Create a simple image with a pattern\n", "def create_test_image():\n", " img = jnp.zeros((28, 28, 1))\n", " # Add vertical edge\n", " img = img.at[5:23, 10:13, 0].set(1.0)\n", " # Add horizontal edge\n", " img = img.at[10:13, 5:23, 0].set(1.0)\n", " return img\n", "\n", "# Create and apply conv\n", "brainstate.random.seed(42)\n", "edge_conv = brainstate.nn.Conv2d(\n", " in_size=(28, 28, 1),\n", " out_channels=4,\n", " kernel_size=(3, 3),\n", " padding='SAME'\n", ")\n", "\n", "test_img = create_test_image()\n", "features = edge_conv(test_img[None, ...])[0] # Add/remove batch dim\n", "\n", "# Visualize\n", "fig, axes = plt.subplots(1, 5, figsize=(15, 3))\n", "\n", "# Original image\n", "axes[0].imshow(test_img[:, :, 0], cmap='gray')\n", "axes[0].set_title('Input Image')\n", "axes[0].axis('off')\n", "\n", "# Feature maps\n", "for i in range(4):\n", " axes[i+1].imshow(np.array(features[:, :, i]), cmap='viridis')\n", " axes[i+1].set_title(f'Feature Map {i}')\n", " axes[i+1].axis('off')\n", "\n", "plt.tight_layout()\n", "plt.show()\n" ] }, { "cell_type": "markdown", "id": "conv_params", "metadata": {}, "source": [ "\n", "### Convolution Parameters\n", "\n", "Understanding stride and padding (`stride=...`, `padding=...`):\n", "\n", "- `stride` controls the step size; pass it with the `stride` keyword (the API no longer accepts `strides`).\n", "- `'SAME'` padding preserves spatial dimensions when the stride is 1; `'VALID'` performs no padding.\n", "- Explicit tuples let you set different values per spatial dimension.\n" ] }, { "cell_type": "code", "execution_count": 8, "id": "conv_stride", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:51.984196Z", "start_time": "2025-10-10T15:36:51.510362Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Input shape: (1, 28, 28, 3)\n", "Stride 1, SAME : (1, 28, 28, 16)\n", "Stride 2, SAME : (1, 14, 14, 16)\n", "Stride 1, VALID : (1, 26, 26, 16)\n" ] } ], "source": [ "\n", "# Different stride values\n", "x = brainstate.random.randn(1, 28, 28, 3)\n", "\n", "configs = [\n", " {'stride': (1, 1), 'padding': 'SAME', 'name': 'Stride 1, SAME'},\n", " {'stride': (2, 2), 'padding': 'SAME', 'name': 'Stride 2, SAME'},\n", " {'stride': (1, 1), 'padding': 'VALID', 'name': 'Stride 1, VALID'},\n", "]\n", "\n", "print(f\"Input shape: {x.shape}\")\n", "\n", "for config in configs:\n", " brainstate.random.seed(0)\n", " conv = brainstate.nn.Conv2d(\n", " in_size=(28, 28, 3),\n", " out_channels=16,\n", " kernel_size=(3, 3),\n", " stride=config['stride'],\n", " padding=config['padding']\n", " )\n", " y = conv(x)\n", " print(f\"{config['name']:20s}: {y.shape}\")\n" ] }, { "cell_type": "markdown", "id": "conv3d", "metadata": {}, "source": [ "### Conv3d - 3D Convolution\n", "\n", "For video or volumetric data:" ] }, { "cell_type": "code", "execution_count": 9, "id": "conv3d_example", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:52.413200Z", "start_time": "2025-10-10T15:36:51.992203Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Video input shape: (2, 16, 64, 64, 3)\n", "Video output shape: (2, 16, 64, 64, 16)\n" ] } ], "source": [ "\n", "# Conv3d: (batch, depth, height, width, channels)\n", "conv3d = brainstate.nn.Conv3d(\n", " in_size=(16, 64, 64, 3),\n", " out_channels=16,\n", " kernel_size=(3, 3, 3),\n", " padding='SAME'\n", ")\n", "\n", "# Video input: (batch=2, frames=16, height=64, width=64, channels=3)\n", "x_video = brainstate.random.randn(2, 16, 64, 64, 3)\n", "y_video = conv3d(x_video)\n", "\n", "print(f\"Video input shape: {x_video.shape}\")\n", "print(f\"Video output shape: {y_video.shape}\")\n" ] }, { "cell_type": "markdown", "id": "pooling", "metadata": {}, "source": [ "## 3. Pooling Layers\n", "\n", "Pooling layers downsample feature maps, reducing spatial dimensions.\n", "\n", "### Max Pooling" ] }, { "cell_type": "code", "execution_count": 10, "id": "maxpool", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:52.588486Z", "start_time": "2025-10-10T15:36:52.443720Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MaxPool2d Layer:\n", "MaxPool2d(\n", " init_value=-inf,\n", " computation=,\n", " pool_dim=2,\n", " return_indices=False,\n", " kernel_size=(2, 2),\n", " stride=(2, 2),\n", " padding=VALID,\n", " channel_axis=-1\n", ")\n", "Input shape: (4, 28, 28, 16)\n", "Output shape: (4, 14, 14, 16)\n", "Spatial reduction: 28x28 → 14x14\n" ] } ], "source": [ "\n", "# MaxPool2d: Takes maximum value in each window\n", "maxpool = brainstate.nn.MaxPool2d(\n", " kernel_size=(2, 2),\n", " stride=(2, 2)\n", ")\n", "\n", "print(\"MaxPool2d Layer:\")\n", "print(maxpool)\n", "\n", "# Input: (batch=4, height=28, width=28, channels=16)\n", "x = brainstate.random.randn(4, 28, 28, 16)\n", "y = maxpool(x)\n", "\n", "print(f\"Input shape: {x.shape}\")\n", "print(f\"Output shape: {y.shape}\")\n", "print(f\"Spatial reduction: {x.shape[1]}x{x.shape[2]} → {y.shape[1]}x{y.shape[2]}\")\n" ] }, { "cell_type": "markdown", "id": "avgpool", "metadata": {}, "source": [ "### Average Pooling" ] }, { "cell_type": "code", "execution_count": 11, "id": "avgpool_example", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:52.670528Z", "start_time": "2025-10-10T15:36:52.617499Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "AvgPool2d Layer:\n", "AvgPool2d(\n", " init_value=0.0,\n", " computation=,\n", " pool_dim=2,\n", " return_indices=False,\n", " kernel_size=(2, 2),\n", " stride=(2, 2),\n", " padding=VALID,\n", " channel_axis=-1\n", ")\n", "Output shape: (4, 14, 14, 16)\n" ] } ], "source": [ "\n", "# AvgPool2d: Takes average value in each window\n", "avgpool = brainstate.nn.AvgPool2d(\n", " kernel_size=(2, 2),\n", " stride=(2, 2)\n", ")\n", "\n", "y_avg = avgpool(x)\n", "\n", "print(\"AvgPool2d Layer:\")\n", "print(avgpool)\n", "print(f\"Output shape: {y_avg.shape}\")\n" ] }, { "cell_type": "markdown", "id": "adaptive_pool", "metadata": {}, "source": [ "### Adaptive Pooling\n", "\n", "Pools to a fixed output size regardless of input size:" ] }, { "cell_type": "code", "execution_count": 12, "id": "adaptive_pool_example", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:54.510435Z", "start_time": "2025-10-10T15:36:52.677534Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "AdaptiveAvgPool2d (output: 7x7)\n", "Input (28, 28) → Output (7, 7)\n", "Input (56, 56) → Output (7, 7)\n", "Input (224, 224) → Output (7, 7)\n" ] } ], "source": [ "\n", "# AdaptiveAvgPool2d: Always outputs specified size\n", "adaptive_pool = brainstate.nn.AdaptiveAvgPool2d(target_size=(7, 7))\n", "\n", "# Works with any input size\n", "inputs = [\n", " brainstate.random.randn(1, 28, 28, 16),\n", " brainstate.random.randn(1, 56, 56, 16),\n", " brainstate.random.randn(1, 224, 224, 16),\n", "]\n", "\n", "print(\"AdaptiveAvgPool2d (output: 7x7)\")\n", "for i, x in enumerate(inputs):\n", " y = adaptive_pool(x)\n", " print(f\"Input {x.shape[1:3]} → Output {y.shape[1:3]}\")\n" ] }, { "cell_type": "markdown", "id": "pool_comparison", "metadata": {}, "source": [ "### Comparing Pooling Operations" ] }, { "cell_type": "code", "execution_count": 13, "id": "pool_visual", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:55.287654Z", "start_time": "2025-10-10T15:36:54.781122Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABLYAAAGGCAYAAABmJAifAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjYsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvq6yFwwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAjdBJREFUeJzt3Qd0VNXWwPE9CaRQQm+hC0hvgiBWUAQREazAsyAKPBUL4AeKT7qKiCIqCKJPsGEXRFQQQUCkN0WUKr33kEASyMy39uHNOJNGksm0O//fWneFuXPv5MwY7557zj772BwOh0MAAAAAAACAEBMR6AYAAAAAAAAAeUHHFgAAAAAAAEISHVsAAAAAAAAISXRsAQAAAAAAICTRsQUAAAAAAICQRMcWAAAAAAAAQhIdWwAAAAAAAAhJdGwBAAAAAAAgJNGxBQAAAAAAgJBExxYAAAAAAABCEh1bAGAxixcvlk6dOkl8fLzYbDaZOXNmhmP++usvufXWW6VYsWJSuHBhufzyy2X37t0BaS8AIPgQSwAAoYKOLQCwmKSkJGncuLFMnDgx0+e3b98uV199tdSpU0cWLlwov//+uwwZMkRiYmL83lYAQHAilgAAQoXN4XA4At0IAIBv6Cj7jBkzpEuXLq593bp1k4IFC8qHH34Y0LYBAEIDsQQAEMzI2AKAMGK32+W7776TSy+9VNq3by9ly5aVli1bZjrFBACAzBBLAADBpECgGwAAVpacnCypqalev44m1+qIubvo6Giz5cbhw4clMTFRXnrpJXn++edlzJgxMmfOHLn99tvl559/luuuu87rtgIA8hexBAAQLLEkKioq6Kad07EFAD4MHtWrFpGDh9O8fq0iRYqYmwh3w4YNk+HDh+d6lF117txZ+vfvb/7dpEkTWbp0qUyePJmbEQAIMsQSAEAwxZLy5cvLjh07gqpzi44tAPARHRHR4LFjTVWJK5r3md8Jp+1Svdku2bNnj8TFxbn253aEXZUuXVoKFCgg9erV89hft25dWbJkSZ7bCADwDWIJACDYYklqaiodWwAQTgoXubDlVdr/lvjQGxH3m5G8pg7rcuybN2/22L9lyxapWrWqV68NAPAdYgkAIFhiSbChYwsALEanmWzbts31WFOF169fLyVLlpQqVarIwIEDpWvXrnLttddKmzZtTF2Ub7/91izXDgCAIpYAAEKFzaFVJAEA+S4hIUGKFSsmBzdX8Trlt3zt3XLq1KkcjbLrTYXeZKTXo0cPmTZtmvn3e++9J6NHj5a9e/dK7dq1ZcSIEaZWCgAguBBLAAChGkv8hY4tAPBxANm/uZLXASS+9t6gCyAAAN8jlgAAvJVg8VjCVEQA8LE0h8Ns3pwPAAhvxBIAgLfSLBpL8t5VBwAAAAAAAAQQGVsA4GN2cZjNm/MBAOGNWAIA8JbdorGEji0A8DENAGkWDCAAAP8hlgAAvGW3aCxhKiIAAAAAAABCEhlbAOBjVk35BQD4D7EEAOAtu0VjCR1bAOBjVl19BADgP8QSAIC30iwaS+jYAgAfs/9v8+Z8AEB4I5YAALxlt2gsocYWAAAAAAAAQhIZWwDgY2lerj7izbkAAGsglgAAvJVm0VhCxxYA+Fia48LmzfkAgPBGLAEAeCvNorGEqYgAAAAAAAAISXRsId/t3LlTbDab2Vq3bp3n13nggQdcr7Nw4UIJ1nb37dvXnFO7dm1xBNkqEbt27ZICBQqY9n3xxReBbo6Ee5FGbzYAwWXatGmumDF8+PB8fW3iETJDLAGsRa/vzmu9Xvfzk8Yl52trvMqJpKQkKVeunDnnhRdekGCjbdK2aRu1rcgbu0VjCR1bkEOHDsngwYOlcePGUrRoUYmNjZVLLrlEevbsKb/99lugmxfUdu/eLe+++6759xNPPGEutpn5/PPPXcFFt2eeeSZffv/JkyclPj7e9brly5f3eL5q1apy++23m3+PGDFC7PZgvRRZm11skubFpucDvuD+xVe3du3aZThmzZo1HsfolpycHPDOJPetSJEictlll8krr7wi586dk3CUVTw6deqUvP7663LrrbdKrVq1pHDhwmZr2rSpjBs3Ts6fP5/n3/ntt9/Kv//9b/P9oUyZMhIVFSVVqlSRBx980HRkuSMeeY9YglDy8MMPe1ynX3rpJQm2ziTnFhkZKWXLlpVOnTr5ZTA9WL355pty+PBhiYmJMdd2p3Xr1pl7lyuvvFIqVqxorvV6zdfP65dffsnz78ttfNK/Kb1P1TZOmDDBq/cazuwWjSV0bIW5xYsXS7169Uyw+f333yUxMdHcsOzYscPcQOiNwhtvvJGr16xQoYK5yOmmF8i8+s9//uN6Hb3ABSP9bFJTU80F/r777sv0mOPHj5ubDF8YOHCgHDhwINtjevXqZX5u3LhRvv/+e5+0A9mzO7zfAH+YP39+hg6Jd955R4KdjtzqF2+9Jt58881h2WmSVTz666+/pF+/fqYTatu2bXLmzBmzrV+/Xp566im588478/w79fOeMmWK+f5w9OhR06m4Z88emTp1qjRr1ky2b9/ucTzxyDvEEoQKvRZ8+eWXHvs+/fRTCVYaM44cOSKzZ8+W66+/Xt5//30JN9qJNH78ePPvLl26SOnSpV3Pvf322zJmzBhZtmyZ7N+/3/z31Wu+fl7aUfj111/n6XfmNj6VKlXKtE1pW70ZmAlndovGEjq2wtjevXvNxUE7XtQ111xjgtCcOXPMaKvzQq8XnJx+AdWLUXR0tFx99dVma9iwYZ7bpz33ztcpVqyYBBu9mH700Ufm3zfeeKPExcVlepxemDUrTkc/8pOOKP33v/+96Ou2adNGSpQoYf6d01RkAOFJr/l6XXHvMJo+fboEoyZNmpiBjwULFsizzz7r2v/TTz/l+Ut2qLpYPNIpgF27dpVPPvlEfvjhB7n//vtdz33zzTfy888/5/l3N2rUyNxgzJs3z/x0/u5jx47JyJEjPY4lHgHhQa8Heg1wp7NANm3aJMFEY4fGEe1Yueqqq8w+ncbdv39/M1AQTjQ26P2KuuOOOzI8r7NCNOlAj9PvBTrl3fm9YcCAAXn+vbmNT87M34MHD5p7VsCJjq0w9vLLL8uJEyfMv/XipEFIL2Tt27c3NzZa48p5gXefOpe+rsjkyZPN+QULFjRT7rKrDaKjuvrFtlChQlKpUiUzHUFvQpzHO39ndjW2nPuqVasmW7duNemrOg2lZMmSJkXVfYqM3pQ98sgj0rx5czMfW0eytZOsVatWHjdvebF06VJXAMhs+o7S96afl/7uPn36ZHqMvreIiAjznlq2bOnKNNDPUd+X7tcsOGcHpNL32Lt3b/PfZujQodm2U/+7XHfddebf3333XdgF6mDgTbqvcwN8TaeiK822cV6HPvvsMzl9+rTruaw673V6gl6ndGDDfVqg+2iqDpg4r9/uUwg0Fjn3u099uBi9luvAh8YUrbvhHm/cp0boNU9HmrUjTKc4aPzRqXOaqZzZ9XDt2rVy1113mS/xGjP0p44a65TMnNKsZ71G6/Q7/Ux0iot+cdfR6cyO1TimbdPjnnzySTNIlF/xSGOt3lBqtkS3bt3kpptuMtkI7pnQq1atylM80qxsfW1tc9u2bc1P984s5+s6EY+8QyxBqHDPztLrTmb7dQDCee3Xa0f6a5rzubvvvjtP9xG5GUS/5ZZb5OOPP3bt1/sjzSx10mwiLdFSuXJlExc0c0izgzXLOT39bq6ZrFdccYWJnToAXadOHdOJplPvcko7dfS6qoMBGkf0Xkvf79mzZzMcq/df9evXN7+rQYMG5nFuzZgxw/zUz1EHSNzde++9JgP3+eefNzGke/fu5vuBk2Z66/TA3Mb63MQnJ/1M0rcZuWPZWOJA2KpUqZImEpptwoQJGZ7fsGGD63ndtm/fbvZPnTrVte+SSy7xOEaf27Fjh+vxdddd53q9v//+21G8eHGP43Vr3Lix6989evRwHa//du7/+eefXfud++Li4hylSpXK8Hr/+c9/XMceOHAgw/Pu24gRI1zHZtXurLz44ouu43/55ZcMzyclJbk+ny+++MIxbNgw1/FPP/20x7GPPfaY67k333zT7GvXrp1r36xZszyO1/N1f8eOHT3aXa5cuUzbOnLkSNcxy5Ytu+h7Q/44deqU+cyXbqzg+H13xTxver6+jr4ekJ/cr0sPPPCAo2DBgubf3333nXm+ZcuW5nGfPn08rp1nz551vUZ0dHSW19iePXu6jjt58qSjYsWKruv3vn37HNu2bXPExsaafdWqVXMkJCRk2173+JP+Ot25c2fXcw8//LDZl5yc7Lj22muzbJ8+l5KS4nqNb775xvUZpN90vz6fWVv0c3Ras2ZNprFOtyJFijhWrFjhOvbYsWOOypUrZziuUaNG+RqPMnPXXXdl+h0gt/EoPf3bcR7bvHnzDM8Tj3KPWIJQovGhaNGi5m+tTJkyjoMHDzoKFChgHteuXdt1nF6fnddKvSex2+2u5/r37++6Tjivu7m9j8iKXlPd71ucTpw44fG6y5cvN/v1mu18P+k3m83meOutt1yvoe+hW7duWcacOnXqOI4fP55pW/T7vNOQIUOyfI1rrrnGI259/vnnph3ZxRH395mVSy+91Bxbo0YNR07ofY777zt9+nS+xfqs4pOT8/7K/e8JF2f1WELGVpjSEXidiuikI9npac+/jq46/fnnnxmO+fvvv02G18yZM12jBVnR9FUtdu6cuqC97FowcMuWLXl6DwkJCaZw4VdffSWjRo3ymAfupCM6Onqsbfvxxx9NOquOCugIjRo7dmyeR4zdR95r1qyZ4fkhQ4aYz0ene16sholmDtSoUcP1OemohrZX6eiTFmd00joyr776qhkFmjRpUo7a6t6+zP47AoBmluqotdIi5Bs2bJAVK1Z41EbKjF6zdAqBTgnQjB8dhddsH6UZq85YoxlWzuLmev3W2oOaZaujzzqC+95772WbGZYVzQqbO3eux5QE5zR4nRqntSSVjrTr9AltqxY4V/rca6+95srwfeihh1zF5zXbV6fhP/roo+ax7tfns1uJScdeevTo4Yp1ms2m13LNGNPixFrHUkf9nSsWagzSmlRKs5B1BFw/M61hkp/xKD1tn07hVPrZaxzPSzzKjMZkpw4dOmR4nngEWJvWXdL7DKXfgTW2ODNqN2/ebL7HKs1Ccn4/1jixfPnyDNcRzYxyXkfy+z7CnWZS6eu7T4/TLCu9Vus12/l+tL2abarf8TW7VZ/Xki3O67jebziz0jTTSjO3tJ3aXqVTMd2nzmdGM5Sc9zWaIaszTDS+dezY0ZWR7IxbaWlpZtqkM6Zo1pO2T/dpdltu4qjOgslpDEl/rddyNprVmx+xPrv45ORso/63188AMALds4bA2Lt3r0cv++bNmzM9rnz58q5jPvroowyj1FWrVnWcO3fO45zMMp/S0tLMSLVzv2aDOT3zzDN5ytjSbd26da79Ogri3K+jBU7ffvut48Ybb3SULl3aERkZmWFE47fffsuy3dnp0KGD63gddXK3atUq87uKFStmRipUdhlbatGiRRlGXHQEy/29nD9/3nHZZZeZ5yZOnJih3VllbP3www+uY8aMGXPR94b8HRlZ8ke8Y/2uSnne9PxgHBlB6Et/XXJm22h20t133+0a9VVZZWwtWbLEZEtpvHCOyrtv7llO6qGHHspwzOOPP56j9rrHn6y2KlWquEaD3UesNRY46b/dR/vV119/7drXrFkzj9+rj53PzZgxI8uMLY1Jzn1NmjQx2VPOrVWrVq7nVq9ebY6vW7eua58zS0698847+RaP0jtz5oyjbdu2ruOfeuqpPMWjzEyZMsV1vI6oZ3Y88Sj3iCUIJXfccYfr//G5c+eafZMnT3btGzRokOtY/X7v3D9gwABXhlT67Nu83EdkxT1LKqtNM8bU2rVrXfs0xqWmpmb6Pl977TWz79Zbb82Q8Zp+FkyJEiVc2WmZZWw9+eSTrn3PPvusK4a4x60GDRpk+Kzi4+M97smuuuqqHGdsHTp0yHWsZpxdjMYwvcfR4zVr2xnTvI31OYlPqmvXrq5jtO3IGavHEjK2wlT6wrK6Ekh6eh/jXvgxswLuOhdaRzUuRudd60i1M4tK5387ab2rvL4H90wzHdVxco7oaOaAji5r/TBdvSOzXn3nsd5wjpQ46eiE/i4d6Y6Pj8/Ra1x77bXSt29fj3060uP+ueuojdZ/0QKXmk2Q1/bBvyw7lx2Wo9d0zWzS7CRnjQ6tFZWVlStXmnonWgtEC7lmtkJR+musLuGty4U7Va9ePV+WgdfRc804W7RokWs02H0k35lFplq0aOH6t/OYrI7N6vjMuD+nqzrpKLZz09Wk0mdYaVav0+WXX57p78ut7K73mnWg2Q9ak0ZpLTHNJstLPEpPMyecdVO0LplmGGR2PPEo74glCHZ6jdGMIaW1b3WFQWfBb81aVZqZ6rwOaM09jTnuGUDuqylqbSdf3UdkRrOstH6wZtOmv6Zr7Uj3mSy5iSPaXm23s35XZvdd6V9Lvfjii64Y4p4t6yzC7x5D9J7I/Z4sr3HkYtfoJUuWmP+umuWmv0+zoHUV3PRyG+tzGp9y0kaEZyyhYytM6Zd+Ldjn/gU8PZ0i4JySoerVq5fhGE0vzi1NK80PzpWVnNwv5s4LnnvRQp1CodMpNIXXvShiXpeFd18G11mE38k5jUS/5DsLJWrBRye9UOu+9J+7pmi7++OPPzJ93V9//dVV4FcDhZMWD9Z9mhbtzr197u2Gf6RJhNcb4A96XdFpF05aiNZ5Y5EZXTzEGSe0U0mn7uk11n1Vo/TXWL1OuS+GoTcszsLneVkVUb9ka4e/dqDpylY6pS+/41B+xS2n7KYz5uX3ZReP3Pdr0V3t+FP33HOPmZrpvNlM72LxyJ0W79e4o7FXb1L1dzin/GfWjszajYsjliDYaWkS5yJOep3XjiC9nunCGM7BZS007uzo1+f+9a9/ufbrNDxnB5d+v9WFSXx5PXauiqjF6rWzSDuchg0bluV10VftyC0dREpJScm39mknpPP4rGKI0vsonRqoUwx1Kql2Qt52222ZHpubWJ/b+ORso7bZPbEB4R1LgrNV8Aud9+40ceLEDLWmtKfdvV7JJZdckueLpgY05wi6fqF3rwfiPoqd3/bt2+f6t67epB1aGiTd9+dV3bp1PVZL8ZbWBtPMMuW8kOuKh/mxNLJ7+zLroAQAJ13RSDu4lK6UW7x48SyPdb+Wjh492oy26gpTWX151U4u7TjTWhvO65yztlVuR2CdqyJqBquuoJRZzY5LL73UI7vMyVk7zP2YrI5N/9j9uOx+n2Yi6HtKv+n7dWY2ucfV1atXZ9q+/IhH+t9D2+N8H5rx++GHH2aZcZ2bePT000/Lc889Z/6tnVna0ZjdZ0Q8AqxLs3dywn11RO3EcNI6V84sJO3wct5n+Oo+wrkqomZ96aqD6TtS3K9lWhvMPSs5N3FEBwacq93qwLzWCM6K+2voKsVZxRHtWHKPITpY7j4zJTdxRGOBczAiq3sarRWmWWP6PnQVX83M69y5s9exPrfxyb2N+lnlpBMS4eHic8hgWYMGDZKPPvrIjHJrgNBOH11uVy9W2gOvxf3cU2G9oTdJejHU3nd13333mcKLu3fvNtMXfEWXWnem9OqXch1l0ItlfhSs1ZspJ80W0DRhJ/1dOprhTqdlaIFjdcMNN5jsBmeKro5SDRw40NVmLQqvRSJ11EszzTRDSy/cOk0o/U2mjoY4i0xq0Ndi+e7L5CpnoU7NvtBUaviXw2ETu8Pm1fmAv+g1SAc7dGrhxRa+0GPdO7a0cPoPP/zgutalpwVv9XqmNMNH449OsdaFPd56660M09+8pTdGzgK6+to61UFvlJ555hnXMbpsuWrXrp0Z+dUp+NrJ9Nhjj5nrsGahOTudNMMo/TLo7ho3bmymnOhNjI48a+aaTqfQrIWdO3eaL+56c+Acbb711ltdN2j6+3Sahl733YsYexuPdJRcHzsLA2v80c/F+d9BaTF9Z0H9nMYjpd8Z3njjDfNvjU36N6BxXTdnzGnevLlHW4lHeUcsQTDTa6ezQ1y/j6a/d9ABdF1QQ33xxRdmcQ+9P9DBcy2urtdq5/nKPVs4UPcRmhmsAwd6nT5w4IDphNProHYa6bVcRUVFmUEgpdfWWbNmue4FtPNJ44b7rI2uXbtmmxigr+F8T1oEXr/n6+ej8XL79u0ma0qvzXqfplMA9V5CB5l0VofGHP3c5s+f73GNz2kc0XumHTt2mGmG7lPJ9b+XxkrtONO2a1abvjcdyHCfTq/7chPrcxuflL6WttHZZuSew6qxJNBFvhBYCxYsyHJZct0iIiIc48aN8zgnqyXOnbIqwp7VMr3uxX1zUzxeC9e7y6wA4xdffJHh98XExHgUAna+dm6Lx2uBRmdx/VtuueWix2dVPF4LSF5//fWu57SwrnrkkUdyVGD3YsXjtdClFqrU5++8886LthP5X6Txxw1VHb/urJ7nTc8PxiKNCH0XW9TCXWbF47Vwbfoi4/rYvVC6s2jtpk2bzPVX91WvXt0sFa7Lq1eocGHZ6MKFC5s4kR33+JOT67QWUtel0bOKcddee63HsukzZ840hfMzO1b3uxfCzyoWrlmzJtu46v7V6+jRo65l0d23WrVq5Vs8ci/OnNXmbH9u45HG4exeN32cJh7lDbEEocC9QLwWVs+MLqrhPOann35y7dfrivu1QxdKSi+39xFZcb9fuFhRdWecK1q0aKbXOI13b731lutYvYa6FzZPv+lCV8ePH8/23kUNGTIk22ur+/v85JNPMj2mZs2auXqf7sXpv/zyS4/n3O/Jstqc7c9NrM9NfHJyv7ebPXv2Rd8XwieWMBUxzGnRXx2F0KkEOsqs2Vra2641SnTkXUepdbQgP+hceR3B1iV/daRWl7DV6Qs6ouHkLKyYXzTbQKdUaHqt/k4dTdDMKfeik3mlKbLO0SQdYXIuBZxbOhruXNZWRyk0K8tZh8tZUFM/I/e069zQ0RFndoCOMsH/0hwRXm9AMNLitDpqrSPueo2tX7++GdnV7Kf00xL0+uOsvaK1ufR6r1k+eg10TlPQqQv5WRRW45lenzUTSke8Y2NjTTu1vZpdpCPfOtrupNMqdFqLxg6d+qLXeZ0yooWPtQaLZlhdjGYh6ZQQXd5cp4no6+v71Lij+3Qk3UkzxBYvXmwyePXz0DonWqxfP8PcIB6FB2IJQmUaYlbXSvcC6O7TEfV645wCrzKr7RiI+whnnFuzZo25L9LsKL3e6nRCvT5qDHFfzEmzmTSrTGOcnue8r9Ipc5opvHz58gw1gjOjsy9mz55tfofGCc361d+t0yY1nrlngHXr1s189ppZpvFGp1RqNpf7FM+c0N+lC384F9/KC3/EemfbtK3OGIXcsWossWnvVqAbgfChf27p02/1Qu9c9ULreuVXR5o/7NmzR2rWrGnSqzW1NjcrFfrL3XffbW6S9IZT07zdvzjAt3Q6qqZy//B7dSlcNO+fe9Jpu3RodCE1PP2KpgCgiEfWRSwBrHkfEWz0c9TPUweBNKYEW2F2ne6qgyxau0vbqmV1kHMJFo8lfKOAX2nhdh2h0TncumltDmd9Dh2N0FHxUKIX1169epl/a62AYOsn1lopzpENXb6Ym4jAsItN7BLhxRakc9kBBA3ikfURSxDurHYfEWy01qNmK2vHkWZbBRttk7ZN26htRd7YLRpLyNiCX2VVLFH366qF+V04GAiGkZFZv9eQwkXzvmpL0uk0ubXR9qAbGQEA+B6xBLiA+wgg7xIsHksYLoNfPf7446bOif5PpSMr8fHxZiURnTNPMAIAAACQGe4jAPi9Y0uXJtWiddqLpwXjHnroIUlMTMz2HC0GqD3u7psWWoV1aLrwb7/9ZpZq1Togujztl19+6bE0OWA1Vi3SGMyxQpcA79ixoylaqinrAwcOlPPnz/v43QCA7xBLskYsCQ/cRwDeS7NoLCngqxfW4HLgwAGzOs+5c+fM6gd9+vQxK0VkR1cD0pUgfLm6BQD4fy573uejB+tc9mCNFWlpaeZGRFfM0ZXs9PXvv/9+M7r74osv+vT9AICvEEuyRiwBgPCOJT7p2NJloOfMmSOrVq2S5s2bm3067/nmm2+WV155xaSNZkUDinOpUQCwAi20mOZFgqxdrFkK0VexQpff/vPPP+Wnn36ScuXKSZMmTWTUqFHy9NNPm6LVuhw2AIQaYknmiCUAkHNWjSU+6dhatmyZSQN2BhfVtm1bswLOihUr5Lbbbsvy3I8//lg++ugjE2Q6deokQ4YMyTZrKyUlxWxOdrvdpCPr8qRZFRgEgIvRdTVOnz5tvhCzepdv+CpW6Os2bNjQ3Ig4tW/fXh555BHZuHGjNG3aNMPrEUsA+AKxJLxiiSKeAMhvxJIAdWwdPHjQzEP3+EUFCkjJkiXNc1n517/+JVWrVjX/wX7//XczIrJ582bX8tCZGT16tIwYMSJf2w8ATnv27JFKlSp59RrezkdPs+jitb6KFXqu+42Icj7O6nWJJQB8iVgSHrFEEU8A+AqxJJ86tp555hkZM2bMRdOB80rnwjvpCEmFChXkhhtukO3bt0uNGjUyPWfw4MEyYMAA12NddrJKlSpytdwsBaRgnttiNQXKeQb8cGcvUzzQTQg6qaWoZ+fu/PkUWf7rS1K0aNF8SfnVLe/nB2cACaVYcTHEEuTF+euaBLoJCHLEkvCKJYp4grwgniA7xJJ87th66qmn5IEHHsj2mEsuucSk8x4+fNhjv64gomm4uamf1bJlS/Nz27ZtWQaY6Ohos6WngaOAjeDhVCCCOgDu7JEZ/2bCnb1ATKCbEJRCcdrA4sWLZezYsbJmzRpT7HbGjBnSpUuXTI/VFaDefvttee2116Rfv3758vsDHSv03JUrV3occ+jQIfMzq9clliBPuG7CwrEk0EIxlijiCfKEeIIcIJbkU8dWmTJlzHYxrVq1Msuw6k1Vs2bNzL4FCxaYOebOoJET69evNz91BAUAQlWaw2Y2b87PjaSkJGncuLE8+OCDcvvtt2d5nHZ4LV++PNvCunkR6Fihr/vCCy+YGx3n9BRdKUuXga9Xr14e3xUAhFcsCTRiCQDkvzSLxhKfVB6rW7eu3HTTTWYJXR3p+PXXX+Wxxx6Tbt26uW6g9u3bJ3Xq1HGNhGjar640okFp586dMmvWLLOk7rXXXiuNGjXyRTMBwC905RFvt9zo0KGDPP/889kWzNVr8OOPP24K5+rS5YHgq1jRrl07c9Nx3333yW+//SZz586V5557Tvr27ZvpKDoAhAJ/x5JQQSwBgJyzaizxSfF4pTdLGlR0rrpW7r/jjjvkjTfecD1/7tw5U6DxzJkz5rEumavL6Y4fP95kG1SuXNmcowEEAEKZ3RFhtryfn79z2XUUW7+oDxw4UOrXry+B5ItYERkZKbNnzzYrV+mIe+HChaVHjx4ycuTIgLxHALBiLAkmxBIACO9Y4rOOLV2JZPr06Vk+X61aNbNspZMGlEWLFvmqOQAQ8hISEnJUx+NitBivrhj1xBNPSKD5KlboSlfff/99vrUTABC8iCUAEN581rEFALjA27TdtP+tPqJfxN0NGzZMhg8fnqvX0mkXr7/+uqxdu5YClAAQhrEEABC+0iwaS4JzgiQAWIjdrVBjXjY9X+3Zs8csG+7cdEnx3Prll19MIVxdelyztnTbtWuXWX1KR7QBANaOJQCA8GUPQCzRFds7depk6h7qwPrMmTM9poo//fTT0rBhQzPlW4/Rmof79+/P1e+gYwsAQoSuxOS+5WUaotbW+v33383qT85NA4jW29LCuAAAAACQX5wrtk+cODHDc1r7UGeSDBkyxPz8+uuvTU3EW2+9NVe/g6mIAOBjdokwmzfn50ZiYqJs27bN9XjHjh2mA0trkGimVqlSpTyO11URy5cvL7Vr185zGwEA1oolAADrsQcgluiK7bplplixYjJv3jyPfRMmTJAWLVrI7t27zb1LTtCxBQA+luaIMJs35+fG6tWrpU2bNq7HAwYMMD91Nadp06bluR0AgPCJJQAA60kLgViiJVd0ymLx4sVzfA4dWwDgY3bR+eh5L9Se23Nbt27tsfrTxezcuTMPrQIAWDmWAACsx55PsSS/VmtPLzk52dTc6t69uym9klMM3QAAAAAAACBHdLV2nUbo3EaPHi3e0kLyd999txmgnzRpUq7OJWMLAHwsFFJ+AQDBjVgCAAiWWLJnzx6PjCpvs7WcnVq6WvuCBQtyla2l6NgCAB9LkwizeXM+ACC8EUsAAMESS+L+t0p7fnB2am3dulV+/vnnDAtd5QQdWwAAAAAAAMh32a3YXqFCBbnzzjtl7dq1Mnv2bElLS5ODBw+a4/T5qKioHP0OOrYAwMfsDpvZvDkfABDeiCUAgFCMJauzWbF9+PDhMmvWLPO4SZMmHudp9pYuipUTdGwBgI/ZvUz51fMBAOGNWAIACMVY0voiK7bnZjX3rNCxBQA+ZndEmM2b8wEA4Y1YAgDwlt2isSQ4WwUAAAAAAABcBBlbAOBjaWIzmzfnAwDCG7EEAOCtNIvGEjq2AMDHrJryCwDwH2IJAMBbdovGEjq2AMDH0rwc3dDzAQDhjVgCAPBWmkVjSXB2twEAAAAAAAAXQcYWAPiYVVN+AQD+QywBAHjLbtFYEpytAgALSXNEeL0BAMKbv2PJ4sWLpVOnThIfHy82m01mzpyZ5bEPP/ywOWb8+PH58E4BAL6SZtH7kuBsFQAAAICASUpKksaNG8vEiROzPW7GjBmyfPly0wEGAEAgMBURAHzMITaxe1GkUc8HAIQ3f8eSDh06mC07+/btk8cff1zmzp0rHTt2zHPbAAD+4bDofQkdWz7Q8Jq6ctf/3SqXNrtESsWXlGG3vSxLv1kl4erux9rJVTc3lko1y0lq8jn5c/Xf8t4L38i+7YclXN1y1+XS8a4WUi6+uHm8a/th+XjKQln969ZANy0odO92hfTp1Vq+/GqVTJw0X0Kdt2m7wZryCwAI31hit9vlvvvuk4EDB0r9+vXz9bUBAOERS/KLz1ul6cvVqlWTmJgYadmypaxcuTLb47/44gupU6eOOb5hw4by/fffS6iJKRwtf/++S9587L+BbkpQaNiqpnw7bbH0v+UVebbbBClQIFJe+OQxiY6NknB15FCCvPfGj/LYvybJ4/+aLL+t2iHDx/9LqtYoK+Gudu3y0qljE9luoY5Pu8Pm9WZVx48fl3vuuUfi4uKkePHi8tBDD0liYmK2x2t2QO3atSU2NlaqVKkiTzzxhJw6dcrjOK31kn779NNP/fCOACC4Y0lCQoLHlpKSkqf2jBkzRgoUKGCuwYFGLAGA8L4v8WnH1meffSYDBgyQYcOGydq1a808/fbt28vhw5nfsC5dulS6d+9ugtG6deukS5cuZvvjjz8klKyas16mDflUfp2ZfSdeuBhyz1vy0+crZPeWg7Ljz30yrt9HUq5SSanVqLKEqxWLN8uqJVtl/+7jsm/3MZk24SdJPpMqdRpWknAWE1NQ/jP4VnnltR/kdGJyoJsDP9AbkY0bN8q8efNk9uzZplhxnz59sjx+//79ZnvllVdMbJg2bZrMmTPHxI30pk6dKgcOHHBtGk8AINxVrlxZihUr5tpGjx6d69dYs2aNvP766+YarJ09gUYsAYDw5tOpiOPGjZPevXtLz549zePJkyfLd999J++9954888wzGY7XAHnTTTeZlGY1atQoE6AmTJhgzoU1FIqLMT9PnzwT6KYEhYgIm1xzYwOTwfbX73sknPV7op0sX7Fd1q7dJffdc5VYRZpEmM2b863or7/+MjcSq1atkubNm5t9b775ptx8883mZiOzQsQNGjSQr776yvW4Ro0a8sILL8i9994r58+fN9kDTjpqX758eT+9GwAIjViyZ88ek9nkFB0dnevX+uWXX8xAtWY6uV4/LU2eeuopszLizp07xV+IJQCQc1a9L/FZq1JTU81oTtu2bf/5ZRER5vGyZcsyPUf3ux+vNMMrq+MRenRU798j7pSNK7fLrs0HJJxVq1lOZi59TmavHCZPPNdJRg6YLrv/PiLhqk3rulKrVjl5592FYjVWTfn1ll7b9YbBeSOiNAZorFixYkWOX0enjuhNmvuNiOrbt6+ULl1aWrRoYQZUHA5HvrYfAEIxluj10n3LS8eW1tb6/fffZf369a5NO5B0cFoLyfsTsQQAcs6q9yU+y9g6evSoGbkpV66cx359vGnTpkzPOXjwYKbH6/6saF0A99oAWisAwavvi3dLtToV5P+6vCbhbu/Oo/Jo17ekUJEYuaZtffm/kXfIwF7/DcvOrTJlispjfdvKwEGfyrlzaYFuDvxEr+1ly3rWldMbipIlS2Z73U8fazS7N/2Uk5EjR8r1118vhQoVkh9//FEeffRRU28lq1owxBIA8KTXzG3btrke79ixw3Rg6TVaM7VKlSrlcXzBggVNZpPWrQrXWKKIJwDgfyG/KqLWBRgxYkSgm4EceOSFu6TFjQ1k4G3j5eiBkxLuzp9Pk/17jpt/b/trv9SuX1G6/KuVvPH8LAk3l9YqLyVLFJYpky9MW1aRkRHSqGFlua1LM2nXYazY7aE7QmqXCLN5c34o0anmWlT4YlNHvKU3C7q8fL169WT48OEezw0ZMsT176ZNm0pSUpKMHTs2y5sRYgmAYOfvWLJ69Wpp06aN67HWzVU9evQwNal8LRRjiSKeAAhmdovel/isY0tTdiMjI+XQoUMe+/VxVvPUdX9ujleDBw92BVpncNKimAi+Tq0rb2osT9/5uhzacyzQzQlKtgibFIyKlHC0dt0u6dnrXY99Tw/sKLt3H5NPPlse0p1aKs1hM5s354cSrbHywAMPZHvMJZdcYq7t6RcT0domulrVxeqZnD592tRkLFq0qMyYMcNkCmRHV+XV0XgdRc9s2g2xBECw83csad26da6m3eV3Xa1QjCWKeAIgmKVZ9L7EZx1bUVFR0qxZM5k/f75r9RC73W4eP/bYY5me06pVK/N8v379XPu0eLzuz4oGlbzUBvClmMIxUrHmP4G0fPWyUqNxNUk4nihH9hyVcJx+2Pq25jKy5xQ5m5gsJcoUNfuTTidLavI5CUc9H79RVv26RY4cPCWxhaKlTYdG0qh5NfnPox9IODp7NlV27vT8fyM5+ZwkJJzNsD8UeTsfPVjnsmelTJkyZrsYvbafPHnS1GPUeKEWLFhgYoXePGRFbxK0/qJe+2fNmiUxMRcWpMiOTp8pUaJElvEiGGMJALgjlgR/LFHEEwDBzG7RWOLTqYg6WqHpylrMUQsu6iopmsLrXCXx/vvvl4oVK7qWGX7yySfluuuuk1dffdWkBH/66acmDXrKlCkSSi5tfom8+vM/KciPjLsw2vTjtIUy9sGJEm5ueeBa8/Plr//psFSv9vtQfvo850U9raR4ycIy8Pk7pGTponImMVl2bDlkOrXWLt8e6KYBflO3bl0zUq6r5+rKt+fOnTMDH926dXOtYrVv3z654YYb5IMPPjBxRG9E2rVrJ2fOnJGPPvrIPHbWL9EbIM0U/vbbb0227xVXXGFuVHSA5MUXX5T/+7//C/A7BgDkN2IJAMCnHVtdu3aVI0eOyNChQ03xxiZNmpjleJ0F4nfv3m1WLHG68sorZfr06fLcc8/Js88+K7Vq1ZKZM2eaJXlDye+L/pQbI+4KdDOCRof4zDP0wtlrI2YGuglBr/9T08UqHI4IsTsivDrfqj7++GNzA6I3HBoP7rjjDnnjjTdcz+sNyubNm83Nh1q7dq1rlauaNWt6vJYWNq5WrZqZSjJx4kTp37+/mUajx40bN87c9ABAqCKWZI1YAgDhHUt8Xjxeg0xWUw8XLlyYYd9dd91lNgCwijSxmc2b861KV63SAY2s6M2Fe42XnNR80ZF73QDASoglWSOWAEB4x5Lg7G4DAAAAAAAAAp2xBQDhThd19K5IY742BwAQgoglAABv2S0aS+jYAgAfs3s5l92bcwEA1kAsAQB4y27RWELHFgD4mF1sZvPmfABAeCOWAAC8ZbdoLAnO7jYAAAAAAADgIsjYAgAfS3PYzObN+QCA8EYsAQB4K82isYSOLQDwMavOZQcA+A+xBADgLbtFY0lwtgoAAAAAAAC4CDK2AMAfRRod1ivSCADwH2IJAMBbdovGEjK2AMDHHP9bfSSvm56fG4sXL5ZOnTpJfHy82Gw2mTlzpuu5c+fOydNPPy0NGzaUwoULm2Puv/9+2b9/vw/eOQAgVGMJAMB6HBaNJXRsAYCP6aiIt1tuJCUlSePGjWXixIkZnjtz5oysXbtWhgwZYn5+/fXXsnnzZrn11lvz8R0DAEI9lgAArMdu0VjCVEQAsJgOHTqYLTPFihWTefPmeeybMGGCtGjRQnbv3i1VqlTxUysBAAAAwHt0bAFAiKw+kpCQ4LE/OjrabN46deqUmbJYvHhxr18LAOAbVl3JCgDgP3aLxpLgbBUAWEh+pfxWrlzZZFw5t9GjR3vdtuTkZFNzq3v37hIXF5cP7xYA4AtWnT4CAPAfu0VjCRlbAOBjzmKL3pyv9uzZ49H55G22lhaSv/vuu8XhcMikSZO8ei0AQGjEEgBA+LJbNJbQsQUAIUI7tfIrq8rZqbVr1y5ZsGAB2VoAAAAAQhIdWwDgY96m7eZ3yq+zU2vr1q3y888/S6lSpfL19QEA1o8lAIDQY7doLKFjCwAsFkASExNl27Ztrsc7duyQ9evXS8mSJaVChQpy5513ytq1a2X27NmSlpYmBw8eNMfp81FRUXluJwDAd6x6MwIA8B+7RWMJHVsAYDGrV6+WNm3auB4PGDDA/OzRo4cMHz5cZs2aZR43adLE4zzN3mrdurWfWwsAAAAAeUfHFgBYbGREO6e0IHxWsnsOABCcrDrKDgDwH7tFYwkdWwDgY1YNIAAA/yGWAAC8ZbdoLKFjCwB8zOHl0rjkVwEAiCUAAG85LBpL6NgCAADZKhVfUnq9dI+06NBUogtFy/5tB+WVByfKljV/B7ppCAIRETbpcf/VcuMN9aVkycJy9FiizJ27QT78eGmgmwbAj255uJ10eridlKtWxjzetXGvfDTqC1k1Z32mx7fr0VoGTu3rsS81OVU6FrrHL+1FYGi8eOD+qz327d59THo8+E6mx7dv11CeGdTRY19q6nlpf/MrPm0nQkuEr3/BxIkTpVq1ahITEyMtW7aUlStXZnnstGnTxGazeWx6HgBYIeXXm83qchMr1BdffCF16tQxxzds2FC+//77DHXEhg4dalaBjI2NlbZt28rWrVt9/C6sqUjxwjJ+yShJO5cmz978ovSq31/e/r/35fSJpEA3DUGie9crpHOnpvLGhHnS48F3Zco7C6Vb15Zye5dmgW6apRBLLo5YElhH9x6T/w7+WPo2f1r6Xv6MrP/5Dxkx82mpWq9SlucknTojd1fo7druqfaoX9uMwNix44jcftebru3xfh9le3xiUrLH8d3+9Zbf2mo1dovGEp92bH322WdmNa5hw4aZpeUbN24s7du3l8OHD2d5TlxcnBw4cMC17dq1y5dNBACfs2oACVSsWLp0qXTv3l0eeughWbdunXTp0sVsf/zxh+uYl19+Wd544w2ZPHmyrFixQgoXLmxeMzk52Y/vzBq6Pt1Fjuw5Jq889JZsXrVNDu48LGvm/S4H/j4U6KYhSNSvX1F+XbpVlq/YLocOnZLFv2yW1Wt2Sp06FQLdNEshlmSPWBJ4y2evkZU/rJN92w7Kvq0HZOpzn8jZxGSpe8WlWZ6jnYcnDp10bScPn/JrmxEYaWl2OXEiybUlJJzN/gSHeBx/4uQZfzXVcuwWjSU+7dgaN26c9O7dW3r27Cn16tUzQaFQoULy3nvvZXmOZmmVL1/etZUrV86XTQQABFhuY8Xrr78uN910kwwcOFDq1q0ro0aNkssuu0wmTJjg+pI8fvx4ee6556Rz587SqFEj+eCDD2T//v0yc+ZMP7+70NeqU3PZsma7DPlsgHx+8F2ZtOZl6dDrhkA3C0Fk48Z9clnTalKpYgnzuMYlZaVBg0qyciVTVeE/xJLgEhERIa27XikxhaPlz2VbsjwutkiMfLTjLfl41yQZMWNQttldsI6KFUvIF5/2lY8/fFj+M7iTlC0bl+3xsbFR8snHj8hn0x+V50feIdWqlvZbWxEafNaxlZqaKmvWrDEpu65fFhFhHi9btizL8xITE6Vq1apSuXJlE0Q2btzoqyYCgF9YdWQkULFC97sfr3QE3Xn8jh075ODBgx7HFCtWzExLyS7+IHMVLilraqbs23ZABt/0vHw7+Ufp+/qDcuP91wW6aQgS0z9dJgsW/invT+0j8+YMlCmTe8pXX6+Snxb8GeimWQqxJGvEkuBRrUEVmZXwoXyfPF2enNRHRtw+Vnb/tTfTY/ds3m+ygYd1eVnG3Pem2CJs8vqvL0jpiiX93m74z19/7ZcxY7+Tpwd/LuNfnyvlyxeT11+7x3ReZWbPnmPy8ivfy3NDv5IXX/rWJMK8+ca9Urp0Ub+33QrsFo0lPisef/ToUUlLS8uQcaWPN23alOk5tWvXNqMqOiJy6tQpeeWVV+TKK680nVuVKmXee5+SkmI2p4SEhHx+JwDgHW+DQLAGkEDFCr3RyOx43e983rkvq2PSI5ZkzRYRIVtWb5f3/vOJebx9/U6p1qCy3PLvdjLvg0WBbh6CQOvr6krb6+vL8y/Okp27jkrNGmWl76Nt5djRRJk7759pXfAOsST4Y4kK93iyd/N+ebjpQClcrJBcc+cVMnDaY/JU62GZdm79tXyL2Zw2Lt0s//1zvHT8943y/tDP/Nxy+MvKVf9k8/6944j8+dd++XT6I9Lmujry/ZzfMxyvz+vm9MfGffL+e72l0y1NZOq0X/zWbquwWzSWBNWqiK1atTKbk3ZqaWrw22+/bdKDMzN69GgZMWJEhv2JtzWXAgUpPO+UXMLn6wSElJQSwfk/ZCClFg/WxVsDw55sF8mne3aHw2Y2b86Hb2UVSyBy/MCJDDcku//aJ9fcfkXA2oTg8nCfNvLJp8vl54V/uYoClytXTP7VvRUdW/mIWBIawj2enD93XvZvv9Dxt3Xt31K7eQ257cmb5fWHp1z03LTzabJ93Q6pWKO8H1qKYJGUlCJ7956Q+P9NZ89Jfa6t2w5JxficHY/wiCU+6+0oXbq0REZGyqFDnsVl9bHWzsqJggULStOmTWXbtm1ZHjN48GCT3eXc9uzZ43XbAQD+kZdYofuzO975MzevSSzJ2sZfN0ulS+M99lW6tIIc2nUkYG1CcImOKSh2h+fgiN1uN9OKgHCKJYp4kjHrNyqqYI6O1emj1RpWkWMHT/q8XQgeMTEFJb5CcTl+LDFHx0dE2OSS6mXk+PGcHY/w4LOOraioKGnWrJnMnz/f40uOPnbPysqOphRv2LDBLLGblejoaLOSovsGAMHELjavN6vKS6zQ/e7Hq3nz5rmOr169urnpcD9Gp4LoilZZvSaxJGtfjZ8tda+oJd0H3ybxNcpLm+5Xy82928qst+YEumkIEsuWbZN7/9VKrmhZw2RqXX3VpXLXHS1kyZKsC0Yj94glwR9Lwj2ePPjiv6ThNXWlXNUyptaWPm7cup7Mn35hutigaY+ZfU73DrlTmt3YSMpXLys1m1aXZz583Jz7w7ue/11gvSzfxo0qm3hRv15FGTXidrHbHTL/5wt1GQc/fYv0euifOp7333uVNG9WTSpUKCa1apaTZ5/pJOXKxcl33/8WwHcRuuwWjSU+nYqoS+726NFDmjdvLi1atDAriyQlJZnVStT9998vFStWNCm7auTIkXLFFVdIzZo15eTJkzJ27FjZtWuX9OrVy5fNBACfsupc9kDFiieffFKuu+46efXVV6Vjx47y6aefyurVq2XKlAvTHLSoaL9+/eT555+XWrVqmZuTIUOGSHx8vFnKHbmj9bWG3z5WHnrxHnMTcnDHYZnUf5osmL4k0E1DkHhjwjx58IFr5Mkn2kmJ4oXk6LFE+fa7dfLBh78GummWQizJHrEk8IqXLSaD3n9MSlYoIUmnzsiO33fJ4JtekLU/XaibVLZKaXHY/8nuLFKisPSf8rCUKF9cEk8kydY1f8uTV/0ny2LzsIYyZYrKc8/eKnFxsXLq1BnZ8Mde6fv4B3Lq1FnzvK6QqB1dTkWKxshTAzpIyRKFJTExWbZsPSiPPfmR7Np9LIDvInTZLRpLfNqx1bVrVzly5IgMHTrUFFls0qSJzJkzx1WEcffu3Sbl1OnEiRNmmV49tkSJEmbkZenSpWbJXgCANeU2Vmj9xenTp5sl2J999llzw6FLrzdo0MB1zKBBg8wNTZ8+fcxAydVXX21eMyaG2ot5seK7tWYDMnP2bKpMnDTfbECgEEsCb1yvSdk+/3/XD/d4PHnA+2ZDeBn1wqxsn+//1HSPx29Nmm82IDs2hyNdUYQQpynCuhRv89tGUTzeDcXjPVE8PiOKx3uyJyfLjuH/MfUx8jqNwHk9ajHjSSlQODrPbTmflCIrb3vdq7Ygb//tWktnKWDLWW0QhJ/z1zcLdBMQ5M6fT5Yli0YQS8IY8QQ5QTxBdoglIbYqIgBYkVVTfgEA/kMsAQB4y27RWELHFgD4mFWX1QUA+A+xBADgLYdFYwnz0wAAAAAAABCSyNgCAB/TkQ27BUdGAAD+QywBAHjLYdFYQscWAPiYluX3ZpkOyvoDAIglAABvOSwaS5iKCAA+Zheb1xsAILwRSwAAoRhLFi9eLJ06dZL4+Hix2Wwyc+ZMj+cdDocMHTpUKlSoILGxsdK2bVvZunVrrn4HHVsAAAAAAADId0lJSdK4cWOZOHFips+//PLL8sYbb8jkyZNlxYoVUrhwYWnfvr0kJyfn+HcwFREAfMyqq48AAPyHWAIACMVY0qFDB7Nl/noOGT9+vDz33HPSuXNns++DDz6QcuXKmcyubt265eh30LEFAD6mBRptXgQQbwo8AgCsgVgCAAiWWJKQkOCxPzo62my5tWPHDjl48KCZfuhUrFgxadmypSxbtizHHVtMRQQAAAAAAECOVK5c2XRAObfRo0dLXminltIMLXf62PlcTpCxBQA+piuPeLX6SLAuPwIA8BtiCQAgWGLJnj17JC4uzrU/L9la+YmOLQDwMeqiAAC8RSwBAARLLImLi/Po2Mqr8uXLm5+HDh0yqyI66eMmTZrk+HWYiggAfgog3mwAgPBGLAEAWC2WVK9e3XRuzZ8/37VP63fp6oitWrXK8euQsQUAAAAAAIB8l5iYKNu2bfMoGL9+/XopWbKkVKlSRfr16yfPP/+81KpVy3R0DRkyROLj46VLly45/h10bAGAj7GSFQDAW8QSAEAoxpLVq1dLmzZtXI8HDBhgfvbo0UOmTZsmgwYNkqSkJOnTp4+cPHlSrr76apkzZ47ExMTk+HfQsQUAPkbBXwCAt4glAIBQjCWtW7cWRzYn2mw2GTlypNnyihpbAAAAAAAACElkbAGAX0ZGvFl9JF+bAwAIQcQSAIC3HBaNJXRsAYCPsUQ7AMBbxBIAgLccFo0ldGwBgI/pwIY3gxtBOjACAPAjYgkAwFsOi8YSamwBAAAAAAAgJJGxBQA+ZtWUXwCA/xBLAADeclg0ltCxlc8euvNK6XXXlR77du07Jt0GTJVwVqZ4EXnytmvkqvrVJCaqoOw5clKGvz9X/tx9SMLRvEEPSsUSxTLsn75svTw/62cJNxE2mzxxdSvpXK+ulClcWA4nJspXf2yUiUtXiCX4Oed38eLFMnbsWFmzZo0cOHBAZsyYIV26dPnn5RwOGTZsmLzzzjty8uRJueqqq2TSpElSq1YtLxoJAAiXWHLu3Dl57rnn5Pvvv5e///5bihUrJm3btpWXXnpJ4uPjvWgkAMCnHNaci0jHlg9s33NUnhj1uetxmj1I/+v7SdFC0TJtYFdZtXmPPDZhhpw4fUaqlC0hCWeSJVzdPfETibT909tdq1xp+W+vO2Tuhq0Sjv7d8nL5V5PGMui7ObL16DFpWKGcvNShvZxOSZUP1qwLdPNCTlJSkjRu3FgefPBBuf322zM8//LLL8sbb7wh77//vlSvXl2GDBki7du3lz///FNiYmIC0mYAQOjEkjNnzsjatWtN/NBjTpw4IU8++aTceuutsnr16oC1GQAQnnxaY0tHejp16mRGbmw2m8ycOfOi5yxcuFAuu+wyiY6Olpo1a8q0adMk1KSl2eX4qTOu7dTpsxLOera7XA4ePy3DP/hRNu48KPuPJcjyv3bJ3qOnJFydSDorRxPPuLbr6laX3cdOyqodeyUcNa0YL/O3bZeFf++QfQkJMmfzVlmyc5c0rlBeLOF/Kb953fT83OjQoYM8//zzctttt2VsisMh48ePNyPtnTt3lkaNGskHH3wg+/fvz9E12lcmTpwo1apVMx1rLVu2lJUrV2Z5rGaaXXPNNVKiRAmzaZZA+uMfeOABE3fct5tuuskP7wQArB9LNENr3rx5cvfdd0vt2rXliiuukAkTJpjsrt27d0ugEEsAILhiiSU6tpwjPRpkcmLHjh3SsWNHadOmjaxfv1769esnvXr1krlz50ooqVy+hMya9LB8+UYvGf74zVKuVFEJZ9c1rmGmHL7c+xaZ//LD8smz98ptVzcMdLOCRsHICOnUpK58vfoPCVfr9u2XVlUrS7USxc3jOmVKS/NK8bLo7x1iBQ6H91t+0evswYMHzRd49xsUvQFYtmyZBMJnn30mAwYMMNMjNQNA44ZmkB0+fDjLAZDu3bvLzz//bNpcuXJladeunezbt8/jOL350Okzzu2TTz7x0zsCAGvHksycOnXKdPwUL34hlvsbsQQAQj+WBOVURB3p0S2nJk+ebKbFvPrqq+Zx3bp1ZcmSJfLaa6+ZwBQKNm47IM9P+kF27T8upUsUkYfuaCWTRnSXe/9vqpxJPifhqGLpYnLXtY3lo5/WyH/nrJD6VcvLoLvbyPnzafLt8j8l3N1Qr6YUjYmWGWvC97OYvHylFImOkh9795Q0u10iIyJk3OIlMuvPTWIF+VWkMSEhwWO/ZrbqlhvaqaXKlSvnsV8fO5/zt3Hjxknv3r2lZ8+erljw3XffyXvvvSfPPPNMhuM//vhjj8fvvvuufPXVVzJ//ny5//77Xfv1sylf3iJZfwDCXjDFkvSSk5Pl6aefNh1FcXFxEgjEEgAI3+LxPs3Yyi0dLXHPIlDaoZVdFkFKSooJ0O5bIC1fv0MWLN8i23cflRW/7ZQBL30tRQtHyw2taku40sLgm3Yflgnf/Cqb9xyRr5dskBlLNsid1zYKdNOCwu3N68svW3bKkdNJEq5urltbbq1XV/p/+710nvaxqbX1UIvmcluDeoFuWlDR0WTNrnJuo0ePllCXmppqpq64X/sjIiLM45xmkGmtFy1kXLJkyQyj8WXLljXTZB555BE5duxYyMQSAAiVWKLXX52SqFPddSGScI4lingCAGFePF6zBTLLItCAcPbsWYmNjc1wjgbjESNGSLBKPJMiuw+ckErlS0i4OnoqSf4+4PklYMfBY3LDZazAFl+8qLSqWUWe/OhbCWfPtL5W3l6+Ur77a7N5vOXoUYmPi5OHr2ghM/6wQCabt/PR/3funj17PEbC8zLC7hx1PnTokFSoUMG1Xx83adJE/O3o0aOSlpaW6bV/06acZexploDWcnS/odGpI1rsWLOAt2/fLs8++6zJINYbnMjIyJCLJQAQTLEkfafWrl27ZMGCBQHL1gqWWKKIJwDCIZYEm6Dq2MqLwYMHm/n0TtoJpiNRwSI2uqBUKldM5ixOlHC1fvt+qVrOs2OvSrkScuAYI1i3NasvxxPPyqLN1qgllVcxBQuIPd2EbbvDbrL9rMDb+ejOc/WGwdubBv1yrp1bOtXC2ZGl180VK1aYkehQo0vLf/rpp2ZE3X1Fx27durn+3bBhQ1Mkv0aNGua4G264IeRiCQAEUyxx79TaunWrqVNVqlQpCVX5FUsU8QRAOMSSYBNUHVt6s6VZA+70sQbfzLK18qsuQH56/N7rZMma7XLgaIKUKVFEet11paTZHTLvV2vUCsqLj+avkWmDusmDN7WQeWu2SP1q5eWOqxvJqI/nSTjTPhvt2Jq59k/zNxLOFmz7Wx69sqXsTzgtW48ek3rlysqDlzeTL37fGOimhaTExETZtm2bR8F4XZBDp1dUqVLFLMyhK13VqlXLdHTpcu06St2lSxe/t7V06dJm1Duza//Fapq88sor5mbkp59+Mjcb2bnkkkvM79LPJbObkWCLJQAQzLFEM37vvPNOU6R99uzZJlvKWadRn4+KigrLWKKIJwAQ5h1brVq1ku+//95jny4lrPtDRZlSRWXEE7dIsaIxcjLhrPy2eZ/0fu5jOXn6rISrP3cdkqcmz5LHu1wjfTpeIfuOnpKxXyyUH1aGb2ef0imI8SXi5Os14bsaotPInxZIv2uukhHtbpBShQrJ4cRE+WT97zLh1+ViCdpv6U3fZS7PXb16tVld1sk5ctyjRw+ZNm2aDBo0yKxa26dPHzl58qRcffXVMmfOHI9Ran/Rm59mzZqZDDJnx5rdbjePH3vssSzPe/nll+WFF14wq+Y2b978or9n7969pi6K+/RLAAgpQRRLhg8fLrNmzTKP009j1+yt1q1biz8RSwAgOGOJJTq2LpY1oKm6uqTuBx98YJ5/+OGHZcKECeam68EHHzRz9T///HOzokmoGPr67EA3ISj9smGH2fCPpVt3S73BrwW6GUEhKfWcvDB/odmsyN+rj+gNhRbxzYouxz5y5EizBQO9WdIbJb2paNGihYwfP950vDlXttLVqSpWrOgqcDxmzBgZOnSoTJ8+XapVq+bKEihSpIjZNPZofZM77rjDjNRrXRSNKzVr1gyZFXYBINhjSXbPBQKxBADCd1VEn3ZsXSxr4MCBA7J7927X8zolRjux+vfvL6+//rpUqlTJLL1L8AAQ8oLr+39Q6dq1qxw5csTcYOiNhY7+awaZswiwxgld3cpJV93SFbB0Goy7YcOGmSwCnY7y+++/y/vvv28y0nSaZbt27WTUqFFMDwEQ2oglWSKWAED4xhKfdmxdbKRHO7cyO2fdunW+bBYAIMjoVJGspotokV53O3fuzPa1tCajTisBAIQXYgkAhKegqrEFAFZk1ZRfAID/EEsAAN5yWDSW0LEFAL5m0SKNAAA/IpYAALzlsGYsoWMLAHxORza8Gd0IzpERAIA/EUsAAN6yWTKW/FNBEQAAAAAAAAghZGwBgK9ZNOUXAOBHxBIAgLcc1owldGwBgK9ZNIAAAPyIWAIA8JbDmrGEqYgAAAAAAAAISWRsAYCv6bK43iyNG6TL6gIA/IhYAgDwlsOasYSOLQDwMYfjwubN+QCA8EYsAQB4y2HRWELHFgD4mkXnsgMA/IhYAgDwlsOasYQaWwAAAAAAAAhJZGwBgK9ZdC47AMCPiCUAAG85rBlL6NgCAB+zOS5s3pwPAAhvxBIAgLdsFo0lTEUEAAAAAABASCJjCwB8zaJFGgEAfkQsAQB4y2HNWELHFgD4mkXnsgMA/IhYAgDwlsOasYSOLQDwNYuOjAAA/IhYAgDwlsOasYQaWwAAAAAAAAhJZGwBgK9ZdGQEAOBHxBIAgLcc1owldGwBgK9ZNIAAAPyIWAIA8JbDmrGEqYgAAAAAAAAISWRsAYCvWXT1EQCAHxFLAADeclgzltCxBQA+ZnNc2Lw5HwAQ3oglAABv2SwaS3w6FXHx4sXSqVMniY+PF5vNJjNnzsz2+IULF5rj0m8HDx70ZTMBwD9z2b3ZLG7ixIlSrVo1iYmJkZYtW8rKlSuzPHbatGkZ4oSe587hcMjQoUOlQoUKEhsbK23btpWtW7f64Z0AgI8QSy6KWAIA4RlLfNqxlZSUJI0bNzZBJjc2b94sBw4ccG1ly5b1WRsBAIH12WefyYABA2TYsGGydu1aEzfat28vhw8fzvKcuLg4jzixa9cuj+dffvlleeONN2Ty5MmyYsUKKVy4sHnN5ORkP7wjAIC/EUsAIHz5tGOrQ4cO8vzzz8ttt92Wq/O0I6t8+fKuLSKCGvcAYFXjxo2T3r17S8+ePaVevXrmBqJQoULy3nvvZXmOjqy7x4ly5cp5jLCPHz9ennvuOencubM0atRIPvjgA9m/f/9FM4cBAKGJWAIA4Ssoe4yaNGliUn5vvPFG+fXXX7M9NiUlRRISEjw2AAgmNrf57HnaxLpSU1NlzZo1ZnqHkw5m6ONly5ZleV5iYqJUrVpVKleubG44Nm7c6Hpux44dZgq7+2sWK1bMTEvJ6jWJJQCCHbEk+GOJIp4ACGY2i8aSoCoer51ZOrrSvHlzExTeffddad26tUn9veyyyzI9Z/To0TJixIgM+w9e65CI2CCdABoABYqdDXQTgkqp4omBbkLQqRJ3ItBNCCrnklJlR369mEVXH8kPR48elbS0NI9RcqWPN23alOk5tWvXNiPwOnp+6tQpeeWVV+TKK680NySVKlVy1WXM7DWzqtmYVSy57Nc0iS4SlGNACALPl/1voJuAIJdw2i4lLg10K6wvWGKJIp4gL4gnyA6x5OKC6uqqAebf//63NGvWzAQWDTb687XXXsvynMGDB5tg5Nz27Nnj1zYDAPyrVatWcv/995vs3uuuu06+/vprKVOmjLz99tt5fk1iCYCg5xwk8WaDT2OJIp4ACGoOa8aSoMrYykyLFi1kyZIlWT4fHR1tNgAIWt6uIGLh5NPSpUtLZGSkHDp0yGO/PtZ6JzlRsGBBadq0qWzbts08dp6nr6GZwO6vqTcwmSGWAAh6xJKgjyWKeAIgqDmsGUuCKmMrM+vXr/cIJgAQciy6rG5+iIqKMlm68+fPd+2z2+3msY6m54ROP9mwYYMrVlSvXt3ckLi/ptY40WntOX1NAAg6xJIsEUsAILxjiU8ztrQgo3PUw1mEUTuqSpYsKVWqVDGpuvv27TMrjChdeUSDSP369c0yulpja8GCBfLjjz/6spkAgADS5dl79Ohh6itqlq7GgqSkJLOyldKpIhUrVjR1S9TIkSPliiuukJo1a8rJkydl7NixZon2Xr16uVa56tevn1mVt1atWiauDBkyROLj46VLly4Bfa8AAN8glgBA+PJpx9bq1aulTZs2HgFHadCZNm2aHDhwQHbv3u2xoslTTz1lOrt0eV4t5vjTTz95vAYAhBrnKiLenG9lXbt2lSNHjsjQoUNNQV6d4jFnzhxXwV6NE7q6ldOJEyfMku56bIkSJcwo/dKlS83y7k6DBg0yNzR9+vQxNyxXX321ec2YmJiAvEcA8BaxJHvEEgAI31hiczgcQdq0vNEUYV2Kt9JrIyUilqDjVKBYaqCbEFRYFTEjVkXMuCrirHZTTeHXuLg4r65H1Z5/QSK8+BJsT06Wnc/9x6u2IG//7Qb8eotEFykY6OYgSD1fdkOgm4CQWMnqb2JJGCOeICeIJ8gOscQCxeMBIORZtEgjAMCPiCUAAG85rBlLgr54PAAg57T4rdYA0VogsbGxUqNGDRk1apRYLDkXAAAAAAwytgDAQnPZx4wZI5MmTZL333/fLMShtQ61cK6mHj/xxBN5bwQAIKCsWhcFAOA/NovGEjq2AMDXHLYLmzfn55AWvu3cubN07NjRPK5WrZp88sknsnLlyrz/fgBAWMUSAIBFOawZS5iKCAAhQos+um8pKSkZjrnyyitl/vz5smXLFvP4t99+kyVLlkiHDh0C0GIAAAAA8C0ytgAgRIo0Vq5c2WP3sGHDZPjw4R77nnnmGdPpVadOHYmMjDQ1t1544QW55557vGgAACDgLFrwFwDgRw5rxhI6tgAgROay79mzx2NZ3ejo6AzHfv755/Lxxx/L9OnTTY2t9evXS79+/SQ+Pl569OiR90YAAALKqnVRAAD+Y7NoLGEqIgD4a2TEm03EdGq5b5l1bA0cONBkbXXr1k0aNmwo9913n/Tv319Gjx7t//cNAAi6WAIACGMO/8YSf63YTsYWAFjImTNnJCLCc8xCpyTa7faAtQkAAABA+BnjpxXb6dgCAF/zMuU3NyMjnTp1MjW1qlSpYoLHunXrZNy4cfLggw960QAAQDjFEgCARTn8G0v8tWI7HVsAYKEijW+++aZJ93300Ufl8OHDprbWv//9bxk6dKgXDQAABJxFC/4CAEIvliQkJHjs1hIpmZVJ0RXbp0yZYlZsv/TSS10rtuvAe36iYwsALKRo0aIyfvx4swEAAABAfsvJau3+XLGdji0A8DVG2QEA3iKWAACCJJbsycFq7f5csZ2OLQDwMasuqwsA8B9iCQAgWGJJ3P9Wab8Y9xXbla7avmvXLrNie352bHkunQUAAAAAAACEyIrtZGwBAAAAAAAgX/lrxXY6tgDA16iLAgDwFrEEABBiseRNP63YTscWAPgYdVEAAN4ilgAAQi2WFPXTiu3U2AIAAAAAAEBIImMLAPyBkXIAgLeIJQAAbznEcujYAgBfoy4KAMBbxBIAgLcc1owldGwBgI9RFwUA4C1iCQDAWzaLxhI6tnygcMGC8tTlV0v7arWkdGwh2Xj0sAxfukB+P3JQwsHlZSpL7zqtpEHJ8lIutqg8/MsXMm/fFo9j+jW4VrrWaCpxBaNlzdG9MnT1D7Iz8YRYVdMS1eTe6tdKnbiKUiYmTgau/VAWHf7T9XzrcvXl9sotpW5cRSkWVUju+fUN2Xr6gFhV3bia0jm+nVxSpIqUjCouYzZNklXHfzPPRdoipHuVztK0eAMpF1NazqSdlQ0nN8lHu2bIiXOnAt10AAAAAEAQ8Wnx+NGjR8vll19uKuGXLVtWunTpIps3b77oeV988YXUqVNHYmJipGHDhvL9999LKBlz3U1yTcVq0v/n76XdF9Nk8d6d8nHHu6VcoSISDgoViJJNJw/J8NVzM32+T51W0uPSy2XI6h/k9nnT5Mz5czK1dXeJiogUq4qJjDIdVWP//CbT52Mjo+S3EztlwpYfJBzERETLzqS98u7fn2Z4LjoiSqoXriJf7v1eBv32oozd9LbEx5aTZ+o+KiGf8uvNZnETJ06UatWqmet+y5YtZeXKlVke27p1a7HZbBm2jh07uo554IEHMjx/0003+endAIAPEEsuilgCAOEZS3yasbVo0SLp27ev6dw6f/68PPvss9KuXTv5888/pXDhwpmes3TpUunevbvpFLvllltk+vTppkNs7dq10qBBAwl20ZEFpEP1S6X33Bmy8sBes2/8mqXStmoNua9+E3ll1RKxukUHtpstKz1rt5CJG5fIT//L4vq/FbNkZZd+0q5SbZm9+58sJitZdnSL2bLyw/515meF2OISDtad3Gi2zJxJS5ZRf77use/dHZ/KmEaDpXRUCTmaGnqZfVZN+c0vn332mQwYMEAmT55sbkR0OeD27dubgRAdFEnv66+/ltTUVNfjY8eOSePGjeWuu+7yOE5vPqZOnep6HB0d7eN3AgC+QyzJHrEEAMI3lvg0Y2vOnDlmpKN+/fomUEybNk12794ta9asyfKc119/3QSQgQMHSt26dWXUqFFy2WWXyYQJEyQUFIiwSYGICElJO++xP/n8eWlevqKEu8qFi0vZ2CLy66Gdrn2J51Jk/bF90rQUnw8yVygyVuwOuySlnZWQZNGRkfwybtw46d27t/Ts2VPq1atnbkoKFSok7733XqbHlyxZUsqXL+/a5s2bZ45PfzOiNx/ux5UoUcJP7wgAQj+WLF68WDp16iTx8fEmU2nmzJmezXE4ZOjQoVKhQgWJjY2Vtm3bytatWyVQiCUAEL73JT7t2Erv1KlTrkCSlWXLlpnA6E5HW3R/KEg6d07WHNwnj1/WSsoWKiwRNpvcVqueXFYuXsqGyVTE7JSJuZCpdzQ5yWO/Pi4Ty+eDjAraCsi9VW+TX4+ulrNpyYFuDvKZjpbrYIf7dT8iIsI8zul1/7///a9069YtQybwwoULzSh97dq15ZFHHjGj8QCAnElKSjID0zq9LzMvv/yyvPHGG6YDacWKFeYarN/Zk5P9H6uJJQAQ3vxWPN5ut0u/fv3kqquuynZK4cGDB6VcuXIe+/Sx7s9MSkqK2ZwSEhIk0Pr9/L2Mve4mWXXfo3Lebpc/jh6SWds3ScPSnu8LQPa0kPyA2r3FJjaZ8vd0CVkWXVY3Pxw9elTS0tIyve5v2rTpoudr/ZQ//vjD3JC408zf22+/XapXry7bt283U+E7dOhgbnAiIyNDIpYAQCBjiV4zdcv0pRwOM9Xvueeek86dO5t9H3zwgbl2a2aXdhCFYyxRxBMAQc1hzfsSv3Vsaa0tDRhLluRvjSmtxTVixAgJJrsTTkrXbz+V2AIFpWhUlBw+kyQT2nYy+8Pdkf9lapWOKSxHkhNd+/XxXycOBbBlCMpOrUv7SJnoUjJ842shna1l1bnswUBvQnSRkRYtWnjsd7+p0ucbNWokNWrUMCPvN9xwQ0jEEgDwRSxJ39GiU+1yWzdqx44dZtDZPUOqWLFipraVdvr4u2MrWGKJIp4ACGY2i96X+GUq4mOPPSazZ8+Wn3/+WSpVqpTtsTp3/dAhzw4Ofaz7MzN48GAzxdG57dmzR4LF2fPnTKdWXFS0XFupmvy4a5uEuz1JJ+Xw2US5slw1174iBaKkSamKsu7YvoC2DcHXqVUhtoyM3DheEs97Tl2FdZQuXdqMeufmuu8+TebTTz+Vhx566KK/55JLLjG/a9u2bSEXSwAgP1WuXNl0Qjk37YjJLedMitzMsgiHWKKIJwBgsYwtTVN+/PHHZcaMGWZkQ9N4L6ZVq1Yyf/58M23RSYs56v7M5GWUyde0E8tmE/n75AmpGldcnr2itWw/eVy+2PyHhINCBQpK1SL/1FGrVLi41C1eTk6mnpUDZxJk6uaV0rf+VbLz9HHT0TWg4XVy6Oxp+XHvZrGq2MgoqVSolOtxfGwJqVW0giScOyOHkk9JXMFYKRdTXMpEx5nnqxYubX4eTzktx1L/yWyzipiIaCkfU8b1uFx0aalWqJLpwDpx7pT8X+1/S/XClWX0XxMlwhYhxQte+Fz0+fOONAk5Fk35zQ9RUVHSrFkzc93XFXCdU9f1sQ6KZOeLL74w0z3uvffei/6evXv3mrooWuQ4VGIJAPgilmhHS1zchbiqrHDtC5ZYoognAIKaw5r3JQV8Pf1w+vTp8s0330jRokVdIzg6OqSrp6j7779fKlas6BotevLJJ+W6666TV199VTp27GhGUFavXi1TpkyRUFE0KlqebnGtlC9SRE4lJ8sPO7bI2FW/mHpb4aBhyQoy/fr7XI+fu+xG8/OrHb/JoBWzZcqmZabz64XLb5a4qBhZfWSP9Fz0qaTaQ7DDIofqFqsok1v0cT3uX/cW83P2vjUycsOXck3ZujKs4T+r8LzY5F/m5zvbfpJ3ts0Xq6lRpKqMaDDA9fiB6hfe+8+Hl8nne2bL5SUbm8evNhnicd6wP8bJxoQtEnIsGkDyiy7P3qNHD2nevLmZBqJ1W3QEXVe2yixOuE8d0RuYUqX+6TRWiYmJZhrIHXfcYUbqtS7KoEGDpGbNmqawMQCEpHyKJdqp5d6xlRfOLCjNiHLv5NHHTZo0kUAglgBA+N6X+LRja9KkSeZn69atPfZPnTpVHnjgAfPv3bt3m1VLnK688krTGabFKLVAY61atUwRyuwKzgeb7/7ebLZwteLwbqnx6QvZHjP+j8VmCxdrj++QFnMGZ/n8d/vWmi1caOfUnUsfzvL57J4LRVady55funbtKkeOHDHLxusAiN4UzZkzxzXFJX2cUJs3bzY1G3/88ccMr6fTUX7//Xd5//335eTJk2ap+nbt2smoUaMYRQcQsoIplugsDO3s0YwoZ0eW1u7S1RF15cBAIJYAQGjFkpCaingxOkUxvbvuustsAIDwoFNFspouklmc0GXXs4oxmhE8d+7cfG8jAIQTzVhyryWlBePXr18vJUuWlCpVqpiyIc8//7wZhNaOriFDhpjOH+dUwEAglgBAePLbqogAELYsmvILALBuLNFSIG3atPGY6qd0ut+0adPMtDyd6tenTx+T0XT11VebDKmYmBgvGgkA8CmHNe9L6NgCAB+zasovAMC6sURLiWQ3+8Jms8nIkSPNBgAIDTaL3pd4TjQHAAAAAAAAQgQZWwDgaxZN+QUA+BGxBADgLYc1YwkdWwDgaxYNIAAAPyKWAAC85bBmLKFjCwB8zPa/zZvzAQDhjVgCAPCWzaKxhBpbAAAAAAAACElkbAGAr1k05RcA4EfEEgCAtxzWjCV0bAGAj1l1WV0AgP8QSwAA3rJZNJYwFREAAAAAAAAhiYwtAPA1i6b8AgD8iFgCAPCWw5qxhI4tAPCHIA0CAIAQQiwBAHjLIZZDxxYA+JhV57IDAPyHWAIA8JbNorGEGlsAAAAAAAAISXRsAYC/5rJ7s+XCvn375N5775VSpUpJbGysNGzYUFavXu2rdwcAsGAsAQBYkMOasYSpiABgoZTfEydOyFVXXSVt2rSRH374QcqUKSNbt26VEiVK5L0BAICAs+r0EQCA/9gsGkvo2AIACxkzZoxUrlxZpk6d6tpXvXr1gLYJAAAAAHyFqYgAYKGU31mzZknz5s3lrrvukrJly0rTpk3lnXfe8eW7AwD4g0WnjwAA/MhhzVhCxxYA+Cnl15tNJSQkeGwpKSkZftfff/8tkyZNklq1asncuXPlkUcekSeeeELef/99/79xAEDQxRIAQPiyWTSW0LEFACEyMqJTDIsVK+baRo8eneFX2e12ueyyy+TFF1802Vp9+vSR3r17y+TJk/3/vgEA+ceio+wAAD9yWDOWUGMLAELEnj17JC4uzvU4Ojo6wzEVKlSQevXqeeyrW7eufPXVV35pIwAAAAD4Ex1bAOBr3o5u/O9c7dRy79jKjK6IuHnzZo99W7ZskapVq3rRAACAVWIJACCMOawZS+jYAgALLavbv39/ufLKK81UxLvvvltWrlwpU6ZMMRuQE9UK15NrynSWirE1JK5gSflw50vyV8LKTI/tXPHf0rJUe5m9/z1ZenS239uKACl4udgK9xIpWF9skeXEfuIRkZSfPI+JrCG2ogNFolroA5G0beI48ZiI/UCgWh3yrLpEO6wbL+rHtZQWpdqb5wsVKCpvbhkgB5J3Zvual5VoI3dWftxj3zl7qgz7o5vP3gcCGzNsRR4XiekoElFB/2uLnPtDHImviZz7LcuX1HNsRZ7w2Oc4v10cR2/y6VuxAptFYwkdWwBgoZGRyy+/XGbMmCGDBw+WkSNHSvXq1WX8+PFyzz33eNEAhJOoiGg5eHanrDm+QO6t9nSWx9WLaymVC10qp84d82v7EARssSLnN4nj7JdiK/FWxucjq4it1CciZ74UR+IbIo5EkQI1RSTjghfIBYuOssO68aJgRIzsSvpLNpxaKrdXejTHr5ucliTjNrt3bvHHa+WY4Ti/UyRhpEjaHhFbtNgK9RRbianiONJWxHE8y5d1nNsijhM93Hak+eodWIvDmrHEp8XjtbCx3mQVLVrULDvfpUuXDFNk0ps2bZrYbDaPLSYmxpfNBABLueWWW2TDhg2SnJwsf/31lykeH8wWL14snTp1kvj4eHPNnzlz5kXPWbhwoSmSr3XGatasaWJHehMnTpRq1aqZGNKyZUuTvYaL23J6ncw79In8mbAiy2PiCpSUTvG95PPd48XOF8nwk7r4wmh6yrxMn7YV6S+SskgciS+LnP9TJG23SMoCEXvWNyiAt4glwRcv1p9cJAsOfyHbTmedeZPVfXPi+ZNu26l8ajGCMWZI8rciqUsvdGyd3yaO06PFFlFUpGDti7xwmoj96D+b44QvWo8Q4dOOrUWLFknfvn1l+fLlMm/ePDl37py0a9dOkpKSsj1Pa8gcOHDAte3atcuXzQQAn7I5HF5vVqYxoXHjxubmISd27NghHTt2lDZt2sj69eulX79+0qtXL5k7d67rmM8++0wGDBggw4YNk7Vr15rXb9++vRw+fNiH7yQ82MQmd1V5Un45MlMOp+wJdHMQdGwi0a3NCLytxHtiK7NcbCW/FIluG+iGhTxiSfaIJdYRFREjA+u8LYPqTJF7qz4jZaMrB7pJ8JuCIrFdxWFPEDm3KftDI6uKrcwSsZVeILZir/5vKiPCNZb4dCrinDlzPB7rKIhmbq1Zs0auvfbaLM/TUZby5cv7smkA4D8WTfnNLx06dDBbTk2ePNlMsXz11Vddqz4uWbJEXnvtNXPDocaNG2cy1Xr27Ok657vvvpP33ntPnnnmGR+9k/BwbZnbTJbW0mPfBbopCEYRpcQWUUSkcJ8LI/Snx4pEXyO24hPFcfw+kXNku+QZsSRbxBJrOJKyT77eM1EOJu+UmMhCcnWZzvJwzRdl/JZ+ksDUd+uKbiO2Yq9dmLZoPyyO4w9km4HlSP1N5NzTImk7RCLKXKi5VeoTcRztKOLIPokm7DmsGUt8mrGV3qlTF9JIS5Ysme1xiYmJZgWvypUrS+fOnWXjxo1ZHpuSkiIJCQkeGwDAupYtWyZt23pmf+hNiO5XqampZgDF/ZiIiAjz2HlMesSSnImPvUSuLN1RvtzzZqCbgqD1v6+WKfNFzkwTOf+XSNIUkZSfxVaoe6AbB/g0lijiiXf2nNki604uNEXmdyT9KR/vfFmSzidIi5LtAt00+FLqcnEcu1Ucx7uKpPwituKvi0Rk02eQulgkZY7I+c0iqUvEcaKXiC1OJCbnnduwFr8Vj7fb7SbFV5eib9CgQZbH1a5d24yCNGrUyHSEvfLKK2aFL+3cqlSpUqZ1vEaMGJFh/90tVkp0kYL5/j5CVYPYvYFuQlBpGL0/0E0IOvWjYgPdhKCScNouJfLptay6+kigHDx4UMqVK+exTx/rzcPZs2flxIkTkpaWlukxmzZlntaeVSxBxhWwChcoJoPq/rPKZqQtUm6u0EOuKn2LjN30cEDbhyBgPyEOxzlxnN/muf/8dpGoZoFqlSUQS4I/lijiSf6yS5rsP7tDSkUzm8fSHGcv1GNM2y2Oc+vFVnqeSOxdIklv5/D80yZ7yxZZNVgTioKGzaKxxG8dW1pr648//jApvtlp1aqV2Zy0U0tTg99++20ZNWpUhuN15S+d++6kwUgzvQAgaFg05ddKiCU5s+7EQtl++nePfQ9cMkTWn1gka04sCFi7EEx0qfYNYitQ3fPSVaCaSBqDSl4hloQE4kn+skmElI+pIptPrw10U+BXEWKzReX8smUrZFbkddi/8W2zrMBhzVjil46txx57TGbPnm1WK8ks6yo7BQsWlKZNm8q2belG/v5HVzHRDQCClVVHRgJFazAeOnTIY58+1oVHYmNjJTIy0myZHZNV/UZiiWfR3lJR/3xOJaPKSoWYanImLVFOnTsqZ9MSPY7Xelunz5+Uoyl0WoQNcwNR9Z/HkZVECtQVsZ8UsR8QR9K7Yis+XiR1lZleItHXikRfL47j9way1SGPWBL8sSTc4snF4kVsZBEpXrC0FC14YUpZ6eiK5ufp/612qO6s/ISpnfXjwY/N4+vL3iW7z2yRY6kHJTaysFxTprMUjyojq4//FJD3CB/HDMdJsRV+RBy6cm7aYZGIEmIrdK9IZDlxJP/wz0uUeF8cuqrimY8uPC76tDiSfxax7xOJKCu2Ik+a/D45OzsQ7zCk2CwaS3zaseVwOOTxxx+XGTNmmOV0tUBjbmkKsC5bf/PNN/ukjQCA0KJZvd9//73HPl1515ntGxUVJc2aNZP58+dLly5dXNPh9bEOtCB7FWNrSO8a/2RId4x/0Pxcc3yBfLV3QgBbhqBRsIFElLxwE6oi4v5jfjrOfi2OU0+bJd0dCcPEVvjfInFDRM7vEMfJx0TOrQlgowFPxBLfx4u6cZfLnZUfdz3fvepT5uf8Q5+ZTWnHl8Nhdx0TE1lEbqv0qBQtUNwMpOw7+7dM3vasHE6hrIo1Y8YQkQI1xBZ724WaWvYTJuvXcay7iPuU9gJVxHauxD/JQhHlxVZ8nOkIE/txkdTV4jh2l4jjuJ/fHMKiY0unH06fPl2++eYbKVq0qJnLrooVK2ZGQtT9998vFStWNPPR1ciRI+WKK66QmjVrysmTJ2Xs2LGya9cus/wuAIQki6b85hddMMQ9K1eXYNel13WhkSpVqphpHfv27ZMPPvjAPP/www/LhAkTZNCgQfLggw/KggUL5PPPPzcrVTnpNJAePXpI8+bNpUWLFjJ+/HizFLxzZStkbUfSRnn299tzfDx1tcJQ6kqxH6yV/TFnvxTH2S/91aLwQCzJFrEk+OLF2hM/my077/491OPx9wemmg3hEzMcJ/te9CUcR9p4Pj7VP1+aFpYc1owlPu3YmjRpkvnZunVrj/1Tp06VBx54wPx79+7dZoURJy3UqMvqaidYiRIlzEjJ0qVLpV69er5sKgD4jFVTfvPL6tWrpU2bf76wOGuT6M3EtGnT5MCBAyZWOGn2r9549O/fX15//XUzxf3dd991Lc+uunbtKkeOHJGhQ4eaeNKkSROZM2dOhiLAABAqiCXZI5YAQPjGEp9PRbwYnaLo7rXXXjMbACA86OBHdvFCb0gyO2fdunXZvq5OFWG6CACEB2IJAIQvv62KCABhy6IpvwAAPyKWAAC85bBmLKFjCwD8IFjTdgEAoYNYAgDwls2CsYSOLQDwNZ0akYOp2dmeDwAIb8QSAIC3HNaMJf9UbQcAAAAAAABCCBlbAOBjVl19BADgP8QSAIC3bBaNJXRsAYCvWbRIIwDAj4glAABvOawZS5iKCAAAAAAAgJBExhYA+JjNfmHz5nwAQHgjlgAAvGWzaCyhYwsAfM2iKb8AAD8ilgAAvOWwZixhKiIA+KlIozcbACC8EUsAAKEYS/bt2yf33nuvlCpVSmJjY6Vhw4ayevXqfH1fZGwBAAAAAAAgX504cUKuuuoqadOmjfzwww9SpkwZ2bp1q5QoUSJffw8dWwDgaw7Hhc2b8wEA4Y1YAgAIsVgyZswYqVy5skydOtW1r3r16pLfmIoIAD7G9BEAgLeIJQCAUIsls2bNkubNm8tdd90lZcuWlaZNm8o777yT7++Lji0AAAAgVAr+erMBAJAPEhISPLaUlJRMj/v7779l0qRJUqtWLZk7d6488sgj8sQTT8j7778v+YmOLQDwNW5GAAAAAFjkvqRy5cpSrFgx1zZ69OhMf53dbpfLLrtMXnzxRZOt1adPH+ndu7dMnjw5X98WNbYAwMe8nQLC9BEAALEEABAssWTPnj0SFxfn2h8dHZ3p8RUqVJB69ep57Ktbt6589dVXkp/o2AIAX6PgLwDAW8QSAECQxJK4uDiPjq2s6IqImzdv9ti3ZcsWqVq1quQnpiICAAAAAAAgX/Xv31+WL19upiJu27ZNpk+fLlOmTJG+ffvm6+8hYwsAfIzpIwAAbxFLAAChFksuv/xymTFjhgwePFhGjhwp1atXl/Hjx8s999wj+YmOLQDwNW8LwHMzAgAglgAAQjCW3HLLLWbzJTq2AMDHGGUHAHiLWAIA8JbNorGEji0vVStcT64p01kqxtaQuIIl5cOdL8lfCStdz99Qrqs0KnaVFIsqLWn287Lv7Hb58eB02Xt2q1hVmZimUq/EvVIipo4UKlBGFu8fKHuTFmV67OVln5FaxW6XNUfGyeaTn4pVFYpuKWXiHpbYgg2lYIHysuvIQ5Jwdq7r+Uolx0mJInd7nHP67ELZeeResaSCl4utcC+RgvXFFllO7CceEUn5yfOYyBpiKzpQJKqFPhBJ2yaOE4+J2A8EqtUAAAAAgCBDx5aXoiKi5eDZnbLm+AK5t9rTGZ4/mrJfZu1/V46nHpKCtii5qkwnefCSofLqpr6SlJYgVlQgIkZOpG6V7QnfyrXxL2d5XKXCraV0TAM5c/6wWF2ErZAkp/4pJxI/k6pl3s30mNNnf5a9xwa4HtsdqWJZtliR85vEcfZLsZV4K+PzkVXEVuoTkTNfiiPxDRFHokiBmiKSIiHJ7riweXM+ACC8EUsAAN6yWzOW+HRVxEmTJkmjRo1cS0G2atVKfvjhh2zP+eKLL6ROnToSExMjDRs2lO+//16C2ZbT62TeoU/kz4QVmT7/28lfZHvi73Ii9ZAcTtkj3++fKjGRhaV8bP4ubxlMDpxZJr8fmyx7kxZmeUxsZBlpXuYpWXpwqNgd58XqEpN/lkOnxkrC2TlZHmN3pMh5+xHXZnecEstKXSyOxNdEUuZl+rStSH+RlEXiSHxZ5PyfImm7RVIWiNiPS0jPZfdms7DFixdLp06dJD4+Xmw2m8ycOTPb47/++mu58cYbpUyZMq7YMnfuPxmQavjw4ea13DeNLQAQsogl2SKWAED4xhKfdmxVqlRJXnrpJVmzZo2sXr1arr/+euncubNs3Lgx0+OXLl0q3bt3l4ceekjWrVsnXbp0Mdsff/whVhBpKyCXl2wnZ9OS5MDZnRK+bNKq/Aj56+RHcir170A3JmgUiWkldSuul0srLJL4Ei9KZERxCU82kejW4ji/U2wl3hNbmeViK/mlSHTbQDcMPpKUlCSNGzeWiRMn5vjmRW9GdOBD40ubNm3MzYzGDXf169eXAwcOuLYlS5b46B0AAAKNWAIA4cunUxE1OLh74YUXTBbX8uXLTZBI7/XXX5ebbrpJBg4caB6PGjVK5s2bJxMmTJDJkydLqKpdtJl0qzJACkZEy+nzJ+S9v0fImbTTEq7qlbhfHI7zsvnkZ4FuStA4nbxQTp39QVLP75HoAlWlXPGnpVrUR7L90K2ayyVhJaKU2CKKiBTucyGr6/RYkehrxFZ8ojiO3ydy7p8adqHC5mWhRT3fyjp06GC2nNIlgt29+OKL8s0338i3334rTZs2de0vUKCAlC9fPl/bCgCBQizJHrEEAMI3lvg0Y8tdWlqafPrpp2Y0RVN9M7Ns2TJp29YzK6N9+/Zmf1ZSUlIkISHBYws2fyf+IW9ufUre3v6sbD29TrpXfUoKRxaTcFQiuo7ULt5Nlh8aGeimBJVTZ2bJ6bPzJOXcJlNUfufhB6RQdBMpHJ35/yvW9r/LUsp8kTPTRM7/JZI0RSTlZ7EV6i4hyeHwfkOW7Ha7nD59WkqWLOmxf+vWrWZKyiWXXCL33HOP7N69O6RjCYAwRywJ+liiiCcAgprDmrHE5x1bGzZskCJFikh0dLQ8/PDDMmPGDKlXr16mxx48eFDKlSvnsU8f6/6sjB49WooVK+baKleuLMHmnCNFjqcelD1ntsjXe98SuyNNmpe8QcJR2dgmEhNZQjpXnyXdai41W5GC8dK09JNya7XsayGEk3Npu+V82jGJLlhNwo79hDgc58Rxfpvn/vPbRSIrSCgvq+vNllc6HVxrgvTr10+s6pVXXpHExES5++5/VhZt2bKlTJs2TebMmWMyhXfs2CHXXHONuWkJ1VgCILz5O5booPSQIUOkevXqEhsbKzVq1DCzKRxBelMTDLFEEU8ABDNbAO9LQnpVxNq1a8v69evl1KlT8uWXX0qPHj1k0aJFWXZu5dbgwYNlwIB/VpLTUZFgDyA2iZACEQUlHO1I+EEOnvGcStam4htm/98J3wasXcGmQGQFiYwoIefSrL9iZEbnRM5tEFuB6p61CQtUE0nbH7hmhaBVq1bJ22+/bRbxsKrp06fLiBEjzPSRsmXLuva7T0fR9683J1WrVpXPP//c1HG0QiwBAF8aM2aM6cx5//33TQkRrZfbs2dP01nzxBNPiJXkVyxRxBMAsGDHVlRUlNSsWdP8u1mzZuZGS2tp6c1Wejp//dChQx779HF289o1E0y3QImKiJFSUf+0r2RUWakQU03OpCXKmfOnpU25O+WvhFVy+twJKVSgqFxRqoPEFSwpG04uFasqYIuVIgUruR4XLhgvxaNqSao9Qc6cPySpqZ6r/emqiMlpx+T0uexTu0NZhK2QRGnHzP8ULFBZYgrWkzT7SbOVLTZATp35Xs6nHZaoAlWlQon/SOr5nZJ4dpFYkq2QSKTbyqCRlUQK1BWxnxSxHxBH0rtiKz5eJHWVSOpykehrRaKvF8fxeyUkebuCSB7O1VFnnTLxzjvvyPPPPy9WpNPbe/XqZVbTTT+NPb3ixYvLpZdeKtu2pcsEDJJYAgDBFkt0USdd9Kljx47mcbVq1eSTTz6RlStDr9alv2KJIp4ACGoO/9+XWKJjK7P56zr3PDNae2v+/PkeU2a0eHxWNbmCQcXYGtK7xijX447xD5qfa44vkG/2vS1loitK06qtpXBknCkYv/fMNpmy/Tk5nLJHrKpkTF1pW+mfYv/NyvQ3P/9OmB22tbVioxrLJeW+cD2OLzHc/DyR+LnsO/GsxBSsIyXK3CkREXFyPu2QJCYvlkMnx4pDUsWSCjaQiJIfux5GxP3H/HSc/Vocp54WSZknjoRhYiv8b5G4ISLnd4jj5GMi59ZIKLI5HGbz5nyVvk5Hdl+e+/bta25G9Eu6FTu29ObqwQcfNDckzpuui3X0bd++Xe677z6/tA8AQj2WXHnllTJlyhTZsmWL6cz57bffzIqA48aNE6sglgAIN7Z8iiVh1bGlqbiawlulShUzF13TfBcuXChz5841z99///1SsWJFMxddPfnkk3LdddfJq6++aoKLBhlNe9agGqx2JG2UZ3+/PcvnP971soSbw2fXyvStLXJ8/KydXcTqklKWyYbd/2SxpbfzSIhmIuVV6kqxH6yV/TFnvxTH2S/91aKQkH4qw7Bhw2T48AudpO702rl27VqTIRsK9EbBffRba5joFHYt4KvxQ2PJvn375IMPPjDPayzRae2a/avTQpx1GLUGjE6RUf/3f/9nVubVKSP79+83n1VkZKR07x6iCxAAgJ9jyTPPPGM6werUqWOun1pzS1c412zgYEQsAYDw5dOOrcOHD5vOqwMHDpgAoXPTtVPrxhtvNM/rqiIREREeI0MaZJ577jl59tlnpVatWjJz5kxp0KCBL5sJAL5l/9/mzfkismfPHomLi3PtzmyEXY/RQQLNdo2JiZFQoAMYbdq0cT121ibRGw4t2qsxxH0VKh3sOH/+vMlK083Jebzau3evufE4duyYlClTRq6++mpZvny5+TcAhCQ/xhKldaQ+/vhj891ca2xpJ5HOqtAVAvV6G2yIJQDgv1gSVh1b//3vf7N9XrO30rvrrrvMBgBWkV8pv3oj4n4zkpk1a9aYQYXLLrvMtU9H2RcvXiwTJkwwU8F1tDmYtG7dOttVtpw3GNnFjsyy1gDASvwZS9TAgQNN1la3bt3M44YNG8quXbvMTItg7NgilgDAxTEVEQAQ9EUab7jhBtmwYYPHPl3FSqeSPP3000HXqQUACM6Cv2fOnPGYWaE0hmi9XABAiHJQPB4AEOSKFi2aYfp24cKFpVSpUkzrBgDkmNaW0ppaWp9KpyKuW7fOFI7XYusAAAQTOrYAwNc0ZdebtN0gTfkFAFg3lrz55psyZMgQefTRR80Ud62t9e9//1uGDh2a9zYAAALLYc37Ejq2AMDHbI4LmzfneyMndUQAAMHN37FEM4DHjx9vNgCANdgCfF/iK54T5wEAAAAAAIAQQcYWAPiaRVN+AQB+RCwBAHjLYc1YQscWAPiYzX5h8+Z8AEB4I5YAALxls2gsoWMLAHzNoiMjAAA/IpYAALzlsGYsocYWAAAAAAAAQhIZWwDgazqw4c3gRnAOjAAA/IlYAgDwlsOasYSOLQDwMZvDYTZvzgcAhDdiCQDAWzaLxhI6tgDA1yw6lx0A4EfEEgCAtxzWjCXU2AIAAAAAAEBIImMLAHxNBza8WRo3OAdGAAD+RCwBAHjLYc1YQscWAPiYVeeyAwD8h1gCAPCWzaKxhKmIAAAAAAAACElkbAGAX5bV9aZIY342BgAQkoglAABvOawZS+jYAgBfs+jqIwAAPyKWAAC85bBmLKFjCwB8TQs02rw8HwAQ3oglAABv2a0ZS6ixBQAAAAAAgJBExhYA+JhVVx8BAPgPsQQA4C2bRWMJHVsA4GsWncsOAPAjYgkAwFsOa8YSpiICAAAAAAAgJPm0Y2vSpEnSqFEjiYuLM1urVq3khx9+yPL4adOmic1m89hiYmJ82UQA8N/IiDebhS1evFg6deok8fHx5ro/c+bMbI9fuHBhhlih28GDBz2OmzhxolSrVs3EkZYtW8rKlSt9/E4AwIeIJdkilgBA+MYSn3ZsVapUSV566SVZs2aNrF69Wq6//nrp3LmzbNy4MctztAPswIEDrm3Xrl2+bCIA+J5FA0h+SUpKksaNG5ubh9zYvHmzR7woW7as67nPPvtMBgwYIMOGDZO1a9ea12/fvr0cPnzYB+8AAPyAWJItYgkAhG8s8WmNLR01cffCCy+YLK7ly5dL/fr1Mz1HR0rKly/vy2YBgH9ZdFnd/NKhQwez5ZbefBQvXjzT58aNGye9e/eWnj17mseTJ0+W7777Tt577z155plnvG4zAPgdsSRbxBIACN9Y4rcaW2lpafLpp5+a0RSdkpiVxMREqVq1qlSuXPmi2V0AgPDVpEkTqVChgtx4443y66+/uvanpqaaTOG2bdu69kVERJjHy5YtC1BrAQDBiFgCAKHP56sibtiwwXRkJScnS5EiRWTGjBlSr169TI+tXbu2GQHRulynTp2SV155Ra688krTuaXTGjOTkpJiNic9z+xPOuejdxSazqSlBboJQSUxNUi7mgMoIYrPxF1C4oXPw5EP6bZWXVY3UPQGREfNmzdvbq7/7777rrRu3VpWrFghl112mRw9etQMppQrV87jPH28adOmTF+TWIK8SIjluonsEUvCK5Yo4gnygniC7BBLgqBjSzur1q9fby7qX375pfTo0UMWLVqUaeeWdoC5Z3Npp1bdunXl7bffllGjRmX6+qNHj5YRI0Zk2D+x3dx8ficAwtHp06elWLFi3r2IRZfVDRSNK7q5x4rt27fLa6+9Jh9++GGeXpNYgrwYF+gGIGQcO3aMWBIGsUQRT5AXxBPkBLEkgB1bUVFRUrNmTfPvZs2ayapVq+T11183nVUXU7BgQWnatKls27Yty2MGDx5sijo62e12OX78uJQqVcrU6wqkhIQEM6Vyz549pih+uOPzyIjPJHg/Dx0R0U4tXV0Jwa9FixayZMkS8+/SpUtLZGSkHDp0yOMYfZxVDcf0seTkyZNmWvzu3bu9/wJhEcH0/2ew4DPxxOeRkQ7sVqlSRUqWLBnopsAPsUQRT7LHdSIjPpOM+Ew8EUuCoGMrPe14ck/PzY6m/+pUxptvvjnLY6Kjo83mLqsCkIGi/zPyP+Q/+Dwy4jMJzs8j376A2h2at+vd+ciWZgbrtBLngIoOpMyfP1+6dOniij36+LHHHstxLHH+DQTD32IwCZb/P4MJn4knPo+MtDaT14glQR9LFPEkZ7hOZMRnkhGfiSdiSYA6tnTEQlcn0d5FzXyYPn26LFy4UObOvZCKe//990vFihVNyq4aOXKkXHHFFSbDS0c3xo4dK7t27ZJevXr5spkA4FsWTfnNL7poiHtm7o4dO8zNhY5KafzQWLJv3z754IMPzPPjx4+X6tWrm9V1tX6j1kVZsGCB/Pjjj67X0NFynfqutVN0BF7P0cVLnCtbAUDIIZZki1gCAOEbS3zasXX48GHTeXXgwAEzSqFF4bVTS1cdUZqS697reOLECbOk7sGDB6VEiRJmlGTp0qVZFpsHgNDgZQDR8y1s9erV0qZNG9dj5xQOvZmYNm2aiSEaL9xXqnrqqafMDUqhQoVMbPnpp588XqNr165y5MgRGTp0qIkpuurVnDlzMhQBBoDQQSzJDrEEAMI3lvi0Y+u///1vts9r9pY7Ldaom1VoGvKwYcMyTUcOR3weGfGZeOLzCE+6ClV2q7zoDYm7QYMGme1idKpIdtNFssPfYkZ8JhnxmXji88iIzyS8Y4nib8ATn0dGfCYZ8Zl44vO4OJsjP9aMBABkWvhSs1XbVn9cCkTkPRCdt6fITzveNIUjqTMAAOGFWAIA8FaCxWOJ34vHA0DYMUUWrVekEQDgR8QSAIC37NaMJflQVh8AAAAAAADwPzK2AMDXHPYLmzfnAwDCG7EEAOAthzVjCRlbPjJx4kSpVq2axMTESMuWLWXlypUSrhYvXiydOnWS+Ph4sdlsMnPmTAlno0ePlssvv1yKFi0qZcuWlS5dusjmzZslnE2aNMmsRqTztHVr1aqV/PDDD2K5ZXW92eBzx48fl3vuucf8DRYvXlweeughs3z8xYoV63XNfXv44YclXGLXF198IXXq1DHHN2zYUL7//nuxmtx8JlqcOv3fg54XzvFcFwq67LLLTMHbmjVrZijgHW6fiX4e6f9GdNMV9y6KWBISiCXEkswQS/5BLMmIWOI9OrZ84LPPPjNLDOvKBWvXrpXGjRtL+/bt5fDhwxKOkpKSzGegF3SILFq0SPr27SvLly+XefPmyblz56Rdu3bmcwpXlSpVkpdeeknWrFljluu+/vrrpXPnzrJx40axBJ2L7u0Gn9MbEf2b0/8vZ8+ebb5k9OnT56Ln9e7d2ywj79xefvllCYfYtXTpUunevbu5aVu3bp3ppNftjz/+kHCO53oz6/73sGvXLgnXeL5jxw7p2LGjtGnTRtavXy/9+vWTXr16ydy5cyXcv+PogJb734kOdF0UsSQkEEuIJekRSzwRSzIilniPVRF9QHvhNSNnwoQJ5rHdbpfKlSvL448/Ls8884yEM+1JnjFjhglYuODIkSPmIqQdXtdee22gmxM0SpYsKWPHjjVfdEJ+9ZGKD3u/+si+yTlafUQzAr/++mvZtGmTxMbGypVXXiljxoyR2rVr5/n3h4O//vpL6tWrJ6tWrZLmzZubfXPmzJGbb75Z9u7da0bQshplb9KkiYwfP17CLXZ17drVfBHTGzenK664wnwekydPFivI7WeiI8j6hfvkyZNidTmJ508//bR89913Hjeo3bp1M5+P/v8Vjp+JjrLrzdmJEydMNk+wxhLkDbGEWJIZYknWiCUZEUvyhoytfJaammqyTtq2bevaFxERYR4vW7YsoG1DcNKLgrMjByJpaWny6aefmi85OiXREvyY8ktGYN7o9Vm/GDhvRJRet/X6vWLFimzP/fjjj6V06dLSoEEDGTx4sJw5c0bCIXbpfvfjlY5AWyXW5TWe65SjqlWrmpsWS2We5oHV/0a8oTftFSpUkBtvvFF+/fXXnJ1k0ekjVkIsIZakRyzxntX/RrxBLPkHxePz2dGjR82Nebly5Tz262PNoADc6YiNjshcddVV5otMONuwYYPpyEpOTpYiRYqYkQod9bQEs6quF0EgF6emH7nSUT/NCNQvVWQEZk1rEqRP3y5QoIDpcM6uXsG//vUv88VTR+F///13M6qoaeGaNWf12KWfS2bH56i+g0U/E82MfO+990zNQB20eOWVV0zWpN6Q6JTrcJPV34iOGp89e9ZklYYbvQHRLBTt+EhJSZF3333XZOtop4fWjwmWWIK8IZYQS9IjlniPWJIRsSQjOraAANLMGk2rXbJkiYQ7DeI6b14D+Jdffik9evQw2UeW6NzydnTDi3PDPSNQU/x1KubFpo7klXvdFC14q180brjhBtm+fbvUqFEjz6+L0KSd8+6ZpnojUrduXXn77bdl1KhRAW0bgifWuU8N178RvV689tpr8uGHHwZtLAl3xBL4E7EEF0MsyYiOrXymKcSRkZFy6NAhj/36uHz58gFrF4LPY4895ioqGo6jL+lFRUWZVU5Us2bNTH2K119/3QRxXKAjU+50ZRjdskJGoMhTTz0lDzzwQLbHXHLJJeb6nL6I6/nz583qVrm5dmsdDbVt27aQuhnJS+zS/VaOdfkRzwsWLChNmzY1fw/hKKu/Ea3JEY4j7Flp0aIFA1xBjliSM8SSjIgl3iOW5EyLMI8l1Njywc253pTPnz/f4+ZSH1umXhC8ous1aKeWTrVbsGCBVK9ePdBNCkr6/42m1lqC3e79JmLqLGjRR+emheJzkhGoNcvCVZkyZcwS4tltet3W67MWIdUpm076/6f+HTpvMHJCsw6VjrZbPXbpfvfjldZ1s0qsy494rtNPdJp1qP095Ber/43kF71u5OhvJJ9iCXKPWJIzxJKMiCXes/rfSH5ZH+axhIwtH9DlXHUalc551Z5TXeFECzf37NlTwpEWP3QfYdAlW/V/PJ0aVaVKFQk32tkwffp0+eabb6Ro0aKuGgLaURGuow5aJLVDhw7m7+H06dPm89HVPiyzjG8+pfzu2bPHY/WR7LK1yAjMHU3xv+mmm8xy61qzQIvu62eoq+44V7Hat2+fmRrywQcfmGu7pnzr36qudlWqVClTF6V///6mlpnWxbBa7Lr//vulYsWKrg7VJ598Uq677jp59dVXzTLc2oG6evVqmTJlilhFbj+TkSNHmtW8NPtUb251ZVddol2XJQ+HeK7Xcv3/RP8fUQ8//LBZBWzQoEHy4IMPmhv8zz//3KxuZRW5/Uz0b0gHtOrXr29qSmpdFP1cfvzxx4v/MotOH7ESYgmxJDPEEk/EkvCOJS+99JJ5P/r/fn6uBEvHlg/osrVHjhyRoUOHmk4LXa1ACzqnL3oXLjQ46XKk7hd3pRd4LWwdbiZNmmR+aoE/d1OnTr1omrtVadq+BvUDBw6YDj79IqedWrrCB/6hnVoXW1ZXMwJ1+WjNCNTOQTICc05XpNIbEL3h0BWL7rjjDnnjjTdcz+sNihbzda5UpaOwP/30k+sLqmbU6TnPPfecWDF27d6923wu7vUc9GZM3++zzz4rtWrVkpkzZ1pq2mtuPxNddltvaPXYEiVKmFH6pUuXWqNWYA7iuV7D9TNx0uuP3njoTbpOLddOdv3yratZWUVuPxNdIU2ntekNSqFChUy80+uI+2sgtBFLiCXpEUs8EUvCN5asWrXKlJnxRae9zaF3QQAAn9TE0o66tqUflAIRUXl+nfP2VPnp6HumEPzFOrYeffRRV0age1HJcM4IBIBQFohYAgCwloQAxxLNStMVG9966y15/vnnTQdvfmZsUWMLAHzN7vB+y0VGoAYazQjUefbO7bPPPvPpWwQAWCeWAAAsyh6YWKLleHS6cdu2bcUXmIoIAD7mcNjN5s35OT+WGxcAsCJ/xhIAgDU58imW5Ga1dq2dt3btWjMV0VfI2AIAAAAAAECO5HS1dl38SgvFa/3BmJgY8RUytgDA1xxeTgEhCwsAQCwBAARJLNmTw9Xa16xZYxYK0/paTmlpaWbldl3tMiUlRSIjI8VbdGwBgK+ZAMDNCADAC8QSAECQxJK4HKzWrnSF2A0bNnjs69mzp9SpU0eefvrpfOnUUnRsAQAAAAAAIF8VLVpUGjRo4LGvcOHCUqpUqQz7vUHHFgD4mt0uYvOiaC8FfwEAxBIAgLfs1owldGwBgK8xfQQA4C1iCQDAArFk4cKFkt/o2AIAH3PY7eLwYmSEJdoBAMQSAIC3HBaNJRGBbgAAAAAAAACQF2RsAUAYpPwCAEIcsQQA4C2HNWMJHVsA4Gt2h4jNegEEAOBHxBIAgLfs1owlTEUEAAAAAABASCJjCwB8zYxs2C03MgIA8CNiCQDAWw5rxhI6tgDAxxx2hzi8SPl1BGkAAQD4D7EEAOAth0VjCVMRAcDXdFlcbzcAQHgLQCzZt2+f3HvvvVKqVCmJjY2Vhg0byurVq33y9gAAfuCw5n0JGVsAAAAAPJw4cUKuuuoqadOmjfzwww9SpkwZ2bp1q5QoUSLQTQMAwAMdWwDgY1ZN+QUAWDeWjBkzRipXrixTp0517atevXqefz8AIPAcFr0voWMLAHzsvCPFq7Td83IuX9sDAAjfWJKQkOCxPzo62mzpzZo1S9q3by933XWXLFq0SCpWrCiPPvqo9O7dO89tAAAE1nmL3pfYHMHa5QYAIS45OdmMbh88eNDr1ypfvrzs2LFDYmJi8qVtAIDwiyVFihSRxMREj33Dhg2T4cOHZzjWGW8GDBhgOrdWrVolTz75pEyePFl69OjhdVsAAP6TbPH7Ejq2AMDHQSQ1NdXr14mKigqq4AEACL1Yol/7bTZbjjK2NO40b95cli5d6tr3xBNPmA6uZcuWed0WAIB/JVv4voSpiADgQ3rRD7YLPwAgtAQillSoUEHq1avnsa9u3bry1Vdf+bUdAID8EWPh+5KIQDcAAAAAQHDRFRE3b97ssW/Lli1StWrVgLUJAIDM0LEFAAAAwEP//v1l+fLl8uKLL8q2bdtk+vTpMmXKFOnbt2+gmwYAgAdqbAEAAADIYPbs2TJ48GDZunWrKTqsheRZFREAEGzo2AIAAAAAAEBIYioiAAAAAAAAQhIdWwAAAAAAAAhJdGwBAAAAAAAgJNGxBQAAAAAAgJBExxYAAAAAAABCEh1bAAAAAAAACEl0bAEAAAAAACAk0bEFAAAAAACAkETHFgAAAAAAAEISHVsAAAAAAAAISXRsAQAAAAAAICTRsQUAAAAAAAAJRf8Pex57bTtQ7qMAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "MaxPool takes the maximum value from each 2x2 window\n", "AvgPool takes the average value from each 2x2 window\n" ] } ], "source": [ "\n", "# Create a simple test pattern\n", "x_test = jnp.array([\n", " [1, 2, 3, 4],\n", " [5, 6, 7, 8],\n", " [9, 10, 11, 12],\n", " [13, 14, 15, 16]\n", "], dtype=jnp.float32)\n", "\n", "x_test = x_test[None, :, :, None] # Add batch and channel dims\n", "\n", "# Apply different pooling\n", "maxpool_2x2 = brainstate.nn.MaxPool2d(kernel_size=(2, 2), stride=(2, 2))\n", "avgpool_2x2 = brainstate.nn.AvgPool2d(kernel_size=(2, 2), stride=(2, 2))\n", "\n", "y_max = maxpool_2x2(x_test)\n", "y_avg = avgpool_2x2(x_test)\n", "\n", "fig, axes = plt.subplots(1, 3, figsize=(12, 4))\n", "\n", "# Original\n", "im1 = axes[0].imshow(x_test[0, :, :, 0], cmap='viridis', interpolation='nearest')\n", "axes[0].set_title('Original (4x4)', fontsize=12, fontweight='bold')\n", "for i in range(4):\n", " for j in range(4):\n", " axes[0].text(j, i, f'{x_test[0,i,j,0]:.0f}', \n", " ha='center', va='center', color='white', fontsize=10)\n", "plt.colorbar(im1, ax=axes[0])\n", "\n", "# Max pooled\n", "im2 = axes[1].imshow(y_max[0, :, :, 0], cmap='viridis', interpolation='nearest')\n", "axes[1].set_title('Max Pooled (2x2)', fontsize=12, fontweight='bold')\n", "for i in range(2):\n", " for j in range(2):\n", " axes[1].text(j, i, f'{y_max[0,i,j,0]:.0f}', \n", " ha='center', va='center', color='white', fontsize=10)\n", "plt.colorbar(im2, ax=axes[1])\n", "\n", "# Avg pooled\n", "im3 = axes[2].imshow(y_avg[0, :, :, 0], cmap='viridis', interpolation='nearest')\n", "axes[2].set_title('Avg Pooled (2x2)', fontsize=12, fontweight='bold')\n", "for i in range(2):\n", " for j in range(2):\n", " axes[2].text(j, i, f'{y_avg[0,i,j,0]:.1f}', \n", " ha='center', va='center', color='white', fontsize=10)\n", "plt.colorbar(im3, ax=axes[2])\n", "\n", "plt.tight_layout()\n", "plt.show()\n", "\n", "print(\"MaxPool takes the maximum value from each 2x2 window\")\n", "print(\"AvgPool takes the average value from each 2x2 window\")\n" ] }, { "cell_type": "markdown", "id": "dropout", "metadata": {}, "source": [ "\n", "## 4. Dropout and Regularization\n", "\n", "Dropout randomly sets activations to zero during training for regularization. Pass `prob` to control the keep probability and enable stochastic behaviour with `environ.context(fit=True)` during training.\n", "\n", "### Standard Dropout\n" ] }, { "cell_type": "code", "execution_count": 14, "id": "dropout_basic", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:55.490420Z", "start_time": "2025-10-10T15:36:55.299660Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Dropout Layer:\n", "Dropout(\n", " prob=0.5,\n", " broadcast_dims=()\n", ")\n", "Original: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n", "Dropout outputs (training mode):\n", " 1: [0. 0. 2. 2. 2. 2. 2. 2. 2. 2.]\n", " 2: [2. 2. 2. 2. 0. 2. 0. 0. 0. 2.]\n", " 3: [2. 2. 2. 2. 0. 0. 2. 0. 0. 0.]\n", "Evaluation mode: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n" ] } ], "source": [ "\n", "# Dropout: Randomly zero out elements\n", "dropout = brainstate.nn.Dropout(prob=0.5) # Keep 50% of activations during training\n", "\n", "print(\"Dropout Layer:\")\n", "print(dropout)\n", "\n", "# Create test input\n", "brainstate.random.seed(42)\n", "x = jnp.ones(10)\n", "\n", "print(\"Original:\", x)\n", "print(\"Dropout outputs (training mode):\")\n", "with environ.context(fit=True):\n", " for i in range(3):\n", " y = dropout(x)\n", " print(f\" {i + 1}: {y}\")\n", "\n", "with environ.context(fit=False):\n", " stable = dropout(x)\n", "\n", "print(\"Evaluation mode:\", stable)\n" ] }, { "cell_type": "markdown", "id": "dropout_modes", "metadata": {}, "source": [ "\n", "### Training vs Evaluation Mode\n", "\n", "Dropout behaves differently during training and evaluation:\n", "\n", "- Wrap forward passes in `with environ.context(fit=True):` to enable dropout.\n", "- Outside that context (or with `fit=False`) dropout becomes a no-op for deterministic evaluation.\n" ] }, { "cell_type": "code", "execution_count": 15, "id": "dropout_modes_example", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:56.301641Z", "start_time": "2025-10-10T15:36:55.500426Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "With dropout (training):\n", " Output 1: [ 1.7709473 -1.7486901 -0.11733629 0.30774632 1.7717932 ]\n", " Output 2: [-2.4300232 0.03660802 1.8875287 -1.3981102 -0.3061459 ]\n", " Outputs differ: True\n", "Evaluation mode (dropout off):\n", " Output: [-0.44592363 -0.25877538 0.6993926 -1.0398884 0.14154091]\n" ] } ], "source": [ "\n", "class NetworkWithDropout(brainstate.nn.Module):\n", " def __init__(self, input_dim, hidden_dim, output_dim):\n", " super().__init__()\n", " self.linear1 = brainstate.nn.Linear((input_dim,), (hidden_dim,))\n", " self.dropout = brainstate.nn.Dropout(prob=0.5)\n", " self.linear2 = brainstate.nn.Linear((hidden_dim,), (output_dim,))\n", "\n", " def update(self, x):\n", " x = self.linear1(x)\n", " x = jnp.maximum(0, x) # ReLU\n", " x = self.dropout(x)\n", " x = self.linear2(x)\n", " return x\n", "\n", "# Create network\n", "brainstate.random.seed(0)\n", "net = NetworkWithDropout(10, 20, 5)\n", "\n", "# Test input\n", "x = brainstate.random.randn(10)\n", "\n", "# Compare outputs in training mode\n", "with environ.context(fit=True):\n", " y1 = net(x)\n", " y2 = net(x)\n", "\n", "with environ.context(fit=False):\n", " y_eval = net(x)\n", "\n", "print(\"With dropout (training):\")\n", "print(f\" Output 1: {y1}\")\n", "print(f\" Output 2: {y2}\")\n", "print(f\" Outputs differ: {not jnp.allclose(y1, y2)}\")\n", "\n", "print(\"Evaluation mode (dropout off):\")\n", "print(f\" Output: {y_eval}\")\n" ] }, { "cell_type": "markdown", "id": "utility_layers", "metadata": {}, "source": [ "## 5. Utility Layers\n", "\n", "### Flatten Layer\n", "\n", "Flattens multi-dimensional inputs:" ] }, { "cell_type": "code", "execution_count": 16, "id": "flatten", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:56.619822Z", "start_time": "2025-10-10T15:36:56.403019Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before flatten: (4, 7, 7, 64)\n", "After flatten: (4, 3136)\n", "Flattened: 3136 = 3136 features per sample\n" ] } ], "source": [ "# Flatten: Reshape to 1D\n", "flatten = brainstate.nn.Flatten(start_axis=1)\n", "\n", "# Example: After convolution, flatten before fully connected\n", "x_conv = brainstate.random.randn(4, 7, 7, 64) # (batch, H, W, C)\n", "x_flat = flatten(x_conv)\n", "\n", "print(f\"Before flatten: {x_conv.shape}\")\n", "print(f\"After flatten: {x_flat.shape}\")\n", "print(f\"Flattened: {7 * 7 * 64} = {x_flat.shape[1]} features per sample\")\n" ] }, { "cell_type": "markdown", "id": "complete_cnn", "metadata": {}, "source": [ "## 6. Building a Complete CNN\n", "\n", "Let's combine everything into a complete convolutional neural network:" ] }, { "cell_type": "code", "execution_count": 17, "id": "complete_cnn_example", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:37:39.668439Z", "start_time": "2025-10-10T15:37:39.560344Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Simple CNN Architecture:\n", "SimpleCNN(\n", " conv1=Conv2d(\n", " in_size=(32, 32, 3),\n", " out_size=(32, 32, 32),\n", " channel_first=False,\n", " channels_last=True,\n", " in_channels=3,\n", " out_channels=32,\n", " stride=(1, 1),\n", " kernel_size=(3, 3),\n", " lhs_dilation=(1, 1),\n", " rhs_dilation=(1, 1),\n", " groups=1,\n", " dimension_numbers=ConvDimensionNumbers(lhs_spec=(0, 3, 1, 2), rhs_spec=(3, 2, 0, 1), out_spec=(0, 3, 1, 2)),\n", " padding=SAME,\n", " kernel_shape=(3, 3, 3, 32),\n", " w_mask=None,\n", " w_initializer=XavierNormal(\n", " scale=1.0,\n", " mode='fan_avg',\n", " in_axis=-2,\n", " out_axis=-1,\n", " distribution='truncated_normal',\n", " rng=RandomState([ 827075891 2691092366]),\n", " unit=Unit(10.0^0)\n", " ),\n", " b_initializer=None,\n", " weight=ParamState(\n", " value={\n", " 'weight': ShapedArray(float32[3,3,3,32])\n", " }\n", " )\n", " ),\n", " pool1=MaxPool2d(\n", " init_value=-inf,\n", " computation=,\n", " pool_dim=2,\n", " return_indices=False,\n", " kernel_size=(2, 2),\n", " stride=(2, 2),\n", " padding=VALID,\n", " channel_axis=-1\n", " ),\n", " conv2=Conv2d(\n", " in_size=(16, 16, 32),\n", " out_size=(16, 16, 64),\n", " channel_first=False,\n", " channels_last=True,\n", " in_channels=32,\n", " out_channels=64,\n", " stride=(1, 1),\n", " kernel_size=(3, 3),\n", " lhs_dilation=(1, 1),\n", " rhs_dilation=(1, 1),\n", " groups=1,\n", " dimension_numbers=ConvDimensionNumbers(lhs_spec=(0, 3, 1, 2), rhs_spec=(3, 2, 0, 1), out_spec=(0, 3, 1, 2)),\n", " padding=SAME,\n", " kernel_shape=(3, 3, 32, 64),\n", " w_mask=None,\n", " w_initializer=XavierNormal(\n", " scale=1.0,\n", " mode='fan_avg',\n", " in_axis=-2,\n", " out_axis=-1,\n", " distribution='truncated_normal',\n", " rng=RandomState([ 827075891 2691092366]),\n", " unit=Unit(10.0^0)\n", " ),\n", " b_initializer=None,\n", " weight=ParamState(\n", " value={\n", " 'weight': ShapedArray(float32[3,3,32,64])\n", " }\n", " )\n", " ),\n", " pool2=MaxPool2d(\n", " init_value=-inf,\n", " computation=,\n", " pool_dim=2,\n", " return_indices=False,\n", " kernel_size=(2, 2),\n", " stride=(2, 2),\n", " padding=VALID,\n", " channel_axis=-1\n", " ),\n", " conv3=Conv2d(\n", " in_size=(8, 8, 64),\n", " out_size=(8, 8, 128),\n", " channel_first=False,\n", " channels_last=True,\n", " in_channels=64,\n", " out_channels=128,\n", " stride=(1, 1),\n", " kernel_size=(3, 3),\n", " lhs_dilation=(1, 1),\n", " rhs_dilation=(1, 1),\n", " groups=1,\n", " dimension_numbers=ConvDimensionNumbers(lhs_spec=(0, 3, 1, 2), rhs_spec=(3, 2, 0, 1), out_spec=(0, 3, 1, 2)),\n", " padding=SAME,\n", " kernel_shape=(3, 3, 64, 128),\n", " w_mask=None,\n", " w_initializer=XavierNormal(\n", " scale=1.0,\n", " mode='fan_avg',\n", " in_axis=-2,\n", " out_axis=-1,\n", " distribution='truncated_normal',\n", " rng=RandomState([ 827075891 2691092366]),\n", " unit=Unit(10.0^0)\n", " ),\n", " b_initializer=None,\n", " weight=ParamState(\n", " value={\n", " 'weight': ShapedArray(float32[3,3,64,128])\n", " }\n", " )\n", " ),\n", " pool3=MaxPool2d(\n", " in_size=(8, 8, 128),\n", " out_size=(4, 4, 128),\n", " init_value=-inf,\n", " computation=,\n", " pool_dim=2,\n", " return_indices=False,\n", " kernel_size=(2, 2),\n", " stride=(2, 2),\n", " padding=VALID,\n", " channel_axis=-1\n", " ),\n", " flatten=Flatten(\n", " in_size=(4, 4, 128),\n", " out_size=(2048,),\n", " start_axis=0,\n", " end_axis=-1\n", " ),\n", " fc1=Linear(\n", " in_size=(2048,),\n", " out_size=(256,),\n", " w_mask=None,\n", " weight=ParamState(\n", " value={\n", " 'bias': ShapedArray(float32[256]),\n", " 'weight': ShapedArray(float32[2048,256])\n", " }\n", " )\n", " ),\n", " dropout=Dropout(\n", " prob=0.5,\n", " broadcast_dims=()\n", " ),\n", " fc2=Linear(\n", " in_size=(256,),\n", " out_size=(10,),\n", " w_mask=None,\n", " weight=ParamState(\n", " value={\n", " 'bias': ShapedArray(float32[10]),\n", " 'weight': ShapedArray(float32[256,10])\n", " }\n", " )\n", " )\n", ")\n", "Input shape: (8, 32, 32, 3)\n", "Output shape: (8, 10)\n", "Logits for first image: [ 0.56677586 0.50766015 0.52835876 -0.21623717 0.56475204 0.07235158\n", " 0.78321373 -0.7866179 -0.6562584 0.10693423]\n" ] } ], "source": [ "\n", "class SimpleCNN(brainstate.nn.Module):\n", " 'Simple CNN for image classification.'\n", "\n", " def __init__(self, num_classes=10):\n", " super().__init__()\n", "\n", " # Conv block 1\n", " self.conv1 = brainstate.nn.Conv2d((32, 32, 3), out_channels=32, kernel_size=(3, 3), padding='SAME')\n", " self.pool1 = brainstate.nn.MaxPool2d(kernel_size=(2, 2), stride=(2, 2))\n", "\n", " # Conv block 2\n", " self.conv2 = brainstate.nn.Conv2d((16, 16, 32), out_channels=64, kernel_size=(3, 3), padding='SAME')\n", " self.pool2 = brainstate.nn.MaxPool2d(kernel_size=(2, 2), stride=(2, 2))\n", "\n", " # Conv block 3\n", " self.conv3 = brainstate.nn.Conv2d((8, 8, 64), out_channels=128, kernel_size=(3, 3), padding='SAME')\n", " self.pool3 = brainstate.nn.MaxPool2d(kernel_size=(2, 2), stride=(2, 2), in_size=self.conv3.out_size)\n", "\n", " # Flatten and classify\n", " self.flatten = brainstate.nn.Flatten(in_size=self.pool3.out_size)\n", " self.fc1 = brainstate.nn.Linear(4 * 4 * 128, (256,)) # Assuming 32x32 input\n", " self.dropout = brainstate.nn.Dropout(prob=0.5)\n", " self.fc2 = brainstate.nn.Linear((256,), (num_classes,))\n", "\n", " def update(self, x):\n", " # Conv block 1\n", " x = self.conv1(x)\n", " x = jnp.maximum(0, x) # ReLU\n", " x = self.pool1(x)\n", "\n", " # Conv block 2\n", " x = self.conv2(x)\n", " x = jnp.maximum(0, x)\n", " x = self.pool2(x)\n", "\n", " # Conv block 3\n", " x = self.conv3(x)\n", " x = jnp.maximum(0, x)\n", " x = self.pool3(x)\n", "\n", " # Classifier\n", " x = self.flatten(x)\n", " x = self.fc1(x)\n", " x = jnp.maximum(0, x)\n", " x = self.dropout(x)\n", " x = self.fc2(x)\n", "\n", " return x\n", "\n", "# Create CNN\n", "brainstate.random.seed(42)\n", "cnn = SimpleCNN(num_classes=10)\n", "\n", "print(\"Simple CNN Architecture:\")\n", "print(cnn)\n", "\n", "# Test with batch of images\n", "batch_size = 8\n", "images = brainstate.random.randn(batch_size, 32, 32, 3) # CIFAR-10 size\n", "with brainstate.environ.context(fit=True):\n", " logits = cnn(images)\n", "\n", "print(f\"Input shape: {images.shape}\")\n", "print(f\"Output shape: {logits.shape}\")\n", "print(f\"Logits for first image: {logits[0]}\")\n" ] }, { "cell_type": "markdown", "id": "feature_viz", "metadata": {}, "source": [ "### Visualizing CNN Feature Maps" ] }, { "cell_type": "code", "execution_count": 18, "id": "feature_visualization", "metadata": { "ExecuteTime": { "end_time": "2025-10-10T15:36:59.428533Z", "start_time": "2025-10-10T15:36:58.928483Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers). Got range [-1.3665657..2.0698853].\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABhoAAAGZCAYAAAB/rOtPAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjYsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvq6yFwwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAUi1JREFUeJzt3QeYVOX5+O93YOlIkQ4iIM2OooBgbIndGEvU2GKNJcYee4nGqNHYookmlqgRW+zGXmIXFMUuKCqIXVREmoC7O//rzO/a/YMl32dOGGHxvq8LcYcP75xdZs+ZnWfOTKFYLBYTAAAAAABADo3y/CUAAAAAAICMQQMAAAAAAJCbQQMAAAAAAJCbQQMAAAAAAJCbQQMAAAAAAJCbQQMAAAAAAJCbQQMAAAAAAJCbQQMAAAAAAJCbQQMAAAAAAJCbQQMAAAAAAJCbQQNlW3/99VOhUEi9e/de1JtS2oZsW04++eT/s82arM1+vf3229/L9gFQWXPmzEnnnntuGjZsWGrTpk1q2bJlGjBgQNpvv/3SxIkTF8k2XXzxxelHP/pRatWqVf1x57XXXlsk2wLA4nm8mDFjRjr00EPTGmuskTp27JhatGhR2p4TTzyx9GcALBqL2/Eis9dee6X+/fun1q1bl37G6Nu3bzr44IPT1KlTF8n2wHep+s4/AQBYjH3++efpJz/5SXr++edLHy+11FKlO93vvPNOuuSSS9Lw4cPTcsst971v1z333FPapk6dOqXJkyd/79cPwOJ/vPjss8/S+eefn5o1a5aWX3759P7776c33ngjnXrqqWns2LHp7rvv/l63B4DF83iRuf3221Pbtm1Lx4tPPvmkNPD4y1/+kiZMmJDuvffe73174Ls4o4GFepbDbrvtlk466aTUrVu31L59+7Trrrsu8IycujMQjjnmmHTggQempZdeurSzPOCAA9LcuXPru7pngF555ZXfuI499tijdEZC9v91D+D8/ve/r/875Zj/LIfsgaFsSp1Nh3fZZZc0a9as0h397IGi7PPJPq/57b777qWJcnbgadq0aerVq1dpojx9+vT6plgslp6VlD1LqV27dqXP+bjjjvvWbb366qvTkCFDStPybM1NN900vfDCC2V9PgA/JNk+te6HgCOPPLL0jJ6XX345ffHFF+nRRx9NAwcOLP3Zv//979IZBtkzgJo3b55WX3319I9//GOBter2y+ecc07p2JXth3v06FE6DtTJ7thnzWGHHVZ/2cyZM0v77ezyv//976XLLrrootKxIHK2HQA/zONFtv5ZZ51VesAou8//7rvvprXWWqvUZj+XZA92AfD9WhyPF5lsGJ0NF5599tnS42DZdWeefPLJ7+XrAmFFKNN6661XzG46vXr1+sZlTZo0KS611FLFPn36lD7Ofh133HH1XfZ3ssuaNWtW7NChwwLdYYcdVt/VXXbFFVd84zp233334gcffFAcNmxYsWnTpqXLevToUfo4+/VdTjrppPp1J02a9I3LWrduXRw4cGD9xyussEKxRYsWxeWWW67+snvvvbd+vbZt25Y+h0GDBi3QbLfddvXNBRdcUH95to2dO3cutmrVqv6yOmeeeWb9ZQMGDCh279699P9ZO27cuP/53wxgSTNt2rRiVVVVaV+Z7Ydra2u/tRs5cmT9/rVLly71x6Hs16mnnlrf1V2WHce6detW7NixY/1l999/f6k5/fTT6/fnNTU1pcuuueaa+uPa559/vsB1Z8ewujXGjx9f0a8HAA33eFHnoIMOKjWNGjUqTp8+vSJfDwAa5vHihBNOKA4dOnSB69t0000r/nWBcjijgYUqm+SOHz8+vfnmm6XXG8385z//+Ua37LLLpkmTJpUmsjvttFPpsgsvvLA0JY7IzjB46qmnSr9nfvWrX5U+zn7llU2Ks9fQXnvttUsfZ5/HAw88UDqFOTtbIfPwww/X99k0+9NPPy09A+mtt95Kxx9/fOny2267rfSafpk//elPpd+z0+vqPt9llllmgeudPXt26YyMTPb766+/XppQr7nmmqWzKk4//fTcnxPAkio7Tbi6urr0/+uss853ntFWt2/OXmM127dm++JtttmmdNlpp51W2gfPL9v3ZmfNZceAJk2aLHAc++Uvf5kaNWpUekbRE088UbrsX//6V+n3rbbaqnTmGgCLl4ZyvJgyZUq6+eabS/+/4447lp75CsD3Z3E/XmSPTY0ZM6b+lT023HDDdMMNNyz0rwP8LwwaWKh+/OMfl04Fy3aU2SlgmY8//vgb3U9/+tP6O8/ZHenMvHnzSjv2RWXLLbcs/V73JtfZSz9lQ4fsc6kbNMz/uTz44INp5ZVXLr1xW3YAyg4omezAlJ0Cnb1sxnvvvVe6LDvoZAeU7GWZtthiiwWu99VXX60/EGUvz5StlbXZKXGZ/2V4ArCk+n9PEvp/vuuHgOxBm+z1VDPbbrtt6XWws7buuPPll1+W9sHz22GHHUovh5e95F3nzp0X2Pdng+LsOJe5/vrrS8Px++67r/Rx9rJ+ACx+GsLxInvSUvYyGB988EHp54+6l8oA4PuzuB8vsj/PHjfLXtopeywqe0zqN7/5zUL8CsD/zqCBhWr+aWtVVdU3dtblqqmpqf//6NkOebVp02aB7a77eP6DTN3ncs0116QjjjiidADJBhJDhw5d4A2B5t/ucqywwgqlqfj8v7L3jQBgQdnro9btr7Nn//wvx5pyjmPZ+/NksmedZr+y9xfKzq7beOONF8r1A/DDOl6MHj269N4M2TNVsyc+3X///c5mAFgEFvfjRSZ7Uupqq62W9tlnn9LHI0eOXKRP2IWvM2hgkbjrrrtKb3CTqTvVK5vw1j2oXjflrdthZi9plL0Bz9dlb5CTyV5i6PtUd5ZB9kNAdprc008//Y2DQDao6NmzZ+n/77zzztKZDtmZC9nnPr+VVlqpdFZEJnsD6OyHjbqXgfrb3/5Wf1oeAP+/tm3blp4dlMme1XPcccfVn+qcyZ7hk72MX/ZSfZlbbrmldKc9u1OfPRsok+17s31wObJnLmX7/uzZTHX75+zN3Ro3brwQPzsAfgjHi5tuuqn0TNbs5VgPOuig0kuw1v18A8D3a3E9XjzzzDPpkUceqe+zsxqybanzfT8eBv+NQQOLRPb6c3369El9+/YtnR2Q+fWvf13asWd+8pOflH4/55xz0gYbbFB6j4NvmybXvTzTBRdckIYMGZL23HPP72X7V1111dLvM2bMKJ3JkP36ttfGO/LII0u/P/bYY6Um+5zffffdBZrsh4kTTzyx9P/nnXde6dS5bELdoUOHNHjw4NKzmgD4pr/85S+l/WXmjDPOKO03Bw0alJZeeum00UYblYbVdS9rlw2Es5fBy/bDt956a+my7I58uQ/oZP12221X+v+PPvpogWch1Tn66KNTv379Sr/X2WSTTUqXZccrAL5fi+PxInuZpOwBrey93bInXGWvuz1ixIjS2Q3Zr+eee26hff4ANNzjRfZKGtnjYtk2ZNuWne1wxx13lP4s+zjbPlhcGDSwSBxyyCGlCe3nn39emtzut99+pZ14nXPPPbf0XgbZNDh7zdJskpy9bunXnXrqqaU74tn7KGTvafBtZz1Uwt57750OP/zw0mvsZcOG9ddfP51yyinf6A488MB0wgknlA4I2Us/Ze/VkP3dujfOrnPsscemf/7zn6VhSfY1yabk2Vkd+++/f2m6DcA3ZfvW7Cyws88+u7T/rK2tTa+//nrpJe1+9atfpXXXXbd0rLn99ttLr3md7a+zO+/ZHfLLLrss9xlj89/xz97c7evPWspeczU7dmXPSqqTvZZrdtnUqVP/h88YgCXleJE9I7XuiVTZ/2cPWM3/K3u/NwC+X4vj8SJ7P4bs1S+yx5DGjRtXeqWM7GW3s5fzfuihh0qPh8HiolBcWC86BgHZGy1Pnjy59KbHJ598clrSZcOF7FS6upeCyp6xlB2sXnnlldKAJDuAAQAAAAA0ZP/vXUiAisjevyEbKGRvFp29AdDYsWNLp0lnr7X3+9//flFvHgAAAADA/8z5NVBBnTp1Kp1aN378+HTPPfeUzm7YfPPN08MPP/yNN48GAAAAAGiIvHQSAAAAAACQmzMaAAAAAACA3AwaWGg+++yz1KZNm9Kv7E2Q+e923333VCgU0iWXXLKoNwWgYhwbFr6ampq03HLLpaqqqvTaa68t6s0BWGgcMxY+xwzgh8DxozzZy3lnj0etvfbai3pTWMIYNLDQnHXWWWnGjBlp7733Tm3btk0vvvhi2nDDDVPXrl1T06ZNU4cOHdKwYcPS5ZdfvsDfO+ecc9L666+funXrlpo1a5Z69epVehB+4sSJubelIVz3b3/729Lvp556avrqq69yXx/A4syxYeFfd+PGjdNhhx1WevDo5JNPzr1NAIv7MWPOnDlpt912S8svv3xq1KhR6UGRtdZa61v/7qxZs9IJJ5yQBgwYUNp/tm/fPo0YMSKNGTMm17Ysyuuuc9lll6UhQ4akVq1apdatW6eVV145XXHFFd/a7rDDDqVtzH7tuOOO9Zc7ZgA/xONH5sYbbyw9kN6xY8fUvHnz1LNnz7THHnukd955J/f1vPXWW2nXXXctrZXt77O111tvvXT77bfnXvPVV19N22yzTerRo0f9fvyYY475RpddNnz48NS5c+fS55MNkQ866KA0ZcqUBbrnnnsubb311ql79+6lbezSpUvabLPN0uOPP17fbLDBBmnw4MFp1KhR6d5778297fAN2Xs0wP9q7ty5xQ4dOmTv91F89tlnS5fdeuutxdatWxdXWGGF4uDBg4tLLbVU6c+zX9ddd1393+3Vq1exUCgUl19++WKfPn3qm65duxa/+OKLBa7n4osvLs6ePXuBy6qrq4sXXnhh6fc6DeG6MyuvvHLp8ltuuSXnVx5g8eXYULnr/vjjj4uNGzcuVlVVlf4fYEk8Znz++eelj3v06FFs27Zt6f+HDRv2jb/75ZdfFocMGVL680aNGhUHDhxYXGWVVUr72pEjR+baby/K684ceOCB9ceAZZddtrjaaqsVO3fuXNx7772/sQ2XX355fZv9+sUvfrHAnztmAD+048dDDz1Uuk9ddz961VVXLe2js4+z++LzGz16dPGpp576xrp33nln8Y033qj/uLa2tv7+ebNmzYqrr756sV27dqWPs+t64YUX6ttZs2YVL7nkkm+sOWXKlOI111yzwGXZzwl19//r9uNHH330N/5udnm2L88eR+revXt9m31cU1NTf+yq26bsZ49sG1u2bFm/zdn11zn77LNLl//sZz8r8ysO382ggYUi2wFnO6hu3botsLPPdsR13nzzzfodYXbHuc6pp55anDx5cv3Hhx56aH03/wPw9913X+myjTbaqDhnzpzSZdn6u+22W+ny+XfiDeG6M8ccc0zp8u23376srzdAQ+DYULnrzqy11lqly7MHqACWxGNG9uD7Bx98UPr/9dZb7zsf7P/jH/9Y/3dfe+21Bf5+9mBPnv32orzuUaNG1Q8uvr7vnz59+gIfZ8eT7MGk4cOHF5dZZplvHTRkHDOAH9Lx4w9/+EP9feiPPvqodFnd/jYbStSZN29e6Yk+2YPzY8eOrb/87rvvLjZt2rQ4YsSI+svefffd+jXPOOOM+oFG3WV33HFHfXvssceWLjvxxBPrL/vss8+KgwYNKu3bx48fX3/5tGnT6p9Q9N8GDccff3z9oCA7xvz85z+v75977rnS5Y8//nj9Zddff/03htEvv/xy/XrZMSu7LBtCZ9sAC4NBAwtF3QPmX5+EZg+sZHfIs4lxmzZtvrHD+zY333xzfXfXXXct8Ge//e1vS5f/9Kc/La2977771t+Z/vqzgBrCdWc/ONRN2AGWNI4Nlb3ugw8+uHT5jjvu+J1/H6ChHzPq/LcH+7Nn+2d/tvbaa5fOLsievZk9M/SCCy6of5Znnv32orruuiFzz549S212zMj+PxtMz39221dffVXapuzPJ06cWHqw7LsGDY4ZwA/p+PHwww9/6xkN2Zlh2f3r+WXD3Wxgmw0gXnrppeKDDz5YbN68eWlwMf8ZDdl+ul+/ft84oyF7oH6vvfZaYD+eDZrXXXfdUnv66aeXHshfc801Sx//9a9//c7P5b8NGr7unHPOqe9fffXV0mVTp04ttm/fvv6Mhuznjuy41KJFi+Jxxx23wN/Pht11Z+zde++9wa82/HcGDSwUdZPU7A7s108lrtvx1U1Kzz///O9cJ9sxb7LJJqV2ueWWq3+2z7edRty7d+/S79tss03pTvbXNYTrzibmdc3MmTO/c9sAGiLHhsped90PF9kPLQBL6jEj8mB/9gBK3b61Y8eOC7z03FlnnZV7v72ornuzzTarXyNbf6WVVqp/wCwbPNQ54YQTSpddffXVpY//26DBMQP4oR0/brrppmKrVq0WuP+dnd01/0sc1Xn00UdLD8h36tSp9Hs2kBg3btw3uuyshjXWWGOBNbO/823DgxkzZtSfTVa3z89erui/iQ4assePsrMj6gbd83vllVdKPzvMv43Z8eGGG274xjrZS/39X8MPKIdBAwvFhhtuWNo5ZadyfZvsFN8rr7yy9Hpy2eT368/KrNtRbrnllvUT57qJ7NdlU9dsGp11Xbp0+cbrnDak686m43U7/vfff/+/rgXQ0Dg2VPa6L7300tKf9+/f/79eH8CScMz4bw/2N2nSpPRnSy+9dOlZo9l+uW697MGV/3W//X1fd/bySnU/I2Qvg5E55ZRT6i+bNGlS8ZlnnikdR3bdddf6v/ffBg2OGcAP6fiRnZmQDQuyffRjjz1WOhtsu+22q79v/W373vnPEPi299HMzlLbYostSn9+yCGHlO6v33jjjfV/J3uvha/LHufJ7u9//YyL/2XQkL18UnY8yrrsDLq6l/nLZNtUd+ZENtTIPq77vLKBdd1LLNXJhhTZn5122mn/57ZBRKNvvj00lK9Nmzal32fOnPmtf77UUkul3XffPa266qpp7ty56dRTT13gzz/66KO03nrrpTvuuCMNGDAgPfnkk2nFFVf81rVOOOGE9NJLL6XOnTunjz/+OO23336ptrb2O7dtcb7u6dOnf+NrCLCkcGyo7HXXHUMcP4AfwjHjv+nRo0fp92yf2bZt21QoFNKaa65Zuuydd975xj653P32933ddWtmhgwZUvp96NCh9Ze9/fbb6ZVXXkk1NTXppptuSq1bty79yq4vc/PNN5c+/uKLL+r/jmMG8EM6fpxxxhlpypQpaYUVVkjrrLNOqdl5553r72u/+uqrC6zx7LPPplNOOaW0H2/atGk6+OCD06RJkxZo/vOf/6S77rqr9P/ZfflWrVql7bbbrv76H3zwwQX6OXPmpN122610fz/b52f37S+99NL/6XN9/fXX01prrZWefvrp0u+PP/546tatW/2fX3vttaXPJbPXXnuVtnHPPfcsfZzNMbLPYX6ODSxsBg0sFP379y/9Pnny5PrLrrnmmvT+++/XfzxhwoT05ptvlv5/1qxZ9ZdnO/hsBzl27NjSAWD06NFpueWW+9br+f3vf59OP/30tMEGG6S33nor7bTTTmnkyJFp3333Le00G9J1z//16tq1a+mHAYAliWNDZa+77uta93UGWNKOGVEbbrhh/b41e9Ak2/9m+9FM3759U6NGjcreby/K665bM1P3gFHd79kgo1+/fgs8kJUdR7JfdWtUV1cv8HHGMQP4IR0/6gat2WWffvrpAvvRTPYAfJ0XXnghbbzxxqV95v33359uvPHG0iA421fXDXDnX3P+tbJ9/4wZM76xZjZc2HrrrUsP7B999NGl+/jZE4ey4fKVV16Z6/N87LHH0ogRI9LEiRNLA46HH344dezYcYHm27bxuz7v7POt+/wcG1hoQuc9wP/hzjvvLJ1ulb1J2fynGGenZmWn8K688sql16KuOw0sO3WrzoABA+ovz95MLTsFrO5Xdopvnfvvv7/+9efq3s8gez3Tutfjm79tCNc9/5sWbb/99gvxXwNg8eDYULnrztS95uuFF164UP69ABa3Y0amb9++pV/ZG3Nmf569BEXdZe+9916pyd4IOXtDzrrXyp7/tamvuuqqXPvtRXnd8+bNq3/pi+y1wrPjRt17NGRvOPpd/ttLJzlmAD+k48fll19evy/O9tErrrhi/cdDhw5dYH+b7Tuz93J44okn6i/PXhIpe3m6ESNG1F/26aef1r/RcvbG0tm+ue49ILKXaHr++efr2+yNl7PLDz300PrLPvzww+LAgQNL644fP77+8qeeeqr+2DL/NmcfZz9D1GnatGn9SyBln8P8PydkX4NMtm5dl/2evQdD3Us3ZW/8PP/LLL322muly7OfS7KX/oOFwaCBhWLu3LnFDh06lHZSdTvXc889t/QmOdkOMtuRZr+vu+66xZEjR37rHeJv+3XSSSct0GYPxmSvbT2/7MBw5plnln6v0xCuO5MdmL7r9f8AGjrHhspd98cff1xaJ/vBIPt/gCXxmJH5rn1i3XsVzP/ml9kbJbdp06b0K3tw6J577vnG9UT324v6uqdOnVrcb7/9Sq8lng06sp8b/vznPxerq6vLHjQ4ZgA/xOPHNddcUxruduzYsdiiRYvSe9QcfvjhpYHB/B5++OHSr6/L3jw527/PL3uD6F122aW47LLLlh7Az/bRm2++eXHUqFELdNl7Qnz9iaZ179lw8cUXf+P6v+tYM/97/fy3Y9IVV1xR340ePbq41VZbFbt3714aNiyzzDKl48LX3/Mtew+H6HtHQFQh+8/COz+CH7JjjjkmnXnmmenwww9P55xzzqLenMVe9tqsgwYNSj179iydOt2kSZNFvUkAC51jQ2VccMEF6ZBDDkk77rhjuu666xb15gAsFI4ZleGYASzpHD/KN3jw4PT888+ne++9N22yySaLenNYQhg0sNB89tlnqU+fPqXXDc1e5y17Ex2+W/amQNlrsl5yySVpn332WdSbA1ARjg0LX/bmn9nrqGavOZu93uvyyy+/qDcJYKFwzFj4HDOAHwLHj/I89NBD6Sc/+UnpPR+efPLJRb05LEEMGgAAAAAAgNwa5f+rAAAAAADAD51BAwAAAAAAkJtBAwAAAAAAkJtBAwAAAAAAkJtBAwAAAAAAkFtVNPyocER40a6dtg23hSm3htujLj4r3P7pT+E0FSfeEG4LX/wivu7P+4fbfR5cJpWjxxmnhtvfH7NPuD2x+Gp83StuCbeFce3D7YVbx29rl3x8V7htuu1/wu0ay+0Ubm+suj3cHjdvfLj97dsdw+3FG+8bbps8sGO43Wvz5cPttvv+O5Vjra3PD7cnp4PC7VN/ezHcDvp1OE3vpGK47ZkK8W2Ib0J6sRjfhkVpo0bbh9vGKw0MtzWvvh5uv9h1rXDb9uqnwu2Ey9cMtwP2ejbcVvXoHm6r3/8glaNR8+bhtnbOnLSofbn10HDb4rYx4bZxp07htuaTT8LtkqxQFb6bmBp37BBui199Vd6GdIkfDwufTw+3M9dYNtw2vzN+WyvHh7etEG57Hhj/3O5558+podi4xa4VWbc4d25F1v3okBEVWXdG/5qKrNv/wKcrsi58X6r69KrY2q//pltF1u17RPy+5eLggdobU0PwyNvxx1cWB9d8Ojw1JM9N6ZkamllzmqaGpFu7+H25xcGk1yqzj6yUA9d/IDU0D65TuWNcJdz72SX/Z+OMBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAILeqaNjtunXCix7z4Fvh9st5r4XbFvsXwu1vHh8Tbtdb5+1wm9oW4+1h8e297Opj4+umlI4ffFe4nb3zv8LtVqfGt7nRJ2eE28PeOCbcDpgc/xq/uFcZX+P0y3C7T+Ndwu3Mmn3CbavJn4bb99p9FW6f3HCLcLvemS+G28IfXwq3s64cn8rxx1+8E26/6npnuB30wOXh9vVbTwq3feM399RxVMtw++Ey66Yfsuq2zcPtB0ePCLc9zhyVKmG5a8o4BpSh2G6pePx+eWvXzpkTbj/ffXi4bft2fN1Gj8f3JS1uix+/y1HTp2u4bVTG16x2xoy0pJq9xeBwu9TLU8JtzcSPy9uQz6ZW5Dbc4cUvwm1tqowZn7QOt9XvlXecBQAA+D45owEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMitKhpet9NW4UV3TDeH20K3X4Tbq9Md4fa4dc4Pt1cVh4Xbn18ZTtPZM54Jtyt3KWPhlNKBH/w13PZrWwi39505Ptyu/O9Xw+3mGxTD7Xn7/TPcfvmXH4Xbw54YGW5/OvuqcLvnRruH2+NOiN+GV29TE26n/L51uO00dc9wO2PaluH2pSFDUzlOnRy/XRY3i69buHvXcDsw/Tq+8DYHh9O+p8wOt198MDf9kL2zactw2+t3o+ILr7VqvH3qpXBa9Z+xqRIKn02Lt2usVNbaxbHxfXWLz+L7nUaPPl+RbW78cfxrkYrxY0v1mJfDbW18C1JVt67xbfjwo3DbeGC/cFvz+pupElrcPibcVqfKKVSF766mqavEbxNLvxQ/DpWlEF+3+/2e81Oc27COg13PL+NYVIYenTpVZN1C72VTpVS//U5qSCafMrwi6/Y66amKrDtzu6GVWbd74wb1vVE9aXKqlL5HVG5tAPgh8tMNAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQW1U03PGB+KKFDbcLt+dMKYbbXZYvhNtdz7s53H62ZW24PaBjfBuOuDL+uU06+M5Ujh8Vfxtu138qvu60i24Mt+e+El+30OymcHvLvIfC7bYTp4Tbw/7903DbbfUXwu0uV8b/LXo8eGK4PWDD+Nfh8Zph4faTO/8QbgdfvWO4XeXi61M57p48IR4fs0s4HZZGhtvpD8a/n8el28LtbT/9S7ideeeH4TbFb8KLVOOB/cJtbfgoVJ73ftw63C5Txj6yUqo/+jjcfrhL37LW7jY23ja/c0yqhPc2ahtue5zxarj96LYVwm3XrT9IlVAz9fOKrPtV16XCbVWz5cNt7UuvpYamZq2Vw23nZ+LrTtgj/jXu/3x83VSM3/+74uxzw+1BN65dxkYAAAB8v5zRAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5FYVLjcshNPiUb8Nt9cuvVe43XO3YrhNx54ZTh+/ojrcnrfSOeH2L7ePCrd7fr5hKseotE64vea1Y8PtmKUuDbf9Hl063I7cc4Nwu83FO4fbPn3OC7fH/Onv4fb23S4Mt1uly8LtNQPahds9C8eH220vOCLcbrXFoeH2mV//K9w+MmNkKsfvtr023K623phwe8ba8a/xZk+F03T96Z3C7cwh8XX3eWfdcLtrahje26JzuO34Um24LQ4fFG67jZ6TFrWvNl4z3Da5/9lw2+qD+Ncs07hfn3Bb8+akcDvlgBHhtvvjs1Ml9Dj6q3BbU8a6jVq2jLcd4sfCcb/rFm5XOHJCuK2Z9kWqhDeuXCPc9rypcbhtfmd8n55p8tLE+NrN+4fbpSY2S4vaz/9yZLjtnuL3KxuSxl3ix4xyfLh9v4qs23ZifL9TjmZ3P1ORddMnqWLmbRI/zpWj6X3xY2I5et05syLrpmIZP6OWofWNT1dm3YqsmlJV72Ursu7cPh1TpTR++LmKrc3Ct88zu6WGZN7MpqkhOW/d61ND8/txP00NyS3Lxx9fWRysNuGQ1JD8ZcyPU0Oz/LKz0pLGGQ0AAAAAAEBuBg0AAAAAAEBuBg0AAAAAAEBuBg0AAAAAAEBuBg0AAAAAAEBuBg0AAAAAAEBuBg0AAAAAAEBuBg0AAAAAAEBuBg0AAAAAAEBuBg0AAAAAAEBuVdFw0B/uCi/aq9tPwu3dNzcLtzWTLw+36YmHwukOvXYOt2enfcLtkT/aJdzuPW9SKsc/j3onHh/ePpwOPfbocNvmzaPC7fT+54Xbfc8Np+nWiduG26/63hpu1/zjg+G22PP8cFsY/EC4vf738dvlNQcdGW4vKfQPt4ekYrj9ojg7leOkg48Jt8Vd1gm3tw6bFl/3yZfC7al3PBJuTzytU7jtv+4l4TZNujg1BN3OHVWRdRsP6Btuaya8FW4LVeFDYSpWV4fbFhOmhNv4qim1ue6pMuqUalJlNKqJ7x8KT75QkW0oNon/20375fBw227k6HBbOzu+72s5adlwW5wzNy1q/fcYmxYHNdOnh9smD8a3uWv8UJ/eP2ZEuF3mwfj2zlq2Ut+hAAAA3y9nNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALlVRcPzL7onvOhVrTaPb0Gzl8Npce5e4fY/y/wq3G65833h9rOn9wi3L8/9Itz+fUyfVI4VC93D7Xl/WiPcHv7mCeH2zq9ahNsrx1wdbn86badwO+2wm8Lteo/9Otw+cP6G4Xb7GVuF2+duj8/2nr78n+G25b+2D7cr9X4j3E57+0/h9sMDj0rl6P7unvG496rhtGWr+Lo/f+fKcHtLz2K4TYWzwukbGwyNr/sDVzPhrXD74eEjwm23c0elSqht1zo1NG/8ZVi47X/Q6LSovb3t0uG21z0zwu20XdcKtzVNC+F22XOfS5Xwzsnx2/uyJ8dv78URg8JtYdSL4Xbin4ancix3VGVua1MOiH/dmn8WPwYUn30l3LYfXN7XYklU271TRdbt/NfK7Nsbmqoe8Z8Zynbfs6lBGRP/ubMctT9arSLrNnrihdSQFGfMrMi605brkSqlw8MVWxoAfpCc0QAAAAAAAORm0AAAAAAAAORm0AAAAAAAAORm0AAAAAAAAORm0AAAAAAAAORm0AAAAAAAAORm0AAAAAAAAORm0AAAAAAAAORm0AAAAAAAAORm0AAAAAAAAORWFQ3/8+E74UUnbhzfgAsfWDUej9stnJ5weW24Xf3aQrjdO30Ybv95dddw+6fCzakcjfefGm7vaPF4uL301qXC7RabbRVup6/ZJNw+dOtl4Xb9d6eH2/6PXRxu32hySbht9MXgcPuX9Ltwe8Xe4TTdUSyG237nbxZuC5M2CLfFtuXdhjv+o3e4Hdt8ULidevSx4Xb/TcNpmvvve8LtlePeC7edVoi3DUWhWbNwW5w7N9zO3XxIuO127qhw+/7RI8Jtj7OeDre1L4xLi4OvNl4z3HYZHT8eLg7m9f8y3Da6dma4bXv1y+H23RPit5+l58wJt4UmTcPtsifHb+/l+Gh4q3DbrYxNGHDR+2VtR3WqjO63Tgy3H23ZpyLbUN0i/j3XuEvnimwDAADAwuCMBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAILeqaLjpgN7hRYccuGG4/dmTbcPtf2btF27bDRwUbh/uWwy3vd7+Z7jdePgn4XbOnE9TOVpu1Cfczljn7HC7z8xr421qGm7veSacpnULB4bbphvtGG7f2PH2cDvjrXvDbaczVw23P2+3bri94vZfhNtfHXRhuP1o6vhw236D+G24sMIpqRw9pp8cbs94q324vfHVqeH29MLQcDvywO3C7fk/Cqep2PrdeDwzNQjFuXMrsu6nqzYJtz0fah5uW79XG9+I2prU0DT6Kv75tfpwXmpImr7ZItx+sk78NrH0m5PCbc9TR4XbRs3j21A7Z064LayxUrgtjn013Db7PH7/6PM9hofb9leOTuV477gR4XaZ0+P/HrNXXSbcftmlEG4btWwZbpf59/vhtvrjKWlJVHw+fpssxwe3rliRdWuea1eRdbs881VF1k33lnEnvEzFEfGft8pRGPViRdad9sv4fqocS780rSLrlnHvZLHw+dXx++vl+Gxy5e57dfhHalAaDVoh/ZBt0jf+c+ziYOLMjqkhmV4Tv5+6uNiv/+OpIdlv8hapIdly6POpIbnj5fhjc4uLYpPGaUnjjAYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACC3qmjY/5WVwos2valzuG01+8RwO/v1EeF2/CMzwu153U4Jt1Mv3yncDtvxX+F26VZnpHIUaqaH2+Ifrw23Jw7dNdz+YZdwmt47Pt4unf4abgcd9EW4fWa3keH2yMc2CrdHff5AuL3pxBfD7QOdBoXba+8shtsbzot/f563fftw++xXx6VyXPXGsHB7071Phduhaelw+8wKL4XbP/R+J9yessGF4bbQO/69Xyzje2NRmr1t/N+25S1Ph9vuj80Kt7WrDQi37V+cFm4LK/QPtzXj30iLgybPxrdj6jYrh9tP/jk43LZu+2W47bb1+HDb4dWacPv+xrXhttOTfcNtzYS3wm1xpfi6jcZPCrfvr9823HYbG07TV60K4bbqy/hx6L1j4/fnMsucPipVwsweTcJtzz/EtyF+S0uptlObeBy/SQAAAHzvnNEAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkVhUNr212YHjRQ4vzwu2sGU+H2zWOvyLcrvDLieF2qePCaerzx3i7TMvLwu3J1b+IL5xS+nDL+8LtT495PdzelbqG21FXF8Pt6+mucPtey7fC7fW/2S3c9u11VbhdfvK74faYwmvhdpvfDQq3Bz7YK9y+fnN8G94fPCXc3v9lqpymT4XTV4ZeGG5XLvw03HZdZvNw++Eh8X1K4Yg24fa8u+P/zg1Fy1vi+/VyTFmjVbgt1MTX7XzRqLSoFYfH9w2F0S+Wt/bA+G2s48PxfV+HZ1qE25rxb6RKaPnBnHDb5bH49hY/jO8ny1Ec+2q4feva1cLtsv+Ymyqh87Ozwm2jOdXxhRsXytqO+L2NlKp6dA+3S18+OlXCjF+sFW7bPRS/z1PGbq1Bmbv5kMqsOyf+80g5rtvrzxVZd9+PD6nIup1XXj5VSvGZ8ZVZtyKrpvRFv/L2PVHtRlbm69DQtNnsrcqsmyqzbkNU+6LbGgCLN2c0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuVVFww+7vxNe9OTVr4m3az8Vbp849a/hdqsvfxFuH6hqGW5f+lkh3FYVNgu3Z+79WCrHX059ONwefdst4XarbVYNtzU3xr8WY7dbL9xu/eSj4Xb19KtwW4zfLFPVrPbh9pHZG4XbH/3ox+G2MPehcHvSiT3D7QFH3Bhuu/zxxHC7y+gbUllWfjOePvtRuN1/75Hh9m81fwm3E1b/ONwWd94/3G71p7Hh9tD4LmWJNL1/Tbgttoy3ncvYhsb9+oTbt/boGm57nzA6Vcq8ds3C7aS9uofbgQc/F24/uHXFcNt9m3HhtvFzr4fbtqPmhNvUqlV8G9rHjxc1n38ebt9Y/8pwu8nOq6VKaPzlV+G28OW8cFvzenz/n6ldb/VwW/34S2lRW+pf8fu28T1VSo1XHJBrewAAAL4PzmgAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByq4qG1z7fNbzoe507hdsB670dbnf55bXh9ud3/iPc7r3aceF2bNo63L5w6EPh9ryVb0/lOG3P8D9dem7beeF2o+Id4fak9Fq4vSn9O9x26vZquD1gr/i6hSt2DLfF0UPC7eNr3R3fhj/GbxPFjeO3id8VWofbzjeH01TcLd6m+6aUEae0xtRfhttztvs43H6ZTg63nR/4MNxec+UG4bbtKxeH29vTReH2h26F8z4Kt9WTJldkG6asHz8WdnqhNtw2btMm3NZMn57K0eTBseF2wIPxdYtlbMPMqS3j6669WritffKFVAm1s2aF28LyvcPtWzsuH243W655uG3cpmm4TS3i6374o3bhtvNfR8W3Yegq8TalNGm/+K2t76M18YULhXhbLOcWXyE18X1KQzK7U/w+bTl6XVRdkXWP22loRdbt2vPdiqxb/e57FVm3Iep1Uhn7qSVY43ZtK7JuzbQvKrIu/78vt67M/qehmDC9c2pIGhUWg/sOZVi7RfyxucXFhvccnhqUhnWTSKlxw9rgLQdX5mfDSnql3appSeOMBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAIDeDBgAAAAAAILeqaPjeL+OL/m2lTcLtzvufHW77drw53Lbc6dxw23y7Qrjtf+aIcPvbdZqG23O7lvEFTint9VK8/7jVneF29Anxr8URt60Rbsdt82y4faR4dLh9ccW/hNst0o7h9sY0NNxunz4Ltyf3KIbbdF7zcHrKaX8Mt9e8d2O4PfmvK4Xbu5pMSOX4RSH+tfjt/vF1izecEG5PuCy+8CP7vBFuN+qzZrhNx+4Wb9N2aYnTqHE4nb5613DbctLkVAmdxnwebgsffBJu39135XDb/exRqVImXDwk3HYaHb4LkVY87cNwWz3x7XBbtVzviqxbjvd/3DbcLnfb7HBbO2dOuG3ct1e4nbVcfHu7Ph6/vdeGy5Qav1be92ffnaenSvhs77XCbYfLRldkG6p6LhNuPx3WqSLbAAAAsDA4owEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMitKhoOX/fy8KL7z9g13P7qoSPC7fhLi+F2pY26hdtLC4PD7Ymttwq3J/T7V7g9d0AhlWNGdZNwO/mFLcLt1i88HG7PXuHZcDsu3R5uZ2zfM9x+2eHlcLvquw+F2+17vhNuCyetGW5/Pu7UcJsmzQ2nu/1tl3BbfeEx4faSsWuF2wGb7pPKcfRxu4fb/X9zVbj9PP5Pl668emS43WPrYeH2tJNuCrdvp97hNl42HG+eMyTc9jvsqYpsQ3Ht1cJt7ZMvVGQbOm7yfjw+O1VMn5vjx9km948Ot9WNGofbxgP7xdd9/c20qLX4JP41K1To9jN1tfbhtu018e+j2lQhXTuVlc/ecIVw2/KWp8Ntx7HTw+2nvxoebjtcFv/eSI3jz/lpVB2/rTUk89qUdx84qtHjz6eGpPrd9xb1JrCYadyvT0XWndtr6YqsW/WfsamhKY4YVJF1C6NerMi6szrH708BwKLgjAYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACC3qmg46sm9wosW7tk73F5e3DXcTtx+i3B7yU1Ph9vfpl7h9r5+y4bbbV9YLdym81+Ptymlvz7dP9weN+y34faWUXeF25GrF8Lt4R3uDrc733hwuO3b9vxw+9spn4fbtFeTcFo8dNNwu+LgC8LtFxs9GG5vrnk73LZd54Zw+7tB8e+je9dcJ5VjwNV/Drd/6/5uuL1rgwPC7e5/2DbcDrrzwnCbDu4TTvt2aRtua46flhqCuZsNCbftXovvRwqrrxRui8+/Gl/3yRfC7fSd1wq3ba59Ktw23WhyWhx8tlLTcNt12irxhce8HE5rXn8z3H614RrhtsmDY8Nt4/btw23He94KtzO3GhpuW9w+Jr5uz/jzRuJ7nJQa94vvywpfVYfb6gnxr1mmZRn9B0eOCLfdzxoVbju8EN9X1a63erydNifctn1jVrgFAAD4vjmjAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyK0qGp55UhmrHhVPHy38Idw+v2PvcLvJ+Pg2bHT9B+H2Z1e+Em6vTfPCbfGC36WyHHNVOD329vPC7VWNrgu3z7x+UbhtOnCzcHtrircdvzgi3A5+rGe4bfyP+I34wbR6uP370AHh9qXTlwq3szucEW5n3XBsuC20XTHcHvlxn1SObefFdyrX7B3/t9v13++F2y0+3T/cHnXKNeF24rsfh9uaVUaE23R8ahBm9ggfWlKn52aG2+Lzr6ZFren02tTQvH3q8HDb7++Tw21x+oxwW5Mqo/nbU8Pt7M2GxNvO8dtwuzdmh9sWt48Jt8Xhg9Ki9sa+XcPtckeNrth2NO4XP758NWTGIt+G9Ojz4bScPcqMX6yVlkQ97ogft8tRXZFVU/pyq6EVWbec/UM55m2yZqqUpvc9W7G1G5KPDivjvlwZqmYVK7Jux0sqt79uaL7o27Ii67YblRrWv93fU4Pw1jPLpoak2MCe1ntYo+1SQ9Nycvw+++Kg5dqfpobkmcE3pIZk09e2SA3N3PYN6zYc0cB2fQAAAAAAwOLEoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMjNoAEAAAAAAMitKhoevPVR4UVfW70YbgesUwi3R713Y7j95JTtw+2R+4+Kt0duHG4ntI5/Hfa46I5Ujo5deoTbu/8Y345xxw4Kty8VXwq3g045INwW04xwe9C0W8LtujXXhtsfr/TrcHvB2CvC7a1H7hdu1+twfrjtWFgl3B76s+nhtus5r4bby38R/17OfHrrtHC7x1dPhNstblst3N51ZYt4+/sB4XafZeLtoSfEv24rpvj38qLU4bLR8bhJ03g7NH47T2NeDqe166webqtbLvr5/LvHjyir731C/BhXXca6b567Vrht+0b8dt7pb/HbT227VvG2aRnbcP+kcFvTvUO4Lec7eNJB8e1tOSZVRN/jx1bkc6tZf3B5G/LIc+G0yZj498en+w4Ptx0vKWO/ViFNZtUu6k0AAAD4Tov+ERMAAAAAAKDBMmgAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByq4qGLY/pG150vyc7htujTz4m3Pa6e7twe/q1l4bbh7e4Ndw+N+PIcLvTH94Jt3d2WSWVY7fxB4fbcdsUwu0T6ehw++eZncNt9xOvCLe/OGFquG1WuDrcXnrq1uH2mnCZ0m0t4tuQfrFcOH3s+ng7sDg73P6rMDHcTn9423D7+DvLpHL86IpDw+2r5/003NZu1T6+EVtPiLdDLgunk0/ZJ9z+bNmx4fbNtOQpDIzfzotjXq7INjR58a1wW917pbSoNZ9arNjajVZdPtz2O/ypcDtzh7VSJRSr4s+XaHH7mHD7/sEjwm3Xi+LrlmOZf4bvoqWq2V+G2zcuHBZul79oWrgtTno33KZHnkuVUhU/HKZpK8a/l+L3bMtTs8HgcNv8zsrc1ha16rfj95cXB7M7Na7Iuo998EJF1u1z29BUKSu+1rMi61ZPLmN/shhoVqHjcpMvK7PuxD8Nr8i6yx01OjU07UZWZptr11m9Ius2nTSlIusCwMLijAYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACA3gwYAAAAAACC3qmi47MOTw4s+NCq+AZfe0D7cvjvsinBb+PvB4faj5oVwe9VfZoXbL068KNx2PebCVI4HCi+G23PfeC3c/rb4ULh9sFH/cHtT8bBwO2iXluF27nXFcLtbp/i/8/rP3BJui4ccF24H/qtTuD38+qvD7fkpvg21n58WbovTw2laY/Qf43G2dmoXbs9e9ohwe2ThrnDb/NYJ4Xa1W+K3iXRTPH3r+DXicYrf3huK2lfi+6dy1Kw/ONw2/uCLcNvh9lfjG9Fh6XBa89nUcNv58ufKe0bBigPi2/Hy6+F27uZDwm3rG54Kt9U/jn9PVD00NlVCbfjeUUrF6uqKfM2a3f1MuH3jgmHhduBv4/cfaubMCbfTdhsebttdNTqVo7j2auF2drf4frLj2Pj9gnI0btMm3H68SvNw2+XhnBsEAADwPXBGAwAAAAAAkJtBAwAAAAAAkJtBAwAAAAAAkJtBAwAAAAAAkJtBAwAAAAAAkJtBAwAAAAAAkJtBAwAAAAAAkJtBAwAAAAAAkJtBAwAAAAAAkJtBAwAAAAAAkFtVNOyw92fhRRs9EG9f+9254fbeLU8PtzffEG+77LZTuL2isHO4bbbDUeF2zB8PSOUoFv4WbguHLx9f+OhiON2wdutwW/zstnB7YIf4NjTpsme4PerclcLt+uttE27brTQi3D5xV5dw22XFVcJtzba9wu1h8ZtOuvaPH4Tbj/d9P75wSum5fy0dbtcfcXS4XSX1DLcvd1o33G775v7h9shNNgm36ZO74u1pqUH4asM1KrJukwfHhtvGjzwXbmtSZTQatEI8/mxqvK2N7yMzH/ykY7j9aqt4u8wfR6VKqHoo/u9cjsKQ+D6127mV+dwKNfF/u+Laq4XbTs/EnzdSO2dOqoSzTo4fXE67Kv65ZQpPvhBuez8ZX/eDo+LH7w6dOoXbmk8+CbfdHol/7xdWGpiWRF9uPbQi67a4bUxF1u3ywHsVWXeTy8r7vogakCrzdSjpE7//uSRr/8/RqSFpfcOi3oIlX6PHn6/IutXph63LoI9TQ/LFl81TQzK0/dupoZlYu1xqSLq0npEakntnN0sNyafXLJsamk4TpqUljTMaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3Kqi4WFb/iq86G6zTw+3hT90DLcP/WLvcLvUqzeG22037RRuV9yge7h99bC7wu20rcekclzw2K/D7dixX4Xbfoc+G27PeP72cPvIpN7h9oIP3w63L+98RbjtcG44TVv+JN6my+8Op9sdMTrcXjHu+nD7r6t7htshn/093J5biN/OCg8+l8rRfs+Lwu2kzk3C7SEzVg+3ew/9KNwWxtwTbj9Y6j/hds2fDAu3u6SGoelnc8Jt8flXw22hKnzISsXq6lQJjVZbMdzO69Ai3MY/s5SKX80ro06p69/j+/XGPePHuHGXrRlu27zcNNx2PX9UqoTCq2+F22JFtiClOR3i/9LvbBZv+x0aP7aU46uN4//Gf5gUPw41Su+mxUHbiTXhtuaTTyqyDbUvvRaPGzWuyDYAAAAsDM5oAAAAAAAAcjNoAAAAAAAAcjNoAAAAAAAAcjNoAAAAAAAAcjNoAAAAAAAAcjNoAAAAAAAAcjNoAAAAAAAAcjNoAAAAAAAAcjNoAAAAAAAAcjNoAAAAAAAAcquKhp9eUh1edL3b/xFuLxrZIdyuuPqV4TZ9uUI43bPLA+F2x1PPCbe9Ltwv3O7yn01TOTocdUS4HXzfIeF23uM9w+3GNxfD7YGrPRNuX2n0t3D7k58eEG5vTW+H26ETzgq3Y4o7hdubD94r3F64Qatw+/ov4+vusOv0cFu455JwWxzXJZXjzr614XbL5mUsXDUovg1/vzvcXrzdDeF2mzQy3HYee0e43SU1DMXnX63MutXx41A5atdZPdx+umL8xtj1vvfDbTmfWaOWLcuoU6qdPTu+HRPj+8kBv4q3k84Yniph3qZDwm2zz+bEF37m5VQJHw+PHze7jCqkRa35e/HjxdTLe4Tbtundsraj0crLh9vaV16Lb8dT71Xke7Sq97Lh9tN141+3dleNTkuiFrfH7yOWY+K1q1Vk3eV2fqEi61b16F6Rdavf/6Ai65bWnjS5Ius2btOmIuvWDuxVkXWLFTpmzNxhrYqs2/qGpyqyLv+/4oj4zyPlaDSmMvexAWBhcUYDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQm0EDAAAAAACQW1U0vP/hV8KLDr6+cbhdPo0Ntxu8+Xy4Xb9Dv3B7xdEfhdsjDj8q3D7x87vD7bKFrqksN1wcTo+Y+2W4LZ7/k3A7rbBtuD29eG24PardnuH273fNDre7Fy8NtxcXDg23O63TP9wWb9gi3F5aiM8Bf5f2CLdbPXlsuN36uE/DbeG57qkc/zzhoHD7t5t2D7f7/3xKuF3+X53C7a2r9w23d957ZbhN+2+WljSNWrUKt7WzZlVkGyZcvma4HbDXs+G24xOFcDtjqyHh9ottlgm3Xc8blRYHteusHm77/3liuK0uYxua3vtMPF59pVQJEy4aGm4HHPB0uJ145vBw2+a6VBE14yaE26oVhlXutvbKa5VZ99PPKrLu5B3j388tphQrsg0AAADfN2c0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuRk0AAAAAAAAuVVFw7267xtedPv/FONb8OM54XRK4bxwe3JxqXDbZZ0u4facv58fbt9M8c8tFX8Vb1NKj6WZ4bb3Dr3D7RMX9A23k8r4Z377zOPC7R7TOoXb0+6+Odx2SweH2xk9jw23Mx+PfyGu3bEQbvd9sXG4venK58Ltqj96INyukVYPt7c1Xy2VY/duL4bb20+N337ar3B4uP3806vD7b1XHBluH0jx28TZbc8NtynFP7dFqXbWrIqsO+HyNcNt0w+bVGQbUjH+b9v8k3nhtsVtY8Jto5WXT+WobVnG12LMy/HtePz5cDtj66HhtsVtH6dKmNW7dbhtGf/U0sDLZ4fbMg6bqbrTV+G2UcuW4bZ2dnx7G/frE24/Hhp/7krnRsNSOVrf+HSqhNo5ZdxPK0OPM0ZVZN3i2uUdZxuMMvar5Vhu5xcqsu6XZezPylHOcaAcb1xQ3vdbOfofXJnvzd2eiR+LyvHm3CkVWfefd29QkXV33uyxiqz71A2VuY806fThFVm33zVTU6XUvPp6RdYtjIr/nFOOYqP4z4dLojbNKnPcrpT1u76RGpJrXh+SGppT947/PL84OPqZn6eGZFzHHqkhWe+AytwvqaRHL6rc/bRFxRkNAAAAAABAbgYNAAAAAABAbgYNAAAAAABAbgYNAAAAAABAbgYNAAAAAABAbgYNAAAAAABAbgYNAAAAAABAbgYNAAAAAABAbgYNAAAAAABAbgYNAAAAAABAbgYNAAAAAABAblXR8O1214YXvWXfL8Pt+9seEG5PKT4abse2PSPcrvFFIdwu33vHcHv79HCamu22aTxOKa3z5xfC7UovPRJuf7XKkeH28OPjX7fdN/l3uP31zJbhdsO5d4Tb01aL34YvevfmcDszfRBu3/nXEeE2bX52OL22z5rh9qZ0Sbi9vhhOU3qqjDaltPGv44uPejx+W7uz7cRwWyjeEm5P7HxWuD1/XrxNf5oUb89MP2gDLpkXj596NpzO+vmwcNv638+H26qJH4Xb6nCZUu0rr5VRp9R4QN9wW1POup06hds2h74bbr+6LVXE+xvE2/63xtvis6+E21nbxW9rKxw6Pty+dsYq4bb/wU+H25o34/unmm7tw227Zz5N5Sjn+6Nxvz4V+fzeP2ZEuO3xp/jXONXGv+sKT8bv+wEAAHzfnNEAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkZtAAAAAAAADkVhUNjxz+YnzVCY+E08l/GhluN0/7hNtz3zk23A49aMNw+6vJ14fbB5+4Ltyefv2PU1mOuDacbtn7sXB788nxTdjwnp+H23+etmV84Yvi6c8eiv/bXTZ1crhdr4zb2oZ/3DrcVhX3DbfFwh7htnDYSuF2VrEQbr+Ip+metFE8Tindf0S8PXvdeHvjhtuE27Oe3jncFt65NNye+2xtuJ1dnBtuT0hLntr1Vg+3czo0DbdtunYJt/P2mhpuC2/1C7fVL4xLldB4YHwbMlPW7RRum6wRb9tc91R8I7Zpnyqh+sdrhNuBR8Xvx8S/g8uz1J3xbaiZMyfc9j/46XBb1XvZcPvawd3DbZN34weM6rffSZVS8+akiqzbdFoxHtfWVGQbKE+jli0rsm6L28akhqTduIb3vLIrBvZKDUnxmvj+uhzPbl6pr8MHFVm1z3GjK7KuPep8HF8AWMw1vHueAAAAAADAYsOgAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyM2gAQAAAAAAyK0qXI5bLZw+d3qfcPtInx3D7atv9w63az49Ityu1+vBcHvZReE0XXb85+H2N/06xBfOtvnyeNv5y1bh9snt7g63l3S6Odw+0O2OcPtav5+F26k7F8PtnIfDafrN2o+G26rj4uv+fMoH4fbQ4l3htvjPn4bbW/YeH243TyuE29Mm3J/KMXhYvF117cPC7bM9Tgu35z+6Xbgdue614bZYWD7cXtNrcLhNW6QlzrS+zcPt0pePDrezN1kz3H76duNw27btvEU+yX/jpNZl9e3vK6N94I1wW1PGNtR8Hj8elqPqobHhtrYiW5DSh7fF95PdT43f1tKzr6RKaHvNjHDbb+2nwu07J8fvd5Wr8YoDwm3NuAkV2YY5HQsVWXfu5kPCbbO7n6nINgAAACwMzmgAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByM2gAAAAAAAByq4qG1x28Y3jRnUZcG24HT2wVbkcWeoTblweE07TG+vH2RyO/CLeP/7xNfOF/tI23KaVHL1sx3HZsMSrc3jTuoXB76c23hNv2/94y3D45+R/h9rwDfhpu71v9tnD7yUpvhNsHB04ItxuccES4fTT9Mtxuusdd4Xbb13cOt6deHr/tDO6WyvK7qQeE20nponDbOxXC7Wf7FMNtYdgu4fbjVeLtDVuMDbe7pMFpSbP05aPD7bRfDg+3s7aZHm57X9Q83E4dGG87PpoqonFVTVl9o+r47bzm08/i6w5aIdzWvjg+VULjTp3i2/D55+G2sHL8TkS3rcfF123ZMtzG/9XK89na8a9Dce3Vwm3v26aF29pUnhnLtw+3LeP/HOmDW+P3pbpeODdVQrO7nwm3harw3fYGpZzv43LUfPJJRdZtaGpaxO8XLekatYr/3FmOvrs8X5F1a9ZYqSLrpvc/qMiyM3ZcqyLrLnX9U6mh6ftM/P5iOe4dFz9uLYlqNqjMbbdSnkmNU0PSaasWqaE58874z92Lg4GPvJcakjuW/3FqSFq8Ff9ZdnHRbHClfupbdJzRAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5GbQAAAAAAAA5FYVDXc6qE940alzOobbpf/xbLhtt8PZ4Xbuvb8Nt7P6nhNuW598X7hdduYO4faGL1dO5Vi/MCrcrlWMr7vD21eH2wMKV4TbR3e6Ltye9+HMcPuzOfGv8R0X7BVut943/kWrfr4Qbju/ODfcbtqiabjdYdvp8XUH7hJuz/hJOE1nvXlvPE4pTVnty3A7svcR4fb16qXD7Zj34rfhNL5TOL1q3z3C7ek1a8S3oXEZ38wNRO16q4fbjg9MDLftRn6cKqFrj+7xuFvXcFr94Ufhtu9hn8W3IVv7/ZdTJUzYvW247Xd4RTYh1XzySbidttvwcNvuqtGpEhq1WSrc1s6eHW5nbzMs3La89elw+2WXZvF1b3khVcongxqH2163xNftvs24cNu4fftwW1imR7itfu/9cJsGDYy3AAAA3zNnNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALkZNAAAAAAAALlVRcNfDng/vOjjf20abod0OT3c7rjWEeH2kPH3hduHz3sg3BYPOyfcFroUw+3rRz6RynHciSeG2wc7zAi3/xqySbgtvLV+uC3usGO4vfyjO8Ltv2+4NdxeeveG4XbZh7uF202e+nW4PXSznuF2t+7bh9sb17893N5QbB9u907Twm3tlfHbZOat1S8PtxcXVwm3vU56Otz22/mX4bbVxEvD7bLHrBtujzxwr3Cb/pYahMKaK4fbRo8+H26ry9iGt65dLdz23fmFcFts1SLc1kx4K1VCzcdTyuoLVeFDfSpWx7/Kve/6Ki1qVb2XDbfT+sfXbZcqpIx/i/ePGRFue5wxKlXCOieODrc3r7F2uO19fHzdTK+TKvP5Ne6/XLiteWNifOHPP0+V8MXApdKS6O39y/jmLMOz+91VkXVXuuc3FVl3wD7PVGTdzs/MTpVSzrG2HE3Ht6zIup1eKOeeRNy7Gxcqsu6Kq7xTkXXfeiR+fClHTfP4z77laP9491Qpb+7fqzILDynvOBe17BaNK7Juiv9IBAD/lTMaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3Kqi4dUrrRVe9Dc3fBluh81rHm6vHh9O020/PjXcLtPugXDbqbh+uH3yJ7eE2xHD30nl2Gyb7cPt0wfGt3lErz+E2332ujHc7vaLQeF2oz+sGG533nDHcHvd5a+H23WHfRRuV/1453C77Ul/C7frnbRTuL3siufD7Qa18X/jexv1Crebztw0laP2qqbh9vQDtg23nZ8eFm7njr4v3I5qdka4nffIm+E2/f3kePu38vYTi0qjt94PtzVlrNt4QN9w2/Hu+LGlHDUT3kqLWmHlAWX1tS+MC7dVPZeJL/yfsRX5tyvna1z9dvx7oveJ8farjdcMt03ufzbcVr8X/974cuXOaVF77NTh4bbbnOqKbcfMHeL3Qdu++nm4rXk1fr9gcfBVq8Ki3gQAAIDv5IwGAAAAAAAgN4MGAAAAAAAgN4MGAAAAAAAgN4MGAAAAAAAgN4MGAAAAAAAgN4MGAAAAAAAgN4MGAAAAAAAgN4MGAAAAAAAgN4MGAAAAAAAgN4MGAAAAAAAgt0KxWCzm/+sAAAAAAMAPmTMaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3AwaAAAAAACA3P4/XJSfaE2xI6UAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Feature map shapes:\n", " Layer 1: (1, 32, 32, 32)\n", " Layer 2: (1, 16, 16, 64)\n", " Layer 3: (1, 8, 8, 128)\n" ] } ], "source": [ "# Get intermediate features\n", "def get_conv_features(model, x):\n", " \"\"\"Extract features from each conv layer.\"\"\"\n", " features = []\n", " \n", " # Conv 1\n", " x = model.conv1(x)\n", " x = jnp.maximum(0, x)\n", " features.append(x)\n", " x = model.pool1(x)\n", " \n", " # Conv 2\n", " x = model.conv2(x)\n", " x = jnp.maximum(0, x)\n", " features.append(x)\n", " x = model.pool2(x)\n", " \n", " # Conv 3\n", " x = model.conv3(x)\n", " x = jnp.maximum(0, x)\n", " features.append(x)\n", " \n", " return features\n", "\n", "# Extract features\n", "single_image = images[0:1] # Take first image\n", "features = get_conv_features(cnn, single_image)\n", "\n", "# Visualize\n", "fig, axes = plt.subplots(1, 4, figsize=(16, 4))\n", "\n", "# Original image\n", "axes[0].imshow((np.array(single_image[0]) + 1) / 2) # Normalize to [0,1]\n", "axes[0].set_title('Input Image\\n(32×32×3)', fontsize=10, fontweight='bold')\n", "axes[0].axis('off')\n", "\n", "# Feature maps\n", "layer_names = ['Conv1\\n(32×32×32)', 'Conv2\\n(16×16×64)', 'Conv3\\n(8×8×128)']\n", "for i, (feat, name) in enumerate(zip(features, layer_names)):\n", " # Show first feature map from each layer\n", " feat_map = np.array(feat[0, :, :, 0])\n", " axes[i+1].imshow(feat_map, cmap='viridis')\n", " axes[i+1].set_title(name, fontsize=10, fontweight='bold')\n", " axes[i+1].axis('off')\n", "\n", "plt.tight_layout()\n", "plt.show()\n", "\n", "print(\"Feature map shapes:\")\n", "for i, feat in enumerate(features):\n", " print(f\" Layer {i+1}: {feat.shape}\")" ] }, { "cell_type": "markdown", "id": "summary", "metadata": {}, "source": [ "\n", "## Summary\n", "\n", "In this tutorial, you learned about:\n", "\n", "✅ **Linear layers** - Fully connected transformations \n", "✅ **Convolutional layers** - 1D, 2D, 3D spatial feature extraction \n", "✅ **Pooling layers** - Max, average, and adaptive pooling \n", "✅ **Dropout** - Regularization through random masking \n", "✅ **Utility layers** - Flatten and reshape operations \n", "✅ **Complete CNN** - Building end-to-end architectures \n", "\n", "### Key Takeaways\n", "\n", "| Layer Type | Use Case | Key Parameters |\n", "|------------|----------|----------------|\n", "| **Linear** | Fully connected | `in_size`, `out_size` |\n", "| **Conv2d** | Image features | `in_size`, `out_channels`, `kernel_size`, `stride`, `padding` |\n", "| **MaxPool2d** | Downsampling | `kernel_size`, `stride` |\n", "| **Dropout** | Regularization | `prob`, `broadcast_dims` |\n", "| **Flatten** | Shape transformation | `start_axis`, `end_axis` |\n", "\n", "### Best Practices\n", "\n", "1. 🎯 **Use SAME padding** to preserve spatial dimensions\n", "2. 🧮 **Provide the correct `in_size`** so layers can validate incoming tensors\n", "3. 💧 **Add dropout** after dense layers for regularization\n", "4. 🏊 **Pool after activation** - standard practice in CNNs\n", "5. 🔍 **Visualize features** - helps debug and understand the network\n", "\n", "### Next Steps\n", "\n", "Continue with:\n", "- **Activations & Normalization** - Improve training stability\n", "- **Recurrent Networks** - Handle sequential data\n", "- **Training** - Put it all together with optimization\n" ] } ], "metadata": { "kernelspec": { "display_name": "Ecosystem-py", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.13" } }, "nbformat": 4, "nbformat_minor": 5 }