preliminaries.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "sPwyFkjYwPIk",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#  CE-40959: Deep Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "wgn-nyxXwPIl",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Homework1: Preliminaries & Numpy (65pts)\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "aTXvPr2QwPIm",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "You can run cells by hitting `Shift` + `Enter`. <br/>\n",
    "We higly recommend you to read each line of code carefully and try to unserstand what it exactly does. You may want to use the techniques mentioned in this notebook in your next assignments.<br/>\n",
    "Please review `Numpy Tutorial` notebook (materials of the first TA class) before coming to this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "dS7No7eVwPIn",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Deadline:   2 Esfand"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "D9FMPPHIwPIo",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "# importing modules\n",
    "import numpy as np\n",
    "import os\n",
    "from svm import SVM\n",
    "import time\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (10.0, 10.0) # set default size of plots\n",
    "\n",
    "# The following two lines let us reload external modules in the notebook\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 122
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 35517,
     "status": "ok",
     "timestamp": 1590744626338,
     "user": {
      "displayName": "sadra barikbein",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GiCq0YqNHWfcZyly3I_AMTkvS8_tO8PZOQpGtkK=s64",
      "userId": "07513140462530453530"
     },
     "user_tz": -270
    },
    "id": "21K8ZI3Jw_B_",
    "outputId": "e1629f81-6481-452c-b6d9-a3e03ae541a2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n",
      "\n",
      "Enter your authorization code:\n",
      "··········\n",
      "Mounted at /content/drive\n"
     ]
    }
   ],
   "source": [
    "from google.colab import drive\n",
    "drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "DjDaaobawPIv",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Problem 1. Computational Graph (13pts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "kZvYCSIKwPIw",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "In this problem, you have to compute the gradient of $out$ which will be computed in function `gradients` with respect to $x$, $y$, $z$, $w$, $t$ (i.e., $\\large{\\frac{\\partial{out}}{\\partial{x}}}$, $\\large{\\frac{\\partial{out}}{\\partial{y}}}$, $\\large{\\frac{\\partial{out}}{\\partial{z}}}$, $\\large{\\frac{\\partial{out}}{\\partial{w}}}$, $\\large{\\frac{\\partial{out}}{\\partial{t}}}$) using the chain rule and save them in $grad\\_x$, $grad\\_y$, $grad\\_z$, $grad\\_w$, $grad\\_t$ and return them.\n",
    "\n",
    "Here is the computations that should be done by `gradients` in a mathematical form:\n",
    "\n",
    "\\begin{equation}\n",
    "z = matmul(x, I - y) \\\\\n",
    "\\\\\n",
    "w = x * x \\\\\n",
    "\\\\\n",
    "t = -\\frac{1}{log(w + abs(z))} \\\\\n",
    "out = \\text{sum  of  the elements  of}\\;t\\\\\n",
    "\\end{equation} \n",
    "\n",
    "\n",
    "Note: $\\mathbf I$ in the first equation is the Identity matrix. $\\mathbf{matmul}$ is matrix multiplication. $\\mathbf *$ is elementwise multiplication. $\\mathbf{log}$ is to the base of $\\mathbf{e}$. $\\mathbf{abs}$ in the third equation is the (elementwise) absolute value. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "rZK6xICBwPIx",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def gradients(x, y):\n",
    "    \"\"\"\n",
    "    x: a numpy 2d-array with shape (N, M)\n",
    "    y: a numpy 2-d array with shape (M, M)\n",
    "    ouput: grad_x, grad_y, grad_z, grad_w, grad_t which are numpy ndarrays\n",
    "    \"\"\"\n",
    "    z, w, t, out = [None] * 4\n",
    "    grad_x, grad_y, grad_z, grad_w, grad_t = [None] * 5\n",
    "    z=np.dot(x,np.eye(y.shape[0])-y)\n",
    "    w=x*x\n",
    "    t=-1./np.log(w+np.abs(z))\n",
    "    out=np.sum(t)\n",
    "    grad_t=np.ones(t.shape)\n",
    "    grad_w=(t**2)/(w+np.abs(z))\n",
    "    grad_z=grad_w*(z/np.abs(z))\n",
    "    grad_y=-np.dot(x.T,grad_z)\n",
    "    grad_x=(t**2)/(w+np.abs(z))*(2*x+(z/np.abs(z)))-np.dot((t**2)/(w+np.abs(z))*(z/np.abs(z)),y.T)\n",
    "    return grad_x, grad_y, grad_z, grad_w, grad_t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "C76KhQdzwPI2",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let's check if you have correctly computed gradients. You have to get small values as the ouput. The largest order of ours is 1e-8."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "Vv8sJ4q8wPI3",
    "new_sheet": false,
    "outputId": "80419540-b690-419d-815e-7959708a1d9f",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relative error of grad_x: 1.0653302011545749e-08\n",
      "relative error of grad_y: 1.2540139465697192e-08\n",
      "relative error of grad_z: 1.068517929546046e-08\n",
      "relative error of grad_w: 1.068517929546046e-08\n",
      "relative error of grad_t: 0.0\n"
     ]
    }
   ],
   "source": [
    "def test_gradients():\n",
    "    np.random.seed(42)\n",
    "    x = np.random.rand(3, 4)\n",
    "    y = np.random.rand(4, 4)\n",
    "    \n",
    "    grad_x, grad_y, grad_z, grad_w, grad_t = gradients(x, y)\n",
    "    \n",
    "    correct_grad_x = [[ 98.53486691, 178.15428636, 101.50170543, 140.87675175],\n",
    "                      [-29.11111973, -50.35135437, -61.51644322, 396.80894273],\n",
    "                      [260.24004063, 319.32608058, 119.95375672, 653.73810858]]\n",
    "    \n",
    "    correct_grad_y = [[ 175.60725449,  221.87847075,    6.64595782, -101.40000624],\n",
    "                      [ 273.07420644,  308.49236288,   11.29541668, -109.51168111],\n",
    "                      [  99.83039306,   74.98883877,    5.30629792,   -5.51204454],\n",
    "                      [ 284.95194676,  363.24817383,   11.92286996, -270.95042005]]\n",
    "    \n",
    "    correct_grad_z = [[-130.17216803,  -93.01792945,   -6.91408473,   -9.96854119],\n",
    "                      [  -3.81552552,   -9.39164072,   -2.01597763,  174.60396305],\n",
    "                      [-210.03844462, -308.71671769,   -6.22480814,  129.57944883]]\n",
    "    \n",
    "    correct_grad_w = [[130.17216803,  93.01792945,   6.91408473,   9.96854119],\n",
    "                      [  3.81552552,   9.39164072,   2.01597763, 174.60396305],\n",
    "                      [210.03844462, 308.71671769,   6.22480814, 129.57944883]]\n",
    "    \n",
    "    correct_grad_t = [[1., 1., 1., 1.],\n",
    "                      [1., 1., 1., 1.],\n",
    "                      [1., 1., 1., 1.]]\n",
    "    \n",
    "    \n",
    "    print('relative error of grad_x:', np.linalg.norm(grad_x - correct_grad_x))\n",
    "    print('relative error of grad_y:', np.linalg.norm(grad_y - correct_grad_y))\n",
    "    print('relative error of grad_z:', np.linalg.norm(grad_z - correct_grad_z))\n",
    "    print('relative error of grad_w:', np.linalg.norm(grad_w - correct_grad_w))\n",
    "    print('relative error of grad_t:', np.linalg.norm(grad_t - correct_grad_t))\n",
    "    \n",
    "\n",
    "test_gradients()    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "jOGzM16KwPI9",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Problem 2. Shy away from for loops! (12pts)\n",
    "Suppose that we have a matrix called $X$ which is a numpy 2d-array with shape $(N_1, D)$. This matrix has $N_1$ rows and the $i$-th row represents a $D-$dimensional vector such as $X_i$ (called an instance). There is also another matrix named $Y$ with the shape $(N_2, D)$. Note that $N_1$ is not necessarily equal to $N_2$. Our goal is to compute the following value for each instance in the first matrix and each instance in the second matrix:\n",
    "\n",
    "\\begin{equation}\n",
    " \\\\\n",
    "V(X_i, Y_j) = {\\sum_{k=1}^{D}{(x_k - y_k)^3}}\n",
    "\\end{equation}\n",
    "\n",
    "where $X_i = [x_1, x_2, ...., x_D]$ and $Y_j = [y_1, y_2, ..., y_D]$ are the $i$-th and $j$-th instance in the first and second matrix respectively.\n",
    "\n",
    "We want to compute a matrix `V` whose $(i, j)$-th entry is $V(X_i, Y_j)$. Complete the code of following functions to do this task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "3A3BXhewwPI-",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def values_with_for_loop(X, Y):\n",
    "    \"\"\"\n",
    "    X: numpy 2d-array with shape (N1, D)\n",
    "    Y: numpy 2d-array with shape (N2, D)\n",
    "    V: Values matrix which should be a 2d-array with shape (N1, N2)\n",
    "    \"\"\"\n",
    "    V=np.zeros((X.shape[0],Y.shape[0]))\n",
    "    for i in range(X.shape[0]):\n",
    "        for j in range(Y.shape[0]):\n",
    "            V[i][j]=np.sum((X[i]-Y[j])**3)\n",
    "    return V"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "ePCpRf7PwPJC",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now let's test your implementation of `values_with_for_loop` with the following function. You have to witness a small value as the ouput. we get an ouput of the order of 1e-6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "ra5Q3R7AwPJD",
    "new_sheet": false,
    "outputId": "2fbe7ac3-8ca2-46f5-afe4-dde4e7ee3c16",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relative error of values: 9.765821008345065e-06\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(42)\n",
    "def test_values_with_for_loop():\n",
    "    X = np.random.randn(10, 8)\n",
    "    Y = np.random.randn(8, 8)\n",
    "    \n",
    "    out_correct = np.asarray([[  8.765334,    20.010048,    18.260977,    43.856083,   -14.216753,\n",
    "                                 29.56002,     29.146101,    51.783203  ],\n",
    "                              [ -28.092327,    -7.90782,     -1.9417393,  -19.624632,   -33.951344,\n",
    "                                -74.167725,   -19.233025,   -34.774654  ],\n",
    "                              [ -9.0146055,   -4.3492904,   19.111053,   -19.567537,   -23.901215,\n",
    "                                -29.252842,   -13.360028,    -6.035372  ],\n",
    "                              [ -17.440336,    34.22961,     17.316256,   -19.815384,   -17.601694,\n",
    "                                -15.11916,     -0.2324852,   -2.339958  ],\n",
    "                              [ -16.793228,   -13.687567,    -2.0563402,  -13.523884,   -61.872227,\n",
    "                                -71.84135,     -3.8474934,  -34.78388   ],\n",
    "                              [ -5.693445,    15.429427,     6.4837265,   -8.542441,   -18.958473,\n",
    "                                -29.938059,     1.6775647,   -7.4752984 ],\n",
    "                              [ -9.28221,      3.8474603,    1.0413681,   12.016386,   -70.26817,\n",
    "                                 4.750604,    11.754794,    16.634857  ],\n",
    "                              [ -10.45044,     -0.9044826,    0.48504904,  -5.3853908,  -39.49779,\n",
    "                                -6.7880898,  -14.213244,     4.442503  ],\n",
    "                              [  5.0958433,   31.653896,    22.115992,    10.347621,    -0.3687537,\n",
    "                                 11.317683,    14.432735,    21.924892  ],\n",
    "                              [ -76.88806,    -18.312935,   -12.917151,   -89.0802,     -35.949398,\n",
    "                                -56.66989,    -34.12826,    -34.17443   ]])\n",
    "    out = values_with_for_loop(X, Y)\n",
    "    print('relative error of values:', np.linalg.norm(out - out_correct))\n",
    "    \n",
    "test_values_with_for_loop()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "VkWIWLLDwPJI",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now, let's make it more challenging!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "-WUvyoNTwPJK",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def values_without_for_loop(X, Y):\n",
    "    V=np.dot(X,(3*Y**2).T)+np.dot(-3*X**2,Y.T)+np.tile(np.sum(X**3,axis=1).reshape(-1,1),Y.shape[0])-np.tile(np.sum(Y**3,axis=1).reshape(-1,1),X.shape[0]).T\n",
    "    return V"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "eB47UaFbwPJP",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now let's test your implementation of `values_without_for_loop`. You should get a small value as the output. Ours is of the order of 1e-13."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "1h04OBfXwPJP",
    "new_sheet": false,
    "outputId": "1e5e9544-f655-4686-d434-f2ad8b7af2ae",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.5613914432908883e-13\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(42)\n",
    "def test_values_without_for_loop():\n",
    "    X = np.random.randn(20, 25)\n",
    "    Y = np.random.randn(30, 25)\n",
    "    \n",
    "    out_correct = values_with_for_loop(X, Y)\n",
    "    out = values_without_for_loop(X, Y)\n",
    "    \n",
    "    print(np.linalg.norm(out - out_correct))\n",
    "    \n",
    "test_values_without_for_loop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "m7dWagC7wPJU",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Finally, let's compare the running time of functions! We have provided the following code snippet to do it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "0J2LpDawwPJV",
    "new_sheet": false,
    "outputId": "0fb3b0cc-5a7b-47a1-d8cc-6d88e7791bd5",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running time of values with for loop: 10.340021848678589 seconds\n",
      "Running time of values without for loop: 0.10039258003234863 seconds\n"
     ]
    }
   ],
   "source": [
    "a = np.random.randn(1000, 150)\n",
    "b = np.random.randn(800, 150)\n",
    "\n",
    "tic = time.time()\n",
    "values_with_for_loop(a, b)\n",
    "toc = time.time()\n",
    "print('Running time of values with for loop:', toc - tic, 'seconds')\n",
    "\n",
    "tic = time.time()\n",
    "values_without_for_loop(a, b)\n",
    "toc = time.time()\n",
    "print('Running time of values without for loop:', toc - tic, 'seconds')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "Dg9zXTBjwPJZ",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "You should see a huge improvement in runnnig time of `values_without_for_loop` compared to `values_with_for_loop`. This experiment shows how vectorized and matrix computations in Numpy could be much more effiecient than just naively using for loops in Python. Numpy is a powerful module with a beautiful functional API and many of its functions are implemented in C which results in a great efficiency. Nevertheless, one of the biggest disadvantages of Numpy is that it cannot run on GPU. By using deep learning frameworks such as TensorFlow, PyTorch, Keras, and ... we will sidestep this problem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "QPSPsn6RwPJa",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Problem 3. Jacobian (10pts)\n",
    "We have provided the function `func` for you below. You have to find out the operations done by this function and then complete the code of `jacobian_func` to compute the jacobian of the output of `func` (i.e. x_4). Note that the Jacobian of a function $\\mathbf{f}$: $\\mathbb{R}^n \\rightarrow \\mathbb{R}^m$ is an $m \\times n$ matrix defined as follows:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "pAkKLf-rwPJb",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<img src=\"image/jacobian.png\" style=\"width: 400px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "uHLyVvfOwPJb",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "**Note:** Write down the computations on a paper and take the derivatives manually. Then, implement `jacobian_func` using Numpy. <br/>\n",
    "Approximating the gradients using numerical methods DOES NOT get any points!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "dL5-F3sawPJc",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def func(x):\n",
    "    \"\"\"\n",
    "    x: input vector as a numpy 1d-array.\n",
    "    \"\"\"\n",
    "    x1 = x - np.ones_like(x)\n",
    "    x2 = x1 ** 2\n",
    "    x3 = np.empty_like(x2)\n",
    "\n",
    "    for i in range(len(x3)-1):\n",
    "        x3[i] = x2[i] * np.exp(x2[i+1])\n",
    "    \n",
    "    x4 = x3[::2].copy()\n",
    "    \n",
    "    return x4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "yOtSVKTLwPJh",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def jacobian_func(x):\n",
    "    \"\"\"\n",
    "    x is the same input passed to func on which the computations are done.\n",
    "    you have to return the jacobian of the output of func.\n",
    "    \"\"\"\n",
    "    x_4=func(x)\n",
    "    jac=np.zeros((x_4.shape[0],x.shape[0]))\n",
    "    for i in range(x_4.shape[0]):\n",
    "        jac[i][2*i]=2*x_4[i]/(x[2*i]-1)\n",
    "        jac[i][2*i+1]=2*(x[2*i+1]-1)*x_4[i]\n",
    "    return jac"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "K_n0hwdhwPJl",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Problem 4. Preprocessing (12pts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "0wrsFHFiwPJm",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### 4.1. Loading Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "E_wDDz_GwPJn",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Fashion-MNIST is a dataset of Zalando’s article images consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28×28 grayscale image, associated with a label from 10 classes. Fashion-MNIST is intended to serve as a direct drop-in replacement of the original MNIST dataset for benchmarking machine learning algorithms.<br/> \n",
    "We have provided this dataset saved in some files under the `data` directory for you.  Training data has been saved in 5 files (`train1.npy`, ..., `train5.npy`) and test data file is `test.npy`. Labels are also stored in two files `labels_train` and `labels_test`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "EV0zl_h-wPJn",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def load_data(root_path):\n",
    "    \"\"\"\n",
    "    root_path: the root directory of data files.\n",
    "    outputs:\n",
    "    1) test_data: test data which is a numpy array of shape (10000, 784).\n",
    "    2) train_data_array: a Python list containing 5 numpy 2d-arrays each one of them has the shape (12000, 784).\n",
    "    3) test_labels: a numpy array with shape (10000, ) containing the labels (0 to 9) of the test set images.\n",
    "    4) train_labels: a numpy array with shape (60000, ) containing the labels (0 to 9) of the training set images.\n",
    "    \"\"\"\n",
    "    train_data_file_names = ['train{}.npy'.format(x) for x in range(1, 6)]\n",
    "    test_data_file_name = 'test.npy'\n",
    "    test_labels_file_name = 'labels_test.npy'\n",
    "    train_labels_file_name = 'labels_train.npy'\n",
    "    \n",
    "    train_data_list = []\n",
    "    for file_name in train_data_file_names:\n",
    "        file_path = os.path.join(root_path, file_name)\n",
    "        train_data_list+=[np.load(file_path)]\n",
    "    \n",
    "    test_data=np.load(os.path.join(root_path,test_data_file_name))\n",
    "    test_labels=np.load(os.path.join(root_path,test_labels_file_name))\n",
    "    train_labels=np.load(os.path.join(root_path,train_labels_file_name))    \n",
    "    return test_data, train_data_list, test_labels, train_labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "SP92E2aTwPJr",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now let's test your implementation of `load_data` with the following function. You have to get no erros after running this function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "UT5AqBVqwPJt",
    "new_sheet": false,
    "outputId": "aa5a3323-2821-4837-93dd-951c4e031a51",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You have successfully loaded the data!\n"
     ]
    }
   ],
   "source": [
    "def test_load_data(root_path):\n",
    "    test_data, train_data_list, test_labels, train_labels = load_data('data')\n",
    "    assert test_data.shape == (10000, 784)\n",
    "    assert len(train_data_list) == 5\n",
    "    assert test_labels.shape == (10000, )\n",
    "    assert train_labels.shape == (60000, )\n",
    "    assert train_data_list[0].shape == (12000, 784)\n",
    "    print('You have successfully loaded the data!')\n",
    "    \n",
    "test_load_data('data')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "QeWd04JawPJz",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Each one of the samples of Fashion-MNIST dataset is a $28\\times28$ graysacle image. Pixel values are between 0 and 255. Here is an image of a t-shirt in the dataset:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "IIXLrlDjwPJ0",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<img src=\"image/mnist.jpg\" >"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "4ghWoHFdwPJ1",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### 4.2. Concatenation\n",
    "You have loaded the training data as a list of five numpy arrays, each one of them has the shape $(12000, 784)$. We need to concatenate these five arrays into one array with shape $(60000, 784)$. <br/> \n",
    "In this part you should Implement a function for cancatenating a set of numpy nd-arrays given to you in a Python list (Note that the list does not necessarily contain five arrays and the arrays might not be 2d-arrays! You have to implement the function for general case). You have to do the concatenation along the first dimension i.e. axis=0. You are **not** allowed to use `np.concatenate`, `np.hstack`, and `np.vstack` in your code and you have to do it from scratch using Numpy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "kH9Hq0f8wPJ2",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def concat(list_of_arrays):\n",
    "    \"\"\"\n",
    "    list_of_arrays is a non-empty list of numpy arrays (with arbitrary number of dimensions) with the SAME shape\n",
    "    output: array_concat which is a numpy ndarray\n",
    "    \"\"\"\n",
    "    array_concat = np.zeros((sum(map(lambda arr:arr.shape[0],list_of_arrays)),)+list_of_arrays[0].shape[1:])\n",
    "    tot=0\n",
    "    for arr in list_of_arrays:\n",
    "        array_concat[tot:tot+arr.shape[0]]=arr\n",
    "        tot+=arr.shape[0]\n",
    "    return array_concat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "2UWd30fswPJ7",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now let's test your implementation with the following function. You have to get no errors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "p8X2IrvtwPJ8",
    "new_sheet": false,
    "outputId": "e8e7f4b9-b8b6-49cc-c88e-c813dea14ec3",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "you have successfully implemented concatenation!\n"
     ]
    }
   ],
   "source": [
    "def test_concat():\n",
    "    t1 = [np.random.randn(10) for _ in range(10)]\n",
    "    t2 = [np.random.randn(5, 3) for _ in range(5)]\n",
    "    t3 = [np.random.randn(1, 5, 3, 4) for _ in range(3)]\n",
    "    t3 = [np.random.randn(100, 5, 2, 5, 14) for _ in range(3)]\n",
    "    t1_concat, t2_concat, t3_concat = concat(t1), concat(t2), concat(t3)\n",
    "    assert np.array_equal(t1_concat, np.concatenate(t1, axis=0))\n",
    "    assert np.array_equal(t2_concat, np.concatenate(t2, axis=0))\n",
    "    assert np.array_equal(t3_concat, np.concatenate(t3, axis=0))\n",
    "    print('you have successfully implemented concatenation!')\n",
    "\n",
    "test_concat()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "XoCZgrU6wPKA",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now let's concat `train_data_list` with your `concat` function and save it in `train_data`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "f1wdojM0wPKB",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "test_data, train_data_list, test_labels, train_labels = load_data('data')\n",
    "train_data = concat(train_data_list)\n",
    "assert train_data.shape == (60000, 784)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "i3mWvGRvwPKF",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Next step is visualizing some of the images we have loaded. But, first we have to reshape each sample into a 2d-array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "I8aVnMQPwPKG",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### 4.3. Reshape\n",
    "Complete the function `reshape` to reshape a flat array with shape $(N, d^2)$ to an array with shape $(N, d, d)$. You can use `np.reshape` in your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "01O6fY-ewPKG",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def reshape(x):\n",
    "    \"\"\"\n",
    "    x is an array with shape (N, d*d)\n",
    "    output: x_reshaped which is numpy 3d-array with shape (N, d, d)\n",
    "    \"\"\"\n",
    "    d=int(np.sqrt(x.shape[1]))\n",
    "    x_reshaped = x.reshape((x.shape[0],d,d))\n",
    "    return x_reshaped"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "nMz0L6LNwPKK",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now let's test you implementation of `reshape` with the following function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "1js5BJiMwPKL",
    "new_sheet": false,
    "outputId": "643c8f62-1a31-4ca9-b401-f17deade3746",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "you have successfully implemented reshape!\n"
     ]
    }
   ],
   "source": [
    "def test_reshape():\n",
    "    a = np.empty([531, 100])\n",
    "    b = np.empty([3000, 4096])\n",
    "    c = np.empty([10000, 10000])\n",
    "    a_reshaped, b_reshaped, c_reshaped = reshape(a), reshape(b), reshape(c) \n",
    "    assert a_reshaped.shape == (531, 10, 10)\n",
    "    assert b_reshaped.shape == (3000, 64, 64)\n",
    "    assert c_reshaped.shape == (10000, 100, 100)\n",
    "    print('you have successfully implemented reshape!')\n",
    "    \n",
    "test_reshape()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "Sk-AiDuwwPKP",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### 4.4. Visualization\n",
    "Let's apply your reshape function on some of the samples in the dataset and visualize them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "zXemSPgtwPKP",
    "new_sheet": false,
    "outputId": "5f739a83-da00-4857-b090-5d9ec34759d1",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV8AAAD8CAYAAADQSqd1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsvXeYXVd5qP+utXY5ffqMZkajPnKRZctFxsZgO9hgB2NTQuiBkEIgBRJI5bk3CfcmISQPJIRwKYEUQgm9gzFgqptsy0WyR5asPpKmt1N3W+v3xz4aSZaLLDnee3457/PMM+2Ub52917e+9a2vCGMMLVq0aNHi2UUmLUCLFi1a/E+kpXxbtGjRIgFayrdFixYtEqClfFu0aNEiAVrKt0WLFi0SoKV8W7Ro0SIBWsq3RYsWLRKgpXxbtGjRIgFayrdFixYtEsB6Nt/shfKXj6XTCQHHZdd13NbJ9m+eTf8ddYSvmbogx9x5mrP/716i8YmTXuv7+oviWRH6MZwwhuOwVg6BlIR795/w98m3XU7vv27FeN5Jz0nbGE6HNI1h5ycu4RUXbmX7xfqkx0+95XJWvuFRqldOnvS/NI3hKNbgANHEFCbwT/6nECAk6GjxT2kag/fizRy60sKqCfruCZChYX6VTemXDzN67wDrPj2L3r7jpNdK0xgAuOx8uGvbCXoKwBpaTnhw9HGf8nTG8Kwq3+NRXZ3UP1fgHat+yD6/mz11myt/aSvnv/Ego34ngVFoI+C7sCF3mHZV4yPD65IS9ykZu245z/+tu3lBaSc35Ws85NfZF3bwN3/2HKZfdxGd/3ZH0iI+KQN3Fnl19xbOdabZE5ToUxUyQjOpXX5UOZdbN+aTFvEJqf7Sc+h9xx7OCo8wsrCM67bvYMCe5d13vIKXnPcgXXaVbx2cJmcFOLd1MlYt4b5oX9Jin4Q6dz0v+uIWznaPcFfVp82qoTC8tvQwGvjE7EUUVYMPffPFrP7TO04yYJJC2A4m8PnHfbfz5YWL+PzuKnYkCQNF5uopvMhiZrSbuW39qJU1xv4a/uCsUb40fgneVWNJi38C+qoLMVJg37UDfdc2xEXnUvnrOm9b/RPes/UlDP/ZLOG+AwCoDWfhLStg/fDe03qvxJRv6euG13Tdxq3z55BVAVIYHBHyYHUIS8Qrut38/lBtMFbEl26ALduSEvlJ0Y7gJ/92KfavRcAjjDSG+fcdl+H2K7KTJ1tiaWPXXA9+l+JwmOX5mZC3jr6QP1p2C9saQ2wrDwJzSYv4hBy6xjA+Oshg9xwZK+D22TVIYbh0eC8zfp6xRomLeg8y2Siwc6qX7kI1aZEfl+5PjnHI6+CQ10E9cthV7aXbrfCG8VcxXi4y1D6HxHDVLzyIe7fLrs0n76aS4Kh1/uKbf5+hmyFfkBghcKqamh4EA4OWIMgLzAM5LC/LP3b8MuLF03STLuWrKj5qtorJZrBKRcJ7HyJ/PXyKIVbzACGg2tsAEHNlHCU43dmdmPL9lb7buae6hl6nTEPb/OQfLqO43wMJRgqkr2n0OExcaPHSm26noDze9pmvptb6dec0kS348Ycu467ZzQQ5ST4nCIpJS3ZqLHx/GV3rK9zfWME5zqPceXgl7QPQMDaNKLHb5JRYuW6CA4e7qPoOtorQRlAJXCIjkRja3DqhVuSsgGK2QaAlbtJCP4bnPuDTbe9lX6ObuSCHFJqsCpjyCpScBt09VaYaecq+y97ZTgoZjwJ7khZ7EXXOMCIb0v3Og4zcOoz0QEYKNMgI0GAsEBGU2wW/cMNW+pwFbsdJWvRF1Pq1RLbCZF2ivbMn7CqEZWHCEITARBpdqaB6uwlLGdTFGzD3PvS03y+xWVXVLhkZ0KZq7Kr3IQMYvzSLtqF7W0DQbZOdDFj5F1vIvcyn2y5zbXaKj1780tMa6H83RgpkBFpBrVsd9w/QViKurKdFYVRT1S5F1eBH9R6+cOEnuLm6kowIOFRpo8RU0iI+IUpqrEzA7HwexwrpcGsU7NgqlMJQCx3a7AahkZTrGQAKSQr8BGREwAp3hnGviDY2AL6O3W+NyCKjQqRrmJwrIIWhbd1qokf3Jix1zMEbe8hvg0d2DBNurBGFEo7ziBgtoKGw2mIr+c5PXkjpQEjm4lpq5nNtuBOrFhGUHKavvZzwynmKXy0SOYLpTZrSLkXbvpDRaySm02ftxzV+h4NtS9RTv/xJJKJ8zRWbmAynKagGy+x5jgTtFPfXaNsRIMamEUJgPJ+5F51FEeh35qhph0/Mn83eP1asenUSUj85XrvArhh08xM1EhCgLQjS6y5dpGPLEa7Jenxgth2FYSxoo2FsJvwSfng6t9azx56DPeRKDfwDecYmXOZWZLGUxvMtdCRBGA5l25ibKPKCjSP8ZM86GjdeSuabW5IWfZHL87uYiQq8tniAA14n+2udaCPwtUWoJZbUjFWK1DyH3974U3qsMn/xnptY+/qkJY8JCoZwXZ2oYdFZrNGebSCFwZYRGRVgSc28l6XNrbP1wBArXruHnZM9tH2lSOn0XKbPOO637ya85mLKy20GfjKH/uDDqA1nsf+lXXQ8JFn2vVFMucK6b86gujp55H+vp+9OyHxz62m9XyLKd/yPYqvE0zYTYYla5FBelaPeKVF+OzIAq2Go9Uoab72cieBHrM8coaFt3nTOXfyEbBJiPylGglHxd3HUCWRApcMt95QcjdIoyga2iIiQ5ISPK0Pm5vN0Jyzfk+Hmfd57/lf5QPFFjN0xgNYSX0u0FthOiKU0Z3VPcNehEn/S/z06nSrfvuByhr6ZtOTHeP+6DVRuXsNI/yPc2H4fH29czYyXw5KaktNgtpHDCyy6C1V+PHUW++c6WPv6+5IWe5FLrh1hsl5ASc3BuXb27OlDlRXSF8gAMPHuz++KuGbzdvKWxwMPrSS4QFD6bLKyH09m7zQHrutHhm20Z86HXaMsf+9OAExnB3S2Y+VyVM/rZ82XGjh7JwhP870SUb7LXjbCd5adS/38Id76z19kxs9z1u8/RF75uDJYfJwrQ6aDPF/eu4nel2aSEPWUsJb1oe3Y5QBNq5dYCbvzhtmzoSc58U6ZiahGROwn1UZQ0y6BUYixtHlIT2Tlq7bxYdZT+fV+zn/zI9x3YAhlRbhuiBTx3lcbwRWbdvJ7K68ADEPcnqzQj0Ph+j185asXcOX5O2i360w14i1TI7RRUhNpSS2wcV+0j17GUxPtIItFHpzo5+7Nn+Ll51/H1gf/k4/OreF7k+cyUS0wPVtASoPjBjiRZHV2is987hp6njdJeUu6Zka4Zx9r/ngfe997OfkxB33OCpQ3iAgiDGCUoLqxh8gRuN+9m1CcvksxMZ9vODaOPTZORgRoBP3uPK4MCYxCCR0flFg1FsIs3pbOpMQ8JUytfrKFK2IlrBU4C+n3+QLYzRtJoVEijjbRRqB7HifWNEUIywKl6N66wMSriwipse0IrQVIsKRmupHneT27mcQ+dniSQo5GYmSVT6AVGRWSsQKOmldhdFxeVAoUL4B3+Vn84OIP4oo80fQMZ3/+d+h6QBDmYmVl9cfuOE9mEQY+LTfT9UjEh97yafo2BvzGJ15FeOhw0sM4AeUJZHAsjkFnLERkQGuc+ZCZs13O9Cw9uQy35kS/ZW4jbXadCInmmJJSQjMb5ulyKhQPpuMme0KUwhznFhWaxcOGMCsQ0eM+K3XYQpIRAbaIZ7oUmg67yu9e/ONkBXsKTBiCNsiGjxcpspkAz7MQIla8oZZMVfJ8Zc8F8eN1eu+nVwzcz88rZzHlF3BViEbgyJCC7dGeq+MF6Ys8Gb/E4co73sqP67E6ecsLf0jlxjILz2swd5GP3xURdoboHh9rdQV/X4HcV+7ibw69mCu+/i4WLh1KeATH0dRLzmx8iI46ppOMEmjXwp5roLwzv4cSU75CxdpqIXTRRpCRARKzaPXaImI+zJKTfrzipBiRz8XKVsSK16j4ZwQIY0449U0r4uINtMksqumwVkLHFjCGS3LpCWl6Mmqr2heVUyEXb0W0ETjWMfcDgJAp3Ik0J70rA6aDPNXQwZFR82/x94Lj4VjRCY9PA3LzHGf3TXBFJnYZfuzeKxFbS9g7szhHbGRNIisKKhaN6SwMNAD44Iqv88LnPEjj12eTFP8JMUoQuYqwYGOkOPZlK9TRzeAZ7D4SW0aPWh++ttBGLiZUHDUgpdBoI2KfY9pzFISIle3jXAeToknyZMxsLHFzzUU2Q8aPfrdFxBWuxnvxZtzv3J2kiE+O0YxdbmGFFo4V0vBtpDREWlLIeNQ8h0Km6RtSClLqdijJOlkVUAlcujMVaqFDVvnUIxspDDk3fS6ggZc/jLfpXG7Srwd2MPLCj/KRzcMM2LPs97uZCoooNFLExtWBeieHgSu+8S7aRhR9W8qps0+Ub9BKoHyNeMxOSTZCrMYStnwx8eS+rH0PUmiipgKWzcugMHTYNSb8ImEm5QrMtmLXQtPPG3vmm18yDi5PO7/wjjuYi3I4zUVQN2+Noqrzobk1/OY/fCVJ8Z4cqTBhSMfmCZTU5JzYAnOsEGNACUOo5aJVbIIUKl4Rf953VtbS78xRD23a7ToZFZJVAXnLW7SEgdT4ewEQAv3ACGYk3iHtCQIeqgyys9FPLXJZm5lg0J2l35mjoBr8ef93ARj+3bvo/fDtmLtTlLXa/FydskFGBqFN7H5oIrQh6MiQG/fis4YzIHG1MBUUcWWILSICo8hJn/koS4QgJ33GdAm/Pd3K17j2YoTDYy1gEZnFMJs080vtd7OtMUSmGW0SmWPr8pA9w1XZI3yKFPnmjkNIgdEwOVtkWecCedtnIiiCE2Cp2OKyVXRsy67T54QXSmF0RL8zTy1ycWS06CppU3XqUZx0odN4LzUV1tE047ftfB1TlTz3qCFqDSeOtW4SVm3+vfMylpOOxIonwmoYRGQQgT7B7ysDjbEl9uF5GF5NNLLr9N/jmRD0THHlMUskJz1qOk45tEVEqBWNzhSt8o+DLmRi/250LMzsKDIEq5aMXE+HS12bH5TbWO5Mk5EByhhUc3fyrZkLuKg/XafRJ9C0Gtctm+TQfBt520cqzUBpgUPzbVR9h5wTUPUcrOWDhKOHEhb4iXFlwIRfos2to40gb3kERlFQXpzlZqXQaj8O89wLWF3aw3N69tFpVRdDRz1tExjF3lo3016eesJyPhXOQrTobtCWRAUaIwVRzkLVQvzl7QhtkCOn/x7JKd/jtk2PtbaOXjApNJaMTogkSCORq8CAeIKzNRmke/EAuNfzqUQuDeOwyppiLsrRq8rs9Pu4vG03tbRfBKDkNNgXdOJFFpalsYTGtUPmq1lKHQ280CJY0Y1Io/J9zCGgIyO0kbgyJKc8PG3hqIiK76aoGsLJ+O0O2z55Hto6LupHgAib80NA5Ar6LuzG3JdO61fmcmglAInSBnPU8lVi0QUhvQi/zcE9g1jr5JRvU+iFMENbpkbQzMsNjNWML5UERmELjbYTk/KUaPS4x5IsjptDMgQMRGn3WQPt0ufq4giTUYkrM3DdwHr+5cDPyclRDodtZNIcL9ecEGPVEvmsx+GZEoWcxyv67uUfHrkWgJ5shbl6hqDkpFJ5CSEwQE768X0vI7LKRwpDRoT0OgvkrG7KKt0JL/tfIhB5DycbIAQodey03Bho1Bx01QbZRm96EvROZN0KZGRQXgSRiV0NUsQuCCkIC3bsC7YEwnEet1b3qZC45Xvv1BBDy2eYDXOLCRaBUURGLoY9GZVuy7H4wxFm15+HaRbQkb7haGRTbZlIf7QGsNYuIJnlcNix+Le7GwOc7YwzZM2x1k5jKZoTCbQkY4csaEnDt2kYh/n5HE4mwBIarSVWPd3b9ryMJ3I5cHFkiCtDpsICbjP2OogUTkoy206gKZNq94lqFuFEAWEg0s0jDxNbwsI10BZSXq3phVQmvBhbxQkWzRBXIwVWPSDK2shQL5aQFKFBdXeddoJI4pbv4f1dfGTsakwt3taqmoyvljDIIM4L7z59n/azQlQuM/i+OF3V6l9GddMQwkD24ALRQ48kLN2pcd3AJgAO/clz+fDzp+hkJ3/4o1eT76kx+Ip0bg8XiWKrfGxvF+6kRemQQUTwt2M30nm/JHIzbBk6B1UTODv3nHYu/n8numk9/edrrqc8XKS8XBLmwevUCA3aNrTtVFhVQ96kN+56zT8brOk50Bpk8wBEH7M+RMMHJYk6inFAUJS+HZWs+diRQWeau/GijenKIiKDsSUi0NizdZzRGUyjcdrvI0zaVtAWLVq0+B9Aq4FmixYtWiRAS/m2aNGiRQIk1734OIJrL2b/DTZWRRC0xR76/Kgk/wsTrO+YYPzyhZOek6ZOp7JYRJfLT/icow0GH0uaxgBxU9NH//CsxULw0TIPE0iybQ1W/dZhoumZk56TtjEAzH8nbjU1X83iVR1MJFj/6/c84WulcQw9t7dTCVweGFnJhrMP0p9d4NZd61n3zxHc+eBJj0/jGPa+93JUXeC3xz5fEQm6N0zS8erxx50vaRzD4T9+LoVRTdvOCgeuL+HOxKnHXZ94/Ia4S6J78fHsv8FmxXlHWFeawpYR7VYNKQy3HDqbN/Xext+xMWkRnxRdLnPwS+fx9nN/hBQGT9u4MkAbwbA7xt+tTbf8R9n57vX84S9+gyFnmnZZIyNCDoXtvCA7wyVvfycr/yJ9NXAfizzvbKbnMjhugJSabLFBECjMFZsQt92ftHinxOybLqeHEVbkZ1i7eZLAKFwZctbgOBMbV9J1Z9ISPjX6qgvpvXAcKQybu/dz1+Qq8rbPC3of4davnQXXPLGxkhYm33o5vS8apWh7vH/Vl7nfG+AzY89hvFZEfa2LaGr6jF4/FW6HtrWzXNB5iPX5scXwmm67zPqOSYry9E8Tn01etvZBxoM2PG3zyuJDaCPY3ejlqzOXJC3aKZNds8CwO8agmqesswRGMWxPYQtFOLwE0vSAqUs7sOyIlZ2zDLQvMNQxh21HTG9IX/eTJ2L6QkOfu8CBaie3HDibb911EeNekTWFaWbOWxoH5OMXZ7GkJmsFLIRZurNV+nILlKMMl3Wno+/cU1FdDrXA5nClxIemruZvd17PaLmd9e2T6NUDZ/z6qVC+1yzfyT2TK2hTdTbmRlnuzLDKmaLbrZAXaQwMOpmb2u4jMIqvHbmAT81fyHjQxoyf5/0DP0eduz5p8U6J/73h2wD4SAaseTIixDOK/aHP9cNnkEf5LDK/HqJIcGShRD2wGS8XcayI2rL0J7oc5dVX3c7IwjIKtseNq7bzl9d8meWZOR5Z6MW4SyBoHChv8Ck4Hgu+y4bCIXKWz665HoqqwWi946lfIAXkz59hvpql5jlcXdrB+d1xPO94vciB68+8LXniylfmcqzKTDE2U+I/9l9GhGTInuag38XI3DJuq69NWsSnRObzbHYF9cjGVSEHG53srnXT5y7wgA+1lW1Ji3hKTIYlAGrapUeGrLQiGsZiX9DOQpjuzKqj2M2uIbWGw8fO/gy1hkOtkcactidmVWaKvmyZVblpdlZ6uW1hGE9b5C0fmQ+e+gVSgJ0LqAYOUhguy+7G14qh4hxtqsah2tKYD5GWSGlw7YCFKENgJEXXY2TnIF3bzzw+OXHlq89fx0PVQXLN4tdfPrcPR0TsbvSwd7KTuxdWJyzhUxOdv44DYY3AqGan1ogZL48tIz47czlz61KeH93koeogu/0+irLBr654HiWZoSh97quvYudsb9LinRIDP60R1ONGq6/5f+/iNzbcBsDK76bfxwhQfeVzON89yLyfYUe5jz8YvIXzC6P0OmWqoYOuLI17yXZC6oHNb6/+CX/2tt/ieZ2PMu9leW5uN49uTWd1vMdSnstRncmiteSjf/lKPrD8uyw0MuT32CjvzHcgiSvfKGsxVi/iWhF9uTIYw0PeIDN+jjW904ul9FJNc0dbj2ILa4U7Q8H2sEVETvroJWJ4bZ/pX+xeAaAxdMqIS3J7Ut8+/iiqHpArNVjjThBm4Cz3CPmsh/CXhvtqbp1ip98X925TIW//29+hx4qjfXoyFWQj8Sl7SoShoj1TZ9gZw/nePZzrHkIIQ6cMyR1ZGmPIb493e7WGQ/vWSbpVnnO6xtCbF/Daznw+pOJTmKrHdQNmmt1aP/rI89lQOELFd+l2K0mKdko0elwmtcuBSgddbo2Pbns+L+jcwcPlfirR0tiuA5S/0Y8Smj4Vb23/cXY9vSrH1ZmA6IddCUt3aqjxObSWDDvjdI5EDFpzhFoi9qe4JOZxVNf73Flex96FTn6j76d0f/wOXlWYpxxl6HRqtO9YGr7r6GCOjAq50IlVzL6gh/7cAu3SYui/9iUr3Cky8NMyBBIhDNHO3Vy9/WVc1b6TwQ/aFPedeVHM5JWvAT9SVBsOy/LxCl9/tI3lzgxHppaGbyjMSMo6g6/jkKBoLD5ZL9oNPG0Rprfr/Qn03l2mql2KMo5A3FYeZHcY32S9W9NegTUmHD1EV7HKdJQnd8RjQPnkXZ9obj5p0U4JtxC73/pyFe6uxy63nUHc0bjdrmFXl0a0g12JVYstYgtxd6MXiSEw+oxDtJ4txNYRRDYiau769o9283BtAK/TjnszniHJK18BGSukXo6rOAGs/8QktghRSnOgmv6T0TAr6JI16oFNu10jMyGpRBlsoSkoj6C4NCYMW7YxG+bZ6serRS10KGub902fg/xZWuv/nUxkBAEK675dtEkHJZbI5w/YdsScn8VRIR974EoA/s+hG8jIIHZjTSwN94mqw/LcHJ6Jd1FfHtlE3vL4Tm3otEswPtuYMMTd67Jh8AgA+Z0Olcjl4IvN4ya6PF0SV75aCYQwFNrrxybJVJxJJdXSCKvRClwREWlJt10mN2ZoU7VFf7Wxl87kL6gGjWYB5elGnsAoPrXj0oSlenooYbCJ0NUqOelQcJbGZAfQWpC3fCyhkQfiRfDeQ0O4MmA2zJEZPTnbM610ORV+UI9DsqxdOVwZstfrSViqp0dxvyF4c7yTFVHc8NeafWZy05JXvo6kw63hWCFtdry1jaZnqGkXy9I0lsKBG1A2NpEWrHKm6NhRo6jqBM3OHDoVeYSnRk4eS4OuBzZF6eNVlo7fGsCSmkHrmJJqd5eGywQgiiQ9ThnruGaZwf74LGTGzyFmlob7xK6AK0IO+vFZQWY6bgu2o7IsYcmeHuUbKkw/N5bZLhsiI3Dnnhm/e+LKV0SGF3U/TGeuzqA7u/j370xtxGvYlJz0Z7gFpfhihFpykTuGuOMBNjpHaEQ2ERJTWhqxmQCXZvay1o6vw/SDPbTJiGwp/dfgeDJWwL5mUfiJqIq1FKrZNyl9u8BNbfdhC01YiuV2ZiXaSF7dswVTWxoLSf/3x6lphx4rDvErr46bJNw/NpiwZE8PrQUzN8Sfec9H72DGyz9jPRkTV77Ho47rgLZ9rJ/+rnny6uSCNGnDCCiKWMGusOLIjV1BNx1ODVtEiJR34jieojy2UHRtM2jgvGVHkhPoNCjYHnNRbC1ORpKFYImceAJdX3yAfUGcoJPpiw/alAer3QmmwwL0L414a7TG0xYb3fjeMR1xrZPqRD5hwZ4eq3umUdaxxbviu9QGnpn5nArlGxiLc9rG2FY5tiqGO4v05spsLqU/D1z50C41qzpmmYriCfOlqUvosqvklA96aYQHATSOa5TZdfsRylrx7sHvJCjR06fNbjAexJEyh8ISw4WJhCU6dXStxrvvejkPL/TzgpVxC5fS/oiMCLhtYRixkP7QS4Do0b3YIuJ7lXMBOH/1KJ62ad+2hHxwwMEfrGRZ+zEX1urSNFHm/yfKN8pKRv0OljkL7Jw75owv7gdtlobSUnWDLQQFy2NPGGdUbJ/sx5YRtogwQeIf8ymTERF+01dtLIXEMBGlv3/b8bgqZD6KD0lmogIla2m5TQa/ZpOxArLNXV92KqCss5SDDCZMX9udJyKnfOxmbZYON96rF44sHfkBGr2aqcoxaz3Qisz4MzOfE9cKQU4y4+fptsscGW+Pe7sBpf0hjchetGDSTHkVPBpkuKn7fn7zgTcC0Li7iw6rSq+9QHvv0khtBehWikeCeGsb7drDNr+fPxt5ecJSPT2y0l9sOLnH62WlO5WwRE+P3Ffu4rziYarNBB31o62MB234WkHKmk0+GR1WlXYVK92z82NklU/+G/cmLNXT480v+DHhcdmdD4wPsOrfdj8jr5248gXYU+7ikN+BqVsIFQ/UHa+hjUCTfuu3fSd8cXYzB4NOGtvbAejeFjJS7Wdfoxv9o86EJXx67PO7F39+qL6c+Z1LS/6S1WA6iK2VjAwW03OXBDK+/zusKvXIxlqzCoADXidlP4OuVBMU7ulROy6vvqFtpDCp61T8VGydG8KrHhuHMYJwbPwZee3EHTD5wz777x3ky8NFlt9yTNGKnfuoBr3sqvQCs0/8Aimg84v3sf0/Ndspsoo7ELZD9mtb2Pc12G85DDj3s1TO22/Y/noq31lGH3Hh9G//05UMjS6daA2Acb/EnnIXcIhvHtlI59DS8JMCYOI75QP3XstrN97D2OH4s//J6DrKOztYG4wmKd3T4iP3XM3Kzwkc7uHW//U8Rl8VsI6lk6wDMHLrMO3jx3y80QPP3E681b24RYsWLRIgFW6HFi1atPifRkv5tmjRokUCtJRvixYtWiRAMq3jhUCefzYv+dzP+d7kBuqhzVw9y8xcHjcT4PsWth0x3DtJwfJoRBYH/mMdXZ881q45TW2mP3ng53RKhy1ehh5V5Qvzl/ALhREGrDKREay0LK649430vnRHfJqt41jHpMcgbIfosg383r9+ntsq6/G0Ra9dZke1j/FaidcObmEmLDAelOh35tiUOcBfvv3Xcb9zDzTPChIdgxBgDGp4DUdeuIyFdRqrKiiMggwgzAlEZLBqML8O/O4IVZP03Aulzx5rAZz0dUAI/BddTN+f72HLvcOomkSGECz3oWxhz0n87giUAWVY8xmDdeuJIVuJj6HJzn/ZzN4b/oWpqMq8NkxGWW6vDXNt4WG0EXSrgHZp8fG5c7llU+cJ0Q9pGcNR5l9/GbPnCLovGWemnMetwXd7AAAgAElEQVS2Q8pjRbruVjgVQ/HzJ7eRfjpjSMbyNYbZ9wbcMbeWauDExa6FIZf3aNTivk/FXIMFL8PhahuhUaz7tUcSEfUJacYjn3OvxcN+B9sDQbus0zCK3+vcQqeqMacdxqMCW7wM923+L/zvr4wVr0hH+JwJfDree4Bl1jzaCDqsGhfk9vOH/bdwXvthhp0xzs0c4pxsXIg8QrD5/x5TvInTlGP3r/ZR74PCfok7K/DaBY3uZr2NrKA6IHDKgraHLewFycRmUnMNADCGhd9dYMv9w2AEUVETtGnklB3Xu+6KEL5ENCTOYZvJt9eQmfSlTKvuLu7/xX9ib1ChYQyugHV2g9/r2EWPDOlTAYGBGR3yzs49BDefeQfg/xaa94ZdM6z6VpXZ25aRv7lA26dKDP+HR/seD/0M1PtKLNTsrWt+yq2zZ3NN7yPUtENRNZAibmHjygCFJkLiaZv7y8vpcSqpCTgTloUJQ753+H7u9Xz2Bd2LweS20OwP4yujMLTLOhGCH9czvGft13nf6pcS7t2fpPgn8JnVt/DlSi+bC3u4vTzMl6Y287y2XRysd3DZMvj3hQ5sEeJpm5vnz+dl7ffyIBclLfYi3g2bsRcEygOj4lRv5cf1NmRgQAuUD0Eeoiw4c2BVJd71l+B+9+6kxV9kdraA21sj2FdA1gVhUaNdg2xItNIgDdIX+D0RwVyOve++kJV/fsdTv/CzyMhfraVNZvFMHIvsG0PNGIKoTgAMqhz/Mj/EG0t7ORBWuPmcr/ISLk5W6MfDGPRVF7KwQjF7VoGORzSRDdoSlFdlMQpmzhWcadBZYso3IwKe27abnPSYCkt8+P6rUAcyuDMCu2qQAUxfFPGOq25hfX6CFc4U21melLgncHSrNOLX2BfE5eYCo0CAY06M6I0QRAgyIuBA2MmR6wfo+Uh6lO+DfsR0VODizD5e1X8Pr9pzDQ9Uh7i2awQlJKN+J7aIqGmHTfkDXJFJ1zHB7LCNtkCG4CwYmpnRaCXAgAwNkSuwm0mGUQYiF2bOsen/bnJyPxbjKXr6Zhn3CwTtGuELjDLoXPwzgDst2fCcXdyzeyXeYPpirwdWHcskjI7bHUVAYEAJyft+egNvvenjRJG/2OUijZSHXLpGfBaGbObXSKwqHC033vVQg/YRj8YNm3G/ffoLeGLK989u/WX6fyLJzIS85oPf4dUb7uXhoX72zHYSmLjA+nvPuZkvTVxM7c1Fbjs8DjxDtdyeIZQwZKRPOcpS1lkkmowMaGgbJTS2iBYLk5d1li5V4R//6CP89Uc2JSz5MSajIj3WAj+obOCzM0XeN/R1akYxrbPsDKpcW9zOFRlJTfvsDAzvmz4vaZFPIChAlDU0hn3cvS7SB6cclyo1CoyKFZffFlcHM8+fQ2xpx29PieukiV3waYQWqiEICgG63pyalsY4AjT47YKVuRnurq89QdGlhRf17zjhd02sdG0Rfx0IK/zsxR9gNCTVSUeNl1yKtgSRI8nOatyywKpplK+RgcaZqOL35plfaVN6yaVkvrXltN4nMeUrtGDqpXWi8Syf+l83MnmRJMwbjDIY2yB8wd9/9zU4ZcPkezz0XC/n/NV+wiNjSYl8Avr5FzIW3U+PKpMRAZNhiZz0UEYToNBGEjXNMCU0Smg2Ogts9dKXqtulKovpnz+rr+LizEECo9gVdHHQ7+Ln1Ryr3Ql6VZl+exZISVlAIQhzBhlCd88C0e3dcSfpo3pVABgEYFcERkLDs7CJLWDhuqlpaaMsjZIaI8HUmtPSAKEEWyNsAwZ2lXvBQKTTtQMB6LbLREYvWr2SWOkejwYyQuCl5dzgMaj2NmbXW8gAlCdx5yIsbQhzCuWDu3+GA68YYMXnD5DrWY5flJyu9z2xK3jppl383SVfJjMuOXKFwJkTZMYl9oLEmVY4s4qgIJgbljxy9Sd53kUjeGelx0E/dnmWwCjWWD6TYYlV9hSOiKMYFAaJxhERjogoygZFWacgbM51pvGvuyRh6WOswQEaxuau6jpy0udXO2/jzvI67qyv5l/GruKOyjBlnaHPnucrkxczFralqh+aKhYxFkQOOP/aSWUlqEa8PbTqYGTsjpAh2FWD1wkrP6xonFvHCODcdUkPIUYIAt/ixsHtaNuArUGALAaxyyGQGF9iLMOF7QfBMvhh+rbsq5xJQo5VLdNAdNztIokt4aNKJzIptH/7e3HmDQubGyyslrG1GxpUQ6NtwcKmPlZ8/gCN9X3Y1Ygwe/oHt4lZvrONHD9bWE9wfpUbhh/GbiouT9tIoZHCoNBsnRnir6bOY8vBlazZ+ihpKUhnRFyHuKybq7zQZESAbxS2CBfHExhFl/Rw0LjCplNqFn53ge7vJSl9jDe8jIwIuKowwg5vgI9MXs3Hlt/BbFRjKixyyGtn3CuxsWuUX+27jd1+L0P2NKqnh2hyMmnxwbFBQ9gWoS2J3xPCLoswE4eYiSi+TtoR2BWD365xRmfI5Ao0lIvOWKko2yRdFx1IXlJ6gM9UryGoKUSHj9EC4zSj0TyJVZF8ff9GVC4kiNKnfK/P1k6yaJWIFbASoICqsegWBltEaNKzkB/FOBZIyOY96gM2tV4bocFqaIwUaEsw/5xBhAZV1yj/9MeQmOW7PD/HC9oeBqAcZjhY72B3pYfRWjujtXb2VLqZ9Atc1HmQ1e4Erxh+gOqVZycl7kn8xhu/g28U89qmx1ogI6I4WuMxLWsmwxK2gJVWxM8aFg8Hinsv/kJCUp/I9LkZttZWcUVGkpceN3Xex19MbqBqNK9tu48/7/sxV7Tt4uOHr+ThxiCD9iyBsWhcuDJp0WN6uxAaUIb2H+zEPWJT7RcIA35REGYFkSuIHAhyAqEh3LsfYwTSF8ggJUv5ulVkdrvcXV8FgPQkZtbBVK1jpzwG/M6I4KddZHMeQZA+5auExDMhDQMZIU9SLhFxje6gafHWTPq61Hh9eay6oT5aRNXk4gGuiEBog1XTWHWN8jTV/tg9cbokpnylMATGIpP1CbRaLJx+tIRkPbRxZYQlNVNhCU9b+KX0+LleXdxOl6rQqQIiI09ogXQ8UmjmtEVO2kQI2qXPlyulZ1naxyfMg2fizc9Y2MY+v4ff7LiLLyycjwQ+NBN3LX5Oxz5eWXqQNfYUDWMTFNIx8aNSBiQIX6IXKogo9uXGYWbxl7bjiQNgmu2cwlBibIPXlZJYWSUwCja4hzAW6KyGUghOcyFXBmMZZJcXj1cYdJSeuXCUeR33OlMCIgyBgYYRNM88OXrXBMZwNEzWGkyPKxEgzEpEBEPf16i6QNsCbUGQj/8uIkOYlVi1iEa3wK6evusksSvoyhDfKIqZ+MBDCkNoJJaIFa4SelEhS6GxpI7Dh1KAGl5Dv1VgQNWY0xaOiAiMJDCKwFjkhY9DhDYSR0TMRDnKOvYNR0bwkvx00kMAwG83TPsFatpnKijywvwObm8MMmDPsicsMFrvYEt5DWdljrAvLJAXIZER+Pl0THy/zcGeF9jzEhMG+O0aoWOFq+04rlcGgIhjgGneT+F0Fqsq8EvpWER01ibMGR7yBvHbNcYyULXiwzar6XbQAhMJIhu68jWMEajuroQlP5F5HSGFQBL7e2NXw4lGiS00DQNKCBQC3ZUOQ+QoRjYTLKoh7lwclhgUBMo3RBlBUIzvGRHGYY36sSeKT4NEZ5HC0JcrxxX6ibv/+tqKu/4ayaRXICMDPj7yPIqqQZhNUtpjTFzVB0CPiq3GdllnWueo6rjzQNU4+M11PiMCxsI2flBbzgXuISZ1Dlc8A+kxzwAiFKzJThIQcWFuP+8fv5a19iQZGXBlBt4zcDNXlHbxSKOfH5Y3UJSChnGIUmIw1nrtONNIA6ZpHfogNBgrtni1Ex+4hTmw6vFEaRtJh9I9it/ukl0/R615/wBg4uuDFghfYjIRZs7B79SsKU6TyfroVf3JCf04rLAKZISFBhrGIDlZweRERNlYixERQVfu2RbzSTEqPqh1dxwGHceLazv+sqoaq6aJbAEyVszKX4KWrxQaW4R0ubXY6m2GzkhhkBhsGVENHCQGb7RATvqYlMyZvpsPABAYTcMoqsZmOiqgkeSkR8PYaCPxjSLT7Gqckx5FqdEmHVYjAMLQaVVQCFbZU1xR2sWd9bX8Ym6W900Ps9wq8PriNDNhnl2VXnpVnsCoMzrhfSZxqhptm2NuUTe2fOPtIYshZ821ffG7XTFoC1SQjgOfyBUoYRh2x4iy8WQ2mQgKIYRxsoXwFEhQdUFWBRQyHtpJyYRoMhpWsFDkjrN+jycitoa1EfSqHIHR+G2J93M4gdAVOBUNSsUZk4FBBrGvVzvNWiENg7YkYdYQOac/nxPRBMJ1KVkN/vXw8+l0mqmI2sJREZLY/aCNIIgUNe0w9P2IPnueyEnHpB/7SBznuie0uNS1WWPVeEluklXWNNpI8sInQuCIiB5V5aLMKF+dvpjlVgE/LSsI4PVEtKsab9n/i1zsOoyHbVyfH+HTC0O0qRpv2Hc1fzR2IdoILm3fy5sPPJ9hZ4zy5nrSogOQPdJAaEF2onlfmDiTLczF6cRBAcJM/LtdBdn0/R5Vwu50OrLEgpzEUprznOlFP69oKMSsjciF4EYYEcf5BiXNMnceJQy1gZRsQZr0qxxKSMYjSb554HY02uEokYH1tuDah19Oh8px8PrExH18RHw9qhcM4HXG2ZHaFlT7FUFO0uhSKF8TZhXOgliabgdbROwY741DToygZDfocqsUbI9Ot0p3pkp7Jp7k2dFK3AU1HbqXrB2yN6jEKcXAc7/9Tn7UKLHJdZuHanWKskFGBKy2FD1SsP3jcWZYw6TD5QCAgIvcMe7cvRqAzdm9fHT6+Yz6nbyk8Agv6txOm1VHG8EaZ4IHJgbiBJIUnbRr2yyaWNacQjscO6HW8ZfyiC3dpttBRket43TEmcrQ0Jmt0S4t8CUiah4+ZzTGU6AFZCPQIIP4f5bUqTFGjjLdPHDrU5qSzKAAG8iI+LsiVsY56bBvd+y6K/Slq8WTtgSWZ5hfaWNkszN5xWDVDULHxXZUQ2PVI8Lcme2cErH5hRBxHOyOAqyHw5U2Co6HF1mL8YtSGGZrWc4pjaEfGKFLVVLj8y29qcLq+wrUTB3PWKx/2xb+ibP5pyd5Tid38IU/jUtxpCa4XMM2v5u2LRm4Fu6qrWWFO0OE4EvleLFoU3UkhpvyNd65s4NlGxeQ8ynZKgpAQvFwXGvDKKgvM5jmz0fDgMICsf+u6VK1awbtCrSjSMMy0v7z/eTfEfDGPTcyuGaK8ZkSWguUNGgtkNIQhRJKIZlSg3tm41C/tpFyqiJlO2SGA2GFmcjmsPCZ0wUAtJHIZghmfCg9z7l/fRjvxgCZoqSdo0S2wGoYgn6f8IjbjJgRiy4t2ROv7nbl+HTKp08ys8i2qWkHI+DzNz+PsBThzKo4KF41D6UFSF/wxYWLGGYrVe2SFndpND7Bi6/5ZT5887+x+e5fo5+R+B9Sxd2XmyemaIOJosX6ve+++xV8/rkf44ZHXg4cSkb448gesuJwvxnNda94I9aRWdDmmPwQ/x4EXD94CdZL4yJBvaeXyv6Mo934nokcCUIQFSJ0EEfzG2nQvjwWJyvBZOPrEDnHoiDSQHhkjPWFiO+//3m84U+/w796l+PaIUpqgkjFCUdSM1fNcsPqh/jml57L1S/byu77GkmLfgK2UPzSg7/G3Rd9Ac8EBKaBLRQSiW5uT2q6RkFmCQ+O8qN6ga5/TteBm9chmFsPPfcb9l73STbs+m0g3k2JKL6dVF0QlOIKedPnCQqnGbafiPLV5TL/9dAldO2E9k83CxI/Ra7373/vV+jbmxKLEYh2PMpr/vcf0XPIR9gOJvBBRxj9xIH77bdmqF7usPCxIYopUL4rbp7nD4Zex/rP3omwHcLoiWUXs3Os2V/gTef8Gis+d3IR6SRwtu1niJU49z5KZAzr33Zqq0LXjw+SmR3AHTlEWhqZ371J0c4dfOs/O+hlx+M+pgDcDwxxO7v/+lkV75S4bmATnezkOk6tcNQ/rDsHm3uf+oHPItkJjV+SFL5wB9d9YRPLm128H4+p37ocfQYatNW9uEWLFi0SICUb+RYtWrT4n0VL+bZo0aJFArSUb4sWLVokQDLdiwF5wTkceg9UKxmENGhfsWJgmlBLIiOYWcgTeBZixkHnIta/5cR2HWnqdPrHu7fx2cnLyaqALRMrqNzZg3vJDLWGw3VrRwiN4rbPXsSyfzjReZ+mMez64GU88soPc0s9T5esEiEIjMWBoJOXFg5y4+++g+zXTj7QStMYABCCnR+/BJkJkcpQKtaYOdJ20v1zPGkawx/v3sbN8+dzfduDDFkLHA6LLOgMj3rLmI+yXFnYwd+t3XjSa6VpDOEPVvCJ4c/y1fL5i6VWp8IiOemz0pmiS1V479rzTypmn6YxIAR/sOthOlUch7zK8pmMJJNRnkvcGps+//usfddS7F4M7PyjDOWpPEIaXDcOyNx/oJtDe7oZ392NP+9iahbGMvQMzjH/hsuSEvXJkYqvz17EpuJBssrnpqFtdFwxxq+s3cLrzr4HW0QUrQZvf8tXkpb0SXnNlbczEgRUtcu+oJvdfi+Hwg5KqsGI77DqTx7/BD5VHG0lP69wdmdxtueobu1GeEtng7fb78OVIVvrq/hZbS07vH72+L3NdPyIyTBdhWgej3WlSW5vrGym1MfhcL32Ajnp8WB9iKgZ42eCtMSanIx3/SXkpMdMFMcq7wjyNJq1ur9UWcHf3Pi5M36PxO7KdW+4j8wBBz3jUi9noGKh5izseYUI42IiyDh7qftNM7R9Oh3hTScgFWvvsrFFxL5GFzN+nqmgwDvW/JB17hizYY5l7jz9zhyD9ixdt3UkLfET8jd9D1LWDhkRLAbEV7XLQpThUNjBp1b+NGEJT4Fm5I7yBFYlLqjjzoJVXjrKd70zxkKYoSgbzEc5yjqDbpYsXZ85QuZMCsg+S1zT/jBdqkJgLAKjyDS7kWdkwMW5vceyPJ8kLDNpbvr7HzJsVfjW7CZ2+cs4HHSwO+jh2/Ob2OiOPiPvkehdueL/3I7bV8M0FKoWtxCSAbhTEndKISsKk4uIptJRgvEkdMTuzQ2Gs+NESFwVUo/iojo/WTibemQTaIt2VeMvd97I9BWzSUv8pATGoqbdxTKAqtkKqV1VE5bs6RE5gIxrOBgVl5dcCqhSCSk0g+4cdrM4f0aEuDIgJ5ulV9EI132KV0qWyzKH+PjhK3lz2yMAi41lB6xZ3vGj1y9aw2nmm+96AVfd9juclRujoePCWTNhgRVurIve98h1Z/weiZsE4d4CTkcDqxZvRYQWKA/CTNzsyT2c8pkjFV89t4edb1xLVvoMuPOMNAbIqoCVmRne1bWdL1x6Nm0vfjRpSZ+ShrGJiOsSt6satgiZi3L86cgvJS3a00JnNFqB8psZk/bSiGUPLljLPr+bj265ipkojy0iIgQZEVDWGf7kh68GwFywPmFJn5w2qbh/3xAFmaGs4+I/UbOW8vrfuoeDQRfeL25OUsSnxPn+fax+7QOsdOISq15zBX9jaS/vXn0p3S/dfcbvkbjyNdLQVqgTuXEXWumBPK67yGLFqjQixLGt0+EJZLM3lcSQkz4ZGeAKm2hhIVk5T5EBax6FpmHsxWagUhiqd3QnLNnTR0bgdRhUY7GGeuqpDrpxTd9IEGgLW0QozGJPwHxfvAOZH05J9+gnQBvDwDfs5s+ximkYh7z0wBgebfQxvTHlRlWz/sr2+lDcKNedoKwzfKMaFwQS6syrgiSrfKVi2R0GR0VEWYPyICgZokxctEI7mtK+EGE7iYr5hByXHRjNztLrLGDLkE6rSptVA2B30KzaJNKvAV73kXdii2hxmzUf5fG0TaMvvb65x2It6yO/X9HoMjjzAr8E2XGJsKzUX4PxGzxWOVM4kxadVoWSrFNUddpVjR5rAXF7G5vdCcavSe9BFYArLPJfugvPBLgywBERGeFzd30NAD94z/OpDKfcdy1i1XhVYYSN7ihzUQ6FIWqqTBOcef+5xC1fu6rjLYluFq/Q4M7G5dsgLjSdao6b0J2qSi1ycWVAUdaRQrMjWDpW49B3Zxb9u41mF2lXBuQOqfRUYnsKys9ZiQwg7AoICtDojbBqEF123lPWD0ma9b85wj+++pWs++e9REhW2VOc6x6iV5WpaZcVXxrlVe94F+f8/XzSoj4p8zpWTIfDONrBb7bX0kair7qQepek2JOuUpIn0dzRTkcFtnnLsUVEOcow6neiNpz1jLxF4rUBg7zEDxWm26faDaZmMb/aR/sKIQ3lFRnyz8Aq899Gc0If/sPnsrNxO332AnnpsbOxjH57jowI2P3ZTax93f0JC/rUyMk59vk9QFz6zxYR81EW5cGsTkcB9SdD5nIculpSOACl7iqVskVxxQL6QAdHrsgx+POkJXxytB/AvQ8R/nA5gVHcWV/LKmeKPV4vgVHs/0CRwVfcRZRiC1719HCXtwzvllU84D9KlxUr2chIiqrOTf/vVlwZsMqe5P1sSFjap+ZRbxnr3DFmorilWWQkR/5G0PvSM3/tZC1fHZEb81gY6SI7kqF4XwZrTuFuz1HY5tJ+u4vXkW5r5SjW82fiGsXAXJSjTdWxRcShoINXnbM1YelOESlpGBsp4u2VNgKFIczFPbnSjlg9hHaPWegigo5cfbGZZuppWluuFcZWopGLbaik0JzXdyR+XIqvhX/eEBkR8KmzP40tQuT/x957x8l1lff/73Nunbq9aFdaSavVqlm2LGw12xhMbw4QU2M6JKGl0FJ+QEgjhG8S+P1CcQghQAATQrG/9GoMRpZtyV1Wl1Z1e51255Zzfn/c2ZWEZBss4rnAvF+veUmaHc08Z/ae5z7nOc95PiiiWt7XFQE5o8LFznHW28mu/AHQ2y6hcIZgoRQKheBZfQ/9St6/7pHvyavTZIegbY+HOVOl2J/FKimMakSx12a6szaZpJHousBV7WOkpU8hcmk1i8yGabKGxEAyF6aAaqLHEGmFTrtxXaYICGptxh0ZoMxz9biSSGVxLu7bK2t91kNBe6rIOFBtUchMBlVKcNlc7ZBIqCSODBaUUiBWfjFF8udCqTvenykpSaQl6oz4TiEXRGZbjWSXywHMDKYJtIFC4msTA40jA05Vm4ELL5ere84XYvWBQy8xKfdlOHU1zPRbHHqRydgmTeqEibh0XWIvNmQ8QV7euYPF9iTFKL6oqjq+r5WUw5X5/XUz7xdl5VfeiN/bDECEJCcrC5G836J4yo3vrKd5vxATl9jIikQbEEYSFGxsOo6Wsbhm5Ylr623iIyIdB2PVAM/q3E2zUaYYucjabc8VAc9tu4/yCzfX2cpHZvx5HiXlMKscjDPUK6BW7ysCStrmNs9FXrKmnqY+KuNbYokzicJAESFIS5+B1BhGR8cFv3/dna8MYsWKeeFAAKF0LIzYWyRKaQoD2Tpa+IvxvZn1lJWz8MvKGh5KC5SWNBvlepv3qKz8ozsI03F9r4FCIbFESKAN7GlJ/5/dXm8THxW/SSMDER+u0LHsS78zhlAgPcnMQMLLm4BqbxOWCDGINzsD4tz7/GN8g1wog0oiyjfY4JyipM+uUJo/UuzKgJz0CLSJHEt26qG1d6ZW926edbIwLauEK3su+P3r6nwrv7OJMFOTdPElfk6is1FcHJ+OCPfm0f1lZgYMzEXd9TT1ETGam+hxZsgZ8VIkI/2FshRPm7H4568BMtQLeTlPWTQb5Xii5JM72c8kWFzFLAuUBYFvxgKh7gkQcf53dkOCN26JN9wmLnGIkFgiIi19JsMsjgyIkJSUTf9VRxOd81356l289Vmv5UlusFCj7MogPiItFNdlR3jJLW/kQwNrCIdH6m3uI7K1+ygT1exCEDKfg+8wC0xccuHyR3V1vsoUGJV4M0T4caRiZ3xkCPiS3BFY3jmJ36RRncnti6D9AIUg0EZts0rEudLamfwoKeJzj4IMFBESW0RnH7QIk7u7fibSUrGeoTwta9gsAR2r/pqphN8EtaKwTFFWdk1o0mA2yiyU/xVUikAlQfLzEZAGB17dxqEwro6JdJz7tUREpCUP+po/vPzWOhv5i/HU5t1xuwDkWUeiM7KKNi58TtTVK0yvMmjdG+L1BLhjBn42VmoN3XiZ2HLA48DxLsJ8RLE/ud2cVLnMqWozfi3PO98bYd4JB4nQyH10rPEyGVklLarxCT0RkJFV3CSfMjwDFUgiV6OlRgqNssAREmVBmFOxAnCS0Zq+NSPMhmmkUAvtDKfCLK7wmQ4zbG4bqq+Nj4aKeOITH8DTBhJFs1HGlX78EAEzKkVTLQ2X2MNTNY77bbTasa2BNkhLnw5zDlcElLsufPVR16sxzGi8lloyPogjYW8yRZgR0F5leqWLVgKhBH4u2RPnifl9lJW9EPnOb1YF2qBQq3MSMtlOTAyPARBgMBHkKGkbT1s0DSV0s/Pn0QJ03M8h9A1kAN8oLUY5cXc8VLK/f4A5z2EmSFOIUhRUCkcGuDKgoFK1WtnkN6VZnzvBVJSmrB08ZXEyaGU8zDMe5ZmJMqx3jwMgjGTP6SX2JEtTE3gq7nniyIC89OISuujCr6W6lpotf98udBTR8ukIYdmIVf10fuzsvrFtn6yTcb8otSqMfzv2RDa1H2XfXBdHy614YXxCrC8zTdGKawW1Sm6uDiCanOITJ57Ise8uo7QiQFgKSiYrv3pHvU37hVj11v0El61k6NkOubtd0uOKz3/mSejXQOaoQd9XRxOjVvxwDLROcMfoUu4c66M7W2BxeoZICyqRxYSXZc+RHgbZWW8zH5Fvr2vmY+9/Fl07Ffm7hznyT3m8k1m6b4fhp0Q032PRyXbUGY3Uk8jO0nKKkcMd08vZ2HScQBvcX1zCmswp0sMXPpcb6hQRL+4AACAASURBVMUNGjRoUAeSHfc3aNCgwW8oDefboEGDBnWg4XwbNGjQoA7UTb14HnnJGgor84xtlKilHoN/PUdxbRvuWJUoZWL+cNd53ytRSqe1s/bV7y0j73jsPr4IPRWX0az4Hx/503vO+15JGoPcsJaP3PwJflZZxi0za5BolqcnmA7SfPWOyxh8053n7SmQhDHMq+D+3t4TdJoFjgVtBNpgyGtndWqY0aCJQBvcNHQxnb+zd6GHQpLG8PP86cE9TEZZIi3IGx6RlnxiMO6HKxwH7fvJHMMZ3+3+T13GE1YOcffdA4jWKuv7ThG8JkV4eAhhmugoSuYYfo5/PHIHnjYZsDyuvP0PWfriB+If/Nx1BL8m6sXz+G1p0ic9MicFb7rkVg7/XZpCr4l1cgotQOZy9Tbx0ak5pJVN41zXtRPLDnnK5gdpWT7NsbdESNdd6AGRVMb+NuRA0MZqZ5iXte/gr3u+Q7tZoNeZ4Q1X3sqRD2yNx5nAcczLj3959DLe9J1X02nO0WtNsz59nNc1jVBWNp99YDNX9dakX5K4yXzG93rsf9bzk8IqMrLKnYUVAHxvZh0r7nIpv3BzPN6kjUGIBWekrtzA/o9vwkr73H3vCrJHJc3NJVwzQHyqyoHPbESHYfLG8HOY3V383t4TzGmHQ0Ent3ld7L3yvzjw0c3oKzZcsP11db5yw1qULfG6HJoP+iy1J/jgpV/Ba4M97+qhuNhm6vkXET1pYz3N/IU5ONfO33/5RahDWW65bT1TJ5oJPBNWLUfaViKVFMov3MzL9p7i4o5hPjVyJR8+9XQ+NXoVv3/oxdxT7GOxPclDxUX0bzrGoX/aksgGR8baQfb/++XsvmcZOhXxtm9dD8DLcqP8yfBlfHH3ZTDh8L1vXs7Bz12Kubi3zhafBxVx+AsbUD9cgmsHZI0qP55bzcvabudk0EKPM8uxUgvpN5/kqvs99n9sU70tPhet2f+Jyzn67BTSk4SjaexpyX++9cP4P23nrjsHeWh3H7pscvKr6zjwmY2JnBMA3zp5Ny+/dScKyUjYxEyUxhUBN5WyvPcpX+P6//jGBTc5qqvz9dtSaAPQkD4wwb+9+gW4ImDJ90sMfrqMOx2RPelT7LUxWhJ4vFiIhYjl1Du2cWpHDwgIs4oorbhu811Ybsjh65pRnrcgTZIkJl5WphClSBkBrhHgK4NpL001MvGVyV9+/WU4MqTVKZMfTGYjlLFtbdhjJvkDEmvMwp4VfHn8Mr5dznHfVC/GEZf0CYlVAOOYy+gz++pt8jkc+MxGXr7uLk78dAnrOkb48tAG9s52MRI2MxHkaDVLhEoSasnn9l7OwKrhept8FsIwMDo6ENX4RKFyFUJBkNPs8pYhfTAqAi01ZpNPadYlnatF8Al0wLv8iIys0moUMdD0WtN42sISId1mrCQy84rCBX1G3byBsWqAyJVYxZD0qQrlwQ7MWY+//5PXICJFpSuFDHWtwxkUr15ZL1MfHq1BKw5/cCulZRFqRQXDE6ROGdiTBjd/dwv+jIM2iBsDJTBqXNk5jiVCRio5pqoZfGWSMgMcI2TWd3nKVffR5czxz0u+zkuW3x2390wYpZ74ZBsi7uVrlgT3fW0t7/zSq5j8Ti/2nAARS8grC6YuSV6joFdsuINvHV/H0557FxmzykDrBOuahrm/soRjlVa+OXoRg/kxFqXnuGb5AY6Ot8TprISgw5Dhl6xENPvYM/GpVG3Gy/KfTA8SpiBsUohQEE06OBmftFNrdJTA9MO8oy0rp6bqEjeG95RNSTm0mUXevuYHF/QZdXO+Ku0QpCXGnE/kmhhehCiWme03MWbKyFCjTEGYNnCnQrzm5OUaF5pf5yNka5WwaJEa00QpDRqC1hC0IMwpZrctrbe152VxegaAzS1DZK0qtgwxZUTa9LGNiNkgRac9R7uRot0sUFqWvPaeytGEGY30NWEuQqhYC9Bvi1C1LWVlxlLyUSaCXPLOuXVascL1ytQY3c4cW5qPUIwcFlkzjFez5G0PKTR9qSnGvCwX9QzjX5GsG+HcCoWes9FWrEpuliQqF+JFJjKIn9PZEG1pBjonmJhI9n5Om4wbGqVllZJysEWEIRSWCOkwCjw3c+SC3r9uzrfYn8WsaKqdKYxKgIg0UWue/NEQlU8hfYUMNNUmA7Mc4bUlb2kizLg/rCxLWn6Qwh41CTICv1khI4EIJUZRsmLtKU4+JXl3d4D3df+Q2SiNIwOWpKZpc0qsyE4wmB1jTXaEi3KnsEREoCMiJFPXJ0/40F8UYFQEXpsgNWwSZkCEgNDIACIn7vcQZDmrb3SSWGmP8NwlD/KFY5fx2bu3MOw3kTWqDAfN5C2PUEkqkcWuqT663QLXdt5HYXGyGtOIzirmrMRvUgglMAsCYSlCZWB6gADDjjvP7T3ZDUULedHqept9DuLy9QTaYIsbB3yetsjIKp62iLQkI6usNAPajcwFfU7dnK+MNEFWYnrxUlxZEqEUyoqdbJA3QMDMSomyJWYpgc5rvqm1gImrfGQk8JsALVBrimhLEbWEjBayYGpk+sJ7gP6qcUTc8i8nPUqRQyWymAlSjPs5Rv18TTIFyjpuJn1J96l6mnte3Fxc7SAjag10QNTSEEKzEP0aVRCeQSqXvOY0GVmlqkwu7zjG5sEjHCh0MhPEWoDl0CLUBvtmulicmUERN+wvdycrIBFCY5ZqNoUijoCjmouZVyOfszCbfFQocMYNqj3JW0nNDmTIiDglMt+L2CL2UxGCknJoMeK5LMzHXq1bv5yvp5GhxpytEqXiCFJbBkFKEqUsrLkIP2fES8qUgZVA6S0dhpjdXaiUYs2yYdBQXeyTPyRxb89ij5kYbkTerWJmA07+4YZ6m3wOSuta39g0gTKwhMIQGikUkRbYMuSO2eVA3KGt3Ule5Juq5Q4rnfEmT5jVBBmg1p5UOTrudJYCs71CRy55F1OzrLIle5Bbjq9kppqizSnRbJXZU1rEytw4J2abcM2AY6UWms0ynWaBaluyApJo3KWyIv5dmCWBWQSExjVrKhAahBK0txR47roHiFzN0O8lbyUys1IihaKs4rFIoTgetOFrAwONpy1OhPE80Jc9dmmqujnfyBGoWkNiEap4Y80L0RKMSoCWoAxwpkS8YeUl75cEgOvQdpfB0GQrmZOa/AM2hX7F3OoQVpZwXB8NRJMOlc5kTRaAW71OAHbMLEfWNLckekE9N1QGebPKn518Bq4IWJtOXuSbdfzadRNvrDlTAsMHYWqEBrMgMMtx5NuaL+OFddeNPYdAS1bb42gtODWXpxTZFCOHZqvMyUozA60TjBZyWDJihTtGtzFHmEvWBq4MwXRD0sOSoDVCWUA1Xrprk1jVVIEfGuyeWQRA36Kputn7cAS5+Pov1lZ784oc8zlfVwTcOHdJ/Nr8Y0/91M/52oKZAYk4OkzQZC90hpdR7IT9JpNSr8QqakpdBl5r8sq0AIZevhhlQz7t8Zq3fYP3v+VTaFPzoad8gT9d/0P+Zv3XGZ/J4i4qxX1lE0azLKMQHJxqp8MuYsoolsjWglBLFIIOu8Adp5ZiiTCR/WRTZoA2NcqJN9oqXQqumsbNVvHWV4hSGr9JE6Y0PdlZKn7ytNxGolgsoLdplpxbJWUEpIyAtOHT5cxhCoVrB1RCi0AbLDZDzKbkyCIJx8EqSNpbCtgzmlx3gTATbzwXg5pSsalACQypOTnVhBbQnZmrr+HnQblxEPKj8mJKtWqHNrO4oMrRZhS5b24JANXmX8O0A8R3GGxrodREpyxCVxC2pHAnA0QIyhJUWwTOjMZ7bvIKy6ttitkrPPqbJnFFwMmgFXMulsjusaYxhOIPL7qNaF+OzPLZept7Dl1GkbT0mZnKMugO4yuTcF4OqdZH9qLUCYI9eTrNApFOVp4R4NRcHtXpY82KhVKz6I4WKpMp0rtSmGWB9AXupKA7VaBYTE6J1jxfm9pITmjGSxla3ArjXpahYhvtZpG5MEWzXSFtBeTsKhtTQxgI8rnkCLPKFUux5mBb5xEqHQL//maCFoUzblD0a87XiOe5F5hctGiYyNUMZMYxey9cjPJXiQgEI1ET/3niioXnYk3GWCrMFSG7TsbOt9j72F1o3ZyvXVAYVYGQsQmRKyFURK7AnK4QZE2aD4egY8WLyBZYxeSVCLHII789Rc6KI8JAxxuFBgpbRJSUQ9bwaH/CKP0tyVti2SJWyQVQSCItqEYmlowwhCZQBt3mDPaMYIk5g6r/ifRzKM2mcNM+RhBHvtqIy8qQpysdjCoLwm6qkLzIF2BKGYRK4hoBSgu6UnOMBnla7RKnyk2kzPj5nAgoaUXeTU4z8qA1jTZhiTtFlIrVx63p+FrJ2tV40zMS2LOCcsllc8sRtKNYZM8QdSfrAJVQ8FCllyNjbcxEadKyys5yvO+RlvF37trxnPHaH/tqtn5pB1dgVMFb24sMNVYxRLlmXB/bnkaGGhlowhSs+PfjtUmUrKhLblhLNGczc3FIIXAJtIFR29aNkPjaoM0o8pr8caa2d+MnUPzQ0wb99hjZ5jL7K934yqSqzIXNNqUFMypNmIFWGS3o1CUJMWURRRKrUFtBSWIdQDdEhhCmNUFeY5Xjm4kzlrwxKC1JiwhTKhanZ3CNAEeGXJo+SlWZ2EbIcCHH/pEOytqkoCRNdnJSQCPb0lSbNddk9tKyV8c59opA2TDjpQjygKGRgUCPORz3WnFaK2RkldHNTfU2/2x6PB4qLiIMDKrKwhUBA84oAIZQdBk+MyM5fuKB3/rY8+51DWMMD4xyiAxjhyUihTJBViO0BBFq7DlNNDYeRy9esjYYvK40zYvmyB42eWbbA7gywBKx7ZYIUUi+P7uOVw09FdODapS8SQ/gigBDaIqRQ6gkSgt8ZVKJLEwZMRI0owXkpMl0eGG1jf8bqLRCSk3kCMJULfq14tPckROXmmkBQVow7mXjQzAJI9SSVsOgUEzR50xRCFy67TlKyiYtfXJmlTVtYzy5/wA3jD2ZHlNgyuTMh2qzJlriMaNcJi6F0tIIrzsicjSOGRLZmtb2ApUlAamlBfbNdqGU4KDXRbknWb+PnrZZtjYfQvtx1YNR24hWWhJok4yQCM/gCkfRvmTmMX9O3Zzv9IBJx71Vhp6XxijVlr2OSdPRkCBvETmSIGfgzGp0tYoIodTj1Mvc86JsSWlPC8XVPm5t6R4hiRZV2e8tYi5yWZs+xbt6voNZhlVNY3W2+FzurS7mSrfEzGSWES9PqAxCLSmFNuXQphg43DK9muoSn4OBYCqBzteaMehpmaXUqxFrC+QumkSZmqhkEuQ0LWsmcdbMUuhXdKXmFmpOk0SvO8OsilBKMOS1sa55mFazyM7icg6X28lZHhNehlLo8IPda3jiztcunE5MAu6EoK21yGu/8kaiphCR91l90XFUU8hTOvcRLPLZ2HmSdYMnGGibYPq/ltB2c5rbxlaQXp+sniGpZxzh/65t44WX3r2g4h0hF1azd1WbyC2d5dm9G2l97v7H/Dl1C8X6vnoKPTFF/1wfYdZG+hEIgT0TUm2xsGfjige7HLdkbPuP25GZTKLmTZCWhFlFS3uB706t5w+6bsFG0dRU5tL0EOvtCU6FKT458URmByN+8MNLWc7t9Tb7LI5W2zGyIyxbMs7yzCSVyCJVyznOI4VmYOkonjZrN5lk5Uy1AMcIWbLhFINNY+yb6SIaFRieRZTSLG+epMnyqHYbjHtZkpW8ipkLXY6HaYxjLpc84TgTYY4Os8ALW3YxFuVQWtJpFbgsfZgHb1zLt9/5YZ5732tp57FP/l8l6THF+vZT7N7ZRu6/Y4HPCBgUw3zyw09m8I93cNxx0NVhAqCFUcxlfTzt3fezY6afJLlfYZroMGSZO4krfAoqRaANPGWRllUCaWCb0Xl7Ev8y1M35hoeHEJaNckyUKZBVkH4ImKRPlqksSiGDuKmO7OpAHT2OKiWrOD733zuwX7cM5+lDnADew+UAdLKX/48zj01WEf/lY/YEdbHzkfjh2CpuHV+J/bSj3AvAvI1nX1CS4/z+TddjCE0ne0kSYafP3uPdrHzl3QwBDkN0MbTw89naA+DQ5y9F9yYnVzrPj08M8N1Da1j+l7fzpb/sBuBH9J/zultZR/olimvueCPmXTlIiPNtvWOMHz97Jf3/veN0lzKtQWsG374TLUTcAP6MBuTh0DEsEXHf91fTx/Y6Wn82Oow39j90yzPZcul+cpZHW+2UV6ANqsqkeFc7reGFffcN9eIGDRo0qAPJqxtq0KBBg98CGs63QYMGDepAw/k2aNCgQR2ou3rxWZyZqJ//98/lpMVlF6F3PphYpVPgvHYDjNy0hu7n70Gm06hyOdlj+AVpjOGx89s4hv03bMKcNej/s7jq5y8O3c8HrnsZ+p7dvzZjeCR+rdSLz+FMp1X7e/kFmzn+nm307MjR+5Gh+tj1SMzfNM7Uoqr9XThxbfKL94xw36Yb45+pJBXMNWjw+LH6bQ+SPwQb7oH9H9vEjZObOfq8hJ1we5xIxpGr+UhRa9i0Hm0ZHP+TCNsKqd7fzNK/2oFV2Mip7y9HeiHwUL0tPpv5G8bP/wnIZUv4xPc/zanIYcs730qTuCMW02zQ4LeQoXduoHXrCFuzB/neQ9s4/pElLO6o1NusupAM51tzVkY+T2hKzKkSXZ9u5tiLFCvfGy9P3HuOcOQtqzGq0HtvPY19GKRxWiBz03qqHS6jl1l4i0Lu89sZCZppu+0kCWwN9LAY61ZRWt6E+407621Kg19n5ueGNFCWRn22k49/YYBOtkNLC/Zs8ddqXvyqSIbznY98LZOht0L3F1tI3XwnK795+iXeE/rpvDtkYr2ZuBZ0wILjNXt7CO98AAfo+ya8/8iddBg+f/o/r2H50dtBiIUTNEnFWNnPmi8Osa8wR681xuQ3as+3tHDkrWvwegNkyWDgbTvqa+j5OCPf7j13E6nhEsNXNtH7rRGiA4fPeunhD2yl/8+TdeLwN5L5oERFLHt3/H0bba3s//NVrHjn7TCTnGPS51C7cVSfdTnuhIe+64HTP6tda8I0QUh08Mv1V05Gzrc2WaLJKZa/7D5SN9+JzJ2tbFrssbBnAvr+7xTVwe56WPkLEZ48tZDnBXjznpfTZTgMbh2Kn9A6lh9K4g2kRtiZZ/fsIg6Nt7O1+TCjb92GsbIf+yabbc++n5Wf9pPpeDktalp80Wbcb97FxIY8cxcFRDf47P/PJ8Cm9QCcesc2dN9v53K3bpyxJ3Lgz1YRzStxiGS4ofNSu3GU3jzL8T9XTL1mKwDCshGmhUyn0WH4SzteSErkex5UoRD/pXZ3KSwVzKxyGfh0Aekne8NKV2t9VqVB07MPsm8oYkPzCX5w/RX8yXv+G4D/etay+hn4KIif3csnB24jWAGHwybe9Kab+Nzx55K6+jAnAMF99TbxbKSBkAIdhsxetxFlCfy8YPT/bCZqCUDD/kOLsCZNDrxZ4zx7G/2fPcX+jc0Ygyvqbf1ZqKsvZXKty6LvDkPVJzyZPNmmx4zWhD/oQ/x9O/3vqq04hDgdGScI6booz+MZD87xkR8/jY1Nh7j3zgHslKDptjYe+P4qckc102shzEXIqiQ1/MvdRBJ8y4kbXAgj7oHrTkLbhjGYmcMoJqeJ9FmcWfUgjQV1428X1vPk3EPs+OANLLEmaTbKhIeHkmEnYC5fSsf25rNe8tQdb+RomMfTFsN+MydeGKdJhGlirFmJuOwi1NWXPq5mPywqQqt49XT523eRPenTsavEinfsYPB1Oxl8/U5a7jYJmiLsVED+sGZ6UzepdJVKf2vdzBaWHf8Oar+Hq+73OHVFiugZMxz+xxwnPtrEkX/Yirjsovg/SOP06898JB0Zz2GjvQ3zqccwbrn7tOpvUtobzH+P0oh7ztQ2xbdP97PyLXdQDBw2bj5ApVtzz22DRKtLTD+9QuRozFkDsyjo/cdfrj9FYiNfON3gYv8Nm1j9kSmm5johnEbdt6fOlj0MZ1Q7CNNAhwqE4M0tD/C7i7fwQeCNBw7SbdRZTugMO0+9axuffuOHecvel/EPh37CP6y4GIClL36Av7/qVVz84fv42u4NmFbE7PVbSI+FRI5EhBoZ1XHinLHBuf+GTeT3mCx5wREeesciDl8vWP1HB9BndJ3quOF2OgDxo17s141zVdsB9pe6OXm0fumf4Kr1OMfjfl7fuvWrPPupL2bJgZ0LS1ijq5Oxa5sxhqcoPH8T2f0zRA8lo5HOL8SZ9e7SIJqYZPb6LbjTEZNrLEoXe6x85d0Iy35My/ZfqZ1Cgo4wshkmXrCOllce56LmU2wfayLPBPqak8wCS7kd45Ye9u/qw+wr0X9jGXbcf/p9fgkS7XyNdas4+jcmTCsOv7SF3BFQy3tgMiFyPELEkbmQ8SQ/Y/k0P+kvvzfilYefB4wDcePyr80+oU4Gx0jXRVWroDU9H9zOh57/dPLPOsTXd20AFMaqAQ6+uoPI0Rzdv56mO1wiB6bWagpLbOxCrCzdeW8dc6ZnfNfLbtIoM2TqhqVEfQLCCFU+V9+s8NIttBtDvKT7Lu4uLuVYsQVx8OjjafVZHH6xgYg6cIcN+r//WrovtvG3tqFN8FoFQU4TdAfMrO4jvWKW48PN9NyyBS2h0i4JsmBUoG1PcoQ0z0Lrc9IK7itHKFRtLC3IqXjhraM6px20Bh3bMPXFTiYmqpS/38eh9R1s6z/EKJx1I5n6ZB/iEoG9Iwc7tp9dKvtLkDjnKywbmXKJ5uZQ/1qkfHgRg2/YBVqjrr6U2cEs+Z31tpKFL/xhqxZqF9xXD17CkuseXIjUesxZvrpnA/3Ur15OXbySo8/JoaXGnRCstHax82+20hfez5F/uBhtQMtDEDmC/HctKu1x+qR1T8TEJRZGRSNNoJ6R7xkcfXn8XWfzFYrH81yz4aE4N23ZyP4+wtYMU2vTlBcJUn/Ty+cmWpATsxhemaiOEdfgH95J021trLtymBNeC+EqSV9qih8OryJvxJJCXakCUiiarQqlbofxtdkFeadS4DDtpShNt9dtDGdxxgnVhYoerTFaWkAKTrxqNYWTVdysj22FFE7k6YaF9Fw9MfuXER4eYuxAO+64JHKg+ccur91yG//AxafbYF7zBOaWSWSgKfcoDn54CyoVYU2bLP8lK2eS5XyFQAc+UeBz5IsXY01HDL7+tKcVoabcJcnX00THAaXRgc97D9/NrcU1vKRpFz8uD/ClNaerMIxVA/zuTbfxpTUgczlUKY7EDDS9n7MW8mD1wDw1RWosh58XRCnYccNGlhz0uO/QxZh9AnsGZKCoNktGL7dxJ2OJHgSkRzR+TqBNsPYer9sYjK5ODr95BcqG3D2C0IXCUhOyIduPL+fJdx3gwalFOGYVqNIuJnjTklv45rWXoLQkY1b59sG1rHh386N+1v8ms1dOsh0biPvFnsIiw+mSuPHan7GCWAAUOXPHI5aePACf/l839dGZj3SJDxd5/W0ceSmIsoHRWmXzsocYr2RZlptkaFOFbkhE2aX/zMu5+P072T3bA7s1MgBrGvy84F1/8wfY3xll5MFOlKNpWz7N+wa/TLMsc5/Xh9KSvaVuhst51NZLfqnPravzNbo6KWxbTvprd8RP1O4uBz+8BRFVWfzSOJciLBsAZQq8tvpGWwuVDMAVriRiHz+rLGNL6ghf4rTzXfa5E/zdjucwyK64cqPmbE+GeZxv3VVX56smJnFml9TyXJCejJi4KAUCmg4qtBFHvc6MpnVPgOlFTK51CV2BM6uwyoLc4SLRVP3qM6PRMZa+N5ZlMpqbEE15VEuOoMUFbXFAr0G12xjHysiyT9CS4p/arkco8HMSdyqirxicU/vb4DFwZm63tkL9yuc/zouf8SrW/GPEwVd3IGZT3HFsDWGXz/6om0HioKrejhfA/s5d/LhnK23XH+PwC/+NiajEV4srOVDpItAGlohIP20fWcPjNU33c+PcWmZkhqvS+9ld7aXXdTlVakIkOu3wcw1notEx0l87W9fMuKWHDy3+LB959YsXntNB3AFfhJr1Vx+g8LgZfC4jf7qN977xcxz12zkWFtlZvpi3tR6mqk3237CJVf9e4uNf/TdeseeVDL5m1zn//yfF1ed518cX5Xk0feEuxMY1jG/MMb7BREQgQlCGxJ3WNB/wEKHCOjZOePIUHbfWKh36FqMNiQgjwoSUCEUzszAzC0fhzFtahliPIyIu60nVnncfbwN/06nN6SPv30pqVKAsuPKDb6P0Sk37xWPo6QpB1cBKBXTkKlzSforF9/nsK3ax93Or6fh4/Q+6tH7qdvSn4BlsOONZDQtn7yzA4gdcsfDTr9B5xmtP1B6/OI+v8/35O4MQPPWBOb7w8WfQ+dG4TOPmwa8z+O0/YPBn5yZ2tSkYzI6xq44Vct0f2s76PxlmvTPMj8vLWGxPsqvq02H4HLn2E3At7PBSpP/2/MmRg6UOYKr+tY0qQu98kPZHyZ+fGZfoMKxviVyDRFJ9zuWMbbQQSiM0hBmoLFJEacX4g520rp3ANiKu6DrMm9tuIy0ENxdX0NpcYubH7STjFv7487g635E/3UZxY4VMzqMwlgUlOPJAFbMFDnx2I08e3M+1z3kFg/fuXGi7uIDWyErIl364jRXU93TVoJWhqgOM1BBDYRMFFcdSJVVmVjm88x1vIv2zO84qhzKyGaK5Oe7Y288gCanWaNDgV0D6yCx9by+xKD1Hu13kSfk9uCKgWVYItMHuau/Ca28urCMtq0xFGQJlEu05UEfL68vj6ny7P3S6CLnrqkvxOmzmlmQoLVEMLh7lp7esZ/m98RLkfKVCRsnn5us+wdvesfVxs/l8PKNnA3Mv28Lt/3wDK6z5+7Zde0D6q3ec83/my2nSzY0jrQ1+s4j2HOB3F41yf3EJM0GaBZ5kZAAAIABJREFUfV4P7eYct1VXscieoRi5eMpiMsjgyBBHhpSVzXJnnNPJoN8+6rbhJn96D2kgXfu3BpZz8hH/T7R7H29bVl/HO0/+xh0848YNj/yiM1IL88rLi3939/+mWQ0aPP5ozVfWdAJVxGUrufWFFxP2+GjPQHoSd1RilsH0NOlxhTUX4oyUeHCii/k6jt9GGurFDRo0aFAHEt3boUGDBg1+U2k43wYNGjSoAw3n26BBgwZ1oD7qxbUSrP0f2wSuQtoR6UyVNR2jzF41xUU7BfdOL+bIiQ6EoWhrLTK3s4Nlf3vnwomYJCqdVr67nO7MHOXQZve+xaAFmc4Sfe8OztuNKkljMNrboLON6KH97P/YJnZf+xFuLPTx1WdtQjVlkBOz5+0tm6QxTL5uK9veuJOXtN6Bpy0uc4ocCSQjUZ4PH3sqJ7679Lxt/5I0hvILNjP8oiqvvmgHx70Wbh0a4HkDD/KU/EPMRGn+9ciTyT3vxDldwOo+htoBqqG/28q2pz7IuJdl78lupFQ8dWAfh+baGS9lGGidIGP6mEKx7/3rSN18WqKq7mM4gw8PbWdKueyt9mAIxQm/FUcGWCJipTNCh1Hgfc/5vXPmdeLVi4VhIC9ezeCqUxhOXBFQHM4yNNvKxO9vYbSa58RkfOZea8H4SBOsKaDD8HQf0ARx4KObMW7pIe94vL77JwxNtSJCyd89+Su8anAH+X+fYPSt2+pt5sNitLdBSxME8Y2t62eSjdtfx0f2P4mxa3oR5So6n8HsX1ZfQx+Byddv5TPv+Rde1fYzRsJmctLjSCAZCtvwtMWH+v+Hr7/pgxReuqXepj4sx963jWvedxvPXfUgI9UmnpA7yta+ITZkjvGAt5hvTa3n6q6DPHHXLJXf2ZSsXr5aM/PKrVx+zR6GCnGP5A19x2nKVahGJqGWXNp5EtcIWOTM4siQJ/51/U+2nY8Tf7GNbxTXc6+3FEuETIVZNqUPscoZpqoshvwOvj13CU//nwvTNqyLJ9OBz74/zjJACSkVQmr8lGRyKova4rP9YH9ceyY1umxiNXtYtXraJJwFX0AIRr62Gn1KM/SDZUSu5t0bnw93NPHnr76Jd9/2AqwxC2WC2aE59PlLWfF799Tb6rMwOjqgJY+oVCEMMbo6afnKvbQ8uBw5MYcujyKyGTDi+7SxaoBo38E6W302Rz6wlZtf+s/cXV2CJUIKUYqSsnFlgIEmQnA4bKWkHP7jA//Ci1/yenpekCwFbPGjXp7gPsQDsz34yqQamWwfXsay5in+Zf9TWNEyiR+ZPOSnaHHK/PH/+SKfuLm/3mafxYa33MvBuQ680KTNjUsrM7bPA5OLWN82zM6RJVzVexgpNI4MSEufpNX5rtpp8det/8p+v4uJMI8rAqRQHPY7yRkeaelTjOJDVcvsCd58YIyPrhx8TJ9Vt5zvwLJRqqGJm/LjU8ehREcSAglKoEOJrhhgK2w7QojklcRVn3UZf73u62BovEURSFjWNEW5R/HBb16LnfVRfR7OlMBvU7zh4tvqbfI5CNs6qx+pME1kRztyrgyWiXAdUAoiBVKCTFC0VSO1Om7wE+i4s0ObWaTZKGOgsUR8s/aUjS3iG/jzlj1YH0Mfgc1tQwD4ysSWITnLoztXoBzadGRKhEoS6ni62jKkpOw6Wnt+DhfaiZSkVLU5PN3GSClP2vKJlOBEqZlISQItUVoghWY6TD/6mz7OvKvzFk6GLUxFWaRQZ9zAJZ6yKCsbS0S1R8jVqcnH/Fl1c76hkjhmyEv770ZFBkY6xHRCjGyAtBRGKgQJomLQ3zaJ51n1MvVhOXl9wNt3vIg3bP0J127dhTsRX1QXbRhi8cUjrFs0jLU/RXBJkY0XH+KG7U+qt8nnIgRitoBqzaEjBbYFQYBO1URA59M8SqGliJ1wwnjNyttRCFqNIraIiLTE0xYRgqkoS1k5GEIRaYkhNE/PJ8/59tmTjFeydLkFprwME5UsgTJwjZByYDPrp3CNgEubjzPuZett7nnJmD6WEeFVbMqejWOGNNsV5gppRgqxIK4l4uunz5liJkhz+APJODQ1zxue9wZWWONUlUWgDU4FzXjaxBUBAJ6K/dCJagvPSXts/tTbHvNn1c35pswAxwhJG1VUKNAalBag5xsyCxAaIliWmcQ0Fca6VfUy97ysXDSGqhp8du8mvrV/Hc971U/xI4N9I52M3N7DAzsGcDdOEcy47Nrdj3CT10JEuzY6UmjbjKNaIcCqRcNSoh0LnUnVnjcRflBvk8/hqvR+vFrU69f+NNC1yDdCopAoSspmKnK51CnV09yHpRqZ9KWmaHbiI+hKC7zIRGlBEBm02BXSRhUpNAWVrOU6QItTJmUG9LTN4tghKTOg0y3guAE5t4plROTNeGxlZcd/T5iCtLpvD5+f3sJSe4Jy5GCgUVoSaIOMrBJogwhBh11ghxex9L2PPW9dN+ebtarMVV22T69AGBohNVIqDDNCShU/as7q2c1xX9+D19dP7PB8tLtFNq85THgsg/lQhu989EpGyzn8KZdqR0jfxpMoLXBaKyA0Vw4eRLrJamgoIoXIpFBWrRlj1Y9TEH4QpyGqAUxOxxGyl0y5mottg5wMKCsHu7YcdKWPIRTNRomM9FG1S31SZcgKp84Wn4srA6qRwWSQoS8zhWVEREoSaYkpFVLohbRDMXCYTeCS/dSWAn5kUKjaGFIxV3XxlYnvG2Ss+NqJiB3Zn7Ud4KY9l9D/8vopujwc92/UXOKcpKpMAm0wG6aQQjMe5lFaMFJtYrkzzl/1X5gcWN2cb4tdJlCSOd/FMBQCkFIjpUZpET+qBtrW5GQFKTVBV7KiLl+Z3HNiMbK3wutf8h3S143wlwPfwh01uWLDft7U9+N4Ag1lIRL89KFBRCZhk0ZrCCP8FhtdKMZpByHQGRfC+OYnUikwjfi1XjKVo2dqG2yRjuNcT9lI4iVuxOk8tadsFMnbPwAwhOZHX76cpzbtxqwtzw2hMKTCMUJ63RkOlju5sv0QI3499VweHn3NSbZ2H0UIjSEVlciiu6VAqCVCaEIlSRs+m//8jQxcn6zNZ2BB5ODu6hIsGSGFwpEhtgiRQpGtrTyajXMbf/3SH3XB7/AYyZseTY7HsakWpBFHvACWdbqaQVgKkQ6ZUWlaM2V6epLVinFJaho1lKG1qcS/bn8Kozu7effu56Ol5s5jfbzj1peg7mmCxRWQYE5YqP7eR3/jxxFtmeiqj1Ag87k44jUkcrYElglhiK76qFwKqn6cF04gM1GaZebkwqYbQKBNolq06GuDZqNMRibz5vHtyfW0p0r0fmD7QpRuSIXWAlMoUmbApemj/OjgKl6Qv4exaq7OFj88pcgmY8dpxelqmla3jBeamIZitJpnkTVD82eTWWaGikAI3vP963hC+giuCClGDmlZpRi5tFsFOu25hRzwhZT71cX5ynSadqvIuqZhKrMuabcmlW0ogsDEMBSGoUCDaUUc99tY1TxGEBmxhlpCcGSINjWzpRSyZOBMCmZn0vh9Pi9efQ+EguxxTTpdRUSCKKso9yQsV6c1qAjDi9DlSlzREEaofHoh9QAgTo7Fm28JEDv8eRTxrvTPKgN0m7MAWCLEEiG2iDDQZGSVqShLq1FEkbwxeJFJh1sEYDKMN9RMoVCIhXRDs1EirBpYQhGqBB5OrUWN7XYx3r8B8naFiUoGpQUZ2ydveYwGTfW08tEREu1GeNrCEiFSxPsHnrJoM4oYaEbCCx9DfZxvVweODFjmTiKseCIoJVBnXFAqkhiWIgwMTvitvLrjp6StANmXnMixz5lkw6aDVMsWKhdS6lOkslU+fMWNfP6uLZhNPlPrIFISbSu2bNhPpa1+2m3nQ3g+BCH2eClOOdSQc+U41ZByEY6Nmp1DteYSWe1QUD4dssz/e/+TWWsXkChcEaCQCxtwy8xpvje5jmWmT0ElL3cdKgNHxqu+0aAJhUAITcb0kbUyS09bOBmftNCEyljQNkwK4owyRCk0kZakjICyH19XbW4JiaYYOZhLFtfLzF8MJTDQtJrxjSRCoBD0mtOkZZXd5Qv3Q3Vxvn5vC8vsCS5JHaWttUioJKapsKwQ1w5oylZIp6vYToBph3zx+1ey0qrQ7FTwe+urNnsm35tYy649y/l/Nn2Lv7jiW6zZcJTrBu7lPR99NWYm4PKlR9m4dT8vH9jJupUn2LFzFYafwHxjbxd66ATCsuKa3jAkaq2VM1W82CkbBuUluYXDFkliu9fBGjtNOJqmRcYbmgqJRMU5U6GwhGKRO0unkeG+BOZLQy2pqtNnnuYjR18ZKB1Hv/eVl/LU5fv5q1PPpM0pEW1bVy9zz8u8YMC+QhdSaCwZYQiNEJq8XSVt+jgyoBLZPPS+7kd5t/ohUy5Llk6wzT1FsyxzRe4AGVnl2nyco97gHmP37KJzNCl/Wepywk2GcelPWTnYZsjUTAaA0DCIQolpRzhOQHkyDRpaDsHeIEMptPGbzcSciTk83YY5ZfKZY1sZm80SHs8wtjpL6uljlO/v4GhrC8MHO9iZXcZlA0MAlLolSVp06apP1NMKpRKy+bRlsuzHG25CQNVHOg5+TpJJYP/nQMeXcc+qsVjeSahapBKrM/vaINCSJe7UWa9PGikjjsjXpk7yEzkAgC0jVC3ve6DcydXN+3j/vc/k6Sv2UW22EjMXEKKmhh0t3DhMqbBliGXE6RNDaAJtUFUmL9t4Z121GB8JVfGwPtjKtuvexro1x1nXNEyXNce4mefWmVVsP7oc9/Ys3Qxf0OfUZfTVVoc2oxiXBJkhq3pHcVM+l/Ud44Obv0I27VEYzdLbN8ngwDDZ4QilJSkzwGtKzrL9yp7DvOSZt3H8VCvVyRQoQU92jtGhVsLmuFTInjTo6Z7m0GcHee2Tf0xxMFkVG7pUotpWK38TZ9T51iodkPElInJZgqxApBNWrXEG1y25m4NBPPEDbS7kfG0Rsdvv5oW5+HCFr5NzDc1jCsU1TXsAWGmNLzzvKyM+NBKZTFfTvCx3khV/55MyfModCRpHbe8A4rE4RkjajG8m7ekSOcujyapgiYiU4dPrTGMMLE9Wf4p5VIT9s92sftcegmvGuPdSuDqzlxtX93BqS4FlL32ARR/fdUFRL9Qr7ZCLL5qc9Dgx2YxrBJiGwlcGu0rLKJRcjGzI2EyWVU2jnHiKoNUoU/AdKl3J+WUdLHSwyJ7ByfhQq0mWQmFUJBiap/bsQ0TQ7FaQ107yX3s2kTqesJN6SqEcEW+WaI0OgviEW8Y9PTGkRBeKaAE6lzznO3+EeKUzwqRKx8dBtSTQJr42cEVcA9xlOBSVl8iKB1k7gSdME4lG16JH1wiwZIQpFaGWOMJCPbiXV7TeTuQkZy6cSSm0ydkeSgum/DRKC1wjxK+lVVJGQLc5y8jTuuNoOYEoz0MVCggj9lXvOnTd6R9qja5e+DVUl5EXF0s+OXY1e6uLCH2DoZlWCnNxIfNdU0tRSmJaIUHJZqyaQ/qCW0urWJabQiYocDx021I+c2QLly0+julEyAAmKll6Lhrl5qf9K1+4dxNBk2KslKXk2WTTHl57wjasDCNuYqQVRBEoDZaFnJiN879CoMYn8DcOYJV0IiOVFdYkXynmucSeYCRsXtiJnq/5nYxOH8d90LfoNor1MvVhSZsB733geWAYjKs0orZhVQ5tIhX3QwiV5F+m4mY6f7T/pVS6kpcCOvql9aTMACk0rhHQapdZkx+h1S6TMapUa8dzD/sddFx3/CydwyQirPiGEUS/+lVGXZyvWdLsne7kmN/Our5hUnZAU1OZ8UqWnOXhugGmqehaNBM34Vhcpqxsjsy10XQkOV3NwmUerhliyYh1vcOYgwU60wUmi2le8aG3kc57iEgQhAaZb+WYnsyRGknWnV6VSpS6awcoTDNutFNrsgNAFCGbm7D3nKDaIk+nIxJEoCUlZbPYzBJpgSUiMrLKTJReOGjhaxOJ5NbS6rMOXSSFjfljuHaArlb5xMiTCJRBJbAo+g7FwKboOwvRMMCc5xDkk+d85QM5trQc4QlNx9iQO8EKd5zNuUNclDnBMneCi7PHWZc6wbOzDzJ0+5J6m/uozG8ijs7U6qrlr84J12XnoeOG2zlw0WZ+9oJmlDdCFpiPTUpAL6dzXrPAcia5tWMZk6/uoeemc5th14uBV8S7n3GL8QIdz1vK4cUrEXlBuVeT/26eJT+bRj24F4DWT9XN1Eek41O70ICamESkUoiWJnSpjLBtiCKiiUl0GNLzZUE4OlZvc8/hd2/6Y5wJyef/fjvGqgEOvi/D0s4p+nMTlEIHLzK5984BvvL2HZSu28xn1jyNJSTnOgL47kV52okbc49vm0E9Y4Bqr0WYEoRpMEuQ2lngu3fGlRod1+6jo54GPwxL/nY7t/5tirmXb2H0mT5NTWWqgYnWAqUE6Z9m6bnpKJ8/sZhlJPSgxRnMpxfUkbgo4FcZqTfUixs0aNCgDiRrDdygQYMGvyU0nG+DBg0a1IGG823QoEGDOlAf9eJfAUlSOn2sNMbw2GmM4WwaY3js1GsMjci3QYMGDepAw/k2aNCgQR1oON8GDRo0qAMN59ugQYMGvwTGwPJfyfskw/nW+gWImkz5WU2izzjOJxwnkb0FfpOZftVWXrnvOM/aPcN3T/3/7b13mCVVtb//7gqn6qTOebon9KSeCJOnmxwEFEQUc1YuV8xZv9dwufrzyjWnq6gYMIGCIoogiGSGIQ3MMDmHzvHkWGH//jjdPTMwhEbtOl7rfZ5+uvt0na61T+1ae9faa6/PZn7RvYGDv17O8BXlJfk9wXFKJ8f0lWP7jmKaZaWIchwT/f0Y29XFCxh6b5dHBv2LoahHr8H4d7W+Hq25CRSVPd9fy9CZx9ciPnhVJ7G3Tf1+KA/nO7HLbrzCkbRKpeh6PtXFJ/YeVTeVhcLfXMZt2hCC9GvXT/78T8W4vYWXruHCj9zH9uwMnkjO5Jux2dycns+VK27lze+7w2MjT4wsFo+WxhzvK30f7zqu77j5PL0fWIVaV+ulqSfGdSYrzAGMXtZJZm4Vb3z30z5vIRArlsD65R4Y+Twoz1TZGL2skwNf7qTv94snjwFI395O5YNlch0UtfT5T2whHv+e7prDjs/ORL+7gXBDhsz5afb8ePXk26p2QXLu1E/nfVXpY6vBL50PX01ww8Ib+WVyAbcOHmJH/qjcyMAHu2i6+Aic0+ORsU9jwqlO3NRnrODgxQbG7BRV4RyjjyhoL1+Lecujk29Z/oTgI3UP8PaZp3ph8TMZ73BCD0wOekjJRdtjJJwHGLEihJQiy6K9RJU8llTZl2+kUU94a/fTmehH49dCra9n8CfVvH7OJm7qHiR4USNb9rfR/BeN+FyFtv9+iN4Pd9H0jfKq8QBQOH8l1gdHcVyFfD5JZUWKrakZdG0Z4Y/fO4O6H2wslTV8cjtqdTVelzoSegChKqAouNlsSROwpQn7Z4KlVX3ErRDBwgFaxgsanbktyQ+2n8qK1h7UYp6A4nULnoaiolZXYi2ayegncuQf02i7w2b/4BwuvuQhbti8CqG7DHy4i2yTRCrghKfeBm+cr6KiBHTcfH7yZtn3ixX87tTv86vYej7RdzYRtcC+wTqydaURVJvVxsaPf5NLL3ybt/KHxzrcY2bh6qL5DH80i9OrozxSSS5VQfCCMY60RAms6yQ4KEgucNizyeXScx73yPgTc6zj7fmPLq5657XcMLyWZjPBDCOGOS5GmXJNXKngIhixy0A99wTXIv/ytcTbNbRzR0ikQnxvw9noMQ1jpcMH1/0VdZ0k5Zhsfmkrr629iRu+4aGczcSAMTHjovS013R2D6OpCJnuKF88/wZq1DQbM/NJ2iarL9vMncvWIYqCjm/10vuKNhq/7d0AIgwDWSggjyn1OvcxkxZjPztSzWxPNDMvOkJLKEFftpL50SH+MrSISxY8RcY2OJysZjAdoY64Z22YxHVQlnYwuqoaxZEMrZOEHwwRHpb0vsYisFfjzh92opySR/SbhM4bJDlQhToSAGXqsWBvnK/r4OYdjvxnF79559f5c2oZLZl9PJxrZ6YxBgYknCDLZ/TRX6yk8JdWOO8Qy6//ID+66Qd86dLXe2I2ULpZ1i8nOSdEcraCHZJoOYFVIXESeYQlmH/RXlpDce77xRpCIVALYIcgEFNQCipXfPP9tCwe8a4Nx+I6qDOaMX5VwFRtkslBru0/hTnhUSq1HJarEdKKhJQCNUqagHCIOyHG7Ah7f+5tHFKsWkK8I0LgLYPctfS3vOngebRoO9l4xzLym2uZ29UNjRDLBxmIR/nO0Fl8fOVfUIVLSzDB1mwr4GGJ0onB23XIXLqOgUsLNNYMENULnDdvJ4/VzeJzT11EVSTL8to+to810z9SiQw6yLDkJbdtRReb+cNjZ3vWBCUUYt4Gl6ZAkpdFn2LAqWBXoZmYHSasFUnbpdj6QC5KvZlme6KZtnCcsWKYI+lqsoUA6YOVuO/xuC+tXoo6kkTmC4RGbAbWaehJQa5e4mqClpsC5GokWl5i7A6Sm2Ez0FeNYjiIGTmMfVMXdJp253vwqk7UeWnWtx1ijfkAX+i5kDnhUZZFenFQyLoBFOGSd3VCWpGRQoTPzr0Fdb9ke2GUXwx3seuKyPOf6B/E4Pu7SM5zMUcUgkMSRsEOgjkChViQ6r0OvY/Po0eAEgGpQLFK4phg11lQUHBrc/BXz5oAwOHPd2JVSIzWNG3VceLZKAHVYUVdLwGlJJdtKBa6cNCFg4tCygmSck2yjsGoFaalwbvZitY6g+KXEsw0+tk53Mj6J97AyfV97E3U45gSNyCpNTO4lB59bzuwhBn1cW4bXoapHluRf9SzNgDs+f5aCLhU18UhHWRgpJK+dB3boi1ICStmdZO1A+yON/KK1i3sq2lg21gzF7Zsp0ZNk3UNCrXeLR7u/Npcqgr7GchXsD9bj4sg5+jUBLIoovSMeiBdS42R5UCqlvMad7Ir08TiSB/D+QhtVXF21pkkVO+UmAsXriHToBEcCxHqzRJ+ohv1pHYqDroMnOaizk+x9tLddOeq2TXSQC5WUnPRDAfXFdjJAOERgcjkpnTeaXW+3Z/p4mXnPYqKi4PCYKGClmAC21V4OD4HU7Wo0AoowqVaz1IfSFOrZ9hbaCIvdQquTp3hrQrBjJsOkfivGdScUqprG88GuXjOVtKOweJQH016gioly0I9SbMWIeZkCSk69+VCHLLqsaRKwg5xU6d3sxUAsSjNsoYhqo0sCpJGI0mlVtLYCikFqtTSzRNWCiVZHgR5N0BUzdEja6nVMzSEUp7Z3/uqWTjJOGNakGDAwnEFCcvE0GzaV3fTn4riImgyk2iKy+mz9vFfTXdiCoW8dLktM48ZeoxvsMizNhRetmb8B4XUzhqchiJCkWgVRaoqsmiqw97ReqpDOeZXDrMpMQvbVTipto/VoQNkpcHcwBDXebhyE6zIY0uFsFYkVgxSGcgTVovEi0Giep7Z4VEeG57JufW7aK0b5Y6xZXRW7uevo4s4vXYvd48s5KRZPWztm+dZG+JzdQIJSXyeRu8ZESLdFbTcn2Xk5BBaSlB8qoo/b1qPtirGK+c8BXMg7+pkbIPt8SaGdszAioDdPzCl807rZZt9zT52fqH0mCfWLCHWEWF4vYs0HFAluALzSABjDIy4pPJADjVdQEnlkGagFCPrG2JB7FH49+m0/Ch2bx+LvhIsqT7kClRoObbYc0BK9sumktKDooAiJlfdpaogwyZSVxG2i1QFjSPeLRqq8+bQfI2BtVXQ119yoD0oQPjoQYqKUNXSQsqEpPx4VX+3aI3HKIfxKgDf+L+PoF5fg4iEGHhJC1ZUsK2iDteQBAcF0R6H2I07iU2+I8fbefoiZxPKSR463yoVLSGQrXk+dMafqVXT1KhpXBSKUiUgHKJKDgeF2xPLefBIO8WeMOaQwuN9K6h/oB/7UDdB99HnP9k/iLZXb2Pw5WvpeZ3FuQt2Mcsc49TIbu5Pd7An08BgroIbl/yMiYTRV7SNcH1qBjuGmtg+2ISqupw/cxf5rwv4uDdtEC4Mn26BIhFpjeQii9B5KeJ9GigSJWTxjiUbiCp57h7r4OSKHvZmG7huzj30NKY57eCHqXpKR62fWnn7aXW+zuAQWvtspCIQQ3Fq9/VQe/t4bq+qgKoiK8JIQ8cNqBRqTewmg3xNFYoFxQqBnqqhes/Upvd/93bs2f+i3zuxROelGJKz7yCmEFjtTRRXtoELgUQRx9SwoipSFehpB1cVoIAUAqmBqwmQYIUVhCOp2u3hU4jrsPLOATbHW7FuEag5qD3iIFxIzlLJNKkkP96FHQTFgny9ixtxwB1fpAu4IGHWTd6lAVZc9zCDF6ykckOQ3/30PADGFhvI8QRQPSWp2ZFFH4iDZdMeSCGVNG51BDeg0n9+M4WqFswxb9MvzVseZd4tcAg4cvIqfrv+bGLLHZSohWsrvOYHH6N6SwwRT+FWV5BYUkXVO4YYTkTg8Uo2XL+O6ODDntnf8L8PYX+ii1y9xJybJGQU6Wo4SLq2j8FcBX3pCn6xby0zq+KEtCI/ueNsjDHBgooOwr2C+U9kULMZkme0T+m80/7AYh849NwH9B790Rj/Cj/bsT4vGmfvAcTe0uc7gcbUOoTXGdePnawC/TTTf9zr/0z9Zf5bnzju98YTrAWcaKAWQP2GY164+u9p1YvH3byD+s08Q+JoMhGrt4/oNuA35XWdWr5cyhhR6+uhMsLOwDzcUADH1DAaDDRdkI2F6Z+tU22BHQYjDlKF1KwgRjxA5JbNz3OW4/E+z9fHx8enTHCGh2H4qIakAoSO+Xvdc7x3qpOR8tjh5uPj4/Mvhu98fXx8fDzAd74+Pj4+HuBLx/v4+Ph4gD/z9fHx8fEA3/n6+Pj4eICvXjz/r0RyAAAgAElEQVRFnqsNE9XBFm3S2H2KVqra9hyUYxugVKlq8LJVFGqg7QvPXTGrXNswFcquDePVzrTmJvZ+YA6BhKDtu1txU8++nbvs2vAi+Fdrgz/z/TsirSLxt3Syc5XNio1Zr815UQg9gCwUqDxkoWVh9N86UUzTa7P+dRh3vHuuXovdP8Cc/9jIjP95CDeVovu3S0vHHKPu4vPPy/Q632dTdJhQHjj2mGNf+ydi6LTSfqThYhR1ycJ/uhtloq6vcdtjpGe5jJ1aLNV2+CdhQoqq8NI1DF/RiXvaCrT22cccUKZ96pjaxMk3rGfBu4+p1zDeh+qiGQ5+sROh+3uj/i8wrVdRjUZJ31hL745G9JQg0g3pNqjcd/QYO1gqdOGqIDWBq4JakAgJdkjgGDDzt/3PfhKPEHqA3AUnM7HP5fCH5nH49SHmfF5FTsiSHFM0u5zZ84M1LHjXY3R8d5g/3HMDF6VWTRbNLmsUFWnb7P/qeuZ+7OHJLa42sOpJlycuXw5b95ZnO6RECYfZ+7llSBWq/hQ9GmYY7zPB8w+ifboZWSigmObzhrWmA6FpSHuiWNYy9n1IIxgqUChoSFfBMC0UxUVXHQKaQ1s0zqbH5jPnj0XUe8a3VpfLfaGoCF077vO9o28znxpczhcbn2L1le+m9pqNpWOfpmLzYphW5zvwxiXUX5kheKaCccoI5pZqMjNUhtc6qGkFu8bG7NEpzs8hYwFkQKLkFMwhBbNrhGRvFdG9GumlU6seNB0IVeHIK11qN5RmiWLjFtxLOpHOuESP45RHB3sBtM8dBEoFhHRRmnWJQKA8ndaxuA4IQdXOo09PQi055F9v6GSeUUBx5fHSVWWEm8mwdM1BChdlkfkTf9Zt/12KwZeD4wUmHS9A83cOcXpoiGsePIP7Lvo6d2bmsczsJuWahEWRTfnZbM/MoPVul8RH0wQ+2U7kggPlc1+4DrIwXrlv/POdc/O/o8dV7j55wVHHC5P9528ZBKfV+VpRQeI/s+R3hmgM5pnxsZ0M37WEhvZRxhJhtO4Q5hgUYgGaFwwzt3KEhx5cQvS0IdJ5A62iyCvfuoGH3rd2Os1+Qbj5PG9Y8Sib3qlMjuThXlG6oFA+Hex5sM9eRbVxmImlncu7TwEyz7nYU24EY+N1LqWcdA5KQaDFczgTOnXlwjGzLYADf5xLc+rEi5wTTx97frqKBe/YNJ1WPitCDyBMgzc/voNbRk7imodPB1Vyxq0fQQQdRFxHzQocUyJcCMQV1A5ID1Zw3rLt3HNVJ3pS0HqVh1p64/fr3u+s46lXfot13/0I29//PQCGnA00qGF+narm1CPdHLAjvPf770FLwzvfeys/+tGFL1oHcFqdrzEmKVga5pwUS6r6eWpsBuec/yR/3roUNaax9pRdPJldxPKTDrHt8Tnoy1xalg/gSMG6lsNYUmFvugE3UJ7rhNc/uZYFPD7paGu3j89ejg07HPt7GXLg1Sor5NG46D37FjCXJ4G/bZT/uzM+exVrlnHk/Ch2WLL1rd+m45b3EuxRCfUsQ03kcCuCKIksc24uMLKmhvQlXagWqHloeCyD2LjF23YcM9sCCCQnlLyf+Vgri6WB46un3MiPG9cjU2mkZR8VPvUAaRWJ3F1BWCmwPNrLE9E2rLhRir6lNFzDxTUp/S7AnpGjoTbBwmCaFZEjJE4P8uS9Cz2zfwJ14Tzed9adHLDhnvd8hX5b4gBRRaXHTvP6KBy0oNNwePADXyPhOtQoGpd9dC8XbX03+l+nPhhOq/NNzYHiaIQl7b1sGmmj0siz69NL4Q2S6EGFPTs66Lr8KR7/zXIiDowMtxAakJz1wY385UgH715wPz/Ydyp1Gev5T/YPxjlrJdnGAFreJfrwYfK/NFjwqQLZV60junMMEUuiDqQRs2fiVoSwakPYQZXQ4STO9t1em/8MBm5exMqmHs4I3ceDo3PZ9431uEEXYducvy3J3SML2fPQbOZ8auPz/7PpQEoaN1ZwJJXipbXbSTsG6x9/C0s6ugkstjm8rpqoYRPL2kRNycq6bYwUIuyL1yGlIFvQObhex3n9ek+bkbm9nbfOepiCq3PnyCJkcpT+ii6av3aC2dS4I/7Yfa/DvEIvSVdVP3864D+a3FtCfPjK17GsvZer1/6SXYUWXKmUVFucIM36UbmpGi1Nn1WNKSx+37+ChZWDND7qcPCqTu8a4Dp87Y6fU6tKUq7EkZCVUKMoFKRLjRJgyMlQpShsL9rM0iQKoKAw4ha5++c/5gN9a9i9emp+aVqnkA1PuAQPBtizcTar6roZ+M0sCh8Zo6omQ2jQofaN3QBk12Qxx1yMU0boeNd2/nRDF80VSa7rXoup255qVk0QbzfINCmkm1QyK2dSH0zTc24UKygYXV1H/yXtjK6soe+iVtLzKhldZJJo18m0V3pt+gkJBiyc8RlvomBy7ilbWLboCFqfQUgplHTdvKwAfwKG3tNGwVGJaAUiagFdczgcq2b/WB0BzcF2FQKaQyJn8tjwTIbzEVTFRQhJVTjHJR1bmHeDtzP5hlCKCiVHXmosqhjgzJZ9XPrWe1mz2aH7s12o1dVoTY3kX74WcfcM3rH7MN8681e85dK7+MwHf8nNl3/FU/udM1diHzrC/572K0y1pPnnSgVd2DgImvU4ipCTX3EnTNoxWWz2MpINsenLqwj+4VGcVm+vw6JAiIwrqVQEqhBEFYE1vnhekEc7flSxyEgXFRhzi1QqKvfmFD7RcM+UzzmtzjfVqmKsHsMxJQ9dvZpMG4xsaiRf1Bm4uEi1meX++5cRDBVIvCJD1CiyeXAGuWaHvt/PpmdLM6c2HiAxy/vUp9CoQ3DYJZCSaDmHR/fNJpAsqT0EMi7hIZdoT4H4Ehs176LlJEbMxdXLNNUJUIXkJ3eexeC2Bu4/PI+BdJQvXHodX/3zy3lt4+NEDntt4fEs+dFOFlcPMsco1V+tCWapDuVorkhSG8xSYeSpMPNUBvNUGnmylk6lkaclkmRWNEZ/vhL90JCnbQhpFrP1EbKOwUxjlLRjsCk+k5v2nUTDaX0suztG6x+T1H/iAJri8tuhVezMz+D86FYezbRzd2aBp/bvf1upP0eVPLNCY6wM5NGFTY2Wpk0fo0rNUqumqVIzhJUCDVqSkFIk5ZrMrR4leqCkhhJ6aurqv39PRpwMqgBLSpzxLwBHSnShsLVYQUQx0AX0OQbDbilokHId2vUkrdrURX2n1flW77FIZ0wq9irElkrC3SDbs8ytH6GtMcYFtdtQirBpzS9prExxasN+Tm7sRQYkjgGKDb/bsYKaXd6vutumQjEqKFQqOIaKiAVILHAIZFyKYYVMg0JypkGoWyPTqJFtFmRaFPR0ecZ7Z1eOUXA0hCNoO6mffNJg5GANfxo9CSfkcqDQgB0ur4Fjw2A7OUcn5QR5YqyNikAeXXXQFJeAYhNQHNqjoyyqHqTGyNIYShHRCxRdlZ50FcO5CDKT8bQNI/kwSdck6wZ4IDafA6k6ao0Ms2vHCOlFdOGwNnqQkVwETbiM5sP87sjJfHTva/ndzpM5Uqj11P5Ab0l1+FcjnXQE+/lo31nMNwZK6uNKYVLBWEViCgtd2CVhVlGkzsig9pQGzrYfbPOsDbG3dTLoKNQoGuqz5IHHnTCWdDCFYMCuZMwJkZUCS1IKU7hTj7tPq/N1dQE9QTKnZXArbbQ8nDZnP41miqEHWrh9dCnFOoevj3XQ+0QzdXqKk6M9BHs10nNtzjpnM9p+k2xjGcx8B4toOVAtSapNo/lBSXRmEj3tIhVQC6XsjlB/aZVXT5c0ubRMmT27j2O7ClE9DxIM1aZlxhgt84Z56MBcMB1MxcLxTt37GYy8q5OGcJr5kSFSjkm6YGC7CpWBHIqQuFLBRRArBunPVWBLhYDqkLUDBBSHpnCSSiOHm/P2cXc4E6ZWzRBV8yyN9nFG/V6ajQRV4+3YlWrkrrFF6KpDayjOgsohWqNxqowcK2d288XGpzy1v+2u0kTorvtPwpIqnRX7adcSdBj9WFJDRaIKF1W46MLGkqUZo4MgrBawBwYZe0cnTjLpWRvGlpbs0YVKUUomIrcTs99hx2bYjjI27mArlDwtWoqsq+FS0pBNuEW05qYpnXdaF9yKEQW7wqHm3hCxrgIjZxXYFW8gWwhgLcqy84YOKoCfjJ6LMi/DX4cX0ZuopPmMHgaTUR79+QqK813C/d6nCyl5BytaGiXVnCTTpPK69if48QVnEewXqOOTcztc2hgSiEtcnVK6k4d2PxsHYrUsmdUPAg4O12I+FEGxwTwnQSYRJuEE0bzVLT2OxvtHeOfHHqRKyfLlIxewtK4fBYmmHP10bVcl5+gEFIe8oxHSLKoCOYKqhSUVdOGS8Dh3ObGnhlWrAjyWS5FyTWJWSdlsYWQQS6rowkEVLjEzhCIkmlRZXXWEGi3Ndd1r+frY1EQb/96o9zzBHX2b+VC/xqBVyb9XP0qzFuGG0fm8ofJJLAmP5dvIuAYJJ4QlVcbsMPVaitnmCNvnrabmp94u4ppzk6hIHi+oLNYVDKEx4hbRAVUIwgJeHd3DTxMn8aHqPSzUk2QlhBSbUdfAFA4bcq0MvmzOlM47rTNf4cJ5K7cSvbSfsxbuwThosqqum0zWQN8ZYv2bniR4/hBzuw6jbI+Qs3Ve374J+1tNzKyO0fDKI7zzrHspVnm/vVIbTSMVkAq4OrgatARiiPrSzSzc0ut2EMS4P9AzYFeUZ52E+nCGvKvjmC7SFWRaJcn1OTJDYQg6GMKeVNUtB5yde/l/17+FQ1YdEb20IDhaCJOyTGLFELFiSXkrrBVRhKQ1FKcmkMFQbTTFQRcuhur9U0hgrPShZt3SInJELWAoNqpwCSlFWgNjVGsZIlrp9QY9halYRNU8FUaeecaAl+ajmCYvO+NVbI210JOv5jujXbx6/7mcH9nGj2PruCl1Ek9l29iUnk3KMekvVlJwS456mdmNs+8gSij0/Cf6B5KJBwkJhzYty27LYJ/tUqNo1KhGKeaLoFIxmRUY4dZsJXlZmimHhKRdK7I538oqs5vRNVObVk2rFytUCXbGmhjLhHhp83YeWJjhqbEZ1FRmSK+w6c1WEVAd5kWHGVxTCmD/+sAq4q9wOTuYRFMcUo5JbL73zrf/vEayXWlcR4UBA62ttHCwoGWQnZkZoEiwFdSohWsLZEFFTWgEMibedrUTk7N1bFcBBRxb4VXnPMwjw7OpC6bZvLmdeeYAilVGu8KEYOafs+y7uJGwViRWDBFQbRQhMRWLvKNTcDWq9BwVWo6kHSSsFajRSzHenZkmGoMpwNuww+wf7Ib3QoseI+6EMDULRypkXQNLqrTpoww7FaUZMOPxU+EybEep0PN88P43lnLLPcLN52HvAbRzoXVLabC4pGoTIcXmsupHGHAMPtu+FsWAg2o1SjSCtG32UMkdIxWl/5H1tghV1aYAfWeW7spDVj09xRrWhfZTlCouEe5PLcRQbFwEd/Uv5NeLf07eFdydm0XKDfLeqm52FiVqemqzk2n1YoGUpPtgPUs6uvnxti7sokpTOEne1tkyWEksFCSeCfKn3SuRmqRqwQAL64Z4NB7isYE2FtcPsrFvNjW7vJ+xhAcc4nETAi6EXJzRIF/YdCHKERMRdRG2wDVcGDAQKmg5gVIUWMHyWrSaIBIoYEkVNaPQMC/BvX3zmVU5xlA2ijRcBq0q7FB52S42buHBobnUBdPkHZ100cBQbaKBPFk7gKlaZG19Mv4bUIJU6nlagzF4V5BtX49Qg7c795yRUe7NKYw6EUxhUZQaKi6KcIkoFhYqllQJCBsFF3f8YTWq5DmzZjdP9Czy1H5gcsPLI2c38eDidfy8UqdQqVCsECChXm6c3JzjerzAeSKkCrpwiAqLUSVH3tXZmm9jVmAYVbicEt2DjsMMLckis28ys8Glm5/HOllw7UW0X7mJudbD8JEXft5pdb7Rw3myO0Psjs/GrrMIDOrsfKKD4PlDmAcNBkcaMcYEhg5aDkb2ttLdIDEKglSLyiNDEbS4RiTk/QwsciiDHosiFRW70kGNWiiqg1VrgyYnZaRFZRFNc7AKGg5Qeav3A8eJcKXAUGykKhnY2cCMxYNs2jWHr57xGz659Y0sNPqwpp5N849jYufXN+vZ396CY4CWlWhZiFEKA+k5iVqUpbCPAOFIRoqSQ0UXa5GKvLk8Ks59ecWpvP3xLVhSY8yOoAubei1JWCmQcoJElTwOAlcq5KWCgouuOPzg0OnMutLbDRbA5LVwRsdQHhjDBMozuHZiGr/9EJ/99pqnvRoAqk94/E+Zddzvc9g4Zdl4mGbnKzZspmnDCf7wLahk3wn+8Bxc/3cx6UUjH9+G8enFZA9XoOQVHEXDUVTQZGn50y3NEqUUFHM60lHQQ0V4dKu3hj8LPfEq2qOjKLZgzopu9uxpAd3lq/tfghuQ/G5stdcmnhDjtsdoeBHv0wBvM0uP4iSTrDJ6+U1iFZVaFgdlPLVJI+6EcKWCg8CSKlElj6lYZB2d4JVRr033+RvwPnj6T0zLpbvJvGo1faeDyCvISCngbnYHUIqAAFfXsSpcIocVGr/tXWzu+Sjsq+BQdQ1KQXB4QxuVIwItIxmb00AoJbhn/wIadpRjnsb/Dd4z61Ryl6xlZKmGcEBIkKK0cHvsl1IslVgNDziYDz/6/P/Yp2zx1Yt9fHx8PKCMkod8fHx8/nXwna+Pj4+PB/jqxVPkhbRBrF7K4QsrMGLQ8ucBnL0HjvnjURWFcm7DC8Vvw4vnOdsw3k8OfaGT2Z95/h1gZdmGKfKv1gZ/we3vyXhF/At/9gDzjQGqlCwtH8thCkFIqEQUk/NbTkaJRv+plCF8PEBK0q9Zx5de9wvmv3mYVg0s6ZaqblFKqLk72871HS1eW/rCEQKEcpyYgLpgLs6e/R4a9cIQmoZ0JUj3uAL3amMDe74+gzPm7qVnfXpK/7O8wg7jFYXU2hqSb1x/3GtPP6YcEWopb/SV0e1UKVlMYdNth7gx1cGwYx+tfOT4WQM+z89LP3sfCi4OgkHHZcwFCyhIGHN0okqZqIo8HSFK8kiGgVpVOakojZSTjldZ3sGeH6/Gutr7Oi0vBGnbJduPcbyLNmnc9uRfeOeyh+iqnGKqLGXmfBXDoPeTXay8axjzHf3s+clqDnzpeKUBtarKI+uenwk5lwN2hM35mSwJaOjC4TXRXeSlwrIbP0DukrWeb6d8IYz8eyfVG2rYe+0qr035mxm9rBN13tSKnnjK+ARjjjHMTC0GQFRIVEoKCrqAWZrDpZEkQ+/r8tDQZ2HcQY28ZSW7vtDBvp8tI3N7O+/eu49vHnqI7x1+kHf97k9cderv+NWCX3P4c2XYhmOQnSdx2lN5LttzkKE/dLB0k8I1Rx7kgsqtbC4UWBrs5mTzyJT/b1mFHdx8nuKKNK+ufBxTsbju3hk4y9N0f7aL2m0OxajC8CpJxxfK+zHFlQo7si28bmQxFzdsYa0xRIMKUpcEb/7nyM0cW20zKzRG07Ikf/jJaha882iO8p7vrcWoyzHrtWW4YWQipj7x/a5Wrpv3VRboYfZYGd74/32M4KiLYknUoot+Z3kIUT4dramRN0U3s7PoMOaapR2IolQQCCAjXaqBd73nD/z+f8tMzXtcNVpcMsqdy35CytVRhCTlBthSmIGDQq2axlQtHso38shlX+O1V3ooI/R8KIKZgRH25Jv56tIbqVJyPJRr4ySjl6zUJstkTpXycL7HiAX+dO21ZKTOuZHtLHpbHwB9S6v50d4usnuquKBrM/uHy/RxC8i/fC1/TsK66AH+ra6XqLD5UP+ZbI83M/99j5StbPkk43Hr68/9PtcMncHM4Bif67qZt/aNMOdPl3Pwomv41KDLXHOIG5ha/dJpQcpSfM62mflImI803sBeq5aUmyQvg3zv098GYKFuc8HWN6P/pQyvhZTs/I/STD3hGqjIUv1bxT3usPvzcEVVL7+n3JyvgrSKXNS2jQdy7ejCwZECU7EIKQXU8QHEFBZ5qfN4oZz2rT8TvWeUBi1Fh9FPrVLg9swiwkqpemFGBrCkisPUw6Hl4XyPcUajToQBu4paNc1sfYRhJ0qjHudzS27BWazQZfbxdk710Njnpu8NRUzF4qlsG8N2lPbAECOFCMN/aCNxTRMLLn+svB3weExue2EGmuJQcDX6rWp+mAhw8KJr+E5sFguCA7RoMShH5yvEpFz8NW0buDVbw2J9hKxUCYnS66ZwcaVCR/UQfV7a+hx8+ryb+WMmRJWaxRQW6jHVAxxZiv3OUNPcnGnCOq+8tn4rAR0377Ao2EufVU29lkIVz+zvcTdE0ilVgShc+PTaCuWDzBfIuAZRpVTQWhcOUSVPQaq4UplU65gq5RHznVhoq64m5QbJugFSrsm7d7wJOFrr1FSK/PfgOdjnlGEcUlFxTz2ZvWdey8UVT/LaqkdRkaTcIPPDQ3z+/dey6CtjpWPL1fGOoyztoKdYw/JID4qQNOoJCq7ODelKdmabOViox1S8V5A+EYpR6iu5V6zl3pzCDDVBXiqEhENAuJjCJStV4q7LD9vu9dbYYxCadnRhCnhFZD8VSh6V0kABEHc1BhyDQSfIqGsw6hq0aDEOvqo8buMJJiqYBYQz7qhy6MKeHEAsqaHgEhIFmrQEurBZeKV3MkLPh8yWnO6AXUWvEyHhlKqC6KK0GDpR/nOqlMdVG3dG3ZctGq/oVErDWt94iLgTIuUEKUqVlYER/vLXlbh6eZh9HK7DkQuCpN08w06UjAzQbgxyf2IBzYE4F4ezHPpikJ7fLfHa0mdFiZYKtRz+L40juRrSjok+Xgk+pBQ4XKwjrBZo0JMcKtZ5aeqzMjHrvf/qH1KUKnWqRUC46ILJr6jiMOYG0EV5VDWDkt0TtqsL5vJ4oYYaNUvGNchLnYzUsaSCKRxC4wPfUl2yWHdY9B97vTT9Wem1qhmyKiblhBzEuEO2yUsdFwVVuKWZ/QlmxuWCm0pRryWZrY/QpqYJKUVq1fSLCjUcy/R7MSEQeqBUvV45vvP/6n1fJ+UEMUUpNnRqxR6gVGi6Vk1zR7adPW+7msDtj0272S+E3e+8mqvjSzjdTLHOsDCFxe54IzP1UXYWs9y19vt8ftkt7PvFCq9NPZoOpGmTTx5uKsW1Rx7kvYvvoyPSz1Axiosg7+rowiZhh6jWSpkaW7NtHPyit4skQtMQeqDUjyaenurr+Oi+7dyfh45AjDFXIy9VChLyUpCVgmEngIpkyCmf2rITs94znsrx2j8+QJsWZ9gp6buZwiIsLOrVIik3wPKASV7qfLjvDH6ZnIsTi3ls/TjHpIGOXt5Jwg6xyOxj1IkQVXJUKVmSromOQ9wJcUGoQJWSo9uq5dAbmj00/DkYb9PpJjyQXcCWYhMqLrsKzYSFjSksFNzjwkIvlOl3vlIirWIp3Wo8vph7xVrWbHYYsKOTU/isa9BTrCUvS2KZvVY1VWqWHntqiczTyfZijpdFtvGl0RX02QWWBZI0hFKlsoAImrUIn3ryEua95UmvTZ3MuZS2DVISe1snl+4c4qfxVQxZFRzO1aEpLg16kpRrknDChJQilVqWhB2iUsthVXuYrzwe25VWcTL/UpvVxq2bbsdFoVbJEXe18RvEIeWWZo6OFISEjSEcBhzvZ75CL6mSTsx6P1W3m9n6CHHXoErJ0WtXTT59bC3WcYqp0PmxK3j7A++kVs8QUrxX8p5EyskJVeubD+BSWmSL2WGGnZJqRZWSpc+u5q0VI6x54rXcl+lgpXkEZ99BLy0/MceszcScLPONgZKzFZK81ClKhZQbpFZLk5FTV5f17Plddp3E6GWd1G6o5jNf+ynLQ0dKI2C+lp35FsbsCO3GEIuNXm4cXs2rIj00qQkSrvc3zCTHjPTxt3aiIGlRJZdWbuJ/Bl/CV4ZPJaA47My3kB+PCc1+nbdqs09n+IpO9nxvLY9edTUqLpZUadCTxy0iTIzq7nhN2YFiBVkngCh6GP45Jm6utc9m9N86uXXjLfwxE6JeTdFrV9BnV7K92MBeq5aM1HEQpUWS8cdFSyrkXrHWqxYAR3PDAYbe18V3423Uq6UZuSVV5usjZGSAuBvkwlCeDXmXiuseJlJZikP2Wycu+D1tHHMPCE1DqCqJN63n87P+wNJgDxtS83llxWbWmEdYHIhxyKqnK3iQl515KTUX7eGuly5mke69GvkEQg8ctylErark8Oc76XMEZ5hxatQ0llSpUdMMuyF0YaPjlOLYU9Sim95i6prGR3ZtISsNDhRKUtEJJ8iGzAIWmAOcFtrPFze9lP1n/5S7cirXDJzBE0fauHzZg/zPyBqWh45QpZTRBoXxkd46+2Qeuuq77LHgzlwzvx1aTVswxpUNG/lo31l8svZJ9tkuliyPnW3a7JmcfstOegtVzHY3E1ELfDM2m95CNbpw6C9WUaHlSTsGe3ONZGyD3mwl5zXsIOsYtAeHqVHTdFy5E97vTRvsc1Zx+fduIqrmcGQpPHVr1izpmznRUkoTLjVKnirFpVIJoAoVDRV7XD96W1ES/IO3edd39G3m2mQDHYF+eu0D1GtJRt3geKjBxkJhtpZmphbh/JaTJ9+X6o+SnBGk0mtJ6WMGQWnb9P6/Lt791lv4av95vKruCdZH9nNHejEAG+PtPHp4Fr97YwMoh0AI3HiCgiyfxVu1tppk52zO/q8H+Vz9dn6caCLlHmaBHqDHzrGr0EpUyTFDj2EKiwAOjhDUq7kpb56aVufb98G1/CkuaDPHWBk8BMBqI82fMq2kHJMHsnNpuSlA521XkJytEOoa4TMrbiXrGtQYGVQkWWmgLpw3nWafmIl4tevQ8vn9xNwcw26Yl4UGOXvWH/lhbCXfHjuJi2ue5Otjy3lp9CnSbnk8Iu76UAsdxR4aAinyrplOS6cAAAa1SURBVM6RXDUNgRQzjBhpxyTv6iRtk1o9g6lYpFSTl1RvQ0VyT3YRG/rm4EhBc3ynZ20w+lM0aYnx9QELVwqiikXcDTBLswgJnUGnSMLVOWwb7Ck2sj3XSne2mqKr0puuJF/UaVg/9cfFvzfOeDZDkxYnL3VUJAHhoow/cczUImzIH5/OFNmvUVhZHpmiAAe+1MmF5z7G9+u+zDVjXZwc7QEg7oTYnGpjWbSHb8/8E2865ZTSG1yn5HwzWbJlMinJvnId137za8zVI9yQruSyI6dSZ6RpDcTYkNfRhWDErqA1MApAlVIk62pkpI75IhYMp/XqNT6e49DFteScAIdzdfTlKqg1sthSYXXFIUKiwBVX/ZYlgT6O2NXcnVzMwUIDj4zNpuBoGKrNmprD7H9zGSSVuw7K0g52fTDKfP0pfhhbybnRbWSlQ15KTo3s5nQTzm85mfTt7Zy+YBeRCYftcZ7vvA89TNv2OMvMbnbkW3FlaVEtYQdxUMjYBi1GnJWhQxSlynfnL+Ax5hC+v57dww1Ylko07O1GF2fHHt7/gytIz7NomzVChZEnoNgEVIe6QIaRYphHn5xP0wOC6gePYPf2cfbWDItq+wgpBSqa8/Ra1Xxn7as8bQcwns0QICyKtKilnNg+u5R5cppp88dMiO/OX3Dce6r2OxiKTUTNA2EPrD6KWl3Nxec+QlC1+EN6KXV6mtbAKKawqNeSvKX+IRwEb2o75bj3CVVF2ja3Z2Y9y3+ePs54KkdPYQu3pZewLdPChdVbuLzhPnrtakylSNwN0aLFOD+ynQEnwkI9wZhbcp9RpciAYyA7T5rSOafV+Sr3PcmrG3u4L97BppFW6kMZGo0klVqWmB2moOj0FqrY4M7HclUU4TJWrKMxmOJQqobBdIQHnbkEh7wtrjN6eSc3ffYrzNQ240iXQ3aWK3svYqBYwcca7iHlqpxuwppPv5saNhJQSyO7Oy79PdHpvOT2lY1c/Z8v5cY3fYPl1Sb350uzlJQbpF5NsiwQ4/TrP077J4+WM/xQ651srWujVk2zKTObbaa3MoktXz4qHmkpKvZJHaRDOqkDA8hcjvnxRwCY+KRv/uI5BIctAqN5lGwBVIXGHQ/BNz/sgfVHMUVJLr4oVLJSQx0fmGvULKow+Y+fvp1WjhfKDB9KoylH5eS9Iva2Tta//3Fq9QyqcInZYeYYwwSEQ1GqjDkRVOHy3Ze/HDg+JU66pXYmHG8HD4CC1Ggx4gxaFdiuSrdVS7dVy/zAwGQ6nCsVEHBSIEnChRrFxjpmDqUfGZ7SOaf3uUVR+VVHK5Cma1OWddH9LAv0YwoXVZRW/+7LzeLu2CIsqTBWKK2QZqwAiZyJdX8tzg6Lxtsegm95d8P85jNf4c7MPOYGhqhXc4DKd9pu48b0PN6w461sWH4T57ecTA0lxzWYiI7fYONXSnifpywLBWZ/eiMf//T6Zz2mnePryP53+0TMsRaQQBlt83Yd5JPbERx1tk+n4vqHgZLl5fGgW2J98CDbi03kbZ3lRi81qoUuXAacCO2//Tfmf/GZCsXKwR4iasHzzS7VP9vI7p/BvR89hy9ccS0Xh4/GPTcVipwbjPPK1rU83fECk9lOKcfbQbz7M13ML26iQsuRdgwajeS4UrTGIaseS6pUqVmG7SiWVKlUc8/YVKELB7euckrnnV7ne0wdz22rXLYxh/zLX4cxWkDYLkrBxt2yE5hIJystygWgrHavf7rnYi5vuo82LYkuSuX9+hyXN0cPsb7jAKe994OEeGTyeNtSSbkmIaXUfqFrx61y+/zrct6r38aRD7nsOvUXjDgZXCAvoV51mafnmf+BR074PieZpk5PlU1ZyeavPcR3v7aA79fWkDxzPrYpCGTcF1RI6rrrzmEGzxxgpgsjDmdV7uTSSHLytf1WmpCAGtXAEBPZGKMnfH/azbPDUrlpx9QKfk2rgOb/1ar3yTesJ1+t0LgxgXxy+4nfuH452mAC++DhyZfKqQ0vFr8NL56XKK+RE4WM/lb86/Di8aoNvnqxj4+Pjwd4H3z08fHx+RfEd74+Pj4+HuA7Xx8fHx8P8J2vj4+Pjwf4ztfHx8fHA3zn6+Pj4+MBvvP18fHx8QDf+fr4+Ph4gO98fXx8fDzAd74+Pj4+HuA7Xx8fHx8P8J2vj4+Pjwf4ztfHx8fHA3zn6+Pj4+MBvvP18fHx8QDf+fr4+Ph4gO98fXx8fDzAd74+Pj4+HuA7Xx8fHx8P8J2vj4+Pjwf4ztfHx8fHA3zn6+Pj4+MBvvP18fHx8YD/H3l8K4iwOjYpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 70 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "samples_per_class = 7\n",
    "\n",
    "for i in range(10):\n",
    "    idxs = np.where(train_labels==i)[0]\n",
    "    idxs = idxs[:samples_per_class]\n",
    "    sample_imgs = reshape(train_data[idxs])\n",
    "    \n",
    "    for j, idx in enumerate(idxs):\n",
    "        plt_idx = i * samples_per_class + j + 1\n",
    "        plt.subplot(10, samples_per_class, plt_idx)\n",
    "        plt.imshow(sample_imgs[j])\n",
    "        plt.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "BkKfOh5CwPKT",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### 4.5. Split\n",
    "During this course, you will see that we mostly partition our training data into train and validation set. Implement the following function to split the given data into two sets provided the ratio between the size of validation set and the whole training dataset. You **cannot** use `np.array_split()` or `np.split()` in the first function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "vcemtxWgwPKU",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def split_train_1(x, val_ratio=0.1):\n",
    "    \"\"\"\n",
    "    x: a numpy ndarray\n",
    "    val_ratio: ratio between the size of validation set and x\n",
    "    ouputs: x_train, x_val: numpy ndarrays\n",
    "    \"\"\"\n",
    "    x_train, x_val = x[:int((1.-val_ratio)*x.shape[0])], x[int((1.-val_ratio)*x.shape[0]):]\n",
    "    \n",
    "    return x_train, x_val\n",
    "\n",
    "def split_train_2(x, val_ratio=0.1):\n",
    "    \"\"\"\n",
    "    x: a numpy ndarray\n",
    "    ratio: ratio between the size of validation set and size of input x\n",
    "    ouputs: x_train, x_val: numpy ndarrays\n",
    "    \"\"\"\n",
    "    x_train,x_val=np.split(x,[int(x.shape[0]*(1.-val_ratio))])\n",
    "    return x_train, x_val"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "6KVMhZ58wPKZ",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### 4.6. Shuffle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "CJAzm-RDwPKb",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Shuffling data serves the purpose of making sure that models remain general and overfit less. You will want to shuffle the data to make sure that the training/test/validation sets are representative of the overall distribution of the data.<br/>\n",
    "Consider the function `shuffle` which takes two parameters: \n",
    "1. $x$ which is a numpy nd-array with shape $(N, d1, d2, ...)$ \n",
    "2. $y$ which is a numpy 1d-array with shape $(N, )$.\n",
    "\n",
    "You have to complete this function so that it shuffles x and y with the **same** order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "h5VCl5aNwPKb",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def shuffle(x, y):\n",
    "    \"\"\"\n",
    "    x: a numpy nd-array with shape (N, d1, d2, ...)\n",
    "    y: a numpy 1d-array with shape (N, )\n",
    "    output: x_shuffled and y_shuffled\n",
    "    \"\"\"\n",
    "    order=np.random.choice(x.shape[0],x.shape[0],replace=False)\n",
    "    x_shuffled, y_shuffled = x[order], y[order]\n",
    "    return x_shuffled, y_shuffled"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "TAd2BZM1wPKf",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Make sure that your implementation is correct with the following simple test:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "n49ASGnYwPKg",
    "new_sheet": false,
    "outputId": "cd32ab1c-3038-48fb-f873-1d23713f7a04",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[3 3 3]\n",
      " [1 1 1]\n",
      " [2 2 2]]\n",
      "[3 1 2]\n"
     ]
    }
   ],
   "source": [
    "def test_shuffle():\n",
    "    x = np.array([[1, 1, 1], [2, 2, 2], [3, 3, 3]])\n",
    "    y = np.array([1, 2, 3])\n",
    "    x_shuffled, y_shuffled = shuffle(x, y)\n",
    "    print(x_shuffled)\n",
    "    print(y_shuffled)\n",
    "\n",
    "test_shuffle()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "s4d8rZ-KwPKj",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Problem 5. Linear Multi-Class SVM with Soft-Margin (18pts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "v-i1iRAwwPKk",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Multiclass SVM aims to assign labels to instances by using support-vector machines, where the labels are drawn from a finite set of several elements.\n",
    "\n",
    "There are several ways to define a loss function for multi-class SVM. In this problem, the following loss function will be used for training the multi-class SVM (Please take a look at p. 46 of the ML Review slides):\n",
    "\n",
    "\\begin{equation}\n",
    "\\large\n",
    "J(W) = \\frac{1}{N} \\sum_{i=1}^{N}{L^{(i)}} + \\lambda ||W||^2\\\\\n",
    "\\large\n",
    "L^{(i)} = \\sum_{j \\neq y^{(i)}}^{}{max(0, 1 + s_j - s_{y^{(i)}})} = \\sum_{j \\neq y^{(i)}}^{}{max(0, 1 + w_j^T x^{(i)} - w_{y^{(i)}}^T x^{(i)})}\n",
    "\\;\\\\\n",
    "\\end{equation} \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "uTaG0j9BwPKl",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "First, let's use the functions of the previous problem to preprocess the data and make 4 sets `X_train`, `X_val`, `y_train`, `y_val`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "Wl4t7xURwPKl",
    "new_sheet": false,
    "outputId": "fd74aaee-9b4b-4ed8-f360-673793faec4f",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000, 784)\n",
      "(10000, 784)\n",
      "(10000,)\n",
      "(60000,)\n",
      "(59000, 784) (1000, 784)\n",
      "(59000,) (1000,)\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(42)\n",
    "\n",
    "test_data, train_data_list, test_labels, train_labels = load_data('data')\n",
    "train_data = concat(train_data_list)\n",
    "\n",
    "print(train_data.shape)  # Expected ouput: (60000, 784)\n",
    "print(test_data.shape)  # Expected ouput: (10000, 784)\n",
    "print(test_labels.shape)  # Expected ouput: (10000,)\n",
    "print(train_labels.shape)  # Expected ouput: (60000,)\n",
    "\n",
    "train_data_shuffled, train_labels_shuffled = shuffle(train_data, train_labels)\n",
    "X_test, y_test = test_data, test_labels\n",
    "X_train, X_val = split_train_1(train_data, val_ratio=1/60)\n",
    "print(X_train.shape, X_val.shape)  # Expected ouput: (59000, 784) (1000, 784)\n",
    "y_train, y_val = split_train_1(train_labels, val_ratio=1/60)\n",
    "print(y_train.shape, y_val.shape)  # Expected ouput: (59000,) (1000,)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "bfYt58J-wPKp",
    "new_sheet": false,
    "outputId": "61880c5a-1ea5-4c33-e4a9-b777bc8bed8d",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(59000, 785)\n",
      "(1000, 785)\n",
      "(10000, 785)\n"
     ]
    }
   ],
   "source": [
    "# >>>>>WARNING: RUN THIS CELL ONLY ONCE!<<<<<\n",
    "\n",
    "# adding 1s to the end of feature vectors to be multiplied by bias term of weights\n",
    "X_val = np.insert(X_val, 0, 1, axis=1)\n",
    "X_train = np.insert(X_train, 0, 1, axis=1)\n",
    "X_test = np.insert(X_test, 0, 1, axis=1)\n",
    "print(X_train.shape)  # Expected ouput: (59000, 785)\n",
    "print(X_val.shape)  # Expected ouput: (1000, 785)\n",
    "print(X_test.shape)  # Expected ouput: (10000, 785)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "7XcgX7fGwPKs",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Open `svm.py` and complete the code of the linear svm model in it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "l71djD-1wPKt",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let's check your implementations first:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "4m-wj6sOwPKu",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let's first check if your model computes the hinge loss correctly. You should witness a small values as the error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "CeY33S0JwPKv",
    "new_sheet": false,
    "outputId": "e76a1607-a5bd-4b39-c401-af8b1a4f2fa2",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relative error of loss: 0.007265668557264959\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(42)\n",
    "toy_model = SVM(n_features=10, n_classes=5, std=0.01)\n",
    "x = np.random.rand(20, 10)\n",
    "y = np.random.randint(0, 5, 20)\n",
    "correct_loss = 3.999079560852596\n",
    "loss = toy_model.loss(x, y, 0.25)\n",
    "\n",
    "print('relative error of loss:', abs(loss - correct_loss))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "xLE0NwUnwPKy",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now let's check if the gradients are computed correctly. You should get a small value as the error. Ours is of the order of 1e-8."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "_6sEGhCBwPKz",
    "new_sheet": false,
    "outputId": "f510a456-12ee-4b5d-fe15-6f01697c7239",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relative error of gradient: 2.2393798129107925\n"
     ]
    }
   ],
   "source": [
    "# >>>>>WARNING: RUN THIS CELL ONLY ONCE AFTER YOU RAN THE ABOVE CELL!<<<<<\n",
    "\n",
    "gradient = toy_model.update_weights(learning_rate=1e-2)\n",
    "correct_gradient = [[-0.24790043, -0.02616584,  0.0862217,  -0.066655,    0.25564707],\n",
    "                    [-0.02056357,  0.05417903, -0.05449371, -0.07649293,  0.09846397],\n",
    "                    [-0.01443777, -0.14332005, -0.00198323, -0.23687162,  0.39444998],\n",
    "                    [-0.09852023,  0.03344655,  0.00652346, -0.15866452,  0.21542415],\n",
    "                    [-0.2999338,  -0.04307378,  0.34938565, -0.32666563,  0.3199567 ],\n",
    "                    [-0.1263978,   0.13268628,  0.15897914, -0.16465115, -0.00139483],\n",
    "                    [-0.00084713, -0.17737171,  0.05615944,  0.03299787,  0.08956249],\n",
    "                    [-0.1513251,  -0.05118482,  0.16787481, -0.12316938,  0.155753  ],\n",
    "                    [-0.13682534,  0.14630157,  0.02634592, -0.10531288,  0.06899802],\n",
    "                    [-0.13455525, -0.04130863,  0.11486015, -0.06620673,  0.12643906]]\n",
    "\n",
    "assert gradient.shape == toy_model.W.shape\n",
    "print('relative error of gradient:', np.linalg.norm(gradient - correct_gradient))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "_LIfr1FMwPK3",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now let's train the main model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "jqmyWIZRwPK3",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from svm import SVM\n",
    "model = SVM(n_features=785, n_classes=10, std=0.001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "83uVUYsPwPK6",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now let's train your model on the training set. Meanwhile, you have to witness that the printed loss drops significantly and the accuracy on validation set increases. Our loss and accuracy will reach values close to 1.0 and 79% respectively at the end of training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "n6MZVDwKwPK7",
    "new_sheet": false,
    "outputId": "7ceff48b-9af8-4e2f-9a03-280e192edbcc",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iteration 0, loss 215.857650, val acc 16.70%\n",
      "iteration 100, loss 85.267539, val acc 43.70%\n",
      "iteration 200, loss 48.210724, val acc 62.50%\n",
      "iteration 300, loss 36.090719, val acc 66.30%\n",
      "iteration 400, loss 32.734956, val acc 67.70%\n",
      "iteration 500, loss 30.950388, val acc 68.10%\n",
      "iteration 600, loss 30.391400, val acc 66.30%\n",
      "iteration 700, loss 30.021544, val acc 67.50%\n",
      "iteration 800, loss 29.332277, val acc 66.70%\n",
      "iteration 900, loss 29.406294, val acc 66.60%\n"
     ]
    }
   ],
   "source": [
    "loss_history = []\n",
    "\n",
    "num_iters, batch_size = 1000, 200\n",
    "\n",
    "for it in range(num_iters):\n",
    "    choice=np.random.choice(X_train.shape[0],batch_size)\n",
    "    X_batch, y_batch = X_train[choice], y_train[choice]\n",
    "    \n",
    "    loss = model.loss(X_batch, y_batch, 2.5e4)\n",
    "    \n",
    "    if it % 100 == 0:\n",
    "        val_preds =  model.predict(X_val)\n",
    "        val_res = val_preds == y_val\n",
    "        test_acc = np.sum(val_res) / len(val_res)\n",
    "        print('iteration %d, loss %f, val acc %.2f%%' % (it, loss, 100*test_acc))\n",
    "    \n",
    "    model.update_weights(learning_rate=1e-7)\n",
    "    loss_history.append(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "p6aG5vyBwPK_",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let's plot the training loss:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "uv4G94mkwPLB",
    "new_sheet": false,
    "outputId": "d5380e40-26c2-4598-c531-056e243154c1",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8XHW9//HXJ/vSNEuTbumSthQoLdBCKCCLiCCIXFyuorhVwFu56r3o1Z+Ket3uz8cPr1e9elUUV1TAjVVEFhHkshRIS1dKoXvTpG2apNm3ST6/P+YEpmXSTtpMTibzfj4e85gz3zkz53Ny2rxztu/X3B0REZFDZYRdgIiIjE0KCBERiUsBISIicSkgREQkLgWEiIjEpYAQEZG4khYQZjbTzB41s41mtsHMrg/av2lmL5rZWjO7y8xKgvYqM+sys9XB40fJqk1ERI7MknUfhJlNA6a5+yozKwJWAm8DZgB/c/eImX0DwN0/a2ZVwH3uvigpBYmIyLAkbQ/C3evdfVUw3QZsBCrd/SF3jwSzrSAaGCIiMsZkjcZCgr2DJcAzh7x1DfC7mNdzzOx5oBX4orv/7+G+t7y83KuqqkauUBGRNLBy5cr97l5xpPmSHhBmNgG4A/iEu7fGtH8BiAC3Bk31wCx3bzSz04G7zWxh7GeCzy0HlgPMmjWLmpqaZK+CiMi4YmY7EpkvqVcxmVk20XC41d3vjGlfBlwOvM+DkyDu3uPujcH0SmALcPyh3+nuN7t7tbtXV1QcMQBFROQoJfMqJgN+Bmx092/HtF8KfBa4wt07Y9orzCwzmJ4LzAe2Jqs+ERE5vGQeYjoH+ACwzsxWB22fB74H5AIPRzOEFe5+HXA+8DUziwD9wHXu3pTE+kRE5DCSFhDu/gRgcd66f4j57yB6OEpERMYA3UktIiJxKSBERCQuBYSIiMSVlgGx+0AX335oE9v3d4RdiojImJWWAdHS2cf3/raZF+pbjzyziEiaSsuAmF6SB0Ddga6QKxERGbvSMiCK87MpyMmk7kB32KWIiIxZaRkQZsb0knztQYiIHEZaBgQQDYgWBYSIyFDSNyCK87QHISJyGGkbENOK89nf3ktPpD/sUkRExqQ0DojolUz7WntCrkREZGxK24CYEgTE3lZdySQiEk/aBkRZQQ4AzZ19IVciIjI2pW1AlBRkA9Dc0RtyJSIiY1PaBkRZ4eAehAJCRCSetA2IgpxMJuRm6VJXEZEhpG1AmBnHT5nAi3vawi5FRGRMSlpAmNlMM3vUzDaa2QYzuz5oLzOzh83s5eC5NGg3M/uemW02s7VmdlqyahtUNamQ3dqDEBGJK5l7EBHgU+6+ADgL+JiZnQR8DnjE3ecDjwSvAd4MzA8ey4GbklgbABVFuTS09eDuyV6UiEjKSVpAuHu9u68KptuAjUAl8FbglmC2W4C3BdNvBX7lUSuAEjOblqz6IBoQPZEB2noiyVyMiEhKGpVzEGZWBSwBngGmuHs9REMEmBzMVgnsivlYbdB26HctN7MaM6tpaGg4proqinIB3U0tIhJP0gPCzCYAdwCfcPfDDeFmcdpec+zH3W9292p3r66oqDim2iomRAOioU0BISJyqKQGhJllEw2HW939zqB57+Cho+B5X9BeC8yM+fgMoC6Z9Q3uQTS0KyBERA6VzKuYDPgZsNHdvx3z1r3AsmB6GXBPTPsHg6uZzgJaBg9FJctgf0zb93ckczEiIikpK4nffQ7wAWCdma0O2j4P3Aj83syuBXYC7wreux+4DNgMdAJXJ7E2ACbmZbNg2kSe296U7EWJiKScpAWEuz9B/PMKAG+MM78DH0tWPUOZW1HIht0to71YEZExL23vpB40oySfugPdDAzoXggRkVhpHxBTi/Po7R+gpUvdfouIxEr7gJgUXOra2KErmUREYikggm6/97er228RkVhpHxDlwR7EPt0sJyJykLQPiNmTCsjKMDbtOdxN3iIi6SftAyIvO5Oq8kJe3tsedikiImNK2gcEwPSSfOpbusMuQ0RkTFFAANOL86hv0cBBIiKxFBDAtOJ89rf30hPpD7sUEZExQwEBTCuJdtq3R4eZREReoYAAphfnA1B3QAEhIjJIAcGrexA6DyEi8ioFBK/uQehKJhGRVykggPycTCpL8lmxtTHsUkRExgwFROCiBZN5fucBosNSiIiIAiIwe1Ih7T0RmjvV7beICCR3TOqfm9k+M1sf0/Y7M1sdPLYPDkVqZlVm1hXz3o+SVddQZpUVALCjUeNTi4hAcsek/iXwfeBXgw3u/u7BaTP7FhA71ucWd1+cxHoOa/akaEDsbOpkyazSsMoQERkzkrYH4e6PA03x3jMzA64Ebk/W8odrZlkBmRnGC3Xq1VVEBMI7B3EesNfdX45pm2Nmz5vZ383svKE+aGbLzazGzGoaGhpGrKC87EzOqCrlaV3JJCIChBcQV3Hw3kM9MMvdlwD/BtxmZhPjfdDdb3b3anevrqioGNGiZpUVsLdV90KIiEAIAWFmWcA7gN8Ntrl7j7s3BtMrgS3A8aNd25SJeTS09dA/oEtdRUTC2IO4CHjR3WsHG8yswswyg+m5wHxg62gXVlmSz4BHT1SLiKS7ZF7mejvwNHCCmdWa2bXBW+/htSenzwfWmtka4I/Ade4e9wR3MlVXlQHw3PZRX7SIyJiTtMtc3f2qIdo/FKftDuCOZNWSqJll0T6Z1O23iIjupD5IblYmJQXZ7GtTQIiIKCAOMXVinvYgRERQQLzG8VOKdLOciAgKiNc4ubKYupZu9rf3hF2KiEioFBCHWFRZDMC63S1HmFNEZHxTQBzipGnRG7g31uswk4ikNwXEIYoLsqksyWdjfVvYpYiIhEoBEceCaRO1ByEiaU8BEceiyolsaWinvScSdikiIqFRQMSxdE4Z7vCnNXVhlyIiEhoFRBxnz51E+YRcarY3h12KiEhoFBBxmBnHTS5k2/72sEsREQmNAmIIc8onsG1/R9hliIiERgExhHkVhTR39tHc0Rt2KSIioVBADGFOeSEA2xq1FyEi6UkBMYTBgNjaoIAQkfSkgBjCzLICMjNMJ6pFJG0pIIaQnZnBrLICtuzTHoSIpKdkjkn9czPbZ2brY9q+Yma7zWx18Lgs5r0bzGyzmW0ys0uSVddwLJlZwtNbG4n0D4RdiojIqEvmHsQvgUvjtH/H3RcHj/sBzOwk4D3AwuAzPzSzzCTWlpDzj6+gpauPLToPISJpKGkB4e6PA00Jzv5W4Lfu3uPu24DNwNJk1ZaoRZXRrr/Xa2wIEUlDRwwIM/tPM5toZtlm9oiZ7Tez9x/DMj9uZmuDQ1ClQVslsCtmntqgLV49y82sxsxqGhoajqGMI5tTPoH87EwNHiQiaSmRPYg3uXsrcDnRX9zHA//nKJd3EzAPWAzUA98K2i3OvB7vC9z9ZnevdvfqioqKoywjMZkZxikzinlue6I7QiIi40ciAZEdPF8G3O7uR/3b0t33unu/uw8AP+HVw0i1wMyYWWcAY6Ir1bPnTeKF+lZau/vCLkVEZFQlEhB/MrMXgWrgETOrALqPZmFmNi3m5duBwSuc7gXeY2a5ZjYHmA88ezTLGGmnzSrFHdbu0mEmEUkvWUeawd0/Z2bfAFrdvd/MOoieVD4sM7sduAAoN7Na4MvABWa2mOjho+3AR4JlbDCz3wMvABHgY+7ef3SrNLIWzyrBDFbtbObc+eVhlyMiMmqOGBBm9i7ggSAcvgicBvxfYM/hPufuV8Vp/tlh5v868PUj1TPaJuZlc1zFBFbvOhB2KSIioyqRQ0z/7u5tZnYucAlwC9GTzWnjlBklbKjTISYRSS+JBMTgoZ63ADe5+z1ATvJKGnvmlBewt7WH7r4xcdRLRGRUJBIQu83sx8CVwP1mlpvg58aN2ZOiPbvqhjkRSSeJ/KK/EngQuNTdDwBlHP19ECnpghMqyM40HnlxX9iliIiMmiMGhLt3AluAS8zs48Bkd38o6ZWNIUV52cwpL+TlvW1hlyIiMmoS6WrjeuBWYHLw+I2Z/UuyCxtrFlUWs2rnAQYG4t7gLSIy7iRyiOla4Ex3/5K7fwk4C/in5JY19pwzr5ymjl42aS9CRNJEIgFhvHolE8F0vL6TxrXqqmi/gqt2NodciYjI6EgkIH4BPBMM9vMVYAWHueFtvJpVVkD5hBxWbldAiEh6SKSrjW+b2WPAuUT3HK529+eTXdhYY2YsnVPG01sbcXfM0m4nSkTSzJABYWZlMS+3B49X3juWXl1T1RlVZdy/bg8NbT1MnpgXdjkiIkl1uD2IlUQ71Rv8U3nw8h0Lpucmsa4xaeH0YgA21LUqIERk3BsyINx9zmgWkgpOnFYEwAv1rbzhxMkhVyMiklxp1WXGsZqYl82ssgJeqGsNuxQRkaRTQAzTwukT+fO6ehrbe8IuRUQkqRQQw3TJwqkAPLmlMeRKRESSK5GuNsriPLKP9Lnx6i2nTCMnM0M9u4rIuJfIHsQqoAF4CXg5mN5mZqvM7PShPmRmPzezfWa2Pqbtm2b2opmtNbO7zKwkaK8ysy4zWx08fnRsq5U82ZkZnDitiHW1CggRGd8SCYgHgMvcvdzdJwFvBn4PfBT44WE+90vg0kPaHgYWufspRAPnhpj3trj74uBxXaIrEIZFlcWsr2vBXR33icj4lUhAVLv7g4Mvgq6+z3f3FUDuUB9y98eBpkPaHnL3SPByBTBj+CWHb/GMEtq6Izyl8xAiMo4lEhBNZvZZM5sdPD4DNJtZJjBwDMu+BvhLzOs5Zva8mf3dzM4b6kNmttzMasyspqGh4RgWf/SuWDyd8gk53PbszlCWLyIyGhIJiPcS/Uv/buAeYFbQlkl0tLlhM7MvABGi40wA1AOz3H0J8G/AbWY2Md5n3f1md6929+qKioqjWfwxy8vO5My5k3QeQkTGtUQ669sPDDVA0ObhLtDMlgGXA2/04CC+u/cAPcH0SjPbAhwP1Az3+0fL8ZOLuH9dPV29/eTnZIZdjojIiDtiQJjZ8cCngarY+d39wuEuzMwuBT4LvD4YynSwvQJocvd+M5sLzAe2Dvf7R9P8KRNwh5odTZw3P5w9GRGRZDpiQAB/AH4E/JSDBw46LDO7HbgAKDezWuDLRK9aygUeDrrLXhFcsXQ+8DUziwTLuG6s9xa7cHr0CNgPHt2sgBCRcSmRgIi4+03D/WJ3vypOc9yBhtz9DuCO4S4jTLMnFbJ0Thmb9rRpfAgRGZcSOUn9JzP7qJlNi72bOumVpYC3nDyN5s4+9rWpXyYRGX8S2YNYFjz/n5i2tBwP4lAnTI12/722toWLT9L4ECIyviRyFZPGhRjCabNKKcrN4r//+hIXnzQl7HJEREbU4YYcvdDd/2Zm74j3vrvfmbyyUkNOVgbVVaU8uqmBtbUHOGVGSdgliYiMmMOdg3h98PwPcR6XJ7mulPGFt5wEwJpdB0KuRERkZB1uyNEvB89Xj145qWdeRSFlhTmsU/ffIjLOJHKjXC7wj7z2RrmvJa+s1GFmnDqjmKe3NhLpHyArU2Mwicj4kMhvs3uAtxLtO6kj5iGBty2pZFdTF/euqQu7FBGREZPIZa4z3P3QcR0kxuWnTOdTv1/DS3vbwy5FRGTEJLIH8ZSZnZz0SlJYZoYxozSfp7fs1yBCIjJuJBIQ5wIrzWxTMFToOjNbm+zCUs2VZ8xkTW0LK7aO6S6kREQSlsghpjcnvYpx4ANnzeabD27iue1NnD1vUtjliIgcsyH3IGIG7Gkb4iExivKyWTh9In9aU8fAgA4ziUjqO9whptuC55VEB+5ZGfMYswP5hOnD587l5X3trNiqsapFJPUd7ka5y4Nn9cWUoDcumEyGwYqtjbzuuPKwyxEROSaJnIPAzEqJjvL2Spel7v54sopKVdHDTMU8taWRfwu7GBGRY3TEq5jM7MPA48CDwFeD568kt6zUdcnCKdTsaOalvTpNIyKpLZHLXK8HzgB2uPsbgCVAQyJfbmY/N7N9ZrY+pq3MzB42s5eD59Kg3czse2a2Obic9rSjWJ/Qve/M2WRnGn9cWRt2KSIixySRgOh2926I9svk7i8CJyT4/b8EDr0L+3PAI+4+H3gkeA3Ry2nnB4/lwLCHOR0LSgtzOGvuJP66cW/YpYiIHJNEAqLWzEqAu4GHzeweIKFOh4LzFIfeOfZW4JZg+hbgbTHtv/KoFUCJmU1LZDljzcUnTWFrQwd/e1EhISKp64gB4e5vd/cD7v4V4N+Bn/HqL/WjMcXd64PvrgcmB+2VwK6Y+WqDtpTzppOmAnDLUztCrkRE5OgdNiDMLCP2/IG7/93d73X33iTUYnHaXnPHmZktN7MaM6tpaEjoVMiom1qcxxWnTmdDXav6ZhKRlHXYgHD3AWCNmc0awWXuHTx0FDzvC9prgZkx880gzqEsd7/Z3avdvbqiomIEyxpZ580vZ397D6t2aqQ5EUlNiZyDmAZsMLNHzOzewccxLPNeYFkwvYzoeBOD7R8MrmY6C2gZPBSVii5dNJW87AzuXKWrmUQkNSVyo9xXj/bLzex24AKg3MxqgS8DNwK/N7NrgZ3Au4LZ7wcuAzYDnUBKD3ValJfNJQun8qc1dXzhLQsoyEnonkQRkTEjkd9al7n7Z2MbzOwbwN+P9EF3v2qIt94YZ14HPpZAPSnjqqWzuGd1HT/932386xvnh12OiMiwJHKI6eI4beoCPAFnzZ3EqTNL+PbDL1Hf0hV2OSIiw3K47r7/2czWAScEdzYPPrYBGjAoQYumR3tN/5+/bQ65EhGR4TncIabbgL8A/49X73YGaHN3DZuWoM9ceiK3PrOTfa09YZciIjIsh+vuuwVoAYY6jyAJKM7P5sPnzuFnT25jV1MnM8sKwi5JRCQhiZyDkGN0zblzMODXK3RntYikDgXEKJheks9bTpnOb1bsoKMnEnY5IiIJUUCMkmVnz6azt58/r0vZe/9EJM0oIEbJ6bNLmVteyJfuWU9rd1/Y5YiIHJECYpSYGVefU0V33wBrdql/JhEZ+xQQo+jyU6YDsHJHc8iViIgcmQJiFJUW5nDe/HJ++NgW1tW2hF2OiMhhKSBG2XffswQc7nxevbyKyNimgBhlZYU5XHTSZG57Zid7WrrDLkdEZEgKiBDc8OYFDLjzg0fVP5OIjF0KiBDMLCvgyuqZ/Pa5nexq6gy7HBGRuBQQIfn4hcfhDrc9uzPsUkRE4lJAhGRacT6LZ5Zw9/O7deOciIxJCogQffqSE9jb2s23H3op7FJERF5j1APCzE4ws9Uxj1Yz+4SZfcXMdse0XzbatY22s+ZO4opTp3PHqlpqm3UuQkTGllEPCHff5O6L3X0xcDrQCdwVvP2dwffc/f7Rri0MH3n9PNzhGw9sCrsUEZGDhH2I6Y3AFndP24ESFkybyIUnTubpLY30RgbCLkdE5BVhB8R7gNtjXn88GPf652ZWGu8DZrbczGrMrKahoWF0qkyyd5xWyf72Hq74/hO4e9jliIgAIQaEmeUAVwB/CJpuAuYBi4F64FvxPufuN7t7tbtXV1RUjEqtyfb64yt43bxJvLinjd89tyvsckREgHD3IN4MrHL3vQDuvtfd+919APgJsDTE2kaVmXHLNUs5dUYx/3HfCzS294RdkohIqAFxFTGHl8xsWsx7bwfWj3pFIcrOzOBbV55KV18/X7vvBR1qEpHQhRIQZlYAXAzcGdP8n2a2zszWAm8APhlGbWE6bnIRH73gOO5ZXcd1v1kZdjkikuZCCQh373T3Se7eEtP2AXc/2d1Pcfcr3D0tB2++/qL5ADy4YS9dvf0hVyMi6Szsq5jkENmZGXz1ioUA/GGlTliLSHgUEGPQB86azZlzyvjSPRt4+IW9YZcjImlKATEGZWQYN3+wmsqSfG78y0b6+nUDnYiMPgXEGFWcn82X/uEktjR08PU/bwy7HBFJQwqIMeyShVN51+kz+PWKHfx6Rdr2RiIiIVFAjHH//g8nMXViHv9+93qe2rw/7HJEJI0oIMa4iXnZ/P66swF470+f0RClIjJqFBApoLIkn59/qBqAz96xlohOWovIKFBApIgLT5zC19++iKe2NPLx254PuxwRSQMKiBTyvjNn86HXVfHAhj08t70p7HJEZJxTQKSY614/j/IJubzvp89w1/O1YZcjIuOYAiLFTC3O46b3n0ZvZIBP/m4Nj2zUndYikhwKiBR0RlUZj336AgA+8uuV1Ohwk4gkgQIiRVWVF/Lra5cSGXDe+5Nn2NfaHXZJIjLOKCBS2HnzK/j1tUvp7R9g2S+eo70nEnZJIjKOKCBS3HnzK7jxHSezaU8r59z4N3Y26kY6ERkZCohx4D1LZ/HVKxbS0tXHsl88y54WHW4SkWMXWkCY2fZgiNHVZlYTtJWZ2cNm9nLwXBpWfanmA2dX8curz6C2uZPL/+d/2b6/I+ySRCTFhb0H8QZ3X+zu1cHrzwGPuPt84JHgtSToghMm8/MPncH+9l6u/PHT3L8uLUdtFZEREnZAHOqtwC3B9C3A20KsJSWdN7+CX12zlK7efj566yq+cNc6Wrv7wi5LRFJQmAHhwENmttLMlgdtU9y9HiB4nhxadSns/OMruPvj55Cfncmtz+zk0u88rl5gRWTYwgyIc9z9NODNwMfM7PxEPmRmy82sxsxqGhoaklthCptXMYHHP/MGTq4spq6lm/P+81FuemxL2GWJSAoJLSDcvS543gfcBSwF9prZNIDgeV+cz93s7tXuXl1RUTGaJaeciqJc/vQv5/LFtywA4BsPvMi7f/w0TR29IVcmIqkglIAws0IzKxqcBt4ErAfuBZYFsy0D7gmjvvHmw+fN5cnPXciFJ07mmW1NnPYfD/Poi6/JXhGRg4S1BzEFeMLM1gDPAn929weAG4GLzexl4OLgtYyAypJ8fvLBak6dUQzA1b98jpO/8iCb97WFXJmIjFXm7mHXcNSqq6u9pqYm7DJSTs32Jt75o6dfef3wJ8/nuMkTMLMQqxKR0WJmK2NuLxjSWLvMVUZBddAb7DuWVAJw8XceZ84N9/Pdv75MY3tPyNWJyFihPYg0t3rXAd72gycPavvaWxdyZfVM8rIzQ6pKRJIp0T0IBYQA0Nrdx0d/s4onNu9/pe0dSypZ/vq5nDh1YoiVichIU0DIUdna0M4z25r4waObqW3uAuCqpTO55pw5Ok8hMk4oIOSYdPZGeOLl/XzqD2to6351nInrXj+Py0+ZxqLK4hCrE5FjoYCQEbNyRxM3P76VBzccPP71u6tncvW5VToEJZJiFBAy4mqbO9m0p43rf7v6oNHrZpTm847TZnDpwqmUT8hh8sS8EKsUkSNRQEhSdff1U9/SzS+e3MYfV9bS2dv/ynvvO3MW00vyuWrpLApyMsnNytC5C5ExRAEho+rPa+v54t3raO58bdfix0+ZwGUnT2PR9GIuOmlKCNWJSCwFhIRmZ2Mnf1lfz9NbG3ls08E97s6tKKRqUiGFuVnsa+3mvWfOYuH0YiZPzKUoN0t7GiKjQAEhY4K7s6+thy372nnspQae3LyfxvZe9rS+dtzsqkkFHOjqY17FBJo7e1l2dhXzJ0/gpOkT2dPazcS8bCbkZTExL5u+/gEG3MnN0s18IsOlgJAxy93pH3DW7W5he2MHn/7DWipL8snKMLYOcyzta86ZQ2FuJmWFORTmZOE4FUW5LJlZSmlhTpLWQCS1KSAkJe1t7aa1q497VtdRWZpPZ28/63e3kJlhrK09wEt72xP+rqLcLMqLctkWhM47llRy2uxSZk8qoLUrwuxJBRTmZjG5KJfC3KxkrZLImKOAkHGrvqWLSL8zMS+bxo4eVu5opigvi+d3HeCFulZmlBbQ1NHDtv0d7GnppjXmRr+hZGYY/QNOhsE5x5VTmJPFmxZOYUNdK529/dz+7E5OnFrEldUzyTCYkJfNnPIC+gdg3e4WsjON02aVUpyfzaY9bZwwtYgZpfm4Q0bGq+dVIv0D3PX8bo6fUsQpM4oZcPj7S/uoripjYl72a+rqjQzQ1dtPccFr3xM5WgoIkcDAgPNCfSsb6lrY3thJSX42Aw5rdh2gvrWbNbsOJHX5hTmZdPT2U1mSz+4DXUecP958bzl5GlOL88jMMLY2tNM/4CyqLGbyxDx+99xO1u9u5XXzJjG9JJ+nNu/n3WfMYmpxLo+/vJ+6A10smDaRypJ8+gec9p4I8ydPoLd/gE172mjrjlA+IYf1u1vZ19ZNWWEORXnZ7G7u4p2nz6Clq4/JE3Np7uhjanEuAw79A06kf4Cpxfm84cQK6g5009TRw6kzSsjKfLWTaHcnMuBkZ2bg7roIIUHxflaDbe7Ogc6+YzqEqoAQGQZ3Z9XOZg509vG6eeXUtXSxfncLZkZHcFNge3eERZXFdPZG6Ojt56U9bVQU5dLd10/NjmYuWjCZugPdbKxvxQy2NnTQ1h3h5BnF1B3ooiAnk6K8bDLMaOvuY3tjB29cMIUDnb1UTSrk7ud3U9fSzZSJuextfW2367lZGfREBkbl51GQk3nQvS2Jmj95ApEBZ2dTJ/0D/ko4AkwqzGH2pAI6e/vpjQxQXpTL7uYuTplRzPbGTqYV5/HUlv2cXFlMb2SAqcV5bNrTxpJZpexv7+HEqUW0dUeoripj1c5mdjV1Uj27jMaOHuaWFzK9JJ9JE3J4aW87Oxo7Wb2rmdrmLi5aMIWlc8po6uhlQ10LhTlZnDqzhDtW1XL6rOi5qtKCHLY3dtATGaCzJ0JBbha5WRlkZRh7Wrt5aMNe3rRwCgunFzMhN5Pmzj7W7W7hlMpiVu86wMb6Vt5/1mye3tJIT2SA3v6BVw5tvm3xdDIyjPIJuTS09RDpd86YU0pOZgYPvbCXR1/cxznHlbOrqZPndx1gycwSntvexCkzSujrH+CZbU1xf9ZfuGwB/3T+3KPavgoIkTHM3V9z+ClWS1cf7k5xfjYtXX2UFOTQ2RthR/CL1MzIyczg+V3NzJ5USKYZ9S1dzC2fwLbGDnY3d9ET6ae0MIfjKibw4p42djR2UF1VRndfPz2RAV7e28bJlcVEBpyOngj5OZnkZmWydE50ns7eflbuaCYr06iYkMtxqTVhAAAJ90lEQVT63S3MLCugpCCbtu4IBzr7aGjrZmdTJ89ua2JNbQuLZ5aQlWFMKc6jp6+frQ0dB114kGEwkLq/cpIqJyuD3gT+AJhWnEd9SzfLz5/L5y9bcFTLUkCIyJgwMOCvBKG70xWET0tXH9kZGRTkZrJ9fwenzy6lrSdCe3eEwuAv+PqWborysigtyGHb/nb2tfbQ2z9AW3eEytLolW99/QPsbe1hYl42TZ29TC7KZVFlMRvrW5k6MY/2ngg7Gjvo63fmlBfS2dvP9v0dnD1vEi/uaaOzN8KKrU1UluSRn5NFb2SAjp4IU4vz6IkMUJSXRWN7L6UF2TR39hHpHyAjwyjIyaRqUiGnzSrlic37mVqcR1NHLzPL8plclEdzZy9ZGUZJfg6dfREa2nq4Y2UtmxvamT+5iOqqUs6cM4mdTR20dkU4//gK+voHqDvQRXlRLoU5WRzo7KU4P7rXmRGcJ8uM+Vke7SG7MRsQZjYT+BUwFRgAbnb375rZV4B/AgbvrPq8u99/uO9SQIiIDF+iARHGtX0R4FPuvsrMioCVZvZw8N533P2/QqhJREQOMeoB4e71QH0w3WZmG4HK0a5DREQOL+PIsySPmVUBS4BngqaPm9laM/u5mZUO8ZnlZlZjZjUNDQ3xZhERkREQWkCY2QTgDuAT7t4K3ATMAxYT3cP4VrzPufvN7l7t7tUVFRWjVq+ISLoJJSDMLJtoONzq7ncCuPted+939wHgJ8DSMGoTEZGoUQ8Ii16X9TNgo7t/O6Z9WsxsbwfWj3ZtIiLyqjCuYjoH+ACwzsxWB22fB64ys8WAA9uBj4RQm4iIBMK4iukJIN7dHYe950FEREZXSt9JbWYNwI5j+IpyYP8IlZMK0m19QeucLrTOwzPb3Y94lU9KB8SxMrOaRO4mHC/SbX1B65wutM7JEep9ECIiMnYpIEREJK50D4ibwy5glKXb+oLWOV1onZMgrc9BiIjI0NJ9D0JERIaQlgFhZpea2SYz22xmnwu7npFiZjPN7FEz22hmG8zs+qC9zMweNrOXg+fSoN3M7HvBz2GtmZ0W7hocHTPLNLPnzey+4PUcM3smWN/fmVlO0J4bvN4cvF8VZt3HwsxKzOyPZvZisL3PToPt/Mng3/V6M7vdzPLG27YOOirdZ2brY9qGvV3NbFkw/8tmtuxo60m7gDCzTOAHwJuBk4jewX1SuFWNmMGxNhYAZwEfC9btc8Aj7j4feCR4DdGfwfzgsZxoh4mp6HpgY8zrbxAdW2Q+0AxcG7RfCzS7+3HAd4L5UtV3gQfc/UTgVKLrP263s5lVAv8KVLv7IiATeA/jb1v/Erj0kLZhbVczKwO+DJxJtE+7Lw/VO/YRRcfGTZ8HcDbwYMzrG4Abwq4rSet6D3AxsAmYFrRNAzYF0z8GroqZ/5X5UuUBzAj+01wI3Ef0Lv39QNah2xt4EDg7mM4K5rOw1+Eo1nkisO3Q2sf5dq4EdgFlwba7D7hkPG5roApYf7TbFbgK+HFM+0HzDeeRdnsQvPoPbVAt43DAokPG2pji0YGaCJ4nB7ONh5/FfwOfITp8LcAk4IC7R4LXsev0yvoG77cE86eauUSH5v1FcGjtp2ZWyDjezu6+G/gvYCfR4QBagJWM/20Nw9+uI7a90zEg4vUDNa4u5Yoz1saQs8ZpS5mfhZldDuxz95WxzXFm9QTeSyVZwGnATe6+BOjg1cMO8aT8egeHSN4KzAGmA4VED7Ecarxt68MZah1HbN3TMSBqgZkxr2cAdSHVMuLijbUB7B3sTj143he0p/rP4hzgCjPbDvyW6GGm/wZKzGywI8rYdXplfYP3i4Gm0Sx4hNQCte4+OBLjH4kGxnjdzgAXAdvcvcHd+4A7gdcx/rc1DH+7jtj2TseAeA6YH1z9kEP0RNe9Idc0Iszij7VBdP0Gr2RYRvTcxGD7B4OrIc4CWgZ3ZVOBu9/g7jPcvYrodvybu78PeBR4ZzDboes7+HN4ZzB/yv1V6e57gF1mdkLQ9EbgBcbpdg7sBM4ys4Lg3/ngOo/rbR0Y7nZ9EHiTmZUGe15vCtqGL+wTMiGdBLoMeAnYAnwh7HpGcL3OJboruRZYHTwuI3rs9RHg5eC5LJjfiF7RtQVYR/QKkdDX4yjX/QLgvmB6LvAssBn4A5AbtOcFrzcH788Nu+5jWN/FQE2wre8GSsf7dga+CrxIdDCxXwO5421bA7cTPcfSR3RP4Nqj2a7ANcG6bwauPtp6dCe1iIjElY6HmEREJAEKCBERiUsBISIicSkgREQkLgWEiIjEpYCQlGRmTwXPVWb23hH+7s/HW9ZYZWYfMrPvh12HjD8KCElJ7v66YLIKGFZABD36Hs5BARGzrHEpgZ+HpCkFhKQkM2sPJm8EzjOz1cF4AZlm9k0zey7oI/8jwfwXWHSsjNuI3lSEmd1tZiuDMQaWB203AvnB990au6zgjtVvBuMRrDOzd8d892P26vgMtwZ3+x5a82Nm9g0ze9bMXjKz84L2g/YAzOw+M7tgcNnBZ1aa2V/NbGnwPVvN7IqYr59pZg9YdJyTL8d81/uD5a02sx8PhkHwvV8zs2eI9oIq8lph3zmohx5H8wDag+cLCO6gDl4vB74YTOcSvdt4TjBfBzAnZt7BO1Lzid6dOyn2u+Ms6x+Bh4mORTCFaPcP04LvbiHa500G8DRwbpyaHwO+FUxfBvw1mP4Q8P2Y+e4DLgimHXhzMH0X8BCQTXQMiNUxn68nesft4LpUAwuAPwHZwXw/BD4Y871Xhr0d9Rjbj8FOrkTGizcBp5jZYP88xUQHVOkFnnX3bTHz/quZvT2YnhnM13iY7z4XuN3d+4l2oPZ34AygNfjuWgAzW0300NcTcb5jsAPFlcE8R9ILPBBMrwN63L3PzNYd8vmH3b0xWP6dQa0R4HTguWCHJp9XO3rrJ9qpo8iQFBAy3hjwL+5+UOdkwSGbjkNeX0R0UJlOM3uMaP89R/ruofTETPcz9P+tnjjzRDj4cG9sHX3uPtgfzsDg5919IKYXU3htd86D3T7f4u43xKmjOwg6kSHpHISkujagKOb1g8A/B92eY2bHB4PpHKqY6JCUnWZ2ItEhWgf1DX7+EI8D7w7Oc1QA5xPtCO5YbQcWm1mGmc0kOkzkcF1s0bGL84G3AU8S7djtnWY2GV4Z23j2CNQraUJ7EJLq1gIRM1tDdDzf7xI99LIqOFHcQPQX5qEeAK4zs7VEh2pcEfPezcBaM1vl0e7DB91F9ITuGqJ/oX/G3fcEAXMsniQ6hOg6oucPVh3FdzxBtIfT44Db3L0GwMy+CDxkZhlEewj9GLDjGOuVNKHeXEVEJC4dYhIRkbgUECIiEpcCQkRE4lJAiIhIXAoIERGJSwEhIiJxKSBERCQuBYSIiMT1/wHwNTZfjohTogAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(loss_history)\n",
    "plt.ylabel('training loss')\n",
    "plt.xlabel('iteration number')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "colab_type": "text",
    "id": "M5JHENEvwPLF",
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Finally let's test your model on the test set. We get an accuracy above 75%."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "button": false,
    "colab": {},
    "colab_type": "code",
    "id": "DdjR3u32wPLG",
    "new_sheet": false,
    "outputId": "274f1d36-a5f6-4480-f892-75bf55e2536d",
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy on test set: 67.25%\n"
     ]
    }
   ],
   "source": [
    "predictions =  model.predict(X_test)\n",
    "test_res = predictions == y_test\n",
    "test_acc = np.sum(test_res) / len(test_res)\n",
    "print(\"accuracy on test set: %.2f%%\" % (100*test_acc))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6FPYpOVAwPLK"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "name": "HW1.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}