Deep_Learning_CourseWorkProject2021.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gBNO-ofgnzxy"
   },
   "source": [
    "# CS5950/CS4950 Course Project\n",
    "\n",
    "This project recaps and systematises work that has been done in the exercise notebooks: if you have worked through the notebooks, little extra work is required.  The aim of the project is to give you the experience of trying to develop a neural network classifier (or regressor), involving setting up a model, optimising the amount of regularisation, investigating its performance, and devising a new model. \n",
    "\n",
    "You should do your work in this notebook, filling in the sections below. To do the work, you may re-use code from ***any of the lab-session sheets provided so far***. (In fact, you should be able to do nearly the entire project using code taken from previous lab sessions.) \n",
    "\n",
    "Please complete this workbook and submit it on Moodle, with all outputs (numbers and graphs) visible and included. \n",
    "\n",
    "The deadline for this assignment is **Wednesday April 7th, 10am**\n",
    "\n",
    "This project has **10%** of the marks of the course.\n",
    "\n",
    "Please write your student number **here**: 100943729    \n",
    "so that I have an identifier of which worksheet I am marking, to prevent any confusion! (I have to download your notebooks before marking them, so I don't want to accidentally give credit for your notebook to somebody else...) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AFElJ_Dznzx-"
   },
   "source": [
    "## Set up dataset\n",
    "\n",
    "You may choose any of the datasets included with Pytorch, or used in the course so far (including, for example, the percolation data). (It is a free country and you may use any other data you wish.)\n",
    "\n",
    "A safe choice would be either the MNIST data or the MNIST-fashion data, which is a drop-in replacement for MNIST (same size data format, same number of classes, same number of training and test examples). \n",
    "\n",
    "A more interesting choice would be CIFAR-10\n",
    "\n",
    "In setting up the data, you should set up a training set and a test set. The test set should be large enough to give a reasonably accurate assessment of the error-rate (or loss) of your models: preferably at least 10,000 examples.\n",
    "\n",
    "For the learning curve experiment (below), you will need to construct training sets of different sizes, with the largest at least 10 times the smallest. For the MNIST data, for example, your smallest training set might be 500, with sizes 500, 1000, 2000, 4000, 8000, 16000, 32000, and perhaps 60000 if you have time. (You get no extra marks for doing very long experiments.) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Importing the Dataset and Loading Libraries**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "DWR6Vr0pXgZ5"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.cm as cm  # colormaps\n",
    "\n",
    "%matplotlib inline\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F  # this includes tensor functions that we can use in backwards pass\n",
    "import torchvision\n",
    "import torchvision.datasets as datasets\n",
    "from torchvision import transforms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Q6VB2wqiX2Ww",
    "outputId": "5104df9d-5b0f-489f-ba25-d044f2987d94"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mounted at /content/drive\n"
     ]
    }
   ],
   "source": [
    "from google.colab import drive\n",
    "drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CXRmfpt3eKK9",
    "outputId": "f240ecb4-1557-4346-e7cd-fb172bd94df0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/content/drive/MyDrive\n"
     ]
    }
   ],
   "source": [
    "cd /content/drive/MyDrive/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "6kWzyo4_X8hE"
   },
   "outputs": [],
   "source": [
    "transform_list = transforms.Compose([ transforms.ToTensor(), transforms.Normalize(mean=[0.0], std=[1.0,]) ] )\n",
    "mnist_trainset = datasets.MNIST(root='./datas', train=True, download=False, transform=transform_list)\n",
    "mnist_testset = datasets.MNIST(root='./data', train=False, download=False, transform=transform_list)\n",
    "mnist_trainset_small =  [ mnist_trainset[i] for i in range(0,4000) ] \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mo9dA5q1nzx_"
   },
   "source": [
    "## Set up model \n",
    "\n",
    "Set up a Pytorch model (you may find it helpful to keep it simple and fast to train). Train on an intermediate-sized dataset (if you are using MNIST, say 2000 or 4000). \n",
    "\n",
    "\n",
    "Plot the loss and error rate as a function of training epochs. \n",
    "\n",
    "\n",
    "###  1.  Assessment of initial model:    ***12 marks***\n",
    "\n",
    "Ensure that your model is complex enough to *overfit* the training data: that is the loss/error-rate on the training set should be below your target loss/error-rate, and the loss/error-rate on the validation set should be higher than this (preferably higher than your target error-rate). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "ojhZABRjqB6K"
   },
   "outputs": [],
   "source": [
    "# Multi Layer Perceptron\n",
    "class NN1( nn.Module ):\n",
    "    \n",
    "    def __init__(self, dropout = 0): \n",
    "        super(NN1, self).__init__()\n",
    "        self.dropout = dropout\n",
    "        self.layers = nn.Sequential( \n",
    "            nn.Flatten(), \n",
    "            nn.Linear(784,256),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(self.dropout),\n",
    "            nn.Linear(256,128),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(self.dropout),\n",
    "            nn.Linear(128,10))\n",
    "    \n",
    "    \n",
    "    def forward( self, x ):\n",
    "        x = self.layers( x )\n",
    "        return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "1bNYNGv_a30j"
   },
   "outputs": [],
   "source": [
    "loss_function = nn.CrossEntropyLoss()\n",
    "import torch.optim as optim\n",
    "trainloader = torch.utils.data.DataLoader( mnist_trainset, batch_size=64, shuffle=True)\n",
    "testloader = torch.utils.data.DataLoader( mnist_testset, batch_size=64, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "id": "CP4u2FjynzyA"
   },
   "outputs": [],
   "source": [
    "# your code here (you can freely used and modify code from course lab-sheets throughout this project)\n",
    "def my_model(trainloader,epo,learn = 0.001,dropout = 0):\n",
    "  global nn1\n",
    "  nn1 = NN1(dropout)\n",
    "  device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "  print(device)\n",
    "  nn1.to(device)\n",
    "  optimizer1 = optim.Adam(nn1.parameters(), lr=learn)\n",
    "\n",
    "  eploss = np.zeros(epo)\n",
    "  eperr = np.zeros(epo)\n",
    "  for epoch in range(epo): # number of times to loop over the dataset\n",
    "      current_loss = 0.0 \n",
    "      n_mini_batches = 0\n",
    "      incorrect = 0\n",
    "      total = 0\n",
    "      for i, mini_batch in enumerate( trainloader, 0 ):\n",
    "          images, labels = mini_batch\n",
    "          \n",
    "          # zero the parameter gradients\n",
    "          optimizer1.zero_grad()\n",
    "          \n",
    "          outputs = nn1(images.cuda())\n",
    "          loss = loss_function ( outputs, labels.cuda() )\n",
    "          loss.backward() # does the backward pass and computes all gradients\n",
    "          optimizer1.step() # does one optimisation step\n",
    "          \n",
    "          n_mini_batches += 1 \n",
    "          current_loss += loss.item() \n",
    "          outputs = nn1(images.cuda())\n",
    "          _, predicted = torch.max( outputs, dim=1)\n",
    "          total += labels.size(0) # the number of labels, which is just the size of the batch \n",
    "          incorrect += (predicted != labels.cuda()).sum().item() \n",
    "      eperr[epoch] = 100*incorrect/total \n",
    "      eploss[epoch]= current_loss / n_mini_batches                                                   \n",
    "      print( \"Error rate on train set: %f %%\" %(100 * incorrect/total))\n",
    "      print('Epoch %d loss: %f '  %(epoch+1, current_loss / n_mini_batches ))\n",
    "\n",
    "  return eperr, eploss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "yBb7P43_dIqa",
    "outputId": "6038b807-77a0-4ca2-fe8f-f11d4969f4bd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda\n",
      "Error rate on train set: 26.525000 %\n",
      "Epoch 1 loss: 1.072640 \n",
      "Error rate on train set: 9.425000 %\n",
      "Epoch 2 loss: 0.343068 \n",
      "Error rate on train set: 6.975000 %\n",
      "Epoch 3 loss: 0.260708 \n",
      "Error rate on train set: 5.375000 %\n",
      "Epoch 4 loss: 0.205904 \n",
      "Error rate on train set: 4.125000 %\n",
      "Epoch 5 loss: 0.164876 \n",
      "Error rate on train set: 2.825000 %\n",
      "Epoch 6 loss: 0.125601 \n",
      "Error rate on train set: 1.875000 %\n",
      "Epoch 7 loss: 0.097747 \n",
      "Error rate on train set: 1.500000 %\n",
      "Epoch 8 loss: 0.076496 \n",
      "Error rate on train set: 0.925000 %\n",
      "Epoch 9 loss: 0.060652 \n",
      "Error rate on train set: 0.650000 %\n",
      "Epoch 10 loss: 0.043765 \n",
      "Error rate on train set: 0.375000 %\n",
      "Epoch 11 loss: 0.030596 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 12 loss: 0.027555 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 13 loss: 0.020955 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 14 loss: 0.015339 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 15 loss: 0.010898 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 16 loss: 0.007781 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 17 loss: 0.005396 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 18 loss: 0.004587 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 19 loss: 0.003551 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 20 loss: 0.003049 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 21 loss: 0.002574 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 22 loss: 0.002297 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 23 loss: 0.002047 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 24 loss: 0.001814 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 25 loss: 0.001596 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 26 loss: 0.001490 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 27 loss: 0.001315 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 28 loss: 0.001190 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 29 loss: 0.001094 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 30 loss: 0.000992 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 31 loss: 0.000943 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 32 loss: 0.000842 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 33 loss: 0.000779 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 34 loss: 0.000729 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 35 loss: 0.000663 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 36 loss: 0.000618 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 37 loss: 0.000573 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 38 loss: 0.000533 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 39 loss: 0.000499 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 40 loss: 0.000470 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 41 loss: 0.000443 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 42 loss: 0.000416 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 43 loss: 0.000387 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 44 loss: 0.000364 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 45 loss: 0.000340 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 46 loss: 0.000323 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 47 loss: 0.000300 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 48 loss: 0.000288 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 49 loss: 0.000266 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 50 loss: 0.000256 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 51 loss: 0.000238 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 52 loss: 0.000229 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 53 loss: 0.000216 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 54 loss: 0.000204 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 55 loss: 0.000194 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 56 loss: 0.000185 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 57 loss: 0.000176 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 58 loss: 0.000165 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 59 loss: 0.000161 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 60 loss: 0.000150 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 61 loss: 0.000145 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 62 loss: 0.000136 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 63 loss: 0.000131 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 64 loss: 0.000124 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 65 loss: 0.000119 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 66 loss: 0.000112 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 67 loss: 0.000107 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 68 loss: 0.000102 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 69 loss: 0.000098 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 70 loss: 0.000095 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 71 loss: 0.000088 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 72 loss: 0.000085 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 73 loss: 0.000082 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 74 loss: 0.000079 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 75 loss: 0.000075 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 76 loss: 0.000071 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 77 loss: 0.000068 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 78 loss: 0.000065 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 79 loss: 0.000062 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 80 loss: 0.000060 \n"
     ]
    }
   ],
   "source": [
    "smalltrainloader = torch.utils.data.DataLoader( mnist_trainset_small, batch_size=64, shuffle=True)\n",
    "eperr, eploss = my_model(smalltrainloader, epo = 80, learn = 0.001)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 298
    },
    "id": "od9anZrjgXVf",
    "outputId": "ae1b026f-5820-4d64-959c-bdf3685076b5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Loss vs Epoch')"
      ]
     },
     "execution_count": 62,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEICAYAAABCnX+uAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de7xcZX3v8c93ZrITciMJ2Ym5kYBEMKJcjFyKVhRQoAqVeoGqhdaKbbHqqZeiPbWUnp7iqcW2p1SLSrVYQcQLEVFULgdRQYIKEgISICH37ECu5Lqzf+ePtfZm2JnZe2fvWTNrFt/3K/PKrMus9dszz/zmWc+z1noUEZiZWfsqtToAMzMbGSdyM7M250RuZtbmnMjNzNqcE7mZWZtzIjcza3NO5Na2JM2TFJIqrY7F2pekiyTd3eo4RiLzRC5puaSdkrZXPf4t6/22G0mXSfpKq+MYiTSpPtvvs/5Yq+N6oUm/c6e3Oo7hkHSqpJ5+ZWi7pJNbHVueNasm8+aI+NFgK0mqRER3v3nliNg31B0d6PoHQpIARURPFtsviGMiYlmrg7C2tiYiZrc6iHbS0qaV9JDmJ5I+I+lp4DJJX5L0WUm3SHoWeJ2kl0q6U9JmSUsknVO1jf3Wr7GfOyX9g6SfS9oq6SZJU6qWnyTpp+n2H5B0ar/X/r2knwA7gMMlvUzSDyU9I2m9pE+k65YkXSrpcUlPS7qhdz9VzQAXSnpK0kZJf5UuOxP4BPCOtPbxQDr/DyUtlbRN0hOS3tfv7/qYpLWS1kj643T7R6TLRkv6dLqv9ZI+J+mgGu/N6PTvPrpqXmd6FDVN0lRJN6frPCPpx5IOuNykRxw3Svpa+vf8QtIxVcsH+owPkvRPklZI2iLp7n5/yzv7v6e2v/Sz/ue0vKxJn49Ol9X9nCX9paTV6ef2qKTTamz7REnrJJWr5r1F0oPp8xMkLU6/f+slXTnMv2Gw7/I5afnZnK770qplcyR9U1JX+v38t37b/rSkTZKelHTWcOJrmYjI9AEsB06vs+wioBv4c5Kjg4OALwFbgFNIfmgmAMtIEl0H8HpgG3Bkuo3+64+psZ87gdXA0cA44BvAV9Jls4CngbPT15+RTndWvfYp4GVpjBOAtcCHgTHp9Inpuh8E7gFmA6OB/wCuS5fNAwL4fPp3HgPsBl6aLr+sN6aquH8HeDEg4LUkPyTHp8vOBNalcY0FvpJu/4h0+WeARcCUNMbvAP9Q53O4Bvj7qulLgO+nz/8B+BwwKn28huSopNZ2+vZfY9llwF7grel2PgI8WbXdgT7jq9LPYRZQBn4rfX8HfE9fqA/qfOeAy9PyOQ3oBH4K/N1AnzNwJLASmFlVjl9cZ7+PA2dUTX8duDR9/jPg3enz8cBJdbZxKrBqgL/tTup/l18CPEvyHR4FfCwtVx1puXkg/V6MI/nuvroqD+0F3puu96fAmnrlPI+PZhWq7cDmqsd7q97Ap/qt/yXgv6qmX0OSsEpV864DLqu1/gAf/hVV0wuAPemH9pfAtf3WvxW4sOq1l1ctuwD4ZZ39LAVOq5qekRaQCs8lndlVy38OnJ8+v4x+ibzG9r8NfDB9fg1ViRk4It3+ESRfwGerv3DAycCTdbZ7OvB41fRPgD9In18O3ESdBN1vOwFs7fdZv7Hq77unat0SyQ/iawb6jNP1dpI02fTf34Dv6Qv1Qf1E/jhwdtX0G4HlA33OaXnakJaRUYPs938B16TPJ6RlcG46fRfwt8DUQbZxKtDTrwxtBsaly++k/nf5r4Eb+pWx1ek2Twa6gEqNfV4ELKuaHpuWqxe1+rMc6qNZTSu/GxGTqh6fr1q2ssb61fNmAivj+e3SK0hqZwNtY6BtriD5xZ4KzAXelh6KbZa0GXg1SRKu9do5JF+IWuYC36razlJgHzC9ap11Vc93kNROapJ0lqR70kPdzSRHDVPTxTP7xVX9vJOkMN5fFcv30/m13AGMTQ+P5wHHAt9Kl/0jSa3mB2nzzqX14k0d3++zvrVWjOnnuSr9Owb6jKeS1J7qvedwAO/pC9xMkve114p0HtT5nCPp7/gQyY/qBknXS5pJbV8Fzkuba84DfhERvft7D0mN+RFJ90l60wBxrulXhiZFxLNVy+t9l5/396XlaSVJOZoDrIh+fXBV1lW9bkf6tG3KUR5OP6x1+8XqeWuAOf3aZQ8l+aUdaBv9zen3+r3ARpIP+tp+hWZcRFxRZ/srgcPr7GMlcFa/bY2JiNV11q/2vL8h/TJ8A/g0MD0iJgG3kNS2IanNVncIVf99G0lqsS+riuPgiKhZMCPpHL6B5GjjAuDmiNiWLtsWER+OiMOBc4C/qNVGOkR9Maaf52ySz3egz3gjsIukiclGZg1JZaPXoem8AT/niPhqRLw6fW0An6q18Yh4mCSRngX8Pkli7132WERcQNKs8yngRknjhvl31PsuP+/vk6R03dUk381DVdBTVfOQyAdzL0kt62OSRinpiHwzcP0BbuddkhZIGktyGHljmsC+ArxZ0hsllSWNUXIKVL1e85uBGZI+lHYeTZB0Yrrsc8DfS5oLfZ2G5w4xvvXAvKpk1kHSDtwFdKedL2+oWv8G4A/TTsKxJIeVQF9N5PPAZyRNS2OZJemNA+z/q8A7gHdS9QWU9CZJR6Rfii0kRxjDPWvnlZLOS79MHyJpz76HAT7j9G+5BrhS0sz0Mzq5t5PO6hqVluXeR4Wkuep/puVyKvBJkvJf93OWdKSk16fv9y6SCsJAn/9XSfqKfpukjZx0+++S1Jl+npvT2cMtR/W+yzcAvyPpNEmjSPqxdpP0BfycpPJzhaRx6XtyyjD3nzvNSuTf0fPPCf3W4C9JRMQeki/1WSS/uv9O0n77yAHGcC1Je/o6kkP1D6TbXwmcS9LR1kXyy/1R6rw3aU31jDSmdcBjPHemzL+QdDD+QNI2kiR1Yq3t1NBb6J+W9It0Px8gKZybSGo4i6ri+B7wryTNIsvSfUFScCFp+18G3CNpK/Ajko6rmiLiXpI2zZnA96oWzU9fu52kw+rfI+KOAf6OB/p91v9ctewmkh+LTcC7gfMiYu8QPuOPAL8G7gOeIanRtUMlpJVuIUm6vY/LSNqwFwMPkryfv0jnQf3PeTRwBcnnso6kRv3xAfZ7HUnH/O0RsbFq/pnAEknbSb4n50fEzjrbmKn9zyP/varl9b7LjwLvAv5vGu+bSU593pMm+jeTtPk/RdKs944B/o62orRxv9Ak3UnSkfiFVseSlfQ0q4eA0QO0A7aMpMtIOtLe1epYrH29EL7Lw+FaTRtTcp7uaEmTSWqp38ljEjezbDmRt7f3kZwa9jhJm+aftjYcM2uFF0TTiplZkblGbmbW5pp6TuXUqVNj3rx5zdylvYDcf//9GyOi3kVPmXLZtiwNVrabmsjnzZvH4sWLm7lLewGRtGLwtbLhsm1ZGqxsu2nFzKzNOZGbmbU5J3IzszbnRG5m1uYGTeTpqBp3SHpYycgbH0znX6Zk1JBfpY+zsw/XzMz6G8pZK93AhyPiF5ImkNzj+ofpss9ExKezC8/MzAYzaCKPiLUkt38kIrZJWsrzB3UwM7MWOqA28nT0mONI7h8N8H5JD0q6Jr1xU63XXKxk0NXFXV1dNbf7+bue4NYl62ouM2tXK55+lit/8CirNu0YfGWzERhyIpc0nmTEmg9FxFbgsySjthxLUmP/p1qvi4irI2JhRCzs7Kx9YdJ//uRJfvTw+gON3SzXVm3ayb/evow1m3e1OhQruCEl8nS0jW8A/x0R3wSIiPURsa9qNJoThh1ESezr8c27rFhKSkbl6+4Z7kA4ZkMzlLNWBHwRWBoRV1bNrx6c+C0kgxoMS6Uk9vkujFYw5VKSyJ3HLWtDOWvlFJJhuX4t6VfpvE8AF0g6lmQw1uUk98YelnJJdLtGbgXTm8hdSbGsDeWslbt5buT2arc0KohySezb58Ju2ZN0DfAmYENEHF1juUjGlDybZEDoiyLiF8PZV18id5XcMpaLKzvLpZJrLdYsXyIZCLies0gGIp4PXEzSqT8slb5EPtwtmA1NThI57uy0poiIu4BnBljlXOC/InEPMKlff9CQ9XZ2ukZuWctJIi85kVtezAJWVk2vos4FcINdI1F2jdyaJB+JXK6RW/sZ7BoJd3Zas+QikVdcI7f8WA3MqZqenc47YO7stGbJRSIv+4Igy49FwB8ocRKwJb3f0AEry00r1hxNHbOznnJJ7O7e1+ow7AVA0nXAqcBUSauAvwFGAUTE50hOqz0bWEZy+uEfDndf5XLvBUGupFi2cpPIfRq5NUNEXDDI8gAuacS+yn2X6LtwW7Zy1LTi408rFnd2WrPkKJG3OgqzxupL5C7clrF8JHK5Rm7F09fZ6Qq5ZSwfibzss1aseNzZac2Sj0QuJ3IrHnd2WrPkIpFXfBtbK6C++5G7s9MylotEXi7Jh59WOM9d2emybdnKTSJ3jdyKJs3jLtuWudwkch9+WtFI8tGmNUVuErlrLVZEZblsW/Zyk8jdjmhFVCq5s9Oyl49E7tMPraB8i2Zrhnwk8rIPP62YSh40xZogF4m84g4hK6hK2TVyy14uErk7hKyoSi7b1gT5SOSlJAzXyq1oyiWXa8teThJ58r9rLlY0lVLJ9yO3zOUkkac1chd4K5hSyZ2dlr2cJPLkf9fIrWh8+qE1Q04SeRLGPt+B3wqmJA/1ZtnLRSKveGxDK6hySa6gWOZykchLpd4b8Hu4NyuWsjs7rQlykch7a+TO41Y0ZXd2WhMMmsglzZF0h6SHJS2R9MF0/hRJP5T0WPr/5OEG8dyQWM7kViy+j5A1w1Bq5N3AhyNiAXAScImkBcClwG0RMR+4LZ0elrJr5FZQvte+NcOgiTwi1kbEL9Ln24ClwCzgXODL6WpfBn53uEGU3UZuBVUuiW53dlrGDqiNXNI84DjgXmB6RKxNF60Dptd5zcWSFkta3NXVVXO7HtvQiqpckjs7LXNDTuSSxgPfAD4UEVurl0VEADVLa0RcHRELI2JhZ2dnzW379EMrKg/1Zs0wpEQuaRRJEv/viPhmOnu9pBnp8hnAhmEH0du04kNQKxjf/dCaYShnrQj4IrA0Iq6sWrQIuDB9fiFw03CD6Dv90DVyy5ikMyU9KmmZpP066CUdmp6l9UtJD0o6eyT7q7iz05pgKDXyU4B3A6+X9Kv0cTZwBXCGpMeA09Pp4QXR19npAm/ZkVQGrgLOAhYAF6RnYFX7n8ANEXEccD7w7yPZpzs7rRkqg60QEXcDqrP4tIYE0Xf6oQu8ZeoEYFlEPAEg6XqSs68erlongInp84OBNSPZoU8/tGbIxZWdz10Q5AJvmZoFrKyaXpXOq3YZ8C5Jq4BbgD+vt7GhnpHls7Esa/lI5D790PLjAuBLETEbOBu4VlLN78lQzsgq+cpOa4JcJPJK2YncmmI1MKdqenY6r9p7gBsAIuJnwBhg6nB3WPF55NYEuUjkJTmRW1PcB8yXdJikDpLOzEX91nmKtO9H0ktJEnntdpMhKLmz05ogF4m80juwhBO5ZSgiuoH3A7eS3GrihohYIulySeekq30YeK+kB4DrgIvSC96GxacfWjMMetZKM5Q81Js1SUTcQtKJWT3vk1XPHyY55bYh3NlpzZCrGrlrLlY07uy0ZshFIvfgy1ZU7uy0ZshJIu9tI/dtbK1YSm5asSbIRSLvu/uh87gVjEcIsmbIRSIv9SVyZ3IrlnLZidyyl4tE7hq5FZVr5NYMuUjkz10Q5ExuxeLOTmuGXCTyiu+1YgVVKokI39nTspWLRO77kVtR9d7Z07Vyy1IuErlr5FZUZd8QzpogF4m87MGXraB6a+S+atmylK9E7rvEWcGU3WxoTZCPRO52RCuosocxtCbIRSIvlYTkdkQrHtfIrRlykcghPd/Whd0KxjVya4bcJHLf7tOKyM2G1gy5SeSukVsR9V0j4Y58y1BuEnmpJLcjWuH0XiPh0w8tS7lJ5K6RWxG5s9OaITeJvFwquR3RCsedndYMOUrkviDIisedndYMuUnkFdfIrYDc2WnNkJtEXir5giArHnd2WjPkJpFXSiUnciucku/saU2Qm0Re8iX6VkC+RbM1w6CJXNI1kjZIeqhq3mWSVkv6Vfo4e6SBVEoluj3UmxVMX2enE7llaCg18i8BZ9aY/5mIODZ93DLSQMolefBlKxw3rVgzDJrII+Iu4JmsA0kSuTO5FUvFg6ZYE4ykjfz9kh5Mm14m11tJ0sWSFkta3NXVVXdj5ZLwGVpWNK6RWzMMN5F/FngxcCywFvineitGxNURsTAiFnZ2dtbdoGvk1gySzpT0qKRlki6ts87bJT0saYmkr45kf+7stGaoDOdFEbG+97mkzwM3jzSQsu+1YhmTVAauAs4AVgH3SVoUEQ9XrTMf+DhwSkRskjRtJPssubPTmmBYNXJJM6om3wI8VG/doSr7fuSWvROAZRHxRETsAa4Hzu23znuBqyJiE0BEbBjJDsu+IMiaYNAauaTrgFOBqZJWAX8DnCrpWCCA5cD7RhxIWezqdmG3TM0CVlZNrwJO7LfOSwAk/QQoA5dFxPdrbUzSxcDFAIceemjNHVZ890NrgkETeURcUGP2FxsdSLkk3yHO8qACzCepvMwG7pL08ojY3H/FiLgauBpg4cKFNQuvOzutGXJzZWdZHljCMrcamFM1PTudV20VsCgi9kbEk8BvSBL7sPiCIGuG/CRyd3Za9u4D5ks6TFIHcD6wqN863yapjSNpKklTyxPD3WHZNXJrAidye8GIiG7g/cCtwFLghohYIulySeekq90KPC3pYeAO4KMR8fRw9+nOTmuGYZ1+mIXkgiAXdstWejuJW/rN+2TV8wD+In2MmDs7rRlcIzfLUMlDvVkT5CqRexQVK5rezk7XyC1LuUnklZLcjmiFUy67s9Oyl5tEXi759EMrnt4auSsplqVcJXK3I1rRlN3ZaU2Qn0TuC4KsgMru7LQmyE8iL5Vc2K1wnruys8WBWKHlKJH78NOKp1QSEr7XvmUqR4m85J59K6SyfLGbZStHidzjGloxlXxGlmUsR4k8qZGHk7kVTMVnZFnGcpPIK303F2pxIGYNlox+1eoorMhyk8ifO9/WJd6KpVz2wOKWrdwlcpd3Kxp3dlrW8pPI5Rq5FVOp5KYVy1Z+ErlHUrGCqpTctGLZciI3y1jJnZ2WMSdys4xV3NlpGctNIu89/dCdQlY0SWdnq6OwIstNIu8dEsujBFnRlHxBkGUsN4m84tHGraAqJflsLMtUbhK5b8BvReXOTsta7hK5OzutaNzZaVnLTyKXE7kVU8mdnZax/CRy18itoDwerWUtN4m8UnYit2Iqu7PTMjZoIpd0jaQNkh6qmjdF0g8lPZb+P3nEgcidnVZMZck3g7NMDaVG/iXgzH7zLgVui4j5wG3p9IhUSkkoPv3QiqZc8t0PLVuDJvKIuAt4pt/sc4Evp8+/DPzuiANJI/EFQVY0ZQ/1Zhkbbhv59IhYmz5fB0wfaSC9NXK3kVuWJJ0p6VFJyyTVPZKU9HuSQtLCke7TnZ2WtRF3dkYyyGbdUirpYkmLJS3u6uqqu51yGokPQS0rksrAVcBZwALgAkkLaqw3AfggcG8j9usauWVtuIl8vaQZAOn/G+qtGBFXR8TCiFjY2dlZd4Plvhq5e4UsMycAyyLiiYjYA1xP0kzY398BnwJ2NWKnSWenE7llZ7iJfBFwYfr8QuCmkQbSd/dD53HLzixgZdX0qnReH0nHA3Mi4ruDbWzoR5vu7LRsDeX0w+uAnwFHSlol6T3AFcAZkh4DTk+nRxZI35WdzuTWGpJKwJXAh4ey/tCPNuW+H8tUZbAVIuKCOotOa2ggZdfILXOrgTlV07PTeb0mAEcDdyqpWLwIWCTpnIhYPNydOpFb1nJzZWfJgy9b9u4D5ks6TFIHcD5JMyEAEbElIqZGxLyImAfcA4woiYMTuWUvN4m84nutWMYioht4P3ArsBS4ISKWSLpc0jlZ7bcsJ3LL1qBNK80yYUwSytade1sciRVZRNwC3NJv3ifrrHtqI/ZZcmenZSw3NfLJYzsol8TG7XtaHYpZQ1XctGIZy00iL5XElHEdbNy+u9WhmDWU28gta7lJ5ABTx492IrfCKfmCIMtYzhJ5B11uWrGCqZR9ib5lK1eJvHP8aDZuc43ciiUZ6s2J3LKTq0Q+dcJourbvJlzorUBGlUX3vh6Xa8tMvhL5+A72dPewbXd3q0Mxa5gp4zroCdi8w6fWWjZylshHA7h5xQpl+sQxAKzf1pCbKZrtJ5+J3B2eViDTJyblev1WV1AsGzlN5C7wVhzTJqQ18q2ukVs28pXIJ3QATuRWLJ0TkgrKBidyy0iuEvmUsR1IbiO3YhkzqsyksaPctGKZyVUir5RLHDLOFwVZ8UyfMMZNK5aZXCVy8GX6VkzTJo5mvY80LSNO5GZNMH3iGLeRW2ZymMh9B0QrnukTR9O1bbdvnmWZyGEiH83GbW4jt2KZPnEM3T3BMztctq3x8pfIJ4xm5959POvL9K1Apk3ovSjIzSvWeLlL5L1Xwa3dsrPFkZg1zrT0Mv0NPgXRMpC7RH5E5wQAlm3Y3uJIzBqn734rrpFbBnKXyF88bRwAj613Irfi6Bzv+61YdnKXyMd2VJg16SAec43cCqSjklzs5jsgWhZyl8gB5k8f70RuhTNt4hjWb3Eit8bLZyKfNp4nurZ75HErlHmHjOXxLldQrPFymsgnsLu7h1WbdrQ6FLOGOXrWwSx/egdbd3mkIGusXCbyI6aPB9zhacXyspkTAXh4zdYWR2JFk89EPi1N5G4ntwJ52cyDAXho9ZYWR2JFk8tEPnHMKKZPHM1jG7a1OhSzhumcMJoXTRzDEtfIrcEqI3mxpOXANmAf0B0RCxsRFCTt5G5asaI5etZE18it4RpRI39dRBzbyCQO8OLOcTy58VkifOaKFcfLZh7M413b2bHH9xKyxsll0wrA3EPGsX13N08/67vFWeNIOlPSo5KWSbq0xvK/kPSwpAcl3SZpbiP3f/Ssg+kJWLrWzYbWOCNN5AH8QNL9ki6utYKkiyUtlrS4q6tryBs+bGpyqf7yjc+OMESzhKQycBVwFrAAuEDSgn6r/RJYGBGvAG4E/k8jYzh6VnLmypI1bl6xxhlpIn91RBxP8sW4RNJv918hIq6OiIURsbCzs3PIG557yFgAlj/tc8mtYU4AlkXEExGxB7geOLd6hYi4IyJ6C909wOxGBvCiiWOYPHYUS9e6w9MaZ0SJPCJWp/9vAL5F8kVpiNmTx1IuiRVPu0ZuDTMLWFk1vSqdV897gO/VWzico01JHPWiiW5asYYadiKXNE7ShN7nwBuAhxoVWEelxKxJB/Gkm1asBSS9C1gI/GO9dYZ7tHnUjAk8um6bh32zhhnJ6YfTgW9J6t3OVyPi+w2JKjVv6jhWuGnFGmc1MKdqenY673kknQ78FfDaiGj4fWdf+qKJ7Ny7j6ee2cG8tC/IbCSGncgj4gngmAbGsp95h4zllys2ERGkPxhmI3EfMF/SYSQJ/Hzg96tXkHQc8B/AmWmTYcMdNSMZPOWRdVudyK0hcnv6ISSnIG7b3c0zPgXRGiAiuoH3A7cCS4EbImKJpMslnZOu9o/AeODrkn4laVGj45g/bQIl+RREa5wRXdmZtcOm9p658iyHpCOsmI1ERNwC3NJv3iernp+edQwHdZSZN3Ucj6zzmSvWGLmvkQMs3+h2ciuWl/rMFWugXCfyOZPHUlJSIzcrkqNeNIGnntnB9t2+VN9GLteJvKNSYs6UsSzz7WytYI6a4XuTW+PkOpEDHD3zYH7tu8VZwbxq3mTKJfH/fpPJiTH2ApP/RD7rYFZt2skmn7liBTJpbAevnDuZ2x8Z+v2HzOrJfSJ/+ax0VBXfZMgK5rSjprF07VbWbN7Z6lCszbVNIn9wlRO5Fcvrj5oGwB2PunnFRib3ifzgsaM4dMpYj6pihXPEtPHMmXIQty91IreRyX0ih6RW7g5PKxpJnHbUdO5etpEtO/a2OhxrY+2RyGe7w9OK6W0LZ7O7u4fr73uq1aFYG2uPRN7bTu5auRXMy2YezEmHT+HLP11O976eVodjbaotEvkxcyYxtqPMTb/c746jZm3vj045jDVbdnHrkvWtDsXaVFsk8vGjK7x94Ry+8+AaNmzd1epwzBrqtJdOZ+4hY/nC3U+0OhRrU22RyAEu+q15dPcE196zotWhmDVUuST+6JTD+OVTm7l/xaZWh2NtqG0S+byp4zjtqOl85Z4V7Nq7r9XhmDXUW185m4ljKnzhx66V24Frm0QO8N7XHMamHXv5+v2rWh2KWUONG13hnSfN5dYl63jKwxvaAWqrRH7CYVM4ds4kPn/XE+7ht8K56LfmUS6Ja37yZKtDsTbTVolcEn/y2hfz1DM7+N5D61odjllDTZ84hnOPncXX7lvpaybsgLRVIgd4w4LpHN45jv+463EiotXhmDXUxb99ODv37nOnvh2QtkvkpZL441cfzkOrt3Lvk8+0OhyzhnrJ9Am87shOvvzT5e7UtyFru0QOcN7xs5g8dhRfvNttiVY8f3rqETz97B6uumNZq0OxNtGWiXzMqDLvPHEuP1q6nuUbPZ6nFcsJh03hvONn8dk7H/dQcDYkbZnIAf7g5LlU3MNvBfXXv7OASWNH8dEbH2DnHjex2MDaNpFPmziGtxw3i+vvW+nL9q1wJo/r4IrzXsHStVu5+NrF7O52Mrf62jaRA1zyuiPY1xNcfZevhrPiOX3BdK447xX8+LGNfPTrD/osLaurrRP53EPGce4xM/nKvSvo2ra71eGYNdzbXzWHj7zhJSx6YA1fu29lq8OxnGrrRA5wyeuTWvmF1/ycDdvcxGLF82enHsEpRxzCZd9ZwtK17vy0/bV9In9x53i+cOGrWP70s7z1sz/jGV8RZwVTKokr334sE8aM4m2f+xk/WOKrmu352j6RA7z2JZ1c+54TWbtlJ39900OtDses4aZPHMO3LzmFwzvHcfG193PhNT/nx491sa/H7eY2wkQu6UxJj0paJunSRgU1HK+cO5kPnauFdkUAAAcWSURBVP4SvvvgWm76lUcSstoGK7OSRkv6Wrr8Xknzmh9lbbMmHcQN7zuZj77xSJas2cq7v/hzTvzfP+IT3/o1P1m20VeCvoBpuD3hksrAb4AzgFXAfcAFEfFwvdcsXLgwFi9ePKz9DUX3vh7e+rmf8cCqzbxhwXTOO342xx06iYljRlEuiVHlQhyAWB2S7o+IhQMsH7TMSvoz4BUR8SeSzgfeEhHvGGzfWZft/nbt3cftj2zgu79ey+1LN7Bz7z5KgkOnjGXyuA4mjhnF5LGjOGT8aDonjGZsR5lKqcT4MRUmjK5wUEeZMaPKjCqLckmUlXw/RlVKjCqLSqmEgHJZVEqipORRLomSkhvYWfMMVrYrI9j2CcCyiHgi3dH1wLlA3USetUq5xH9e9Cq+ePeTXHvPiv3GQDxkXAeTx3XgIti+OiolvvuB1wz35UMps+cCl6XPbwT+TZIiZ+f+jRlV5uyXz+Dsl89g55593L1sI79evYXHu7azdedeNu3YwxMbt/P09j3syOCCIglKEiL5n+RfmvCTRC/omy8JVT9Pt5FurWpZ35y+/aSb2e/Ho3qy/+tqrrPf37B/JqiZGwZIGEPJJUP90TvnmJl84LT5Q1q3v5Ek8llA9flQq4AT+68k6WLgYoBDDz10BLsbmsnjOvjIG4/kz087godWb+GBlVvY1b2Pvd3Buq272LLTnaHtbIRHVUMps33rRES3pC3AIcDG/htrdtmu56COMmcsmM4ZC6bXXL59dze79u5j774etu/qZtvubnbu2cfu7n3s6Q4igu6eoLunhz3dPezdF3Tv6yGAfT3Jsp4IenqCnkjm9UQQQfI/yf+k0z0ByWSyDkCk6+03P53H89btXRZ9072/ov2X9W3kuf+em13127v/sv3fp1q/1AP9fg/pl/0Afv6nTRg99JX7GUkiH5KIuBq4GpLDz6z312t0pcwr507hlXOnNGuX9gLTqrJ9oMaPrjB+dPpVP7i1sVg2RlK9WQ3MqZqenc4zy6uhlNm+dSRVSFLf002JzmyYRpLI7wPmSzpMUgdwPrCoMWGZZWIoZXYRcGH6/K3A7XlrHzfrb9hNK2n74fuBW4EycE1ELGlYZGYNVq/MSrocWBwRi4AvAtdKWgY8Q5LszXJtRG3kEXELcEuDYjHLXK0yGxGfrHq+C3hbs+MyGwmfWG1m1uacyM3M2pwTuZlZm3MiNzNrc8O+18qwdiZ1ASvqLJ5KjavnWsSx1Jb3WOZGRGcrghmgbOfpPYN8xeNYajvgst3URD4QSYsHuilMMzmW2hzLgctbnHmKx7HUNpxY3LRiZtbmnMjNzNpcnhL51a0OoIpjqc2xHLi8xZmneBxLbQccS27ayM3MbHjyVCM3M7NhcCI3M2tzLU/krRzAWdIcSXdIeljSEkkfTOdPkfRDSY+l/09uYkxlSb+UdHM6fVg6CPCydFDgjibFMUnSjZIekbRU0smtel8k/Y/083lI0nWSxrTqfTkQLtv7xeSyvX8sDSnbLU3kSgbDvQo4C1gAXCBpQRND6AY+HBELgJOAS9L9XwrcFhHzgdvS6Wb5ILC0avpTwGci4ghgE/CeJsXxL8D3I+Io4Jg0pqa/L5JmAR8AFkbE0SS3nz2f1r0vQ+KyXZPLdpWGlu2IaNkDOBm4tWr648DHWxjPTSQjrD8KzEjnzQAebdL+Z5MUotcDN5OM7boRqNR6vzKM42DgSdLO8Kr5TX9feG4MzSkkt12+GXhjK96XA4zbZfv5+3fZ3j+WhpXtVjet1BoMd1YrApE0DzgOuBeYHhFr00XrgNqj2jbePwMfA3rS6UOAzRHRnU436/05DOgC/jM9FP6CpHG04H2JiNXAp4GngLXAFuB+WvO+HAiX7edz2e6nkWW71Yk8FySNB74BfCgitlYvi+RnMfNzNCW9CdgQEfdnva8hqADHA5+NiOOAZ+l3qNnE92UycC7JF3AmMA44M+v9FoXL9n4KWbZbnchbPoCzpFEkBf2/I+Kb6ez1kmaky2cAG5oQyinAOZKWA9eTHIL+CzBJySDA0Lz3ZxWwKiLuTadvJCn8rXhfTgeejIiuiNgLfJPkvWrF+3IgXLaf47JdW8PKdqsTeUsHcJYkkjEal0bElVWLqgfgvZCkfTFTEfHxiJgdEfNI3ofbI+KdwB0kgwA3M5Z1wEpJR6azTgMepgXvC8lh50mSxqafV28sTX9fDpDLdsplu67Gle2sG/SH0OB/NvAb4HHgr5q871eTHEI9CPwqfZxN0n53G/AY8CNgSpPjOhW4OX1+OPBzYBnwdWB0k2I4FlicvjffBia36n0B/hZ4BHgIuBYY3ar35QDjdtnePy6X7efH0pCy7Uv0zczaXKubVszMbIScyM3M2pwTuZlZm3MiNzNrc07kZmZtzonczKzNOZGbmbW5/w9YUhowL7+Q0wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.subplot(1,2,1)\n",
    "plt.plot(eperr)\n",
    "plt.title('Error percentage vs Epoch')\n",
    "plt.subplot(1,2,2)\n",
    "plt.plot(eploss)\n",
    "plt.title('Loss vs Epoch')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1RKDthYy2AmF",
    "outputId": "df0e4b36-1c8f-44aa-d6e9-fedd55c4ca31"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error Rate on test set: 6.090000 %\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "6.09"
      ]
     },
     "execution_count": 98,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "validate(testloader) # this function is defined below, I forgot to run this cell before"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IYes1SV-1Sk0"
   },
   "source": [
    "**The Model is Overfitting and giving 100% classification accuracy/ 5% error on training set, I desired 3% error on both training and test set. The validations set has 6.09% error**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VsiCjz9KnzyA"
   },
   "source": [
    "### 2.  Regularisation experiment:  ***12 marks***\n",
    "\n",
    "Now select a type of regularisation (which may be L2, or dropout) and train your initial model with different amounts of regularisation. \n",
    "\n",
    "A small amount of regularisation may improve the validation set performance of your overfitted model; too much regularisation may make performance on the validation set worse. \n",
    "\n",
    "Train your model multiple times, applying different amounts of regularisation, and plot a graoph showing the effect of different amounts of regularisation. \n",
    "\n",
    "Plot the amount of regularisation along the x axis, and the validation set performance (loss/error-rate) on the y axis.  How does regularisation affect the performance of your initial network, and what is the optimal amount? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_42DnVKc2wem"
   },
   "source": [
    "**Choosing Dropout for regularisation**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "id": "ubxakNaskEki"
   },
   "outputs": [],
   "source": [
    "def validate(testloader):\n",
    "  # this code calculates the error rate on the validation set\n",
    "  incorrect = 0 \n",
    "  total = 0 \n",
    "  with torch.no_grad():  # we do not neet to compute the gradients when making predictions on the validation set\n",
    "      for data in testloader: \n",
    "          images, labels = data\n",
    "          outputs = nn1(images.cuda())\n",
    "          _, predicted = torch.max( outputs, dim=1)\n",
    "          total += labels.size(0) # the number of labels, which is just the size of the batch \n",
    "          incorrect += (predicted != labels.cuda()).sum().item() \n",
    "                                                        \n",
    "  print( \"Error Rate on test set: %f %%\" %(100 * incorrect/total))\n",
    "  return (100 * incorrect/total)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8_hn14KcnzyB"
   },
   "outputs": [],
   "source": [
    "# your code here \n",
    "reg = [0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]\n",
    "valerr = np.zeros(len(reg))\n",
    "for i in range(len(reg)):\n",
    "  my_model(smalltrainloader, epo = 50, learn = 0.001, dropout = reg[i])\n",
    "  valerr[i] = validate(testloader)\n",
    "plt.plot(valerr, reg)\n",
    "plt.title('Error Rate vs Dropout Rate' )\n",
    "plt.xlabel('Dropout Rate')\n",
    "plt.ylabel('Error rate in percantage')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 312
    },
    "id": "hjGxeCTNm1a7",
    "outputId": "0caaceaf-80cf-4eb8-81e2-69386ffa77c5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Error rate in percantage')"
      ]
     },
     "execution_count": 65,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEWCAYAAABhffzLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZhdVZnv8e+vplSSypxKyByGBAQkISkRFGkQtR0YbEUQtUXbFrUd27nVvmI79nW6rXa3YmvLtUXACVC0lVYE5YJalYQxSBhSGUkqyamkUhlqeu8fe1dyUtRwMpyxfp/nOc85e37PquTd+6y99lqKCMzMbPSoKnYAZmZWWE78ZmajjBO/mdko48RvZjbKOPGbmY0yTvxmZqOME7+Z2SjjxD/KSVoraa+k3VmvrxU4ht9K2pcee5ukH0ualeO250vakO8Yc4ijvxw7JLVL+n+S3iqp5P6PSVooKSTVDLPONZK6079J//c55zCOEZJOOjYR27FWcv8orSgujoiGrNc7BltpsEQhqfpwDjTM+u+IiAbgJKAB+MLh7LdEXBwRE4AFwOeADwHfGmrlwy27Irgx/ZtMB+4AflDkeOwYceK3IUl6g6S7JX1Z0nbgGknfkfTvkn4uqRO4QNIz0qv2dkkPSbokax9PW3+4Y0ZEO3AzsDRrH2+UtDq9mn5C0lvS+eOBXwCzs36tzJZUJenDkh6XtF3STZKmDvEdV0u6KGu6RlKbpGWS6iX9V7qPdkl/kjRzpHKLiJ0RcStwBXCVpNOHKoscyu7rkm5Pv/udkhZkLX9OGtPO9P05WcvWSnpB1vQ1kv4rnbwrfW9Py2zYK/mI6AG+B8yR1Jju7yxJ96Rxb5b0NUl16bL+/d+X7v+KdP5FklZl/YI4Y6SytPxw4reRPBt4ApgJfDqd95r08wTgD8BPgV8BM4B3At+TdHLWPrLX//1wB5M0DXgF8FjW7K3ARcBE4I3AlyUti4hO4CXApqxfK5vSGF4O/AUwG8gA/zrEIb8PXJk1/ZfAtohYAVwFTALmAdOAtwJ7h4s/W0T8EdgAPC9r9uGW3WuBT5Jcda8iScCkJ7LbgK+ksX0JuC0tv5Gcl75PTsvsnuFWThP664HtJGUJ0Av8fRrXOcCFwN+l37t//0vS/d8o6Uzg28Bb0ni/AdwqaUwO8dox5sRvADenV2H9rzdnLdsUEV+NiJ6I6E96t0TE3RHRR3Jl3gB8LiK6IuI3wM84NJkeWD8i9g0Rw1ck7QS2kSSTd/YviIjbIuLxSNxJkiifN8R+IEnQH42IDRGxH7gGuGyIOu3rgUskjUunX0NyMgDoJklSJ0VEb0S0RMSuYY47mE1A9q+Nwy272yLirvR7fBQ4R9I84GXAmoj4bvq3+T7wCHDxYcY3nMsltZOc7N4MXJZe/ZOWxb3psdeSJPK/GGZfVwPfiIg/pGV5HbAfOPsYxms5cuI3gJdHxOSs1zezlq0fZP3sebOB9Wki69cKzBlhHwO9KyImAWcAU4C5/QskvUTSvZJ2pInopSQnh6EsAH7SfyIDVpNcoT6tmiYiHkuXX5wm/0tITgYA3wV+CdwgaZOk/y2pNofvkm0OsCNr+ojLLiJ2p/uanb5aBxxr4LZH66aImExSbg8Cy/sXSFos6WeSnpK0C/gMI/9N3pd9gUHyS2r2MYzXcuTEbyMZrPvW7HmbgHk6tPXKfGDjCPsY/GARDwCfAv5ViTHAj0hu9s5ME9HPAQ2z7/XASwaczOojYuMg68LB6p5LgYfTkwER0R0Rn4iIU4HnkFQ3vT7X7yLpWSSJOLt663DLbl7W/hpIfj1sSl8LOFT2tp3AuKxlxw0Rw4giYhvJFfs1Otja6t9JfmEsioiJwEc4+DcZzHrg0wP+JuPSXypWYE78drT+AOwBPiipVtL5JNUNNxzFPq8jucq8BKgDxgBtQI+klwAvylp3CzBN0qSseV8HPt1/I1RSo6RLhzneDek+38bBq30kXSDpmUpa3+wiqfrpG3wXB0mamN4wvgH4r/RkNphcyu6lks5N69k/CdwbEetJTn6LJb0mvSF9BXAqSVURJPcDXp3utwm4LGufben3OGGk79IvIv5M8uvng+msCSRlslvSKSRll23LgP1/E3irpGenJ/Txkl4maUKuMdix48RvAD/Voe34f5LrhhHRRZKsXkJSP/9vwOsj4pEjDSbd578A/xgRHcC7gJtIbiy+Brg1a91HSK7Yn0irEGan294K/EpSB3AvyU3qoY63GbiH5Kr+xqxFxwE/JElwq4E7Sap/hvLT9HjrSerjv0RyM3q47zlS2V0PfJykimc58Lp02+0kv0DeR3LT9YPARenVOcA/AieSlNknyDqhRcQekhvMd6dllms9++eBqyXNAN5P8rfoIEnqNw5Y9xrgunT/l0dEM8l9gq+lMT0GvCHH49oxJg/EYlaaJH0H2BARHyt2LFZZfMVvZjbKOPGbmY0yruoxMxtlfMVvZjbKDNk7XymZPn16LFy4sNhhmJmVlZaWlm0R0Thwflkk/oULF9Lc3FzsMMzMyoqkgU93A67qMTMbdZz4zcxGGSd+M7NRxonfzGyUceI3MxtlnPjNzEYZJ34zs1HGid/MrARlOrv47M9X80Tb7mO+byd+M7MS1NKa4Rt3PUFbx/5jvm8nfjOzEtSyLkNNlVgyb/Ix37cTv5lZCWpZm+G0OZOor60+5vt24jczKzFdPX3ct6GdpgVT8rJ/J34zsxLz0Kad7O/pc+I3MxstWlozACx34jczGx1aWjPMmzqWGRPr87J/J34zsxISETS3Zlg+Pz9X++DEb2ZWUjZk9tLWsZ/lC6fm7Rh5S/yS6iX9UdJ9kh6S9Il0/nckPSlpVfpamq8YzMzKTXPrDoC83diF/A69uB94fkTsllQL/F7SL9JlH4iIH+bx2GZmZal5bYYJY2pYPHNC3o6Rt8QfEQH0dzJRm74iX8czM6sELa0Zls6fTHWV8naMvNbxS6qWtArYCtweEX9IF31a0v2SvixpzBDbXi2pWVJzW1tbPsM0MysJu/Z18+ctHXlrxtkvr4k/InojYikwFzhL0unAPwCnAM8CpgIfGmLbayOiKSKaGhsb8xmmmVlJWLWunQhoWpC/G7tQoFY9EdEO3AG8OCI2R2I/8J/AWYWIwcys1DW3ZqgSLJ1/7Dtmy5bPVj2Nkiann8cCLwQekTQrnSfg5cCD+YrBzKyctLTu4JTjJtIwJp/tbvLbqmcWcJ2kapITzE0R8TNJv5HUCAhYBbw1jzGYmZWFnt4+Vq1r55XL5+b9WPls1XM/cOYg85+fr2OamZWrR57qoLOrN+83dsFP7pqZlYQV6/LbMVs2J34zsxLQvDbDcRPrmTN5bN6P5cRvZlYCWlozLF8whaTdS3458ZuZFdlTO/exsX1vQap5wInfzKzo+jtmc+I3MxslWloz1NdWcersiQU5nhO/mVmRtbRmWDJ3MrXVhUnJTvxmZkW0p6uHhzbtomlhYap5wInfzKyo7lu/k96+yHvHbNmc+M3MiqglvbF7Zp47ZsvmxG9mVkQtrRkWzWhg8ri6gh3Tid/MrEj6+uLAg1uF5MRvZlYkj7ftZte+Hid+M7PRork16ZitaWHhbuyCE7+ZWdE0r80wbXwdC6eNK+hxnfjNzIpkxboMywrUMVs2J34zsyLYtns/T27rLHj9Pjjxm5kVxYr++n0nfjOz0aGlNUNddRWnz5lU8GM78ZuZFUFza4bT50ykvra64Md24jczK7D9Pb08sGFnwZtx9nPiNzMrsAc37qSrt49l8wtfvw9O/GZmBdeS3tgtRoseyDHxS1og6QXp57GSJuSwTb2kP0q6T9JDkj6Rzj9e0h8kPSbpRkmF65nIzKwENK/NsHDaOBonjCnK8UdM/JLeDPwQ+EY6ay5wcw773g88PyKWAEuBF0s6G/hn4MsRcRKQAd50JIGbmZWjiKRjtmVFutqH3K743w48F9gFEBFrgBkjbRSJ3elkbfoK4PkkJxKA64CXH2bMZmZlq3X7HrZ3dhV04JWBckn8+yOiq39CUg1JAh+RpGpJq4CtwO3A40B7RPSkq2wA5gyx7dWSmiU1t7W15XI4M7OS11zk+n3ILfHfKekjwFhJLwR+APw0l51HRG9ELCWpHjoLOCXXwCLi2ohoioimxsbGXDczMytpLa0ZJtbXsGhGQ9FiyCXxfxhoAx4A3gL8HPjY4RwkItqBO4BzgMnprwZITggbD2dfZmblrKV1B8sWTKGqqrAds2UbMfFHRF9EfDMiXhURl6WfR6zqkdQoaXL6eSzwQmA1yQngsnS1q4Bbjjx8M7PysXNPN49u2c3yIrXf71cz0gqSHuDpdfo7gWbgUxGxfYhNZwHXSaomOcHcFBE/k/QwcIOkTwErgW8dcfRmZmVkxfq0fn9hiSd+4BdAL3B9Ov1qYBzwFPAd4OLBNoqI+4EzB5n/BEl9v5nZqNKyNkN1lVg6b3JR48gl8b8gIpZlTT8gaUVELJP0unwFZmZWaVpaM5w6ayLj6nJJvfmTy83dakkHrtAlPQvo706uZ/BNzMwsW3dvH6vWtxe1GWe/XE47fwt8W1IDIJIHuf5W0njgs/kMzsysUqzevIu93b3lkfgj4k/AMyVNSqd3Zi2+KV+BmZlVkv6O2ZqKfGMXcrviR9LLgNOA+v5BgSPin/IYl5lZRWluzTB7Uj2zJo0tdig5ddL2deAK4J0kVT2vAhbkOS4zs4qyojXD8iINvDJQLjd3nxMRrwcyEfEJkqdvF+c3LDOzyrGxfS+bd+5j+fziNuPsl0vi35u+75E0G+gmeTjLzMxy0Lx2B0DRhlocKJc6/p+lXS98HlhB8hTvf+Q1KjOzCrKiNcO4umpOOW7EMawKIpfE/78jYj/wI0k/A+qBffkNy8yscjS3Zlg6bzI11aUx2m0uUdzT/yEi9qfNOe8ZZn0zM0t17u9h9eZdNJVA+/1+Q17xSzqOZJCUsZLOJGnRAzCRpK8eMzMbwar17fQFRR1qcaDhqnr+EngDSZ/5X8qa3wF8JI8xmZlVjOa1GaQySfwRcR1Jt8qvjIgfFTAmM7OK0bIuw8kzJzCxvrbYoRyQa6ue1wALs9f3k7tmZsPr7QtWtma4eOnsYodyiFwS/y0kA6+0APvzG46ZWeVYs7WDjv09JXVjF3JL/HMj4sV5j8TMrMI0r01H3CqxxJ9Lc87/J+mZeY/EzKzCtLRmmN4whvlTS6shZC5X/OcCb5D0JElVj4CIiDPyGpmZWZlrac3QtGAK/b0al4pcEv9L8h6FmVmF2dqxj3U79vDXZ5deZ8a5DMTSCiBpBkl3DWZmNoIV6cAry0tg4JWBcumP/xJJa4AngTuBtcAv8hyXmVlZa16boa6mitNmTyx2KE+Ty83dTwJnA49GxPHAhcC9eY3KzKzMNbdmWDJ3EmNqqosdytPkkvi7I2I7UCWpKiLuAJpG2kjSPEl3SHpY0kOS3p3Ov0bSRkmr0tdLj/I7mJmVlH3dvTy0aWdJddOQLZebu+2SGoC7gO9J2gp05rBdD/C+iFghaQLQIun2dNmXI+ILRxaymVlpu3/DTrp7g6YFpTHwykC5XPFfCuwB/h74b+Bx4OKRNoqIzRGxIv3cAawm6e3TzKyitbSW5oNb/XJJ/DOAuojoSTtu+yZwWMPISFoInAn8IZ31Dkn3S/q2pEFLRtLVkpolNbe1tR3O4czMiqqldQcnTB/P1PF1xQ5lULkk/h8AfVnTvem8nKTVRD8C3hMRu4B/B04ElgKbgS8Otl1EXBsRTRHR1NjYmOvhzMyKKiJoac2U7NU+5Jb4ayKiq38i/ZzTaUxSLUnS/15E/DjdfktE9EZEH8mvh7MOP2wzs9L0xLZOMnu6yz7xt0m6pH9C0qXAtpE2UvKM8reA1RHxpaz5s7JW+yvgwdzDNTMrbS1px2xNJfjgVr9cWvW8laQ1z9fS6Q3AX+ew3XPT9R6QtCqd9xHgSklLgSB5GOwthxWxmVkJa2nNMHlcLSdMbyh2KEMaNvFLqgbeFhFnp3X1RMTuXHYcEb/n4Di92X5+2FGamZWJ5tYdLJs/haqq0uqYLduwVT0R0UvSOycRsTvXpG9mNhplOrt4vK2zpOv3IbeqnpWSbiVpyXPgwa3+m7VmZpZYsa602+/3yyXx1wPbgednzQvAid/MLEtza4aaKrFk7uRihzKsXLplfmMhAjEzK3ctrRlOmzOJsXWl1zFbtly6ZV4s6deSHkynz5D0sfyHZmZWPrp6+rhvfTvL55d2NQ/k1o7/m8A/AN0AEXE/8Op8BmVmVm4e2rST/T19Jd1+v18uiX9cRPxxwLyefARjZlauSr1jtmy5JP5tkk4kuaGLpMtI+tgxM7NUS2uGuVPGMnNi6Y9Qm0urnrcD1wKnSNpIMgTja/MalZlZGYkImlszPPfEacUOJSe5tOp5AniBpPFAVdq3vpmZpTZk9tLWsb8sqnkgt1Y90yR9Bfgd8FtJ/yKpPE5rZmYF0Ny6A4DlJTri1kC51PHfALQBrwQuSz/fmM+gzMzKSUtrhoYxNZx83GGNUVU0udTxz4qIT2ZNf0rSFfkKyMys3DSvzXDm/MlUl3DHbNlyueL/laRXS6pKX5cDv8x3YGZm5aBjXzd/3tJRNvX7kFvifzNwPbA/fd0AvEVSh6Rd+QzOzKzUrVzXTkR5tN/vl0urnvKotDIzK4KW1gxVgqXzSrtjtmy5XPGbmdkQWloznHzcRCbU1xY7lJw58ZuZHaGe3j5WrsvQVEbVPODEb2Z2xP68pYPOrt6yqt+H3Jpz9o+9OzN7/YhYl6+gzMzKQTl1zJZtxMQv6Z3Ax4EtQF86O4Az8hiXmVnJa2nNMHPiGOZOGVvsUA5LLlf87wZOjojt+Q7GzKycNK/NsHzBFKTyeHCrXy51/OuBnfkOxMysnDy1cx8b2/eWTf882XK54n+CpHO220ge4AIgIr6Ut6jMzEpcudbvQ25X/OuA24E6YELWa1iS5km6Q9LDkh6S9O50/lRJt0tak76XX6mZ2ajX3LqD+toqTps9sdihHLZcntz9xBHuuwd4X0SskDQBaJF0O/AG4NcR8TlJHwY+DHzoCI9hZlYUK1ozLJk7mdrq8msVP2Til/R/IuI9kn5KOuxitoi4ZLgdR8Rm0iEaI6JD0mpgDnApcH662nXAb3HiN7Mysrerl4c27eLq804odihHZLgr/u+m71842oNIWgicCfwBmJmeFACeInk+YLBtrgauBpg/f/7RhmBmdsysWt9OT1/QtLA8a6qHTPwR0ZK+33k0B5DUAPwIeE9E7Mpu9hQRIelpvybSZdeSjPVLU1PToOuYmRXDinXJjd1l88sz8ee1ckpSLUnS/15E/DidvUXSrHT5LGBrPmMwMzvWmtfu4KQZDUweV1fsUI5I3hK/kkv7bwGrBzT9vBW4Kv18FXBLvmIwMzvW+vqCFevay65jtmw59dUDIGlcROw5jH0/F/hr4AFJq9J5HwE+B9wk6U1AK3D5YezTzKyoHm/bzc693Syr5MQv6TnAfwANwHxJS4C3RMTfDbddRPweGOo55gsPN1Azs1LQnD64Vc5X/LlU9XwZ+EtgO0BE3Aecl8+gzMxKVUtrhqnj6zh++vhih3LEcqrjj4j1A2b15iEWM7OS19KaYdn88uuYLVtOnbSl1T0hqVbS+4HVeY7LzKzkbN+9nye3dZZt+/1+uST+twJvJ3nqdiOwFBi2ft/MrBKVc8ds2XJp1XNyRLw2e4ak5wJ35yckM7PS1NKaobZaPHPOpGKHclRyueL/ao7zzMwqWktrhtPnTKK+trrYoRyV4TppOwd4DtAo6b1ZiyYC5f2tzcwO0/6eXu7fuJOrzllQ7FCO2nBVPXUkbfdrOLT//V3AZfkMysys1Dy4cRddPX1lX78Pw3fSdidwp6TvRERrAWMyMys5La07AMpyqMWBcrm5u0fS54HTgPr+mRHx/LxFZWZWYprXZlgwbRyNE8YUO5SjlsvN3e8BjwDHA58A1gJ/ymNMZmYlJSJYsS7D8jLthnmgXBL/tIj4FtAdEXdGxN8Avto3s1Gjdfsetu3uYnmZP7jVL5eqnu70fbOklwGbgPKv5DIzy1GlPLjVL5fE/ylJk4D3kbTfnwj8fV6jMjMrIc2tGSbU17B4xoSRVy4DwyZ+SdXAooj4GbATuKAgUZmZlZCW1h0smz+Fqqry7Zgt27B1/BHRC1xZoFjMzErOzr3dPLpld8VU80BuVT13S/oacCPQ2T8zIlbkLSozsxLRP7B6OQ+8MlAuiX9p+v5PWfMCt+wxs1FgRWuG6iqxZN7kYodyzIyY+CPC9fpmNmo1r83wjFkTGD8m5yHKS15OI3CZmY1G3b19rFrfTlMFdNOQzYnfzGwIj2zuYG93L8sqqH4fRkj8kqrSYRfNzEad5rRjtkq6sQsjN+fsA/61QLGYmZWUltYMsybVM3vy2GKHckzlUtXza0mv1GEOKS/p25K2Snowa941kjZKWpW+XnrYEZuZFUhLa6ai2u/3yyXxvwX4AdAlaZekDkm7ctjuO8CLB5n/5YhYmr5+fhixmpkVzMb2vWzeua/iqnkgt+acR9Q5RUTcJWnhkWxrZlZsBztmq6wWPZBjqx5Jl0j6Qvq66CiP+Q5J96dVQUOeSiVdLalZUnNbW9tRHtLM7PC0rN3B2NpqnjGrMjpmyzZi4pf0OeDdwMPp692SPnuEx/t34ESSp4E3A18casWIuDYimiKiqbGx8QgPZ2Z2ZFrWZVg6bzI11ZXX6j2Xb/RS4IUR8e2I+DZJvf3LjuRgEbElInrT1kLfBM46kv2YmeVT5/4eVm/uoKlCBl4ZKNdTWXYnFZOO9GCSZmVN/hXw4FDrmpkVy6r17fT2RUW26IHcOmn7DLBS0h2AgPOAD4+0kaTvA+cD0yVtAD4OnC9pKUknb2tJWgyZmZWUltYMEpxZIWPsDjTSQCxVQB9wNvCsdPaHIuKpkXYcEYP14/+tw47QzKzAmlszLJ4xgUlja4sdSl7k8uTuByNic0Tcmr5GTPpmZuVqY/teVrZmKq5/nmy51PH/j6T3S5onaWr/K++RmZkV2C8feoqX/svv6IvgVU1zix1O3uRSx39F+v72rHkBnHDswzEzK7x93b189uerue6eVk6fM5GvXbmMhdPHFzusvMmljv/DEXFjgeIxMyuoJ9p2847rV/Lw5l286dzj+eCLT2ZMTXWxw8qrYRN/RPRJ+gDJeLtmZhXlxys28LGbH6SupopvXdXEhc+YWeyQCiKXqp7/kfR+nj7Y+o68RWVmlked+3v4x1se5McrNnLWwqn8y5VLmTWpsrpeHo7r+M1sVHlo007eef1KntzeybsvXMQ7n39SRXbLMJxceuc8vhCBmJnlU0Twf+9p5dO3rWbK+Fqu/9uzOefEacUOqyiGPM1J+mDW51cNWPaZfAZlZnYste/p4i3fbeHjtz7Ec0+axs/f9bxRm/Rh+Hb8r876/A8Dlg02wIqZWclpXruDl33l99zx56187GXP4FtXPYtpDWOKHVZRDVfVoyE+DzZtZlZSevuCr9/5OF+6/VHmTB7LD9/6HJbMmzzyhqPAcIk/hvg82LSZWcnY2rGPv79xFXc/tp2LzpjFZ17xTCbWV2a/O0diuMS/JB1bV8DYrHF2BdTnPTIzsyNw16NtvPemVeze38PnXvFMrnjWPCRXUmQbMvFHRGU/umZmFaW7t48v/upRvn7n4yye2cD1bz6bxTMrb9jEYyGXdvxmZiVt/Y49vOuGlaxc186VZ83nf110KmPrfO06FCd+Mytrv3hgMx/80f0Q8NUrz+TiJbOLHVLJc+I3s7K0r7uXT932MP917zqWzJ3EV69cxvxp44odVllw4jezsvPY1g7ecf1KHnmqg6vPO4H3v+hk6mpGV7cLR8OJ38zKRkTwg5YNfPyWhxhbV81/vvFZXHDyjGKHVXac+M2sLOze38NHf/IAt6zaxDknTOP/vHopMye6ZfmRcOI3s5L34MadvOP6FazbsYf3vnAxb7/gJKqr3Db/SDnxm1nJigj+8+61fPYXq5neMIYbrj6Hs473kN9Hy4nfzEpSprOLD/zwPv5n9VZe8IyZfP6yM5gyvq7YYVWEvCV+Sd8GLgK2RsTp6bypJCN5LQTWApdHRCZfMZhZefrjkzt41/dXsr1zP//rolN543MXutuFYyif7Z++w9O7b/4w8OuIWAT8Op02MwOSHjW/8us1vPrae6ivreLHb3suf3Pu8U76x1jervgj4i5JCwfMvhQ4P/18HfBb4EP5isHMysOOzi5ue2AzN/1pPQ9s3MnLl87mU3/1TBrGuDY6HwpdqjMjYnP6+SlgyCHtJV0NXA0wf/78AoRmZoW0t6uX21dv4ZaVG7nz0TZ6+oLFMxv44quW8Iplc3yVn0dFO51GREgasl//iLgWuBagqanJ/f+bVYCe3j7ufnw7t6zcyC8feorOrl5mTarnTecez6VL5/CMWROc8Aug0Il/i6RZEbFZ0ixga4GPb2YFFhHct2EnN6/cyM/u38S23V1MrK/h4iWzuXTpHJ59/FSq3Ca/oAqd+G8FrgI+l77fUuDjm1mBPLmtk5tXbuTW+zbx5LZO6mqquPCUGVy6dA4XnNLImBp3m1ws+WzO+X2SG7nTJW0APk6S8G+S9CagFbg8X8c3s8Jr69jPT+/bxC2rNnLfhp1IcM4J03jbX5zIX55+HJPGevjDUpDPVj1XDrHownwd08wKb/f+Hn754FPcvGojdz+2jb6A02ZP5KMvfQYXL5nNcZPcn06pcVspMztsXT193PVoGzev2sj/rN7Cvu4+5k4Zy9vOP5GXL53DIg95WNKc+M0sJxFBS2uGm1dt5Lb7N5PZ082UcbVctnwuL186h+ULprhFTplw4jezYa3Z0sHNqzZyy6pNbMjspb62iheeehwvXzqb8xY3UlvtAVDKjRO/mT3NUzv3cet9G7l55SYe3ryLKsG5ixp57wsX86LTjvMTtWXOfz0zA2Dn3m7++8HN3LxyE/c+uZ0IWDJvMh+/+FQuOmM2jRPGFDtEO0ac+M2KaN32Pax+ahc9vUFPXx+9fUFPXxx8793Q4MQAAAwpSURBVO07dDp97zswPWB57+Dz+w7ZfsBxepP3J7d30tXTx/HTx/PuCxdx6dI5HD99fLGLyPLAid+sgDr2dXPP49u5a00bv1uzjdbte45oPzVVorpKB9+rq6hS9nT28qqnrV9bXUV97aHLz100nUuWzOaMuZN8k7bCOfGb5VFvX/Dgxp3c9WiS6Fesy9DTF4yrq+Y5J07jb557PMvmT6G+tipNzFVUVycJ+UAir85K6FVVVAknZjsqTvxmx9jmnXv53aPbuGtNG3c/to3Mnm4AnjlnElefdwLnLW5k2fwp1NW4NYwVhxO/2VHa29XLH57czu/WbOOuR9tYs3U3ADMmjOH5p8zkvMXTOfek6Uxr8M1RKw1O/GaHKSJ45KkOfremjbse3cYf1+6gq6ePupoqnn38VC5vmsfzFk/n5JnuYthKkxO/WQ627d7P3Y9t4860rr6tYz8Ai2c28PqzF/C8xY08+/ip1Ne6x0krfU78VhARwcb2vazZupu12zoZU1PN5HG1yWtsHVPG1zJlXF3JJM6unj5aWjNp65s2Hty4C4Ap42o5d1Ejz1s0nfMWNboDMitLTvx2TPX1BZt27mXNlt2s2drBo1t2s2ZLB49t3U1nV++I24+pqWLKuLoDJ4WDn+uYkp4kDpkeV8eksbVHfaM0InhyW+eB1jf3PLGdPV291FSJZQum8P4XLea8xY2cNnsS1R40xMqcE78dkb6+5Ar+sa27eXRLB2u2Jgl+zdbd7MlK8I0TxrBoRgOvaprHSTMaWDxzAic0jqenN8js6SKzp4ude7rJ7OmmfW8X7Xu6yXR20b63m/Y9XTy2dXeybE8XPX1Dj8DZMKaGSWNrD/xymDR2kJNG+nlyukwibVOf3JTd2L4XgIXTxvHKZXM5b3EjZ58wlQn17kPeKosT/xD6+oLuvj66evro7g26e5PPXb19Bz4n75HMG7Csf153b7K8rrqKKePrmJompqnj65gyvo4JY2pK+gZgf4Jfs7WDNVt2J1fwW5Mr+IEJfvHMBi5vmseimUmCXzSjgcnj6obc9+FUk0QEnV29ZDq72Lm3Oz1pdLMzfW9PTw7t6bINmb2070nWHeZ8ASQnjeecOI23nX8i5y1qZP60cTnHZVaOKjrx39S8nrsebaO7N03AaULuT9oHE3g8bX53b2HGd6+pUnJCGJfUc08dX3fgxND/yj5RTB1Xx9i6Y18Pnp3gH92SXMU/tnX30xL8jAljWDxzApc3zUuS+8yGERP8sSCJhjE1NIypYd5hbNfXF3Ts6znw66L/l0T7nm729/SxfMEUls6b7B4mbVSp6MS/MbOXhzftora6irqaKmqrRV1NFRPqa6g7MK/qwPK6dPmh8/q3qz6wfV3W8uRd1FVXU1uTPApfl71tetyunj4ynd3s2NNFprOL7Z3Je//0js4kMf35qQ4ye5Kr1hji3FNfW5WeKAacGMYlvyimjh9z4CQydVwdk8fVHagD70/wj27pOHD1vmZLkuD3dj89wV/xrHksmjGBxTMbWDRjApPGlVe1R1WVmDSulknjalmI+50xA1AMlV1KSFNTUzQ3Nxc7jILq7Qt27e0+5MSw45ATRXJy6D9h7OjsomNfz5D7mzCmhknjatm+u+uQBD9zYpLg++vfF80ozwRvZk8nqSUimgbOr+gr/nJWnVYBTRlfB425bdPV00f73q7kl0XWCaH/F0b7ni6mjh+T1sE3cFKjE7zZaOTEX0HqaqqYMaGeGRPcttzMhuY7WmZmo4wTv5nZKOPEb2Y2yhSljl/SWqAD6AV6BrvrbGZm+VHMm7sXRMS2Ih7fzGxUclWPmdkoU6zEH8CvJLVIunqwFSRdLalZUnNbW1uBwzMzq1zFSvznRsQy4CXA2yWdN3CFiLg2IpoioqmxMccnmMzMbERF77JB0jXA7oj4wjDrtAGtR3iI6YDvJRzk8jjIZXEol8ehKqE8FkTE066cC35zV9J4oCoiOtLPLwL+abhtBgv8MI7X7FZDB7k8DnJZHMrlcahKLo9itOqZCfwk7YO+Brg+Iv67CHGYmY1KBU/8EfEEsKTQxzUzs8RoaM55bbEDKDEuj4NcFodyeRyqYsuj6Dd3zcyssEbDFb+ZmWVx4jczG2UqJvFLerGkP0t6TNKHB1k+RtKN6fI/SFpY+CgLI4eyeK+khyXdL+nXkhYUI85CGak8stZ7paSQVJFN+PrlUh6SLk//jTwk6fpCx1goOfxfmS/pDkkr0/8vLy1GnMdcRJT9C6gGHgdOAOqA+4BTB6zzd8DX08+vBm4sdtxFLIsLgHHp57dValnkWh7pehOAu4B7gaZix13kfx+LgJXAlHR6RrHjLmJZXAu8Lf18KrC22HEfi1elXPGfBTwWEU9ERBdwA3DpgHUuBa5LP/8QuFDpwwQVZsSyiIg7ImJPOnkvMLfAMRZSLv82AD4J/DOwr5DBFUEu5fFm4F8jIgMQEVsLHGOh5FIWAUxMP08CNhUwvryplMQ/B1ifNb0hnTfoOhHRA+wEphUkusLKpSyyvQn4RV4jKq4Ry0PSMmBeRNxWyMCKJJd/H4uBxZLulnSvpBcXLLrCyqUsrgFeJ2kD8HPgnYUJLb882PooJul1QBPwF8WOpVgkVQFfAt5Q5FBKSQ1Jdc/5JL8G75L0zIhoL2pUxXEl8J2I+KKkc4DvSjo9IvqKHdjRqJQr/o3AvKzpuem8QdeRVEPys217QaIrrFzKAkkvAD4KXBIR+wsUWzGMVB4TgNOB36Yjw50N3FrBN3hz+fexAbg1Iroj4kngUZITQaXJpSzeBNwEEBH3APUknbeVtUpJ/H8CFkk6XlIdyc3bWwescytwVfr5MuA3kd6xqTAjloWkM4FvkCT9Sq2/7TdseUTEzoiYHhELI2IhyT2PSyKiuTjh5l0u/1duJrnaR9J0kqqfJwoZZIHkUhbrgAsBJD2DJPGX/QAhFZH40zr7dwC/BFYDN0XEQ5L+SdIl6WrfAqZJegx4LzBks75ylmNZfB5oAH4gaZWkgf/YK0aO5TFq5FgevwS2S3oYuAP4QERU3K/jHMvifcCbJd0HfB94QyVcMLrLBjOzUaYirvjNzCx3TvxmZqOME7+Z2SjjxG9mNso48ZuZjTJO/FbWJPWmTVIfknSfpPelT+MWK573SBo3xLLfpj1B3ifpT5KWjrCvyZL+Lj+R2mjmxG/lbm9ELI2I04AXAi8BPj5wpfRp7UJ4DzBo4k+9NiKWAP9G8jzFcCaT9Cprdkw58VvFSJ9Cvhp4hxJvkHSrpN8Av5Y0VdLNab/q90o6A0DSNZK+K+keSWskvTmdL0mfl/SgpAckXZHOP1/Sz/qPK+lr6bHeBcwG7pB0xwjh3kPaIZikhnRchBXpcfp7iPwccGL6i+bz6bofSH8t3C/pE8es8GxUcSdtVlEi4glJ1cCMdNYy4IyI2CHpq8DKiHi5pOcD/xfor245g6SfnvHASkm3Aeeky5eQ9M/yJ0l3DXPsr0h6L3BBRGwbIdQXk3SNAElX0H8VEbvSLhLuTZ+m/jBwekQsBZD0IpI+c84CRNKn0HkRMWRMZoNx4rdKd3tE7Eg/nwu8EiAifiNpmqT+vtZviYi9wN70av2sdP3vR0QvsEXSncCzgF1HEc/30n5hGjh40hHwGUnnAX0kvwRmDrLti9LXynS6geRE4MRvh8WJ3yqKpBOAXqC/87nOHDcd2HfJcH2Z9HBoNWl9jscAeC3QQlK//1XgFem8RmB5RHSnvYQOtk8Bn42IbxzG8cyexnX8VjEkNQJfB742REdavyNJskg6H9gWEf1X75dKqpc0jaRnyj+l618hqTrd93nAH4FW4FQl4zhPJu29MdVB0tXzkNLY/hE4W9IpJF2Eb02T/gVA/xjIA/f1S+BvJDWk32GOpBmYHSZf8Vu5GytpFVBLciX+XZKBVQZzDfBtSfcDezjYTTfA/SQ9UU4HPhkRmyT9hKSe/z6SXwAfjIinACTdBDwIPMnBqhdIxmj9b0mbIuKCoYKOiL2Svgh8APgQ8FNJDwDNwCPpOtuVjIL1IPCLiPhA2jXwPUpGDd0NvI6Dv27McuLeOW3Uk3QNsDsivlDsWMwKwVU9ZmajjK/4zcxGGV/xm5mNMk78ZmajjBO/mdko48RvZjbKOPGbmY0y/x8F3b2N/yR3qgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(reg,valerr)\n",
    "plt.title('Error Rate vs Dropout Rate' )\n",
    "plt.xlabel('Dropout Rate')\n",
    "plt.ylabel('Error rate in percantage')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "X6hHeLONnFZO",
    "outputId": "100ee8ab-5eed-4297-b0cf-bed74e9d9b59"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 6.5   6.63  6.22  7.09  6.71  8.15  8.23 10.09 13.94 36.79]\n"
     ]
    }
   ],
   "source": [
    "# Dropout at 0.2 appears to be the best. Choosing it for applying in future\n",
    "valerr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jdLeI9oo26_1"
   },
   "source": [
    "**Regularisation improved the initial network with 0.2 but has decreased model performance after some fluctuation. The optimal value apears to be 0.2**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "txrVtWDNnzyB"
   },
   "source": [
    "### 3.  Learning curve experiment:  ***12 marks***\n",
    "\n",
    "Now train your model (with the amount of regularisation you selected) on different amounts of training data. \n",
    "\n",
    "Plot the performance (loss/error-rate) of the model on the validation set against the size of the training set. \n",
    "This is typically best done on a log-log plot. \n",
    "\n",
    "Describe the approximate relationship between the training set size and loss / error-rate.  Does the network performance appear to improve as some power of the amount of data in the training set?  If so, by what power ? \n",
    "\n",
    "For example, a very good rate of improvement is for error-rate to be proportional to $\\dfrac{1}{\\sqrt{n}}$ where $n$ is the training set size.  For your model, the rate of improvement of validation error with training set size may not be as fast as this (or it may, who knows?)  The aim of this exercise is to find out what it actually is. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "dhuZD11WnzyC",
    "outputId": "1d8f9868-7004-427e-f158-cdce83af9b46"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda\n",
      "Error rate on train set: 55.000000 %\n",
      "Epoch 1 loss: 2.196109 \n",
      "Error rate on train set: 34.000000 %\n",
      "Epoch 2 loss: 1.736537 \n",
      "Error rate on train set: 26.400000 %\n",
      "Epoch 3 loss: 1.164568 \n",
      "Error rate on train set: 18.800000 %\n",
      "Epoch 4 loss: 0.795980 \n",
      "Error rate on train set: 15.600000 %\n",
      "Epoch 5 loss: 0.562055 \n",
      "Error rate on train set: 11.800000 %\n",
      "Epoch 6 loss: 0.449399 \n",
      "Error rate on train set: 10.800000 %\n",
      "Epoch 7 loss: 0.360325 \n",
      "Error rate on train set: 9.400000 %\n",
      "Epoch 8 loss: 0.300278 \n",
      "Error rate on train set: 5.600000 %\n",
      "Epoch 9 loss: 0.221560 \n",
      "Error rate on train set: 4.200000 %\n",
      "Epoch 10 loss: 0.206207 \n",
      "Error rate on train set: 2.800000 %\n",
      "Epoch 11 loss: 0.163322 \n",
      "Error rate on train set: 1.800000 %\n",
      "Epoch 12 loss: 0.126651 \n",
      "Error rate on train set: 2.800000 %\n",
      "Epoch 13 loss: 0.112708 \n",
      "Error rate on train set: 1.800000 %\n",
      "Epoch 14 loss: 0.105737 \n",
      "Error rate on train set: 1.200000 %\n",
      "Epoch 15 loss: 0.087044 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 16 loss: 0.079295 \n",
      "Error rate on train set: 0.400000 %\n",
      "Epoch 17 loss: 0.054726 \n",
      "Error rate on train set: 0.400000 %\n",
      "Epoch 18 loss: 0.049696 \n",
      "Error rate on train set: 0.800000 %\n",
      "Epoch 19 loss: 0.051013 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 20 loss: 0.038653 \n",
      "Error Rate on test set: 16.880000 %\n",
      "cuda\n",
      "Error rate on train set: 52.600000 %\n",
      "Epoch 1 loss: 2.078407 \n",
      "Error rate on train set: 27.200000 %\n",
      "Epoch 2 loss: 1.198946 \n",
      "Error rate on train set: 17.100000 %\n",
      "Epoch 3 loss: 0.665845 \n",
      "Error rate on train set: 13.700000 %\n",
      "Epoch 4 loss: 0.496620 \n",
      "Error rate on train set: 11.400000 %\n",
      "Epoch 5 loss: 0.409796 \n",
      "Error rate on train set: 8.100000 %\n",
      "Epoch 6 loss: 0.329797 \n",
      "Error rate on train set: 7.800000 %\n",
      "Epoch 7 loss: 0.269244 \n",
      "Error rate on train set: 6.200000 %\n",
      "Epoch 8 loss: 0.236142 \n",
      "Error rate on train set: 5.100000 %\n",
      "Epoch 9 loss: 0.205472 \n",
      "Error rate on train set: 4.400000 %\n",
      "Epoch 10 loss: 0.166724 \n",
      "Error rate on train set: 2.700000 %\n",
      "Epoch 11 loss: 0.141376 \n",
      "Error rate on train set: 2.700000 %\n",
      "Epoch 12 loss: 0.126144 \n",
      "Error rate on train set: 2.300000 %\n",
      "Epoch 13 loss: 0.109020 \n",
      "Error rate on train set: 2.300000 %\n",
      "Epoch 14 loss: 0.090470 \n",
      "Error rate on train set: 1.500000 %\n",
      "Epoch 15 loss: 0.078983 \n",
      "Error rate on train set: 1.500000 %\n",
      "Epoch 16 loss: 0.064260 \n",
      "Error rate on train set: 1.500000 %\n",
      "Epoch 17 loss: 0.058091 \n",
      "Error rate on train set: 1.200000 %\n",
      "Epoch 18 loss: 0.047968 \n",
      "Error rate on train set: 0.600000 %\n",
      "Epoch 19 loss: 0.051955 \n",
      "Error rate on train set: 0.500000 %\n",
      "Epoch 20 loss: 0.038592 \n",
      "Error Rate on test set: 12.880000 %\n",
      "cuda\n",
      "Error rate on train set: 39.250000 %\n",
      "Epoch 1 loss: 1.588028 \n",
      "Error rate on train set: 18.400000 %\n",
      "Epoch 2 loss: 0.654405 \n",
      "Error rate on train set: 13.000000 %\n",
      "Epoch 3 loss: 0.463726 \n",
      "Error rate on train set: 9.600000 %\n",
      "Epoch 4 loss: 0.334401 \n",
      "Error rate on train set: 7.950000 %\n",
      "Epoch 5 loss: 0.288794 \n",
      "Error rate on train set: 7.050000 %\n",
      "Epoch 6 loss: 0.250239 \n",
      "Error rate on train set: 5.800000 %\n",
      "Epoch 7 loss: 0.208685 \n",
      "Error rate on train set: 4.300000 %\n",
      "Epoch 8 loss: 0.188822 \n",
      "Error rate on train set: 4.200000 %\n",
      "Epoch 9 loss: 0.143048 \n",
      "Error rate on train set: 3.650000 %\n",
      "Epoch 10 loss: 0.130496 \n",
      "Error rate on train set: 3.000000 %\n",
      "Epoch 11 loss: 0.118778 \n",
      "Error rate on train set: 2.150000 %\n",
      "Epoch 12 loss: 0.089042 \n",
      "Error rate on train set: 1.650000 %\n",
      "Epoch 13 loss: 0.073862 \n",
      "Error rate on train set: 2.250000 %\n",
      "Epoch 14 loss: 0.079144 \n",
      "Error rate on train set: 1.400000 %\n",
      "Epoch 15 loss: 0.067680 \n",
      "Error rate on train set: 1.200000 %\n",
      "Epoch 16 loss: 0.056432 \n",
      "Error rate on train set: 0.950000 %\n",
      "Epoch 17 loss: 0.047560 \n",
      "Error rate on train set: 0.550000 %\n",
      "Epoch 18 loss: 0.040522 \n",
      "Error rate on train set: 0.600000 %\n",
      "Epoch 19 loss: 0.038334 \n",
      "Error rate on train set: 0.450000 %\n",
      "Epoch 20 loss: 0.035744 \n",
      "Error Rate on test set: 9.820000 %\n",
      "cuda\n",
      "Error rate on train set: 32.775000 %\n",
      "Epoch 1 loss: 1.179147 \n",
      "Error rate on train set: 11.375000 %\n",
      "Epoch 2 loss: 0.412380 \n",
      "Error rate on train set: 8.600000 %\n",
      "Epoch 3 loss: 0.297415 \n",
      "Error rate on train set: 6.575000 %\n",
      "Epoch 4 loss: 0.242195 \n",
      "Error rate on train set: 5.150000 %\n",
      "Epoch 5 loss: 0.191720 \n",
      "Error rate on train set: 4.425000 %\n",
      "Epoch 6 loss: 0.164902 \n",
      "Error rate on train set: 3.550000 %\n",
      "Epoch 7 loss: 0.137531 \n",
      "Error rate on train set: 2.700000 %\n",
      "Epoch 8 loss: 0.113923 \n",
      "Error rate on train set: 2.650000 %\n",
      "Epoch 9 loss: 0.098270 \n",
      "Error rate on train set: 1.975000 %\n",
      "Epoch 10 loss: 0.077832 \n",
      "Error rate on train set: 1.800000 %\n",
      "Epoch 11 loss: 0.071921 \n",
      "Error rate on train set: 1.250000 %\n",
      "Epoch 12 loss: 0.058384 \n",
      "Error rate on train set: 1.075000 %\n",
      "Epoch 13 loss: 0.045921 \n",
      "Error rate on train set: 0.875000 %\n",
      "Epoch 14 loss: 0.046455 \n",
      "Error rate on train set: 0.850000 %\n",
      "Epoch 15 loss: 0.037567 \n",
      "Error rate on train set: 0.850000 %\n",
      "Epoch 16 loss: 0.033696 \n",
      "Error rate on train set: 0.875000 %\n",
      "Epoch 17 loss: 0.031099 \n",
      "Error rate on train set: 0.525000 %\n",
      "Epoch 18 loss: 0.024699 \n",
      "Error rate on train set: 0.450000 %\n",
      "Epoch 19 loss: 0.026483 \n",
      "Error rate on train set: 0.675000 %\n",
      "Epoch 20 loss: 0.022602 \n",
      "Error Rate on test set: 7.440000 %\n",
      "cuda\n",
      "Error rate on train set: 22.075000 %\n",
      "Epoch 1 loss: 0.830415 \n",
      "Error rate on train set: 9.350000 %\n",
      "Epoch 2 loss: 0.330292 \n",
      "Error rate on train set: 6.762500 %\n",
      "Epoch 3 loss: 0.243310 \n",
      "Error rate on train set: 5.400000 %\n",
      "Epoch 4 loss: 0.193842 \n",
      "Error rate on train set: 4.162500 %\n",
      "Epoch 5 loss: 0.149556 \n",
      "Error rate on train set: 3.675000 %\n",
      "Epoch 6 loss: 0.129976 \n",
      "Error rate on train set: 2.875000 %\n",
      "Epoch 7 loss: 0.104651 \n",
      "Error rate on train set: 2.462500 %\n",
      "Epoch 8 loss: 0.091249 \n",
      "Error rate on train set: 2.187500 %\n",
      "Epoch 9 loss: 0.075979 \n",
      "Error rate on train set: 1.425000 %\n",
      "Epoch 10 loss: 0.061521 \n",
      "Error rate on train set: 1.425000 %\n",
      "Epoch 11 loss: 0.057538 \n",
      "Error rate on train set: 1.025000 %\n",
      "Epoch 12 loss: 0.045905 \n",
      "Error rate on train set: 0.962500 %\n",
      "Epoch 13 loss: 0.039372 \n",
      "Error rate on train set: 0.912500 %\n",
      "Epoch 14 loss: 0.042470 \n",
      "Error rate on train set: 0.825000 %\n",
      "Epoch 15 loss: 0.031600 \n",
      "Error rate on train set: 0.625000 %\n",
      "Epoch 16 loss: 0.024585 \n",
      "Error rate on train set: 0.600000 %\n",
      "Epoch 17 loss: 0.025890 \n",
      "Error rate on train set: 0.550000 %\n",
      "Epoch 18 loss: 0.018964 \n",
      "Error rate on train set: 0.500000 %\n",
      "Epoch 19 loss: 0.018352 \n",
      "Error rate on train set: 0.525000 %\n",
      "Epoch 20 loss: 0.016016 \n",
      "Error Rate on test set: 5.020000 %\n",
      "cuda\n",
      "Error rate on train set: 16.768750 %\n",
      "Epoch 1 loss: 0.607496 \n",
      "Error rate on train set: 6.937500 %\n",
      "Epoch 2 loss: 0.248321 \n",
      "Error rate on train set: 5.037500 %\n",
      "Epoch 3 loss: 0.179463 \n",
      "Error rate on train set: 3.606250 %\n",
      "Epoch 4 loss: 0.135077 \n",
      "Error rate on train set: 2.881250 %\n",
      "Epoch 5 loss: 0.107480 \n",
      "Error rate on train set: 2.356250 %\n",
      "Epoch 6 loss: 0.085656 \n",
      "Error rate on train set: 1.862500 %\n",
      "Epoch 7 loss: 0.073871 \n",
      "Error rate on train set: 1.518750 %\n",
      "Epoch 8 loss: 0.060518 \n",
      "Error rate on train set: 1.343750 %\n",
      "Epoch 9 loss: 0.049723 \n",
      "Error rate on train set: 1.175000 %\n",
      "Epoch 10 loss: 0.040249 \n",
      "Error rate on train set: 1.068750 %\n",
      "Epoch 11 loss: 0.039168 \n",
      "Error rate on train set: 1.087500 %\n",
      "Epoch 12 loss: 0.040715 \n",
      "Error rate on train set: 0.762500 %\n",
      "Epoch 13 loss: 0.028014 \n",
      "Error rate on train set: 0.850000 %\n",
      "Epoch 14 loss: 0.028442 \n",
      "Error rate on train set: 0.712500 %\n",
      "Epoch 15 loss: 0.026014 \n",
      "Error rate on train set: 0.668750 %\n",
      "Epoch 16 loss: 0.024588 \n",
      "Error rate on train set: 0.681250 %\n",
      "Epoch 17 loss: 0.026718 \n",
      "Error rate on train set: 0.587500 %\n",
      "Epoch 18 loss: 0.024242 \n",
      "Error rate on train set: 0.662500 %\n",
      "Epoch 19 loss: 0.022166 \n",
      "Error rate on train set: 0.575000 %\n",
      "Epoch 20 loss: 0.019199 \n",
      "Error Rate on test set: 3.590000 %\n",
      "cuda\n",
      "Error rate on train set: 11.971875 %\n",
      "Epoch 1 loss: 0.430696 \n",
      "Error rate on train set: 4.925000 %\n",
      "Epoch 2 loss: 0.175660 \n",
      "Error rate on train set: 3.353125 %\n",
      "Epoch 3 loss: 0.124421 \n",
      "Error rate on train set: 2.650000 %\n",
      "Epoch 4 loss: 0.096576 \n",
      "Error rate on train set: 2.090625 %\n",
      "Epoch 5 loss: 0.076388 \n",
      "Error rate on train set: 1.687500 %\n",
      "Epoch 6 loss: 0.064662 \n",
      "Error rate on train set: 1.550000 %\n",
      "Epoch 7 loss: 0.053374 \n",
      "Error rate on train set: 1.300000 %\n",
      "Epoch 8 loss: 0.048839 \n",
      "Error rate on train set: 1.084375 %\n",
      "Epoch 9 loss: 0.040768 \n",
      "Error rate on train set: 1.084375 %\n",
      "Epoch 10 loss: 0.036748 \n",
      "Error rate on train set: 0.900000 %\n",
      "Epoch 11 loss: 0.033263 \n",
      "Error rate on train set: 0.843750 %\n",
      "Epoch 12 loss: 0.031796 \n",
      "Error rate on train set: 0.771875 %\n",
      "Epoch 13 loss: 0.028329 \n",
      "Error rate on train set: 0.762500 %\n",
      "Epoch 14 loss: 0.026242 \n",
      "Error rate on train set: 0.703125 %\n",
      "Epoch 15 loss: 0.027384 \n",
      "Error rate on train set: 0.615625 %\n",
      "Epoch 16 loss: 0.022788 \n",
      "Error rate on train set: 0.571875 %\n",
      "Epoch 17 loss: 0.019953 \n",
      "Error rate on train set: 0.637500 %\n",
      "Epoch 18 loss: 0.024968 \n",
      "Error rate on train set: 0.653125 %\n",
      "Epoch 19 loss: 0.024037 \n",
      "Error rate on train set: 0.637500 %\n",
      "Epoch 20 loss: 0.019768 \n",
      "Error Rate on test set: 3.040000 %\n",
      "cuda\n",
      "Error rate on train set: 8.743333 %\n",
      "Epoch 1 loss: 0.320886 \n",
      "Error rate on train set: 3.611667 %\n",
      "Epoch 2 loss: 0.133010 \n",
      "Error rate on train set: 2.650000 %\n",
      "Epoch 3 loss: 0.094317 \n",
      "Error rate on train set: 2.161667 %\n",
      "Epoch 4 loss: 0.075778 \n",
      "Error rate on train set: 1.793333 %\n",
      "Epoch 5 loss: 0.065318 \n",
      "Error rate on train set: 1.533333 %\n",
      "Epoch 6 loss: 0.056510 \n",
      "Error rate on train set: 1.335000 %\n",
      "Epoch 7 loss: 0.049702 \n",
      "Error rate on train set: 1.220000 %\n",
      "Epoch 8 loss: 0.045208 \n",
      "Error rate on train set: 1.150000 %\n",
      "Epoch 9 loss: 0.039919 \n",
      "Error rate on train set: 0.975000 %\n",
      "Epoch 10 loss: 0.037326 \n",
      "Error rate on train set: 0.950000 %\n",
      "Epoch 11 loss: 0.032406 \n",
      "Error rate on train set: 0.910000 %\n",
      "Epoch 12 loss: 0.031892 \n",
      "Error rate on train set: 0.820000 %\n",
      "Epoch 13 loss: 0.030998 \n",
      "Error rate on train set: 0.841667 %\n",
      "Epoch 14 loss: 0.028652 \n",
      "Error rate on train set: 0.801667 %\n",
      "Epoch 15 loss: 0.027465 \n",
      "Error rate on train set: 0.740000 %\n",
      "Epoch 16 loss: 0.023959 \n",
      "Error rate on train set: 0.680000 %\n",
      "Epoch 17 loss: 0.022167 \n",
      "Error rate on train set: 0.693333 %\n",
      "Epoch 18 loss: 0.025904 \n",
      "Error rate on train set: 0.733333 %\n",
      "Epoch 19 loss: 0.023818 \n",
      "Error rate on train set: 0.606667 %\n",
      "Epoch 20 loss: 0.021538 \n",
      "Error Rate on test set: 2.150000 %\n"
     ]
    }
   ],
   "source": [
    "# your code here\n",
    "amount = [500, 1000, 2000, 4000, 8000, 16000, 32000,60000]\n",
    "valerr = np.zeros(len(amount))\n",
    "for i in range(len(amount)):\n",
    "  mnist_trainset_small =  [ mnist_trainset[i] for i in range(0,amount[i]) ] \n",
    "  smalltrainloader = torch.utils.data.DataLoader( mnist_trainset_small, batch_size=64, shuffle=True)\n",
    "  my_model(smalltrainloader, epo = 20, learn = 0.001, dropout = 0.2)\n",
    "  valerr[i] = validate(testloader)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 312
    },
    "id": "mw6Eib9zotuZ",
    "outputId": "637da2d9-ad6b-4df8-f86b-47a529339cb1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Error rate in percantage')"
      ]
     },
     "execution_count": 118,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEWCAYAAACOv5f1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxcVZ338c+316S7s3XSJCENWdhJ2KOyCEYFCciijo44OoMLguioM+qDoDOCuw86oo4LoCLMAwOIIiKjIDISUDSQAEIgrCEhCYR09nQndGf5PX/c20l1p5fqpbq6qr7v16teuXXr1r2/21X5nVPnnnuOIgIzMystZfkOwMzMhp6Tv5lZCXLyNzMrQU7+ZmYlyMnfzKwEOfmbmZUgJ3+zHJD0O0nnDva2xURSs6QZ+Y6jVDn5FwlJSyVtTf9DtT++P8Qx3Cvp1fTYayTdKmlylu+dI2lFrmPsJYbMv93OTn/P9/ZlXxFxWkRcN9jb9kX6N92ZcQ4rJP1c0mv6sI/LJF0/2LEBRERdRCzJxb6td07+xeXM9D9U++Ofu9pIUkUX68r7cqAetv/niKgD9gfqgG/1Zb/5lPm3A16k49/zhvbtuvr7DWMvpeczCjgWeAq4X9Kb8xuW5ZuTfwmQ9H5Jf5Z0haS1wGWSrpX0I0m/ldQCvFHSIWntfYOkJySdlbGPPbbv6ZgRsQG4DTgyYx8fkLRY0mZJSyRdkK6vBX4H7J1RS91bUpmkiyU9L2ltWmut7+YcF0s6I+N5haQmSUdLGiHp+nQfGyQ9JGliH/5+c9Ja82clrQJ+JmmcpDvSY6xPlxsz3nOvpPMy/v5/kvStdNsXJJ3Wz22nS7ov/Rv+QdIPsqmZR2JFRHwB+AnwfzP2+V1JyyVtkrRQ0onp+rnA54B3p5/J39L1XX6O3fzt9pc0T9LG9NfgzRmvRfp65ufeLGmLpMjY7oPp8dZLukvS1N7O13rn5F86XgcsASYCX03X/UO6PAqYD/wG+D2wF/Bx4AZJB2XsI3P7P/V0MEnjgXcAz2WsXg2cAYwGPgBcIenoiGgBTiOtpaaPl9IY3ga8AdgbWA/8oJtD3gi8J+P5qcCaiHgYOBcYA+wDjAc+AmztKf4uTALqganA+ST/d36WPt833V9PzWyvA54GJgCXAz+VpH5s+9/Ag+l5XAb8Yx/PA+BW4Oi00AV4iKSQrk/3f4ukERFxJ/A14Ob0Mzki3b7Lz7GbY32Z5Ds1DmgE/rPzBhHxUqdfXb8CbgKQdDZJAfQOoAG4n+SztoGKCD+K4AEsBZqBDRmPD6evvR94sdP21wL/lfH8RGAVUJax7kbgsq627yaGe4EtwEYggEeBfXvY/jbgk+nyHGBFp9cXA2/OeD4Z2AZUdLGv/YHNQE36/AbgC+nyB4EHgMP7+Pc8OSO2NmBED9sfCazv9Lc4L+Pv/1zGazXp32dSX7YlKWS2t59j+vr1wPXdxLTH3zRdf3C6zyndvG89cES6fFl3++/qc+zitf8CrgYau3gtgP07rfsssBAYmT7/HfChjNfL0u/Y1KH+P1ZsD9f8i8vbImJsxuPHGa8t72L7zHV7A8sjYmfGumXAlF720dknImIMcDi7a3sASDpN0l8lrZO0ATidpHbbnanAr9Kmmg0khcEOkl8vHUTEc+nrZ0qqAc4iqcUC/D/gLuAmSS9JulxSZRbnkqkpIl7NOJcaSVdJWiZpE3AfMFbdXwtZlRHrlnSxro/b7g2sy1gH2X0mnU0hSbwbACR9Jm1W2Zj+ncfQw+fSx8/xIkDAg0qaEj/Y036BT5J8j9t/mU0FvpvxHViX7m9KN7uxLDn5l46uhm/NXPcSsI+kzO/EvsDKXvbR9cEiHge+AvxAiWrglyQXgCdGxFjgtyT/kbvb93LgtE4F2oiIWNnFtrC76eds4Mm0QCAitkXEFyPiUOB4kiaLf8r2XLqJ79PAQcDrImI0cFK6vrumnMHwMlCfFm7t9unHft4OPBwRLWn7/kXA3wPj0s9lI918Lll8jh1ExKqI+HBE7A1cAPxQ0v6dt0ubF68D/j4iMgu05cAFnb4DIyPigX6ct2Vw8rd280l+Tl8kqVLSHOBM0rbXfrqOpJZ+FlAFVANNwPa0lveWjG1fAcZLGpOx7krgq+0X+CQ1pG3A3bkp3eeF7K71I+mNkg5La+WbSJqOdna9i6yNImnn36DkIvSlA9xfryJiGbCA5IJ9laTjSD6jXqUF8BRJlwLnkbSjQ3Ie20k+lwpJXyBpy2/3CjAto1LQ2+fY+bjv0u4L4etJCpOdnbYZDfwa+HxEdL6WdCVwiaSZ6bZjJL0rm3O2njn5F5ffdOo18ats3xgRbSSJ5DRgDfBD4J8i4qn+BpPu87vAv0fEZuATwM9JksA/ALdnbPsUSc19SfoTf+/0vbcDv5e0GfgrycXQ7o73MvAXktr9zRkvTQJ+QZL4FwPzSJqCBuI7wEiSv9VfgTsHuL9svRc4DlhL8svqZqC1h+33ltRMcj3oIeAwYE5E/D59/S6S2J8haeZ7lY5NSbek/66V9HBvn2MXXgPMT2O4neTaQOe+/UeT/Iq6IvP7CxARvyLpmXRT2ry2iOQ7agOkCE/mYlao0q6TT0VEzn95WHFxzd+sgEh6jaT9lNwDMZfk+sZt+Y7LCk8h3aloZkkT1q0k/fxXABdGxCP5DckKkZt9zMxKkJt9zMxKUEE0+0yYMCGmTZuW7zDMzArKwoUL10REQ1evFUTynzZtGgsWLMh3GGZmBUXSsu5ec7OPmVkJcvI3MytBTv5mZiXIyd/MrAQ5+ZuZlSAnfzOzEuTkb2ZWgoo6+d+z+BV+eO9zvW9oZlZiijr53/dME1fN6zx0uJmZFXXyr6muYEvb9nyHYWY27BR18q+rrmDbjqB1+458h2JmNqwUdfKvqSoHYEurk7+ZWaaiTv61Vcm4dS1u+jEz66Cok39NdVrzb3PN38wsU1En/101/1bX/M3MMhV38q9uT/6u+ZuZZSrq5N9+wddt/mZmHeUs+Uu6RtJqSYs6rf+4pKckPSHp8lwdH3bX/N3X38yso1zW/K8F5maukPRG4GzgiIiYCXwrh8entr3m72YfM7MOcpb8I+I+YF2n1RcC34iI1nSb1bk6PiR3+IJr/mZmnQ11m/+BwImS5kuaJ+k13W0o6XxJCyQtaGpq6tfBaiqTmn+za/5mZh0MdfKvAOqBY4H/A/xckrraMCKujojZETG7oaGhXwcrKxM1VeVscVdPM7MOhjr5rwBujcSDwE5gQi4PWFNVQYtv8jIz62Cok/9twBsBJB0IVAFrcnnA2upyt/mbmXVSkasdS7oRmANMkLQCuBS4Brgm7f7ZBpwbEZGrGCCt+bvN38ysg5wl/4h4TzcvvS9Xx+xKbZVr/mZmnRX1Hb6Q3OjlsX3MzDoqgeRf7gu+ZmadFH3yr6mqcFdPM7NOij7511a55m9m1lnRJ39P4m5mtqeiT/6exN3MbE9Fn/w9ibuZ2Z6KPvl7Enczsz0VffL3JO5mZnsq+uTvSdzNzPZU/Mnfk7ibme2h6JO/J3E3M9tT0Sd/T+JuZran4k/+nsTdzGwPRZ/8PYm7mdmeij/5V7rmb2bWWdEn//ZJ3N3V08xst6JP/uBJ3M3MOstZ8pd0jaTV6Xy9nV/7tKSQNCFXx8/kSdzNzDrKZc3/WmBu55WS9gHeAryYw2N34Enczcw6ylnyj4j7gHVdvHQFcBEQuTp2Z3Wu+ZuZdTCkbf6SzgZWRsTfstj2fEkLJC1oamoa0HGTmr+Tv5lZuyFL/pJqgM8BX8hm+4i4OiJmR8TshoaGAR3bk7ibmXWUVfKXNFXSyenySEmj+nGs/YDpwN8kLQUagYclTerHvvrEk7ibmXVU0dsGkj4MnA/UkyTwRuBK4M19OVBEPA7slbHfpcDsiFjTl/30hydxNzPrKJua/8eAE4BNABHxLBlJvDuSbgT+AhwkaYWkDw0k0IGo9STuZmYd9FrzB1ojok0SAJIqyKKnTkS8p5fXp2UT4GCozZjEvbqifKgOa2Y2bGVT858n6XPASEmnALcAv8ltWIPLk7ibmXWUTfK/GGgCHgcuAH4L/FsugxpsnsTdzKyjXpt9ImIn8OP0UZA8ibuZWUfZ9PZ5nD3b+DcCC4CvRMTaXAQ2mDyJu5lZR9lc8P0dsAP47/T5OUANsIpk/J4zcxLZINo9laNr/mZmkF3yPzkijs54/rikhyPiaEnvy1Vgg6n9gm+za/5mZkB2F3zLJb22/Ymk1wDt/SULIpt6Enczs46yqfmfB1wjqQ4Qyc1e50mqBb6ey+AGiydxNzPrKJvePg8Bh0kakz7fmPHyz3MV2GBqn8TdF3zNzBLZ1PyR9FZgJjCi/U7fiPhSDuMaVLVV5YyoLKNpc2u+QzEzGxZ6bfOXdCXwbuDjJM0+7wKm5jiuQSWJxnE1rFi/Nd+hmJkNC9lc8D0+Iv4JWB8RXwSOAw7MbViDr3HcSFZs2JLvMMzMhoVskn97dXmLpL2BbcDk3IWUG1PGjnTN38wslU3yv0PSWOCbwMPAUuDGXAaVC43jatiwZRubX92W71DMzPIumwu+l0dEK/BLSXcAI4BXcxvW4GscNxKAlRu2cvCkyjxHY2aWX9nU/P/SvhARrWlXz7/0sP2wtCv5u+nHzKz7mn86t+4UknH8jyLp6QMwmmRsn4LSOC4J2e3+ZmY9N/ucCryfZM7eb2es3wx8Locx5cSEuiqqK8pYsd49fszMuk3+EXEdcJ2kv4uIX/Z1x5KuAc4AVkfErHTdN0lGAW0Dngc+EBEb+hV53+NJunu65m9mlnVvn3+Q9DlJX2h/ZPG+a4G5ndbdDcyKiMOBZ4BL+hbuwPhGLzOzRDbJ/9fA2SQjeLZkPHoUEfcB6zqt+31EtA+w81eSJqUhk9T83exjZpZNV8/GiOhcgx8MHwRu7u5FSecD5wPsu+++g3LAxnE1rN+yjebW7dRVZzWskZlZUcqm5v+ApMMG86CSPk/yS+KG7raJiKsjYnZEzG5oaBiU47q7p5lZIpvk/3pgoaSnJT0m6XFJj/X3gJLeT3Ih+L0R0Xlu4JyakiZ/N/2YWanLpu3jtME6mKS5wEXAGyJiyDNw467k75q/mZW2Xmv+EbEsIpaRDPAWGY8eSbqR5E7ggyStkPQh4PvAKOBuSY+mw0UPmYa6aqoryli5wcnfzEpbrzV/SWcB/wHsDawmGct/McnkLt2KiPd0sfqn/Yhx0Ehiinv8mJll1eb/ZeBY4JmImA68maSbZkFyX38zs+yS/7aIWAuUSSqLiD8Cs3McV874Ll8zs+wu+G6QVAfcB9wgaTVZ3OQ1XDWOG8m6ljZaWrdT677+Zlaisqn5nw1sAf4VuJNkTJ4zcxlULrWP7umLvmZWyrJJ/nsBVRGxPR3s7cckPXYKUqP7+puZZZX8bwF2Zjzfka4rSO7rb2aWXfKviIi29ifpclXuQsqt9r7+Tv5mVsqySf5NaV9/ACSdDazJXUi5JYkpY93X38xKWzbdXT5C0svn++nzFcA/5i6k3Jvi7p5mVuJ6TP6SyoELI+LYtLsnEdE8JJHlUOO4Gp58aVW+wzAzy5sem30iYgfJqJ5ERHMxJH5ILvqubWljS9v23jc2MytC2TT7PCLpdpIePrtu7oqIW3MWVY5ljut/wMSC7bVqZtZv2ST/EcBa4E0Z6wIo4OSf3Oi1wsnfzEpUr8k/Ij4wFIEMpX18o5eZlbheu3pKOlDSPZIWpc8Pl/RvuQ8tdybUVVPlvv5mVsKy6ef/Y+ASYBtARDwGnJPLoHKtrEw0jnV3TzMrXdkk/5qIeLDTuoLvJuNJXcyslGWT/NdI2o906kZJ7wRe7u1Nkq6RtLq9uShdVy/pbknPpv+O63fkA+Rx/c2slGWT/D8GXAUcLGkl8C8kd/325lpgbqd1FwP3RMQBwD3p87xoHFfjvv5mVrKymcB9SUScDDQAB0fE69MJ3Xt7333Auk6rzwauS5evA97Wx3gHTXtf/5c8rr+ZlaBsevuMl/Q94H7gXknflTS+n8ebGBHtTUargIn93M+AtSf/5W76MbMSlE2zz01AE/B3wDvT5ZsHeuCICNLrCF2RdL6kBZIWNDU1DfRwe8i80cvMrNRkk/wnR8SXI+KF9PEV+l9jf0XSZID039XdbRgRV0fE7IiY3dDQ0M/Dda+hrpqq8jKWr3OPHzMrPdkk/99LOkdSWfr4e+Cufh7vduDcdPlc4Nf93M+AlZWJQ/cezV+XrM1XCGZmeZNN8v8w8N9Aa/q4CbhA0mZJm7p7k6Qbgb8AB0laIelDwDeAUyQ9C5ycPs+bubMm8diKjZ7M3cxKTja9fUZFRFlEVKaPsnTdqIgY3cP73hMRk9P3NEbETyNibUS8OSIOiIiTI6Jzb6AhderMSQDctchj+5tZacmm5l+0pk+o5eBJo7jzCSd/MystJZ38Ian9P7R0HU2bW/MdipnZkCn55D931iQi4A+LX8l3KGZmQyar5C+pXNLekvZtf+Q6sKFy8KRRTB1fw51u9zezEtLrZC6SPg5cCrwC7ExXB3B4DuMaMpKYO3MS1/z5BTZu3caYkZX5DsnMLOeyqfl/EjgoImZGxGHpoygSf7tTZ01i247gj091e8+ZmVlRySb5Lwc25jqQfDqycSwTR1e76cfMSkY2E7gvIRnQ7X9IbvICICK+nbOohlhZmTh15iR+vmA5W9t2MLKqPN8hmZnlVDY1/xeBu4EqYFTGo6jMnTmJV7ftZN4zgz+InJnZcNNrzT8ivjgUgeTba6fXM7amkrueWMXcWZPyHY6ZWU51m/wlfSci/kXSb+hi6OWIOCunkQ2xivIyTjlkInc+sYq27Tupqij5WyDMrIj1VPP/f+m/3xqKQIaDubMmccvCFfxlyVrecODgDyNtZjZcdJv8I2Jh+u+8oQsnv07YfwK1VeXcuWiVk7+ZFTW3bWQYUVnOGw/ei7ufXMWOnd1OMmZmVvCc/DuZO2sSa5rbWLhsfb5DMTPLmayTv6SaXAYyXMw5aC+qKsp8w5eZFbVek7+k4yU9CTyVPj9C0g9zHlme1FVXcNIBE7jriVUkc8ybmRWfbGr+VwCnAmsBIuJvwEm5DCrfTp05iZUbtrJoZbezVJqZFbSsmn0iYnmnVTsGclBJ/yrpCUmLJN0oacRA9jfYTj5kIuVl4s4nXs53KGZmOZHVwG6SjgdCUqWkzwCL+3tASVOATwCzI2IWUA6c09/95cK42iqOnVHvdn8zK1rZJP+PAB8DpgArgSOBjw7wuBXASEkVQA3w0gD3N+jmzpzE800tPLd6c75DMTMbdNkk/4Mi4r0RMTEi9oqI9wGH9PeAEbGS5K7hF4GXgY0R8fvO20k6X9ICSQuamoZ+sLW3zEzG93Ht38yKUTbJ/z+zXJcVSeOAs4HpwN5AraT3dd4uIq6OiNkRMbuhYejvtp04egRH7zuWO59w8jez4tPTwG7HAccDDZI+lfHSaJJ2+v46GXghIprS49yaHuf6AewzJ+bOmsTXfvsUy9dtYZ/6krjNwcxKRE81/yqgjqSAyBzHfxPwzgEc80XgWEk1kgS8mQFcQM6lU9Omn7tc+zezItPTwG7zgHmSro2IZYN1wIiYL+kXwMPAduAR4OrB2v9gmjq+lkMmj+auJ1Zx3okz8h2OmdmgyWYaxy2SvgnMBHb1x4+IN/X3oBFxKXBpf98/lObOnMR37nmG1ZtfZa9Rw+p2BDOzfsvmgu8NJEM7TAe+CCwFHsphTMPK3FmTiIC7n3wl36GYmQ2abJL/+Ij4KbAtIuZFxAeBftf6C82BE+uYPqGW2x99yWP9mFnRyCb5b0v/fVnSWyUdBdTnMKZhRRLvfd2+zH9hnS/8mlnRyCb5f0XSGODTwGeAnwD/mtOohpn3Hz+NQyeP5tLbn2DTq9t6f4OZ2TDXY/KXVA4cEBEbI2JRRLwxIo6JiNuHKL5hoaK8jK+/4zCaNrfyzTufznc4ZmYD1mPyj4gdwHuGKJZh7Yh9xvJPx03j+vnLPMuXmRW8bJp9/izp+5JOlHR0+yPnkQ1Dnzn1ICaNHsHnbn2cbTt25jscM7N+yyb5H0nSx/9LwH+kj2/lMqjhqq66gi+dPYunX9nM1fctyXc4Zmb91utNXhHxxqEIpFCccuhE5s6cxPfueZYzDp/M1PG1+Q7JzKzPsp7A3Xa77KyZVJaX8flfLXLffzMrSE7+/TBpzAg+O/cg/vTcGm57dGW+wzEz67PeunqWpVM4Wifvfd1Ujtp3LF++YzHrWtryHY6ZWZ/01tVzJ/CDIYqloJSVia+/4zA2bd3G1347LEekNjPrVjbNPvdI+rt07H3LcPCk0Zx/0gx+sXAFDzy/Jt/hmJllLZvkfwFwC9AmaZOkzZI25TiugvGJNx/A1PE1fP5Xi3h12458h2NmlpVek39EjIqIsoiojIjR6fPRQxFcIRhRWc5X33YYL6xp4Qd/fC7f4ZiZZSWr3j6SzpL0rfRxRq6DKjSvP2ACbz9qClfOe55nX9mc73DMzHrVa/KX9A3gk8CT6eOTkr6e68AKzb+99RBqqyu45NbH2bnTff/NbHjLpuZ/OnBKRFwTEdcAc4G3DuSgksZK+oWkpyQtlnTcQPY3HIyvq+bzpx/CgmXrufGhF/MdjplZj7K9yWtsxvKYQTjud4E7I+Jg4AigKPpKvvOYRo6bMZ5v/O4pVm96Nd/hmJl1K5vk/zXgEUnXSroOWAh8tb8HTCeGOQn4KUBEtEXEhv7ubziRxFffPovW7Tv54h1P5jscM7Nu9XqHL7ATOBa4FfglcFxE3DyAY04HmoCfSXpE0k8k7TE6mqTzJS2QtKCpqWkAhxtaMxrq+Pgb9+d/HnuZ/33Kk76b2fCUzR2+F0XEyxFxe/oY6ES2FcDRwI8i4iigBbi4i2NfHRGzI2J2Q0PDAA85tC54w34csFcd/37bE7S0bs93OGZme8im2ecPkj4jaR9J9e2PARxzBbAiIuanz39BUhgUjaqKMr72jsNYuWErV9z9TL7DMTPbQzbJ/93Ax4D7SNr7FwIL+nvA9JfDckkHpaveTNKFtKi8Zlo9//C6fbnmzy+waOXGfIdjZtZBNm3+F0fE9E6PGQM87seBGyQ9RjJT2NcGuL9h6bNzD2Z8XTUX3/oY2z3to5kNI9m0+f+fwT5oRDyatucfHhFvi4iinBF9zMhKLj3zUBat3MS1DyzNdzhmZrvko82/pLz1sMm86eC9+Pbdz7Byw9Z8h2NmBuShzb/USOJLZ88kAv79Nk/7aGbDQzajenZu7x+MNv+S0jiuhk+/5UD+96nV/PbxgfaUNTMbuG6Tv6SLMpbf1em1orxAm0vvP34as6aM5rLfPMHGrdvyHY6Zlbieav7nZCxf0um1uTmIpahVlJfx9bcfztrmVi6/86l8h2NmJa6n5K9ulrt6blk4rHEMHzhhOjfMf5EFS9flOxwzK2E9Jf/oZrmr55alT51yIFPGjuSSWx+n2UM/mFme9JT8j2ifsxc4PF1uf37YEMVXdGqrK/jy22by7OpmTvjG//Ltu59hXUtbvsMysxKjQuh6OHv27FiwoLh6lz7y4np+dO/z/P7JVxhRWcY5r9mX806cTuO4mnyHZmZFQtLCiJjd5WtO/vn13OrNXDlvCbc9shKAs47cm4+8YT8OnDgqz5GZWaFz8i8AL23Yyk/uf4EbH3yRrdt2cPIhE7lwzn4cM3VcvkMzswLl5F9A1re0cd1flnLtA0vZsGUbr51ez4Vz9mPOgQ1I7mRlZtlz8i9AW9q2c9ODy/nx/Ut4eeOrHDxpFBfO2Y+3HjaZivJsp142s1Lm5F/A2rbv5Pa/vcSV857nudXN7FM/kvNP2o93HdPIiMryfIdnZsOYk38R2Lkz+MPiV/jhvc/z6PINTKir4gMnTOd9x05lzMjKfIdnZsOQk38RiQjmv7COH937PPOeaaKuuoL3HrsvHzphOnuNHpHv8MxsGHHyL1KLVm7kqvuW8D+PvURFWRl/d0wjF5w0g2kTavMdmpkNA07+RW7Z2hauvm8JtyxcwfYdOzntsMlc+Ib9mDVlTL5DM7M8GpbJX1I5yaQwKyPijJ62dfLPzupNr3LNn5dy/V+X0dy6nRMPmMCFc/bjuBnj3U3UrAQN1+T/KWA2MNrJf3Bt3LqNG+Yv45o/vcCa5jaO2GcsF75hP95y6ETKylwImJWKnpJ/XjqMS2oE3gr8JB/HL3ZjRlby0Tn786fPvomvvG0W61va+Mj1CznlinncsmA5bdt35jtEM8uzvNT8Jf0C+DowCvhMVzV/SecD5wPsu+++xyxbtmxogywi23fs5LeLVvGje59n8cubmDxmBOedOINzXrMPtdUV+Q7PzHJkWDX7SDoDOD0iPippDt0k/0xu9hkcEcG8Z5r40b3PM/+FdYytqeTc46Zx7vHTqK+tynd4ZjbIhlvy/zrwj8B2YAQwGrg1It7X3Xuc/AffwmXruXLe89z95CuMrCznnNfuw3knzmDK2JH5Ds3MBsmwSv4dDu6af949+0oypPSvH02GlD77yCl85A0zOMBDSpsVvJ6Svxt8S9wBE0fxH39/BJ96y4H85P4l3PTgcn758ApOOXQi571+OgdNGsWYkZXuKmpWZHyTl3WwrqWN6x5IhpTeuHUbAJXlYnxtNRNGVTGhrjrjUUXDqGS5/d+xIyvdndRsmBi2zT7ZcvIfei2t2/nj06tZtfFV1jS3saa5dfdjcxtrW1rZtmPP7055mRhfmxYSo9ICor2wSAuP9oJiXE0V5S4ozHLGzT7WZ7XVFZxx+N7dvh4RbNy6jTXNrTRt3l04NG1uLySSdc+9spk1zW207djz3oIyQX1tx18QE+oyfl2Mqk4KjlFV1NdUeR4Ds0Hk5G/9IomxNVWMrali/7163jYi2PTq9vRXQ2uHXxLthUVTcxtLmlpY09xKaxc3oUlQX1PV4RdEd81P9bVVVLqgMOuRk7/lnCTGjKxkzMhK9muo63HbiKC5dfvuAiKjcGh/3tTcysMvrmfN5ja2btvR5X7G1VR2+AXR/ouiIaPZacKoKsbXVlNV4YLCSo+Tvw0rkhg1opJRIyqZnrXZ9IAAAAu+SURBVMXQ1C2t2zN+Rex5bWJNcyuPrdjAms2ttLR1XVCMGVm5u7mpvampruP1ifYCpLrCs6dZcXDyt4JWW11BbXUFU8f3XlBsbduR/orYs/mpvbBY/NIm7mtuZfOr27vcx6gRFXtcwG5/NIzqWGh4mk0bzpz8rWSMrCpnn/oa9qmv6XXbV7ft2H3henNGAdHctqvweHrVZv7cvHZXl9jO6qordhUG9bVVjK+rZnxtVbqcNDnV11Yxoa6Kcb5OYUPMyd+sCyMqy2kcV0PjuN4LirbtO1nbktHTaXNaQGQUHsvWbuHhFzewfksbO3Z23b16zMjKDoVDfW1SWCTLSWGRFBouLGzgnPzNBqiqoozJY0YyeUzv4yLt3Jl0kV3b0sba5lbWtbSly22sa2ndtbx0zRYWLtvAupZWuikrXFjYgDj5mw2hsjIxLk3G++/Vc88n6L2wWNPSxrpdhcV61rW0ubCwrDj5mw1j/S8sWtMCom1XAdHfwqK9cGi/Ia8+LUB2XctwYVGQnPzNikjHwqL37XsqLNamzVDrmtt4YU2LC4si4+RvVsL6U1hs2LotuT7RnDZBDaSwSJuZ9uwN1bFnlIf3GHxO/maWtbIy7arJ97WwWJP+stjz+kWrC4s8cPI3s5zpa2GxI22G6rawSH9hvLCmhQVL17N+iwuL/nLyN7Nho7yfhcXa5rTJKS0odi9nX1iMrancdU1ifG019XXty1XU13XsGVUMhYWTv5kVrMzC4oAstu+psFi765dGK883NfPQ0raiLiyc/M2sZPSnsNiwpa3bm/H6U1hMSIf1qK+rYkIaS31ddbI8hIXFkCd/SfsA/wVMBAK4OiK+O9RxmJn1prxMyfWBuuo+Fxbt1ywyr18k63cXFuu2tNHdZIrthcXX3n4Yx84YP6jnBfmp+W8HPh0RD0saBSyUdHdEPJmHWMzMBk2HwmJi79tnU1iMGVmZk1iHPPlHxMvAy+nyZkmLgSmAk7+ZlZS+FhaDKa9XICRNA44C5nfx2vmSFkha0NTUNNShmZkVtbwlf0l1wC+Bf4mITZ1fj4irI2J2RMxuaGgY+gDNzIpYXpK/pEqSxH9DRNyajxjMzErZkCd/SQJ+CiyOiG8P9fHNzCw/Nf8TgH8E3iTp0fRxeh7iMDMrWfno7fMnQEN9XDMz22143W9sZmZDwsnfzKwEKbq7t3gYkdQELOvj2yYAa3IQTj74XIYnn8vwVEznAgM7n6kR0WVf+YJI/v0haUFEzM53HIPB5zI8+VyGp2I6F8jd+bjZx8ysBDn5m5mVoGJO/lfnO4BB5HMZnnwuw1MxnQvk6HyKts3fzMy6V8w1fzMz64aTv5lZCSq65C9prqSnJT0n6eJ8x9NO0jWSVktalLGuXtLdkp5N/x2Xrpek76Xn8JikozPec266/bOSzs1Yf4ykx9P3fC8dQC9X57KPpD9KelLSE5I+WajnI2mEpAcl/S09ly+m66dLmp8e/2ZJVen66vT5c+nr0zL2dUm6/mlJp2asH9LvpKRySY9IuqMIzmVp+j14VNKCdF3Bfc/SY42V9AtJT0laLOm4vJ5LRBTNAygHngdmAFXA34BD8x1XGttJwNHAoox1lwMXp8sXA/83XT4d+B3JGEjHAvPT9fXAkvTfcenyuPS1B9Ntlb73tByey2Tg6HR5FPAMcGghnk+6/7p0uZJkYqFjgZ8D56TrrwQuTJc/ClyZLp8D3JwuH5p+36qB6en3sDwf30ngU8B/A3ekzwv5XJYCEzqtK7jvWXqs64Dz0uUqYGw+zyVnH1o+HsBxwF0Zzy8BLsl3XBnxTKNj8n8amJwuTwaeTpevAt7TeTvgPcBVGeuvStdNBp7KWN9huyE4r18DpxT6+QA1wMPA60juqKzo/L0C7gKOS5cr0u3U+bvWvt1QfyeBRuAe4E3AHWlsBXku6TGWsmfyL7jvGTAGeIG0k81wOJdia/aZAizPeL4iXTdcTYxkTmOAVUD7LJ7dnUdP61d0sT7n1HEqzoI8n7SZ5FFgNXA3Se12Q0Rs7+L4u2JOX98IjKfv55gr3wEuAnamz8dTuOcCEMDvJS2UdH66rhC/Z9OBJuBnaZPcTyTVksdzKbbkX7AiKa4Lqt+tepiKs5DOJyJ2RMSRJLXm1wIH5zmkfpF0BrA6IhbmO5ZB9PqIOBo4DfiYpJMyXyyg71kFSbPvjyLiKKCFpJlnl6E+l2JL/iuBfTKeN6brhqtXJE0GSP9dna7v7jx6Wt/YxfqcUddTcRbs+QBExAbgjyTNG2Mltc93kXn8XTGnr48B1tL3c8yFE4CzJC0FbiJp+vkuhXkuAETEyvTf1cCvSArnQvyerQBWRMT89PkvSAqD/J1LLtvrhvpBUrouIfmJ1X5Bama+48qIbxod2/y/SceLPZeny2+l48WeB9P19STthuPSxwtAffpa54s9p+fwPAT8F/CdTusL7nyABmBsujwSuB84A7iFjhdJP5ouf4yOF0l/ni7PpONF0iUkF0jz8p0E5rD7gm9BngtQC4zKWH4AmFuI37P0WPcDB6XLl6XnkbdzyekXMB8Pkqvkz5C0234+3/FkxHUj8DKwjaQW8CGS9tV7gGeBP2R8iAJ+kJ7D48DsjP18EHgufXwgY/1sYFH6nu/T6cLSIJ/L60l+nj4GPJo+Ti/E8wEOBx5Jz2UR8IV0/Yz0P9NzJMmzOl0/In3+XPr6jIx9fT6N92kyelrk4ztJx+RfkOeSxv239PFE+/EK8XuWHutIYEH6XbuNJHnn7Vw8vIOZWQkqtjZ/MzPLgpO/mVkJcvI3MytBTv5mZiXIyd/MrAQ5+duwIWl8Onrjo5JWSVqZ8byql/fOlvS9LI7xwCDFOkfSxvRW/acl3ZfeYZvN+47v47FqJN2Qjti4SNKf0rurB+18rPRU9L6J2dCIiLUkfaGRdBnQHBHfan9dUkXsHqOm83sXkPSh7u0YfUq8vbg/Is5IYzsSuE3S1oi4p4f3zAGaSW5YytYngVci4rD0WAeR3C8y2OdjJcQ1fxvWJF0r6UpJ84HLJb1W0l/SGvcDaSJsr1G3j19/mZL5E+6VtETSJzL215yx/b0Z46vf0D7+uaTT03UL03HR7+gtzoh4FPgS8M/pPs5UMkb+I5L+IGliOgjeR4B/TX/NnNjVdl3sfjIZt+pHxNMR0drpfL6U8StppaSfpevfp2S+gkclXSWpvI8fgRUpJ38rBI3A8RHxKeAp4MRIBsf6AvC1bt5zMHAqyVgwl6ZjEXV2FPAvJOPXzwBOkDSCZJjc0yLiGJLhH7L1MLsHhfsTcGwa503ARRGxlGR4hSsi4siIuL+r7brY7zXAZ9NC7yuSDui8QUR8IZLB6eYA64DvSzoEeDdwQvraDuC9fTgfK2Ju9rFCcEtE7EiXxwDXpQkwSCZg6cr/pLXjVkmrSYbKXdFpmwcjYgVAOqTzNJImmSUR8UK6zY3A+WQnc+akRuDmdLCuKpIxWLrS63YR8aikGcBbgJOBhyQdFxGLOxw8+eVyPfDtiFgo6Z+BY9LtIRm7aDVmuOZvhaElY/nLwB8jYhZwJsn4NF1pzVjeQdcVnWy26YujgPaE/J/A99N2+gt6iDOr7SKiOSJujYiPkiT407vY7DKSkSN/lj4XcF36K+PIiDgoIi7rx3lZEXLyt0Izht3t3+/Pwf6fBmZo93y2787mTZIOB/6dZDAu6BjnuRmbbiaZ+pJetsvc9wnaPbdrFUkz1bJO25xJ8qvgExmr7wHeKWmvdJt6SVOzOR8rfk7+VmguB74u6RFy0GwZEVtJ5ra9U9JCkmS9sZvNT2zv6kmS9D+R0dPnMuCWdB9rMt7zG+Dt7Rd8e9gu037APEmPk4xAuoBkLoVMnyKZuan94u6XIuJJ4N9IZsJ6jGSWsslZ/SGs6HlUT7NOJNVFRHPahv4D4NmIuCLfcZkNJtf8zfb04fQC8BMkzTJX5Tkes0Hnmr+ZWQlyzd/MrAQ5+ZuZlSAnfzOzEuTkb2ZWgpz8zcxK0P8HwKWMbGAru0QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(amount,valerr)\n",
    "plt.title('Error Rate vs Training Data size' )\n",
    "plt.xlabel('Training Data Size')\n",
    "plt.ylabel('Error rate in percantage')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 282
    },
    "id": "8bxNAi3pqA8w",
    "outputId": "d38b18d6-e4f8-4927-b2a6-cee29991fd8a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f50107bd310>]"
      ]
     },
     "execution_count": 72,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAD4CAYAAAAKA1qZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dfXTc1X3n8fdXMxo92ZZsWbZlS7JM7EBsQsDIJmlIWkIBkSZx0kAxSRNo6bppQptuTnfrbE9YwnZ7QnpOSFM4aWiAAFvWpt7SOImJ80Ba2jwYCzDYBgyKMZZsg2X5UZL1/N0/5koejcbS2JY8kn6f1zk6+s393fnNvYdBH997fw/m7oiISDTl5boBIiKSOwoBEZEIUwiIiESYQkBEJMIUAiIiERbPdQPOxOzZs722tjbXzRARmTRmz57N5s2bN7t7fab9kyoEamtraWhoyHUzREQmFTObfbp9mg4SEYkwhYCISIQpBEREIkwhICISYVmFgJnVm9kuM2s0s7UZ9heY2fqwf4uZ1abtrzGzNjP7i5SyPWa23cy2mZlWe0VEcmDUEDCzGHAfcD2wFLjZzJamVbsNOOLui4F7gLvT9n8NeDLD4a9y90vdve6MWy4iIucsm5HASqDR3Xe7ezewDliVVmcV8HDY3gBcbWYGYGYfBV4Hdo5Nk0VEZKxkEwILgKaU182hLGMdd+8FjgHlZjYN+EvgyxmO68CPzOxZM1tzug83szVm1mBmDS0tLVk0d7iHf7GH772w/6zeKyIylY33wvCdwD3u3pZh35XuvpzkNNPnzOz9mQ7g7ve7e52711VUVJxVIx7bspcfvHjgrN4rIjKVZXPF8D6gOuV1VSjLVKfZzOJAKdAKXAHcYGZfBcqAfjPrdPd73X0fgLsfNLMnSE47PX1OvTmNokSMjp6+8Ti0iMikls1IYCuwxMwWmVkCWA1sTKuzEbglbN8APOVJ73P3WnevBb4O/I2732tmJWY2HcDMSoBrgR1j0J+MivJjdHYrBERE0o06EnD3XjO7HdgMxIAH3X2nmd0FNLj7RuAB4FEzawQOkwyKkcwFnghrx3HgMXf/4Tn0Y0RFiRgtJ7rG6/AiIpNWVjeQc/dNwKa0sjtStjuBG0c5xp0p27uBd51JQ89FUX6Mk5oOEhEZJhJXDBclYpzUdJCIyDDRCAGNBEREMopGCGgkICKSUTRCIIwE3D3XTRERmVCiEQKJGACdPf05bomIyMQSjRDIT4aA1gVERIZSCIiIRFg0QiBMB53s7s1xS0REJpZohMDASKBbawIiIqmiEQIJTQeJiGQSiRAo1JqAiEhGkQiBYq0JiIhkFIkQ0NlBIiKZRSMEEloYFhHJJBIhoDUBEZHMsgoBM6s3s11m1mhmazPsLzCz9WH/FjOrTdtfY2ZtZvYX2R5zLGlNQEQks1FDwMxiwH0kHwi/FLjZzJamVbsNOOLui4F7gLvT9n8NePIMjzlm8mN5xPNMIwERkTTZjARWAo3uvtvdu4F1wKq0OquAh8P2BuBqC8+ONLOPAq8DO8/wmGOqKD+mNQERkTTZhMACoCnldXMoy1jH3XuBY0C5mU0D/hL48lkcEwAzW2NmDWbW0NLSkkVzMytMxDjZo+kgEZFU470wfCdwj7u3ne0B3P1+d69z97qKioqzbkixHiwjIjJMNg+a3wdUp7yuCmWZ6jSbWRwoBVqBK4AbzOyrQBnQb2adwLNZHHNM6RGTIiLDZRMCW4ElZraI5B/q1cAn0upsBG4BfgncADzlycd4vW+ggpndCbS5+70hKEY75pgqzI9xUg+VEREZYtQQcPdeM7sd2AzEgAfdfaeZ3QU0uPtG4AHgUTNrBA6T/KN+xsc8x76MKDkdpDUBEZFU2YwEcPdNwKa0sjtStjuBG0c5xp2jHXM8FeXHON7Zc74+TkRkUojEFcMQzg7SwrCIyBCRCYGi/JgeNC8ikiYyIVCciNGhNQERkSEiEwI6RVREZLjIhEBhmA7q7/dcN0VEZMKITAgMPFOgs1ejARGRAZEJgVO3k1YIiIgMiEwI6MEyIiLDRSYEBp4z3KkQEBEZFLkQ6NB0kIjIoMiEgNYERESGi0wIFCa0JiAiki4yITAwHaSRgIjIKZEJgWKNBEREholMCBTpFFERkWGyCgEzqzezXWbWaGZrM+wvMLP1Yf8WM6sN5SvNbFv4ecHMPpbynj1mtj3saxirDp1OoRaGRUSGGfWhMmYWA+4DrgGaga1mttHdX0qpdhtwxN0Xm9lq4G7gJmAHUBeeJFYJvGBm33P3gdt5XuXuh8ayQ6ejNQERkeGyGQmsBBrdfbe7dwPrgFVpdVYBD4ftDcDVZmbu3pHyB78QyNnd2/JjeeTHTNNBIiIpsgmBBUBTyuvmUJaxTvijfwwoBzCzK8xsJ7Ad+ExKKDjwIzN71szWnH0Xsleo20mLiAyR1TOGz4W7bwGWmdk7gIfN7MnwTOIr3X2fmc0Bfmxmr7j70+nvDwGxBqCmpuac2pJ8uphCQERkQDYjgX1AdcrrqlCWsY6ZxYFSoDW1gru/DLQBF4fX+8Lvg8ATJKedhnH3+929zt3rKioqsmju6RUlYrpthIhIimxCYCuwxMwWmVkCWA1sTKuzEbglbN8APOXuHt4TBzCzhcBFwB4zKzGz6aG8BLiW5CLyuCrK18PmRURSjTodFM7suR3YDMSAB919p5ndBTS4+0bgAeBRM2sEDpMMCoArgbVm1gP0A59190NmdgHwhJkNtOExd//hWHcuXVFCawIiIqmyWhNw903AprSyO1K2O4EbM7zvUeDRDOW7gXedaWPPlUYCIiJDReaKYdDD5kVE0kUrBDQdJCIyRLRCID9Gp6aDREQGRSsEEjE6NBIQERkUrRDQwrCIyBDRCoFEjK7efvr7c3YLIxGRCSVaIRDuJNrZq9GAiAhELQTCMwV06wgRkaRohYCeKSAiMkS0QiCMBHQnURGRpGiFQL6mg0REUkUqBMqKEwAcbu/OcUtERCaGSIXA/LJCAPYdPZnjloiITAyRCoE50wuJ5RkHjikEREQgYiEQyzPmTi/gwNHOXDdFRGRCiFQIAFSWFbFfIwERESDLEDCzejPbZWaNZrY2w/4CM1sf9m8xs9pQvtLMtoWfF8zsY9kec7zMLyviwDGNBEREIIsQMLMYcB9wPbAUuNnMlqZVuw044u6LgXuAu0P5DqDO3S8F6oFvmVk8y2OOi/mlhRw41om77h8kIpLNSGAl0Ojuu929G1gHrEqrswp4OGxvAK42M3P3DnfvDeWFwMBf3myOOS4qSwvp7u2nVaeJiohkFQILgKaU182hLGOd8Ef/GFAOYGZXmNlOYDvwmbA/m2MS3r/GzBrMrKGlpSWL5o6ssqwIQIvDIiKch4Vhd9/i7suAFcAXzazwDN9/v7vXuXtdRUXFObdnfmkyBLQ4LCKSXQjsA6pTXleFsox1zCwOlAKtqRXc/WWgDbg4y2OOi8pwwdgBXTAmIpJVCGwFlpjZIjNLAKuBjWl1NgK3hO0bgKfc3cN74gBmthC4CNiT5THHRXlJgkQ8T2cIiYgA8dEquHuvmd0ObAZiwIPuvtPM7gIa3H0j8ADwqJk1AodJ/lEHuBJYa2Y9QD/wWXc/BJDpmGPct4zMjMrSQvYrBERERg8BAHffBGxKK7sjZbsTuDHD+x4FHs32mOdLZWmhpoNERIjgFcOQXBzWdJCISERDoLKskDePd9KnB86LSMRFMwRKi+jrdw6e0GhARKItkiGwIFwwtl8XjIlIxEUyBAavFdAFYyIScdEMgVLdOkJEBCIaAjMK45QkYrp1hIhEXiRDwMyoLCvSSEBEIi+SIQDhgjGNBEQk4iIbAvNLi3TrCBGJvMiGQGVZIYfauuju7c91U0REciayITC/tAh3eOu4RgMiEl2RDYGBawX260ZyIhJh0Q2BgWsFtC4gIhEW2RCYH0YC+zQSEJEIyyoEzKzezHaZWaOZrc2wv8DM1of9W8ysNpRfY2bPmtn28PsDKe/5t3DMbeFnzlh1KhvFiThlxfk6TVREIm3Uh8qYWQy4D7gGaAa2mtlGd38ppdptwBF3X2xmq4G7gZuAQ8CH3X2/mV1M8kliC1Le90l3bxijvpyxylJdMCYi0ZbNSGAl0Ojuu929G1gHrEqrswp4OGxvAK42M3P35919fyjfCRSZWcFYNHwszNdjJkUk4rIJgQVAU8rrZob+a35IHXfvBY4B5Wl1Pg485+5dKWUPhamgL5mZZfpwM1tjZg1m1tDS0pJFc7NXWaarhkUk2s7LwrCZLSM5RfTHKcWfdPd3Au8LP5/K9F53v9/d69y9rqKiYkzbVVlaxNGOHk52943pcUVEJotsQmAfUJ3yuiqUZaxjZnGgFGgNr6uAJ4BPu/uvB97g7vvC7xPAYySnnc6rgTOEdDdREYmqbEJgK7DEzBaZWQJYDWxMq7MRuCVs3wA85e5uZmXAD4C17v7zgcpmFjez2WE7H/gQsOPcunLm9FwBEYm6UUMgzPHfTvLMnpeBx919p5ndZWYfCdUeAMrNrBH4AjBwGuntwGLgjrRTQQuAzWb2IrCN5EjiH8eyY9mYH0JAIwERiapRTxEFcPdNwKa0sjtStjuBGzO876+Bvz7NYS/PvpnjY25p8kQljQREJKoie8UwQEE8xuxpBTpDSEQiK9IhAMnFYV0rICJRFfkQqCwt5IDuHyQiERX5EJhfVsT+oydx91w3RUTkvFMIlBbR3t3H8c7eXDdFROS8i3wIDDxcRovDIhJFCoGBawW0LiAiERT5EFgydxrxPGPL64dz3RQRkfMu8iEwozCf31g8m8073tTisIhETuRDAKB+2Tz2tHaw660TuW6KiMh5pRAArl02FzN4cvubuW6KiMh5pRAAZk8rYEXtLDbvVAiISLQoBIL6ZfN45c0TvH6oPddNERE5bxQCwXUXzwPQaEBEIkUhECwoK+KSqlJ+uEMhICLRkVUImFm9me0ys0YzW5thf4GZrQ/7t5hZbSi/xsyeNbPt4fcHUt5zeShvNLNvnO5B8+dT/cXz2NZ0VFcPi0hkjBoCZhYD7gOuB5YCN5vZ0rRqtwFH3H0xcA/Jh8oDHAI+HB4ofwvwaMp7vgn8F2BJ+Kk/h36MifplYUpIowERiYhsRgIrgUZ33+3u3cA6YFVanVXAw2F7A3C1mZm7P+/u+0P5TqAojBoqgRnu/itPXqH1CPDRc+7NObqgYhpvnzuNH2pdQEQiIpsQWAA0pbxuDmUZ64RnEh8DytPqfBx4zt27Qv3mUY4JgJmtMbMGM2toaWnJornnpn7ZPJ55/TCtbV3j/lkiIrl2XhaGzWwZySmiPz7T97r7/e5e5+51FRUVY9+4NNddPI9+h5+8/Na4f5aISK5lEwL7gOqU11WhLGMdM4sDpUBreF0FPAF82t1/nVK/apRj5sTSyhnUzCrmSa0LiEgEZBMCW4ElZrbIzBLAamBjWp2NJBd+AW4AnnJ3N7My4AfAWnf/+UBldz8AHDezd4ezgj4NfPcc+zImzIz6i+fx88ZDHO/syXVzRETG1aghEOb4bwc2Ay8Dj7v7TjO7y8w+Eqo9AJSbWSPwBWDgNNLbgcXAHWa2LfzMCfs+C3wbaAR+DTw5Vp06V9ctm0dPn/OzVw7muikiIuPKJtPtk+vq6ryhoWHcP6e/33nPV37K8pqZfPP3Lx/3zxMRGU9m9qy712XapyuGM8jLM65bNo9/29XCye6+XDdHRGTcKAROo37ZPE729PHvr47/aakiIrmiEDiNlYtmUVacrxvKiciUphA4jXgsj2veMZefvPwW3b39uW6OiMi4UAiMoP7ieZzo7OWXu1tz3RQRkXGhEBjBexfPZlpBXLeXFpEpSyEwgsL8GFddNIcfv/Qmff2T51RaEZFsKQRGUb9sHofaumnYczjXTRERGXMKgVH81oUVFMTzdHtpEZmSFAKjKCmI8/63V7B5x5tMpqurRUSyoRDIQv2yeew/1sn2fcdy3RQRkTGlEMjC1e+YQzzPdHtpEZlyFAJZKCtO8J63lfNDTQmJyBSjEMjSdcvm8fqhdl472JbrpoiIjBmFQJauXTYXM3hsy95cN0VEZMwoBLI0Z3ohN6+s4Tu/2MOjv3oj180RERkTWYWAmdWb2S4zazSztRn2F5jZ+rB/i5nVhvJyM/uZmbWZ2b1p7/m3cMz0J45NWHd9ZBlXXzSHO767g03bD+S6OSIi52zUEDCzGHAfcD2wFLjZzJamVbsNOOLui4F7gLtDeSfwJeAvTnP4T7r7peFnwj/LMR7L495PLOfympn8+bpt/KLxUK6bJCJyTrIZCawEGt19t7t3A+uAVWl1VgEPh+0NwNVmZu7e7u7/STIMpoSiRIxv31JH7exi1jz6LDt07YCITGLZhMACoCnldXMoy1gnPJj+GFCexbEfClNBXzIzy1TBzNaYWYOZNbS0TIynfJUVJ3j4D1cyozDOrQ9t5Y3W9lw3SUTkrORyYfiT7v5O4H3h51OZKrn7/e5e5+51FRUV57WBI6ksLeKR266gr7+fTz3wDC0nunLdJBGRM5ZNCOwDqlNeV4WyjHXMLA6UAiM+icXd94XfJ4DHSE47TSqL50zjwVtX0HKii1sfeoYTnT25bpKIyBnJJgS2AkvMbJGZJYDVwMa0OhuBW8L2DcBTPsKltWYWN7PZYTsf+BCw40wbPxFcVjOTb/7+cna9eYI1jzxLV29frpskIpK1UUMgzPHfDmwGXgYed/edZnaXmX0kVHsAKDezRuALwOBppGa2B/gacKuZNYcziwqAzWb2IrCN5EjiH8euW+fXb104h7+98RJ+ubuV/7p+mx5AIyKTRjybSu6+CdiUVnZHynYncONp3lt7msNenl0TJ4ePXVZFa1s3f/2Dl5lVsoP/tepiTrPWLSIyYWQVApKdP3rfBbS0dfGtf99NxbRCPv/bS3LdJBGRESkExtja+os4dKKbe37yKuXTEvz+uxfmukkiIqelEBhjZsZXPv5OjnR086Xv7qC8JMH176zMdbNERDLSDeTGQX4sj/s+sZzLqsv4/Lpt/PLXI54tKyKSMwqBcVKUiPHgrStYWF7Mmkca2Llft5cQkYlHITCOyooTPHLbSqYXxrnlwa3sbe3IdZNERIZQCIyz5O0lVtLb38+nHtyi20uIyISiEDgPFs+ZzoO3ruCt4538wXd0ewkRmTgUAufJ8pqZfPOTl/PygRN85v/o9hIiMjEoBM6jqy6aw1c/fgk/b2zlC4+/oNtLiEjO6TqB8+zjl1fR2t7F32x6hfKSBF/+yDLdXkJEckYhkANr3v82DrV1c//TuyktyudPP7CERFyDMhE5/xQCObK2/iIOtXXx90818k9b9vK7ly3gphXVLJk7PddNE5EIsRFu+z/h1NXVeUNDQ66bMWb6+52nX2vh8YYmfvzSW/T0Octryli9oobfuaSSkgJltIicOzN71t3rMu5TCEwMrW1dPPH8PtZtbaLxYBsliRgfftd8blpRzaXVZVo3EJGzds4hYGb1wN8BMeDb7v6VtP0FwCMknxHQCtzk7nvMrBzYAKwAvuPut6e853LgO0ARyWcVfH6kp5HB1A6BAe7Oc3uPsH5rE9974QAne/p4+9xp3LSiho9dtoBZJYlcN1FEJplzCgEziwGvAtcAzSQfN3mzu7+UUuezwCXu/hkzWw18zN1vMrMS4DLgYuDitBB4BvgzYAvJEPiGuz85UluiEAKpTnT28P0XD7B+axPbmo6SiOVxzbK5rF5RzXvfNpu8PI0ORGR0I4VANpPOK4FGd98dDrYOWAW8lFJnFXBn2N4A3Gtm5u7twH+a2eK0BlUCM9z9V+H1I8BHgRFDIGqmF+Zz88oabl5ZwytvHmf91iaeeH4fP3jxAFUzi7jx8mpurKtifllRrpsqIpNUNuclLgCaUl43h7KMdcIziY8B5aMcs3mUYwJgZmvMrMHMGlpaWrJo7tR00bwZ/M8PL2PL/7iav7/5MmrLS7jnJ6/y3ruf4taHnuHJ7Qfo7u3PdTNFZJKZ8KefuPv9wP2QnA7KcXNyriCeXDD+8Lvm03S4g39uaOLxhmb+5J+eo7wkwe8uT55quniOTjUVkdFlEwL7gOqU11WhLFOdZjOLA6UkF4hHOmbVKMeUUVTPKuYL117I53/77Tz9Wgvrn2nioZ/v4R//43XqFs7kphXV/M4llRQnJnzWi0iOZPPXYSuwxMwWkfxDvRr4RFqdjcAtwC+BG4CnRjrTx90PmNlxM3s3yYXhTwN/fxbtFyCWZ1x14RyuunAOLSe6eOL5ZtZtbeK/bXiRL3/vpcFTTd9VVapTTUVkiGxPEf0g8HWSp4g+6O7/28zuAhrcfaOZFQKPkjwT6DCwOmUheQ8wA0gAR4Fr3f0lM6vj1CmiTwJ/qlNEx4670/BG8lTTH7yYPNX0onnTuWlFNR+9dAEzdaqpSGToYrGIO9HZw/deOMD6rXt5ofkYiVge1108j9UrqnnPBeU61VRkilMIyKCX9h/n8YbkqabHTvZQPauI37u8mhvqqqgs1ammIlORQkCG6ezpY/PON3m8oYmfN7aSZ/Cbb6/gphU1XP2OOeTHdFdTkalCISAj2tvaweMNTfzzs028dbyL2dMSfHx5Fb+3opq3VUzLdfNE5BwpBCQrvX39PP1aC+ueaeKpVw7S2++sqJ3Jh981n7dVTKNmVjHzy4qIaQ1BZFJRCMgZO3iik395bh/rtzbx+qH2wfL8mFE9s5ia8mIWziqmpryE2vJiFpYXUzWzmML8WA5bLSKZKATkrLk7+4918kZrO3tbO9jT2sHew+280drB3tYOTnT1DtY1g3kzCllYXszCWSXJoCgvprY8uT2jMD+HPRGJrnO9gZxEmJmxoKyIBWVF/Mbbhu5zdw63d/PG4WQgvNHawRshIH76ykEOtXUNqT+zOJ+a8hIWziqmtjw5ilgYRhQV0wt0IZtIDigE5KyZGeXTCiifVsDympnD9rd39bL3cAdvtLaHgEiGxXN7j/D9F/fTnzIILcqPsbC8mJpZydHD4DTTrBLmlxUS19lKIuNCISDjpqQgzjsqZ/COyhnD9nX39rPv6MlTARGmmV4/1M6/v9pCV8odUeN5xoKZRSwMo4iBsKidXULNLK1DiJwLhYDkRCKex6LZJSyaXTJsX3+/89aJzsF1hzcOtyfXIlo72Lb3CMc7e4fUnzujYHANYnCaKYRFWbFujyEyEoWATDh5eUZlaRGVpUW8+4Lhj6U42tHNntaOwcXqgWmmp19tYcOJoesQpUX5Q6aZFs4K6xDlJcyZXqBbZkjkKQRk0ikrTnBpcYJLq8uG7TvZ3Ze2DpH8vX3fMZ7c8SZ9KQsRBfG8EA4DwVA8+LpqZpGumpZIUAjIlFKUiHHhvOlcOG/4Q3V6+vrZf/Tk4CL1G4faB0cR/9nYQmfPqXWIWJ4xv6xw6DTTrFNhoWc0yFShb7JERn4sL/yrf/g6hLtz8ERXWKRuZ+/hcE1Eazubth/gaEfPkPoV0wvCxXLJKaba2adGETOL83W6q0waCgERkqe7zp1RyNwZhaxcNGvY/mMne8LFcu1Dppt+0djKvxwf+lC86QXxMHooSbmyuph5MwqZVhCnpCBOcSKmoJAJIasQMLN64O9IPlTm2+7+lbT9BcAjwOUkHyt5k7vvCfu+CNwG9AF/5u6bQ/ke4EQo7z3d1WwiE0FpUT7vrCrlnVWlw/Z19vTRdDh5muupkOjgpQPH2bzzTXr7h1+VbwYliXgIhdhgOJQUnCorKYgzLRFnWmFqeZxpYd+p98dJxLV+IWdn1BAwsxhwH3AN0AxsNbON7v5SSrXbgCPuvtjMVgN3AzeZ2VKSj6NcBswHfmJmb3f3vvC+q9z90Bj2R+S8K8yPsWTudJbMHb4O0dvXz4FjydNdW9o6aevqo72rl/auXtrC7/auPk6E7cPtHUPKu/v6M3zicIl43qnwSKQGxqlAmT4saIYG0OAoJT+ms6YiJJuRwEqgMeVxkeuAVUBqCKwC7gzbG4B7LTnWXQWsc/cu4HUzawzH++XYNF9kYovH8qieVUz1rOKzen93b/+pwOjupa1zIDz6hgRJW/ep4GjrStY70tFN05GOwfL27l6yuVXYwChlcDQSRh2po5BphclRykiBolHK5JBNCCwAmlJeNwNXnK6Ou/ea2TGgPJT/Ku29C8K2Az8yMwe+5e73Z/pwM1sDrAGoqanJorkiU0cinkcinhiTZ0L39zsne1LDo48TXT3DAiW53TcYLm2dybLmIx20d58Kmu7eLEcpsbyhgZIyrZU+ChkMmtTRTKFGKeMplwvDV7r7PjObA/zYzF5x96fTK4VwuB+SdxE9340UmSry8mzwD+2cMTheT1//kEBp6+o5FR4ZAmWgvK2rl2Md3ew7kjKiyXKUAlCSSA+UGNMK8k+NUtIDJW10kvq+grhuOZJNCOwDqlNeV4WyTHWazSwOlJJcID7te9194PdBM3uC5DTRsBAQkYkpP5ZHWXFiTG7N4Z4cpQwGSmdKkHSnTHtlCJT2rl72HT05pLwry1FKfszSFtljTCvMHxyNZFyQz7i+kqw/GUcp2YTAVmCJmS0i+Qd8NfCJtDobgVtIzvXfADzl7m5mG4HHzOxrJBeGlwDPmFkJkOfuJ8L2tcBdY9IjEZl0zIziRDx5Ed7w9fUz1tPXT0dX3+BayYnOXoYtyHf3pYRLmPbq7uXYyR72h1AZ2J/hBK+MilNGKac98ytxqnzImV+JoQv5BfG883Ia8aghEOb4bwc2kzxF9EF332lmdwEN7r4ReAB4NCz8HiYZFIR6j5NcRO4FPufufWY2F3gidDAOPObuPxyH/olIBOXH8igtzqO0+NwfZOTudPb0Dw2MIYGSNgXWnRyxtHUm11v2H+0MaynJOqlXpo8kHqbvBgLlXz/3XooSYz99pSeLiYicR719/bR3n5rWGjg9eMiCfFpZW1cP3/zk5Wc93aQni4mITBDxWB6lRXmUFk2Mx63qBF4RkQhTCIiIRJhCQEQkwhQCIiIRphAQEYkwhYCISIQpBEREIkwhICISYZPqimEzawHeOIO3zAam0kNrplJ/1JeJSR4BysIAAATBSURBVH2ZmM6lL4cA3L0+085JFQJnyswaptJjK6dSf9SXiUl9mZjGsy+aDhIRiTCFgIhIhE31EMj4yMpJbCr1R32ZmNSXiWnc+jKl1wRERGRkU30kICIiI1AIiIhE2JQNATOrN7NdZtZoZmtz3Z4BZvagmR00sx0pZbPM7Mdm9lr4PTOUm5l9I/ThRTNbnvKeW0L918zslpTyy81se3jPN2wcH1JqZtVm9jMze8nMdprZ5ydrf8ys0MyeMbMXQl++HMoXmdmW8PnrzSwRygvC68awvzblWF8M5bvM7LqU8vP6nTSzmJk9b2bfn8x9MbM94TuwzcwaQtmk+46Fzyozsw1m9oqZvWxm78l5X9x9yv2QfBbyr4ELgATwArA01+0KbXs/sBzYkVL2VWBt2F4L3B22Pwg8CRjwbmBLKJ8F7A6/Z4btmWHfM6GuhfdeP459qQSWh+3pwKvA0snYn3D8aWE7H9gSPvdxYHUo/wfgT8L2Z4F/CNurgfVhe2n4vhUAi8L3MJaL7yTwBeAx4Pvh9aTsC7AHmJ1WNum+Y+GzHgb+KGwngLJc92XcvoC5/AHeA2xOef1F4Iu5bldKe2oZGgK7gMqwXQnsCtvfAm5OrwfcDHwrpfxboawSeCWlfEi989Cv7wLXTPb+AMXAc8AVJK+2jKd/r4DNwHvCdjzUs/Tv2kC98/2dBKqAnwIfAL4f2jZZ+7KH4SEw6b5jQCnwOuGEnInSl6k6HbQAaEp53RzKJqq57n4gbL8JzA3bp+vHSOXNGcrHXZhCuIzkv6AnZX/C9Mk24CDwY5L/2j3q7r0ZPn+wzWH/MaCcM+/jePk68N+B/vC6nMnbFwd+ZGbPmtmaUDYZv2OLgBbgoTBN920zKyHHfZmqITBpeTLCJ9V5u2Y2Dfh/wJ+7+/HUfZOpP+7e5+6XkvxX9Ergohw36ayY2YeAg+7+bK7bMkaudPflwPXA58zs/ak7J9F3LE5yKvib7n4Z0E5y+mdQLvoyVUNgH1Cd8roqlE1Ub5lZJUD4fTCUn64fI5VXZSgfN2aWTzIA/snd/yUUT9r+ALj7UeBnJKc9yswsnuHzB9sc9pcCrZx5H8fDe4GPmNkeYB3JKaG/Y3L2BXffF34fBJ4gGdCT8TvWDDS7+5bwegPJUMhtX8ZrHi+XPyQTdzfJ4dfAwtWyXLcrpX21DF0T+FuGLgx9NWz/DkMXhp4J5bNIzi3ODD+vA7PCvvSFoQ+OYz8MeAT4elr5pOsPUAGUhe0i4D+ADwH/zNDF1M+G7c8xdDH18bC9jKGLqbtJLqTm5DsJ/BanFoYnXV+AEmB6yvYvgPrJ+B0Ln/UfwIVh+87Qj5z2ZVy/gLn8Ibmy/irJed2/ynV7Utr1f4EDQA/JfxncRnL+9afAa8BPUv6DGnBf6MN2oC7lOH8INIafP0gprwN2hPfcS9oi1Bj35UqSQ9cXgW3h54OTsT/AJcDzoS87gDtC+QXhf6xGkn9EC0J5YXjdGPZfkHKsvwrt3UXK2Rm5+E4yNAQmXV9Cm18IPzsHPmsyfsfCZ10KNITv2b+S/COe077othEiIhE2VdcEREQkCwoBEZEIUwiIiESYQkBEJMIUAiIiEaYQEBGJMIWAiEiE/X/ePnOXYCh8yQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(amount,1/np.array(amount)**0.5)          #for comparision with 1/sqrt(n) expectation of error decrease with size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SDkUpq3Y3alI"
   },
   "source": [
    "**error-rate seems to be proportional to  1/sqrt(2) where  n  is the training set size, just as expected**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The plot below shows the same data but on a log-log scale**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 312
    },
    "id": "kmcNMdqkqH3u",
    "outputId": "81e8e053-36d4-4ce0-d95b-9215a387ad5b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Error rate in percantage, log')"
      ]
     },
     "execution_count": 119,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3gU5fbA8e9JQiihSYfQi3QhEJAOIhaUarlgQbAB9oLXn957VWy3eL0qKigqUq6KiAICig3pPfSuSA+9hV6SnN8fM7muIWVDspls9nyeZ5/szL47c2ZmN2fnfd95R1QVY4wxoSvM6wCMMcZ4yxKBMcaEOEsExhgT4iwRGGNMiLNEYIwxIc4SgTHGhDhLBCZfEZEZItI/p8vmJyJyUkRqeh1HWkRkgIjM9zoOf4jIdhHp4nUcOcESQTa5H4Yz7pcr5fFuLscwW0TOuus+JCKTRKSin+/tJCK7Ax1jJjH47rvkVPvzjqwsS1W7qurYnC6bFe4+TfbZht0i8oWItMjCMoaKyCc5HRuAqhZV1a2BWLYJTpYIckZ398uV8ng4rUIiEpHGvPCsrCiD8g+ralGgNlAUeD0ry/WS774DdvLH/flpSrm09l8etsfdnmJAK2ATME9ErvY2LGMuZokggNzT3AUi8qaIHAaGisgYEXlPRL4VkVPAVSJS3/1Vf0xE1otID59lXFQ+o3Wq6jFgCtDUZxl3i8hGETkhIltFZJA7PwqYAVTy+fVaSUTCROQZEflNRA67v2ZLpbONG0Wkm890hIgcFJFmIlJIRD5xl3FMRJaJSPks7L9O7q/p/xORfcBoEblMRKa76zjqPq/s857ZInKfz/6fLyKvu2W3iUjXSyxbQ0TmuvvwJxEZ7s8vdnXsVtXngY+Af/ksc5iI7BKR4yKyXETau/OvB/4C9HGPyWp3fprHMZ19V1tE5ohIgnuWOMHnNXVf9z3uJ0XktIioT7l73PUdFZHvRaRaBuvr4X52j7n7tb7Pa9tF5CkRWePGM0FECmW279z3tnE/Nwnu3zY+r/l9TESkjPtZOSYiR0RknoiEua9VEecs+qD7WX3XnV9LRH525x0SkU9FpGQ6y/f7O5MXWSIIvCuBrUB54FV33u3u82LAEmAa8ANQDngE+FRE6vosw7d8hvWnIlIauAnY4jP7ANANKA7cDbwpIs1U9RTQFffXq/vY48bQC+gIVAKOAsPTWeV44Daf6euAQ6q6AugPlACqAKWBwcCZjOJPQwWgFFANGIjzmR3tTld1l5dRVdyVwGagDPAaMEpE5BLKfgYsdbdjKNAvi9sBMAloJk4CBliGk7BLucufKCKFVPU74O/ABPeYNHHLp3kc01nXyzifqcuAysA7qQuo6p5UZ2OTgc8BRKQnTjK6CSgLzMM51hcRkcvd1x53y34LTBORSJ9ifwKuB2oAVwAD0ttJPsstBXwDvI2z398AvnE/45C1YzIE2O3GV97dNhXnDHs6sAOoDkSn7ANAgH/gfAfq43yOh6az/Kx8Z/IeVbVHNh7AduAkcMzncb/72gBgZ6ryY4BxPtPtgX1AmM+88cDQtMqnE8Ns4DSQACiwCqiaQfkpwGPu807A7lSvbwSu9pmuCFwAItJYVm3gBFDEnf4UeN59fg+wELgii/uzi09s54FCGZRvChxNtS/u89n/W3xeK+LunwpZKYuTcBJTttF9/RPgk3RiumifuvPrucuMTud9R4Em7vOh6S0/reOYxmvjgA+Aymm8pkDtVPP+D1gOFHanZwD3+rwe5n7GqqWxvOeAL1KVjQc6+RzTO31efw14P524BwDz3ef9gKWpXl/klsnqMXkJ+DqN7W4NHEzrs53GMnoBK9P5rPr9ncmLDzsjyBm9VLWkz+NDn9d2pVHed14lYJeqJvvM24HzyySjZaT2qKqWwPm1lfIrEAAR6Soii91T4mPADTi/etNTDZjsnkYfw/mQJ+H8kvoDVd3ivt5dRIoAPXB+qQH8F/ge+FxE9ojIayJSwI9t8XVQVc/6bEsRERkpIjtE5DgwFygp6bed7POJ9bT7tGgWy1YCjvjMA/+OSWrROP+EjwG41SUb3WqPYzhnT+kelywex6dxftEudats7sloucBjOJ/jlDO2asAwn8/AEXd50WksohLOZxYA97O8K1XZfT7PT5P+MUh3ua6U70ZWj8m/cc6Sf3Cr1Z5x51cBdqhqYuo3iEh5EflcROLdz9onpL+//f7O5EWWCAIvreFdfeftAaqk1Fe6quL8ospoGWmvTHUt8AowXBwFga9wGo/Lq2pJnFP3lCqPtJa9C+iaKrkVUtX4NMrC79VDPYENbnJAVS+o6ouq2gBog1OtcZe/25JOfEOAusCVqloc6ODOT6+6JyfsBUq5iS5FlUtYTm9ghaqeEqc94GmcKpPL3OOSQDrHxY/j+Aequk9V71fVSsAgYISI1E5dzq2CHAv8SVV9/5HuAgal+gwUVtWFaaxuD84/wpRlCs7+Se/z4q8/LNeV8t3I0jFR1ROqOkRVa+L8WHlSnIb7XUBVSbsjwt9xjkNj97N2J+l/zrL6nclTLBF4bwnOL6SnRaSAiHQCuvN7PeWlGIvzS6QHEAkUxDn9TXR//V3rU3Y/UFpESvjMex94NaVxUETKunXG6fncXeYD/H42gIhcJSKN3V/rx3FOlZPTXoTfiuG0Cxxz65BfyObyMqWqO4A4nMb+SBFpjXOMMuUm42gReQG4D6duGpztSMStlhCR53Hq/lPsB6r7/EDI7DimXu+t8nsj+lGcf2jJqcoUx6ku+auqpm57eh94VkQaumVLiMit6azuC+BGEbnaPeMbApzDqRbMjm+By0XkdnE6IfQBGgDTs3pMRKSbOA3kgpNwk3D2x1KcpPJPEYkSp4NDW/dtxXCqfRNEJBr4cwaxZvU7k6dYIsgZ0+SPvS8m+/tGVT2P8wHuChwCRgB3qeqmSw3GXeYw4DlVPQE8ivNlPYrT8DzVp+wmnF/0W93T2krue6finEafABbjNKSmt769OHW3bYAJPi9VAL7ESQIbgTk41UXZ8RZQGGdfLQa+y+by/HUHTn3yYZwzrgk4/+zSU0lETuL8I1kGNMapM//Bff17nNh/wanuOMsfqzYmun8Pi8iKzI5jGloAS9wYpuK0JaS+dqAZztnVm76fXwBVnYzTw+lzt1pkHc5n9CKquhnn1/I7OMelO04X4PMZxJcpVT2McxY5BGe/Pw10U9VDbpGsHJM6wE84x2MRMEJVZ6lqkhtvbZyuy7uBPu57XsTZRwk4jdaTMgg3S9+ZvEbchg1jTBaI0x1zk6oG/IzE+MeOyaWzMwJj/CAiLcTpVx4mTj//nji9doxH7JjknGC6UtMYL1XAqRoojVN98ICqrvQ2pJBnxySHWNWQMcaEOKsaMsaYEBd0VUNlypTR6tWrex2GMcYEleXLlx9S1bJpvRZ0iaB69erExcV5HYYxxgQVEUl9lfb/WNWQMcaEOEsExhgT4iwRGGNMiLNEYIwxIc4SgTHGhDhLBMYYE+IsERhjTIgLmURw6OQ5Xpm+gWOnszUyrjHG5DshkwgW/naYjxds46rXZ/Ppkh0kJdsYS8YYAyGUCHo0qcS3j7Xn8vLF+OvkdfQcPp/lO454HZYxxnguZBIBQL0Kxfl8YCveuS2GwyfPc/N7i3hiwioOHD+b+ZuNMSafCqlEACAidG9SiZlDOvLwVbX5Zs1ernp9NiPn/Mb5xOzeTtcYY4JPyCWCFEUiI3jqurr8+GQHWtcqzT9mbOL6t+Yye/MBr0MzxphcFbKJIEW10lF81L8Fo+9uAcCA0cu4b2wcOw+f9jgyY4zJHSGfCFJcVbcc3z3egWe61mPRb4fo8uYc/vPDZk6fT/Q6NGOMCShLBD4iI8IY3LEWPz/ViRsbV+Sdn7fQ5T9zmL5mD3ZLT2NMfmWJIA3lixfizT5N+XJwa0oWieThz1Zy24eL2bTvuNehGWNMjrNEkIHY6qWY9kg7Xu3diE37TnDj2/MZOnU9CacveB2aMcbkGEsEmQgPE+64shqzn+rE7S2rMm7Rdq76z2zGL91pVycbY/IFSwR+Klkkkpd7NWL6I+2pXbYoz05aS6/hC1ix86jXoRljTLZYIsiiBpWKM2FQK4b1bcrBE+e4acRChnyxmgMn7OpkY0xwskRwCUSEnk2jmTmkIw92qsW01Xvo/PocPpy71a5ONsYEHUsE2RBVMIKnr6/HD090oGWNUrz67Ua6DpvL3F8Oeh2aMcb4zRJBDqheJoqPB7Tg4wGxJCUrd328lIHj4th1xK5ONsbkfZYIclDneuX5/okOPH19XeZvOcTVb8zhjR82c+Z8ktehGWNMuiwR5LCCEeE82Kk2Pw/pRNdGFXj75y10eWMO367da1cnG2PyJEsEAVKhRCGG9Y3hi0GtKV64AA9+uoI7PlrCL/tPeB2aMcb8gSWCAGtZoxTTH2nHy70asWHvcboOm8eL09aTcMauTjbG5A2WCHJBeJjQr1U1Zg3pRN8WVRizcDudX5/NhGU7Sbark40xHrNEkIsui4rk1d6NmfZwO2qUieL/vlpL7xELWGlXJxtjPGSJwAONokswcXBr3urTlL0JZ+k9YiFPTVzNwRPnvA7NGBOCLBF4REToFRPNz091YnDHWny9Kp7Or8/mo3lbuZBkVycbY3KPJQKPFS0YwTNd6/H94x1oXv0yXvlmI12HzWP+r4e8Ds0YEyIsEeQRNcsWZfSAFozqH8uFpGTuHLWEwf9dblcnG2MCLsLrAMzvRISr65enbe0yjJq/jXd/3sKszQd4oFMtBnesRaEC4V6HaIzJh+yMIA8qVCCch66qzcwhHbmmQXne+ulXurwxh+/W7bOrk40xOS5giUBEqojILBHZICLrReSxNMp0EpEEEVnlPp4PVDzBqFLJwrx7ezPG39+KqMgIBn+ynLs+XsqWAye9Ds0Yk49IoH5hikhFoKKqrhCRYsByoJeqbvAp0wl4SlW7+bvc2NhYjYuLy/F487rEpGT+u3gHb/z4C2fOJ3FPuxo80rk2xQoV8Do0Y0wQEJHlqhqb1msBOyNQ1b2qusJ9fgLYCEQHan35XUR4GHe3rcGspzpxc7PKfDhvK53/M4dJK3ZbdZExJlsyPSMQkSfTmJ0ALFfVVX6tRKQ6MBdopKrHfeZ3Ar4CdgN7cM4O1me0rFA9I0ht9a5jPD91Pat3HaN5tct4sUdDGkWX8DosY0weldEZgT+J4DMgFpjmzuoGrAGqAxNV9bVM3l8UmAO8qqqTUr1WHEhW1ZMicgMwTFXrpLGMgcBAgKpVqzbfsWNHhjGHiuRk5csVu/nXjE0cOX2e21tW5alr63JZVKTXoRlj8pjsJoK5wA2qetKdLgp8A1yPc1bQIIP3FgCmA9+r6ht+BLodiFXVdK+msjOCiyWcucBbP/3CuEU7KFYogiHX1uX2llUJDxOvQzPG5BHZbSMoB/gOgnMBKK+qZ1LNT71SAUYBG9NLAiJSwS2HiLR04znsR0zGR4nCBXihe0O+fbQ99SsU57kp6+j+znyWbT/idWjGmCDgzwVlnwJLRORrQHCqhj4TkShgQwbvawv0A9aKSEpbwl+AqgCq+j5wC/CAiCQCZ4C+ai2fl6xuhWJ8dv+VfLN2L69+s5Fb319Er6aVePaG+pQvXsjr8IwxeZRf3UdFJBbnHzvAAlX1rG7Gqob8c/p8IiNm/cYHc7dSIFx49Oo63N22BpERdg2hMaEoJ7qPXgCSgST3ucnjikRG8NR1dfnxyQ60rlWaf8zYxPXD5jLnl4Neh2aMyWMyTQTuFcGfAmVw2gs+EZFHAh2YyRnVSkfxUf8WjL67BarQ/+Ol3D8uzgazM8b8jz+9htYArVX1lDsdBSxS1StyIb6LWNXQpTuXmMTH87fzzs+/kpisDO5Qkwc61aZwpA1mZ0x+l92qIcGpEkqR5M4zQaZgRDgPdKrFz0M6cX3DCrz98xa6vDGHb9futauTjQlh/iSC0Ti9hoaKyFBgMU63UBOkKpQoxNu3xTBhYCuKFYrgwU9XcOeoJfy6/4TXoRljPOBvr6FmQDt3cp6qrgxoVBmwqqGclZiUzGdLd/L695s5fT6J/m2q81iXOhS3weyMyVcu6cpiESmV0UJV1ZOrlSwRBMbhk+d4/YfNfL5sF6WjCvJM13rcFBNNmF2dbEy+cKmJYBug/N4ekFJQAFXVmjkdqD8sEQTWmt3HeGHqelbuPEZM1ZK81KMRjSvbYHbGBLtsjTWU11giCLzkZGXSynj+OWMTh0+do2+LKjx1bV1KFy3odWjGmEvkyf0ITPAKCxNuaV6Zn5/qyD1tazAxbjdXvT6bsQu3k5iU7HV4xpgcZonApKt4oQI8160BMx5rT+PKJXhh6nq6vTOfJVttXEBj8hNLBCZTdcoX45N7r+S9O5px4mwifT5YzKPjV7Iv4azXoRljcoAlAuMXEaFr44r89GRHHr26Dt+t30fn/8xmxOwtnEtMynwBxpg8K8uJQEQ2uo+HAxGQydsKR4bz5DWX89MTHWlbuwyvfbeZ69+ax6xNB7wOzRhzibKcCFS1Ps7FZdtyPhwTLKqWLsKHd8Uy9p6WCHD3mGXcO2YZOw/bYHbGBBu/EoGIVBORLu7zwsB5Vf0moJGZoNDx8rJ893gHnu1aj8VbD9PlzTm89dMvnL1g1UXGBAt/hqG+H/gSGOnOqgxMCWRQJrhERoQxqGMtZg7pxLUNyvPWT79y7ZtzrbrImCDhzxnBQzh3JzsOoKq/4tyXwJg/qFCiEO/e3oxP7r2SiHDh7jHLGDgujt1HrbrImLzMn0RwTlXPp0yISAS/DzdhzEXa1SnDd4914Onr6zLv10N0eWMOw2dZ7yJj8ip/EsEcEfkLUFhErgEmAtMCG5YJdpERYTzYqTY/DelIx8vL8u/vN9P1rXnM//WQ16EZY1LxJxE8AxwE1gKDgG+BvwUyKJN/RJcszMh+sYy+uwVJqtw5agkPfbaCvQlnvA7NGOOyQedMrjl7IYkP5m5l+KwthIcJj3epw91ta1Ag3K5rNCbQsjX6qIis5eI2gQQgDnhFVXN14BlLBMFv5+HTvDhtPTM3HaBOuaK83KsRrWqW9josY/K17I4+OgP4BrjDfUzDSQL7gDE5FKMJIVVLF2HUgBZ8eFcsp88n0feDxTz++UoOnLCxi4zxQoQfZbqoajOf6bUiskJVm4nInYEKzOR/1zQoT7vaZRgxewsj52xl5sYDPHHN5dzVuhoRVl1kTK7x59sWLiItUyZEpAUQ7k4mBiQqEzIKR4Yz5Nq6fP9EB5pWLclL0zfQ/d0FLN/hyZ1QjQlJ/iSC+4BRIrJNRLYDo4D7RSQK+EcggzOho0aZKMbd05L37mjGsdPnufm9Rfx54moOnTzndWjG5Ht+9xoSkRIAqpoQ0IgyYY3F+d+pc4m88/MWPpq3lSKR4fz5+nrc3rIq4WGS+ZuNMWnK9j2LReRGoCFQKGWeqr6UYxFmgSWC0LHlwAmem7KeRVsP0zi6BC/3akTTKiW9DsuYoJStXkMi8j7QB3gEEOBWoFqORmhMGmqXK8Zn91/JsL5N2X/8LL1HLODZSWs5eup85m82xvjNnzaCNqp6F3BUVV8EWgOXBzYsYxwiQs+m0cwc0pF72tbgi7hddP7PbD5fupPk5OC6GNKYvMqfRJAyFsBpEakEXAAqBi4kYy5WrFABnuvWgG8ebUedcsV4ZtJabn5/IeviPW2yMiZf8CcRTBeRksC/gRXAdmB8IIMyJj31KhRnwqBW/OfWJuw6cpoe787nha/XkXDmgtehGRO0/BlioqCqnkt5jtNgfDZlXm6zxmKTIuHMBd74YTP/XbyDUlGRPNu1Pjc1i0bEehcZk1p2h5hYlPJEVc+53UcXZVDemFxRonABXuzZiKkPt6NKqSIMmbiaP41cxKZ9x70OzZigkm4iEJEKItIc5z4EMSLSzH10AorkWoTGZKJRdAm+GtyGf93cmC0HTnLj2/N5efoGTpy16iJj/JHRWEPXAQNw7lH8hs/8E8BfAhiTMVkWFib0aVGVaxtU4LXvN/Pxgm1MW72Hv95Ynx5NKll1kTEZ8KeN4GZV/SqX4smUtREYf6zadYznpqxjbXwCrWuW5uVeDaldrpjXYRnjmezej6AgcDNQHZ8zCLuy2OR1ScnKZ0t38u/vNnH6fBL3tq/Bo53rEFXQn0F3jclfsttY/DXQE2ek0VM+j8xWWkVEZonIBhFZLyKPpVFGRORtEdkiImtEpFlayzLmUoSHCf1aVWPWU53oHRPNyDlb6fLGHGas3Uuw3ZnPmEDy54xgnao2yvKCRSoCFVV1hYgUA5YDvVR1g0+ZG3CGrrgBuBIYpqpXZrRcOyMwlypu+xGe+3o9G/cep32dMrzUsxE1ykR5HZYxuSK7ZwQLRaRxVleqqntVdYX7/ASwEYhOVawnME4di4GSbgIxJsfFVi/FtIfb8kL3BqzaeYzr3pzLp0t2eB2WMZ7zJxG0A5aLyGa3+matiKzJykpEpDoQAyxJ9VI0sMtnejcXJwtEZKCIxIlI3MGDB7OyamP+ICI8jLvb1mDmUx1pXas0f528jle/2WDjFpmQ5k+rWdfsrEBEigJfAY+r6iVd6aOqHwAfgFM1lJ14jAEoV6wQo/rH8tL0DXw4bxs7j5zmrT4xFI4Mz/zNxuQzmZ4RqOoOVd2BM/ic+jwyJSIFcJLAp6o6KY0i8UAVn+nK7jxjAi4iPIwXezTk+W4N+GHDfvp8sIgDx896HZYxuc6f+xH0EJFfgW3AHJxB52b48T7Bua3lRlV9I51iU4G73N5DrYAEVd3rb/DGZJeIcE+7GnzQL5Zf95+k1/AFNkSFCTn+tBG8DLQCflHVGsDVwGI/3tcW6Ad0FpFV7uMGERksIoPdMt8CW4EtwIfAg1neAmNywDUNyjNxcGuSVLnlvUXM3nzA65CMyTX+dB+NU9VYEVkNxKhqsoisVtUmuRPiH1n3URNIexPOcM+YOH7Zf4KhPRrSr5XdjM/kD9ntPnrMbfCdC3wqIsPw44IyY4JRxRKFmTi4NR0vL8tzU9bxyvQNJFmPIpPP+ZMIegKngSeA74DfgO6BDMoYLxUtGMEH/ZozoE11Ppq/jcGfLOf0+USvwzImYPxJBOWASFVNVNWxOHX5NnqXydciwsMY2qMhL3RvwMyN++kzcrH1KDL5lj+JYCKQ7DOd5M4zJt+7u20NPrwrlt8OOj2KNu61HkUm//EnEUSo6vmUCfd5ZOBCMiZvubp+eb4Y5PQouvV961Fk8h9/EsFBEemRMiEiPYFDgQvJmLynUXQJpjzUlqqlinDPmGX8d7GNUWTyD38SwWDgLyKyU0R2Av8HDAxsWMbkPSk9ijrVLcdzU9bxsvUoMvlEhmMNiUg48ICqtnK7kKKqJ3MlMmPyoKiCEXx4VywvT9/AqPnb2HH4NG/f1pQikXazGxO8MjwjUNUknNFHUdWTlgSMcW54M7RHQ4Z2b8DPm/bzp5GL2G89ikwQ86dqaKWITBWRfiJyU8oj4JEZk8cNaFuDj/rHsvXgKXoNX8CGPdajyAQnfxJBIeAw0BnnQrLuQLdABmVMsOhczxmjSBVufX8hszZZjyITfDIdayivsbGGTF60L+Es945dxsa9xxnaoyF3ta7udUjG/EG2xhoSkctFZKaIrHOnrxCRv+V0kMYEswolCvHFoNZ0rleO579ez4vT1luPIhM0/Kka+hB4FrgAoKprgL6BDMqYYBRVMIKR/WK5u211Ri/YzqD/xnHqnI1RZPI+fxJBEVVdmmqefbqNSUN4mPBC94a82KMhP286YD2KTFDwJxEcEpFauLenFJFbALuLmDEZ6N+mOh/1j2X7IetRZPI+fxLBQ8BIoJ6IxAOP41xtbIzJgNOjqI31KDJ5nj83r9+qql2AskA9VW3n3szeGJOJBpWKM+WhtlQvE8W9Y5cxduF2r0My5iL+9BoqLSJvA/OA2SIyTERKBz40Y/IH3x5FL0xdz9Cp1qPI5C3+VA19DhwEbgZucZ9PCGRQxuQ3KT2K7mlbgzELtzNwnPUoMnmHP4mgoqq+rKrb3McrQPlAB2ZMfhMeJjzfvQEv92zIrM1Oj6J9CdajyHjPn0Twg4j0FZEw9/En4PtAB2ZMftWvdXVGDWjxvx5F6/ckeB2SCXH+JIL7gc+Ac+7jc2CQiJwQEesTZ8wluKpuOSYOboMI3Pr+ImZu3O91SCaE+dNrqJiqhqlqAfcR5s4rpqrFcyNIY/KjlB5FNctGcf+4OEYv2OZ1SCZE+XNGYIwJkPLFU3oUlefFaRusR5HxhCUCYzxWJDKCkf2ac287p0fR/ePiOGk9ikwuskRgTB4QHiY8160BL/dqxOzNB7j1/UXsTTjjdVgmRPiVCEQkXEQqiUjVlEegAzMmFPVrVY1RA1qw87DTo2hdvPUoMoHnz5XFjwD7gR+Bb9zH9ADHZUzIuqpuOb58oA3hIvxp5CJ+2mA9ikxg+XNG8BhQV1Ubqmpj93FFoAMzJpTVr+j0KKpVtigD/2s9ikxg+ZMIdgF2fmpMLitXvBATBrWiS32nR9ELX68jMSnZ67BMPhThR5mtOIPNfYNzQRkAqvpGwKIyxgBOj6L37mzOP2ds5MN529i47wTD+jalYonCXodm8hF/zgh24rQPRALFfB7GmFwQHib89cYG/OfWJqyLT6DrsHn8aO0GJgeJanBdvBIbG6txcXFeh2GMJ7YePMkj41eyfs9x+reuxrM31KdQgXCvwzJBQESWq2psWq+lWzUkIm+p6uMiMg33NpW+VLVHDsZojPFDzbJFmfRgG/45YxOjF2xn6fajvHNbDLXLFfU6NBPE0j0jEJHmqrpcRDqm9bqqzgloZOmwMwJjHDM37ufPX67hzPkkhvZowJ9iqyAiXodl8qiMzgisasiYILb/+Fke/3wVi7YeptsVFfn7TY0pXqiA12GZPCijRGBDTBgTxMoXL8Qn913JU9dezox1+7jx7Xms3HnU67BMkLFEYEyQCw8THu5chy8GtSI52bm/wXuzfyPZRjE1fvI7EYhIkawsWEQ+FpEDIrIundc7iUiCiKxyH89nZfnGmD9qXq0U342aifEAABRNSURBVD7anmsalOdf323iro+XcuCE3QrTZM6fsYbaiMgGYJM73URERvix7DHA9ZmUmaeqTd3HS34s0xiTgRJFCjDijmb8vXdjlm0/Qte35jF78wGvwzJ5nD9nBG8C1wGHAVR1NdAhszep6lzgSLaiM8ZkmYhw+5VVmfZIO0oXjWTA6GX8/duNnE+04SlM2vyqGlLVXalmJeXQ+luLyGoRmSEiDdMrJCIDRSROROIOHjyYQ6s2Jn+7vHwxpj7cjjuurMoHc7dyy/sL2X7olNdhmTzIr0HnRKQNoCJSQESeAjbmwLpXANVUtQnwDjAlvYKq+oGqxqpqbNmyZXNg1caEhkIFwnm1d2Peu6MZ2w+d4sa35zFlZbzXYZk8xp9EMBh4CIgG4oGmwIPZXbGqHlfVk+7zb4ECIlImu8s1xlysa+OKzHi8A/UrFufxCasY8sVqTtntMI3Ln0RQV1XvUNXyqlpOVe8E6md3xSJSQdzLIEWkpRvL4ewu1xiTtuiShfl8YCse7VybSSt30/2d+XYHNAP4lwje8XPeH4jIeGARUFdEdovIvSIyWEQGu0VuAdaJyGrgbaCvBttlzsYEmYjwMJ68ti6f3deKU+cTuWnEQkbN34Z99UJbRmMNtQbaAI/j9BxKURzo7dbt5zobYsKYnHHk1Hme/nI1P208QOd65fj3LVdQumhBr8MyAXKpQ0xEAkVxRij1vQ/BcZxf88aYIFYqKpIP74plaPcGzP/1EF2HzWPhb4e8Dst4INNB50SkmqruyKV4MmVnBMbkvPV7Enhk/Eq2HTrFQ51q83iXOkSE2wg0+Ul2B507LSL/FpFvReTnlEcOx2iM8VDDSiWY9nA7bmlWmXdnbaHPB4vZffS012GZXOJPIvgUZ3iJGsCLwHZgWQBjMsZ4IKpgBP++tQnD+jZl874T3DBsHjPW7vU6LJML/EkEpVV1FHBBVeeo6j1A5wDHZYzxSM+m0XzzaDtqlInigU9X8OyktZw5n1ODCZi8yJ9EcMH9u1dEbhSRGKBUAGMyxnisWukoJg5uw6CONRm/dCc9h89n874TXodlAsSfRPCKiJQAhgBPAR8BTwQ0KmOM5yIjwni2a33G3dOSI6fO0+Pd+XyyeIddc5APZZgIRCQcqKOqCaq6TlWvUtXmqjo1l+Izxnisw+VlmfFYB1rWKMXfpqzjwU9XkHD6QuZvNEEjw0SgqknAbbkUizEmjypbrCBj727Js13r8eOG/XQdNpdl222U+fzCn6qhBSLyroi0F5FmKY+AR2aMyVPCwoRBHWvx5QNtiAgPo8/IRbw981eS7JaYQc+fC8pmpTFbVdWTnkN2QZkx3jtx9gJ/nbyOqav30KpmKd7qE0OFEoW8DstkIKMLyjJNBHmNJQJj8gZV5cvlu3n+6/UUKhDGv29pQpcG5b0Oy6Qju1cWG2PMRUSEW2OrMP3RdlQsUZj7xsUxdOp6zl6waw6CjSUCY0y21CpblMkPtWFAm+qMWbidm0Ys5LeDJ70Oy2RBZt1Hw9zbVBpjTLoKRoQztEdDProrlr0JZ+j+znwmxu2yaw6CRGbdR5OB4bkUizEmyHVpUJ4Zj3Xgisol+POXa+jx7gI+nr+NQyfPeR2ayYA/vYZex7nT2KS8cAcxayw2Ju9LSlY+W7KDz5ftYv2e44SHCR3qlKF3s8pcU788hSPDvQ4x5GSr15CInACigCTgDCA43UeL53Sg/rBEYExw+WX/CSavjOfrlfHsSThL0YIRXN+oAjfFRHNlzdKEh4nXIYYE6z5qjPFccrKyeNthpqyM59u1+zh5LpEKxQvRM6YSN8VUpm6FYl6HmK9lOxGISA+ggzs5W1Wn52B8WWKJwJjgd/ZCEj9u2M+UlfHM+eUgiclK/YrFuSkmmp5NK1GuuF2cltOyWzX0T6AFzg1qwBl7KE5Vn83RKP1kicCY/OXwyXNMW72Hyav2sHrXMcIE2tYuQ++YaK5rWIGoghFeh5gvZDcRrAGauj2IUkYkXamqV+R4pH6wRGBM/vXbwZNMWRnP5JXx7D56hsIFwrm+UQV6xUTTtlZpu49yNmSUCPxNtSWBlKEGS+RIVMYYk0qtskUZcm1dnrzmcuJ2HGXSini+WbOHySvjKVusID2bVKJXTDQNKxVHxBqZc4o/ZwR9gX8Bs3B6DHUAnlHVCYEP72J2RmBMaDmXmMSsTQeYtCKeWZsPcCFJubx8UXrFRNOraTSVShb2OsSgcMlVQyISBtwCzMNpJwBYqqr7cjxKP1kiMCZ0HTt9nulr9jJlZTxxO44iAq1qlKZ3TDRdG1egWKECXoeYZ2W3jSAuvTd7wRKBMQZg5+HTTF4Zz+SVu9l++DQFI8K4pkF5bmoWTfs6ZSlg7Ql/kBO9hg4BE4BTKfNV1ZPbE1kiMMb4UlVW7TrG5JXxTFu9h6OnL1A6KpLuTSrROyaaKyqXsPYEsp8ItqUxW1W1Zk4El1WWCIwx6TmfmMzcXw4yeWU8P27cz/nEZGqWjaJ302h6xURTpVQRr0P0THbbCG71qmE4LZYIjDH+OH72AjPW7mXSiniWbHMqMFpUv4zeMZW5sXFFShQJrfYEayMwxoS0+GNn/nd9wpYDJ4kMD+Pq+uXoFRPNVXXLERmR/9sTrI3AGGNw2hPWxR9n8sp4pq6O59DJ85QsUoBuV1Skd0w0zapelm/bE6yNwBhjUklMSmb+lkNMXhnP9+v3cfZCMlfVLcuw22Iong+7odroo8YYk4GT5xL5fOlO/jljE9XLRDGqfyzVSkd5HVaOuqSb14vI0z7Pb0312t9zLjxjjPFW0YIR3Ne+JuPubcmhk+foNXwBi7ce9jqsXJNRC0lfn+epRxq9PgCxGGOMp9rUKsOUB9tSumhB7vxoCeOX7vQ6pFyRUSKQdJ6nNW2MMflC9TJRTHqwDe3qlOHZSWt5cdp6EpOSvQ4roDJKBJrO87SmjTEm3yheqACj+rfg3nY1GL1gO/eMjeP42QtehxUwGSWCJiJy3L1n8RXu85TpxrkUnzHGeCI8THiuWwP+dXNjFv12iN7DF7D90KnM3xiE0k0EqhquqsVVtZiqRrjPU6bzX98qY4xJQ58WVfnk3is5cuo8PYcvYOGWQ16HlOMCdjmdiHwsIgdEZF06r4uIvC0iW0RkjYg0C1QsxhiTHVfWLM3XD7WjXLGC3PXxUj5ZvMPrkHJUIK+rHkPGvYu6AnXcx0DgvQDGYowx2VK1dBEmPdiG9nXK8Lcp63jh63X5phE5YIlAVefy++0t09ITGKeOxUBJEakYqHiMMSa7ihUqwEf9W3B/+xqMXbSDAaOXkXA6+BuRvRxpKRrY5TO92513EREZKCJxIhJ38ODBXAnOGGPSEh4m/PXGBrx2yxUs2XaY3iMWsPXgSa/DypagGHJPVT9Q1VhVjS1btqzX4RhjDH+KrcJn97fi2JkL9Bq+gPm/Bm8jspeJIB6o4jNd2Z1njDFBoUX1Unz9UFsqlihM/9FLGbdou9chXRIvE8FU4C6391ArIEFV93oYjzHGZFmVUkX46sE2XFW3LM9/vZ6/TVnLhSBrRI4I1IJFZDzQCSgjIruBF4ACAKr6PvAtcAOwBTgN3B2oWIwxJpCKFoxgZL9YXvt+EyPnbGXboVMMv70ZJYtEeh2aX2wYamOMyUFfLt/NXyatpVLJQnzUvwW1yxX1OiTgEoehNsYYk3W3NK/M+IFXcvJcIr1HLGDuL3m/p6MlAmOMyWHNq5ViykNtiS5ZmAGjlzJ6wTbycu2LJQJjjAmAypcV4asH2tClfnlenLaBv0xel2cbkS0RGGNMgEQVjOD9O5vzYKdajF+6k36jlnD01Hmvw7qIJQJjjAmgsDDh6evr8VafpqzYeYyewxfw6/4TXof1B5YIjDEmF/SKiebzga04fT6Jm0YsZNbmA16H9D+WCIwxJpc0q3oZUx9uS5VSRbh3zDI+mrc1TzQiWyIwxphcVKlkYb58oDXXNCjPK99s5Jmv1nI+0dtGZEsExhiTy4pERvDeHc15pHNtJsTt4s5RSzjiYSOyJQJjjPFAWJgw5Nq6DOvblFW7jtFz+Hx+8agR2RKBMcZ4qGfTaL4Y1JqzF5K5acRCft60P9djsERgjDEea1qlJFMfbkv1MkW4d2wcH8z9LVcbkS0RGGNMHlCxRGEmDmrDDY0q8vdvN/H0l2s4l5iUK+sO2DDUxhhjsqZwZDjv3BZD7XJFGTbzV7YfPsV7dzanTNGCAV2vnREYY0weEhYmPHHN5bxzWwxrdifQ890FbNp3PLDrDOjSjTHGXJLuTSoxcXBrEpOTuXnEQn7cELhGZEsExhiTR11RuSRfP9SOWuWKMvC/cXw8f1tA1mOJwBhj8rAKJQoxYWBrejSpRI2yUQFZhzUWG2NMHlc4MpxhfWMCtnw7IzDGmBBnicAYY0KcJQJjjAlxlgiMMSbEWSIwxpgQZ4nAGGNCnCUCY4wJcZYIjDEmxEleuHFyVojIQWAHUAY45HE4XgnlbYfQ3v5Q3naw7c/O9ldT1bJpvRB0iSCFiMSpaqzXcXghlLcdQnv7Q3nbwbY/UNtvVUPGGBPiLBEYY0yIC+ZE8IHXAXgolLcdQnv7Q3nbwbY/INsftG0ExhhjckYwnxEYY4zJAZYIjDEmxAVdIhCRkiLypYhsEpGNItLa65hyi4jUFZFVPo/jIvK413HlFhF5QkTWi8g6ERkvIoW8jik3ichj7ravD4XjLiIfi8gBEVnnM6+UiPwoIr+6fy/zMsZASWfbb3WPfbKI5GgX0qBLBMAw4DtVrQc0ATZ6HE+uUdXNqtpUVZsCzYHTwGSPw8oVIhINPArEqmojIBzo621UuUdEGgH3Ay1xPvfdRKS2t1EF3Bjg+lTzngFmqmodYKY7nR+N4eJtXwfcBMzN6ZUFVSIQkRJAB2AUgKqeV9Vj3kblmauB31R1h9eB5KIIoLCIRABFgD0ex5Ob6gNLVPW0qiYCc3D+KeRbqjoXOJJqdk9grPt8LNArV4PKJWltu6puVNXNgVhfUCUCoAZwEBgtIitF5CMRCczdnPO+vsB4r4PILaoaD7wO7AT2Agmq+oO3UeWqdUB7ESktIkWAG4AqHsfkhfKqutd9vg8o72Uw+UWwJYIIoBnwnqrGAKfIv6eG6RKRSKAHMNHrWHKLWxfcE+fHQCUgSkTu9Daq3KOqG4F/AT8A3wGrgCRPg/KYOn3frf97Dgi2RLAb2K2qS9zpL3ESQ6jpCqxQ1f1eB5KLugDbVPWgql4AJgFtPI4pV6nqKFVtrqodgKPAL17H5IH9IlIRwP17wON48oWgSgSqug/YJSJ13VlXAxs8DMkrtxFC1UKunUArESkiIoJz7EOmowCAiJRz/1bFaR/4zNuIPDEV6O8+7w987WEs+UbQXVksIk2Bj4BIYCtwt6oe9Taq3OO2iewEaqpqgtfx5CYReRHoAyQCK4H7VPWct1HlHhGZB5QGLgBPqupMj0MKKBEZD3TCGXp5P/ACMAX4AqiKMxz9n1Q1dYNy0Etn248A7wBlgWPAKlW9LkfWF2yJwBhjTM4KqqohY4wxOc8SgTHGhDhLBMYYE+IsERhjTIizRGCMMSHOEoHJVe4QCSmjp+4TkXif6chM3hsrIm/7sY6FORRrJxFJcIcz2Swic0Wkm5/vy9LFbu71EZ+KyFp3hNH5IlLUfS1HtsdnXWNE5JacXKYJbhFeB2BCi6oeBpoCiMhQ4KSqvp7yuohEuIOqpfXeOCDOj3Xk5BXH81S1mxtbU2CKiJzJpA9/J+AkkJV/4I8B+1W1sbuuujjXC+T09hhzETsjMJ5zf6G+LyJLgNdEpKWILHJ/iS9MuZLc/aU93X0+1B2zfbaIbBWRR32Wd9Kn/Gyf+1d86l6VjIjc4M5bLiJvpyw3I6q6CngJeNhdRncRWeLG+ZOIlBeR6sBg4An3LKd9WuXSWHxFIN5nXZtTLpbz2Z6XfM6e4kVktDv/ThFZ6s4fKSLhWdj3V7txrXX3Z8FL3T8meFkiMHlFZaCNqj4JbALauwMLPg/8PZ331AOuwxmj/wURKZBGmRjgcaABUBNoK84NbUYCXVW1Oc6Vmv5a4a4XYD7Qyo3zc+BpVd0OvA+86d47Yl5a5dJY7sfA/7kJ8BURqZO6gKo+796LohPOVabvikh9nKut27qvJQF3+LMh7n4YA/Rxz0QigAeyuX9MELKqIZNXTFTVlNE0SwBj3X+GCqT1Dx7gG/dX8zkROYAzJPHuVGWWqupuABFZBVTHqbbZqqrb3DLjgYF+xik+zysDE9zBzyKBbWm/JfNyqrpKRGoC1+IMsLdMRFq7o47+vnLnjOYT4A1VXS4iD+PcpGiZe7JTGP8HYquLM5BfyuB1Y4GHgNlc+v4xQcjOCExeccrn+cvALPdOZN2B9G5J6TvOUBJp/7Dxp0xWxPD7YHfvAO+6v6YHZRCnX+VU9aSqTlLVB3H+2d+QRrGhOCPwjnanBRibcuc6Va2rqkMvYbtMCLNEYPKiEvxeXz4gAMvfDNR06/PBqVrJlIhcATwHDHdn+cbZ36foCaCYz3R65XyX3Vbc+++6vaca4Ayq5lumO87ZwqM+s2cCt8jvI5OWEpFq7vNxItIyg03aDFSX32952Q/nzmeXtH9M8LJEYPKi14B/iMhKAlB9qapngAeB70RkOc4/7vRGcm2f0n0UJwE86tNjaCgw0V3GIZ/3TAN6pzQWZ1DOVy1gjoisxRlZNQ74KlWZJ4FoIKVh+CVV3QD8DfhBRNYAP+I0PANcQQa381TVs8DdbmxrgWTg/SzuH5MP2OijJiSJSFFVPenWuQ8HflXVN72OK6eISHFglKreeonvz9f7x/yRnRGYUHW/23i8HqfqZqTH8eQoVT1+qUnAla/3j/kjOyMwxpgQZ2cExhgT4iwRGGNMiLNEYIwxIc4SgTHGhDhLBMYYE+L+HzwocT8fOQO/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.log(amount),np.log(valerr))\n",
    "plt.title('Error Rate vs Training Data size on log scale' )\n",
    "plt.xlabel('Training Data Size, log')\n",
    "plt.ylabel('Error rate in percantage, log')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qc3e40p4nzyC"
   },
   "source": [
    "### 4. Second model:   *** 14 marks ***\n",
    "\n",
    "Devise what you believe to be a better neural network architecture for the problem (e.g. for MNIST you might use a fully-connected network for your initial model, and then try a convolutional net for your second model).\n",
    "\n",
    "Repeat sections 2 (regularisation experiment), and determine whether it has better validation set performance than the first model, for an intermediate size of training set (e.g. 2000 or 4000 for the MNIST data). \n",
    "\n",
    "Repeat section 3. Does your model have a different learning curve from the first? Plot the learning curves for the first and second model on the same graph, to compare them. Comment: is there a more rapid reduction of error-rate with training set size for your second model?  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mpdbXxN1rT1i"
   },
   "source": [
    "**Since it is an image dataset, I beleive convolutional network to work better.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "id": "Dcm3He0fnzyD"
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "# CNN model\n",
    "class NN2( nn.Module ):\n",
    "    \n",
    "    def __init__(self, dropout = 0): \n",
    "        super(NN2, self).__init__()\n",
    "        self.dropout = dropout\n",
    "        self.layers = nn.Sequential( \n",
    "            nn.Conv2d(1, 32, 3, 1),\n",
    "            nn.ReLU(),\n",
    "            nn.Conv2d(32, 64, 3, 1),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(2),\n",
    "            nn.Dropout(self.dropout),\n",
    "            nn.Flatten(),\n",
    "            nn.Linear(9216, 128),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(self.dropout),\n",
    "            nn.Linear(128,10))\n",
    "    \n",
    "    \n",
    "    def forward( self, x ):\n",
    "        x = self.layers( x )\n",
    "        return x\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "id": "EcjWxsPfrkGQ"
   },
   "outputs": [],
   "source": [
    "# It is a similar function like above but it defines Convolutional Model \n",
    "def my_model_2(trainloader,epo,learn = 0.001,dropout = 0):\n",
    "  global nn2\n",
    "  nn2 = NN2(dropout)\n",
    "  device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "  print(device)\n",
    "  nn2.to(device)\n",
    "  optimizer1 = optim.Adam(nn2.parameters(), lr=learn)\n",
    "\n",
    "  eploss = np.zeros(epo)\n",
    "  eperr = np.zeros(epo)\n",
    "  for epoch in range(epo): # number of times to loop over the dataset\n",
    "      current_loss = 0.0 \n",
    "      n_mini_batches = 0\n",
    "      incorrect = 0\n",
    "      total = 0\n",
    "      for i, mini_batch in enumerate( trainloader, 0 ):\n",
    "          images, labels = mini_batch\n",
    "          \n",
    "          # zero the parameter gradients\n",
    "          optimizer1.zero_grad()\n",
    "          \n",
    "          outputs = nn2(images.cuda())\n",
    "          loss = loss_function ( outputs, labels.cuda() )\n",
    "          loss.backward() # does the backward pass and computes all gradients\n",
    "          optimizer1.step() # does one optimisation step\n",
    "          \n",
    "          n_mini_batches += 1 \n",
    "          current_loss += loss.item() \n",
    "          outputs = nn2(images.cuda())\n",
    "          _, predicted = torch.max( outputs, dim=1)\n",
    "          total += labels.size(0) \n",
    "          incorrect += (predicted != labels.cuda()).sum().item()\n",
    "      eperr[epoch] = 100*incorrect/total \n",
    "      eploss[epoch]= current_loss / n_mini_batches                                                   \n",
    "      print( \"Error rate on train set: %f %%\" %(100 * incorrect/total))\n",
    "      print('Epoch %d loss: %f '  %(epoch+1, current_loss / n_mini_batches ))\n",
    "\n",
    "  return eperr, eploss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Q-K2zwCdsuDF",
    "outputId": "accb4d1c-8115-4c06-b59d-72c9f20f2fe7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda\n",
      "Error rate on train set: 18.225000 %\n",
      "Epoch 1 loss: 0.718868 \n",
      "Error rate on train set: 6.475000 %\n",
      "Epoch 2 loss: 0.254525 \n",
      "Error rate on train set: 3.250000 %\n",
      "Epoch 3 loss: 0.142429 \n",
      "Error rate on train set: 1.675000 %\n",
      "Epoch 4 loss: 0.079925 \n",
      "Error rate on train set: 0.850000 %\n",
      "Epoch 5 loss: 0.051130 \n",
      "Error rate on train set: 0.475000 %\n",
      "Epoch 6 loss: 0.029867 \n",
      "Error rate on train set: 0.100000 %\n",
      "Epoch 7 loss: 0.017355 \n",
      "Error rate on train set: 0.050000 %\n",
      "Epoch 8 loss: 0.011503 \n",
      "Error rate on train set: 0.050000 %\n",
      "Epoch 9 loss: 0.007450 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 10 loss: 0.002834 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 11 loss: 0.001348 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 12 loss: 0.000910 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 13 loss: 0.000578 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 14 loss: 0.000412 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 15 loss: 0.000337 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 16 loss: 0.000291 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 17 loss: 0.000258 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 18 loss: 0.000231 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 19 loss: 0.000202 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 20 loss: 0.000181 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 21 loss: 0.000162 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 22 loss: 0.000149 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 23 loss: 0.000135 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 24 loss: 0.000124 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 25 loss: 0.000114 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 26 loss: 0.000105 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 27 loss: 0.000096 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 28 loss: 0.000089 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 29 loss: 0.000082 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 30 loss: 0.000077 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 31 loss: 0.000071 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 32 loss: 0.000067 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 33 loss: 0.000063 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 34 loss: 0.000058 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 35 loss: 0.000054 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 36 loss: 0.000051 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 37 loss: 0.000048 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 38 loss: 0.000046 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 39 loss: 0.000043 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 40 loss: 0.000041 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 41 loss: 0.000038 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 42 loss: 0.000036 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 43 loss: 0.000034 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 44 loss: 0.000032 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 45 loss: 0.000030 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 46 loss: 0.000029 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 47 loss: 0.000027 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 48 loss: 0.000026 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 49 loss: 0.000024 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 50 loss: 0.000023 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 51 loss: 0.000022 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 52 loss: 0.000021 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 53 loss: 0.000020 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 54 loss: 0.000019 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 55 loss: 0.000018 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 56 loss: 0.000017 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 57 loss: 0.000016 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 58 loss: 0.000015 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 59 loss: 0.000015 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 60 loss: 0.000014 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 61 loss: 0.000013 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 62 loss: 0.000013 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 63 loss: 0.000012 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 64 loss: 0.000012 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 65 loss: 0.000011 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 66 loss: 0.000011 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 67 loss: 0.000010 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 68 loss: 0.000010 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 69 loss: 0.000009 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 70 loss: 0.000009 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 71 loss: 0.000009 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 72 loss: 0.000008 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 73 loss: 0.000008 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 74 loss: 0.000008 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 75 loss: 0.000007 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 76 loss: 0.000007 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 77 loss: 0.000007 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 78 loss: 0.000006 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 79 loss: 0.000006 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 80 loss: 0.000006 \n"
     ]
    }
   ],
   "source": [
    "mnist_trainset_small =  [ mnist_trainset[i] for i in range(0,4000) ] \n",
    "smalltrainloader = torch.utils.data.DataLoader( mnist_trainset_small, batch_size=64, shuffle=True)\n",
    "eperr, eploss = my_model_2(smalltrainloader, epo = 80, learn = 0.001)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 298
    },
    "id": "bRf1MbhatBkJ",
    "outputId": "41b92b0b-ece3-4c41-a7f3-2366cd020640"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Loss vs Epoch')"
      ]
     },
     "execution_count": 77,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEICAYAAAC+iFRkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de7xcZX3v8c93ZrL3Jjcu5gK5EcQUjZxyMUXwVhTUQBGs16TaQquNeqRitVWsPZTS01e1Kt6gUlQO3gARRSKNIiK+vIIEBCQEJFyTEMjmmoQQkp38zh9rTTIMM3vv7Jk1s9fa3/frNa89a61n1vPMzLN++zfPuikiMDOzYit1uwFmZpY9B3szszHAwd7MbAxwsDczGwMc7M3MxgAHezOzMcDB3gpP0lxJIanS7bZYPkk6RdIvu92OVoyaYC/pPklPS9pU8zin2+0abSSdKemb3W5HK9LA+1Tdd/2RbrdrLEm3t2O73Y6RkHS0pB11/WeTpKO63bbRbLRlOm+IiJ8MVUhSJSIG6uaVI2L7cCva3fK7Q5IARcSOLNZfEIdExKpuN8Jy68GImNXtRuTJqMnsB5P+hPqVpM9KehQ4U9KFkr4kaZmkp4BXS3qRpJ9JekLSCkkn1qzjOeUb1PMzSf8h6beSNki6QtI+NcuPlPTrdP23SDq67rX/LulXwGbg+ZJeLOlqSY9JeljSP6VlS5JOl3S3pEclXVqtp2bI4WRJD0h6RNLH02ULgX8C3p5mMrek8/9a0kpJGyXdI+k9de/rI5LWSXpQ0rvT9b8gXdYr6dNpXQ9LOk/SHg0+m970fR9cM29q+mtsmqQpkq5Myzwm6ReSdrt/pb9cLpP07fT93CTpkJrlg33He0j6jKT7JT0p6Zd17+Ud9Z+pPVv6PX8u7SsPps9702VNv2NJH5W0Nv3O7pR0TIN1v1TSQ5LKNfP+XNKt6fMjJC1Pt72HJZ09wvcw1HZ8Ytp3nkjLvqhm2WxJ35PUn26b59St+9OSHpd0r6TjRtK+romIUfEA7gOObbLsFGAA+DuSXyN7ABcCTwIvJ/mnNQlYRRIMe4DXABuBg9J11Jfva1DPz4C1wMHABOC7wDfTZTOBR4Hj09e/Np2eWvPaB4AXp22cBKwDPgz0pdMvTcueBlwHzAJ6gf8GLk6XzQUC+HL6Pg8BngFelC4/s9qmmnb/GXAgIOBPSf7ZHJ4uWwg8lLZrPPDNdP0vSJd/FlgK7JO28QfAfzT5Hi4A/r1m+v3Aj9Ln/wGcB4xLH68k+XXTaD0762+w7ExgG/CWdD3/ANxbs97BvuNz0+9hJlAGXpZ+voN+pmPxQZPtDTgr7ZvTgKnAr4F/G+w7Bg4CVgMzavrwgU3qvRt4bc30d4DT0+e/Af4yfT4ROLLJOo4G1gzy3n5G8+34j4CnSLbfccBH0j7Vk/aZW9JtYgLJdvuKmhi0DfjbtNz7gAeb9fHR+Oh6A+o63ybgiZrH39Z80A/Ulb8Q+HrN9CtJglqpZt7FwJmNyg/SST5RMz0f2Jp+uR8FvlFX/irg5JrXnlWzbDHwuyb1rASOqZneL+1IFXYFplk1y38LLEqfn0ldsG+w/u8Dp6XPL6AmeAMvSNf/ApIN9anaDRM4Cri3yXqPBe6umf4V8Ffp87OAK2gSxOvWE8CGuu/69TXv77qasiWSf5qvHOw7Tss9TTI8VF/foJ/pWHzQPNjfDRxfM/164L7BvuO0L61P+8e4Ier9v8AF6fNJaf/bP53+OfCvwJQh1nE0sKOu/zwBTEiX/4zm2/H/AS6t619r03UeBfQDlQZ1ngKsqpken/apfbv9XQ73MdqGcd4YEXvVPL5cs2x1g/K182YAq+PZ4+T3k2R5g61jsHXeT/LffwqwP/DW9KffE5KeAF5BEqgbvXY2yYbTyP7A5TXrWQlsB6bXlHmo5vlmkkynIUnHSbou/Wn9BMmvjynp4hl17ap9PpWk095Y05YfpfMbuRYYn/4cnwscClyeLvsUSYb043Qo6fRm7U0dXvddX9Wojen3uSZ9H4N9x1NIMrFmnznsxmc6hs0g+Uyr7k/nQZPvOJJ9Lx8k+ae7XtIlkmbQ2EXAm9KhoTcBN0VEtb53kWTed0i6QdIJg7Tzwbr+s1dEPFWzvNl2/Kz3l/al1SR9aDZwf9TtD6zxUM3rNqdPc9OHRluwH0yjy3PWznsQmF03TjyH5L/2YOuoN7vu9duAR0g6xDfqOteEiPhEk/WvBp7fpI7VwHF16+qLiLVNytd61ntIN5rvAp8GpkfEXsAykqwdkqy4dkdW7ft7hCQbfnFNO/aMiIYdOJId2peS/GpZDFwZERvTZRsj4sMR8XzgROBDjcZth2lnG9PvcxbJ9zvYd/wIsIVkOMtG7kGSZKRqTjpv0O84Ii6KiFekrw3gk41WHhG3kwTb44C/IAn+1WV3RcRikiGkTwKXSZowwvfRbDt+1vuTpLTsWpLtco4KeohunoL9UK4nydY+Immckp2nbwAu2c31vFPSfEnjSX62XpYGuW8Cb5D0ekllSX1KDgFrdkTAlcB+kj6Y7vSaJOml6bLzgH+XtD/s3NF50jDb9zAwtybg9ZCMS/cDA+lOo9fVlL8U+Ot0x+Z4kp+xwM6s5svAZyVNS9syU9LrB6n/IuDtwDuo2VAlnSDpBenG8yTJL5WRHo30EklvSje6D5KMr1/HIN9x+l4uAM6WNCP9jo6q7ly0hsal/bj6qJAMi/1z2ienAGeQ9P2m37GkgyS9Jv2st5AkEIN99xeR7Ld6FcmYPen63ylpavpdPpHOHmkfarYdXwr8maRjJI0j2af2DMm+id+SJEefkDQh/UxePsL6R53RFux/oGcfN3v50C9JRMRWkg3/OJL/4P9FMp58x2624Rsk4/sPkQwLfCBd/2rgJJKdg/0kWcA/0uQzTDPe16Ztegi4i11HAH2eZKfojyVtJAlkL220ngaqG8ejkm5K6/kASSd+nCRbWlrTjh8CXyAZglmV1gVJB4dkX8Qq4DpJG4CfkOxwaygiricZZ50B/LBm0bz0tZtIdrT9V0RcO8j7uKXuu/5czbIrSP6hPA78JfCmiNg2jO/4H4DfAzcAj5Fkh6Otj48my0gCc/VxJsmY+nLgVpLP8qZ0HjT/jnuBT5B8Jw+RZOYfG6Tei0kOJPhpRDxSM38hsELSJpJtZFFEPN1kHTP03OPs31yzvNl2fCfwTuCLaXvfQHLI99b0n8EbSPZBPEAyfPj2Qd5Hrijd2WAkh2yR7Pz8SrfbkpX0MLPbgN5Bxia7RtKZJDsA39nttlg+jYXteCSc9YwBSo5l7pW0N0m2+4PRGOjNLDsO9mPDe0gOjbubZJz1fd1tjpl1modxzMzGAGf2ZmZjwKg8nnTKlCkxd+7cbjfDCurGG298JCKanTiWGfdry9JQ/XpUBvu5c+eyfPnybjfDCkrS/UOXaj/3a8vSUP3awzhmZmOAg72Z2RjgYG9mNgY42JuZjQEO9mZmY4CDvZnZGOBgb2Y2BuQm2N//6FOc/eM7WfP45qELm+XIst+v46u/vLfbzbCCy02wX/v403zhp6tY+3izy1ub5dPVtz/M1359X7ebYQWXm2BfKiV32du+wxdus2IpSe7XlrncBPtKNdj7Kp3WAZIWSrpT0qpGN0+X9FlJN6ePP6Q3ax+RSsnB3rI3Kq+N00g1sx/wRmEZk1QGziW5reQa4AZJS9ObZQMQEX9fU/7vgMNGWl+pJPdry1zuMvsd3igse0cAqyLinvS+t5eQ3H+4mcUk91UdkUpJ7PAvVsvYkJm9pAuAE4D1EXFwOu/b7Lop9V7AExFxaIPX3gdsJLk70kBELBhpQ0tyZm8dM5PkhvJVa2hyQ3hJ+wMHAD9tsnwJsARgzpw5DSsrl8TA9h0tNNdsaMMZxrkQOAf4enVGROy847qkzwBPDvL6V9fdQX5EKmVn9jYqLQIui4jtjRZGxPnA+QALFixo2HnLJeFubVkbchgnIn4OPNZomSQBb6OFn7DDVXZmb52zFphdMz0rndfIIlrs/+WSGNjhzN6y1eqY/SuBhyPiribLA/ixpBvTn7NNSVoiabmk5f39/c9ZXq6O2Xts07J3AzBP0gGSekgC+tL6QpJeCOwN/KaVysol4VhvWWs12A+1Y+oVEXE4cBzwfkmvalYwIs6PiAURsWDq1OfeWasa7Ae2O9hbtiJiADgVuApYCVwaESsknSXpxJqii4BLIlrLQMpyZm/ZG/Ghl5IqwJuAlzQrExFr07/rJV1OcpTDz0dSX9nH2VsHRcQyYFndvDPqps9sR13VMfuIQOlwpVm7tZLZHwvcERFrGi2UNEHSpOpz4HXAbSOtrOwzaK2g3LetE4YM9pIuJhmTPEjSGknvShc9Z8eUpBmSqtnQdOCXkm4Bfgv8T0T8aKQNLfukKiso/2q1ThhyGCciFjeZf0qDeQ8Cx6fP7wEOabF9O1WPxvGhl1Y0zuytE3J0Bm3SVGf2VjQVB3vrgNwE+7JPqrKCqp4d7mBvWcpPsPdJVVZQ1bPD3bctS/kJ9j6pygqq5P1R1gG5C/Y+qcqKpuIjzawDchPs0+3Bh6dZ4fgubNYJuQn2kiiXxHafVm4F46NxrBNyE+yBNNh3uxVm7eWTqqwT8hXs5czeiscnVVkn5CrYV5zZWwF5GMc6IVfBvuQxeysgn1RlnZCrYF8pyeOaVjjVk6oc7C1LuQr2SWbvDcKKpeSzw60DchXsKw72VkDVi/z57HDLUq6CfUly9mOFk8Z6nx1umcpVsK+U5euHWOE4s7dOyFWwLzuztw6RtFDSnZJWSTq9SZm3Sbpd0gpJF420rnI1s3fftgyN+Ibj3ZDcmNkbhGVLUhk4F3gtsAa4QdLSiLi9psw84GPAyyPicUnTRlpfuZrZO9hbhoZzD9oLJK2XdFvNvDMlrZV0c/o4vslrh8yOdke5JI9rWiccAayKiHsiYitwCXBSXZm/Bc6NiMcBImL9SCvzvRqsE4YzjHMhsLDB/M9GxKHpY1n9wprs6DhgPrBY0vxWGuvM3jpkJrC6ZnpNOq/WHwF/JOlXkq6T1GgbQdISScslLe/v729YmS+XYJ0wZLCPiJ8Dj41g3cPJjnZLueQxexs1KsA84GhgMfBlSXvVF4qI8yNiQUQsmDp1asMVOdhbJ7Syg/ZUSbemwzx7N1g+nOxop+FmQN4grAPWArNrpmel82qtAZZGxLaIuBf4A0nw322+6qV1wkiD/ZeAA4FDgXXAZ1ptyLAyIDnYW0fcAMyTdICkHmARsLSuzPdJsnokTSEZ1rlnJJXtyux93SfLzoiCfUQ8HBHbI2IH8GWSIZt6w8mOdosze+uEiBgATgWuAlYCl0bECklnSToxLXYV8Kik24FrgX+MiEdHUt+uq1623HSzpkZ06KWk/SJiXTr558BtDYrtzI5Igvwi4C9G1MpUuSS2DniLsOylBx0sq5t3Rs3zAD6UPlpScmZvHTBksJd0McnP1SmS1gD/Ahwt6VAggPuA96RlZwBfiYjjI2JAUjU7KgMXRMSKVhrrHbRWRM7srROGDPYRsbjB7K82KfsgcHzN9HOyo1b40Esrol3Xs3e0t+zk6nIJFZ9UZQXkO1VZJ+Qq2DuztyKqjtl7iNKylLtg7w3Ciqaa2TuRsSzlLNiXfLEoK5yyM3vrgHwFe3mDsOKpBnsnMpalfAX7Usk7saxwfNVL64ScBXsfsWDFUyoJyZm9ZStnwb7ki0VZIfkubJa1nAV7Z/ZWTOWSnMhYpnIV7Cses7eCKpfEdp8waBnKVbAv+RLHVlDO7C1ruQr2lbKDvRWTL99tWctVsHdmb0VVcbC3jOUq2Ff8U9cKyomMZS1Xwb6UZj/hgG8F48zesparYL/rglFdbohZm5Uc7C1juQr2uy4Y5Zs8WLYkLZR0p6RVkk5vsPwUSf2Sbk4f726lPg9RWtZGdA/abtl1waguN8QKTVIZOBd4LbAGuEHS0oi4va7otyPi1HbUWfLluy1jQ2b2ki6QtF7SbTXzPiXpDkm3Srpc0l5NXnufpN+nmc/yVhu764JRjvaWqSOAVRFxT0RsBS4BTsqywopPqrKMDWcY50JgYd28q4GDI+KPgT8AHxvk9a+OiEMjYsHImriLM3vrkJnA6prpNem8em9OE57LJM1utCJJSyQtl7S8v7+/aYUleRjHsjVksI+InwOP1c37cUQMpJPXAbMyaNtzeMzeRpEfAHPThOdq4GuNCkXE+RGxICIWTJ06tenKfMKgZa0dO2j/Bvhhk2UB/FjSjZKWDLaS4WRA1WDvDMgythaozdRnpfN2iohHI+KZdPIrwEtaqbDs4+wtYy0Fe0kfBwaAbzUp8oqIOBw4Dni/pFc1W9dwMqCdwd4bhWXrBmCepAMk9QCLgKW1BSTtVzN5IrCylQp9uQTL2oiPxpF0CnACcEw0OcspItamf9dLupxkx9fPR1qng711QkQMSDoVuAooAxdExApJZwHLI2Ip8AFJJ5IkO48Bp7RSp4O9ZW1EwV7SQuAjwJ9GxOYmZSYApYjYmD5/HXDWiFvKrqNxvFFY1iJiGbCsbt4ZNc8/xuAHJuwWB3vL2nAOvbwY+A1wkKQ1kt4FnANMAq5OD6s8Ly07Q1J1A5kO/FLSLcBvgf+JiB+10thK2ffqtGLyJY4ta0Nm9hGxuMHsrzYp+yBwfPr8HuCQllpXp6TqoZfeKKxYyqUSAzu2d7sZVmC5ulxCpeTM3oqpUpKTGMtUroJ9yTtoraBKvuG4ZSxXwb7iYG8F5czesparYO+TqqyoyiX5zHDLVD6DvTMgK5hySb5Pg2XKwd5sFHBmb1nLV7D3SVVWUOWSfDVXy1Sugn31pCoHeyuaspzZW7ZyFexLzuytoMplsd2x3jKUq2BfKSXNdbC3okkucexob9nJVbBPY71PPrHC8YXQLGu5CvbVzH6Hj7O3gnGwt6zlKtiXndlbQVV81UvLWM6CfZrZO9hbwZSc2VvG8hXs5ateWjFVHOwtY/kK9mVfz96KqaTkcglN7vBp1rJ8BXtn9lZQvqKrZS1fwd5XvbQOkbRQ0p2SVkk6fZByb5YUkha0Ul/JN+axjA0r2Eu6QNJ6SbfVzNtH0tWS7kr/7t3ktSenZe6SdHIrjd0Z7H2qoWVIUhk4FzgOmA8sljS/QblJwGnA9a3WWc3sfVixZWW4mf2FwMK6eacD10TEPOCadPpZJO0D/AvwUuAI4F+a/VMYjl2Z/UjXYDYsRwCrIuKeiNgKXAKc1KDcvwGfBLa0WmHZmb1lbFjBPiJ+DjxWN/sk4Gvp868Bb2zw0tcDV0fEYxHxOHA1z/2nMWy7LnHszN4yNRNYXTO9Jp23k6TDgdkR8T+DrUjSEknLJS3v7+9vWq7at33wgWWllTH76RGxLn3+EDC9QZkhN5qq4WwUu3ZijbTJZq2TVALOBj48VNmIOD8iFkTEgqlTpzYt58zestaWHbSRHC/WUi8dzkax66qXjvaWqbXA7JrpWem8qknAwcDPJN0HHAksbWUnrTN7y1orwf5hSfsBpH/XNygz1EazW5zZW4fcAMyTdICkHmARsLS6MCKejIgpETE3IuYC1wEnRsTykVbow4ota60E+6VA9eiak4ErGpS5CnidpL3THbOvS+eNSMlj9tYBETEAnErSV1cCl0bECklnSToxizp9y03LWmU4hSRdDBwNTJG0huQIm08Al0p6F3A/8La07ALgvRHx7oh4TNK/kWRKAGdFRP2O3t1rcEnOfixzEbEMWFY374wmZY9utT4He8vasIJ9RCxusuiYBmWXA++umb4AuGBErWug5KsDWgH5hEHLWq7OoIX0glE+0N4Kxpm9ZS13wb4sZ/ZWPL42jmUtf8G+7EvBWvHsOqzYfduykbtg7+t+WxFVyg72lq3cBfuSHOyteEo+zt4ylrtg78zeiqiS3nLTfduykrtg73t1WhGlsd592zKTu2Bf8XH2VkDO7C1ruQv2JZ9BawVUrmb2TmQsI7kL9pWSfGVAK5zyzsze132ybOQu2JfkzN6Kpyxf0dWylbtgXyk7s7fi8V3YLGu5C/ZlZ/ZWQGXfq8Eylr9gXxI7vBPLCmbXbQkd7S0buQz2A77qpRXMztsSOpGxjOQy2PvwNCua6lUvnchYVvIZ7D1mbwVTcmZvGcthsC852FvmJC2UdKekVZJOb7D8vZJ+L+lmSb+UNL+V+nZm9u7blpERB3tJB6UdvfrYIOmDdWWOlvRkTZmG9/DcHWX5lHLLlqQycC5wHDAfWNwgmF8UEf8rIg4F/hM4u5U6d47Zu29bRoZ1D9pGIuJO4FDYuXGsBS5vUPQXEXHCSOup58zeOuAIYFVE3AMg6RLgJOD2aoGI2FBTfgLQUqcs+xLHlrERB/s6xwB3R8T9bVpfU+WSM3vL3Exgdc30GuCl9YUkvR/4ENADvKbRiiQtAZYAzJkzp2mFZd+8xDLWrjH7RcDFTZYdJekWST+U9OJWK6qUSj4ax0aFiDg3Ig4EPgr8c5My50fEgohYMHXq1KbrKvu2hJaxloO9pB7gROA7DRbfBOwfEYcAXwS+P8h6lkhaLml5f39/0/p8PXvrgLXA7JrpWem8Zi4B3thKhTvPoHUiYxlpR2Z/HHBTRDxcvyAiNkTEpvT5MmCcpCmNVjLcDMh3qrIOuAGYJ+mANJlZBCytLSBpXs3knwF3tVLhzmDv4+wtI+0Ys19MkyEcSfsCD0dESDqC5J/Lo61U5nvQWtYiYkDSqcBVQBm4ICJWSDoLWB4RS4FTJR0LbAMeB05upc6dwzjO7C0jLQV7SROA1wLvqZn3XoCIOA94C/A+SQPA08CiiNZ6c6UkXz/EMpf+El1WN++MmuentbO+UknIhxVbhloK9hHxFPC8unnn1Tw/BzinlTrqJWP27Vyj2ejgIUrLUu7OoE02CEd7Kx4PUVqWchfsfW0cKypn9pYlB3uzUaJU8o15LDv5DPY+YsEKqOIb81iG8hnsnf1YAZWd2VuGchfsK+kG0eIRnGajTrkkn1RlmcldsO8bVyYCnhnwETlWLH3jymwZ2N7tZlhB5S7YT+xNTg146pmBLrfErL0m9VXYuMX92rKRu2A/YWewdwZkxTKxt8ImB3vLSO6C/cTeMgCbnNlbwUzqG8eGLdu63QwrqNwF+52Z/VYHeyuWSb0VJzGWmdwF+/E9SbD3RmFF4zF7y1Lugr130FpRTexLMnsfVmxZyF2wn5CO2W/2DlormEl949i+I3h6m/u2tV/ugn01s/cwjhVNtW97KMeykLtgXx2z9zCOFc2kPgd7y07ugn1PpURPucQmH41jBbMr2PvwS2u/3AV7SMbtndlb0UzqGwd4iNKykdNgX/EOWsuUpIWS7pS0StLpDZZ/SNLtkm6VdI2k/Vut02P2lqWWg72k+yT9XtLNkpY3WC5JX0g3mlslHd5qnRN98ollSFIZOBc4DpgPLJY0v67Y74AFEfHHwGXAf7Zab3UYx5dMsCy0K7N/dUQcGhELGiw7DpiXPpYAX2q1sgm9FZ9Ba1k6AlgVEfdExFbgEuCk2gIRcW1EbE4nrwNmtVrppN5kGMeXTLAsdGIY5yTg65G4DthL0n6trHBCb4VNHsax7MwEVtdMr0nnNfMu4IeNFkhaImm5pOX9/f2DVjqxz4cVW3baEewD+LGkGyUtabB8WBvO7mwUE3q8g9ZGB0nvBBYAn2q0PCLOj4gFEbFg6tSpg66rXBLje8oes7dMtCPYvyIiDicZrnm/pFeNZCW7s1FM6K042FuW1gKza6ZnpfOeRdKxwMeBEyPimXZUPKnPlzm2bLQc7CNibfp3PXA5yXhnrWFtOLtjooO9ZesGYJ6kAyT1AIuApbUFJB0G/DdJoF/froon9lbY+IzH7K39Wgr2kiZImlR9DrwOuK2u2FLgr9Kjco4EnoyIda3UO6G3zFNbt/uCUZaJiBgATgWuAlYCl0bECklnSToxLfYpYCLwnfRItKVNVrdbJvWN8zCOZaLS4uunA5dLqq7rooj4kaT3AkTEecAy4HhgFbAZ+OsW62RCb4XtO4JnBnbQN67c6urMniMilpH03dp5Z9Q8PzaLen2ZY8tKS8E+Iu4BDmkw/7ya5wG8v5V66tVeDM3B3opkUl+FdU9u6XYzrIByeQatL4ZmRTWpd5yvjWOZyGWwr96H1jcdt6KZ6KNxLCO5DPa+D60V1aS+Ck9t3c72HT74wNor18HeZxpa0fjmPJaVXAZ734fWimpyepljj9tbu+Uy2I/vqY7ZO9hbsfj6OJaVXAb7XT91vYPWisW3JrSs5DLYV8fsNzv7sYLZmcg42Fub5TLYjyuX6Kn4PrRWPNVbE/qa9tZuuQz24IuhWTFNmdgDQP/GtlxE02yn3Ab75KbjHrO3Ytlzj3H0VEqsd7C3NstvsO/xfWiteCSx7+Q+Ht7g6+NYe+U22E/s9WnlVkzTJ/c62Fvb5TbYT5nYyyOb/FPXimfa5D7Wb3DftvbKbbB39mNFNX2Sh3Gs/XIb7KdN7mPDlgGe3uqdtFYs0yf38tTW7d4nZW2V22C/7+Q+AGdAVjjT3bctA7kN9t4gLEuSFkq6U9IqSac3WP4qSTdJGpD0lnbWPW1yL+C+be014mAvabakayXdLmmFpNMalDla0pPpDZlvlnRGo3WNxPTqBuHjka3NJJWBc4HjgPnAYknz64o9AJwCXNTu+quJjHfSWju1cg/aAeDDEXGTpEnAjZKujojb68r9IiJOaKGehqZVM3vfr9Pa7whgVXqPZSRdApwE7OzbEXFfumxHuyuvBvuHnNlbG404s4+IdRFxU/p8I7ASmNmuhg1lcl+FPcaV/VPXsjATWF0zvYYR9m1JSyQtl7S8v79/WK+Z2FthQo/7trVXW8bsJc0FDgOub7D4KEm3SPqhpBcPso7d2igkJYdfehjHRrGIOD8iFkTEgqlTpw77ddP39LH21l4tB3tJE4HvAh+MiA11i28C9o+IQ4AvAt9vtp6RbBTTfFq5ZWMtMLtmelY6r2N8rL21W0vBXtI4kkD/rYj4Xv3yiNgQEZvS58uAcZKmtFJnrekO9paNG4B5kg6Q1AMsApZ2sgHJr1b3bWufVo7GEfBVYGVEnItXgSAAAAl2SURBVN2kzL5pOSQdkdb36EjrrLdvehZtRLRrlWZExABwKnAVyb6oSyNihaSzJJ0IIOlPJK0B3gr8t6QV7WxDksg8475tbdPK0TgvB/4S+L2km9N5/wTMAYiI84C3AO+TNAA8DSyKNvbe6ZP72LJtBxu2DLDnHuPatVqz6i/RZXXzzqh5fgPJ8E4mpk3uY+vADp7YvI29J/RkVY2NISMO9hHxS0BDlDkHOGekdQxl2s7jkbc42FuhHDh1AgArH9rAyw5s28injWG5PYMWYPqk5MQqH49sRXPY7L0B+N0DT3S5JVYUuQ72++5ZvWSCD1GzYtlz/DgOnDqBm+5/vNtNsYLIdbCfNik90/DJp7vcErP2O3zO3vxu9RPeSWttketgv0dPmVl778EdD23sdlPM2u6wOXvz2FNbuf/Rzd1uihVAroM9wMEz9mTFg/Xncpnl3+H77wXATQ94KMdal/9gP3My9z7yFBu3bOt2U8zaat60SUzsrTjYW1vkPti/eOaeANzu7N4KplwSh8zekxvv9xE51rrcB/uDZyTB/jYHeyuglx04hZXrNviyINay3Af7qZN6mT65l9vWPtntppi13TEvmgbAtXes73JLLO9yH+whye4d7K2IDpo+iZl77cFPVjrYW2sKEexfPHNP7u7fxOatA91uillbSeKYF03jV6seYcu27d1ujuVYIYL9wTMmsyPgZp9abgX0mhdO4+lt2/nN3W27YKyNQYUI9q+YN4VJfRW+9dsHut0Us7Y78vnPY3xPmStvXdftpliOFSLYj++psOhPZvOj2x5inS+dYAXTN67MW18yiytuXssDPpvWRqgQwR7gr46aS0Twzevu73ZTzNruf7/6BZRL4gs/vavbTbGcKkywn73PeI590XQuuv4B78iywpk+uY93Hrk/37tpDSvX+ZwS232FCfYAp7x8Lo9v3sbSWx7sdlPM2u69f3oge4/v4R1fuZ7fr/GhxrZ7ChXsj3r+8zho+iQu/NV9viysFc7USb18571Hsce4Mm8//zd8/Tf3sWOH+7kNT0vBXtJCSXdKWiXp9AbLeyV9O11+vaS5rdQ3jPZwysvncvu6Ddxwny8eZSM32vp21fOnTuS773sZL9l/b864YgVvP/83rH7MO21taCMO9pLKwLnAccB8YLGk+XXF3gU8HhEvAD4LfHKk9Q3XGw+dyV7jx/EP37mFz//kLr6zfDVX3LzWR+nYsI3Wvl217559fP1vjuAzbz2EO9Zt5Pgv/ILP/eQP/HrVI9zTv4lHNz3Dlm3b/evWnmXENxwHjgBWRcQ9AJIuAU4Cbq8pcxJwZvr8MuAcSYoMe+EePWXOftshnHvt3Xzumj9QW9Pc541nXLlQI1fWxAl/PIPTjp030pePyr5dSxJvfsks/mTuPnz0u7fy+Wvuor7mkqCnUmJcqUSpJEqCkoQEkExLIKrzQOm6kzrq66yW0XOX1bWtYZsHfUODLdztYs1f36Rto81grfziXxzGC/edvNvrbCXYzwRW10yvAV7arExEDEh6Enge8Ej9yiQtAZYAzJkzp4VmwWteOJ3XvHA6T2zeysYtA2zcMsDP/rCeFWs3EDjbGQumTe5t5eVt69vt7NeNzHneeC5eciRPPr2NW9c8wSObnuHJzdt4aut2Nm8dYOvADgZ2BDt2JD1/e/o3AiIi+ZtuE8lznjVvp9j1p/7/We1Us391g211w/3/2PKWm5NNf6gY1Vcpj2i9rQT7toqI84HzARYsWNCWr2Wv8T3sNb4HgPkzdv8/oVmrsujXjey5xzheOW9qVqu3AmhlTGMtMLtmelY6r2EZSRVgT8AX+LDRzn3bCqeVYH8DME/SAZJ6gEXA0royS4GT0+dvAX7aqTFNsxa4b1vhjHgYJx2nPBW4CigDF0TECklnAcsjYinwVeAbklYBj5FsNGajmvu2FVFLY/YRsQxYVjfvjJrnW4C3tlKHWTe4b1vR+DhEM7MxwMHezGwMcLA3MxsDHOzNzMYAjcajxST1A43uQjKFBmffdtFoao/b0lijtuwfER0/A2mQfg2j/zPrFrelsd3u16My2DcjaXlELOh2O6pGU3vclsZGU1sGM5ra6bY0lve2eBjHzGwMcLA3MxsD8hbsz+92A+qMpva4LY2NprYMZjS1021pLNdtydWYvZmZjUzeMnszMxsBB3szszEgN8F+qBtAZ1z3bEnXSrpd0gpJp6Xz95F0taS70r97d7BNZUm/k3RlOn1AeuPrVemNsHs61I69JF0m6Q5JKyUd1a3PRdLfp9/PbZIultTXrc9ld7hvP6s9o6Jfp3UXqm/nIthreDeAztIA8OGImA8cCbw/rf904JqImAdck053ymnAyprpTwKfTW+A/TjJDbE74fPAjyLihcAhaZs6/rlImgl8AFgQEQeTXJp4Ed37XIbFffs5Rku/hqL17eQ+lKP7ARwFXFUz/THgY11szxXAa4E7gf3SefsBd3ao/lkkHe01wJUk9yd+BKg0+rwybMeewL2kO/pr5nf8c2HXPWH3Ibl095XA67vxuexmu923d9U9Kvp1Wlfh+nYuMnsa3wB6ZjcaImkucBhwPTA9Italix4CpneoGZ8DPgLsSKefBzwREQPpdKc+nwOAfuD/pT+9vyJpAl34XCJiLfBp4AFgHfAkcCPd+Vx2h/v2LqOlX0MB+3Zegv2oIGki8F3ggxGxoXZZJP9eMz+OVdIJwPqIuDHruoahAhwOfCkiDgOeou5nbQc/l72Bk0g20hnABGBh1vUWRbf79ijr11DAvp2XYD+cG0BnStI4ko3hWxHxvXT2w5L2S5fvB6zvQFNeDpwo6T7gEpKfvJ8H9lJy42vo3OezBlgTEden05eRbCDd+FyOBe6NiP6I2AZ8j+Sz6sbnsjvctxOjqV9DAft2XoL9cG4AnRlJIrnn6MqIOLtmUe1Np08mGe/MVER8LCJmRcRcks/hpxHxDuBakhtfd7ItDwGrJR2UzjoGuJ0ufC4kP3GPlDQ+/b6qben457Kb3LcZXf06bU/x+nYndna0aSfF8cAfgLuBj3e47leQ/Fy7Fbg5fRxPMqZ4DXAX8BNgnw6362jgyvT584HfAquA7wC9HWrDocDy9LP5PrB3tz4X4F+BO4DbgG8Avd36XHaz3e7bz25T1/t1Wneh+rYvl2BmNgbkZRjHzMxa4GBvZjYGONibmY0BDvZmZmOAg72Z2RjgYG9mNgY42JuZjQH/H3ksECgQ1yl0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.subplot(1,2,1)\n",
    "plt.plot(eperr)\n",
    "plt.title('Error percentage vs Epoch')\n",
    "plt.subplot(1,2,2)\n",
    "plt.plot(eploss)\n",
    "plt.title('Loss vs Epoch')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "id": "VLvllSg8vFDp"
   },
   "outputs": [],
   "source": [
    "def validate_2(testloader):\n",
    "  # this code calculates the error rate on the validation set\n",
    "  incorrect = 0 \n",
    "  total = 0 \n",
    "  with torch.no_grad():  # we do not neet to compute the gradients when making predictions on the validation set\n",
    "      for data in testloader: \n",
    "          images, labels = data\n",
    "          outputs = nn2(images.cuda())\n",
    "          _, predicted = torch.max( outputs, dim=1)\n",
    "          total += labels.size(0) # the number of labels, which is just the size of the batch \n",
    "          incorrect += (predicted != labels.cuda()).sum().item()\n",
    "  print( \"Error Rate on test set: %f %%\" %(100 * incorrect/total))\n",
    "  return (100 * incorrect/total)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "twOjP0GH347x"
   },
   "source": [
    "**Doing Task 2 of checking the effect of regularisation**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "rNAImnT6vdmH",
    "outputId": "35e88b95-6ff2-40c6-c531-d35f4326ac4c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda\n",
      "Error rate on train set: 19.125000 %\n",
      "Epoch 1 loss: 0.722370 \n",
      "Error rate on train set: 5.925000 %\n",
      "Epoch 2 loss: 0.243439 \n",
      "Error rate on train set: 3.350000 %\n",
      "Epoch 3 loss: 0.140753 \n",
      "Error rate on train set: 2.075000 %\n",
      "Epoch 4 loss: 0.090757 \n",
      "Error rate on train set: 0.950000 %\n",
      "Epoch 5 loss: 0.050788 \n",
      "Error rate on train set: 0.350000 %\n",
      "Epoch 6 loss: 0.029260 \n",
      "Error rate on train set: 0.075000 %\n",
      "Epoch 7 loss: 0.013070 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 8 loss: 0.006489 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 9 loss: 0.006970 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 10 loss: 0.004020 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 11 loss: 0.003136 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 12 loss: 0.001379 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 13 loss: 0.001076 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 14 loss: 0.000710 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 15 loss: 0.000349 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 16 loss: 0.000289 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 17 loss: 0.000247 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 18 loss: 0.000205 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 19 loss: 0.000182 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 20 loss: 0.000164 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 21 loss: 0.000148 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 22 loss: 0.000135 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 23 loss: 0.000124 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 24 loss: 0.000111 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 25 loss: 0.000103 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 26 loss: 0.000097 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 27 loss: 0.000088 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 28 loss: 0.000082 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 29 loss: 0.000076 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 30 loss: 0.000070 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 31 loss: 0.000066 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 32 loss: 0.000062 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 33 loss: 0.000057 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 34 loss: 0.000055 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 35 loss: 0.000051 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 36 loss: 0.000048 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 37 loss: 0.000045 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 38 loss: 0.000042 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 39 loss: 0.000040 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 40 loss: 0.000038 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 41 loss: 0.000036 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 42 loss: 0.000034 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 43 loss: 0.000032 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 44 loss: 0.000030 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 45 loss: 0.000029 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 46 loss: 0.000027 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 47 loss: 0.000026 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 48 loss: 0.000025 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 49 loss: 0.000023 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 50 loss: 0.000022 \n",
      "Error Rate on test set: 3.290000 %\n",
      "cuda\n",
      "Error rate on train set: 24.400000 %\n",
      "Epoch 1 loss: 0.877719 \n",
      "Error rate on train set: 6.925000 %\n",
      "Epoch 2 loss: 0.265947 \n",
      "Error rate on train set: 4.000000 %\n",
      "Epoch 3 loss: 0.148325 \n",
      "Error rate on train set: 2.150000 %\n",
      "Epoch 4 loss: 0.093956 \n",
      "Error rate on train set: 1.325000 %\n",
      "Epoch 5 loss: 0.057904 \n",
      "Error rate on train set: 0.800000 %\n",
      "Epoch 6 loss: 0.038100 \n",
      "Error rate on train set: 0.625000 %\n",
      "Epoch 7 loss: 0.032215 \n",
      "Error rate on train set: 0.525000 %\n",
      "Epoch 8 loss: 0.028707 \n",
      "Error rate on train set: 0.650000 %\n",
      "Epoch 9 loss: 0.028632 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 10 loss: 0.010122 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 11 loss: 0.008229 \n",
      "Error rate on train set: 0.075000 %\n",
      "Epoch 12 loss: 0.005688 \n",
      "Error rate on train set: 0.125000 %\n",
      "Epoch 13 loss: 0.006263 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 14 loss: 0.010413 \n",
      "Error rate on train set: 0.225000 %\n",
      "Epoch 15 loss: 0.016328 \n",
      "Error rate on train set: 0.425000 %\n",
      "Epoch 16 loss: 0.013539 \n",
      "Error rate on train set: 0.125000 %\n",
      "Epoch 17 loss: 0.008111 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 18 loss: 0.008027 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 19 loss: 0.003507 \n",
      "Error rate on train set: 0.125000 %\n",
      "Epoch 20 loss: 0.010375 \n",
      "Error rate on train set: 0.100000 %\n",
      "Epoch 21 loss: 0.004815 \n",
      "Error rate on train set: 0.125000 %\n",
      "Epoch 22 loss: 0.006611 \n",
      "Error rate on train set: 0.125000 %\n",
      "Epoch 23 loss: 0.006407 \n",
      "Error rate on train set: 0.075000 %\n",
      "Epoch 24 loss: 0.004556 \n",
      "Error rate on train set: 0.175000 %\n",
      "Epoch 25 loss: 0.007766 \n",
      "Error rate on train set: 0.075000 %\n",
      "Epoch 26 loss: 0.004378 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 27 loss: 0.002106 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 28 loss: 0.000901 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 29 loss: 0.001375 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 30 loss: 0.004370 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 31 loss: 0.002039 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 32 loss: 0.000731 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 33 loss: 0.001395 \n",
      "Error rate on train set: 0.100000 %\n",
      "Epoch 34 loss: 0.011265 \n",
      "Error rate on train set: 0.275000 %\n",
      "Epoch 35 loss: 0.011650 \n",
      "Error rate on train set: 0.125000 %\n",
      "Epoch 36 loss: 0.006933 \n",
      "Error rate on train set: 0.100000 %\n",
      "Epoch 37 loss: 0.008773 \n",
      "Error rate on train set: 0.125000 %\n",
      "Epoch 38 loss: 0.006714 \n",
      "Error rate on train set: 0.100000 %\n",
      "Epoch 39 loss: 0.002489 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 40 loss: 0.001958 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 41 loss: 0.000652 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 42 loss: 0.000704 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 43 loss: 0.004154 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 44 loss: 0.000886 \n",
      "Error rate on train set: 0.050000 %\n",
      "Epoch 45 loss: 0.000773 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 46 loss: 0.000341 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 47 loss: 0.000289 \n",
      "Error rate on train set: 0.050000 %\n",
      "Epoch 48 loss: 0.005245 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 49 loss: 0.009371 \n",
      "Error rate on train set: 0.100000 %\n",
      "Epoch 50 loss: 0.002772 \n",
      "Error Rate on test set: 3.180000 %\n",
      "cuda\n",
      "Error rate on train set: 22.775000 %\n",
      "Epoch 1 loss: 0.844095 \n",
      "Error rate on train set: 6.700000 %\n",
      "Epoch 2 loss: 0.262986 \n",
      "Error rate on train set: 3.825000 %\n",
      "Epoch 3 loss: 0.145361 \n",
      "Error rate on train set: 2.450000 %\n",
      "Epoch 4 loss: 0.096589 \n",
      "Error rate on train set: 1.925000 %\n",
      "Epoch 5 loss: 0.078335 \n",
      "Error rate on train set: 1.300000 %\n",
      "Epoch 6 loss: 0.050600 \n",
      "Error rate on train set: 0.925000 %\n",
      "Epoch 7 loss: 0.037298 \n",
      "Error rate on train set: 0.575000 %\n",
      "Epoch 8 loss: 0.024800 \n",
      "Error rate on train set: 0.750000 %\n",
      "Epoch 9 loss: 0.029251 \n",
      "Error rate on train set: 0.500000 %\n",
      "Epoch 10 loss: 0.029179 \n",
      "Error rate on train set: 0.450000 %\n",
      "Epoch 11 loss: 0.016588 \n",
      "Error rate on train set: 0.225000 %\n",
      "Epoch 12 loss: 0.011026 \n",
      "Error rate on train set: 0.300000 %\n",
      "Epoch 13 loss: 0.015022 \n",
      "Error rate on train set: 0.350000 %\n",
      "Epoch 14 loss: 0.011093 \n",
      "Error rate on train set: 0.450000 %\n",
      "Epoch 15 loss: 0.013280 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 16 loss: 0.012622 \n",
      "Error rate on train set: 0.325000 %\n",
      "Epoch 17 loss: 0.010597 \n",
      "Error rate on train set: 0.175000 %\n",
      "Epoch 18 loss: 0.010296 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 19 loss: 0.008397 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 20 loss: 0.006315 \n",
      "Error rate on train set: 0.175000 %\n",
      "Epoch 21 loss: 0.005355 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 22 loss: 0.007983 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 23 loss: 0.009444 \n",
      "Error rate on train set: 0.225000 %\n",
      "Epoch 24 loss: 0.008886 \n",
      "Error rate on train set: 0.225000 %\n",
      "Epoch 25 loss: 0.011104 \n",
      "Error rate on train set: 0.125000 %\n",
      "Epoch 26 loss: 0.005653 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 27 loss: 0.004169 \n",
      "Error rate on train set: 0.050000 %\n",
      "Epoch 28 loss: 0.004467 \n",
      "Error rate on train set: 0.100000 %\n",
      "Epoch 29 loss: 0.003445 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 30 loss: 0.003493 \n",
      "Error rate on train set: 0.075000 %\n",
      "Epoch 31 loss: 0.005040 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 32 loss: 0.010720 \n",
      "Error rate on train set: 0.300000 %\n",
      "Epoch 33 loss: 0.010877 \n",
      "Error rate on train set: 0.325000 %\n",
      "Epoch 34 loss: 0.009459 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 35 loss: 0.007532 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 36 loss: 0.006239 \n",
      "Error rate on train set: 0.050000 %\n",
      "Epoch 37 loss: 0.004311 \n",
      "Error rate on train set: 0.225000 %\n",
      "Epoch 38 loss: 0.008522 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 39 loss: 0.010167 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 40 loss: 0.011144 \n",
      "Error rate on train set: 0.175000 %\n",
      "Epoch 41 loss: 0.003814 \n",
      "Error rate on train set: 0.075000 %\n",
      "Epoch 42 loss: 0.003279 \n",
      "Error rate on train set: 0.100000 %\n",
      "Epoch 43 loss: 0.002042 \n",
      "Error rate on train set: 0.050000 %\n",
      "Epoch 44 loss: 0.002150 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 45 loss: 0.003314 \n",
      "Error rate on train set: 0.075000 %\n",
      "Epoch 46 loss: 0.001860 \n",
      "Error rate on train set: 0.125000 %\n",
      "Epoch 47 loss: 0.008174 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 48 loss: 0.011135 \n",
      "Error rate on train set: 0.125000 %\n",
      "Epoch 49 loss: 0.005266 \n",
      "Error rate on train set: 0.175000 %\n",
      "Epoch 50 loss: 0.005301 \n",
      "Error Rate on test set: 3.210000 %\n",
      "cuda\n",
      "Error rate on train set: 22.625000 %\n",
      "Epoch 1 loss: 0.836477 \n",
      "Error rate on train set: 7.600000 %\n",
      "Epoch 2 loss: 0.287069 \n",
      "Error rate on train set: 4.375000 %\n",
      "Epoch 3 loss: 0.177189 \n",
      "Error rate on train set: 3.075000 %\n",
      "Epoch 4 loss: 0.116879 \n",
      "Error rate on train set: 2.625000 %\n",
      "Epoch 5 loss: 0.098201 \n",
      "Error rate on train set: 1.775000 %\n",
      "Epoch 6 loss: 0.066721 \n",
      "Error rate on train set: 1.575000 %\n",
      "Epoch 7 loss: 0.053888 \n",
      "Error rate on train set: 1.150000 %\n",
      "Epoch 8 loss: 0.043564 \n",
      "Error rate on train set: 0.975000 %\n",
      "Epoch 9 loss: 0.033659 \n",
      "Error rate on train set: 1.050000 %\n",
      "Epoch 10 loss: 0.033769 \n",
      "Error rate on train set: 0.825000 %\n",
      "Epoch 11 loss: 0.024057 \n",
      "Error rate on train set: 0.750000 %\n",
      "Epoch 12 loss: 0.024699 \n",
      "Error rate on train set: 0.600000 %\n",
      "Epoch 13 loss: 0.019684 \n",
      "Error rate on train set: 0.600000 %\n",
      "Epoch 14 loss: 0.023901 \n",
      "Error rate on train set: 0.875000 %\n",
      "Epoch 15 loss: 0.022364 \n",
      "Error rate on train set: 0.550000 %\n",
      "Epoch 16 loss: 0.022873 \n",
      "Error rate on train set: 0.550000 %\n",
      "Epoch 17 loss: 0.015130 \n",
      "Error rate on train set: 0.375000 %\n",
      "Epoch 18 loss: 0.013263 \n",
      "Error rate on train set: 0.225000 %\n",
      "Epoch 19 loss: 0.010951 \n",
      "Error rate on train set: 0.550000 %\n",
      "Epoch 20 loss: 0.016568 \n",
      "Error rate on train set: 0.500000 %\n",
      "Epoch 21 loss: 0.018068 \n",
      "Error rate on train set: 0.525000 %\n",
      "Epoch 22 loss: 0.013315 \n",
      "Error rate on train set: 0.450000 %\n",
      "Epoch 23 loss: 0.011827 \n",
      "Error rate on train set: 0.525000 %\n",
      "Epoch 24 loss: 0.013607 \n",
      "Error rate on train set: 0.325000 %\n",
      "Epoch 25 loss: 0.008184 \n",
      "Error rate on train set: 0.425000 %\n",
      "Epoch 26 loss: 0.017422 \n",
      "Error rate on train set: 0.375000 %\n",
      "Epoch 27 loss: 0.014728 \n",
      "Error rate on train set: 0.400000 %\n",
      "Epoch 28 loss: 0.011943 \n",
      "Error rate on train set: 0.375000 %\n",
      "Epoch 29 loss: 0.012206 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 30 loss: 0.010827 \n",
      "Error rate on train set: 0.300000 %\n",
      "Epoch 31 loss: 0.009552 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 32 loss: 0.009819 \n",
      "Error rate on train set: 0.225000 %\n",
      "Epoch 33 loss: 0.004480 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 34 loss: 0.005352 \n",
      "Error rate on train set: 0.225000 %\n",
      "Epoch 35 loss: 0.013313 \n",
      "Error rate on train set: 0.425000 %\n",
      "Epoch 36 loss: 0.013138 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 37 loss: 0.009799 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 38 loss: 0.006430 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 39 loss: 0.004907 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 40 loss: 0.005608 \n",
      "Error rate on train set: 0.275000 %\n",
      "Epoch 41 loss: 0.009977 \n",
      "Error rate on train set: 0.275000 %\n",
      "Epoch 42 loss: 0.012219 \n",
      "Error rate on train set: 0.300000 %\n",
      "Epoch 43 loss: 0.008219 \n",
      "Error rate on train set: 0.475000 %\n",
      "Epoch 44 loss: 0.013792 \n",
      "Error rate on train set: 0.375000 %\n",
      "Epoch 45 loss: 0.008550 \n",
      "Error rate on train set: 0.225000 %\n",
      "Epoch 46 loss: 0.007167 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 47 loss: 0.009211 \n",
      "Error rate on train set: 0.225000 %\n",
      "Epoch 48 loss: 0.004721 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 49 loss: 0.005031 \n",
      "Error rate on train set: 0.350000 %\n",
      "Epoch 50 loss: 0.012292 \n",
      "Error Rate on test set: 3.800000 %\n",
      "cuda\n",
      "Error rate on train set: 29.825000 %\n",
      "Epoch 1 loss: 0.985923 \n",
      "Error rate on train set: 10.750000 %\n",
      "Epoch 2 loss: 0.384140 \n",
      "Error rate on train set: 6.700000 %\n",
      "Epoch 3 loss: 0.235066 \n",
      "Error rate on train set: 5.475000 %\n",
      "Epoch 4 loss: 0.181845 \n",
      "Error rate on train set: 4.350000 %\n",
      "Epoch 5 loss: 0.136353 \n",
      "Error rate on train set: 3.075000 %\n",
      "Epoch 6 loss: 0.110039 \n",
      "Error rate on train set: 2.825000 %\n",
      "Epoch 7 loss: 0.100514 \n",
      "Error rate on train set: 2.100000 %\n",
      "Epoch 8 loss: 0.076896 \n",
      "Error rate on train set: 1.975000 %\n",
      "Epoch 9 loss: 0.063852 \n",
      "Error rate on train set: 1.925000 %\n",
      "Epoch 10 loss: 0.054162 \n",
      "Error rate on train set: 1.375000 %\n",
      "Epoch 11 loss: 0.051008 \n",
      "Error rate on train set: 1.575000 %\n",
      "Epoch 12 loss: 0.044080 \n",
      "Error rate on train set: 1.075000 %\n",
      "Epoch 13 loss: 0.046322 \n",
      "Error rate on train set: 1.125000 %\n",
      "Epoch 14 loss: 0.043430 \n",
      "Error rate on train set: 1.125000 %\n",
      "Epoch 15 loss: 0.032570 \n",
      "Error rate on train set: 0.950000 %\n",
      "Epoch 16 loss: 0.033548 \n",
      "Error rate on train set: 0.900000 %\n",
      "Epoch 17 loss: 0.030509 \n",
      "Error rate on train set: 1.175000 %\n",
      "Epoch 18 loss: 0.028323 \n",
      "Error rate on train set: 0.700000 %\n",
      "Epoch 19 loss: 0.026787 \n",
      "Error rate on train set: 1.050000 %\n",
      "Epoch 20 loss: 0.028150 \n",
      "Error rate on train set: 1.050000 %\n",
      "Epoch 21 loss: 0.026733 \n",
      "Error rate on train set: 0.600000 %\n",
      "Epoch 22 loss: 0.027126 \n",
      "Error rate on train set: 0.625000 %\n",
      "Epoch 23 loss: 0.022226 \n",
      "Error rate on train set: 0.650000 %\n",
      "Epoch 24 loss: 0.016610 \n",
      "Error rate on train set: 0.675000 %\n",
      "Epoch 25 loss: 0.016935 \n",
      "Error rate on train set: 0.650000 %\n",
      "Epoch 26 loss: 0.022692 \n",
      "Error rate on train set: 0.650000 %\n",
      "Epoch 27 loss: 0.015233 \n",
      "Error rate on train set: 0.650000 %\n",
      "Epoch 28 loss: 0.019469 \n",
      "Error rate on train set: 0.525000 %\n",
      "Epoch 29 loss: 0.020160 \n",
      "Error rate on train set: 0.700000 %\n",
      "Epoch 30 loss: 0.017894 \n",
      "Error rate on train set: 0.325000 %\n",
      "Epoch 31 loss: 0.018107 \n",
      "Error rate on train set: 0.925000 %\n",
      "Epoch 32 loss: 0.021148 \n",
      "Error rate on train set: 0.625000 %\n",
      "Epoch 33 loss: 0.021814 \n",
      "Error rate on train set: 0.700000 %\n",
      "Epoch 34 loss: 0.018696 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 35 loss: 0.011441 \n",
      "Error rate on train set: 0.700000 %\n",
      "Epoch 36 loss: 0.012760 \n",
      "Error rate on train set: 0.500000 %\n",
      "Epoch 37 loss: 0.019483 \n",
      "Error rate on train set: 0.600000 %\n",
      "Epoch 38 loss: 0.022008 \n",
      "Error rate on train set: 0.525000 %\n",
      "Epoch 39 loss: 0.019318 \n",
      "Error rate on train set: 0.625000 %\n",
      "Epoch 40 loss: 0.019530 \n",
      "Error rate on train set: 0.525000 %\n",
      "Epoch 41 loss: 0.018103 \n",
      "Error rate on train set: 0.300000 %\n",
      "Epoch 42 loss: 0.015982 \n",
      "Error rate on train set: 0.275000 %\n",
      "Epoch 43 loss: 0.011384 \n",
      "Error rate on train set: 0.450000 %\n",
      "Epoch 44 loss: 0.010543 \n",
      "Error rate on train set: 0.475000 %\n",
      "Epoch 45 loss: 0.015375 \n",
      "Error rate on train set: 0.575000 %\n",
      "Epoch 46 loss: 0.014497 \n",
      "Error rate on train set: 0.600000 %\n",
      "Epoch 47 loss: 0.016773 \n",
      "Error rate on train set: 0.325000 %\n",
      "Epoch 48 loss: 0.016641 \n",
      "Error rate on train set: 0.325000 %\n",
      "Epoch 49 loss: 0.015587 \n",
      "Error rate on train set: 0.350000 %\n",
      "Epoch 50 loss: 0.007499 \n",
      "Error Rate on test set: 3.710000 %\n",
      "cuda\n",
      "Error rate on train set: 32.000000 %\n",
      "Epoch 1 loss: 1.059268 \n",
      "Error rate on train set: 12.150000 %\n",
      "Epoch 2 loss: 0.430903 \n",
      "Error rate on train set: 7.700000 %\n",
      "Epoch 3 loss: 0.297977 \n",
      "Error rate on train set: 6.225000 %\n",
      "Epoch 4 loss: 0.220377 \n",
      "Error rate on train set: 5.000000 %\n",
      "Epoch 5 loss: 0.181932 \n",
      "Error rate on train set: 4.550000 %\n",
      "Epoch 6 loss: 0.153481 \n",
      "Error rate on train set: 3.725000 %\n",
      "Epoch 7 loss: 0.128409 \n",
      "Error rate on train set: 3.600000 %\n",
      "Epoch 8 loss: 0.110984 \n",
      "Error rate on train set: 3.050000 %\n",
      "Epoch 9 loss: 0.111082 \n",
      "Error rate on train set: 3.500000 %\n",
      "Epoch 10 loss: 0.091774 \n",
      "Error rate on train set: 3.075000 %\n",
      "Epoch 11 loss: 0.085069 \n",
      "Error rate on train set: 2.225000 %\n",
      "Epoch 12 loss: 0.085052 \n",
      "Error rate on train set: 2.325000 %\n",
      "Epoch 13 loss: 0.064971 \n",
      "Error rate on train set: 2.225000 %\n",
      "Epoch 14 loss: 0.071682 \n",
      "Error rate on train set: 1.550000 %\n",
      "Epoch 15 loss: 0.057773 \n",
      "Error rate on train set: 1.575000 %\n",
      "Epoch 16 loss: 0.060339 \n",
      "Error rate on train set: 1.950000 %\n",
      "Epoch 17 loss: 0.056122 \n",
      "Error rate on train set: 1.925000 %\n",
      "Epoch 18 loss: 0.055390 \n",
      "Error rate on train set: 1.950000 %\n",
      "Epoch 19 loss: 0.056862 \n",
      "Error rate on train set: 1.975000 %\n",
      "Epoch 20 loss: 0.048999 \n",
      "Error rate on train set: 1.925000 %\n",
      "Epoch 21 loss: 0.044739 \n",
      "Error rate on train set: 1.850000 %\n",
      "Epoch 22 loss: 0.047288 \n",
      "Error rate on train set: 1.250000 %\n",
      "Epoch 23 loss: 0.047428 \n",
      "Error rate on train set: 1.250000 %\n",
      "Epoch 24 loss: 0.031420 \n",
      "Error rate on train set: 1.350000 %\n",
      "Epoch 25 loss: 0.036897 \n",
      "Error rate on train set: 1.550000 %\n",
      "Epoch 26 loss: 0.040111 \n",
      "Error rate on train set: 1.225000 %\n",
      "Epoch 27 loss: 0.044340 \n",
      "Error rate on train set: 1.450000 %\n",
      "Epoch 28 loss: 0.034404 \n",
      "Error rate on train set: 1.575000 %\n",
      "Epoch 29 loss: 0.039994 \n",
      "Error rate on train set: 1.300000 %\n",
      "Epoch 30 loss: 0.030240 \n",
      "Error rate on train set: 0.900000 %\n",
      "Epoch 31 loss: 0.026445 \n",
      "Error rate on train set: 0.975000 %\n",
      "Epoch 32 loss: 0.035598 \n",
      "Error rate on train set: 1.275000 %\n",
      "Epoch 33 loss: 0.035326 \n",
      "Error rate on train set: 1.125000 %\n",
      "Epoch 34 loss: 0.033435 \n",
      "Error rate on train set: 0.925000 %\n",
      "Epoch 35 loss: 0.033657 \n",
      "Error rate on train set: 0.825000 %\n",
      "Epoch 36 loss: 0.027465 \n",
      "Error rate on train set: 1.150000 %\n",
      "Epoch 37 loss: 0.025917 \n",
      "Error rate on train set: 1.125000 %\n",
      "Epoch 38 loss: 0.037863 \n",
      "Error rate on train set: 1.300000 %\n",
      "Epoch 39 loss: 0.027817 \n",
      "Error rate on train set: 1.200000 %\n",
      "Epoch 40 loss: 0.040085 \n",
      "Error rate on train set: 0.875000 %\n",
      "Epoch 41 loss: 0.030293 \n",
      "Error rate on train set: 1.025000 %\n",
      "Epoch 42 loss: 0.031716 \n",
      "Error rate on train set: 0.925000 %\n",
      "Epoch 43 loss: 0.025319 \n",
      "Error rate on train set: 0.725000 %\n",
      "Epoch 44 loss: 0.025462 \n",
      "Error rate on train set: 0.900000 %\n",
      "Epoch 45 loss: 0.031279 \n",
      "Error rate on train set: 0.800000 %\n",
      "Epoch 46 loss: 0.027969 \n",
      "Error rate on train set: 0.600000 %\n",
      "Epoch 47 loss: 0.020151 \n",
      "Error rate on train set: 0.825000 %\n",
      "Epoch 48 loss: 0.028012 \n",
      "Error rate on train set: 0.900000 %\n",
      "Epoch 49 loss: 0.022467 \n",
      "Error rate on train set: 0.825000 %\n",
      "Epoch 50 loss: 0.020100 \n",
      "Error Rate on test set: 4.530000 %\n",
      "cuda\n",
      "Error rate on train set: 35.750000 %\n",
      "Epoch 1 loss: 1.147085 \n",
      "Error rate on train set: 14.400000 %\n",
      "Epoch 2 loss: 0.507101 \n",
      "Error rate on train set: 10.275000 %\n",
      "Epoch 3 loss: 0.361208 \n",
      "Error rate on train set: 7.525000 %\n",
      "Epoch 4 loss: 0.271229 \n",
      "Error rate on train set: 6.525000 %\n",
      "Epoch 5 loss: 0.247663 \n",
      "Error rate on train set: 5.850000 %\n",
      "Epoch 6 loss: 0.186024 \n",
      "Error rate on train set: 5.475000 %\n",
      "Epoch 7 loss: 0.174617 \n",
      "Error rate on train set: 5.125000 %\n",
      "Epoch 8 loss: 0.166270 \n",
      "Error rate on train set: 4.700000 %\n",
      "Epoch 9 loss: 0.150273 \n",
      "Error rate on train set: 4.650000 %\n",
      "Epoch 10 loss: 0.134702 \n",
      "Error rate on train set: 4.000000 %\n",
      "Epoch 11 loss: 0.128742 \n",
      "Error rate on train set: 3.375000 %\n",
      "Epoch 12 loss: 0.118825 \n",
      "Error rate on train set: 3.300000 %\n",
      "Epoch 13 loss: 0.095915 \n",
      "Error rate on train set: 3.225000 %\n",
      "Epoch 14 loss: 0.093376 \n",
      "Error rate on train set: 3.700000 %\n",
      "Epoch 15 loss: 0.097380 \n",
      "Error rate on train set: 3.150000 %\n",
      "Epoch 16 loss: 0.094736 \n",
      "Error rate on train set: 2.750000 %\n",
      "Epoch 17 loss: 0.097540 \n",
      "Error rate on train set: 2.900000 %\n",
      "Epoch 18 loss: 0.085560 \n",
      "Error rate on train set: 2.475000 %\n",
      "Epoch 19 loss: 0.071681 \n",
      "Error rate on train set: 2.400000 %\n",
      "Epoch 20 loss: 0.068446 \n",
      "Error rate on train set: 2.575000 %\n",
      "Epoch 21 loss: 0.063171 \n",
      "Error rate on train set: 2.400000 %\n",
      "Epoch 22 loss: 0.072385 \n",
      "Error rate on train set: 2.275000 %\n",
      "Epoch 23 loss: 0.069631 \n",
      "Error rate on train set: 2.175000 %\n",
      "Epoch 24 loss: 0.062814 \n",
      "Error rate on train set: 1.750000 %\n",
      "Epoch 25 loss: 0.067910 \n",
      "Error rate on train set: 1.925000 %\n",
      "Epoch 26 loss: 0.061586 \n",
      "Error rate on train set: 1.825000 %\n",
      "Epoch 27 loss: 0.056322 \n",
      "Error rate on train set: 1.875000 %\n",
      "Epoch 28 loss: 0.055174 \n",
      "Error rate on train set: 2.025000 %\n",
      "Epoch 29 loss: 0.062348 \n",
      "Error rate on train set: 1.700000 %\n",
      "Epoch 30 loss: 0.067983 \n",
      "Error rate on train set: 2.200000 %\n",
      "Epoch 31 loss: 0.068788 \n",
      "Error rate on train set: 1.500000 %\n",
      "Epoch 32 loss: 0.057450 \n",
      "Error rate on train set: 1.600000 %\n",
      "Epoch 33 loss: 0.055379 \n",
      "Error rate on train set: 1.775000 %\n",
      "Epoch 34 loss: 0.044989 \n",
      "Error rate on train set: 1.275000 %\n",
      "Epoch 35 loss: 0.051811 \n",
      "Error rate on train set: 2.025000 %\n",
      "Epoch 36 loss: 0.044259 \n",
      "Error rate on train set: 1.700000 %\n",
      "Epoch 37 loss: 0.051141 \n",
      "Error rate on train set: 1.375000 %\n",
      "Epoch 38 loss: 0.044270 \n",
      "Error rate on train set: 1.650000 %\n",
      "Epoch 39 loss: 0.043335 \n",
      "Error rate on train set: 1.125000 %\n",
      "Epoch 40 loss: 0.051074 \n",
      "Error rate on train set: 1.800000 %\n",
      "Epoch 41 loss: 0.042798 \n",
      "Error rate on train set: 1.775000 %\n",
      "Epoch 42 loss: 0.047442 \n",
      "Error rate on train set: 1.400000 %\n",
      "Epoch 43 loss: 0.042499 \n",
      "Error rate on train set: 1.425000 %\n",
      "Epoch 44 loss: 0.036477 \n",
      "Error rate on train set: 1.325000 %\n",
      "Epoch 45 loss: 0.032473 \n",
      "Error rate on train set: 1.600000 %\n",
      "Epoch 46 loss: 0.041432 \n",
      "Error rate on train set: 1.525000 %\n",
      "Epoch 47 loss: 0.044670 \n",
      "Error rate on train set: 1.200000 %\n",
      "Epoch 48 loss: 0.035737 \n",
      "Error rate on train set: 1.450000 %\n",
      "Epoch 49 loss: 0.040168 \n",
      "Error rate on train set: 1.600000 %\n",
      "Epoch 50 loss: 0.036647 \n",
      "Error Rate on test set: 4.970000 %\n",
      "cuda\n",
      "Error rate on train set: 40.925000 %\n",
      "Epoch 1 loss: 1.280231 \n",
      "Error rate on train set: 19.825000 %\n",
      "Epoch 2 loss: 0.655204 \n",
      "Error rate on train set: 15.200000 %\n",
      "Epoch 3 loss: 0.494058 \n",
      "Error rate on train set: 11.750000 %\n",
      "Epoch 4 loss: 0.371455 \n",
      "Error rate on train set: 9.900000 %\n",
      "Epoch 5 loss: 0.331818 \n",
      "Error rate on train set: 8.425000 %\n",
      "Epoch 6 loss: 0.306062 \n",
      "Error rate on train set: 7.825000 %\n",
      "Epoch 7 loss: 0.263553 \n",
      "Error rate on train set: 7.425000 %\n",
      "Epoch 8 loss: 0.239674 \n",
      "Error rate on train set: 7.100000 %\n",
      "Epoch 9 loss: 0.213959 \n",
      "Error rate on train set: 6.575000 %\n",
      "Epoch 10 loss: 0.229580 \n",
      "Error rate on train set: 6.300000 %\n",
      "Epoch 11 loss: 0.196487 \n",
      "Error rate on train set: 6.350000 %\n",
      "Epoch 12 loss: 0.202865 \n",
      "Error rate on train set: 5.125000 %\n",
      "Epoch 13 loss: 0.184630 \n",
      "Error rate on train set: 5.550000 %\n",
      "Epoch 14 loss: 0.164454 \n",
      "Error rate on train set: 5.375000 %\n",
      "Epoch 15 loss: 0.164429 \n",
      "Error rate on train set: 4.575000 %\n",
      "Epoch 16 loss: 0.149290 \n",
      "Error rate on train set: 4.425000 %\n",
      "Epoch 17 loss: 0.162130 \n",
      "Error rate on train set: 4.925000 %\n",
      "Epoch 18 loss: 0.136403 \n",
      "Error rate on train set: 4.025000 %\n",
      "Epoch 19 loss: 0.146821 \n",
      "Error rate on train set: 4.400000 %\n",
      "Epoch 20 loss: 0.126818 \n",
      "Error rate on train set: 4.825000 %\n",
      "Epoch 21 loss: 0.132443 \n",
      "Error rate on train set: 4.250000 %\n",
      "Epoch 22 loss: 0.124962 \n",
      "Error rate on train set: 3.850000 %\n",
      "Epoch 23 loss: 0.124264 \n",
      "Error rate on train set: 4.025000 %\n",
      "Epoch 24 loss: 0.130162 \n",
      "Error rate on train set: 3.850000 %\n",
      "Epoch 25 loss: 0.125456 \n",
      "Error rate on train set: 4.050000 %\n",
      "Epoch 26 loss: 0.102702 \n",
      "Error rate on train set: 3.675000 %\n",
      "Epoch 27 loss: 0.119080 \n",
      "Error rate on train set: 3.725000 %\n",
      "Epoch 28 loss: 0.116215 \n",
      "Error rate on train set: 3.725000 %\n",
      "Epoch 29 loss: 0.114070 \n",
      "Error rate on train set: 2.950000 %\n",
      "Epoch 30 loss: 0.102995 \n",
      "Error rate on train set: 3.225000 %\n",
      "Epoch 31 loss: 0.103780 \n",
      "Error rate on train set: 3.475000 %\n",
      "Epoch 32 loss: 0.103390 \n",
      "Error rate on train set: 3.075000 %\n",
      "Epoch 33 loss: 0.088425 \n",
      "Error rate on train set: 3.300000 %\n",
      "Epoch 34 loss: 0.097518 \n",
      "Error rate on train set: 2.750000 %\n",
      "Epoch 36 loss: 0.086079 \n",
      "Error rate on train set: 3.425000 %\n",
      "Epoch 37 loss: 0.094026 \n",
      "Error rate on train set: 3.425000 %\n",
      "Epoch 38 loss: 0.083835 \n",
      "Error rate on train set: 3.050000 %\n",
      "Epoch 39 loss: 0.093508 \n",
      "Error rate on train set: 2.825000 %\n",
      "Epoch 40 loss: 0.079335 \n",
      "Error rate on train set: 3.100000 %\n",
      "Epoch 41 loss: 0.081821 \n",
      "Error rate on train set: 3.000000 %\n",
      "Epoch 42 loss: 0.084104 \n",
      "Error rate on train set: 2.550000 %\n",
      "Epoch 43 loss: 0.094551 \n",
      "Error rate on train set: 3.000000 %\n",
      "Epoch 44 loss: 0.078611 \n",
      "Error rate on train set: 2.975000 %\n",
      "Epoch 45 loss: 0.088529 \n",
      "Error rate on train set: 2.750000 %\n",
      "Epoch 46 loss: 0.083830 \n",
      "Error rate on train set: 2.475000 %\n",
      "Epoch 47 loss: 0.081206 \n",
      "Error rate on train set: 2.500000 %\n",
      "Epoch 48 loss: 0.074526 \n",
      "Error rate on train set: 2.725000 %\n",
      "Epoch 49 loss: 0.076357 \n",
      "Error rate on train set: 2.425000 %\n",
      "Epoch 50 loss: 0.072590 \n",
      "Error Rate on test set: 5.380000 %\n",
      "cuda\n",
      "Error rate on train set: 58.025000 %\n",
      "Epoch 1 loss: 1.695042 \n",
      "Error rate on train set: 33.975000 %\n",
      "Epoch 2 loss: 1.010970 \n",
      "Error rate on train set: 26.025000 %\n",
      "Epoch 3 loss: 0.820050 \n",
      "Error rate on train set: 22.825000 %\n",
      "Epoch 4 loss: 0.704639 \n",
      "Error rate on train set: 21.025000 %\n",
      "Epoch 5 loss: 0.651426 \n",
      "Error rate on train set: 20.025000 %\n",
      "Epoch 6 loss: 0.597474 \n",
      "Error rate on train set: 16.400000 %\n",
      "Epoch 7 loss: 0.512565 \n",
      "Error rate on train set: 16.675000 %\n",
      "Epoch 8 loss: 0.537633 \n",
      "Error rate on train set: 16.375000 %\n",
      "Epoch 9 loss: 0.483084 \n",
      "Error rate on train set: 15.425000 %\n",
      "Epoch 10 loss: 0.459503 \n",
      "Error rate on train set: 13.750000 %\n",
      "Epoch 11 loss: 0.446867 \n",
      "Error rate on train set: 14.100000 %\n",
      "Epoch 12 loss: 0.420309 \n",
      "Error rate on train set: 13.775000 %\n",
      "Epoch 13 loss: 0.403298 \n",
      "Error rate on train set: 12.725000 %\n",
      "Epoch 14 loss: 0.393934 \n",
      "Error rate on train set: 12.150000 %\n",
      "Epoch 15 loss: 0.376187 \n",
      "Error rate on train set: 11.900000 %\n",
      "Epoch 16 loss: 0.390751 \n",
      "Error rate on train set: 11.525000 %\n",
      "Epoch 17 loss: 0.344911 \n",
      "Error rate on train set: 11.900000 %\n",
      "Epoch 18 loss: 0.347206 \n",
      "Error rate on train set: 11.300000 %\n",
      "Epoch 19 loss: 0.354536 \n",
      "Error rate on train set: 11.550000 %\n",
      "Epoch 20 loss: 0.355946 \n",
      "Error rate on train set: 10.550000 %\n",
      "Epoch 21 loss: 0.315210 \n",
      "Error rate on train set: 10.000000 %\n",
      "Epoch 22 loss: 0.327049 \n",
      "Error rate on train set: 9.875000 %\n",
      "Epoch 23 loss: 0.312777 \n",
      "Error rate on train set: 9.575000 %\n",
      "Epoch 24 loss: 0.305590 \n",
      "Error rate on train set: 9.250000 %\n",
      "Epoch 25 loss: 0.316092 \n",
      "Error rate on train set: 9.550000 %\n",
      "Epoch 26 loss: 0.296589 \n",
      "Error rate on train set: 10.300000 %\n",
      "Epoch 27 loss: 0.313203 \n",
      "Error rate on train set: 9.175000 %\n",
      "Epoch 28 loss: 0.283591 \n",
      "Error rate on train set: 9.225000 %\n",
      "Epoch 29 loss: 0.278773 \n",
      "Error rate on train set: 9.100000 %\n",
      "Epoch 30 loss: 0.279399 \n",
      "Error rate on train set: 9.400000 %\n",
      "Epoch 31 loss: 0.259633 \n",
      "Error rate on train set: 8.775000 %\n",
      "Epoch 32 loss: 0.283333 \n",
      "Error rate on train set: 8.500000 %\n",
      "Epoch 33 loss: 0.266013 \n",
      "Error rate on train set: 8.950000 %\n",
      "Epoch 34 loss: 0.268017 \n",
      "Error rate on train set: 8.250000 %\n",
      "Epoch 35 loss: 0.261782 \n",
      "Error rate on train set: 8.650000 %\n",
      "Epoch 36 loss: 0.246963 \n",
      "Error rate on train set: 8.150000 %\n",
      "Epoch 37 loss: 0.239968 \n",
      "Error rate on train set: 9.050000 %\n",
      "Epoch 38 loss: 0.252694 \n",
      "Error rate on train set: 8.575000 %\n",
      "Epoch 39 loss: 0.244113 \n",
      "Error rate on train set: 8.375000 %\n",
      "Epoch 40 loss: 0.252923 \n",
      "Error rate on train set: 8.700000 %\n",
      "Epoch 41 loss: 0.237068 \n",
      "Error rate on train set: 7.500000 %\n",
      "Epoch 42 loss: 0.220740 \n",
      "Error rate on train set: 7.675000 %\n",
      "Epoch 43 loss: 0.244054 \n",
      "Error rate on train set: 7.050000 %\n",
      "Epoch 44 loss: 0.228453 \n",
      "Error rate on train set: 7.925000 %\n",
      "Epoch 45 loss: 0.229536 \n",
      "Error rate on train set: 7.850000 %\n",
      "Epoch 46 loss: 0.243400 \n",
      "Error rate on train set: 7.450000 %\n",
      "Epoch 47 loss: 0.208821 \n",
      "Error rate on train set: 7.250000 %\n",
      "Epoch 48 loss: 0.222813 \n",
      "Error rate on train set: 6.950000 %\n",
      "Epoch 49 loss: 0.217220 \n",
      "Error rate on train set: 6.850000 %\n",
      "Epoch 50 loss: 0.221298 \n",
      "Error Rate on test set: 9.640000 %\n",
      "cuda\n",
      "Error rate on train set: 82.225000 %\n",
      "Epoch 1 loss: 2.220607 \n",
      "Error rate on train set: 64.700000 %\n",
      "Epoch 2 loss: 1.803664 \n",
      "Error rate on train set: 54.625000 %\n",
      "Epoch 3 loss: 1.505196 \n",
      "Error rate on train set: 51.100000 %\n",
      "Epoch 4 loss: 1.416931 \n",
      "Error rate on train set: 47.425000 %\n",
      "Epoch 5 loss: 1.328596 \n",
      "Error rate on train set: 46.700000 %\n",
      "Epoch 6 loss: 1.288913 \n",
      "Error rate on train set: 43.550000 %\n",
      "Epoch 7 loss: 1.183121 \n",
      "Error rate on train set: 42.200000 %\n",
      "Epoch 8 loss: 1.167213 \n",
      "Error rate on train set: 40.600000 %\n",
      "Epoch 9 loss: 1.131647 \n",
      "Error rate on train set: 39.300000 %\n",
      "Epoch 10 loss: 1.092644 \n",
      "Error rate on train set: 39.725000 %\n",
      "Epoch 11 loss: 1.061262 \n",
      "Error rate on train set: 38.125000 %\n",
      "Epoch 12 loss: 1.026318 \n",
      "Error rate on train set: 36.525000 %\n",
      "Epoch 13 loss: 1.046548 \n",
      "Error rate on train set: 35.775000 %\n",
      "Epoch 14 loss: 1.006150 \n",
      "Error rate on train set: 35.900000 %\n",
      "Epoch 15 loss: 0.999501 \n",
      "Error rate on train set: 34.550000 %\n",
      "Epoch 16 loss: 0.944587 \n",
      "Error rate on train set: 34.350000 %\n",
      "Epoch 17 loss: 0.942574 \n",
      "Error rate on train set: 34.675000 %\n",
      "Epoch 18 loss: 0.932058 \n",
      "Error rate on train set: 33.450000 %\n",
      "Epoch 19 loss: 0.910405 \n",
      "Error rate on train set: 33.475000 %\n",
      "Epoch 20 loss: 0.919199 \n",
      "Error rate on train set: 32.950000 %\n",
      "Epoch 21 loss: 0.918908 \n",
      "Error rate on train set: 30.425000 %\n",
      "Epoch 22 loss: 0.879209 \n",
      "Error rate on train set: 31.150000 %\n",
      "Epoch 23 loss: 0.866892 \n",
      "Error rate on train set: 31.275000 %\n",
      "Epoch 24 loss: 0.862350 \n",
      "Error rate on train set: 31.650000 %\n",
      "Epoch 25 loss: 0.844863 \n",
      "Error rate on train set: 31.575000 %\n",
      "Epoch 26 loss: 0.873610 \n",
      "Error rate on train set: 29.700000 %\n",
      "Epoch 27 loss: 0.850393 \n",
      "Error rate on train set: 29.550000 %\n",
      "Epoch 28 loss: 0.845937 \n",
      "Error rate on train set: 28.400000 %\n",
      "Epoch 29 loss: 0.822115 \n",
      "Error rate on train set: 29.725000 %\n",
      "Epoch 30 loss: 0.812706 \n",
      "Error rate on train set: 29.550000 %\n",
      "Epoch 31 loss: 0.799664 \n",
      "Error rate on train set: 29.600000 %\n",
      "Epoch 32 loss: 0.771688 \n",
      "Error rate on train set: 30.000000 %\n",
      "Epoch 33 loss: 0.809137 \n",
      "Error rate on train set: 28.850000 %\n",
      "Epoch 34 loss: 0.814092 \n",
      "Error rate on train set: 28.425000 %\n",
      "Epoch 35 loss: 0.808578 \n",
      "Error rate on train set: 27.675000 %\n",
      "Epoch 36 loss: 0.774673 \n",
      "Error rate on train set: 28.225000 %\n",
      "Epoch 37 loss: 0.765481 \n",
      "Error rate on train set: 27.000000 %\n",
      "Epoch 38 loss: 0.773838 \n",
      "Error rate on train set: 28.325000 %\n",
      "Epoch 39 loss: 0.769336 \n",
      "Error rate on train set: 27.800000 %\n",
      "Epoch 40 loss: 0.767500 \n",
      "Error rate on train set: 28.600000 %\n",
      "Epoch 41 loss: 0.767816 \n",
      "Error rate on train set: 27.175000 %\n",
      "Epoch 42 loss: 0.735629 \n",
      "Error rate on train set: 27.275000 %\n",
      "Epoch 43 loss: 0.756470 \n",
      "Error rate on train set: 26.850000 %\n",
      "Epoch 44 loss: 0.754677 \n",
      "Error rate on train set: 28.200000 %\n",
      "Epoch 45 loss: 0.749407 \n",
      "Error rate on train set: 26.300000 %\n",
      "Epoch 46 loss: 0.751892 \n",
      "Error rate on train set: 26.225000 %\n",
      "Epoch 47 loss: 0.771850 \n",
      "Error rate on train set: 26.325000 %\n",
      "Epoch 48 loss: 0.741661 \n",
      "Error rate on train set: 27.075000 %\n",
      "Epoch 49 loss: 0.721098 \n",
      "Error rate on train set: 26.950000 %\n",
      "Epoch 50 loss: 0.746146 \n",
      "Error Rate on test set: 28.070000 %\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Error rate in percantage')"
      ]
     },
     "execution_count": 86,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEWCAYAAABhffzLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxddZ3/8dc7adI1bWm6QEsXQFEBAaEiqD8FxV3AGR33ETdwHx0dt1keg8uov3H7jeKo+NOH6ICi44bbqD9FUAccbtgLKggJ0JY2bdIkXZI0yef3xzk3PQ1Nctrm7u/n43Ef95xzzz3nc79NP/d7v+ecz1FEYGZmjaOp0gGYmVl5OfGbmTUYJ34zswbjxG9m1mCc+M3MGowTv5lZg3HiNzNrME78DU5Sp6Q9knZmHpeWOYZfSxpM971N0nclHZXzvWdLerDUMeaIo9iOA5J2SPpvSW+UVHX/xyStkxSSZk2xziWS9qb/JsXPc9ZB7CMkPWJmIraZVnV/lFYR50XEgszjrQda6UCJQlLzwexoivXfGhELgEcAC4BPHMx2q8R5EdEGrAU+BrwX+PJkKx9s21XAVem/yVLgGuDbFY7HZogTv01K0qsl/U7SpyVtBy6R9FVJn5f0E0m7gHMkPSbtte+QtEHS+ZltPGz9qfYZETuA7wOnZrbxGkl3pb3peyW9IV0+H/gpsDLza2WlpCZJ75P0Z0nbJX1L0pJJPuNdkp6fmZ8lqVvSaZLmSPqPdBs7JN0oacV07RYRfRFxNfAS4EJJJ03WFjna7guSfpF+9mslrc28/sQ0pr70+YmZ1zolnZuZv0TSf6Sz16XPO9I2m7InHxEjwBXAKknL0u2dIen6NO7Nki6V1Jq+Vtz+ren2X5Iuf76kWzK/IE6eri2tNJz4bTpPAO4FVgD/ki57eTrdBvwe+CHwc2A58DbgCkmPymwju/5vp9qZpHbgL4F7Mou3As8HFgKvAT4t6bSI2AU8B9iU+bWyKY3hBcBTgZVAL/C5SXb5DeBlmflnAdsi4ibgQmARsBpoB94I7Jkq/qyI+B/gQeB/ZRYfbNu9AvgQSa/7FpIETPpF9mPgM2lsnwJ+nLbfdJ6SPi9O2+z6qVZOE/qrgO0kbQkwCvxtGtdZwNOBN6efu7j9U9LtXyXpccBXgDek8X4RuFrS7Bzx2gxz4jeA76e9sOLjosxrmyLisxExEhHFpPeDiPhdRIyR9MwXAB+LiOGI+BXwI/ZPpuPrR8TgJDF8RlIfsI0kmbyt+EJE/Dgi/hyJa0kS5f+aZDuQJOh/iIgHI2IIuAR40SRj2lcC50ual86/nOTLAGAvSZJ6RESMRkRHRPRPsd8D2QRkf20cbNv9OCKuSz/HPwBnSVoNPA+4OyK+nv7bfAP4A3DeQcY3lRdL2kHyZXcR8KK090/aFjek++4kSeRPnWJbFwNfjIjfp215OTAEnDmD8VpOTvwG8IKIWJx5fCnz2gMHWD+7bCXwQJrIirqAVdNsY6K/iYhFwMnAEcDRxRckPUfSDZJ60kT0XJIvh8msBb5X/CID7iLpoT5smCYi7klfPy9N/ueTfBkAfB34GfBNSZsk/auklhyfJWsV0JOZP+S2i4id6bZWpo+uCfua+N7D9a2IWEzSbncApxdfkHS8pB9JekhSP/ARpv83eVe2g0HyS2rlDMZrOTnx23QOVL41u2wTsFr7n72yBtg4zTYOvLOI24EPA59TYjbwHZKDvSvSRPQTQFNs+wHgORO+zOZExMYDrAv7hnsuAO5MvwyIiL0R8YGIOAF4Islw06vyfhZJjydJxNnhrYNtu9WZ7S0g+fWwKX2sZX/Z9+4C5mVeO3KSGKYVEdtIeuyXaN/ZVp8n+YXxyIhYCPw9+/5NDuQB4F8m/JvMS3+pWJk58dvh+j2wG3iPpBZJZ5MMN3zzMLZ5OUkv83ygFZgNdAMjkp4DPDOz7hagXdKizLIvAP9SPBAqaZmkC6bY3zfTbb6Jfb19JJ0j6bFKzr7pJxn6GTvwJvaRtDA9YPxN4D/SL7MDydN2z5X05HSc/UPADRHxAMmX3/GSXp4ekH4JcALJUBEkxwNemm53PfCizDa7089x7HSfpSgi/kjy6+c96aI2kjbZKenRJG2XtWXC9r8EvFHSE9Iv9PmSniepLW8MNnOc+A3gh9r/PP7v5X1jRAyTJKvnkIzP/zvwqoj4w6EGk27z34B/iogB4G+Ab5EcWHw5cHVm3T+Q9NjvTYcQVqbvvRr4uaQB4AaSg9ST7W8zcD1Jr/6qzEtHAv9JkuDuAq4lGf6ZzA/T/T1AMh7/KZKD0VN9zuna7krgn0mGeE4HXpm+dzvJL5B3kRx0fQ/w/LR3DvBPwHEkbfYBMl9oEbGb5ADz79I2yzvO/nHgYknLgb8j+bcYIEnqV01Y9xLg8nT7L46IAslxgkvTmO4BXp1zvzbD5BuxmFUnSV8FHoyIf6x0LFZf3OM3M2swTvxmZg3GQz1mZg3GPX4zswYzaXW+arJ06dJYt25dpcMwM6spHR0d2yJi2cTlNZH4161bR6FQqHQYZmY1RdLEq7sBD/WYmTUcJ34zswbjxG9m1mCc+M3MGowTv5lZg3HiNzNrME78ZmYNxonfzKwK9e4a5qM/vYt7u3fO+Lad+M3MqlBHVy9fvPZeugeGZnzbTvxmZlWo0NVLS7M4ZfXiGd+2E7+ZWRUqdPZw0qpFzGlpnvFtO/GbmVWZoZFRbtvYx/q1R5Rk+078ZmZV5o6NfQyPjHH62iUl2b4Tv5lZlSl09gJwunv8ZmaNodDVy7r2eSxrm12S7Tvxm5lVkYjgpq7ekg3zgBO/mVlVuW/bLrbvGubx60ozzANO/GZmVaU4vr/eid/MrDEUunpYPK+FY5cuKNk+nPjNzKpIoauX09ccQVOTSrYPJ34zsyrRs2uYe7t3cXoJh3nAid/MrGp0dKXj+yU8owec+M3Mqkahq4eWZnHy0YtKuh8nfjOzKlHo7OWxJSrMluXEb2ZWBQb3jnL7g32sX1faYR5w4jczqwp3bOxjeHSsZPV5spz4zcyqQKGrtIXZspz4zcyqQKGzl2OWzmfpgtIUZsty4jczq7CI4Kb7e8vS24cSJn5JqyVdI+lOSRskvT1dfomkjZJuSR/PLVUMZma14N5tu+jZNVyyO25NNKuE2x4B3hURN0lqAzok/SJ97dMR8YkS7tvMrGYUOnsAynJGD5Qw8UfEZmBzOj0g6S5gVan2Z2ZWqwqdvRwxr4Xjls0vy/7KMsYvaR3wOOD36aK3SrpN0lcklee3jZlZleroSsb3pdIVZssqeeKXtAD4DvCOiOgHPg8cB5xK8ovgk5O872JJBUmF7u7uUodpZlYR23cOce+2XSW949ZEJU38klpIkv4VEfFdgIjYEhGjETEGfAk440DvjYjLImJ9RKxftmxZKcM0M6uY8cJsJa7ImVXKs3oEfBm4KyI+lVl+VGa1vwDuKFUMZmbVrqOrl9bmJh67qrSF2bJKeVbPk4C/Bm6XdEu67O+Bl0k6FQigE3hDCWMwM6tqha5eTlq1sOSF2bJKeVbPb4EDHan4San2aWZWS4qF2V7zpHVl3a+v3DUzq5Dby1iYLcuJ38ysQgqd5SvMluXEb2ZWIR1dPRy7dD7tZSjMluXEb2ZWARExfuFWuTnxm5lVwJ+7d9G7e29Zz98vcuI3M6uAjq6kMFs5r9gtcuI3M6uAG8tcmC3Lid/MrAKS8f0lZSvMluXEb2ZWZtt2DnHftl0VGd8HJ34zs7IbL8xWgTN6wInfzKzsioXZTipjYbasXIlf0lpJ56bTc9NbKZqZ2SEodPbw2KMXlbUwW9a0iV/SRcB/Al9MFx0NfL+UQZmZ1avBvaPcvrGvYsM8kK/H/xaSEsv9ABFxN7C8lEGZmdWr2x7sY+9olO3G6geSJ/EPRcRwcUbSLJJa+mZmdpAK4xduVXeP/1pJfw/MlfQM4NvAD0sblplZfero7OXYZfNZMr+1YjHkSfzvA7qB20nulvUT4B9LGZSZWT0aGws67u+t6Pg+5LgDV+am6F8qfThmZvXr3m072bF7L+srUJ8na9rEL+l2Hj6m3wcUgA9HxPZSBGZmVm/Gb7xSoSt2i/Lcc/enwChwZTr/UmAe8BDwVeC8kkRmZlZnCl29LJnfyrFLy1+YLStP4j83Ik7LzN8u6aaIOE3SK0sVmJlZvSl09nD62iMqUpgtK8/B3WZJZxRnJD0eKF5uNlKSqMzM6kz3wBCd23dX/MAu5Ovxvx74iqQFgEgu5Hq9pPnAR0sZnJlZvRgvzFbh8X3Id1bPjcBjJS1K5/syL3+rVIGZmdWTjq4eWmdVrjBbVp4eP5KeB5wIzCmOTUXEB0sYl5lZXSl09XLyqkXMnlWZwmxZeYq0fQF4CfA2kqGevwLWljguM7O6Mbh3lDs29lX8NM6iPAd3nxgRrwJ6I+IDwFnA8aUNy8ysfowXZqvwhVtFeRL/nvR5t6SVwF7gqNKFZGZWX27srHxhtqw8Y/w/krQY+DhwE8lVvP+3pFGZmdWRjq5ejqtwYbasPIn/XyNiCPiOpB8Bc4DB0oZlZlYfxsaCjq5enn3ikZUOZVyeoZ7rixMRMZSeznn9FOubmVnqz9076duzt2oO7MIUPX5JRwKrSOrwP47kjB6AhSS1eqYkaTXwNWAFyfDQZRHxb5KWAFcB64BO4MUR0XsYn8HMrGoVihduVcn4Pkw91PMs4NUk99j9VGb5APD3ObY9ArwrIm5Kb87eIekX6TZ/GREfk/Q+knr/7z2E2M3Mql6hs5f2+a0cU+HCbFmTJv6IuBy4XNILI+I7B7vhiNgMbE6nByTdRfIL4gLg7HS1y4Ff48RvZnWq0NXDaVVQmC0r71k9LycZmhlf/2Cu3JW0Dngc8HtgRfqlAElp5xWTvOdi4GKANWvW5N2VmVnV6B4Yomv7bl7xhOrKYXkO7v6ApJc+AuzKPHJJi7t9B3hHRPRnX4uIYJIbt0fEZRGxPiLWL1u2LO/uzMyqRsf4jdWr48Ktojw9/qMj4tmHsnFJLSRJ/4qI+G66eIukoyJis6SjgK2Hsm0zs2pX6OxNC7MtrHQo+8nT4/9vSY892A0rGdD6MnBXRGQPDl8NXJhOX0jyi8LMrO4Uuno55ejqKMyWlSfxP5nkjJw/SrpN0u2SbsvxvicBfw08TdIt6eO5wMeAZ0i6Gzg3nTczqyuDe0fZsKmv6oZ5IN9Qz3MOZcMR8Vv2nfs/0dMPZZtmZrXi1gd2pIXZquf8/aI8N2LpApC0nKRcg5mZTaN44Va1FGbLylOP//x0WOY+4FqSq21/WuK4zMxqWqGzh0csX8ARVVKYLSvPGP+HgDOBP0XEMSTDNDeUNCozsxpWLMxWjcM8kC/x742I7UCTpKaIuAZYX+K4zMxq1j3dO+kfHKnKYR7Id3B3R3oR1nXAFZK2chAXcJmZNZpCZ1qYbV31ndED+Xr8FwC7gb8F/gv4M3BeKYMyM6tlha4e2ue3sq592kLGFZGnx78c2BwRgyRF2+aS1NfZXtLIzMxqVEdXL6dXWWG2rDw9/m8DY5n50XSZmZlNsHVgkK7tu1lfRTdemShP4p8VEcPFmXS6+s5PMjOrAh1VPr4P+RJ/t6TzizOSLgC2lS4kM7PaVejqZfasJk5auajSoUwqzxj/G0nO5rk0nX+QpAaPmZlNkBRmW0zrrDz96sqYMvFLagbeFBFnpqd0EhE7yxKZmVmN2TM8yoaNfVz0lGMrHcqUpkz8ETEq6cnptBO+mdkUbn1wByNj1VmYLSvPUM/Nkq4mOZNn/MKtzI1VzMyM5DROqM7CbFl5Ev8cknP2n5ZZFoATv5lZxo1pYbbF86r7xMc8ZZlfU45AzMxq2dhYcFNXL887+ahKhzKtPGWZj5f0S0l3pPMnS/rH0odmZlY77t5aLMxWvefvF+U53+hLwPuBvQARcRvw0lIGZWZWawpdPQBVf2AX8iX+eRHxPxOWjZQiGDOzWtXR2cvSBa2srdLCbFl5Ev82SceRHNBF0ouAzSWNysysxhSqvDBbVp6zet4CXAY8WtJGklswvqKkUZmZ1ZCt/YPc37Obvz5zbaVDySXPWT33AudKmg80RcRA6cMyM6sd4zdWr+KKnFl5zuppl/QZ4DfAryX9m6T20odmZlYbCp3VX5gtK88Y/zeBbuCFwIvS6atKGZSZWS3p6OrhlNXVXZgtK0+UR0XEhyLivvTxYZI7cJmZNbw9w6Ns2NRfE6dxFuVJ/D+X9FJJTenjxcDPSh2YmVktuOWBtDBbjYzvQ77EfxFwJTCUPr4JvEHSgKT+UgZnZlbtOtILt05bUzuJP89ZPW3lCMTMrBYVunp5ZA0UZsuqjSMRZmZVaGws6Ojqrer76x6IE7+Z2SH609YBBgZHaurALjjxm5kdskJncuFWLR3YhZyJX1KzpJWS1hQfOd7zFUlbi+Wc02WXSNoo6Zb08dzDCd7MrJI6unpZumA2a5ZUf2G2rGkP7kp6G/DPwBZgLF0cwMnTvPWrwKXA1yYs/3REfOLgwjQzqz6Frh7W10hhtqw8RdreDjwqIrYfzIYj4jpJ6w4lKDOzarelf5AHevZw4VnrKh3KQcsz1PMA0DeD+3yrpNvSoaBJB8YkXSypIKnQ3d09g7s3Mzt8xfH9ar+x+oHkSfz3khRne7+kdxYfh7i/zwPHAaeS1PT/5GQrRsRlEbE+ItYvW7bsEHdnZlYaha4e5rQ0cWKNFGbLyjPUc3/6aE0fhywithSnJX0J+NHhbM/MrFI6uno55ejaKcyWlefK3Q/M1M4kHRURxbt3/QVwx1Trm5lVo93DI2zY1M8bn3pspUM5JJMmfkn/JyLeIemHpLddzIqI86fasKRvAGcDSyU9SHJm0NmSTk231wm84dBDNzOrjFse2MHoWLB+bW1dsVs0VY//6+nzIZ16GREvO8DiLx/KtszMqklHemC3lgqzZU2a+COiI32+tnzhmJlVv0JXL8evWMCieS2VDuWQ1N5RCTOzChodC27q6uX0Gh3mASd+M7OD8qctAwwMjfD4GqvPk5U78UuqrWIUZmYlUOhKC7PVc49f0hMl3Qn8IZ0/RdK/lzwyM7Mq1NHZw7K22axeMrfSoRyyPD3+TwPPArYDRMStwFNKGZSZWbUqdPXWZGG2rFxDPRHxwIRFoyWIxcysqm3pH+TB3j01WZ8nK0/JhgckPREISS0k1TrvKm1YZmbVZ9+NV2p3fB/y9fjfCLwFWAVsJCmw9uZSBmVmVo1u7CwWZltY6VAOS54e/6Mi4hXZBZKeBPyuNCGZmVWnjq5eTl29mJbm2j4TPk/0n825zMysbu0aGuHOzf01fRpn0VRF2s4Cnggsm1B/fyHQXOrAzMyqya1pYbbTa/jCraKphnpagQXpOm2Z5f3Ai0oZlJlZtSl09SLVbmG2rKmKtF0LXCvpqxHRVcaYzMyqTqGrl+OXt7Fobm0WZsvKc3B3t6SPAycCc4oLI+JpJYvKzKyKjI4FN3f1ct6pKysdyozIc3D3CpJyDccAHyC5gcqNJYzJzKyq/PGhpDDb+hq/cKsoT+Jvj4gvA3sj4tqIeC3g3r6ZNYyOrh4AHl/jF24V5Rnq2Zs+b5b0PGATUB+f3swsh0JXL8vbZnP0EbVbmC0rT+L/sKRFwLtIzt9fCPxtSaMyM6sihc5e1q+r7cJsWVMmfknNwCMj4kdAH3BOWaIyM6sSD/UNsnHHHl775GMqHcqMmXKMPyJGgQPdNN3MrCEU0vH9ejmwC/mGen4n6VLgKmBXcWFE3FSyqMzMqkShs5e5Lc2cUOOF2bLyJP5T0+cPZpYFPrPHzBpAoauHU1YvqvnCbFnTJv6I8Li+mTWkXUMj3LV5gDeffVylQ5lR9fMVZmY2w24pFmaro/F9cOI3M5tUoTMtzNZIiV9SU3rbRTOzhlPo6uFRK9pYOKf2C7NlTXc65xjwuTLFYmZWNUbHgpvv31F3wzyQb6jnl5JeqHq5ZM3MLIc/PjTAzqER1tfBjVcmypP43wB8GxiW1C9pQFJ/ieMyM6uofRdu1V9psjync7ZNt46ZWb0pdPayYmH9FGbLynVWj6TzJX0ifTw/53u+ImmrpDsyy5ZI+oWku9Pn+vsNZWZ1oaOrl/Vrl9RNYbasaRO/pI8BbwfuTB9vl/TRHNv+KvDsCcveB/wyIh4J/DKdNzOrKpv79rBxx566PLAL+Uo2PBc4NT3DB0mXAzcD75/qTRFxnaR1ExZfAJydTl8O/Bp4b+5ozczKoNDZC1CXB3Yh/wVcizPTiw5jfysiYnM6/RCwYrIVJV0sqSCp0N3dfRi7NDM7OB1dSWG2xxxVP4XZsvL0+D8C3CzpGkDAU5iBIZqICEkxxeuXAZcBrF+/ftL1zMxmWqGrh1NXL66rwmxZ0165C4wBZwLfBb4DnBURVx3i/rZIOird9lHA1kPcjplZSVz3p27u3NTP44+pv9M4i/JcufueiNgcEVenj4cOY39XAxem0xcCPziMbZmZzajr/tTNRV8r8KgjF/LaJ62rdDglk+d3zP+T9HeSVqenYy6RNO1XoaRvANcDj5L0oKTXAR8DniHpbuDcdN7MrOKKSf/YZQu48vVPYPG81kqHVDJ5xvhfkj6/JbMsgGOnelNETHbLxqfn2KeZWdlMTPpHzK/fpA/T32y9CXjfYYzpm5lVtUZL+pBvjP/dZYrFzKysGjHpQwnH+M3MqlmjJn0o4Ri/mVm1auSkD/mqcx5TjkDMzMqh0ZM+TDHUI+k9mem/mvDaR0oZlJlZKTjpJ6Ya439pZnpiQbaJVTfNzKpaNulf0cBJH6ZO/Jpk+kDzZmZV6zd375/0lzRw0oepE39MMn2geTOzqvSbu7t5/eVO+llTHdw9Jb23roC5mfvsCphT8sjMzA6Tk/6BTZr4I6K5nIGYmc0kJ/3J1WexaTNraE76U3PiN7O64qQ/PSd+M6sbTvr5OPGbWV0oJv1jls530p+GE7+Z1bxs0r/yojOd9KfhxG9mNc1J/+A58ZtZzXLSPzRO/GZWk3579zYn/UPkxG9mNee3d2/jdZff6KR/iJz4zaymOOkfPid+M6sZTvozw4nfzGqCk/7MceI3s6rnpD+znPjNrKo56c88J34zq1pO+qXhxG9mVclJv3Sc+M2s6jjpl5YTv5lVFSf90pvqnrtmZiU3Nhb07h5m68AQd23u5/3fvd1Jv8QqkvgldQIDwCgwEhHrKxGHmZXO0Mgo3QNDdA8MsTV9JPODbO0fonvnEFv7h9i2c4iRsRh/36OPbHPSL7FK9vjPiYhtFdy/mR2kiKB/cCRN5oP7J/b+wfFk3r1ziB279z7s/RK0z29lWdsclrfN5vgVbSxvm82yttksb5vDsrbZnHz0Iua0NFfg0zUOD/WYGaNjwfad+3rlWyf0yrcO7EvqQyNjD3t/66wmlrfNZnnbbI5dNp8zj21Pk/lsli+czbIFc1i+cDbt81uZ1exDi5VWqcQfwM8lBfDFiLhs4gqSLgYuBlizZk2ZwzOrDxFB/54RHuofZEv/IA/1D7I1fd7SP8SWdHn3wBCZ0ZZxi+a2jPfIT19zxHjPPEnmaVJvm8PCObOQVP4PaIekUon/yRGxUdJy4BeS/hAR12VXSL8MLgNYv379Af4kzRrb4N7RJJn3DbIlHWopTm/pG2TLQJLUB/c+vIe+eF4LRy6cw/KFc3j0kW2sWDgnTfBJUl/eNpulC2Z7yKVOVSTxR8TG9HmrpO8BZwDXTf0us8YwMjrG9l3DSRLvLz6Gxnvtxfm+PQ8fQ5/T0sSRC+ewYuEcTjl6MSsWzmZFOn/kojmsSBO7E3pjK3vilzQfaIqIgXT6mcAHyx2HWSXtHBrhrs39bNjYxz3dO3moLxlHn2zYpblJLFswmxWL5rCuPRlDLyb0FQtnj/fePeRieVSix78C+F76xzkLuDIi/qsCcZiVRc+uYTZs6mPDpn7u2NjHnZv6uW/7LiJN7ovmtnDUoiSJP/rItvEkXuy5r1g0m/b5s2luckK3mVH2xB8R9wKnlHu/ZqUWEWzuG+SOjUmSTx59bO4bHF9n1eK5nLRqIS943CpOXLmQk1YtYnnbbPfSrax8OqfZIRgbC+7bvms8uW/YmDz3pueuS3DcsgWcccySJMGvXMQJKxeyeJ4vSrLKc+I3m8bwyBh3bx0YT+4bNvVz1+Z+dg2PAtDa3MTxRy7gWSceyYkrF3LCykU85qg25rX6v5dVJ/9lmmXsHk4Pum7qZ8PGfu7Y1MfdW3YyPJqcEjm/tZkTVi7kr9av5oS0J/+I5QtoneWLkqx2OPFbSfUP7uX+7bvp3L6Lru276dq+i4079jA2BrOaRWtzE7OaRUtzEy3NTcxqEi2zmmhpSpbNam6itVnMStdr3W+dJlpmiVlNTbQ071t/fLpp33ZbsvtoFi1NTewdG+OPDw2wYVMfd6S9+Xu37TvoumR+KyeuXMhrnryOk1Yu4sSVC1nXPp8mH2S1GufEb4clIti+a5iuNLF3bt/N/dt3Jc89u+nZNbzf+svaZrP6iLnMampiz95RRsbG2DsS7B0bY2Q02Ds6xt70eaQ4PTY2noxLZeWiOZy4ahHnnbKSE9Mkf9SiOT7oanXJid+mNTYWbO4fpGvbLrp6kt77/dt3j/fgi2PdAE2ClYvnsrZ9Hs8+6UjWLpnH2vZ5rG2fz5ol85g/+9D+5EbHil8KyZfByOgYe8eCvSNjjIyNMTwSyZfI+OsT1h8bY3hkjJGx5L3Do4GA41e0ccLKha4EaQ3Fid+A5ADmg7276erZPZ7gi4n9gZ4942PckBzMPHrJXNa1z+eMY5awLk3sa9vncfQR80oy3t3cJJqbmn3FqdkMqOvE//XrO7nmj920NjfRMquJ1uYmWmclY8ats5LplvFl+57Hl2WXF9dtbqJ1lmhtbk6Xab91KzE0EBFEwFgEYwHB/vNjxdfHgi0Dg3Ru2839PelwTDr+vmnHnv2uFp3X2sza9vk8cnkb556wgrVL5rOufR5r2udx1KK5vpjIrIbVdeLfOTTK1oFBhkeSn/vDI2MMjSQ//4fT55EDlSQ8DMWDiNkvkdmzmpCSkqT7EnIwlnaix6qy/K8AAAc2SURBVOfj4Um8mLRjYhKPINg3fyiOmNfCmvb5nL72CP7ycavGe+1r2+ezdEGrx7fN6lRdJ/43nX0cbzr7uCnXKY4dD6dfBsOZL4b9lwXDo6Pp8th/vey6o2PsLT6PJl80wyPJwUkJmiSa0mcmzEtK1ykuT+ZFuk6Txrch9r1XmXWz88n0vm03CZYumM269vmsaZ/Horkt5fmHMLOqUteJPw+PHZtZo/FVJ2ZmDcaJ38yswTjxm5k1GCd+M7MG48RvZtZgnPjNzBqME7+ZWYNx4jczazCKUte7nQGSuoGuQ3z7UmDbDIZT69we+7gt9uf22F89tMfaiFg2cWFNJP7DIakQEesrHUe1cHvs47bYn9tjf/XcHh7qMTNrME78ZmYNphES/2WVDqDKuD32cVvsz+2xv7ptj7of4zczs/01Qo/fzMwynPjNzBpM3SR+Sc+W9EdJ90h63wFeny3pqvT130taV/4oyyNHW7xT0p2SbpP0S0lrKxFnuUzXHpn1XigpJNXlKXxFedpD0ovTv5ENkq4sd4zlkuP/yhpJ10i6Of3/8txKxDnjknu81vYDaAb+DBwLtAK3AidMWOfNwBfS6ZcCV1U67gq2xTnAvHT6TfXaFnnbI12vDbgOuAFYX+m4K/z38UjgZuCIdH55peOuYFtcBrwpnT4B6Kx03DPxqJce/xnAPRFxb0QMA98ELpiwzgXA5en0fwJPV33eTXzatoiIayJidzp7A3B0mWMspzx/GwAfAv43MFjO4CogT3tcBHwuInoBImJrmWMslzxtEcDCdHoRsKmM8ZVMvST+VcADmfkH02UHXCciRoA+oL0s0ZVXnrbIeh3w05JGVFnTtoek04DVEfHjcgZWIXn+Po4Hjpf0O0k3SHp22aIrrzxtcQnwSkkPAj8B3lae0Eqr4W+23sgkvRJYDzy10rFUiqQm4FPAqyscSjWZRTLcczbJr8HrJD02InZUNKrKeBnw1Yj4pKSzgK9LOikixiod2OGolx7/RmB1Zv7odNkB15E0i+Rn2/ayRFdeedoCSecC/wCcHxFDZYqtEqZrjzbgJODXkjqBM4Gr6/gAb56/jweBqyNib0TcB/yJ5Iug3uRpi9cB3wKIiOuBOSTF22pavST+G4FHSjpGUivJwdurJ6xzNXBhOv0i4FeRHrGpM9O2haTHAV8kSfr1On5bNGV7RERfRCyNiHURsY7kmMf5EVGoTLgll+f/yvdJevtIWkoy9HNvOYMskzxtcT/wdABJjyFJ/N1ljbIE6iLxp2P2bwV+BtwFfCsiNkj6oKTz09W+DLRLugd4JzDpaX21LGdbfBxYAHxb0i2SJv6x142c7dEwcrbHz4Dtku4ErgHeHRF19+s4Z1u8C7hI0q3AN4BX10OH0SUbzMwaTF30+M3MLD8nfjOzBuPEb2bWYJz4zcwajBO/mVmDceK3miZpND0ldYOkWyW9K70at1LxvEPSvEle+3VaCfJWSTdKOnWabS2W9ObSRGqNzInfat2eiDg1Ik4EngE8B/jniSulV2uXwzuAAyb+1Csi4hTg30mup5jKYpKqsmYzyonf6kZ6FfLFwFuVeLWkqyX9CvilpCWSvp/WVb9B0skAki6R9HVJ10u6W9JF6XJJ+rikOyTdLukl6fKzJf2ouF9Jl6b7+htgJXCNpGumCfd60oJgkhak90W4Kd1PsULkx4Dj0l80H0/XfXf6a+E2SR+YscazhuIibVZXIuJeSc3A8nTRacDJEdEj6bPAzRHxAklPA74GFIdbTiap0zMfuFnSj4Gz0tdPIanPcqOk66bY92ckvRM4JyK2TRPqs0lKI0BSCvovIqI/LZFwQ3o19fuAkyLiVABJzySpmXMGIJKaQk+JiEljMjsQJ36rd7+IiJ50+snACwEi4leS2iUVa63/ICL2AHvS3voZ6frfiIhRYIuka4HHA/2HEc8VaV2YBez70hHwEUlPAcZIfgmsOMB7n5k+bk7nF5B8ETjx20Fx4re6IulYYBQoFp/blfOtE2uXTFXLZIT9h0nn5NwHwCuADpLx/c8Cf5kuWwacHhF70yqhB9qmgI9GxBcPYn9mD+MxfqsbkpYBXwAunaSQ1m9IkiySzga2RUSx936BpDmS2kkqU96Yrv8SSc3ptp8C/A/QBZyg5D7Oi0mrN6YGSEo9TyqN7Z+AMyU9mqRE+NY06Z8DFO+BPHFbPwNeK2lB+hlWSVqO2UFyj99q3VxJtwAtJD3xr5PcWOVALgG+Iuk2YDf7ynQD3EZSiXIp8KGI2CTpeyTj/LeS/AJ4T0Q8BCDpW8AdwH3sG3qB5B6t/yVpU0ScM1nQEbFH0ieBdwPvBX4o6XagAPwhXWe7krtg3QH8NCLenZYGvl7JXUN3Aq9k368bs1xcndManqRLgJ0R8YlKx2JWDh7qMTNrMO7xm5k1GPf4zcwajBO/mVmDceI3M2swTvxmZg3Gid/MrMH8f3exa+yy1iAHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# your code here \n",
    "reg = [0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]\n",
    "valerr = np.zeros(len(reg))\n",
    "for i in range(len(reg)):\n",
    "  my_model_2(smalltrainloader, epo = 50, learn = 0.001, dropout = reg[i])\n",
    "  valerr[i] = validate_2(testloader)\n",
    "plt.plot(reg,valerr)\n",
    "plt.title('Error Rate vs Dropout Rate' )\n",
    "plt.xlabel('Dropout Rate')\n",
    "plt.ylabel('Error rate in percantage')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "acToqyUB4Gzm"
   },
   "source": [
    "**It has a better performance than the MLP model for intermediate dataset and the optimal value is at 0.1 dropout. The error is half as lower as the MLP one.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "H00N6q_Tv9ym",
    "outputId": "757ec0c6-6119-45ce-f547-09dd0f0fd01c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3.29,  3.18,  3.21,  3.8 ,  3.71,  4.53,  4.97,  5.38,  9.64,\n",
       "       28.07])"
      ]
     },
     "execution_count": 87,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "valerr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RvcGd3owwCjB"
   },
   "source": [
    "**the model seems to be working best with 0.1 dropout, however, the change being not significant, I choose to apply no dropout in the next step**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now, doing task 3-checking effect of size of the training data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "pD7NJqz_wfIO",
    "outputId": "41cf8ae9-af64-4b4e-be6a-ac2c0c496979"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda\n",
      "Error rate on train set: 41.400000 %\n",
      "Epoch 1 loss: 2.011344 \n",
      "Error rate on train set: 16.200000 %\n",
      "Epoch 2 loss: 0.807275 \n",
      "Error rate on train set: 10.800000 %\n",
      "Epoch 3 loss: 0.430487 \n",
      "Error rate on train set: 11.000000 %\n",
      "Epoch 4 loss: 0.398587 \n",
      "Error rate on train set: 6.200000 %\n",
      "Epoch 5 loss: 0.241523 \n",
      "Error rate on train set: 5.400000 %\n",
      "Epoch 6 loss: 0.180856 \n",
      "Error rate on train set: 1.800000 %\n",
      "Epoch 7 loss: 0.114422 \n",
      "Error rate on train set: 1.200000 %\n",
      "Epoch 8 loss: 0.079325 \n",
      "Error rate on train set: 1.000000 %\n",
      "Epoch 9 loss: 0.055221 \n",
      "Error rate on train set: 0.200000 %\n",
      "Epoch 10 loss: 0.033421 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 11 loss: 0.022630 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 12 loss: 0.016912 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 13 loss: 0.010876 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 14 loss: 0.005504 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 15 loss: 0.003655 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 16 loss: 0.002257 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 17 loss: 0.001807 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 18 loss: 0.001369 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 19 loss: 0.001136 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 20 loss: 0.000924 \n",
      "Error Rate on test set: 12.780000 %\n",
      "cuda\n",
      "Error rate on train set: 38.800000 %\n",
      "Epoch 1 loss: 1.598164 \n",
      "Error rate on train set: 14.500000 %\n",
      "Epoch 2 loss: 0.516355 \n",
      "Error rate on train set: 7.900000 %\n",
      "Epoch 3 loss: 0.305476 \n",
      "Error rate on train set: 6.500000 %\n",
      "Epoch 4 loss: 0.233557 \n",
      "Error rate on train set: 3.200000 %\n",
      "Epoch 5 loss: 0.150767 \n",
      "Error rate on train set: 2.400000 %\n",
      "Epoch 6 loss: 0.114057 \n",
      "Error rate on train set: 1.700000 %\n",
      "Epoch 7 loss: 0.087075 \n",
      "Error rate on train set: 1.700000 %\n",
      "Epoch 8 loss: 0.071278 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 9 loss: 0.030210 \n",
      "Error rate on train set: 0.400000 %\n",
      "Epoch 10 loss: 0.031356 \n",
      "Error rate on train set: 0.300000 %\n",
      "Epoch 11 loss: 0.023869 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 12 loss: 0.014909 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 13 loss: 0.007742 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 14 loss: 0.005511 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 15 loss: 0.002748 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 16 loss: 0.001759 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 17 loss: 0.001336 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 18 loss: 0.001023 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 19 loss: 0.000826 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 20 loss: 0.000722 \n",
      "Error Rate on test set: 7.920000 %\n",
      "cuda\n",
      "Error rate on train set: 26.450000 %\n",
      "Epoch 1 loss: 1.062015 \n",
      "Error rate on train set: 9.900000 %\n",
      "Epoch 2 loss: 0.372236 \n",
      "Error rate on train set: 6.250000 %\n",
      "Epoch 3 loss: 0.252569 \n",
      "Error rate on train set: 4.500000 %\n",
      "Epoch 4 loss: 0.188070 \n",
      "Error rate on train set: 2.950000 %\n",
      "Epoch 5 loss: 0.130178 \n",
      "Error rate on train set: 1.350000 %\n",
      "Epoch 6 loss: 0.073367 \n",
      "Error rate on train set: 1.050000 %\n",
      "Epoch 7 loss: 0.056384 \n",
      "Error rate on train set: 0.300000 %\n",
      "Epoch 8 loss: 0.025872 \n",
      "Error rate on train set: 0.050000 %\n",
      "Epoch 9 loss: 0.011625 \n",
      "Error rate on train set: 0.050000 %\n",
      "Epoch 10 loss: 0.010098 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 11 loss: 0.004590 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 12 loss: 0.003154 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 13 loss: 0.001818 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 14 loss: 0.001456 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 15 loss: 0.001096 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 16 loss: 0.000875 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 17 loss: 0.000743 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 18 loss: 0.000688 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 19 loss: 0.000651 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 20 loss: 0.000551 \n",
      "Error Rate on test set: 5.010000 %\n",
      "cuda\n",
      "Error rate on train set: 20.600000 %\n",
      "Epoch 1 loss: 0.782774 \n",
      "Error rate on train set: 6.025000 %\n",
      "Epoch 2 loss: 0.246495 \n",
      "Error rate on train set: 2.950000 %\n",
      "Epoch 3 loss: 0.126002 \n",
      "Error rate on train set: 1.575000 %\n",
      "Epoch 4 loss: 0.075133 \n",
      "Error rate on train set: 0.975000 %\n",
      "Epoch 5 loss: 0.049420 \n",
      "Error rate on train set: 0.325000 %\n",
      "Epoch 6 loss: 0.029049 \n",
      "Error rate on train set: 0.300000 %\n",
      "Epoch 7 loss: 0.022080 \n",
      "Error rate on train set: 0.050000 %\n",
      "Epoch 8 loss: 0.010834 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 9 loss: 0.007945 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 10 loss: 0.008002 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 11 loss: 0.003617 \n",
      "Error rate on train set: 0.150000 %\n",
      "Epoch 12 loss: 0.011998 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 13 loss: 0.004072 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 14 loss: 0.001103 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 15 loss: 0.000468 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 16 loss: 0.000308 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 17 loss: 0.000239 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 18 loss: 0.000206 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 19 loss: 0.000179 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 20 loss: 0.000157 \n",
      "Error Rate on test set: 3.160000 %\n",
      "cuda\n",
      "Error rate on train set: 15.300000 %\n",
      "Epoch 1 loss: 0.569087 \n",
      "Error rate on train set: 3.512500 %\n",
      "Epoch 2 loss: 0.153726 \n",
      "Error rate on train set: 1.712500 %\n",
      "Epoch 3 loss: 0.078701 \n",
      "Error rate on train set: 0.962500 %\n",
      "Epoch 4 loss: 0.051929 \n",
      "Error rate on train set: 0.437500 %\n",
      "Epoch 5 loss: 0.030214 \n",
      "Error rate on train set: 0.250000 %\n",
      "Epoch 6 loss: 0.018124 \n",
      "Error rate on train set: 0.112500 %\n",
      "Epoch 7 loss: 0.012026 \n",
      "Error rate on train set: 0.075000 %\n",
      "Epoch 8 loss: 0.009188 \n",
      "Error rate on train set: 0.012500 %\n",
      "Epoch 9 loss: 0.006281 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 10 loss: 0.003754 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 11 loss: 0.002130 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 12 loss: 0.002170 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 13 loss: 0.001511 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 14 loss: 0.004750 \n",
      "Error rate on train set: 0.137500 %\n",
      "Epoch 15 loss: 0.024438 \n",
      "Error rate on train set: 0.175000 %\n",
      "Epoch 16 loss: 0.024055 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 17 loss: 0.007423 \n",
      "Error rate on train set: 0.012500 %\n",
      "Epoch 18 loss: 0.002067 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 19 loss: 0.000417 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 20 loss: 0.000182 \n",
      "Error Rate on test set: 2.140000 %\n",
      "cuda\n",
      "Error rate on train set: 8.487500 %\n",
      "Epoch 1 loss: 0.344376 \n",
      "Error rate on train set: 1.756250 %\n",
      "Epoch 2 loss: 0.079399 \n",
      "Error rate on train set: 0.793750 %\n",
      "Epoch 3 loss: 0.043397 \n",
      "Error rate on train set: 0.343750 %\n",
      "Epoch 4 loss: 0.026998 \n",
      "Error rate on train set: 0.206250 %\n",
      "Epoch 5 loss: 0.016801 \n",
      "Error rate on train set: 0.100000 %\n",
      "Epoch 6 loss: 0.013066 \n",
      "Error rate on train set: 0.087500 %\n",
      "Epoch 7 loss: 0.012360 \n",
      "Error rate on train set: 0.106250 %\n",
      "Epoch 8 loss: 0.012690 \n",
      "Error rate on train set: 0.043750 %\n",
      "Epoch 9 loss: 0.008208 \n",
      "Error rate on train set: 0.012500 %\n",
      "Epoch 10 loss: 0.005965 \n",
      "Error rate on train set: 0.056250 %\n",
      "Epoch 11 loss: 0.009479 \n",
      "Error rate on train set: 0.018750 %\n",
      "Epoch 12 loss: 0.004226 \n",
      "Error rate on train set: 0.031250 %\n",
      "Epoch 13 loss: 0.006539 \n",
      "Error rate on train set: 0.012500 %\n",
      "Epoch 14 loss: 0.003112 \n",
      "Error rate on train set: 0.006250 %\n",
      "Epoch 15 loss: 0.001982 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 16 loss: 0.006928 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 17 loss: 0.008827 \n",
      "Error rate on train set: 0.012500 %\n",
      "Epoch 18 loss: 0.004638 \n",
      "Error rate on train set: 0.006250 %\n",
      "Epoch 19 loss: 0.002875 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 20 loss: 0.002379 \n",
      "Error Rate on test set: 1.740000 %\n",
      "cuda\n",
      "Error rate on train set: 5.928125 %\n",
      "Epoch 1 loss: 0.226111 \n",
      "Error rate on train set: 1.162500 %\n",
      "Epoch 2 loss: 0.054956 \n",
      "Error rate on train set: 0.550000 %\n",
      "Epoch 3 loss: 0.032702 \n",
      "Error rate on train set: 0.331250 %\n",
      "Epoch 4 loss: 0.021603 \n",
      "Error rate on train set: 0.168750 %\n",
      "Epoch 5 loss: 0.015298 \n",
      "Error rate on train set: 0.100000 %\n",
      "Epoch 6 loss: 0.011733 \n",
      "Error rate on train set: 0.043750 %\n",
      "Epoch 7 loss: 0.007599 \n",
      "Error rate on train set: 0.028125 %\n",
      "Epoch 8 loss: 0.008224 \n",
      "Error rate on train set: 0.018750 %\n",
      "Epoch 9 loss: 0.005045 \n",
      "Error rate on train set: 0.034375 %\n",
      "Epoch 10 loss: 0.009955 \n",
      "Error rate on train set: 0.021875 %\n",
      "Epoch 11 loss: 0.007003 \n",
      "Error rate on train set: 0.021875 %\n",
      "Epoch 12 loss: 0.005808 \n",
      "Error rate on train set: 0.006250 %\n",
      "Epoch 13 loss: 0.003453 \n",
      "Error rate on train set: 0.003125 %\n",
      "Epoch 14 loss: 0.003607 \n",
      "Error rate on train set: 0.006250 %\n",
      "Epoch 15 loss: 0.004399 \n",
      "Error rate on train set: 0.025000 %\n",
      "Epoch 16 loss: 0.007024 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 17 loss: 0.001408 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 18 loss: 0.000152 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 19 loss: 0.000032 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 20 loss: 0.000015 \n",
      "Error Rate on test set: 1.010000 %\n",
      "cuda\n",
      "Error rate on train set: 3.313333 %\n",
      "Epoch 1 loss: 0.139145 \n",
      "Error rate on train set: 0.815000 %\n",
      "Epoch 2 loss: 0.040496 \n",
      "Error rate on train set: 0.400000 %\n",
      "Epoch 3 loss: 0.024635 \n",
      "Error rate on train set: 0.196667 %\n",
      "Epoch 4 loss: 0.016547 \n",
      "Error rate on train set: 0.113333 %\n",
      "Epoch 5 loss: 0.012149 \n",
      "Error rate on train set: 0.056667 %\n",
      "Epoch 6 loss: 0.009577 \n",
      "Error rate on train set: 0.066667 %\n",
      "Epoch 7 loss: 0.008364 \n",
      "Error rate on train set: 0.018333 %\n",
      "Epoch 8 loss: 0.005021 \n",
      "Error rate on train set: 0.020000 %\n",
      "Epoch 9 loss: 0.005747 \n",
      "Error rate on train set: 0.023333 %\n",
      "Epoch 10 loss: 0.004999 \n",
      "Error rate on train set: 0.010000 %\n",
      "Epoch 11 loss: 0.003425 \n",
      "Error rate on train set: 0.008333 %\n",
      "Epoch 12 loss: 0.004128 \n",
      "Error rate on train set: 0.016667 %\n",
      "Epoch 13 loss: 0.003437 \n",
      "Error rate on train set: 0.008333 %\n",
      "Epoch 14 loss: 0.003897 \n",
      "Error rate on train set: 0.010000 %\n",
      "Epoch 15 loss: 0.002677 \n",
      "Error rate on train set: 0.008333 %\n",
      "Epoch 16 loss: 0.002312 \n",
      "Error rate on train set: 0.003333 %\n",
      "Epoch 17 loss: 0.002269 \n",
      "Error rate on train set: 0.010000 %\n",
      "Epoch 18 loss: 0.003571 \n",
      "Error rate on train set: 0.013333 %\n",
      "Epoch 19 loss: 0.003545 \n",
      "Error rate on train set: 0.000000 %\n",
      "Epoch 20 loss: 0.000879 \n",
      "Error Rate on test set: 0.820000 %\n"
     ]
    }
   ],
   "source": [
    "#Now, doing task 3-checking effect of size of the training data\n",
    "amount = [500, 1000, 2000, 4000, 8000, 16000, 32000,60000]\n",
    "valerr_2 = np.zeros(len(amount))\n",
    "for i in range(len(amount)):\n",
    "  mnist_trainset_small =  [ mnist_trainset[i] for i in range(0,amount[i]) ] \n",
    "  smalltrainloader = torch.utils.data.DataLoader( mnist_trainset_small, batch_size=64, shuffle=True)\n",
    "  my_model_2(smalltrainloader, epo = 20, learn = 0.001, dropout = 0)\n",
    "  valerr_2[i] = validate_2(testloader)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MvVNlJVy0DF2",
    "outputId": "20b3df65-a1a9-4833-b742-3f4720f00161"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([12.78,  7.92,  5.01,  3.16,  2.14,  1.74,  1.01,  0.82])"
      ]
     },
     "execution_count": 128,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "valerr_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 312
    },
    "id": "F9wcvQ8Jz2bo",
    "outputId": "980761ab-85db-422f-a90d-e816e990409f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Error rate in percantage')"
      ]
     },
     "execution_count": 129,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEWCAYAAACOv5f1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5xdVX338c/3nLklk2Rym9yBJIBBEQSMyqUqKq0gF7W1KtWnWFS8a2ufh4JtBa3a1vpUrWgVFaWFCoKIigIqj1wEuSSA3C8hCSaQkEkgIReSzOX3/LHXTPZM5nIymTMnc873/Xrt1+yz9jp7r33Omd9ee62191ZEYGZmtaVQ6QKYmdnoc/A3M6tBDv5mZjXIwd/MrAY5+JuZ1SAHfzOzGuTgb1YGkq6VdMZI560mkrZIWljpctQqB/8qIWmlpBfSP1T3dMEol+FGSdvTttdLukrS7BLfe7yk1eUu4xBlyH92XX0+z3ftyboi4qSIuHik8+6J9Jl25fZhtaQfSnrFHqzjfEmXjHTZACJiQkQsL8e6bWgO/tXl1PQP1T19tL9Mkur6SSvuyYYGyf/RiJgAHARMAL60J+utpPxnB/yB3p/npd35+vv89mFPp/2ZCBwNPALcIukNlS2WVZqDfw2Q9B5Jt0r6sqQNwPmSvi/pPyX9QtJW4HWSXpxq7xslPSjptNw6dss/2DYjYiNwNXBEbh1/JelhSZslLZf0gZTeDFwLzMnVUudIKkg6R9ITkjakWuvUAfbxYUmn5F7XSWqTdJSkJkmXpHVslHSXpJl78Pkdn2rNfydpLfA9SVMkXZO28Vyan5d7z42S3pf7/H8r6Usp7wpJJw0z7wJJN6fP8NeSvl5KzTwyqyPi08B3gH/NrfOrklZJel7SUkmvTuknAp8C3pG+k9+n9H6/xwE+u4Mk3SRpUzobvDy3LNLy/Pe+RdI2SZHLd2ba3nOSrpd0wFD7a0Nz8K8drwKWAzOBz6e0v0jzE4E7gJ8BvwRmAB8DLpW0KLeOfP7fDrYxSdOAPwWW5ZLXAacAk4C/Ar4s6aiI2AqcRKqlpunpVIa3AK8F5gDPAV8fYJM/AE7PvX4jsD4i7gbOAFqA/YBpwAeBFwYrfz9mAVOBA4CzyP53vpde75/WN1gz26uAR4HpwBeB70rSMPL+D3Bn2o/zgf+1h/sBcBVwVDroAtxFdpCemtZ/haSmiLgO+AJwefpOXpby9/s9DrCtfyL7TU0B5gFf65shIp7uc9b1Y+AyAElvJjsA/SnQCtxC9l3b3ooIT1UwASuBLcDG3PT+tOw9wB/65P8+8F+5168G1gKFXNoPgPP7yz9AGW4EtgGbgADuBfYfJP/VwCfS/PHA6j7LHwbekHs9G2gH6vpZ10HAZmB8en0p8Ok0fyZwG3D4Hn6eJ+TKthNoGiT/EcBzfT6L9+U+/2W5ZePT5zNrT/KSHWQ6uvcxLb8EuGSAMu32mab0Q9I65w7wvueAl6X58wdaf3/fYz/L/gu4EJjXz7IADuqT9nfAUmBcen0t8N7c8kL6jR0w2v9j1Ta55l9d3hIRk3PTt3PLVvWTP582B1gVEV25tCeBuUOso6+PR0QLcDi7ansASDpJ0u2SnpW0EXgTWe12IAcAP05NNRvJDgadZGcvvUTEsrT8VEnjgdPIarEA/w1cD1wm6WlJX5RUX8K+5LVFxPbcvoyX9C1JT0p6HrgZmKyB+0LW5sq6Lc1O2MO8c4Bnc2lQ2nfS11yywLsRQNL/Ts0qm9Ln3MIg38sefo9nAwLuVNaUeOZg6wU+QfY77j4zOwD4au438Gxa39wBVmMlcvCvHf3dvjWf9jSwn6T8b2J/4Kkh1tH/xiLuBz4HfF2ZRuBHZB3AMyNiMvALsn/kgda9CjipzwGtKSKe6icv7Gr6eTPwUDogEBHtEfGZiHgJcCxZk8VflrovA5Tvb4FFwKsiYhLwmpQ+UFPOSFgDTE0Ht277DWM9bwXujoitqX3/bODtwJT0vWxigO+lhO+xl4hYGxHvj4g5wAeAb0g6qG++1Lx4MfD2iMgf0FYBH+jzGxgXEbcNY78tx8Hfut1Bdjp9tqR6SccDp5LaXofpYrJa+mlAA9AItAEdqZb3J7m8zwDTJLXk0r4JfL67g09Sa2oDHshlaZ0fYletH0mvk3RYqpU/T9Z01NX/Kko2kaydf6OyTujz9nJ9Q4qIJ4ElZB32DZKOIfuOhpQOwHMlnQe8j6wdHbL96CD7XuokfZqsLb/bM8D8XKVgqO+x73b/XLs6wp8jO5h09ckzCfgJ8PcR0bcv6ZvAuZIOTXlbJP15Kftsg3Pwry4/6zNq4selvjEidpIFkpOA9cA3gL+MiEeGW5i0zq8C/xgRm4GPAz8kCwJ/Afw0l/cRspr78nSKPye996fALyVtBm4n6wwdaHtrgN+R1e4vzy2aBVxJFvgfBm4iawraG18BxpF9VrcD1+3l+kr1LuAYYAPZmdXlwI5B8s+RtIWsP+gu4DDg+Ij4ZVp+PVnZHyNr5ttO76akK9LfDZLuHup77McrgDtSGX5K1jfQd2z/UWRnUV/O/34BIuLHZCOTLkvNaw+Q/UZtLynCD3MxG6vS0MlHIqLsZx5WXVzzNxtDJL1C0oHKroE4kax/4+pKl8vGnrF0paKZZU1YV5GN818NfCgi7qlskWwscrOPmVkNcrOPmVkNGhPNPtOnT4/58+dXuhhmZmPK0qVL10dEa3/LxkTwnz9/PkuWLKl0MczMxhRJTw60zM0+ZmY1yMHfzKwGOfibmdUgB38zsxrk4G9mVoMc/M3MapCDv5lZDarq4H/Dw8/wjRuXDZ3RzKzGVHXwv+mxNi68ue+tw83MrKqDf1N9kR3te/vAJjOz6lPVwb+xrsCOjk5851Izs96qPvh3BXR0OfibmeVVefAvArC9vbPCJTEz27dUd/Cvz3ZvR4fb/c3M8qo7+Nc5+JuZ9afKg3/W7LPDzT5mZr1UdfBvcrOPmVm/qjr499T8HfzNzHopW/CXdJGkdZIeyKX9m6RHJN0n6ceSJpdr+5Br83ezj5lZL+Ws+X8fOLFP2q+Al0bE4cBjwLll3H7PaJ/trvmbmfVStuAfETcDz/ZJ+2VEdKSXtwPzyrV9cIevmdlAKtnmfyZw7UALJZ0laYmkJW1tbcPagId6mpn1ryLBX9LfAx3ApQPliYgLI2JxRCxubW0d1naa6t3ha2bWn7rR3qCk9wCnAG+IMt9xbVfN380+ZmZ5oxr8JZ0InA28NiK2lXt7u9r8XfM3M8sr51DPHwC/AxZJWi3pvcAFwETgV5LulfTNcm0ffG8fM7OBlK3mHxGn95P83XJtrz8NxTTU06N9zMx6qeorfAsF0VAsuOZvZtZHVQd/2PU0LzMz26X6g3990TV/M7M+qj/41xU82sfMrI/qD/71bvYxM+ur+oN/nZt9zMz6qoHgX/BQTzOzPmoi+Lvmb2bWW/UHf4/2MTPbTdUH/6a6gu/nb2bWR9UH/8b6Ijtd8zcz66X6g7/b/M3MdlMjwd/NPmZmeTUQ/Its9xW+Zma9VH/w9xW+Zma7qfrg31RXpL0z6Owq6xMjzczGlKoP/t1P8/KIHzOzXao/+Psh7mZmu6mB4J8e4u6av5lZjxoI/qnm7xE/ZmY9qj/4pzb/7W72MTPrUf3Bv7vZxzV/M7MeJQV/SQdIOiHNj5M0sbzFGjlN9e7wNTPra8jgL+n9wJXAt1LSPODqchZqJLnD18xsd6XU/D8CHAc8DxARjwMzhnqTpIskrZP0QC5tqqRfSXo8/Z0y3IKXykM9zcx2V0rw3xERO7tfSKoDSrlc9vvAiX3SzgFuiIiDgRvS67Lq7vB1m7+Z2S6lBP+bJH0KGCfpj4ErgJ8N9aaIuBl4tk/ym4GL0/zFwFv2oKzD4mYfM7PdlRL8zwHagPuBDwC/AP5hmNubGRFr0vxaYOZAGSWdJWmJpCVtbW3D3NyuZh8/xN3MbJe6oTJERBfw7TSNmIgISQM2H0XEhcCFAIsXLx72Xdl2tfm75m9m1m3I4C/pfnZv498ELAE+FxEb9mB7z0iaHRFrJM0G1u3Be4elqb672cc1fzOzbqU0+1wL/Bx4V5p+Rhb415J16u6JnwJnpPkzgJ/s4fv3mG/vYGa2uyFr/sAJEXFU7vX9ku6OiKMkvXugN0n6AXA8MF3SauA84F+AH0p6L/Ak8PbhF700dcUCxYLc7GNmllNK8C9KemVE3Akg6RVAMS3rGOhNEXH6AIvesGdF3Ht+jq+ZWW+lBP/3ARdJmgCI7GKv90lqBv65nIUbKVnwd83fzKxbKaN97gIOk9SSXm/KLf5huQo2krKHuLvmb2bWrZSaP5JOBg4FmiQBEBGfLWO5RlRTvWv+ZmZ5pdzY7ZvAO4CPkTX7/DlwQJnLNaIa64oe7WNmllPKUM9jI+Ivgeci4jPAMcCLyluskdVY7w5fM7O8UoL/C+nvNklzgHZgdvmKNPLc4Wtm1lspbf7XSJoM/BtwN9nVvt8pa6lGWGNdkRfc4Wtm1qOU4P/FiNgB/EjSNUATsL28xRpZjXUFNr6wc+iMZmY1opRmn991z0TEjjTU83eD5N/nNNYX2O4OXzOzHgPW/CXNAuaS3cf/SLKRPgCTgPGjULYR01RXdIevmVnOYM0+bwTeQ/bM3n/PpW8GPlXGMo24xvqCh3qameUMGPwj4mLgYkl/FhE/GsUyjbjGuqJH+5iZ5ZQ62ucvgPn5/GPpCl/f2M3MrLdSgv9PyB7eshTYUd7ilEf3OP+IoPv2FGZmtayU4D8vIk4se0nKqLG+SAS0dwYNdQ7+ZmalDPW8TdJhZS9JGfU8xN1NP2ZmQGk1/z8C3iNpBVmzj8iev354WUs2ghq7n+Pb3pVdomZmVuNKCf4nlb0UZdbzHF/X/M3MgNIe5vIkgKQZjNF6867g7+GeZmZQ2v38T5P0OLACuAlYCVxb5nKNqMa6XLOPmZmV1OH7T8DRwGMRsYDsAey3l7VUI6yx3s0+ZmZ5pQT/9ojYABQkFSLiN8DiMpdrRLnZx8yst1I6fDdKmgDcDFwqaR2wtbzFGllNabSPH+JuZpYppeb/ZmAb8DfAdcATwKnlLNRIc83fzKy3UoL/DKAhIjrSzd6+DUzcm41K+htJD0p6QNIPJJV1FFFPh6+Dv5kZUFrwvwLIR83OlDYskuYCHwcWR8RLgSLwzuGurxQ9NX83+5iZAaUF/7qI6HkGYppv2Mvt1pE9JKaO7MEwT+/l+ga1a7SPa/5mZlBa8G+TdFr3C0lvBtYPd4MR8RTwJeAPwBpgU0T8sm8+SWdJWiJpSVtb23A3B7jZx8ysr1KC/weBT0n6g6Q/AH8HnDXcDUqaQtaJvACYAzRLenfffBFxYUQsjojFra2tw90cAE2p5v/Czo69Wo+ZWbUYdKinpCLwoYg4Og33JCK27OU2TwBWRERb2sZVwLHAJXu53gE11hWZ1FTHus1j8nEEZmYjbtCaf0R0kt3Vk4jYMgKBH7LmnqMljVf2ZJU3AA+PwHoHNaulibWbtpd7M2ZmY0IpF3ndI+mnZCN8ei7uioirhrPBiLhD0pXA3UAHcA9w4XDWtSdmTmrimecd/M3MoLTg3wRsAF6fSwtgWMEfICLOA84b7vuHY3ZLE489s3k0N2lmts8q5ZbOfzUaBSm3WZOaaNu8g47OLuqKpfRzm5lVr1Ju6fwiSTdIeiC9PlzSP5S/aCNrZksTXQFtW9zpa2ZWShX428C5QDtARNxHma/ILYdZk7I7SLjT18ystOA/PiLu7JM25gbMz2rJgr87fc3MSgv+6yUdSNbJi6S3kV2ZO6Z01/zXuOZvZlbSaJ+PkA3FPETSU2SPc3xXWUtVBlObG2goFljrmr+ZWUmjfZYDJ0hqBgoRMSbHS0pixqRGnnHN38yspNE+0yT9B3ALcKOkr0qaVv6ijbxZk5pc8zczo7Q2/8uANuDPgLel+cvLWahy8S0ezMwypQT/2RHxTxGxIk2fA2aWu2Dl0F3zj4hKF8XMrKJKCf6/lPROSYU0vR24vtwFK4dZLU1sb+/i+RfG3EhVM7MRVUrwfz/wP8CONF0GfEDSZknPl7NwI21m94Vebvc3sxo3ZPCPiIkRUYiI+jQVUtrEiJg0GoUcKbNbHPzNzKC0mn/V6Kn5b3qhwiUxM6usGg3+vrmbmdW2mgr+DXUFpjU3uNnHzGpeKbd36H6W78x8/oj4Q7kKVU5+opeZWQnBX9LHyJ669QzQlZIDOLyM5Sqb2S1NPO0LvcysxpVS8/8EsCgiNpS7MKNhZksT96zaWOlimJlVVClt/quATeUuyGiZNamJZ7fuZEdHZ6WLYmZWMaXU/JeT3dDt52QXeQEQEf9etlKVUfd9/dc9v4P9po6vcGnMzCqjlOD/hzQ1pGlM636i15pN2x38zaxmlXI//8+MRkFGyyxf5WtmNnDwl/SViPhrST8jPcIxLyJOK2vJyqT7Qi8/1MXMatlgNf//Tn+/NNIblTQZ+A7wUrIDy5kR8buR3k5/JjXVMa6+6Jq/mdW0AYN/RCxNf28qw3a/ClwXEW+T1ACMWuO7JGb7oS5mVuNKusJ3JElqAV4DvAcgInYCO0ezDDP9OEczq3GVuLfPArJHQX5P0j2SvpMeDt+LpLMkLZG0pK2tbUQL4Mc5mlmtKzn4Sxqpppk64CjgPyPiSGArcE7fTBFxYUQsjojFra2tI7TpzMxJTazbvJ2uLj/O0cxq05DBX9Kxkh4CHkmvXybpG3uxzdXA6oi4I72+kuxgMGpmTWqkvTN4dtuotjaZme0zSqn5fxl4I7ABICJ+T9ZmPywRsRZYJWlRSnoD8NBw1zccs1rGAbjpx8xqVknNPhGxqk/S3t4Y52PApZLuA44AvrCX69sjPRd6OfibWY0qZbTPKknHAiGpnuwunw/vzUYj4l5g8d6sY2/M8oPczazGlVLz/yDwEWAu8BRZTf3D5SxUuU2f0EBB+KEuZlazSqn5L4qId+UTJB0H3FqeIpVfXbHAjIlNrHGzj5nVqFJq/l8rMW1MmdnixzmaWe0a7MZuxwDHAq2SPplbNAkolrtg5TZrUiPL27ZWuhhmZhUxWM2/AZhAdoCYmJueB95W/qKV1yzf4sHMathgN3a7CbhJ0vcj4slRLNOomNnSxObtHWzd0UFz46jf4sjMrKJKiXrbJP0bcCjQ1J0YEa8vW6lGwezcQ10ObJ1Q4dKYmY2uUjp8LyW7tcMC4DPASuCuMpZpVPihLmZWy0oJ/tMi4rtAe0TcFBFnAmO61g++0MvMalspzT7t6e8aSScDTwNTy1ek0ZF/kLuZWa0pJfh/Lj2A5W/JxvdPAv6mrKUaBeMb6mgZV8+qZ7dVuihmZqNu0OAvqQgcHBHXAJuA141KqUbJsQdO44ZH1tHZFRQLqnRxzMxGzaBt/hHRCZw+SmUZdW86bDZtm3dw18pnK10UM7NRVUqH762SLpD0aklHdU9lL9koeP0hM2iqL/CL+9dUuihmZqOqlDb/I9Lfz+bSgioY8dPcWMfrFs3gF/ev5bxTD3XTj5nVjCGDf0RUVTt/XycfPptrH1jLnSue5ZgDp1W6OGZmo6LkB7hXKzf9mFktqvngP76hjtcfMoNrH1hDZ1dUujhmZqNi0OAvqZAe4VjVTj5sDuu37OSOFRsqXRQzs1Ex1FDPLuDro1SWinndIa2Mqy+66cfMakYpzT43SPozSVU7FKa76ee6B9bS0dlV6eKYmZVdKcH/A8AVwE5Jz0vaLOn5Mpdr1J18+GzWb9nJnSt8wZeZVb8hg39ETIyIQkTUR8Sk9HrSaBRuNL1u0QzG1Rf5uZt+zKwGlDTaR9Jpkr6UplPKXahKGNdQ5PUvdtOPmdWGIYO/pH8BPgE8lKZPSPrnvd2wpKKkeyRds7frGimnHDabDVvd9GNm1a+Umv+bgD+OiIsi4iLgRODkEdj2J4CHR2A9I+b4RTMY31DkGjf9mFmVK/Uir8m5+Za93aikeWQHkO/s7bpG0riGokf9mFlNKCX4fwG4R9L3JV0MLAU+v5fb/QpwNjBghJV0lqQlkpa0tbXt5eZKd8rhs3l2607ucNOPmVWxIa/wJQvQRwNXAT8CjomIy4e7wdRhvC4ilg6WLyIujIjFEbG4tbV1uJvbYz1NP/e56cfMqlcpV/ieHRFrIuKnaVq7l9s8DjhN0krgMuD1ki7Zy3WOmKb6Im948Uyuf9BNP2ZWvUpp9vm1pP8taT9JU7un4W4wIs6NiHkRMR94J/D/IuLdw11fOZx8WNb0c/tyN/2YWXUq5WEu70h/P5JLC2DhyBdn33D8olaaG4r8/P6n+aODp1e6OGZmI66UNv9zImJBn2lEAn9E3BgR+9xFY91NP9c9sJZ2N/2YWRUqpc3//4xSWfYpJx8+m+e2tXP7ct/m2cyqz6i3+Y8Vr31RavrxqB8zq0KlBP93kLX330w2xn8psKSchdoXNNUXOeElM7nuQTf9mFn1KeWunn3b+0eszX9fd/Jhs9m4rZ3fPeGmHzOrLgMGf0ln5+b/vM+yL5SzUPuK17yolQmNdW76MbOqM1jN/525+XP7LDuxDGXZ5zTVFznhxTO4/iE3/ZhZdRks+GuA+f5eV62TD5/Dxm3t3OamHzOrIoMF/xhgvr/XVevVB09nQmMdP7nnqUoXxcxsxAx2he/L0rN6BYzLPbdXQFPZS7aPaKov8tYj5/Lftz/JlOYGzj3pEOqKpd4J28xs3zRg8I+I4mgWZF923qkvoVgQ3/3tCh5ft4WvnX4kLePqK10sM7NhcxW2BHXFAuefdij//KeH8bsn1vPWb9zK8rYtlS6WmdmwOfjvgdNfuT+XvPdVbNzWzlu+fis3PzZ6D5kxMxtJDv576FULp/GTjxzHnMnjeM/37uSi364gomb6v82sSjj4D8N+U8fzow8dywkvnslnr3mIc350Pzs6OitdLDOzkjn4D1NzYx3ffPfL+djrD+LyJat417fvYP2WHZUulplZSRz890KhIP72TxbxtdOP5P6nNvHmC27loaefH/qNZmYV5uA/Ak592Ryu/OCxdHYFf/aft3HdA74XkJnt2xz8R8hh81r46UePY9GsiXzwkrv56q8fd0ewme2zHPxH0IxJTVx21tH86ZFz+fKvH+Oj/3MPL+x0R7CZ7XtKeYC77YGm+iL/9+0vY9GsifzLdY+wcsNWvv2Xi5kzeVyli2Zm1sM1/zKQxAdeeyAXnfEK/rBhG6ddcCtLn3yu0sUyM+vh4F9GrztkBld9+FiaG4ucfuHtXLFkVaWLZGYGOPiX3cEzJ3L1h49j8fwp/J8r7+PzP3+Izi53BJtZZTn4j4IpzQ1cfOYrOeOYA/j2LSs48/t38fz29koXy8xq2Kh3+EraD/gvYCbZQ2EujIivjnY5Rlt9scBn3vxSXjRrIuf95EHecsGtnHL4bA6cMYGDZ0xkYWszTfW+i7aZjY5KjPbpAP42Iu6WNBFYKulXEfFQBcoy6t71qgM4sHUC/3j1A1zwm2V0twBJsP/U8RzUOoGDZvSeJjb52QFmNrJGPfhHxBpgTZrfLOlhYC5QE8Ef4OiF0/jVJ1/L9vZOVm7YyuPPbGHZui0sa9vCsme2cMvj69mZe2D8rElNux0QDp4xgWkTGiu4F2Y2llV0nL+k+cCRwB39LDsLOAtg//33H9VyjZam+iKHzJrEIbMm9Urv6Oxi1XMv8Pgzm3sOCMvatvDDJavYlrtobMr4+nQwmNjroDC7pQlJo707ZjaGqFK3IJA0AbgJ+HxEXDVY3sWLF8eSJUtGp2D7sIjg6U3bs7OEdVtYtm4zy9Zt4fF1W9i4bVcHcnNDkQPzZwqtEzh45kT2mzLOzx82qyGSlkbE4v6WVaTmL6ke+BFw6VCB33aRxNzJ45g7eRyvfVFrT3pEsGHrzp4DwRPp4HDrsvVcdfdTPfkaigUWTG/moJndB4Ts4LBgejONde5sNqsllRjtI+C7wMMR8e+jvf1qJInpExqZPqGRoxdO67Xs+e3tPJE7KDy+bgv3r97EL+5fQ/dJX0FwwLRmDmzd1XR00IwJHDhjAhMafQcQs2pUif/s44D/Bdwv6d6U9qmI+EUFylL1JjXVc+T+Uzhy/ym90re3d/JEW3aG0H1QWLZuCzc9to72zl1NgTMnNTJ/WjMLW5tZML2ZBdMnsGD6ePabOt5nC2ZjWCVG+/wWcG9khTXVFzl0TguHzmnpld7e2cWTG7ZlB4W2LaxYv5UV67dy/YPP8OzWnT35CoJ5U8Yzf3ozC6d3Hxiyac7kcRQL/orN9mU+p7de6ouFno7ivjZta2fFhq2sWL+FFW1bWb5+Kys3bGXpymfZmhuF1FAscMC08b0OCAumN7OgtZnWCY0eiWS2D3Dwt5K1jK/niPGTOWK/yb3SI4K2zTuyg0E6U1iephsfbet1zUJzQ5EFrd3NR80smD6+Z75lnC9mMxstDv621yQxY1ITMyY17dbh3NkVPL3xBZav38qKti2s3LCN5eu3cu+q5/j5fU+Tv8fd1OaGXmcKC6c3M396M/OnNTOuwf0LZiPJwd/KqlgQ+03NOojzw1MBdnR0surZbSxvy84WVm7YyvK2rdz8WBtXLl3dK++cliYWtGYHggXTuzugJzBvyjjqfe2C2R5z8LeKaawrpquTJ+62bMuOjp4mpPz0s98/zfPbO3ryFQti/6lZ/8L8aVm/QncH9KxJTRTc8WzWLwd/2ydNaKzjpXNbeOnc3qORIoLntrVnnc7rt6W/W1mxfhu3PbGe7e27+hca6wq9mpHyI5OmNje449lqmoO/jSmSmNrcwNTmqbz8gKm9lnV1Bc9s3s6Ktq3ZqKTUnPTo2s386qFn6Mh1MExqqssdGCZkndDTmpk/fbzvomo1wcHfqkahIGa3jGN2yziOPWh6r2UdnV2sfu6FnpFI3U1Kd618jqvvfbpX3taJjSyYtmt4avdBYv+p4/3MBasaDv5WE+qKhWzk0PRmXtdn2fb2Tp7ckDUhZaOSss7nGx55hvVLdl3YJsHcyeN2u35h4fQJzJ3iC9tsbHHwt5rXVF9k0ayJLJq1e8fzphfaWZkbidQ9KunHdz/F5h27Op7ri1LzfB8AAAqFSURBVN0dzxNY2GdU0oyJvrDN9j0O/maDaBlXz8v2m8zL+rmwbf2WndnBIDUldXc+3/x4Gzs7dnU81xdFQ7FAXbFAfbFAQ1FpXtSntPqUluVTyrdrvj6Xvy6tLz9fVxD1dX3yFQo01A08X1copPeI+jRfV8jW51FS1c/B32wYJNE6sZHWiY28ckHvjufOrmDNphd6hqc+vXE77Z1ddHR2sbMz6Ojsor2zi/bOSH+76OgKdnZ0sW1nR898R1da3tFFez/z5XwUR7GgngNB90Ghvp8DVv0AB6uBD1z5g1/v+fy6+z3A9bfuQoH6noOafIa1Bxz8zUZYsSDmTRnPvCnjefXBrUO/YZg600FgZ2cXHbkDSXuf+Y6Upz134BnoINQ9372+nQPM9zpwdQab2zvo6OqivSOlp/mOri52dqRtd3X1umNsOew66xENdYV0dtP7QFFXKKSDWjr7KebOhIZ9gNv9wNV9dpef77ve+mKhYn1FDv5mY1SxIIqF4pgagRQRuw4EHZEdJDp7z3d0RnawSmc/Qx3c8mdV3fP95em17s4udrR3saWzY+CzsZS3oyvo7CrfQasgBm3y+8JbD9vt7HIkOPib2aiRREOdaKAADZUuTem6z7I6uiJreutMzW8dXenspnt5Nt+RDmT5+Z4DS+6g1nN2NMABrqMzaG4sz8Hdwd/MbAjdZ1kANFa2LCPFd8QyM6tBDv5mZjXIwd/MrAY5+JuZ1SAHfzOzGuTgb2ZWgxz8zcxqkIO/mVkNUpTz7lAjRFIb8OQevm06sL4MxakE78u+yfuyb6qmfYG9258DIqLfG0yNieA/HJKWRMTiSpdjJHhf9k3el31TNe0LlG9/3OxjZlaDHPzNzGpQNQf/CytdgBHkfdk3eV/2TdW0L1Cm/anaNn8zMxtYNdf8zcxsAA7+ZmY1qOqCv6QTJT0qaZmkcypdnm6SLpK0TtIDubSpkn4l6fH0d0pKl6T/SPtwn6Sjcu85I+V/XNIZufSXS7o/vec/VMYnWUvaT9JvJD0k6UFJnxir+yOpSdKdkn6f9uUzKX2BpDvS9i+X1JDSG9PrZWn5/Ny6zk3pj0p6Yy59VH+TkoqS7pF0TRXsy8r0O7hX0pKUNuZ+Z2lbkyVdKekRSQ9LOqai+xIRVTMBReAJYCHZQ+J+D7yk0uVKZXsNcBTwQC7ti8A5af4c4F/T/JuAawEBRwN3pPSpwPL0d0qan5KW3ZnyKr33pDLuy2zgqDQ/EXgMeMlY3J+0/glpvh64I233h8A7U/o3gQ+l+Q8D30zz7wQuT/MvSb+3RmBB+h0WK/GbBD4J/A9wTXo9lvdlJTC9T9qY+52lbV0MvC/NNwCTK7kvZfvSKjEBxwDX516fC5xb6XLlyjOf3sH/UWB2mp8NPJrmvwWc3jcfcDrwrVz6t1LabOCRXHqvfKOwXz8B/nis7w8wHrgbeBXZFZV1fX9XwPXAMWm+LuVT399ad77R/k0C84AbgNcD16Syjcl9SdtYye7Bf8z9zoAWYAVpkM2+sC/V1uwzF1iVe706pe2rZkbEmjS/FpiZ5gfaj8HSV/eTXnapqeBIshrzmNyf1ExyL7AO+BVZ7XZjRHT0s/2eMqflm4Bp7Pk+lstXgLOBrvR6GmN3XwAC+KWkpZLOSmlj8Xe2AGgDvpea5L4jqZkK7ku1Bf8xK7LD9ZgadytpAvAj4K8j4vn8srG0PxHRGRFHkNWaXwkcUuEiDYukU4B1EbG00mUZQX8UEUcBJwEfkfSa/MIx9DurI2v2/c+IOBLYStbM02O096Xagv9TwH651/NS2r7qGUmzAdLfdSl9oP0YLH1eP+llI6meLPBfGhFXpeQxuz8AEbER+A1Z88ZkSXX9bL+nzGl5C7CBPd/HcjgOOE3SSuAysqafrzI29wWAiHgq/V0H/Jjs4DwWf2ergdURcUd6fSXZwaBy+1LO9rrRnsiOrsvJTrG6O6QOrXS5cuWbT+82/3+jd2fPF9P8yfTu7LkzpU8lazeckqYVwNS0rG9nz5vKuB8C/gv4Sp/0Mbc/QCswOc2PA24BTgGuoHcn6YfT/Efo3Un6wzR/KL07SZeTdZBW5DcJHM+uDt8xuS9AMzAxN38bcOJY/J2lbd0CLErz56f9qNi+lPUHWImJrJf8MbJ227+vdHly5foBsAZoJ6sFvJesffUG4HHg17kvUcDX0z7cDyzOredMYFma/iqXvhh4IL3nAvp0LI3wvvwR2enpfcC9aXrTWNwf4HDgnrQvDwCfTukL0z/TMrLg2ZjSm9LrZWn5wty6/j6V91FyIy0q8Zukd/Afk/uSyv37ND3Yvb2x+DtL2zoCWJJ+a1eTBe+K7Ytv72BmVoOqrc3fzMxK4OBvZlaDHPzNzGqQg7+ZWQ1y8Dczq0EO/rbPkDQt3b3xXklrJT2Ve90wxHsXS/qPErZx2wiV9XhJm9Kl+o9KujldYVvK+47dw22Nl3RpumPjA5J+m66uHrH9sdpTN3QWs9ERERvIxkIj6XxgS0R8qXu5pLrYdY+avu9dQjaGeqht7FHgHcItEXFKKtsRwNWSXoiIGwZ5z/HAFrILlkr1CeCZiDgsbWsR2fUiI70/VkNc87d9mqTvS/qmpDuAL0p6paTfpRr3bSkQdteou+9ff76y5yfcKGm5pI/n1rcll//G3P3VL+2+/7mkN6W0pem+6NcMVc6IuBf4LPDRtI5Tld0j/x5Jv5Y0M90E74PA36SzmVf3l6+f1c8md6l+RDwaETv67M9nc2dJT0n6Xkp/t7LnFdwr6VuSinv4FViVcvC3sWAecGxEfBJ4BHh1ZDfH+jTwhQHecwjwRrJ7wZyX7kXU15HAX5Pdv34hcJykJrLb5J4UES8nu/1Dqe5m103hfgscncp5GXB2RKwku73ClyPiiIi4pb98/az3IuDv0kHvc5IO7pshIj4d2c3pjgeeBS6Q9GLgHcBxaVkn8K492B+rYm72sbHgiojoTPMtwMUpAAbZA1j68/NUO94haR3ZrXJX98lzZ0SsBki3dJ5P1iSzPCJWpDw/AM6iNPknJ80DLk8362oguwdLf4bMFxH3SloI/AlwAnCXpGMi4uFeG8/OXC4B/j0ilkr6KPDylB+yexetwwzX/G1s2Jqb/yfgNxHxUuBUsvvT9GdHbr6T/is6peTZE0cC3QH5a8AFqZ3+A4OUs6R8EbElIq6KiA+TBfg39ZPtfLI7R34vvRZwcTrLOCIiFkXE+cPYL6tCDv421rSwq/37PWVY/6PAQu16nu07SnmTpMOBfyS7GRf0LucZuaybyR59yRD58us+True7dpA1kz1ZJ88p5KdFXw8l3wD8DZJM1KeqZIOKGV/rPo5+NtY80XgnyXdQxmaLSPiBbJn214naSlZsN40QPZXdw/1JAv6H8+N9DkfuCKtY33uPT8D3trd4TtIvrwDgZsk3U92B9IlZM9SyPsk2ZObujt3PxsRDwH/QPYkrPvInlI2u6QPwqqe7+pp1oekCRGxJbWhfx14PCK+XOlymY0k1/zNdvf+1AH8IFmzzLcqXB6zEeeav5lZDXLN38ysBjn4m5nVIAd/M7Ma5OBvZlaDHPzNzGrQ/weI4/IYFblwsAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(amount,valerr_2)\n",
    "plt.title('Error Rate vs Training Data size' )\n",
    "plt.xlabel('Training Data Size')\n",
    "plt.ylabel('Error rate in percantage')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**for comparision with 1/sqrt(n) expectation of error decrease with size**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 282
    },
    "id": "xGXiL9n-z5k3",
    "outputId": "adf80e83-9a54-4854-faf1-4712967dbf70"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f4fb66de490>]"
      ]
     },
     "execution_count": 114,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAD4CAYAAAAKA1qZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dfXTc1X3n8fdXMxo92ZZsWbZlS7JM7EBsQsDIJmlIWkIBkSZx0kAxSRNo6bppQptuTnfrbE9YwnZ7QnpOSFM4aWiAAFvWpt7SOImJ80Ba2jwYCzDYBgyKMZZsg2X5UZL1/N0/5koejcbS2JY8kn6f1zk6+s393fnNvYdBH997fw/m7oiISDTl5boBIiKSOwoBEZEIUwiIiESYQkBEJMIUAiIiERbPdQPOxOzZs722tjbXzRARmTRmz57N5s2bN7t7fab9kyoEamtraWhoyHUzREQmFTObfbp9mg4SEYkwhYCISIQpBEREIkwhICISYVmFgJnVm9kuM2s0s7UZ9heY2fqwf4uZ1abtrzGzNjP7i5SyPWa23cy2mZlWe0VEcmDUEDCzGHAfcD2wFLjZzJamVbsNOOLui4F7gLvT9n8NeDLD4a9y90vdve6MWy4iIucsm5HASqDR3Xe7ezewDliVVmcV8HDY3gBcbWYGYGYfBV4Hdo5Nk0VEZKxkEwILgKaU182hLGMdd+8FjgHlZjYN+EvgyxmO68CPzOxZM1tzug83szVm1mBmDS0tLVk0d7iHf7GH772w/6zeKyIylY33wvCdwD3u3pZh35XuvpzkNNPnzOz9mQ7g7ve7e52711VUVJxVIx7bspcfvHjgrN4rIjKVZXPF8D6gOuV1VSjLVKfZzOJAKdAKXAHcYGZfBcqAfjPrdPd73X0fgLsfNLMnSE47PX1OvTmNokSMjp6+8Ti0iMikls1IYCuwxMwWmVkCWA1sTKuzEbglbN8APOVJ73P3WnevBb4O/I2732tmJWY2HcDMSoBrgR1j0J+MivJjdHYrBERE0o06EnD3XjO7HdgMxIAH3X2nmd0FNLj7RuAB4FEzawQOkwyKkcwFnghrx3HgMXf/4Tn0Y0RFiRgtJ7rG6/AiIpNWVjeQc/dNwKa0sjtStjuBG0c5xp0p27uBd51JQ89FUX6Mk5oOEhEZJhJXDBclYpzUdJCIyDDRCAGNBEREMopGCGgkICKSUTRCIIwE3D3XTRERmVCiEQKJGACdPf05bomIyMQSjRDIT4aA1gVERIZSCIiIRFg0QiBMB53s7s1xS0REJpZohMDASKBbawIiIqmiEQIJTQeJiGQSiRAo1JqAiEhGkQiBYq0JiIhkFIkQ0NlBIiKZRSMEEloYFhHJJBIhoDUBEZHMsgoBM6s3s11m1mhmazPsLzCz9WH/FjOrTdtfY2ZtZvYX2R5zLGlNQEQks1FDwMxiwH0kHwi/FLjZzJamVbsNOOLui4F7gLvT9n8NePIMjzlm8mN5xPNMIwERkTTZjARWAo3uvtvdu4F1wKq0OquAh8P2BuBqC8+ONLOPAq8DO8/wmGOqKD+mNQERkTTZhMACoCnldXMoy1jH3XuBY0C5mU0D/hL48lkcEwAzW2NmDWbW0NLSkkVzMytMxDjZo+kgEZFU470wfCdwj7u3ne0B3P1+d69z97qKioqzbkixHiwjIjJMNg+a3wdUp7yuCmWZ6jSbWRwoBVqBK4AbzOyrQBnQb2adwLNZHHNM6RGTIiLDZRMCW4ElZraI5B/q1cAn0upsBG4BfgncADzlycd4vW+ggpndCbS5+70hKEY75pgqzI9xUg+VEREZYtQQcPdeM7sd2AzEgAfdfaeZ3QU0uPtG4AHgUTNrBA6T/KN+xsc8x76MKDkdpDUBEZFU2YwEcPdNwKa0sjtStjuBG0c5xp2jHXM8FeXHON7Zc74+TkRkUojEFcMQzg7SwrCIyBCRCYGi/JgeNC8ikiYyIVCciNGhNQERkSEiEwI6RVREZLjIhEBhmA7q7/dcN0VEZMKITAgMPFOgs1ejARGRAZEJgVO3k1YIiIgMiEwI6MEyIiLDRSYEBp4z3KkQEBEZFLkQ6NB0kIjIoMiEgNYERESGi0wIFCa0JiAiki4yITAwHaSRgIjIKZEJgWKNBEREholMCBTpFFERkWGyCgEzqzezXWbWaGZrM+wvMLP1Yf8WM6sN5SvNbFv4ecHMPpbynj1mtj3saxirDp1OoRaGRUSGGfWhMmYWA+4DrgGaga1mttHdX0qpdhtwxN0Xm9lq4G7gJmAHUBeeJFYJvGBm33P3gdt5XuXuh8ayQ6ejNQERkeGyGQmsBBrdfbe7dwPrgFVpdVYBD4ftDcDVZmbu3pHyB78QyNnd2/JjeeTHTNNBIiIpsgmBBUBTyuvmUJaxTvijfwwoBzCzK8xsJ7Ad+ExKKDjwIzN71szWnH0Xsleo20mLiAyR1TOGz4W7bwGWmdk7gIfN7MnwTOIr3X2fmc0Bfmxmr7j70+nvDwGxBqCmpuac2pJ8uphCQERkQDYjgX1AdcrrqlCWsY6ZxYFSoDW1gru/DLQBF4fX+8Lvg8ATJKedhnH3+929zt3rKioqsmju6RUlYrpthIhIimxCYCuwxMwWmVkCWA1sTKuzEbglbN8APOXuHt4TBzCzhcBFwB4zKzGz6aG8BLiW5CLyuCrK18PmRURSjTodFM7suR3YDMSAB919p5ndBTS4+0bgAeBRM2sEDpMMCoArgbVm1gP0A59190NmdgHwhJkNtOExd//hWHcuXVFCawIiIqmyWhNw903AprSyO1K2O4EbM7zvUeDRDOW7gXedaWPPlUYCIiJDReaKYdDD5kVE0kUrBDQdJCIyRLRCID9Gp6aDREQGRSsEEjE6NBIQERkUrRDQwrCIyBDRCoFEjK7efvr7c3YLIxGRCSVaIRDuJNrZq9GAiAhELQTCMwV06wgRkaRohYCeKSAiMkS0QiCMBHQnURGRpGiFQL6mg0REUkUqBMqKEwAcbu/OcUtERCaGSIXA/LJCAPYdPZnjloiITAyRCoE50wuJ5RkHjikEREQgYiEQyzPmTi/gwNHOXDdFRGRCiFQIAFSWFbFfIwERESDLEDCzejPbZWaNZrY2w/4CM1sf9m8xs9pQvtLMtoWfF8zsY9kec7zMLyviwDGNBEREIIsQMLMYcB9wPbAUuNnMlqZVuw044u6LgXuAu0P5DqDO3S8F6oFvmVk8y2OOi/mlhRw41om77h8kIpLNSGAl0Ojuu929G1gHrEqrswp4OGxvAK42M3P3DnfvDeWFwMBf3myOOS4qSwvp7u2nVaeJiohkFQILgKaU182hLGOd8Ef/GFAOYGZXmNlOYDvwmbA/m2MS3r/GzBrMrKGlpSWL5o6ssqwIQIvDIiKch4Vhd9/i7suAFcAXzazwDN9/v7vXuXtdRUXFObdnfmkyBLQ4LCKSXQjsA6pTXleFsox1zCwOlAKtqRXc/WWgDbg4y2OOi8pwwdgBXTAmIpJVCGwFlpjZIjNLAKuBjWl1NgK3hO0bgKfc3cN74gBmthC4CNiT5THHRXlJgkQ8T2cIiYgA8dEquHuvmd0ObAZiwIPuvtPM7gIa3H0j8ADwqJk1AodJ/lEHuBJYa2Y9QD/wWXc/BJDpmGPct4zMjMrSQvYrBERERg8BAHffBGxKK7sjZbsTuDHD+x4FHs32mOdLZWmhpoNERIjgFcOQXBzWdJCISERDoLKskDePd9KnB86LSMRFMwRKi+jrdw6e0GhARKItkiGwIFwwtl8XjIlIxEUyBAavFdAFYyIScdEMgVLdOkJEBCIaAjMK45QkYrp1hIhEXiRDwMyoLCvSSEBEIi+SIQDhgjGNBEQk4iIbAvNLi3TrCBGJvMiGQGVZIYfauuju7c91U0REciayITC/tAh3eOu4RgMiEl2RDYGBawX260ZyIhJh0Q2BgWsFtC4gIhEW2RCYH0YC+zQSEJEIyyoEzKzezHaZWaOZrc2wv8DM1of9W8ysNpRfY2bPmtn28PsDKe/5t3DMbeFnzlh1KhvFiThlxfk6TVREIm3Uh8qYWQy4D7gGaAa2mtlGd38ppdptwBF3X2xmq4G7gZuAQ8CH3X2/mV1M8kliC1Le90l3bxijvpyxylJdMCYi0ZbNSGAl0Ojuu929G1gHrEqrswp4OGxvAK42M3P35919fyjfCRSZWcFYNHwszNdjJkUk4rIJgQVAU8rrZob+a35IHXfvBY4B5Wl1Pg485+5dKWUPhamgL5mZZfpwM1tjZg1m1tDS0pJFc7NXWaarhkUk2s7LwrCZLSM5RfTHKcWfdPd3Au8LP5/K9F53v9/d69y9rqKiYkzbVVlaxNGOHk52943pcUVEJotsQmAfUJ3yuiqUZaxjZnGgFGgNr6uAJ4BPu/uvB97g7vvC7xPAYySnnc6rgTOEdDdREYmqbEJgK7DEzBaZWQJYDWxMq7MRuCVs3wA85e5uZmXAD4C17v7zgcpmFjez2WE7H/gQsOPcunLm9FwBEYm6UUMgzPHfTvLMnpeBx919p5ndZWYfCdUeAMrNrBH4AjBwGuntwGLgjrRTQQuAzWb2IrCN5EjiH8eyY9mYH0JAIwERiapRTxEFcPdNwKa0sjtStjuBGzO876+Bvz7NYS/PvpnjY25p8kQljQREJKoie8UwQEE8xuxpBTpDSEQiK9IhAMnFYV0rICJRFfkQqCwt5IDuHyQiERX5EJhfVsT+oydx91w3RUTkvFMIlBbR3t3H8c7eXDdFROS8i3wIDDxcRovDIhJFCoGBawW0LiAiERT5EFgydxrxPGPL64dz3RQRkfMu8iEwozCf31g8m8073tTisIhETuRDAKB+2Tz2tHaw660TuW6KiMh5pRAArl02FzN4cvubuW6KiMh5pRAAZk8rYEXtLDbvVAiISLQoBIL6ZfN45c0TvH6oPddNERE5bxQCwXUXzwPQaEBEIkUhECwoK+KSqlJ+uEMhICLRkVUImFm9me0ys0YzW5thf4GZrQ/7t5hZbSi/xsyeNbPt4fcHUt5zeShvNLNvnO5B8+dT/cXz2NZ0VFcPi0hkjBoCZhYD7gOuB5YCN5vZ0rRqtwFH3H0xcA/Jh8oDHAI+HB4ofwvwaMp7vgn8F2BJ+Kk/h36MifplYUpIowERiYhsRgIrgUZ33+3u3cA6YFVanVXAw2F7A3C1mZm7P+/u+0P5TqAojBoqgRnu/itPXqH1CPDRc+7NObqgYhpvnzuNH2pdQEQiIpsQWAA0pbxuDmUZ64RnEh8DytPqfBx4zt27Qv3mUY4JgJmtMbMGM2toaWnJornnpn7ZPJ55/TCtbV3j/lkiIrl2XhaGzWwZySmiPz7T97r7/e5e5+51FRUVY9+4NNddPI9+h5+8/Na4f5aISK5lEwL7gOqU11WhLGMdM4sDpUBreF0FPAF82t1/nVK/apRj5sTSyhnUzCrmSa0LiEgEZBMCW4ElZrbIzBLAamBjWp2NJBd+AW4AnnJ3N7My4AfAWnf/+UBldz8AHDezd4ezgj4NfPcc+zImzIz6i+fx88ZDHO/syXVzRETG1aghEOb4bwc2Ay8Dj7v7TjO7y8w+Eqo9AJSbWSPwBWDgNNLbgcXAHWa2LfzMCfs+C3wbaAR+DTw5Vp06V9ctm0dPn/OzVw7muikiIuPKJtPtk+vq6ryhoWHcP6e/33nPV37K8pqZfPP3Lx/3zxMRGU9m9qy712XapyuGM8jLM65bNo9/29XCye6+XDdHRGTcKAROo37ZPE729PHvr47/aakiIrmiEDiNlYtmUVacrxvKiciUphA4jXgsj2veMZefvPwW3b39uW6OiMi4UAiMoP7ieZzo7OWXu1tz3RQRkXGhEBjBexfPZlpBXLeXFpEpSyEwgsL8GFddNIcfv/Qmff2T51RaEZFsKQRGUb9sHofaumnYczjXTRERGXMKgVH81oUVFMTzdHtpEZmSFAKjKCmI8/63V7B5x5tMpqurRUSyoRDIQv2yeew/1sn2fcdy3RQRkTGlEMjC1e+YQzzPdHtpEZlyFAJZKCtO8J63lfNDTQmJyBSjEMjSdcvm8fqhdl472JbrpoiIjBmFQJauXTYXM3hsy95cN0VEZMwoBLI0Z3ohN6+s4Tu/2MOjv3oj180RERkTWYWAmdWb2S4zazSztRn2F5jZ+rB/i5nVhvJyM/uZmbWZ2b1p7/m3cMz0J45NWHd9ZBlXXzSHO767g03bD+S6OSIi52zUEDCzGHAfcD2wFLjZzJamVbsNOOLui4F7gLtDeSfwJeAvTnP4T7r7peFnwj/LMR7L495PLOfympn8+bpt/KLxUK6bJCJyTrIZCawEGt19t7t3A+uAVWl1VgEPh+0NwNVmZu7e7u7/STIMpoSiRIxv31JH7exi1jz6LDt07YCITGLZhMACoCnldXMoy1gnPJj+GFCexbEfClNBXzIzy1TBzNaYWYOZNbS0TIynfJUVJ3j4D1cyozDOrQ9t5Y3W9lw3SUTkrORyYfiT7v5O4H3h51OZKrn7/e5e5+51FRUV57WBI6ksLeKR266gr7+fTz3wDC0nunLdJBGRM5ZNCOwDqlNeV4WyjHXMLA6UAiM+icXd94XfJ4DHSE47TSqL50zjwVtX0HKii1sfeoYTnT25bpKIyBnJJgS2AkvMbJGZJYDVwMa0OhuBW8L2DcBTPsKltWYWN7PZYTsf+BCw40wbPxFcVjOTb/7+cna9eYI1jzxLV29frpskIpK1UUMgzPHfDmwGXgYed/edZnaXmX0kVHsAKDezRuALwOBppGa2B/gacKuZNYcziwqAzWb2IrCN5EjiH8euW+fXb104h7+98RJ+ubuV/7p+mx5AIyKTRjybSu6+CdiUVnZHynYncONp3lt7msNenl0TJ4ePXVZFa1s3f/2Dl5lVsoP/tepiTrPWLSIyYWQVApKdP3rfBbS0dfGtf99NxbRCPv/bS3LdJBGRESkExtja+os4dKKbe37yKuXTEvz+uxfmukkiIqelEBhjZsZXPv5OjnR086Xv7qC8JMH176zMdbNERDLSDeTGQX4sj/s+sZzLqsv4/Lpt/PLXI54tKyKSMwqBcVKUiPHgrStYWF7Mmkca2Llft5cQkYlHITCOyooTPHLbSqYXxrnlwa3sbe3IdZNERIZQCIyz5O0lVtLb38+nHtyi20uIyISiEDgPFs+ZzoO3ruCt4538wXd0ewkRmTgUAufJ8pqZfPOTl/PygRN85v/o9hIiMjEoBM6jqy6aw1c/fgk/b2zlC4+/oNtLiEjO6TqB8+zjl1fR2t7F32x6hfKSBF/+yDLdXkJEckYhkANr3v82DrV1c//TuyktyudPP7CERFyDMhE5/xQCObK2/iIOtXXx90818k9b9vK7ly3gphXVLJk7PddNE5EIsRFu+z/h1NXVeUNDQ66bMWb6+52nX2vh8YYmfvzSW/T0Octryli9oobfuaSSkgJltIicOzN71t3rMu5TCEwMrW1dPPH8PtZtbaLxYBsliRgfftd8blpRzaXVZVo3EJGzds4hYGb1wN8BMeDb7v6VtP0FwCMknxHQCtzk7nvMrBzYAKwAvuPut6e853LgO0ARyWcVfH6kp5HB1A6BAe7Oc3uPsH5rE9974QAne/p4+9xp3LSiho9dtoBZJYlcN1FEJplzCgEziwGvAtcAzSQfN3mzu7+UUuezwCXu/hkzWw18zN1vMrMS4DLgYuDitBB4BvgzYAvJEPiGuz85UluiEAKpTnT28P0XD7B+axPbmo6SiOVxzbK5rF5RzXvfNpu8PI0ORGR0I4VANpPOK4FGd98dDrYOWAW8lFJnFXBn2N4A3Gtm5u7twH+a2eK0BlUCM9z9V+H1I8BHgRFDIGqmF+Zz88oabl5ZwytvHmf91iaeeH4fP3jxAFUzi7jx8mpurKtifllRrpsqIpNUNuclLgCaUl43h7KMdcIziY8B5aMcs3mUYwJgZmvMrMHMGlpaWrJo7tR00bwZ/M8PL2PL/7iav7/5MmrLS7jnJ6/y3ruf4taHnuHJ7Qfo7u3PdTNFZJKZ8KefuPv9wP2QnA7KcXNyriCeXDD+8Lvm03S4g39uaOLxhmb+5J+eo7wkwe8uT55quniOTjUVkdFlEwL7gOqU11WhLFOdZjOLA6UkF4hHOmbVKMeUUVTPKuYL117I53/77Tz9Wgvrn2nioZ/v4R//43XqFs7kphXV/M4llRQnJnzWi0iOZPPXYSuwxMwWkfxDvRr4RFqdjcAtwC+BG4CnRjrTx90PmNlxM3s3yYXhTwN/fxbtFyCWZ1x14RyuunAOLSe6eOL5ZtZtbeK/bXiRL3/vpcFTTd9VVapTTUVkiGxPEf0g8HWSp4g+6O7/28zuAhrcfaOZFQKPkjwT6DCwOmUheQ8wA0gAR4Fr3f0lM6vj1CmiTwJ/qlNEx4670/BG8lTTH7yYPNX0onnTuWlFNR+9dAEzdaqpSGToYrGIO9HZw/deOMD6rXt5ofkYiVge1108j9UrqnnPBeU61VRkilMIyKCX9h/n8YbkqabHTvZQPauI37u8mhvqqqgs1ammIlORQkCG6ezpY/PON3m8oYmfN7aSZ/Cbb6/gphU1XP2OOeTHdFdTkalCISAj2tvaweMNTfzzs028dbyL2dMSfHx5Fb+3opq3VUzLdfNE5BwpBCQrvX39PP1aC+ueaeKpVw7S2++sqJ3Jh981n7dVTKNmVjHzy4qIaQ1BZFJRCMgZO3iik395bh/rtzbx+qH2wfL8mFE9s5ia8mIWziqmpryE2vJiFpYXUzWzmML8WA5bLSKZKATkrLk7+4918kZrO3tbO9jT2sHew+280drB3tYOTnT1DtY1g3kzCllYXszCWSXJoCgvprY8uT2jMD+HPRGJrnO9gZxEmJmxoKyIBWVF/Mbbhu5zdw63d/PG4WQgvNHawRshIH76ykEOtXUNqT+zOJ+a8hIWziqmtjw5ilgYRhQV0wt0IZtIDigE5KyZGeXTCiifVsDympnD9rd39bL3cAdvtLaHgEiGxXN7j/D9F/fTnzIILcqPsbC8mJpZydHD4DTTrBLmlxUS19lKIuNCISDjpqQgzjsqZ/COyhnD9nX39rPv6MlTARGmmV4/1M6/v9pCV8odUeN5xoKZRSwMo4iBsKidXULNLK1DiJwLhYDkRCKex6LZJSyaXTJsX3+/89aJzsF1hzcOtyfXIlo72Lb3CMc7e4fUnzujYHANYnCaKYRFWbFujyEyEoWATDh5eUZlaRGVpUW8+4Lhj6U42tHNntaOwcXqgWmmp19tYcOJoesQpUX5Q6aZFs4K6xDlJcyZXqBbZkjkKQRk0ikrTnBpcYJLq8uG7TvZ3Ze2DpH8vX3fMZ7c8SZ9KQsRBfG8EA4DwVA8+LpqZpGumpZIUAjIlFKUiHHhvOlcOG/4Q3V6+vrZf/Tk4CL1G4faB0cR/9nYQmfPqXWIWJ4xv6xw6DTTrFNhoWc0yFShb7JERn4sL/yrf/g6hLtz8ERXWKRuZ+/hcE1Eazubth/gaEfPkPoV0wvCxXLJKaba2adGETOL83W6q0waCgERkqe7zp1RyNwZhaxcNGvY/mMne8LFcu1Dppt+0djKvxwf+lC86QXxMHooSbmyuph5MwqZVhCnpCBOcSKmoJAJIasQMLN64O9IPlTm2+7+lbT9BcAjwOUkHyt5k7vvCfu+CNwG9AF/5u6bQ/ke4EQo7z3d1WwiE0FpUT7vrCrlnVWlw/Z19vTRdDh5muupkOjgpQPH2bzzTXr7h1+VbwYliXgIhdhgOJQUnCorKYgzLRFnWmFqeZxpYd+p98dJxLV+IWdn1BAwsxhwH3AN0AxsNbON7v5SSrXbgCPuvtjMVgN3AzeZ2VKSj6NcBswHfmJmb3f3vvC+q9z90Bj2R+S8K8yPsWTudJbMHb4O0dvXz4FjydNdW9o6aevqo72rl/auXtrC7/auPk6E7cPtHUPKu/v6M3zicIl43qnwSKQGxqlAmT4saIYG0OAoJT+ms6YiJJuRwEqgMeVxkeuAVUBqCKwC7gzbG4B7LTnWXQWsc/cu4HUzawzH++XYNF9kYovH8qieVUz1rOKzen93b/+pwOjupa1zIDz6hgRJW/ep4GjrStY70tFN05GOwfL27l6yuVXYwChlcDQSRh2po5BphclRykiBolHK5JBNCCwAmlJeNwNXnK6Ou/ea2TGgPJT/Ku29C8K2Az8yMwe+5e73Z/pwM1sDrAGoqanJorkiU0cinkcinhiTZ0L39zsne1LDo48TXT3DAiW53TcYLm2dybLmIx20d58Kmu7eLEcpsbyhgZIyrZU+ChkMmtTRTKFGKeMplwvDV7r7PjObA/zYzF5x96fTK4VwuB+SdxE9340UmSry8mzwD+2cMTheT1//kEBp6+o5FR4ZAmWgvK2rl2Md3ew7kjKiyXKUAlCSSA+UGNMK8k+NUtIDJW10kvq+grhuOZJNCOwDqlNeV4WyTHWazSwOlJJcID7te9194PdBM3uC5DTRsBAQkYkpP5ZHWXFiTG7N4Z4cpQwGSmdKkHSnTHtlCJT2rl72HT05pLwry1FKfszSFtljTCvMHxyNZFyQz7i+kqw/GUcp2YTAVmCJmS0i+Qd8NfCJtDobgVtIzvXfADzl7m5mG4HHzOxrJBeGlwDPmFkJkOfuJ8L2tcBdY9IjEZl0zIziRDx5Ed7w9fUz1tPXT0dX3+BayYnOXoYtyHf3pYRLmPbq7uXYyR72h1AZ2J/hBK+MilNGKac98ytxqnzImV+JoQv5BfG883Ia8aghEOb4bwc2kzxF9EF332lmdwEN7r4ReAB4NCz8HiYZFIR6j5NcRO4FPufufWY2F3gidDAOPObuPxyH/olIBOXH8igtzqO0+NwfZOTudPb0Dw2MIYGSNgXWnRyxtHUm11v2H+0MaynJOqlXpo8kHqbvBgLlXz/3XooSYz99pSeLiYicR719/bR3n5rWGjg9eMiCfFpZW1cP3/zk5Wc93aQni4mITBDxWB6lRXmUFk2Mx63qBF4RkQhTCIiIRJhCQEQkwhQCIiIRphAQEYkwhYCISIQpBEREIkwhICISYZPqimEzawHeOIO3zAam0kNrplJ/1JeJSR4BysIAAATBSURBVH2ZmM6lL4cA3L0+085JFQJnyswaptJjK6dSf9SXiUl9mZjGsy+aDhIRiTCFgIhIhE31EMj4yMpJbCr1R32ZmNSXiWnc+jKl1wRERGRkU30kICIiI1AIiIhE2JQNATOrN7NdZtZoZmtz3Z4BZvagmR00sx0pZbPM7Mdm9lr4PTOUm5l9I/ThRTNbnvKeW0L918zslpTyy81se3jPN2wcH1JqZtVm9jMze8nMdprZ5ydrf8ys0MyeMbMXQl++HMoXmdmW8PnrzSwRygvC68awvzblWF8M5bvM7LqU8vP6nTSzmJk9b2bfn8x9MbM94TuwzcwaQtmk+46Fzyozsw1m9oqZvWxm78l5X9x9yv2QfBbyr4ELgATwArA01+0KbXs/sBzYkVL2VWBt2F4L3B22Pwg8CRjwbmBLKJ8F7A6/Z4btmWHfM6GuhfdeP459qQSWh+3pwKvA0snYn3D8aWE7H9gSPvdxYHUo/wfgT8L2Z4F/CNurgfVhe2n4vhUAi8L3MJaL7yTwBeAx4Pvh9aTsC7AHmJ1WNum+Y+GzHgb+KGwngLJc92XcvoC5/AHeA2xOef1F4Iu5bldKe2oZGgK7gMqwXQnsCtvfAm5OrwfcDHwrpfxboawSeCWlfEi989Cv7wLXTPb+AMXAc8AVJK+2jKd/r4DNwHvCdjzUs/Tv2kC98/2dBKqAnwIfAL4f2jZZ+7KH4SEw6b5jQCnwOuGEnInSl6k6HbQAaEp53RzKJqq57n4gbL8JzA3bp+vHSOXNGcrHXZhCuIzkv6AnZX/C9Mk24CDwY5L/2j3q7r0ZPn+wzWH/MaCcM+/jePk68N+B/vC6nMnbFwd+ZGbPmtmaUDYZv2OLgBbgoTBN920zKyHHfZmqITBpeTLCJ9V5u2Y2Dfh/wJ+7+/HUfZOpP+7e5+6XkvxX9Ergohw36ayY2YeAg+7+bK7bMkaudPflwPXA58zs/ak7J9F3LE5yKvib7n4Z0E5y+mdQLvoyVUNgH1Cd8roqlE1Ub5lZJUD4fTCUn64fI5VXZSgfN2aWTzIA/snd/yUUT9r+ALj7UeBnJKc9yswsnuHzB9sc9pcCrZx5H8fDe4GPmNkeYB3JKaG/Y3L2BXffF34fBJ4gGdCT8TvWDDS7+5bwegPJUMhtX8ZrHi+XPyQTdzfJ4dfAwtWyXLcrpX21DF0T+FuGLgx9NWz/DkMXhp4J5bNIzi3ODD+vA7PCvvSFoQ+OYz8MeAT4elr5pOsPUAGUhe0i4D+ADwH/zNDF1M+G7c8xdDH18bC9jKGLqbtJLqTm5DsJ/BanFoYnXV+AEmB6yvYvgPrJ+B0Ln/UfwIVh+87Qj5z2ZVy/gLn8Ibmy/irJed2/ynV7Utr1f4EDQA/JfxncRnL+9afAa8BPUv6DGnBf6MN2oC7lOH8INIafP0gprwN2hPfcS9oi1Bj35UqSQ9cXgW3h54OTsT/AJcDzoS87gDtC+QXhf6xGkn9EC0J5YXjdGPZfkHKsvwrt3UXK2Rm5+E4yNAQmXV9Cm18IPzsHPmsyfsfCZ10KNITv2b+S/COe077othEiIhE2VdcEREQkCwoBEZEIUwiIiESYQkBEJMIUAiIiEaYQEBGJMIWAiEiE/X/ePnOXYCh8yQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(amount,1/np.array(amount)**0.5)          #for comparision with 1/sqrt(n) expectation of error decrease with size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Below is the same plot but in log sclae now**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 312
    },
    "id": "Oers0Ejq0AES",
    "outputId": "d45cc637-b258-4de6-9a5c-ef2f8796d9b3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Error rate in percantage, log')"
      ]
     },
     "execution_count": 130,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5gUVfb/8fdnEkNWchiimFAyklV2XXcFUQwoijkHUHd1f66631XXDW5wcUFUzMjqmhAlmHUVQYIMqOQkEoacc+b8/qgabceZoSf09Mz0eT1PP1NddbvqVHVPn7636t6SmeGccy5xJcU7AOecc/HlicA55xKcJwLnnEtwngiccy7BeSJwzrkE54nAOecSnCcCV65Iek/SVcVdtjyRtFNS83jHkRtJV0uaFO84oiFpmaRfxDuO4uCJoIjCD8Oe8J8r+zGshGP4TNLecNsbJY2WVD/K1/aUlBXrGI8QQ+SxO5zjeF5WkHWZWS8ze7G4yxZEeEwPR+xDlqTXJZ1SgHU8KOml4o4NwMyqmNnSWKzblU2eCIrHOeE/V/ZjUG6FJKXkMi+5IBvKp/wgM6sCtACqAI8UZL3xFHnsgBX8+Hi+nF0ut+NXiq0O96cq0AVYAEyUdEZ8w3LupzwRxFBYzf1C0qOSNgEPShoh6UlJ70raBfxM0onhr/qtkuZKOjdiHT8pn982zWwr8DbQNmId10iaL2mHpKWSbgrnVwbeAxpE/HptIClJ0j2SvpW0Kfw1WyOPfZwvqU/E8xRJGyS1l5Qu6aVwHVslTZdUtwDHr2f4a/p3ktYCL0g6WtL4cBtbwumMiNd8Jun6iOM/SdIjYdnvJPUqZNlmkj4Pj+HHkh6P5he7BbLM7H7gWeDvEescImmlpO2SZkg6NZx/FnAf0D98T74J5+f6PuZx7FpImiBpW1hLfC1imYXLI9/3nZJ2S7KIcteG29si6QNJTfLZ3rnhZ3dreFxPjFi2TNJvJc0K43lNUvqRjl342m7h52Zb+LdbxLKo3xNJtcLPylZJmyVNlJQULmukoBa9IfysDgvnHyPpf+G8jZJelnRUHuuP+n+mNPJEEHudgaVAXeAv4bwB4XRVYBowDvgQqAPcBrws6fiIdUSWz7f9VFJN4AJgScTs9UAfoBpwDfCopPZmtgvoRfjrNXysDmM4DzgdaABsAR7PY5OvAJdGPP8VsNHMZgJXAdWBRkBN4GZgT37x56IeUANoAtxI8Jl9IXzeOFxffk1xnYGFQC3gH8BzklSIsv8Fvgz340HgigLuB8BooL2CBAwwnSBh1wjX/4akdDN7H/gr8Fr4nrQJy+f6PuaxrT8RfKaOBjKAx3IWMLPVOWpjbwGvAkjqS5CMLgBqAxMJ3uufkHRcuOzXYdl3gXGS0iKKXQycBTQDWgNX53WQItZbA3gHGEpw3AcD74SfcSjYe3IXkBXGVzfcN1NQwx4PLAeaAg2zjwEg4GGC/4ETCT7HD+ax/oL8z5Q+ZuaPIjyAZcBOYGvE44Zw2dXAihzlRwAjI56fCqwFkiLmvQI8mFv5PGL4DNgNbAMM+BponE/5t4E7wumeQFaO5fOBMyKe1wcOACm5rKsFsAOoFD5/Gbg/nL4WmAy0LuDx/EVEbPuB9HzKtwW25DgW10cc/yURyyqFx6deQcoSJJyD2fsYLn8JeCmPmH5yTMP5J4TrbJjH67YAbcLpB/Naf27vYy7LRgJPAxm5LDOgRY55vwNmABXD5+8B10UsTwo/Y01yWd8fgNdzlF0F9Ix4Ty+PWP4PYHgecV8NTAqnrwC+zLF8SlimoO/JQ8CYXPa7K7Aht892Lus4D/gqj89q1P8zpfHhNYLicZ6ZHRXxeCZi2cpcykfOawCsNLPDEfOWE/wyyW8dOd1uZtUJfm1l/woEQFIvSVPDKvFWoDfBr968NAHeCqvRWwk+5IcIfkn9iJktCZefI6kScC7BLzWA/wAfAK9KWi3pH5JSo9iXSBvMbG/EvlSS9JSk5ZK2A58DRynvcydrI2LdHU5WKWDZBsDmiHkQ3XuSU0OCL+GtAGFzyfyw2WMrQe0pz/elgO/j3QS/aL8Mm2yuzW+9wB0En+PsGlsTYEjEZ2BzuL6GuayiAcFnFoDws7wyR9m1EdO7yfs9yHO9oez/jYK+J/8kqCV/GDar3RPObwQsN7ODOV8gqa6kVyWtCj9rL5H38Y76f6Y08kQQe7kN7xo5bzXQKLu9MtSY4BdVfuvIfWNms4E/A48rUAF4k+DkcV0zO4qg6p7d5JHbulcCvXIkt3QzW5VLWfiheagvMC9MDpjZATP7o5m1BLoRNGtcGe2+5BHfXcDxQGczqwacFs7Pq7mnOKwBaoSJLlujQqznfGCmme1ScD7gboImk6PD92UbebwvUbyPP2Jma83sBjNrANwEPCGpRc5yYRPki8DFZhb5RboSuCnHZ6CimU3OZXOrCb4Is9cpguOT1+clWj9abyj7f6NA74mZ7TCzu8ysOcGPlTsVnLhfCTRW7hci/JXgfWgVftYuJ+/PWUH/Z0oVTwTxN43gF9LdklIl9QTO4Yd2ysJ4keCXyLlAGlCBoPp7MPz198uIsuuAmpKqR8wbDvwl++SgpNphm3FeXg3XeQs/1AaQ9DNJrcJf69sJqsqHc19F1KoSnBfYGrYhP1DE9R2RmS0HMglO9qdJ6krwHh1RmIwbSnoAuJ6gbRqC/ThI2Cwh6X6Ctv9s64CmET8QjvQ+5tzuRfrhJPoWgi+0wznKVCNoLvm9meU89zQcuFfSSWHZ6pIuymNzrwNnSzojrPHdBewjaBYsineB4yQNUHARQn+gJTC+oO+JpD4KTpCLIOEeIjgeXxIklb9JqqzgAofu4cuqEjT7bpPUEPh/+cRa0P+ZUsUTQfEYpx9fffFWtC80s/0EH+BewEbgCeBKM1tQ2GDCdQ4B/mBmO4DbCf5ZtxCceB4bUXYBwS/6pWG1tkH42rEE1egdwFSCE6l5bW8NQdttN+C1iEX1gFEESWA+MIGguago/g1UJDhWU4H3i7i+aF1G0J68iaDG9RrBl11eGkjaSfBFMh1oRdBm/mG4/AOC2BcRNHfs5cdNG2+EfzdJmnmk9zEXpwDTwhjGEpxLyNl3oD1B7erRyM8vgJm9RXCF06ths8gcgs/oT5jZQoJfy48RvC/nEFwCvD+f+I7IzDYR1CLvIjjudwN9zGxjWKQg78mxwMcE78cU4Akz+9TMDoXxtiC4dDkL6B++5o8Ex2gbwUnr0fmEW6D/mdJG4YkN51wBKLgcc4GZxbxG4qLj70nheY3AuShIOkXBdeVJCq7z70tw1Y6LE39Pik9Z6qnpXDzVI2gaqEnQfHCLmX0V35ASnr8nxcSbhpxzLsF505BzziW4Mtc0VKtWLWvatGm8w3DOuTJlxowZG82sdm7LylwiaNq0KZmZmfEOwznnyhRJOXtpf8+bhpxzLsF5InDOuQTnicA55xKcJwLnnEtwngiccy7BeSJwzrkE54nAOecSXMIkgs279vPQuHns3v+TGxE551xCS5hE8MWSjbww+TvOf3wy323cFe9wnHOu1IhZIpDUSNKnkuaF90y9I5cyPcP7tX4dPu6PVTzntGnAi9d0Yv2OvZz72CQ+mrcuVptyzrkyJZY1goPAXeH9arsAAyW1zKXcRDNrGz4eimE8nHZcbcbd1oNmtStzw8hMHvlgIYcO++irzrnEFrNEYGZrzGxmOL2D4FaFDWO1vWhlHF2J12/qyqWdGjHs0yVc/cKXbN5VpDvqOedcmVYi5wgkNQXaEdyoPaeukr6R9F72jbJzef2NkjIlZW7YsKHI8aSnJvPwBa35+4WtmPbdZs55bBLfrNxa5PU651xZFPNEIKkK8CbwazPbnmPxTKCJmbUhuPF1rreZM7OnzayjmXWsXTvXUVQLpf8pjXnz5m4AXDR8Cq9+uaLY1u2cc2VFTBOBpFSCJPCymY3OudzMtpvZznD6XSBVUq1YxpRTq4zqjL+tB52b1+Ce0bP53ahZ7D1wqCRDcM65uIrlVUMCngPmm9ngPMrUC8shqVMYz6ZYxZSXoyunMeKaTtz28xa8lrmSi4ZPYeXm3SUdhnPOxUUsawTdgSuAn0dcHtpb0s2Sbg7L9APmSPoGGApcYnG6iXJykrjrl8fz7JUdWbZpF+cMm8SERUU/H+Gcc6Vdmbt5fceOHS3WdyhbtnEXN780g4XrdnDnL45j4M9akJSkmG7TOediSdIMM+uY27KE6VlcEE1rVeatW7vTt00D/vXRIm4Ymcm2PQfiHZZzzsWEJ4I8VExL5tH+bXmo70lMWLSBc4dNYv6anBc9Oedc2eeJIB+SuLJrU167qQt7Dxzi/Ce+4K2vsuIdlnPOFStPBFHo0KQG427rQZuMo/jNa99w/5g57D94ON5hOedcsfBEEKU6VdN5+frO3HBqM0ZOWc4lT09h7ba98Q7LOeeKzBNBAaQkJ/H7s1vy+ID2LFi7gz6PTWTKtyXe7cE554qVJ4JCOLt1fcYO6k61iqlc/tw0nvl8KWXtMlznnMvmiaCQWtSpypiB3flly7r85d35DPzvTHbu87ufOefKHk8ERVA1PZUnLmvPfb1P4P05a+k7bBJL1u+Id1jOOVcgngiKSBI3nnYML13fma27D9B32Be8O3tNvMNyzrmoeSIoJt2OqcX423twXL2q3PryTP767nwOHvJLTJ1zpZ8ngmJUv3pFXr2xC1d0acLTny/l8uemsWHHvniH5Zxz+fJEUMwqpCTzp/NO5l8XteGrFVs557FJzFi+Jd5hOedcnjwRxMiFHTIYfWs30lKSuOTpKYycsswvMXXOlUqeCGLopAbVGTeoB6ceW5v7x8zlzte/Yc9+v/uZc6508UQQY9UrpfLslR2588zjePvrVZz/xBcs37Qr3mE559z3PBGUgKQkcfsZx/LC1aewZtte+jw2iU/mr4t3WM45B3giKFE9j6/D+Nt60LhGJa57MZPBHy7k0GE/b+Cciy9PBCWsUY1KvHlLNy7qkMHQ/y3hmhHT2bJrf7zDcs4lME8EcZCemsw/+rXmr+e3Yuq3m+jz2CRmZ22Ld1jOuQTliSBOJDGgc2Nev7krZsaFwyfz+vSV8Q7LOZeAUo5UQNKduczeBswws6+LP6TE0rbRUYy7rQe3v/oVd785i69WbuGBc04iPTU53qE55xJENDWCjsDNQMPwcRNwFvCMpLtjGFvCqFmlAiOv7cytPY/hlS9XcvFTU8jasjveYTnnEkQ0iSADaG9md5nZXUAHoA5wGnB1DGNLKMlJ4u6zTuCpKzrw3YZdnPPYJCYu3hDvsJxzCSCaRFAHiBw57QBQ18z25JjvisGvTqrHmEHdqV21Alc9/yVDPl7sl5g652IqmkTwMjBN0gOSHgS+AP4rqTIwL5bBJarmtavw1q3d6du2IY9+vIgrnpvG+h174x2Wc66cUjQDoUnqCHQPn35hZpkxjSofHTt2tMzMuG2+RJkZb2Rmcf/YOVSpkMrQS9rSrUWteIflnCuDJM0ws465LYv28tEDwGHgUDgdzUYbSfpU0jxJcyXdkUsZSRoqaYmkWZLaRxlPQpDExac0YszAHlSvmMJlz01j8EeLvKnIOVesjpgIwi/wl4FaBOcLXpJ0WxTrPgjcZWYtgS7AQEktc5TpBRwbPm4EnixA7Anj+HpVGTuoB+e3a8jQTxZz+bPTWL/dm4qcc8UjmhrBdUBnM3vAzO4n+FK/4UgvMrM1ZjYznN4BzCe4/DRSX2CkBaYCR0mqX6A9SBCVK6Qw+OK2/LNfa75auYXeQycyafHGeIflnCsHokkEImgSynYonBc1SU2BdsC0HIsaApHdabP4abJA0o2SMiVlbtiQ2JdUXtSxEWMH9eDoSmlc8fw0H7jOOVdk0SSCFwiuGnowvGpoKvBctBuQVAV4E/i1mW0vTJBm9rSZdTSzjrVr1y7MKsqV4+pWZcyg7vRrHwxcN+CZqazzpiLnXCEdMRGY2WDgGmBz+LjGzP4dzcolpRIkgZfNbHQuRVYBjSKeZ4Tz3BFUSkvhnxe14ZGL2jAraxu9h0zk80WJXVtyzhVOnolAUo3sB7AMeCl8LA/n5UuSCGoO88NkkpuxwJXh1UNdgG1mtqagO5HI+nXIYOyg7tSsksZVL3zJIx8s5OChw/EOyzlXhuQ36NwMwPjhfEB2Q7TC6eZHWHd34ApgtqTswenuAxoDmNlw4F2gN7AE2E1Q83AFdGzdqowZ2IMHx85l2KdL+PK7zQy9tB31qqfHOzTnXBkQVYey0iSROpQVxltfZfH7t+aQnprM4Ivb0PP4OvEOyTlXChRHhzJXRpzfLoOxg3pQp2oFrn5hOn9/f4E3FTnn8uWJoBxqUacKbw/szqWdGvHkZ99y6TNTWbNtT7zDcs6VUp4Iyqn01GQevqA1Qy5py7zV2+k9ZCKfLlwf77Ccc6WQJ4Jyrm/bhoy9rQd1q6VzzQvT+dt7CzjgTUXOuQgFTgSS5oePQbEIyBW/Y2oHTUUDOjdm+IRvueTpqaze6k1FzrlAgROBmZ0I9AC+K/5wXKykpybz1/NbMfTSdixYs53eQyfyyfx18Q7LOVcKRJUIJDWR9ItwuiKw38zeiWlkLibObdOA8befSv3qFbnuxUwefne+NxU5l+CiGYb6BmAU8FQ4KwN4O5ZBudhqVqsyb93ajcu7NOapz5fS/6kprPKmIucSVjQ1goEEvYS3A5jZYoL7ErgyLD01mT+f14phA9qxaN1Oeg+ZyMfzvKnIuUQUTSLYZ2b7s59ISuGH4SZcGdendQPG39aDjKMrcv3ITP48fh77D3pTkXOJJJpEMEHSfUBFSWcCbwDjYhuWK0lNa1XmzVu6cWXXJjw76TsufmoKWVt2xzss51wJiSYR3ANsAGYDNxEMFPd/sQzKlbz01GQe6nsyT1zWnm/XB01FH85dG++wnHMlwAedcz+xfNMuBv33K2av2sa13ZtxT68TSEvxvofOlWX5DTqX3zDU2S+ezU/PCWwDMoE/m9mmoofoSpMmNSsz6pauPPzuAp7/4jtmLN/MsAHtaVSjUrxDc87FQDQ/894D3gEuCx/jCJLAWmBEzCJzcVUhJZkHzz2J4Ze3Z+nGXfQeOpH353hTkXPl0RFrBMAvzKx9xPPZkmaaWXtJl8cqMFc6nHVyfVrWr85tr8zk5pdmcE33ptzb60RvKnKuHInmvzlZUqfsJ5JOAZLDpwdjEpUrVRrXrMQbN3fj2u7NeOGLZfQbPpkVm/yqIufKi2gSwfXAc5K+k7SM4D7EN0iqDDwcy+Bc6ZGWksT957TkqSs6sGzjLs4eOpH3ZvvtpZ0rD6K+akhSdQAz2xbTiI7ArxqKv5WbdzPola/4ZuVWrurahPvOPpEKKclHfqFzLm6KdNVQuIKzgZOAdCm4l72ZPVRsEboypVGNSrxxU1f+/v4Cnpv0HTNXbGXYgHY0qVk53qE55wohmkHnhgP9gdsAARcBTWIclyvl0lKS+EOfljxzZUdWbN5Nn6GTeNebipwrk6I5R9DNzK4EtpjZH4GuwHGxDcuVFWe2rMs7t/fgmDpVuPXlmdw/Zg77Dh6Kd1jOuQKIJhFkj0+8W1ID4ABQP3YhubIm4+hKvHFzV244tRkjpyznwicns3zTrniH5ZyLUjSJYLyko4B/AjOBZcArsQzKlT2pyUn8/uywqWhT0FTkVxU5VzYc8aohSRXMbF/2NJAO7M2eV9L8qqHSL/Kqoqu7NeXe3if4VUXOxVl+Vw1FUyOYkj1hZvvCy0en5FPeJbjsq4qu7d6MEZOXcfHwKazc7B3QnCut8kwEkupJ6kBwH4J2ktqHj56Ajz7m8pXdAW345R1YGnZA+8CHtXauVMqvH8GvgKsJ7lE8OGL+DuC+I61Y0vNAH2C9mZ2cy/KewBjgu3DWaO+bUP6cdXI9TmpQjYH/nclN/5nBdT2a8buzfFhr50qTaM4RXGhmbxZ4xdJpwE5gZD6J4Ldm1qcg6/VzBGXTvoOHePjdBYyYvIy2jY5i2IB2ZBztFUvnSkpRexaPlzQAaBpZ/ki/3s3sc0lNow/TlWfZw1p3blaDu0fNoveQifzr4rac2bJuvENzLuFFUz8fA/QlGGl0V8SjOHSV9I2k9ySdlFchSTdKypSUuWHDhmLatIuHXq3qM/72HjSuWYkbRmbyl3fmceDQ4XiH5VxCi6ZpaE5uTTtRrTyoEYzPo2moGnDYzHZK6g0MMbNjj7RObxoqH/YdPMRf3pnPyCnLadf4KIYNaE/DoyrGOyznyq2iXj46WVKrYo4JM9tuZjvD6XeBVEm1ins7rnSqkJLMQ31PZtiAdixet5Ozh07kfwvWxTss5xJSNImgBzBD0kJJsyTNljSrqBsOL09VON0pjMXvf5xg+rRuwLjbetCgekWuHZHJw+/N96Yi50pYNCeLexVmxZJeAXoCtSRlAQ8AqQBmNhzoB9wi6SDBeEaXWLQ3R3DlSrNalRl9azf+NH4eT01YyoxlW3hsQDvqV/emIudKQkFuTFOHYHgJAMxsRayCyo+fIyjfxny9ivtGzyYtJYnB/dvys+PrxDsk58qFIp0jkHSupMUEHb8mEAw6916xRuhcqG/bhoy9rQd1q6VzzQvT+cf7CzjoTUXOxVQ05wj+BHQBFplZM+AMYGpMo3IJ7ZjaVXh7YHcu7dSIJz77lgHPTGPttr3xDsu5ciuaRHDAzDYBSZKSzOxTINfqhXPFJT01mYcvaM2/+7dlzupt9B46kQmLvA+Jc7EQTSLYKqkK8DnwsqQhFF+HMufydV67howd1IPaVSpw9Qtf8sgHC72pyLliFk0i6AvsBn4DvA98C5wTy6Cci9SiTtBUdHGHRgz7dAmXPTuN9du9qci54hJNIqgDpJnZQTN7EXgGqBrbsJz7sYppyfy9X2v+dVEbZmUFTUWTFm+Md1jOlQvRJII3gMi6+KFwnnMl7sIOGYwd1J2jK6VxxfPTGPzRIg4d9u4nzhVFNIkgxcz2Zz8Jp9NiF5Jz+Tu2blXGDOrOBe0yGPrJYq54bhrrd3hTkXOFFU0i2CDp3OwnkvoCXid3cVUpLYV/XdyGf/ZrzcwVW+g9ZBKTl/jH0rnCiCYR3AzcJ2mFpBXA74AbYxuWc9G5qGMjxgzsQfWKKVz23DT+/bE3FTlXUPkmAknJwC1m1gVoCbQ0s25m9m2JROdcFI6vV5Wxg3pwXtuG/PvjxVz1/Jds2LEv3mE5V2bkmwjM7BDB6KOY2c7sYaOdK20qV0hh8MVt+PuFrZi+bDO9h05kyrc+mK1z0YimaegrSWMlXSHpguxHzCNzroAk0f+Uxrw9sDtVK6Rw2bNTeeyTxRz2piLn8hVNIkgnuE/Azwk6kp0DFOiG886VpBPrV2PsbT3o07oB//poEVe98CWbdnpTkXN5iXoY6tLCh6F20TIzXvlyJQ+Om8vRlVJ57NL2dGpWI95hORcXRR2G+jhJn0iaEz5vLen/ijtI54qbJAZ0bszbt3anUloKlz4zlcc/XeJNRc7lEE3T0DPAvcABADObBVwSy6CcK04tG1Rj7KDu9Dq5Hv/8YCHXjJjO5l37j/xC5xJENImgkpl9mWPewVgE41ysVE1P5bFL2/Gn805myreb6D1kIpnLNsc7LOdKhWgSwUZJxwAGIKkfsCamUTkXA5K4oksTRt/ajQqpSfR/eiqPfLCQvQcOxTs05+IqmkQwEHgKOEHSKuDXBL2NnSuTTm5YnXG39aBv2wYM+3QJvYdMZOpS73PgEldBbl5fGUgysx2xDSl/ftWQK06fL9rA79+ezcrNe7jklEbc2+tEqldKjXdYzhW7ol41VFPSUGAi8JmkIZJqFneQzsXDacfV5oNfn8aNpzXn9cyVnDF4Au/MWkNZu6zauaKIpmnoVWADcCHQL5x+LZZBOVeSKqWlcF/vExk7qAf1qldg4H9ncsPITFZv3RPv0JwrEUdsGpI0x8xOzjFvtpm1imlkefCmIRdLBw8d5oUvlvGvjxaSLHH3WSdweZcmJCcp3qE5VyRFahoCPpR0iaSk8HEx8EHxhuhc6ZCSnMQNpzXno9+cTvsmR/PA2Ln0Gz6ZhWvjemrMuZiKpkawA6hMcItKgGRgVzhtZlYtduH9lNcIXEkxM97+ehUPjZvHzn0Hufn0Yxj4sxakpybHOzTnCqxINQIzq2pmSWaWGj6SwnlV80sCkp6XtD57aIpclkvSUElLJM2S1D76XXIu9iRxfrsMPrmrJ+e0bsBj/1tC76ETmeaXmrpyJpqmocIaAZyVz/JewLHh40bgyRjG4lyh1aicxuD+bRl5bScOHDpM/6encu/oWWzbcyDeoTlXLGKWCMzscyC/Pvx9gZEWmAocJal+rOJxrqgiLzV9bfpKfjF4Au/O9ktNXdkXyxrBkTQEVkY8zwrnOVdqZV9qOmZgD+pUrcCtL8/khpEzWLPNLzV1ZVdUiUBSsqQGkhpnP2IdWI7t3ygpU1Lmhg0bSnLTzuWqVUZ1xgzszr29TmDSkg2cOfhzRk5Z5kNcuzIpmp7FtwHrgI+Ad8LH+GLY9iqgUcTzjHDeT5jZ02bW0cw61q5duxg27VzRpSQncdPpx/Dhr0+nbaOjuH9McKnponV+qakrW6KpEdwBHG9mJ5lZq/DRuhi2PRa4Mrx6qAuwzcx8VFNX5jSuWYn/XNeJwRe34buNuzh76EQGf+ijmrqyIyWKMiuBbQVdsaRXgJ5ALUlZwANAKoCZDQfeBXoDS4DdwDUF3YZzpYUkLmifwenH1ebP78xn6P+WMH72Gv52QWu/PaYr9aLpUPYccDxBk9D3dwA3s8GxDS133qHMlQUTFm3g92/NJmvLHi7t1Jh7ep1A9Yo+qqmLn6IOMbGC4PxAGlA14uGcy8Ppx9Xmw9+cxg2nNuO16Ss4c/AE3vNLTV0pFfX9CEoLrxG4smZ21jbuGT2Luau3c2bLuvyp78nUq54e77BcgsmvRpBnIpD0bzP7taRxhLepjGRm5xZvmNHxRODKooOHDvPcpO949ONFpCQl8buzjueyzk1I8lFNXQkpbCLoYGYzJJ2e23Izm1CMMUbNE4Ery5Zv2sXv35rDpCUb6dDkaP52QSuOrXkBczUAABTqSURBVOstrS72CpUISitPBK6sMzNGz1zFn96Zx659B7mlZwsG/uwYKqT4qKYudop6stg5V4wkcWGHDD6583TOblWfoZ8spveQiUxflt/QXM7FjicC5+KkZpUK/PuSdoy45hT2HjjMRcOncN9bs9m+10c1dSUr6kQgqVIsA3EuUfU8vg4f3Xka1/doxqtfruAX/5rA+3O8k70rOdGMNdRN0jxgQfi8jaQnYh6ZcwmkUloK/9enJW8P7E6tKhW4+aWZ3Dgyk7Xb9sY7NJcAoqkRPAr8CtgEYGbfAKfFMijnElXrjKMYM6g79/Q6gQmLNnDm4An8Z+pyH9XUxVRUTUNmtjLHLB9Ny7kYSU1O4ubTj+HD35xG60bV+cPbc7j4qSks9lFNXYxEkwhWSuoGmKRUSb8F5sc4LucSXpOalXnpus48clEblmzYSe+hE7n9la/4cO5aH9nUFatoRh+9GRhCcPewVcCHwK2xDMo5F5BEvw4Z9Dy+NkM+Xsz4WasZ+81qqlZI4cyWdenTpj49WtQmLcUvAHSFF83oo93N7IsjzSsp3qHMJbIDhw4z+dtNvDNrNe/PWcv2vQeplp7Cr06qx9mt69O9RS1Skz0puJ8qUs9iSTPNrP2R5pUUTwTOBfYfPMwXSzYybtZqPpq7jh37DnJUpVTOOqkefVo3oEvzGqR4UnCh/BJBnk1DkroC3YDaku6MWFQN8L7wzsVZWkoSPzuhDj87oQ77Dh7i80UbeWfWasZ9s5pXp6+kZuU0zjo5qCl0blaTZB/gzuUhv3MEaUCVsEzkqFjbgX6xDMo5VzAVUpI5s2VdzmxZl70HDvHZwg2Mn7Wa0TNX8fK0FdSqUoHerYKaQscmR/uop+5HomkaamJmy0soniPypiHnordn/yH+t2A978xezf8WrGfvgcPUrVaB3q3q06d1fdo18qSQKIp6jqA2cDdwEvD93TTM7OfFGWS0PBE4Vzi79h3kkwXrGf/Naj5btIH9Bw/ToHp6kBTaNKBNRnUkTwrlVVETwYfAa8BvCS4lvQrYYGa/K+5Ao+GJwLmi27H3AB/PX8c7s9YwYdEGDhwyMo6uyNmt63NO6wac1KCaJ4VypqiJYIaZdZA0y8xah/Omm9kpMYj1iDwROFe8tu05wEfz1jF+1momLd7IwcNG05qVOLt1fc5u1YAT61f1pFAOFOqqoQjZY+KukXQ2sBqoUVzBOefiq3rFVPp1yKBfhwy27NrPh/PWMn7WGoZPWMrjn35L89qV6RM2Hx3nd1Mrl6KpEfQBJgKNgMcILh/9o5mNjX14P+U1AudKxqad+3h/7lrembWGqUs3cdjg2DpV6NO6AWe3rk+LOlXiHaIrgEI3DUlKBm43s0djFVxBeSJwruSt37GXD+asZdysNUxfthkzOKFeVfq0rk+f1g1oWqtyvEN0R1DUcwRfmlmnmERWCJ4InIuvddv38u7sNbwzaw2Zy7cAcFKDakFNoVV9Gtf0e1iVRkVNBI8CqQRXDu3Knm9mM4szyGh5InCu9Fi9dQ/vzl7D+Flr+HrlVgDaZFQPTjS3bkDDoyrGOUKXraiJ4NNcZpv3I3DORVq5eXdQU5i9hllZ2wC4ultT/tCnpQ9vUQoUKREUccNnEQxhnQw8a2Z/y7H8auCfBMNbAwwzs2fzW6cnAudKv+WbdvHsxO/4z9Tl/Oqkugy5pB3pqT5EWTzllwhiNjRheKL5caAX0BK4VFLLXIq+ZmZtw0e+ScA5VzY0qVmZP513Mvf3acmH89Zx+bPT2Lp7f7zDcnmI5Ri1nYAlZrbUzPYDrwJ9Y7g951wpc22PZgy7tD2zsrZx4ZOTydqyO94huVzkmwgkJYW3qSyMhkDkvY6zwnk5XShplqRRkhoVclvOuVLq7Nb1GXldJzbs2Mf5T0xm7upt8Q7J5ZBvIjCzwwTNO7EyDmgaDl3xEfBiboUk3SgpU1Lmhg0bYhiOcy4WujSvyahbupGaJPo/NZWJi/3/uDSJpmnoE0kXquCDjawi6I2cLYMfTgoDYGabzGxf+PRZoENuKzKzp82so5l1rF27dgHDcM6VBsfVrcroW7uTcXRFrnlhOqNnZsU7JBeKJhHcBLwB7Je0XdIOSdujeN104FhJzSSlAZcAPxqWQlL9iKfnAvOjjNs5VwbVq57O6zd35ZSmNbjz9W944rMlxPLKRRedIyYCM6tqZklmlmpm1cLn1aJ43UFgEPABwRf862Y2V9JDks4Ni90uaa6kb4DbgasLvyvOubKgWnoqI649hXPbNOAf7y/kgbFzOXTYk0E8RdWPIPziPi18+pmZjY9pVPnwfgTOlQ+HDxt/f38BT32+1PsalIAi9SOQ9DfgDmBe+LhD0sPFG6JzLtEkJYl7e5/4fV+Dy56dxpZd3tcgHqI5R9AbONPMnjez54GzgLNjG5ZzLlFc26MZjw9oz+xV27hw+GRWbva+BiUt2g5lR0VMV49FIM65xNW7VX3+c20nNu7YxwVPTmbOKu9rUJKiSQR/Bb6SNELSi8AM4C+xDcs5l2g6/6ivwRTva1CCjtizGDgMdAFGA28CXc3stRKIzTmXYLL7GjSqUcn7GpSgaHoW321ma8xsbPhYW0KxOecSUHZfg07Ngr4Gj3/qfQ1iLZqmoY8l/VZSI0k1sh8xj8w5l7Cqpacy4ppO9G3bgH9+sJD7x3hfg1hKiaJM//DvwIh5BjQv/nCccy6QlpLEoxe3pV61dJ76fCnrtu9l6KXe1yAWojlHcI+ZNcvx8CTgnIu57L4GD5zTko/me1+DWInmHMH/K6FYnHMuV9d0b8YT3tcgZvwcgXOuTOjVqj4vXdeZTTv3c/4T3tegOEWTCPoTnB/4nKAPwQzAB/txzpW4Ts1q8OYtXamQkkT/p6bw+SLva1Acohl9NOf5AT9H4JyLmxZ1qjL61m40qlGJa0dM580Z3tegqPJMBJLujpi+KMeyv8YyKOecy0/dakFfg87Na3DXG97XoKjyqxFcEjF9b45lZ8UgFueci1q19FReuLoT54V9Df4wZo73NSik/PoRKI/p3J4751yJS0tJYvDFbalbPZ2nJixl/fZ93tegEPKrEVge07k9d865uEhKEvf2OpEHw74GA56Zymbva1Ag+SWCNtn3KAZah9PZz1uVUHzOOReVq8O+BnNWb6ffk97XoCDyTARmlhxxj+KUcDr7eWpJBumcc9Ho1ao+L1/fmU27vK9BQUR7YxrnnCsTTmn6474GE7yvwRF5InDOlTvZfQ0a16zMdSOmM8r7GuTLE4FzrlyqWy2d12/qQufmNfit9zXIlycC51y5VTVHX4P/e9v7GuQmmvsROOdcmZXd16Be9YoMn/At63fsY+gl7aiY5n0NsnmNwDlX7iUliXt6ncAfzz2Jj+evY8Cz3tcgkicC51zCuKpbU568rD3zVm/nwicns2KT9zUATwTOuQRz1slBX4PNu/ZzwZNfMDvL+xrENBFIOkvSQklLJN2Ty/IKkl4Ll0+T1DSW8TjnHEDH7/saJNP/6Sl8tnB9vEOKq5glAknJwONAL6AlcKmkljmKXQdsMbMWwKPA32MVj3PORWpRpypv3dqNpjUrc/2LmbyRuTLeIcVNLGsEnYAlZrbUzPYDrwJ9c5TpC7wYTo8CzpDkI5s650pEnWrpvHZTF7o0r8n/GzWLO179islLNnI4wS4xjeXlow2ByBSbBXTOq4yZHZS0DagJbIxhXM45972q6ak8f/Up/PODBbw6fSVjvl5Nw6MqcmGHDC5s35AmNSvHO8SYKxP9CCTdCNwI0Lhx4zhH45wrb9JSkvj92S2565fH88HctYyakcVj/1vM0E8W06lZDfp1yKB3q/pUqVAmvjILTLHqci2pK/Cgmf0qfH4vgJk9HFHmg7DMFEkpwFqgtuUTVMeOHS0zMzMmMTvnXLY12/YweuYq3pyRxdKNu6iYmkyvVvXo1yGDLs1qkpRUtlqxJc0ws465LothIkgBFgFnAKuA6cAAM5sbUWYg0MrMbpZ0CXCBmV2c33o9ETjnSpKZMXPFVkbNyGL8N6vZse/g901H/dpn0LhmpXiHGJW4JIJww72BfwPJwPNm9hdJDwGZZjZWUjrwH6AdsBm4xMyW5rdOTwTOuXjZs/8QH84Lmo4mLdmIGXSOaDqqXIqbjuKWCGLBE4FzrjRYvXUPb321ilEzsvhu4y4qpSXT6+T69OuQQedmNUpd05EnAueci5Gg6WgLo2ZkMe6bNezcd5CMoytyYfsM+nXIoFGN0tF05InAOedKQGluOvJE4JxzJWzV1j28NTOLUTOyWLZpN5XSkundKmg66tS05JuOPBE451ycmBkzlgdNR+NnBU1HjWoETUcXti+5piNPBM45Vwrs2X/o+w5rX3wbNB11aV6Dfh0a0evkejFtOvJE4JxzpUxeTUcXdcjglBg0HXkicM65UsrMyFy+hVGZWbwzO2g6alyjEhe2z+CC9g2LrenIE4FzzpUBu/cf/L7paPK3mzCDrs1r0q9DBr1a1aNSWuGbjjwROOdcGZO1ZTdvzVzFqJlZLN+0m8ppyfzmzOO4/tTmhVpffomg9PaHds65BJZxdCVuO+NYBv28BdOXbWHUjJXUr14xJtvyROCcc6WYJDo1q0GnZjVitg2/eb1zziU4TwTOOZfgPBE451yC80TgnHMJzhOBc84lOE8EzjmX4DwROOdcgvNE4JxzCa7MDTEhaQOwHKgFbIxzOPGSyPsOib3/ibzv4PtflP1vYma1c1tQ5hJBNkmZeY2bUd4l8r5DYu9/Iu87+P7Hav+9acg55xKcJwLnnEtwZTkRPB3vAOIokfcdEnv/E3nfwfc/JvtfZs8ROOecKx5luUbgnHOuGHgicM65BFfmEoGkoySNkrRA0nxJXeMdU0mRdLykryMe2yX9Ot5xlRRJv5E0V9IcSa9ISo93TCVJ0h3hvs9NhPdd0vOS1kuaEzGvhqSPJC0O/x4dzxhjJY99vyh87w9LKtZLSMtcIgCGAO+b2QlAG2B+nOMpMWa20MzamllboAOwG3grzmGVCEkNgduBjmZ2MpAMXBLfqEqOpJOBG4BOBJ/7PpJaxDeqmBsBnJVj3j3AJ2Z2LPBJ+Lw8GsFP930OcAHweXFvrEwlAknVgdOA5wDMbL+ZbY1vVHFzBvCtmS2PdyAlKAWoKCkFqASsjnM8JelEYJqZ7Tazg8AEgi+FcsvMPgc255jdF3gxnH4ROK9Egyohue27mc03s4Wx2F6ZSgRAM2AD8IKkryQ9K6lyvIOKk0uAV+IdREkxs1XAI8AKYA2wzcw+jG9UJWoOcKqkmpIqAb2BRnGOKR7qmtmacHotUDeewZQXZS0RpADtgSfNrB2wi/JbNcyTpDTgXOCNeMdSUsK24L4EPwYaAJUlXR7fqEqOmc0H/g58CLwPfA0cimtQcWbBte9+/XsxKGuJIAvIMrNp4fNRBIkh0fQCZprZungHUoJ+AXxnZhvM7AAwGugW55hKlJk9Z2YdzOw0YAuwKN4xxcE6SfUBwr/r4xxPuVCmEoGZrQVWSjo+nHUGMC+OIcXLpSRQs1BoBdBFUiVJInjvE+ZCAQBJdcK/jQnOD/w3vhHFxVjgqnD6KmBMHGMpN8pcz2JJbYFngTRgKXCNmW2Jb1QlJzwnsgJobmbb4h1PSZL0R6A/cBD4CrjezPbFN6qSI2kiUBM4ANxpZp/EOaSYkvQK0JNg6OV1wAPA28DrQGOC4egvNrOcJ5TLvDz2fTPwGFAb2Ap8bWa/KpbtlbVE4JxzrniVqaYh55xzxc8TgXPOJThPBM45l+A8ETjnXILzROCccwnOE4ErUeEQCdmjp66VtCriedoRXttR0tAotjG5mGLtKWlbOJzJQkmfS+oT5esK1Nkt7B/xsqTZ4QijkyRVCZcVy/5EbGuEpH7FuU5XtqXEOwCXWMxsE9AWQNKDwE4zeyR7uaSUcFC13F6bCWRGsY3i7HE80cz6hLG1Bd6WtOcI1/D3BHYCBfkCvwNYZ2atwm0dT9BfoLj3x7mf8BqBi7vwF+pwSdOAf0jqJGlK+Et8cnZP8vCX9vhw+sFwzPbPJC2VdHvE+nZGlP8s4v4VL4e9kpHUO5w3Q9LQ7PXmx8y+Bh4CBoXrOEfStDDOjyXVldQUuBn4TVjLOTW3crmsvj6wKmJbC7M7y0Xsz0MRtadVkl4I518u6ctw/lOSkgtw7M8I45odHs8KhT0+ruzyROBKiwygm5ndCSwATg0HFrwf+GserzkB+BXBGP0PSErNpUw74NdAS6A50F3BDW2eAnqZWQeCnprRmhluF2AS0CWM81XgbjNbBgwHHg3vHTExt3K5rPd54HdhAvyzpGNzFjCz+8N7UfQk6GU6TNKJBL2tu4fLDgGXRbMj4XEYAfQPayIpwC1FPD6uDPKmIVdavGFm2aNpVgdeDL8MDcjtCx7gnfBX8z5J6wmGJM7KUeZLM8sCkPQ10JSg2WapmX0XlnkFuDHKOBUxnQG8Fg5+lgZ8l/tLjlzOzL6W1Bz4JcEAe9MldQ1HHf1h40GN5iVgsJnNkDSI4CZF08PKTkWiH4jteIKB/LIHr3sRGAh8RuGPjyuDvEbgSotdEdN/Aj4N70R2DpDXLSkjxxk6RO4/bKIpUxDt+GGwu8eAYeGv6ZvyiTOqcma208xGm9mtBF/2vXMp9iDBCLwvhM8FvJh95zozO97MHizEfrkE5onAlUbV+aG9/OoYrH8h0Dxsz4egaeWIJLUG/gA8Hs6KjPOqiKI7gKoRz/MqF7nu7grvvxtePdWSYFC1yDLnENQWbo+Y/QnQTz+MTFpDUpNweqSkTvns0kKgqX645eUVBHc+K9TxcWWXJwJXGv0DeFjSV8Sg+dLM9gC3Au9LmkHwxZ3XSK6nZl8+SpAAbo+4YuhB4I1wHRsjXjMOOD/7ZHE+5SIdA0yQNJtgZNVM4M0cZe4EGgLZJ4YfMrN5wP8BH0qaBXxEcOIZoDX53M7TzPYC14SxzQYOA8MLeHxcOeCjj7qEJKmKme0M29wfBxab2aPxjqu4SKoGPGdmFxXy9eX6+Lgf8xqBS1Q3hCeP5xI03TwV53iKlZltL2wSCJXr4+N+zGsEzjmX4LxG4JxzCc4TgXPOJThPBM45l+A8ETjnXILzROCccwnu/wPzg/+PnLe97QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#This is the same plot but in log sclae now\n",
    "plt.plot(np.log(amount),np.log(valerr_2))\n",
    "plt.title('Error Rate vs Training Data size on log scale' )\n",
    "plt.xlabel('Training Data Size, log')\n",
    "plt.ylabel('Error rate in percantage, log')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 312
    },
    "id": "FMzXKwVf5YtE",
    "outputId": "734f0ab2-241b-419f-d077-a51acaa6a7dc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'Training Data Size')"
      ]
     },
     "execution_count": 131,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEWCAYAAACNJFuYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZwcdZ3/8denj5nJ9MzkmiEZcpCJhnAE5IgaBDWIKIeKu+u6wKIgIt7itSyou3jrz2vV1V3lEhTEkxVBQQ4FPLgSiBASwpEDck8CJJOZZM7P74+qnnTm7Jk+qrvzfj4e/Zjq6uqqz/RUz6e+R32/5u6IiIhkikUdgIiIlB4lBxERGUTJQUREBlFyEBGRQZQcRERkECUHEREZRMlBREQGUXIoE2a21sy6zKxxwPpHzMzNbI6ZXWNmXxzm/W5m7Wa2y8w2mNm3zCxenOhFhmdmZ5vZkvDc3GRmt5rZCWb22fC8fXvGton0+R4+vyZ8/oqMbV5qZrqBK0dKDuVlDXBW+omZHQHUjuH9L3P3OuAk4GzgPfkNT2RszOzjwLeBLwPTgNnA/wBnhJs8D3xulAuZ54EhL4pk/JQcystPgHdmPD8X+PFYd+LuTwB/BhbkKS6RMTOzicDngQ+6+43u3u7u3e5+s7v/W7jZbUAXcM4Iu7oWONLMXlvgkPcrSg7l5X6gwcwODa+kzgSuG+tOzOww4NXAI3mOT2QsjgNqgP8bYRsH/gO4zMySw2zTQVDy+FJ+w9u/KTmUn3Tp4WRgJbBhDO992MxeAG4GrgR+lP/wRLI2Fdjm7j0jbeTuvwVagQtG2OyHwGwzOzWP8e3XElEHIGP2E+BeoIWxVykd4+5P5z8kkXHZDjSaWWK0BAF8huBi5idDvejunWb2BeALBCVqyZFKDmXG3dcRNEyfBtwYcTgiubgP6ATeOtqG7n4H8DTwgRE2+xEwCfjHvES3n1PJoTy9G5js7u1mNvBvGDezmoznfe7eVcTYRLLi7jvM7D+B75tZD3A70A28HjiRoC0h06eBm0bYX4+ZXQZ8t0Ah71dUcihD7v6Muy8Z5uVLgN0Zjz8WLTCRMXL3bwIfJ6g2agWeAz4E/GaIbf8KPDjKLm8ANuU5zP2SabIfEREZSCUHEREZRMlBREQGUXIQEZFBlBxERGSQsujK2tjY6HPmzIk6DKlQS5cu3ebuTVEcW+e2FFIu53ZZJIc5c+awZMlwPTdFcmNm66I6ts5tKaRczm1VK4mIyCBKDiIiMoiSg4iIDKLkICIigyg5iIjIIEoOIiIyiJKDiIgMUtbJ4Yp7V3Pb8s1RhyGSV+u2t/Ot21ex4cXdUYci+7GyTg7X/G0td67cEnUYInm1/oXdfPePT7NRyUEiVNbJIRE3unv7og5DJK/iMQOgp1dzrUh0yjo5JOMxJQepOOnk0Nun5CDRKevkkIgZ3bq6kgrTX3Lo04WPRKesk0NVQiUHqTyJMDn0aQpfiVBZJ4dEzFQvKxVHbQ5SCso6OSTjMbpUcpAKozYHKQVlnRyqEjF6lBykwqSrlXpVrSQRKuvkoAZpqUTxWPC1VMlBolSw5GBmV5vZVjNbPmD9h83sCTN73My+lssx1JVVKlHc1OYg0StkyeEa4JTMFWZ2InAG8DJ3Pxz4Ri4HUHKQKAx14WNmXw8veh41s/8zs0nj3X88rjYHiV7BkoO73ws8P2D1+4GvuntnuM3WXI6RjBs9+gJJ8V3DgAsf4A5ggbsfCTwJXDrenavNQUpBsdscDgZebWYPmNk9Zvby4TY0swvNbImZLWltbR1ym0Q8RnePSg5SXENd+Lj77e7eEz69H5g53v3H0tVKuvCRCBU7OSSAKcAi4N+AX5iF34QB3P1yd1/o7gubmpqG3FnQlVVfICk55wO3DvfiaBc+/SUHVZlKhIqdHNYDN3rgQaAPaBzvzoJqJX2BpHSY2aeBHuD64bYZ7cKnv81B1z0SoWInh98AJwKY2cFAFbBtvDtLqlpJSoiZnQe8CfhX9/E3GPSXHHThIxFKFGrHZnYDsBhoNLP1wGXA1cDVYS+PLuDcXL5EyXiMbtXLSgkws1OAi4HXuntHLvtSm4OUgoIlB3c/a5iXzsnXMZKaz0EiMMyFz6VANXBH2Ix2v7u/bzz739vmoOQg0SlYciiGZDyGe9AfPD0ejUihDXPhc1W+9h9XV1YpAeU9fEbYcKfSg1QSMyNmuglOolXWyaEqHoSv5CCVJhGLqc1BIlXWySFdN6vB96TSxGNGn5KDRKisk0MyEYSvYbul0sRjGhpGolXeySEc2lgT/kilicdMbQ4SqfJODglVK0llSsR0979Eq6yTQyKmaiWpTEHJIeooZH9W1skhGVe1klSmIDnovJbolHVyqEpoxiypTGqQlqiVdXJIVyvpPgepNAl1ZZWIlXVySPbfBKcvkVSWmEoOErEyTw4aPkMqU0JdWSViZZ4cwt5KariTChPX8BkSsbJODumB97p69CWSyhKPoTYHiVRZJwcNvCeVSiUHiVpZJ4eEqpWkQqnNQaJW1smhv0Fa1UpSYTS2kkStYMnBzK42s63hfNEDX/uEmbmZNeZyjP6urCo5SIWJm5KDRKuQJYdrgFMGrjSzWcAbgGdzPUB/cuhRcpDKkohr4D2JVsGSg7vfCzw/xEv/BVwM5HxZlK5WUsOdVBpVK0nUitrmYGZnABvc/e9ZbHuhmS0xsyWtra1DbqOB96RSxc3odSUHiU7RkoOZ1QKfAv4zm+3d/XJ3X+juC5uamobcpv8mOA2fIRUmHjOd1xKprJKDmR1kZq8PlyeYWf04jvUSoAX4u5mtBWYCD5vZ9HHsCwi+QGa6z0EqTyKuaiWJVmK0DczsPcCFwBSCf/AzgR8AJ43lQO7+GHBAxn7XAgvdfdtY9jNQMh7TwHtScWKqVpKIZVNy+CBwPLATwN2fIuOf/HDM7AbgPmC+ma03s3fnEuhwkjFTyUGKaqhu2mY2xczuMLOnwp+TczmGboKTqGWTHDrdvSv9xMwSZNHTyN3Pcvdmd0+6+0x3v2rA63NyLTUAJBMxJQcptmsY3E37EuAud58H3BU+H7d4LKY2B4lUNsnhHjP7FDDBzE4GfgncXNiwspeIqVpJimuYbtpnANeGy9cCb83lGCo5SNSySQ6XAK3AY8B7gd8DnylkUGNRFVe1kpSEae6+KVzeDEwbbsNsumnHYmpzkGiN2iDt7n3AFeGj5CTiMXqUHKSEuLub2bD/2d39cuBygIULFw65nUoOErVseis9xuA2hh3AEuCL7r69EIFlKxk3VStJKdhiZs3uvsnMmoGtuewsuM9BFz0SnWyqlW4Ffgf8a/i4mSAxbCZomItU0JVVXyKJ3G+Bc8Plc4GbctlZPGao4CBRGrXkALze3Y/JeP6YmT3s7seY2TmFCixbSg5SbGE37cVAo5mtBy4Dvgr8IuyyvQ54ey7HSMQ08J5EK5vkEDezV7j7gwBm9nIgHr7WU7DIspSMmwbek6Jy97OGeWlMN4aORAPvSdSySQ4XAFebWR1gBDfDXWBmKeArhQwuG4l4jC4N2S0VJh7TRY9EK5veSg8BR5jZxPD5joyXf1GowLJVFY/R0RV5AUYkr+Ixwx36+pxYzKIOR/ZD2ZQcMLPTgcOBGrPgRHX3zxcwrqzVJONs29UZdRgieZUIE0KvOzGUHKT4Ru2tZGY/AP4F+DBBtdI/AwcVOK6sNdQk2NWpkoNUlngs+Gqq3UGikk1X1le5+zuBF9z9c8BxwMGFDSt7dUoOUoHCqUqUHCQy2SSH3eHPDjM7EOgGmgsX0tjU1yRo29ODa6gBqSDpkoMapSUq2bQ53GJmk4CvAw8T3C19ZUGjGoO66iS9fc6e7j4mVMVHf4NIGehvc1BykIhkkxy+5u6dwK/N7BagBthT2LCyV18T/Apte7qVHKRixJQcJGLZVCvdl15w986wK+t9I2xfVP3JQe0OUkFUcpCoDVtyCOd2nkEwj8PR0N+frgGoLUJsWdlbclBykMoRD5ODhtCQqIxUrfRG4DyCOaO/lbG+DfhUAWMak7rqJAC7lBykgsRNJQeJ1rDJwd2vBa41s39y91+PdcdmdjXwJmCruy8I130deDPQBTwDvMvdXxxX5KHMNgeRSpGIKzlItLJpc7jFzM42s0+Z2X+mH1m87xoGz7N7B7DA3Y8EngQuHVu4g9VVq81BKk9cbQ4SsWySw00E8+P2AO0ZjxENNc+uu9/u7un/4vcTVFnlpKEmqFZSm4NUkkR/m4OSg0Qjm66sM919YAkgH84Hfj7ci2Z2IXAhwOzZs4fdSao66L6qNgepJDG1OUjEsik5/M3MjsjnQc3s0wQlkeuH28bdL3f3he6+sKmpadh9JeIxaqvianOQiqI2B4laNiWHE4DzzGwN0EnQpdXDdoMxM7PzCBqqT/I8jXlRV63xlaSyaPgMiVo2yeHUfB3MzE4BLgZe6+4d+dpvenwlkUqhrqwStVGrldx9nbuvIxiAzzMeIwrn2b0PmG9m68O5db8H1AN3mNmycDjwnNXVJNVbSSqKeitJ1EYtOZjZW4BvAgcCWwnmclhJMPnPsIaZZ/eqccQ4qoaahNocpKKozUGilk2D9BeARcCT7t5CMIn6/QWNaozqqhPqrSQVJd1bScNnSFSySQ7d7r4diJlZzN3/BCwscFxjojYHqTTp+xz6NE+JRCSbBukXzawOuBe43sy2ksVNcMVUV51UbyWpKP0D7/UqOUg0sik5nAF0AB8DbiMYE+nNhQxqrOrDqUJVPyuVQm0OErVsksMBQJW794SD8V1B0OOoZKQH32vvUulBomVmHzOzx81suZndYGY149lP3DR8hkQrm+TwSyCzVaw3XFcyNKeDlAIzmwF8BFgYjkQcB84cz77ianOQiGWTHBLu3pV+Ei5XFS6ksdOcDlJCEgQTZCUIJsXaOK6dpO+QVpuDRCSb5NAa3usAgJmdAWwrXEhjpzkdpBS4+wbgG8CzwCZgh7vfPnA7M7vQzJaY2ZLW1tYh9xXmBrU5SGSySQ7vAz5lZs+a2bPAvxOOlloq6jSPtJQAM5tM0IGjheCm0ZSZnTNwu2wGlUzGg69mt+5zkIiM2JXVzOLA+919UdidFXffVZTIxqBBbQ5SGl4PrHH3VgAzuxF4FXDdWHc0oSoYin53V28+4xPJ2oglB3fvJRiVFXffVYqJAdTmICXjWWCRmdWamRGMJrByPDuqTQbJob1TyUGikc1NcI+Y2W8Jeij13/zm7jcWLKoxSrc57OpUm4NEx90fMLNfAQ8TzFfyCHD5ePaViMeoTsToUPdsiUg2yaEG2A68LmOdAyWTHGqr4sRM1UoSPXe/DLgsH/vSPCUSpVGTg7u/qxiB5MLMqKvW+EpSWWqr43SozUEiMmpvJTM72MzuMrPl4fMjzewzhQ9tbOprkkoOUlFSVQnaVXKQiGTTlfUK4FKgG8DdH2Wcd30WUjC+ktocpHKkqhMaEkYik01yqHX3BwesK7kzVtVKUmlqq+LqrSSRySY5bDOzlxBODWpmbyO4+3NEZna1mW1NV0eF66aY2R1m9lT4c/K4Ix8gPTKrSKVIVSXUW0kik01y+CDwQ+AQM9sAfJTgrunRXAOcMmDdJcBd7j4PuCt8nhd1anOQCpOqTqjkIJHJprfSauD1ZpYCYu7els2O3f1eM5szYPUZwOJw+VrgboLhOHKm2eCk0qSq42pzkMhk01tpqpl9F/gzcLeZfcfMpo7zeNPcPV0ltRmYNs79DFJfndDAe1JRUtUJOlRykIhkU630M6AV+CfgbeHyz3M9sLs7YTvGULIZuTJTfU2Czp4+uno0UJlUhlRVnK5endMSjWySQ7O7f8Hd14SPLzL+K/4tZtYMEP7cOtyG2YxcmamuOj2EhorhUhlqq4JzWo3SEoVsksPtZnammcXCx9uBP4zzeL8Fzg2XzwVuGud+BmmqD2Zj3LRjd752KRIpXfBIlLJJDu8Bfgp0ho+fAe81szYz2zncm8zsBuA+YL6ZrTezdwNfBU42s6cIhjf+aq6/QFpLYwqAtds68rVLkUjVVgcjs2oIDYlCNr2V6sezY3c/a5iXThrP/kYzp7EWgLXb20fZUqQ8pMJqJQ2hIVHIpuRQFmqrEkxrqGZ1q5KDVIZUdTo5qOQgxVcxyQFgztSUSg5SMWrD2eB0r4NEoaKSw9ymFGu3KTlIZUg3SKu3kkQhm8l+0nNJT8vc3t2fLVRQ4zVnaort7V3s2N3NxAnJqMMRyUm6QXqXqpUkAqMmBzP7MMHMVluA9N04DhxZwLjGZU5/j6V2XjZrUsTRiOQm3SDdoQZpiUA2JYeLgPnuvr3QweSqvzvrdiUHKX8TknHMoF1dWSUC2bQ5PAfsKHQg+TB7Si1msEbtDlIBYjGjNhlXV1aJRDYlh9UEA+79juAmOADc/VsFi2qcapJxDpw4QclBKkZtteZ0kGhkkxyeDR9V4aOktTSqx5JUjrrqhBqkJRLZ3CH9uWIEki9zGmv57bKNuDtmFnU4IjmprYqrQVoiMWxyMLNvu/tHzexmhhha293fUtDIxqmlsY6de3p4vr2LqXXVUYcjkpNUVUI3wUkkRio5/CT8+Y1iBJIvLRljLCk5SLGZ2STgSmABwUXV+e5+33j3l6qOs21XV77CE8nasMnB3ZeGP+8pXji5mzM16M66ZlsHxx40JeJoZD/0HeA2d3+bmVUBtbnsrLY6QfvzGmlYii+rO6TLyawptcRjpkZpKTozmwi8BjgPwN27gJwu++uqNFWoRKOixlYCSMZjzJqs7qwSiRaCaXR/ZGaPmNmVZpYauNFYpsCtrdZ9DhKNrJODmeVUPC6mOY0pJQeJQgI4Bvhfdz8aaAcuGbjRWKbATTdIB1OuixTPqMnBzF5lZiuAJ8LnLzOz/yl4ZDlID92tL5QU2Xpgvbs/ED7/FUGyGLdUdYI+h86evtE3FsmjbEoO/wW8EdgO4O5/J6hXLVktjSk6unrZ2tY5+sYieeLum4HnzGx+uOokYEUu+0z1j8yqqiUprqyqldz9uQGrcmohM7OPmdnjZrbczG4ws5pc9jdQegA+VS1JBD4MXG9mjwJHAV/OZWe1/SOzqlFaiiurgffM7FWAm1nSzD4JrBzvAc1sBvARYKG7LwDiwJnj3d9QWjKG7hYpJndfFrYnHOnub3X3F3LZX51KDhKRbJLD+4APAjOADQRXQx/I8bgJYIKZJQj6gW/McX/7OHDSBKriMdZoylApc/0lB90lLUWWzX0O8939XzNXmNnxwF/Hc0B332Bm3yAYzG83cLu73z5wOzO7ELgQYPbs2WM6RjxmzJoygTWtSg5S3lLhVKGa00GKLZuSw39nuS4rZjYZOIOgT/iBQMrMzhm43Vi6+w2lpbGOtSo5SJlLN0jrXgcptpEG3jsOeBXQZGYfz3ipgaCdYLxeD6xx99bwODeGx7kuh30O0tJYy71PtdLX58RiGp1VylN6qlAlBym2kUoOVUAdQQKpz3jsBN6WwzGfBRaZWa0FY2qfRA4N3MOZ05iiq6ePjTt253vXIkWTrlbqULWSFNlIA+/dA9xjZte4+7p8HdDdHzCzXwEPAz3AI8Dl+dp/WsvUdI+lDmZOLpubu0X2UVul3koSjWwapDvM7OvA4UD//Qju/rrxHtTdLwMuG+/7s9HSFN7rsL2dE+Y1FvJQIgVTnYgRj5l6K0nRZdMgfT3B0BktwOeAtcBDBYwpL6bV11CTjKnHkpQ1MyNVFaddN8FJkWWTHKa6+1VAt7vf4+7nA+MuNRRLLGbMmZpizbZdUYcikpNUdUIN0lJ02SSH7vDnJjM73cyOBspiFp0jZ05kyboX6OnVoGVSvmqr4mqQlqLLJjl8MZzE5BPAJwmmQPxYQaPKk8XzD6BtTw8PP/ti1KGIjFtdTZKde7pH31Akj0ZMDmYWB+a5+w53X+7uJ7r7se7+2yLFl5MT5jUSjxl3r9oadSgi4zZz0gSe01ShUmQjJgd37wXOKlIseddQk+TYgyZz96qRZ9sSKWVzm1I898JuujSngxRRNtVKfzWz75nZq83smPSj4JHlyeL5TazYtJMtO/dEHYrIuMxtStHb5zz7vHreSfFkkxyOIrjH4fPAN8PHNwoZVD6dOP8AAO5R6UHK1NzGOgCeUbdsKaJRb4Jz9xOLEUihHDK9nukNNdz95Fbe/vJZUYcjMmZzwxs6Vys5SBFlNRNcOTMzXntwE39+chvd6tIqZai+JklTfTWrW3XPjhRPxScHgBMPaaKts4eH1+U0KZdIZOY2plitmQ2liEbryhoLpwgta8e/tJFEzLj7SbU7SHma21SnkoMU1WhdWfuA7xcploKpD7u0/ukJ3e8g5eklTSle6OjmhfauqEOR/UQ21Up3mdk/hXMvlK0TDzmAJza3sXmHurRK+elvlNZYYVIk2SSH9wK/BLrMbKeZtZnZzgLHlXeL5wdTjd7zpEoPUn7UnVWKbdTk4O717h5z96S7N4TPG4oRXD7NnxZ2adX9DlKGZk6eQDJu6s4qRZPNZD+Y2VuA14RP73b3WwoXUmGYGYvnN/G7RzfR3dtHMr5fdNSSCpGIxzhoakqN0lI0o/6HNLOvAhcBK8LHRWb2lUIHVgiL5x9AW2cPS9WlVQrIzOJm9oiZ5fUiSt1ZpZiyuXw+DTjZ3a9296uBU4DTczmomU0ys1+Z2RNmttLMjstlf9k6/qVTgy6tqlqSwroIWJnvnbY0pVi3vV3zk0hRZFu3MiljeWIejvsd4DZ3PwR4GQX4Ig2lvibJwjmTNYS3FIyZzSS4eLoy3/t+SWMd3b3O+hd253vXIoNkkxy+DDxiZteY2bXAUuBL4z1gOHHQa4CrANy9y92LNhvPifODLq2bdugLJgXxbeBiYNjLezO70MyWmNmS1tbsS7Hp7qxrVLUkRTDqHdIEJ/ki4Ebg18Bx7v7zHI7ZArQCPwrrZa80s9QQxx7XF2g0izVKqxSImb0J2OruS0fazt0vd/eF7r6wqakp6/3PbUp3Z1WjtBReNndIX+zum9z9t+Fjc47HTADHAP/r7kcD7cAlQxx7XF+g0Rw8rY7miTX8SVVLkn/HA28xs7XAz4DXmdl1+dr5lFQVk2qTapSWosimWulOM/ukmc0ysynpRw7HXA+sd/cHwue/IkgWRRF0aT2Avz69XTNrSV65+6XuPtPd5wBnAn9093PyeYy5jerOKsWRTXL4F+CDwL0E7Q1LgSXjPWBY8njOzOaHq04i6CJbNIvnN7FLXVqlDAUD8KnkIIWXTZvDJe7eMuAxN8fjfhi43sweJZhp7ss57m9Mjn9pI8m4qdeSFIy73+3ub8r3fuc2pdja1knbnu5871pkH9m0Ofxbvg/q7svC9oQj3f2t7l7US/i66gQvnzNF9ztI2UmPsaQeS1JoUbQ5lITF85tYtaWNjS+qS6uUj5doylApkqK3OZSK/i6tmgBIysjsqbXEDDVKS8GNOvCeu7cUI5Bim3dAHTMmTeBPT2zlrFfMjjockaxUJ+LMmlLLM6pWkgIbtuRgZhdnLP/zgNeK2oBcCGbGa+c38dent6lLq5SVoDurkoMU1kjVSmdmLF864LVTChBL0S0+uIn2rl6WrHs+6lBEsja3qY4123bR1+dRhyIVbKTkYMMsD/W8LO3t0qp2Bykfc5tS7OnuY9NOTXkrhTNScvBhlod6XpZS1Qle0TJF9ztIWUl3Z1WjtBTSSMnhZek5o4Ejw+X08yOKFF/BnXzoNJ7csosr7l0ddSgiWTlkej1V8Rg3LdsYdShSwYZNDu4ez5gzOhEup58nixlkIZ2z6CBOP6KZL/1+JT++b23U4YiManKqivOOn8OvH17Pio07ow5HKtR+P5FyIh7j22cexcmHTeM/b3qcnz34bNQhiYzqg4tfysQJSb5ya1HmyZL90H6fHACS8RjfO/toXntwE5f+32Pc+PD6qEMSGdHE2iQfft08/vzUNt3IKQWh5BCqTsT54TuO5bi5U/nkL//OLY+qPldK2zsWHcTsKbV85fcr6VW3VskzJYcMNck4V567kIUHTeGiny3jD4/nOq+RSOFUJWJcfMp8ntjcxq9V2pU8U3IYoLYqwdXvejlHzpzIh376MH96Qt1cpXSdfkQzR82axDdvX0VHV0/U4UgFUXIYQl11gmve9QrmT6/nvdct5S9PbYs6JJEhmRmfPv1Qtuzs5Ko/r4k6HKkgSg7DmDghyU/OfyVzG1Nc8OOHeGD19qhDEhnSy+dM4Y2HT+MH9zxDa1tn1OFIhVByGMHkVBXXXfBKZkyawPnXPKRpRaVk/fsph9DZ08e373wy6lCkQig5jKKxrpqfvmcRTfXVnHf1gzy6/sWoQxIZZG5THWe/cjY/e+g5nt6qYTUkd5ElBzOLm9kjZnZLVDFka1pDDT99zyIm1iZ5x1UP6q5UKUkXnTSPCck4X731iahDkQoQZcnhIqBsbu88cNIEbnjPImqr4rzjqgd4aktb1CGJ7GNqXTXvX/wS7ly5hfvVRiY5iiQ5mNlM4HTgyiiOP16zptTy0/csIhYzzr7yAU3yLiXn3Se00Dyxhi//fqXme5CcRFVy+DZwMTDsFGxmdqGZLTGzJa2tpTM8QEtjip9e8Er6+pyzr7if557viDokkX41yTifeMN8Hl2/g5t1l7/koOjJwczeBGx196Ujbeful7v7Qndf2NTUVKTosjNvWj3XXfBKdnf3ctYV97Pxxd1RhyTS7x+OnsGhzQ187bZV7OnujTocKVNRlByOB95iZmuBnwGvM7PrIogjJ4c2N/CT81/Jjo5uzr7ifrZoVq79npnNMrM/mdkKM3vczC6KIo54zPjUaYew4cXdXHbT46xV9aeMQ9GTg7tf6u4z3X0OwTzVf3T3c4odRz4cMXMi15z/ClrbOjn7ivvZtks3IO3neoBPuPthwCLgg2Z2WBSBvHpeE2e9Yha/WPoci79xN2d8/6/86K9rdJOcZE33OeTo2IMmc/V5L2fDi7s558oH2NqmEsT+yt03ufvD4XIbQW+8GVHF85V/PJK/XfI6Lj31ELp6+vjczStY9JW7eOfVD3Ljw+vZ1amxmGR45l76PRoWLlzoS5YsieBVi9UAABBNSURBVDqMEf3lqW2cf+1DdPX0cdDUWhbMmMgRMyay4MCJLJjRwKTaqqhDlGGY2VJ3X5jnfc4B7gUWuPvOAa9dCFwIMHv27GPXrVuXz0MP68ktbfzmkQ3ctGwjG17cTU0yxsmHTeetRx3Iaw5uIhnXtWKlyeXcVnLIo5WbdvLHJ7ayfMMOHtuwg/Uv7G2onjVlQpAswoRxxIyJTE4pYZSCfCcHM6sD7gG+5O43jrRtFOd2X5+z9NkX+M0jG/jdY5t4saObybVJTj+ymbceNYNjD5qMmRU1JikMJYcS9UJ7F8s3Boni8Q07eWzDDp7N6Po6Y1I6YTT0lzSm1lVHGPH+KZ/JwcySwC3AH9z9W6NtH/W53dXTx71PtvKbZRu4c+UW9nT3MXPyBM446kDeetQM5k2rjyw2yZ2SQxnZ0dHN8o07+ksXyzfsYO32vQnjwIk1HB4minRJo6leCaOQ8pUcLLjcvhZ43t0/ms17Sunc3tXZwx+Wb+Y3yzbw16e30edwWHMDbz36QN7yshlMn1gTdYgyRkoOZW7H7m4e37i3dLF8ww5WZ3Q/nNZQ3Z8o0knjgAZ9UfMlj8nhBODPwGPsvcHzU+7+++HeU6rn9ta2Pdzy903ctGwDf1+/AzM4dvZkTlkwnVMWTGfm5NqoQ5QsKDlUoLY93Ty+cSfLN+wtZaze1k76z9VUPzhhTGuoVl3xOBSiQTpb5XBur9nWzm+XbeTW5Zt4YnMwptiRMydyyoLpnLqgmZbGVMQRynCUHPYTuzp7WDEgYTzTuov0EDqNdVV7e0mFjwMn1ihhjELJIXtrt7Vz6/LN3LZ8E39fvwOAQ6bX9yeKg6fV6XwrIUoO+7GOrr0J47ENwc+ntrb1J4wpqSoOP7CBw5obODR8zG1KqdtiBiWH8dnw4m5uCxPFknUv4A5zG1P9iWLBjAYliogpOcg+dnf1smLTTh7fuIPH1u/g8Y07eXrrLrp6g2rwqniMedPq+pPFoc31HNa8/96LoeSQu6079/CHFVu4bfkm7l/9PL19zszJEzh1wXROWdDM0bMmEYspURSbkoOMqru3j2dad7Fy005WbmoLf+5k266u/m2aJ9ZwyPT6jKTRQEtjiniFf6mVHPLr+fYu7lyxhVuXb+IvT2+ju9eZ1lDNKYcHieLlcyaTUMm1KJQcZNy2tu3ZJ1ms3LSTZ1rb6Q3rpWqSMeZP2zdhHNJcT0NNMuLI80fJoXB27unmjyu3cuvyTdy9qpXOnj6mpqp4w+HTOGVBM8fNnUpVQomiUJQcJK/2dPfy9NYBpYzNO3mxo7t/m5mTJ/Qni8Oag+Qxa3JtWVYdKDkUR0dXD3evauXW5Zv548ottHf10lCT4PWHTePUBc28el4jNcl41GFWlFzO7US+g5HyV5OM9/d2SnN3Nu/c058wVoSljDtXbunvXpuqinNI2IbRX8qYXk9tlU4zgdqqBKcd0cxpRzSzp7uXvzy1jVuXb+aOFZu58eENpKrinHjIAZy6oJnF85tIVeu8iZJKDpKT3V29rNqyb7XUE5vaaAtH/DSDOVNTQcKYHlZNHdhQUl1sVXKIVndvH/c9s51bl2/m9sc3s729i+pEjNce3MSpR0znpEOnVVQ1ZjGpWklKiruz/oXd/aWLdGkjc1yphprEPr2lDm1u4OBp9ZFUKyg5lI7ePuehtc+HXWQ3s3nnHpJx4/iXNnLqgumcfNh0pmjAyqwpOUhZaNvTzZNb2liR0QC+anMbHV3BVJbxmNHSmNonYRzW3MAB9YW981vJoTT19TnL1r/Ibcs3c+vyTTz3/G7iMeOVLVM4dcF03nj4dA0jMwolBylbfX3Ouuc79qmWWrmpjQ0Z83JPSVXtWy3V3MBLD6jLWy8XJYfS5+48vnFnf6J4pjUYe6y+OkGqOkFtdZxUVYJU+LO2OkFddZzaqgSpqni4zd7lYJs4ddUJaqvS701UXM8pNUhL2YqFpYWWxhSnHdHcv35HRzcrN++bMH58/zq6eoIb+ZJx4yVNdfvc+X1oc72GPK9QZtbfSeKTb5zPU1vauHPlVra27aGjs5f2rh7aO3to7+pl8849dHT1Bs/DddlKxm1v8kgnkjDJ9CeS8PX0+r0JKZ1swnXVCWqT8bK9p0PJQUrSxNoki+ZOZdHcqf3renr7WLOtPWzLCKqm/vL0Nm58ZEP/NgfUVw+687ulMVW2X1AZ2rxp9VnPNdHX5+zp6WVXZ09GIgl+dnSGSSQjuXR09rCrs5eOruB5e2cP23d17E04XT3s6e4b/cChmmSsP3mkSyi1VYMTSaoqIwkN2DadpFJVCSYk40XpMl705GBms4AfA9MABy539+8UOw4pP4l4rP+fwhlH7V2/fVfnPvdjrNzUxt+eWU13b1BlWp2IMX96Pb9636sqrtpARheLGbVVwT9i8jR3UU9vHx3dvXR0hkknnXDC5LG35BIkmWCbvdu27elh846whBMmpvT5OhozqE3Gw6qzBP9w9Aw+ctK8/PxiGaIoOfQAn3D3h82sHlhqZne4+4oIYpEKMLWumhPmVXPCvMb+dV09mcOF7KS1rVOJQfImEY/REI/ltYttV0/f4EQyoMqsI6OqLNi2lwMKNBlY0ZODu28CNoXLbWa2EpgBKDlI3lQlYv3VSyLloCoRoypRVTIDYEZ6KWVmc4CjgQeGeO1CM1tiZktaW1uLHZqIyH4tsuRgZnXAr4GPuvvOga+7++XuvtDdFzY1NRU/QBGR/VgkycHMkgSJ4Xp3vzGKGEREZHhFTw4W3Op6FbDS3b9V7OOLiMjooig5HA+8A3idmS0LH6dFEIeIiAwjit5KfwFKYzhOEREZkjp+i4jIIEoOIiIySFmMympmrcC6IV5qBLYVOZx8KNe4oXxjHynug9w9kv7Sw5zb5foZQ/nGXq5xQ4HO7bJIDsMxsyVRDbWci3KNG8o39nKKu5xiHahcYy/XuKFwsataSUREBlFyEBGRQco9OVwedQDjVK5xQ/nGXk5xl1OsA5Vr7OUaNxQo9rJucxARkcIo95KDiIgUgJKDiIgMUpbJwcxOMbNVZva0mV0SUQyzzOxPZrbCzB43s4vC9VPM7A4zeyr8OTlcb2b23TDmR83smIx9nRtu/5SZnZux/lgzeyx8z3fDQQvz+TvEzewRM7slfN5iZg+Ex/u5mVWF66vD50+Hr8/J2Mel4fpVZvbGjPUF+RuZ2SQz+5WZPWFmK83suHL6zLP4/XRu5x5/2Z3X4b5L69x297J6AHHgGWAuUAX8HTgsgjiagWPC5XrgSeAw4GvAJeH6S4D/Fy6fBtxKMK7UIuCBcP0UYHX4c3K4PDl87cFwWwvfe2qef4ePAz8Fbgmf/wI4M1z+AfD+cPkDwA/C5TOBn4fLh4WffzXQEv5d4oX8GwHXAheEy1XApHL6zHVuF/5zLsfzuhTP7aKedHn6AI8D/pDx/FLg0hKI6ybgZGAV0ByuawZWhcs/BM7K2H5V+PpZwA8z1v8wXNcMPJGxfp/t8hDvTOAu4HXALeEJsw1IDPycgT8Ax4XLiXA7G/jZp7cr1N8ImAisIexIMfCzLPXPXOd24T/ncjyvS/XcLsdqpRnAcxnP14frImP7Tnc6zYN5sgE2A9PC5eHiHmn9+iHW58u3gYuBvvD5VOBFd+8Z4nj9MYav7wi3H+vvlKsWoBX4UVhtcKWZpSifz3w0OrdzV47nNZTguV2OyaGk2AjTnXqQokuur7CZvQnY6u5Lo45ljBLAMcD/uvvRQDtBUbtfqX7m5ajczu0yPq+hBM/tckwOG4BZGc9nhuuKzoae7nSLmTWHrzcDW8P1w8U90vqZQ6zPh+OBt5jZWuBnBEXw7wCTzCw9x0fm8fpjDF+fCGwfx++Uq/XAend/IHz+K4IvVDl85tnQuZ2bcj2voRTP7ajrM8dRN5cgaGRpYW+j0OERxGHAj4FvD1j/dfZtQPpauHw6+zYgPRiun0JQ1zg5fKwBpoSvDWxAOq0Av8di9jbc/ZJ9G+4+EC5/kH0b7n4RLh/Ovg13qwka7Qr2NwL+DMwPlz8bft5l9Znr3C7851xu53UpnttFPeny+CGeRtCD4hng0xHFcAJBEe9RYFn4OI2gzvIu4Cngzow/jAHfD2N+DFiYsa/zgafDx7sy1i8Elofv+R4DGqvy9HtkfonmhifQ0+EXqjpcXxM+fzp8fW7G+z8dxreKjN4PhfobAUcBS8LP/TfhF6CsPnOd24X/nMvtvC7Fc1vDZ4iIyCDl2OYgIiIFpuQgIiKDKDmIiMggSg4iIjKIkoOIiAyi5DAKM5tqZsvCx2Yz25DxvGqU9y40s+9mcYy/5SnWxWa2I7z9fpWZ3RveNZrN+141xmPVmtn14SiPy83sL+EdtXn7faSwdG4P+x6d2wQ3dcgI3H07Qf9jzOyzwC53/0b6dTNL+N5xWwa+dwlBv+XRjjGmk3cUf3b3N4WxHQX8xsx2u/tdI7xnMbALGMuJfxGwxd2PCI81H+iGvP8+UiA6t4elcxuVHMbFzK4xsx+Y2QPA18zsFWZ2X3hV87fwZEpftaTHlP+smV1tZneb2Woz+0jG/nZlbH+37R3T/fr0mOtmdlq4bmk4Fvsto8Xp7suAzwMfCvfxZgvGrX/EzO40s2nhwGrvAz4WXjG+eqjthth9Mxm337v7KnfvHPD7fD7jSnSDmf0oXH+OmT0Yrv+hmcXH+CeQAtG5DejcDkRxB2a5Pghuaf8kcA3BcMDxcH0De4cEfj3wax98l+ZnCa5eqoFGgjFckuFruzK230Ew7kkMuI/gbtUagpEWW8Ltbkjvd0B8iweuJ7gyXBkuT2bvvOEXAN/M/L0y3jPkdkPsd2sY4xeBeRmv7Rqw7SSCuziPBQ4Fbs743f8HeGfUf9v9/aFzW+f2wIeqlcbvl+7eGy5PBK41s3kEww4kh3nP7zy4Auk0s60Ew++uH7DNg+6+HsDMlgFzCIrFq919TbjNDcCFWcaZOdvTTODnFgzgVUUw7spQRt3O3ZeZ2VzgDQT/NB4ys+PcfeU+Bw+uDq8DvuXuS83sQwRfpIfCC8cJ7B1MTEqDzm2d26pWykF7xvIXgD+5+wLgzQRXQ0PpzFjuZeg2n2y2GYujgfRJ/d/A9zyoS33vCHFmtZ2773L3G939AwRfktOG2OyzBKNN/ih8bsC17n5U+Jjv7p8dx+8lhaNzW+e2kkOeTGRvHeV5Bdj/KmCu7Z3j9l+yeZOZHQn8B8EAXbBvnOdmbNpGMB0ko2yXue/jbe98tlUEUyuuG7DNmwmuvD6Ssfou4G1mdkC4zRQzOyib30cioXN7Pz23lRzy42vAV8zsEQrQA8zddxPMd3ubmS0lOOF3DLP5q8PGtlUEX5yP+N7eHJ8FfhnuY1vGe24G/iHdaDfCdpleAtxjZo8BjxD0XPn1gG0+TjDbVLqB7vPuvgL4DHC7mT0K3EHQACilSef2fnpua1TWMmFmde6+K6zn/D7wlLv/V9RxieRK53ZpUsmhfLwnbMR7nKBo/MOI4xHJF53bJUglBxERGUQlBxERGUTJQUREBlFyEBGRQZQcRERkECUHEREZ5P8D5lGDX28JuiAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.subplot(1,2,1)\n",
    "plt.plot(amount,valerr)\n",
    "plt.title('MLP' )\n",
    "plt.xlabel('Training Data Size')\n",
    "plt.ylabel('Error rate in percantage')\n",
    "plt.subplot(1,2,2)\n",
    "plt.plot(amount,valerr_2)\n",
    "plt.title('CNN' )\n",
    "plt.xlabel('Training Data Size')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o0MdoL6q43S1"
   },
   "source": [
    "### Comment:\n",
    "**The CNN model's plot for task 3 is a bit different as there is a rapid reduction of error rate with CNN model. The shapes i similar but the elbow looks sharper. This shows high suitability of CNN for this dataset.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "Deep Learning CourseWorkProject2021.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}