Profiling and Optimizing Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "b5c78042-c045-4c41-8c87-e1653c735842"
    }
   },
   "source": [
    "# Profiling Python\n",
    "\n",
    "Famously, premature optimization has been called the source of all evil, at least when it comes to computer programming. The key part of this quote is the word premature. In order to optimize your code, you need to know what parts need optimization. This is where profiling comes in. With profiling, you can figure out just how much time is being used in various parts of your code, as well as how much memory is being used. With this information, you will be armed to figure what where to focus your energy to get the largest improvement. In this workshop, I will be assuming that you have some Python experience and ready to try and get the most out the code you have written."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "0fc0f506-117f-4cd3-afda-3a4bce378e7b"
    }
   },
   "source": [
    "## Time Profiling\n",
    "\n",
    "The first step in getting better, faster code is to do some profiling to see where your time is being spent. The three broad categories of profiling are\n",
    "* course time measurements\n",
    "* function profiling\n",
    "* line profiling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "679cf2de-72a5-44a6-9b96-47d6cad7ab52"
    }
   },
   "source": [
    "## Course Time Measurements\n",
    "\n",
    "The most basic form of profiling is just seeing how long different chunks of code take to run. You can do this most simply by measuring the start and end times and finding the difference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "nbpresent": {
     "id": "d84eef1e-d245-457b-97be-0adf5eae629c"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.38340199661512153\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "start = time.clock()\n",
    "for a in range(1000000):\n",
    "    b = a ^ a\n",
    "end = time.clock()\n",
    "print(end-start)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "7c336aa9-56a4-4214-8f54-5499f51474ae"
    }
   },
   "source": [
    "A more organized way would be to have a timer object that you can easily reuse rather than sticking a number of start-end time commands. An example class might look like the code below (which was blatantly borrowed from the website https://www.huyng.com/posts/python-performance-analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "nbpresent": {
     "id": "43d216d0-896b-4891-82b1-8f07951e828b"
    }
   },
   "outputs": [],
   "source": [
    "class Timer(object):\n",
    "    def __init__(self, verbose=False):\n",
    "        self.verbose = verbose\n",
    "\n",
    "    def __enter__(self):\n",
    "        self.start = time.time()\n",
    "        return self\n",
    "\n",
    "    def __exit__(self, *args):\n",
    "        self.end = time.time()\n",
    "        self.secs = self.end - self.start\n",
    "        self.msecs = self.secs * 1000  # millisecs\n",
    "        if self.verbose:\n",
    "            print('elapsed time: %f ms' % self.msecs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "a56d1cd1-3100-4718-b5d8-09f19e7d395e"
    }
   },
   "source": [
    "You can then use it with the 'with' statement to time chunks of code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "nbpresent": {
     "id": "b8f2af9e-6434-4fa0-a9ab-a18e97dbeba5"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=> elasped 1 million: 0.42665958404541016 s\n",
      "=> elasped 10 million: 3.5925371646881104 s\n"
     ]
    }
   ],
   "source": [
    "with Timer() as t:\n",
    "    for a in range(1000000):\n",
    "        b = a^a\n",
    "print(\"=> elasped 1 million: %s s\" % t.secs)\n",
    "\n",
    "with Timer() as t:\n",
    "    for a in range(10000000):\n",
    "        b = a^a\n",
    "print(\"=> elasped 10 million: %s s\" % t.secs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you use the IPython shell, there are special functions called magics that let you interact with the Python interpreter. Two of these magics are %timeit and %%timeit. The first one times how long a single statement takes to run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000000 loops, best of 3: 1.1 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%timeit range(1000000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000000 loops, best of 3: 1.07 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "a = range(1000000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "b95b638c-0db2-49a5-90f0-639a64c552f1"
    }
   },
   "source": [
    "## Function Profiling\n",
    "\n",
    "While this might be good enough for simple code, it quickly becomes too coarse to do any real analysis with. Luckily, Python includes two profiling modules as part of a standard installation. The first a pure Python module named 'profile'. Because it is pure Python, it introduces quite a lot overhead. It is ideal if you want to alter the behavior of the profiler, however. The second profiler is named 'cProfile', which is a C implementation of the same behavior as 'profile'. It is therefore much faster.\n",
    "\n",
    "The easiest way to use it is to use the run method. We will start by creating our load function to make the code easier to read."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true,
    "nbpresent": {
     "id": "e79c2d55-a4f3-4452-ae6f-a77c8c34769e"
    }
   },
   "outputs": [],
   "source": [
    "def myload(num):\n",
    "    for a in range(num):\n",
    "        b = a^a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "6cbee6a8-a2eb-4f67-8fe5-d6274f73a34f"
    }
   },
   "source": [
    "Now we can profile it and see what happens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "nbpresent": {
     "id": "5accdfdc-c8f9-496b-974e-3525d995d861"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "         4 function calls in 0.240 seconds\n",
      "\n",
      "   Ordered by: standard name\n",
      "\n",
      "   ncalls  tottime  percall  cumtime  percall filename:lineno(function)\n",
      "        1    0.240    0.240    0.240    0.240 <ipython-input-5-421e6c049a13>:1(myload)\n",
      "        1    0.000    0.000    0.240    0.240 <string>:1(<module>)\n",
      "        1    0.000    0.000    0.240    0.240 {built-in method builtins.exec}\n",
      "        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import cProfile\n",
    "\n",
    "cProfile.run('myload(1000000)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "6475efa9-1af8-40d9-999a-39f273c42dc8"
    }
   },
   "source": [
    "As you can see, cProfile gives a breakdown of each function call, how many times it gets called, and various measures of the amount of time used. This is a good starting point for larger programs. The major issue with the run method is that you can only use a string that can be run with the exec function. You can also use cProfile directly (like the timer class above) to collect stats for sections of code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Line Profiler\n",
    "\n",
    "If you need more detail, you can use another module named line_profiler. This one usually is not included in most Python installations. You will likely need to install it with\n",
    "\n",
    "* pip install line_profiler\n",
    "\n",
    "It comes with a command line utility called kernprof that you can use to profile your scripts. You need to add a decorator to any functions you want to profile like"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-7-cd3c5e2218d4>, line 2)",
     "output_type": "error",
     "traceback": [
      "\u001b[1;36m  File \u001b[1;32m\"<ipython-input-7-cd3c5e2218d4>\"\u001b[1;36m, line \u001b[1;32m2\u001b[0m\n\u001b[1;33m    myload(1000000)\u001b[0m\n\u001b[1;37m         ^\u001b[0m\n\u001b[1;31mSyntaxError\u001b[0m\u001b[1;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "@profile\n",
    "myload(1000000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then run it with\n",
    "\n",
    "* kernprof -l myscript.py\n",
    "\n",
    "This generates a binary output file with the profiling information. You can process the results with\n",
    "\n",
    "* python -m line_profiler myscript.py.lprof\n",
    "\n",
    "If you want to use line_profiler directly within your script, you can import the LineProfiler class and use it to profile statements in your script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "nbpresent": {
     "id": "7d6e498f-e91d-4059-8d2f-32a6ce8c587e"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Timer unit: 4.72616e-07 s\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\berna_000\\Anaconda3\\lib\\site-packages\\ipykernel\\__main__.py:4: UserWarning: Could not extract a code object for the object None\n"
     ]
    }
   ],
   "source": [
    "import line_profiler\n",
    "\n",
    "profiler = line_profiler.LineProfiler()\n",
    "profiler.add_function(myload(1000000))\n",
    "profiler.print_stats()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "The above warning is because we are running this through Jupyter. If you were to run this as a regular script in IPython, it won't occur. It will also give you the line-by-line profiling information that looks like\n",
    "\n",
    "\n",
    "File: test.py\n",
    "\n",
    "Function: get_number at line 43\n",
    "\n",
    "Total time: 4.44195 s\n",
    "\n",
    "Line #      Hits         Time  Per Hit   % Time  Line Contents\n",
    "==============================================================\n",
    "    43                                           def get_number():\n",
    "    44   5000001      2223313      0.4     50.1      for x in xrange(5000000):\n",
    "    45   5000000      2218638      0.4     49.9          yield x\n",
    "\n",
    "You can also use it in a fashion similar to cProfile, using the run() method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Timer unit: 4.72616e-07 s\n",
      "\n"
     ]
    }
   ],
   "source": [
    "profiler.run('myload(1000000)')\n",
    "profiler.print_stats()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Memory Profiling\n",
    "\n",
    "The other resource that you may need to optimize for is memory. Similarly to line_profilr is a module named memory_profiler. You can import is with\n",
    "\n",
    "* pip install memory_profiler\n",
    "\n",
    "The most basic way to use it is again similar to line_profiler. It includes a decorator named @profile which you can add to your script to profile the functions in question. You can then run it with\n",
    "\n",
    "* python -m memory_profiler myscript.py\n",
    "\n",
    "Similarly to line_profiler, memory_profiler includes a command line utility called mprof. You can run it with\n",
    "\n",
    "* mprof run myscript.py\n",
    "* mprof list\n",
    "\n",
    "It can even generate a plot of memory usage with\n",
    "\n",
    "* mprof plot\n",
    "\n",
    "In an interactive way, you can use memory_profiler directly in a few different ways. The easiest is similar to the coarse timing examples above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[34.93359375]\n",
      "[421.328125]\n"
     ]
    }
   ],
   "source": [
    "import memory_profiler\n",
    "\n",
    "usage = memory_profiler.memory_usage()\n",
    "print(usage)\n",
    "b = list()\n",
    "for a in range(10000000):\n",
    "    b.append(a)\n",
    "usage = memory_profiler.memory_usage()\n",
    "print(usage)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can analyze the memory usage for a given function using the same function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[421.3671875, 421.4375, 434.71875, 448.64453125, 456.4296875, 421.6328125]\n"
     ]
    }
   ],
   "source": [
    "def myload2(a):\n",
    "    b = list()\n",
    "    for c in range(a):\n",
    "        b.append(c)\n",
    "\n",
    "usage = memory_profiler.memory_usage((myload2,(1000000,)))\n",
    "print(usage)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the above gives you information about raw memory usage, the issue in your code might be an inefficient use of memory by having more objects around than are strictly needed. You can use objgraph to get access to what is taking up chunks of memory. You can install it with\n",
    "\n",
    "* pip install objgraph\n",
    "\n",
    "Then you can start with a report of the most common objects in memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "function                   7610\n",
      "dict                       4990\n",
      "tuple                      3633\n",
      "weakref                    1891\n",
      "list                       1369\n",
      "wrapper_descriptor         1195\n",
      "getset_descriptor          1060\n",
      "builtin_function_or_method 961\n",
      "method_descriptor          910\n",
      "type                       890\n",
      "Show growth of objects\n",
      "function                       7608     +7608\n",
      "dict                           4819     +4819\n",
      "tuple                          3228     +3228\n",
      "weakref                        1892     +1892\n",
      "list                           1332     +1332\n",
      "wrapper_descriptor             1195     +1195\n",
      "getset_descriptor              1060     +1060\n",
      "builtin_function_or_method      960      +960\n",
      "method_descriptor               910      +910\n",
      "type                            890      +890\n",
      "Interactive        1        +1\n"
     ]
    }
   ],
   "source": [
    "import objgraph\n",
    "\n",
    "objgraph.show_most_common_types()\n",
    "print('Show growth of objects')\n",
    "objgraph.show_growth()\n",
    "myload2(1000000)\n",
    "objgraph.show_growth()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Optimizing Python\n",
    "\n",
    "Now that you have some idea of what parts of your code are most problematic, you can now start to look at what you could do to get better performance out of it. One of the major efficiency problems with Python is also one of its greatest powers, the combination of untyped variables and object oriented programming. Because variables are untyped, Python ends up having to check what object it refers to each time it is ever accessed. Many of the optimization ideas are based on minimizing this issue. The following is simply a list of possible ideas in no particular order."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use keys and sort\n",
    "\n",
    "Sorting is something that can easily be done inefficiently. Luckily, Python has an extremely efficient sort method in lists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The slowest run took 6.98 times longer than the fastest. This could mean that an intermediate result is being cached.\n",
      "1000000 loops, best of 3: 812 ns per loop\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[1, 2, 2, 3, 5, 8, 9, 10, 74]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = [1,74,5,2,8,3,9,10,2]\n",
    "%timeit a.sort()\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the sort is done in place. You can sort more complicated lists using keys. Say you had a list of tuples; you can sort based any of the values contained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100000 loops, best of 3: 4.17 µs per loop\n",
      "[(1, 2, 3), (3, 4, 5), (9, 1, 7)]\n",
      "100000 loops, best of 3: 4.15 µs per loop\n",
      "[(9, 1, 7), (1, 2, 3), (3, 4, 5)]\n"
     ]
    }
   ],
   "source": [
    "import operator\n",
    "b = [(1,2,3),(9,1,7),(3,4,5)]\n",
    "%timeit b.sort(key=operator.itemgetter(0))\n",
    "print(b)\n",
    "%timeit b.sort(key=operator.itemgetter(1))\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Avoiding dots\n",
    "\n",
    "Whenever you use object methods or module functions and give the full path to them, Python needs to resolve and check every step along the way. This happens everytime it is referenced. For example, making a list of words upper case involves:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100000 loops, best of 3: 7.53 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "lowercase = ['this', 'is', 'a', 'lower', 'case', 'string']\n",
    "uppercase = []\n",
    "for word in lowercase:\n",
    "    uppercase.append(str.upper(word))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can speed up the references to str.upper and uppercase.append, as below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "upper = str.upper\n",
    "uppercase = []\n",
    "append = uppercase.append"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100000 loops, best of 3: 5.37 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "lowercase = ['this', 'is', 'a', 'lower', 'case', 'string']\n",
    "for word in lowercase:\n",
    "    append(upper(word))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using different coding techniques\n",
    "\n",
    "Sometimes, using a different technique to get the same result will alter the amount of time used. Say you were creating a dictionary of characters and how many times they were used. As an example, we'll use the string 'abcd' and simply cycle over them. The first way to create the dictionary would be to create an empty one, and add new entries if the character isn't already being counted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100000 loops, best of 3: 14.7 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "n = 16\n",
    "myDict = {}\n",
    "for i in range(0, n):\n",
    "    char = 'abcd'[i%4]\n",
    "    if char not in myDict:\n",
    "        myDict[char] = 0\n",
    "    myDict[char] += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A different technique would be to use a try/except block instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000 loops, best of 3: 19.2 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "n = 16\n",
    "myDict = {}\n",
    "for i in range(0, n):\n",
    "    char = 'abcd'[i%4]\n",
    "    try:\n",
    "        myDict[char] += 1\n",
    "    except KeyError:\n",
    "        myDict[char] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For small data, the first is slightly faster. For larger data, the reverse is true."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000 loops, best of 3: 1.24 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "n = 1600\n",
    "myDict = {}\n",
    "for i in range(0, n):\n",
    "    char = 'abcde'[i%5]\n",
    "    if char not in myDict:\n",
    "        myDict[char] = 0\n",
    "    myDict[char] += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000 loops, best of 3: 1.08 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "n = 1600\n",
    "myDict = {}\n",
    "for i in range(0, n):\n",
    "    char = 'abcde'[i%5]\n",
    "    try:\n",
    "        myDict[char] += 1\n",
    "    except KeyError:\n",
    "        myDict[char] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## List Comprehensions\n",
    "\n",
    "When you need to create lists of values, you can do the normal thing of using a for loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000 loops, best of 3: 1.19 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "a = []\n",
    "for b in range(1000):\n",
    "    a.append(b**2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use comprehensions, instead, where you define the creation rule within the list definition. This may be faster in some cases, as below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000 loops, best of 3: 1 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "a = [x**2 for x in range(1000)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can make very complex creation rules, as long as they can be defined with if and for statements."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use Numpy\n",
    "\n",
    "In scientific computations, numerical work is the bulk of what your code needs to handle. You can speed these types of operations up by having external libraries written in C do all of the heavy lifting. The main library is numpy. As an example, let's say we want to scale a matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100 loops, best of 3: 5.54 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "w = 100\n",
    "h = 100\n",
    "matrix1 = [[1 for x in range(w)] for y in range(h)]\n",
    "for x in range(h):\n",
    "    for y in range(w):\n",
    "        matrix1[x][y] = 5 * matrix1[x][y]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The slowest run took 1388.21 times longer than the fastest. This could mean that an intermediate result is being cached.\n",
      "10000 loops, best of 3: 54.7 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "w = 100\n",
    "h = 100\n",
    "matrix1 = numpy.ones((w, h))\n",
    "matrix1 = 5 * matrix1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Replace for with map\n",
    "\n",
    "The map function takes the explicit looping of a for loop and turns it into an implicit looping. This moves the bulk of the loop to the interpreter level."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000 loops, best of 3: 23.4 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "uppercase = []\n",
    "lowercase = ['this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string']\n",
    "for word in lowercase:\n",
    "    uppercase.append(str.upper(word))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000000 loops, best of 3: 1.69 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "lowercase = ['this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string']\n",
    "uppercase = map(str.upper, lowercase)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Moving to local variables\n",
    "\n",
    "In cases where you can't use map, you can move your for loop to a function so that all of the intermediate variables are local. Local variables are accessed much more efficiently than global variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def myupper(lowercase):\n",
    "    upper = str.upper\n",
    "    newlist = []\n",
    "    append = newlist.append\n",
    "    for word in lowercase:\n",
    "        append(upper(word))\n",
    "    return newlist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000 loops, best of 3: 23 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "uppercase = []\n",
    "lowercase = ['this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string']\n",
    "for word in lowercase:\n",
    "    uppercase.append(str.upper(word))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000 loops, best of 3: 17 µs per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "lowercase = ['this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string','this', 'is', 'a', 'lower', 'case', 'string']\n",
    "uppercase = myupper(lowercase)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Minimizing function calls\n",
    "\n",
    "One of the more expensive operations in Python is the function call. The example below is a function that is too general."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = 0\n",
    "def doit1(i):\n",
    "    global x\n",
    "    x = x + i\n",
    "\n",
    "list = range(100000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10 loops, best of 3: 59.9 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "for i in list:\n",
    "    doit1(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Moving the for loop into the function, and aggregating your data into a list, gives you:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = 0\n",
    "def doit2(list):\n",
    "    global x\n",
    "    for i in list:\n",
    "        x = x + i\n",
    "\n",
    "list = range(100000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10 loops, best of 3: 30.3 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "doit2(list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Don't import ideas from other languages\n",
    "\n",
    "There are tricks that work well in other languages that do not port to Python. An example is doubling a value, which can be done with\n",
    "\n",
    "* a * 2\n",
    "* a<<1\n",
    "* a + a\n",
    "\n",
    "In C, the middle one works fastest. What about Python?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The slowest run took 12.85 times longer than the fastest. This could mean that an intermediate result is being cached.\n",
      "1000000 loops, best of 3: 147 ns per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "x=42\n",
    "y = 2*x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The slowest run took 8.68 times longer than the fastest. This could mean that an intermediate result is being cached.\n",
      "1000000 loops, best of 3: 218 ns per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "x=42\n",
    "y = x<<1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The slowest run took 10.09 times longer than the fastest. This could mean that an intermediate result is being cached.\n",
      "10000000 loops, best of 3: 140 ns per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "x=42\n",
    "y = x + x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The shift method actually is the worst version in Python."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cython\n",
    "\n",
    "You can move particularly problematic code into its own external C module of code. Cython lets you import the compiled C code back into your Python program. Scientific distributions, such as Anaconda, have it included.\n",
    "\n",
    "You can use it with IPython notebooks with"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%load_ext Cython"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The easiest way to take advantage of it is to use statically typed variables. This way, Python knows what the variables refer to and doesn't need to check."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The slowest run took 18.27 times longer than the fastest. This could mean that an intermediate result is being cached.\n",
      "10000000 loops, best of 3: 77.6 ns per loop\n"
     ]
    }
   ],
   "source": [
    "def sum1():\n",
    "    a = 0\n",
    "    for i in range(1000000):\n",
    "        a += i\n",
    "%timeit sum1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "ename": "DistutilsPlatformError",
     "evalue": "Unable to find vcvarsall.bat",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mDistutilsPlatformError\u001b[0m                    Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-41-53889f58b608>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mget_ipython\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrun_cell_magic\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'cython'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m''\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'def sum2():\\n    cdef int a = 0\\n    for i in range(10):\\n        a += i'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32mC:\\Users\\berna_000\\Anaconda3\\lib\\site-packages\\IPython\\core\\interactiveshell.py\u001b[0m in \u001b[0;36mrun_cell_magic\u001b[1;34m(self, magic_name, line, cell)\u001b[0m\n\u001b[0;32m   2118\u001b[0m             \u001b[0mmagic_arg_s\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mvar_expand\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mline\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstack_depth\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   2119\u001b[0m             \u001b[1;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbuiltin_trap\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 2120\u001b[1;33m                 \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mfn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmagic_arg_s\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcell\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   2121\u001b[0m             \u001b[1;32mreturn\u001b[0m \u001b[0mresult\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   2122\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<decorator-gen-129>\u001b[0m in \u001b[0;36mcython\u001b[1;34m(self, line, cell)\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\berna_000\\Anaconda3\\lib\\site-packages\\IPython\\core\\magic.py\u001b[0m in \u001b[0;36m<lambda>\u001b[1;34m(f, *a, **k)\u001b[0m\n\u001b[0;32m    191\u001b[0m     \u001b[1;31m# but it's overkill for just that one bit of state.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    192\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mmagic_deco\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0marg\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 193\u001b[1;33m         \u001b[0mcall\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;32mlambda\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m*\u001b[0m\u001b[0ma\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mk\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0ma\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mk\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    194\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    195\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mcallable\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0marg\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\berna_000\\Anaconda3\\lib\\site-packages\\Cython\\Build\\IpythonMagic.py\u001b[0m in \u001b[0;36mcython\u001b[1;34m(self, line, cell)\u001b[0m\n\u001b[0;32m    276\u001b[0m             \u001b[0mbuild_extension\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbuild_temp\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mos\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdirname\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mpyx_file\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    277\u001b[0m             \u001b[0mbuild_extension\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbuild_lib\u001b[0m  \u001b[1;33m=\u001b[0m \u001b[0mlib_dir\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 278\u001b[1;33m             \u001b[0mbuild_extension\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    279\u001b[0m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_code_cache\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mmodule_name\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    280\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\berna_000\\Anaconda3\\lib\\distutils\\command\\build_ext.py\u001b[0m in \u001b[0;36mrun\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    336\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    337\u001b[0m         \u001b[1;31m# Now actually compile and link everything.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 338\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbuild_extensions\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    339\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    340\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mcheck_extensions_list\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mextensions\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\berna_000\\Anaconda3\\lib\\distutils\\command\\build_ext.py\u001b[0m in \u001b[0;36mbuild_extensions\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    445\u001b[0m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_build_extensions_parallel\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    446\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 447\u001b[1;33m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_build_extensions_serial\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    448\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    449\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m_build_extensions_parallel\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\berna_000\\Anaconda3\\lib\\distutils\\command\\build_ext.py\u001b[0m in \u001b[0;36m_build_extensions_serial\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    470\u001b[0m         \u001b[1;32mfor\u001b[0m \u001b[0mext\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mextensions\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    471\u001b[0m             \u001b[1;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_filter_build_errors\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mext\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 472\u001b[1;33m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbuild_extension\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mext\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    473\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    474\u001b[0m     \u001b[1;33m@\u001b[0m\u001b[0mcontextlib\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcontextmanager\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\berna_000\\Anaconda3\\lib\\distutils\\command\\build_ext.py\u001b[0m in \u001b[0;36mbuild_extension\u001b[1;34m(self, ext)\u001b[0m\n\u001b[0;32m    530\u001b[0m                                          \u001b[0mdebug\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdebug\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    531\u001b[0m                                          \u001b[0mextra_postargs\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mextra_args\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 532\u001b[1;33m                                          depends=ext.depends)\n\u001b[0m\u001b[0;32m    533\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    534\u001b[0m         \u001b[1;31m# XXX outdated variable, kept here in case third-part code\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\Users\\berna_000\\Anaconda3\\lib\\distutils\\_msvccompiler.py\u001b[0m in \u001b[0;36mcompile\u001b[1;34m(self, sources, output_dir, macros, include_dirs, debug, extra_preargs, extra_postargs, depends)\u001b[0m\n\u001b[0;32m    315\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    316\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minitialized\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 317\u001b[1;33m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minitialize\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    318\u001b[0m         compile_info = self._setup_compile(output_dir, macros, include_dirs,\n\u001b[0;32m    319\u001b[0m                                            sources, depends, extra_postargs)\n",
      "\u001b[1;32mC:\\Users\\berna_000\\Anaconda3\\lib\\distutils\\_msvccompiler.py\u001b[0m in \u001b[0;36minitialize\u001b[1;34m(self, plat_name)\u001b[0m\n\u001b[0;32m    208\u001b[0m             )\n\u001b[0;32m    209\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 210\u001b[1;33m         \u001b[0mvc_env\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_get_vc_env\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mplat_spec\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    211\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mvc_env\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    212\u001b[0m             raise DistutilsPlatformError(\"Unable to find a compatible \"\n",
      "\u001b[1;32mC:\\Users\\berna_000\\Anaconda3\\lib\\distutils\\_msvccompiler.py\u001b[0m in \u001b[0;36m_get_vc_env\u001b[1;34m(plat_spec)\u001b[0m\n\u001b[0;32m     83\u001b[0m     \u001b[0mvcvarsall\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mvcruntime\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_find_vcvarsall\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mplat_spec\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     84\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mvcvarsall\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 85\u001b[1;33m         \u001b[1;32mraise\u001b[0m \u001b[0mDistutilsPlatformError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"Unable to find vcvarsall.bat\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     86\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     87\u001b[0m     \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mDistutilsPlatformError\u001b[0m: Unable to find vcvarsall.bat"
     ]
    }
   ],
   "source": [
    "%%cython\n",
    "def sum2():\n",
    "    cdef int a = 0\n",
    "    for i in range(10):\n",
    "        a += i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%timeit sum2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more complex problems, you can write your Cython code in source files with '.pyx' as a filename ending. You then go through a compile step. It is then usable from your Python code. There is a large amount of information available at\n",
    "\n",
    "* http://docs.cython.org\n",
    "\n",
    "Setting it up correctly on Windows is painful, since you need to have a C compiler that Cython can find and use."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Moving from CPython to PyPy\n",
    "\n",
    "There are many different implementations of Python, written in different languages. CPython is the standard implementation. Jython is one that has been implemented in Java.\n",
    "\n",
    "Pypy is a highly optimized implementation that optimizes your Python code for a JIT (Just-In-Time) compiler. This way, many of the performance issues of checking objects before using them is optimized away. You only get speed-ups for code that runs for long periods of time."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  },
  "nbpresent": {
   "slides": {
    "c770abe6-1191-4d98-89f4-2e00a305cc24": {
     "id": "c770abe6-1191-4d98-89f4-2e00a305cc24",
     "prev": null,
     "regions": {
      "7e4d9e40-38b2-4a36-bb76-e9b8ab65d115": {
       "attrs": {
        "height": 1,
        "width": 1,
        "x": 0,
        "y": 0
       },
       "id": "7e4d9e40-38b2-4a36-bb76-e9b8ab65d115"
      }
     }
    }
   },
   "themes": {}
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}