Improve import time

[gvanrossum]

We've been thinking a lot about startup time, and we've done some good work (freeze and deep-freeze).

As that work is winding down, let's take some time to recognize that when most users talk about slow startup time, they include the time to start their application in that. And the fraction that they have the least control over is the cost of import itself.

There are a couple of parts to this:

• find the module to be imported (either a .py, a .pyc, or a .pyd/.dll/.so)
• loading bytecode from a .pyc file
• loading a shared library

Let's measure the times each of these takes and see how we can improve that time.

[penguin-wwy]

Hello, I have made some experiments before in the old version, hope the conclusions help :)
I generate unrolling interpreter loop code for certain functions, which are executed at import time(e.g. _find_and_load() in Lib/importlib/_bootstrap.py) , and execute them through the eval_frame, replacing the frozen code.

Then, I test a sample:

def test():
    import dis
    del sys.modules['dis']
    del sys.modules['opcode']

Call this function 10, 000 times

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
10000    0.120    0.000   10.640    0.001   import_test.py:5(import_test)
    1    0.015    0.015    10.655   10.655  import_test.py:9(test)

After optimization, ~1% speedup.

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
10000    0.009    0.000   10.543    0.001  import_test.py:5(import_test)
    1    0.016    0.016   10.559   10.559  import_test.py:9(test)

Most of the time in this process is spent on exec_module

20000/10000    0.035    0.000    9.575    0.001 <frozen importlib._bootstrap_external>:672(exec_module)
40000/10000    0.015    0.000    8.415    0.001 <frozen importlib._bootstrap>:211(_call_with_frames_removed)

Since the use case was removed more details are missing, only some simple conclusions :(

• The perceived effectiveness of the optimization depends heavily on the complexity of the user module.
• In the _find_and_load function and its successor, many accesses global variable occurs, generating many complex instructions(e.g. LOAD_GLOBAL, LOAD_ATTR, BINARY_SUBSCR...). Can make some security guarantees that make it possible to localize.

[sthagen]

I think that knowing the hardness of the typical atomic tasks listed in the initial post of that discussion is important and helpful.

Disclosure:

I never had to measure the impact of the CPython import strategy in my use cases. I either accepted or rewrote the application.

I expect that during import the variable number of file system trees the system calls are issued across are the cause of slow start at least for use cases with deep and wide dependency trees.

Any measurement of these actions should in my experience document (and consider)

• the operating system strategy (read-ahead, memory buffering) and
• the varying device (driver) capabilities

Does the reference implementation team have access to specific reference environments (hardware included) to conduct such trial and ideally operate benchmarks to guide and rate implementation alternatives?

In the absence of such reference environments, I fear that the data may not provide actionable metrics for guiding implementations of the import task (as a whole, not the steps in isolation).

I remember that operators of a distributed analytical database with columnar stores were touching a complete file system tree to achieve a boost in performance for application specific chores (off-index full table scans, purges) on Debian linux systems. This user level spidering was effective for them to reach their goals. The optimizations we (as vendor) introduced in the C++ code of the database itself did not suffice.

[gvanrossum]

Here are some quick findings. On a Windows 10 box with a heavy WindowsDefender profile (file points to the PYC file for typing.py and is ~109 kbytes long):

• f = open(file, 'rb') 190 usec
• data = f.read() 35 usec
• marshal.loads(data[16:]) ~700 usec
• import typing 2.68 msec (this includes the above times, plus sys.path search overhead in importlib, but most of it is executing the module)

I suspect there's not much I can do about the ~200 usec just to open the file or the 35 usec to read it. We could potentially speed up marshal. But clearly the big win would be improving execution time.

[gvanrossum]

(Opening a much smaller file takes much less time, 10 usec, so it does seem that the cost of open() may be mostly Windows Defender.)

[lpereira]

Been thinking about this a bit, and maybe we could do something like how Firefox ZIPs files, and store uncompressed pyc files inside, say, a pyz file, and unmarshal directly from the memory-mapped file. The order can be the same as the import order so we benefit quite a bit from OS readahead. (Being a single file, too, also reduces the overhead of opening and closing multiple files, which can be expensive in some common circumstances and add up quite quickly.)

[sthagen]

Maybe some typical scenarios and using strace (or a similar tool) might help to map the system call counts to use cases.
Would that single zip like file behave similar to the "prelude" in other languages or is it only a virtual file system? In the latter case as @gvanrossum observed the "blocking readers in the middle" may have already eaten our (otherwise) free lunch.
How would the "watch" work (change detection would still involve operations per individual import source file, right)?

[methane]

Another idea is __pycache__.zip.
Add an option to compileall that produce zip file instead of __pycache__ directory. Python windows installer and pip install may use the option.

This idea may be nonsense on most systems. But it will help startup time on Windows.

[gvanrossum]

I doubt that the I/O proper is the main cost. On my Mac (no Windows Defender running :-) the open+read take only 28 usec, compared to 255 usec for marshal.loads. (And the whole import is 1.6 msec.) The best way to speed up I/O when Windows defender is running is probably turning it off. (Steve Dower would tell me that we should complain to the Windows Defender team though. :-)

If you write a little snippet of code that creates a much larger file and then tests how fast it is to read part of it I can see what the timings are on my Windows box though.