-
-
Notifications
You must be signed in to change notification settings - Fork 113
Use lazy imports in topology to speed up library import. #942
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Conversation
Loads IPython and scipy only when actually needed. Made build123d load about 1 second faster on my machine. Python caches modules so all in all this should only be a net benefit.
I'm not convinced about this one. Users can skip reloading all of the build123d (and ocp_vscode) content by following the best practices shown here: https://github.com/bernhard-42/vscode-ocp-cad-viewer?tab=readme-ov-file#best-practices (yes, this should be in the build123d documentation too). Use the Jupyter extension for a more interactive experience. This allows to have one cell (separated by # %%) at the beginning to import all libraries # %%
from build123d import *
from ocp_vscode import *
# %%
b = Box(1,2,3)
show(b)
# %% In addition, there is already a proposal to optimize the IPython content see: #924 |
My use case is to have helper code in a separate python package. This helper code needs to import build123d objects. I thought the only way to refresh the help code in jupyter was to restart the kernel, since IPython extension autoload didn't seem to work - after adddional testing, autoreload does seem to work, but it just behaves weird in quite a few circumstances. All in all I still believe that it makes sense to try to make the library load faster and lazy load heavier dependencies when they are required, but if you do not agree, feel free to close the PR :) |
Getting faster loading times is clearly a good thing. Let me think about it a bit... |
I haven't had time to fully digest, but PEP about lazy imports is here https://peps.python.org/pep-0690/ I personally prefer that lazy imports are not adopted at this time, but I would like to see some more performance profiling on different platforms to understand the impact this could have. Perhaps the profiling could be done on CI for the same platforms we run tests against? |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## dev #942 +/- ##
==========================================
- Coverage 96.93% 96.93% -0.01%
==========================================
Files 32 32
Lines 9470 9469 -1
==========================================
- Hits 9180 9179 -1
Misses 290 290 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Do note that that PEP does not actually apply to what this PR does, and was eventually rejected. It only specifies a mechanism for handling how libraries can implement lazy submodules. Whereas all this code change does, is delay the import of a module until it is actually needed. > uv run python -m timeit -n 1 -r 1 "import build123d"
1 loop, best of 1: 1.14 sec per loop after: >uv run python -m timeit -n 1 -r 1 "import build123d"
1 loop, best of 1: 832 msec per loop After reading that PEP I do think build123d should adopt some of what it describes. I see quite a lot of benefit in splitting export and import of different filetypes into submodules. This could even allow users to maybe install the library without most of the dependencies it currently has. Much in a similar way to how VTK is currently somewhat optional (if you can figure out how to compile OCP without). Supporting lazy imports would also help with #924 and even #946, cutting down the amount of mandatory dependencies to Other libraries seem to support this using https://peps.python.org/pep-0562/ . I propose that the exporters and importers should get migrated to a folder structure like this: def export_step(...):
from build123d.exporters import ExportStep
exporter = ExportStep(unit, write_pcurves, precision_mode)
exporter.add_shape(to_export)
exporter.write(file_path) Hope this all makes sense - I got a bit rambly :/ |
I'm gonna go ahead and close this PR, as it seems svgpathtools also loads scipy, and that svgpathtools is used a ton inside the exporter for svg. Meaning that it's unfortunately not quite this simple to avoid loading scipy. Instead I will submit a draft PR of my suggestion above soonish :) Edit: I'm gonna hold off on submitting the draft PR. Started writing it and it is overall looking pretty good in my opinon, but it's also a lot of work, and I don't really see a need to do it, unless y'all think it's a good idea. |
Found out how to properly profile this, so I'm gonna open this back up. Sorry for all the back and forth. I definitely still think there is value in doing this change. |
Whoops, forgot pylint warns on non top-level imports
Loads IPython and scipy only when actually needed. Made build123d load about 1 second (on average) faster on my machine. Python caches modules so all in all this should only be a net benefit.
This is nice when needing to restart the jupyter kernel and having to not wait as long for the library to import.
It would be even better if we could perhaps split the exporters and importers into seperate submodules, and require the user to be explicit about importing them. This would allow ezdxf and svgpathtools to be lazily imported, allowing for about half a second speed up (theoretically).