Examples require usage of pickled objects

[Jashcraf]

This issue is part of my JOSS review: openjournals/joss-reviews#7917

Hello, thanks for having me as a reviewer - super cool to see an optical propagation library tailored to modeling starshades!

I was running the examples to verify functionality, and encountered the following in visualize_throughput.py

PyStarshade % python examples/visualize_throughput.py
Traceback (most recent call last):
  File "/Users/kaladin/joss_reviews/PyStarshade/examples/visualize_throughput.py", line 22, in <module>
    data = np.load(file_path_on)
  File "/opt/anaconda3/envs/joss/lib/python3.10/site-packages/numpy/lib/_npyio_impl.py", line 486, in load
    raise ValueError(
ValueError: This file contains pickled (object) data. If you trust the file you can load it unsafely using the `allow_pickle=` keyword argument or `pickle.load()`.

Unless I'm mistaken, it looks like the HWO pupil models are written as binaries using the .npz extension, and that's what is being "un-pickled" here. The error above to prevent users from executing pickled data is because pickling is unsafe.

I would suggest switching the data required to use PyStarshade to another format so that users don't need to worry about pickles (what a funny world we live in). I don't have a particular preference on what PyStarshade uses, but here are some options:

• .txt or .csv by loading / saving using np.savetxt / np.loadtxt, pandas dataframes, or something similar
• .fits files, which are a commonly-used file format in astronomy / astronomical instrumentation if you didn't already know about them

[xiaziyna]

Hi Jaren! Thanks for agreeing to review PyStarshade :)

From what I can tell, if an .npz file only contains Numpy arrays, they do not get pickled. If saving dicts or objects, they get pickled.

I have checked all files in PyStarshade (as well as those in this example) load with the 'allow_pickle = False' setting in np.load, and also made sure they only contain numpy arrays. I have also tried a fresh install.

Can you tell me any other info around this error (did you create a new pupil file/did you git clone and use LFS to download the files)?
Thanks - I'm sorry about this!

[Jashcraf]

Sorry for dropping off of this - had a bit of a work sprint at an observatory and I dropped every other ball I was "juggling"

Ah I think my error was not installing the files via LFS - so np.load() was looking for a file that didn't exist and threw the pickle error. I'm currently downloading the files now and will re-run the examples to verify functionality. Will close issue once I've run all the examples.

[Jashcraf]

No longer getting the pickle issue! Closing