You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
OG-Core has been using pkl as the format for saving model outputs such as SS_vars.pkl, TPI_vars.pkl, and model_params.pkl.
The binary file type makes is impossible to see diffs, is not human readable, and poses more security risks.
In addition, the custom Specifications class object saved to model_params.pkl is saved with CloudPickle and cannot be saved and opened in different version of Python and so is not a good file type for long-term storage or for sharing across users/platforms.
What about using JSON for all model output?
Steady-state results saved to SS_vars.pkl are in a single dictionary with values that are scalars or Numpy arrays. This dictionary could easily be saved to JSON.
Transition path results saved to SS_vars.pkl are in a single dictionary with values that are scalars or Numpy arrays. This dictionary could easily be saved to JSON.
Model parameters saved to model_params.pkl are almost all data types that can be put into a JSON. The exception is if tax functions use the mono or mono2D type.
Should we move over to JSON? The issue with certain tax function types (that are rarely used at this point) could be solved by saving just those function to a pickle file in parameterizations of the model that use those functions.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
OG-Core has been using
pkl
as the format for saving model outputs such asSS_vars.pkl
,TPI_vars.pkl
, andmodel_params.pkl
.The binary file type makes is impossible to see diffs, is not human readable, and poses more security risks.
In addition, the custom
Specifications
class object saved tomodel_params.pkl
is saved with CloudPickle and cannot be saved and opened in different version of Python and so is not a good file type for long-term storage or for sharing across users/platforms.What about using JSON for all model output?
SS_vars.pkl
are in a single dictionary with values that are scalars or Numpy arrays. This dictionary could easily be saved to JSON.SS_vars.pkl
are in a single dictionary with values that are scalars or Numpy arrays. This dictionary could easily be saved to JSON.model_params.pkl
are almost all data types that can be put into a JSON. The exception is if tax functions use themono
ormono2D
type.Should we move over to JSON? The issue with certain tax function types (that are rarely used at this point) could be solved by saving just those function to a pickle file in parameterizations of the model that use those functions.
cc @rickecon
Beta Was this translation helpful? Give feedback.
All reactions