v2.2.4

Minor update v2.2.4

This updates some of the fetch_obsloc_tap_visits function so that the actual returned values match the docstring. Logging surrounding the use of this function has also been extended, to help with debugging.

New stacker to identify the healpix matching the boresight of a visit and adding timestamp format time values were added to maf.

New prenight simulations for lsstcam were added.

Additional small bugfixes included a stellar magnitude recarray creation fix, likely related to moving to numpy 2.

What's Changed

• tickets/SP-2115: fix return type of fetch_obsloc_tap_visits to match docstring by @ehneilsen in #458
• stellar mag fix by @yoachim in #461
• tickets/SP-2005: Add healpix and timestamp stackers in support of comparing completed to simulated visits by @ehneilsen in #460
• tickets/SP-2149: Improved logging and regularization of archive bucket in prenight sim archive code by @ehneilsen in #459
• tickets/SP-2165: Set up automatic pre-night opsim simulations for lsstcam by @ehneilsen in #462

Full Changelog: v2.2.3...v2.2.4

v2.2.3

v2.2.3 is a minor update for rubin-sim.
The major changes involve updates in the requirements specified in requirements.txt and pyproject.toml.
Some additional packages (george and scikit-learn) have been moved to "optional-requirements.txt" as they are only used for a single metric. The current packages in optional-requirements.txt include:

openorb
openorb-data-de405
george
scikit-learn
jinja2
tornado

which are used for moving object metrics, the 3x2pt FoM metric, and the show_maf webpages.
Running conda install --file optional-requirements.txt after installing rubin-sim will provide these packages if needed for a conda installation; they are also listed under 'optional' requirements in the pyproject.toml and are available via pip by running pip install "rubin-sim[extras]".

Other changes include updates to the sim_archive code used to generate simulations, including generating simulations for the next three nights (to provide 48 hour look-ahead) instead of just two nights.

An internal update to how gather_summaries runs and identifies the simulation source means that this will now work on results databases hosting summary stats from more than one simulation.

What's Changed

• SP-2099: gather_summaries reads run_name per metric from resultsDb by @rhiannonlynne in #456
• tickets/SP-2093: Change default pre-night nights from 2 to 3 by @ehneilsen in #454
• tickets/SP-2094: Fix and expand version metadata in pre-night briefing archive by @ehneilsen in #455
• SP-2104: update dependency requirements by @rhiannonlynne in #457

Full Changelog: v2.2.2...v2.2.3

v2.2.2

v2.2.2

The sim_archive module is moved from rubin_scheduler as of rubin_scheduler v3.7.0 and into rubin_sim as of v2.2.2.
This module is used to generate pre-night simulations, as well as potentially long-term simulations during operations.

Support for fetching visits from these pre-night simulations for obsloctap 24-hour lookahead is added.

Show_maf now will automatically update its tracking database information, for easier updates of the MAF metric results data.

We begin support for vector metrics to more easily and efficiently evaluate metrics over time.

Between sqlite3 and python 3.12, requirements are stricter about using string-literals in sql queries - these must be single quotes, rather than double quotes. MAF queries that are part of our codebase have been updated; users will also need to look out for errors when doing queries such as "select * from observations where scheduler_note not like 'DD%'" (note the order of the single/double quotes).

Bugfixes in the scripts to run metrics over batches allow the proper setting of "clobber" so that data is overwritten when desired rather than added onto in the resultsDb.

Build updates - import rubin_scheduler.version is no longer supported, but import rubin_scheduler.__version__ continues to work (this coincides with an update to our versions of setuptools and setuptools_scm).
And github actions workflows are updated

Many future deprecation warnings have been addressed
Even more improvements to align with the DM style guide have been implemented

What's Changed

• SP-1922: Bugfixes in scripts - set clobber properly, set order of batches by @rhiannonlynne in #441
• sqlite3 compatibility + note-scheduler_note fixes by @humnaawan in #444
• updates to align more with DM guide by @humnaawan in #443
• U/yoachim/futures by @yoachim in #442
• update SQL queries to proper quotes by @yoachim in #445
• Tickets/sp 1966 by @yoachim in #446
• Update requirements and version info by @rhiannonlynne in #447
• use fresh example database to test batches by @yoachim in #448
• update for vector metrics by @yoachim in #449
• SP-2039: Update workflows - miniforge conda and py3.12 by @rhiannonlynne in #450
• tickets/SP-2004: move opsim simulation archive submodule out of rubin_scheduler into rubin_sim by @ehneilsen in #451
• show_maf refresh when tracking database file changes by @yoachim in #453
• tickets/SP-2082: Support getting obsloctac visits from the pre-night sim archive by @ehneilsen in #452

Full Changelog: v2.2.1...v2.2.2

v2.2.1

Patch v2.2.0 release to ensure backward compatibility with numpy 1.26
Also adds compatibility with skyproj 2.0.0.

What's Changed

• SP-1906: backwards compatibility with numpy 1. by @rhiannonlynne in #439

Full Changelog: v2.2.0...v2.2.1

v2.2.0

Updates with v2.2.0 --

IMPORTANT:
OpenOrb has been removed as a required dependency. This means, if you desire to use openorb to calculate ephemeris values (the utilities are still present in rubin_sim.moving_objects) you will need to install openorb separately.
This needed to happen to allow the overall Rubin environment (rubin_env) progress to Python 3.12, as the conda-forge distribution of openorb is not available for 3.12 at the moment. Work is in progress to update this, but it has been challenging. OpenOrb should still install for python 3.11 and numpy < 2, or from pip.
Hopefully, a future update will either restore openorb as a dependency or will move to an alternate python-compatible ephemeris generator.

Utilities added to support use of MAF or predicted values, now that visits are starting to arrive in the ConsDB:

• Added DayObs and other time related stackers to support schedview relating to information about visits in the consdb -- i.e. dayObs is available as a stacker,
• Added an "Age" metric was added to show the time since the last visit at a given point to support schedview plots
• Function to calculate predicted zeropoints for any given observation are now available as functions in rubin_sim.phot_utils

Metric updates

• An error in handling the stellar luminosity function when calculating crowding effects was fixed - this means stellar counts which include crowding will change if calculated with MAF 2.2.0 compared to earlier calculations.
• The run_selfcal script is updated to send an appropriate subset of visits to the selfcal metric.
• magtype is used to set cometary magnitudes for cometary populations properly.
• Some plots that took significant time to generate are removed or simplified in the default metric outputs.
• Some issues relating to calculating SNR in the SN metrics are updated, to allow for (more) variable exposure times.
• Additional stackers to handle scheduler_note labels more easily for calculating statistics or identifying types of visits

Other work

• The survey_start_mjd function was deprecated in rubin_scheduler and has been updated appropriately here to SURVEY_START_MJD.
• scheduler_note is used by default instead of note.
• Support for building a docker container for use at usdf-maf.slac.stanford.edu has been added.

What's Changed

• swap in usual m52snr by @yoachim in #419
• tickets/PREOPS-5360: clean up the nightsum and prenight Times Square notebooks by updating schedview to support them better by @ehneilsen in #418
• U/yoachim/fo plot fix by @yoachim in #417
• Tickets/opsim 1140 by @yoachim in #416
• Add dockerfile and workflow by @rhiannonlynne in #420
• Docker Build by @rhiannonlynne in #421
• Move dockerfile to port 8080 by @rhiannonlynne in #422
• DM-45888: Remove mambaforge for miniforge by @roceb in #424
• update sim_baseline to v4.0 by @yoachim in #426
• remove survey_start function by @yoachim in #423
• SP-1601: clean up rubin-sim developer guide issues by @rhiannonlynne in #425
• Have sed.resample_sed actually resample rather than interpolate. by @yoachim in #427
• tickets/SP-1685: handle unknown filters gracefully by @ehneilsen in #428
• update selfcal by @yoachim in #430
• remove slow plots from batch by @yoachim in #431
• SP-1663: moving object magtype being used appropriately by @rhiannonlynne in #432
• add option to clip trailing index value to StringCountMetric by @yoachim in #433
• SP-1763: Add predicted zeropoint utilities by @rhiannonlynne in #435
• update sky generation by @yoachim in #434
• SP-1791: update some stats to make analyzing scheduler_note values easier by @yoachim in #436
• SP-1776 : interpret LF as cumulative and use appropriately by @rhiannonlynne in #437
• tickets/SP-1600: updates to pass ruff checks by @humnaawan in #429
• SP-1305: Remove openorb dependency by @rhiannonlynne in #438

New Contributors

• @roceb made their first contribution in #424
• @humnaawan made their first contribution in #429

Full Changelog: v2.1.0...v2.2.0

v2.1.0

v2.1.0

In this update, significant new metrics have been added:

• a photometric calibration metric (contributed by @erykoff)
• an incremental template metric (contributed by @ebellm, building on prior work by the MAF team)
• color-slope metrics were added to capture short timescale visit timings
• updates to the DDF metrics, including coadded depths per night
• Stackers that provide T_eff and dayobs values have been added, and the corresponding T_eff metrics updated

Significant changes occurred in the plotting methods:

• plotters now return the matplotlib Figure objects, instead of just the "fig.num" integer values
• a new set of plotters has been added that uses skyproj instead of healpy to create plots (this is in support of schedview, but is useful for projections beyond those provided by healpy)

Code maintenance and updates:

• Use of palpy has now been replaced by astropy - rubin-sim now drops its dependency on palpy.
• Several pandas and numpy deprecation warnings have been addressed
• Default verbosity levels have been reduced
• "get_baseline" should now be called from rubin-sim, not rubin-scheduler, and the sim_baseline is downloaded with rs_download_data from rubin-sim.

Significant documentation updates have been achieved - please see rubin-sim.lsst.io for the latest version.

rubin_sim is maintained by @rhiannonlynne, @yoachim, and @ehneilsen of the Rubin Survey Scheduling Team.
rubin_sim is an open-software project and we are grateful for the many contributions from our community.

What's Changed

• update import by @yoachim in #384
• reduce default verbosity and fix deprecation warnings by @yoachim in #385
• Switched interp2d to RectBivariateSpline by @rjassef in #386
• Add rubin-scheduler to all_req.txt by @ebellm in #388
• update rubin_sim.utils to rubin_scheduler by @yoachim in #390
• Incremental template metric by @ebellm in #389
• swap astropy for palpy by @yoachim in #391
• Tickets/opsim 1122 by @yoachim in #392
• OPSIM-1107 Documentation restructure by @rhiannonlynne in #387
• OPSIM 1132 - fix CI by @rhiannonlynne in #393
• Proper name of download testing data script by @rhiannonlynne in #394
• OPSIM-1141: Plotters return Figure objects, not fig.number by @rhiannonlynne in #400
• fix pandas dep warnings by @yoachim in #401
• PREOPS-4949: Add stackers useful for schedview (Teff and dayobs related) by @ehneilsen in #399
• Bugfix/opsim 1141 by @rhiannonlynne in #402
• tickets/PREOPS-5023: implement sky maps with skyproj in maf by @ehneilsen in #403
• Tickets/opsim 1144 by @yoachim in #404
• color and slope metrics by @yoachim in #406
• catch when astroplan sends a masked array by @yoachim in #407
• OPSIM-1149: bugfix to add cols_req to new stackers by @rhiannonlynne in #409
• OPSIM-1155: Update DDF metrics / metric_subsets by @rhiannonlynne in #411
• U/erykoff/uniformity metrics by @rhiannonlynne in #414

New Contributors

• @rjassef made their first contribution in #386

Full Changelog: v2.0.0...v2.1.0

v2.0.0

This release is a major update, as portions of this package have now been moved to rubin_scheduler. In particular:

• imports from rubin_sim.scheduler now become rubin_scheduler.scheduler
• imports from rubin_sim.utils now become rubin_scheduler.utils
• imports from rubin_sim.skybrightness_pre now become rubin_scheduler.skybrightness_pre
• imports from rubin_sim.data now mostly become imports from rubin_scheduler.data -- with the caveat that "rs_download_data" and the dictionary of data files relevant to only rubin_sim still are held in rubin_sim.data.

Downloads of data should now include
scheduler_download_data as well as rs_download_data -- these two separate steps will download the necessary data for rubin_sim.

rubin_sim is maintained by @yoachim, @rhiannonlynne, @ehneilsen of the Rubin Survey Scheduling Team.
rubin_sim is an open-software project and we are grateful for the many contributions from our community.

What's Changed

• tickets/PREOPS-4124 Support restricting the scheduler reward summary stats to a region of interest by @ehneilsen in #376
• Address future Warning by @rhiannonlynne in #379
• tickets/PREOPS-4503: replace nans in sky brightness array before IntRounded is applied by @ehneilsen in #378
• tickets/PREOPS-4501: Do not delete old SkyModelPre attributes until we know if new ones are valid by @ehneilsen in #377
• Use the same code for basis function & reward area by @ehneilsen in #380
• lingering camel-snake-pep8 updates by @yoachim in #381
• Tickets/opsim 1093 (split rubin_sim and rubin_scheduler) by @yoachim in #382

Full Changelog: 1.3.3...v2.0.0

v1.3.3

This update include a bug fix in the Skybrightness Model, so that SEDs returned by return_sed do not contain NaNs.

There is also some extension of "constant weather" options in site_models that is useful for work we're doing to build dashboards to interact with the scheduler in real-time at the telescope, as well as modifications in the "example_scheduler" code that should simplify incorporating future updates to the scheduler configurations used in simulations. This includes moving the survey_start_date provided by rubin_sim.utils.survey_start_date() to the date used in the most recent (v3.2 and v3.3) simulations and updating the sample of skybrightness_pre files provided in rubin-sim-data to match this date.

There is also an update in the DDF generation for simulations, that corrects a small bug that could arise with the DDF scheduling when the survey start date changes.

What's Changed

• Remove six by @rhiannonlynne in #369
• handle constant weather by @ehneilsen in #371
• OPSIM-1086: remove obsolete code and simplify upkeep of example_scheduler by @rhiannonlynne in #372
• Tickets/opsim 1084 by @yoachim in #373
• update mjd0 to match v3.3 sims by @yoachim in #374
• force wavelenth arrays to be idenical for twilight solar spectra by @yoachim in #375

Full Changelog: 1.3.2...1.3.3

v1.3.2

Updates with 1.3.2 -

rubin-sim /rubin_sim is now available on pypi!

There are rubin_sim_data updates as well -

• skybrightness_pre contains a smaller set of default skybrightness files, to speed up download — the full skybrightness_pre files continue to be available via rs_download_sky.
• site_models has an updated almanac, extending 20 years into the future
• scheduler data contains a dust map, so now contains all of the data files necessary to set up the example_scheduler
  Running rs_download_data —versions will confirm which of your data files should be updated to match the current version of rubin_sim.

Full Changelog: 1.3.0...1.3.2

v1.3.0

Updates in v1.3.0

Please note that there is an associated rubin_sim_data update in v1.3.0. The new data files include updates in the throughputs , skybrightness and skybrightness_pre files. To get up to date copies of these data files, please run
rs_download_data --force --dirs throughputs,skybrightness,skybrightness_pre
You can check your data versions at any time by running rs_download_data --version (and then update whichever files are necessary).

The throughputs update in v1.3.0 brings the throughput files associated with rubin_sim from the Rubin throughput from release 1.7 to 1.9. These new files correspond to as-measured mirror coatings, filter throughputs, lens coatings and move to a "triple-silver" set of mirror coatings. For more information, please check the README in the syseng_throughputs repository. The skybrightness and skybrightness_pre files are updated to match these new throughput files.

Updates in MAF:

• the dither stackers have been removed, as dithering is now implemented within the FBS. Some data columns that were used for these stackers are no longer available, which further complicated maintenance of this code.
• a bugfix in the one-d slicer makes per-night counting metrics work as expected
• an update in stackers allow rubin_sim.maf to import even without any data files available on disk (previously at least the throughput data files were required)
• a bugfix in the coadd stacker (which allows coaddition of survey visits over a night) allows it to work even if the "note" column is passed in the data to the stacker
• pandas h5 files are now compatible with the run comparison functions, instead of only csv files
• the MAF m52snr function now calculates SNR according to equation 5 in the LSST Overview paper (having the effect of correctly calculating SNR for sources with SNR greater than about 15)
• the astrometric precision function now optionally accepts a systematic noise floor value (instead of users having to include this separately in their metrics)
• a y-band-only parallax metric has been added to the standard metrics calculated for astrometry

Updates in FBS:

• the default behavior for the filter-change function is now to swap u band with y band during the lunar cycle (previous behavior was to swap u with z)
• basis functions associated with the twilight (greedy and pairs_15) survey have been updated, reducing occasional rapid filter swaps that occurred in baseline_v3.2 and previous runs
• the m5 basis function has been updated to work as-expected with the provided skybrightness_pre "dark sky" maps (this ensures that even with changes in the dark sky values, the appropriate weights per basis function are retained)
• if the conditions MJD value is updated, the entire conditions object resets its value (previously, stale values could have been kept)
• a (non-critical) warning issued when calculating skybrightness near or during twilight time has been fixed

rubin_sim is maintained by @yoachim, @rhiannonlynne, @ehneilsen, and @tribeiro
rubin_sim is an open-software project and we are grateful for the many contributions from our community.