v.1.3.3

Release v1.3.3

New Features:

1. util_extract_items.py - STAC Date Extraction Script:

   • Purpose: Extracts all dates from items in a STAC API VHI collection and exports them to a CSV file.
   • Key Features:
     • Dynamically retrieves and processes items from a STAC collection.
     • Supports error handling and logging for robust execution.
     • Exports unique dates in YYYY-MM-DD format to a CSV file.
     • Adapted from an earlier STAC deletion script.

2. upload_all_csv_from_tools() Functionality:

   • Introduced a new function in util_upload_s3.py to upload all CSV files from the tools/ directory to Amazon S3.
   • Dynamically identifies and uploads all CSV files, improving flexibility and reducing manual intervention.
   • Provides detailed feedback on upload success or failure for individual files.

---

Enhancements:

1. util_get_acquisition_plans.py:
   • Added now S2A S2B and S2C orbits
   • Improved code readability and formatting by adding spaces and removing redundant lines.

Bug Fixes:

• Resolved minor formatting issues and improved code consistency across all modified scripts.

---

Deprecations:

• Removed redundant hardcoded upload steps for specific CSV files in util_upload_s3.py.

---

Additional Notes:

• The changes enhance the repository's capabilities for handling STAC collections and streamline interactions with Amazon S3.
• The new features and improvements ensure better maintainability, scalability, and usability.

---

What's Changed

• adding Acquistione planning and s3 push by @davidoesch in #115

Full Changelog: v1.3.2...v1.3.3

v1.3.2

Release v1.3.2

New Features

• Created reference_values.ipynb: Enables generation of full time series per region for the climate reference period. This notebook provides a reproducible workflow to extract and analyze long-term climate statistics region-wise.

Bug Fixes

• Added a check for missing orbit information in step0_processor_s2_sr: This fixes in the Sentinel-2 surface reflectance preprocessing step by ensuring graceful handling of scenes without orbit data.

What's Changed

• Dev 20250407 by @davidoesch in #114

Full Changelog: v1.3.1...v1.3.2

Release notes v1.3.1

Enhancements

• 2 Orbits a day are now supported: Enhanced support for orbit selection and mosaicking in step0_processor_s2_sr.generate_s2_sr_mosaic_for_single_date. Corrected handling of multiple Sentinel-2 orbits per day; improved cloud classification logic using both CSplus and sen2cloudless checks.
• Refactored util_upload_LST_MAX_DOY.py: updated function definitions, enhanced inline documentation, and streamlined overall structure for better readability and usability.
• Improved dataset merging in util_create_LSTMAX.py for daily data processing across the full year.

Bug Fixes

• Fixed band naming issue in lst_clima, ensuring correct handling in validation processes.
• Applied security prompt when confirming file deletions to prevent accidental data loss.

New Features

• Created util_upload_LST_MAX_DOY.py for uploading LST MAX history to GEE.
• Added util_create_LSTMAX.py to compute and export maximum land surface temperature.
• Introduced util_cdi_vhi_merge_csv.py and util_cdi_vhi_maps_hist.py to process and visualize CDI and historical VHI data.
• Added support for MSG LST CLIMA to work under all-sky conditions.
• Developed validation script for VHI and CDI data.
• Implemented VHI_HIST batch processing via config and .bat file.
• Created batch_testSUMMER.bat for testing summer LST MAX scenarios.

Documentation

• Cleaned and updated LST MAX AS documentation.
• Updated requirements.txt to include xarray for processing MCH data.

Other Changes

• Removed vfs-cache mode from satromo_publish.py to streamline GCS uploads.
• Updated util_moveassets for improved asset relocation handling.

Added PR #111: #111

What's Changed

• Dev 20250226 by @davidoesch in #112

Full Changelog: v1.3.0...v1.3.1

v1.3.0

Dev to Main Branch Merge

Overview

This PR focuses on VHI (Vegetation Health Index) processing system improvements, bug fixes, and new utilities.

Key Changes

VHI Processing

• Fixed TCI formula and topographic correction
• Added scene filtering based on valid pixel coverage
• Improved timestamp and date handling
• Standardized Landsat band naming

New Features

• VHI Map Grid Generator for monthly maps
• CDI/VHI map generation utilities
• VHI Extractor class and CSV processing
• Support for Sentinel-2C acquisition plans
• STAC asset management tools

Documentation & Fixes

• Added Jupyter notebooks for tutorials and analysis
• Fixed CMS paths and error handling
• Enhanced subprocess logging (#108)
• Fixed date handling in warnregions extraction (#107)

Contributors

• @davidoesch
• @Tschoun
• @metaodi

v1.2.5

🚀 New Features

Enhanced VHI Processing

• Added support for rerunning in step1_processor_vhi.py
  • Loads and exports existing GEE assets to GCS/drive if present
  • Eliminates the need to delete GEE assets before rerunning

Calendar Generation

• Implemented initial version for calendar generation

VHI History Initialization

• Added functionality to initialize VHI history

🔧 Improvements

Streamlined Operations

• Commented out batch VHI processing for optimized workflow: prompt for next step deactivated

🔗 Additional Information

What's Changed

• Updated step1_processor_vhi.py with rerunning support
• Implemented calendar generation functionality
• Added VHI history initialization
• Prepared codebase for release
• Dev 20241204 merge to v1.2.5 by @davidoesch in #106

Contributors

• Co-authored by Joan Sturm [email protected]

Full Changelog: v1.2.4...v1.2.5

Release Notes v1.2.4

🚀 New Features

Improved Upload Functionality

• Implemented multipart upload capability from KOGIS
• Addresses issues in #103
• Based on KOGIS implementation
• Supports upload of TIFF and JSON files

Enhanced Logging

• Added logging for single part uploads

🔧 Improvements

Streamlined Operations

• Removed verbose mode for cleaner output

🔗 Additional Information

What's Changed

• v1.2.4 by @davidoesch in #105

For a detailed list of all changes, please refer to the Full Changelog.

Release Notes v1.2.3

New Features

• Batch Scripts for VHI and Sentinel-2 Processing:
  • Added a .bat file to support batch processing for both the Vegetation Health Index (VHI) and Sentinel-2 Surface Reflectance (S2_SR) data.
  • Enables streamlined month-wise processing for faster, consistent workflows.

Enhancements

• Requirements Update:

  • Updated requirements.txt to include the latest version of the requests module for improved compatibility and security.

• Google Cloud Storage (GCS) Support for Production Environment:

  • Added support for GCS to enhance storage and accessibility of data in the production environment.
  • Addresses issue #103.

Bug Fixes

• Resolved minor bugs to improve stability and performance.

Documentation

• No new documentation changes in this release.

Contributors

• @Tschoun
• @davidoesch

What's Changed

• Dev 20241105 to v1.2.3 by @davidoesch in #104

Full Changelog: v1.2.2...v1.2.3

v1.2.2

Release Notes v1.2.2

Bug Fixes

• Resolved issue #100 in the VHI calculation process. This fix has also been applied to the production environment.

Enhancements

• Improved NDSI (Normalized Difference Snow Index) calculation:
  • NDSI is now added to all Sentinel-2 assets within the specified timeframe.
  • Masking is applied based on the NDSI of each specific date, enhancing the accuracy of snow coverage detection.

Documentation

• Updated README with latest information.
• Added VHI Data Flow documentation to provide a clearer understanding of the process.

New Features

• Introduced a batch script (batch_VHI.bat) to facilitate month-wise VHI processing.

Other Changes

• Made preparations for merging the updated code.

Contributors

• @Tschoun
• @davidoesch

For a complete list of changes, please refer to the full changelog.

v1.2.1

What's Changed

• v1.2.1 by @Tschoun in #99

New Features

Enhanced VHI Calculation

Snow Coverage Masking

• Added functionality to mask snow-covered areas and low illumination angles in VHI calculations.
• Introduced a new function calculateSnowMask to compute the Normalized Difference Snow Index (NDSI).

Illumination Angle Filtering

• Implemented masking for areas with NDSI values of 0.43 or higher, effectively filtering out snow-covered regions.
• Added masking for areas with illumination angles of 65° or higher to improve accuracy in poorly lit terrain.

Technical Details

• The new calculateSnowMask function utilizes NDSI for snow detection.
• Threshold values:
  • NDSI: ≥ 0.43 for snow coverage
  • Illumination angle: ≥ 65° for low light conditions

Full Changelog: v1.2.0...v1.2.1

v1.2.0

Release Notes for Version 1.2.0

• Merging dev-20240917 to main by @Tschoun in #98

General Updates

• Fixed the wrong collection name for LST_reference_data.

Bug Fixes and Security

• Fix for Issue #95.
  • Solved with sorting the unique_filename_day list.
• Small changes to util_moveasset.py.
• "Fix" for bug #97:
  • Updated the terrainShadowMask band to now contain values from 0-90° representing the illumination angle based on the solar position and the SRTM 30m DEM. Values above 90° have been clamped to 90.
  • Adapted the functions addTerrainShadow and addTerrainShadow_predefined to add the mask (calculated from SwissALTI3D or SwissSURFACE3D) as the value 100 to the terrainShadowMask band, which now contains the illumination angle.
  • Adapted the function addMaskedPixelCount to set the threshold for the terrainShadowMask to greater than 99.

Configuration Updates

• Updated prod_config.py.

Notes

This release includes important fixes related to the terrainShadowMask and the handling of illumination angles. Users should be aware of the changes made to the DEM sources and the adjustments in the associated functions.

Full Changelog: v1.1.0...v1.2.0