Skip to content

Test adding metopa and metopb as placeholder drafts to stac database #426

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Jul 11, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
59e101e
testing adding metopa and metopb as placeholder drafts to stac database
shmh40 Jul 1, 2025
2492e0f
added the actual json files because I think we have to
shmh40 Jul 1, 2025
82b001c
Merge branch 'develop' into shmh40/dev/stac_test
shmh40 Jul 3, 2025
1566193
updated metopa metopb jsons and ets
shmh40 Jul 7, 2025
0c81cc7
add fy3 and update metops
shmh40 Jul 7, 2025
248cc80
updated names of metops
shmh40 Jul 7, 2025
2a7f56c
updated metopb untarred size inodes and end date
shmh40 Jul 7, 2025
841fbdb
update names to instrument, satellite
shmh40 Jul 7, 2025
3eee5bb
add untarred data size and inodes for metopa
shmh40 Jul 7, 2025
07600df
updated to oscar naming, with format platform, instrument, and added …
shmh40 Jul 8, 2025
bbc357b
update size and inodes of fy3c mwhs
shmh40 Jul 8, 2025
e65d7e5
add fengyun jsons, missing before, and update unique ids of metopa and b
shmh40 Jul 8, 2025
6935a67
add processing_level field to metopa as a test
shmh40 Jul 8, 2025
8c29a15
adding processing level field
shmh40 Jul 8, 2025
a11c1e6
fix up processing level
shmh40 Jul 8, 2025
b9bdb69
updated jsons and jsonnets for provenance
shmh40 Jul 8, 2025
56522d3
actually include provenance
shmh40 Jul 8, 2025
7471f54
Merge remote-tracking branch 'origin/develop' into shmh40/dev/stac_test
shmh40 Jul 8, 2025
89a1b3c
updated to include processor and provider, remove provenance
shmh40 Jul 8, 2025
de93b1a
add abi-goes
iluise Jul 9, 2025
80c9112
fix abi goes geometry
iluise Jul 9, 2025
1ae4456
fix latitude and longitude
iluise Jul 9, 2025
d9a2d08
fix typo
iluise Jul 9, 2025
a8586c8
hopefully this time lat is right..
iluise Jul 9, 2025
7c80a42
Merge remote-tracking branch 'origin/develop' into shmh40/dev/stac_test
shmh40 Jul 11, 2025
681e1b5
update catalogue json for develop
shmh40 Jul 11, 2025
1695a9f
check catalogue on this branch
shmh40 Jul 11, 2025
9b1c202
jsonneted for develop
shmh40 Jul 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions stac/abi-goes16.jsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
local common = import 'common.jsonnet';

{
name: 'ABI-GOES16',
filename: 'abigoes.json',
description: 'The Advanced Baseline Imager (ABI) on GOES16 (Geostationary Operational Environmental Satellite 16) provides high-resolution, multispectral imagery for real-time weather monitoring and forecasting. GOES-16 is also called GOES-East as it covers the Eastern side of the Americas.',
title: 'ABI-GOES16',
unique_id: '13',
start_datetime: '2017-12-17T00:00:41',
end_datetime: '2024-12-31 23:50:20',
frequency: '10min',
fixed_timesteps: 'True',
keywords: [
'atmosphere',
'observation',
'geostationary',
'satellite',
],
providers: [
common.providers.nasa,
common.providers.nasa_processor
],
processing_level: 'Operational L1b',

variables: {
names: [
'time_bounds_swaths',
'time_bounds_rows',
'a_h_NRTH',
'b_h_NRTH',
'Rad',
'DQF',
't',
'time_bounds',
'goes_imager_projection',
'kappa0',
'band_id',
'band_wavelength',
'min_radiance_value_of_valid_pixels',
'max_radiance_value_of_valid_pixels',
'mean_radiance_value_of_valid_pixels',
'std_dev_radiance_value_of_valid_pixels',
'esun',
'earth_sun_distance_anomaly_in_AU',
'nominal_satellite_height',
'nominal_satellite_subpoint_lat',
'nominal_satellite_subpoint_lon',
'x_image',
'y_image',
'geospatial_lat_lon_extent',
'yaw_flip_flag',
'x_image_bounds',
'y_image_bounds',
'planck_bc1',
'planck_bc2',
'planck_fk1',
'planck_fk2',
'focal_plane_temperature_threshold_decreasing',
'focal_plane_temperature_threshold_increasing',
'focal_plane_temperature_threshold_exceeded_count',
'percent_uncorrectable_L0_errors',
'saturated_pixel_count',
'undersaturated_pixel_count',
'valid_pixel_count',
'maximum_focal_plane_temperature',
'algorithm_dynamic_input_data_container',
'algorithm_product_version_container',
'reprocessing_version',
'missing_pixel_count',
'processing_parm_version_container',
't_star_look',
'star_id',
'band_wavelength_star_look',
'y',
'x',
]
},

geometry: [-156, 6, -81, 81],

dataset: {
dataset_name: 'RP_ABI-L1b-RadF-M6C*_G16_s*_e*_c*.nc',
type: 'application/vnd+netcdf',
description: 'Observation dataset',
locations: [common.hpc.hpc2020],
size: '230.8 TB',
inodes: '5.426.532',
roles: ['data'],
},
}
1 change: 1 addition & 0 deletions stac/cerra.jsonnet
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ local common = import 'common.jsonnet';
providers: [
common.providers.copernicus,
],
processing_level: 'NA',

// retrieved from anemoi dataset with:
// ds.variables, ds.statistics["minimum"], ds.statistics["maximum"],
Expand Down
15 changes: 15 additions & 0 deletions stac/common.jsonnet
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,26 @@
roles: ['provider'],
url: 'https://www.nasa.gov',
},
nasa_processor: {
name: 'NASA',
roles: ['processor'],
url: 'https://www.nasa.gov',
},
eumetsat: {
name: 'EUMETSAT',
roles: ['provider'],
url: 'https://eumetsat.int',
},
eumetsat_processor: {
name: 'EUMETSAT',
roles: ['processor'],
url: 'https://eumetsat.int',
},
cma: {
name: 'CMA',
roles: ['provider'],
url: 'https://www.cma.gov.cn/',
},
},
hpc: {
leonardo: 'leonardo',
Expand Down
2 changes: 2 additions & 0 deletions stac/era5_v8.jsonnet
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ local common = import 'common.jsonnet';
providers: [
common.providers.ecmwf_provider,
],
processing_level: 'NA',


// retrieved from anemoi dataset with:
// ds.variables, ds.statistics["minimum"], ds.statistics["maximum"],
Expand Down
1 change: 1 addition & 0 deletions stac/functions.libsonnet
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ local fill_properties(ds) = {
variables: fill_variables(ds.variables),
frequency: ds.frequency,
fixed_timesteps: ds.fixed_timesteps,
processing_level: ds.processing_level,
};

local fill_geometry(vars) = {
Expand Down
62 changes: 62 additions & 0 deletions stac/fy3a.jsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
local common = import 'common.jsonnet';

{
name: 'FY-3A, MWHS',
filename: 'fy3a.json',
description: "The data from the MWHS microwave radiometer onboard FY-3A, a Fengyun satellite. Data is available for three FY-3 satellites, FY-3A, FY-3B and FY-3C.",
title: 'FY-3A, MWHS',
unique_id: '10',
start_datetime: '2008-07-01T00:19:46',
end_datetime: '2014-05-05T00:33:45',
frequency: 'NA',
fixed_timesteps: 'False',
keywords: [
'atmosphere',
'observation',
'polar-orbiter',
'satellite',
],
providers: [
common.providers.ecmwf_host,
common.providers.cma,
common.providers.eumetsat_processor,
],
processing_level: '1C',



variables: {
names: [
'quality_pixel_bitmask',
'instrtemp',
'scnlin',
'satellite_azimuth_angle',
'satellite_zenith_angle',
'solar_azimuth_angle',
'solar_zenith_angle',
'data_quality_bitmask',
'quality_scanline_bitmask',
'time',
'warmnedt',
'coldnedt',
'btemps',
'u_independent_btemps',
'u_structured_btemps',
'u_common_btemps',
'quality_issue_pixel_bitmask',
],

},

geometry: [-180, 180, -90, 90],

dataset: {
dataset_name: 'MICROWAVE_FCDR_V1.1-20200512/FY3A/*/*.nc',
type: 'application/vnd+netcdf',
description: 'Observation dataset',
locations: [common.hpc.hpc2020],
size: '664.9 GB',
inodes: '31039',
roles: ['data'],
},
}
61 changes: 61 additions & 0 deletions stac/fy3b.jsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
local common = import 'common.jsonnet';

{
name: 'FY-3B, MWHS',
filename: 'fy3b.json',
description: "The data from the MWHS microwave radiometer onboard FY-3B, a Fengyun satellite. Data is available for three FY-3 satellites, FY-3A, FY-3B and FY-3C.",
title: 'FY-3B, MWHS',
unique_id: '11',
start_datetime: '2010-11-18T22:23:16',
end_datetime: '2018-12-31T22:29:55',
frequency: 'NA',
fixed_timesteps: 'False',
keywords: [
'atmosphere',
'observation',
'polar-orbiter',
'satellite',
],
providers: [
common.providers.ecmwf_host,
common.providers.cma,
common.providers.eumetsat_processor,
],
processing_level: '1C',


variables: {
names: [
'quality_pixel_bitmask',
'instrtemp',
'scnlin',
'satellite_azimuth_angle',
'satellite_zenith_angle',
'solar_azimuth_angle',
'solar_zenith_angle',
'data_quality_bitmask',
'quality_scanline_bitmask',
'time',
'warmnedt',
'coldnedt',
'btemps',
'u_independent_btemps',
'u_structured_btemps',
'u_common_btemps',
'quality_issue_pixel_bitmask',
],

},

geometry: [-180, 180, -90, 90],

dataset: {
dataset_name: 'MICROWAVE_FCDR_V1.1-20200512/FY3C/*/*.nc',
type: 'application/vnd+netcdf',
description: 'Observation dataset',
locations: [common.hpc.hpc2020],
size: '961.4 GB',
inodes: '44204',
roles: ['data'],
},
}
61 changes: 61 additions & 0 deletions stac/fy3c.jsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
local common = import 'common.jsonnet';

{
name: 'FY-3C, MWHS',
filename: 'fy3c.json',
description: "The data from the MWHS microwave radiometer onboard FY-3C, a Fengyun satellite. Data is available for three FY-3 satellites, FY-3A, FY-3B and FY-3C.",
title: 'FY-3C, MWHS',
unique_id: '12',
start_datetime: '2013-09-30T21:05:22',
end_datetime: '2018-12-31T19:58:58',
frequency: 'NA',
fixed_timesteps: 'False',
keywords: [
'atmosphere',
'observation',
'polar-orbiter',
'satellite',
],
providers: [
common.providers.ecmwf_host,
common.providers.cma,
common.providers.eumetsat_processor,
],
processing_level: '1C',


variables: {
names: [
'quality_pixel_bitmask',
'instrtemp',
'scnlin',
'satellite_azimuth_angle',
'satellite_zenith_angle',
'solar_azimuth_angle',
'solar_zenith_angle',
'data_quality_bitmask',
'quality_scanline_bitmask',
'time',
'warmnedt',
'coldnedt',
'btemps',
'u_independent_btemps',
'u_structured_btemps',
'u_common_btemps',
'quality_issue_pixel_bitmask',
],

},

geometry: [-180, 180, -90, 90],

dataset: {
dataset_name: 'MICROWAVE_FCDR_V1.1-20200512/FY3C/*/*.nc',
type: 'application/vnd+netcdf',
description: 'Observation dataset',
locations: [common.hpc.hpc2020],
size: '1.51 TB',
inodes: '27805',
roles: ['data'],
},
}
3 changes: 2 additions & 1 deletion stac/imerg.jsonnet
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ local common = import 'common.jsonnet';
common.providers.ecmwf_host,
common.providers.nasa,
],

processing_level: 'NA',

variables: {
names: ['tp'],
mins: [0],
Expand Down
Loading