Unhelpful error message

[lucas-johnson]

Downloading some data from the microsoft planetary computer for a large area in northeastern US. I'm getting an error that i'm not sure how to resolve. Thanks for the package and your help!

See the reprex below in the subsequent comments.

bbox.zip

[lucas-johnson]

packageVersion('rsi')
#> [1] '0.3.0'

bbox_dl <- tempfile()
download.file('https://github.com/user-attachments/files/17284035/bbox.zip', bbox_dl)
bbox_file <- unzip(bbox_dl)
bbox <- sf::st_read(bbox_file)
#> Reading layer `bbox' from data source 
#>   `/private/tmp/Rtmp769K6P/reprex-39db373ce1a2-alive-lark/bbox.gpkg' 
#>   using driver `GPKG'
#> Simple feature collection with 1 feature and 0 fields
#> Geometry type: POLYGON
#> Dimension:     XY
#> Bounding box:  xmin: -335805.7 ymin: 1179539 xmax: 2261755 ymax: 3023828
#> Projected CRS: NAD83 / Conus Albers

test <- rsi::get_stac_data(
    # Spatial AOI:
    aoi = bbox,
    # Which asset do we want, from which collection, from which API:
    asset_names = "aws0_999",
    stac_source = "https://planetarycomputer.microsoft.com/api/stac/v1",
    collection = "gnatsgo-rasters",
    output_filename = tempfile(),
    start_date = "2019-07-01",
    end_date = "2021-07-01",
    pixel_x_size = 30,
    pixel_y_size = 30
)
#> Error:
#> ! Failed to evaluate glue component {items$features[[which_item]]$id %||% 'UNKNOWN'}
#> Caused by error in `items$features[[which_item]]`:
#> ! recursive indexing failed at level 3

^{Created on 2024-10-07 with reprex v2.1.1}

[mikemahoney218]

Do you have the rsi_pc_key environment variable set?

I'm hoping to test this later, but the failed download might be due to rate limiting. This actual error message is a logic error in the code, though...

[lucas-johnson]

> Sys.getenv('rsi_pc_key')
[1] ""

[mikemahoney218]

Well, unfortunately when testing on my machine (which does have a key set) this gets the same error. So, no silver bullet 😆

[lucas-johnson]

Rats!

[pokyah]

Hi!

I've got the same issue invoking the following:

location = "Namur Belgium"

aoi = st_as_sf(tidygeocoder::geo_osm(location), coords = c("long", "lat"), crs = 4326)

sentinel2_imagery <- get_sentinel2_imagery(
  aoi,
  stac_source = "https://planetarycomputer.microsoft.com/api/stac/v1",
  start_date = "2022-06-01",
  end_date = "2022-07-01",
  asset_names = sentinel2_band_mapping$planetary_computer_v1["B01"],
  sign_function = sign_planetary_computer,
  output_filename = tempfile(fileext = ".tif")
)

Entering in debug mode allowed me to spot the lines where the problem occured :

  function (asset) 
{
  feature_iter <- seq_len(length(items$features))
  if (length(download_locations[[asset]]) == 1) {
    feature_iter <- list(feature_iter)
  }
  tryCatch({
    future.apply::future_mapply(function(which_item, dl_location) {
      p(glue::glue("Downloading {asset}"))
      signed_items <- maybe_sign_items(items, sign_function)
      url <- rstac::assets_url(signed_items, asset)[which_item]
      if (!merge) {
        item_bbox <- items$features[[which_item]]$bbox
        current_options <- set_gdalwarp_extent(gdalwarp_options, 
          aoi, item_bbox)
      }
      sf::gdal_utils("warp", paste0("/vsicurl/", url), 
        dl_location, options = current_options, quiet = TRUE, 
        config_options = gdal_config_options)
    }, which_item = feature_iter, dl_location = download_locations[[asset]], 
      future.seed = TRUE)
  }, error = function(e) {
    rlang::warn(glue::glue("Failed to download {items$features[[i]]$id %||% 'UNKNOWN'} from {items$features[[i]]$properties$datetime %||% 'UNKNOWN'}"))
    download_locations[i, ] <- NA
  })
}

But at the moment this seems cryptic to me. Will dig further unless someone has found a solution :)

Thanks

[h-a-graham]

Okay so I've just submitted a PR that resolves the issues mentioned here: #89

@lucas-johnson It seems that the main issue is that the output raster is too large - I believe the below error is more helpful?

A good solution here might be to use composite = NULL and then wrap the result in terra::vrt or another vrt wrapper like sf::gdalutils.

packageVersion("rsi")
#> [1] '0.3.2'
bbox_dl <- tempfile()
download.file("https://github.com/user-attachments/files/17284035/bbox.zip", bbox_dl)
bbox_file <- unzip(bbox_dl)
bbox <- sf::read_sf(bbox_file)

test <- rsi::get_stac_data(
  # Spatial AOI:
  aoi = bbox,
  # Which asset do we want, from which collection, from which API:
  asset_names = "aws0_999",
  stac_source = "https://planetarycomputer.microsoft.com/api/stac/v1",
  collection = "gnatsgo-rasters",
  output_filename = tempfile(),
  start_date = "2019-07-01",
  end_date = "2021-07-01",
  pixel_x_size = 30,
  pixel_y_size = 30
)
#> Error in `...future.FUN()`:
#> ! GDAL warp failed when attempting to merge 181 items
#> Caused by error:
#> ! C stack usage  2842657010660 is too close to the limit

^{Created on 2025-01-20 with reprex v2.1.1}

@pokyah The main issue here is that aoi is a point geometry and rsi requires an extent to set the extent of the output raster. That said, this wasn't being caught so the PR adds a new check to catch this early on. With a buffer, your code works as expected.

suppressPackageStartupMessages({
  library(sf)
  library(rsi)
})
location <- "Namur Belgium"
aoi <- st_as_sf(tidygeocoder::geo(location),
  coords = c("long", "lat"), crs = 4326
)
#> Passing 1 address to the Nominatim single address geocoder
#> Query completed in: 1.1 seconds

s2_img <- get_sentinel2_imagery(
  aoi,
  stac_source = "https://planetarycomputer.microsoft.com/api/stac/v1",
  start_date = "2022-06-01",
  end_date = "2022-07-01",
  asset_names = sentinel2_band_mapping$planetary_computer_v1["B01"],
  sign_function = sign_planetary_computer,
  output_filename = tempfile(fileext = ".tif")
)
#> Error:
#> ! `aoi` has no extent.
#> ℹ This can occur when `aoi` is a single point geometry.

aoi_buff <- sf::st_transform(aoi, 3857) |>
  sf::st_buffer(1000)

s2_img_poly_aoi <- get_sentinel2_imagery(
  aoi_buff,
  stac_source = "https://planetarycomputer.microsoft.com/api/stac/v1",
  start_date = "2022-06-01",
  end_date = "2022-07-01",
  asset_names = sentinel2_band_mapping$planetary_computer_v1["B01"],
  sign_function = sign_planetary_computer,
  output_filename = tempfile(fileext = ".tif")
)

terra::rast(s2_img_poly_aoi) |> terra::plot()

^{Created on 2025-01-21 with reprex v2.1.1}

I'll wait for @mikemahoney218 to review PR but hopefully we can resolve this issue soon. ✌️

[github-actions]

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.