Skip to content

Rendering Hints extension (WIP) #879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from
Closed

Rendering Hints extension (WIP) #879

Changes from 7 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
bf87644
first draft of rendering-hints extension
cholmes Aug 18, 2020
23b931c
markdown fixes
cholmes Aug 18, 2020
4b92dfb
tweaks from PR comments
cholmes Aug 19, 2020
c9520a3
updates from latest discussions
cholmes Aug 20, 2020
be2bee9
fixed markdown
cholmes Aug 20, 2020
41f4365
fixed image
cholmes Aug 20, 2020
dd4e2be
another image fix
cholmes Aug 20, 2020
24fc4d8
Update extensions/rendering-hints/README.md
cholmes Jan 26, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions extensions/rendering-hints/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Rendering Hints Extension Specification

- **Title: Rendering Hints**
- **Identifier: rendering_hints**
- **Field Name Prefix: render**
- **Scope: Item**
- **Extension [Maturity Classification](../README.md#extension-maturity): Proposal**

One of the things many people want to do with data cataloged in STAC is to render it, often doing so directly
from the assets (usually [Cloud Optimized GeoTIFF's](http://cogeo.org)) with dynamic web tile services. To
perform well these services benefit from a few 'hints' that can help the renderer perform more effectively.

This document explains the fields of the STAC Rendering Hints (`render`) Extension to a STAC Item. They are meant
to be 'hints' - things that could be discovered by inspecting the data, but provided in STAC to help out tools
that can take advantage of them. It is written with geospatial raster data in mind, but could apply to other data types.

The field names defined herein should be added as fields in the Item Properties object. When specified on an Item,
the values are assumed to apply to all non-thumbnail Assets in that Item.

## Examples

- TODO

## Schema

- TODO

## Item Properties fields

| Field Name | Type | Description |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(throwing this here because it seems like the most appropriate place to ask about included fields)

I don't know what the appropriate scope is for this extension, but I think the main hint I would want, if it's available, is a colormap from the tif. I'm a little bit confused about what I'd do with the min and max zoom hints.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been interested in having that type of stuff in here, though I'm far from an expert on it. I think it can be in scope, and I was anticipating it'd grow to include similar things. But I'm more than happy to include it now. Just give me more details of what that actually looks like - what is the field, what's in it, how do you explain it. Or (preferred) feel free to add a 'commitable suggestion' (or whatever it is called) on the PR and I'll add it in.

Copy link
Contributor

@kylebarron kylebarron Aug 19, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little bit confused about what I'd do with the min and max zoom hints

Min and max zoom hints are helpful for on-demand rendering. The max_zoom is generally discoverable from the gsd separately, but the min_zoom depends on the number of overview levels in the file (usually a Cloud-Optimized GeoTIFF).

That gets me thinking... what if a different way to describe min/max zoom is the gsd of the full-resolution data vs the gsd of the smallest overview? the main gsd is of course already stored per band. Storing the min_overview_gsd [of each band?] would allow the dynamic tiler to derive the min/max zoom levels, and would additionally support non-mercator rendering by being able to derive those zoom levels from the gsd values

Copy link
Collaborator

@m-mohr m-mohr Aug 19, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, for visualization it's sometimes important to know the min/max values (not zoom, actual values) for the data. Sometimes that can't be well determined from the data_type. For example, normalized_differences are usually between -1 and 1, but non of the data types really caters for that.

Additionally, I would like to use some of the fields from this extension, but don't always care about zoom levels. So having that required seems to make this less useful. But I'm not exactly sure what to require. Maybe just one of the properties? (Schema: minProperties: 1) Otherwise I'd probably just fill the min/max values for zoom. They should be mentioned in the docs, I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

min_overview_gsd does seem a bit cleaner. Is there an easy way to calculate that? Would want to give people some reference of how to get it.

min/max values sounds good to add. I'm for it. What should the field name be?

And minProperties: 1 sounds good - I only knew the one use case, but if you have a use case that involves not using them then I'm happy to not make that one required.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Responded in main thread.

| ---------------- | ------------------------ | ----------- |
| render:overview_max_gsd | number | The maximum Ground Sample Distance represented in an overview. This should be the GSD of the highest level overview, generally of a [Cloud Optimized GeoTIFF](http://cogeo.org), but should work with any format. |
| render:data_type | string | The data `type` (float, int, complex, etc) to let the renderer apply any needed rescaling up front. The full set of options is listed below. |

**render:overview_max_gsd**: This field helps renderers of understand what zoom levels they can efficiently show. It is
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**render:overview_max_gsd**: This field helps renderers of understand what zoom levels they can efficiently show. It is
**render:overview_max_gsd**: This field helps renderers understand what zoom levels they can efficiently show. It is

generally used in conjunction with `gsd` (from [common metadata](https://github.com/radiantearth/stac-spec/blob/master/item-spec/common-metadata.md#instrument)). `overview_max_gsd` enables the calculuation of the 'minimum' zoom level that a renderer
would want to show, and then the maximum zoom level is calculated from the `gsd` - the resolution of the image. The former
is based on the highest level of overview (also known as a [pyramid](https://en.wikipedia.org/wiki/Pyramid_(image_processing)))
contained in the asset.

<img src="https://user-images.githubusercontent.com/407017/90821250-75ce5280-e2e7-11ea-9008-6c073e083be0.png" alt="image pyramid" width="300">

So in the above image it would be the ground sample distance of 'level 4', which will be a much higher gsd than the image,
as each pixel is greatly down-sampled. Dynamic tile servers (like [titiler](https://github.com/developmentseed/titiler)) will
generally convert the gsd to [zoom
levels](https://wiki.openstreetmap.org/wiki/Zoom_levels), [Web Mercator](https://en.wikipedia.org/wiki/Web_Mercator_projection)
or others, which is easily done (example python [to webmercator](https://github.com/cogeotiff/rio-cogeo/blob/b9b57301c2b7a4be560c887176c282e68ca63c27/rio_cogeo/utils.py#L62-L66) or arbitrary [TileMatrixSet](https://github.com/cogeotiff/rio-tiler-crs/blob/834bcf3d39cdc555b3ce930439ab186d00fd5fc5/rio_tiler_crs/cogeo.py#L98-L105))

**render:data_type** - Specifies the data storage `type` of the assets. Can be used at the asset level if the assets are of
different types. This is used to let the renderer know if any type of rescaling is needed up front. The possible values
for the type in STAC are specified in the table below.

| Type Name | Description |
|-----------|-------------|
| `unknown` | Not known |
| `int8` | 8-bit integer |
| `int16` | 16-bit integer |
| `int32` | 32-bit integer |
| `uint8` | unsigned 8-bit integer (common for 8-bit rgb png's) |
| `unit16` | unsigned 16-bit integer |
| `uint32` | unsigned 32-bit integer |
| `float32` | 32-bit float |
| `float64` | 64-big float |
| `cint16` | 16-bit complex integer |
| `cint32` | 32-bit complex integer |
| `cfloat32` | 32-bit complex float |
| `cfloat64` | 64-bit complex float |