Open
Description
Based on this mailing-list question and discussion in #4381, I think we have an option to include the choice sequence and spans in observability outputs (respectively as metadata.choices
and metadata.choice_spans
).
- The choice sequence can mostly be represented directly in JSON, plus
{"nan": <integer value>}
and{"bytes": <base64 value>}
. And{"integer", <string value>}
for large integers... - We might want to store the choice nodes though, which have the constraints on each choice? Hmm.
metadata.choice_nodes
if so. - Spans also seem useful, though it's not entirely clear how to store them - maybe a dict with keys label/start/end, and lists of integers?
- We should probably also ship functions to transform these into various forms though, including (choice_sequence, spans) -> labelled_tree.
Finally, I'm inclined to say that this means observability is graduating from experimental to a normal supported part of Hypothesis; and we should therefore think a bit about how it's configured. We'll have options (none, basic, coverage, choices, all)
; might we end up adding more? If not, it's easy to make that a setting and HYPOTHESIS_OBSERVABILITY=
env var.