Skip to content

New option to include choice sequence and spans in observability output #4387

Open
@Zac-HD

Description

@Zac-HD

Based on this mailing-list question and discussion in #4381, I think we have an option to include the choice sequence and spans in observability outputs (respectively as metadata.choices and metadata.choice_spans).

  • The choice sequence can mostly be represented directly in JSON, plus {"nan": <integer value>} and {"bytes": <base64 value>}. And {"integer", <string value>} for large integers...
  • We might want to store the choice nodes though, which have the constraints on each choice? Hmm. metadata.choice_nodes if so.
  • Spans also seem useful, though it's not entirely clear how to store them - maybe a dict with keys label/start/end, and lists of integers?
    • We should probably also ship functions to transform these into various forms though, including (choice_sequence, spans) -> labelled_tree.

Finally, I'm inclined to say that this means observability is graduating from experimental to a normal supported part of Hypothesis; and we should therefore think a bit about how it's configured. We'll have options (none, basic, coverage, choices, all); might we end up adding more? If not, it's easy to make that a setting and HYPOTHESIS_OBSERVABILITY= env var.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementit's not broken, but we want it to be betterinterophow to play nicely with other packages

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions