-
Notifications
You must be signed in to change notification settings - Fork 616
Add PrimitiveProvider.on_observation
#4416
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add PrimitiveProvider.on_observation
#4416
Conversation
33094e2
to
b2be76d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
per comment below, maybe we can use Provider.observe_test_case()
instead of this new mechanism?
unrelated note: this line suggests that monkeypatching |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm, let's ship it!
return ( | ||
with_observation_callback(on_observation) | ||
if ( | ||
self.settings.backend != "hypothesis" | ||
# only for lifetime = "test_function" providers (guaranteed | ||
# by this isinstance check) | ||
and isinstance(self.provider, PrimitiveProvider) | ||
# and the provider opted-in to observations | ||
and self.provider.add_observability_callback | ||
) | ||
else nullcontext() | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems cleaner as an if-statement and two returns than a ternary expression and one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
interesting! I disagree, but only very weakly
I'm finding this useful as a "post test-case callback" in hypofuzz. I'm hoping to put things like
observation.metadata.data_status: Status
andobservation.metadata.choice_nodes
in here, and replace any manual inspection ofConjectureData
fromHypofuzzProvider
.