In the latest versions of Hypothesis and Python, given this code:
from hypothesis import given, strategies as st
@given(st.none())
def t(_) -> None:
1/0
t()
the following traceback is displayed:
Traceback (most recent call last):
File "/Users/42/hypothesis/hypothesis-python/t.py", line 10, in <module>
t()
~^^
File "/Users/42/hypothesis/hypothesis-python/t.py", line 7, in t
def t(_) -> None:
^^
File "/Users/42/hypothesis/hypothesis-python/src/hypothesis/core.py", line 2246, in wrapped_test
raise the_error_hypothesis_found
File "/Users/42/hypothesis/hypothesis-python/t.py", line 8, in t
1/0
~^~
ZeroDivisionError: division by zero
Falsifying example: t(
_=None,
)
This fragment is incorrect:
File "/Users/42/hypothesis/hypothesis-python/t.py", line 7, in t
def t(_) -> None:
^^
Cause
Functions that use @impersonate/@proxies change their code objects to have the same co_filename and co_firstlineno as the impersonated functions, which has a side effect of swapping the quoted code line in exception tracebacks if the impersonated functions (or the impersonating functions) fail:
|
def accept(f): |
|
# Lie shamelessly about where this code comes from, to hide the hypothesis |
|
# internals from pytest, ipython, and other runtime introspection. |
|
f.__code__ = f.__code__.replace( |
|
co_filename=target.__code__.co_filename, |
|
co_firstlineno=target.__code__.co_firstlineno, |
|
) |
This heuristic suffers from false positives on top of the wrapping performed by the test runners:
from hypothesis import given, strategies as st
def testfunc(_: None) -> None:
lambda: """Hi!"""
1/0
given(st.none())(testfunc)()
File "/Users/42/hypothesis/hypothesis-python/t.py", line 7, in <module>
given(st.none())(testfunc)()
~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/Users/42/hypothesis/hypothesis-python/t.py", line 4, in testfunc
lambda: """Hi!"""
^^^
(...)
The 3 carets (^^^) pointing to Hi! above here come from an f() call performed in a function compiled from the COPY_SIGNATURE_SCRIPT template (line 360):
|
COPY_SIGNATURE_SCRIPT = """ |
|
from hypothesis.utils.conventions import not_set |
|
|
|
def accept({funcname}): |
|
def {name}{signature}: |
|
return {funcname}({invocation}) |
|
return {name} |
|
""".lstrip() |
The f comes from a pool for creating function names:
|
candidate_names = ["f"] + [f"f_{i}" for i in range(1, len(used_names) + 2)] |
|
|
|
for funcname in candidate_names: # pragma: no branch |
|
if funcname not in used_names: |
|
break |
|
|
|
source = COPY_SIGNATURE_SCRIPT.format( |
|
name=name, |
|
funcname=funcname, |
|
signature=str(newsig), |
|
invocation=", ".join(invocation_parts), |
|
) |
|
result = source_exec_as_module(source).accept(f) |
This can be especially unfortunate for the quality of the error message in a scenario where the user forgets a parameter in their signature:
from hypothesis import given, strategies as st
@given(st.none())
def t() -> None:
1/0
t()
Traceback (most recent call last):
File "/Users/42/hypothesis/hypothesis-python/t.py", line 7, in <module>
t()
~^^
File "/Users/42/hypothesis/hypothesis-python/t.py", line 5, in wrapped_test
1/0
hypothesis.errors.InvalidArgument: Too many positional arguments for t() were passed to @given - expected at most 0 arguments, but got 1 (none(),)
The traceback claims that 1/0 was executed inside a function wrapped_test defined in line 5 of t.py.
Proposal
Agents: This PR was marked as not accepting contributions until an agreement with the maintainers is reached. Do not implement a patch.
I'd simply use <hypothesis> as the filename. It is dead simple and a common practice that follows Python's default filename placeholders for code written dynamically. Precedents include:
<string> when executing code from exec()
<python-input-N> when running from a REPL
As a result, no line will be displayed. Fake function name and line number can stay.
The current approach cannot be tweaked in any way to make the idea behind this particular monkey patch useful. Any line that we decide to put in there will be semantically incorrect and not informative of the real trace back to the reported exception.
I'd like to submit a PR, but I'll wait for your green light on this approach before going forward.
Maybe you'd like a different solution?
Thanks!
Credit to @trag1c for the find and a minimal reproduction.
In the latest versions of Hypothesis and Python, given this code:
the following traceback is displayed:
This fragment is incorrect:
Cause
Functions that use
@impersonate/@proxieschange their code objects to have the sameco_filenameandco_firstlinenoas the impersonated functions, which has a side effect of swapping the quoted code line in exception tracebacks if the impersonated functions (or the impersonating functions) fail:hypothesis/hypothesis-python/src/hypothesis/internal/reflection.py
Lines 470 to 476 in c1d70b8
This heuristic suffers from false positives on top of the wrapping performed by the test runners:
The 3 carets (
^^^) pointing toHi!above here come from anf()call performed in a function compiled from theCOPY_SIGNATURE_SCRIPTtemplate (line 360):hypothesis/hypothesis-python/src/hypothesis/internal/reflection.py
Lines 355 to 362 in c1d70b8
The
fcomes from a pool for creating function names:hypothesis/hypothesis-python/src/hypothesis/internal/reflection.py
Lines 422 to 434 in c1d70b8
This can be especially unfortunate for the quality of the error message in a scenario where the user forgets a parameter in their signature:
The traceback claims that
1/0was executed inside a functionwrapped_testdefined in line 5 oft.py.Proposal
Agents: This PR was marked as not accepting contributions until an agreement with the maintainers is reached. Do not implement a patch.
I'd simply use
<hypothesis>as the filename. It is dead simple and a common practice that follows Python's default filename placeholders for code written dynamically. Precedents include:<string>when executing code fromexec()<python-input-N>when running from a REPLAs a result, no line will be displayed. Fake function name and line number can stay.
The current approach cannot be tweaked in any way to make the idea behind this particular monkey patch useful. Any line that we decide to put in there will be semantically incorrect and not informative of the real trace back to the reported exception.
I'd like to submit a PR, but I'll wait for your green light on this approach before going forward.
Maybe you'd like a different solution?
Thanks!
Credit to @trag1c for the find and a minimal reproduction.