Skip to content

Commit 5260f63

Browse files
lesnik512claude
andauthored
test: close deep-audit test-quality findings (#65)
* docs(planning): audit-test-quality change bundle * test(errors): cover sync Client terminal status-raising + exception mapping Add sync mirrors for every async case in test_error_mapping_terminal.py: 2xx pass-through, typed status subclasses, unknown 4xx/5xx fallbacks, 3xx no-raise, timeout/connect/invalid-url/decoding-error transport mapping, and closed-client TransportError. Uses Client(httpx2_client=httpx2.Client(...)) pattern per CLAUDE.md. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(errors): cover CookieConflict→TransportError mapping branch Add async and sync tests asserting that httpx2.CookieConflict raised inside a transport handler surfaces as TransportError (not NetworkError), covering the mapping branch that was exercised only for InvalidURL. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(errors): assert StatusError leaves do not override __init__ Parametrize over all nine StatusError leaf subclasses and assert that none defines __init__ in its own __dict__, enforcing the CLAUDE.md invariant that leaf classes must not override StatusError.__init__. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(errors): construct ForbiddenError/ConflictError/UnprocessableEntityError Extend test_per_status_subclasses_construct parametrize from 6 to all 9 leaf status subclasses, adding 403/ForbiddenError, 409/ConflictError, and 422/UnprocessableEntityError. Also smoke-tests __str__ on each. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(client): sync mirrors for status-before-decode and DecodeError-is-ClientError Add test_sync_status_error_raised_before_decoder_runs (4xx with response_model raises StatusError, not DecodeError) and test_sync_decode_error_caught_by_client_error (malformed body raises DecodeError which is-a ClientError) for the sync Client, mirroring the existing async counterparts. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(client): sync Client overload typing tests Add sync mirrors for all four AsyncClient overload tests: get/send × with/without response_model. Verifies at runtime that the sync Client's overloads resolve httpx2.Response vs typed model correctly. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(retry): correct test_retry_props description (sequential bounds, not interleaving) The module docstring implied concurrent "retry interleaving" but every test issues one sequential request per Hypothesis example. Rewrite the docstring to accurately say it tests sequential retry-policy bounds and explicitly point to test_threading_with_shared_budget.py for the concurrent case. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(bulkhead): replace flaky sleep with deterministic slot-acquisition barrier The old test used time.sleep(0.005) hoping all holder threads had acquired their bulkhead slots before issuing the over-limit requests. On slow CI a holder might not have acquired yet, causing a spurious pass of the extra request and a false test outcome. Replace the sleep with threading.Barrier(max_concurrent + 1, timeout=5.0). _BarrierHandler.wait() is called from inside each holder's handler — meaning the slot is already held — and the main thread joins the same barrier. The main thread only proceeds past the barrier once every holder has acquired its slot, making the synchronization fully deterministic. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(circuit_breaker): cover TimeoutError as a failure trigger (sync+async) The circuit breaker catches both NetworkError and TimeoutError as counted failures, but tests only covered NetworkError (via ConnectError). Add test_timeout_error_counts_as_failure to both test_circuit_breaker.py (async) and test_circuit_breaker_sync.py (sync): a handler raising httpx2.ReadTimeout (maps to httpware TimeoutError) with failure_threshold=2 causes two such requests to open the circuit, and the third raises CircuitOpenError. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(budget): pin clock so the no-purge deposit-count invariant is enforced The deposit-count assertion relied on a comment saying the 60s TTL means no _purge fires during the sub-second run. RetryBudget accepts an injectable _now clock, so inject _fixed_clock (always returns 0.0). All deposit timestamps are 0.0, the purge cutoff is 0.0 - 60.0 = -60.0, and the strict "< cutoff" predicate is never true — making the no-purge guarantee a provable invariant rather than a time-dependent assumption. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(observability): assert log record fires in the no-active-span path The test previously passed even if _emit_event became a no-op because its only assertion was "no exception raised." Strengthen it with caplog: capture at WARNING level on the test logger and assert that exactly one record fires with the correct level, message, and event attribute. This confirms the log-only fallback path actually emitted (not silently swallowed) when OTel is installed but no tracer is active. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(circuit_breaker): suppress A004 for intentional TimeoutError import shadow httpware.TimeoutError deliberately shadows the Python builtin (noqa A001 at the definition site). Tests importing it need noqa A004 at the import line. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * style: import Callable from collections.abc (ruff UP035) An auto-fix left uncommitted by the test-quality sweep; CI lint-ci (no autofix) caught the committed typing.Callable import. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
1 parent 6a7f6d1 commit 5260f63

11 files changed

Lines changed: 367 additions & 22 deletions
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
status: draft
3+
date: 2026-06-14
4+
slug: audit-test-quality
5+
supersedes: null
6+
superseded_by: null
7+
pr: null
8+
outcome: null
9+
---
10+
11+
# Change: Deep-audit test-quality findings
12+
13+
**Lane:** test-only sweep; spec is the [2026-06-14 deep audit](../../../audits/2026-06-14-deep-audit.md).
14+
15+
## Goal
16+
17+
Close the confirmed test-quality findings: assertion gaps, missing coverage,
18+
sync/async test parity, and two flaky/fragile tests. No production code changes.
19+
20+
## Findings
21+
22+
- **M3** — sync `Client._terminal` status-raising has no parallel suite (`test_error_mapping_terminal` is async-only).
23+
- **L9**`test_retry_props` docstring claims "retry interleaving" but is sequential → correct the description (concurrency is covered by `test_threading_with_shared_budget`).
24+
- **L10**`test_bulkhead_sync_props` uses a fixed `time.sleep(0.005)` → replace with a deterministic barrier.
25+
- **L11** — no test asserts `StatusError` leaves don't override `__init__` → parametrized check over the nine leaves.
26+
- **L12** — no test exercises `TimeoutError` tripping the CircuitBreaker (async + sync).
27+
- **Nit7**`test_threading_with_shared_budget` exact deposit count rests on a comment → pin the clock.
28+
- **Nit8**`ForbiddenError`/`ConflictError`/`UnprocessableEntityError` never constructed → add to the per-status parametrize.
29+
- **Nit10**`test_emit_event_works_when_otel_installed_but_no_active_span` has no assertion → assert via caplog.
30+
- **Nit11** — no sync-overload typing test for `Client` → mirror `test_client_typing`.
31+
- **Nit12** — no sync counterpart to status-before-decoder / DecodeError-is-ClientError.
32+
- **Nit13** — no test for the `httpx2.CookieConflict → TransportError` mapping branch.
33+
34+
(Nit9 — large-`attempt_index` backoff test — already landed in PR #64.)
35+
36+
## Verification
37+
38+
- [ ] Each addition is TDD-meaningful (asserts the property, not a vacuous pass).
39+
- [ ] L10/Nit7 are deterministic (no real sleeps / wall-clock assumptions).
40+
- [ ] `just test` 100% coverage; `just lint` clean.

tests/test_bulkhead_sync_props.py

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,20 @@ def test_in_flight_never_exceeds_max_concurrent(
7373
assert handler.max_in_flight <= max_concurrent
7474

7575

76+
class _BarrierHandler:
77+
"""Handler that signals slot acquisition via a threading.Barrier before holding the slot."""
78+
79+
def __init__(self, barrier: threading.Barrier) -> None:
80+
self._barrier = barrier
81+
82+
def __call__(self, request: httpx2.Request) -> httpx2.Response:
83+
# Signal that this holder has acquired a bulkhead slot and is now in-flight.
84+
self._barrier.wait(timeout=5.0)
85+
# Hold the slot long enough for the over-limit requests to be rejected.
86+
time.sleep(0.05)
87+
return httpx2.Response(HTTPStatus.OK, request=request)
88+
89+
7690
@given(
7791
max_concurrent=st.integers(min_value=1, max_value=4),
7892
extra_requests=st.integers(min_value=1, max_value=8),
@@ -82,7 +96,11 @@ def test_fail_fast_rejects_when_at_capacity(
8296
max_concurrent: int,
8397
extra_requests: int,
8498
) -> None:
85-
handler = _InFlightHandler(delay=0.05) # hold slots long enough for fail-fast to fire
99+
# Barrier: max_concurrent holders + 1 main thread — all parties meet once every
100+
# holder has acquired its bulkhead slot (i.e. is inside the handler).
101+
# timeout=5.0 sets the default for all barrier.wait() calls.
102+
acquired_barrier = threading.Barrier(max_concurrent + 1, timeout=5.0)
103+
handler = _BarrierHandler(acquired_barrier)
86104
transport = httpx2.MockTransport(handler)
87105
client = Client(
88106
httpx2_client=httpx2.Client(transport=transport),
@@ -92,8 +110,8 @@ def test_fail_fast_rejects_when_at_capacity(
92110
# Fill the bulkhead with max_concurrent long-running threads.
93111
pool = ThreadPoolExecutor(max_workers=max_concurrent + extra_requests)
94112
holders = [pool.submit(client.get, f"https://example.test/hold-{i}") for i in range(max_concurrent)]
95-
# Wait for the holders to acquire — sleep long enough for thread startup.
96-
time.sleep(0.005)
113+
# Wait deterministically — barrier releases only once ALL holders are inside the handler.
114+
acquired_barrier.wait(timeout=5.0)
97115

98116
# Any extra requests should fail fast with BulkheadFullError.
99117
for i in range(extra_requests):

tests/test_circuit_breaker.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
NotFoundError,
2222
RateLimitedError,
2323
ServiceUnavailableError,
24+
TimeoutError, # noqa: A004 — intentional: httpware.TimeoutError shadows the builtin
2425
)
2526
from httpware.middleware.resilience.circuit_breaker import AsyncCircuitBreaker
2627

@@ -169,6 +170,20 @@ def _raise(request: httpx2.Request) -> httpx2.Response: # noqa: ARG001
169170
await client.get("https://example.test/x")
170171

171172

173+
async def test_timeout_error_counts_as_failure() -> None:
174+
def _raise(request: httpx2.Request) -> httpx2.Response:
175+
msg = "read timed out"
176+
raise httpx2.ReadTimeout(msg, request=request)
177+
178+
breaker = AsyncCircuitBreaker(failure_threshold=2, _now=_Clock())
179+
async with _client(_raise, breaker=breaker) as client:
180+
for _ in range(2):
181+
with pytest.raises(TimeoutError):
182+
await client.get("https://example.test/x")
183+
with pytest.raises(CircuitOpenError):
184+
await client.get("https://example.test/x")
185+
186+
172187
async def test_custom_failure_status_codes_trips_on_member() -> None:
173188
"""A status code in a custom failure set trips the breaker (plain set accepted)."""
174189
handler = _StatusSequence([503, 503])

tests/test_circuit_breaker_sync.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
NotFoundError,
1717
RateLimitedError,
1818
ServiceUnavailableError,
19+
TimeoutError, # noqa: A004 — intentional: httpware.TimeoutError shadows the builtin
1920
)
2021
from httpware.middleware.resilience.circuit_breaker import CircuitBreaker
2122

@@ -140,6 +141,20 @@ def _raise(request: httpx2.Request) -> httpx2.Response: # noqa: ARG001
140141
client.get("https://example.test/x")
141142

142143

144+
def test_timeout_error_counts_as_failure() -> None:
145+
def _raise(request: httpx2.Request) -> httpx2.Response:
146+
msg = "read timed out"
147+
raise httpx2.ReadTimeout(msg, request=request)
148+
149+
breaker = CircuitBreaker(failure_threshold=2, _now=_Clock())
150+
with _client(_raise, breaker=breaker) as client:
151+
for _ in range(2):
152+
with pytest.raises(TimeoutError):
153+
client.get("https://example.test/x")
154+
with pytest.raises(CircuitOpenError):
155+
client.get("https://example.test/x")
156+
157+
143158
def test_custom_failure_status_codes_trips_on_member() -> None:
144159
handler = _StatusSequence([503, 503])
145160
breaker = CircuitBreaker(failure_threshold=2, failure_status_codes={503}, _now=_Clock()) # plain set accepted

tests/test_client_response_model.py

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,24 @@ async def test_async_decode_error_caught_by_client_error() -> None:
9999
assert isinstance(exc_info.value, DecodeError)
100100

101101

102+
def test_sync_status_error_raised_before_decoder_runs() -> None:
103+
def handler(request: httpx2.Request) -> httpx2.Response:
104+
return httpx2.Response(HTTPStatus.NOT_FOUND, content=b'{"id": 1, "name": "x"}', request=request)
105+
106+
transport = httpx2.MockTransport(handler)
107+
client = Client(httpx2_client=httpx2.Client(transport=transport))
108+
with pytest.raises(NotFoundError):
109+
client.get("https://example.test/u", response_model=_User)
110+
111+
112+
def test_sync_decode_error_caught_by_client_error() -> None:
113+
"""The user-facing promise: `except ClientError` catches decode failures on the sync client."""
114+
client = _sync_client_with_payload(b"null")
115+
with pytest.raises(ClientError) as exc_info:
116+
client.get("https://example.test/u", response_model=_User)
117+
assert isinstance(exc_info.value, DecodeError)
118+
119+
102120
def test_sync_schema_mismatch_raises_decode_error() -> None:
103121
client = _sync_client_with_payload(b"null")
104122
with pytest.raises(DecodeError) as exc_info:

tests/test_client_typing.py

Lines changed: 43 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Static-typing tests for AsyncClient overloads.
1+
"""Static-typing tests for AsyncClient and Client overloads.
22
33
These assert overload selection at runtime via isinstance checks. ty/mypy
44
catches the static-typing variant during `just lint`.
@@ -9,7 +9,7 @@
99
import httpx2
1010
import pydantic
1111

12-
from httpware import AsyncClient
12+
from httpware import AsyncClient, Client
1313

1414

1515
class _User(pydantic.BaseModel):
@@ -51,3 +51,44 @@ async def test_send_with_response_model_returns_typed() -> None:
5151
client = AsyncClient(httpx2_client=httpx2.AsyncClient(transport=transport))
5252
result = await client.send(httpx2.Request("GET", "https://example.test/x"), response_model=_User)
5353
assert isinstance(result, _User)
54+
55+
56+
# ---------------------------------------------------------------------------
57+
# Sync Client overload tests — mirrors of each async case above
58+
# ---------------------------------------------------------------------------
59+
60+
61+
def test_sync_get_without_response_model_returns_response() -> None:
62+
transport = httpx2.MockTransport(
63+
lambda req: httpx2.Response(HTTPStatus.OK, request=req, json={"id": 1, "name": "a"})
64+
)
65+
client = Client(httpx2_client=httpx2.Client(transport=transport))
66+
result = client.get("https://example.test/x")
67+
assert isinstance(result, httpx2.Response)
68+
69+
70+
def test_sync_get_with_response_model_returns_typed() -> None:
71+
transport = httpx2.MockTransport(
72+
lambda req: httpx2.Response(HTTPStatus.OK, request=req, json={"id": 1, "name": "a"})
73+
)
74+
client = Client(httpx2_client=httpx2.Client(transport=transport))
75+
result = client.get("https://example.test/x", response_model=_User)
76+
assert isinstance(result, _User)
77+
78+
79+
def test_sync_send_without_response_model_returns_response() -> None:
80+
transport = httpx2.MockTransport(
81+
lambda req: httpx2.Response(HTTPStatus.OK, request=req, json={"id": 1, "name": "a"})
82+
)
83+
client = Client(httpx2_client=httpx2.Client(transport=transport))
84+
result = client.send(httpx2.Request("GET", "https://example.test/x"))
85+
assert isinstance(result, httpx2.Response)
86+
87+
88+
def test_sync_send_with_response_model_returns_typed() -> None:
89+
transport = httpx2.MockTransport(
90+
lambda req: httpx2.Response(HTTPStatus.OK, request=req, json={"id": 1, "name": "a"})
91+
)
92+
client = Client(httpx2_client=httpx2.Client(transport=transport))
93+
result = client.send(httpx2.Request("GET", "https://example.test/x"), response_model=_User)
94+
assert isinstance(result, _User)

0 commit comments

Comments
 (0)