fix(api): add explicit connect timeout to two api/core/ bare-httpx timeouts

[Harsh23Kashyap]

Problem

Two outbound HTTP calls in api/core/ use a bare integer timeout=30 instead of a httpx.Timeout object with a separate connect cap. A bare integer sets the read timeout to 30s and lets the connect phase default to httpx's 5s, but the two phases share the same 30s budget — so a slow or unreachable endpoint can keep a worker waiting up to 30s before failing.

The adjacent api/core/rag/extractor/watercrawl/client.py:15 already has the tighter pattern:

WATERCRAWL_REQUEST_TIMEOUT: httpx.Timeout = httpx.Timeout(30.0, connect=5.0)

The two sites below are the only remaining timeout=30-style calls in api/ that don't use a httpx.Timeout object.

Motivation

#39860 (cycle 6 in this session's history) added httpx.Timeout(30.0, connect=5.0) to the class-level WaterCrawl client and the _STREAM_TIMEOUT constant. The download_result method on the same class was missed. The creators.py site was a separate, pre-existing gap that no prior cycle addressed.

Aligning both with the same httpx.Timeout(30.0, connect=5.0) shape so a slow or unreachable endpoint fails in ≤5s on the connect phase (vs up to 30s on the read+connect budget) and the read budget is bounded cleanly.

Affected sites

File:line	Current	Target
api/core/rag/extractor/watercrawl/client.py:235	httpx.get(..., timeout=30)	httpx.get(..., timeout=WATERCRAWL_REQUEST_TIMEOUT)
api/core/helper/creators.py:23	httpx.post(..., timeout=30)	httpx.post(..., timeout=httpx.Timeout(30.0, connect=5.0))

The watercrawl site reuses the existing module-level constant WATERCRAWL_REQUEST_TIMEOUT (defined at line 15 of the same file, cycle 6 PR #39860). The creators site gets a new module-level constant _CREATORS_REQUEST_TIMEOUT to match the same naming convention.

Why no existing issue covered it

Searched gh search issues for core timeout, connect timeout, outbound timeout, httpx.Timeout, download result, creators platform — 0 matches. The two sites are quiet inconsistencies in a pattern that was already addressed for the WaterCrawl crawler and the auth validation clients in cycles 6 (#39860) and 14 (#40803) of this session. This is the natural third sweep in the same family.

Proposed solution

Two-file change:

# api/core/rag/extractor/watercrawl/client.py
-def download_result(self, result_object: dict[str, Any]):
-    response = httpx.get(result_object["result"], timeout=30)
+def download_result(self, result_object: dict[str, Any]):
+    response = httpx.get(result_object["result"], timeout=WATERCRAWL_REQUEST_TIMEOUT)

# api/core/helper/creators.py
+_CREATORS_REQUEST_TIMEOUT: httpx.Timeout = httpx.Timeout(30.0, connect=5.0)
+
 def upload_dsl(dsl_file_bytes: bytes, filename: str = "template.yaml") -> str:
     url = str(creators_platform_api_url / "api/v1/templates/anonymous-upload")
-    response = httpx.post(url, files={"file": (filename, dsl_file_bytes)}, timeout=30)
+    response = httpx.post(url, files={"file": (filename, dsl_file_bytes)}, timeout=_CREATORS_REQUEST_TIMEOUT)

Plus tests:

• test_watercrawl.py::test_download_result_fetches_json_and_closes — extend to assert the timeout is a httpx.Timeout object with connect == 5.0 and read == 30.0. The existing assertion captured["timeout"] is not None is too loose to catch a regression.
• test_creators.py::test_returns_claim_code — update the call_kwargs.kwargs["timeout"] == 30 assertion to assert the new httpx.Timeout shape, and add a new test that asserts the module-level constant has connect == 5.0 and read == 30.0.

Alternatives considered

• Use httpx.Timeout(30.0) (no connect) — same as the auth fix in cycle 14 fix(api): bound connect phase to 3.0s in auth credential validation clients #40803. Rejected here because the watercrawl file already has the connect=5.0 constant established; using the existing constant is more consistent than introducing a new shape.
• Use httpx.Timeout(30.0, connect=10.0) — looser connect bound. Rejected: 5s is the existing convention in the same file.
• Add a global default in app_factory.py — bigger refactor, out of scope. The pattern in this codebase is per-module constants.

Out of scope

• The orphan api/services/auth/jina.py (tracked under a separate cleanup, mentioned in cycle 14 fix(api): bound connect phase to 3.0s in auth credential validation clients #40803 PR body).
• The api/services/auth/jina.py 179-line standalone test test_jina_auth_standalone_module.py.
• The wider PEP 3134 sweep (cycle 13 fix(api): chain original exception in 13 except blocks per PEP 3134 #40738 covered 13 sites; upstream fix: replace bare except with except Exception in core agent, storage and vector backends #40801 is the opposite direction).
• Per-PR timeouts on tools / datasource clients (different code areas, different fix shapes).

Acceptance criteria

• Both sites use httpx.Timeout(30.0, connect=5.0) (or the existing WATERCRAWL_REQUEST_TIMEOUT constant).
• The existing test_watercrawl.py::test_download_result_fetches_json_and_closes test still passes and is extended to assert the connect timeout is 5.0.
• The existing test_creators.py::test_returns_claim_code test still passes after the timeout assertion is updated.
• A new test in test_creators.py asserts the module-level _CREATORS_REQUEST_TIMEOUT constant has connect == 5.0 and read == 30.0.
• No other source files change.
• Backward compatible: callers that observe latency on unreachable endpoints will see it drop from up to 30s to ≤5s on the connect phase; success-path latency is unaffected.

Backward compatibility

The public API of WaterCrawlAPIClient.download_result and core.helper.creators.upload_dsl is unchanged. Only the internal timeout shape changes (bare int → httpx.Timeout(30.0, connect=5.0)).

Risks

The 5s connect timeout matches the existing WATERCRAWL_REQUEST_TIMEOUT constant. If creators.dify.dev has a real connect latency > 5s under normal operation, the upload will start failing. Unlikely for the Creators Platform API; the read=30s budget still bounds the total request.

[Harsh23Kashyap]

Opened PR #40807 for this. The watercrawl site reuses the existing WATERCRAWL_REQUEST_TIMEOUT constant from cycle 6 (#39860); the creators site gets a new _CREATORS_REQUEST_TIMEOUT in the same naming convention. 41 tests pass locally (--noconftest); ruff + pyrefly clean on the 2 source files and 2 test files.