Skip to content

fix(setup-higress): surface HTTP errors instead of silent WARNINGs - #1104

Open
RerankerGuo wants to merge 1 commit into
agentscope-ai:mainfrom
RerankerGuo:fix/issue-908-higress-setup-silent-failures
Open

fix(setup-higress): surface HTTP errors instead of silent WARNINGs#1104
RerankerGuo wants to merge 1 commit into
agentscope-ai:mainfrom
RerankerGuo:fix/issue-908-higress-setup-silent-failures

Conversation

@RerankerGuo

Copy link
Copy Markdown
Contributor

Summary

Closes #908

The higress_api helper in setup-higress.sh used to swallow nearly every Console API failure as a log-level WARNING with exit 0:

  • "success":false responses → only logged WARNING, caller never noticed
  • HTTP 405 Method Not Allowed → fell through to generic WARNING
  • HTML responses only matched <!DOCTYPE html> (not <html>)
  • curl transport errors (|| true) → completely ignored
  • Callers never checked the return value anyway

This produced the exact symptom reported in #908: after a K8s token-plan hot-reload, the every-boot IDEMPOTENT section would log "OK" but workers would later fail with "Model not exist / HTTP 400" because Higress had never accepted the new routes (HTTP 405 or a session-expired HTML page being silently dropped).

Changes

manager/scripts/init/setup-higress.sh

  1. higress_api now returns real exit codes — 0 only on success, non-zero on error (curl err, HTML response, 401/403, 405, success:false, or any unexpected HTTP code)
  2. Extended HTML detection — case-insensitive match for both <!DOCTYPE html> and <html...> tags
  3. Truncated error bodies — error messages keep the first 200-500 chars of response to avoid flooding logs, but keep enough context to diagnose
  4. Added higress_api_soft wrapper — for NON-IDEMPOTENT first-boot steps where a prior boot may have already created the resource and an error is non-fatal
  5. NON-IDEMPOTENT section uses higress_api_soft — preserves the old tolerant behaviour for one-time setup, since those resources are marker-protected and may already exist on a retry
  6. IDEMPOTENT section hard-fails on higress_api error — every call appends || exit 1 so the AI Gateway / LLM provider hot-reload cannot silently produce a broken config

changelog/current.md

Added bug-fix entry referencing #908.

Test Plan

  • Shell syntax validation: bash -n setup-higress.sh passes
  • CI run on GitHub
  • Manually simulate bad Console endpoint (HTTP 405 proxy) — confirm script exits with ERROR, not a silent WARNING
  • Manually re-run on healthy system — IDEMPOTENT provider update still succeeds after re-reading env vars

The higress_api helper used to swallow nearly every Console API failure
as a log-level WARNING with exit 0: success:false responses, HTML
responses (missing </html> variant), HTTP 405 Method Not Allowed, and
curl transport errors were all silently accepted. This caused the
every-boot IDEMPOTENT token-plan reload (LLM provider + AI route
update) to appear to succeed in logs while workers later failed with
"Model not exist / HTTP 400" because Higress never received the new
configuration.

Fix:
- higress_api now returns the real exit code (non-zero on error)
- Extended HTML detection to match any <html[> tag (not only <!DOCTYPE html>)
- Added explicit HTTP 405 branch with actionable error
- success:false is now ERROR (exit 1) instead of WARNING
- Introduced higress_api_soft wrapper for first-boot NON-IDEMPOTENT steps
  where a prior run may have already created the resource
- All IDEMPOTENT callers use  so a real failure stops the
  boot sequence instead of producing a silently broken AI gateway

Closes agentscope-ai#908
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@RerankerGuo
RerankerGuo force-pushed the fix/issue-908-higress-setup-silent-failures branch from 2a8a511 to cff5d04 Compare July 30, 2026 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant