fix: recover & rotate endpoints when an RPC endpoint misbehaves

[sinzii]

Problem

With a multi-endpoint WsProvider, if the endpoint picked at connect time is unreachable/misbehaving, the client crashed with Error: [object Object] and stopped instead of retrying a different endpoint. Reported via examples/scripts/reconnection.ts.

Root cause

Endpoint rotation itself worked. The failure was one layer up: WsProvider emitted the raw WebSocket Event (not an Error) on socket errors, and BaseSubstrateClient rejected the pending connect() on the first error — killing the client while the provider was about to rotate. A few related gaps also left a client stuck on a connected-but-broken endpoint.

Changes

• Wrap socket errors in a WsConnectionError (with endpoint context) instead of emitting a raw Event.
• connect() no longer fails fast on transient provider errors; it rejects only on init errors or MaxRetryAttemptedError, letting the provider rotate.
• Escalate init failures to an endpoint switch (disconnect(true)) when retry is enabled; JsonRpcV2NotSupportedError still rejects so the v2→legacy fallback is preserved.
• ChainHead stop: bounded re-follow retries, then switch endpoint instead of dead-ending.
• Staling watchdog armed right after init (not only on the next block).
• Provider hardening: immediate retries count toward maxRetryAttempts; attempt counter resets only after a connection proves healthy (grace window / first message); array failover remembers all recently-failed endpoints; new connectTimeoutMs (default 30s) force-closes a stalled handshake so reconnection rotates onward.

Behavior changes to note

• maxRetryAttempts now also counts immediate retries (e.g. disconnect(true) switches).
• Init errors during the initial connect() no longer fail fast when retry is enabled — they rotate endpoints (bounded by maxRetryAttempts if set).

Testing

• providers 60/60, api 448/448; new unit tests cover each behavior.
• Manual repro with a bad endpoint first now logs a readable WsConnectionError, rotates to a working endpoint, and returns the correct genesis hash — no crash.

🤖 Generated with Claude Code