Starting Core through the API while Supervisor is still booting apps produces a Core instance that cannot bind its own port. Core comes up without the http component, never answers /api/, and Supervisor waits the full startup timeout before failing.
What causes it
Since #7131 Supervisor reserves Core's host port with a transient systemd socket unit (hassio-port-reserve.socket) before booting apps, so host-network apps cannot claim it. The reservation is taken in Core.start() before the initialize phase and released only after the services phase, immediately before Supervisor starts Core itself:
core_port_reserved = await self._reserve_core_port(hosts, port)
await self.sys_apps.boot(AppStartup.INITIALIZE)
...
await self.sys_apps.boot(AppStartup.SERVICES)
if core_port_reserved:
core_port_reserved = not await self._release_core_port()
# run HomeAssistantThat ordering is correct for Supervisor's own start of Core, but the reservation is known only to Core.start() in supervisor/core.py. HomeAssistantCore.start() and HomeAssistantCore.restart() do not consult or release it, so any caller that reaches them during the reservation window starts a Core that finds its port occupied by Supervisor's own reservation unit:
POST /core/start and POST /core/restart (supervisor/api/homeassistant.py) — how this was hit in practice- backup restore (
supervisor/backups/manager.py)
- the Core container watchdog (
homeassistant/core.py), which is registered before the reservation is taken
The window lasts for the whole app boot, i.e. as long as the initialize, system and services phases take. Each phase blocks until every app in it reports started, up to STARTUP_TIMEOUT (120 s) per app, so on a system with apps that are slow to report healthy the window can be minutes rather than seconds. That is also the situation in which a user is most likely to press "Start" on a Core that looks stuck.
Observed
POST /core/start was issued 32 s into a boot, while the services phase was still waiting on app healthchecks. Core started, failed to bind 8123, and Supervisor released the reservation about 26 ms later. Supervisor's own boot flow then found the container already running and logged Skipping start of Home Assistant, so it never noticed Core was broken; the API call's _block_till_run() spun for ten minutes and raised HomeAssistantStartupTimeout.
Supervisor:
15:21:06.975 INFO [supervisor.apps.manager] Phase 'services' starting 7 apps
15:21:32.705 INFO [supervisor.docker.manager] Starting homeassistant
15:21:32.844 INFO [supervisor.homeassistant.core] Wait until Home Assistant is ready
15:21:37.401 INFO [supervisor.core] Skipping start of Home Assistant
15:22:32.885 INFO [supervisor.homeassistant.core] Still waiting for Home Assistant Core to start...
...
15:31:33.259 ERROR [supervisor.homeassistant.core] No Home Assistant Core response, assuming a fatal startup error
15:31:33.259 ERROR [supervisor.api.utils] Unexpected error during API call: No Home Assistant Core response, assuming a fatal startup error
Traceback (most recent call last):
File "/usr/src/supervisor/supervisor/api/utils.py", line 146, in wrap_api
answer = await method(*args, **kwargs)
File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 478, in start
await self._block_till_run()
File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 657, in _block_till_run
raise HomeAssistantStartupTimeout(
supervisor.exceptions.HomeAssistantStartupTimeout: No Home Assistant Core response, assuming a fatal startup error
Core:
15:21:37.375 ERROR [homeassistant.setup] Error during setup of component http: Failed to create HTTP server at port 8123: [Errno 98] error while attempting to bind on address ('::', 8123, 0, 0): [errno 98] address in use
(The recovery-mode fallback to port 80 failed as well, but for an unrelated reason — an app was listening on 80.)
Two things make the failure hard to diagnose: the reservation's release logs only at debug level, so nothing at INFO indicates the port was ever held, and the resulting Core has no http component at all rather than a visible bind error at the Supervisor level.
Possible directions
Not investigated in depth, but either or both of:
- reject or defer Core start/restart API calls while Supervisor is still booting apps
- make the reservation release part of the Core start path itself, so every caller of
HomeAssistantCore.start()/restart() releases it rather than only Core.start() in supervisor/core.py
Starting Core through the API while Supervisor is still booting apps produces a Core instance that cannot bind its own port. Core comes up without the
httpcomponent, never answers/api/, and Supervisor waits the full startup timeout before failing.What causes it
Since #7131 Supervisor reserves Core's host port with a transient systemd socket unit (
hassio-port-reserve.socket) before booting apps, so host-network apps cannot claim it. The reservation is taken inCore.start()before theinitializephase and released only after theservicesphase, immediately before Supervisor starts Core itself:That ordering is correct for Supervisor's own start of Core, but the reservation is known only to
Core.start()insupervisor/core.py.HomeAssistantCore.start()andHomeAssistantCore.restart()do not consult or release it, so any caller that reaches them during the reservation window starts a Core that finds its port occupied by Supervisor's own reservation unit:POST /core/startandPOST /core/restart(supervisor/api/homeassistant.py) — how this was hit in practicesupervisor/backups/manager.py)homeassistant/core.py), which is registered before the reservation is takenThe window lasts for the whole app boot, i.e. as long as the
initialize,systemandservicesphases take. Each phase blocks until every app in it reportsstarted, up toSTARTUP_TIMEOUT(120 s) per app, so on a system with apps that are slow to report healthy the window can be minutes rather than seconds. That is also the situation in which a user is most likely to press "Start" on a Core that looks stuck.Observed
POST /core/startwas issued 32 s into a boot, while theservicesphase was still waiting on app healthchecks. Core started, failed to bind 8123, and Supervisor released the reservation about 26 ms later. Supervisor's own boot flow then found the container already running and loggedSkipping start of Home Assistant, so it never noticed Core was broken; the API call's_block_till_run()spun for ten minutes and raisedHomeAssistantStartupTimeout.Supervisor:
Core:
(The recovery-mode fallback to port 80 failed as well, but for an unrelated reason — an app was listening on 80.)
Two things make the failure hard to diagnose: the reservation's release logs only at debug level, so nothing at INFO indicates the port was ever held, and the resulting Core has no
httpcomponent at all rather than a visible bind error at the Supervisor level.Possible directions
Not investigated in depth, but either or both of:
HomeAssistantCore.start()/restart()releases it rather than onlyCore.start()insupervisor/core.py