Skip to content

EventHub: Don't report lost connection while server is busy - #379

Merged
iLLiCiTiT merged 1 commit into
developfrom
bugfix/websocket-ping-when-idle
Nov 4, 2026
Merged

iLLiCiTiT merged 1 commit into
developfrom
bugfix/websocket-ping-when-idle

Conversation

@iLLiCiTiT

Copy link
Copy Markdown
Member

Changelog Description

EventHub no longer reports lost connection while the server is busy but fine. When nothing arrives on the websocket for 5s the client pings the server. Connection is considered lost only when nothing, not even the pong, arrives for 15s.

Same change as ynput/ayon-core#2292, for ynput/ayon-core#2291.

Additional info

Server sends heartbeat only when nothing else was published for 2s, and other messages only to clients subscribed to their topic. So a client could receive nothing for a long time. This works with already deployed servers. Server side fix: ynput/ayon-backend#1245.

  • Pings are sent only while nothing is received, so there is no extra traffic while messages flow.
  • Pings sent by the server (uvicorn every 20s) also count as signs of life. websocket-client answers them with pong on its own.
  • Ping and pong don't report connection.opened, because they don't tell if auth succeeded. Known limitation: after a real reconnect to an unfixed server under continuous load, connection.opened is reported only once the load drops.

Testing notes:

  1. pytest tests/test_event_hub_ping.py tests/test_event_hub_lifecycle.py (no server needed).
  2. Start an event hub, run the script from Tools show 'Connection to AYON server lost' while server is busy but fine ayon-core#2291 against the server. No connection.closed event.
  3. Pause the server (e.g. docker pause <server container>). connection.closed after ~15s, connection.opened after unpause.

Server sends heartbeat only when it did not publish any other message
for 2 seconds, so while busy a client could receive nothing for longer
than the receive timeout and consider the connection lost. Ping the
server after 5 seconds without any frame and consider the connection
lost only when nothing, not even pong, arrived for 15 seconds.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@iLLiCiTiT
iLLiCiTiT merged commit a6c7d89 into develop Nov 4, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant