Add WS heartbeat and harden broadcast against dead sockets (#76)

Messages could silently stop arriving until a room switch happened to
paper over it with a REST refetch. Root cause: a connection that dies
without a clean close (laptop sleep, an idle connection recycled by a
proxy/NAT) looks identical to a healthy-but-quiet one -- readyState
stays OPEN, onclose never fires, so the already-correct reconnect and
rejoin logic never runs.

The backend now pings every 30s and the frontend force-closes the
socket if nothing (any frame, not just a pong) arrives within 65s,
routing straight through the existing reconnect/rejoin path instead of
needing separate recovery logic.

Also found and fixed a more severe adjacent bug while tracing this:
ConnectionManager.broadcast()/send_to_user() had no per-socket error
handling, so one dead or racing socket could raise inside the app's
single, never-restarted Redis pubsub listener task and silently kill
live delivery for every room and every user on that process until
restart. Now caught and logged per-socket instead of propagating.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
ksmithandClaude Sonnet 5 committed 2026-09-23 12:37:58 -06:00
1 parent b1c9a85ab3
commit 6f67f36578
5 files changed
+137 -16

No files matched your search

+19 -2
View File
@@ -1,8 +1,11 @@
import logging
import uuid
from collections import defaultdict
from fastapi import WebSocket
logger = logging.getLogger(__name__)
class ConnectionManager:
"""Local, single-process WebSocket socket registry.
@@ -43,8 +46,22 @@ class ConnectionManager:
async def broadcast(self, room_id: uuid.UUID, payload: dict) -> None:
for websocket in list(self._rooms.get(room_id, ())):
await websocket.send_json(payload)
# #76: one dead/racing socket (e.g. its own disconnect cleanup
# hasn't finished removing it from _rooms yet) must not take the
# rest of this broadcast down with it -- this runs inside
# Broadcaster.listen()'s single, never-restarted per-process
# pubsub loop, so an unhandled exception here previously meant
# one bad socket silently killed *live delivery for every room
# on this instance*, not just the one socket, until the process
# was restarted.
try:
await websocket.send_json(payload)
except Exception:
logger.warning("Dropping dead socket during broadcast to room %s", room_id, exc_info=True)
async def send_to_user(self, user_id: uuid.UUID, payload: dict) -> None:
for websocket in list(self._users.get(user_id, ())):
await websocket.send_json(payload)
try:
await websocket.send_json(payload)
except Exception:
logger.warning("Dropping dead socket sending to user %s", user_id, exc_info=True)