Private
Public Access
Add WS heartbeat and harden broadcast against dead sockets (#76)
Messages could silently stop arriving until a room switch happened to paper over it with a REST refetch. Root cause: a connection that dies without a clean close (laptop sleep, an idle connection recycled by a proxy/NAT) looks identical to a healthy-but-quiet one -- readyState stays OPEN, onclose never fires, so the already-correct reconnect and rejoin logic never runs. The backend now pings every 30s and the frontend force-closes the socket if nothing (any frame, not just a pong) arrives within 65s, routing straight through the existing reconnect/rejoin path instead of needing separate recovery logic. Also found and fixed a more severe adjacent bug while tracing this: ConnectionManager.broadcast()/send_to_user() had no per-socket error handling, so one dead or racing socket could raise inside the app's single, never-restarted Redis pubsub listener task and silently kill live delivery for every room and every user on that process until restart. Now caught and logged per-socket instead of propagating. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b1c9a85ab3
commit
6f67f36578
5 files changed
+137
-16
No files matched your search
@@ -1,8 +1,11 @@
|
||||
import logging
|
||||
import uuid
|
||||
from collections import defaultdict
|
||||
|
||||
from fastapi import WebSocket
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ConnectionManager:
|
||||
"""Local, single-process WebSocket socket registry.
|
||||
@@ -43,8 +46,22 @@ class ConnectionManager:
|
||||
|
||||
async def broadcast(self, room_id: uuid.UUID, payload: dict) -> None:
|
||||
for websocket in list(self._rooms.get(room_id, ())):
|
||||
await websocket.send_json(payload)
|
||||
# #76: one dead/racing socket (e.g. its own disconnect cleanup
|
||||
# hasn't finished removing it from _rooms yet) must not take the
|
||||
# rest of this broadcast down with it -- this runs inside
|
||||
# Broadcaster.listen()'s single, never-restarted per-process
|
||||
# pubsub loop, so an unhandled exception here previously meant
|
||||
# one bad socket silently killed *live delivery for every room
|
||||
# on this instance*, not just the one socket, until the process
|
||||
# was restarted.
|
||||
try:
|
||||
await websocket.send_json(payload)
|
||||
except Exception:
|
||||
logger.warning("Dropping dead socket during broadcast to room %s", room_id, exc_info=True)
|
||||
|
||||
async def send_to_user(self, user_id: uuid.UUID, payload: dict) -> None:
|
||||
for websocket in list(self._users.get(user_id, ())):
|
||||
await websocket.send_json(payload)
|
||||
try:
|
||||
await websocket.send_json(payload)
|
||||
except Exception:
|
||||
logger.warning("Dropping dead socket sending to user %s", user_id, exc_info=True)
|
||||
Reference in new issue
Block a user