Add WS heartbeat and harden broadcast against dead sockets (#76)

Messages could silently stop arriving until a room switch happened to
paper over it with a REST refetch. Root cause: a connection that dies
without a clean close (laptop sleep, an idle connection recycled by a
proxy/NAT) looks identical to a healthy-but-quiet one -- readyState
stays OPEN, onclose never fires, so the already-correct reconnect and
rejoin logic never runs.

The backend now pings every 30s and the frontend force-closes the
socket if nothing (any frame, not just a pong) arrives within 65s,
routing straight through the existing reconnect/rejoin path instead of
needing separate recovery logic.

Also found and fixed a more severe adjacent bug while tracing this:
ConnectionManager.broadcast()/send_to_user() had no per-socket error
handling, so one dead or racing socket could raise inside the app's
single, never-restarted Redis pubsub listener task and silently kill
live delivery for every room and every user on that process until
restart. Now caught and logged per-socket instead of propagating.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
ksmithandClaude Sonnet 5 committed 2026-09-23 12:37:58 -06:00
1 parent b1c9a85ab3
commit 6f67f36578
5 files changed
+137 -16

No files matched your search

+33 -1
View File
@@ -1,3 +1,4 @@
import asyncio
import uuid
from fastapi import APIRouter, Depends, WebSocket, WebSocketDisconnect
@@ -31,6 +32,18 @@ router = APIRouter(tags=["ws"])
WS_UNAUTHENTICATED = 4401
# #76: comfortably under the ~60s idle-connection timeout common to reverse
# proxies/load balancers sitting in front of this endpoint, and short enough
# that the frontend's own zombie-detection watchdog (useChatSocket.ts) can
# notice a dead connection and reconnect well within a session -- a socket
# can go silently dead (no close frame at all, e.g. a laptop sleeping or a
# NAT dropping an idle mapping) without this, and no live message reaches it
# again until *something* else finally triggers a close, which used to be
# "switch rooms and back" purely by accident (that remounts the message
# list, which refetches history over plain REST -- unrelated to the socket
# itself actually recovering).
WS_PING_INTERVAL_SECONDS = 30
class ClientEnvelope(BaseModel):
type: str
@@ -124,7 +137,18 @@ async def chat_endpoint(websocket: WebSocket, db: AsyncSession = Depends(get_db)
try:
while True:
raw = await websocket.receive_json()
try:
raw = await asyncio.wait_for(
websocket.receive_json(), timeout=WS_PING_INTERVAL_SECONDS
)
except asyncio.TimeoutError:
# Nothing received in a while -- ping rather than assume the
# connection's still good. A genuinely dead socket fails this
# send (raising, caught below same as any other disconnect);
# a merely quiet one just gets proded and the loop continues
# waiting.
await websocket.send_json({"type": "ping"})
continue
try:
try:
envelope = ClientEnvelope.model_validate(raw)
@@ -146,6 +170,14 @@ async def chat_endpoint(websocket: WebSocket, db: AsyncSession = Depends(get_db)
joined_rooms.add(envelope.room_id)
await websocket.send_json({"type": "joined", "room_id": str(envelope.room_id)})
elif envelope.type == "pong":
# #76: the client's reply to our "ping" above -- nothing
# to do with it. Its only purpose is resetting the
# frontend's own liveness watchdog; the timeout on the
# receive_json() above already gets reset by receiving
# any frame at all, this one included.
pass
elif envelope.type == "leave":
if envelope.room_id is None:
await websocket.send_json({"type": "error", "detail": "room_id required"})