Private
Public Access
Add WS heartbeat and harden broadcast against dead sockets (#76)
Messages could silently stop arriving until a room switch happened to paper over it with a REST refetch. Root cause: a connection that dies without a clean close (laptop sleep, an idle connection recycled by a proxy/NAT) looks identical to a healthy-but-quiet one -- readyState stays OPEN, onclose never fires, so the already-correct reconnect and rejoin logic never runs. The backend now pings every 30s and the frontend force-closes the socket if nothing (any frame, not just a pong) arrives within 65s, routing straight through the existing reconnect/rejoin path instead of needing separate recovery logic. Also found and fixed a more severe adjacent bug while tracing this: ConnectionManager.broadcast()/send_to_user() had no per-socket error handling, so one dead or racing socket could raise inside the app's single, never-restarted Redis pubsub listener task and silently kill live delivery for every room and every user on that process until restart. Now caught and logged per-socket instead of propagating. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b1c9a85ab3
commit
6f67f36578
5 files changed
+137
-16
No files matched your search
+33
-1
@@ -1,3 +1,4 @@
|
||||
import asyncio
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, Depends, WebSocket, WebSocketDisconnect
|
||||
@@ -31,6 +32,18 @@ router = APIRouter(tags=["ws"])
|
||||
|
||||
WS_UNAUTHENTICATED = 4401
|
||||
|
||||
# #76: comfortably under the ~60s idle-connection timeout common to reverse
|
||||
# proxies/load balancers sitting in front of this endpoint, and short enough
|
||||
# that the frontend's own zombie-detection watchdog (useChatSocket.ts) can
|
||||
# notice a dead connection and reconnect well within a session -- a socket
|
||||
# can go silently dead (no close frame at all, e.g. a laptop sleeping or a
|
||||
# NAT dropping an idle mapping) without this, and no live message reaches it
|
||||
# again until *something* else finally triggers a close, which used to be
|
||||
# "switch rooms and back" purely by accident (that remounts the message
|
||||
# list, which refetches history over plain REST -- unrelated to the socket
|
||||
# itself actually recovering).
|
||||
WS_PING_INTERVAL_SECONDS = 30
|
||||
|
||||
|
||||
class ClientEnvelope(BaseModel):
|
||||
type: str
|
||||
@@ -124,7 +137,18 @@ async def chat_endpoint(websocket: WebSocket, db: AsyncSession = Depends(get_db)
|
||||
|
||||
try:
|
||||
while True:
|
||||
raw = await websocket.receive_json()
|
||||
try:
|
||||
raw = await asyncio.wait_for(
|
||||
websocket.receive_json(), timeout=WS_PING_INTERVAL_SECONDS
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
# Nothing received in a while -- ping rather than assume the
|
||||
# connection's still good. A genuinely dead socket fails this
|
||||
# send (raising, caught below same as any other disconnect);
|
||||
# a merely quiet one just gets proded and the loop continues
|
||||
# waiting.
|
||||
await websocket.send_json({"type": "ping"})
|
||||
continue
|
||||
try:
|
||||
try:
|
||||
envelope = ClientEnvelope.model_validate(raw)
|
||||
@@ -146,6 +170,14 @@ async def chat_endpoint(websocket: WebSocket, db: AsyncSession = Depends(get_db)
|
||||
joined_rooms.add(envelope.room_id)
|
||||
await websocket.send_json({"type": "joined", "room_id": str(envelope.room_id)})
|
||||
|
||||
elif envelope.type == "pong":
|
||||
# #76: the client's reply to our "ping" above -- nothing
|
||||
# to do with it. Its only purpose is resetting the
|
||||
# frontend's own liveness watchdog; the timeout on the
|
||||
# receive_json() above already gets reset by receiving
|
||||
# any frame at all, this one included.
|
||||
pass
|
||||
|
||||
elif envelope.type == "leave":
|
||||
if envelope.room_id is None:
|
||||
await websocket.send_json({"type": "error", "detail": "room_id required"})
|
||||
|
||||
Reference in new issue
Block a user