Phase 5: Redis pub/sub for horizontal scaling

Splits the WebSocket layer into three pieces so one app instance and many
behave identically: ConnectionManager stays a purely local socket registry;
RoomBroadcaster publishes chat messages to a per-room Redis channel and
every instance (including the publisher) forwards received messages to its
own local sockets via a single psubscribe("room:*") listener started in
main.py's lifespan; Presence is a Redis-backed refcounted hash per room
tracking who's connected across all instances.

Presence replaces the old process-local connected_user_ids check that
Phase 4's offline-push logic used -- without it, a user connected on a
different instance would look offline and get a redundant push. Fixing
this was scoped in beyond the issue's literal ask (message fan-out only)
since it's a real correctness gap in a phase specifically about running
more than one instance; a known limitation (no heartbeat/TTL, so a hard
crash leaks a presence increment) is documented in the README instead of
solved here.

New tests/test_broadcast.py spins up two independent app instances sharing
one Postgres + Redis to prove delivery and presence both actually cross
the Redis boundary, not just work in-process. Manually verified the same
thing against two real uvicorn processes on different ports.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-14 07:13:06 -06:00
co-authored by Claude Sonnet 5
parent d09bf4a30a
commit 0b995ef75f
11 changed files with 328 additions and 57 deletions
+6 -18
View File
@@ -5,22 +5,19 @@ from fastapi import WebSocket
class ConnectionManager:
"""In-memory, single-process WebSocket registry.
"""Local, single-process WebSocket socket registry.
Correct for a single app-server instance only; cross-instance fan-out via
Redis pub/sub is a later phase (ARCHITECTURE.md phase 5).
Purely about delivering to sockets connected to *this* process --
cross-instance fan-out lives in RoomBroadcaster, and cross-instance
"who's connected" for push lives in Presence, both backed by Redis
(ARCHITECTURE.md phase 5).
"""
def __init__(self) -> None:
self._rooms: dict[uuid.UUID, set[WebSocket]] = defaultdict(set)
# A single connection can be joined to multiple rooms at once (one
# `join` message per room over the same socket), so this is keyed on
# the socket alone, not per-room.
self._ws_user: dict[WebSocket, uuid.UUID] = {}
def join(self, room_id: uuid.UUID, websocket: WebSocket, user_id: uuid.UUID) -> None:
def join(self, room_id: uuid.UUID, websocket: WebSocket) -> None:
self._rooms[room_id].add(websocket)
self._ws_user[websocket] = user_id
def leave(self, room_id: uuid.UUID, websocket: WebSocket) -> None:
self._rooms[room_id].discard(websocket)
@@ -30,15 +27,6 @@ class ConnectionManager:
def leave_all(self, websocket: WebSocket) -> None:
for room_id in list(self._rooms.keys()):
self.leave(room_id, websocket)
self._ws_user.pop(websocket, None)
def connected_user_ids(self, room_id: uuid.UUID) -> set[uuid.UUID]:
"""Users (not just sockets) with an active connection to this room --
used to skip push notifications for anyone already watching, per
ARCHITECTURE.md's "members with no active connection" push flow."""
return {
self._ws_user[ws] for ws in self._rooms.get(room_id, ()) if ws in self._ws_user
}
async def broadcast(self, room_id: uuid.UUID, payload: dict) -> None:
for websocket in list(self._rooms.get(room_id, ())):