Phase 5: Redis pub/sub for horizontal scaling

Splits the WebSocket layer into three pieces so one app instance and many
behave identically: ConnectionManager stays a purely local socket registry;
RoomBroadcaster publishes chat messages to a per-room Redis channel and
every instance (including the publisher) forwards received messages to its
own local sockets via a single psubscribe("room:*") listener started in
main.py's lifespan; Presence is a Redis-backed refcounted hash per room
tracking who's connected across all instances.

Presence replaces the old process-local connected_user_ids check that
Phase 4's offline-push logic used -- without it, a user connected on a
different instance would look offline and get a redundant push. Fixing
this was scoped in beyond the issue's literal ask (message fan-out only)
since it's a real correctness gap in a phase specifically about running
more than one instance; a known limitation (no heartbeat/TTL, so a hard
crash leaks a presence increment) is documented in the README instead of
solved here.

New tests/test_broadcast.py spins up two independent app instances sharing
one Postgres + Redis to prove delivery and presence both actually cross
the Redis boundary, not just work in-process. Manually verified the same
thing against two real uvicorn processes on different ports.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-14 07:13:06 -06:00
co-authored by Claude Sonnet 5
parent d09bf4a30a
commit 0b995ef75f
11 changed files with 328 additions and 57 deletions
+38 -15
View File
@@ -1,3 +1,4 @@
import contextlib
import os
from pathlib import Path
@@ -6,6 +7,9 @@ os.environ.setdefault(
)
os.environ.setdefault("SESSION_SECRET", "test-secret")
os.environ.setdefault("SESSION_HTTPS_ONLY", "false")
# DB index 15 keeps test presence/pub-sub state separate from whatever a
# developer's local Redis is doing on db 0.
os.environ.setdefault("REDIS_URL", "redis://localhost:6379/15")
import pytest
import pytest_asyncio
@@ -76,15 +80,15 @@ async def client(app):
@pytest.fixture
def ws_client():
def ws_client_factory():
# Starlette's TestClient (needed for websocket_connect, which httpx's
# async client doesn't support) runs the ASGI app on a background thread
# with its own event loop via anyio's BlockingPortal. asyncpg connections
# are bound to the loop they're opened on, so this app gets its own
# engine created here (no connections opened yet) rather than reusing
# the `db_session`/`app` fixtures' engine, which belongs to pytest's
# loop. No per-test rollback here (see test_ws_chat.py for the
# unique-name convention that keeps tests independent without it).
# are bound to the loop they're opened on, so each app built here gets
# its own engine (no connections opened yet) rather than reusing the
# `db_session`/`app` fixtures' engine, which belongs to pytest's loop.
# No per-test rollback here (see test_ws_chat.py for the unique-name
# convention that keeps tests independent without it).
#
# poolclass=NullPool: with pooling, a WS test that does more than one
# DB round trip per message (e.g. the offline-push lookup) can hit a
@@ -94,19 +98,38 @@ def ws_client():
# surfaces as "connection is closed". A fresh connection per session
# sidesteps it; fine for tests, not something prod needs (prod isn't
# juggling a background portal thread against the main test thread).
application = create_app()
test_engine = create_async_engine(TEST_DATABASE_URL, poolclass=NullPool)
test_session_factory = async_sessionmaker(test_engine, expire_on_commit=False)
#
# A factory (not a single client) so tests can spin up more than one
# independent app instance -- sharing the same test Postgres and Redis,
# like separate app-server processes behind Nginx would -- to exercise
# cross-instance broadcast/presence (see test_broadcast.py). Each
# TestClient is entered via an ExitStack so its lifespan (which opens
# the Redis connection/pubsub listener) starts immediately and all of
# them get torn down together at fixture teardown.
stack = contextlib.ExitStack()
async def _get_db():
async with test_session_factory() as session:
yield session
def _make() -> TestClient:
application = create_app()
test_engine = create_async_engine(TEST_DATABASE_URL, poolclass=NullPool)
test_session_factory = async_sessionmaker(test_engine, expire_on_commit=False)
application.dependency_overrides[get_db] = _get_db
async def _get_db():
async with test_session_factory() as session:
yield session
with TestClient(application) as tc:
application.dependency_overrides[get_db] = _get_db
tc = stack.enter_context(TestClient(application))
tc.session_factory = test_session_factory # type: ignore[attr-defined]
yield tc
return tc
yield _make
stack.close()
@pytest.fixture
def ws_client(ws_client_factory):
return ws_client_factory()
async def register_and_login(