Presence was a per-user refcount in Redis, decremented only when a
connection's cleanup ran. Anything that ended a connection without
cleanup (a crash, an unclean restart, a laptop asleep with the page
frozen) leaked its count forever, so those users never showed offline.
Production had 19 counts for 2 live connections.
Presence is now one expiring entry per connection (90s TTL) for global,
per-room and focus presence, in new presence:v2:* keys. Clients renew
theirs by sending any frame, which the existing 30s ping/pong already
does. The server closes a human connection that sends nothing for 90s
(code 4408), which catches a frozen page whose browser still answers
transport pings. Bot connections can't be expected to answer pings, so
the server renews theirs itself while the handler is alive.
An expiry has no disconnect event to announce it, so a sweeper on each
worker (one per 15s overall, via a Redis lock) prunes expired entries
and broadcasts the offline transition. The old presence:* hashes are
deleted at startup.
Tests cover the expiry store, the sweeper, and the websocket cases.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Messages could silently stop arriving until a room switch happened to
paper over it with a REST refetch. Root cause: a connection that dies
without a clean close (laptop sleep, an idle connection recycled by a
proxy/NAT) looks identical to a healthy-but-quiet one -- readyState
stays OPEN, onclose never fires, so the already-correct reconnect and
rejoin logic never runs.
The backend now pings every 30s and the frontend force-closes the
socket if nothing (any frame, not just a pong) arrives within 65s,
routing straight through the existing reconnect/rejoin path instead of
needing separate recovery logic.
Also found and fixed a more severe adjacent bug while tracing this:
ConnectionManager.broadcast()/send_to_user() had no per-socket error
handling, so one dead or racing socket could raise inside the app's
single, never-restarted Redis pubsub listener task and silently kill
live delivery for every room and every user on that process until
restart. Now caught and logged per-socket instead of propagating.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Site-wide, any user can upload -- usable both as reactions and inline
in message text via :shortcode:, alongside the existing built-in
Unicode picker. A :shortcode: reference is stored/sent as literal
text (same as the built-in shortcode convention) and resolved to an
image at render time, so it degrades to plain text if the emoji is
later deleted.
Backend: new custom_emoji table (shortcode unique, sized to fit
MessageReaction.emoji's existing column alongside its colons), upload/
list/delete endpoints (delete restricted to uploader or site admin).
Frontend: a CustomEmojiProvider context feeds a new "Custom" category
in the emoji picker (inline upload + hover-to-remove), extends the
composer's shortcode autocomplete, and a shared EmojiGlyph resolver
renders custom emoji wherever a value can appear -- message text,
reaction pills, and the picker itself.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replaces the stateless signed-cookie session (bare user_id) with a
real server-side sessions table -- the cookie now just carries an
opaque session id, resolved against the DB on every request. Each
session records IP address (respects X-Forwarded-For), a parsed
device label, and last-seen time (throttled updates, not written on
every request).
New GET/DELETE /api/auth/sessions endpoints and an "Active sessions"
section in Profile settings let a user see every device they're
logged in from and revoke one they don't recognize -- including their
own current session, which just signs them out.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Scoped to direct messages only, and deliberately narrower than the
existing push/desktop "offline" (not connected to this room's channel
right now, which fires on every message) -- email uses GlobalPresence
instead (no open connection anywhere, or appear_offline), since the
other participant could easily just be active in a different room.
Debounced to the first unread message in the conversation rather than
firing on every message in a burst, resetting once they mark it read.
Reuses the existing SMTP/send_email infrastructure from the invite
feature, so it silently no-ops if SMTP isn't configured, same as
everywhere else that already uses it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Message.deleted_at has existed since the initial schema but was never
wired up -- no WS envelope, no permission check, no frontend concept of
it at all. Soft delete, author-only (mirrors the existing edit
permission exactly): content and any attached image/file are cleared
and the underlying MessageImage/MessageFile row and stored file are
actually removed, not just detached, so the message becomes a "This
message was deleted" tombstone with nothing left to recover through a
stale attachment URL. A deleted message can no longer be edited or
reacted to.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
is_archived was previously only exposed on the admin-only AdminRoom
schema and checked in one place (excluding a room from Browse rooms) --
for anyone already a member it was a complete no-op: still in their
sidebar, still fully postable, no indication anywhere it was archived.
Expose is_archived on the regular RoomRead/MyRoomItem schemas, drop
archived rooms from the sidebar list (while keeping them directly
reachable via URL so history stays readable), and reject new messages
in one -- both the WS "message" handler and incoming webhooks -- with a
clear "archived and read-only" response instead of silently no-op'ing
or a confusing membership error.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The sidebar shows every DM's online/offline dot at once, but the only
existing signal for a presence change (member_updated) is broadcast to
a room's own channel, which Presence only delivers to a connection that
currently has that specific room joined -- never true for a DM sitting
unopened in the sidebar. Add a dedicated per-user broadcast
(dm_presence_update) sent to each of a user's DM partners on their own
per-user channel whenever their global online/offline state changes, so
the sidebar dot updates without needing that DM to be the open room.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Desktop mode's focus gating (from #49) made losing OS focus send "leave"
for every open room, which stopped live message delivery to that room,
not just notification eligibility -- so a message wouldn't render until
the room was manually left and rejoined. Room join/leave is now gated on
visibility alone, matching the browser; notification eligibility gets its
own separate signal (a "focus"/"blur" WS frame tracked by a new
Redis-backed FocusPresence), so a connected-but-unfocused desktop member
still gets notified without losing live delivery.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two related fixes:
1. A hidden DM's un-hide-on-new-message path only cleared
RoomMembership.hidden_at in the DB -- it never told an already-open
client to refresh. The only existing signal for that room
(unread_update) does setRooms(prev => prev.map(...)), which is a
no-op for a room that isn't in `prev` at all -- exactly what a
hidden DM is. Now broadcasts the same room_added signal a brand new
DM gets (via UPDATE ... RETURNING to know exactly who was
un-hidden), reusing the fix already established for that class of
bug.
2. While debugging #1's test, found the actual root cause behind the
deploy-blocking migrations from earlier this session: every
WebSocket connection shares one AsyncSession for its entire
lifetime, and SQLAlchemy opens a transaction implicitly on first
use. Nothing ever committed it -- not the initial auth lookup, not
any of the several read-then-continue branches in the message loop
(join/message/edit/reaction all check membership this way). A
connection that's just sitting open (which for a real user can be
hours) was holding that transaction open the entire time, which is
exactly what blocked ALTER TABLE twice in production this session
(confirmed both times via pg_stat_activity -- idle in transaction
for 30+ minutes on this exact query shape). Now commits once after
connection setup and once after every frame via a try/finally
wrapping the whole dispatch, so no exit path (including the many
`continue`s) can leave a transaction open while idling on the next
receive_json().
Verified end-to-end in the browser (a hidden DM reappears in an
already-open tab with zero reload when the other person messages
again) and via a new WS-level test reproducing the exact scenario.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Shows a dot on rooms with unread messages in the sidebar, updated live
over WebSocket. Reuses the same offline-member audience computation
already used for push notifications: a member gets the real-time signal
whenever they aren't currently connected to that room's channel, which
correctly covers both "room not open" and "room open but tab
backgrounded" (the client leaves a room's channel while hidden).
Persisted server-side via a new room_memberships.last_read_at column so
state survives reload and stays consistent across devices, advanced by
an explicit mark-read call the frontend makes on room-open and on each
live message received while the room is genuinely visible -- gated on a
live visibility check, not a cached ref, so a backgrounded-but-open room
keeps accumulating unread instead of auto-marking-read the instant a
message arrives somewhere it can't be seen.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Every avatar in the app (chat messages, room member list, your own
avatar in the top bar/profile, the admin user list, the room-invite
search) now shows a green/red presence dot. Also adds a global
"Appear offline" toggle in the account menu, letting a user lurk in a
room undetected -- it overrides the real connection state everywhere,
not per-room.
Backend: new GlobalPresence (backend/app/ws/global_presence.py), a
cross-instance Redis-backed connection tracker parallel to the
existing per-room Presence, incremented/decremented on WS connect/
disconnect. A new users.appear_offline column (migration
f0f6e494454a) always wins over actual connection state when computing
displayed status. RoomMemberRead gained a computed `status` field;
add_member/change_member_role/list_room_members all compute it via a
shared _member_status() helper. Connect/disconnect and profile
updates (display_name, avatar, appear_offline) all broadcast
member_updated to every room the user belongs to, reusing the
broadcast infrastructure from the earlier avatar-staleness fix, so
chat surfaces update live with no new WS envelope type needed. A new
GET /api/users/online gives the admin list and user-search a snapshot
(deliberately not live -- see backend/app/routers/users.py) for
surfaces where "accurate as of page load" is good enough.
Frontend: UserAvatar renders an optional status dot; every call site
threads status/appear_offline through from whichever data source it
already has (room members, the current user, or the new online-ids
snapshot for admin/search).
4 new backend tests (backend/tests/test_presence.py); existing
broadcast-adjacent WS tests updated to tolerate the new member_updated
noise on connect. Verified end-to-end in the browser with two real
users: presence dot flips live on connect/disconnect via the existing
room-broadcast channel, and the lurk toggle correctly forces offline
while still connected.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Previously GET /api/rooms/mine was only ever fetched once at app mount,
so a room added mid-session stayed invisible until a full page reload
-- add_member had no way to reach an already-open client at all.
Backend: ConnectionManager and Broadcaster (renamed from RoomBroadcaster)
now support per-user channels alongside the existing per-room ones, so a
signal can reach a user's socket even for a room they haven't joined
(and by definition can't have, until this fires). add_member publishes
a room_added event on the target user's channel.
Frontend: the WebSocket connection is no longer scoped to whichever
room is open -- ChatShellPage now owns one persistent connection for
the whole session (including while no room is open, which is exactly
when this bug showed), and ChatPane joins/leaves rooms on top of it.
A room_added event triggers a room-list refetch with no reload needed.
Verified end-to-end in the browser: a user sitting on the empty room
list saw a newly-added room appear live, then chatted in it normally.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Messages can now carry an arbitrary file (MessageFile), parallel to the
existing MessageImage feature rather than a refactor of it. Files serve
with Content-Disposition: attachment to force a download and prevent an
uploaded HTML/SVG from executing same-origin. No content-type allowlist,
same 8MB cap as images for now (a separate size-limit redesign is tracked
as its own issue).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Composer gains an emoji picker (static curated unicode list, insert at
cursor position) and messages gain Slack/Mattermost-style reactions:
react with any emoji, toggle off by reacting again, see who reacted via
a tooltip on each pill.
Backend: MessageReaction model (unique on message_id+user_id+emoji backs
toggle semantics), WS "reaction" envelope broadcasts the full recomputed
reaction list per message (same approach as message edits), REST message
list embeds reactions so a reload doesn't lose state that only arrived
over WS.
Frontend: shared EmojiPicker component (anchored popover, Escape/outside-
click dismiss via new useEscapeKey hook) used by both the composer and a
new hover-revealed reaction trigger on each message row.
Images live on the app server's local disk (uploads/), served through an
authenticated, room-membership-gated endpoint since rooms can be private.
Uploads are streamed with a byte-count cap, validated as genuine decodable
images with Pillow (not just a spoofed Content-Type), and downscaled to
2000px on the longer side (except GIF, to preserve animation).
Backend: MessageImage model + nullable Message.content/image_id with a
content-or-image CheckConstraint, upload/serve endpoints in rooms.py, WS
message envelope gains image_id, push notification body says "sent an
image" for image-only messages.
Frontend: Composer gets an attach button with upload progress and a
thumbnail chip; MessageList renders images inline with a click-to-zoom
ImageLightbox.
Bot accounts (User rows with is_bot=True), scoped API tokens (read:messages,
write:messages, manage:rooms) authenticated via Authorization: Bearer on both
REST and the WS handshake, live bot WebSocket access on the same /ws/chat
endpoint humans use, message editing (WS "edit" envelope -> message_update
broadcast, fans out cross-instance for free via the existing broadcaster),
incoming webhooks (room-scoped, no auth beyond the URL token), and outgoing
webhooks/event subscriptions (HMAC-SHA256 signed, backgrounded delivery,
creation-time SSRF validation against private/loopback/link-local targets).
Token auth is additive, not a parallel system: a bearer-token-authenticated
bot goes through the exact same room-membership/role checks a session-
authenticated human does everywhere; only read:messages/write:messages are
separately scope-gated (the two message endpoints). manage:rooms scope
enforcement, full per-delivery SSRF re-validation, and bot API rate limiting
were explicitly scoped out (confirmed with the repo owner) as disproportionate
to this phase -- documented as known gaps in backend/README.md rather than
silently skipped.
Admin portal gains a Bots tab (create bots, issue/revoke scoped tokens,
cross-room webhook visibility); RoomInfoPanel gains room-scoped webhook/
subscription management, mirroring how invites already work there. The chat
UI also gets a minimal "edit your own message" affordance -- not asked for
by the issue, but the only practical way to exercise the edit pipeline by
hand instead of only via a scripted bot client.
Along the way: fixed a real bug caught while writing the incoming-webhook
test -- offline-push notification relied on the sender being "connected" to
exclude themselves, true for WS-originated messages but not for the new
webhook path, which has no WS connection for the attributed sender at all.
Now explicitly excluded. Also discovered the REST-only test fixture never
triggered ASGI lifespan, so app.state.broadcaster/presence didn't exist for
it; moved their construction out of the lifespan into create_app() itself
(Redis client construction is synchronous/lazy) so both the WS and
REST-only paths always have them.
New tests/test_bots.py, test_message_edit.py, test_webhooks.py (full suite
now 78/78, stable across repeated runs) plus a scripted end-to-end smoke
test (bot WS join/post/edit, incoming webhook, SSRF rejection, outgoing
delivery) and a full browser walkthrough of the new admin/room UI.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Splits the WebSocket layer into three pieces so one app instance and many
behave identically: ConnectionManager stays a purely local socket registry;
RoomBroadcaster publishes chat messages to a per-room Redis channel and
every instance (including the publisher) forwards received messages to its
own local sockets via a single psubscribe("room:*") listener started in
main.py's lifespan; Presence is a Redis-backed refcounted hash per room
tracking who's connected across all instances.
Presence replaces the old process-local connected_user_ids check that
Phase 4's offline-push logic used -- without it, a user connected on a
different instance would look offline and get a redundant push. Fixing
this was scoped in beyond the issue's literal ask (message fan-out only)
since it's a real correctness gap in a phase specifically about running
more than one instance; a known limitation (no heartbeat/TTL, so a hard
crash leaks a presence increment) is documented in the README instead of
solved here.
New tests/test_broadcast.py spins up two independent app instances sharing
one Postgres + Redis to prove delivery and presence both actually cross
the Redis boundary, not just work in-process. Manually verified the same
thing against two real uvicorn processes on different ports.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Backend: PushSubscription model/migration, VAPID config + `cli.py
generate-vapid-keys`, push_service.send_push_to_user (upsert-by-endpoint
subscribe/unsubscribe, auto-cleanup of expired 404/410 subscriptions),
/api/push/* router, and ConnectionManager now tracks connected user IDs
per room so chat.py can push only to offline members after broadcasting
to online ones.
Two test-infra bugs found and fixed along the way: send_push_to_user
takes the caller's AsyncSession and is awaited inline rather than fired
via asyncio.create_task with its own session (background tasks were
outliving the test event loop); and the ws_client fixture now uses
NullPool to eliminate a connection-pool checkout race that was failing
WS tests intermittently.
Frontend: service worker rebuilt with vite-plugin-pwa's injectManifest
strategy (custom src/sw.ts) so it can add push/notificationclick
handlers alongside the existing precaching and StaleWhileRevalidate
routes ported over from generateSW. New subscribe/unsubscribe flow
(lib/push.ts, api/push.ts) with a toggle in the account menu.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Invite-only FastAPI + SQLAlchemy(async) + Postgres backend (session-cookie
auth via CLI-provisioned accounts, open-room CRUD, single-instance /ws/chat)
and a React + Vite PWA frontend (login, room list, chat view). Backend tests
pass against a local Postgres DB. See README.md and backend/README.md for
setup, and ARCHITECTURE.md for the full phased design.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ksmithandClaude Sonnet 5.5