Commit Graph
4 Commits
Author SHA1 Message Date
ksmithandClaude Sonnet 5.5 d6e310b127 Make the Postgres search migration online-safe (R2-T3, #39)
One migration did a full-table UPDATE, a blocking GIN build and the
trigger install together — long locks and a deploy that could outlast
its health-check window on a large archive.

Now two stages. 0002 only adds the nullable column and the trigger
(metadata-only; new and updated rows get a vector from then on). 0003
is non-atomic: it backfills in 5,000-id batches, each its own short
transaction and only touching rows still NULL, then builds the index
with CREATE INDEX CONCURRENTLY (dropping a stale INVALID one from a
failed earlier run first). Re-running is safe, and it's a cheap no-op on
an install that applied the earlier single-step version.

Measured on a real PostgreSQL 16 with 150,000 existing articles while a
second connection kept inserting/updating: the migration took ~20s, the
slowest concurrent write was 37ms (nothing blocked), no NULL vectors
remained, the index was valid, and unapply/re-apply worked. Full suite
passes on Postgres.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 12:02:20 -07:00
ksmithandClaude Sonnet 5.5 70b0d4f277 Index Postgres search and harden SQLite search/concurrency (C5, C6)
Postgres recomputed a tsvector over every article's full text on every
search. A migration now adds a stored, GIN-indexed search_vector column
kept current by a trigger (so no ingest/update path can forget it), and
the backend reads that column. Untested here: no Postgres in the dev
sandbox, so the SQL is reviewed but not executed — worth one run of the
(skipped-on-SQLite) Postgres test module against a real instance.

SQLite search built one bound parameter per subscribed feed, so enough
feeds would exceed the variable limit, and the OperationalError was
swallowed into "no results". It now scopes through a subquery on
Subscription and logs failures. SQLite connections also get WAL mode,
a 20s busy timeout, and IMMEDIATE transactions so gunicorn, Celery, and
the FTS triggers stop tripping over each other.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 08:51:20 -07:00
ksmithandClaude Sonnet 5 8653dbeaf9 Add full-text search with per-backend SQLite/Postgres implementations
Implements the backend-agnostic search interface from the plan,
selected at runtime by connection.vendor:

- SQLite: an FTS5 virtual table (external-content, mirroring
  Article.title/content_html) created via a vendor-conditional
  migration, kept in sync by insert/update/delete triggers. Queries
  wrap the whole user input as a single FTS5 phrase so punctuation in
  a search box can't break MATCH syntax. `reindex_search` management
  command rebuilds it after bulk imports.
- Postgres: computes the search vector on the fly per the plan's
  decision (title weighted above content, no stored/indexed column)
  rather than diverging the schema from SQLite.

Both backends scope results to the requesting user's own subscriptions
and share the stream/detail article-row template and read/star/save
actions via a new apps.articles.queries.with_user_state() helper,
extracted from apps.reader.views to avoid duplicating that annotation
logic for search results.

Verified live: the SQLite path end-to-end in-browser against the real
Hacker News subscription (search UI, phrase matching, no-match case),
and the Postgres path against a throwaway Postgres container (search,
feed-scoping, and title-vs-content ranking), torn down afterward — not
part of this project's own infrastructure.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 16:37:22 -06:00
ksmithandClaude Sonnet 5 c635cd9330 Scaffold Django project and add first-run auth flow
Sets up the Evection project structure: split settings supporting both
SQLite and PostgreSQL via DATABASE_URL, Celery+Redis wiring, and the
planned app layout (accounts, feeds, articles, rules, search, reader,
pwa, core). Implements multi-user auth with a FreshRSS-style first-run
setup wizard, login/logout, and admin-only user management, covered by
automated tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 15:56:58 -06:00