Private
Public Access
One migration did a full-table UPDATE, a blocking GIN build and the trigger install together — long locks and a deploy that could outlast its health-check window on a large archive. Now two stages. 0002 only adds the nullable column and the trigger (metadata-only; new and updated rows get a vector from then on). 0003 is non-atomic: it backfills in 5,000-id batches, each its own short transaction and only touching rows still NULL, then builds the index with CREATE INDEX CONCURRENTLY (dropping a stale INVALID one from a failed earlier run first). Re-running is safe, and it's a cheap no-op on an install that applied the earlier single-step version. Measured on a real PostgreSQL 16 with 150,000 existing articles while a second connection kept inserting/updating: the migration took ~20s, the slowest concurrent write was 37ms (nothing blocked), no NULL vectors remained, the index was valid, and unapply/re-apply worked. Full suite passes on Postgres. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
64 lines
2.3 KiB
Python
64 lines
2.3 KiB
Python
"""Stored tsvector for Postgres search (C5), kept current by a trigger so
|
|
no ingestion/update code path can forget to maintain it.
|
|
|
|
Stage 1 of 2, and deliberately cheap: it only adds the (nullable) column
|
|
and installs the trigger, so new and updated rows get a vector from this
|
|
point on. Both are metadata-only/brief-lock operations even on a huge
|
|
table. The slow work — backfilling existing rows and building the GIN
|
|
index — is 0003, done in batches and CONCURRENTLY so a large existing
|
|
archive doesn't lock the table for the length of a deploy (R2-T3).
|
|
|
|
No-ops entirely on SQLite — its FTS5 table (0001_sqlite_fts) already plays
|
|
this role. The column is deliberately not a Django model field: a
|
|
SearchVectorField would break SQLite, and nothing but the search backend
|
|
ever reads it (apps/search/backends/postgres.py).
|
|
"""
|
|
|
|
from django.db import migrations
|
|
|
|
VECTOR_EXPR = (
|
|
"setweight(to_tsvector('english', coalesce({row}title, '')), 'A') || "
|
|
"setweight(to_tsvector('english', coalesce({row}content_html, '')), 'B')"
|
|
)
|
|
|
|
CREATE = f"""
|
|
ALTER TABLE articles_article ADD COLUMN IF NOT EXISTS search_vector tsvector;
|
|
|
|
CREATE OR REPLACE FUNCTION articles_article_search_vector_update() RETURNS trigger AS $$
|
|
BEGIN
|
|
NEW.search_vector := {VECTOR_EXPR.format(row="NEW.")};
|
|
RETURN NEW;
|
|
END
|
|
$$ LANGUAGE plpgsql;
|
|
|
|
DROP TRIGGER IF EXISTS articles_article_search_vector_trg ON articles_article;
|
|
CREATE TRIGGER articles_article_search_vector_trg
|
|
BEFORE INSERT OR UPDATE OF title, content_html ON articles_article
|
|
FOR EACH ROW EXECUTE FUNCTION articles_article_search_vector_update();
|
|
"""
|
|
|
|
DROP = """
|
|
DROP TRIGGER IF EXISTS articles_article_search_vector_trg ON articles_article;
|
|
DROP FUNCTION IF EXISTS articles_article_search_vector_update();
|
|
DROP INDEX IF EXISTS articles_article_search_vector_gin;
|
|
ALTER TABLE articles_article DROP COLUMN IF EXISTS search_vector;
|
|
"""
|
|
|
|
|
|
def create_vector(apps, schema_editor):
|
|
if schema_editor.connection.vendor != "postgresql":
|
|
return
|
|
schema_editor.execute(CREATE)
|
|
|
|
|
|
def drop_vector(apps, schema_editor):
|
|
if schema_editor.connection.vendor != "postgresql":
|
|
return
|
|
schema_editor.execute(DROP)
|
|
|
|
|
|
class Migration(migrations.Migration):
|
|
dependencies = [("search", "0001_sqlite_fts"), ("articles", "0001_initial")]
|
|
|
|
operations = [migrations.RunPython(create_vector, drop_vector)]
|