Private
Public Access
Index Postgres search and harden SQLite search/concurrency (C5, C6)
Postgres recomputed a tsvector over every article's full text on every search. A migration now adds a stored, GIN-indexed search_vector column kept current by a trigger (so no ingest/update path can forget it), and the backend reads that column. Untested here: no Postgres in the dev sandbox, so the SQL is reviewed but not executed — worth one run of the (skipped-on-SQLite) Postgres test module against a real instance. SQLite search built one bound parameter per subscribed feed, so enough feeds would exceed the variable limit, and the OperationalError was swallowed into "no results". It now scopes through a subquery on Subscription and logs failures. SQLite connections also get WAL mode, a 20s busy timeout, and IMMEDIATE transactions so gunicorn, Celery, and the FTS triggers stop tripping over each other. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
cb0ce5a336
commit
70b0d4f277
4 files changed
+104
-21
No files matched your search
@@ -1,27 +1,28 @@
|
||||
from django.contrib.postgres.search import SearchQuery, SearchRank, SearchVector
|
||||
|
||||
from apps.articles.models import Article
|
||||
|
||||
from .base import SearchBackend
|
||||
|
||||
# Matches the stored search_vector column, GIN index, and maintaining
|
||||
# trigger created by apps/search/migrations/0002_postgres_search_vector.py
|
||||
# — the query reads that column directly instead of recomputing a
|
||||
# tsvector over every article's full text on every search (C5).
|
||||
_VECTOR = "articles_article.search_vector"
|
||||
_QUERY = "plainto_tsquery('english', %s)"
|
||||
|
||||
|
||||
class PostgresSearchBackend(SearchBackend):
|
||||
"""Computes the search vector on the fly at query time rather than
|
||||
storing/indexing it on Article — keeps the schema identical to the
|
||||
SQLite backend. Revisit with a stored+GIN-indexed column only if an
|
||||
instance's archive grows large enough to need it."""
|
||||
|
||||
def search(self, query: str, user, limit: int = 50):
|
||||
if not query.strip():
|
||||
return Article.objects.none()
|
||||
|
||||
vector = SearchVector("title", weight="A") + SearchVector("content_html", weight="B")
|
||||
search_query = SearchQuery(query)
|
||||
|
||||
return (
|
||||
Article.objects.filter(feed__subscriptions__user=user)
|
||||
.select_related("feed")
|
||||
.annotate(search=vector, rank=SearchRank(vector, search_query))
|
||||
.filter(search=search_query)
|
||||
.extra(
|
||||
select={"rank": f"ts_rank({_VECTOR}, {_QUERY})"},
|
||||
select_params=[query],
|
||||
where=[f"{_VECTOR} @@ {_QUERY}"],
|
||||
params=[query],
|
||||
)
|
||||
.order_by("-rank", "-published_at")[:limit]
|
||||
)
|
||||
Reference in new issue
Block a user