Batch feed ingestion, add a readiness check, fail open without Redis (C2, C8)

Ingestion did a get_or_create per entry. It now prefetches existing
articles for the poll in one query, bulk-inserts new ones and
bulk-updates changed ones (falling back to row-by-row only if a
concurrent fetch wins a race), and dispatches rule evaluation in
bounded chunks so a huge first fetch is several small tasks.

/health/ stays a DB-only liveness check (upgrade.sh gates on it right
after restart). New /health/ready/ also checks Redis and that
poll_due_feeds ticked within five minutes — proving beat and a worker
are both alive — and returns 503 with per-check detail otherwise.

The sidebar unread-count cache and login throttle now degrade instead
of erroring when Redis is unreachable: every authenticated page and
every login would otherwise 500 during a cache outage. Found while
verifying in a sandbox with no Redis.

Also includes the dashboard's oldest-article metric (core/views.py).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
ksmithandClaude Sonnet 5.5 committed 2026-11-01 08:51:20 -07:00
1 parent 8d2900424a
commit e5f1277014
14 files changed
+354 -48

No files matched your search

+16 -2
View File
@@ -1,3 +1,5 @@
import logging
from django.core.cache import cache
from django.db.models import Count
@@ -5,6 +7,8 @@ from apps.core.cache_utils import SIDEBAR_UNREAD_CACHE_TTL_SECONDS, sidebar_unre
from .models import SAVED_TAG_NAME, STARRED_TAG_NAME, Article, ArticleTag, UserArticleState
logger = logging.getLogger(__name__)
def with_user_state(articles, user):
"""Annotates an iterable of Article instances with is_read/is_starred/
@@ -56,7 +60,14 @@ def unread_counts_by_feed(user):
the current user's own read state, so this is rarely visibly stale.
"""
cache_key = sidebar_unread_cache_key(user.pk)
counts = cache.get(cache_key)
# The cache is an optimization here, never a dependency: every
# authenticated page render goes through this, so a Redis outage must
# degrade to recomputing the count, not to a 500 on every page.
try:
counts = cache.get(cache_key)
except Exception:
logger.warning("Sidebar cache unavailable; computing unread counts directly")
counts = None
if counts is not None:
return counts
@@ -68,5 +79,8 @@ def unread_counts_by_feed(user):
.annotate(unread=Count("id"))
)
counts = {row["feed_id"]: row["unread"] for row in rows}
cache.set(cache_key, counts, SIDEBAR_UNREAD_CACHE_TTL_SECONDS)
try:
cache.set(cache_key, counts, SIDEBAR_UNREAD_CACHE_TTL_SECONDS)
except Exception:
logger.warning("Sidebar cache unavailable; skipping cache write")
return counts