Private
Public Access
Make the readiness check catch fetching that is scheduled but failing (#27)
/health/ready/ proved beat and a worker were alive but not that fetching worked: every feed could be failing, or the queue could be hours behind, and it would still report ok. apps.core.health adds three checks: Celery queue depth (real Redis list length, over 1000 fails), a successful fetch within three poll intervals (30 min minimum, with a grace period for a brand-new install), and the share of active feeds failing (over half, once there are at least four). The payload also reports queue depth, active/failing feed counts, and the age of the last successful fetch. Verified against real Redis with queued Celery tasks, and the suite on PostgreSQL. docs/install.md lists each check. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
f8cd1eb33e
commit
fd494a9c08
4 files changed
+252
-11
No files matched your search
+22
-6
@@ -314,17 +314,33 @@ misconfigured feed or an outage at the source.
|
||||
|
||||
`/health/` is an unauthenticated liveness check (DB connectivity only) —
|
||||
`scripts/upgrade.sh` uses it to confirm a deploy actually came back up.
|
||||
Point an external uptime monitor at `/health/ready/` instead: it also checks
|
||||
Redis and that the background poller has ticked in the last five minutes
|
||||
(proving Celery beat is scheduling *and* a worker is consuming), and returns
|
||||
503 with a per-check breakdown when any of them fails — so "the site loads
|
||||
but nothing is being fetched" actually alerts.
|
||||
Point an external uptime monitor at `/health/ready/` instead. It returns 503
|
||||
with a per-check breakdown when any of these fail:
|
||||
|
||||
- `database` and `redis` — reachable.
|
||||
- `poll_heartbeat` — the background poller ticked in the last five minutes
|
||||
(proves Celery beat is scheduling *and* a worker is consuming).
|
||||
- `queue_depth` — no more than 1000 tasks waiting in the Celery queue
|
||||
(workers aren't keeping up, or aren't running).
|
||||
- `recent_fetch_success` — at least one feed has been fetched successfully
|
||||
within three poll intervals (minimum 30 minutes), so "scheduled but every
|
||||
fetch is failing" is caught. A brand-new install gets that long before it
|
||||
counts as stale.
|
||||
- `feed_failures` — no more than half of your active feeds are failing, once
|
||||
you have at least four (a handful of dead feeds is normal; most of them
|
||||
failing at once points at your server, DNS, or outbound network).
|
||||
|
||||
The payload also reports `queue_depth`, `active_feeds`, `failing_feeds`, and
|
||||
the age of the last successful fetch, so you can graph them.
|
||||
|
||||
**Retention.** By default every fetched article is kept forever. In the Django
|
||||
admin, "Site settings" has a *Retention days* field: set it to delete articles
|
||||
fetched longer ago than that, nightly (03:30 UTC, via Celery beat). Articles
|
||||
anyone has starred or saved for later are always kept; read state and tags for
|
||||
deleted articles go with them. The dashboard shows the article count and the
|
||||
deleted articles go with them. A small record (feed and entry id) is kept for
|
||||
each deleted article so a feed that still lists it can't bring it back as new
|
||||
and unread, and entries whose own publish date is already older than the window
|
||||
aren't fetched in the first place. The dashboard shows the article count and the
|
||||
age of the oldest one, so you can see what retention is doing.
|
||||
|
||||
**SQLite.** SQLite installs run in WAL mode with a 20-second busy timeout (set
|
||||
|
||||
Reference in new issue
Block a user