Expire purge tombstones and time-box the retention purge (R2-T2, R2-T4; #38, #40)

Tombstones were kept forever, so enabling retention left a permanent row
per deleted article. They now expire after 365 days (cleaned up by the
same nightly task, even when retention is off); the documented price is
that an entry a feed still lists after a year, with no or a recent
publish date, can reappear.

The purge looped until every eligible row was gone with no run budget.
A run now works for at most four minutes, then queues a continuation, so
a huge first purge can't monopolise a worker or starve polling; hard
time limits back that up, and each chunk is its own transaction so an
interrupted run just resumes. Adds indexes on Article.fetched_at and
PurgedArticle.purged_at for the scans these do.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
ksmithandClaude Sonnet 5.5 committed 2026-11-01 12:00:52 -07:00
1 parent 2e1c0e31a8
commit 72bffbe97f
4 files changed
+153 -9

No files matched your search

+6 -1
View File
@@ -25,7 +25,11 @@ class Article(models.Model):
class Meta:
unique_together = [("feed", "guid")]
indexes = [models.Index(fields=["feed", "published_at"])]
indexes = [
models.Index(fields=["feed", "published_at"]),
# The retention purge scans for the oldest articles by this.
models.Index(fields=["fetched_at"]),
]
ordering = ["-published_at", "-fetched_at"]
def __str__(self):
@@ -47,6 +51,7 @@ class PurgedArticle(models.Model):
class Meta:
unique_together = [("feed", "guid")]
indexes = [models.Index(fields=["purged_at"])]
class UserArticleState(models.Model):