Expire purge tombstones and time-box the retention purge (R2-T2, R2-T4; #38, #40)

Tombstones were kept forever, so enabling retention left a permanent row
per deleted article. They now expire after 365 days (cleaned up by the
same nightly task, even when retention is off); the documented price is
that an entry a feed still lists after a year, with no or a recent
publish date, can reappear.

The purge looped until every eligible row was gone with no run budget.
A run now works for at most four minutes, then queues a continuation, so
a huge first purge can't monopolise a worker or starve polling; hard
time limits back that up, and each chunk is its own transaction so an
interrupted run just resumes. Adds indexes on Article.fetched_at and
PurgedArticle.purged_at for the scans these do.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
ksmithandClaude Sonnet 5.5 committed 2026-11-01 12:00:52 -07:00
1 parent 2e1c0e31a8
commit 72bffbe97f
4 files changed
+153 -9

No files matched your search

@@ -0,0 +1,22 @@
# Generated by Django 5.2.18 on 2026-11-01 19:00
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('articles', '0005_purgedarticle'),
('feeds', '0006_backfill_feed_next_fetch_at'),
]
operations = [
migrations.AddIndex(
model_name='article',
index=models.Index(fields=['fetched_at'], name='articles_ar_fetched_eb5501_idx'),
),
migrations.AddIndex(
model_name='purgedarticle',
index=models.Index(fields=['purged_at'], name='articles_pu_purged__27d417_idx'),
),
]