Add core data models and the feed fetch pipeline

Implements Folder/Feed/Subscription (feeds app) and Article/
UserArticleState/Tag/ArticleTag (articles app), preserving the
shared-Feed/shared-Article dedup design: a feed is fetched once
regardless of subscriber count, and per-user read/star/save/hide state
is created lazily rather than eagerly per subscriber.

Adds the fetch pipeline: feedparser-based parsing with conditional GET
(ETag/Last-Modified), nh3 HTML sanitization, guid-based dedup with
content-hash change detection that never resets a user's read/star
state, and exponential backoff on fetch errors respecting a configured
minimum interval floor. Wired into Celery via poll_due_feeds/fetch_feed
tasks, plus a minimal "add feed by URL" view that fetches synchronously
so new subscribers see results immediately.

Verified live against a real feed (hnrss.org) for ingestion, 304
dedup, and error backoff, in addition to the unit/integration test
suite (28 tests, all passing).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
ksmithandClaude Sonnet 5 committed 2026-10-30 16:03:20 -06:00
1 parent c635cd9330
commit d200ab80f7
19 files changed
+1355 -8

No files matched your search

+27 -1
View File
@@ -1 +1,27 @@
# Register your models here.
from django.contrib import admin
from .models import Article, ArticleTag, Tag, UserArticleState
@admin.register(Article)
class ArticleAdmin(admin.ModelAdmin):
list_display = ["title", "feed", "published_at", "fetched_at"]
list_filter = ["feed"]
search_fields = ["title", "guid", "link"]
@admin.register(UserArticleState)
class UserArticleStateAdmin(admin.ModelAdmin):
list_display = ["user", "article", "is_read", "is_starred", "is_saved_for_later", "is_hidden"]
list_filter = ["is_read", "is_starred", "is_saved_for_later", "is_hidden"]
@admin.register(Tag)
class TagAdmin(admin.ModelAdmin):
list_display = ["name", "user", "color"]
list_filter = ["user"]
@admin.register(ArticleTag)
class ArticleTagAdmin(admin.ModelAdmin):
list_display = ["user", "article", "tag"]