Private
Public Access
Add core data models and the feed fetch pipeline
Implements Folder/Feed/Subscription (feeds app) and Article/ UserArticleState/Tag/ArticleTag (articles app), preserving the shared-Feed/shared-Article dedup design: a feed is fetched once regardless of subscriber count, and per-user read/star/save/hide state is created lazily rather than eagerly per subscriber. Adds the fetch pipeline: feedparser-based parsing with conditional GET (ETag/Last-Modified), nh3 HTML sanitization, guid-based dedup with content-hash change detection that never resets a user's read/star state, and exponential backoff on fetch errors respecting a configured minimum interval floor. Wired into Celery via poll_due_feeds/fetch_feed tasks, plus a minimal "add feed by URL" view that fetches synchronously so new subscribers see results immediately. Verified live against a real feed (hnrss.org) for ingestion, 304 dedup, and error backoff, in addition to the unit/integration test suite (28 tests, all passing). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
c635cd9330
commit
d200ab80f7
19 files changed
+1355
-8
No files matched your search
+45
-1
@@ -1 +1,45 @@
|
||||
# Create your views here.
|
||||
from django.contrib import messages
|
||||
from django.contrib.auth.decorators import login_required
|
||||
from django.shortcuts import redirect, render
|
||||
|
||||
from .forms import AddFeedForm
|
||||
from .models import Feed, Subscription
|
||||
from .services.fetcher import fetch_and_ingest
|
||||
|
||||
|
||||
@login_required
|
||||
def add_feed(request):
|
||||
form = AddFeedForm(request.POST or None, user=request.user)
|
||||
if request.method == "POST" and form.is_valid():
|
||||
feed_url = form.cleaned_data["feed_url"]
|
||||
feed, _created = Feed.objects.get_or_create(feed_url=feed_url)
|
||||
|
||||
subscription, created = Subscription.objects.get_or_create(
|
||||
user=request.user,
|
||||
feed=feed,
|
||||
defaults={"folder": form.cleaned_data["folder"]},
|
||||
)
|
||||
if not created:
|
||||
messages.info(request, "You're already subscribed to that feed.")
|
||||
return redirect("reader:stream")
|
||||
|
||||
# Fetch synchronously so the user sees results immediately; the
|
||||
# periodic Celery poll takes over from here for subsequent fetches.
|
||||
result = fetch_and_ingest(feed)
|
||||
feed.save()
|
||||
|
||||
if result.status == Feed.STATUS_ERROR:
|
||||
messages.warning(
|
||||
request,
|
||||
f"Subscribed to {feed_url}, but the initial fetch failed: {result.error_message}",
|
||||
)
|
||||
else:
|
||||
article_count = len(result.new_article_ids)
|
||||
messages.success(
|
||||
request,
|
||||
f"Subscribed to '{subscription.display_title}' — "
|
||||
f"{article_count} article(s) fetched.",
|
||||
)
|
||||
return redirect("reader:stream")
|
||||
|
||||
return render(request, "feeds/add_feed.html", {"form": form})
|
||||
Reference in new issue
Block a user