Private
Public Access
Route every feed fetch through one atomic claim (R2-T1, #32)
Adding a feed and the admin retry queued fetch_feed directly without claiming the feed, so the periodic poller (which sees a never-fetched or just-reset feed as due) could claim and fetch it too: duplicate fetches, and one task overwriting another's claim/ETag/next-fetch state. The retry path also queued the backoff reset and the fetch as two independent tasks that could run in either order. claim_feed/enqueue_fetch in apps.feeds.tasks are now the only way a fetch gets dispatched — the poller, add-feed and retry all use them, so a feed can only have one fetch in flight. If queueing fails the claim is released (the poller recovers it), and add-feed still saves the subscription during a broker outage. The retry resets backoff synchronously before claiming, and says so when a fetch is already running. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
0975a7634b
commit
85914f9107
5 files changed
+184
-33
No files matched your search
+11
-2
@@ -1,3 +1,4 @@
|
||||
import logging
|
||||
from io import BytesIO
|
||||
|
||||
from django.contrib import messages
|
||||
@@ -12,7 +13,9 @@ from apps.articles.models import SubscriptionTag, Tag
|
||||
from .forms import AddFeedForm, EditSubscriptionForm, OPMLImportForm
|
||||
from .models import Feed, Subscription
|
||||
from .opml import export_opml, extract_opml_bytes, import_opml, parse_opml
|
||||
from .tasks import fetch_feed, import_opml_task
|
||||
from .tasks import enqueue_fetch, import_opml_task
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
OPML_ASYNC_THRESHOLD = 50
|
||||
|
||||
@@ -42,7 +45,13 @@ def add_feed(request):
|
||||
# slow or large feed would otherwise tie up a Gunicorn worker for
|
||||
# the duration of the request; fetch_feed handles the save/error
|
||||
# bookkeeping the same way the periodic poll's dispatch does.
|
||||
fetch_feed.delay(feed.pk)
|
||||
try:
|
||||
enqueue_fetch(feed.pk)
|
||||
except Exception:
|
||||
# The subscription itself is saved; with the broker down the
|
||||
# periodic poller picks the (never-fetched) feed up once it's
|
||||
# back, so this isn't worth failing the request over.
|
||||
logger.exception("Could not queue the initial fetch for feed %s", feed.pk)
|
||||
messages.success(
|
||||
request,
|
||||
f"Subscribed to {feed_url}. Fetching its articles now — "
|
||||
|
||||
Reference in new issue
Block a user