Compare commits

..
Author SHA1 Message Date
ksmithandClaude Sonnet 5 cb0ce5a336 Batch rule/tag lookups and index FilterRule for the ingest query (C9)
FilterRule had no indexes beyond the default primary/foreign-key ones,
and apply_filter_rules_for_feed issued 2 queries per subscriber (its
enabled rules, its subscription tags) inside a loop over every
subscriber of the feed — both cost grew with subscriber/rule count on
every single feed poll.

Adds a composite index on (user, is_enabled, priority), matching the
exact filter+order shape every caller already uses (this task,
apps.rules.tasks' backlog re-apply). apply_filter_rules_for_feed now
fetches rules and subscription tags once for the whole feed's subscriber
set, grouped by user/subscription in Python, instead of once per
subscriber.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 16:09:36 -06:00
ksmithandClaude Sonnet 5 abf4cef55b Cache sidebar unread counts for a short TTL (C3)
apps.core.context_processors.sidebar runs on every authenticated page
render and called unread_counts_by_feed, a join/group-by over the
user's whole article corpus, every single time — cost grew with the
archive's size regardless of which page was being viewed.

unread_counts_by_feed now caches its {feed_id: count} result per user
for 30 seconds (apps.core.cache_utils). The reader views that change
the current user's own read state (toggle_read, mark_all_read, viewing
an article) invalidate it immediately so the sidebar never visibly lags
behind the action that just caused it; other paths (rule-driven state
changes during background ingestion) aren't explicitly invalidated and
simply self-heal within the TTL, which is an acceptable tradeoff
already made elsewhere in this app (the stream's own "new articles"
banner polls on the same 30s cadence).

Also adds a global cache.clear() autouse fixture (conftest.py) — the
test CACHES backend (LocMemCache) lives in process memory and survives
Django's per-test DB rollback, so a cached count keyed by a user's pk
could otherwise leak into an unrelated later test reusing that pk.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 16:09:28 -06:00
ksmithandClaude Sonnet 5 b33fe905d2 Push due-feed scheduling into an indexed DB query (C1)
poll_due_feeds loaded every active feed's timing fields and recomputed
each one's error-backoff due-ness in Python on every beat tick (every
1-2 minutes) — scheduler CPU and DB reads grew linearly with total feed
count regardless of how many were actually due.

Feed.next_fetch_at (indexed together with is_active) is now set by
fetch_feed itself right after every fetch attempt, success or error,
using the same backoff formula that used to run per-feed in
poll_due_feeds. The scheduler's query is now a direct indexed range
scan (next_fetch_at <= now), ordered so the most overdue feeds claim
first, and bounded to POLL_BATCH_SIZE so a large backlog spreads across
several ticks instead of dispatching everything at once. A migration
backfills next_fetch_at for existing feeds using the same formula.
reset_feed_backoff also clears next_fetch_at so a manually-reset feed
is immediately due again, matching its prior behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 16:09:18 -06:00
ksmithandClaude Sonnet 5 b84cfe6474 Document the Nginx Proxy Manager fix for S8's X-Forwarded-For spoofing
docs/install.md covered the raw-nginx case via nginx.conf.example, but
gave no concrete guidance for Nginx Proxy Manager specifically, whose
generated config also defaults to the appending (spoofable) form of
X-Forwarded-For — the same production deployment this review is based
on uses NPM. Documents the Advanced tab / Custom Nginx Configuration
override, why it takes effect ahead of NPM's own default header, and a
note for anyone with something in front of NPM itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 15:35:07 -06:00
17 changed files with 469 additions and 65 deletions

No files matched your search

+19 -4
View File
@@ -1,5 +1,8 @@
from django.core.cache import cache
from django.db.models import Count
from apps.core.cache_utils import SIDEBAR_UNREAD_CACHE_TTL_SECONDS, sidebar_unread_cache_key
from .models import SAVED_TAG_NAME, STARRED_TAG_NAME, Article, ArticleTag, UserArticleState
@@ -44,9 +47,19 @@ def with_user_state(articles, user):
def unread_counts_by_feed(user):
"""{feed_id: unread_count} for every feed the user subscribes to, in one
query — used by the sidebar, so it must stay cheap regardless of how
many feeds/articles exist."""
"""{feed_id: unread_count} for every feed the user subscribes to — used
by the sidebar's context processor, which runs on every authenticated
page render, so the underlying join/group-by over the user's whole
article corpus is cached for a short TTL rather than recomputed on
every single page view (C3). apps.core.cache_utils.
invalidate_sidebar_unread_cache is called wherever a request changes
the current user's own read state, so this is rarely visibly stale.
"""
cache_key = sidebar_unread_cache_key(user.pk)
counts = cache.get(cache_key)
if counts is not None:
return counts
rows = (
Article.objects.filter(feed__subscriptions__user=user)
.exclude(user_states__user=user, user_states__is_read=True)
@@ -54,4 +67,6 @@ def unread_counts_by_feed(user):
.values("feed_id")
.annotate(unread=Count("id"))
)
return {row["feed_id"]: row["unread"] for row in rows}
counts = {row["feed_id"]: row["unread"] for row in rows}
cache.set(cache_key, counts, SIDEBAR_UNREAD_CACHE_TTL_SECONDS)
return counts
+21 -11
View File
@@ -17,6 +17,10 @@ def apply_filter_rules_for_feed(feed_id: int, article_ids: list[int]):
Per-rule scoping is left to evaluate_rules_for_article/rule_in_scope
rather than filtered into this query, so this and the backlog
re-apply task (apps.rules.tasks) can't scope rules differently.
Rules and subscription tags are fetched once for every subscriber of
this feed, not once per subscriber (C9) — a popular feed with many
subscribers previously issued 2 queries per subscriber here.
"""
if not article_ids:
return 0
@@ -25,22 +29,28 @@ def apply_filter_rules_for_feed(feed_id: int, article_ids: list[int]):
if not articles:
return 0
subscriptions = Subscription.objects.filter(feed_id=feed_id).select_related("user")
subscriptions = list(Subscription.objects.filter(feed_id=feed_id).select_related("user"))
if not subscriptions:
return 0
rules_by_user_id: dict[int, list[FilterRule]] = {}
for rule in FilterRule.objects.filter(
user_id__in={s.user_id for s in subscriptions}, is_enabled=True
).order_by("priority", "id"):
rules_by_user_id.setdefault(rule.user_id, []).append(rule)
tag_ids_by_subscription_id: dict[int, set[int]] = {}
for subscription_id, tag_id in SubscriptionTag.objects.filter(
subscription_id__in=[s.pk for s in subscriptions]
).values_list("subscription_id", "tag_id"):
tag_ids_by_subscription_id.setdefault(subscription_id, set()).add(tag_id)
evaluated = 0
for subscription in subscriptions:
rules = list(
FilterRule.objects.filter(user=subscription.user, is_enabled=True).order_by(
"priority", "id"
)
)
rules = rules_by_user_id.get(subscription.user_id)
if not rules:
continue
subscription_tag_ids = set(
SubscriptionTag.objects.filter(
user=subscription.user, subscription=subscription
).values_list("tag_id", flat=True)
)
subscription_tag_ids = tag_ids_by_subscription_id.get(subscription.pk, set())
for article in articles:
evaluate_rules_for_article(subscription.user, article, rules, subscription_tag_ids)
+60
View File
@@ -0,0 +1,60 @@
import pytest
from django.contrib.auth import get_user_model
from apps.core.cache_utils import invalidate_sidebar_unread_cache
from apps.feeds.models import Feed, Subscription
from .models import Article
from .queries import unread_counts_by_feed
User = get_user_model()
@pytest.fixture
def alice(db):
return User.objects.create_user("alice", password="pw-alice-123")
@pytest.fixture
def feed(db):
return Feed.objects.create(feed_url="https://example.com/feed.xml")
@pytest.mark.django_db
def test_unread_counts_by_feed_is_cached_across_calls(alice, feed):
"""unread_counts_by_feed (C3) caches its result — a second call right
after a new unread article appears must still see the stale cached
value until invalidated or the TTL elapses, proving it isn't
recomputing the query every time."""
Subscription.objects.create(user=alice, feed=feed)
Article.objects.create(feed=feed, guid="a-1")
first = unread_counts_by_feed(alice)
assert first == {feed.pk: 1}
Article.objects.create(feed=feed, guid="a-2")
second = unread_counts_by_feed(alice)
assert second == {feed.pk: 1}, "expected the cached count, not a fresh recount"
@pytest.mark.django_db
def test_invalidate_sidebar_unread_cache_forces_a_recount(alice, feed):
Subscription.objects.create(user=alice, feed=feed)
Article.objects.create(feed=feed, guid="a-1")
unread_counts_by_feed(alice)
Article.objects.create(feed=feed, guid="a-2")
invalidate_sidebar_unread_cache(alice.pk)
assert unread_counts_by_feed(alice) == {feed.pk: 2}
@pytest.mark.django_db
def test_unread_counts_by_feed_cache_is_per_user(alice, feed):
bob = User.objects.create_user("bob", password="pw-bob-123")
Subscription.objects.create(user=alice, feed=feed)
Subscription.objects.create(user=bob, feed=feed)
Article.objects.create(feed=feed, guid="a-1")
assert unread_counts_by_feed(alice) == {feed.pk: 1}
assert unread_counts_by_feed(bob) == {feed.pk: 1}
+37
View File
@@ -150,3 +150,40 @@ def test_disabled_rule_does_not_apply(feed, alice):
@pytest.mark.django_db
def test_dispatch_with_empty_article_ids_is_a_noop(feed):
assert apply_filter_rules_for_feed(feed.id, []) == 0
@pytest.mark.django_db
def test_dispatch_fetches_rules_and_tags_once_per_feed_not_once_per_subscriber(feed):
"""Rules/subscription tags are batched across every subscriber of the
feed in one query each (C9), not re-queried per subscriber — assert
this by checking the query count stays flat as subscribers grow."""
from django.db import connection
from django.test.utils import CaptureQueriesContext
users = [User.objects.create_user(f"user{i}", password="pw-123456") for i in range(5)]
for user in users:
sub = Subscription.objects.create(user=user, feed=feed)
tag = Tag.objects.create(user=user, name="Tech")
SubscriptionTag.objects.create(user=user, subscription=sub, tag=tag)
FilterRule.objects.create(
user=user,
tag=tag,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_STAR,
)
article = Article.objects.create(feed=feed, guid="guid-1", title="Python news")
with CaptureQueriesContext(connection) as ctx:
apply_filter_rules_for_feed(feed.id, [article.pk])
# Exactly one SELECT against each table regardless of subscriber
# count (5 here) — a regression back to per-subscription querying
# would show up as 5 of each instead of 1.
rule_selects = [q for q in ctx.captured_queries if "rules_filterrule" in q["sql"]]
tag_selects = [q for q in ctx.captured_queries if "articles_subscriptiontag" in q["sql"]]
assert len(rule_selects) == 1
assert len(tag_selects) == 1
for user in users:
assert is_starred(user, article) is True
+22
View File
@@ -0,0 +1,22 @@
from django.core.cache import cache
# Short enough that a missed invalidation point self-heals within one
# page load's worth of staleness; long enough to actually absorb repeat
# authenticated-page renders (apps.core.context_processors.sidebar runs
# on every one) without recomputing unread_counts_by_feed's join/group-by
# over the user's whole article corpus each time (C3).
SIDEBAR_UNREAD_CACHE_TTL_SECONDS = 30
def sidebar_unread_cache_key(user_id: int) -> str:
return f"sidebar-unread:{user_id}"
def invalidate_sidebar_unread_cache(user_id: int) -> None:
"""Call this wherever a request changes the current user's own
read/unread state (toggle_read, mark_all_read, viewing an article) so
the sidebar reflects it on the very next render instead of waiting
out the TTL above. Rule-driven state changes during background feed
ingestion aren't invalidated this way — those self-heal within the
TTL, same as a missed call site here would."""
cache.delete(sidebar_unread_cache_key(user_id))
-27
View File
@@ -1,13 +1,6 @@
from datetime import timedelta
from unittest.mock import patch
import pytest
from django.contrib.auth import get_user_model
from django.urls import reverse
from django.utils import timezone
from apps.feeds.models import Feed
from apps.feeds.tasks import poll_due_feeds
from .models import SiteSettings
@@ -30,26 +23,6 @@ def test_load_returns_the_existing_row_without_creating_a_second_one():
assert SiteSettings.objects.count() == 1
@pytest.mark.django_db
def test_poll_due_feeds_uses_the_site_wide_interval_for_every_feed():
SiteSettings.objects.create(fetch_interval_minutes=120)
due = Feed.objects.create(
feed_url="https://a.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(minutes=150),
)
not_due = Feed.objects.create(
feed_url="https://b.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(minutes=90),
)
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
poll_due_feeds()
dispatched_ids = {call.args[0] for call in mock_delay.call_args_list}
assert dispatched_ids == {due.id}
assert not_due.id not in dispatched_ids
@pytest.mark.django_db
def test_admin_changelist_redirects_straight_to_the_change_form(client):
assert not SiteSettings.objects.exists()
@@ -0,0 +1,24 @@
# Generated by Django 5.2.18 on 2026-10-31 22:00
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("feeds", "0004_remove_feed_fetch_interval_minutes"),
]
operations = [
migrations.AddField(
model_name="feed",
name="next_fetch_at",
field=models.DateTimeField(blank=True, null=True),
),
migrations.AddIndex(
model_name="feed",
index=models.Index(
fields=["is_active", "next_fetch_at"], name="feeds_feed_is_acti_5c232e_idx"
),
),
]
@@ -0,0 +1,49 @@
from datetime import timedelta
from django.conf import settings
from django.db import migrations
def backfill_next_fetch_at(apps, schema_editor):
"""One-time backfill for feeds that existed before next_fetch_at (C1) —
everything created after this migration gets it set by fetch_feed
itself. Mirrors the exact formula apps.feeds.services.fetcher.
effective_interval_minutes used to compute on every poll tick, just
run once here instead of forever after.
"""
Feed = apps.get_model("feeds", "Feed")
SiteSettings = apps.get_model("core", "SiteSettings")
configured_interval = 60
settings_row = SiteSettings.objects.filter(pk=1).first()
if settings_row is not None:
configured_interval = settings_row.fetch_interval_minutes
floor = getattr(settings, "EVECTION_MIN_FETCH_INTERVAL_MINUTES", 15)
base = max(configured_interval, floor)
max_backoff = 24 * 60
for feed in Feed.objects.filter(last_fetched_at__isnull=False).only(
"id", "last_fetched_at", "fetch_error_count"
):
if feed.fetch_error_count <= 0:
interval = base
else:
interval = min(base * (2**feed.fetch_error_count), max_backoff)
feed.next_fetch_at = feed.last_fetched_at + timedelta(minutes=interval)
feed.save(update_fields=["next_fetch_at"])
def noop_reverse(apps, schema_editor):
pass
class Migration(migrations.Migration):
dependencies = [
("feeds", "0005_feed_next_fetch_at_and_more"),
("core", "0001_initial"),
]
operations = [
migrations.RunPython(backfill_next_fetch_at, noop_reverse),
]
+12
View File
@@ -42,6 +42,15 @@ class Feed(models.Model):
fetch_error_count = models.PositiveIntegerField(default=0)
is_active = models.BooleanField(default=True)
# Set by fetch_feed right after every fetch attempt (success or error)
# to last_fetched_at + effective_interval_minutes — precomputed once,
# at write time, instead of poll_due_feeds re-deriving due-ness (which
# depends on fetch_error_count's exponential backoff) for every active
# feed on every tick (C1). NULL means "due now", same as a feed that's
# never been fetched. Indexed with is_active below so the scheduler's
# query is a direct index range scan, not a full table scan.
next_fetch_at = models.DateTimeField(null=True, blank=True)
# Set by poll_due_feeds' atomic claim (an UPDATE ... WHERE clause, not
# select_for_update — Celery Beat and fetch_feed run in separate
# transactions) and cleared once fetch_feed finishes, so overlapping
@@ -53,6 +62,9 @@ class Feed(models.Model):
created_at = models.DateTimeField(auto_now_add=True)
class Meta:
indexes = [models.Index(fields=["is_active", "next_fetch_at"])]
def __str__(self):
return self.title or self.feed_url
+34 -23
View File
@@ -22,27 +22,31 @@ logger = logging.getLogger(__name__)
# while a worker that died mid-fetch doesn't wedge the feed forever.
CLAIM_STALE_MINUTES = 10
# Caps how many feeds one poll_due_feeds tick will claim/dispatch, so a
# large backlog (first run after a long outage, a huge import) spreads
# across several beat ticks instead of enqueuing everything — and every
# bounded Feed query below stays index-only work, not a full scan (C1).
POLL_BATCH_SIZE = 500
@shared_task
def poll_due_feeds():
"""Find every active Feed whose polling interval has elapsed, atomically
claim it (so overlapping beat ticks or a slow-running previous fetch
can't dispatch the same feed twice), and dispatch a fetch_feed task for
each claimed feed — fan-out happens here, not via one Celery Beat entry
per feed.
"""Find a bounded batch of active Feeds whose next_fetch_at has
elapsed (set by fetch_feed itself after every attempt — see Feed.
next_fetch_at), atomically claim them (so overlapping beat ticks or a
slow-running previous fetch can't dispatch the same feed twice), and
dispatch a fetch_feed task for each claimed feed — fan-out happens
here, not via one Celery Beat entry per feed.
"""
now = timezone.now()
claim_stale_before = now - timedelta(minutes=CLAIM_STALE_MINUTES)
# Loaded once per call, not per feed — the interval is site-wide
# (apps.core.models.SiteSettings), so every feed below shares it.
configured_interval = SiteSettings.load().fetch_interval_minutes
due_feed_ids = [
feed.id
for feed in Feed.objects.filter(is_active=True).only(
"id", "last_fetched_at", "fetch_error_count"
)
if _is_due(feed, configured_interval)
]
due_feed_ids = list(
Feed.objects.filter(is_active=True)
.filter(Q(next_fetch_at__isnull=True) | Q(next_fetch_at__lte=now))
.filter(Q(claimed_at__isnull=True) | Q(claimed_at__lt=claim_stale_before))
.order_by("next_fetch_at")
.values_list("id", flat=True)[:POLL_BATCH_SIZE]
)
claimed_ids = []
for feed_id in due_feed_ids:
# The UPDATE's WHERE clause re-checks staleness at the database
@@ -61,11 +65,14 @@ def poll_due_feeds():
return len(claimed_ids)
def _is_due(feed: Feed, configured_interval_minutes: int) -> bool:
if feed.last_fetched_at is None:
return True
interval = effective_interval_minutes(feed, configured_interval_minutes)
return feed.last_fetched_at <= timezone.now() - timedelta(minutes=interval)
def _schedule_next_fetch(feed: Feed) -> None:
"""Sets feed.next_fetch_at from its (just-updated) fetch_error_count —
called right before every feed.save() in fetch_feed, so the next
poll_due_feeds tick can find it with a plain indexed range query
instead of recomputing every active feed's backoff in Python (C1)."""
configured_interval = SiteSettings.load().fetch_interval_minutes
interval = effective_interval_minutes(feed, configured_interval)
feed.next_fetch_at = timezone.now() + timedelta(minutes=interval)
# Bounds a single feed's fetch+parse+ingest, independent of
@@ -102,10 +109,12 @@ def fetch_feed(feed_id: int):
record_fetch_error(feed, "Unexpected error — see worker logs")
feed.last_fetched_at = timezone.now()
feed.claimed_at = None
_schedule_next_fetch(feed)
feed.save()
raise
feed.claimed_at = None
_schedule_next_fetch(feed)
feed.save()
changed_article_ids = result.new_article_ids + result.updated_article_ids
@@ -127,9 +136,11 @@ def fetch_feed(feed_id: int):
@shared_task
def reset_feed_backoff(feed_id: int):
"""Admin-triggered: clear a feed's error count so it's retried at its
normal configured interval instead of the backed-off one."""
Feed.objects.filter(pk=feed_id).update(fetch_error_count=0)
"""Admin-triggered: clear a feed's error count and due-date backoff so
it's retried at its normal configured interval instead of the backed-
off one (next_fetch_at=None also makes it immediately due for the
next poll_due_feeds tick, same as a feed that's never been fetched)."""
Feed.objects.filter(pk=feed_id).update(fetch_error_count=0, next_fetch_at=None)
@shared_task
+67
View File
@@ -12,18 +12,23 @@ from apps.feeds.tasks import fetch_feed, poll_due_feeds
@pytest.mark.django_db
def test_poll_due_feeds_dispatches_only_due_feeds():
# next_fetch_at (set by fetch_feed itself, C1), not last_fetched_at,
# is what poll_due_feeds' query filters on now.
never_fetched = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
recently_fetched = Feed.objects.create(
feed_url="https://b.example.com/feed.xml",
last_fetched_at=timezone.now(),
next_fetch_at=timezone.now() + timedelta(hours=1),
)
overdue = Feed.objects.create(
feed_url="https://c.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2),
next_fetch_at=timezone.now() - timedelta(hours=1),
)
inactive_but_overdue = Feed.objects.create(
feed_url="https://d.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2),
next_fetch_at=timezone.now() - timedelta(hours=1),
is_active=False,
)
@@ -45,6 +50,7 @@ def test_poll_due_feeds_does_not_reclaim_a_feed_it_just_claimed():
overdue = Feed.objects.create(
feed_url="https://c.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2),
next_fetch_at=timezone.now() - timedelta(hours=1),
)
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
@@ -65,6 +71,7 @@ def test_poll_due_feeds_reclaims_a_stale_claim():
overdue = Feed.objects.create(
feed_url="https://c.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2),
next_fetch_at=timezone.now() - timedelta(hours=1),
claimed_at=timezone.now() - timedelta(minutes=CLAIM_STALE_MINUTES + 1),
)
@@ -141,3 +148,63 @@ def test_fetch_feed_task_records_error_and_clears_claim_on_unexpected_exception(
assert feed.fetch_error_count == 1
assert feed.claimed_at is None
assert feed.last_fetched_at is not None
assert feed.next_fetch_at is not None
@pytest.mark.django_db
@responses.activate
def test_fetch_feed_task_sets_next_fetch_at_so_poll_due_feeds_can_find_it_later():
"""next_fetch_at (C1) is what poll_due_feeds' query filters on — a
feed fetch_feed just succeeded on must come back due again only after
the configured interval, not immediately."""
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
responses.add(responses.GET, feed.feed_url, body=GOOD_FEED_XML, status=200)
fetch_feed(feed.id)
feed.refresh_from_db()
assert feed.next_fetch_at is not None
assert feed.next_fetch_at > timezone.now()
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
poll_due_feeds()
mock_delay.assert_not_called()
@pytest.mark.django_db
def test_reset_feed_backoff_makes_the_feed_immediately_due():
from apps.feeds.tasks import reset_feed_backoff
feed = Feed.objects.create(
feed_url="https://example.com/feed.xml",
fetch_error_count=5,
last_fetched_at=timezone.now(),
next_fetch_at=timezone.now() + timedelta(hours=12),
)
reset_feed_backoff(feed.id)
feed.refresh_from_db()
assert feed.fetch_error_count == 0
assert feed.next_fetch_at is None
@pytest.mark.django_db
@responses.activate
def test_fetch_feed_schedules_the_next_fetch_using_the_site_wide_interval():
"""fetch_feed (not poll_due_feeds — see apps.feeds.tasks.
_schedule_next_fetch, C1) is what reads SiteSettings' site-wide
interval and sets Feed.next_fetch_at from it after every fetch."""
from apps.core.models import SiteSettings
SiteSettings.objects.create(fetch_interval_minutes=120)
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
responses.add(responses.GET, feed.feed_url, body=GOOD_FEED_XML, status=200)
before = timezone.now()
fetch_feed(feed.id)
feed.refresh_from_db()
assert feed.next_fetch_at is not None
expected = before + timedelta(minutes=120)
assert abs((feed.next_fetch_at - expected).total_seconds()) < 5
+54
View File
@@ -289,3 +289,57 @@ def test_stream_exposes_the_latest_article_id_and_scope_for_the_banner(client, a
assert f"since={article.pk}" in content
assert "filter=all" in content
@pytest.mark.django_db
def test_sidebar_unread_count_updates_immediately_after_reading_an_article(client, alice):
"""unread_counts_by_feed caches its result for a short TTL (C3) — the
reader views that change the current user's own read state must
invalidate it, or the sidebar would show a stale count until the TTL
elapses on every one of these very common actions."""
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
article = Article.objects.create(feed=feed, guid="a-1", title="Hello")
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
assert '<span class="unread-count">1</span>' in response.content.decode()
client.get(reverse("reader:article_detail", args=[article.pk]))
response = client.get(reverse("reader:stream"))
assert "unread-count" not in response.content.decode()
@pytest.mark.django_db
def test_sidebar_unread_count_updates_immediately_after_toggle_read(client, alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
article = Article.objects.create(feed=feed, guid="a-1", title="Hello")
UserArticleState.objects.create(user=alice, article=article, is_read=True)
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
assert "unread-count" not in response.content.decode()
client.post(reverse("reader:toggle_read", args=[article.pk]))
response = client.get(reverse("reader:stream"))
assert '<span class="unread-count">1</span>' in response.content.decode()
@pytest.mark.django_db
def test_sidebar_unread_count_updates_immediately_after_mark_all_read(client, alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
Article.objects.create(feed=feed, guid="a-1", title="Hello")
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
assert '<span class="unread-count">1</span>' in response.content.decode()
client.post(reverse("reader:mark_all_read"))
response = client.get(reverse("reader:stream"))
assert "unread-count" not in response.content.decode()
+4
View File
@@ -19,6 +19,7 @@ from apps.articles.models import (
get_system_tag,
)
from apps.articles.queries import with_user_state
from apps.core.cache_utils import invalidate_sidebar_unread_cache
from apps.feeds.models import Feed
FILTER_CHOICES = [
@@ -240,6 +241,7 @@ def mark_all_read(request):
for aid in new_article_ids
]
)
invalidate_sidebar_unread_cache(request.user.pk)
stream_url = reverse("reader:stream")
query = request.GET.urlencode()
@@ -254,6 +256,7 @@ def _load_article_and_mark_read(user, article_id):
state.is_read = True
state.read_at = timezone.now()
state.save(update_fields=["is_read", "read_at"])
invalidate_sidebar_unread_cache(user.pk)
article.is_read = state.is_read
article.is_starred, article.is_saved_for_later = _article_tag_flags(user, article)
@@ -285,6 +288,7 @@ def toggle_read(request, article_id):
state.is_read = not state.is_read
state.read_at = timezone.now() if state.is_read else None
state.save(update_fields=["is_read", "read_at"])
invalidate_sidebar_unread_cache(request.user.pk)
article.is_read = state.is_read
article.is_starred, article.is_saved_for_later = _article_tag_flags(request.user, article)
@@ -0,0 +1,23 @@
# Generated by Django 5.2.18 on 2026-10-31 22:05
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("articles", "0004_starred_saved_as_system_tags"),
("feeds", "0006_backfill_feed_next_fetch_at"),
("rules", "0002_folder_scope_to_tag_scope"),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.AddIndex(
model_name="filterrule",
index=models.Index(
fields=["user", "is_enabled", "priority"], name="rules_filte_user_id_dcb19b_idx"
),
),
]
+10
View File
@@ -75,6 +75,16 @@ class FilterRule(models.Model):
class Meta:
ordering = ["priority", "id"]
indexes = [
# Matches the ingest-time query shape exactly (C9): every
# caller — apps.articles.tasks.apply_filter_rules_for_feed,
# apps.rules.tasks.reapply_rules_for_user_backlog — filters by
# (user, is_enabled) and orders by priority. `user` alone
# already has an index from the ForeignKey, but it doesn't
# cover is_enabled or let the database satisfy the ORDER BY
# from the index itself.
models.Index(fields=["user", "is_enabled", "priority"]),
]
def __str__(self):
label = self.name or f"{self.get_match_field_display()} {self.get_match_type_display()}"
+14
View File
@@ -1,8 +1,22 @@
import pytest
from django.core.cache import cache
from apps.accounts.middleware import SetupWizardMiddleware
@pytest.fixture(autouse=True)
def _reset_django_cache():
"""config.settings.test overrides CACHES to LocMemCache, which — like
SetupWizardMiddleware's own cache below — lives in process memory and
survives Django's per-test DB rollback. Without this, a cached
sidebar unread count (apps.core.cache_utils) keyed by a user's pk
could leak into a later, unrelated test that happens to create a user
with the same auto-incrementing pk."""
cache.clear()
yield
cache.clear()
@pytest.fixture(autouse=True)
def _reset_setup_wizard_cache():
"""SetupWizardMiddleware caches "does any User exist" on the class
+19
View File
@@ -274,6 +274,25 @@ next paragraph for why it matters). Whichever proxy you use,
headers that need forwarding either way, including that overwrite-not-
append distinction for `X-Forwarded-For`.
**Nginx Proxy Manager specifically:** NPM's generated config already sets
`X-Forwarded-For` to the appending (unsafe) form by default — override it
per proxy host rather than editing NPM's own templates: open the proxy
host entry for Evection, go to its **Advanced** tab, and add this to the
**Custom Nginx Configuration** box:
```nginx
proxy_set_header X-Forwarded-For $remote_addr;
```
This lands inside NPM's own `location` block, after its default headers,
so nginx uses this one (the last `proxy_set_header` for a given name in
the same block wins) — no edit to any file NPM manages, and it survives
NPM regenerating its config. Save, and NPM reloads nginx for you. If
something else sits in front of NPM itself (a CDN, another load balancer,
Cloudflare, ...), `$remote_addr` at NPM's layer is that upstream's
address, not the original client — forward whatever *that* layer's own
trusted client-IP header says instead.
Either way, the proxy must set `X-Forwarded-Proto: https` on requests it
forwards once it's terminating TLS — Evection's `prod` settings
(`SECURE_PROXY_SSL_HEADER` in `config/settings/prod.py`) trust that header to