Batch rule/tag lookups and index FilterRule for the ingest query (C9)

FilterRule had no indexes beyond the default primary/foreign-key ones,
and apply_filter_rules_for_feed issued 2 queries per subscriber (its
enabled rules, its subscription tags) inside a loop over every
subscriber of the feed — both cost grew with subscriber/rule count on
every single feed poll.

Adds a composite index on (user, is_enabled, priority), matching the
exact filter+order shape every caller already uses (this task,
apps.rules.tasks' backlog re-apply). apply_filter_rules_for_feed now
fetches rules and subscription tags once for the whole feed's subscriber
set, grouped by user/subscription in Python, instead of once per
subscriber.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
ksmithandClaude Sonnet 5 committed 2026-10-31 16:09:36 -06:00
1 parent abf4cef55b
commit cb0ce5a336
4 files changed
+91 -11

No files matched your search

+21 -11
View File
@@ -17,6 +17,10 @@ def apply_filter_rules_for_feed(feed_id: int, article_ids: list[int]):
Per-rule scoping is left to evaluate_rules_for_article/rule_in_scope
rather than filtered into this query, so this and the backlog
re-apply task (apps.rules.tasks) can't scope rules differently.
Rules and subscription tags are fetched once for every subscriber of
this feed, not once per subscriber (C9) — a popular feed with many
subscribers previously issued 2 queries per subscriber here.
"""
if not article_ids:
return 0
@@ -25,22 +29,28 @@ def apply_filter_rules_for_feed(feed_id: int, article_ids: list[int]):
if not articles:
return 0
subscriptions = Subscription.objects.filter(feed_id=feed_id).select_related("user")
subscriptions = list(Subscription.objects.filter(feed_id=feed_id).select_related("user"))
if not subscriptions:
return 0
rules_by_user_id: dict[int, list[FilterRule]] = {}
for rule in FilterRule.objects.filter(
user_id__in={s.user_id for s in subscriptions}, is_enabled=True
).order_by("priority", "id"):
rules_by_user_id.setdefault(rule.user_id, []).append(rule)
tag_ids_by_subscription_id: dict[int, set[int]] = {}
for subscription_id, tag_id in SubscriptionTag.objects.filter(
subscription_id__in=[s.pk for s in subscriptions]
).values_list("subscription_id", "tag_id"):
tag_ids_by_subscription_id.setdefault(subscription_id, set()).add(tag_id)
evaluated = 0
for subscription in subscriptions:
rules = list(
FilterRule.objects.filter(user=subscription.user, is_enabled=True).order_by(
"priority", "id"
)
)
rules = rules_by_user_id.get(subscription.user_id)
if not rules:
continue
subscription_tag_ids = set(
SubscriptionTag.objects.filter(
user=subscription.user, subscription=subscription
).values_list("tag_id", flat=True)
)
subscription_tag_ids = tag_ids_by_subscription_id.get(subscription.pk, set())
for article in articles:
evaluate_rules_for_article(subscription.user, article, rules, subscription_tag_ids)