Compare commits

...
Author SHA1 Message Date
ksmithandClaude Sonnet 5 9439f9cc59 Add SSRF protection to feed fetching: validate scheme/address, bound redirects and response size (S1)
fetch_and_ingest fetched a feed_url supplied straight from the "add
feed" form or an uploaded OPML file, with requests following redirects
on its own and no check that the target (or a redirect's target) was
actually a public, fetchable feed rather than an internal service,
localhost, or a cloud metadata endpoint — reachable from wherever this
app runs, on any authenticated user's behalf.

apps.feeds.services.ssrf_guard.safe_get() replaces the direct
requests.get() call: it resolves and validates the hostname (rejecting
anything that isn't a globally routable address) before the initial
request and before following each redirect hop manually, caps the
number of hops, and bounds the response body size by streaming it
instead of buffering it unbounded. OPML-imported feeds go through the
same check the first time poll_due_feeds fetches them, since this is
the one chokepoint every feed URL passes through regardless of which
path created the Feed row.

Known residual gap, documented in ssrf_guard.py: this validates then
connects rather than pinning the TCP connection to the exact address it
validated, so a narrow DNS-rebinding race isn't closed. Fixing that
needs a custom transport and felt like more machinery than this
self-hosted app's threat model currently justifies.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:25:20 -06:00
ksmithandClaude Sonnet 5 fd8eeb605e Scope backlog re-apply per-feed and always queue it through Celery (T5, S5)
Completes the T5 fix: reapply_rules_for_user_backlog now computes each
feed's subscription-tag ids up front and passes them to
evaluate_rules_for_article per article, so a feed/tag-scoped rule from
a different feed can't fire on this backlog (see the previous commit's
rule_in_scope() in apps.rules.engine).

Also completes S5's second half: the "Re-apply" view now always
dispatches reapply_rules_for_user.delay() instead of calling the
backlog function inline from the request — a large backlog or a
regex that outruns the new timeout guard no longer risks tying up a
web worker.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:20:25 -06:00
ksmithandClaude Sonnet 5 91c0b1dd26 Centralize rule feed/tag scoping in the engine and fix backlog re-apply (T5)
reapply_rules_for_user_backlog narrowed its *backlog* to the triggering
rule's feed/tag scope, but then loaded every enabled rule for the user
and ran all of them against that backlog with no per-rule scope check
— so re-applying one feed- or tag-scoped rule could also fire an
unrelated rule scoped to a different feed/tag, if it happened to match.

rule_in_scope() moves the feed/tag scope check into the engine itself
(apply_filter_rules_for_feed already computed this per-subscription via
query filters; it now just passes the tag ids and lets the engine
decide), so backlog re-apply and ingest-time dispatch can't drift into
scoping rules differently again.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:19:56 -06:00
ksmithandClaude Sonnet 5 f2ee793cf8 upgrade.sh: auto-bump EVECTION_PWA_CACHE_VERSION on every deploy
The release-notes reminder to manually bump this and restart gunicorn
was easy to miss or forget, leaving installed PWAs on a stale cached
shell/pages after an upgrade that changed static assets. upgrade.sh
now sets it to the deployed tag itself before restarting services, so
there's no manual step and no cache-invalidation window to miss.
install.sh does the same on first install for consistency.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:08:05 -06:00
ksmithandClaude Sonnet 5 b7d9ee672c Add an atomic per-feed claim to poll_due_feeds (T1)
poll_due_feeds scanned for due feeds and dispatched fetch_feed without
marking them claimed, so overlapping beat ticks or a slow-running
fetch could dispatch the same feed twice, racing on ETag/status/error-
count writes. Feed.claimed_at is now set via an UPDATE ... WHERE
claimed_at IS NULL OR claimed_at < stale-cutoff, so only one caller's
UPDATE can ever match a given row; fetch_feed clears it when done.
A claim older than CLAIM_STALE_MINUTES is treated as abandoned so a
worker dying mid-fetch doesn't wedge the feed forever.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:03:18 -06:00
ksmithandClaude Sonnet 5 d7b433e580 Purge the service worker's authenticated-page cache on logout (S3)
PAGES_CACHE held every cached navigation response (including
authenticated article pages) with no logout-time purge, so a later
offline navigation on a shared or account-switched device could still
serve another user's cached pages. Logout is a plain POST, so the
fetch handler now special-cases that request: it deletes PAGES_CACHE
and lets the real logout request through, with no race against the
navigation that follows since it's synchronous with the same request.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:03:10 -06:00
ksmithandClaude Sonnet 5 b52ba2be8a Make first-run setup race-safe with an atomic SetupLock claim (S4)
The setup view's User.objects.exists() check and the admin account's
save() weren't in the same transaction, so two concurrent requests
during first deployment could both pass the check and both create an
admin account. SetupLock is a singleton row (pk=1); claiming it and
creating the admin now happen in one atomic block, so only the first
of any racing requests can succeed — the rest hit IntegrityError and
are sent to the login page instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:03:05 -06:00
ksmithandClaude Sonnet 5 56eb2bc498 Fail closed in production if SECRET_KEY is missing or the placeholder default (S2)
base.py's env.str() fallback is a publicly known string in this repo's
history. Running prod settings with it (an unset or forgotten .env
value) would sign sessions/CSRF tokens with a key anyone can read.
install.sh already generates a real random key for fresh installs, so
this only blocks the broken case.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:02:58 -06:00
ksmithandClaude Sonnet 5 9d91da6fc6 Fix /health/ 400ing when hit directly over loopback (as upgrade.sh does)
scripts/upgrade.sh's health check connects straight to gunicorn's bind
address, bypassing the reverse proxy entirely, so the request's Host
header is "127.0.0.1:<port>" and it has no X-Forwarded-Proto header.
Against prod settings that's exactly a DisallowedHost (ALLOWED_HOSTS only
has the public domain) — which Django turns into a 400 — and would have
then hit a redirect-to-HTTPS loop from SECURE_SSL_REDIRECT right after,
since there's no TLS listener on that port to redirect to. Allow the
loopback addresses in ALLOWED_HOSTS (harmless: gunicorn's bind address is
never reachable from outside this host per docs/install.md) and exempt
/health/ from the HTTPS redirect specifically.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:19:27 -06:00
ksmithandClaude Sonnet 5 c316f411f8 Make the active-tag sidebar highlight as visible as the active-feed one
.sidebar-tag-link.active only changed text color, which is easy to miss
next to .sidebar-feed-link.active's background pill + bold weight. Give
the tag link the same background-pill treatment (padding/margin offset
so the box doesn't shift layout when inactive).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:17:45 -06:00
ksmithandClaude Sonnet 5 e1907ff587 install.sh: fail hard (or auto-install) when Redis isn't actually reachable
The old check only looked for redis-cli on PATH and printed a warning if
missing — it never tested an actual connection, so a host with no Redis
installed at all still sailed through the entire install looking
successful. The real failure only showed up later as every feed stuck on
"pending" forever, with the worker/beat units quietly retrying their
connection in the background rather than surfacing anything to whoever
ran the script. Actually connect to the REDIS_URL that will end up in
.env; on Debian/Ubuntu with the default local address, install and start
redis-server automatically; otherwise fail the install outright with a
clear message instead of a warning that's easy to miss.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:12:41 -06:00
25 changed files with 825 additions and 50 deletions

No files matched your search

@@ -0,0 +1,22 @@
# Generated by Django 5.2.18 on 2026-10-31 18:00
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("accounts", "0001_initial"),
]
operations = [
migrations.CreateModel(
name="SetupLock",
fields=[
(
"id",
models.PositiveSmallIntegerField(default=1, primary_key=True, serialize=False),
),
],
),
]
+12
View File
@@ -5,3 +5,15 @@ from django.db import models
class User(AbstractUser):
timezone = models.CharField(max_length=64, default="UTC")
created_at = models.DateTimeField(auto_now_add=True)
class SetupLock(models.Model):
"""Singleton row that gates first-run setup against a race between
concurrent requests. The setup view always tries to INSERT the row
with pk=1 in the same transaction as creating the first admin user:
the database's primary-key uniqueness makes exactly one of those
INSERTs win even under concurrent requests, so only one admin account
can ever be created this way, instead of the User.objects.exists()
check alone (which has a window between the check and the save)."""
id = models.PositiveSmallIntegerField(primary_key=True, default=1)
+23 -1
View File
@@ -1,7 +1,7 @@
import pytest
from django.urls import reverse
from .models import User
from .models import SetupLock, User
# SetupWizardMiddleware's cache is reset around every test by the autouse
# fixture in the top-level conftest.py, not just here.
@@ -43,6 +43,28 @@ def test_setup_creates_admin_and_logs_in(client):
assert stream_response.status_code == 200
@pytest.mark.django_db
def test_setup_refuses_a_second_concurrent_submission(client):
# Simulates two requests racing the setup form: the first claims the
# SetupLock row directly (standing in for a request that's already
# past the User.objects.exists() check), so a second submission's
# INSERT must lose instead of also creating an admin account.
SetupLock.objects.create()
response = client.post(
reverse("accounts:setup"),
{
"username": "raced",
"email": "raced@example.com",
"password": "correct-horse-battery",
"password_confirm": "correct-horse-battery",
},
)
assert response.status_code == 302
assert response.url == reverse("accounts:login")
assert not User.objects.filter(username="raced").exists()
@pytest.mark.django_db
def test_setup_is_unreachable_once_a_user_exists(client):
User.objects.create_user("existing", password="irrelevant")
+12 -1
View File
@@ -1,11 +1,12 @@
from django.contrib import messages
from django.contrib.auth import login
from django.contrib.auth.decorators import user_passes_test
from django.db import IntegrityError, transaction
from django.shortcuts import redirect, render
from .forms import CreateUserForm, SetupForm
from .middleware import SetupWizardMiddleware
from .models import User
from .models import SetupLock, User
is_admin = user_passes_test(lambda u: u.is_authenticated and u.is_staff)
@@ -16,11 +17,21 @@ def setup(request):
form = SetupForm(request.POST or None)
if request.method == "POST" and form.is_valid():
try:
with transaction.atomic():
# Claims the one-time setup slot: the first of any
# concurrent requests to reach this INSERT wins it, and
# every other one raises IntegrityError instead of also
# creating an admin account. See SetupLock's docstring.
SetupLock.objects.create()
user = form.save(commit=False)
user.set_password(form.cleaned_data["password"])
user.is_staff = True
user.is_superuser = True
user.save()
except IntegrityError:
messages.info(request, "Setup was already completed. Log in with that account.")
return redirect("accounts:login")
SetupWizardMiddleware._a_user_exists = True
login(request, user)
messages.success(request, "Welcome to Evection! Your admin account is ready.")
+12 -12
View File
@@ -1,5 +1,4 @@
from celery import shared_task
from django.db.models import Q
from apps.feeds.models import Subscription
from apps.rules.engine import evaluate_rules_for_article
@@ -14,6 +13,10 @@ def apply_filter_rules_for_feed(feed_id: int, article_ids: list[int]):
FilterRules (scoped to this feed/tag or unscoped) against the given
articles. Dispatched by the fetch pipeline after new/updated articles
are committed for a feed.
Per-rule scoping is left to evaluate_rules_for_article/rule_in_scope
rather than filtered into this query, so this and the backlog
re-apply task (apps.rules.tasks) can't scope rules differently.
"""
if not article_ids:
return 0
@@ -26,24 +29,21 @@ def apply_filter_rules_for_feed(feed_id: int, article_ids: list[int]):
evaluated = 0
for subscription in subscriptions:
rules = FilterRule.objects.filter(user=subscription.user, is_enabled=True).filter(
Q(feed_id=feed_id) | Q(feed__isnull=True)
rules = list(
FilterRule.objects.filter(user=subscription.user, is_enabled=True).order_by(
"priority", "id"
)
subscription_tag_ids = list(
)
if not rules:
continue
subscription_tag_ids = set(
SubscriptionTag.objects.filter(
user=subscription.user, subscription=subscription
).values_list("tag_id", flat=True)
)
if subscription_tag_ids:
rules = rules.filter(Q(tag_id__in=subscription_tag_ids) | Q(tag__isnull=True))
else:
rules = rules.filter(tag__isnull=True)
rules = list(rules.order_by("priority", "id"))
if not rules:
continue
for article in articles:
evaluate_rules_for_article(subscription.user, article, rules)
evaluate_rules_for_article(subscription.user, article, rules, subscription_tag_ids)
evaluated += 1
return evaluated
+21
View File
@@ -0,0 +1,21 @@
import socket
import pytest
@pytest.fixture(autouse=True)
def _fake_public_dns(monkeypatch):
"""apps.feeds.services.ssrf_guard resolves a feed's real hostname via
socket.getaddrinfo before every fetch. Tests use made-up hostnames
like example.com/a.example.com purely as fixture data — they must
not depend on real DNS/network access to pass that check, and must
resolve deterministically regardless of this sandbox's network
access. Every lookup here resolves to a fixed, definitely-public IP;
tests that specifically exercise the "resolves to a non-public
address" rejection path patch socket.getaddrinfo themselves instead.
"""
def _fake_getaddrinfo(host, *args, **kwargs):
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("93.184.216.34", 0))]
monkeypatch.setattr(socket, "getaddrinfo", _fake_getaddrinfo)
@@ -0,0 +1,18 @@
# Generated by Django 5.2.18 on 2026-10-31 18:00
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("feeds", "0002_remove_folder"),
]
operations = [
migrations.AddField(
model_name="feed",
name="claimed_at",
field=models.DateTimeField(blank=True, null=True),
),
]
+9
View File
@@ -40,6 +40,15 @@ class Feed(models.Model):
fetch_error_count = models.PositiveIntegerField(default=0)
is_active = models.BooleanField(default=True)
# Set by poll_due_feeds' atomic claim (an UPDATE ... WHERE clause, not
# select_for_update — Celery Beat and fetch_feed run in separate
# transactions) and cleared once fetch_feed finishes, so overlapping
# beat ticks or a fetch that runs long can't both dispatch the same
# feed. A claim older than CLAIM_STALE_MINUTES (tasks.py) is treated as
# abandoned and can be reclaimed, so a worker dying mid-fetch doesn't
# wedge the feed forever.
claimed_at = models.DateTimeField(null=True, blank=True)
created_at = models.DateTimeField(auto_now_add=True)
def __str__(self):
+6 -1
View File
@@ -23,6 +23,8 @@ from django.utils import timezone
from apps.articles.models import Article
from apps.feeds.models import Feed
from .ssrf_guard import SSRFBlocked, safe_get
logger = logging.getLogger(__name__)
FETCH_TIMEOUT_SECONDS = 15
@@ -139,7 +141,10 @@ def fetch_and_ingest(feed: Feed) -> FetchResult:
headers["If-Modified-Since"] = feed.last_modified
try:
response = requests.get(feed.feed_url, headers=headers, timeout=FETCH_TIMEOUT_SECONDS)
response = safe_get(feed.feed_url, headers=headers, timeout=FETCH_TIMEOUT_SECONDS)
except SSRFBlocked as exc:
feed.last_fetched_at = timezone.now()
return _record_error(feed, f"Refused to fetch: {exc}")
except requests.RequestException as exc:
feed.last_fetched_at = timezone.now()
return _record_error(feed, f"Request failed: {exc}")
+121
View File
@@ -0,0 +1,121 @@
"""Rejects feed-fetch targets that could turn this server/worker's own
network access into an SSRF vector.
Feed URLs are entirely attacker-controlled (any authenticated user via
"add feed", or an uploaded OPML file — see apps.feeds.opml) and this app
fetches them from the server's own network position. fetch_and_ingest
(apps.feeds.services.fetcher) is the single chokepoint every feed URL
passes through before any HTTP request is made, regardless of which
path created the Feed row, so enforcing this here covers all of them.
This validates the resolved address before connecting and again before
following each redirect hop, denying by default anything that isn't a
globally routable address (ip.is_global) — covering loopback, RFC1918
private ranges, link-local (including the 169.254.169.254 cloud
metadata address), and so on, for both IPv4 and IPv6.
What this does *not* close: the TCP connection isn't pinned to the
specific IP that was validated, so a DNS answer that changes between
the validation lookup and the actual connect (classic DNS rebinding)
could still slip through. Closing that fully needs a custom transport
that connects to the exact validated IP while still verifying TLS
against the original hostname — more machinery than this self-hosted
app's threat model currently justifies; this stops the much more
common case of a feed URL (or a redirect) simply pointing straight at
an internal address.
"""
from __future__ import annotations
import ipaddress
import socket
from dataclasses import dataclass
from urllib.parse import urljoin, urlparse
import requests
from requests.structures import CaseInsensitiveDict
ALLOWED_SCHEMES = {"http", "https"}
MAX_REDIRECTS = 5
MAX_RESPONSE_BYTES = 10 * 1024 * 1024 # 10 MB — generous for an RSS/Atom feed
READ_CHUNK_BYTES = 65536
class SSRFBlocked(Exception):
"""Raised when a fetch target — or one of its redirects — isn't safe
to connect to on a user's behalf."""
@dataclass
class BoundedResponse:
status_code: int
# requests' own CaseInsensitiveDict — server header casing varies
# (ETag/Etag/etag) and fetcher.py looks these up by a fixed-case key.
headers: CaseInsensitiveDict
content: bytes
def validate_fetch_url(url: str) -> None:
"""Raises SSRFBlocked unless `url` has an http(s) scheme, a hostname,
and every address that hostname resolves to is globally routable."""
parsed = urlparse(url)
if parsed.scheme not in ALLOWED_SCHEMES:
raise SSRFBlocked(f"Unsupported URL scheme: {parsed.scheme!r}")
if not parsed.hostname:
raise SSRFBlocked("URL has no hostname")
try:
addrinfo = socket.getaddrinfo(parsed.hostname, None)
except OSError as exc:
raise SSRFBlocked(f"Could not resolve {parsed.hostname!r}: {exc}") from exc
for *_, sockaddr in addrinfo:
ip = ipaddress.ip_address(sockaddr[0])
if not ip.is_global:
raise SSRFBlocked(f"{parsed.hostname!r} resolves to a non-public address: {ip}")
def safe_get(url: str, *, headers: dict, timeout: float) -> BoundedResponse:
"""GETs `url` the way fetch_and_ingest needs it: every redirect hop is
validated and followed manually (never left to requests/urllib3, so
a redirect can't bypass validate_fetch_url), capped at MAX_REDIRECTS,
and the body is read in bounded chunks rather than trusting
Content-Length or buffering an unbounded response.
"""
current_url = url
with requests.Session() as session:
for _ in range(MAX_REDIRECTS + 1):
validate_fetch_url(current_url)
response = session.get(
current_url,
headers=headers,
timeout=timeout,
stream=True,
allow_redirects=False,
)
try:
if response.is_redirect or response.is_permanent_redirect:
location = response.headers.get("Location")
if not location:
raise SSRFBlocked("Redirect response had no Location header")
current_url = urljoin(current_url, location)
continue
return BoundedResponse(
status_code=response.status_code,
headers=response.headers,
content=_read_bounded(response),
)
finally:
response.close()
raise SSRFBlocked(f"Too many redirects (> {MAX_REDIRECTS})")
def _read_bounded(response: requests.Response) -> bytes:
chunks = []
total = 0
for chunk in response.iter_content(chunk_size=READ_CHUNK_BYTES):
total += len(chunk)
if total > MAX_RESPONSE_BYTES:
raise SSRFBlocked(f"Response exceeded {MAX_RESPONSE_BYTES} bytes")
chunks.append(chunk)
return b"".join(chunks)
+29 -4
View File
@@ -4,6 +4,7 @@ from io import BytesIO
from celery import shared_task
from django.contrib.auth import get_user_model
from django.db.models import Q
from django.utils import timezone
from apps.articles.tasks import apply_filter_rules_for_feed
@@ -14,13 +15,23 @@ from .services.fetcher import effective_interval_minutes, fetch_and_ingest
logger = logging.getLogger(__name__)
# How long a claimed-but-unfinished fetch blocks the same feed from being
# claimed again — must comfortably exceed FETCH_TIMEOUT_SECONDS plus parse/
# ingest time, so a slow-but-healthy fetch is never raced by the next tick,
# while a worker that died mid-fetch doesn't wedge the feed forever.
CLAIM_STALE_MINUTES = 10
@shared_task
def poll_due_feeds():
"""Find every active Feed whose polling interval has elapsed and
dispatch a fetch_feed task for each — fan-out happens here, not via one
Celery Beat entry per feed.
"""Find every active Feed whose polling interval has elapsed, atomically
claim it (so overlapping beat ticks or a slow-running previous fetch
can't dispatch the same feed twice), and dispatch a fetch_feed task for
each claimed feed — fan-out happens here, not via one Celery Beat entry
per feed.
"""
now = timezone.now()
claim_stale_before = now - timedelta(minutes=CLAIM_STALE_MINUTES)
due_feed_ids = [
feed.id
for feed in Feed.objects.filter(is_active=True).only(
@@ -28,9 +39,22 @@ def poll_due_feeds():
)
if _is_due(feed)
]
claimed_ids = []
for feed_id in due_feed_ids:
# The UPDATE's WHERE clause re-checks staleness at the database
# level: it only claims rows nobody else has already claimed more
# recently than CLAIM_STALE_MINUTES ago, so two workers racing this
# same loop can't both get rowcount=1 for the same feed.
claimed = (
Feed.objects.filter(id=feed_id, is_active=True)
.filter(Q(claimed_at__isnull=True) | Q(claimed_at__lt=claim_stale_before))
.update(claimed_at=now)
)
if claimed:
claimed_ids.append(feed_id)
for feed_id in claimed_ids:
fetch_feed.delay(feed_id)
return len(due_feed_ids)
return len(claimed_ids)
def _is_due(feed: Feed) -> bool:
@@ -48,6 +72,7 @@ def fetch_feed(feed_id: int):
return None
result = fetch_and_ingest(feed)
feed.claimed_at = None
feed.save()
changed_article_ids = result.new_article_ids + result.updated_article_ids
+138
View File
@@ -0,0 +1,138 @@
import socket
import pytest
import responses
from apps.feeds.services.ssrf_guard import (
MAX_REDIRECTS,
MAX_RESPONSE_BYTES,
SSRFBlocked,
safe_get,
validate_fetch_url,
)
# apps/feeds/conftest.py's autouse _fake_public_dns fixture pins every
# hostname to this address unless a test below overrides it directly.
PUBLIC_IP = "93.184.216.34"
def _patch_dns(monkeypatch, ip: str):
def _fake_getaddrinfo(host, *args, **kwargs):
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", (ip, 0))]
monkeypatch.setattr(socket, "getaddrinfo", _fake_getaddrinfo)
class TestValidateFetchUrl:
def test_allows_a_public_https_url(self):
validate_fetch_url("https://example.com/feed.xml")
def test_rejects_a_non_http_scheme(self):
with pytest.raises(SSRFBlocked, match="scheme"):
validate_fetch_url("file:///etc/passwd")
def test_rejects_a_url_with_no_hostname(self):
with pytest.raises(SSRFBlocked, match="hostname"):
validate_fetch_url("https:///feed.xml")
@pytest.mark.parametrize(
"ip",
[
"127.0.0.1", # loopback
"10.0.0.5", # RFC1918 private
"192.168.1.1", # RFC1918 private
"169.254.169.254", # link-local / cloud metadata
"0.0.0.0", # unspecified
"::1", # IPv6 loopback
"fc00::1", # IPv6 unique local
],
)
def test_rejects_a_url_resolving_to_a_non_public_address(self, monkeypatch, ip):
_patch_dns(monkeypatch, ip)
with pytest.raises(SSRFBlocked, match="non-public"):
validate_fetch_url("https://internal.example.com/feed.xml")
def test_rejects_a_url_whose_hostname_does_not_resolve(self, monkeypatch):
def _raise(*args, **kwargs):
raise socket.gaierror("nope")
monkeypatch.setattr(socket, "getaddrinfo", _raise)
with pytest.raises(SSRFBlocked, match="Could not resolve"):
validate_fetch_url("https://nowhere.example.com/feed.xml")
class TestSafeGet:
@responses.activate
def test_returns_the_response_body_for_a_normal_feed(self):
responses.add(responses.GET, "https://example.com/feed.xml", body="<rss></rss>")
response = safe_get("https://example.com/feed.xml", headers={}, timeout=5)
assert response.status_code == 200
assert response.content == b"<rss></rss>"
@responses.activate
def test_follows_a_safe_redirect(self):
responses.add(
responses.GET,
"https://example.com/feed.xml",
status=302,
headers={"Location": "https://example.com/final.xml"},
)
responses.add(responses.GET, "https://example.com/final.xml", body="<rss>final</rss>")
response = safe_get("https://example.com/feed.xml", headers={}, timeout=5)
assert response.content == b"<rss>final</rss>"
@responses.activate
def test_blocks_a_redirect_to_a_private_address(self, monkeypatch):
responses.add(
responses.GET,
"https://example.com/feed.xml",
status=302,
headers={"Location": "http://internal.example.com/secret"},
)
real_getaddrinfo = socket.getaddrinfo
def _fake_getaddrinfo(host, *args, **kwargs):
if host == "internal.example.com":
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("127.0.0.1", 0))]
return real_getaddrinfo(host, *args, **kwargs)
monkeypatch.setattr(socket, "getaddrinfo", _fake_getaddrinfo)
with pytest.raises(SSRFBlocked, match="non-public"):
safe_get("https://example.com/feed.xml", headers={}, timeout=5)
@responses.activate
def test_blocks_a_redirect_loop_past_max_redirects(self):
for _ in range(MAX_REDIRECTS + 2):
responses.add(
responses.GET,
"https://example.com/feed.xml",
status=302,
headers={"Location": "https://example.com/feed.xml"},
)
with pytest.raises(SSRFBlocked, match="Too many redirects"):
safe_get("https://example.com/feed.xml", headers={}, timeout=5)
@responses.activate
def test_blocks_a_response_over_the_size_cap(self):
responses.add(
responses.GET,
"https://example.com/feed.xml",
body=b"x" * (MAX_RESPONSE_BYTES + 1),
)
with pytest.raises(SSRFBlocked, match="exceeded"):
safe_get("https://example.com/feed.xml", headers={}, timeout=5)
@responses.activate
def test_preserves_case_insensitive_header_lookup(self):
responses.add(
responses.GET,
"https://example.com/feed.xml",
body="<rss></rss>",
headers={"etag": "abc123"},
)
response = safe_get("https://example.com/feed.xml", headers={}, timeout=5)
assert response.headers.get("ETag") == "abc123"
+52
View File
@@ -40,6 +40,58 @@ def test_poll_due_feeds_dispatches_only_due_feeds():
assert inactive_but_overdue.id not in dispatched_ids
@pytest.mark.django_db
def test_poll_due_feeds_does_not_reclaim_a_feed_it_just_claimed():
# Models a second beat tick (or an overlapping worker) running
# poll_due_feeds before the first claimed fetch_feed has finished —
# the feed must only be dispatched once.
overdue = Feed.objects.create(
feed_url="https://c.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2),
fetch_interval_minutes=60,
)
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
first_count = poll_due_feeds()
second_count = poll_due_feeds()
assert first_count == 1
assert second_count == 0
assert mock_delay.call_count == 1
overdue.refresh_from_db()
assert overdue.claimed_at is not None
@pytest.mark.django_db
def test_poll_due_feeds_reclaims_a_stale_claim():
from apps.feeds.tasks import CLAIM_STALE_MINUTES
overdue = Feed.objects.create(
feed_url="https://c.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2),
fetch_interval_minutes=60,
claimed_at=timezone.now() - timedelta(minutes=CLAIM_STALE_MINUTES + 1),
)
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
dispatched_count = poll_due_feeds()
assert dispatched_count == 1
assert mock_delay.call_args.args[0] == overdue.id
@pytest.mark.django_db
@responses.activate
def test_fetch_feed_task_clears_its_claim():
feed = Feed.objects.create(feed_url="https://example.com/feed.xml", claimed_at=timezone.now())
responses.add(responses.GET, feed.feed_url, body=GOOD_FEED_XML, status=200)
fetch_feed(feed.id)
feed.refresh_from_db()
assert feed.claimed_at is None
GOOD_FEED_XML = """<?xml version="1.0"?>
<rss version="2.0"><channel>
<title>Example Blog</title>
+5
View File
@@ -32,6 +32,11 @@ def test_service_worker_is_served_at_root_with_correct_type_and_scope(client):
assert 'self.addEventListener("fetch"' in body
# Mutations must stay network-only, never intercepted.
assert 'request.method !== "GET"' in body
# Logging out must purge PAGES_CACHE so a later offline navigation on
# a shared/account-switched device can't serve another user's
# previously cached authenticated pages.
assert "caches.delete(PAGES_CACHE)" in body
assert reverse("accounts:logout") in body
@pytest.mark.django_db
+68 -4
View File
@@ -9,6 +9,7 @@ from __future__ import annotations
import logging
import re
import signal
from django.utils import timezone
@@ -27,6 +28,42 @@ logger = logging.getLogger(__name__)
_TAG_RE = re.compile(r"<[^>]+>")
# Bounds how much CPU/time a single user-authored regex rule can burn
# against one article. re has no built-in match timeout, so this uses
# SIGALRM to interrupt a runaway match (catastrophic backtracking) —
# only effective on the main thread of a process with SIGALRM available,
# which covers gunicorn's sync workers and Celery's (default) prefork
# pool; elsewhere it's skipped and MAX_REGEX_HAYSTACK_CHARS is the only
# bound. reapply_rules_for_user (apps.rules.tasks) also carries a Celery
# task time_limit as a second, coarser backstop.
REGEX_TIMEOUT_SECONDS = 1
MAX_REGEX_HAYSTACK_CHARS = 20_000
class _RegexTimeout(Exception):
pass
def _raise_regex_timeout(signum, frame):
raise _RegexTimeout()
def _regex_search_with_timeout(pattern: str, haystack: str, flags: int):
if not hasattr(signal, "SIGALRM"):
return re.search(pattern, haystack, flags)
try:
previous_handler = signal.signal(signal.SIGALRM, _raise_regex_timeout)
except ValueError:
# Not the main thread (e.g. a threaded dev server) — signal
# timers only work there.
return re.search(pattern, haystack, flags)
signal.setitimer(signal.ITIMER_REAL, REGEX_TIMEOUT_SECONDS)
try:
return re.search(pattern, haystack, flags)
finally:
signal.setitimer(signal.ITIMER_REAL, 0)
signal.signal(signal.SIGALRM, previous_handler)
def _haystack(article: Article, match_field: str) -> str:
if match_field == FilterRule.FIELD_TITLE:
@@ -47,8 +84,12 @@ def rule_matches(rule: FilterRule, article: Article) -> bool:
if rule.match_type == FilterRule.MATCH_REGEX:
flags = 0 if rule.case_sensitive else re.IGNORECASE
haystack = raw_haystack[:MAX_REGEX_HAYSTACK_CHARS]
try:
return re.search(rule.match_value, raw_haystack, flags) is not None
return _regex_search_with_timeout(rule.match_value, haystack, flags) is not None
except _RegexTimeout:
logger.warning("Regex timed out for rule %s", rule.pk)
return False
except re.error as exc:
logger.warning("Invalid regex in rule %s: %s", rule.pk, exc)
return False
@@ -100,13 +141,36 @@ def apply_rule_action(rule: FilterRule, user, article: Article, state: UserArtic
return False
def evaluate_rules_for_article(user, article: Article, rules) -> None:
"""Evaluates an ordered iterable of rules (already scoped/enabled)
against one article for one user, applying actions as they match."""
def rule_in_scope(rule: FilterRule, feed_id, subscription_tag_ids) -> bool:
"""Whether `rule`'s own feed/tag scope (see FilterRule's docstring)
covers an article from `feed_id`, given the user's subscription-tag
ids for that feed.
Centralized here — rather than left for each caller to re-derive via
its own query filters — so ingest-time dispatch and backlog re-apply
can't drift into scoping rules differently (see apps.articles.tasks
and apps.rules.tasks).
"""
if rule.feed_id is not None and rule.feed_id != feed_id:
return False
if rule.tag_id is not None and rule.tag_id not in subscription_tag_ids:
return False
return True
def evaluate_rules_for_article(user, article: Article, rules, subscription_tag_ids=()) -> None:
"""Evaluates an ordered iterable of a user's enabled rules against one
article, applying actions as they match. `rules` doesn't need to be
pre-scoped to this article's feed/tags — out-of-scope rules (per
rule_in_scope) are skipped here, so callers can pass one user's whole
enabled ruleset regardless of which feed(s) the article(s) belong to.
"""
matched_any = False
state = None
for rule in rules:
if not rule_in_scope(rule, article.feed_id, subscription_tag_ids):
continue
if not rule_matches(rule, article):
continue
if state is None:
+24 -6
View File
@@ -1,6 +1,6 @@
from celery import shared_task
from apps.articles.models import Article
from apps.articles.models import Article, SubscriptionTag
from apps.feeds.models import Subscription
from .engine import evaluate_rules_for_article
@@ -12,9 +12,20 @@ def reapply_rules_for_user_backlog(rule_id: int) -> int:
rule, so priority/short-circuit ordering stays correct) against their
unread backlog, scoped by the triggering rule's own feed/tag scope.
Exists as a plain function so it can be called synchronously for
immediate feedback (e.g. from a "Re-apply" view) or wrapped in a task
for background use — see reapply_rules_for_user below.
The *backlog* is narrowed to the triggering rule's own feed/tag scope
(re-applying a feed-scoped rule has no reason to touch other feeds'
articles), but which of the user's *other* rules apply to each of
those articles is still per-article/per-feed — evaluate_rules_for_article
checks that via rule_in_scope, so a feed- or tag-scoped rule for a
different feed can't fire on this backlog just because it happened to
match (see apps.rules.engine.rule_in_scope).
Exists as a plain function, separate from the reapply_rules_for_user
task below, so it's unit-testable without Celery — but every caller
should go through that task (see apps.rules.views.reapply_rule), not
call this directly from a request: a large backlog or a pathological
regex (bounded but not eliminated by apps.rules.engine's timeout
guard) could otherwise tie up a web worker for a long time.
"""
try:
rule = FilterRule.objects.select_related("user").get(pk=rule_id)
@@ -45,13 +56,20 @@ def reapply_rules_for_user_backlog(rule_id: int) -> int:
rules = list(FilterRule.objects.filter(user=user, is_enabled=True).order_by("priority", "id"))
subscription_tag_ids_by_feed: dict[int, set[int]] = {}
for feed_id, tag_id in SubscriptionTag.objects.filter(
user=user, subscription__feed_id__in=feed_ids
).values_list("subscription__feed_id", "tag_id"):
subscription_tag_ids_by_feed.setdefault(feed_id, set()).add(tag_id)
count = 0
for article in backlog:
evaluate_rules_for_article(user, article, rules)
subscription_tag_ids = subscription_tag_ids_by_feed.get(article.feed_id, set())
evaluate_rules_for_article(user, article, rules, subscription_tag_ids)
count += 1
return count
@shared_task
@shared_task(time_limit=120, soft_time_limit=90)
def reapply_rules_for_user(rule_id: int):
return reapply_rules_for_user_backlog(rule_id)
+37 -1
View File
@@ -1,3 +1,5 @@
import time
import pytest
from django.contrib.auth import get_user_model
@@ -10,7 +12,7 @@ from apps.articles.models import (
)
from apps.feeds.models import Feed
from .engine import evaluate_rules_for_article, rule_matches
from .engine import evaluate_rules_for_article, rule_in_scope, rule_matches
from .models import FilterRule
User = get_user_model()
@@ -87,6 +89,22 @@ class TestRuleMatches:
rule = make_rule(user, match_type=FilterRule.MATCH_REGEX, match_value="(unclosed")
assert rule_matches(rule, article) is False
def test_catastrophic_backtracking_regex_times_out_instead_of_hanging(self, user, article):
# (a+)+$ against a long run of a's followed by a non-matching
# character is a textbook ReDoS pattern — left unguarded this
# takes on the order of a minute (confirmed locally); the
# REGEX_TIMEOUT_SECONDS guard in apps.rules.engine must interrupt
# it well before that instead of hanging the caller.
article.title = "a" * 30 + "!"
rule = make_rule(user, match_type=FilterRule.MATCH_REGEX, match_value=r"(a+)+$")
start = time.monotonic()
result = rule_matches(rule, article)
elapsed = time.monotonic() - start
assert result is False
assert elapsed < 5
def test_content_field_strips_html_tags(self, user, article):
rule = make_rule(user, match_field=FilterRule.FIELD_CONTENT, match_value="Lots of details")
assert rule_matches(rule, article) is True
@@ -104,6 +122,24 @@ class TestRuleMatches:
assert rule_matches(rule, article) is True
@pytest.mark.django_db
class TestRuleInScope:
def test_unscoped_rule_applies_to_any_feed(self, user):
rule = make_rule(user)
assert rule_in_scope(rule, feed_id=999, subscription_tag_ids=set()) is True
def test_feed_scoped_rule_only_applies_to_its_own_feed(self, user, feed):
rule = make_rule(user, feed=feed)
assert rule_in_scope(rule, feed_id=feed.id, subscription_tag_ids=set()) is True
assert rule_in_scope(rule, feed_id=feed.id + 1, subscription_tag_ids=set()) is False
def test_tag_scoped_rule_only_applies_when_the_feed_carries_that_tag(self, user):
tag = Tag.objects.create(user=user, name="work")
rule = make_rule(user, tag=tag)
assert rule_in_scope(rule, feed_id=1, subscription_tag_ids={tag.id}) is True
assert rule_in_scope(rule, feed_id=1, subscription_tag_ids=set()) is False
@pytest.mark.django_db
class TestApplyRuleAction:
def test_star_action(self, user, article):
+103 -1
View File
@@ -1,8 +1,17 @@
from unittest.mock import patch
import pytest
from django.contrib.auth import get_user_model
from django.urls import reverse
from apps.articles.models import STARRED_TAG_NAME, Article, ArticleTag, UserArticleState
from apps.articles.models import (
STARRED_TAG_NAME,
Article,
ArticleTag,
SubscriptionTag,
Tag,
UserArticleState,
)
from apps.feeds.models import Feed, Subscription
from .models import FilterRule
@@ -119,6 +128,28 @@ def test_reapply_rule_view_applies_to_unread_backlog(client, alice, feed):
assert is_starred(alice, article) is True
@pytest.mark.django_db
def test_reapply_rule_view_always_queues_through_celery_never_runs_inline(client, alice, feed):
# A large backlog or a pathological regex must not be able to tie up
# the request/response cycle — the view should dispatch a task, not
# call reapply_rules_for_user_backlog directly.
Subscription.objects.create(user=alice, feed=feed)
rule = FilterRule.objects.create(
user=alice,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_STAR,
)
client.login(username="alice", password="pw-alice-123")
with patch("apps.rules.views.reapply_rules_for_user.delay") as mock_delay:
response = client.post(reverse("rules:reapply_rule", args=[rule.pk]))
assert response.status_code == 302
mock_delay.assert_called_once_with(rule.pk)
@pytest.mark.django_db
def test_reapply_skips_already_read_articles(alice, feed):
article = Article.objects.create(feed=feed, guid="guid-1", title="Python 4 released")
@@ -160,3 +191,74 @@ def test_reapply_respects_feed_scope(alice):
assert is_starred(alice, article_a) is True
assert not UserArticleState.objects.filter(user=alice, article=article_b).exists()
@pytest.mark.django_db
def test_reapply_does_not_apply_a_different_feeds_scoped_rule_to_the_backlog():
# Regression test for a bug where re-applying one feed-scoped rule
# loaded *every* enabled rule and ran it against the whole backlog
# with no scope check — so a same-matching rule scoped to a feed
# the user isn't re-applying would still fire on this backlog.
feed_a = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
feed_b = Feed.objects.create(feed_url="https://b.example.com/feed.xml")
alice = User.objects.create_user("alice", password="pw-alice-123")
Subscription.objects.create(user=alice, feed=feed_a)
Subscription.objects.create(user=alice, feed=feed_b)
article_a = Article.objects.create(feed=feed_a, guid="guid-1", title="Python in A")
triggering_rule = FilterRule.objects.create(
user=alice,
feed=feed_a,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_STAR,
)
FilterRule.objects.create(
user=alice,
feed=feed_b,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_TAG,
action_value="feed-b-only",
)
reapply_rules_for_user_backlog(triggering_rule.pk)
assert is_starred(alice, article_a) is True
assert not Tag.objects.filter(user=alice, name="feed-b-only").exists()
@pytest.mark.django_db
def test_reapply_applies_a_tag_scoped_rule_only_to_that_tags_feeds():
feed_a = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
feed_b = Feed.objects.create(feed_url="https://b.example.com/feed.xml")
alice = User.objects.create_user("alice", password="pw-alice-123")
sub_a = Subscription.objects.create(user=alice, feed=feed_a)
Subscription.objects.create(user=alice, feed=feed_b)
tag = Tag.objects.create(user=alice, name="work")
SubscriptionTag.objects.create(user=alice, subscription=sub_a, tag=tag)
article_a = Article.objects.create(feed=feed_a, guid="guid-1", title="Python in A")
article_b = Article.objects.create(feed=feed_b, guid="guid-2", title="Python in B")
unscoped_trigger = FilterRule.objects.create(
user=alice,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="unrelated-trigger-value",
action=FilterRule.ACTION_MARK_READ,
)
FilterRule.objects.create(
user=alice,
tag=tag,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_STAR,
)
reapply_rules_for_user_backlog(unscoped_trigger.pk)
assert is_starred(alice, article_a) is True
assert is_starred(alice, article_b) is False
+6 -5
View File
@@ -5,7 +5,7 @@ from django.views.decorators.http import require_POST
from .forms import FilterRuleForm
from .models import FilterRule
from .tasks import reapply_rules_for_user_backlog
from .tasks import reapply_rules_for_user
@login_required
@@ -48,8 +48,9 @@ def delete_rule(request, rule_id):
@require_POST
def reapply_rule(request, rule_id):
rule = get_object_or_404(FilterRule, user=request.user, pk=rule_id)
# Run synchronously for immediate feedback, same pattern as add_feed —
# a Celery worker isn't required for this to work.
count = reapply_rules_for_user_backlog(rule.pk)
messages.success(request, f"Re-applied your rules to {count} unread article(s).")
# Always queued, never run inline from the request/response cycle —
# a large backlog or a pathological regex could otherwise tie up a
# web worker for a long time (see apps.rules.tasks).
reapply_rules_for_user.delay(rule.pk)
messages.success(request, "Re-applying your rules to your unread backlog…")
return redirect("rules:rule_list")
+28 -1
View File
@@ -1,8 +1,30 @@
from django.core.exceptions import ImproperlyConfigured
from .base import * # noqa: F403
from .base import env
from .base import SECRET_KEY, env
DEBUG = False
# base.py falls back to a publicly known placeholder if SECRET_KEY isn't
# set — fine for a throwaway dev DB, but production must never sign
# sessions/CSRF tokens with a key anyone can read in this repo's history.
# Fail startup rather than quietly running with a known key.
if not SECRET_KEY or SECRET_KEY == "django-insecure-change-me-in-production":
raise ImproperlyConfigured(
"SECRET_KEY is missing or still the placeholder default. Set a unique, "
"random SECRET_KEY in .env before running with prod settings — generate "
'one with: python -c "from django.core.management.utils import '
'get_random_secret_key; print(get_random_secret_key())"'
)
# Health checks (scripts/upgrade.sh, an external uptime monitor, ...) hit
# gunicorn directly over loopback, bypassing the reverse proxy entirely —
# that request's Host header is "127.0.0.1:<port>", not the public domain
# in ALLOWED_HOSTS, so without this Django 400s it. Harmless to always
# allow: gunicorn's bind address is never reachable from outside this host
# in the first place (see docs/install.md's "Reverse proxy" section).
ALLOWED_HOSTS = [*ALLOWED_HOSTS, "localhost", "127.0.0.1", "::1"] # noqa: F405
# A reverse proxy (nginx, Caddy, Traefik, Nginx Proxy Manager on another
# host, ...) is required in front of gunicorn in every deployment — see
# docs/install.md — and it's always the one terminating TLS. This tells
@@ -10,6 +32,11 @@ DEBUG = False
# sets, and the redirect/cookie/HSTS settings below assume it's set.
SECURE_SSL_REDIRECT = env.bool("SECURE_SSL_REDIRECT", default=True)
SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")
# /health/ is also exempted from the HTTPS redirect above: hit directly
# over loopback (see ALLOWED_HOSTS above) it has no X-Forwarded-Proto
# header to prove it's "secure", and there's no TLS listener on gunicorn's
# own port to redirect to anyway.
SECURE_REDIRECT_EXEMPT = [r"^health/$"]
# WhiteNoise serves /static/ directly from gunicorn instead of relying on
# the reverse proxy to alias the staticfiles directory on disk — the proxy
+5 -4
View File
@@ -295,6 +295,7 @@ git fetch --tags --force
LATEST_TAG="$(git tag --sort=-creatordate | head -n1)"
git checkout --detach "$LATEST_TAG"
chown -R evection:evection /opt/evection/app
sed -i "s/^EVECTION_PWA_CACHE_VERSION=.*/EVECTION_PWA_CACHE_VERSION=$LATEST_TAG/" .env
sudo -u evection -H /opt/evection/venv/bin/pip install .
sudo -u evection -H env DJANGO_SETTINGS_MODULE=config.settings.prod \
/opt/evection/venv/bin/python manage.py migrate
@@ -311,7 +312,7 @@ git tag v$(date +%Y.%m.%d)
git push --tags
```
Bump `EVECTION_PWA_CACHE_VERSION` in `.env` whenever a release changes cached
static assets (CSS/JS/icons) — it's baked into the service worker's cache name, so
changing it forces every installed PWA to drop its old cache instead of serving
stale assets after the upgrade.
`upgrade.sh` sets `EVECTION_PWA_CACHE_VERSION` in `.env` to the deployed tag on every
run — it's baked into the service worker's cache name, so this forces every installed
PWA to drop its old cache instead of serving stale assets after the upgrade. You don't
need to bump it by hand.
+40 -4
View File
@@ -63,9 +63,45 @@ if ! "$PYTHON_BIN" -c 'import ensurepip' >/dev/null 2>&1; then
exit 1
fi
if ! command -v redis-cli >/dev/null 2>&1; then
echo "Warning: 'redis-cli' not found on PATH. Install it before going live" \
"(see docs/install.md's prerequisites section: apt install redis-server)." >&2
# Celery (and therefore all feed fetching) is entirely dependent on Redis
# being reachable — it fails silently from the installer's point of view
# (the worker/beat processes just retry forever in the background), so this
# has to be a hard failure here, not a warning that's easy to miss while
# every later step "succeeds" at installing an app that can never fetch a
# single feed. Check the actual REDIS_URL that will end up in .env (an
# explicit EVECTION_REDIS_URL override, if set — the default local Redis
# otherwise), not just whether some redis-cli happens to be on PATH.
REDIS_URL_TO_CHECK="${EVECTION_REDIS_URL:-redis://localhost:6379/0}"
check_redis() {
"$PYTHON_BIN" -c "
import sys, socket, urllib.parse
u = urllib.parse.urlparse('$REDIS_URL_TO_CHECK')
try:
with socket.create_connection((u.hostname or 'localhost', u.port or 6379), timeout=3):
pass
except OSError as e:
print(e, file=sys.stderr)
sys.exit(1)
"
}
REDIS_HOST="$("$PYTHON_BIN" -c "import urllib.parse; print(urllib.parse.urlparse('$REDIS_URL_TO_CHECK').hostname or 'localhost')")"
if ! check_redis; then
if [[ "$REDIS_HOST" =~ ^(localhost|127\.0\.0\.1|::1)$ ]] && command -v apt-get >/dev/null 2>&1; then
echo "==> Redis isn't reachable at $REDIS_URL_TO_CHECK — installing redis-server"
apt-get install -y redis-server
systemctl enable --now redis-server
if ! check_redis; then
echo "Redis still isn't reachable after installing redis-server —" \
"check: systemctl status redis-server" >&2
exit 1
fi
else
echo "Redis isn't reachable at $REDIS_URL_TO_CHECK. Evection's background" \
"feed-fetching depends on it entirely, so this has to be fixed before" \
"continuing — either start/install Redis there, or set" \
"EVECTION_REDIS_URL to point at wherever it actually runs." >&2
exit 1
fi
fi
# --- 2. System user -----------------------------------------------------
@@ -168,7 +204,7 @@ REDIS_URL=${EVECTION_REDIS_URL:-redis://localhost:6379/0}
EVECTION_GUNICORN_BIND=$GUNICORN_BIND
EVECTION_SIGNUP_ENABLED=False
EVECTION_MIN_FETCH_INTERVAL_MINUTES=15
EVECTION_PWA_CACHE_VERSION=v1
EVECTION_PWA_CACHE_VERSION=${LATEST_TAG:-v1}
EOF
chown "$EVECTION_USER:$EVECTION_USER" "$ENV_FILE"
chmod 600 "$ENV_FILE"
+12 -3
View File
@@ -72,6 +72,18 @@ echo "Deploying $LATEST_TAG (currently on: $CURRENT_REF)"
git checkout --quiet --detach "$LATEST_TAG"
chown -R "$EVECTION_USER:$EVECTION_USER" "$APP_DIR"
# Bump the PWA's cache-busting version to this release's tag on every
# deploy, not just ones that happened to touch static assets — a missed
# manual bump otherwise leaves installed PWAs serving a stale cached
# shell/pages indefinitely. Cheap even when nothing cacheable changed:
# it just makes the service worker open a fresh cache name next time it
# activates.
if grep -q '^EVECTION_PWA_CACHE_VERSION=' "$APP_DIR/.env"; then
sed -i "s/^EVECTION_PWA_CACHE_VERSION=.*/EVECTION_PWA_CACHE_VERSION=$LATEST_TAG/" "$APP_DIR/.env"
else
echo "EVECTION_PWA_CACHE_VERSION=$LATEST_TAG" >> "$APP_DIR/.env"
fi
echo "==> Installing updated dependencies"
sudo -u "$EVECTION_USER" -H "$VENV_DIR/bin/pip" install --upgrade "$APP_DIR"
@@ -131,7 +143,4 @@ systemctl status evection-gunicorn --no-pager -l | head -10
echo
echo "==> Deployed $LATEST_TAG."
echo " If this release changed cached static assets (CSS/JS/icons), bump"
echo " EVECTION_PWA_CACHE_VERSION in $APP_DIR/.env and restart evection-gunicorn"
echo " again so installed PWAs drop their old cache instead of serving stale assets."
echo " journalctl -u evection-gunicorn -f to watch logs."
+7 -1
View File
@@ -300,12 +300,18 @@ form p {
}
.sidebar-tag-link {
display: inline-block;
padding: 0.15rem 0.4rem;
margin: -0.15rem -0.4rem;
border-radius: 0.375rem;
color: inherit;
text-decoration: none;
}
.sidebar-tag-link.active {
color: var(--accent);
background: var(--bg-row-selected);
color: var(--text);
font-weight: 600;
}
.sidebar-item-list {
+15 -1
View File
@@ -66,8 +66,23 @@ async function cacheFirstForAsset(request) {
return response;
}
const LOGOUT_PATH = "{% url 'accounts:logout' %}";
self.addEventListener("fetch", (event) => {
const { request } = event;
const url = new URL(request.url);
// Logging out ends this device's session, so any authenticated HTML
// sitting in PAGES_CACHE must not outlive it — otherwise a later
// offline navigation (shared device, different account logging in
// next) could still serve a previous user's cached article pages.
// Purging it synchronously with the logout request itself, rather
// than via a postMessage from the page, means there's no race with
// the navigation that follows.
if (request.method === "POST" && url.pathname === LOGOUT_PATH) {
event.respondWith(caches.delete(PAGES_CACHE).then(() => fetch(request)));
return;
}
// Mutations (mark-read, star, OPML upload, forms...) are always
// network-only — never intercept anything but GET.
@@ -78,7 +93,6 @@ self.addEventListener("fetch", (event) => {
return;
}
const url = new URL(request.url);
if (url.pathname.startsWith("/static/")) {
event.respondWith(cacheFirstForAsset(request));
}