Compare commits

..
Author SHA1 Message Date
ksmithandClaude Sonnet 5 90837b339c Make the Site settings admin page go straight to the edit form
It's a true singleton, not a collection, so the normal changelist (an
"Add" link, a list with 0 or 1 rows, an extra click to get to the one
row) was just friction — and visually inconsistent with every other
app's admin section, which do show an "Add" link. The changelist view
now redirects straight to the single row's change form (creating it
first if this is a fresh install), and "Add" is never offered at all.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 13:23:47 -06:00
ksmithandClaude Sonnet 5 dc8b9ff83b Make the feed-poll interval site-wide instead of per-feed
Replaces Feed.fetch_interval_minutes with a single admin-editable
SiteSettings.fetch_interval_minutes (apps.core), following the same
singleton-row pattern as accounts.SetupLock. Per-feed error backoff is
unchanged — only the configured baseline interval is now shared across
every feed, loaded once per poll_due_feeds run rather than queried per
feed. EVECTION_MIN_FETCH_INTERVAL_MINUTES keeps working as the floor
underneath it.

SiteSettings is registered in Django admin as a singleton: "Add" is
only offered when no row exists yet, and delete is always refused.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:59:47 -06:00
ksmithandClaude Sonnet 5 a1ea2e5662 Add login rate limiting (S6)
Django's default LoginView had no throttle — an internet-facing
instance was exposed to unlimited password-guessing/credential-
stuffing attempts. apps.accounts.views.LoginView adds two independent,
Redis-cache-backed throttles (shared across all gunicorn workers,
unlike the per-process default LocMemCache): by client address (catches
one attacker spraying many usernames) and by the submitted username
(catches a distributed attack on one specific account). Either blocks
further attempts for a 5-minute window; both are generous enough not
to lock out a user who's just mistyped their own password a few times.

Client address prefers X-Forwarded-For's first hop, falling back to
REMOTE_ADDR — the same reverse-proxy trust boundary config.settings.prod
already relies on for X-Forwarded-Proto (gunicorn is never meant to be
reachable except through that proxy).

CACHES now points at Redis (already a hard install requirement for
Celery) instead of Django's implicit per-process default, since the
whole point is a shared counter across every gunicorn worker; test
settings override it back to LocMemCache so the suite doesn't need a
real Redis instance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:46:49 -06:00
ksmithandClaude Sonnet 5 9241a54958 Cap entries per poll and stored content size (T4)
ssrf_guard.MAX_RESPONSE_BYTES (S1) already bounds the raw download, but
a feed within that byte cap can still carry an unreasonable number of
entries, or one entry with an unreasonably large body — both cost real
CPU/DB time (nh3 sanitization, an Article row/UPDATE) per entry.
fetch_and_ingest now processes at most MAX_ENTRIES_PER_POLL (200)
entries per poll (logging a warning when truncated) and sanitize_html
bounds both its input (sanitizer cost) and output (what's actually
stored/rendered) length.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:46:40 -06:00
ksmithandClaude Sonnet 5 64f25789f5 Isolate unexpected fetch failures and harden task durability (T2)
fetch_feed had no time limit and no handling for anything beyond the
network/parse errors fetch_and_ingest already catches internally — an
unexpected exception (a parser/sanitizer bug, a DB error) crashed the
task with no error recorded on the Feed, the claim left held for the
full CLAIM_STALE_MINUTES, and no visibility beyond a bare traceback.
fetch_feed now catches that, records it the same way a handled failure
would be (reusing fetcher.record_fetch_error, promoted from a private
helper), clears the claim, and re-raises so Celery still marks the task
FAILED. soft_time_limit/time_limit bound a single fetch that somehow
outruns FETCH_TIMEOUT_SECONDS + ssrf_guard's response cap during
parsing/ingestion.

Also: a worker process killed mid-task (OOM, deploy restart, crash)
previously lost that task silently — Celery's default is to ack a
message the moment a worker *starts* it. CELERY_TASK_ACKS_LATE +
CELERY_TASK_REJECT_ON_WORKER_LOST (config/settings/base.py) make the
message redeliverable instead, and CELERY_WORKER_PREFETCH_MULTIPLIER=1
keeps a worker from grabbing a batch of tasks it could lose all at once.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:46:29 -06:00
ksmithandClaude Sonnet 5 0466ccc334 Fix sidebar disclosure triangle rendering above the tag name
summary's native ::marker triangle is positioned for inline content —
once the tag-name + unread-count row needed to be a flex box (for
v2026.10.31-15's tag unread counts), the marker got pushed onto its
own line above instead of sitting beside it, confirmed via a
screenshot after deploying that release.

Replaces the native marker with a plain ::before triangle (hiding
-webkit-details-marker too) so `summary` itself can safely be the flex
row, with the triangle as a normal flex item that stays inline no
matter what the rest of the row contains. Verified in the browser,
including that collapse/expand still works and the triangle still
rotates on open.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:39:55 -06:00
ksmithandClaude Sonnet 5 e760eeda1d Show each tag's total unread count and remember collapsed sections
Closes gitea#24: the sidebar's per-feed unread counts were already
computed by sidebar() (apps.core.context_processors); each tag group's
total is now the sum of its own feeds' counts (a feed can carry more
than one tag, so this reuses the same per-feed data rather than a
second query) and renders next to the tag name like a feed's own
count.

Closes gitea#25: each sidebar <details> section (one per tag, plus
"Untagged") now carries a stable data-sidebar-key, and
static/js/sidebar-state.js remembers which ones are collapsed in
localStorage — this isn't a SPA, so every click re-renders the sidebar
server-side, and previously every section reset back to expanded on
the next page load. Per-browser, same pattern as theme.js/theme-init.js,
not synced across devices/accounts.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:34:57 -06:00
ksmithandClaude Sonnet 5 9439f9cc59 Add SSRF protection to feed fetching: validate scheme/address, bound redirects and response size (S1)
fetch_and_ingest fetched a feed_url supplied straight from the "add
feed" form or an uploaded OPML file, with requests following redirects
on its own and no check that the target (or a redirect's target) was
actually a public, fetchable feed rather than an internal service,
localhost, or a cloud metadata endpoint — reachable from wherever this
app runs, on any authenticated user's behalf.

apps.feeds.services.ssrf_guard.safe_get() replaces the direct
requests.get() call: it resolves and validates the hostname (rejecting
anything that isn't a globally routable address) before the initial
request and before following each redirect hop manually, caps the
number of hops, and bounds the response body size by streaming it
instead of buffering it unbounded. OPML-imported feeds go through the
same check the first time poll_due_feeds fetches them, since this is
the one chokepoint every feed URL passes through regardless of which
path created the Feed row.

Known residual gap, documented in ssrf_guard.py: this validates then
connects rather than pinning the TCP connection to the exact address it
validated, so a narrow DNS-rebinding race isn't closed. Fixing that
needs a custom transport and felt like more machinery than this
self-hosted app's threat model currently justifies.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:25:20 -06:00
ksmithandClaude Sonnet 5 fd8eeb605e Scope backlog re-apply per-feed and always queue it through Celery (T5, S5)
Completes the T5 fix: reapply_rules_for_user_backlog now computes each
feed's subscription-tag ids up front and passes them to
evaluate_rules_for_article per article, so a feed/tag-scoped rule from
a different feed can't fire on this backlog (see the previous commit's
rule_in_scope() in apps.rules.engine).

Also completes S5's second half: the "Re-apply" view now always
dispatches reapply_rules_for_user.delay() instead of calling the
backlog function inline from the request — a large backlog or a
regex that outruns the new timeout guard no longer risks tying up a
web worker.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:20:25 -06:00
ksmithandClaude Sonnet 5 91c0b1dd26 Centralize rule feed/tag scoping in the engine and fix backlog re-apply (T5)
reapply_rules_for_user_backlog narrowed its *backlog* to the triggering
rule's feed/tag scope, but then loaded every enabled rule for the user
and ran all of them against that backlog with no per-rule scope check
— so re-applying one feed- or tag-scoped rule could also fire an
unrelated rule scoped to a different feed/tag, if it happened to match.

rule_in_scope() moves the feed/tag scope check into the engine itself
(apply_filter_rules_for_feed already computed this per-subscription via
query filters; it now just passes the tag ids and lets the engine
decide), so backlog re-apply and ingest-time dispatch can't drift into
scoping rules differently again.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:19:56 -06:00
ksmithandClaude Sonnet 5 f2ee793cf8 upgrade.sh: auto-bump EVECTION_PWA_CACHE_VERSION on every deploy
The release-notes reminder to manually bump this and restart gunicorn
was easy to miss or forget, leaving installed PWAs on a stale cached
shell/pages after an upgrade that changed static assets. upgrade.sh
now sets it to the deployed tag itself before restarting services, so
there's no manual step and no cache-invalidation window to miss.
install.sh does the same on first install for consistency.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:08:05 -06:00
ksmithandClaude Sonnet 5 b7d9ee672c Add an atomic per-feed claim to poll_due_feeds (T1)
poll_due_feeds scanned for due feeds and dispatched fetch_feed without
marking them claimed, so overlapping beat ticks or a slow-running
fetch could dispatch the same feed twice, racing on ETag/status/error-
count writes. Feed.claimed_at is now set via an UPDATE ... WHERE
claimed_at IS NULL OR claimed_at < stale-cutoff, so only one caller's
UPDATE can ever match a given row; fetch_feed clears it when done.
A claim older than CLAIM_STALE_MINUTES is treated as abandoned so a
worker dying mid-fetch doesn't wedge the feed forever.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:03:18 -06:00
ksmithandClaude Sonnet 5 d7b433e580 Purge the service worker's authenticated-page cache on logout (S3)
PAGES_CACHE held every cached navigation response (including
authenticated article pages) with no logout-time purge, so a later
offline navigation on a shared or account-switched device could still
serve another user's cached pages. Logout is a plain POST, so the
fetch handler now special-cases that request: it deletes PAGES_CACHE
and lets the real logout request through, with no race against the
navigation that follows since it's synchronous with the same request.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:03:10 -06:00
ksmithandClaude Sonnet 5 b52ba2be8a Make first-run setup race-safe with an atomic SetupLock claim (S4)
The setup view's User.objects.exists() check and the admin account's
save() weren't in the same transaction, so two concurrent requests
during first deployment could both pass the check and both create an
admin account. SetupLock is a singleton row (pk=1); claiming it and
creating the admin now happen in one atomic block, so only the first
of any racing requests can succeed — the rest hit IntegrityError and
are sent to the login page instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:03:05 -06:00
ksmithandClaude Sonnet 5 56eb2bc498 Fail closed in production if SECRET_KEY is missing or the placeholder default (S2)
base.py's env.str() fallback is a publicly known string in this repo's
history. Running prod settings with it (an unset or forgotten .env
value) would sign sessions/CSRF tokens with a key anyone can read.
install.sh already generates a real random key for fresh installs, so
this only blocks the broken case.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:02:58 -06:00
ksmithandClaude Sonnet 5 9d91da6fc6 Fix /health/ 400ing when hit directly over loopback (as upgrade.sh does)
scripts/upgrade.sh's health check connects straight to gunicorn's bind
address, bypassing the reverse proxy entirely, so the request's Host
header is "127.0.0.1:<port>" and it has no X-Forwarded-Proto header.
Against prod settings that's exactly a DisallowedHost (ALLOWED_HOSTS only
has the public domain) — which Django turns into a 400 — and would have
then hit a redirect-to-HTTPS loop from SECURE_SSL_REDIRECT right after,
since there's no TLS listener on that port to redirect to. Allow the
loopback addresses in ALLOWED_HOSTS (harmless: gunicorn's bind address is
never reachable from outside this host per docs/install.md) and exempt
/health/ from the HTTPS redirect specifically.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:19:27 -06:00
ksmithandClaude Sonnet 5 c316f411f8 Make the active-tag sidebar highlight as visible as the active-feed one
.sidebar-tag-link.active only changed text color, which is easy to miss
next to .sidebar-feed-link.active's background pill + bold weight. Give
the tag link the same background-pill treatment (padding/margin offset
so the box doesn't shift layout when inactive).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:17:45 -06:00
ksmithandClaude Sonnet 5 e1907ff587 install.sh: fail hard (or auto-install) when Redis isn't actually reachable
The old check only looked for redis-cli on PATH and printed a warning if
missing — it never tested an actual connection, so a host with no Redis
installed at all still sailed through the entire install looking
successful. The real failure only showed up later as every feed stuck on
"pending" forever, with the worker/beat units quietly retrying their
connection in the background rather than surfacing anything to whoever
ran the script. Actually connect to the REDIS_URL that will end up in
.env; on Debian/Ubuntu with the default local address, install and start
redis-server automatically; otherwise fail the install outright with a
clear message instead of a warning that's easy to miss.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:12:41 -06:00
ksmithandClaude Sonnet 5 b0d62c91f8 upgrade.sh: use the venv's Python for the health check, not curl
curl isn't installed by default on this install's host, and nothing else
in install.sh/upgrade.sh actually requires it — the venv's own Python
(which obviously exists by the time this check runs) can make the same
HTTP request via urllib, so drop the curl dependency entirely rather than
adding it to the prerequisites.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:08:48 -06:00
ksmithandClaude Sonnet 5 2068df471f install.sh: actually restart services when their unit files change
systemctl enable --now only starts a service if it's stopped — on a
re-run where a unit file's ExecStart changed (e.g. picking up a new
EVECTION_GUNICORN_BIND), the already-running process just kept running
on its old command line indefinitely. This is exactly what happened
switching gunicorn from a Unix socket to TCP: the new unit file was
written and daemon-reload'd, but gunicorn itself never restarted, so it
kept listening on the old socket. Restart explicitly instead of relying
on --now.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:05:46 -06:00
ksmithandClaude Sonnet 5 21c33e8188 Make the reverse proxy a required, pluggable part of every deployment
gunicorn previously only ever bound to a Unix socket, which assumes the
reverse proxy is nginx running on this same host — it can't be reached by
a proxy on a different machine at all (e.g. Nginx Proxy Manager on its own
box), which is this project's actual deployment target. Switch gunicorn to
listening on a plain TCP address instead, controlled by a new
EVECTION_GUNICORN_BIND setting in .env.

install.sh now asks whether the proxy is local or remote and picks a
sensible bind default either way (127.0.0.1:8000 vs 0.0.0.0:8000, with a
reminder to firewall the port in the remote case), and no longer tries to
automate nginx/certbot itself — docs/install.md's new "Reverse proxy"
section covers both shapes instead, since which proxy and where it runs is
the operator's choice, not something this script should assume or set up.
upgrade.sh's health check now curls the TCP bind instead of the old socket
path.

Also add WhiteNoise so gunicorn serves /static/ itself: the old approach
(nginx aliasing the staticfiles directory on disk) silently breaks the
moment the proxy isn't on this host and can't see this filesystem.
nginx.conf.example's own /static/ alias becomes an optional same-host perf
optimization rather than a requirement.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:01:46 -06:00
44 changed files with 1705 additions and 175 deletions

No files matched your search

@@ -0,0 +1,22 @@
# Generated by Django 5.2.18 on 2026-10-31 18:00
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("accounts", "0001_initial"),
]
operations = [
migrations.CreateModel(
name="SetupLock",
fields=[
(
"id",
models.PositiveSmallIntegerField(default=1, primary_key=True, serialize=False),
),
],
),
]
+12
View File
@@ -5,3 +5,15 @@ from django.db import models
class User(AbstractUser): class User(AbstractUser):
timezone = models.CharField(max_length=64, default="UTC") timezone = models.CharField(max_length=64, default="UTC")
created_at = models.DateTimeField(auto_now_add=True) created_at = models.DateTimeField(auto_now_add=True)
class SetupLock(models.Model):
"""Singleton row that gates first-run setup against a race between
concurrent requests. The setup view always tries to INSERT the row
with pk=1 in the same transaction as creating the first admin user:
the database's primary-key uniqueness makes exactly one of those
INSERTs win even under concurrent requests, so only one admin account
can ever be created this way, instead of the User.objects.exists()
check alone (which has a window between the check and the save)."""
id = models.PositiveSmallIntegerField(primary_key=True, default=1)
+74
View File
@@ -0,0 +1,74 @@
import pytest
from django.contrib.auth import get_user_model
from django.core.cache import cache
from django.urls import reverse
from .views import (
LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_ADDRESS,
LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_USERNAME,
)
User = get_user_model()
@pytest.fixture(autouse=True)
def _clear_rate_limit_cache():
cache.clear()
yield
cache.clear()
@pytest.fixture
def alice(db):
return User.objects.create_user("alice", password="pw-alice-123")
def _bad_login(client, username="alice"):
return client.post(reverse("accounts:login"), {"username": username, "password": "wrong"})
@pytest.mark.django_db
def test_a_single_failed_login_still_shows_the_normal_error(client, alice):
response = _bad_login(client)
assert response.status_code == 200
assert b"Please enter a correct" in response.content or response.context["form"].errors
@pytest.mark.django_db
def test_correct_credentials_succeed_before_the_limit_is_hit(client, alice):
for _ in range(3):
_bad_login(client)
response = client.post(
reverse("accounts:login"), {"username": "alice", "password": "pw-alice-123"}
)
assert response.status_code == 302
@pytest.mark.django_db
def test_too_many_failed_attempts_from_one_address_blocks_further_tries(client, alice):
for _ in range(LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_ADDRESS):
_bad_login(client)
# Even the correct password is now refused — the throttle runs
# before the form is ever validated.
response = client.post(
reverse("accounts:login"), {"username": "alice", "password": "pw-alice-123"}
)
assert response.status_code == 302
assert response.url == reverse("accounts:login")
assert not response.wsgi_request.user.is_authenticated
@pytest.mark.django_db
def test_too_many_failed_attempts_against_one_username_blocks_that_username(client, alice):
bob = User.objects.create_user("bob", password="pw-bob-123")
for _ in range(LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_USERNAME):
_bad_login(client, username="alice")
# Same client address, different username — the per-username
# throttle for "alice" shouldn't affect bob logging in.
response = client.post(reverse("accounts:login"), {"username": "bob", "password": "pw-bob-123"})
assert response.status_code == 302
assert response.wsgi_request.user == bob
+23 -1
View File
@@ -1,7 +1,7 @@
import pytest import pytest
from django.urls import reverse from django.urls import reverse
from .models import User from .models import SetupLock, User
# SetupWizardMiddleware's cache is reset around every test by the autouse # SetupWizardMiddleware's cache is reset around every test by the autouse
# fixture in the top-level conftest.py, not just here. # fixture in the top-level conftest.py, not just here.
@@ -43,6 +43,28 @@ def test_setup_creates_admin_and_logs_in(client):
assert stream_response.status_code == 200 assert stream_response.status_code == 200
@pytest.mark.django_db
def test_setup_refuses_a_second_concurrent_submission(client):
# Simulates two requests racing the setup form: the first claims the
# SetupLock row directly (standing in for a request that's already
# past the User.objects.exists() check), so a second submission's
# INSERT must lose instead of also creating an admin account.
SetupLock.objects.create()
response = client.post(
reverse("accounts:setup"),
{
"username": "raced",
"email": "raced@example.com",
"password": "correct-horse-battery",
"password_confirm": "correct-horse-battery",
},
)
assert response.status_code == 302
assert response.url == reverse("accounts:login")
assert not User.objects.filter(username="raced").exists()
@pytest.mark.django_db @pytest.mark.django_db
def test_setup_is_unreachable_once_a_user_exists(client): def test_setup_is_unreachable_once_a_user_exists(client):
User.objects.create_user("existing", password="irrelevant") User.objects.create_user("existing", password="irrelevant")
+1 -5
View File
@@ -7,11 +7,7 @@ app_name = "accounts"
urlpatterns = [ urlpatterns = [
path("setup/", views.setup, name="setup"), path("setup/", views.setup, name="setup"),
path( path("login/", views.LoginView.as_view(), name="login"),
"login/",
auth_views.LoginView.as_view(template_name="accounts/login.html"),
name="login",
),
path("logout/", auth_views.LogoutView.as_view(), name="logout"), path("logout/", auth_views.LogoutView.as_view(), name="logout"),
path("users/", views.manage_users, name="manage_users"), path("users/", views.manage_users, name="manage_users"),
path("users/create/", views.create_user, name="create_user"), path("users/create/", views.create_user, name="create_user"),
+76 -1
View File
@@ -1,14 +1,79 @@
from django.contrib import messages from django.contrib import messages
from django.contrib.auth import login from django.contrib.auth import login
from django.contrib.auth.decorators import user_passes_test from django.contrib.auth.decorators import user_passes_test
from django.contrib.auth.views import LoginView as DjangoLoginView
from django.core.cache import cache
from django.db import IntegrityError, transaction
from django.shortcuts import redirect, render from django.shortcuts import redirect, render
from .forms import CreateUserForm, SetupForm from .forms import CreateUserForm, SetupForm
from .middleware import SetupWizardMiddleware from .middleware import SetupWizardMiddleware
from .models import User from .models import SetupLock, User
is_admin = user_passes_test(lambda u: u.is_authenticated and u.is_staff) is_admin = user_passes_test(lambda u: u.is_authenticated and u.is_staff)
# Two independent throttles, either of which can block an attempt: by
# client address (catches one attacker spraying many usernames) and by
# the submitted username (catches a distributed/low-and-slow attack on
# one specific account from many addresses). Both are generous enough
# not to lock out a user who's just fat-fingered their own password a
# few times.
LOGIN_RATE_LIMIT_WINDOW_SECONDS = 5 * 60
LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_ADDRESS = 15
LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_USERNAME = 8
def _client_address(request) -> str:
# gunicorn is always run behind a reverse proxy in every deployment
# (see docs/install.md) and its bind address is never meant to be
# reachable from outside that proxy — the same trust boundary
# config.settings.prod already relies on for X-Forwarded-Proto. A
# direct request (e.g. local dev) has no X-Forwarded-For and falls
# back to REMOTE_ADDR.
forwarded_for = request.META.get("HTTP_X_FORWARDED_FOR", "")
if forwarded_for:
return forwarded_for.split(",")[0].strip()
return request.META.get("REMOTE_ADDR", "unknown")
def _attempts_key(kind: str, value: str) -> str:
return f"login-attempts:{kind}:{value}"
def _too_many_attempts(key: str, limit: int) -> bool:
return cache.get(key, 0) >= limit
def _record_failed_attempt(key: str) -> None:
try:
cache.incr(key)
except ValueError:
cache.set(key, 1, LOGIN_RATE_LIMIT_WINDOW_SECONDS)
class LoginView(DjangoLoginView):
"""Wraps Django's own LoginView with the throttle above — everything
else (form, template, redirect behavior) is unchanged."""
template_name = "accounts/login.html"
def post(self, request, *args, **kwargs):
address_key = _attempts_key("addr", _client_address(request))
username_key = _attempts_key("user", request.POST.get("username", "")[:150])
if _too_many_attempts(
address_key, LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_ADDRESS
) or _too_many_attempts(username_key, LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_USERNAME):
messages.error(
request, "Too many login attempts. Please wait a few minutes and try again."
)
return redirect("accounts:login")
return super().post(request, *args, **kwargs)
def form_invalid(self, form):
_record_failed_attempt(_attempts_key("addr", _client_address(self.request)))
_record_failed_attempt(_attempts_key("user", self.request.POST.get("username", "")[:150]))
return super().form_invalid(form)
def setup(request): def setup(request):
if User.objects.exists(): if User.objects.exists():
@@ -16,11 +81,21 @@ def setup(request):
form = SetupForm(request.POST or None) form = SetupForm(request.POST or None)
if request.method == "POST" and form.is_valid(): if request.method == "POST" and form.is_valid():
try:
with transaction.atomic():
# Claims the one-time setup slot: the first of any
# concurrent requests to reach this INSERT wins it, and
# every other one raises IntegrityError instead of also
# creating an admin account. See SetupLock's docstring.
SetupLock.objects.create()
user = form.save(commit=False) user = form.save(commit=False)
user.set_password(form.cleaned_data["password"]) user.set_password(form.cleaned_data["password"])
user.is_staff = True user.is_staff = True
user.is_superuser = True user.is_superuser = True
user.save() user.save()
except IntegrityError:
messages.info(request, "Setup was already completed. Log in with that account.")
return redirect("accounts:login")
SetupWizardMiddleware._a_user_exists = True SetupWizardMiddleware._a_user_exists = True
login(request, user) login(request, user)
messages.success(request, "Welcome to Evection! Your admin account is ready.") messages.success(request, "Welcome to Evection! Your admin account is ready.")
+12 -12
View File
@@ -1,5 +1,4 @@
from celery import shared_task from celery import shared_task
from django.db.models import Q
from apps.feeds.models import Subscription from apps.feeds.models import Subscription
from apps.rules.engine import evaluate_rules_for_article from apps.rules.engine import evaluate_rules_for_article
@@ -14,6 +13,10 @@ def apply_filter_rules_for_feed(feed_id: int, article_ids: list[int]):
FilterRules (scoped to this feed/tag or unscoped) against the given FilterRules (scoped to this feed/tag or unscoped) against the given
articles. Dispatched by the fetch pipeline after new/updated articles articles. Dispatched by the fetch pipeline after new/updated articles
are committed for a feed. are committed for a feed.
Per-rule scoping is left to evaluate_rules_for_article/rule_in_scope
rather than filtered into this query, so this and the backlog
re-apply task (apps.rules.tasks) can't scope rules differently.
""" """
if not article_ids: if not article_ids:
return 0 return 0
@@ -26,24 +29,21 @@ def apply_filter_rules_for_feed(feed_id: int, article_ids: list[int]):
evaluated = 0 evaluated = 0
for subscription in subscriptions: for subscription in subscriptions:
rules = FilterRule.objects.filter(user=subscription.user, is_enabled=True).filter( rules = list(
Q(feed_id=feed_id) | Q(feed__isnull=True) FilterRule.objects.filter(user=subscription.user, is_enabled=True).order_by(
"priority", "id"
) )
subscription_tag_ids = list( )
if not rules:
continue
subscription_tag_ids = set(
SubscriptionTag.objects.filter( SubscriptionTag.objects.filter(
user=subscription.user, subscription=subscription user=subscription.user, subscription=subscription
).values_list("tag_id", flat=True) ).values_list("tag_id", flat=True)
) )
if subscription_tag_ids:
rules = rules.filter(Q(tag_id__in=subscription_tag_ids) | Q(tag__isnull=True))
else:
rules = rules.filter(tag__isnull=True)
rules = list(rules.order_by("priority", "id"))
if not rules:
continue
for article in articles: for article in articles:
evaluate_rules_for_article(subscription.user, article, rules) evaluate_rules_for_article(subscription.user, article, rules, subscription_tag_ids)
evaluated += 1 evaluated += 1
return evaluated return evaluated
+28 -1
View File
@@ -1 +1,28 @@
# Register your models here. from django.contrib import admin
from django.shortcuts import redirect
from django.urls import reverse
from .models import SiteSettings
@admin.register(SiteSettings)
class SiteSettingsAdmin(admin.ModelAdmin):
"""A true singleton, not a collection — browsing a changelist to pick
the one row and click into it is an extra, pointless step, so
"Core > Site settings" in the sidebar goes straight to the (only)
change form instead, creating the row first if this is a fresh
install. No changelist/add/delete views exist at all here.
"""
fields = ["fetch_interval_minutes"]
def has_add_permission(self, request):
return False
def has_delete_permission(self, request, obj=None):
return False
def changelist_view(self, request, extra_context=None):
settings_row = SiteSettings.load()
url = reverse("admin:core_sitesettings_change", args=[settings_row.pk])
return redirect(url)
+13 -1
View File
@@ -44,7 +44,19 @@ def sidebar(request):
return { return {
"sidebar_tag_groups": [ "sidebar_tag_groups": [
{"tag": tag, "subscriptions": feeds_by_tag.get(tag.pk, [])} for tag in tags {
"tag": tag,
"subscriptions": feeds_by_tag.get(tag.pk, []),
# A feed can carry more than one tag, so this is a sum
# over whichever of the user's feeds this tag happens to
# cover right now — not a separately-tracked counter — and
# reuses the per-feed counts already computed above rather
# than a second query.
"unread_count": sum(
subscription.unread_count for subscription in feeds_by_tag.get(tag.pk, [])
),
}
for tag in tags
], ],
"sidebar_untagged": untagged, "sidebar_untagged": untagged,
"sidebar_starred_count": ArticleTag.objects.filter( "sidebar_starred_count": ArticleTag.objects.filter(
+33
View File
@@ -0,0 +1,33 @@
# Generated by Django 5.2.18 on 2026-10-31 18:56
from django.db import migrations, models
class Migration(migrations.Migration):
initial = True
dependencies = []
operations = [
migrations.CreateModel(
name="SiteSettings",
fields=[
(
"id",
models.PositiveSmallIntegerField(default=1, primary_key=True, serialize=False),
),
(
"fetch_interval_minutes",
models.PositiveIntegerField(
default=60,
help_text="How often, in minutes, every feed is polled for new articles. Can't be set lower than EVECTION_MIN_FETCH_INTERVAL_MINUTES (see .env).",
),
),
],
options={
"verbose_name": "Site settings",
"verbose_name_plural": "Site settings",
},
),
]
+35 -1
View File
@@ -1 +1,35 @@
# Create your models here. from django.db import models
class SiteSettings(models.Model):
"""Singleton row (pk=1, same pattern as accounts.SetupLock) holding
instance-wide configuration an operator reasonably wants to change
without touching the server or .env — editable via Django admin.
Currently just the feed-poll interval, which used to be a per-feed
field; it's site-wide now (see apps.feeds.tasks.poll_due_feeds).
"""
id = models.PositiveSmallIntegerField(primary_key=True, default=1)
fetch_interval_minutes = models.PositiveIntegerField(
default=60,
help_text=(
"How often, in minutes, every feed is polled for new articles. "
"Can't be set lower than EVECTION_MIN_FETCH_INTERVAL_MINUTES (see .env)."
),
)
class Meta:
verbose_name = "Site settings"
verbose_name_plural = "Site settings"
def __str__(self):
return "Site settings"
@classmethod
def load(cls):
"""Returns the singleton row, creating it with defaults on first
use so callers never need to special-case "no row exists yet"
(e.g. a fresh install whose first poll_due_feeds run hasn't
created it, and whose admin hasn't visited the settings page)."""
obj, _ = cls.objects.get_or_create(pk=1)
return obj
+82
View File
@@ -0,0 +1,82 @@
import pytest
from django.contrib.auth import get_user_model
from django.test import RequestFactory
from django.urls import reverse
from apps.articles.models import Article, SubscriptionTag, Tag
from apps.feeds.models import Feed, Subscription
from .context_processors import sidebar
User = get_user_model()
@pytest.fixture
def alice(db):
return User.objects.create_user("alice", password="pw-alice-123")
def _sidebar_context(user):
request = RequestFactory().get("/")
request.user = user
return sidebar(request)
@pytest.mark.django_db
def test_tag_unread_count_sums_its_feeds(alice):
feed_a = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
feed_b = Feed.objects.create(feed_url="https://b.example.com/feed.xml")
sub_a = Subscription.objects.create(user=alice, feed=feed_a)
sub_b = Subscription.objects.create(user=alice, feed=feed_b)
tag = Tag.objects.create(user=alice, name="work")
SubscriptionTag.objects.create(user=alice, subscription=sub_a, tag=tag)
SubscriptionTag.objects.create(user=alice, subscription=sub_b, tag=tag)
for i in range(3):
Article.objects.create(feed=feed_a, guid=f"a-{i}", title=f"A{i}")
for i in range(4):
Article.objects.create(feed=feed_b, guid=f"b-{i}", title=f"B{i}")
context = _sidebar_context(alice)
[group] = context["sidebar_tag_groups"]
assert group["tag"] == tag
assert group["unread_count"] == 7
@pytest.mark.django_db
def test_tag_unread_count_is_zero_with_no_unread_articles(alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
sub = Subscription.objects.create(user=alice, feed=feed)
tag = Tag.objects.create(user=alice, name="empty")
SubscriptionTag.objects.create(user=alice, subscription=sub, tag=tag)
context = _sidebar_context(alice)
[group] = context["sidebar_tag_groups"]
assert group["unread_count"] == 0
@pytest.mark.django_db
def test_sidebar_sections_carry_a_stable_collapse_state_key(client, alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
sub = Subscription.objects.create(user=alice, feed=feed)
tag = Tag.objects.create(user=alice, name="work")
SubscriptionTag.objects.create(user=alice, subscription=sub, tag=tag)
other_feed = Feed.objects.create(feed_url="https://other.example.com/feed.xml")
Subscription.objects.create(user=alice, feed=other_feed) # untagged
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
content = response.content.decode()
# static/js/sidebar-state.js reads these to restore/persist each
# section's collapsed state across page loads.
assert f'data-sidebar-key="tag-{tag.pk}"' in content
assert 'data-sidebar-key="untagged"' in content
@pytest.mark.django_db
def test_base_template_loads_the_sidebar_state_script(client, alice):
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
assert "sidebar-state.js" in response.content.decode()
+84
View File
@@ -0,0 +1,84 @@
from datetime import timedelta
from unittest.mock import patch
import pytest
from django.contrib.auth import get_user_model
from django.urls import reverse
from django.utils import timezone
from apps.feeds.models import Feed
from apps.feeds.tasks import poll_due_feeds
from .models import SiteSettings
User = get_user_model()
@pytest.mark.django_db
def test_load_creates_the_singleton_row_with_defaults():
assert not SiteSettings.objects.exists()
settings_row = SiteSettings.load()
assert settings_row.fetch_interval_minutes == 60
assert SiteSettings.objects.count() == 1
@pytest.mark.django_db
def test_load_returns_the_existing_row_without_creating_a_second_one():
SiteSettings.objects.create(fetch_interval_minutes=30)
settings_row = SiteSettings.load()
assert settings_row.fetch_interval_minutes == 30
assert SiteSettings.objects.count() == 1
@pytest.mark.django_db
def test_poll_due_feeds_uses_the_site_wide_interval_for_every_feed():
SiteSettings.objects.create(fetch_interval_minutes=120)
due = Feed.objects.create(
feed_url="https://a.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(minutes=150),
)
not_due = Feed.objects.create(
feed_url="https://b.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(minutes=90),
)
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
poll_due_feeds()
dispatched_ids = {call.args[0] for call in mock_delay.call_args_list}
assert dispatched_ids == {due.id}
assert not_due.id not in dispatched_ids
@pytest.mark.django_db
def test_admin_changelist_redirects_straight_to_the_change_form(client):
assert not SiteSettings.objects.exists()
User.objects.create_superuser("admin", password="pw-admin-123")
client.login(username="admin", password="pw-admin-123")
changelist_url = reverse("admin:core_sitesettings_changelist")
response = client.get(changelist_url)
settings_row = SiteSettings.objects.get() # created by the redirect itself
assert response.status_code == 302
assert response.url == reverse("admin:core_sitesettings_change", args=[settings_row.pk])
@pytest.mark.django_db
def test_admin_never_offers_add_or_delete(client):
# ModelAdmin.has_add_permission/has_delete_permission returning False
# makes Django raise PermissionDenied rather than build a 403 response
# directly — the test client's default behavior re-raises that for
# debugging convenience instead of returning it as a response.
client.raise_request_exception = False
User.objects.create_superuser("admin", password="pw-admin-123")
client.login(username="admin", password="pw-admin-123")
response = client.get(reverse("admin:core_sitesettings_add"))
assert response.status_code == 403
settings_row = SiteSettings.load()
delete_url = reverse("admin:core_sitesettings_delete", args=[settings_row.pk])
response = client.get(delete_url)
assert response.status_code == 403
+21
View File
@@ -0,0 +1,21 @@
import socket
import pytest
@pytest.fixture(autouse=True)
def _fake_public_dns(monkeypatch):
"""apps.feeds.services.ssrf_guard resolves a feed's real hostname via
socket.getaddrinfo before every fetch. Tests use made-up hostnames
like example.com/a.example.com purely as fixture data — they must
not depend on real DNS/network access to pass that check, and must
resolve deterministically regardless of this sandbox's network
access. Every lookup here resolves to a fixed, definitely-public IP;
tests that specifically exercise the "resolves to a non-public
address" rejection path patch socket.getaddrinfo themselves instead.
"""
def _fake_getaddrinfo(host, *args, **kwargs):
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("93.184.216.34", 0))]
monkeypatch.setattr(socket, "getaddrinfo", _fake_getaddrinfo)
@@ -0,0 +1,18 @@
# Generated by Django 5.2.18 on 2026-10-31 18:00
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("feeds", "0002_remove_folder"),
]
operations = [
migrations.AddField(
model_name="feed",
name="claimed_at",
field=models.DateTimeField(blank=True, null=True),
),
]
@@ -0,0 +1,17 @@
# Generated by Django 5.2.18 on 2026-10-31 18:56
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
("feeds", "0003_feed_claimed_at"),
]
operations = [
migrations.RemoveField(
model_name="feed",
name="fetch_interval_minutes",
),
]
+13 -2
View File
@@ -35,11 +35,22 @@ class Feed(models.Model):
) )
last_error_message = models.TextField(blank=True) last_error_message = models.TextField(blank=True)
# Instance-wide, not a per-user preference — see class docstring. # The poll interval itself is site-wide now (apps.core.models.
fetch_interval_minutes = models.PositiveIntegerField(default=60) # SiteSettings), not per-feed — only the error-driven backoff on top
# of it still varies per feed (see apps.feeds.services.fetcher.
# effective_interval_minutes).
fetch_error_count = models.PositiveIntegerField(default=0) fetch_error_count = models.PositiveIntegerField(default=0)
is_active = models.BooleanField(default=True) is_active = models.BooleanField(default=True)
# Set by poll_due_feeds' atomic claim (an UPDATE ... WHERE clause, not
# select_for_update — Celery Beat and fetch_feed run in separate
# transactions) and cleared once fetch_feed finishes, so overlapping
# beat ticks or a fetch that runs long can't both dispatch the same
# feed. A claim older than CLAIM_STALE_MINUTES (tasks.py) is treated as
# abandoned and can be reclaimed, so a worker dying mid-fetch doesn't
# wedge the feed forever.
claimed_at = models.DateTimeField(null=True, blank=True)
created_at = models.DateTimeField(auto_now_add=True) created_at = models.DateTimeField(auto_now_add=True)
def __str__(self): def __str__(self):
+42 -13
View File
@@ -23,12 +23,23 @@ from django.utils import timezone
from apps.articles.models import Article from apps.articles.models import Article
from apps.feeds.models import Feed from apps.feeds.models import Feed
from .ssrf_guard import SSRFBlocked, safe_get
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
FETCH_TIMEOUT_SECONDS = 15 FETCH_TIMEOUT_SECONDS = 15
MAX_BACKOFF_MINUTES = 24 * 60 MAX_BACKOFF_MINUTES = 24 * 60
USER_AGENT = "Evection/0.1 (+https://github.com/; self-hosted RSS reader)" USER_AGENT = "Evection/0.1 (+https://github.com/; self-hosted RSS reader)"
# ssrf_guard.MAX_RESPONSE_BYTES already bounds the raw download; these
# bound what a single poll does *with* that response — a feed within the
# byte cap can still carry an unreasonable number of entries, or one
# entry with an unreasonably large body, and both cost real CPU/DB time
# (nh3 sanitization, an Article row/UPDATE) per entry processed.
MAX_ENTRIES_PER_POLL = 200
MAX_CONTENT_HTML_INPUT_CHARS = 200_000 # bounds sanitizer cost, not just storage
MAX_CONTENT_HTML_CHARS = 50_000 # bounds what's actually stored/rendered
ALLOWED_TAGS = { ALLOWED_TAGS = {
"a", "a",
"p", "p",
@@ -81,7 +92,9 @@ class FetchResult:
def sanitize_html(raw_html: str) -> str: def sanitize_html(raw_html: str) -> str:
if not raw_html: if not raw_html:
return "" return ""
return nh3.clean(raw_html, tags=ALLOWED_TAGS, attributes=ALLOWED_ATTRIBUTES, link_rel=None) bounded = raw_html[:MAX_CONTENT_HTML_INPUT_CHARS]
cleaned = nh3.clean(bounded, tags=ALLOWED_TAGS, attributes=ALLOWED_ATTRIBUTES, link_rel=None)
return cleaned[:MAX_CONTENT_HTML_CHARS]
def _content_hash(title: str, content_html: str) -> str: def _content_hash(title: str, content_html: str) -> str:
@@ -111,14 +124,17 @@ def _entry_content_html(entry) -> str:
return getattr(entry, "summary", "") or "" return getattr(entry, "summary", "") or ""
def effective_interval_minutes(feed: Feed) -> int: def effective_interval_minutes(feed: Feed, configured_interval_minutes: int) -> int:
"""The actual polling interval to use, applying exponential backoff on """The actual polling interval to use for `feed`, applying exponential
top of the admin-configured ``fetch_interval_minutes`` for feeds that backoff on top of the site-wide `configured_interval_minutes` (see
are currently erroring. ``fetch_interval_minutes`` itself is never apps.core.models.SiteSettings) for a feed that's currently erroring.
mutated by the fetcher — it's the admin's chosen baseline. Takes the configured value as a parameter rather than querying
SiteSettings itself, since this runs once per feed (apps.feeds.tasks
calls it from a list comprehension over every active feed) and the
site-wide value doesn't vary by feed.
""" """
floor = getattr(settings, "EVECTION_MIN_FETCH_INTERVAL_MINUTES", 15) floor = getattr(settings, "EVECTION_MIN_FETCH_INTERVAL_MINUTES", 15)
base = max(feed.fetch_interval_minutes, floor) base = max(configured_interval_minutes, floor)
if feed.fetch_error_count <= 0: if feed.fetch_error_count <= 0:
return base return base
backoff = base * (2**feed.fetch_error_count) backoff = base * (2**feed.fetch_error_count)
@@ -139,10 +155,13 @@ def fetch_and_ingest(feed: Feed) -> FetchResult:
headers["If-Modified-Since"] = feed.last_modified headers["If-Modified-Since"] = feed.last_modified
try: try:
response = requests.get(feed.feed_url, headers=headers, timeout=FETCH_TIMEOUT_SECONDS) response = safe_get(feed.feed_url, headers=headers, timeout=FETCH_TIMEOUT_SECONDS)
except SSRFBlocked as exc:
feed.last_fetched_at = timezone.now()
return record_fetch_error(feed, f"Refused to fetch: {exc}")
except requests.RequestException as exc: except requests.RequestException as exc:
feed.last_fetched_at = timezone.now() feed.last_fetched_at = timezone.now()
return _record_error(feed, f"Request failed: {exc}") return record_fetch_error(feed, f"Request failed: {exc}")
feed.last_fetched_at = timezone.now() feed.last_fetched_at = timezone.now()
@@ -154,12 +173,12 @@ def fetch_and_ingest(feed: Feed) -> FetchResult:
) )
if response.status_code >= 400: if response.status_code >= 400:
return _record_error(feed, f"HTTP {response.status_code}") return record_fetch_error(feed, f"HTTP {response.status_code}")
parsed = feedparser.parse(response.content) parsed = feedparser.parse(response.content)
if parsed.bozo and not parsed.entries: if parsed.bozo and not parsed.entries:
return _record_error(feed, f"Unparseable feed: {parsed.get('bozo_exception')}") return record_fetch_error(feed, f"Unparseable feed: {parsed.get('bozo_exception')}")
if parsed.bozo: if parsed.bozo:
logger.warning( logger.warning(
@@ -181,7 +200,17 @@ def fetch_and_ingest(feed: Feed) -> FetchResult:
new_ids: list[int] = [] new_ids: list[int] = []
updated_ids: list[int] = [] updated_ids: list[int] = []
for entry in parsed.entries: entries = parsed.entries
if len(entries) > MAX_ENTRIES_PER_POLL:
logger.warning(
"Feed %s: %d entries in one poll, only processing the first %d",
feed.feed_url,
len(entries),
MAX_ENTRIES_PER_POLL,
)
entries = entries[:MAX_ENTRIES_PER_POLL]
for entry in entries:
guid = _entry_guid(entry) guid = _entry_guid(entry)
title = (getattr(entry, "title", "") or "")[:1000] title = (getattr(entry, "title", "") or "")[:1000]
content_html = sanitize_html(_entry_content_html(entry)) content_html = sanitize_html(_entry_content_html(entry))
@@ -215,7 +244,7 @@ def fetch_and_ingest(feed: Feed) -> FetchResult:
) )
def _record_error(feed: Feed, message: str) -> FetchResult: def record_fetch_error(feed: Feed, message: str) -> FetchResult:
feed.last_fetch_status = Feed.STATUS_ERROR feed.last_fetch_status = Feed.STATUS_ERROR
feed.last_error_message = message[:2000] feed.last_error_message = message[:2000]
feed.fetch_error_count += 1 feed.fetch_error_count += 1
+121
View File
@@ -0,0 +1,121 @@
"""Rejects feed-fetch targets that could turn this server/worker's own
network access into an SSRF vector.
Feed URLs are entirely attacker-controlled (any authenticated user via
"add feed", or an uploaded OPML file — see apps.feeds.opml) and this app
fetches them from the server's own network position. fetch_and_ingest
(apps.feeds.services.fetcher) is the single chokepoint every feed URL
passes through before any HTTP request is made, regardless of which
path created the Feed row, so enforcing this here covers all of them.
This validates the resolved address before connecting and again before
following each redirect hop, denying by default anything that isn't a
globally routable address (ip.is_global) — covering loopback, RFC1918
private ranges, link-local (including the 169.254.169.254 cloud
metadata address), and so on, for both IPv4 and IPv6.
What this does *not* close: the TCP connection isn't pinned to the
specific IP that was validated, so a DNS answer that changes between
the validation lookup and the actual connect (classic DNS rebinding)
could still slip through. Closing that fully needs a custom transport
that connects to the exact validated IP while still verifying TLS
against the original hostname — more machinery than this self-hosted
app's threat model currently justifies; this stops the much more
common case of a feed URL (or a redirect) simply pointing straight at
an internal address.
"""
from __future__ import annotations
import ipaddress
import socket
from dataclasses import dataclass
from urllib.parse import urljoin, urlparse
import requests
from requests.structures import CaseInsensitiveDict
ALLOWED_SCHEMES = {"http", "https"}
MAX_REDIRECTS = 5
MAX_RESPONSE_BYTES = 10 * 1024 * 1024 # 10 MB — generous for an RSS/Atom feed
READ_CHUNK_BYTES = 65536
class SSRFBlocked(Exception):
"""Raised when a fetch target — or one of its redirects — isn't safe
to connect to on a user's behalf."""
@dataclass
class BoundedResponse:
status_code: int
# requests' own CaseInsensitiveDict — server header casing varies
# (ETag/Etag/etag) and fetcher.py looks these up by a fixed-case key.
headers: CaseInsensitiveDict
content: bytes
def validate_fetch_url(url: str) -> None:
"""Raises SSRFBlocked unless `url` has an http(s) scheme, a hostname,
and every address that hostname resolves to is globally routable."""
parsed = urlparse(url)
if parsed.scheme not in ALLOWED_SCHEMES:
raise SSRFBlocked(f"Unsupported URL scheme: {parsed.scheme!r}")
if not parsed.hostname:
raise SSRFBlocked("URL has no hostname")
try:
addrinfo = socket.getaddrinfo(parsed.hostname, None)
except OSError as exc:
raise SSRFBlocked(f"Could not resolve {parsed.hostname!r}: {exc}") from exc
for *_, sockaddr in addrinfo:
ip = ipaddress.ip_address(sockaddr[0])
if not ip.is_global:
raise SSRFBlocked(f"{parsed.hostname!r} resolves to a non-public address: {ip}")
def safe_get(url: str, *, headers: dict, timeout: float) -> BoundedResponse:
"""GETs `url` the way fetch_and_ingest needs it: every redirect hop is
validated and followed manually (never left to requests/urllib3, so
a redirect can't bypass validate_fetch_url), capped at MAX_REDIRECTS,
and the body is read in bounded chunks rather than trusting
Content-Length or buffering an unbounded response.
"""
current_url = url
with requests.Session() as session:
for _ in range(MAX_REDIRECTS + 1):
validate_fetch_url(current_url)
response = session.get(
current_url,
headers=headers,
timeout=timeout,
stream=True,
allow_redirects=False,
)
try:
if response.is_redirect or response.is_permanent_redirect:
location = response.headers.get("Location")
if not location:
raise SSRFBlocked("Redirect response had no Location header")
current_url = urljoin(current_url, location)
continue
return BoundedResponse(
status_code=response.status_code,
headers=response.headers,
content=_read_bounded(response),
)
finally:
response.close()
raise SSRFBlocked(f"Too many redirects (> {MAX_REDIRECTS})")
def _read_bounded(response: requests.Response) -> bytes:
chunks = []
total = 0
for chunk in response.iter_content(chunk_size=READ_CHUNK_BYTES):
total += len(chunk)
if total > MAX_RESPONSE_BYTES:
raise SSRFBlocked(f"Response exceeded {MAX_RESPONSE_BYTES} bytes")
chunks.append(chunk)
return b"".join(chunks)
+68 -10
View File
@@ -4,50 +4,108 @@ from io import BytesIO
from celery import shared_task from celery import shared_task
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.db.models import Q
from django.utils import timezone from django.utils import timezone
from apps.articles.tasks import apply_filter_rules_for_feed from apps.articles.tasks import apply_filter_rules_for_feed
from apps.core.models import SiteSettings
from .models import Feed from .models import Feed
from .opml import import_opml from .opml import import_opml
from .services.fetcher import effective_interval_minutes, fetch_and_ingest from .services.fetcher import effective_interval_minutes, fetch_and_ingest, record_fetch_error
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# How long a claimed-but-unfinished fetch blocks the same feed from being
# claimed again — must comfortably exceed FETCH_TIMEOUT_SECONDS plus parse/
# ingest time, so a slow-but-healthy fetch is never raced by the next tick,
# while a worker that died mid-fetch doesn't wedge the feed forever.
CLAIM_STALE_MINUTES = 10
@shared_task @shared_task
def poll_due_feeds(): def poll_due_feeds():
"""Find every active Feed whose polling interval has elapsed and """Find every active Feed whose polling interval has elapsed, atomically
dispatch a fetch_feed task for each — fan-out happens here, not via one claim it (so overlapping beat ticks or a slow-running previous fetch
Celery Beat entry per feed. can't dispatch the same feed twice), and dispatch a fetch_feed task for
each claimed feed — fan-out happens here, not via one Celery Beat entry
per feed.
""" """
now = timezone.now()
claim_stale_before = now - timedelta(minutes=CLAIM_STALE_MINUTES)
# Loaded once per call, not per feed — the interval is site-wide
# (apps.core.models.SiteSettings), so every feed below shares it.
configured_interval = SiteSettings.load().fetch_interval_minutes
due_feed_ids = [ due_feed_ids = [
feed.id feed.id
for feed in Feed.objects.filter(is_active=True).only( for feed in Feed.objects.filter(is_active=True).only(
"id", "last_fetched_at", "fetch_interval_minutes", "fetch_error_count" "id", "last_fetched_at", "fetch_error_count"
) )
if _is_due(feed) if _is_due(feed, configured_interval)
] ]
claimed_ids = []
for feed_id in due_feed_ids: for feed_id in due_feed_ids:
# The UPDATE's WHERE clause re-checks staleness at the database
# level: it only claims rows nobody else has already claimed more
# recently than CLAIM_STALE_MINUTES ago, so two workers racing this
# same loop can't both get rowcount=1 for the same feed.
claimed = (
Feed.objects.filter(id=feed_id, is_active=True)
.filter(Q(claimed_at__isnull=True) | Q(claimed_at__lt=claim_stale_before))
.update(claimed_at=now)
)
if claimed:
claimed_ids.append(feed_id)
for feed_id in claimed_ids:
fetch_feed.delay(feed_id) fetch_feed.delay(feed_id)
return len(due_feed_ids) return len(claimed_ids)
def _is_due(feed: Feed) -> bool: def _is_due(feed: Feed, configured_interval_minutes: int) -> bool:
if feed.last_fetched_at is None: if feed.last_fetched_at is None:
return True return True
interval = effective_interval_minutes(feed) interval = effective_interval_minutes(feed, configured_interval_minutes)
return feed.last_fetched_at <= timezone.now() - timedelta(minutes=interval) return feed.last_fetched_at <= timezone.now() - timedelta(minutes=interval)
@shared_task # Bounds a single feed's fetch+parse+ingest, independent of
# FETCH_TIMEOUT_SECONDS (apps.feeds.services.fetcher) which only covers
# the HTTP request itself: parsing/sanitizing/writing a large-but-under-
# the-byte-cap feed could still run long. soft_time_limit lets the
# except-Exception handler below record a normal feed error (and clear
# the claim) before the hard time_limit kills the worker process outright.
FETCH_FEED_SOFT_TIME_LIMIT = 60
FETCH_FEED_TIME_LIMIT = 90
@shared_task(time_limit=FETCH_FEED_TIME_LIMIT, soft_time_limit=FETCH_FEED_SOFT_TIME_LIMIT)
def fetch_feed(feed_id: int): def fetch_feed(feed_id: int):
try: try:
feed = Feed.objects.get(pk=feed_id, is_active=True) feed = Feed.objects.get(pk=feed_id, is_active=True)
except Feed.DoesNotExist: except Feed.DoesNotExist:
return None return None
try:
result = fetch_and_ingest(feed) result = fetch_and_ingest(feed)
except Exception:
# fetch_and_ingest already handles the expected failure modes
# (network errors, SSRF-blocked targets, unparseable feeds) by
# returning an error FetchResult rather than raising — anything
# that reaches here is unexpected (a parser/sanitizer bug, a DB
# error, this task's own soft time limit), so it's recorded the
# same way a handled failure would be rather than silently
# leaving the feed's status stale, the claim held for the full
# CLAIM_STALE_MINUTES, and no record of what happened. Re-raised
# afterward so Celery still marks the task FAILED (visible in
# worker logs/monitoring) instead of swallowing it.
logger.exception("Feed %s: unexpected error during fetch", feed.feed_url)
record_fetch_error(feed, "Unexpected error — see worker logs")
feed.last_fetched_at = timezone.now()
feed.claimed_at = None
feed.save()
raise
feed.claimed_at = None
feed.save() feed.save()
changed_article_ids = result.new_article_ids + result.updated_article_ids changed_article_ids = result.new_article_ids + result.updated_article_ids
+34 -2
View File
@@ -5,7 +5,13 @@ from django.utils import timezone
from apps.articles.models import Article, UserArticleState from apps.articles.models import Article, UserArticleState
from apps.feeds.models import Feed from apps.feeds.models import Feed
from apps.feeds.services.fetcher import effective_interval_minutes, fetch_and_ingest from apps.feeds.services.fetcher import (
MAX_CONTENT_HTML_CHARS,
MAX_ENTRIES_PER_POLL,
effective_interval_minutes,
fetch_and_ingest,
sanitize_html,
)
FEED_URL = "https://example.com/feed.xml" FEED_URL = "https://example.com/feed.xml"
@@ -130,7 +136,7 @@ def test_unparseable_feed_records_error_and_backs_off(feed):
assert result.status == Feed.STATUS_ERROR assert result.status == Feed.STATUS_ERROR
assert feed.fetch_error_count == 1 assert feed.fetch_error_count == 1
assert effective_interval_minutes(feed) > feed.fetch_interval_minutes assert effective_interval_minutes(feed, 60) > 60
@pytest.mark.django_db @pytest.mark.django_db
@@ -196,3 +202,29 @@ def test_refetching_updated_content_does_not_reset_user_state():
state.refresh_from_db() state.refresh_from_db()
assert state.is_read is True assert state.is_read is True
assert ArticleTag.objects.filter(user=user, article=article, tag=starred_tag).exists() assert ArticleTag.objects.filter(user=user, article=article, tag=starred_tag).exists()
def test_sanitize_html_caps_stored_output_length():
huge = "<p>" + ("a" * (MAX_CONTENT_HTML_CHARS + 10_000)) + "</p>"
cleaned = sanitize_html(huge)
assert len(cleaned) <= MAX_CONTENT_HTML_CHARS
@pytest.mark.django_db
@responses.activate
def test_fetch_only_processes_the_first_max_entries_per_poll(feed):
items = "".join(
f"<item><title>Item {i}</title><guid>guid-{i}</guid></item>"
for i in range(MAX_ENTRIES_PER_POLL + 5)
)
xml = f"""<?xml version="1.0"?>
<rss version="2.0"><channel><title>Big feed</title>{items}</channel></rss>
"""
responses.add(responses.GET, FEED_URL, body=xml, status=200)
result = fetch_and_ingest(feed)
feed.save()
assert len(result.new_article_ids) == MAX_ENTRIES_PER_POLL
assert Article.objects.filter(feed=feed).count() == MAX_ENTRIES_PER_POLL
assert not Article.objects.filter(feed=feed, guid=f"guid-{MAX_ENTRIES_PER_POLL}").exists()
+138
View File
@@ -0,0 +1,138 @@
import socket
import pytest
import responses
from apps.feeds.services.ssrf_guard import (
MAX_REDIRECTS,
MAX_RESPONSE_BYTES,
SSRFBlocked,
safe_get,
validate_fetch_url,
)
# apps/feeds/conftest.py's autouse _fake_public_dns fixture pins every
# hostname to this address unless a test below overrides it directly.
PUBLIC_IP = "93.184.216.34"
def _patch_dns(monkeypatch, ip: str):
def _fake_getaddrinfo(host, *args, **kwargs):
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", (ip, 0))]
monkeypatch.setattr(socket, "getaddrinfo", _fake_getaddrinfo)
class TestValidateFetchUrl:
def test_allows_a_public_https_url(self):
validate_fetch_url("https://example.com/feed.xml")
def test_rejects_a_non_http_scheme(self):
with pytest.raises(SSRFBlocked, match="scheme"):
validate_fetch_url("file:///etc/passwd")
def test_rejects_a_url_with_no_hostname(self):
with pytest.raises(SSRFBlocked, match="hostname"):
validate_fetch_url("https:///feed.xml")
@pytest.mark.parametrize(
"ip",
[
"127.0.0.1", # loopback
"10.0.0.5", # RFC1918 private
"192.168.1.1", # RFC1918 private
"169.254.169.254", # link-local / cloud metadata
"0.0.0.0", # unspecified
"::1", # IPv6 loopback
"fc00::1", # IPv6 unique local
],
)
def test_rejects_a_url_resolving_to_a_non_public_address(self, monkeypatch, ip):
_patch_dns(monkeypatch, ip)
with pytest.raises(SSRFBlocked, match="non-public"):
validate_fetch_url("https://internal.example.com/feed.xml")
def test_rejects_a_url_whose_hostname_does_not_resolve(self, monkeypatch):
def _raise(*args, **kwargs):
raise socket.gaierror("nope")
monkeypatch.setattr(socket, "getaddrinfo", _raise)
with pytest.raises(SSRFBlocked, match="Could not resolve"):
validate_fetch_url("https://nowhere.example.com/feed.xml")
class TestSafeGet:
@responses.activate
def test_returns_the_response_body_for_a_normal_feed(self):
responses.add(responses.GET, "https://example.com/feed.xml", body="<rss></rss>")
response = safe_get("https://example.com/feed.xml", headers={}, timeout=5)
assert response.status_code == 200
assert response.content == b"<rss></rss>"
@responses.activate
def test_follows_a_safe_redirect(self):
responses.add(
responses.GET,
"https://example.com/feed.xml",
status=302,
headers={"Location": "https://example.com/final.xml"},
)
responses.add(responses.GET, "https://example.com/final.xml", body="<rss>final</rss>")
response = safe_get("https://example.com/feed.xml", headers={}, timeout=5)
assert response.content == b"<rss>final</rss>"
@responses.activate
def test_blocks_a_redirect_to_a_private_address(self, monkeypatch):
responses.add(
responses.GET,
"https://example.com/feed.xml",
status=302,
headers={"Location": "http://internal.example.com/secret"},
)
real_getaddrinfo = socket.getaddrinfo
def _fake_getaddrinfo(host, *args, **kwargs):
if host == "internal.example.com":
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("127.0.0.1", 0))]
return real_getaddrinfo(host, *args, **kwargs)
monkeypatch.setattr(socket, "getaddrinfo", _fake_getaddrinfo)
with pytest.raises(SSRFBlocked, match="non-public"):
safe_get("https://example.com/feed.xml", headers={}, timeout=5)
@responses.activate
def test_blocks_a_redirect_loop_past_max_redirects(self):
for _ in range(MAX_REDIRECTS + 2):
responses.add(
responses.GET,
"https://example.com/feed.xml",
status=302,
headers={"Location": "https://example.com/feed.xml"},
)
with pytest.raises(SSRFBlocked, match="Too many redirects"):
safe_get("https://example.com/feed.xml", headers={}, timeout=5)
@responses.activate
def test_blocks_a_response_over_the_size_cap(self):
responses.add(
responses.GET,
"https://example.com/feed.xml",
body=b"x" * (MAX_RESPONSE_BYTES + 1),
)
with pytest.raises(SSRFBlocked, match="exceeded"):
safe_get("https://example.com/feed.xml", headers={}, timeout=5)
@responses.activate
def test_preserves_case_insensitive_header_lookup(self):
responses.add(
responses.GET,
"https://example.com/feed.xml",
body="<rss></rss>",
headers={"etag": "abc123"},
)
response = safe_get("https://example.com/feed.xml", headers={}, timeout=5)
assert response.headers.get("ETag") == "abc123"
+69 -3
View File
@@ -16,17 +16,14 @@ def test_poll_due_feeds_dispatches_only_due_feeds():
recently_fetched = Feed.objects.create( recently_fetched = Feed.objects.create(
feed_url="https://b.example.com/feed.xml", feed_url="https://b.example.com/feed.xml",
last_fetched_at=timezone.now(), last_fetched_at=timezone.now(),
fetch_interval_minutes=60,
) )
overdue = Feed.objects.create( overdue = Feed.objects.create(
feed_url="https://c.example.com/feed.xml", feed_url="https://c.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2), last_fetched_at=timezone.now() - timedelta(hours=2),
fetch_interval_minutes=60,
) )
inactive_but_overdue = Feed.objects.create( inactive_but_overdue = Feed.objects.create(
feed_url="https://d.example.com/feed.xml", feed_url="https://d.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2), last_fetched_at=timezone.now() - timedelta(hours=2),
fetch_interval_minutes=60,
is_active=False, is_active=False,
) )
@@ -40,6 +37,56 @@ def test_poll_due_feeds_dispatches_only_due_feeds():
assert inactive_but_overdue.id not in dispatched_ids assert inactive_but_overdue.id not in dispatched_ids
@pytest.mark.django_db
def test_poll_due_feeds_does_not_reclaim_a_feed_it_just_claimed():
# Models a second beat tick (or an overlapping worker) running
# poll_due_feeds before the first claimed fetch_feed has finished —
# the feed must only be dispatched once.
overdue = Feed.objects.create(
feed_url="https://c.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2),
)
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
first_count = poll_due_feeds()
second_count = poll_due_feeds()
assert first_count == 1
assert second_count == 0
assert mock_delay.call_count == 1
overdue.refresh_from_db()
assert overdue.claimed_at is not None
@pytest.mark.django_db
def test_poll_due_feeds_reclaims_a_stale_claim():
from apps.feeds.tasks import CLAIM_STALE_MINUTES
overdue = Feed.objects.create(
feed_url="https://c.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2),
claimed_at=timezone.now() - timedelta(minutes=CLAIM_STALE_MINUTES + 1),
)
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
dispatched_count = poll_due_feeds()
assert dispatched_count == 1
assert mock_delay.call_args.args[0] == overdue.id
@pytest.mark.django_db
@responses.activate
def test_fetch_feed_task_clears_its_claim():
feed = Feed.objects.create(feed_url="https://example.com/feed.xml", claimed_at=timezone.now())
responses.add(responses.GET, feed.feed_url, body=GOOD_FEED_XML, status=200)
fetch_feed(feed.id)
feed.refresh_from_db()
assert feed.claimed_at is None
GOOD_FEED_XML = """<?xml version="1.0"?> GOOD_FEED_XML = """<?xml version="1.0"?>
<rss version="2.0"><channel> <rss version="2.0"><channel>
<title>Example Blog</title> <title>Example Blog</title>
@@ -75,3 +122,22 @@ def test_fetch_feed_task_skips_inactive_feed():
@pytest.mark.django_db @pytest.mark.django_db
def test_fetch_feed_task_skips_missing_feed(): def test_fetch_feed_task_skips_missing_feed():
assert fetch_feed(999999) is None assert fetch_feed(999999) is None
@pytest.mark.django_db
def test_fetch_feed_task_records_error_and_clears_claim_on_unexpected_exception():
feed = Feed.objects.create(
feed_url="https://example.com/feed.xml",
claimed_at=timezone.now(),
fetch_error_count=0,
)
with patch("apps.feeds.tasks.fetch_and_ingest", side_effect=RuntimeError("boom")):
with pytest.raises(RuntimeError, match="boom"):
fetch_feed(feed.id)
feed.refresh_from_db()
assert feed.last_fetch_status == Feed.STATUS_ERROR
assert feed.fetch_error_count == 1
assert feed.claimed_at is None
assert feed.last_fetched_at is not None
+5
View File
@@ -32,6 +32,11 @@ def test_service_worker_is_served_at_root_with_correct_type_and_scope(client):
assert 'self.addEventListener("fetch"' in body assert 'self.addEventListener("fetch"' in body
# Mutations must stay network-only, never intercepted. # Mutations must stay network-only, never intercepted.
assert 'request.method !== "GET"' in body assert 'request.method !== "GET"' in body
# Logging out must purge PAGES_CACHE so a later offline navigation on
# a shared/account-switched device can't serve another user's
# previously cached authenticated pages.
assert "caches.delete(PAGES_CACHE)" in body
assert reverse("accounts:logout") in body
@pytest.mark.django_db @pytest.mark.django_db
+24
View File
@@ -164,3 +164,27 @@ def test_saved_filter_only_shows_saved_articles(client, alice, alices_article):
response = client.get(reverse("reader:stream"), {"filter": "starred"}) response = client.get(reverse("reader:stream"), {"filter": "starred"})
assert alices_article.title not in response.content.decode() assert alices_article.title not in response.content.decode()
@pytest.mark.django_db
def test_sidebar_shows_a_tags_total_unread_count(client, alice):
from apps.articles.models import SubscriptionTag, Tag
feed_a = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
feed_b = Feed.objects.create(feed_url="https://b.example.com/feed.xml")
sub_a = Subscription.objects.create(user=alice, feed=feed_a)
sub_b = Subscription.objects.create(user=alice, feed=feed_b)
tag = Tag.objects.create(user=alice, name="work")
SubscriptionTag.objects.create(user=alice, subscription=sub_a, tag=tag)
SubscriptionTag.objects.create(user=alice, subscription=sub_b, tag=tag)
for i in range(3):
Article.objects.create(feed=feed_a, guid=f"a-{i}", title=f"A{i}")
for i in range(4):
Article.objects.create(feed=feed_b, guid=f"b-{i}", title=f"B{i}")
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
# The only "7" on the page — the individual feeds show 3 and 4.
content = response.content.decode()
assert '<span class="unread-count">7</span>' in content
+68 -4
View File
@@ -9,6 +9,7 @@ from __future__ import annotations
import logging import logging
import re import re
import signal
from django.utils import timezone from django.utils import timezone
@@ -27,6 +28,42 @@ logger = logging.getLogger(__name__)
_TAG_RE = re.compile(r"<[^>]+>") _TAG_RE = re.compile(r"<[^>]+>")
# Bounds how much CPU/time a single user-authored regex rule can burn
# against one article. re has no built-in match timeout, so this uses
# SIGALRM to interrupt a runaway match (catastrophic backtracking) —
# only effective on the main thread of a process with SIGALRM available,
# which covers gunicorn's sync workers and Celery's (default) prefork
# pool; elsewhere it's skipped and MAX_REGEX_HAYSTACK_CHARS is the only
# bound. reapply_rules_for_user (apps.rules.tasks) also carries a Celery
# task time_limit as a second, coarser backstop.
REGEX_TIMEOUT_SECONDS = 1
MAX_REGEX_HAYSTACK_CHARS = 20_000
class _RegexTimeout(Exception):
pass
def _raise_regex_timeout(signum, frame):
raise _RegexTimeout()
def _regex_search_with_timeout(pattern: str, haystack: str, flags: int):
if not hasattr(signal, "SIGALRM"):
return re.search(pattern, haystack, flags)
try:
previous_handler = signal.signal(signal.SIGALRM, _raise_regex_timeout)
except ValueError:
# Not the main thread (e.g. a threaded dev server) — signal
# timers only work there.
return re.search(pattern, haystack, flags)
signal.setitimer(signal.ITIMER_REAL, REGEX_TIMEOUT_SECONDS)
try:
return re.search(pattern, haystack, flags)
finally:
signal.setitimer(signal.ITIMER_REAL, 0)
signal.signal(signal.SIGALRM, previous_handler)
def _haystack(article: Article, match_field: str) -> str: def _haystack(article: Article, match_field: str) -> str:
if match_field == FilterRule.FIELD_TITLE: if match_field == FilterRule.FIELD_TITLE:
@@ -47,8 +84,12 @@ def rule_matches(rule: FilterRule, article: Article) -> bool:
if rule.match_type == FilterRule.MATCH_REGEX: if rule.match_type == FilterRule.MATCH_REGEX:
flags = 0 if rule.case_sensitive else re.IGNORECASE flags = 0 if rule.case_sensitive else re.IGNORECASE
haystack = raw_haystack[:MAX_REGEX_HAYSTACK_CHARS]
try: try:
return re.search(rule.match_value, raw_haystack, flags) is not None return _regex_search_with_timeout(rule.match_value, haystack, flags) is not None
except _RegexTimeout:
logger.warning("Regex timed out for rule %s", rule.pk)
return False
except re.error as exc: except re.error as exc:
logger.warning("Invalid regex in rule %s: %s", rule.pk, exc) logger.warning("Invalid regex in rule %s: %s", rule.pk, exc)
return False return False
@@ -100,13 +141,36 @@ def apply_rule_action(rule: FilterRule, user, article: Article, state: UserArtic
return False return False
def evaluate_rules_for_article(user, article: Article, rules) -> None: def rule_in_scope(rule: FilterRule, feed_id, subscription_tag_ids) -> bool:
"""Evaluates an ordered iterable of rules (already scoped/enabled) """Whether `rule`'s own feed/tag scope (see FilterRule's docstring)
against one article for one user, applying actions as they match.""" covers an article from `feed_id`, given the user's subscription-tag
ids for that feed.
Centralized here — rather than left for each caller to re-derive via
its own query filters — so ingest-time dispatch and backlog re-apply
can't drift into scoping rules differently (see apps.articles.tasks
and apps.rules.tasks).
"""
if rule.feed_id is not None and rule.feed_id != feed_id:
return False
if rule.tag_id is not None and rule.tag_id not in subscription_tag_ids:
return False
return True
def evaluate_rules_for_article(user, article: Article, rules, subscription_tag_ids=()) -> None:
"""Evaluates an ordered iterable of a user's enabled rules against one
article, applying actions as they match. `rules` doesn't need to be
pre-scoped to this article's feed/tags — out-of-scope rules (per
rule_in_scope) are skipped here, so callers can pass one user's whole
enabled ruleset regardless of which feed(s) the article(s) belong to.
"""
matched_any = False matched_any = False
state = None state = None
for rule in rules: for rule in rules:
if not rule_in_scope(rule, article.feed_id, subscription_tag_ids):
continue
if not rule_matches(rule, article): if not rule_matches(rule, article):
continue continue
if state is None: if state is None:
+24 -6
View File
@@ -1,6 +1,6 @@
from celery import shared_task from celery import shared_task
from apps.articles.models import Article from apps.articles.models import Article, SubscriptionTag
from apps.feeds.models import Subscription from apps.feeds.models import Subscription
from .engine import evaluate_rules_for_article from .engine import evaluate_rules_for_article
@@ -12,9 +12,20 @@ def reapply_rules_for_user_backlog(rule_id: int) -> int:
rule, so priority/short-circuit ordering stays correct) against their rule, so priority/short-circuit ordering stays correct) against their
unread backlog, scoped by the triggering rule's own feed/tag scope. unread backlog, scoped by the triggering rule's own feed/tag scope.
Exists as a plain function so it can be called synchronously for The *backlog* is narrowed to the triggering rule's own feed/tag scope
immediate feedback (e.g. from a "Re-apply" view) or wrapped in a task (re-applying a feed-scoped rule has no reason to touch other feeds'
for background use — see reapply_rules_for_user below. articles), but which of the user's *other* rules apply to each of
those articles is still per-article/per-feed — evaluate_rules_for_article
checks that via rule_in_scope, so a feed- or tag-scoped rule for a
different feed can't fire on this backlog just because it happened to
match (see apps.rules.engine.rule_in_scope).
Exists as a plain function, separate from the reapply_rules_for_user
task below, so it's unit-testable without Celery — but every caller
should go through that task (see apps.rules.views.reapply_rule), not
call this directly from a request: a large backlog or a pathological
regex (bounded but not eliminated by apps.rules.engine's timeout
guard) could otherwise tie up a web worker for a long time.
""" """
try: try:
rule = FilterRule.objects.select_related("user").get(pk=rule_id) rule = FilterRule.objects.select_related("user").get(pk=rule_id)
@@ -45,13 +56,20 @@ def reapply_rules_for_user_backlog(rule_id: int) -> int:
rules = list(FilterRule.objects.filter(user=user, is_enabled=True).order_by("priority", "id")) rules = list(FilterRule.objects.filter(user=user, is_enabled=True).order_by("priority", "id"))
subscription_tag_ids_by_feed: dict[int, set[int]] = {}
for feed_id, tag_id in SubscriptionTag.objects.filter(
user=user, subscription__feed_id__in=feed_ids
).values_list("subscription__feed_id", "tag_id"):
subscription_tag_ids_by_feed.setdefault(feed_id, set()).add(tag_id)
count = 0 count = 0
for article in backlog: for article in backlog:
evaluate_rules_for_article(user, article, rules) subscription_tag_ids = subscription_tag_ids_by_feed.get(article.feed_id, set())
evaluate_rules_for_article(user, article, rules, subscription_tag_ids)
count += 1 count += 1
return count return count
@shared_task @shared_task(time_limit=120, soft_time_limit=90)
def reapply_rules_for_user(rule_id: int): def reapply_rules_for_user(rule_id: int):
return reapply_rules_for_user_backlog(rule_id) return reapply_rules_for_user_backlog(rule_id)
+37 -1
View File
@@ -1,3 +1,5 @@
import time
import pytest import pytest
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
@@ -10,7 +12,7 @@ from apps.articles.models import (
) )
from apps.feeds.models import Feed from apps.feeds.models import Feed
from .engine import evaluate_rules_for_article, rule_matches from .engine import evaluate_rules_for_article, rule_in_scope, rule_matches
from .models import FilterRule from .models import FilterRule
User = get_user_model() User = get_user_model()
@@ -87,6 +89,22 @@ class TestRuleMatches:
rule = make_rule(user, match_type=FilterRule.MATCH_REGEX, match_value="(unclosed") rule = make_rule(user, match_type=FilterRule.MATCH_REGEX, match_value="(unclosed")
assert rule_matches(rule, article) is False assert rule_matches(rule, article) is False
def test_catastrophic_backtracking_regex_times_out_instead_of_hanging(self, user, article):
# (a+)+$ against a long run of a's followed by a non-matching
# character is a textbook ReDoS pattern — left unguarded this
# takes on the order of a minute (confirmed locally); the
# REGEX_TIMEOUT_SECONDS guard in apps.rules.engine must interrupt
# it well before that instead of hanging the caller.
article.title = "a" * 30 + "!"
rule = make_rule(user, match_type=FilterRule.MATCH_REGEX, match_value=r"(a+)+$")
start = time.monotonic()
result = rule_matches(rule, article)
elapsed = time.monotonic() - start
assert result is False
assert elapsed < 5
def test_content_field_strips_html_tags(self, user, article): def test_content_field_strips_html_tags(self, user, article):
rule = make_rule(user, match_field=FilterRule.FIELD_CONTENT, match_value="Lots of details") rule = make_rule(user, match_field=FilterRule.FIELD_CONTENT, match_value="Lots of details")
assert rule_matches(rule, article) is True assert rule_matches(rule, article) is True
@@ -104,6 +122,24 @@ class TestRuleMatches:
assert rule_matches(rule, article) is True assert rule_matches(rule, article) is True
@pytest.mark.django_db
class TestRuleInScope:
def test_unscoped_rule_applies_to_any_feed(self, user):
rule = make_rule(user)
assert rule_in_scope(rule, feed_id=999, subscription_tag_ids=set()) is True
def test_feed_scoped_rule_only_applies_to_its_own_feed(self, user, feed):
rule = make_rule(user, feed=feed)
assert rule_in_scope(rule, feed_id=feed.id, subscription_tag_ids=set()) is True
assert rule_in_scope(rule, feed_id=feed.id + 1, subscription_tag_ids=set()) is False
def test_tag_scoped_rule_only_applies_when_the_feed_carries_that_tag(self, user):
tag = Tag.objects.create(user=user, name="work")
rule = make_rule(user, tag=tag)
assert rule_in_scope(rule, feed_id=1, subscription_tag_ids={tag.id}) is True
assert rule_in_scope(rule, feed_id=1, subscription_tag_ids=set()) is False
@pytest.mark.django_db @pytest.mark.django_db
class TestApplyRuleAction: class TestApplyRuleAction:
def test_star_action(self, user, article): def test_star_action(self, user, article):
+103 -1
View File
@@ -1,8 +1,17 @@
from unittest.mock import patch
import pytest import pytest
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.urls import reverse from django.urls import reverse
from apps.articles.models import STARRED_TAG_NAME, Article, ArticleTag, UserArticleState from apps.articles.models import (
STARRED_TAG_NAME,
Article,
ArticleTag,
SubscriptionTag,
Tag,
UserArticleState,
)
from apps.feeds.models import Feed, Subscription from apps.feeds.models import Feed, Subscription
from .models import FilterRule from .models import FilterRule
@@ -119,6 +128,28 @@ def test_reapply_rule_view_applies_to_unread_backlog(client, alice, feed):
assert is_starred(alice, article) is True assert is_starred(alice, article) is True
@pytest.mark.django_db
def test_reapply_rule_view_always_queues_through_celery_never_runs_inline(client, alice, feed):
# A large backlog or a pathological regex must not be able to tie up
# the request/response cycle — the view should dispatch a task, not
# call reapply_rules_for_user_backlog directly.
Subscription.objects.create(user=alice, feed=feed)
rule = FilterRule.objects.create(
user=alice,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_STAR,
)
client.login(username="alice", password="pw-alice-123")
with patch("apps.rules.views.reapply_rules_for_user.delay") as mock_delay:
response = client.post(reverse("rules:reapply_rule", args=[rule.pk]))
assert response.status_code == 302
mock_delay.assert_called_once_with(rule.pk)
@pytest.mark.django_db @pytest.mark.django_db
def test_reapply_skips_already_read_articles(alice, feed): def test_reapply_skips_already_read_articles(alice, feed):
article = Article.objects.create(feed=feed, guid="guid-1", title="Python 4 released") article = Article.objects.create(feed=feed, guid="guid-1", title="Python 4 released")
@@ -160,3 +191,74 @@ def test_reapply_respects_feed_scope(alice):
assert is_starred(alice, article_a) is True assert is_starred(alice, article_a) is True
assert not UserArticleState.objects.filter(user=alice, article=article_b).exists() assert not UserArticleState.objects.filter(user=alice, article=article_b).exists()
@pytest.mark.django_db
def test_reapply_does_not_apply_a_different_feeds_scoped_rule_to_the_backlog():
# Regression test for a bug where re-applying one feed-scoped rule
# loaded *every* enabled rule and ran it against the whole backlog
# with no scope check — so a same-matching rule scoped to a feed
# the user isn't re-applying would still fire on this backlog.
feed_a = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
feed_b = Feed.objects.create(feed_url="https://b.example.com/feed.xml")
alice = User.objects.create_user("alice", password="pw-alice-123")
Subscription.objects.create(user=alice, feed=feed_a)
Subscription.objects.create(user=alice, feed=feed_b)
article_a = Article.objects.create(feed=feed_a, guid="guid-1", title="Python in A")
triggering_rule = FilterRule.objects.create(
user=alice,
feed=feed_a,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_STAR,
)
FilterRule.objects.create(
user=alice,
feed=feed_b,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_TAG,
action_value="feed-b-only",
)
reapply_rules_for_user_backlog(triggering_rule.pk)
assert is_starred(alice, article_a) is True
assert not Tag.objects.filter(user=alice, name="feed-b-only").exists()
@pytest.mark.django_db
def test_reapply_applies_a_tag_scoped_rule_only_to_that_tags_feeds():
feed_a = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
feed_b = Feed.objects.create(feed_url="https://b.example.com/feed.xml")
alice = User.objects.create_user("alice", password="pw-alice-123")
sub_a = Subscription.objects.create(user=alice, feed=feed_a)
Subscription.objects.create(user=alice, feed=feed_b)
tag = Tag.objects.create(user=alice, name="work")
SubscriptionTag.objects.create(user=alice, subscription=sub_a, tag=tag)
article_a = Article.objects.create(feed=feed_a, guid="guid-1", title="Python in A")
article_b = Article.objects.create(feed=feed_b, guid="guid-2", title="Python in B")
unscoped_trigger = FilterRule.objects.create(
user=alice,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="unrelated-trigger-value",
action=FilterRule.ACTION_MARK_READ,
)
FilterRule.objects.create(
user=alice,
tag=tag,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_STAR,
)
reapply_rules_for_user_backlog(unscoped_trigger.pk)
assert is_starred(alice, article_a) is True
assert is_starred(alice, article_b) is False
+6 -5
View File
@@ -5,7 +5,7 @@ from django.views.decorators.http import require_POST
from .forms import FilterRuleForm from .forms import FilterRuleForm
from .models import FilterRule from .models import FilterRule
from .tasks import reapply_rules_for_user_backlog from .tasks import reapply_rules_for_user
@login_required @login_required
@@ -48,8 +48,9 @@ def delete_rule(request, rule_id):
@require_POST @require_POST
def reapply_rule(request, rule_id): def reapply_rule(request, rule_id):
rule = get_object_or_404(FilterRule, user=request.user, pk=rule_id) rule = get_object_or_404(FilterRule, user=request.user, pk=rule_id)
# Run synchronously for immediate feedback, same pattern as add_feed — # Always queued, never run inline from the request/response cycle —
# a Celery worker isn't required for this to work. # a large backlog or a pathological regex could otherwise tie up a
count = reapply_rules_for_user_backlog(rule.pk) # web worker for a long time (see apps.rules.tasks).
messages.success(request, f"Re-applied your rules to {count} unread article(s).") reapply_rules_for_user.delay(rule.pk)
messages.success(request, "Re-applying your rules to your unread backlog…")
return redirect("rules:rule_list") return redirect("rules:rule_list")
+25
View File
@@ -106,9 +106,34 @@ EVECTION_SIGNUP_ENABLED = env.bool("EVECTION_SIGNUP_ENABLED", default=False)
EVECTION_MIN_FETCH_INTERVAL_MINUTES = env.int("EVECTION_MIN_FETCH_INTERVAL_MINUTES", default=15) EVECTION_MIN_FETCH_INTERVAL_MINUTES = env.int("EVECTION_MIN_FETCH_INTERVAL_MINUTES", default=15)
EVECTION_PWA_CACHE_VERSION = env.str("EVECTION_PWA_CACHE_VERSION", default="v1") EVECTION_PWA_CACHE_VERSION = env.str("EVECTION_PWA_CACHE_VERSION", default="v1")
# Redis-backed rather than Django's default per-process LocMemCache: login
# rate limiting (apps.accounts.views) must share counters across every
# gunicorn worker, not just whichever one happened to handle a given
# request. Redis is already a hard install requirement for Celery, so
# this adds no new infrastructure — just a distinct key prefix so cache
# keys can't collide with anything else sharing that Redis instance.
CACHES = {
"default": {
"BACKEND": "django.core.cache.backends.redis.RedisCache",
"LOCATION": env.str("REDIS_URL", default="redis://localhost:6379/0"),
"KEY_PREFIX": "evection-cache",
}
}
# Celery # Celery
CELERY_BROKER_URL = env.str("REDIS_URL", default="redis://localhost:6379/0") CELERY_BROKER_URL = env.str("REDIS_URL", default="redis://localhost:6379/0")
CELERY_RESULT_BACKEND = None CELERY_RESULT_BACKEND = None
CELERY_TASK_SERIALIZER = "json" CELERY_TASK_SERIALIZER = "json"
CELERY_ACCEPT_CONTENT = ["json"] CELERY_ACCEPT_CONTENT = ["json"]
CELERY_TIMEZONE = TIME_ZONE CELERY_TIMEZONE = TIME_ZONE
# Without these, a worker process killed mid-task (OOM, deploy restart,
# crash) silently loses that task: the default is to ack a message the
# moment a worker *starts* it, not when it finishes. acks_late + reject-
# on-worker-lost means the message is only consumed on completion (success
# or a handled failure) and is redelivered to another worker otherwise.
# prefetch_multiplier=1 keeps a worker from grabbing a batch of tasks it
# could similarly lose all at once.
CELERY_TASK_ACKS_LATE = True
CELERY_TASK_REJECT_ON_WORKER_LOST = True
CELERY_WORKER_PREFETCH_MULTIPLIER = 1
+49 -4
View File
@@ -1,13 +1,58 @@
from django.core.exceptions import ImproperlyConfigured
from .base import * # noqa: F403 from .base import * # noqa: F403
from .base import env from .base import SECRET_KEY, env
DEBUG = False DEBUG = False
# nginx terminates TLS and proxies to gunicorn over a Unix socket (see # base.py falls back to a publicly known placeholder if SECRET_KEY isn't
# docs/install.md) — this tells Django's request.is_secure() to trust the # set — fine for a throwaway dev DB, but production must never sign
# header nginx sets, and the redirect/cookie/HSTS settings below assume it. # sessions/CSRF tokens with a key anyone can read in this repo's history.
# Fail startup rather than quietly running with a known key.
if not SECRET_KEY or SECRET_KEY == "django-insecure-change-me-in-production":
raise ImproperlyConfigured(
"SECRET_KEY is missing or still the placeholder default. Set a unique, "
"random SECRET_KEY in .env before running with prod settings — generate "
'one with: python -c "from django.core.management.utils import '
'get_random_secret_key; print(get_random_secret_key())"'
)
# Health checks (scripts/upgrade.sh, an external uptime monitor, ...) hit
# gunicorn directly over loopback, bypassing the reverse proxy entirely —
# that request's Host header is "127.0.0.1:<port>", not the public domain
# in ALLOWED_HOSTS, so without this Django 400s it. Harmless to always
# allow: gunicorn's bind address is never reachable from outside this host
# in the first place (see docs/install.md's "Reverse proxy" section).
ALLOWED_HOSTS = [*ALLOWED_HOSTS, "localhost", "127.0.0.1", "::1"] # noqa: F405
# A reverse proxy (nginx, Caddy, Traefik, Nginx Proxy Manager on another
# host, ...) is required in front of gunicorn in every deployment — see
# docs/install.md — and it's always the one terminating TLS. This tells
# Django's request.is_secure() to trust the X-Forwarded-Proto header it
# sets, and the redirect/cookie/HSTS settings below assume it's set.
SECURE_SSL_REDIRECT = env.bool("SECURE_SSL_REDIRECT", default=True) SECURE_SSL_REDIRECT = env.bool("SECURE_SSL_REDIRECT", default=True)
SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https") SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")
# /health/ is also exempted from the HTTPS redirect above: hit directly
# over loopback (see ALLOWED_HOSTS above) it has no X-Forwarded-Proto
# header to prove it's "secure", and there's no TLS listener on gunicorn's
# own port to redirect to anyway.
SECURE_REDIRECT_EXEMPT = [r"^health/$"]
# WhiteNoise serves /static/ directly from gunicorn instead of relying on
# the reverse proxy to alias the staticfiles directory on disk — the proxy
# may not even be on this host (see docs/install.md), so it can't be
# trusted to have filesystem access to $APP_DIR/staticfiles.
MIDDLEWARE = [
"django.middleware.security.SecurityMiddleware",
"whitenoise.middleware.WhiteNoiseMiddleware",
*MIDDLEWARE[1:], # noqa: F405
]
STORAGES = {
"default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
"staticfiles": {
"BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage",
},
}
SESSION_COOKIE_SECURE = True SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True CSRF_COOKIE_SECURE = True
+7
View File
@@ -5,3 +5,10 @@ CELERY_TASK_ALWAYS_EAGER = True
CELERY_TASK_EAGER_PROPAGATES = True CELERY_TASK_EAGER_PROPAGATES = True
PASSWORD_HASHERS = ["django.contrib.auth.hashers.MD5PasswordHasher"] PASSWORD_HASHERS = ["django.contrib.auth.hashers.MD5PasswordHasher"]
# base.py's CACHES needs a real Redis instance — tests must not depend on
# one being available, and a plain per-process cache is also what keeps
# login-rate-limit tests isolated from each other (LocMemCache's default
# location is per-process but shared across tests in that process; see
# apps.accounts.test_rate_limit's cache.clear() fixture).
CACHES = {"default": {"BACKEND": "django.core.cache.backends.locmem.LocMemCache"}}
+6 -5
View File
@@ -9,11 +9,12 @@ Group=evection
WorkingDirectory=/opt/evection/app WorkingDirectory=/opt/evection/app
EnvironmentFile=/opt/evection/app/.env EnvironmentFile=/opt/evection/app/.env
Environment=DJANGO_SETTINGS_MODULE=config.settings.prod Environment=DJANGO_SETTINGS_MODULE=config.settings.prod
RuntimeDirectory=evection # Evection is always meant to sit behind a reverse proxy (see
ExecStart=/opt/evection/venv/bin/gunicorn \ # docs/install.md) — it listens on a plain TCP address, never a Unix
--workers 3 \ # socket, so that proxy can be on this host or a different one.
--bind unix:/run/evection/gunicorn.sock \ # EVECTION_GUNICORN_BIND (set in .env by scripts/install.sh) controls
config.wsgi:application # what address:port that is.
ExecStart=/bin/sh -c 'exec /opt/evection/venv/bin/gunicorn --workers 3 --bind "$EVECTION_GUNICORN_BIND" config.wsgi:application'
ExecReload=/bin/kill -s HUP $MAINPID ExecReload=/bin/kill -s HUP $MAINPID
Restart=on-failure Restart=on-failure
+15 -4
View File
@@ -1,6 +1,12 @@
# Reverse proxy for Evection. Copy to /etc/nginx/sites-available/evection, # Reverse proxy for Evection, for the case where nginx runs on the SAME host
# adjust server_name and the certificate paths (certbot will fill those in # as Evection (see docs/install.md's "Reverse proxy" section — if your proxy
# for you if you use `certbot --nginx`), then symlink into sites-enabled. # is on a different host/VM instead, e.g. Nginx Proxy Manager, this file
# doesn't apply to you directly, but the proxy_set_header lines below still
# show what any proxy needs to forward). Copy to
# /etc/nginx/sites-available/evection, adjust server_name, the certificate
# paths (certbot will fill those in for you if you use `certbot --nginx`),
# and the port in proxy_pass if you didn't use EVECTION_GUNICORN_BIND's
# default of 127.0.0.1:8000, then symlink into sites-enabled.
server { server {
listen 80; listen 80;
@@ -24,6 +30,11 @@ server {
client_max_body_size 10M; # OPML uploads are small, but leave headroom client_max_body_size 10M; # OPML uploads are small, but leave headroom
# Optional: gunicorn already serves /static/ itself via WhiteNoise (with
# its own far-future cache headers, since filenames are content-hashed),
# so this block is purely a same-host perf optimization that skips
# gunicorn for static assets — not required, and not an option at all
# once the proxy is on a different host than $APP_DIR/staticfiles.
location /static/ { location /static/ {
alias /opt/evection/app/staticfiles/; alias /opt/evection/app/staticfiles/;
expires 30d; expires 30d;
@@ -36,7 +47,7 @@ server {
# else below, so no special-casing is needed here. # else below, so no special-casing is needed here.
location / { location / {
proxy_pass http://unix:/run/evection/gunicorn.sock; proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+71 -31
View File
@@ -1,9 +1,16 @@
# Installing Evection # Installing Evection
Evection is a traditional Python/Django application: `pip install`, a WSGI server Evection is a traditional Python/Django application: `pip install`, a WSGI server
(gunicorn), a Celery worker + beat scheduler for background feed polling, and an (gunicorn), and a Celery worker + beat scheduler for background feed polling.
nginx reverse proxy in front of it. There's no Docker image for v1 — everything There's no Docker image for v1 — everything below runs directly on the host.
below runs directly on the host.
**A reverse proxy in front of gunicorn is required, always** — Evection has no
TLS support of its own and assumes one is terminating HTTPS for it (see "Reverse
proxy" below). It can be nginx/Caddy/Traefik running on the same host, or
something like Nginx Proxy Manager running on an entirely different host/VM —
`scripts/install.sh` doesn't set one up for you, since which one and where it
runs is up to you; it just leaves gunicorn listening on a TCP address for that
proxy to reach.
Both SQLite and PostgreSQL are fully supported. Steps that differ are marked Both SQLite and PostgreSQL are fully supported. Steps that differ are marked
**SQLite** / **Postgres**; everything else applies to both. **SQLite** / **Postgres**; everything else applies to both.
@@ -32,9 +39,11 @@ run at all until a tag exists.
## Quick install ## Quick install
Once the prerequisites below are in place, `scripts/install.sh` automates Once the prerequisites below are in place, `scripts/install.sh` automates
everything from "create a system user" through "systemd units are running" — everything from "create a system user" through "systemd units are running",
and optionally nginx + a Let's Encrypt certificate too. It's interactive including asking where your reverse proxy will run so it can set up gunicorn's
(asks for your domain and database choice) and safe to re-run. listen address accordingly. It's interactive (asks for your domain, database
choice, and proxy location) and safe to re-run. It does **not** set up the
proxy itself — see "Reverse proxy" below for that, once this finishes.
```bash ```bash
sudo mkdir -p /opt/evection sudo mkdir -p /opt/evection
@@ -57,18 +66,20 @@ and 4 below first — prerequisites and what `.env` controls.
- Python 3.11 or 3.12, with its `venv` module (Debian/Ubuntu split this into a - Python 3.11 or 3.12, with its `venv` module (Debian/Ubuntu split this into a
separate `python3.X-venv` package — see below) separate `python3.X-venv` package — see below)
- Redis (Celery's broker) - Redis (Celery's broker, and the app's cache — e.g. login rate limiting)
- **Postgres only**: a PostgreSQL server (12+) and its client headers if you're not - **Postgres only**: a PostgreSQL server (12+) and its client headers if you're not
using the bundled `psycopg[binary]` wheel (most platforms are covered by the wheel, using the bundled `psycopg[binary]` wheel (most platforms are covered by the wheel,
so this is rarely needed) so this is rarely needed)
- nginx (or another reverse proxy) - A reverse proxy, already running or that you're about to set up — on this
- A domain name pointed at the host, if you want this reachable outside your LAN host or a different one (see "Reverse proxy" below)
- A domain name pointed at wherever that proxy terminates TLS, if you want
this reachable outside your LAN
Debian/Ubuntu example for the non-Python pieces (swap `python3.13-venv` for Debian/Ubuntu example for the non-Python pieces (swap `python3.13-venv` for
whatever `python3 --version` reports on your system): whatever `python3 --version` reports on your system):
```bash ```bash
sudo apt install redis-server nginx python3.13-venv sudo apt install redis-server python3.13-venv
# Postgres only: # Postgres only:
sudo apt install postgresql sudo apt install postgresql
``` ```
@@ -129,11 +140,19 @@ At minimum, set the following in `.env`:
sudo -u postgres createdb evection -O evection sudo -u postgres createdb evection -O evection
``` ```
- `REDIS_URL` — `redis://localhost:6379/0` is fine for a single-host install - `REDIS_URL` — `redis://localhost:6379/0` is fine for a single-host install
- `EVECTION_GUNICORN_BIND` — the `address:port` gunicorn listens on; see
"Reverse proxy" below for how to pick this
How often every feed is polled is a site-wide setting (Django admin → Site
settings), not a `.env` value — `EVECTION_MIN_FETCH_INTERVAL_MINUTES` only sets
a floor under it, so it can't be configured lower than that without editing `.env`.
Evection reads `config.settings.prod` when run via the commands below (see each Evection reads `config.settings.prod` when run via the commands below (see each
systemd unit's `DJANGO_SETTINGS_MODULE`). That module turns on HTTPS-only cookies, systemd unit's `DJANGO_SETTINGS_MODULE`). That module turns on HTTPS-only cookies,
HSTS, and a Content-Security-Policy header on top of the base settings — see HSTS, and a Content-Security-Policy header on top of the base settings, and
`config/settings/prod.py` if you need to adjust it for your proxy setup. serves static files itself via WhiteNoise rather than relying on the proxy to
have filesystem access to this host — see `config/settings/prod.py` if you need
to adjust any of that.
## 5. Initialize the database and static files ## 5. Initialize the database and static files
@@ -192,28 +211,47 @@ sudo systemctl status evection-gunicorn evection-celery-worker evection-celery-b
sudo journalctl -u evection-gunicorn -f sudo journalctl -u evection-gunicorn -f
``` ```
## 7. nginx ## 7. Reverse proxy
Copy `docs/deploy/nginx.conf.example` to `/etc/nginx/sites-available/evection`, gunicorn only ever listens on plain HTTP, on the `EVECTION_GUNICORN_BIND`
adjust `server_name` and the certificate paths, then enable it: address you set in step 4 — it never terminates TLS itself. A reverse proxy
in front of it is not optional: it's what the setup wizard, login, and every
other page actually get reached through. Two shapes this takes:
**Proxy on the same host as Evection.** Set `EVECTION_GUNICORN_BIND=127.0.0.1:8000`
(install.sh's default when you say the proxy is local) so gunicorn isn't
reachable from outside this machine at all — only the proxy, listening on
80/443, is. `docs/deploy/nginx.conf.example` is a working nginx config for
this case; copy it into place and get a certificate the usual way:
```bash ```bash
sudo cp docs/deploy/nginx.conf.example /etc/nginx/sites-available/evection sudo cp docs/deploy/nginx.conf.example /etc/nginx/sites-available/evection
# edit it: server_name, and the proxy_pass port if you didn't use 8000
sudo ln -s /etc/nginx/sites-available/evection /etc/nginx/sites-enabled/ sudo ln -s /etc/nginx/sites-available/evection /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx sudo nginx -t && sudo systemctl reload nginx
```
Get a TLS certificate (Let's Encrypt via certbot is the easiest path):
```bash
sudo apt install certbot python3-certbot-nginx sudo apt install certbot python3-certbot-nginx
sudo certbot --nginx -d your-domain.example sudo certbot --nginx -d your-domain.example
``` ```
Evection's `prod` settings enable `SECURE_SSL_REDIRECT` and HSTS by default, so **Proxy on a different host or VM** (Nginx Proxy Manager, a Caddy box, a
plan to have TLS working before your first real visit — see container host, etc.). Set `EVECTION_GUNICORN_BIND=0.0.0.0:8000` (or to this
`config/settings/prod.py` to disable the redirect temporarily if you need to debug host's specific IP) so the proxy can actually reach gunicorn over the
over plain HTTP first. network, then **firewall that port to only accept connections from the
proxy's host** — gunicorn itself has no auth in front of it, so it must
never be reachable from the open internet directly. Point the proxy's
upstream/forward target at `this-host:8000`, and make sure it forwards
`X-Forwarded-Proto` (whatever sets up TLS passthrough/termination normally
handles this, but double-check — see the next paragraph for why it matters).
Whichever proxy you use, `docs/deploy/nginx.conf.example`'s `location /`
block shows the three headers that need forwarding either way.
Either way, the proxy must set `X-Forwarded-Proto: https` on requests it
forwards once it's terminating TLS — Evection's `prod` settings
(`SECURE_PROXY_SSL_HEADER` in `config/settings/prod.py`) trust that header to
know the original request was HTTPS, which is what keeps `SECURE_SSL_REDIRECT`
from causing a redirect loop. Plan to have TLS working end-to-end before your
first real visit; see `config/settings/prod.py` to disable the redirect
temporarily if you need to debug over plain HTTP first.
## 8. Visit the site ## 8. Visit the site
@@ -257,9 +295,11 @@ Equivalent by hand:
```bash ```bash
cd /opt/evection/app cd /opt/evection/app
sudo -u evection -H git fetch --tags --force git fetch --tags --force
LATEST_TAG="$(sudo -u evection -H git tag --sort=-creatordate | head -n1)" LATEST_TAG="$(git tag --sort=-creatordate | head -n1)"
sudo -u evection -H git checkout --detach "$LATEST_TAG" git checkout --detach "$LATEST_TAG"
chown -R evection:evection /opt/evection/app
sed -i "s/^EVECTION_PWA_CACHE_VERSION=.*/EVECTION_PWA_CACHE_VERSION=$LATEST_TAG/" .env
sudo -u evection -H /opt/evection/venv/bin/pip install . sudo -u evection -H /opt/evection/venv/bin/pip install .
sudo -u evection -H env DJANGO_SETTINGS_MODULE=config.settings.prod \ sudo -u evection -H env DJANGO_SETTINGS_MODULE=config.settings.prod \
/opt/evection/venv/bin/python manage.py migrate /opt/evection/venv/bin/python manage.py migrate
@@ -276,7 +316,7 @@ git tag v$(date +%Y.%m.%d)
git push --tags git push --tags
``` ```
Bump `EVECTION_PWA_CACHE_VERSION` in `.env` whenever a release changes cached `upgrade.sh` sets `EVECTION_PWA_CACHE_VERSION` in `.env` to the deployed tag on every
static assets (CSS/JS/icons) — it's baked into the service worker's cache name, so run — it's baked into the service worker's cache name, so this forces every installed
changing it forces every installed PWA to drop its old cache instead of serving PWA to drop its old cache instead of serving stale assets after the upgrade. You don't
stale assets after the upgrade. need to bump it by hand.
+1
View File
@@ -22,6 +22,7 @@ dependencies = [
"psycopg[binary]>=3.1,<4", "psycopg[binary]>=3.1,<4",
"gunicorn>=21,<24", "gunicorn>=21,<24",
"django-csp>=4.0,<5", "django-csp>=4.0,<5",
"whitenoise>=6.6,<7",
] ]
[project.optional-dependencies] [project.optional-dependencies]
+80 -52
View File
@@ -1,6 +1,11 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Automates docs/install.md's manual steps: system user, virtualenv, .env, # Automates docs/install.md's manual steps: system user, virtualenv, .env,
# migrate/collectstatic, systemd units, and (optionally) nginx + TLS. # migrate/collectstatic, systemd units. Does NOT set up a reverse proxy —
# Evection always requires one in front of gunicorn (see docs/install.md),
# but which one (local nginx/Caddy/Traefik, or something like Nginx Proxy
# Manager on a different host entirely) is up to you, so this script just
# leaves gunicorn listening on EVECTION_GUNICORN_BIND for you to point it
# at.
# #
# Run as root, from inside the checkout you want to serve: # Run as root, from inside the checkout you want to serve:
# sudo ./scripts/install.sh # sudo ./scripts/install.sh
@@ -11,7 +16,7 @@
# Non-interactive use (e.g. scripted provisioning) — set these before # Non-interactive use (e.g. scripted provisioning) — set these before
# running, and the script won't prompt: # running, and the script won't prompt:
# EVECTION_ALLOWED_HOSTS, EVECTION_DATABASE_URL, EVECTION_REDIS_URL, # EVECTION_ALLOWED_HOSTS, EVECTION_DATABASE_URL, EVECTION_REDIS_URL,
# EVECTION_USER (defaults to "evection") # EVECTION_GUNICORN_BIND, EVECTION_USER (defaults to "evection")
set -euo pipefail set -euo pipefail
@@ -58,12 +63,46 @@ if ! "$PYTHON_BIN" -c 'import ensurepip' >/dev/null 2>&1; then
exit 1 exit 1
fi fi
for bin in redis-cli nginx; do # Celery (and therefore all feed fetching) is entirely dependent on Redis
if ! command -v "$bin" >/dev/null 2>&1; then # being reachable — it fails silently from the installer's point of view
echo "Warning: '$bin' not found on PATH. Install it before going live" \ # (the worker/beat processes just retry forever in the background), so this
"(see docs/install.md's prerequisites section: apt install redis-server nginx)." >&2 # has to be a hard failure here, not a warning that's easy to miss while
# every later step "succeeds" at installing an app that can never fetch a
# single feed. Check the actual REDIS_URL that will end up in .env (an
# explicit EVECTION_REDIS_URL override, if set — the default local Redis
# otherwise), not just whether some redis-cli happens to be on PATH.
REDIS_URL_TO_CHECK="${EVECTION_REDIS_URL:-redis://localhost:6379/0}"
check_redis() {
"$PYTHON_BIN" -c "
import sys, socket, urllib.parse
u = urllib.parse.urlparse('$REDIS_URL_TO_CHECK')
try:
with socket.create_connection((u.hostname or 'localhost', u.port or 6379), timeout=3):
pass
except OSError as e:
print(e, file=sys.stderr)
sys.exit(1)
"
}
REDIS_HOST="$("$PYTHON_BIN" -c "import urllib.parse; print(urllib.parse.urlparse('$REDIS_URL_TO_CHECK').hostname or 'localhost')")"
if ! check_redis; then
if [[ "$REDIS_HOST" =~ ^(localhost|127\.0\.0\.1|::1)$ ]] && command -v apt-get >/dev/null 2>&1; then
echo "==> Redis isn't reachable at $REDIS_URL_TO_CHECK — installing redis-server"
apt-get install -y redis-server
systemctl enable --now redis-server
if ! check_redis; then
echo "Redis still isn't reachable after installing redis-server —" \
"check: systemctl status redis-server" >&2
exit 1
fi
else
echo "Redis isn't reachable at $REDIS_URL_TO_CHECK. Evection's background" \
"feed-fetching depends on it entirely, so this has to be fixed before" \
"continuing — either start/install Redis there, or set" \
"EVECTION_REDIS_URL to point at wherever it actually runs." >&2
exit 1
fi
fi fi
done
# --- 2. System user ----------------------------------------------------- # --- 2. System user -----------------------------------------------------
@@ -138,15 +177,34 @@ if [ ! -f "$ENV_FILE" ]; then
fi fi
DATABASE_URL="${DATABASE_URL:-sqlite:///$APP_DIR/db.sqlite3}" DATABASE_URL="${DATABASE_URL:-sqlite:///$APP_DIR/db.sqlite3}"
GUNICORN_BIND="${EVECTION_GUNICORN_BIND:-}"
if [ -z "$GUNICORN_BIND" ] && [ -t 0 ]; then
echo
echo "Evection must run behind a reverse proxy — gunicorn needs an" \
"address:port that proxy can reach."
read -rp "Is the reverse proxy on this same machine, or a different one? [same/different] [same]: " PROXY_LOCATION
if [[ "${PROXY_LOCATION:-}" =~ ^[Dd] ]]; then
read -rp "Address:port for gunicorn to listen on (e.g. 0.0.0.0:8000, or this host's own IP:8000) [0.0.0.0:8000]: " GUNICORN_BIND
GUNICORN_BIND="${GUNICORN_BIND:-0.0.0.0:8000}"
echo " Remember to firewall this port to only accept connections" \
"from the reverse proxy's host."
else
read -rp "Address:port for gunicorn to listen on [127.0.0.1:8000]: " GUNICORN_BIND
GUNICORN_BIND="${GUNICORN_BIND:-127.0.0.1:8000}"
fi
fi
GUNICORN_BIND="${GUNICORN_BIND:-127.0.0.1:8000}"
cat > "$ENV_FILE" <<EOF cat > "$ENV_FILE" <<EOF
SECRET_KEY=$SECRET_KEY SECRET_KEY=$SECRET_KEY
DEBUG=False DEBUG=False
ALLOWED_HOSTS=$ALLOWED_HOSTS ALLOWED_HOSTS=$ALLOWED_HOSTS
DATABASE_URL=$DATABASE_URL DATABASE_URL=$DATABASE_URL
REDIS_URL=${EVECTION_REDIS_URL:-redis://localhost:6379/0} REDIS_URL=${EVECTION_REDIS_URL:-redis://localhost:6379/0}
EVECTION_GUNICORN_BIND=$GUNICORN_BIND
EVECTION_SIGNUP_ENABLED=False EVECTION_SIGNUP_ENABLED=False
EVECTION_MIN_FETCH_INTERVAL_MINUTES=15 EVECTION_MIN_FETCH_INTERVAL_MINUTES=15
EVECTION_PWA_CACHE_VERSION=v1 EVECTION_PWA_CACHE_VERSION=${LATEST_TAG:-v1}
EOF EOF
chown "$EVECTION_USER:$EVECTION_USER" "$ENV_FILE" chown "$EVECTION_USER:$EVECTION_USER" "$ENV_FILE"
chmod 600 "$ENV_FILE" chmod 600 "$ENV_FILE"
@@ -154,6 +212,11 @@ else
echo "==> $ENV_FILE already exists — leaving it alone" echo "==> $ENV_FILE already exists — leaving it alone"
fi fi
# Needed below for the final "gunicorn is listening on..." message even
# when $ENV_FILE already existed and the block above was skipped.
GUNICORN_BIND="$(grep -m1 '^EVECTION_GUNICORN_BIND=' "$ENV_FILE" | cut -d= -f2-)"
GUNICORN_BIND="${GUNICORN_BIND:-127.0.0.1:8000}"
# --- 5. Migrate + static files -------------------------------------------- # --- 5. Migrate + static files --------------------------------------------
echo "==> Running migrations" echo "==> Running migrations"
@@ -174,56 +237,21 @@ for unit in evection-gunicorn evection-celery-worker evection-celery-beat; do
"$APP_DIR/docs/deploy/${unit}.service" > "/etc/systemd/system/${unit}.service" "$APP_DIR/docs/deploy/${unit}.service" > "/etc/systemd/system/${unit}.service"
done done
systemctl daemon-reload systemctl daemon-reload
systemctl enable --now evection-gunicorn evection-celery-worker evection-celery-beat systemctl enable evection-gunicorn evection-celery-worker evection-celery-beat
# Not "enable --now": that only starts a service if it's stopped, so a
# re-run after a unit file changed (e.g. a new EVECTION_GUNICORN_BIND)
# would reload the daemon but leave the already-running process on its
# stale old ExecStart indefinitely. Always restart explicitly instead.
systemctl restart evection-gunicorn evection-celery-worker evection-celery-beat
echo echo
echo "==> Services installed. Checking status..." echo "==> Services installed. Checking status..."
systemctl --no-pager status evection-gunicorn evection-celery-worker evection-celery-beat \ systemctl --no-pager status evection-gunicorn evection-celery-worker evection-celery-beat \
|| echo "One or more services didn't start cleanly — see 'journalctl -u evection-gunicorn' etc." >&2 || echo "One or more services didn't start cleanly — see 'journalctl -u evection-gunicorn' etc." >&2
# --- 7. nginx (optional) ---------------------------------------------------
SET_UP_NGINX="${EVECTION_SETUP_NGINX:-}"
if [ -z "$SET_UP_NGINX" ] && [ -t 0 ]; then
echo
read -rp "Set up nginx and an HTTPS certificate now? [y/N]: " SET_UP_NGINX
fi
if [[ "${SET_UP_NGINX:-}" =~ ^[Yy]$ ]]; then
if ! command -v nginx >/dev/null 2>&1; then
echo "nginx isn't installed — install it first (apt install nginx), then" \
"run docs/install.md's nginx section manually." >&2
else
DOMAIN="${EVECTION_DOMAIN:-}"
if [ -z "$DOMAIN" ] && [ -t 0 ]; then
read -rp "Domain name for nginx/TLS (must match a hostname in ALLOWED_HOSTS): " DOMAIN
fi
if [ -z "$DOMAIN" ]; then
echo "No domain given — skipping nginx setup. Run docs/install.md's nginx section manually." >&2
else
sed -e "s/your-domain.example/$DOMAIN/g" \
-e "s#/opt/evection#$EVECTION_HOME#g" \
"$APP_DIR/docs/deploy/nginx.conf.example" > "/etc/nginx/sites-available/evection"
ln -sf /etc/nginx/sites-available/evection /etc/nginx/sites-enabled/evection
if nginx -t; then
systemctl reload nginx
echo "==> nginx configured for $DOMAIN (HTTP only so far)"
if command -v certbot >/dev/null 2>&1; then
certbot --nginx -d "$DOMAIN" || echo "certbot failed — run it manually: certbot --nginx -d $DOMAIN" >&2
else
echo "certbot isn't installed. Install it and run:" \
"apt install certbot python3-certbot-nginx && certbot --nginx -d $DOMAIN" >&2
fi
else
echo "nginx config test failed — not enabling it. Check /etc/nginx/sites-available/evection." >&2
fi
fi
fi
else
echo "==> Skipping nginx — see docs/install.md step 7 to set it up yourself."
fi
echo echo
echo "==> Done." echo "==> Done."
echo " Visit the site (once nginx/TLS is in place) — the setup wizard creates your admin account." echo " gunicorn is listening on $GUNICORN_BIND — point your reverse proxy at it"
echo " (see docs/install.md's \"Reverse proxy\" section) before visiting the site."
echo " Once that's in place, the setup wizard creates your admin account on first visit."
echo " To upgrade later: sudo ./scripts/upgrade.sh" echo " To upgrade later: sudo ./scripts/upgrade.sh"
+36 -5
View File
@@ -26,7 +26,6 @@ APP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
EVECTION_HOME="$(dirname "$APP_DIR")" EVECTION_HOME="$(dirname "$APP_DIR")"
VENV_DIR="$EVECTION_HOME/venv" VENV_DIR="$EVECTION_HOME/venv"
EVECTION_USER="${EVECTION_USER:-evection}" EVECTION_USER="${EVECTION_USER:-evection}"
SOCKET="/run/evection/gunicorn.sock"
# $APP_DIR is owned by $EVECTION_USER, but the git steps below run as # $APP_DIR is owned by $EVECTION_USER, but the git steps below run as
# root — without this, git refuses to touch a repo it doesn't own # root — without this, git refuses to touch a repo it doesn't own
@@ -73,6 +72,18 @@ echo "Deploying $LATEST_TAG (currently on: $CURRENT_REF)"
git checkout --quiet --detach "$LATEST_TAG" git checkout --quiet --detach "$LATEST_TAG"
chown -R "$EVECTION_USER:$EVECTION_USER" "$APP_DIR" chown -R "$EVECTION_USER:$EVECTION_USER" "$APP_DIR"
# Bump the PWA's cache-busting version to this release's tag on every
# deploy, not just ones that happened to touch static assets — a missed
# manual bump otherwise leaves installed PWAs serving a stale cached
# shell/pages indefinitely. Cheap even when nothing cacheable changed:
# it just makes the service worker open a fresh cache name next time it
# activates.
if grep -q '^EVECTION_PWA_CACHE_VERSION=' "$APP_DIR/.env"; then
sed -i "s/^EVECTION_PWA_CACHE_VERSION=.*/EVECTION_PWA_CACHE_VERSION=$LATEST_TAG/" "$APP_DIR/.env"
else
echo "EVECTION_PWA_CACHE_VERSION=$LATEST_TAG" >> "$APP_DIR/.env"
fi
echo "==> Installing updated dependencies" echo "==> Installing updated dependencies"
sudo -u "$EVECTION_USER" -H "$VENV_DIR/bin/pip" install --upgrade "$APP_DIR" sudo -u "$EVECTION_USER" -H "$VENV_DIR/bin/pip" install --upgrade "$APP_DIR"
@@ -99,7 +110,30 @@ fi
echo "==> Verifying" echo "==> Verifying"
sleep 2 sleep 2
if curl -sf --unix-socket "$SOCKET" http://localhost/health/ >/dev/null; then # gunicorn always listens on a plain TCP address (EVECTION_GUNICORN_BIND in
# .env, set by scripts/install.sh) rather than a Unix socket, since the
# reverse proxy in front of it may be on a different host entirely — see
# docs/install.md. 0.0.0.0 isn't a valid address to connect *to*, so check
# against localhost in that case; this health check only needs to prove
# gunicorn itself is answering on this host, not reach it the way the
# proxy does.
HEALTH_BIND="$(grep -m1 '^EVECTION_GUNICORN_BIND=' "$APP_DIR/.env" | cut -d= -f2-)"
HEALTH_BIND="${HEALTH_BIND:-127.0.0.1:8000}"
HEALTH_HOST="${HEALTH_BIND%:*}"
HEALTH_PORT="${HEALTH_BIND##*:}"
if [ "$HEALTH_HOST" = "0.0.0.0" ]; then
HEALTH_HOST="127.0.0.1"
fi
# Use the venv's own Python rather than curl — curl isn't guaranteed to be
# installed on a minimal host, but the venv obviously already is.
if "$VENV_DIR/bin/python" -c "
import sys, urllib.request
try:
urllib.request.urlopen('http://$HEALTH_HOST:$HEALTH_PORT/health/', timeout=5)
except Exception as e:
print(e, file=sys.stderr)
sys.exit(1)
"; then
echo "Health check OK" echo "Health check OK"
else else
echo "Health check FAILED — check: journalctl -u evection-gunicorn -n 50" >&2 echo "Health check FAILED — check: journalctl -u evection-gunicorn -n 50" >&2
@@ -109,7 +143,4 @@ systemctl status evection-gunicorn --no-pager -l | head -10
echo echo
echo "==> Deployed $LATEST_TAG." echo "==> Deployed $LATEST_TAG."
echo " If this release changed cached static assets (CSS/JS/icons), bump"
echo " EVECTION_PWA_CACHE_VERSION in $APP_DIR/.env and restart evection-gunicorn"
echo " again so installed PWAs drop their old cache instead of serving stale assets."
echo " journalctl -u evection-gunicorn -f to watch logs." echo " journalctl -u evection-gunicorn -f to watch logs."
+42 -1
View File
@@ -290,22 +290,63 @@ form p {
padding: 0 0.5rem; padding: 0 0.5rem;
} }
/* `summary`'s native disclosure triangle is a ::marker, positioned for
list-item/inline content — once the tag-name + unread-count row below
needed to be a flex box (block-level), the marker got pushed onto its
own line above instead of sitting beside it. Replacing it with a
plain ::before triangle lets `summary` itself be a flex row with that
triangle as a normal flex item, so it stays inline no matter what the
rest of the row contains. */
.sidebar-section summary { .sidebar-section summary {
display: flex;
align-items: center;
gap: 0.4rem;
padding: 0.4rem 0.75rem; padding: 0.4rem 0.75rem;
cursor: pointer; cursor: pointer;
color: var(--text-dim); color: var(--text-dim);
font-size: 0.75rem; font-size: 0.75rem;
text-transform: uppercase; text-transform: uppercase;
letter-spacing: 0.04em; letter-spacing: 0.04em;
list-style: none;
}
.sidebar-section summary::-webkit-details-marker {
display: none;
}
.sidebar-section summary::before {
content: "▶";
flex: 0 0 auto;
font-size: 0.6rem;
transition: transform 0.1s ease;
}
.sidebar-section[open] > summary::before {
transform: rotate(90deg);
}
.sidebar-tag-row {
display: flex;
align-items: center;
justify-content: space-between;
gap: 0.4rem;
flex: 1 1 auto;
min-width: 0;
} }
.sidebar-tag-link { .sidebar-tag-link {
display: inline-block;
padding: 0.15rem 0.4rem;
margin: -0.15rem -0.4rem;
border-radius: 0.375rem;
color: inherit; color: inherit;
text-decoration: none; text-decoration: none;
} }
.sidebar-tag-link.active { .sidebar-tag-link.active {
color: var(--accent); background: var(--bg-row-selected);
color: var(--text);
font-weight: 600;
} }
.sidebar-item-list { .sidebar-item-list {
+46
View File
@@ -0,0 +1,46 @@
(function () {
// Remembers which sidebar <details> sections (one per tag, plus
// "Untagged" — see templates/core/_sidebar.html's data-sidebar-key)
// the user has collapsed, so a full page navigation (this isn't an
// SPA — every click re-renders the sidebar server-side) doesn't keep
// snapping them back open. Per-browser via localStorage, same pattern
// as theme.js/theme-init.js — not synced across devices/accounts.
var STORAGE_KEY = "evection-sidebar-collapsed";
function readCollapsed() {
try {
var raw = localStorage.getItem(STORAGE_KEY);
return raw ? JSON.parse(raw) : [];
} catch (e) {
return [];
}
}
function writeCollapsed(keys) {
try {
localStorage.setItem(STORAGE_KEY, JSON.stringify(keys));
} catch (e) {
/* localStorage unavailable (private mode, etc.) — state just won't persist */
}
}
var collapsed = readCollapsed();
document.querySelectorAll(".sidebar-section[data-sidebar-key]").forEach(function (details) {
var key = details.dataset.sidebarKey;
if (collapsed.indexOf(key) !== -1) {
details.open = false;
}
details.addEventListener("toggle", function () {
var current = readCollapsed();
var index = current.indexOf(key);
if (details.open) {
if (index !== -1) current.splice(index, 1);
} else if (index === -1) {
current.push(key);
}
writeCollapsed(current);
});
});
})();
+4 -1
View File
@@ -53,7 +53,10 @@
{% endif %} {% endif %}
<script src="{% static 'js/stream.js' %}" defer></script> <script src="{% static 'js/stream.js' %}" defer></script>
{% if user.is_authenticated %}<script src="{% static 'js/theme.js' %}" defer></script>{% endif %} {% if user.is_authenticated %}
<script src="{% static 'js/theme.js' %}" defer></script>
<script src="{% static 'js/sidebar-state.js' %}" defer></script>
{% endif %}
<script src="{% static 'js/register-sw.js' %}" defer></script> <script src="{% static 'js/register-sw.js' %}" defer></script>
</body> </body>
</html> </html>
+5 -2
View File
@@ -31,10 +31,13 @@
</ul> </ul>
{% for entry in sidebar_tag_groups %} {% for entry in sidebar_tag_groups %}
<details class="sidebar-section" open> <details class="sidebar-section" data-sidebar-key="tag-{{ entry.tag.pk }}" open>
<summary> <summary>
<span class="sidebar-tag-row">
<a href="{% url 'reader:stream' %}?tag={{ entry.tag.pk }}" <a href="{% url 'reader:stream' %}?tag={{ entry.tag.pk }}"
class="sidebar-tag-link{% if request.GET.tag == entry.tag.pk|stringformat:'s' %} active{% endif %}">{{ entry.tag.name }}</a> class="sidebar-tag-link{% if request.GET.tag == entry.tag.pk|stringformat:'s' %} active{% endif %}">{{ entry.tag.name }}</a>
{% if entry.unread_count %}<span class="unread-count">{{ entry.unread_count }}</span>{% endif %}
</span>
</summary> </summary>
{% if entry.subscriptions %} {% if entry.subscriptions %}
<ul class="sidebar-item-list"> <ul class="sidebar-item-list">
@@ -54,7 +57,7 @@
{% endfor %} {% endfor %}
{% if sidebar_untagged %} {% if sidebar_untagged %}
<details class="sidebar-section" open> <details class="sidebar-section" data-sidebar-key="untagged" open>
<summary>Untagged</summary> <summary>Untagged</summary>
<ul class="sidebar-item-list"> <ul class="sidebar-item-list">
{% for subscription in sidebar_untagged %} {% for subscription in sidebar_untagged %}
+15 -1
View File
@@ -66,8 +66,23 @@ async function cacheFirstForAsset(request) {
return response; return response;
} }
const LOGOUT_PATH = "{% url 'accounts:logout' %}";
self.addEventListener("fetch", (event) => { self.addEventListener("fetch", (event) => {
const { request } = event; const { request } = event;
const url = new URL(request.url);
// Logging out ends this device's session, so any authenticated HTML
// sitting in PAGES_CACHE must not outlive it — otherwise a later
// offline navigation (shared device, different account logging in
// next) could still serve a previous user's cached article pages.
// Purging it synchronously with the logout request itself, rather
// than via a postMessage from the page, means there's no race with
// the navigation that follows.
if (request.method === "POST" && url.pathname === LOGOUT_PATH) {
event.respondWith(caches.delete(PAGES_CACHE).then(() => fetch(request)));
return;
}
// Mutations (mark-read, star, OPML upload, forms...) are always // Mutations (mark-read, star, OPML upload, forms...) are always
// network-only — never intercept anything but GET. // network-only — never intercept anything but GET.
@@ -78,7 +93,6 @@ self.addEventListener("fetch", (event) => {
return; return;
} }
const url = new URL(request.url);
if (url.pathname.startsWith("/static/")) { if (url.pathname.startsWith("/static/")) {
event.respondWith(cacheFirstForAsset(request)); event.respondWith(cacheFirstForAsset(request));
} }