Compare commits

..
Author SHA1 Message Date
ksmithandClaude Sonnet 5.5 fed5bcc49e Document the release tag format: v<year>.<month>.<day>, then .2, .3
The docs and upgrade.sh's hint showed zero-padded `date +%Y.%m.%d` and said
nothing about a second release in a day. Tags are unpadded
(v2026.11.1) with .2, .3, ... appended for each further release that day.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 12:17:49 -07:00
ksmithandClaude Sonnet 5.5 ac5e951749 Harden the login throttle and cache readiness aggregates (R2-S4, R2-C5; #36, #45)
S4: the throttle keyed on the first X-Forwarded-For entry, which a client
or a proxy that appends can choose, and disabled itself entirely when
Redis was down.

EVECTION_TRUSTED_PROXIES (IPs/CIDRs of your reverse proxy) makes it
strict: X-Forwarded-For is only believed when the TCP peer is a trusted
proxy, and the rightmost entry that isn't itself a trusted proxy — the
address the proxy actually saw — is used instead of the spoofable first
one. Unset keeps the previous behavior so existing installs don't
suddenly share one bucket behind a proxy, and `manage.py check --deploy`
warns (evection.W001) until it's set. During a Redis outage the throttle
now falls back to a bounded in-process limiter instead of switching off.

C5: /health/ready/ reuses its queue/feed aggregates for 15s
(EVECTION_HEALTH_CACHE_SECONDS) so a fast-polling monitor can't create
steady load; the database, Redis and poller probes stay live. Docs
cover both settings, restricting the endpoint at the proxy, and the
tombstone/purge behavior from the previous changes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 12:08:13 -07:00
ksmithandClaude Sonnet 5.5 aacb41d1d3 Purge the PWA page cache on every login/logout/expiry transition (R2-S3, #35)
The service worker cached authenticated pages in one origin-wide cache
and only purged it when the logout request itself passed through it —
not on session expiry, externally cleared cookies, or an account
switch, so a shared device could show the previous session's pages
offline.

Every HTML response now carries X-Evection-Auth: an opaque HMAC of the
user (or "anon"), via a small middleware. The service worker remembers
the marker its cached pages belong to and, whenever a navigation brings
a different one, deletes the pages cache before storing anything; "anon"
responses (the login page) are never cached. So the cache always holds
exactly one session's pages, and any transition the worker can observe
online clears it. (What no design can detect is a session ending while
the device stays offline.)

Verified in a real browser: pages cached after login; expiring the
session server-side (no logout request) and loading a page left the
cache deleted and the marker "anon"; logging in as a second user
afterward cached only that user's pages, none of the first user's.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 12:05:45 -07:00
ksmithandClaude Sonnet 5.5 91a4c360a4 Retry transient fetch failures promptly instead of waiting a poll interval (R2-T5, #41)
An unexpected exception while fetching was recorded and re-raised, then
the feed waited for its next scheduled poll — which under error backoff
can be hours — even for a locked SQLite file or a dropped database
connection that clears in seconds.

fetch_feed now retries OperationalError/InterfaceError up to three times
with doubling delays (30s, 60s, 120s), holding the feed's claim so the
poller can't also fetch it and without counting it against the feed.
Anything else, and a transient error after the retries are spent, keep
the existing path: record the error, back off, release the claim,
re-raise. Time limits are unchanged.

Verified with a real Celery worker against a real Redis broker, with
the fetch rigged to fail twice: retries fired after 1s and 2s (delays
shortened for the probe), the third attempt succeeded, and the feed
ended healthy, unclaimed and scheduled. (Celery's eager test mode
mishandles retries, so the unit tests mock retry() and assert the
countdowns, claim and error-count behavior instead.)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 12:04:01 -07:00
ksmithandClaude Sonnet 5.5 d6e310b127 Make the Postgres search migration online-safe (R2-T3, #39)
One migration did a full-table UPDATE, a blocking GIN build and the
trigger install together — long locks and a deploy that could outlast
its health-check window on a large archive.

Now two stages. 0002 only adds the nullable column and the trigger
(metadata-only; new and updated rows get a vector from then on). 0003
is non-atomic: it backfills in 5,000-id batches, each its own short
transaction and only touching rows still NULL, then builds the index
with CREATE INDEX CONCURRENTLY (dropping a stale INVALID one from a
failed earlier run first). Re-running is safe, and it's a cheap no-op on
an install that applied the earlier single-step version.

Measured on a real PostgreSQL 16 with 150,000 existing articles while a
second connection kept inserting/updating: the migration took ~20s, the
slowest concurrent write was 37ms (nothing blocked), no NULL vectors
remained, the index was valid, and unapply/re-apply worked. Full suite
passes on Postgres.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 12:02:20 -07:00
ksmithandClaude Sonnet 5.5 72bffbe97f Expire purge tombstones and time-box the retention purge (R2-T2, R2-T4; #38, #40)
Tombstones were kept forever, so enabling retention left a permanent row
per deleted article. They now expire after 365 days (cleaned up by the
same nightly task, even when retention is off); the documented price is
that an entry a feed still lists after a year, with no or a recent
publish date, can reappear.

The purge looped until every eligible row was gone with no run budget.
A run now works for at most four minutes, then queues a continuation, so
a huge first purge can't monopolise a worker or starve polling; hard
time limits back that up, and each chunk is its own transaction so an
interrupted run just resumes. Adds indexes on Article.fetched_at and
PurgedArticle.purged_at for the scans these do.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 12:00:52 -07:00
ksmithandClaude Sonnet 5.5 2e1c0e31a8 Pin feed connections to the address that was validated (R2-S2, #34)
The SSRF guard checked that a hostname resolved to public addresses,
then let requests resolve it again to connect — so a DNS answer that
changed in between (DNS rebinding) could point the connection at an
internal address.

Connections now go through a pinned adapter whose socket factory
resolves, checks and connects inside one function: whatever the name
resolves to at connect time is what gets validated and used. Only the
socket is replaced, so TLS still verifies against the original
hostname. The up-front validate_fetch_url check stays as defense in
depth. Tests cover a rebinding answer at connect time being refused and
a real loopback round trip with the original Host header preserved;
checked live: a real HTTPS feed fetches, 127.0.0.1/169.254.169.254/
localhost are blocked, and a wrong-hostname certificate is still
rejected.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 12:00:03 -07:00
ksmithandClaude Sonnet 5.5 10ba491e10 Make the manual install/upgrade docs match the hardened scripts (R2-C4, #44)
docs/install.md still showed 'git fetch --tags --force' and a bare
'pip install .', reintroducing the rewritten-tag and unpinned-dependency
risks the scripts closed. The manual steps now use a non-forced fetch
(with an explanation of what a clobber error means) and the hash-pinned
requirements.txt install followed by pip install --no-deps.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 11:59:03 -07:00
ksmithandClaude Sonnet 5.5 698940763f Bound OPML uploads in size, depth, count and broker payload (R2-S5, #37)
An OPML upload was read fully into memory, parsed into a whole tree,
walked recursively, and (for large imports) embedded in a Celery
message. Nothing bounded raw size, ZIP entries, nesting depth, outline
count or feed count.

Uploads are capped at 5 MB (checked on the reported size before reading)
and ZIPs at 50 entries, with the decompressed member bounded by what it
actually yields rather than only its header. Parsing now streams with
iterparse and aborts the moment a limit is crossed — 24 levels deep,
20,000 outlines, 5,000 feeds — so a hostile file never builds a huge or
deep tree; a 50,000-level nesting bomb is rejected cleanly. Limit errors
are shown to the user. Large imports go to the worker by cache reference
(deleted once read, expires after an hour) instead of as the whole
document inside the broker message.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 11:58:49 -07:00
ksmithandClaude Sonnet 5.5 41f99e9405 Allow only http(s) URLs from feeds and OPML, at storage and render (R2-S1, #33)
A feed's channel link, an entry's link and an OPML htmlUrl were stored
as given and rendered in href attributes (bulk writes skip model
validation), so a malicious feed could supply a javascript: or data:
link. CSP is a second layer, not the control.

apps.core.urlutils accepts only absolute http(s) URLs. Fetching stores
channel/entry links through it — relative links now resolve against the
feed's own URL instead of being stored as dead relative paths — OPML
import skips non-http(s) feed URLs and blanks unsafe htmlUrl, and the
add-feed form rejects non-http(s) (URLField also allows ftp). A
safe_http_url template filter guards the two places a feed-supplied URL
becomes a link (the article's "Original" link and the dashboard), so
values already in the database from before this fix are neutralized too.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 11:57:23 -07:00
ksmithandClaude Sonnet 5.5 85914f9107 Route every feed fetch through one atomic claim (R2-T1, #32)
Adding a feed and the admin retry queued fetch_feed directly without
claiming the feed, so the periodic poller (which sees a never-fetched
or just-reset feed as due) could claim and fetch it too: duplicate
fetches, and one task overwriting another's claim/ETag/next-fetch state.
The retry path also queued the backoff reset and the fetch as two
independent tasks that could run in either order.

claim_feed/enqueue_fetch in apps.feeds.tasks are now the only way a
fetch gets dispatched — the poller, add-feed and retry all use them, so
a feed can only have one fetch in flight. If queueing fails the claim is
released (the poller recovers it), and add-feed still saves the
subscription during a broker outage. The retry resets backoff
synchronously before claiming, and says so when a fetch is already
running.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 11:56:23 -07:00
ksmithandClaude Sonnet 5.5 0975a7634b Mark the browser tab with a red dot when new articles arrive (#31)
The in-page "new articles" banner is invisible from a background tab.
Every authenticated page now polls a small JSON endpoint
(reader:new_since, every 30s) and redraws the tab's favicon with a red
dot when unread, non-hidden articles from the user's feeds have
appeared since the page loaded. A reload (which is what clicking the
banner does) resets the baseline and the icon.

The endpoint is deliberately not scoped to the stream's current
filter/tag/feed, so the tab lights up for anything new whichever page
is open. Scope is the dot only: no desktop notifications, per the
request. Verified in a dev server enforcing the production CSP: the
icon swapped to the dotted version after a new article arrived and
reset on reload.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 10:18:29 -07:00
ksmithandClaude Sonnet 5.5 fd494a9c08 Make the readiness check catch fetching that is scheduled but failing (#27)
/health/ready/ proved beat and a worker were alive but not that
fetching worked: every feed could be failing, or the queue could be
hours behind, and it would still report ok.

apps.core.health adds three checks: Celery queue depth (real Redis list
length, over 1000 fails), a successful fetch within three poll intervals
(30 min minimum, with a grace period for a brand-new install), and the
share of active feeds failing (over half, once there are at least four).
The payload also reports queue depth, active/failing feed counts, and
the age of the last successful fetch. Verified against real Redis with
queued Celery tasks, and the suite on PostgreSQL. docs/install.md lists
each check.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 09:30:00 -07:00
ksmithandClaude Sonnet 5.5 f8cd1eb33e Stop retention from bringing deleted articles back (#29)
The nightly purge deleted articles, but a feed that still listed an
entry would re-insert it on its next poll as a brand-new, unread
article. For feeds with long archives, old items could reappear after
every retention run.

The purge now leaves a (feed, guid) tombstone per deleted article, and
ingestion skips tombstoned guids. With retention on, entries whose own
publish date is already outside the window are not ingested at all, so
a long archive isn't fetched only to be purged and tombstoned the same
night. Undated entries rely on the tombstone.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 09:30:00 -07:00
ksmithandClaude Sonnet 5.5 73dbee9400 Document running the test suite against PostgreSQL
The Postgres search backend's tests are skipped on SQLite, so they only
run when pointed at a real Postgres. Documents the throwaway-container
recipe used to verify the stored/GIN-indexed search vector migration.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 09:23:02 -07:00
ksmithandClaude Sonnet 5.5 6c6227af57 Treat an article with no publish date as published when we fetched it
Feeds that give no publish date left the row with no time at all and
grouped the article under "Undated". The stream's sort order already
fell back to fetched_at; the list row, its date-group header, and the
open article's header now use the same fallback. The article's own
published_at stays empty — nothing is invented in the database, so a
later refetch can't see a spurious change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 09:14:09 -07:00
ksmithandClaude Sonnet 5.5 c339d456e4 Show the date as well as the time on each stream row
Rows only showed a time of day, leaving the date to the group header
above. Each row now shows "Oct 31, 5:30 PM" in the viewer's local
timezone, with the year added only for articles from a previous year.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 09:07:36 -07:00
ksmithandClaude Sonnet 5.5 7afda7d0ba Fix inline article expand and back-to-stream scope in production
Clicking an article title opened the detail page instead of expanding
it inline, and "Back to stream" then reset to every feed. Cause: the
title (and the new-articles banner) used inline onclick= handlers,
which production's CSP (script-src 'self') blocks — dev sends no CSP,
so it worked there. The title link just fell through to its href.

Behavior moves to delegated listeners in static/js/stream.js (modified
clicks still open in a new tab). A test now fails if any template gains
an inline on*= handler. The stream also passes its filter/tag/feed
scope through the article link so the detail page's back link returns
to it. Verified in a dev server enforcing the same CSP: title click
expands and collapses in place, URL unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 09:02:51 -07:00
ksmithandClaude Sonnet 5.5 37e658c49a Add configurable article retention (C7)
Articles and full HTML bodies accumulated forever, growing the
database, search index, and backups without bound. Site settings gets
a "Retention days" field (0, the default, keeps everything); a nightly
beat task deletes articles fetched longer ago than that in bounded
chunks, always keeping anything a user starred or saved for later.
Read state and tags cascade with the article and the FTS triggers drop
search entries. The dashboard shows the article count and oldest
article's age, and docs/install.md covers retention, the readiness
check, and SQLite's WAL tuning.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 08:51:20 -07:00
ksmithandClaude Sonnet 5.5 e5f1277014 Batch feed ingestion, add a readiness check, fail open without Redis (C2, C8)
Ingestion did a get_or_create per entry. It now prefetches existing
articles for the poll in one query, bulk-inserts new ones and
bulk-updates changed ones (falling back to row-by-row only if a
concurrent fetch wins a race), and dispatches rule evaluation in
bounded chunks so a huge first fetch is several small tasks.

/health/ stays a DB-only liveness check (upgrade.sh gates on it right
after restart). New /health/ready/ also checks Redis and that
poll_due_feeds ticked within five minutes — proving beat and a worker
are both alive — and returns 503 with per-check detail otherwise.

The sidebar unread-count cache and login throttle now degrade instead
of erroring when Redis is unreachable: every authenticated page and
every login would otherwise 500 during a cache outage. Found while
verifying in a sandbox with no Redis.

Also includes the dashboard's oldest-article metric (core/views.py).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 08:51:20 -07:00
ksmithandClaude Sonnet 5.5 8d2900424a Use keyset pagination for the stream (C4)
The stream did a COUNT(*) and OFFSET slice on every request, ordered by
a non-unique (published_at, fetched_at) — deep pages got slower and
articles sharing a timestamp could repeat or vanish across pages.

It now orders by (Coalesce(published_at, fetched_at), id) and pages with
before/after cursors, fetching one extra row to know whether another
page exists instead of counting. Pagination links also keep the feed
scope, which the old links silently dropped.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 08:51:20 -07:00
ksmithandClaude Sonnet 5.5 70b0d4f277 Index Postgres search and harden SQLite search/concurrency (C5, C6)
Postgres recomputed a tsvector over every article's full text on every
search. A migration now adds a stored, GIN-indexed search_vector column
kept current by a trigger (so no ingest/update path can forget it), and
the backend reads that column. Untested here: no Postgres in the dev
sandbox, so the SQL is reviewed but not executed — worth one run of the
(skipped-on-SQLite) Postgres test module against a real instance.

SQLite search built one bound parameter per subscribed feed, so enough
feeds would exceed the variable limit, and the OperationalError was
swallowed into "no results". It now scopes through a subquery on
Subscription and logs failures. SQLite connections also get WAL mode,
a 20s busy timeout, and IMMEDIATE transactions so gunicorn, Celery, and
the FTS triggers stop tripping over each other.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 08:51:20 -07:00
ksmithandClaude Sonnet 5 cb0ce5a336 Batch rule/tag lookups and index FilterRule for the ingest query (C9)
FilterRule had no indexes beyond the default primary/foreign-key ones,
and apply_filter_rules_for_feed issued 2 queries per subscriber (its
enabled rules, its subscription tags) inside a loop over every
subscriber of the feed — both cost grew with subscriber/rule count on
every single feed poll.

Adds a composite index on (user, is_enabled, priority), matching the
exact filter+order shape every caller already uses (this task,
apps.rules.tasks' backlog re-apply). apply_filter_rules_for_feed now
fetches rules and subscription tags once for the whole feed's subscriber
set, grouped by user/subscription in Python, instead of once per
subscriber.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 16:09:36 -06:00
ksmithandClaude Sonnet 5 abf4cef55b Cache sidebar unread counts for a short TTL (C3)
apps.core.context_processors.sidebar runs on every authenticated page
render and called unread_counts_by_feed, a join/group-by over the
user's whole article corpus, every single time — cost grew with the
archive's size regardless of which page was being viewed.

unread_counts_by_feed now caches its {feed_id: count} result per user
for 30 seconds (apps.core.cache_utils). The reader views that change
the current user's own read state (toggle_read, mark_all_read, viewing
an article) invalidate it immediately so the sidebar never visibly lags
behind the action that just caused it; other paths (rule-driven state
changes during background ingestion) aren't explicitly invalidated and
simply self-heal within the TTL, which is an acceptable tradeoff
already made elsewhere in this app (the stream's own "new articles"
banner polls on the same 30s cadence).

Also adds a global cache.clear() autouse fixture (conftest.py) — the
test CACHES backend (LocMemCache) lives in process memory and survives
Django's per-test DB rollback, so a cached count keyed by a user's pk
could otherwise leak into an unrelated later test reusing that pk.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 16:09:28 -06:00
ksmithandClaude Sonnet 5 b33fe905d2 Push due-feed scheduling into an indexed DB query (C1)
poll_due_feeds loaded every active feed's timing fields and recomputed
each one's error-backoff due-ness in Python on every beat tick (every
1-2 minutes) — scheduler CPU and DB reads grew linearly with total feed
count regardless of how many were actually due.

Feed.next_fetch_at (indexed together with is_active) is now set by
fetch_feed itself right after every fetch attempt, success or error,
using the same backoff formula that used to run per-feed in
poll_due_feeds. The scheduler's query is now a direct indexed range
scan (next_fetch_at <= now), ordered so the most overdue feeds claim
first, and bounded to POLL_BATCH_SIZE so a large backlog spreads across
several ticks instead of dispatching everything at once. A migration
backfills next_fetch_at for existing feeds using the same formula.
reset_feed_backoff also clears next_fetch_at so a manually-reset feed
is immediately due again, matching its prior behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 16:09:18 -06:00
ksmithandClaude Sonnet 5 b84cfe6474 Document the Nginx Proxy Manager fix for S8's X-Forwarded-For spoofing
docs/install.md covered the raw-nginx case via nginx.conf.example, but
gave no concrete guidance for Nginx Proxy Manager specifically, whose
generated config also defaults to the appending (spoofable) form of
X-Forwarded-For — the same production deployment this review is based
on uses NPM. Documents the Advanced tab / Custom Nginx Configuration
override, why it takes effect ahead of NPM's own default header, and a
note for anyone with something in front of NPM itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 15:35:07 -06:00
ksmithandClaude Sonnet 5 ca03a5994b Stop nginx from forwarding a client-spoofable X-Forwarded-For (S8)
The example nginx config set X-Forwarded-For to $proxy_add_x_forwarded_for,
which APPENDS to whatever X-Forwarded-For a client already sent rather
than replacing it. A client could send its own "X-Forwarded-For:
1.2.3.4" and have it forwarded as the first (trusted) entry — exactly
what apps.accounts.views._client_address reads to key the login rate
limiter, letting an attacker spoof a different address on every request
to dodge the per-address throttle. Changed to $remote_addr, nginx's own
view of who actually connected, which overwrites rather than appends.

docs/install.md's "different host" proxy guidance also gets a concrete
ufw example instead of just "firewall that port" prose, explains why
that firewall rule isn't optional (gunicorn trusts whoever can reach it
directly as if they were the proxy), and documents an optional Unix
socket setup for the same-host case for anyone who wants gunicorn
unreachable by anything but the proxy, not just this nginx config.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 15:27:36 -06:00
ksmithandClaude Sonnet 5 cb5617edb8 Pin dependencies by hash and refuse to follow a rewritten release tag (S7)
pyproject.toml used broad version ranges with no lock file, so two
deployments of the same release could end up with different dependency
versions, and there was no check that a package actually matches what
was reviewed/tested — a compromised or substituted package on the index
could install silently.

requirements.txt and requirements-dev.txt (generated via `pip-compile
--generate-hashes`, see the new README section on regenerating them)
pin every dependency to an exact version with its hash. install.sh and
upgrade.sh now install from requirements.txt with --require-hashes,
then the app itself with --no-deps — a tampered or substituted package
fails the hash check instead of installing.

Both scripts also no longer `git fetch --tags --force`: a plain fetch
refuses to move a tag ref they already have onto a different commit,
so a rewritten/tampered release tag is reported and stops the
deploy/install instead of being silently followed into production.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 15:27:29 -06:00
ksmithandClaude Sonnet 5 03dd37e69d Notify the open stream when new articles arrive
The stream had no way to learn about articles the background poll
ingested while the page was open — only a manual reload or navigation
ever picked them up.

Adds a lightweight "N new articles — click to show" banner: the stream
records the highest article id in its current scope (filter/tag/feed)
when rendered, and an htmx poll every 30s checks reader:check_new for
anything newer in that same scope. It never swaps the list out from
under you mid-read — clicking the banner is what actually reloads it.
id, not published_at, is the cursor, since a feed can backdate an
entry's published date below everything currently on screen while it's
still genuinely new to this stream.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 15:04:55 -06:00
ksmithandClaude Sonnet 5 17e7f63e61 Update link/author/summary/published_at on refetch, not just title (T6)
A changed guid's article only ever had its title/content_html/
content_hash refreshed on a subsequent poll — legitimate feed edits to
an entry's link, author, summary, or publication date were silently
ignored forever, which can affect navigation, ordering, and rules that
key off those fields.

content_hash is now computed over every mutable field feedparser gives
us for an entry (title, link, author, summary, content_html,
published_at), and all of them are written together when the hash
changes. Per-user read/star state is untouched either way, same as
before.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 14:44:15 -06:00
ksmithandClaude Sonnet 5 0bfe9c8076 Enqueue the initial feed fetch instead of running it inline (T3)
add_feed ran fetch_and_ingest synchronously during the POST request, so
a slow or large feed tied up a Gunicorn worker for the duration of the
fetch (up to the 15s HTTP timeout, plus parse/ingest time) and a user
could repeat it with distinct URLs to reduce request capacity.

The subscription is now created synchronously as before, but the first
fetch is dispatched via fetch_feed.delay() — the same task the periodic
poll uses, so the save/error bookkeeping is unchanged. The user sees an
immediate redirect with a message that the fetch is in progress rather
than waiting on it in the request/response cycle.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 14:44:09 -06:00
ksmithandClaude Sonnet 5 7bb981b545 Rebuild stream date-group headers in the viewer's local timezone
The server still groups articles into "Received today/yesterday/<date>"
headers using UTC, so an article published near UTC midnight could land
in a mismatched day group. Each article row now ships its raw UTC
instant (data-published-utc); static/js/date-groups.js discards the
server-rendered headers on load and rebuilds them using the browser's
own local calendar day, same approach as the article-time localization
this follows up on.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 14:36:27 -06:00
ksmithandClaude Sonnet 5 fd68400822 Render article/feed times in the viewer's local timezone, client-side
Everything is stored and server-rendered in UTC (config.settings.base's
TIME_ZONE), so article times, the article detail header, and the
dashboard's "last fetched" column were always showing UTC regardless of
where the viewer actually is.

Article/feed timestamps now render as <time datetime="..."> elements
carrying the UTC instant plus a UTC-formatted fallback as their initial
text; static/js/localize-times.js reformats them using the browser's
own timezone via toLocaleString(), both on page load and after every
htmx swap (pagination, the inline article expand, the dashboard's retry
action all inject new <time> elements without a full page load).

No server-side change needed — this doesn't touch the (currently
unused) per-user timezone field, and works correctly even if the
viewer travels or changes their system timezone, with no settings page
required. Verified live in the browser against a UTC-stored timestamp
rendering correctly shifted to the sandbox's local zone.

Known related issue, out of scope here: the stream's "Received
today/yesterday" date-group headers (apps.reader.views._group_by_date)
are still computed server-side via timezone.localdate(), which is also
effectively UTC — an article just before/after UTC midnight could land
in a date group that doesn't match its now-correctly-localized
time-of-day. Flagged for the user rather than fixed, since correcting
it properly would mean moving date-grouping to the client too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 13:31:07 -06:00
ksmithandClaude Sonnet 5 90837b339c Make the Site settings admin page go straight to the edit form
It's a true singleton, not a collection, so the normal changelist (an
"Add" link, a list with 0 or 1 rows, an extra click to get to the one
row) was just friction — and visually inconsistent with every other
app's admin section, which do show an "Add" link. The changelist view
now redirects straight to the single row's change form (creating it
first if this is a fresh install), and "Add" is never offered at all.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 13:23:47 -06:00
ksmithandClaude Sonnet 5 dc8b9ff83b Make the feed-poll interval site-wide instead of per-feed
Replaces Feed.fetch_interval_minutes with a single admin-editable
SiteSettings.fetch_interval_minutes (apps.core), following the same
singleton-row pattern as accounts.SetupLock. Per-feed error backoff is
unchanged — only the configured baseline interval is now shared across
every feed, loaded once per poll_due_feeds run rather than queried per
feed. EVECTION_MIN_FETCH_INTERVAL_MINUTES keeps working as the floor
underneath it.

SiteSettings is registered in Django admin as a singleton: "Add" is
only offered when no row exists yet, and delete is always refused.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:59:47 -06:00
ksmithandClaude Sonnet 5 a1ea2e5662 Add login rate limiting (S6)
Django's default LoginView had no throttle — an internet-facing
instance was exposed to unlimited password-guessing/credential-
stuffing attempts. apps.accounts.views.LoginView adds two independent,
Redis-cache-backed throttles (shared across all gunicorn workers,
unlike the per-process default LocMemCache): by client address (catches
one attacker spraying many usernames) and by the submitted username
(catches a distributed attack on one specific account). Either blocks
further attempts for a 5-minute window; both are generous enough not
to lock out a user who's just mistyped their own password a few times.

Client address prefers X-Forwarded-For's first hop, falling back to
REMOTE_ADDR — the same reverse-proxy trust boundary config.settings.prod
already relies on for X-Forwarded-Proto (gunicorn is never meant to be
reachable except through that proxy).

CACHES now points at Redis (already a hard install requirement for
Celery) instead of Django's implicit per-process default, since the
whole point is a shared counter across every gunicorn worker; test
settings override it back to LocMemCache so the suite doesn't need a
real Redis instance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:46:49 -06:00
ksmithandClaude Sonnet 5 9241a54958 Cap entries per poll and stored content size (T4)
ssrf_guard.MAX_RESPONSE_BYTES (S1) already bounds the raw download, but
a feed within that byte cap can still carry an unreasonable number of
entries, or one entry with an unreasonably large body — both cost real
CPU/DB time (nh3 sanitization, an Article row/UPDATE) per entry.
fetch_and_ingest now processes at most MAX_ENTRIES_PER_POLL (200)
entries per poll (logging a warning when truncated) and sanitize_html
bounds both its input (sanitizer cost) and output (what's actually
stored/rendered) length.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:46:40 -06:00
ksmithandClaude Sonnet 5 64f25789f5 Isolate unexpected fetch failures and harden task durability (T2)
fetch_feed had no time limit and no handling for anything beyond the
network/parse errors fetch_and_ingest already catches internally — an
unexpected exception (a parser/sanitizer bug, a DB error) crashed the
task with no error recorded on the Feed, the claim left held for the
full CLAIM_STALE_MINUTES, and no visibility beyond a bare traceback.
fetch_feed now catches that, records it the same way a handled failure
would be (reusing fetcher.record_fetch_error, promoted from a private
helper), clears the claim, and re-raises so Celery still marks the task
FAILED. soft_time_limit/time_limit bound a single fetch that somehow
outruns FETCH_TIMEOUT_SECONDS + ssrf_guard's response cap during
parsing/ingestion.

Also: a worker process killed mid-task (OOM, deploy restart, crash)
previously lost that task silently — Celery's default is to ack a
message the moment a worker *starts* it. CELERY_TASK_ACKS_LATE +
CELERY_TASK_REJECT_ON_WORKER_LOST (config/settings/base.py) make the
message redeliverable instead, and CELERY_WORKER_PREFETCH_MULTIPLIER=1
keeps a worker from grabbing a batch of tasks it could lose all at once.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:46:29 -06:00
ksmithandClaude Sonnet 5 0466ccc334 Fix sidebar disclosure triangle rendering above the tag name
summary's native ::marker triangle is positioned for inline content —
once the tag-name + unread-count row needed to be a flex box (for
v2026.10.31-15's tag unread counts), the marker got pushed onto its
own line above instead of sitting beside it, confirmed via a
screenshot after deploying that release.

Replaces the native marker with a plain ::before triangle (hiding
-webkit-details-marker too) so `summary` itself can safely be the flex
row, with the triangle as a normal flex item that stays inline no
matter what the rest of the row contains. Verified in the browser,
including that collapse/expand still works and the triangle still
rotates on open.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:39:55 -06:00
ksmithandClaude Sonnet 5 e760eeda1d Show each tag's total unread count and remember collapsed sections
Closes gitea#24: the sidebar's per-feed unread counts were already
computed by sidebar() (apps.core.context_processors); each tag group's
total is now the sum of its own feeds' counts (a feed can carry more
than one tag, so this reuses the same per-feed data rather than a
second query) and renders next to the tag name like a feed's own
count.

Closes gitea#25: each sidebar <details> section (one per tag, plus
"Untagged") now carries a stable data-sidebar-key, and
static/js/sidebar-state.js remembers which ones are collapsed in
localStorage — this isn't a SPA, so every click re-renders the sidebar
server-side, and previously every section reset back to expanded on
the next page load. Per-browser, same pattern as theme.js/theme-init.js,
not synced across devices/accounts.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:34:57 -06:00
ksmithandClaude Sonnet 5 9439f9cc59 Add SSRF protection to feed fetching: validate scheme/address, bound redirects and response size (S1)
fetch_and_ingest fetched a feed_url supplied straight from the "add
feed" form or an uploaded OPML file, with requests following redirects
on its own and no check that the target (or a redirect's target) was
actually a public, fetchable feed rather than an internal service,
localhost, or a cloud metadata endpoint — reachable from wherever this
app runs, on any authenticated user's behalf.

apps.feeds.services.ssrf_guard.safe_get() replaces the direct
requests.get() call: it resolves and validates the hostname (rejecting
anything that isn't a globally routable address) before the initial
request and before following each redirect hop manually, caps the
number of hops, and bounds the response body size by streaming it
instead of buffering it unbounded. OPML-imported feeds go through the
same check the first time poll_due_feeds fetches them, since this is
the one chokepoint every feed URL passes through regardless of which
path created the Feed row.

Known residual gap, documented in ssrf_guard.py: this validates then
connects rather than pinning the TCP connection to the exact address it
validated, so a narrow DNS-rebinding race isn't closed. Fixing that
needs a custom transport and felt like more machinery than this
self-hosted app's threat model currently justifies.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:25:20 -06:00
ksmithandClaude Sonnet 5 fd8eeb605e Scope backlog re-apply per-feed and always queue it through Celery (T5, S5)
Completes the T5 fix: reapply_rules_for_user_backlog now computes each
feed's subscription-tag ids up front and passes them to
evaluate_rules_for_article per article, so a feed/tag-scoped rule from
a different feed can't fire on this backlog (see the previous commit's
rule_in_scope() in apps.rules.engine).

Also completes S5's second half: the "Re-apply" view now always
dispatches reapply_rules_for_user.delay() instead of calling the
backlog function inline from the request — a large backlog or a
regex that outruns the new timeout guard no longer risks tying up a
web worker.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:20:25 -06:00
ksmithandClaude Sonnet 5 91c0b1dd26 Centralize rule feed/tag scoping in the engine and fix backlog re-apply (T5)
reapply_rules_for_user_backlog narrowed its *backlog* to the triggering
rule's feed/tag scope, but then loaded every enabled rule for the user
and ran all of them against that backlog with no per-rule scope check
— so re-applying one feed- or tag-scoped rule could also fire an
unrelated rule scoped to a different feed/tag, if it happened to match.

rule_in_scope() moves the feed/tag scope check into the engine itself
(apply_filter_rules_for_feed already computed this per-subscription via
query filters; it now just passes the tag ids and lets the engine
decide), so backlog re-apply and ingest-time dispatch can't drift into
scoping rules differently again.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:19:56 -06:00
ksmithandClaude Sonnet 5 f2ee793cf8 upgrade.sh: auto-bump EVECTION_PWA_CACHE_VERSION on every deploy
The release-notes reminder to manually bump this and restart gunicorn
was easy to miss or forget, leaving installed PWAs on a stale cached
shell/pages after an upgrade that changed static assets. upgrade.sh
now sets it to the deployed tag itself before restarting services, so
there's no manual step and no cache-invalidation window to miss.
install.sh does the same on first install for consistency.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:08:05 -06:00
ksmithandClaude Sonnet 5 b7d9ee672c Add an atomic per-feed claim to poll_due_feeds (T1)
poll_due_feeds scanned for due feeds and dispatched fetch_feed without
marking them claimed, so overlapping beat ticks or a slow-running
fetch could dispatch the same feed twice, racing on ETag/status/error-
count writes. Feed.claimed_at is now set via an UPDATE ... WHERE
claimed_at IS NULL OR claimed_at < stale-cutoff, so only one caller's
UPDATE can ever match a given row; fetch_feed clears it when done.
A claim older than CLAIM_STALE_MINUTES is treated as abandoned so a
worker dying mid-fetch doesn't wedge the feed forever.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:03:18 -06:00
ksmithandClaude Sonnet 5 d7b433e580 Purge the service worker's authenticated-page cache on logout (S3)
PAGES_CACHE held every cached navigation response (including
authenticated article pages) with no logout-time purge, so a later
offline navigation on a shared or account-switched device could still
serve another user's cached pages. Logout is a plain POST, so the
fetch handler now special-cases that request: it deletes PAGES_CACHE
and lets the real logout request through, with no race against the
navigation that follows since it's synchronous with the same request.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:03:10 -06:00
ksmithandClaude Sonnet 5 b52ba2be8a Make first-run setup race-safe with an atomic SetupLock claim (S4)
The setup view's User.objects.exists() check and the admin account's
save() weren't in the same transaction, so two concurrent requests
during first deployment could both pass the check and both create an
admin account. SetupLock is a singleton row (pk=1); claiming it and
creating the admin now happen in one atomic block, so only the first
of any racing requests can succeed — the rest hit IntegrityError and
are sent to the login page instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:03:05 -06:00
ksmithandClaude Sonnet 5 56eb2bc498 Fail closed in production if SECRET_KEY is missing or the placeholder default (S2)
base.py's env.str() fallback is a publicly known string in this repo's
history. Running prod settings with it (an unset or forgotten .env
value) would sign sessions/CSRF tokens with a key anyone can read.
install.sh already generates a real random key for fresh installs, so
this only blocks the broken case.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:02:58 -06:00
ksmithandClaude Sonnet 5 9d91da6fc6 Fix /health/ 400ing when hit directly over loopback (as upgrade.sh does)
scripts/upgrade.sh's health check connects straight to gunicorn's bind
address, bypassing the reverse proxy entirely, so the request's Host
header is "127.0.0.1:<port>" and it has no X-Forwarded-Proto header.
Against prod settings that's exactly a DisallowedHost (ALLOWED_HOSTS only
has the public domain) — which Django turns into a 400 — and would have
then hit a redirect-to-HTTPS loop from SECURE_SSL_REDIRECT right after,
since there's no TLS listener on that port to redirect to. Allow the
loopback addresses in ALLOWED_HOSTS (harmless: gunicorn's bind address is
never reachable from outside this host per docs/install.md) and exempt
/health/ from the HTTPS redirect specifically.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:19:27 -06:00
ksmithandClaude Sonnet 5 c316f411f8 Make the active-tag sidebar highlight as visible as the active-feed one
.sidebar-tag-link.active only changed text color, which is easy to miss
next to .sidebar-feed-link.active's background pill + bold weight. Give
the tag link the same background-pill treatment (padding/margin offset
so the box doesn't shift layout when inactive).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:17:45 -06:00
ksmithandClaude Sonnet 5 e1907ff587 install.sh: fail hard (or auto-install) when Redis isn't actually reachable
The old check only looked for redis-cli on PATH and printed a warning if
missing — it never tested an actual connection, so a host with no Redis
installed at all still sailed through the entire install looking
successful. The real failure only showed up later as every feed stuck on
"pending" forever, with the worker/beat units quietly retrying their
connection in the background rather than surfacing anything to whoever
ran the script. Actually connect to the REDIS_URL that will end up in
.env; on Debian/Ubuntu with the default local address, install and start
redis-server automatically; otherwise fail the install outright with a
clear message instead of a warning that's easy to miss.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:12:41 -06:00
ksmithandClaude Sonnet 5 b0d62c91f8 upgrade.sh: use the venv's Python for the health check, not curl
curl isn't installed by default on this install's host, and nothing else
in install.sh/upgrade.sh actually requires it — the venv's own Python
(which obviously exists by the time this check runs) can make the same
HTTP request via urllib, so drop the curl dependency entirely rather than
adding it to the prerequisites.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:08:48 -06:00
ksmithandClaude Sonnet 5 2068df471f install.sh: actually restart services when their unit files change
systemctl enable --now only starts a service if it's stopped — on a
re-run where a unit file's ExecStart changed (e.g. picking up a new
EVECTION_GUNICORN_BIND), the already-running process just kept running
on its old command line indefinitely. This is exactly what happened
switching gunicorn from a Unix socket to TCP: the new unit file was
written and daemon-reload'd, but gunicorn itself never restarted, so it
kept listening on the old socket. Restart explicitly instead of relying
on --now.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:05:46 -06:00
ksmithandClaude Sonnet 5 21c33e8188 Make the reverse proxy a required, pluggable part of every deployment
gunicorn previously only ever bound to a Unix socket, which assumes the
reverse proxy is nginx running on this same host — it can't be reached by
a proxy on a different machine at all (e.g. Nginx Proxy Manager on its own
box), which is this project's actual deployment target. Switch gunicorn to
listening on a plain TCP address instead, controlled by a new
EVECTION_GUNICORN_BIND setting in .env.

install.sh now asks whether the proxy is local or remote and picks a
sensible bind default either way (127.0.0.1:8000 vs 0.0.0.0:8000, with a
reminder to firewall the port in the remote case), and no longer tries to
automate nginx/certbot itself — docs/install.md's new "Reverse proxy"
section covers both shapes instead, since which proxy and where it runs is
the operator's choice, not something this script should assume or set up.
upgrade.sh's health check now curls the TCP bind instead of the old socket
path.

Also add WhiteNoise so gunicorn serves /static/ itself: the old approach
(nginx aliasing the staticfiles directory on disk) silently breaks the
moment the proxy isn't on this host and can't see this filesystem.
nginx.conf.example's own /static/ alias becomes an optional same-host perf
optimization rather than a requirement.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 09:01:46 -06:00
ksmithandClaude Sonnet 5 62702e13e3 Detect and recreate a half-created venv, not just a missing one
The idempotency check only looked for the venv directory's existence, so
a venv left behind by an earlier failed run (e.g. missing ensurepip, as
just hit) — present but with no bin/pip — made every later re-run skip
venv creation entirely and then fail on a pip that was never created.
Check for bin/pip specifically and wipe+recreate if it's missing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 08:52:36 -06:00
ksmithandClaude Sonnet 5 78611c0eae Check for a working venv module before install.sh does anything else
On Debian/Ubuntu, python3-venv (ensurepip support) is a separate package
from python3 itself, and apt installing just python3 leaves venv creation
failing partway through install.sh — after the system user, chown, and
git checkout have all already happened. Check for it up front alongside
the Python version check, and document the apt package in install.md's
prerequisites.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 08:51:28 -06:00
ksmithandClaude Sonnet 5 50d2ba9404 Mark the app checkout as a safe.directory for root's git commands
install.sh chowns $APP_DIR to the service user, but its own (and
upgrade.sh's) git fetch/checkout steps run as root per the previous fix —
git refuses that combination by default ("detected dubious ownership"),
which is exactly what happens re-running install.sh after an earlier run
got far enough to chown the checkout. Add the directory to root's
safe.directory list before touching it with git.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 08:49:37 -06:00
99 changed files with 6779 additions and 361 deletions

No files matched your search

+6
View File
@@ -23,3 +23,9 @@ EVECTION_MIN_FETCH_INTERVAL_MINUTES=15
# Bump this on deploys that change cached static assets to invalidate old # Bump this on deploys that change cached static assets to invalidate old
# service worker caches. # service worker caches.
EVECTION_PWA_CACHE_VERSION=v1 EVECTION_PWA_CACHE_VERSION=v1
# IP address(es) or CIDR(s) of the reverse proxy in front of gunicorn,
# comma-separated, e.g. 192.168.1.10 or 10.0.0.0/24. When set, the login
# throttle trusts X-Forwarded-For only from these addresses. Strongly
# recommended; `manage.py check --deploy` warns until it's set.
# EVECTION_TRUSTED_PROXIES=
+25
View File
@@ -33,6 +33,19 @@ python manage.py runserver
Visit `http://localhost:8000/` — you'll land on the first-run setup wizard to Visit `http://localhost:8000/` — you'll land on the first-run setup wizard to
create the instance's admin account. create the instance's admin account.
### Updating pinned dependencies
`requirements.txt`/`requirements-dev.txt` are hash-pinned lock files generated
from `pyproject.toml` — production installs (`scripts/install.sh`,
`scripts/upgrade.sh`) install from `requirements.txt` with `--require-hashes`
rather than re-resolving version ranges at install time. After changing a
dependency in `pyproject.toml`, regenerate both and commit them alongside:
```bash
pip-compile --generate-hashes --allow-unsafe --output-file=requirements.txt pyproject.toml
pip-compile --generate-hashes --allow-unsafe --extra=dev --output-file=requirements-dev.txt pyproject.toml
```
## Tests ## Tests
```bash ```bash
@@ -41,6 +54,18 @@ ruff check .
black --check . black --check .
``` ```
### Testing against PostgreSQL
The suite runs on SQLite by default; the Postgres-only tests (search backend)
are skipped there. To run everything against a real Postgres:
```bash
docker run -d --name evection-pg-test -e POSTGRES_PASSWORD=testpw -e POSTGRES_DB=evection \
-p 127.0.0.1:55432:5432 postgres:16
DATABASE_URL=postgres://postgres:testpw@127.0.0.1:55432/evection pytest
docker rm -f evection-pg-test
```
## License ## License
[GNU AGPL-3.0-or-later](LICENSE) — if you run a modified version of Evection as [GNU AGPL-3.0-or-later](LICENSE) — if you run a modified version of Evection as
+3
View File
@@ -4,3 +4,6 @@ from django.apps import AppConfig
class AccountsConfig(AppConfig): class AccountsConfig(AppConfig):
default_auto_field = "django.db.models.BigAutoField" default_auto_field = "django.db.models.BigAutoField"
name = "apps.accounts" name = "apps.accounts"
def ready(self):
from . import checks # noqa: F401 (registers the deploy check)
+20
View File
@@ -0,0 +1,20 @@
from django.conf import settings
from django.core.checks import Tags, Warning, register
@register(Tags.security, deploy=True)
def trusted_proxies_configured(app_configs, **kwargs):
if settings.DEBUG or settings.EVECTION_TRUSTED_PROXIES:
return []
return [
Warning(
"EVECTION_TRUSTED_PROXIES is not set.",
hint=(
"The login throttle then trusts the first X-Forwarded-For entry from "
"anyone, which only holds while gunicorn is reachable by nothing but a "
"proxy that overwrites that header. Set EVECTION_TRUSTED_PROXIES to your "
"reverse proxy's IP/CIDR in .env (see docs/install.md)."
),
id="evection.W001",
)
]
+1
View File
@@ -45,4 +45,5 @@ def _setup_exempt_paths():
reverse("pwa:service_worker"), reverse("pwa:service_worker"),
reverse("pwa:offline"), reverse("pwa:offline"),
reverse("core:health"), reverse("core:health"),
reverse("core:health_ready"),
} }
@@ -0,0 +1,22 @@
# Generated by Django 5.2.18 on 2026-10-31 18:00
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("accounts", "0001_initial"),
]
operations = [
migrations.CreateModel(
name="SetupLock",
fields=[
(
"id",
models.PositiveSmallIntegerField(default=1, primary_key=True, serialize=False),
),
],
),
]
+12
View File
@@ -5,3 +5,15 @@ from django.db import models
class User(AbstractUser): class User(AbstractUser):
timezone = models.CharField(max_length=64, default="UTC") timezone = models.CharField(max_length=64, default="UTC")
created_at = models.DateTimeField(auto_now_add=True) created_at = models.DateTimeField(auto_now_add=True)
class SetupLock(models.Model):
"""Singleton row that gates first-run setup against a race between
concurrent requests. The setup view always tries to INSERT the row
with pk=1 in the same transaction as creating the first admin user:
the database's primary-key uniqueness makes exactly one of those
INSERTs win even under concurrent requests, so only one admin account
can ever be created this way, instead of the User.objects.exists()
check alone (which has a window between the check and the save)."""
id = models.PositiveSmallIntegerField(primary_key=True, default=1)
+217
View File
@@ -0,0 +1,217 @@
import pytest
from django.contrib.auth import get_user_model
from django.core.cache import cache
from django.urls import reverse
from .views import (
LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_ADDRESS,
LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_USERNAME,
)
User = get_user_model()
@pytest.fixture(autouse=True)
def _clear_rate_limit_cache():
cache.clear()
yield
cache.clear()
@pytest.fixture
def alice(db):
return User.objects.create_user("alice", password="pw-alice-123")
def _bad_login(client, username="alice"):
return client.post(reverse("accounts:login"), {"username": username, "password": "wrong"})
@pytest.mark.django_db
def test_a_single_failed_login_still_shows_the_normal_error(client, alice):
response = _bad_login(client)
assert response.status_code == 200
assert b"Please enter a correct" in response.content or response.context["form"].errors
@pytest.mark.django_db
def test_correct_credentials_succeed_before_the_limit_is_hit(client, alice):
for _ in range(3):
_bad_login(client)
response = client.post(
reverse("accounts:login"), {"username": "alice", "password": "pw-alice-123"}
)
assert response.status_code == 302
@pytest.mark.django_db
def test_too_many_failed_attempts_from_one_address_blocks_further_tries(client, alice):
for _ in range(LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_ADDRESS):
_bad_login(client)
# Even the correct password is now refused — the throttle runs
# before the form is ever validated.
response = client.post(
reverse("accounts:login"), {"username": "alice", "password": "pw-alice-123"}
)
assert response.status_code == 302
assert response.url == reverse("accounts:login")
assert not response.wsgi_request.user.is_authenticated
@pytest.mark.django_db
def test_too_many_failed_attempts_against_one_username_blocks_that_username(client, alice):
bob = User.objects.create_user("bob", password="pw-bob-123")
for _ in range(LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_USERNAME):
_bad_login(client, username="alice")
# Same client address, different username — the per-username
# throttle for "alice" shouldn't affect bob logging in.
response = client.post(reverse("accounts:login"), {"username": "bob", "password": "pw-bob-123"})
assert response.status_code == 302
assert response.wsgi_request.user == bob
@pytest.fixture
def cache_down(monkeypatch):
from unittest.mock import Mock
broken = Mock(side_effect=ConnectionError("redis down"))
for name in ("get", "incr", "set"):
monkeypatch.setattr(f"apps.accounts.throttle.cache.{name}", broken)
@pytest.fixture(autouse=True)
def _reset_local_limiter():
from .views import login_throttle
login_throttle._local.clear()
yield
login_throttle._local.clear()
@pytest.mark.django_db
def test_login_still_works_when_the_throttle_cache_is_down(client, alice, cache_down):
assert _bad_login(client).status_code == 200
good = client.post(reverse("accounts:login"), {"username": "alice", "password": "pw-alice-123"})
assert good.status_code == 302
@pytest.mark.django_db
def test_a_redis_outage_degrades_to_a_local_limit_instead_of_no_limit(client, alice, cache_down):
"""R2-S4: previously every attempt was allowed while the cache was
down, so an outage switched brute-force protection off entirely."""
for _ in range(LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_USERNAME):
_bad_login(client)
blocked = client.post(
reverse("accounts:login"), {"username": "alice", "password": "pw-alice-123"}
)
assert blocked.status_code == 302
assert blocked.url == reverse("accounts:login"), "even the right password is refused"
class TestLocalLimiter:
def test_counts_per_key_and_expires(self, monkeypatch):
import time as time_module
from .throttle import LoginThrottle
throttle = LoginThrottle(window_seconds=60)
for _ in range(3):
throttle._local_record("k")
assert throttle._local_count("k") == 3
assert throttle._local_count("other") == 0
real = time_module.monotonic()
monkeypatch.setattr("apps.accounts.throttle.time.monotonic", lambda: real + 61)
assert throttle._local_count("k") == 0
def test_memory_is_bounded(self, monkeypatch):
from . import throttle as throttle_module
monkeypatch.setattr(throttle_module, "LOCAL_MAX_KEYS", 5)
throttle = throttle_module.LoginThrottle(window_seconds=60)
for i in range(50):
throttle._local_record(f"k{i}")
assert len(throttle._local) <= 5
class TestClientAddress:
def _request(self, remote, forwarded=None):
from django.test import RequestFactory
extra = {"REMOTE_ADDR": remote}
if forwarded is not None:
extra["HTTP_X_FORWARDED_FOR"] = forwarded
return RequestFactory().get("/", **extra)
def test_unset_keeps_the_legacy_first_hop_behavior(self, settings):
from .throttle import client_address
settings.EVECTION_TRUSTED_PROXIES = []
assert client_address(self._request("10.0.0.5", "1.2.3.4, 10.0.0.5")) == "1.2.3.4"
assert client_address(self._request("10.0.0.5")) == "10.0.0.5"
def test_trusted_proxy_uses_the_rightmost_untrusted_hop_not_the_spoofable_first(self, settings):
import ipaddress
from .throttle import client_address
settings.EVECTION_TRUSTED_PROXIES = [ipaddress.ip_network("10.0.0.5/32")]
# The client sent "X-Forwarded-For: 6.6.6.6"; the proxy appended the
# address it actually saw (203.0.113.9).
request = self._request("10.0.0.5", "6.6.6.6, 203.0.113.9")
assert client_address(request) == "203.0.113.9"
def test_forwarded_for_from_an_untrusted_peer_is_ignored(self, settings):
import ipaddress
from .throttle import client_address
settings.EVECTION_TRUSTED_PROXIES = [ipaddress.ip_network("10.0.0.5/32")]
# Someone reaching gunicorn directly and claiming to be 1.2.3.4.
assert client_address(self._request("198.51.100.7", "1.2.3.4")) == "198.51.100.7"
def test_chained_trusted_proxies_are_skipped(self, settings):
import ipaddress
from .throttle import client_address
settings.EVECTION_TRUSTED_PROXIES = [ipaddress.ip_network("10.0.0.0/24")]
request = self._request("10.0.0.5", "203.0.113.9, 10.0.0.7")
assert client_address(request) == "203.0.113.9"
def test_trusted_peer_with_no_forwarded_header_falls_back_to_the_peer(self, settings):
import ipaddress
from .throttle import client_address
settings.EVECTION_TRUSTED_PROXIES = [ipaddress.ip_network("10.0.0.5/32")]
assert client_address(self._request("10.0.0.5")) == "10.0.0.5"
class TestDeployCheck:
def test_warns_when_trusted_proxies_is_unset_in_production(self, settings):
from .checks import trusted_proxies_configured
settings.DEBUG = False
settings.EVECTION_TRUSTED_PROXIES = []
warnings = trusted_proxies_configured(None)
assert [w.id for w in warnings] == ["evection.W001"]
def test_quiet_when_configured_or_in_debug(self, settings):
import ipaddress
from .checks import trusted_proxies_configured
settings.DEBUG = False
settings.EVECTION_TRUSTED_PROXIES = [ipaddress.ip_network("10.0.0.5/32")]
assert trusted_proxies_configured(None) == []
settings.DEBUG = True
settings.EVECTION_TRUSTED_PROXIES = []
assert trusted_proxies_configured(None) == []
+23 -1
View File
@@ -1,7 +1,7 @@
import pytest import pytest
from django.urls import reverse from django.urls import reverse
from .models import User from .models import SetupLock, User
# SetupWizardMiddleware's cache is reset around every test by the autouse # SetupWizardMiddleware's cache is reset around every test by the autouse
# fixture in the top-level conftest.py, not just here. # fixture in the top-level conftest.py, not just here.
@@ -43,6 +43,28 @@ def test_setup_creates_admin_and_logs_in(client):
assert stream_response.status_code == 200 assert stream_response.status_code == 200
@pytest.mark.django_db
def test_setup_refuses_a_second_concurrent_submission(client):
# Simulates two requests racing the setup form: the first claims the
# SetupLock row directly (standing in for a request that's already
# past the User.objects.exists() check), so a second submission's
# INSERT must lose instead of also creating an admin account.
SetupLock.objects.create()
response = client.post(
reverse("accounts:setup"),
{
"username": "raced",
"email": "raced@example.com",
"password": "correct-horse-battery",
"password_confirm": "correct-horse-battery",
},
)
assert response.status_code == 302
assert response.url == reverse("accounts:login")
assert not User.objects.filter(username="raced").exists()
@pytest.mark.django_db @pytest.mark.django_db
def test_setup_is_unreachable_once_a_user_exists(client): def test_setup_is_unreachable_once_a_user_exists(client):
User.objects.create_user("existing", password="irrelevant") User.objects.create_user("existing", password="irrelevant")
+106
View File
@@ -0,0 +1,106 @@
"""Login attempt throttling: counters live in the shared cache (Redis) so
every gunicorn worker sees them, with an in-process fallback when the
cache is unreachable (R2-S4) — so a Redis outage degrades throttling to
per-worker limits instead of switching it off for anyone who happens to
choose that moment."""
import ipaddress
import logging
import threading
import time
from django.conf import settings
from django.core.cache import cache
logger = logging.getLogger(__name__)
LOCAL_MAX_KEYS = 10_000
class LoginThrottle:
def __init__(self, window_seconds: int):
self.window = window_seconds
self._lock = threading.Lock()
self._local: dict[str, tuple[int, float]] = {}
def is_blocked(self, key: str, limit: int) -> bool:
try:
return cache.get(key, 0) >= limit
except Exception:
logger.warning("Login throttle cache unavailable; using local limiter")
return self._local_count(key) >= limit
def record(self, key: str) -> None:
try:
try:
cache.incr(key)
except ValueError:
cache.set(key, 1, self.window)
except Exception:
logger.warning("Login throttle cache unavailable; using local limiter")
self._local_record(key)
def _local_count(self, key: str) -> int:
with self._lock:
entry = self._local.get(key)
if entry is None or entry[1] <= time.monotonic():
return 0
return entry[0]
def _local_record(self, key: str) -> None:
now = time.monotonic()
with self._lock:
if len(self._local) >= LOCAL_MAX_KEYS:
self._local = {k: v for k, v in self._local.items() if v[1] > now}
if len(self._local) >= LOCAL_MAX_KEYS:
self._local.clear() # under a flood, bounded memory beats perfect memory
count, expires = self._local.get(key, (0, 0.0))
if expires <= now:
count, expires = 0, now + self.window
self._local[key] = (count + 1, expires)
def _parse_ip(value: str):
try:
return ipaddress.ip_address(value.strip())
except ValueError:
return None
def _is_trusted_proxy(value: str, trusted) -> bool:
ip = _parse_ip(value)
return ip is not None and any(ip in network for network in trusted)
def client_address(request) -> str:
"""The address to throttle a login attempt by.
EVECTION_TRUSTED_PROXIES (comma-separated IPs/CIDRs of your reverse
proxy) makes this strict: X-Forwarded-For is only believed when the
TCP peer is one of those proxies, and then the *rightmost* entry that
isn't itself a trusted proxy is used — the address your proxy actually
saw — rather than the first entry, which is whatever the client chose
to send. A client reaching gunicorn directly, or a proxy outside the
list, gets its own peer address.
Unset (the default), it keeps the previous behavior — trust the first
X-Forwarded-For entry — which is only safe while gunicorn is reachable
by nothing but a proxy that overwrites that header (docs/install.md);
`manage.py check --deploy` warns until it's set.
"""
remote = request.META.get("REMOTE_ADDR", "unknown")
forwarded = [
part.strip()
for part in request.META.get("HTTP_X_FORWARDED_FOR", "").split(",")
if part.strip()
]
trusted = settings.EVECTION_TRUSTED_PROXIES
if not trusted:
return forwarded[0] if forwarded else remote
if not _is_trusted_proxy(remote, trusted):
return remote
for address in reversed(forwarded):
if not _is_trusted_proxy(address, trusted):
return address
return forwarded[0] if forwarded else remote
+1 -5
View File
@@ -7,11 +7,7 @@ app_name = "accounts"
urlpatterns = [ urlpatterns = [
path("setup/", views.setup, name="setup"), path("setup/", views.setup, name="setup"),
path( path("login/", views.LoginView.as_view(), name="login"),
"login/",
auth_views.LoginView.as_view(template_name="accounts/login.html"),
name="login",
),
path("logout/", auth_views.LogoutView.as_view(), name="logout"), path("logout/", auth_views.LogoutView.as_view(), name="logout"),
path("users/", views.manage_users, name="manage_users"), path("users/", views.manage_users, name="manage_users"),
path("users/create/", views.create_user, name="create_user"), path("users/create/", views.create_user, name="create_user"),
+59 -1
View File
@@ -1,14 +1,62 @@
import logging
from django.contrib import messages from django.contrib import messages
from django.contrib.auth import login from django.contrib.auth import login
from django.contrib.auth.decorators import user_passes_test from django.contrib.auth.decorators import user_passes_test
from django.contrib.auth.views import LoginView as DjangoLoginView
from django.db import IntegrityError, transaction
from django.shortcuts import redirect, render from django.shortcuts import redirect, render
from .forms import CreateUserForm, SetupForm from .forms import CreateUserForm, SetupForm
from .middleware import SetupWizardMiddleware from .middleware import SetupWizardMiddleware
from .models import User from .models import SetupLock, User
from .throttle import LoginThrottle, client_address
logger = logging.getLogger(__name__)
is_admin = user_passes_test(lambda u: u.is_authenticated and u.is_staff) is_admin = user_passes_test(lambda u: u.is_authenticated and u.is_staff)
# Two independent throttles, either of which can block an attempt: by
# client address (catches one attacker spraying many usernames) and by
# the submitted username (catches a distributed/low-and-slow attack on
# one specific account from many addresses). Both are generous enough
# not to lock out a user who's just fat-fingered their own password a
# few times.
LOGIN_RATE_LIMIT_WINDOW_SECONDS = 5 * 60
LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_ADDRESS = 15
LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_USERNAME = 8
login_throttle = LoginThrottle(LOGIN_RATE_LIMIT_WINDOW_SECONDS)
def _attempts_key(kind: str, value: str) -> str:
return f"login-attempts:{kind}:{value}"
class LoginView(DjangoLoginView):
"""Wraps Django's own LoginView with the throttle above — everything
else (form, template, redirect behavior) is unchanged."""
template_name = "accounts/login.html"
def post(self, request, *args, **kwargs):
address_key = _attempts_key("addr", client_address(request))
username_key = _attempts_key("user", request.POST.get("username", "")[:150])
if login_throttle.is_blocked(
address_key, LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_ADDRESS
) or login_throttle.is_blocked(username_key, LOGIN_RATE_LIMIT_MAX_ATTEMPTS_PER_USERNAME):
messages.error(
request, "Too many login attempts. Please wait a few minutes and try again."
)
return redirect("accounts:login")
return super().post(request, *args, **kwargs)
def form_invalid(self, form):
login_throttle.record(_attempts_key("addr", client_address(self.request)))
login_throttle.record(_attempts_key("user", self.request.POST.get("username", "")[:150]))
return super().form_invalid(form)
def setup(request): def setup(request):
if User.objects.exists(): if User.objects.exists():
@@ -16,11 +64,21 @@ def setup(request):
form = SetupForm(request.POST or None) form = SetupForm(request.POST or None)
if request.method == "POST" and form.is_valid(): if request.method == "POST" and form.is_valid():
try:
with transaction.atomic():
# Claims the one-time setup slot: the first of any
# concurrent requests to reach this INSERT wins it, and
# every other one raises IntegrityError instead of also
# creating an admin account. See SetupLock's docstring.
SetupLock.objects.create()
user = form.save(commit=False) user = form.save(commit=False)
user.set_password(form.cleaned_data["password"]) user.set_password(form.cleaned_data["password"])
user.is_staff = True user.is_staff = True
user.is_superuser = True user.is_superuser = True
user.save() user.save()
except IntegrityError:
messages.info(request, "Setup was already completed. Log in with that account.")
return redirect("accounts:login")
SetupWizardMiddleware._a_user_exists = True SetupWizardMiddleware._a_user_exists = True
login(request, user) login(request, user)
messages.success(request, "Welcome to Evection! Your admin account is ready.") messages.success(request, "Welcome to Evection! Your admin account is ready.")
@@ -0,0 +1,27 @@
# Generated by Django 5.2.18 on 2026-11-01 16:28
import django.db.models.deletion
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('articles', '0004_starred_saved_as_system_tags'),
('feeds', '0006_backfill_feed_next_fetch_at'),
]
operations = [
migrations.CreateModel(
name='PurgedArticle',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('guid', models.CharField(max_length=1000)),
('purged_at', models.DateTimeField(auto_now_add=True)),
('feed', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='purged_articles', to='feeds.feed')),
],
options={
'unique_together': {('feed', 'guid')},
},
),
]
@@ -0,0 +1,22 @@
# Generated by Django 5.2.18 on 2026-11-01 19:00
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('articles', '0005_purgedarticle'),
('feeds', '0006_backfill_feed_next_fetch_at'),
]
operations = [
migrations.AddIndex(
model_name='article',
index=models.Index(fields=['fetched_at'], name='articles_ar_fetched_eb5501_idx'),
),
migrations.AddIndex(
model_name='purgedarticle',
index=models.Index(fields=['purged_at'], name='articles_pu_purged__27d417_idx'),
),
]
+23 -1
View File
@@ -25,13 +25,35 @@ class Article(models.Model):
class Meta: class Meta:
unique_together = [("feed", "guid")] unique_together = [("feed", "guid")]
indexes = [models.Index(fields=["feed", "published_at"])] indexes = [
models.Index(fields=["feed", "published_at"]),
# The retention purge scans for the oldest articles by this.
models.Index(fields=["fetched_at"]),
]
ordering = ["-published_at", "-fetched_at"] ordering = ["-published_at", "-fetched_at"]
def __str__(self): def __str__(self):
return self.title or self.guid return self.title or self.guid
class PurgedArticle(models.Model):
"""Tombstone left by the retention purge (apps.articles.tasks.
purge_old_articles) for every article it deletes: a feed that still
lists an entry after we purged it would otherwise be re-inserted as a
brand-new, unread article on its next poll. Ingestion skips any guid
that has a tombstone here. Only the (feed, guid) pair is kept — a
small fraction of the article it replaces.
"""
feed = models.ForeignKey(Feed, on_delete=models.CASCADE, related_name="purged_articles")
guid = models.CharField(max_length=1000)
purged_at = models.DateTimeField(auto_now_add=True)
class Meta:
unique_together = [("feed", "guid")]
indexes = [models.Index(fields=["purged_at"])]
class UserArticleState(models.Model): class UserArticleState(models.Model):
"""Per-user read/hide state for an Article. """Per-user read/hide state for an Article.
+33 -4
View File
@@ -1,7 +1,14 @@
import logging
from django.core.cache import cache
from django.db.models import Count from django.db.models import Count
from apps.core.cache_utils import SIDEBAR_UNREAD_CACHE_TTL_SECONDS, sidebar_unread_cache_key
from .models import SAVED_TAG_NAME, STARRED_TAG_NAME, Article, ArticleTag, UserArticleState from .models import SAVED_TAG_NAME, STARRED_TAG_NAME, Article, ArticleTag, UserArticleState
logger = logging.getLogger(__name__)
def with_user_state(articles, user): def with_user_state(articles, user):
"""Annotates an iterable of Article instances with is_read/is_starred/ """Annotates an iterable of Article instances with is_read/is_starred/
@@ -44,9 +51,26 @@ def with_user_state(articles, user):
def unread_counts_by_feed(user): def unread_counts_by_feed(user):
"""{feed_id: unread_count} for every feed the user subscribes to, in one """{feed_id: unread_count} for every feed the user subscribes to — used
query — used by the sidebar, so it must stay cheap regardless of how by the sidebar's context processor, which runs on every authenticated
many feeds/articles exist.""" page render, so the underlying join/group-by over the user's whole
article corpus is cached for a short TTL rather than recomputed on
every single page view (C3). apps.core.cache_utils.
invalidate_sidebar_unread_cache is called wherever a request changes
the current user's own read state, so this is rarely visibly stale.
"""
cache_key = sidebar_unread_cache_key(user.pk)
# The cache is an optimization here, never a dependency: every
# authenticated page render goes through this, so a Redis outage must
# degrade to recomputing the count, not to a 500 on every page.
try:
counts = cache.get(cache_key)
except Exception:
logger.warning("Sidebar cache unavailable; computing unread counts directly")
counts = None
if counts is not None:
return counts
rows = ( rows = (
Article.objects.filter(feed__subscriptions__user=user) Article.objects.filter(feed__subscriptions__user=user)
.exclude(user_states__user=user, user_states__is_read=True) .exclude(user_states__user=user, user_states__is_read=True)
@@ -54,4 +78,9 @@ def unread_counts_by_feed(user):
.values("feed_id") .values("feed_id")
.annotate(unread=Count("id")) .annotate(unread=Count("id"))
) )
return {row["feed_id"]: row["unread"] for row in rows} counts = {row["feed_id"]: row["unread"] for row in rows}
try:
cache.set(cache_key, counts, SIDEBAR_UNREAD_CACHE_TTL_SECONDS)
except Exception:
logger.warning("Sidebar cache unavailable; skipping cache write")
return counts
+130 -18
View File
@@ -1,11 +1,19 @@
from celery import shared_task import logging
from django.db.models import Q import time
from datetime import timedelta
from celery import shared_task
from django.db import transaction
from django.utils import timezone
from apps.core.models import SiteSettings
from apps.feeds.models import Subscription from apps.feeds.models import Subscription
from apps.rules.engine import evaluate_rules_for_article from apps.rules.engine import evaluate_rules_for_article
from apps.rules.models import FilterRule from apps.rules.models import FilterRule
from .models import Article, SubscriptionTag from .models import Article, ArticleTag, PurgedArticle, SubscriptionTag
logger = logging.getLogger(__name__)
@shared_task @shared_task
@@ -14,6 +22,14 @@ def apply_filter_rules_for_feed(feed_id: int, article_ids: list[int]):
FilterRules (scoped to this feed/tag or unscoped) against the given FilterRules (scoped to this feed/tag or unscoped) against the given
articles. Dispatched by the fetch pipeline after new/updated articles articles. Dispatched by the fetch pipeline after new/updated articles
are committed for a feed. are committed for a feed.
Per-rule scoping is left to evaluate_rules_for_article/rule_in_scope
rather than filtered into this query, so this and the backlog
re-apply task (apps.rules.tasks) can't scope rules differently.
Rules and subscription tags are fetched once for every subscriber of
this feed, not once per subscriber (C9) — a popular feed with many
subscribers previously issued 2 queries per subscriber here.
""" """
if not article_ids: if not article_ids:
return 0 return 0
@@ -22,28 +38,124 @@ def apply_filter_rules_for_feed(feed_id: int, article_ids: list[int]):
if not articles: if not articles:
return 0 return 0
subscriptions = Subscription.objects.filter(feed_id=feed_id).select_related("user") subscriptions = list(Subscription.objects.filter(feed_id=feed_id).select_related("user"))
if not subscriptions:
return 0
rules_by_user_id: dict[int, list[FilterRule]] = {}
for rule in FilterRule.objects.filter(
user_id__in={s.user_id for s in subscriptions}, is_enabled=True
).order_by("priority", "id"):
rules_by_user_id.setdefault(rule.user_id, []).append(rule)
tag_ids_by_subscription_id: dict[int, set[int]] = {}
for subscription_id, tag_id in SubscriptionTag.objects.filter(
subscription_id__in=[s.pk for s in subscriptions]
).values_list("subscription_id", "tag_id"):
tag_ids_by_subscription_id.setdefault(subscription_id, set()).add(tag_id)
evaluated = 0 evaluated = 0
for subscription in subscriptions: for subscription in subscriptions:
rules = FilterRule.objects.filter(user=subscription.user, is_enabled=True).filter( rules = rules_by_user_id.get(subscription.user_id)
Q(feed_id=feed_id) | Q(feed__isnull=True)
)
subscription_tag_ids = list(
SubscriptionTag.objects.filter(
user=subscription.user, subscription=subscription
).values_list("tag_id", flat=True)
)
if subscription_tag_ids:
rules = rules.filter(Q(tag_id__in=subscription_tag_ids) | Q(tag__isnull=True))
else:
rules = rules.filter(tag__isnull=True)
rules = list(rules.order_by("priority", "id"))
if not rules: if not rules:
continue continue
subscription_tag_ids = tag_ids_by_subscription_id.get(subscription.pk, set())
for article in articles: for article in articles:
evaluate_rules_for_article(subscription.user, article, rules) evaluate_rules_for_article(subscription.user, article, rules, subscription_tag_ids)
evaluated += 1 evaluated += 1
return evaluated return evaluated
PURGE_CHUNK_SIZE = 500
# One run works for at most this long, then queues itself again (R2-T4) so
# a huge first purge never holds a worker for hours or starves feed
# polling; the hard limits below are the backstop for a single stuck chunk.
PURGE_RUN_BUDGET_SECONDS = 240
PURGE_CONTINUATION_DELAY_SECONDS = 30
PURGE_SOFT_TIME_LIMIT = 540
PURGE_TIME_LIMIT = 600
# How long a purged article's tombstone is kept (R2-T2). It only needs to
# outlive the time a feed keeps *listing* the entry; after this, an entry a
# feed still publishes (and that has no publish date, or a recent one) can
# be ingested as new again — the price of the tombstone table not growing
# forever. Documented in docs/install.md.
TOMBSTONE_TTL_DAYS = 365
def _expire_tombstones(out_of_time) -> int:
cutoff = timezone.now() - timedelta(days=TOMBSTONE_TTL_DAYS)
removed = 0
while not out_of_time():
chunk = list(
PurgedArticle.objects.filter(purged_at__lt=cutoff).values_list("pk", flat=True)[
:PURGE_CHUNK_SIZE
]
)
if not chunk:
break
PurgedArticle.objects.filter(pk__in=chunk).delete()
removed += len(chunk)
return removed
@shared_task(time_limit=PURGE_TIME_LIMIT, soft_time_limit=PURGE_SOFT_TIME_LIMIT)
def purge_old_articles():
"""Deletes articles older than SiteSettings.retention_days (C7), in
chunks so one run never holds a huge delete in a single transaction,
and within a time budget — if it runs out it queues a continuation
(R2-T4). Anything a user starred or saved for later (a system
ArticleTag) is kept — those are explicit "keep this" signals.
UserArticleState and ArticleTag rows go with their article via ON
DELETE CASCADE, and the SQLite FTS triggers drop the search entries.
0 days means keep everything forever (the default).
Each deleted article leaves a PurgedArticle tombstone first, so a feed
that still lists the entry can't bring it back as a new, unread
article on its next poll (see apps.feeds.services.fetcher). Tombstones
older than TOMBSTONE_TTL_DAYS are removed here too, whether or not
retention is currently on (R2-T2). Returns the number of articles
deleted by this run.
"""
started = time.monotonic()
def out_of_time() -> bool:
return time.monotonic() - started > PURGE_RUN_BUDGET_SECONDS
expired = _expire_tombstones(out_of_time)
if expired:
logger.info("Removed %d expired purge tombstone(s)", expired)
retention_days = SiteSettings.load().retention_days
if retention_days <= 0:
return 0
cutoff = timezone.now() - timedelta(days=retention_days)
keep = ArticleTag.objects.filter(tag__is_system=True).values("article_id")
deleted_total = 0
while True:
chunk = list(
Article.objects.filter(fetched_at__lt=cutoff)
.exclude(pk__in=keep)
.values_list("pk", "feed_id", "guid")[:PURGE_CHUNK_SIZE]
)
if not chunk:
return deleted_total
with transaction.atomic():
PurgedArticle.objects.bulk_create(
[PurgedArticle(feed_id=feed_id, guid=guid) for _pk, feed_id, guid in chunk],
ignore_conflicts=True,
)
Article.objects.filter(pk__in=[pk for pk, _feed_id, _guid in chunk]).delete()
deleted_total += len(chunk)
if out_of_time():
logger.info(
"Retention purge hit its %ds budget after %d article(s); continuing shortly",
PURGE_RUN_BUDGET_SECONDS,
deleted_total,
)
purge_old_articles.apply_async(countdown=PURGE_CONTINUATION_DELAY_SECONDS)
return deleted_total
+76
View File
@@ -0,0 +1,76 @@
import pytest
from django.contrib.auth import get_user_model
from apps.core.cache_utils import invalidate_sidebar_unread_cache
from apps.feeds.models import Feed, Subscription
from .models import Article
from .queries import unread_counts_by_feed
User = get_user_model()
@pytest.fixture
def alice(db):
return User.objects.create_user("alice", password="pw-alice-123")
@pytest.fixture
def feed(db):
return Feed.objects.create(feed_url="https://example.com/feed.xml")
@pytest.mark.django_db
def test_unread_counts_by_feed_is_cached_across_calls(alice, feed):
"""unread_counts_by_feed (C3) caches its result — a second call right
after a new unread article appears must still see the stale cached
value until invalidated or the TTL elapses, proving it isn't
recomputing the query every time."""
Subscription.objects.create(user=alice, feed=feed)
Article.objects.create(feed=feed, guid="a-1")
first = unread_counts_by_feed(alice)
assert first == {feed.pk: 1}
Article.objects.create(feed=feed, guid="a-2")
second = unread_counts_by_feed(alice)
assert second == {feed.pk: 1}, "expected the cached count, not a fresh recount"
@pytest.mark.django_db
def test_invalidate_sidebar_unread_cache_forces_a_recount(alice, feed):
Subscription.objects.create(user=alice, feed=feed)
Article.objects.create(feed=feed, guid="a-1")
unread_counts_by_feed(alice)
Article.objects.create(feed=feed, guid="a-2")
invalidate_sidebar_unread_cache(alice.pk)
assert unread_counts_by_feed(alice) == {feed.pk: 2}
@pytest.mark.django_db
def test_unread_counts_by_feed_cache_is_per_user(alice, feed):
bob = User.objects.create_user("bob", password="pw-bob-123")
Subscription.objects.create(user=alice, feed=feed)
Subscription.objects.create(user=bob, feed=feed)
Article.objects.create(feed=feed, guid="a-1")
assert unread_counts_by_feed(alice) == {feed.pk: 1}
assert unread_counts_by_feed(bob) == {feed.pk: 1}
@pytest.mark.django_db
def test_unread_counts_still_work_when_the_cache_is_down(alice, feed, monkeypatch):
"""A Redis outage must degrade the sidebar to recomputing counts, not
500 every authenticated page."""
from unittest.mock import Mock
Subscription.objects.create(user=alice, feed=feed)
Article.objects.create(feed=feed, guid="a-1")
broken = Mock(side_effect=ConnectionError("redis down"))
monkeypatch.setattr("apps.articles.queries.cache.get", broken)
monkeypatch.setattr("apps.articles.queries.cache.set", broken)
assert unread_counts_by_feed(alice) == {feed.pk: 1}
invalidate_sidebar_unread_cache(alice.pk)
+152
View File
@@ -0,0 +1,152 @@
from datetime import timedelta
import pytest
from django.contrib.auth import get_user_model
from django.utils import timezone
from apps.core.models import SiteSettings
from apps.feeds.models import Feed
from .models import STARRED_TAG_NAME, Article, ArticleTag, UserArticleState, get_system_tag
from .tasks import purge_old_articles
User = get_user_model()
def _article(feed, guid, age_days):
article = Article.objects.create(feed=feed, guid=guid, title=guid)
Article.objects.filter(pk=article.pk).update(
fetched_at=timezone.now() - timedelta(days=age_days)
)
return article
@pytest.fixture
def feed(db):
return Feed.objects.create(feed_url="https://example.com/feed.xml")
@pytest.mark.django_db
def test_purge_is_a_noop_with_the_default_keep_forever_setting(feed):
_article(feed, "ancient", age_days=5000)
assert purge_old_articles() == 0
assert Article.objects.count() == 1
@pytest.mark.django_db
def test_purge_deletes_only_articles_older_than_the_retention_window(feed):
SiteSettings.objects.create(retention_days=30)
old = _article(feed, "old", age_days=45)
recent = _article(feed, "recent", age_days=5)
assert purge_old_articles() == 1
assert not Article.objects.filter(pk=old.pk).exists()
assert Article.objects.filter(pk=recent.pk).exists()
@pytest.mark.django_db
def test_purge_keeps_starred_articles_and_cascades_user_state(feed):
SiteSettings.objects.create(retention_days=30)
user = User.objects.create_user("alice", password="pw-alice-123")
starred = _article(feed, "starred-old", age_days=90)
ArticleTag.objects.create(
user=user, article=starred, tag=get_system_tag(user, STARRED_TAG_NAME)
)
doomed = _article(feed, "plain-old", age_days=90)
state = UserArticleState.objects.create(user=user, article=doomed, is_read=True)
purge_old_articles()
assert Article.objects.filter(pk=starred.pk).exists()
assert not Article.objects.filter(pk=doomed.pk).exists()
assert not UserArticleState.objects.filter(pk=state.pk).exists()
@pytest.mark.django_db
def test_purge_works_through_more_than_one_chunk(feed, monkeypatch):
monkeypatch.setattr("apps.articles.tasks.PURGE_CHUNK_SIZE", 2)
SiteSettings.objects.create(retention_days=30)
for i in range(5):
_article(feed, f"old-{i}", age_days=60)
assert purge_old_articles() == 5
assert Article.objects.count() == 0
@pytest.mark.django_db
def test_purge_leaves_a_tombstone_per_deleted_article(feed):
from .models import PurgedArticle
SiteSettings.objects.create(retention_days=30)
_article(feed, "gone-1", age_days=60)
_article(feed, "gone-2", age_days=60)
_article(feed, "kept", age_days=1)
purge_old_articles()
assert set(PurgedArticle.objects.filter(feed=feed).values_list("guid", flat=True)) == {
"gone-1",
"gone-2",
}
@pytest.mark.django_db
def test_expired_tombstones_are_removed_even_when_retention_is_off(feed):
from .models import PurgedArticle
from .tasks import TOMBSTONE_TTL_DAYS
old = PurgedArticle.objects.create(feed=feed, guid="old")
fresh = PurgedArticle.objects.create(feed=feed, guid="fresh")
PurgedArticle.objects.filter(pk=old.pk).update(
purged_at=timezone.now() - timedelta(days=TOMBSTONE_TTL_DAYS + 1)
)
assert SiteSettings.load().retention_days == 0
purge_old_articles()
assert not PurgedArticle.objects.filter(pk=old.pk).exists()
assert PurgedArticle.objects.filter(pk=fresh.pk).exists()
@pytest.mark.django_db
def test_a_run_that_exhausts_its_budget_queues_a_continuation(feed, monkeypatch):
"""R2-T4: a huge purge doesn't hold a worker for its whole length —
after the budget it does what it can and queues another run."""
from unittest.mock import patch
monkeypatch.setattr("apps.articles.tasks.PURGE_CHUNK_SIZE", 2)
monkeypatch.setattr("apps.articles.tasks.PURGE_RUN_BUDGET_SECONDS", -1) # already over
SiteSettings.objects.create(retention_days=30)
for i in range(5):
_article(feed, f"old-{i}", age_days=60)
with patch("apps.articles.tasks.purge_old_articles.apply_async") as mock_continue:
deleted = purge_old_articles()
assert deleted == 2, "one chunk, then it yields"
assert Article.objects.count() == 3
mock_continue.assert_called_once()
assert mock_continue.call_args.kwargs["countdown"] > 0
@pytest.mark.django_db
def test_continuation_runs_finish_the_job(feed, monkeypatch):
from unittest.mock import patch
monkeypatch.setattr("apps.articles.tasks.PURGE_CHUNK_SIZE", 2)
SiteSettings.objects.create(retention_days=30)
for i in range(5):
_article(feed, f"old-{i}", age_days=60)
total = 0
for run in range(5):
# Budget runs out after every chunk, as it would on a huge archive.
monkeypatch.setattr("apps.articles.tasks.PURGE_RUN_BUDGET_SECONDS", -1)
with patch("apps.articles.tasks.purge_old_articles.apply_async") as mock_continue:
total += purge_old_articles()
if not mock_continue.called:
break
assert total == 5
assert Article.objects.count() == 0
+37
View File
@@ -150,3 +150,40 @@ def test_disabled_rule_does_not_apply(feed, alice):
@pytest.mark.django_db @pytest.mark.django_db
def test_dispatch_with_empty_article_ids_is_a_noop(feed): def test_dispatch_with_empty_article_ids_is_a_noop(feed):
assert apply_filter_rules_for_feed(feed.id, []) == 0 assert apply_filter_rules_for_feed(feed.id, []) == 0
@pytest.mark.django_db
def test_dispatch_fetches_rules_and_tags_once_per_feed_not_once_per_subscriber(feed):
"""Rules/subscription tags are batched across every subscriber of the
feed in one query each (C9), not re-queried per subscriber — assert
this by checking the query count stays flat as subscribers grow."""
from django.db import connection
from django.test.utils import CaptureQueriesContext
users = [User.objects.create_user(f"user{i}", password="pw-123456") for i in range(5)]
for user in users:
sub = Subscription.objects.create(user=user, feed=feed)
tag = Tag.objects.create(user=user, name="Tech")
SubscriptionTag.objects.create(user=user, subscription=sub, tag=tag)
FilterRule.objects.create(
user=user,
tag=tag,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_STAR,
)
article = Article.objects.create(feed=feed, guid="guid-1", title="Python news")
with CaptureQueriesContext(connection) as ctx:
apply_filter_rules_for_feed(feed.id, [article.pk])
# Exactly one SELECT against each table regardless of subscriber
# count (5 here) — a regression back to per-subscription querying
# would show up as 5 of each instead of 1.
rule_selects = [q for q in ctx.captured_queries if "rules_filterrule" in q["sql"]]
tag_selects = [q for q in ctx.captured_queries if "articles_subscriptiontag" in q["sql"]]
assert len(rule_selects) == 1
assert len(tag_selects) == 1
for user in users:
assert is_starred(user, article) is True
+28 -1
View File
@@ -1 +1,28 @@
# Register your models here. from django.contrib import admin
from django.shortcuts import redirect
from django.urls import reverse
from .models import SiteSettings
@admin.register(SiteSettings)
class SiteSettingsAdmin(admin.ModelAdmin):
"""A true singleton, not a collection — browsing a changelist to pick
the one row and click into it is an extra, pointless step, so
"Core > Site settings" in the sidebar goes straight to the (only)
change form instead, creating the row first if this is a fresh
install. No changelist/add/delete views exist at all here.
"""
fields = ["fetch_interval_minutes", "retention_days"]
def has_add_permission(self, request):
return False
def has_delete_permission(self, request, obj=None):
return False
def changelist_view(self, request, extra_context=None):
settings_row = SiteSettings.load()
url = reverse("admin:core_sitesettings_change", args=[settings_row.pk])
return redirect(url)
+29
View File
@@ -0,0 +1,29 @@
import logging
from django.core.cache import cache
logger = logging.getLogger(__name__)
# Short enough that a missed invalidation point self-heals within one
# page load's worth of staleness; long enough to actually absorb repeat
# authenticated-page renders (apps.core.context_processors.sidebar runs
# on every one) without recomputing unread_counts_by_feed's join/group-by
# over the user's whole article corpus each time (C3).
SIDEBAR_UNREAD_CACHE_TTL_SECONDS = 30
def sidebar_unread_cache_key(user_id: int) -> str:
return f"sidebar-unread:{user_id}"
def invalidate_sidebar_unread_cache(user_id: int) -> None:
"""Call this wherever a request changes the current user's own
read/unread state (toggle_read, mark_all_read, viewing an article) so
the sidebar reflects it on the very next render instead of waiting
out the TTL above. Rule-driven state changes during background feed
ingestion aren't invalidated this way — those self-heal within the
TTL, same as a missed call site here would."""
try:
cache.delete(sidebar_unread_cache_key(user_id))
except Exception:
logger.warning("Sidebar cache unavailable; nothing to invalidate")
+13 -1
View File
@@ -44,7 +44,19 @@ def sidebar(request):
return { return {
"sidebar_tag_groups": [ "sidebar_tag_groups": [
{"tag": tag, "subscriptions": feeds_by_tag.get(tag.pk, [])} for tag in tags {
"tag": tag,
"subscriptions": feeds_by_tag.get(tag.pk, []),
# A feed can carry more than one tag, so this is a sum
# over whichever of the user's feeds this tag happens to
# cover right now — not a separately-tracked counter — and
# reuses the per-feed counts already computed above rather
# than a second query.
"unread_count": sum(
subscription.unread_count for subscription in feeds_by_tag.get(tag.pk, [])
),
}
for tag in tags
], ],
"sidebar_untagged": untagged, "sidebar_untagged": untagged,
"sidebar_starred_count": ArticleTag.objects.filter( "sidebar_starred_count": ArticleTag.objects.filter(
+111
View File
@@ -0,0 +1,111 @@
from datetime import timedelta
import redis
from django.conf import settings
from django.core.cache import cache
from django.db.models import Max, Min
from django.utils import timezone
from apps.feeds.models import Feed
from .models import SiteSettings
# More than this many tasks waiting means workers aren't keeping up (or
# aren't running) — fetches are piling up faster than they complete.
MAX_QUEUE_DEPTH = 1000
# Past this share of active feeds failing, it's far more likely something
# on our side (DNS, outbound network, a bad deploy) than every publisher
# breaking at once. Needs a handful of feeds first so one broken feed out
# of two doesn't read as an outage.
MAX_FAILING_RATIO = 0.5
MIN_FEEDS_FOR_RATIO = 4
MIN_SUCCESS_WINDOW_MINUTES = 30
def queue_depth() -> int | None:
"""Tasks waiting in Celery's default queue, or None if the broker
can't be reached. Celery's Redis transport keeps each queue as a Redis
list named after the queue."""
try:
client = redis.Redis.from_url(
settings.CELERY_BROKER_URL, socket_connect_timeout=2, socket_timeout=2
)
return int(client.llen("celery"))
except Exception:
return None
def _success_window() -> timedelta:
floor = getattr(settings, "EVECTION_MIN_FETCH_INTERVAL_MINUTES", 15)
base = max(SiteSettings.load().fetch_interval_minutes, floor)
return timedelta(minutes=max(3 * base, MIN_SUCCESS_WINDOW_MINUTES))
PIPELINE_CACHE_KEY = "health-ready-pipeline"
def cached_fetch_pipeline_status() -> tuple[dict[str, bool], dict]:
"""fetch_pipeline_status, reused for EVECTION_HEALTH_CACHE_SECONDS
(R2-C5): the queue/feed aggregates behind it cost real queries, and a
monitor that polls /health/ready/ every second shouldn't turn that
into recurring database and Redis load. The cheap liveness probes
(database, Redis, poll heartbeat) are not cached — those stay live.
A cache failure just means computing it fresh."""
ttl = settings.EVECTION_HEALTH_CACHE_SECONDS
if ttl > 0:
try:
cached = cache.get(PIPELINE_CACHE_KEY)
except Exception:
cached = None
if cached is not None:
return cached
result = fetch_pipeline_status()
if ttl > 0:
try:
cache.set(PIPELINE_CACHE_KEY, result, timeout=ttl)
except Exception:
pass
return result
def fetch_pipeline_status(now=None) -> tuple[dict[str, bool], dict]:
"""Checks that fetching is actually *succeeding*, not just scheduled
(the poll heartbeat only proves beat and a worker are alive): returns
({check: passed}, detail) covering queue depth, whether any feed has
been fetched successfully recently, and the share of feeds failing.
"""
now = now or timezone.now()
detail: dict = {}
depth = queue_depth()
detail["queue_depth"] = depth
checks = {"queue_depth": depth is not None and depth <= MAX_QUEUE_DEPTH}
active = Feed.objects.filter(is_active=True)
active_count = active.count()
failing = active.filter(fetch_error_count__gt=0).count()
detail["active_feeds"] = active_count
detail["failing_feeds"] = failing
window = _success_window()
last_success = active.filter(
last_fetch_status__in=[Feed.STATUS_OK, Feed.STATUS_NOT_MODIFIED]
).aggregate(latest=Max("last_fetched_at"))["latest"]
if last_success is not None:
detail["last_successful_fetch_age_seconds"] = int((now - last_success).total_seconds())
recent_success = now - last_success <= window
elif active_count == 0:
recent_success = True
else:
# Nothing has ever fetched successfully — only a problem once
# the oldest feed has had a full window to be fetched.
oldest = active.aggregate(oldest=Min("created_at"))["oldest"]
recent_success = now - oldest <= window
checks["recent_fetch_success"] = recent_success
checks["feed_failures"] = not (
active_count >= MIN_FEEDS_FOR_RATIO and failing / active_count > MAX_FAILING_RATIO
)
return checks, detail
+33
View File
@@ -0,0 +1,33 @@
# Generated by Django 5.2.18 on 2026-10-31 18:56
from django.db import migrations, models
class Migration(migrations.Migration):
initial = True
dependencies = []
operations = [
migrations.CreateModel(
name="SiteSettings",
fields=[
(
"id",
models.PositiveSmallIntegerField(default=1, primary_key=True, serialize=False),
),
(
"fetch_interval_minutes",
models.PositiveIntegerField(
default=60,
help_text="How often, in minutes, every feed is polled for new articles. Can't be set lower than EVECTION_MIN_FETCH_INTERVAL_MINUTES (see .env).",
),
),
],
options={
"verbose_name": "Site settings",
"verbose_name_plural": "Site settings",
},
),
]
@@ -0,0 +1,18 @@
# Generated by Django 5.2.18 on 2026-11-01 15:48
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0001_initial'),
]
operations = [
migrations.AddField(
model_name='sitesettings',
name='retention_days',
field=models.PositiveIntegerField(default=0, help_text="Articles older than this many days (by when they were fetched) are deleted daily, along with everyone's read state and tags for them — except articles anyone has starred or saved for later. 0 keeps everything forever."),
),
]
+45 -1
View File
@@ -1 +1,45 @@
# Create your models here. from django.db import models
class SiteSettings(models.Model):
"""Singleton row (pk=1, same pattern as accounts.SetupLock) holding
instance-wide configuration an operator reasonably wants to change
without touching the server or .env — editable via Django admin.
Currently just the feed-poll interval, which used to be a per-feed
field; it's site-wide now (see apps.feeds.tasks.poll_due_feeds).
"""
id = models.PositiveSmallIntegerField(primary_key=True, default=1)
fetch_interval_minutes = models.PositiveIntegerField(
default=60,
help_text=(
"How often, in minutes, every feed is polled for new articles. "
"Can't be set lower than EVECTION_MIN_FETCH_INTERVAL_MINUTES (see .env)."
),
)
retention_days = models.PositiveIntegerField(
default=0,
help_text=(
"Articles older than this many days (by when they were fetched) are "
"deleted daily, along with everyone's read state and tags for them — "
"except articles anyone has starred or saved for later. 0 keeps "
"everything forever."
),
)
class Meta:
verbose_name = "Site settings"
verbose_name_plural = "Site settings"
def __str__(self):
return "Site settings"
@classmethod
def load(cls):
"""Returns the singleton row, creating it with defaults on first
use so callers never need to special-case "no row exists yet"
(e.g. a fresh install whose first poll_due_feeds run hasn't
created it, and whose admin hasn't visited the settings page)."""
obj, _ = cls.objects.get_or_create(pk=1)
return obj
View File
Whitespace-only changes.
+12
View File
@@ -0,0 +1,12 @@
from django import template
from apps.core.urlutils import http_url_or_empty
register = template.Library()
@register.filter
def safe_http_url(value):
""" "" unless value is an absolute http(s) URL — use before putting any
feed-supplied URL in an href (see apps.core.urlutils)."""
return http_url_or_empty(value)
+82
View File
@@ -0,0 +1,82 @@
import pytest
from django.contrib.auth import get_user_model
from django.test import RequestFactory
from django.urls import reverse
from apps.articles.models import Article, SubscriptionTag, Tag
from apps.feeds.models import Feed, Subscription
from .context_processors import sidebar
User = get_user_model()
@pytest.fixture
def alice(db):
return User.objects.create_user("alice", password="pw-alice-123")
def _sidebar_context(user):
request = RequestFactory().get("/")
request.user = user
return sidebar(request)
@pytest.mark.django_db
def test_tag_unread_count_sums_its_feeds(alice):
feed_a = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
feed_b = Feed.objects.create(feed_url="https://b.example.com/feed.xml")
sub_a = Subscription.objects.create(user=alice, feed=feed_a)
sub_b = Subscription.objects.create(user=alice, feed=feed_b)
tag = Tag.objects.create(user=alice, name="work")
SubscriptionTag.objects.create(user=alice, subscription=sub_a, tag=tag)
SubscriptionTag.objects.create(user=alice, subscription=sub_b, tag=tag)
for i in range(3):
Article.objects.create(feed=feed_a, guid=f"a-{i}", title=f"A{i}")
for i in range(4):
Article.objects.create(feed=feed_b, guid=f"b-{i}", title=f"B{i}")
context = _sidebar_context(alice)
[group] = context["sidebar_tag_groups"]
assert group["tag"] == tag
assert group["unread_count"] == 7
@pytest.mark.django_db
def test_tag_unread_count_is_zero_with_no_unread_articles(alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
sub = Subscription.objects.create(user=alice, feed=feed)
tag = Tag.objects.create(user=alice, name="empty")
SubscriptionTag.objects.create(user=alice, subscription=sub, tag=tag)
context = _sidebar_context(alice)
[group] = context["sidebar_tag_groups"]
assert group["unread_count"] == 0
@pytest.mark.django_db
def test_sidebar_sections_carry_a_stable_collapse_state_key(client, alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
sub = Subscription.objects.create(user=alice, feed=feed)
tag = Tag.objects.create(user=alice, name="work")
SubscriptionTag.objects.create(user=alice, subscription=sub, tag=tag)
other_feed = Feed.objects.create(feed_url="https://other.example.com/feed.xml")
Subscription.objects.create(user=alice, feed=other_feed) # untagged
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
content = response.content.decode()
# static/js/sidebar-state.js reads these to restore/persist each
# section's collapsed state across page loads.
assert f'data-sidebar-key="tag-{tag.pk}"' in content
assert 'data-sidebar-key="untagged"' in content
@pytest.mark.django_db
def test_base_template_loads_the_sidebar_state_script(client, alice):
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
assert "sidebar-state.js" in response.content.decode()
+57
View File
@@ -0,0 +1,57 @@
import pytest
from django.contrib.auth import get_user_model
from django.urls import reverse
from .models import SiteSettings
User = get_user_model()
@pytest.mark.django_db
def test_load_creates_the_singleton_row_with_defaults():
assert not SiteSettings.objects.exists()
settings_row = SiteSettings.load()
assert settings_row.fetch_interval_minutes == 60
assert SiteSettings.objects.count() == 1
@pytest.mark.django_db
def test_load_returns_the_existing_row_without_creating_a_second_one():
SiteSettings.objects.create(fetch_interval_minutes=30)
settings_row = SiteSettings.load()
assert settings_row.fetch_interval_minutes == 30
assert SiteSettings.objects.count() == 1
@pytest.mark.django_db
def test_admin_changelist_redirects_straight_to_the_change_form(client):
assert not SiteSettings.objects.exists()
User.objects.create_superuser("admin", password="pw-admin-123")
client.login(username="admin", password="pw-admin-123")
changelist_url = reverse("admin:core_sitesettings_changelist")
response = client.get(changelist_url)
settings_row = SiteSettings.objects.get() # created by the redirect itself
assert response.status_code == 302
assert response.url == reverse("admin:core_sitesettings_change", args=[settings_row.pk])
@pytest.mark.django_db
def test_admin_never_offers_add_or_delete(client):
# ModelAdmin.has_add_permission/has_delete_permission returning False
# makes Django raise PermissionDenied rather than build a 403 response
# directly — the test client's default behavior re-raises that for
# debugging convenience instead of returning it as a response.
client.raise_request_exception = False
User.objects.create_superuser("admin", password="pw-admin-123")
client.login(username="admin", password="pw-admin-123")
response = client.get(reverse("admin:core_sitesettings_add"))
assert response.status_code == 403
settings_row = SiteSettings.load()
delete_url = reverse("admin:core_sitesettings_delete", args=[settings_row.pk])
response = client.get(delete_url)
assert response.status_code == 403
+44
View File
@@ -0,0 +1,44 @@
import pytest
from django.template import Context, Template
from .urlutils import http_url_or_empty, resolve_http_url
@pytest.mark.parametrize(
"value",
[
"javascript:alert(1)",
"JaVaScRiPt:alert(1)",
"java\tscript:alert(1)",
" javascript:alert(1)",
"data:text/html,<script>alert(1)</script>",
"vbscript:msgbox(1)",
"file:///etc/passwd",
"ftp://example.com/feed.xml",
"//example.com/no-scheme",
"/relative/path",
"http://",
"",
None,
],
)
def test_unsafe_or_non_absolute_urls_are_rejected(value):
assert http_url_or_empty(value) == ""
@pytest.mark.parametrize("value", ["http://example.com/a", "https://example.com/a?b=1#c"])
def test_http_and_https_urls_pass_through(value):
assert http_url_or_empty(value) == value
def test_relative_links_resolve_against_the_feed_url_and_are_still_checked():
base = "https://blog.example.com/feed.xml"
assert resolve_http_url(base, "/2026/post") == "https://blog.example.com/2026/post"
assert resolve_http_url(base, "javascript:alert(1)") == ""
assert resolve_http_url(base, "https://other.example.com/x") == "https://other.example.com/x"
def test_template_filter_blanks_unsafe_urls():
template = Template("{% load safe_urls %}[{{ url|safe_http_url }}]")
assert template.render(Context({"url": "javascript:alert(1)"})) == "[]"
assert template.render(Context({"url": "https://example.com/"})) == "[https://example.com/]"
+236 -6
View File
@@ -22,6 +22,61 @@ def test_health_check_works_before_first_run_setup(client):
assert response.status_code == 200 assert response.status_code == 200
@pytest.fixture
def broker_queue(monkeypatch):
"""Stands in for the Redis broker's queue length (tests have no Redis):
set `broker_queue["depth"]` to a number, or None for "unreachable"."""
state = {"depth": 0}
monkeypatch.setattr("apps.core.health.queue_depth", lambda: state["depth"])
return state
@pytest.mark.django_db
def test_health_ready_is_degraded_until_a_poll_tick_has_happened(client, broker_queue):
response = client.get(reverse("core:health_ready"))
assert response.status_code == 503
body = response.json()
assert body["status"] == "degraded"
assert body["checks"]["database"] is True
assert body["checks"]["redis"] is True
assert body["checks"]["poll_heartbeat"] is False
@pytest.mark.django_db
def test_health_ready_is_ok_after_poll_due_feeds_runs(client, broker_queue):
from apps.feeds.tasks import poll_due_feeds
poll_due_feeds()
response = client.get(reverse("core:health_ready"))
assert response.status_code == 200
checks = response.json()["checks"]
assert checks == {
"database": True,
"redis": True,
"poll_heartbeat": True,
"queue_depth": True,
"recent_fetch_success": True,
"feed_failures": True,
}
@pytest.mark.django_db
def test_health_ready_flags_a_stale_poll_heartbeat(client, broker_queue):
from datetime import timedelta
from django.core.cache import cache
from django.utils import timezone
from apps.feeds.tasks import POLL_HEARTBEAT_CACHE_KEY
cache.set(POLL_HEARTBEAT_CACHE_KEY, timezone.now() - timedelta(minutes=30))
response = client.get(reverse("core:health_ready"))
assert response.status_code == 503
assert response.json()["checks"]["poll_heartbeat"] is False
@pytest.mark.django_db @pytest.mark.django_db
def test_dashboard_requires_staff(client, django_user_model): def test_dashboard_requires_staff(client, django_user_model):
django_user_model.objects.create_user("regular", password="pw-regular-123") django_user_model.objects.create_user("regular", password="pw-regular-123")
@@ -58,7 +113,7 @@ def test_dashboard_lists_erroring_feeds(client, django_user_model):
@pytest.mark.django_db @pytest.mark.django_db
def test_retry_feed_resets_backoff_and_requeues(client, django_user_model): def test_retry_feed_resets_backoff_then_claims_and_queues_one_fetch(client, django_user_model):
django_user_model.objects.create_user("admin", password="pw-admin-123", is_staff=True) django_user_model.objects.create_user("admin", password="pw-admin-123", is_staff=True)
client.login(username="admin", password="pw-admin-123") client.login(username="admin", password="pw-admin-123")
@@ -68,12 +123,187 @@ def test_retry_feed_resets_backoff_and_requeues(client, django_user_model):
last_fetch_status=Feed.STATUS_ERROR, last_fetch_status=Feed.STATUS_ERROR,
) )
with ( with patch("apps.feeds.tasks.fetch_feed.delay") as mock_fetch:
patch("apps.core.views.reset_feed_backoff.delay") as mock_reset,
patch("apps.core.views.fetch_feed.delay") as mock_fetch,
):
response = client.post(reverse("core:retry_feed", args=[feed.id])) response = client.post(reverse("core:retry_feed", args=[feed.id]))
assert response.status_code == 302 assert response.status_code == 302
mock_reset.assert_called_once_with(feed.id)
mock_fetch.assert_called_once_with(feed.id) mock_fetch.assert_called_once_with(feed.id)
feed.refresh_from_db()
assert feed.fetch_error_count == 0
assert feed.claimed_at is not None, "the fetch is claimed so the poller can't also take it"
@pytest.mark.django_db
def test_retry_feed_does_not_queue_a_second_fetch_while_one_is_in_flight(client, django_user_model):
from django.utils import timezone
django_user_model.objects.create_user("admin", password="pw-admin-123", is_staff=True)
client.login(username="admin", password="pw-admin-123")
feed = Feed.objects.create(feed_url="https://busy.example.com/feed", claimed_at=timezone.now())
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_fetch:
client.post(reverse("core:retry_feed", args=[feed.id]))
mock_fetch.assert_not_called()
def _ready(client):
from django.core.cache import cache
from django.utils import timezone
from apps.feeds.tasks import POLL_HEARTBEAT_CACHE_KEY
# Record the heartbeat directly: running poll_due_feeds here would
# execute fetches eagerly (CELERY_TASK_ALWAYS_EAGER) and rewrite the
# feed states each test sets up.
cache.set(POLL_HEARTBEAT_CACHE_KEY, timezone.now())
response = client.get(reverse("core:health_ready"))
return response.status_code, response.json()
@pytest.mark.django_db
def test_health_ready_flags_a_backed_up_queue(client, broker_queue):
from .health import MAX_QUEUE_DEPTH
broker_queue["depth"] = MAX_QUEUE_DEPTH + 1
status, body = _ready(client)
assert status == 503
assert body["checks"]["queue_depth"] is False
assert body["queue_depth"] == MAX_QUEUE_DEPTH + 1
@pytest.mark.django_db
def test_health_ready_flags_an_unreachable_broker_queue(client, broker_queue):
broker_queue["depth"] = None
status, body = _ready(client)
assert status == 503
assert body["checks"]["queue_depth"] is False
@pytest.mark.django_db
def test_health_ready_flags_no_recent_successful_fetch(client, broker_queue):
from datetime import timedelta
from django.utils import timezone
feed = Feed.objects.create(
feed_url="https://a.example.com/feed.xml",
last_fetch_status=Feed.STATUS_OK,
last_fetched_at=timezone.now() - timedelta(days=2),
)
Feed.objects.filter(pk=feed.pk).update(created_at=timezone.now() - timedelta(days=30))
status, body = _ready(client)
assert status == 503
assert body["checks"]["recent_fetch_success"] is False
assert body["last_successful_fetch_age_seconds"] > 86400
@pytest.mark.django_db
def test_health_ready_is_ok_with_a_recent_successful_fetch(client, broker_queue):
from django.utils import timezone
Feed.objects.create(
feed_url="https://a.example.com/feed.xml",
last_fetch_status=Feed.STATUS_NOT_MODIFIED,
last_fetched_at=timezone.now(),
)
status, body = _ready(client)
assert status == 200
assert body["active_feeds"] == 1
assert body["failing_feeds"] == 0
@pytest.mark.django_db
def test_health_ready_gives_a_brand_new_feed_time_before_it_counts_as_stale(client, broker_queue):
Feed.objects.create(feed_url="https://new.example.com/feed.xml")
status, body = _ready(client)
assert status == 200
assert body["checks"]["recent_fetch_success"] is True
@pytest.mark.django_db
def test_health_ready_flags_most_feeds_failing_but_not_a_single_broken_one(client, broker_queue):
from django.utils import timezone
now = timezone.now()
healthy = Feed.objects.create(
feed_url="https://ok.example.com/feed.xml",
last_fetch_status=Feed.STATUS_OK,
last_fetched_at=now,
)
broken = [
Feed.objects.create(
feed_url=f"https://bad{i}.example.com/feed.xml",
last_fetch_status=Feed.STATUS_ERROR,
fetch_error_count=3,
last_fetched_at=now,
)
for i in range(2)
]
status, body = _ready(client)
assert status == 200, "2 of 3 failing is below the minimum feed count for the ratio"
Feed.objects.create(
feed_url="https://bad9.example.com/feed.xml",
last_fetch_status=Feed.STATUS_ERROR,
fetch_error_count=1,
last_fetched_at=now,
)
status, body = _ready(client)
assert status == 503
assert body["checks"]["feed_failures"] is False
assert body["failing_feeds"] == 3
assert healthy.pk and broken
@pytest.mark.django_db
def test_readiness_reuses_the_pipeline_aggregates_for_a_short_time(client, broker_queue, settings):
"""R2-C5: the expensive part of the readiness check is cached briefly,
while the cheap liveness probes stay live."""
from django.core.cache import cache
from django.utils import timezone
from apps.feeds.tasks import POLL_HEARTBEAT_CACHE_KEY
settings.EVECTION_HEALTH_CACHE_SECONDS = 30
cache.set(POLL_HEARTBEAT_CACHE_KEY, timezone.now())
Feed.objects.create(feed_url="https://a.example.com/feed.xml")
first = client.get(reverse("core:health_ready")).json()
Feed.objects.create(feed_url="https://b.example.com/feed.xml")
second = client.get(reverse("core:health_ready")).json()
assert first["active_feeds"] == 1
assert second["active_feeds"] == 1, "served from the short-lived cache"
cache.delete("health-ready-pipeline")
third = client.get(reverse("core:health_ready")).json()
assert third["active_feeds"] == 2
@pytest.mark.django_db
def test_readiness_liveness_probes_are_never_cached(client, broker_queue, settings):
from django.core.cache import cache
settings.EVECTION_HEALTH_CACHE_SECONDS = 30
first = client.get(reverse("core:health_ready"))
assert first.json()["checks"]["poll_heartbeat"] is False
from django.utils import timezone
from apps.feeds.tasks import POLL_HEARTBEAT_CACHE_KEY
cache.set(POLL_HEARTBEAT_CACHE_KEY, timezone.now())
second = client.get(reverse("core:health_ready"))
assert second.json()["checks"]["poll_heartbeat"] is True
+1
View File
@@ -6,6 +6,7 @@ app_name = "core"
urlpatterns = [ urlpatterns = [
path("health/", views.health, name="health"), path("health/", views.health, name="health"),
path("health/ready/", views.health_ready, name="health_ready"),
path("dashboard/", views.dashboard, name="dashboard"), path("dashboard/", views.dashboard, name="dashboard"),
path("dashboard/feeds/<int:feed_id>/retry/", views.retry_feed, name="retry_feed"), path("dashboard/feeds/<int:feed_id>/retry/", views.retry_feed, name="retry_feed"),
] ]
+30
View File
@@ -0,0 +1,30 @@
from urllib.parse import urljoin, urlsplit
ALLOWED_SCHEMES = {"http", "https"}
def http_url_or_empty(value: str | None) -> str:
"""The URL if it's an absolute http(s) URL, else "". Feed content is
untrusted: a link of "javascript:..." or "data:..." is harmless inside
an escaped attribute value but becomes script the moment someone
clicks it, so every URL we store from a feed or OPML file, and every
one we render as a link, goes through this. (urlsplit already ignores
the tab/newline tricks browsers also ignore inside a scheme.)"""
if not value:
return ""
value = value.strip()
try:
parts = urlsplit(value)
except ValueError:
return ""
if parts.scheme.lower() not in ALLOWED_SCHEMES or not parts.netloc:
return ""
return value
def resolve_http_url(base: str, value: str | None) -> str:
"""Like http_url_or_empty, but first resolves a relative link (feeds
often give "/2026/post") against the feed's own URL."""
if not value:
return ""
return http_url_or_empty(urljoin(base, value.strip()))
+67 -6
View File
@@ -1,20 +1,29 @@
from django.contrib import messages from django.contrib import messages
from django.contrib.auth.decorators import user_passes_test from django.contrib.auth.decorators import user_passes_test
from django.core.cache import cache
from django.db import connection from django.db import connection
from django.db.models import Count, Q from django.db.models import Count, Min, Q
from django.http import JsonResponse from django.http import JsonResponse
from django.shortcuts import get_object_or_404, redirect, render from django.shortcuts import get_object_or_404, redirect, render
from django.utils import timezone
from django.views.decorators.http import require_POST from django.views.decorators.http import require_POST
from apps.articles.models import Article from apps.articles.models import Article
from apps.feeds.models import Feed, Subscription from apps.feeds.models import Feed, Subscription
from apps.feeds.tasks import fetch_feed, reset_feed_backoff from apps.feeds.tasks import POLL_HEARTBEAT_CACHE_KEY, enqueue_fetch, reset_backoff
from .health import cached_fetch_pipeline_status
is_admin = user_passes_test(lambda u: u.is_authenticated and u.is_staff) is_admin = user_passes_test(lambda u: u.is_authenticated and u.is_staff)
# Beat ticks poll_due_feeds every minute; this many minutes without a tick
# means beat or the worker (or the broker between them) isn't working.
POLL_HEARTBEAT_MAX_AGE_SECONDS = 5 * 60
def health(request): def health(request):
"""Unauthenticated liveness/readiness check — reachable even before """Unauthenticated liveness check — reachable even before
first-run setup (see SetupWizardMiddleware). Used by scripts/upgrade.sh first-run setup (see SetupWizardMiddleware). Used by scripts/upgrade.sh
to confirm a freshly restarted deploy actually came up, and is a to confirm a freshly restarted deploy actually came up, and is a
reasonable target for any external uptime monitor too.""" reasonable target for any external uptime monitor too."""
@@ -26,6 +35,52 @@ def health(request):
return JsonResponse({"status": "ok"}) return JsonResponse({"status": "ok"})
def health_ready(request):
"""Readiness: liveness plus everything background fetching depends on
(C8) — the database, the Redis cache/broker, and a recent
poll_due_feeds tick, which proves beat is scheduling AND a worker is
consuming. Beyond that tick, apps.core.health checks fetching is
actually succeeding: queue depth, a recent successful fetch, and the
share of feeds failing. /health/ above stays DB-only on purpose: scripts/upgrade.sh
gates a deploy on it right after restart, before beat has had a
chance to tick. Point an uptime monitor at this one instead."""
checks = {"database": False, "redis": False, "poll_heartbeat": False}
detail = {}
try:
with connection.cursor() as cursor:
cursor.execute("SELECT 1")
checks["database"] = True
except Exception:
pass
try:
cache.set("health-ready-probe", "1", timeout=10)
checks["redis"] = cache.get("health-ready-probe") == "1"
except Exception:
pass
try:
last_tick = cache.get(POLL_HEARTBEAT_CACHE_KEY)
except Exception:
last_tick = None
if last_tick is not None:
age = (timezone.now() - last_tick).total_seconds()
detail["poll_heartbeat_age_seconds"] = int(age)
checks["poll_heartbeat"] = age <= POLL_HEARTBEAT_MAX_AGE_SECONDS
if checks["database"]:
pipeline_checks, pipeline_detail = cached_fetch_pipeline_status()
checks.update(pipeline_checks)
detail.update(pipeline_detail)
ok = all(checks.values())
return JsonResponse(
{"status": "ok" if ok else "degraded", "checks": checks, **detail},
status=200 if ok else 503,
)
@is_admin @is_admin
def dashboard(request): def dashboard(request):
feeds = Feed.objects.annotate(subscriber_count=Count("subscriptions")) feeds = Feed.objects.annotate(subscriber_count=Count("subscriptions"))
@@ -39,6 +94,7 @@ def dashboard(request):
"error_feed_count": erroring_feeds.count(), "error_feed_count": erroring_feeds.count(),
"subscription_count": Subscription.objects.count(), "subscription_count": Subscription.objects.count(),
"article_count": Article.objects.count(), "article_count": Article.objects.count(),
"oldest_article_at": Article.objects.aggregate(oldest=Min("fetched_at"))["oldest"],
} }
return render( return render(
@@ -52,7 +108,12 @@ def dashboard(request):
@require_POST @require_POST
def retry_feed(request, feed_id): def retry_feed(request, feed_id):
feed = get_object_or_404(Feed, pk=feed_id) feed = get_object_or_404(Feed, pk=feed_id)
reset_feed_backoff.delay(feed.id) # Reset first, synchronously: as separate queued tasks the reset could
fetch_feed.delay(feed.id) # land after the fetch and undo the schedule it just set.
messages.success(request, f"Queued an immediate retry for '{feed.title or feed.feed_url}'.") reset_backoff(feed.id)
name = feed.title or feed.feed_url
if enqueue_fetch(feed.id):
messages.success(request, f"Queued an immediate retry for '{name}'.")
else:
messages.info(request, f"'{name}' is already being fetched right now.")
return redirect("core:dashboard") return redirect("core:dashboard")
+21
View File
@@ -0,0 +1,21 @@
import socket
import pytest
@pytest.fixture(autouse=True)
def _fake_public_dns(monkeypatch):
"""apps.feeds.services.ssrf_guard resolves a feed's real hostname via
socket.getaddrinfo before every fetch. Tests use made-up hostnames
like example.com/a.example.com purely as fixture data — they must
not depend on real DNS/network access to pass that check, and must
resolve deterministically regardless of this sandbox's network
access. Every lookup here resolves to a fixed, definitely-public IP;
tests that specifically exercise the "resolves to a non-public
address" rejection path patch socket.getaddrinfo themselves instead.
"""
def _fake_getaddrinfo(host, *args, **kwargs):
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("93.184.216.34", 0))]
monkeypatch.setattr(socket, "getaddrinfo", _fake_getaddrinfo)
+23
View File
@@ -1,10 +1,22 @@
from django import forms from django import forms
from apps.core.urlutils import http_url_or_empty
from .models import Subscription from .models import Subscription
from .opml import MAX_UPLOAD_BYTES
class AddFeedForm(forms.Form): class AddFeedForm(forms.Form):
feed_url = forms.URLField(label="Feed URL", max_length=1000, assume_scheme="https") feed_url = forms.URLField(label="Feed URL", max_length=1000, assume_scheme="https")
def clean_feed_url(self):
# URLField also accepts ftp:// and ftps://; feeds are only ever
# fetched over http(s).
url = http_url_or_empty(self.cleaned_data["feed_url"])
if not url:
raise forms.ValidationError("Enter a feed URL starting with http:// or https://.")
return url
tag_names = forms.CharField( tag_names = forms.CharField(
label="Tags", label="Tags",
max_length=500, max_length=500,
@@ -26,3 +38,14 @@ class EditSubscriptionForm(forms.ModelForm):
class OPMLImportForm(forms.Form): class OPMLImportForm(forms.Form):
opml_file = forms.FileField(label="OPML file") opml_file = forms.FileField(label="OPML file")
def clean_opml_file(self):
uploaded = self.cleaned_data["opml_file"]
# Checked on the upload's reported size before anything reads it
# into memory (R2-S5); the reverse proxy's client_max_body_size is
# a second, coarser limit.
if uploaded.size > MAX_UPLOAD_BYTES:
raise forms.ValidationError(
f"That file is too large (limit {MAX_UPLOAD_BYTES // (1024 * 1024)} MB)."
)
return uploaded
@@ -0,0 +1,18 @@
# Generated by Django 5.2.18 on 2026-10-31 18:00
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("feeds", "0002_remove_folder"),
]
operations = [
migrations.AddField(
model_name="feed",
name="claimed_at",
field=models.DateTimeField(blank=True, null=True),
),
]
@@ -0,0 +1,17 @@
# Generated by Django 5.2.18 on 2026-10-31 18:56
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
("feeds", "0003_feed_claimed_at"),
]
operations = [
migrations.RemoveField(
model_name="feed",
name="fetch_interval_minutes",
),
]
@@ -0,0 +1,24 @@
# Generated by Django 5.2.18 on 2026-10-31 22:00
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("feeds", "0004_remove_feed_fetch_interval_minutes"),
]
operations = [
migrations.AddField(
model_name="feed",
name="next_fetch_at",
field=models.DateTimeField(blank=True, null=True),
),
migrations.AddIndex(
model_name="feed",
index=models.Index(
fields=["is_active", "next_fetch_at"], name="feeds_feed_is_acti_5c232e_idx"
),
),
]
@@ -0,0 +1,49 @@
from datetime import timedelta
from django.conf import settings
from django.db import migrations
def backfill_next_fetch_at(apps, schema_editor):
"""One-time backfill for feeds that existed before next_fetch_at (C1) —
everything created after this migration gets it set by fetch_feed
itself. Mirrors the exact formula apps.feeds.services.fetcher.
effective_interval_minutes used to compute on every poll tick, just
run once here instead of forever after.
"""
Feed = apps.get_model("feeds", "Feed")
SiteSettings = apps.get_model("core", "SiteSettings")
configured_interval = 60
settings_row = SiteSettings.objects.filter(pk=1).first()
if settings_row is not None:
configured_interval = settings_row.fetch_interval_minutes
floor = getattr(settings, "EVECTION_MIN_FETCH_INTERVAL_MINUTES", 15)
base = max(configured_interval, floor)
max_backoff = 24 * 60
for feed in Feed.objects.filter(last_fetched_at__isnull=False).only(
"id", "last_fetched_at", "fetch_error_count"
):
if feed.fetch_error_count <= 0:
interval = base
else:
interval = min(base * (2**feed.fetch_error_count), max_backoff)
feed.next_fetch_at = feed.last_fetched_at + timedelta(minutes=interval)
feed.save(update_fields=["next_fetch_at"])
def noop_reverse(apps, schema_editor):
pass
class Migration(migrations.Migration):
dependencies = [
("feeds", "0005_feed_next_fetch_at_and_more"),
("core", "0001_initial"),
]
operations = [
migrations.RunPython(backfill_next_fetch_at, noop_reverse),
]
+25 -2
View File
@@ -35,13 +35,36 @@ class Feed(models.Model):
) )
last_error_message = models.TextField(blank=True) last_error_message = models.TextField(blank=True)
# Instance-wide, not a per-user preference — see class docstring. # The poll interval itself is site-wide now (apps.core.models.
fetch_interval_minutes = models.PositiveIntegerField(default=60) # SiteSettings), not per-feed — only the error-driven backoff on top
# of it still varies per feed (see apps.feeds.services.fetcher.
# effective_interval_minutes).
fetch_error_count = models.PositiveIntegerField(default=0) fetch_error_count = models.PositiveIntegerField(default=0)
is_active = models.BooleanField(default=True) is_active = models.BooleanField(default=True)
# Set by fetch_feed right after every fetch attempt (success or error)
# to last_fetched_at + effective_interval_minutes — precomputed once,
# at write time, instead of poll_due_feeds re-deriving due-ness (which
# depends on fetch_error_count's exponential backoff) for every active
# feed on every tick (C1). NULL means "due now", same as a feed that's
# never been fetched. Indexed with is_active below so the scheduler's
# query is a direct index range scan, not a full table scan.
next_fetch_at = models.DateTimeField(null=True, blank=True)
# Set by poll_due_feeds' atomic claim (an UPDATE ... WHERE clause, not
# select_for_update — Celery Beat and fetch_feed run in separate
# transactions) and cleared once fetch_feed finishes, so overlapping
# beat ticks or a fetch that runs long can't both dispatch the same
# feed. A claim older than CLAIM_STALE_MINUTES (tasks.py) is treated as
# abandoned and can be reclaimed, so a worker dying mid-fetch doesn't
# wedge the feed forever.
claimed_at = models.DateTimeField(null=True, blank=True)
created_at = models.DateTimeField(auto_now_add=True) created_at = models.DateTimeField(auto_now_add=True)
class Meta:
indexes = [models.Index(fields=["is_active", "next_fetch_at"])]
def __str__(self): def __str__(self):
return self.title or self.feed_url return self.title or self.feed_url
+67 -20
View File
@@ -17,6 +17,7 @@ from io import BytesIO
import defusedxml.ElementTree as DefusedET import defusedxml.ElementTree as DefusedET
from apps.articles.models import SubscriptionTag, Tag from apps.articles.models import SubscriptionTag, Tag
from apps.core.urlutils import http_url_or_empty
from .models import Feed, Subscription from .models import Feed, Subscription
@@ -25,6 +26,14 @@ from .models import Feed, Subscription
# that directly instead of making people unzip it themselves first. # that directly instead of making people unzip it themselves first.
MAX_ZIPPED_OPML_SIZE = 20 * 1024 * 1024 MAX_ZIPPED_OPML_SIZE = 20 * 1024 * 1024
# An upload is untrusted: bound everything an attacker controls (R2-S5).
# A real export of thousands of feeds is well under a megabyte.
MAX_UPLOAD_BYTES = 5 * 1024 * 1024
MAX_ZIP_ENTRIES = 50
MAX_OPML_DEPTH = 24
MAX_OPML_OUTLINES = 20_000
MAX_OPML_FEEDS = 5_000
def extract_opml_bytes(uploaded_bytes: bytes) -> bytes: def extract_opml_bytes(uploaded_bytes: bytes) -> bytes:
"""If the upload is a zip (e.g. FreshRSS's export), returns the bytes """If the upload is a zip (e.g. FreshRSS's export), returns the bytes
@@ -34,6 +43,8 @@ def extract_opml_bytes(uploaded_bytes: bytes) -> bytes:
return uploaded_bytes return uploaded_bytes
with zipfile.ZipFile(BytesIO(uploaded_bytes)) as archive: with zipfile.ZipFile(BytesIO(uploaded_bytes)) as archive:
if len(archive.infolist()) > MAX_ZIP_ENTRIES:
raise ValueError("The uploaded zip contains too many files.")
candidates = [ candidates = [
info info
for info in archive.infolist() for info in archive.infolist()
@@ -44,7 +55,13 @@ def extract_opml_bytes(uploaded_bytes: bytes) -> bytes:
member = min(candidates, key=lambda info: info.file_size) member = min(candidates, key=lambda info: info.file_size)
if member.file_size > MAX_ZIPPED_OPML_SIZE: if member.file_size > MAX_ZIPPED_OPML_SIZE:
raise ValueError("The .opml file inside the zip is implausibly large.") raise ValueError("The .opml file inside the zip is implausibly large.")
return archive.read(member) # file_size is just a header field an attacker controls, so bound
# what actually decompresses too, not only what it claims.
with archive.open(member) as handle:
data = handle.read(MAX_ZIPPED_OPML_SIZE + 1)
if len(data) > MAX_ZIPPED_OPML_SIZE:
raise ValueError("The .opml file inside the zip is implausibly large.")
return data
def parse_opml(file_obj) -> list[dict]: def parse_opml(file_obj) -> list[dict]:
@@ -57,38 +74,68 @@ def parse_opml(file_obj) -> list[dict]:
on the feed's own outline, the convention export_opml below writes and on the feed's own outline, the convention export_opml below writes and
many other readers use for a flat multi-category representation. many other readers use for a flat multi-category representation.
""" """
tree = DefusedET.parse(file_obj)
root = tree.getroot()
body = root.find("body")
entries: list[dict] = [] entries: list[dict] = []
if body is None: # Streamed with iterparse instead of building the whole tree first, so
return entries # a hostile file (absurd nesting, millions of outlines) is rejected as
# soon as it crosses a limit rather than after it has used the memory.
in_body = False
depth = 0
outline_count = 0
open_feed_outlines = 0 # >0 while inside a feed outline: its children are ignored
pushed_name: list[bool] = [] # per open category outline: did it add a tag name?
ancestor_names: list[str] = []
def walk(container, ancestor_names): for event, elem in DefusedET.iterparse(file_obj, events=("start", "end")):
for outline in container.findall("outline"): if event == "start":
xml_url = outline.get("xmlUrl") depth += 1
if xml_url: if depth > MAX_OPML_DEPTH:
raise ValueError("The OPML file is nested too deeply.")
if elem.tag == "body":
in_body = True
elif elem.tag == "outline" and in_body:
outline_count += 1
if outline_count > MAX_OPML_OUTLINES:
raise ValueError("The OPML file contains too many entries.")
if open_feed_outlines:
open_feed_outlines += 1
continue
raw_xml_url = elem.get("xmlUrl")
if raw_xml_url:
open_feed_outlines = 1
xml_url = http_url_or_empty(raw_xml_url)
if not xml_url:
continue # a feed URL that isn't http(s) can never be fetched
tag_names = list(ancestor_names) tag_names = list(ancestor_names)
category_attr = outline.get("category") for name in (elem.get("category") or "").split(","):
if category_attr:
for name in category_attr.split(","):
name = name.strip() name = name.strip()
if name and name not in tag_names: if name and name not in tag_names:
tag_names.append(name) tag_names.append(name)
entries.append( entries.append(
{ {
"feed_url": xml_url, "feed_url": xml_url,
"title": outline.get("title") or outline.get("text") or "", "title": elem.get("title") or elem.get("text") or "",
"site_url": outline.get("htmlUrl") or "", "site_url": http_url_or_empty(elem.get("htmlUrl")),
"tag_names": tag_names, "tag_names": tag_names,
} }
) )
if len(entries) > MAX_OPML_FEEDS:
raise ValueError("The OPML file contains too many feeds.")
else: else:
nested_name = outline.get("title") or outline.get("text") nested_name = elem.get("title") or elem.get("text")
next_ancestors = ancestor_names + [nested_name] if nested_name else ancestor_names pushed_name.append(bool(nested_name))
walk(outline, next_ancestors) if nested_name:
ancestor_names.append(nested_name)
walk(body, []) else:
depth -= 1
if elem.tag == "body":
in_body = False
elif elem.tag == "outline" and in_body:
if open_feed_outlines:
open_feed_outlines -= 1
elif pushed_name:
if pushed_name.pop():
ancestor_names.pop()
elem.clear()
return entries return entries
+154 -42
View File
@@ -11,24 +11,38 @@ import hashlib
import logging import logging
from calendar import timegm from calendar import timegm
from dataclasses import dataclass from dataclasses import dataclass
from datetime import datetime from datetime import datetime, timedelta
from datetime import timezone as dt_timezone from datetime import timezone as dt_timezone
import feedparser import feedparser
import nh3 import nh3
import requests import requests
from django.conf import settings from django.conf import settings
from django.db import IntegrityError, transaction
from django.utils import timezone from django.utils import timezone
from apps.articles.models import Article from apps.articles.models import Article, PurgedArticle
from apps.core.models import SiteSettings
from apps.core.urlutils import resolve_http_url
from apps.feeds.models import Feed from apps.feeds.models import Feed
from .ssrf_guard import SSRFBlocked, safe_get
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
FETCH_TIMEOUT_SECONDS = 15 FETCH_TIMEOUT_SECONDS = 15
MAX_BACKOFF_MINUTES = 24 * 60 MAX_BACKOFF_MINUTES = 24 * 60
USER_AGENT = "Evection/0.1 (+https://github.com/; self-hosted RSS reader)" USER_AGENT = "Evection/0.1 (+https://github.com/; self-hosted RSS reader)"
# ssrf_guard.MAX_RESPONSE_BYTES already bounds the raw download; these
# bound what a single poll does *with* that response — a feed within the
# byte cap can still carry an unreasonable number of entries, or one
# entry with an unreasonably large body, and both cost real CPU/DB time
# (nh3 sanitization, an Article row/UPDATE) per entry processed.
MAX_ENTRIES_PER_POLL = 200
MAX_CONTENT_HTML_INPUT_CHARS = 200_000 # bounds sanitizer cost, not just storage
MAX_CONTENT_HTML_CHARS = 50_000 # bounds what's actually stored/rendered
ALLOWED_TAGS = { ALLOWED_TAGS = {
"a", "a",
"p", "p",
@@ -81,11 +95,20 @@ class FetchResult:
def sanitize_html(raw_html: str) -> str: def sanitize_html(raw_html: str) -> str:
if not raw_html: if not raw_html:
return "" return ""
return nh3.clean(raw_html, tags=ALLOWED_TAGS, attributes=ALLOWED_ATTRIBUTES, link_rel=None) bounded = raw_html[:MAX_CONTENT_HTML_INPUT_CHARS]
cleaned = nh3.clean(bounded, tags=ALLOWED_TAGS, attributes=ALLOWED_ATTRIBUTES, link_rel=None)
return cleaned[:MAX_CONTENT_HTML_CHARS]
def _content_hash(title: str, content_html: str) -> str: def _content_hash(
return hashlib.sha256(f"{title}\n{content_html}".encode("utf-8")).hexdigest() title: str, link: str, author: str, summary: str, content_html: str, published_at
) -> str:
"""Covers every mutable Article field an entry carries (T6) — a feed
editing its link/author/summary/published date, not just title or
body, must still be detected as a change worth re-saving."""
published_key = published_at.isoformat() if published_at else ""
payload = "\n".join([title, link, author, summary, content_html, published_key])
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
def _entry_guid(entry) -> str: def _entry_guid(entry) -> str:
@@ -111,14 +134,37 @@ def _entry_content_html(entry) -> str:
return getattr(entry, "summary", "") or "" return getattr(entry, "summary", "") or ""
def effective_interval_minutes(feed: Feed) -> int: UPDATABLE_ARTICLE_FIELDS = (
"""The actual polling interval to use, applying exponential backoff on "title",
top of the admin-configured ``fetch_interval_minutes`` for feeds that "link",
are currently erroring. ``fetch_interval_minutes`` itself is never "author",
mutated by the fetcher — it's the admin's chosen baseline. "summary",
"content_html",
"published_at",
"content_hash",
)
def _ensure_pks(feed: Feed, articles: list[Article]) -> list[int]:
"""bulk_create only populates primary keys on backends that can
RETURN them (Postgres, SQLite 3.35+); look the rest up by guid."""
if all(a.pk for a in articles):
return [a.pk for a in articles]
guids = [a.guid for a in articles]
return list(Article.objects.filter(feed=feed, guid__in=guids).values_list("pk", flat=True))
def effective_interval_minutes(feed: Feed, configured_interval_minutes: int) -> int:
"""The actual polling interval to use for `feed`, applying exponential
backoff on top of the site-wide `configured_interval_minutes` (see
apps.core.models.SiteSettings) for a feed that's currently erroring.
Takes the configured value as a parameter rather than querying
SiteSettings itself, since this runs once per feed (apps.feeds.tasks
calls it from a list comprehension over every active feed) and the
site-wide value doesn't vary by feed.
""" """
floor = getattr(settings, "EVECTION_MIN_FETCH_INTERVAL_MINUTES", 15) floor = getattr(settings, "EVECTION_MIN_FETCH_INTERVAL_MINUTES", 15)
base = max(feed.fetch_interval_minutes, floor) base = max(configured_interval_minutes, floor)
if feed.fetch_error_count <= 0: if feed.fetch_error_count <= 0:
return base return base
backoff = base * (2**feed.fetch_error_count) backoff = base * (2**feed.fetch_error_count)
@@ -139,10 +185,13 @@ def fetch_and_ingest(feed: Feed) -> FetchResult:
headers["If-Modified-Since"] = feed.last_modified headers["If-Modified-Since"] = feed.last_modified
try: try:
response = requests.get(feed.feed_url, headers=headers, timeout=FETCH_TIMEOUT_SECONDS) response = safe_get(feed.feed_url, headers=headers, timeout=FETCH_TIMEOUT_SECONDS)
except SSRFBlocked as exc:
feed.last_fetched_at = timezone.now()
return record_fetch_error(feed, f"Refused to fetch: {exc}")
except requests.RequestException as exc: except requests.RequestException as exc:
feed.last_fetched_at = timezone.now() feed.last_fetched_at = timezone.now()
return _record_error(feed, f"Request failed: {exc}") return record_fetch_error(feed, f"Request failed: {exc}")
feed.last_fetched_at = timezone.now() feed.last_fetched_at = timezone.now()
@@ -154,12 +203,12 @@ def fetch_and_ingest(feed: Feed) -> FetchResult:
) )
if response.status_code >= 400: if response.status_code >= 400:
return _record_error(feed, f"HTTP {response.status_code}") return record_fetch_error(feed, f"HTTP {response.status_code}")
parsed = feedparser.parse(response.content) parsed = feedparser.parse(response.content)
if parsed.bozo and not parsed.entries: if parsed.bozo and not parsed.entries:
return _record_error(feed, f"Unparseable feed: {parsed.get('bozo_exception')}") return record_fetch_error(feed, f"Unparseable feed: {parsed.get('bozo_exception')}")
if parsed.bozo: if parsed.bozo:
logger.warning( logger.warning(
@@ -174,48 +223,111 @@ def fetch_and_ingest(feed: Feed) -> FetchResult:
channel_title = parsed.feed.get("title") channel_title = parsed.feed.get("title")
if channel_title and not feed.title: if channel_title and not feed.title:
feed.title = channel_title[:500] feed.title = channel_title[:500]
channel_link = parsed.feed.get("link") channel_link = resolve_http_url(feed.feed_url, parsed.feed.get("link"))
if channel_link and not feed.site_url: if channel_link and not feed.site_url:
feed.site_url = channel_link[:1000] feed.site_url = channel_link[:1000]
new_ids: list[int] = [] new_ids: list[int] = []
updated_ids: list[int] = [] updated_ids: list[int] = []
for entry in parsed.entries: entries = parsed.entries
guid = _entry_guid(entry) if len(entries) > MAX_ENTRIES_PER_POLL:
title = (getattr(entry, "title", "") or "")[:1000] logger.warning(
content_html = sanitize_html(_entry_content_html(entry)) "Feed %s: %d entries in one poll, only processing the first %d",
content_hash = _content_hash(title, content_html) feed.feed_url,
len(entries),
article, created = Article.objects.get_or_create( MAX_ENTRIES_PER_POLL,
feed=feed,
guid=guid,
defaults={
"link": (getattr(entry, "link", "") or "")[:2000],
"title": title,
"author": (getattr(entry, "author", "") or "")[:500],
"content_html": content_html,
"summary": (getattr(entry, "summary", "") or "")[:5000],
"content_hash": content_hash,
"published_at": _entry_published_at(entry),
},
) )
entries = entries[:MAX_ENTRIES_PER_POLL]
# One prefetch + bulk writes for the whole poll (C2), instead of a
# get_or_create (SELECT, then INSERT) per entry. Entries are keyed by
# guid so a feed that repeats one within a single response is only
# processed once.
parsed_entries: dict[str, dict] = {}
for entry in entries:
guid = _entry_guid(entry)
if guid in parsed_entries:
continue
title = (getattr(entry, "title", "") or "")[:1000]
link = resolve_http_url(feed.feed_url, getattr(entry, "link", ""))[:2000]
author = (getattr(entry, "author", "") or "")[:500]
summary = (getattr(entry, "summary", "") or "")[:5000]
content_html = sanitize_html(_entry_content_html(entry))
published_at = _entry_published_at(entry)
parsed_entries[guid] = {
"title": title,
"link": link,
"author": author,
"summary": summary,
"content_html": content_html,
"published_at": published_at,
"content_hash": _content_hash(title, link, author, summary, content_html, published_at),
}
# Entries the retention purge already deleted (tombstoned) and, when
# retention is on, entries so old by their own publish date that the
# next purge would delete them again, are not (re-)ingested — see
# apps.articles.tasks.purge_old_articles.
retention_days = SiteSettings.load().retention_days
if retention_days > 0:
too_old_before = timezone.now() - timedelta(days=retention_days)
for guid in [
g
for g, f in parsed_entries.items()
if f["published_at"] is not None and f["published_at"] < too_old_before
]:
del parsed_entries[guid]
for guid in PurgedArticle.objects.filter(feed=feed, guid__in=parsed_entries).values_list(
"guid", flat=True
):
del parsed_entries[guid]
existing = {a.guid: a for a in Article.objects.filter(feed=feed, guid__in=parsed_entries)}
to_create = [
Article(feed=feed, guid=guid, **fields)
for guid, fields in parsed_entries.items()
if guid not in existing
]
to_update = []
for guid, article in existing.items():
fields = parsed_entries[guid]
if article.content_hash != fields["content_hash"]:
for name, value in fields.items():
setattr(article, name, value)
to_update.append(article)
if to_update:
Article.objects.bulk_update(to_update, list(UPDATABLE_ARTICLE_FIELDS), batch_size=100)
updated_ids.extend(a.pk for a in to_update)
if to_create:
try:
with transaction.atomic():
Article.objects.bulk_create(to_create)
except IntegrityError:
# Another worker inserted some of these guids between the
# prefetch above and now (two fetches of the same feed can
# overlap) — fall back to row-by-row so nothing is lost or
# duplicated, at the cost of the old per-entry round trips
# for this rare case only.
for article in to_create:
fields = {name: getattr(article, name) for name in UPDATABLE_ARTICLE_FIELDS}
obj, created = Article.objects.get_or_create(
feed=feed, guid=article.guid, defaults=fields
)
if created: if created:
new_ids.append(article.pk) new_ids.append(obj.pk)
elif article.content_hash != content_hash: else:
article.title = title new_ids.extend(_ensure_pks(feed, to_create))
article.content_html = content_html
article.content_hash = content_hash
article.save(update_fields=["title", "content_html", "content_hash"])
updated_ids.append(article.pk)
return FetchResult( return FetchResult(
status=Feed.STATUS_OK, new_article_ids=new_ids, updated_article_ids=updated_ids status=Feed.STATUS_OK, new_article_ids=new_ids, updated_article_ids=updated_ids
) )
def _record_error(feed: Feed, message: str) -> FetchResult: def record_fetch_error(feed: Feed, message: str) -> FetchResult:
feed.last_fetch_status = Feed.STATUS_ERROR feed.last_fetch_status = Feed.STATUS_ERROR
feed.last_error_message = message[:2000] feed.last_error_message = message[:2000]
feed.fetch_error_count += 1 feed.fetch_error_count += 1
+203
View File
@@ -0,0 +1,203 @@
"""Rejects feed-fetch targets that could turn this server/worker's own
network access into an SSRF vector.
Feed URLs are entirely attacker-controlled (any authenticated user via
"add feed", or an uploaded OPML file — see apps.feeds.opml) and this app
fetches them from the server's own network position. fetch_and_ingest
(apps.feeds.services.fetcher) is the single chokepoint every feed URL
passes through before any HTTP request is made, regardless of which
path created the Feed row, so enforcing this here covers all of them.
This validates the resolved address before connecting and again before
following each redirect hop, denying by default anything that isn't a
globally routable address (ip.is_global) — covering loopback, RFC1918
private ranges, link-local (including the 169.254.169.254 cloud
metadata address), and so on, for both IPv4 and IPv6.
The connection is pinned too (R2-S2): the address is resolved, checked and
connected to inside one function (_connect_validated, used by the adapter
below as the connection's socket factory), so a DNS answer that changes
between a separate validation lookup and the connect (DNS rebinding) can't
steer the connection to an internal address — whatever it resolves to *at
connect time* is what gets validated. TLS still verifies against the
original hostname, because only the socket is replaced, not the host the
TLS layer is told about. The up-front validate_fetch_url check stays as
defense in depth and gives a clear error before any connection attempt.
"""
from __future__ import annotations
import ipaddress
import socket
from dataclasses import dataclass
from urllib.parse import urljoin, urlparse
import requests
from requests.adapters import HTTPAdapter
from requests.structures import CaseInsensitiveDict
from urllib3.connection import HTTPConnection, HTTPSConnection
from urllib3.connectionpool import HTTPConnectionPool, HTTPSConnectionPool
ALLOWED_SCHEMES = {"http", "https"}
MAX_REDIRECTS = 5
MAX_RESPONSE_BYTES = 10 * 1024 * 1024 # 10 MB — generous for an RSS/Atom feed
READ_CHUNK_BYTES = 65536
class SSRFBlocked(Exception):
"""Raised when a fetch target — or one of its redirects — isn't safe
to connect to on a user's behalf."""
@dataclass
class BoundedResponse:
status_code: int
# requests' own CaseInsensitiveDict — server header casing varies
# (ETag/Etag/etag) and fetcher.py looks these up by a fixed-case key.
headers: CaseInsensitiveDict
content: bytes
def _is_allowed_ip(ip) -> bool:
return ip.is_global
def validate_fetch_url(url: str) -> None:
"""Raises SSRFBlocked unless `url` has an http(s) scheme, a hostname,
and every address that hostname resolves to is globally routable."""
parsed = urlparse(url)
if parsed.scheme not in ALLOWED_SCHEMES:
raise SSRFBlocked(f"Unsupported URL scheme: {parsed.scheme!r}")
if not parsed.hostname:
raise SSRFBlocked("URL has no hostname")
try:
addrinfo = socket.getaddrinfo(parsed.hostname, None)
except OSError as exc:
raise SSRFBlocked(f"Could not resolve {parsed.hostname!r}: {exc}") from exc
for *_, sockaddr in addrinfo:
ip = ipaddress.ip_address(sockaddr[0])
if not _is_allowed_ip(ip):
raise SSRFBlocked(f"{parsed.hostname!r} resolves to a non-public address: {ip}")
def _connect_validated(host, port, timeout, source_address, socket_options):
"""Resolves `host`, requires every address to be globally routable, and
connects to one of those exact addresses — the same lookup result is
both validated and used, so there is nothing for a rebinding DNS
answer to change in between."""
try:
addrinfo = socket.getaddrinfo(host, port, type=socket.SOCK_STREAM)
except OSError as exc:
raise SSRFBlocked(f"Could not resolve {host!r}: {exc}") from exc
candidates = []
for family, socktype, proto, _canonname, sockaddr in addrinfo:
ip = ipaddress.ip_address(sockaddr[0])
if not _is_allowed_ip(ip):
raise SSRFBlocked(f"{host!r} resolves to a non-public address: {ip}")
candidates.append((family, socktype, proto, sockaddr))
if not candidates:
raise SSRFBlocked(f"{host!r} did not resolve to any address")
last_error: OSError | None = None
for family, socktype, proto, sockaddr in candidates:
sock = None
try:
sock = socket.socket(family, socktype, proto)
for level, option, value in socket_options or ():
sock.setsockopt(level, option, value)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
sock.connect(sockaddr)
return sock
except OSError as exc:
last_error = exc
if sock is not None:
sock.close()
raise last_error or OSError(f"Could not connect to {host!r}")
class _PinnedHTTPConnection(HTTPConnection):
def _new_conn(self):
return _connect_validated(
self._dns_host, self.port, self.timeout, self.source_address, self.socket_options
)
class _PinnedHTTPSConnection(HTTPSConnection):
def _new_conn(self):
return _connect_validated(
self._dns_host, self.port, self.timeout, self.source_address, self.socket_options
)
class _PinnedHTTPConnectionPool(HTTPConnectionPool):
ConnectionCls = _PinnedHTTPConnection
class _PinnedHTTPSConnectionPool(HTTPSConnectionPool):
ConnectionCls = _PinnedHTTPSConnection
class PinnedAdapter(HTTPAdapter):
"""requests adapter whose connections go through _connect_validated."""
def init_poolmanager(self, *args, **kwargs):
super().init_poolmanager(*args, **kwargs)
self.poolmanager.pool_classes_by_scheme = {
"http": _PinnedHTTPConnectionPool,
"https": _PinnedHTTPSConnectionPool,
}
def safe_get(url: str, *, headers: dict, timeout: float) -> BoundedResponse:
"""GETs `url` the way fetch_and_ingest needs it: every redirect hop is
validated and followed manually (never left to requests/urllib3, so
a redirect can't bypass validate_fetch_url), capped at MAX_REDIRECTS,
and the body is read in bounded chunks rather than trusting
Content-Length or buffering an unbounded response.
"""
current_url = url
with requests.Session() as session:
adapter = PinnedAdapter()
session.mount("http://", adapter)
session.mount("https://", adapter)
for _ in range(MAX_REDIRECTS + 1):
validate_fetch_url(current_url)
response = session.get(
current_url,
headers=headers,
timeout=timeout,
stream=True,
allow_redirects=False,
)
try:
if response.is_redirect or response.is_permanent_redirect:
location = response.headers.get("Location")
if not location:
raise SSRFBlocked("Redirect response had no Location header")
current_url = urljoin(current_url, location)
continue
return BoundedResponse(
status_code=response.status_code,
headers=response.headers,
content=_read_bounded(response),
)
finally:
response.close()
raise SSRFBlocked(f"Too many redirects (> {MAX_REDIRECTS})")
def _read_bounded(response: requests.Response) -> bytes:
chunks = []
total = 0
for chunk in response.iter_content(chunk_size=READ_CHUNK_BYTES):
total += len(chunk)
if total > MAX_RESPONSE_BYTES:
raise SSRFBlocked(f"Response exceeded {MAX_RESPONSE_BYTES} bytes")
chunks.append(chunk)
return b"".join(chunks)
+184 -27
View File
@@ -4,50 +4,185 @@ from io import BytesIO
from celery import shared_task from celery import shared_task
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.core.cache import cache
from django.db import InterfaceError, OperationalError
from django.db.models import Q
from django.utils import timezone from django.utils import timezone
from apps.articles.tasks import apply_filter_rules_for_feed from apps.articles.tasks import apply_filter_rules_for_feed
from apps.core.models import SiteSettings
from .models import Feed from .models import Feed
from .opml import import_opml from .opml import import_opml
from .services.fetcher import effective_interval_minutes, fetch_and_ingest from .services.fetcher import effective_interval_minutes, fetch_and_ingest, record_fetch_error
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# How long a claimed-but-unfinished fetch blocks the same feed from being
# claimed again — must comfortably exceed FETCH_TIMEOUT_SECONDS plus parse/
# ingest time, so a slow-but-healthy fetch is never raced by the next tick,
# while a worker that died mid-fetch doesn't wedge the feed forever.
CLAIM_STALE_MINUTES = 10
# Caps how many feeds one poll_due_feeds tick will claim/dispatch, so a
# large backlog (first run after a long outage, a huge import) spreads
# across several beat ticks instead of enqueuing everything — and every
# bounded Feed query below stays index-only work, not a full scan (C1).
POLL_BATCH_SIZE = 500
RULE_DISPATCH_CHUNK_SIZE = 50
# Written on every poll_due_feeds tick; apps.core.views.health_ready
# reads it to tell whether beat is scheduling and a worker is consuming.
POLL_HEARTBEAT_CACHE_KEY = "poll-due-feeds-heartbeat"
POLL_HEARTBEAT_TTL_SECONDS = 3600
@shared_task @shared_task
def poll_due_feeds(): def poll_due_feeds():
"""Find every active Feed whose polling interval has elapsed and """Find a bounded batch of active Feeds whose next_fetch_at has
dispatch a fetch_feed task for each — fan-out happens here, not via one elapsed (set by fetch_feed itself after every attempt — see Feed.
Celery Beat entry per feed. next_fetch_at), atomically claim them (so overlapping beat ticks or a
slow-running previous fetch can't dispatch the same feed twice), and
dispatch a fetch_feed task for each claimed feed — fan-out happens
here, not via one Celery Beat entry per feed.
""" """
due_feed_ids = [ now = timezone.now()
feed.id cache.set(POLL_HEARTBEAT_CACHE_KEY, now, timeout=POLL_HEARTBEAT_TTL_SECONDS)
for feed in Feed.objects.filter(is_active=True).only( due_feed_ids = list(
"id", "last_fetched_at", "fetch_interval_minutes", "fetch_error_count" Feed.objects.filter(is_active=True)
.filter(Q(next_fetch_at__isnull=True) | Q(next_fetch_at__lte=now))
.filter(
Q(claimed_at__isnull=True)
| Q(claimed_at__lt=now - timedelta(minutes=CLAIM_STALE_MINUTES))
) )
if _is_due(feed) .order_by("next_fetch_at")
] .values_list("id", flat=True)[:POLL_BATCH_SIZE]
)
dispatched = 0
for feed_id in due_feed_ids: for feed_id in due_feed_ids:
try:
if enqueue_fetch(feed_id):
dispatched += 1
except Exception:
# Broker trouble: enqueue_fetch already released this feed's
# claim, so it's simply picked up again next tick.
logger.exception("Could not queue a fetch for feed %s", feed_id)
return dispatched
def claim_feed(feed_id: int) -> bool:
"""Atomically marks a feed as having a fetch in flight (an UPDATE whose
WHERE clause re-checks staleness, so two callers racing for the same
feed can't both get rowcount=1). Every path that dispatches a fetch —
the periodic poller, a newly added feed, an admin retry — goes through
this, so they can never run the same feed concurrently."""
now = timezone.now()
stale_before = now - timedelta(minutes=CLAIM_STALE_MINUTES)
return bool(
Feed.objects.filter(id=feed_id, is_active=True)
.filter(Q(claimed_at__isnull=True) | Q(claimed_at__lt=stale_before))
.update(claimed_at=now)
)
def enqueue_fetch(feed_id: int) -> bool:
"""Claim a feed and queue its fetch. False means another fetch of it is
already in flight (nothing was queued — that one will do the work).
If queueing itself fails the claim is released and the error re-raised,
so a broker outage doesn't leave the feed claimed until it goes stale.
"""
if not claim_feed(feed_id):
return False
try:
fetch_feed.delay(feed_id) fetch_feed.delay(feed_id)
return len(due_feed_ids) except Exception:
Feed.objects.filter(pk=feed_id).update(claimed_at=None)
raise
def _is_due(feed: Feed) -> bool:
if feed.last_fetched_at is None:
return True return True
interval = effective_interval_minutes(feed)
return feed.last_fetched_at <= timezone.now() - timedelta(minutes=interval)
@shared_task def _schedule_next_fetch(feed: Feed) -> None:
def fetch_feed(feed_id: int): """Sets feed.next_fetch_at from its (just-updated) fetch_error_count —
called right before every feed.save() in fetch_feed, so the next
poll_due_feeds tick can find it with a plain indexed range query
instead of recomputing every active feed's backoff in Python (C1)."""
configured_interval = SiteSettings.load().fetch_interval_minutes
interval = effective_interval_minutes(feed, configured_interval)
feed.next_fetch_at = timezone.now() + timedelta(minutes=interval)
# Bounds a single feed's fetch+parse+ingest, independent of
# FETCH_TIMEOUT_SECONDS (apps.feeds.services.fetcher) which only covers
# the HTTP request itself: parsing/sanitizing/writing a large-but-under-
# the-byte-cap feed could still run long. soft_time_limit lets the
# except-Exception handler below record a normal feed error (and clear
# the claim) before the hard time_limit kills the worker process outright.
FETCH_FEED_SOFT_TIME_LIMIT = 60
FETCH_FEED_TIME_LIMIT = 90
# Failures that say nothing about the feed and usually clear up in
# seconds — a locked SQLite file, a dropped database connection — are
# retried promptly (R2-T5) instead of waiting out the next poll interval,
# which under error backoff can be hours. Bounded: three attempts, 30s /
# 60s / 120s apart, then the normal "record the error and back off" path.
# Anything else (a parser bug, the soft time limit) is not retried: that
# would just repeat the same failure. The claim is held throughout so the
# poller can't also fetch the feed; it's far shorter than CLAIM_STALE_MINUTES.
TRANSIENT_FETCH_ERRORS = (OperationalError, InterfaceError)
FETCH_MAX_RETRIES = 3
FETCH_RETRY_BASE_DELAY_SECONDS = 30
def _record_unexpected_failure(feed: Feed) -> None:
logger.exception("Feed %s: unexpected error during fetch", feed.feed_url)
record_fetch_error(feed, "Unexpected error — see worker logs")
feed.last_fetched_at = timezone.now()
feed.claimed_at = None
_schedule_next_fetch(feed)
feed.save()
@shared_task(
bind=True, time_limit=FETCH_FEED_TIME_LIMIT, soft_time_limit=FETCH_FEED_SOFT_TIME_LIMIT
)
def fetch_feed(self, feed_id: int):
try: try:
feed = Feed.objects.get(pk=feed_id, is_active=True) feed = Feed.objects.get(pk=feed_id, is_active=True)
except Feed.DoesNotExist: except Feed.DoesNotExist:
return None return None
try:
result = fetch_and_ingest(feed) result = fetch_and_ingest(feed)
except TRANSIENT_FETCH_ERRORS as exc:
if self.request.retries < FETCH_MAX_RETRIES:
delay = FETCH_RETRY_BASE_DELAY_SECONDS * 2**self.request.retries
logger.warning(
"Feed %s: transient error (%s); retry %d/%d in %ds",
feed.feed_url,
type(exc).__name__,
self.request.retries + 1,
FETCH_MAX_RETRIES,
delay,
)
raise self.retry(exc=exc, countdown=delay, max_retries=FETCH_MAX_RETRIES)
_record_unexpected_failure(feed)
raise
except Exception:
# fetch_and_ingest already handles the expected failure modes
# (network errors, SSRF-blocked targets, unparseable feeds) by
# returning an error FetchResult rather than raising — anything
# that reaches here is unexpected (a parser/sanitizer bug, a DB
# error, this task's own soft time limit), so it's recorded the
# same way a handled failure would be rather than silently
# leaving the feed's status stale, the claim held for the full
# CLAIM_STALE_MINUTES, and no record of what happened. Re-raised
# afterward so Celery still marks the task FAILED (visible in
# worker logs/monitoring) instead of swallowing it.
_record_unexpected_failure(feed)
raise
feed.claimed_at = None
_schedule_next_fetch(feed)
feed.save() feed.save()
changed_article_ids = result.new_article_ids + result.updated_article_ids changed_article_ids = result.new_article_ids + result.updated_article_ids
@@ -58,7 +193,13 @@ def fetch_feed(feed_id: int):
len(result.new_article_ids), len(result.new_article_ids),
len(result.updated_article_ids), len(result.updated_article_ids),
) )
apply_filter_rules_for_feed.delay(feed_id, changed_article_ids) # Chunked so one huge poll (a first fetch of a long archive feed)
# becomes several bounded rule-evaluation tasks, each well inside
# the worker's time limits, instead of one unbounded one (C2).
for start in range(0, len(changed_article_ids), RULE_DISPATCH_CHUNK_SIZE):
apply_filter_rules_for_feed.delay(
feed_id, changed_article_ids[start : start + RULE_DISPATCH_CHUNK_SIZE]
)
return { return {
"status": result.status, "status": result.status,
@@ -67,20 +208,36 @@ def fetch_feed(feed_id: int):
} }
def reset_backoff(feed_id: int) -> None:
"""Clear a feed's error count and due-date backoff so it's retried at
its normal configured interval instead of the backed-off one
(next_fetch_at=None also makes it immediately due, same as a feed
that's never been fetched). A plain synchronous function so a caller
can finish the reset *before* dispatching the fetch — as an async task
it could land after the fetch and undo its freshly scheduled
next_fetch_at."""
Feed.objects.filter(pk=feed_id).update(fetch_error_count=0, next_fetch_at=None)
@shared_task @shared_task
def reset_feed_backoff(feed_id: int): def reset_feed_backoff(feed_id: int):
"""Admin-triggered: clear a feed's error count so it's retried at its reset_backoff(feed_id)
normal configured interval instead of the backed-off one."""
Feed.objects.filter(pk=feed_id).update(fetch_error_count=0)
@shared_task @shared_task
def import_opml_task(user_id: int, opml_content: str): def import_opml_task(user_id: int, upload_key: str):
"""Background path for large OPML imports — see OPML_ASYNC_THRESHOLD """Background path for large OPML imports — see OPML_ASYNC_THRESHOLD
in apps.feeds.views. opml_content is passed as text rather than a in apps.feeds.views. The document itself waits in the cache under
file handle since Celery tasks must be JSON-serializable.""" upload_key (set by the view) rather than travelling in the broker
message; it's removed once read, and expires on its own if this task
never runs."""
content = cache.get(upload_key)
if content is None:
logger.warning("OPML import %s expired or missing; nothing imported", upload_key)
return None
cache.delete(upload_key)
try: try:
user = get_user_model().objects.get(pk=user_id) user = get_user_model().objects.get(pk=user_id)
except get_user_model().DoesNotExist: except get_user_model().DoesNotExist:
return None return None
return import_opml(user, BytesIO(opml_content.encode("utf-8"))) return import_opml(user, BytesIO(content))
+229 -2
View File
@@ -1,3 +1,5 @@
import datetime as dt
import pytest import pytest
import requests import requests
import responses import responses
@@ -5,7 +7,13 @@ from django.utils import timezone
from apps.articles.models import Article, UserArticleState from apps.articles.models import Article, UserArticleState
from apps.feeds.models import Feed from apps.feeds.models import Feed
from apps.feeds.services.fetcher import effective_interval_minutes, fetch_and_ingest from apps.feeds.services.fetcher import (
MAX_CONTENT_HTML_CHARS,
MAX_ENTRIES_PER_POLL,
effective_interval_minutes,
fetch_and_ingest,
sanitize_html,
)
FEED_URL = "https://example.com/feed.xml" FEED_URL = "https://example.com/feed.xml"
@@ -130,7 +138,7 @@ def test_unparseable_feed_records_error_and_backs_off(feed):
assert result.status == Feed.STATUS_ERROR assert result.status == Feed.STATUS_ERROR
assert feed.fetch_error_count == 1 assert feed.fetch_error_count == 1
assert effective_interval_minutes(feed) > feed.fetch_interval_minutes assert effective_interval_minutes(feed, 60) > 60
@pytest.mark.django_db @pytest.mark.django_db
@@ -196,3 +204,222 @@ def test_refetching_updated_content_does_not_reset_user_state():
state.refresh_from_db() state.refresh_from_db()
assert state.is_read is True assert state.is_read is True
assert ArticleTag.objects.filter(user=user, article=article, tag=starred_tag).exists() assert ArticleTag.objects.filter(user=user, article=article, tag=starred_tag).exists()
@pytest.mark.django_db
@responses.activate
def test_refetching_picks_up_link_author_summary_and_published_at_changes():
"""Previously only title/content_html/content_hash were updated on
refetch (T6) — a feed correcting its link, author, summary, or
publication date without touching the title went unnoticed forever."""
feed = Feed.objects.create(feed_url=FEED_URL)
responses.add(responses.GET, FEED_URL, body=GOOD_FEED_XML, status=200)
fetch_and_ingest(feed)
feed.save()
article = Article.objects.get(feed=feed)
assert article.link == "https://example.com/1"
assert article.author == ""
edited_xml = GOOD_FEED_XML.replace(
"<link>https://example.com/1</link>",
"<link>https://example.com/1-moved</link>",
).replace(
"<pubDate>Mon, 01 Jan 2024 00:00:00 GMT</pubDate>",
"<pubDate>Tue, 02 Jan 2024 00:00:00 GMT</pubDate>\n <author>jane@example.com</author>",
)
responses.add(responses.GET, FEED_URL, body=edited_xml, status=200)
result = fetch_and_ingest(feed)
feed.save()
assert result.updated_article_ids == [article.pk]
article.refresh_from_db()
assert article.link == "https://example.com/1-moved"
assert article.author == "jane@example.com"
assert article.published_at == dt.datetime(2024, 1, 2, tzinfo=dt.timezone.utc)
def test_sanitize_html_caps_stored_output_length():
huge = "<p>" + ("a" * (MAX_CONTENT_HTML_CHARS + 10_000)) + "</p>"
cleaned = sanitize_html(huge)
assert len(cleaned) <= MAX_CONTENT_HTML_CHARS
@pytest.mark.django_db
@responses.activate
def test_fetch_only_processes_the_first_max_entries_per_poll(feed):
items = "".join(
f"<item><title>Item {i}</title><guid>guid-{i}</guid></item>"
for i in range(MAX_ENTRIES_PER_POLL + 5)
)
xml = f"""<?xml version="1.0"?>
<rss version="2.0"><channel><title>Big feed</title>{items}</channel></rss>
"""
responses.add(responses.GET, FEED_URL, body=xml, status=200)
result = fetch_and_ingest(feed)
feed.save()
assert len(result.new_article_ids) == MAX_ENTRIES_PER_POLL
assert Article.objects.filter(feed=feed).count() == MAX_ENTRIES_PER_POLL
assert not Article.objects.filter(feed=feed, guid=f"guid-{MAX_ENTRIES_PER_POLL}").exists()
def _feed_xml(count, title_suffix=""):
items = "".join(
f"<item><title>Item {i}{title_suffix}</title><guid>guid-{i}</guid></item>"
for i in range(count)
)
return (
f'<?xml version="1.0"?><rss version="2.0"><channel><title>T</title>{items}</channel></rss>'
)
@pytest.mark.django_db
@responses.activate
def test_ingest_uses_a_constant_number_of_article_queries(feed):
"""C2: one prefetch + bulk writes per poll, not a get_or_create per
entry — query count must not grow with the number of entries."""
from django.db import connection
from django.test.utils import CaptureQueriesContext
responses.add(responses.GET, FEED_URL, body=_feed_xml(40), status=200)
with CaptureQueriesContext(connection) as ctx:
result = fetch_and_ingest(feed)
assert len(result.new_article_ids) == 40
assert len(ctx.captured_queries) < 10
assert Article.objects.filter(feed=feed).count() == 40
@pytest.mark.django_db
@responses.activate
def test_ingest_dedups_a_guid_repeated_within_one_response(feed):
xml = (
'<?xml version="1.0"?><rss version="2.0"><channel><title>T</title>'
"<item><title>A</title><guid>same</guid></item>"
"<item><title>B</title><guid>same</guid></item>"
"</channel></rss>"
)
responses.add(responses.GET, FEED_URL, body=xml, status=200)
result = fetch_and_ingest(feed)
assert len(result.new_article_ids) == 1
assert Article.objects.filter(feed=feed).count() == 1
@pytest.mark.django_db
@responses.activate
def test_ingest_reports_updates_separately_from_new_articles(feed):
responses.add(responses.GET, FEED_URL, body=_feed_xml(3), status=200)
fetch_and_ingest(feed)
responses.replace(responses.GET, FEED_URL, body=_feed_xml(4, " edited"), status=200)
result = fetch_and_ingest(feed)
assert len(result.new_article_ids) == 1
assert len(result.updated_article_ids) == 3
assert Article.objects.get(feed=feed, guid="guid-0").title == "Item 0 edited"
@pytest.mark.django_db
@responses.activate
def test_ingest_falls_back_to_row_by_row_when_a_concurrent_insert_wins(feed):
from unittest.mock import patch
from django.db import IntegrityError
responses.add(responses.GET, FEED_URL, body=_feed_xml(3), status=200)
with patch.object(Article.objects, "bulk_create", side_effect=IntegrityError("race")):
result = fetch_and_ingest(feed)
assert len(result.new_article_ids) == 3
assert Article.objects.filter(feed=feed).count() == 3
@pytest.mark.django_db
@responses.activate
def test_purged_article_is_not_resurrected_by_a_feed_that_still_lists_it(feed):
"""#29: a feed still listing an entry the retention purge deleted must
not bring it back as a brand-new, unread article."""
from apps.articles.tasks import purge_old_articles
from apps.core.models import SiteSettings
SiteSettings.objects.create(retention_days=30)
responses.add(responses.GET, FEED_URL, body=_feed_xml(3), status=200)
fetch_and_ingest(feed)
Article.objects.filter(feed=feed, guid="guid-0").update(
fetched_at=timezone.now() - dt.timedelta(days=90)
)
assert purge_old_articles() == 1
assert not Article.objects.filter(feed=feed, guid="guid-0").exists()
result = fetch_and_ingest(feed)
assert result.new_article_ids == []
assert not Article.objects.filter(feed=feed, guid="guid-0").exists()
assert Article.objects.filter(feed=feed).count() == 2
@pytest.mark.django_db
@responses.activate
def test_entries_older_than_the_retention_window_are_not_ingested(feed):
"""With retention on, a feed's long archive isn't ingested only to be
purged (and tombstoned) the same night."""
from apps.core.models import SiteSettings
SiteSettings.objects.create(retention_days=30)
xml = (
'<?xml version="1.0"?><rss version="2.0"><channel><title>T</title>'
"<item><title>Ancient</title><guid>old</guid>"
"<pubDate>Mon, 01 Jan 2024 00:00:00 GMT</pubDate></item>"
"<item><title>Undated</title><guid>undated</guid></item>"
"</channel></rss>"
)
responses.add(responses.GET, FEED_URL, body=xml, status=200)
result = fetch_and_ingest(feed)
assert len(result.new_article_ids) == 1
assert Article.objects.filter(feed=feed).get().guid == "undated"
@pytest.mark.django_db
@responses.activate
def test_old_entries_are_ingested_normally_when_retention_is_off(feed):
xml = (
'<?xml version="1.0"?><rss version="2.0"><channel><title>T</title>'
"<item><title>Ancient</title><guid>old</guid>"
"<pubDate>Mon, 01 Jan 2024 00:00:00 GMT</pubDate></item>"
"</channel></rss>"
)
responses.add(responses.GET, FEED_URL, body=xml, status=200)
assert len(fetch_and_ingest(feed).new_article_ids) == 1
@pytest.mark.django_db
@responses.activate
def test_unsafe_urls_in_a_feed_are_not_stored(feed):
"""R2-S1: a feed's channel link and entry links are untrusted — only
absolute http(s) URLs may be persisted (they end up in href attributes)."""
xml = (
'<?xml version="1.0"?><rss version="2.0"><channel><title>T</title>'
"<link>javascript:alert(1)</link>"
"<item><title>Bad</title><guid>bad</guid><link>javascript:alert(2)</link></item>"
"<item><title>Relative</title><guid>rel</guid><link>/posts/1</link></item>"
"<item><title>Good</title><guid>good</guid><link>https://example.com/good</link></item>"
"</channel></rss>"
)
responses.add(responses.GET, FEED_URL, body=xml, status=200)
fetch_and_ingest(feed)
assert feed.site_url == ""
links = dict(Article.objects.filter(feed=feed).values_list("guid", "link"))
assert links["bad"] == ""
assert links["rel"] == "https://example.com/posts/1"
assert links["good"] == "https://example.com/good"
+92
View File
@@ -192,3 +192,95 @@ class TestExportOPML:
exported = export_opml(alice) exported = export_opml(alice)
assert b"<opml" in exported assert b"<opml" in exported
assert b"<outline" not in exported assert b"<outline" not in exported
class TestUnsafeOPMLUrls:
def test_non_http_feed_urls_are_skipped_and_unsafe_html_urls_blanked(self):
from io import BytesIO
from apps.feeds.opml import parse_opml
xml = b"""<?xml version="1.0"?><opml version="2.0"><head/><body>
<outline text="Bad" xmlUrl="javascript:alert(1)"/>
<outline text="FTP" xmlUrl="ftp://example.com/feed.xml"/>
<outline text="Good" xmlUrl="https://example.com/feed.xml" htmlUrl="javascript:alert(2)"/>
</body></opml>"""
entries = parse_opml(BytesIO(xml))
assert [e["feed_url"] for e in entries] == ["https://example.com/feed.xml"]
assert entries[0]["site_url"] == ""
class TestOPMLLimits:
def _opml(self, body):
from io import BytesIO
return BytesIO(
f'<?xml version="1.0"?><opml version="2.0"><head/><body>{body}</body></opml>'.encode()
)
def test_deep_nesting_is_rejected_without_recursing(self):
from apps.feeds.opml import MAX_OPML_DEPTH, parse_opml
depth = MAX_OPML_DEPTH + 50
body = '<outline text="x">' * depth + "</outline>" * depth
with pytest.raises(ValueError, match="nested too deeply"):
parse_opml(self._opml(body))
def test_a_pathologically_deep_file_does_not_crash_the_parser(self):
from apps.feeds.opml import parse_opml
depth = 50_000
body = "<outline>" * depth + "</outline>" * depth
with pytest.raises(ValueError):
parse_opml(self._opml(body))
def test_too_many_outlines_is_rejected(self, monkeypatch):
from apps.feeds import opml
monkeypatch.setattr(opml, "MAX_OPML_OUTLINES", 5)
body = '<outline text="folder"/>' * 6
with pytest.raises(ValueError, match="too many entries"):
opml.parse_opml(self._opml(body))
def test_too_many_feeds_is_rejected(self, monkeypatch):
from apps.feeds import opml
monkeypatch.setattr(opml, "MAX_OPML_FEEDS", 2)
body = "".join(f'<outline xmlUrl="https://e.example.com/{i}.xml"/>' for i in range(3))
with pytest.raises(ValueError, match="too many feeds"):
opml.parse_opml(self._opml(body))
def test_a_zip_with_too_many_files_is_rejected(self, monkeypatch):
import zipfile
from io import BytesIO
from apps.feeds import opml
monkeypatch.setattr(opml, "MAX_ZIP_ENTRIES", 3)
buffer = BytesIO()
with zipfile.ZipFile(buffer, "w") as archive:
for i in range(5):
archive.writestr(f"f{i}.txt", "x")
archive.writestr("subs.opml", "<opml/>")
with pytest.raises(ValueError, match="too many files"):
opml.extract_opml_bytes(buffer.getvalue())
def test_a_feed_outlines_children_are_not_imported_as_separate_feeds(self):
from apps.feeds.opml import parse_opml
body = (
'<outline xmlUrl="https://a.example.com/f.xml">'
'<outline xmlUrl="https://child.example.com/f.xml"/></outline>'
'<outline xmlUrl="https://b.example.com/f.xml"/>'
)
urls = [e["feed_url"] for e in parse_opml(self._opml(body))]
assert urls == ["https://a.example.com/f.xml", "https://b.example.com/f.xml"]
+209
View File
@@ -0,0 +1,209 @@
import socket
import pytest
import responses
from apps.feeds.services.ssrf_guard import (
MAX_REDIRECTS,
MAX_RESPONSE_BYTES,
SSRFBlocked,
safe_get,
validate_fetch_url,
)
# apps/feeds/conftest.py's autouse _fake_public_dns fixture pins every
# hostname to this address unless a test below overrides it directly.
PUBLIC_IP = "93.184.216.34"
def _patch_dns(monkeypatch, ip: str):
def _fake_getaddrinfo(host, *args, **kwargs):
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", (ip, 0))]
monkeypatch.setattr(socket, "getaddrinfo", _fake_getaddrinfo)
class TestValidateFetchUrl:
def test_allows_a_public_https_url(self):
validate_fetch_url("https://example.com/feed.xml")
def test_rejects_a_non_http_scheme(self):
with pytest.raises(SSRFBlocked, match="scheme"):
validate_fetch_url("file:///etc/passwd")
def test_rejects_a_url_with_no_hostname(self):
with pytest.raises(SSRFBlocked, match="hostname"):
validate_fetch_url("https:///feed.xml")
@pytest.mark.parametrize(
"ip",
[
"127.0.0.1", # loopback
"10.0.0.5", # RFC1918 private
"192.168.1.1", # RFC1918 private
"169.254.169.254", # link-local / cloud metadata
"0.0.0.0", # unspecified
"::1", # IPv6 loopback
"fc00::1", # IPv6 unique local
],
)
def test_rejects_a_url_resolving_to_a_non_public_address(self, monkeypatch, ip):
_patch_dns(monkeypatch, ip)
with pytest.raises(SSRFBlocked, match="non-public"):
validate_fetch_url("https://internal.example.com/feed.xml")
def test_rejects_a_url_whose_hostname_does_not_resolve(self, monkeypatch):
def _raise(*args, **kwargs):
raise socket.gaierror("nope")
monkeypatch.setattr(socket, "getaddrinfo", _raise)
with pytest.raises(SSRFBlocked, match="Could not resolve"):
validate_fetch_url("https://nowhere.example.com/feed.xml")
class TestSafeGet:
@responses.activate
def test_returns_the_response_body_for_a_normal_feed(self):
responses.add(responses.GET, "https://example.com/feed.xml", body="<rss></rss>")
response = safe_get("https://example.com/feed.xml", headers={}, timeout=5)
assert response.status_code == 200
assert response.content == b"<rss></rss>"
@responses.activate
def test_follows_a_safe_redirect(self):
responses.add(
responses.GET,
"https://example.com/feed.xml",
status=302,
headers={"Location": "https://example.com/final.xml"},
)
responses.add(responses.GET, "https://example.com/final.xml", body="<rss>final</rss>")
response = safe_get("https://example.com/feed.xml", headers={}, timeout=5)
assert response.content == b"<rss>final</rss>"
@responses.activate
def test_blocks_a_redirect_to_a_private_address(self, monkeypatch):
responses.add(
responses.GET,
"https://example.com/feed.xml",
status=302,
headers={"Location": "http://internal.example.com/secret"},
)
real_getaddrinfo = socket.getaddrinfo
def _fake_getaddrinfo(host, *args, **kwargs):
if host == "internal.example.com":
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("127.0.0.1", 0))]
return real_getaddrinfo(host, *args, **kwargs)
monkeypatch.setattr(socket, "getaddrinfo", _fake_getaddrinfo)
with pytest.raises(SSRFBlocked, match="non-public"):
safe_get("https://example.com/feed.xml", headers={}, timeout=5)
@responses.activate
def test_blocks_a_redirect_loop_past_max_redirects(self):
for _ in range(MAX_REDIRECTS + 2):
responses.add(
responses.GET,
"https://example.com/feed.xml",
status=302,
headers={"Location": "https://example.com/feed.xml"},
)
with pytest.raises(SSRFBlocked, match="Too many redirects"):
safe_get("https://example.com/feed.xml", headers={}, timeout=5)
@responses.activate
def test_blocks_a_response_over_the_size_cap(self):
responses.add(
responses.GET,
"https://example.com/feed.xml",
body=b"x" * (MAX_RESPONSE_BYTES + 1),
)
with pytest.raises(SSRFBlocked, match="exceeded"):
safe_get("https://example.com/feed.xml", headers={}, timeout=5)
@responses.activate
def test_preserves_case_insensitive_header_lookup(self):
responses.add(
responses.GET,
"https://example.com/feed.xml",
body="<rss></rss>",
headers={"etag": "abc123"},
)
response = safe_get("https://example.com/feed.xml", headers={}, timeout=5)
assert response.headers.get("ETag") == "abc123"
class TestConnectionPinning:
"""R2-S2: the address that's validated is the address that's connected
to — a hostname that resolves differently at connect time can't slip an
internal address past a pre-check."""
def test_a_rebinding_dns_answer_at_connect_time_is_blocked(self, monkeypatch):
import socket
calls = []
def rebinding_getaddrinfo(host, *args, **kwargs):
calls.append(host)
ip = "93.184.216.34" if len(calls) == 1 else "127.0.0.1"
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", (ip, 0))]
monkeypatch.setattr(socket, "getaddrinfo", rebinding_getaddrinfo)
# The up-front check sees the public answer...
validate_fetch_url("http://rebind.example.com/feed.xml")
# ...but the connection's own lookup gets the internal one and must refuse.
with pytest.raises(SSRFBlocked, match="non-public"):
safe_get("http://rebind.example.com/feed.xml", headers={}, timeout=2)
def test_connects_to_the_validated_address_and_keeps_the_original_host_header(
self, monkeypatch
):
import socket
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer
from apps.feeds.services import ssrf_guard
seen = {}
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
seen["host"] = self.headers["Host"]
body = b"<rss/>"
self.send_response(200)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *args):
pass
server = HTTPServer(("127.0.0.1", 0), Handler)
port = server.server_address[1]
threading.Thread(target=server.handle_request, daemon=True).start()
real_getaddrinfo = socket.getaddrinfo
monkeypatch.setattr(
socket,
"getaddrinfo",
lambda host, *a, **k: (
[(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("127.0.0.1", port))]
if host == "feed.test"
else real_getaddrinfo(host, *a, **k)
),
)
# Allow loopback for this test only, standing in for a public address.
monkeypatch.setattr(ssrf_guard, "_is_allowed_ip", lambda ip: True)
response = safe_get(f"http://feed.test:{port}/feed.xml", headers={}, timeout=5)
server.server_close()
assert response.status_code == 200
assert response.content == b"<rss/>"
assert seen["host"] == f"feed.test:{port}"
+50 -3
View File
@@ -173,9 +173,9 @@ def test_import_opml_dispatches_background_task_above_threshold(client, alice, m
dispatched = {} dispatched = {}
def fake_delay(user_id, content): def fake_delay(user_id, upload_key):
dispatched["user_id"] = user_id dispatched["user_id"] = user_id
dispatched["content"] = content dispatched["upload_key"] = upload_key
from . import views as feeds_views from . import views as feeds_views
@@ -187,6 +187,11 @@ def test_import_opml_dispatches_background_task_above_threshold(client, alice, m
assert response.status_code == 302 assert response.status_code == 302
assert dispatched["user_id"] == alice.id assert dispatched["user_id"] == alice.id
# The task gets a reference to the document, not the document itself.
from django.core.cache import cache
assert dispatched["upload_key"].startswith("opml-import:")
assert cache.get(dispatched["upload_key"]) == SIMPLE_OPML
# The background task was dispatched instead of a synchronous import. # The background task was dispatched instead of a synchronous import.
assert not Subscription.objects.filter(user=alice).exists() assert not Subscription.objects.filter(user=alice).exists()
@@ -204,9 +209,51 @@ def test_export_opml_view_returns_downloadable_file(client, alice, alices_subscr
@pytest.mark.django_db @pytest.mark.django_db
def test_import_opml_task_runs_import(alice): def test_import_opml_task_runs_import(alice):
from django.core.cache import cache
from .tasks import import_opml_task from .tasks import import_opml_task
result = import_opml_task(alice.id, SIMPLE_OPML.decode("utf-8")) cache.set("opml-import:test", SIMPLE_OPML, timeout=60)
result = import_opml_task(alice.id, "opml-import:test")
assert result["created"] == 2 assert result["created"] == 2
assert Subscription.objects.filter(user=alice).count() == 2 assert Subscription.objects.filter(user=alice).count() == 2
assert cache.get("opml-import:test") is None, "the stored upload is removed once read"
@pytest.mark.django_db
def test_import_opml_task_does_nothing_if_the_upload_expired(alice):
from .tasks import import_opml_task
assert import_opml_task(alice.id, "opml-import:gone") is None
assert not Subscription.objects.filter(user=alice).exists()
@pytest.mark.django_db
def test_import_opml_rejects_an_oversized_upload(client, alice, monkeypatch):
from . import forms
monkeypatch.setattr(forms, "MAX_UPLOAD_BYTES", 100)
client.login(username="alice", password="pw-alice-123")
upload = SimpleUploadedFile("big.opml", SIMPLE_OPML, content_type="text/x-opml")
response = client.post(reverse("feeds:import_opml"), {"opml_file": upload})
assert response.status_code == 200
assert "too large" in response.content.decode()
assert not Subscription.objects.filter(user=alice).exists()
@pytest.mark.django_db
def test_import_opml_reports_a_limit_error_instead_of_a_generic_one(client, alice, monkeypatch):
from . import opml
monkeypatch.setattr(opml, "MAX_OPML_FEEDS", 1)
client.login(username="alice", password="pw-alice-123")
upload = SimpleUploadedFile("subs.opml", SIMPLE_OPML, content_type="text/x-opml")
response = client.post(reverse("feeds:import_opml"), {"opml_file": upload}, follow=True)
assert "too many feeds" in response.content.decode()
assert not Subscription.objects.filter(user=alice).exists()
+331 -3
View File
@@ -3,6 +3,7 @@ from unittest.mock import patch
import pytest import pytest
import responses import responses
from django.urls import reverse
from django.utils import timezone from django.utils import timezone
from apps.articles.models import Article from apps.articles.models import Article
@@ -12,21 +13,23 @@ from apps.feeds.tasks import fetch_feed, poll_due_feeds
@pytest.mark.django_db @pytest.mark.django_db
def test_poll_due_feeds_dispatches_only_due_feeds(): def test_poll_due_feeds_dispatches_only_due_feeds():
# next_fetch_at (set by fetch_feed itself, C1), not last_fetched_at,
# is what poll_due_feeds' query filters on now.
never_fetched = Feed.objects.create(feed_url="https://a.example.com/feed.xml") never_fetched = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
recently_fetched = Feed.objects.create( recently_fetched = Feed.objects.create(
feed_url="https://b.example.com/feed.xml", feed_url="https://b.example.com/feed.xml",
last_fetched_at=timezone.now(), last_fetched_at=timezone.now(),
fetch_interval_minutes=60, next_fetch_at=timezone.now() + timedelta(hours=1),
) )
overdue = Feed.objects.create( overdue = Feed.objects.create(
feed_url="https://c.example.com/feed.xml", feed_url="https://c.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2), last_fetched_at=timezone.now() - timedelta(hours=2),
fetch_interval_minutes=60, next_fetch_at=timezone.now() - timedelta(hours=1),
) )
inactive_but_overdue = Feed.objects.create( inactive_but_overdue = Feed.objects.create(
feed_url="https://d.example.com/feed.xml", feed_url="https://d.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2), last_fetched_at=timezone.now() - timedelta(hours=2),
fetch_interval_minutes=60, next_fetch_at=timezone.now() - timedelta(hours=1),
is_active=False, is_active=False,
) )
@@ -40,6 +43,58 @@ def test_poll_due_feeds_dispatches_only_due_feeds():
assert inactive_but_overdue.id not in dispatched_ids assert inactive_but_overdue.id not in dispatched_ids
@pytest.mark.django_db
def test_poll_due_feeds_does_not_reclaim_a_feed_it_just_claimed():
# Models a second beat tick (or an overlapping worker) running
# poll_due_feeds before the first claimed fetch_feed has finished —
# the feed must only be dispatched once.
overdue = Feed.objects.create(
feed_url="https://c.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2),
next_fetch_at=timezone.now() - timedelta(hours=1),
)
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
first_count = poll_due_feeds()
second_count = poll_due_feeds()
assert first_count == 1
assert second_count == 0
assert mock_delay.call_count == 1
overdue.refresh_from_db()
assert overdue.claimed_at is not None
@pytest.mark.django_db
def test_poll_due_feeds_reclaims_a_stale_claim():
from apps.feeds.tasks import CLAIM_STALE_MINUTES
overdue = Feed.objects.create(
feed_url="https://c.example.com/feed.xml",
last_fetched_at=timezone.now() - timedelta(hours=2),
next_fetch_at=timezone.now() - timedelta(hours=1),
claimed_at=timezone.now() - timedelta(minutes=CLAIM_STALE_MINUTES + 1),
)
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
dispatched_count = poll_due_feeds()
assert dispatched_count == 1
assert mock_delay.call_args.args[0] == overdue.id
@pytest.mark.django_db
@responses.activate
def test_fetch_feed_task_clears_its_claim():
feed = Feed.objects.create(feed_url="https://example.com/feed.xml", claimed_at=timezone.now())
responses.add(responses.GET, feed.feed_url, body=GOOD_FEED_XML, status=200)
fetch_feed(feed.id)
feed.refresh_from_db()
assert feed.claimed_at is None
GOOD_FEED_XML = """<?xml version="1.0"?> GOOD_FEED_XML = """<?xml version="1.0"?>
<rss version="2.0"><channel> <rss version="2.0"><channel>
<title>Example Blog</title> <title>Example Blog</title>
@@ -75,3 +130,276 @@ def test_fetch_feed_task_skips_inactive_feed():
@pytest.mark.django_db @pytest.mark.django_db
def test_fetch_feed_task_skips_missing_feed(): def test_fetch_feed_task_skips_missing_feed():
assert fetch_feed(999999) is None assert fetch_feed(999999) is None
@pytest.mark.django_db
def test_fetch_feed_task_records_error_and_clears_claim_on_unexpected_exception():
feed = Feed.objects.create(
feed_url="https://example.com/feed.xml",
claimed_at=timezone.now(),
fetch_error_count=0,
)
with patch("apps.feeds.tasks.fetch_and_ingest", side_effect=RuntimeError("boom")):
with pytest.raises(RuntimeError, match="boom"):
fetch_feed(feed.id)
feed.refresh_from_db()
assert feed.last_fetch_status == Feed.STATUS_ERROR
assert feed.fetch_error_count == 1
assert feed.claimed_at is None
assert feed.last_fetched_at is not None
assert feed.next_fetch_at is not None
@pytest.mark.django_db
@responses.activate
def test_fetch_feed_task_sets_next_fetch_at_so_poll_due_feeds_can_find_it_later():
"""next_fetch_at (C1) is what poll_due_feeds' query filters on — a
feed fetch_feed just succeeded on must come back due again only after
the configured interval, not immediately."""
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
responses.add(responses.GET, feed.feed_url, body=GOOD_FEED_XML, status=200)
fetch_feed(feed.id)
feed.refresh_from_db()
assert feed.next_fetch_at is not None
assert feed.next_fetch_at > timezone.now()
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
poll_due_feeds()
mock_delay.assert_not_called()
@pytest.mark.django_db
def test_reset_feed_backoff_makes_the_feed_immediately_due():
from apps.feeds.tasks import reset_feed_backoff
feed = Feed.objects.create(
feed_url="https://example.com/feed.xml",
fetch_error_count=5,
last_fetched_at=timezone.now(),
next_fetch_at=timezone.now() + timedelta(hours=12),
)
reset_feed_backoff(feed.id)
feed.refresh_from_db()
assert feed.fetch_error_count == 0
assert feed.next_fetch_at is None
@pytest.mark.django_db
@responses.activate
def test_fetch_feed_schedules_the_next_fetch_using_the_site_wide_interval():
"""fetch_feed (not poll_due_feeds — see apps.feeds.tasks.
_schedule_next_fetch, C1) is what reads SiteSettings' site-wide
interval and sets Feed.next_fetch_at from it after every fetch."""
from apps.core.models import SiteSettings
SiteSettings.objects.create(fetch_interval_minutes=120)
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
responses.add(responses.GET, feed.feed_url, body=GOOD_FEED_XML, status=200)
before = timezone.now()
fetch_feed(feed.id)
feed.refresh_from_db()
assert feed.next_fetch_at is not None
expected = before + timedelta(minutes=120)
assert abs((feed.next_fetch_at - expected).total_seconds()) < 5
@pytest.mark.django_db
@responses.activate
def test_fetch_feed_dispatches_rule_evaluation_in_bounded_chunks(monkeypatch):
monkeypatch.setattr("apps.feeds.tasks.RULE_DISPATCH_CHUNK_SIZE", 2)
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
items = "".join(f"<item><title>T{i}</title><guid>g{i}</guid></item>" for i in range(5))
responses.add(
responses.GET,
feed.feed_url,
body=f'<?xml version="1.0"?><rss version="2.0"><channel>{items}</channel></rss>',
status=200,
)
with patch("apps.feeds.tasks.apply_filter_rules_for_feed.delay") as mock_delay:
fetch_feed(feed.id)
chunk_sizes = [len(call.args[1]) for call in mock_delay.call_args_list]
assert chunk_sizes == [2, 2, 1]
@pytest.mark.django_db
def test_enqueue_fetch_claims_the_feed_and_queues_once():
from apps.feeds.tasks import enqueue_fetch
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
first = enqueue_fetch(feed.id)
second = enqueue_fetch(feed.id)
assert (first, second) == (True, False)
mock_delay.assert_called_once_with(feed.id)
@pytest.mark.django_db
def test_the_poller_does_not_dispatch_a_feed_a_direct_enqueue_already_claimed():
"""T1: a just-added feed (next_fetch_at NULL, i.e. due) is claimed by
the add-feed path, so the periodic poller must not fetch it too."""
from apps.feeds.tasks import enqueue_fetch
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
enqueue_fetch(feed.id)
dispatched = poll_due_feeds()
assert dispatched == 0
assert mock_delay.call_count == 1
@pytest.mark.django_db
def test_enqueue_fetch_releases_the_claim_if_queueing_fails():
from apps.feeds.tasks import enqueue_fetch
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
with patch("apps.feeds.tasks.fetch_feed.delay", side_effect=ConnectionError("broker down")):
with pytest.raises(ConnectionError):
enqueue_fetch(feed.id)
feed.refresh_from_db()
assert feed.claimed_at is None
@pytest.mark.django_db
def test_poller_survives_a_broker_failure_and_leaves_feeds_unclaimed():
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
with patch("apps.feeds.tasks.fetch_feed.delay", side_effect=ConnectionError("broker down")):
assert poll_due_feeds() == 0
feed.refresh_from_db()
assert feed.claimed_at is None
@pytest.mark.django_db
def test_add_feed_claims_its_initial_fetch(client):
from django.contrib.auth import get_user_model
get_user_model().objects.create_user("alice", password="pw-alice-123")
client.login(username="alice", password="pw-alice-123")
with patch("apps.feeds.tasks.fetch_feed.delay") as mock_delay:
client.post(reverse("feeds:add_feed"), {"feed_url": "https://example.com/new.xml"})
poll_due_feeds()
assert mock_delay.call_count == 1
@pytest.mark.django_db
def test_add_feed_still_subscribes_when_the_broker_is_down(client):
from django.contrib.auth import get_user_model
from apps.feeds.models import Subscription
user = get_user_model().objects.create_user("alice", password="pw-alice-123")
client.login(username="alice", password="pw-alice-123")
with patch("apps.feeds.tasks.fetch_feed.delay", side_effect=ConnectionError("broker down")):
response = client.post(
reverse("feeds:add_feed"), {"feed_url": "https://example.com/new.xml"}
)
assert response.status_code == 302
assert Subscription.objects.filter(
user=user, feed__feed_url="https://example.com/new.xml"
).exists()
class TestTransientFailureRetries:
"""R2-T5: a locked database or dropped connection is retried promptly
with bounded backoff; everything else keeps the record-and-back-off
behavior."""
@pytest.mark.django_db
def test_a_transient_database_error_is_retried_without_penalizing_the_feed(self):
from celery.exceptions import Retry
from django.db import OperationalError
feed = Feed.objects.create(
feed_url="https://example.com/feed.xml", claimed_at=timezone.now()
)
with (
patch("apps.feeds.tasks.fetch_and_ingest", side_effect=OperationalError("locked")),
patch.object(fetch_feed, "retry", side_effect=Retry()) as mock_retry,
):
with pytest.raises(Retry):
fetch_feed(feed.id)
assert mock_retry.call_args.kwargs["countdown"] == 30
feed.refresh_from_db()
assert feed.fetch_error_count == 0, "the feed wasn't at fault"
assert feed.claimed_at is not None, "claim is held so the poller can't double-fetch"
@pytest.mark.django_db
def test_backoff_doubles_each_attempt(self):
from celery.exceptions import Retry
from django.db import InterfaceError
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
delays = []
for attempt in range(3):
fetch_feed.push_request(retries=attempt)
try:
with (
patch("apps.feeds.tasks.fetch_and_ingest", side_effect=InterfaceError("gone")),
patch.object(fetch_feed, "retry", side_effect=Retry()) as mock_retry,
):
with pytest.raises(Retry):
fetch_feed(feed.id)
delays.append(mock_retry.call_args.kwargs["countdown"])
finally:
fetch_feed.pop_request()
assert delays == [30, 60, 120]
@pytest.mark.django_db
def test_after_the_retries_are_spent_it_is_recorded_as_a_normal_failure(self):
from django.db import OperationalError
from apps.feeds.tasks import FETCH_MAX_RETRIES
feed = Feed.objects.create(
feed_url="https://example.com/feed.xml", claimed_at=timezone.now()
)
fetch_feed.push_request(retries=FETCH_MAX_RETRIES)
try:
with patch("apps.feeds.tasks.fetch_and_ingest", side_effect=OperationalError("locked")):
with pytest.raises(OperationalError):
fetch_feed(feed.id)
finally:
fetch_feed.pop_request()
feed.refresh_from_db()
assert feed.fetch_error_count == 1
assert feed.claimed_at is None
assert feed.next_fetch_at is not None
@pytest.mark.django_db
def test_other_exceptions_are_not_retried(self):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
with (
patch("apps.feeds.tasks.fetch_and_ingest", side_effect=RuntimeError("parser bug")),
patch.object(fetch_feed, "retry") as mock_retry,
):
with pytest.raises(RuntimeError):
fetch_feed(feed.id)
mock_retry.assert_not_called()
+19 -3
View File
@@ -54,7 +54,11 @@ GOOD_FEED_XML = """<?xml version="1.0"?>
@pytest.mark.django_db @pytest.mark.django_db
@responses.activate @responses.activate
def test_add_feed_view_subscribes_and_fetches_immediately(client, two_users): def test_add_feed_view_subscribes_and_enqueues_the_initial_fetch(client, two_users):
"""The subscription is created synchronously; the first fetch is
dispatched via fetch_feed.delay() (T3) rather than run inline — in
tests CELERY_TASK_ALWAYS_EAGER still runs it immediately, so the
article it ingests is visible by the time the response returns."""
alice, _ = two_users alice, _ = two_users
client.login(username="alice", password="pw-alice-123") client.login(username="alice", password="pw-alice-123")
responses.add(responses.GET, "https://example.com/feed.xml", body=GOOD_FEED_XML, status=200) responses.add(responses.GET, "https://example.com/feed.xml", body=GOOD_FEED_XML, status=200)
@@ -62,9 +66,10 @@ def test_add_feed_view_subscribes_and_fetches_immediately(client, two_users):
response = client.post(reverse("feeds:add_feed"), {"feed_url": "https://example.com/feed.xml"}) response = client.post(reverse("feeds:add_feed"), {"feed_url": "https://example.com/feed.xml"})
assert response.status_code == 302 assert response.status_code == 302
assert Subscription.objects.filter( subscription = Subscription.objects.get(
user=alice, feed__feed_url="https://example.com/feed.xml" user=alice, feed__feed_url="https://example.com/feed.xml"
).exists() )
assert subscription.feed.articles.filter(guid="guid-1").exists()
@pytest.mark.django_db @pytest.mark.django_db
@@ -112,3 +117,14 @@ def test_add_feed_view_rejects_duplicate_subscription(client, two_users):
assert response.status_code == 302 assert response.status_code == 302
assert Subscription.objects.filter(user=alice, feed=feed).count() == 1 assert Subscription.objects.filter(user=alice, feed=feed).count() == 1
@pytest.mark.django_db
@pytest.mark.parametrize("url", ["ftp://example.com/feed.xml", "javascript:alert(1)"])
def test_add_feed_rejects_non_http_urls(client, two_users, url):
client.login(username="alice", password="pw-alice-123")
response = client.post(reverse("feeds:add_feed"), {"feed_url": url})
assert response.status_code == 200
assert not Feed.objects.filter(feed_url=url).exists()
+36 -19
View File
@@ -1,7 +1,10 @@
import logging
import uuid
from io import BytesIO from io import BytesIO
from django.contrib import messages from django.contrib import messages
from django.contrib.auth.decorators import login_required from django.contrib.auth.decorators import login_required
from django.core.cache import cache
from django.http import HttpResponse from django.http import HttpResponse
from django.shortcuts import get_object_or_404, redirect, render from django.shortcuts import get_object_or_404, redirect, render
from django.views.decorators.http import require_POST from django.views.decorators.http import require_POST
@@ -11,11 +14,13 @@ from apps.articles.models import SubscriptionTag, Tag
from .forms import AddFeedForm, EditSubscriptionForm, OPMLImportForm from .forms import AddFeedForm, EditSubscriptionForm, OPMLImportForm
from .models import Feed, Subscription from .models import Feed, Subscription
from .opml import export_opml, extract_opml_bytes, import_opml, parse_opml from .opml import MAX_UPLOAD_BYTES, export_opml, extract_opml_bytes, import_opml, parse_opml
from .services.fetcher import fetch_and_ingest from .tasks import enqueue_fetch, import_opml_task
from .tasks import import_opml_task
logger = logging.getLogger(__name__)
OPML_ASYNC_THRESHOLD = 50 OPML_ASYNC_THRESHOLD = 50
OPML_UPLOAD_TTL_SECONDS = 60 * 60
@login_required @login_required
@@ -39,22 +44,21 @@ def add_feed(request):
user=request.user, subscription=subscription, tag=tag user=request.user, subscription=subscription, tag=tag
) )
# Fetch synchronously so the user sees results immediately; the # Enqueue the initial fetch instead of running it inline (T3) — a
# periodic Celery poll takes over from here for subsequent fetches. # slow or large feed would otherwise tie up a Gunicorn worker for
result = fetch_and_ingest(feed) # the duration of the request; fetch_feed handles the save/error
feed.save() # bookkeeping the same way the periodic poll's dispatch does.
try:
if result.status == Feed.STATUS_ERROR: enqueue_fetch(feed.pk)
messages.warning( except Exception:
request, # The subscription itself is saved; with the broker down the
f"Subscribed to {feed_url}, but the initial fetch failed: {result.error_message}", # periodic poller picks the (never-fetched) feed up once it's
) # back, so this isn't worth failing the request over.
else: logger.exception("Could not queue the initial fetch for feed %s", feed.pk)
article_count = len(result.new_article_ids)
messages.success( messages.success(
request, request,
f"Subscribed to '{subscription.display_title}' — " f"Subscribed to {feed_url}. Fetching its articles now — "
f"{article_count} article(s) fetched.", "refresh in a few seconds if you don't see them yet.",
) )
return redirect("reader:stream") return redirect("reader:stream")
@@ -148,16 +152,29 @@ def unsubscribe(request, subscription_id):
def import_opml_view(request): def import_opml_view(request):
form = OPMLImportForm(request.POST or None, request.FILES or None) form = OPMLImportForm(request.POST or None, request.FILES or None)
if request.method == "POST" and form.is_valid(): if request.method == "POST" and form.is_valid():
uploaded = form.cleaned_data["opml_file"].read() uploaded = form.cleaned_data["opml_file"].read(MAX_UPLOAD_BYTES + 1)
try: try:
content = extract_opml_bytes(uploaded) content = extract_opml_bytes(uploaded)
entry_count = len(parse_opml(BytesIO(content))) entry_count = len(parse_opml(BytesIO(content)))
except ValueError as exc:
# Our own limit/format errors carry a user-presentable message.
messages.error(request, str(exc))
return redirect("feeds:import_opml")
except Exception: except Exception:
messages.error(request, "That file doesn't look like a valid OPML export.") messages.error(request, "That file doesn't look like a valid OPML export.")
return redirect("feeds:import_opml") return redirect("feeds:import_opml")
if entry_count > OPML_ASYNC_THRESHOLD: if entry_count > OPML_ASYNC_THRESHOLD:
import_opml_task.delay(request.user.id, content.decode("utf-8")) # Hand the worker a reference, not the document: a multi-MB
# file doesn't belong inside a broker message.
upload_key = f"opml-import:{uuid.uuid4().hex}"
try:
cache.set(upload_key, content, timeout=OPML_UPLOAD_TTL_SECONDS)
import_opml_task.delay(request.user.id, upload_key)
except Exception:
logger.exception("Could not queue the OPML import")
messages.error(request, "Couldn't queue that import right now — try again shortly.")
return redirect("feeds:import_opml")
messages.success(request, f"Importing {entry_count} feeds in the background.") messages.success(request, f"Importing {entry_count} feeds in the background.")
else: else:
result = import_opml(request.user, BytesIO(content)) result = import_opml(request.user, BytesIO(content))
+38
View File
@@ -0,0 +1,38 @@
import hashlib
import hmac
from django.conf import settings
AUTH_MARKER_HEADER = "X-Evection-Auth"
ANONYMOUS_MARKER = "anon"
def auth_marker(user) -> str:
"""An opaque, stable identifier for who is logged in — "anon" if no
one. Derived with an HMAC of the user's pk under SECRET_KEY, so it's
constant for one user but reveals neither the pk nor anything
guessable."""
if user is None or not user.is_authenticated:
return ANONYMOUS_MARKER
digest = hmac.new(
settings.SECRET_KEY.encode(), f"evection-sw-auth:{user.pk}".encode(), hashlib.sha256
).hexdigest()
return digest[:16]
class AuthMarkerMiddleware:
"""Stamps every HTML response with who it was rendered for. The service
worker (templates/pwa/sw.js) caches authenticated pages for offline
reading in a single cache; it compares this marker on every navigation
and purges that cache whenever it changes — session expiry (the login
page comes back as "anon"), logging in, switching accounts — so one
session's pages can't be served to the next (R2-S3)."""
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
response = self.get_response(request)
if response.get("Content-Type", "").startswith("text/html"):
response[AUTH_MARKER_HEADER] = auth_marker(getattr(request, "user", None))
return response
+44
View File
@@ -32,6 +32,11 @@ def test_service_worker_is_served_at_root_with_correct_type_and_scope(client):
assert 'self.addEventListener("fetch"' in body assert 'self.addEventListener("fetch"' in body
# Mutations must stay network-only, never intercepted. # Mutations must stay network-only, never intercepted.
assert 'request.method !== "GET"' in body assert 'request.method !== "GET"' in body
# Logging out must purge PAGES_CACHE so a later offline navigation on
# a shared/account-switched device can't serve another user's
# previously cached authenticated pages.
assert "caches.delete(PAGES_CACHE)" in body
assert reverse("accounts:logout") in body
@pytest.mark.django_db @pytest.mark.django_db
@@ -61,3 +66,42 @@ def test_base_template_links_manifest_and_registers_service_worker(client, djang
assert 'rel="manifest"' in content assert 'rel="manifest"' in content
assert "register-sw.js" in content assert "register-sw.js" in content
assert "apple-touch-icon" in content assert "apple-touch-icon" in content
@pytest.mark.django_db
def test_html_responses_are_stamped_with_who_they_were_rendered_for(client, django_user_model):
"""R2-S3: the service worker purges its page cache whenever this marker
changes, so it has to be stable per user, different between users,
and "anon" when logged out."""
django_user_model.objects.create_user("alice", password="pw-alice-123")
django_user_model.objects.create_user("bob", password="pw-bob-123")
anon = client.get(reverse("accounts:login"))["X-Evection-Auth"]
assert anon == "anon"
client.login(username="alice", password="pw-alice-123")
alice_1 = client.get(reverse("reader:stream"))["X-Evection-Auth"]
alice_2 = client.get(reverse("reader:stream"))["X-Evection-Auth"]
client.logout()
client.login(username="bob", password="pw-bob-123")
bob = client.get(reverse("reader:stream"))["X-Evection-Auth"]
assert alice_1 == alice_2
assert len({anon, alice_1, bob}) == 3
assert "alice" not in alice_1 and not alice_1.isdigit()
@pytest.mark.django_db
def test_non_html_responses_are_not_stamped(client, django_user_model):
django_user_model.objects.create_user("alice", password="pw-alice-123")
client.login(username="alice", password="pw-alice-123")
assert "X-Evection-Auth" not in client.get(reverse("core:health"))
def test_service_worker_purges_its_page_cache_when_the_auth_marker_changes(client):
body = client.get(reverse("pwa:service_worker")).content.decode()
assert "X-Evection-Auth" in body
assert "caches.delete(PAGES_CACHE)" in body
assert "previous !== marker" in body
+420
View File
@@ -1,3 +1,5 @@
import re
import pytest import pytest
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.urls import reverse from django.urls import reverse
@@ -164,3 +166,421 @@ def test_saved_filter_only_shows_saved_articles(client, alice, alices_article):
response = client.get(reverse("reader:stream"), {"filter": "starred"}) response = client.get(reverse("reader:stream"), {"filter": "starred"})
assert alices_article.title not in response.content.decode() assert alices_article.title not in response.content.decode()
@pytest.mark.django_db
def test_sidebar_shows_a_tags_total_unread_count(client, alice):
from apps.articles.models import SubscriptionTag, Tag
feed_a = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
feed_b = Feed.objects.create(feed_url="https://b.example.com/feed.xml")
sub_a = Subscription.objects.create(user=alice, feed=feed_a)
sub_b = Subscription.objects.create(user=alice, feed=feed_b)
tag = Tag.objects.create(user=alice, name="work")
SubscriptionTag.objects.create(user=alice, subscription=sub_a, tag=tag)
SubscriptionTag.objects.create(user=alice, subscription=sub_b, tag=tag)
for i in range(3):
Article.objects.create(feed=feed_a, guid=f"a-{i}", title=f"A{i}")
for i in range(4):
Article.objects.create(feed=feed_b, guid=f"b-{i}", title=f"B{i}")
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
# The only "7" on the page — the individual feeds show 3 and 4.
content = response.content.decode()
assert '<span class="unread-count">7</span>' in content
@pytest.mark.django_db
def test_article_rows_carry_their_utc_instant_for_client_side_date_grouping(client, alice):
"""The server still groups date headers in UTC, but ships each row's
raw UTC instant so date-groups.js can rebuild the headers using the
viewer's local calendar day instead."""
from django.utils import timezone
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
dated = Article.objects.create(
feed=feed, guid="dated", title="Dated", published_at=timezone.now()
)
undated = Article.objects.create(feed=feed, guid="undated", title="No publish date")
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"), {"filter": "all"})
content = response.content.decode()
expected_iso = dated.published_at.isoformat()
assert f'id="article-{dated.pk}"' in content
assert f'data-published-utc="{expected_iso}"' in content
# A feed that gave no publish date falls back to when it was fetched
# (the same fallback the stream's sort order uses), so the row still
# carries a time and groups under a real date, not "Undated".
undated_row = re.search(rf'<li id="article-{undated.pk}".*?</li>', content, flags=re.DOTALL)
assert undated_row is not None
row_html = undated_row.group(0)
assert f'data-published-utc="{undated.fetched_at.isoformat()}"' in row_html
assert 'class="article-time"' in row_html
assert "Undated" not in content
@pytest.mark.django_db
def test_check_new_articles_reports_nothing_below_the_given_cursor(client, alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
existing = Article.objects.create(feed=feed, guid="existing", title="Existing")
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:check_new"), {"filter": "all", "since": existing.pk})
assert response.status_code == 200
assert response.content.decode().strip() == ""
@pytest.mark.django_db
def test_check_new_articles_reports_articles_created_after_the_cursor(client, alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
existing = Article.objects.create(feed=feed, guid="existing", title="Existing")
Article.objects.create(feed=feed, guid="new-1", title="New one")
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:check_new"), {"filter": "all", "since": existing.pk})
content = response.content.decode()
assert "1 new article" in content
assert "click to show" in content
@pytest.mark.django_db
def test_check_new_articles_respects_the_active_tag_scope(client, alice):
from apps.articles.models import SubscriptionTag, Tag
feed_a = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
feed_b = Feed.objects.create(feed_url="https://b.example.com/feed.xml")
sub_a = Subscription.objects.create(user=alice, feed=feed_a)
Subscription.objects.create(user=alice, feed=feed_b)
tag = Tag.objects.create(user=alice, name="work")
SubscriptionTag.objects.create(user=alice, subscription=sub_a, tag=tag)
cursor = Article.objects.create(feed=feed_a, guid="cursor", title="Cursor").pk
# New in feed_b, which isn't tagged "work" — shouldn't count for this scope.
Article.objects.create(feed=feed_b, guid="untagged-new", title="Untagged new")
client.login(username="alice", password="pw-alice-123")
response = client.get(
reverse("reader:check_new"), {"filter": "all", "tag": tag.pk, "since": cursor}
)
assert response.content.decode().strip() == ""
# New in feed_a (tagged "work") — should count.
Article.objects.create(feed=feed_a, guid="tagged-new", title="Tagged new")
response = client.get(
reverse("reader:check_new"), {"filter": "all", "tag": tag.pk, "since": cursor}
)
assert "1 new article" in response.content.decode()
@pytest.mark.django_db
def test_stream_exposes_the_latest_article_id_and_scope_for_the_banner(client, alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
article = Article.objects.create(feed=feed, guid="g1", title="Hello")
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"), {"filter": "all"})
content = response.content.decode()
assert f"since={article.pk}" in content
assert "filter=all" in content
@pytest.mark.django_db
def test_sidebar_unread_count_updates_immediately_after_reading_an_article(client, alice):
"""unread_counts_by_feed caches its result for a short TTL (C3) — the
reader views that change the current user's own read state must
invalidate it, or the sidebar would show a stale count until the TTL
elapses on every one of these very common actions."""
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
article = Article.objects.create(feed=feed, guid="a-1", title="Hello")
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
assert '<span class="unread-count">1</span>' in response.content.decode()
client.get(reverse("reader:article_detail", args=[article.pk]))
response = client.get(reverse("reader:stream"))
assert "unread-count" not in response.content.decode()
@pytest.mark.django_db
def test_sidebar_unread_count_updates_immediately_after_toggle_read(client, alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
article = Article.objects.create(feed=feed, guid="a-1", title="Hello")
UserArticleState.objects.create(user=alice, article=article, is_read=True)
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
assert "unread-count" not in response.content.decode()
client.post(reverse("reader:toggle_read", args=[article.pk]))
response = client.get(reverse("reader:stream"))
assert '<span class="unread-count">1</span>' in response.content.decode()
@pytest.mark.django_db
def test_sidebar_unread_count_updates_immediately_after_mark_all_read(client, alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
Article.objects.create(feed=feed, guid="a-1", title="Hello")
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"))
assert '<span class="unread-count">1</span>' in response.content.decode()
client.post(reverse("reader:mark_all_read"))
response = client.get(reverse("reader:stream"))
assert "unread-count" not in response.content.decode()
def _stream_article_ids(client, query):
response = client.get(reverse("reader:stream"), query)
content = response.content.decode()
ids = [int(m) for m in re.findall(r'<li id="article-(\d+)"', content)]
return ids, content
def _link_query(content, label):
match = re.search(rf'<a href="\?([^"]+)">{label}', content)
if not match:
return None
from urllib.parse import parse_qsl
return dict(parse_qsl(match.group(1).replace("&amp;", "&")))
@pytest.mark.django_db
def test_stream_keyset_pagination_pages_through_tied_timestamps_without_gaps(client, alice):
"""Keyset pagination (C4) on (sort_at, id): articles sharing an exact
timestamp must neither repeat nor go missing across page boundaries,
which offset paging over a non-unique order can't guarantee."""
from django.utils import timezone
from apps.reader.views import PAGE_SIZE
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
same_moment = timezone.now()
total = PAGE_SIZE * 2 + 5
created = [
Article.objects.create(feed=feed, guid=f"g{i}", title=f"T{i}", published_at=same_moment)
for i in range(total)
]
client.login(username="alice", password="pw-alice-123")
seen = []
query = {"filter": "all"}
pages = 0
while query:
ids, content = _stream_article_ids(client, query)
assert len(ids) <= PAGE_SIZE
seen.extend(ids)
query = _link_query(content, "Older")
pages += 1
assert pages < 10
assert pages == 3
assert len(seen) == total
assert sorted(seen) == sorted(a.pk for a in created)
@pytest.mark.django_db
def test_stream_newer_link_returns_to_the_previous_page(client, alice):
from django.utils import timezone
from apps.reader.views import PAGE_SIZE
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
now = timezone.now()
for i in range(PAGE_SIZE + 5):
Article.objects.create(
feed=feed,
guid=f"g{i}",
title=f"T{i}",
published_at=now - timezone.timedelta(minutes=i),
)
client.login(username="alice", password="pw-alice-123")
first_ids, first_content = _stream_article_ids(client, {"filter": "all"})
assert _link_query(first_content, "Newer") is None
second_ids, second_content = _stream_article_ids(client, _link_query(first_content, "Older"))
assert len(second_ids) == 5
assert _link_query(second_content, "Older") is None
back_ids, _ = _stream_article_ids(client, _link_query(second_content, "Newer"))
assert back_ids == first_ids
@pytest.mark.django_db
def test_stream_pagination_links_keep_the_feed_scope(client, alice):
from apps.reader.views import PAGE_SIZE
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
for i in range(PAGE_SIZE + 1):
Article.objects.create(feed=feed, guid=f"g{i}", title=f"T{i}")
client.login(username="alice", password="pw-alice-123")
_, content = _stream_article_ids(client, {"filter": "all", "feed": feed.pk})
assert _link_query(content, "Older")["feed"] == str(feed.pk)
def test_templates_use_no_inline_event_handlers():
"""Production's CSP (script-src 'self') blocks inline on*= handlers —
they work in dev, where no CSP is sent, and silently do nothing in
production. Behavior belongs in static/js."""
from pathlib import Path
from django.conf import settings
pattern = re.compile(r"""<[^>]+\s(on[a-z]+)\s*=""", re.IGNORECASE)
offenders = []
for template in Path(settings.BASE_DIR, "templates").rglob("*.html"):
for match in pattern.finditer(template.read_text()):
offenders.append(f"{template.name}: {match.group(1)}")
assert offenders == []
@pytest.mark.django_db
def test_article_detail_back_link_keeps_the_stream_scope(client, alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
article = Article.objects.create(feed=feed, guid="g1", title="Hello")
client.login(username="alice", password="pw-alice-123")
stream = client.get(
reverse("reader:stream"), {"filter": "all", "feed": feed.pk}
).content.decode()
link = re.search(r'href="(/articles/\d+/\?[^"]+)"', stream)
assert link is not None
detail = client.get(link.group(1).replace("&amp;", "&")).content.decode()
assert f"filter=all&amp;feed={feed.pk}" in detail
assert "Back to stream" in detail
assert article.title in detail
@pytest.mark.django_db
def test_article_with_no_publish_date_shows_when_it_was_fetched(client, alice):
"""A feed that gives no date is treated as published when we fetched
it — in the list row, its group header, and the open article's header
— without storing an invented value on the article itself."""
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
article = Article.objects.create(feed=feed, guid="g1", title="No publish date")
assert article.published_at is None
client.login(username="alice", password="pw-alice-123")
iso = article.fetched_at.isoformat()
stream = client.get(reverse("reader:stream"), {"filter": "all"}).content.decode()
assert f'datetime="{iso}"' in stream
assert "Received today" in stream
detail = client.get(reverse("reader:article_detail", args=[article.pk])).content.decode()
assert f'datetime="{iso}"' in detail
article.refresh_from_db()
assert article.published_at is None
@pytest.mark.django_db
def test_new_since_requires_login(client):
response = client.get(reverse("reader:new_since"))
assert response.status_code == 302
@pytest.mark.django_db
def test_new_since_without_a_cursor_returns_the_baseline_id(client, alice):
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
article = Article.objects.create(feed=feed, guid="g1", title="Hello")
client.login(username="alice", password="pw-alice-123")
body = client.get(reverse("reader:new_since")).json()
assert body == {"latest_id": article.pk, "new_count": 0}
@pytest.mark.django_db
def test_new_since_counts_only_new_unread_visible_articles_from_my_feeds(client, alice, bob):
mine = Feed.objects.create(feed_url="https://mine.example.com/feed.xml")
theirs = Feed.objects.create(feed_url="https://theirs.example.com/feed.xml")
Subscription.objects.create(user=alice, feed=mine)
Subscription.objects.create(user=bob, feed=theirs)
cursor = Article.objects.create(feed=mine, guid="cursor", title="Cursor").pk
new_unread = Article.objects.create(feed=mine, guid="n1", title="New unread")
already_read = Article.objects.create(feed=mine, guid="n2", title="New but read")
hidden = Article.objects.create(feed=mine, guid="n3", title="New but hidden")
Article.objects.create(feed=theirs, guid="n4", title="Someone else's feed")
UserArticleState.objects.create(user=alice, article=already_read, is_read=True)
UserArticleState.objects.create(user=alice, article=hidden, is_hidden=True)
client.login(username="alice", password="pw-alice-123")
body = client.get(reverse("reader:new_since"), {"since": cursor}).json()
assert body["new_count"] == 1
assert new_unread.pk > cursor
@pytest.mark.django_db
def test_authenticated_pages_load_the_tab_badge_script_with_its_endpoint(client, alice):
client.login(username="alice", password="pw-alice-123")
content = client.get(reverse("reader:stream")).content.decode()
assert "js/tab-badge" in content
assert f'data-url="{reverse("reader:new_since")}"' in content
client.logout()
assert "tab-badge" not in client.get(reverse("accounts:login")).content.decode()
@pytest.mark.django_db
def test_unsafe_stored_links_are_not_rendered_as_hrefs(client, alice, django_user_model):
"""R2-S1 render boundary: even if a bad URL is already in the database
(stored before validation existed), it must not become a clickable
javascript: link."""
feed = Feed.objects.create(
feed_url="https://example.com/feed.xml", site_url="javascript:alert(1)"
)
Subscription.objects.create(user=alice, feed=feed)
article = Article.objects.create(
feed=feed, guid="g1", title="Hello", link="javascript:alert(2)"
)
client.login(username="alice", password="pw-alice-123")
detail = client.get(reverse("reader:article_detail", args=[article.pk])).content.decode()
assert "javascript:" not in detail
assert "Original" not in detail
article.link = "https://example.com/real"
article.save()
detail = client.get(reverse("reader:article_detail", args=[article.pk])).content.decode()
assert 'href="https://example.com/real"' in detail
django_user_model.objects.create_user("admin", password="pw-admin-123", is_staff=True)
client.login(username="admin", password="pw-admin-123")
Feed.objects.filter(pk=feed.pk).update(fetch_error_count=2, last_fetch_status="error")
dashboard = client.get(reverse("core:dashboard")).content.decode()
assert "javascript:" not in dashboard
+2
View File
@@ -6,6 +6,8 @@ app_name = "reader"
urlpatterns = [ urlpatterns = [
path("", views.stream, name="stream"), path("", views.stream, name="stream"),
path("check-new/", views.check_new_articles, name="check_new"),
path("new-since/", views.new_since, name="new_since"),
path("mark-all-read/", views.mark_all_read, name="mark_all_read"), path("mark-all-read/", views.mark_all_read, name="mark_all_read"),
path("articles/<int:article_id>/", views.article_detail, name="article_detail"), path("articles/<int:article_id>/", views.article_detail, name="article_detail"),
path( path(
+134 -15
View File
@@ -1,8 +1,11 @@
from datetime import datetime
from itertools import groupby from itertools import groupby
from urllib.parse import urlencode
from django.contrib.auth.decorators import login_required from django.contrib.auth.decorators import login_required
from django.db.models import Q from django.db.models import Max, Q
from django.http import Http404 from django.db.models.functions import Coalesce
from django.http import Http404, JsonResponse
from django.shortcuts import get_object_or_404, redirect, render from django.shortcuts import get_object_or_404, redirect, render
from django.urls import reverse from django.urls import reverse
from django.utils import timezone from django.utils import timezone
@@ -18,6 +21,7 @@ from apps.articles.models import (
get_system_tag, get_system_tag,
) )
from apps.articles.queries import with_user_state from apps.articles.queries import with_user_state
from apps.core.cache_utils import invalidate_sidebar_unread_cache
from apps.feeds.models import Feed from apps.feeds.models import Feed
FILTER_CHOICES = [ FILTER_CHOICES = [
@@ -109,7 +113,8 @@ def _group_by_date(articles):
today = timezone.localdate() today = timezone.localdate()
def day_of(article): def day_of(article):
return timezone.localtime(article.published_at).date() if article.published_at else None when = article.published_at or article.fetched_at
return timezone.localtime(when).date() if when else None
return [ return [
{"label": _date_label(day, today), "articles": list(items)} {"label": _date_label(day, today), "articles": list(items)}
@@ -117,8 +122,7 @@ def _group_by_date(articles):
] ]
@login_required def _resolve_active_filter(request):
def stream(request):
requested_filter = request.GET.get("filter") requested_filter = request.GET.get("filter")
if requested_filter in VALID_FILTERS: if requested_filter in VALID_FILTERS:
active_filter = requested_filter active_filter = requested_filter
@@ -128,7 +132,14 @@ def stream(request):
# selected, rather than always resetting to "unread". # selected, rather than always resetting to "unread".
active_filter = request.session.get("stream_filter", "unread") active_filter = request.session.get("stream_filter", "unread")
request.session["stream_filter"] = active_filter request.session["stream_filter"] = active_filter
return active_filter
def _filtered_queryset(request, active_filter):
"""The stream's fully-scoped queryset (filter tab + tag/feed scope),
unpaginated — shared by stream() and check_new_articles() so the "any
new articles?" check always matches exactly what the stream itself
would show."""
queryset = _base_queryset(request.user) queryset = _base_queryset(request.user)
if active_filter == "unread": if active_filter == "unread":
queryset = queryset.exclude(user_states__user=request.user, user_states__is_read=True) queryset = queryset.exclude(user_states__user=request.user, user_states__is_read=True)
@@ -146,17 +157,70 @@ def stream(request):
article_tags__tag__name=SAVED_TAG_NAME, article_tags__tag__name=SAVED_TAG_NAME,
article_tags__tag__is_system=True, article_tags__tag__is_system=True,
) )
return _apply_scope(request, queryset)
queryset, active_tag, active_feed = _apply_scope(request, queryset)
def _encode_cursor(article) -> str:
return f"{article.sort_at.isoformat()},{article.pk}"
def _decode_cursor(raw):
"""(sort_at, id) from a cursor produced by _encode_cursor, or None for
a missing/garbled one (treated as "no cursor" — the first page)."""
if not raw:
return None
try: try:
page = max(int(request.GET.get("page", 1)), 1) timestamp, pk = raw.rsplit(",", 1)
return datetime.fromisoformat(timestamp), int(pk)
except ValueError: except ValueError:
page = 1 return None
start, end = (page - 1) * PAGE_SIZE, page * PAGE_SIZE
total_count = queryset.count()
articles = with_user_state(queryset[start:end], request.user) @login_required
def stream(request):
active_filter = _resolve_active_filter(request)
queryset, active_tag, active_feed = _filtered_queryset(request, active_filter)
scope_params = {"filter": active_filter}
if active_tag:
scope_params["tag"] = active_tag.pk
if active_feed:
scope_params["feed"] = active_feed.pk
# Keyset pagination (C4) over (sort_at, id) — sort_at falls back to
# fetched_at for entries a feed gave no published date, and id breaks
# ties so pages neither shift nor repeat articles that share a
# timestamp. No COUNT(*): fetching one row past the page size is
# enough to know whether another page exists in that direction.
ordered = queryset.annotate(sort_at=Coalesce("published_at", "fetched_at"))
after = _decode_cursor(request.GET.get("after"))
before = _decode_cursor(request.GET.get("before"))
if after:
sort_at, pk = after
rows = list(
ordered.filter(Q(sort_at__gt=sort_at) | Q(sort_at=sort_at, id__gt=pk)).order_by(
"sort_at", "id"
)[: PAGE_SIZE + 1]
)
has_newer_page = len(rows) > PAGE_SIZE
has_older_page = True
rows = rows[:PAGE_SIZE][::-1]
else:
if before:
sort_at, pk = before
ordered = ordered.filter(Q(sort_at__lt=sort_at) | Q(sort_at=sort_at, id__lt=pk))
rows = list(ordered.order_by("-sort_at", "-id")[: PAGE_SIZE + 1])
has_older_page = len(rows) > PAGE_SIZE
has_newer_page = bool(before)
rows = rows[:PAGE_SIZE]
articles = with_user_state(rows, request.user)
newer_qs = older_qs = ""
if articles:
newer_qs = urlencode({**scope_params, "after": _encode_cursor(articles[0])})
older_qs = urlencode({**scope_params, "before": _encode_cursor(articles[-1])})
context = { context = {
"date_groups": _group_by_date(articles), "date_groups": _group_by_date(articles),
@@ -165,13 +229,58 @@ def stream(request):
"active_tag": active_tag, "active_tag": active_tag,
"active_feed": active_feed, "active_feed": active_feed,
"user_tags": Tag.objects.filter(user=request.user), "user_tags": Tag.objects.filter(user=request.user),
"page": page, "has_newer_page": has_newer_page and bool(articles),
"has_next_page": end < total_count, "has_older_page": has_older_page and bool(articles),
"has_previous_page": page > 1, "newer_qs": newer_qs,
"older_qs": older_qs,
# Lets the "new articles" banner ask, on this exact same scope,
# whether anything with a higher id has shown up since this page
# was rendered — id (not published_at) because it's monotonic
# regardless of how a feed backdates an entry's published date.
"latest_article_id": queryset.aggregate(Max("id"))["id__max"] or 0,
"new_check_qs": urlencode(scope_params),
"scope_qs": urlencode(scope_params),
} }
return render(request, "reader/stream.html", context) return render(request, "reader/stream.html", context)
@login_required
def check_new_articles(request):
active_filter = _resolve_active_filter(request)
queryset, _active_tag, _active_feed = _filtered_queryset(request, active_filter)
try:
since_id = int(request.GET.get("since", 0))
except ValueError:
since_id = 0
new_count = queryset.filter(id__gt=since_id).count()
return render(request, "reader/_new_articles_banner.html", {"new_count": new_count})
@login_required
def new_since(request):
"""Feeds the browser-tab dot (static/js/tab-badge.js) on every page:
without ?since it returns the current highest article id, which the
page keeps as its baseline; with it, how many unread, non-hidden
articles from the user's subscriptions have appeared after that id.
Deliberately not scoped to the stream's current filter/tag/feed — the
tab should light up for anything new, whichever page is open."""
latest_id = Article.objects.aggregate(Max("id"))["id__max"] or 0
try:
since_id = int(request.GET["since"])
except (KeyError, ValueError):
return JsonResponse({"latest_id": latest_id, "new_count": 0})
new_count = (
_base_queryset(request.user)
.filter(id__gt=since_id)
.exclude(user_states__user=request.user, user_states__is_read=True)
.count()
)
return JsonResponse({"latest_id": latest_id, "new_count": new_count})
@login_required @login_required
@require_POST @require_POST
def mark_all_read(request): def mark_all_read(request):
@@ -202,6 +311,7 @@ def mark_all_read(request):
for aid in new_article_ids for aid in new_article_ids
] ]
) )
invalidate_sidebar_unread_cache(request.user.pk)
stream_url = reverse("reader:stream") stream_url = reverse("reader:stream")
query = request.GET.urlencode() query = request.GET.urlencode()
@@ -216,6 +326,7 @@ def _load_article_and_mark_read(user, article_id):
state.is_read = True state.is_read = True
state.read_at = timezone.now() state.read_at = timezone.now()
state.save(update_fields=["is_read", "read_at"]) state.save(update_fields=["is_read", "read_at"])
invalidate_sidebar_unread_cache(user.pk)
article.is_read = state.is_read article.is_read = state.is_read
article.is_starred, article.is_saved_for_later = _article_tag_flags(user, article) article.is_starred, article.is_saved_for_later = _article_tag_flags(user, article)
@@ -225,7 +336,14 @@ def _load_article_and_mark_read(user, article_id):
@login_required @login_required
def article_detail(request, article_id): def article_detail(request, article_id):
article = _load_article_and_mark_read(request.user, article_id) article = _load_article_and_mark_read(request.user, article_id)
return render(request, "reader/article_detail.html", {"article": article}) # The stream passes its filter/tag/feed scope through the link so
# "Back to stream" returns to it instead of resetting to every feed.
back_params = {k: request.GET[k] for k in ("filter", "tag", "feed") if request.GET.get(k)}
return render(
request,
"reader/article_detail.html",
{"article": article, "back_qs": urlencode(back_params)},
)
@login_required @login_required
@@ -247,6 +365,7 @@ def toggle_read(request, article_id):
state.is_read = not state.is_read state.is_read = not state.is_read
state.read_at = timezone.now() if state.is_read else None state.read_at = timezone.now() if state.is_read else None
state.save(update_fields=["is_read", "read_at"]) state.save(update_fields=["is_read", "read_at"])
invalidate_sidebar_unread_cache(request.user.pk)
article.is_read = state.is_read article.is_read = state.is_read
article.is_starred, article.is_saved_for_later = _article_tag_flags(request.user, article) article.is_starred, article.is_saved_for_later = _article_tag_flags(request.user, article)
+68 -4
View File
@@ -9,6 +9,7 @@ from __future__ import annotations
import logging import logging
import re import re
import signal
from django.utils import timezone from django.utils import timezone
@@ -27,6 +28,42 @@ logger = logging.getLogger(__name__)
_TAG_RE = re.compile(r"<[^>]+>") _TAG_RE = re.compile(r"<[^>]+>")
# Bounds how much CPU/time a single user-authored regex rule can burn
# against one article. re has no built-in match timeout, so this uses
# SIGALRM to interrupt a runaway match (catastrophic backtracking) —
# only effective on the main thread of a process with SIGALRM available,
# which covers gunicorn's sync workers and Celery's (default) prefork
# pool; elsewhere it's skipped and MAX_REGEX_HAYSTACK_CHARS is the only
# bound. reapply_rules_for_user (apps.rules.tasks) also carries a Celery
# task time_limit as a second, coarser backstop.
REGEX_TIMEOUT_SECONDS = 1
MAX_REGEX_HAYSTACK_CHARS = 20_000
class _RegexTimeout(Exception):
pass
def _raise_regex_timeout(signum, frame):
raise _RegexTimeout()
def _regex_search_with_timeout(pattern: str, haystack: str, flags: int):
if not hasattr(signal, "SIGALRM"):
return re.search(pattern, haystack, flags)
try:
previous_handler = signal.signal(signal.SIGALRM, _raise_regex_timeout)
except ValueError:
# Not the main thread (e.g. a threaded dev server) — signal
# timers only work there.
return re.search(pattern, haystack, flags)
signal.setitimer(signal.ITIMER_REAL, REGEX_TIMEOUT_SECONDS)
try:
return re.search(pattern, haystack, flags)
finally:
signal.setitimer(signal.ITIMER_REAL, 0)
signal.signal(signal.SIGALRM, previous_handler)
def _haystack(article: Article, match_field: str) -> str: def _haystack(article: Article, match_field: str) -> str:
if match_field == FilterRule.FIELD_TITLE: if match_field == FilterRule.FIELD_TITLE:
@@ -47,8 +84,12 @@ def rule_matches(rule: FilterRule, article: Article) -> bool:
if rule.match_type == FilterRule.MATCH_REGEX: if rule.match_type == FilterRule.MATCH_REGEX:
flags = 0 if rule.case_sensitive else re.IGNORECASE flags = 0 if rule.case_sensitive else re.IGNORECASE
haystack = raw_haystack[:MAX_REGEX_HAYSTACK_CHARS]
try: try:
return re.search(rule.match_value, raw_haystack, flags) is not None return _regex_search_with_timeout(rule.match_value, haystack, flags) is not None
except _RegexTimeout:
logger.warning("Regex timed out for rule %s", rule.pk)
return False
except re.error as exc: except re.error as exc:
logger.warning("Invalid regex in rule %s: %s", rule.pk, exc) logger.warning("Invalid regex in rule %s: %s", rule.pk, exc)
return False return False
@@ -100,13 +141,36 @@ def apply_rule_action(rule: FilterRule, user, article: Article, state: UserArtic
return False return False
def evaluate_rules_for_article(user, article: Article, rules) -> None: def rule_in_scope(rule: FilterRule, feed_id, subscription_tag_ids) -> bool:
"""Evaluates an ordered iterable of rules (already scoped/enabled) """Whether `rule`'s own feed/tag scope (see FilterRule's docstring)
against one article for one user, applying actions as they match.""" covers an article from `feed_id`, given the user's subscription-tag
ids for that feed.
Centralized here — rather than left for each caller to re-derive via
its own query filters — so ingest-time dispatch and backlog re-apply
can't drift into scoping rules differently (see apps.articles.tasks
and apps.rules.tasks).
"""
if rule.feed_id is not None and rule.feed_id != feed_id:
return False
if rule.tag_id is not None and rule.tag_id not in subscription_tag_ids:
return False
return True
def evaluate_rules_for_article(user, article: Article, rules, subscription_tag_ids=()) -> None:
"""Evaluates an ordered iterable of a user's enabled rules against one
article, applying actions as they match. `rules` doesn't need to be
pre-scoped to this article's feed/tags — out-of-scope rules (per
rule_in_scope) are skipped here, so callers can pass one user's whole
enabled ruleset regardless of which feed(s) the article(s) belong to.
"""
matched_any = False matched_any = False
state = None state = None
for rule in rules: for rule in rules:
if not rule_in_scope(rule, article.feed_id, subscription_tag_ids):
continue
if not rule_matches(rule, article): if not rule_matches(rule, article):
continue continue
if state is None: if state is None:
@@ -0,0 +1,23 @@
# Generated by Django 5.2.18 on 2026-10-31 22:05
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("articles", "0004_starred_saved_as_system_tags"),
("feeds", "0006_backfill_feed_next_fetch_at"),
("rules", "0002_folder_scope_to_tag_scope"),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.AddIndex(
model_name="filterrule",
index=models.Index(
fields=["user", "is_enabled", "priority"], name="rules_filte_user_id_dcb19b_idx"
),
),
]
+10
View File
@@ -75,6 +75,16 @@ class FilterRule(models.Model):
class Meta: class Meta:
ordering = ["priority", "id"] ordering = ["priority", "id"]
indexes = [
# Matches the ingest-time query shape exactly (C9): every
# caller — apps.articles.tasks.apply_filter_rules_for_feed,
# apps.rules.tasks.reapply_rules_for_user_backlog — filters by
# (user, is_enabled) and orders by priority. `user` alone
# already has an index from the ForeignKey, but it doesn't
# cover is_enabled or let the database satisfy the ORDER BY
# from the index itself.
models.Index(fields=["user", "is_enabled", "priority"]),
]
def __str__(self): def __str__(self):
label = self.name or f"{self.get_match_field_display()} {self.get_match_type_display()}" label = self.name or f"{self.get_match_field_display()} {self.get_match_type_display()}"
+24 -6
View File
@@ -1,6 +1,6 @@
from celery import shared_task from celery import shared_task
from apps.articles.models import Article from apps.articles.models import Article, SubscriptionTag
from apps.feeds.models import Subscription from apps.feeds.models import Subscription
from .engine import evaluate_rules_for_article from .engine import evaluate_rules_for_article
@@ -12,9 +12,20 @@ def reapply_rules_for_user_backlog(rule_id: int) -> int:
rule, so priority/short-circuit ordering stays correct) against their rule, so priority/short-circuit ordering stays correct) against their
unread backlog, scoped by the triggering rule's own feed/tag scope. unread backlog, scoped by the triggering rule's own feed/tag scope.
Exists as a plain function so it can be called synchronously for The *backlog* is narrowed to the triggering rule's own feed/tag scope
immediate feedback (e.g. from a "Re-apply" view) or wrapped in a task (re-applying a feed-scoped rule has no reason to touch other feeds'
for background use — see reapply_rules_for_user below. articles), but which of the user's *other* rules apply to each of
those articles is still per-article/per-feed — evaluate_rules_for_article
checks that via rule_in_scope, so a feed- or tag-scoped rule for a
different feed can't fire on this backlog just because it happened to
match (see apps.rules.engine.rule_in_scope).
Exists as a plain function, separate from the reapply_rules_for_user
task below, so it's unit-testable without Celery — but every caller
should go through that task (see apps.rules.views.reapply_rule), not
call this directly from a request: a large backlog or a pathological
regex (bounded but not eliminated by apps.rules.engine's timeout
guard) could otherwise tie up a web worker for a long time.
""" """
try: try:
rule = FilterRule.objects.select_related("user").get(pk=rule_id) rule = FilterRule.objects.select_related("user").get(pk=rule_id)
@@ -45,13 +56,20 @@ def reapply_rules_for_user_backlog(rule_id: int) -> int:
rules = list(FilterRule.objects.filter(user=user, is_enabled=True).order_by("priority", "id")) rules = list(FilterRule.objects.filter(user=user, is_enabled=True).order_by("priority", "id"))
subscription_tag_ids_by_feed: dict[int, set[int]] = {}
for feed_id, tag_id in SubscriptionTag.objects.filter(
user=user, subscription__feed_id__in=feed_ids
).values_list("subscription__feed_id", "tag_id"):
subscription_tag_ids_by_feed.setdefault(feed_id, set()).add(tag_id)
count = 0 count = 0
for article in backlog: for article in backlog:
evaluate_rules_for_article(user, article, rules) subscription_tag_ids = subscription_tag_ids_by_feed.get(article.feed_id, set())
evaluate_rules_for_article(user, article, rules, subscription_tag_ids)
count += 1 count += 1
return count return count
@shared_task @shared_task(time_limit=120, soft_time_limit=90)
def reapply_rules_for_user(rule_id: int): def reapply_rules_for_user(rule_id: int):
return reapply_rules_for_user_backlog(rule_id) return reapply_rules_for_user_backlog(rule_id)
+37 -1
View File
@@ -1,3 +1,5 @@
import time
import pytest import pytest
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
@@ -10,7 +12,7 @@ from apps.articles.models import (
) )
from apps.feeds.models import Feed from apps.feeds.models import Feed
from .engine import evaluate_rules_for_article, rule_matches from .engine import evaluate_rules_for_article, rule_in_scope, rule_matches
from .models import FilterRule from .models import FilterRule
User = get_user_model() User = get_user_model()
@@ -87,6 +89,22 @@ class TestRuleMatches:
rule = make_rule(user, match_type=FilterRule.MATCH_REGEX, match_value="(unclosed") rule = make_rule(user, match_type=FilterRule.MATCH_REGEX, match_value="(unclosed")
assert rule_matches(rule, article) is False assert rule_matches(rule, article) is False
def test_catastrophic_backtracking_regex_times_out_instead_of_hanging(self, user, article):
# (a+)+$ against a long run of a's followed by a non-matching
# character is a textbook ReDoS pattern — left unguarded this
# takes on the order of a minute (confirmed locally); the
# REGEX_TIMEOUT_SECONDS guard in apps.rules.engine must interrupt
# it well before that instead of hanging the caller.
article.title = "a" * 30 + "!"
rule = make_rule(user, match_type=FilterRule.MATCH_REGEX, match_value=r"(a+)+$")
start = time.monotonic()
result = rule_matches(rule, article)
elapsed = time.monotonic() - start
assert result is False
assert elapsed < 5
def test_content_field_strips_html_tags(self, user, article): def test_content_field_strips_html_tags(self, user, article):
rule = make_rule(user, match_field=FilterRule.FIELD_CONTENT, match_value="Lots of details") rule = make_rule(user, match_field=FilterRule.FIELD_CONTENT, match_value="Lots of details")
assert rule_matches(rule, article) is True assert rule_matches(rule, article) is True
@@ -104,6 +122,24 @@ class TestRuleMatches:
assert rule_matches(rule, article) is True assert rule_matches(rule, article) is True
@pytest.mark.django_db
class TestRuleInScope:
def test_unscoped_rule_applies_to_any_feed(self, user):
rule = make_rule(user)
assert rule_in_scope(rule, feed_id=999, subscription_tag_ids=set()) is True
def test_feed_scoped_rule_only_applies_to_its_own_feed(self, user, feed):
rule = make_rule(user, feed=feed)
assert rule_in_scope(rule, feed_id=feed.id, subscription_tag_ids=set()) is True
assert rule_in_scope(rule, feed_id=feed.id + 1, subscription_tag_ids=set()) is False
def test_tag_scoped_rule_only_applies_when_the_feed_carries_that_tag(self, user):
tag = Tag.objects.create(user=user, name="work")
rule = make_rule(user, tag=tag)
assert rule_in_scope(rule, feed_id=1, subscription_tag_ids={tag.id}) is True
assert rule_in_scope(rule, feed_id=1, subscription_tag_ids=set()) is False
@pytest.mark.django_db @pytest.mark.django_db
class TestApplyRuleAction: class TestApplyRuleAction:
def test_star_action(self, user, article): def test_star_action(self, user, article):
+103 -1
View File
@@ -1,8 +1,17 @@
from unittest.mock import patch
import pytest import pytest
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.urls import reverse from django.urls import reverse
from apps.articles.models import STARRED_TAG_NAME, Article, ArticleTag, UserArticleState from apps.articles.models import (
STARRED_TAG_NAME,
Article,
ArticleTag,
SubscriptionTag,
Tag,
UserArticleState,
)
from apps.feeds.models import Feed, Subscription from apps.feeds.models import Feed, Subscription
from .models import FilterRule from .models import FilterRule
@@ -119,6 +128,28 @@ def test_reapply_rule_view_applies_to_unread_backlog(client, alice, feed):
assert is_starred(alice, article) is True assert is_starred(alice, article) is True
@pytest.mark.django_db
def test_reapply_rule_view_always_queues_through_celery_never_runs_inline(client, alice, feed):
# A large backlog or a pathological regex must not be able to tie up
# the request/response cycle — the view should dispatch a task, not
# call reapply_rules_for_user_backlog directly.
Subscription.objects.create(user=alice, feed=feed)
rule = FilterRule.objects.create(
user=alice,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_STAR,
)
client.login(username="alice", password="pw-alice-123")
with patch("apps.rules.views.reapply_rules_for_user.delay") as mock_delay:
response = client.post(reverse("rules:reapply_rule", args=[rule.pk]))
assert response.status_code == 302
mock_delay.assert_called_once_with(rule.pk)
@pytest.mark.django_db @pytest.mark.django_db
def test_reapply_skips_already_read_articles(alice, feed): def test_reapply_skips_already_read_articles(alice, feed):
article = Article.objects.create(feed=feed, guid="guid-1", title="Python 4 released") article = Article.objects.create(feed=feed, guid="guid-1", title="Python 4 released")
@@ -160,3 +191,74 @@ def test_reapply_respects_feed_scope(alice):
assert is_starred(alice, article_a) is True assert is_starred(alice, article_a) is True
assert not UserArticleState.objects.filter(user=alice, article=article_b).exists() assert not UserArticleState.objects.filter(user=alice, article=article_b).exists()
@pytest.mark.django_db
def test_reapply_does_not_apply_a_different_feeds_scoped_rule_to_the_backlog():
# Regression test for a bug where re-applying one feed-scoped rule
# loaded *every* enabled rule and ran it against the whole backlog
# with no scope check — so a same-matching rule scoped to a feed
# the user isn't re-applying would still fire on this backlog.
feed_a = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
feed_b = Feed.objects.create(feed_url="https://b.example.com/feed.xml")
alice = User.objects.create_user("alice", password="pw-alice-123")
Subscription.objects.create(user=alice, feed=feed_a)
Subscription.objects.create(user=alice, feed=feed_b)
article_a = Article.objects.create(feed=feed_a, guid="guid-1", title="Python in A")
triggering_rule = FilterRule.objects.create(
user=alice,
feed=feed_a,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_STAR,
)
FilterRule.objects.create(
user=alice,
feed=feed_b,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_TAG,
action_value="feed-b-only",
)
reapply_rules_for_user_backlog(triggering_rule.pk)
assert is_starred(alice, article_a) is True
assert not Tag.objects.filter(user=alice, name="feed-b-only").exists()
@pytest.mark.django_db
def test_reapply_applies_a_tag_scoped_rule_only_to_that_tags_feeds():
feed_a = Feed.objects.create(feed_url="https://a.example.com/feed.xml")
feed_b = Feed.objects.create(feed_url="https://b.example.com/feed.xml")
alice = User.objects.create_user("alice", password="pw-alice-123")
sub_a = Subscription.objects.create(user=alice, feed=feed_a)
Subscription.objects.create(user=alice, feed=feed_b)
tag = Tag.objects.create(user=alice, name="work")
SubscriptionTag.objects.create(user=alice, subscription=sub_a, tag=tag)
article_a = Article.objects.create(feed=feed_a, guid="guid-1", title="Python in A")
article_b = Article.objects.create(feed=feed_b, guid="guid-2", title="Python in B")
unscoped_trigger = FilterRule.objects.create(
user=alice,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="unrelated-trigger-value",
action=FilterRule.ACTION_MARK_READ,
)
FilterRule.objects.create(
user=alice,
tag=tag,
match_field=FilterRule.FIELD_TITLE,
match_type=FilterRule.MATCH_CONTAINS,
match_value="Python",
action=FilterRule.ACTION_STAR,
)
reapply_rules_for_user_backlog(unscoped_trigger.pk)
assert is_starred(alice, article_a) is True
assert is_starred(alice, article_b) is False
+6 -5
View File
@@ -5,7 +5,7 @@ from django.views.decorators.http import require_POST
from .forms import FilterRuleForm from .forms import FilterRuleForm
from .models import FilterRule from .models import FilterRule
from .tasks import reapply_rules_for_user_backlog from .tasks import reapply_rules_for_user
@login_required @login_required
@@ -48,8 +48,9 @@ def delete_rule(request, rule_id):
@require_POST @require_POST
def reapply_rule(request, rule_id): def reapply_rule(request, rule_id):
rule = get_object_or_404(FilterRule, user=request.user, pk=rule_id) rule = get_object_or_404(FilterRule, user=request.user, pk=rule_id)
# Run synchronously for immediate feedback, same pattern as add_feed — # Always queued, never run inline from the request/response cycle —
# a Celery worker isn't required for this to work. # a large backlog or a pathological regex could otherwise tie up a
count = reapply_rules_for_user_backlog(rule.pk) # web worker for a long time (see apps.rules.tasks).
messages.success(request, f"Re-applied your rules to {count} unread article(s).") reapply_rules_for_user.delay(rule.pk)
messages.success(request, "Re-applying your rules to your unread backlog…")
return redirect("rules:rule_list") return redirect("rules:rule_list")
+13 -12
View File
@@ -1,27 +1,28 @@
from django.contrib.postgres.search import SearchQuery, SearchRank, SearchVector
from apps.articles.models import Article from apps.articles.models import Article
from .base import SearchBackend from .base import SearchBackend
# Matches the stored search_vector column, GIN index, and maintaining
# trigger created by apps/search/migrations/0002_postgres_search_vector.py
# — the query reads that column directly instead of recomputing a
# tsvector over every article's full text on every search (C5).
_VECTOR = "articles_article.search_vector"
_QUERY = "plainto_tsquery('english', %s)"
class PostgresSearchBackend(SearchBackend): class PostgresSearchBackend(SearchBackend):
"""Computes the search vector on the fly at query time rather than
storing/indexing it on Article — keeps the schema identical to the
SQLite backend. Revisit with a stored+GIN-indexed column only if an
instance's archive grows large enough to need it."""
def search(self, query: str, user, limit: int = 50): def search(self, query: str, user, limit: int = 50):
if not query.strip(): if not query.strip():
return Article.objects.none() return Article.objects.none()
vector = SearchVector("title", weight="A") + SearchVector("content_html", weight="B")
search_query = SearchQuery(query)
return ( return (
Article.objects.filter(feed__subscriptions__user=user) Article.objects.filter(feed__subscriptions__user=user)
.select_related("feed") .select_related("feed")
.annotate(search=vector, rank=SearchRank(vector, search_query)) .extra(
.filter(search=search_query) select={"rank": f"ts_rank({_VECTOR}, {_QUERY})"},
select_params=[query],
where=[f"{_VECTOR} @@ {_QUERY}"],
params=[query],
)
.order_by("-rank", "-published_at")[:limit] .order_by("-rank", "-published_at")[:limit]
) )
+15 -9
View File
@@ -1,12 +1,14 @@
import logging
import sqlite3 import sqlite3
from django.db import connection from django.db import connection
from apps.articles.models import Article from apps.articles.models import Article
from apps.feeds.models import Subscription
from .base import SearchBackend from .base import SearchBackend
logger = logging.getLogger(__name__)
def _fts_phrase_query(raw_query: str) -> str: def _fts_phrase_query(raw_query: str) -> str:
"""Wraps the whole user input as a single FTS5 phrase so arbitrary """Wraps the whole user input as a single FTS5 phrase so arbitrary
@@ -21,24 +23,28 @@ class SQLiteFTSBackend(SearchBackend):
if not query.strip(): if not query.strip():
return [] return []
feed_ids = list(Subscription.objects.filter(user=user).values_list("feed_id", flat=True)) # Scoped through a subquery on Subscription rather than an IN
if not feed_ids: # list built from every subscribed feed id (C6): a user with
return [] # thousands of feeds would otherwise exceed SQLite's bound-
# parameter limit and the whole search would fail.
placeholders = ",".join(["%s"] * len(feed_ids)) sql = """
sql = f"""
SELECT a.id SELECT a.id
FROM articles_article a FROM articles_article a
JOIN articles_fts f ON f.rowid = a.id JOIN articles_fts f ON f.rowid = a.id
WHERE articles_fts MATCH %s AND a.feed_id IN ({placeholders}) WHERE articles_fts MATCH %s
AND a.feed_id IN (SELECT feed_id FROM feeds_subscription WHERE user_id = %s)
ORDER BY rank ORDER BY rank
LIMIT %s LIMIT %s
""" """
try: try:
with connection.cursor() as cursor: with connection.cursor() as cursor:
cursor.execute(sql, [_fts_phrase_query(query), *feed_ids, limit]) cursor.execute(sql, [_fts_phrase_query(query), user.pk, limit])
ordered_ids = [row[0] for row in cursor.fetchall()] ordered_ids = [row[0] for row in cursor.fetchall()]
except sqlite3.OperationalError: except sqlite3.OperationalError:
# A genuinely bad FTS query is rare (the input is quoted as a
# phrase), so this is far more likely a real operational
# failure — log it instead of silently showing "no results".
logger.exception("SQLite FTS search failed for user %s", user.pk)
return [] return []
if not ordered_ids: if not ordered_ids:
@@ -0,0 +1,63 @@
"""Stored tsvector for Postgres search (C5), kept current by a trigger so
no ingestion/update code path can forget to maintain it.
Stage 1 of 2, and deliberately cheap: it only adds the (nullable) column
and installs the trigger, so new and updated rows get a vector from this
point on. Both are metadata-only/brief-lock operations even on a huge
table. The slow work — backfilling existing rows and building the GIN
index — is 0003, done in batches and CONCURRENTLY so a large existing
archive doesn't lock the table for the length of a deploy (R2-T3).
No-ops entirely on SQLite — its FTS5 table (0001_sqlite_fts) already plays
this role. The column is deliberately not a Django model field: a
SearchVectorField would break SQLite, and nothing but the search backend
ever reads it (apps/search/backends/postgres.py).
"""
from django.db import migrations
VECTOR_EXPR = (
"setweight(to_tsvector('english', coalesce({row}title, '')), 'A') || "
"setweight(to_tsvector('english', coalesce({row}content_html, '')), 'B')"
)
CREATE = f"""
ALTER TABLE articles_article ADD COLUMN IF NOT EXISTS search_vector tsvector;
CREATE OR REPLACE FUNCTION articles_article_search_vector_update() RETURNS trigger AS $$
BEGIN
NEW.search_vector := {VECTOR_EXPR.format(row="NEW.")};
RETURN NEW;
END
$$ LANGUAGE plpgsql;
DROP TRIGGER IF EXISTS articles_article_search_vector_trg ON articles_article;
CREATE TRIGGER articles_article_search_vector_trg
BEFORE INSERT OR UPDATE OF title, content_html ON articles_article
FOR EACH ROW EXECUTE FUNCTION articles_article_search_vector_update();
"""
DROP = """
DROP TRIGGER IF EXISTS articles_article_search_vector_trg ON articles_article;
DROP FUNCTION IF EXISTS articles_article_search_vector_update();
DROP INDEX IF EXISTS articles_article_search_vector_gin;
ALTER TABLE articles_article DROP COLUMN IF EXISTS search_vector;
"""
def create_vector(apps, schema_editor):
if schema_editor.connection.vendor != "postgresql":
return
schema_editor.execute(CREATE)
def drop_vector(apps, schema_editor):
if schema_editor.connection.vendor != "postgresql":
return
schema_editor.execute(DROP)
class Migration(migrations.Migration):
dependencies = [("search", "0001_sqlite_fts"), ("articles", "0001_initial")]
operations = [migrations.RunPython(create_vector, drop_vector)]
@@ -0,0 +1,79 @@
"""Stage 2 of 2 (see 0002): backfill existing rows and build the GIN index
without holding a long lock (R2-T3).
Not atomic, on purpose. The backfill runs in id-range batches, each its
own short transaction, so concurrent reads and writes keep flowing and a
failed deploy can simply be re-run (it only touches rows whose vector is
still NULL). CREATE INDEX CONCURRENTLY can't run inside a transaction at
all. Rows written during the backfill already get a vector from the
trigger installed in 0002.
Safe to re-run, and a cheap no-op on an install that already has the
index and a full column (e.g. one that applied the earlier single-step
version of 0002).
No-ops entirely on SQLite.
"""
import time
from django.db import migrations
BATCH_SIZE = 5000
INDEX_NAME = "articles_article_search_vector_gin"
BACKFILL = """
UPDATE articles_article
SET search_vector =
setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
setweight(to_tsvector('english', coalesce(content_html, '')), 'B')
WHERE id >= %s AND id < %s AND search_vector IS NULL
"""
def backfill_and_index(apps, schema_editor):
connection = schema_editor.connection
if connection.vendor != "postgresql":
return
with connection.cursor() as cursor:
cursor.execute("SELECT min(id), max(id) FROM articles_article")
low, high = cursor.fetchone()
if low is not None:
started = time.monotonic()
done = 0
for start in range(low, high + 1, BATCH_SIZE):
cursor.execute(BACKFILL, [start, start + BATCH_SIZE])
done += cursor.rowcount
print(f"\n backfilled {done} search vector(s) in {time.monotonic() - started:.1f}s")
# A failed CONCURRENTLY build leaves an INVALID index behind, which
# IF NOT EXISTS would then happily skip — drop it so a re-run
# actually rebuilds.
cursor.execute(
"SELECT 1 FROM pg_index i JOIN pg_class c ON c.oid = i.indexrelid "
"WHERE c.relname = %s AND NOT i.indisvalid",
[INDEX_NAME],
)
if cursor.fetchone():
cursor.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {INDEX_NAME}")
cursor.execute(
f"CREATE INDEX CONCURRENTLY IF NOT EXISTS {INDEX_NAME} "
"ON articles_article USING GIN (search_vector)"
)
def drop_index(apps, schema_editor):
if schema_editor.connection.vendor != "postgresql":
return
with schema_editor.connection.cursor() as cursor:
cursor.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {INDEX_NAME}")
class Migration(migrations.Migration):
atomic = False
dependencies = [("search", "0002_postgres_search_vector")]
operations = [migrations.RunPython(backfill_and_index, drop_index)]
+4
View File
@@ -17,4 +17,8 @@ app.conf.beat_schedule = {
"task": "apps.feeds.tasks.poll_due_feeds", "task": "apps.feeds.tasks.poll_due_feeds",
"schedule": crontab(minute="*/1"), "schedule": crontab(minute="*/1"),
}, },
"purge-old-articles": {
"task": "apps.articles.tasks.purge_old_articles",
"schedule": crontab(hour=3, minute=30),
},
} }
+54
View File
@@ -4,6 +4,7 @@ Base Django settings for Evection, shared by all environments.
Environment-specific overrides live in dev.py / prod.py / test.py. Environment-specific overrides live in dev.py / prod.py / test.py.
""" """
import ipaddress
from pathlib import Path from pathlib import Path
import environ import environ
@@ -49,6 +50,7 @@ MIDDLEWARE = [
"django.middleware.clickjacking.XFrameOptionsMiddleware", "django.middleware.clickjacking.XFrameOptionsMiddleware",
"django_htmx.middleware.HtmxMiddleware", "django_htmx.middleware.HtmxMiddleware",
"apps.accounts.middleware.SetupWizardMiddleware", "apps.accounts.middleware.SetupWizardMiddleware",
"apps.pwa.middleware.AuthMarkerMiddleware",
] ]
ROOT_URLCONF = "config.urls" ROOT_URLCONF = "config.urls"
@@ -77,6 +79,21 @@ ASGI_APPLICATION = "config.asgi.application"
# postgres://user:pass@host:5432/dbname # postgres://user:pass@host:5432/dbname
DATABASES = {"default": env.db_url("DATABASE_URL", default=f"sqlite:///{BASE_DIR / 'db.sqlite3'}")} DATABASES = {"default": env.db_url("DATABASE_URL", default=f"sqlite:///{BASE_DIR / 'db.sqlite3'}")}
# gunicorn, Celery workers, and the FTS triggers all write to the same
# SQLite file (C6): without WAL, readers and a writer block each other,
# and without a busy timeout a contended write fails immediately with
# "database is locked" instead of waiting its turn. IMMEDIATE takes the
# write lock at transaction start so a read-then-write transaction can't
# deadlock upgrading it mid-way.
if DATABASES["default"]["ENGINE"] == "django.db.backends.sqlite3":
DATABASES["default"].setdefault("OPTIONS", {}).update(
{
"timeout": 20,
"transaction_mode": "IMMEDIATE",
"init_command": "PRAGMA journal_mode=WAL; PRAGMA synchronous=NORMAL;",
}
)
AUTH_USER_MODEL = "accounts.User" AUTH_USER_MODEL = "accounts.User"
AUTH_PASSWORD_VALIDATORS = [ AUTH_PASSWORD_VALIDATORS = [
@@ -106,9 +123,46 @@ EVECTION_SIGNUP_ENABLED = env.bool("EVECTION_SIGNUP_ENABLED", default=False)
EVECTION_MIN_FETCH_INTERVAL_MINUTES = env.int("EVECTION_MIN_FETCH_INTERVAL_MINUTES", default=15) EVECTION_MIN_FETCH_INTERVAL_MINUTES = env.int("EVECTION_MIN_FETCH_INTERVAL_MINUTES", default=15)
EVECTION_PWA_CACHE_VERSION = env.str("EVECTION_PWA_CACHE_VERSION", default="v1") EVECTION_PWA_CACHE_VERSION = env.str("EVECTION_PWA_CACHE_VERSION", default="v1")
# How long /health/ready/ reuses its queue/feed aggregates (0 = never).
EVECTION_HEALTH_CACHE_SECONDS = env.int("EVECTION_HEALTH_CACHE_SECONDS", default=15)
# IPs/CIDRs of the reverse proxy in front of gunicorn. When set, the login
# throttle only believes X-Forwarded-For from these peers (see
# apps.accounts.throttle.client_address). Empty keeps the legacy behavior.
EVECTION_TRUSTED_PROXIES = [
ipaddress.ip_network(entry.strip(), strict=False)
for entry in env.list("EVECTION_TRUSTED_PROXIES", default=[])
if entry.strip()
]
# Redis-backed rather than Django's default per-process LocMemCache: login
# rate limiting (apps.accounts.views) must share counters across every
# gunicorn worker, not just whichever one happened to handle a given
# request. Redis is already a hard install requirement for Celery, so
# this adds no new infrastructure — just a distinct key prefix so cache
# keys can't collide with anything else sharing that Redis instance.
CACHES = {
"default": {
"BACKEND": "django.core.cache.backends.redis.RedisCache",
"LOCATION": env.str("REDIS_URL", default="redis://localhost:6379/0"),
"KEY_PREFIX": "evection-cache",
}
}
# Celery # Celery
CELERY_BROKER_URL = env.str("REDIS_URL", default="redis://localhost:6379/0") CELERY_BROKER_URL = env.str("REDIS_URL", default="redis://localhost:6379/0")
CELERY_RESULT_BACKEND = None CELERY_RESULT_BACKEND = None
CELERY_TASK_SERIALIZER = "json" CELERY_TASK_SERIALIZER = "json"
CELERY_ACCEPT_CONTENT = ["json"] CELERY_ACCEPT_CONTENT = ["json"]
CELERY_TIMEZONE = TIME_ZONE CELERY_TIMEZONE = TIME_ZONE
# Without these, a worker process killed mid-task (OOM, deploy restart,
# crash) silently loses that task: the default is to ack a message the
# moment a worker *starts* it, not when it finishes. acks_late + reject-
# on-worker-lost means the message is only consumed on completion (success
# or a handled failure) and is redelivered to another worker otherwise.
# prefetch_multiplier=1 keeps a worker from grabbing a batch of tasks it
# could similarly lose all at once.
CELERY_TASK_ACKS_LATE = True
CELERY_TASK_REJECT_ON_WORKER_LOST = True
CELERY_WORKER_PREFETCH_MULTIPLIER = 1
+49 -4
View File
@@ -1,13 +1,58 @@
from django.core.exceptions import ImproperlyConfigured
from .base import * # noqa: F403 from .base import * # noqa: F403
from .base import env from .base import SECRET_KEY, env
DEBUG = False DEBUG = False
# nginx terminates TLS and proxies to gunicorn over a Unix socket (see # base.py falls back to a publicly known placeholder if SECRET_KEY isn't
# docs/install.md) — this tells Django's request.is_secure() to trust the # set — fine for a throwaway dev DB, but production must never sign
# header nginx sets, and the redirect/cookie/HSTS settings below assume it. # sessions/CSRF tokens with a key anyone can read in this repo's history.
# Fail startup rather than quietly running with a known key.
if not SECRET_KEY or SECRET_KEY == "django-insecure-change-me-in-production":
raise ImproperlyConfigured(
"SECRET_KEY is missing or still the placeholder default. Set a unique, "
"random SECRET_KEY in .env before running with prod settings — generate "
'one with: python -c "from django.core.management.utils import '
'get_random_secret_key; print(get_random_secret_key())"'
)
# Health checks (scripts/upgrade.sh, an external uptime monitor, ...) hit
# gunicorn directly over loopback, bypassing the reverse proxy entirely —
# that request's Host header is "127.0.0.1:<port>", not the public domain
# in ALLOWED_HOSTS, so without this Django 400s it. Harmless to always
# allow: gunicorn's bind address is never reachable from outside this host
# in the first place (see docs/install.md's "Reverse proxy" section).
ALLOWED_HOSTS = [*ALLOWED_HOSTS, "localhost", "127.0.0.1", "::1"] # noqa: F405
# A reverse proxy (nginx, Caddy, Traefik, Nginx Proxy Manager on another
# host, ...) is required in front of gunicorn in every deployment — see
# docs/install.md — and it's always the one terminating TLS. This tells
# Django's request.is_secure() to trust the X-Forwarded-Proto header it
# sets, and the redirect/cookie/HSTS settings below assume it's set.
SECURE_SSL_REDIRECT = env.bool("SECURE_SSL_REDIRECT", default=True) SECURE_SSL_REDIRECT = env.bool("SECURE_SSL_REDIRECT", default=True)
SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https") SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")
# /health/ is also exempted from the HTTPS redirect above: hit directly
# over loopback (see ALLOWED_HOSTS above) it has no X-Forwarded-Proto
# header to prove it's "secure", and there's no TLS listener on gunicorn's
# own port to redirect to anyway.
SECURE_REDIRECT_EXEMPT = [r"^health/$", r"^health/ready/$"]
# WhiteNoise serves /static/ directly from gunicorn instead of relying on
# the reverse proxy to alias the staticfiles directory on disk — the proxy
# may not even be on this host (see docs/install.md), so it can't be
# trusted to have filesystem access to $APP_DIR/staticfiles.
MIDDLEWARE = [
"django.middleware.security.SecurityMiddleware",
"whitenoise.middleware.WhiteNoiseMiddleware",
*MIDDLEWARE[1:], # noqa: F405
]
STORAGES = {
"default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
"staticfiles": {
"BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage",
},
}
SESSION_COOKIE_SECURE = True SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True CSRF_COOKIE_SECURE = True
+10
View File
@@ -5,3 +5,13 @@ CELERY_TASK_ALWAYS_EAGER = True
CELERY_TASK_EAGER_PROPAGATES = True CELERY_TASK_EAGER_PROPAGATES = True
PASSWORD_HASHERS = ["django.contrib.auth.hashers.MD5PasswordHasher"] PASSWORD_HASHERS = ["django.contrib.auth.hashers.MD5PasswordHasher"]
# base.py's CACHES needs a real Redis instance — tests must not depend on
# one being available, and a plain per-process cache is also what keeps
# login-rate-limit tests isolated from each other (LocMemCache's default
# location is per-process but shared across tests in that process; see
# apps.accounts.test_rate_limit's cache.clear() fixture).
CACHES = {"default": {"BACKEND": "django.core.cache.backends.locmem.LocMemCache"}}
# Tests change state between calls to /health/ready/; caching it would hide that.
EVECTION_HEALTH_CACHE_SECONDS = 0
+14
View File
@@ -1,8 +1,22 @@
import pytest import pytest
from django.core.cache import cache
from apps.accounts.middleware import SetupWizardMiddleware from apps.accounts.middleware import SetupWizardMiddleware
@pytest.fixture(autouse=True)
def _reset_django_cache():
"""config.settings.test overrides CACHES to LocMemCache, which — like
SetupWizardMiddleware's own cache below — lives in process memory and
survives Django's per-test DB rollback. Without this, a cached
sidebar unread count (apps.core.cache_utils) keyed by a user's pk
could leak into a later, unrelated test that happens to create a user
with the same auto-incrementing pk."""
cache.clear()
yield
cache.clear()
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
def _reset_setup_wizard_cache(): def _reset_setup_wizard_cache():
"""SetupWizardMiddleware caches "does any User exist" on the class """SetupWizardMiddleware caches "does any User exist" on the class
+6 -5
View File
@@ -9,11 +9,12 @@ Group=evection
WorkingDirectory=/opt/evection/app WorkingDirectory=/opt/evection/app
EnvironmentFile=/opt/evection/app/.env EnvironmentFile=/opt/evection/app/.env
Environment=DJANGO_SETTINGS_MODULE=config.settings.prod Environment=DJANGO_SETTINGS_MODULE=config.settings.prod
RuntimeDirectory=evection # Evection is always meant to sit behind a reverse proxy (see
ExecStart=/opt/evection/venv/bin/gunicorn \ # docs/install.md) — it listens on a plain TCP address, never a Unix
--workers 3 \ # socket, so that proxy can be on this host or a different one.
--bind unix:/run/evection/gunicorn.sock \ # EVECTION_GUNICORN_BIND (set in .env by scripts/install.sh) controls
config.wsgi:application # what address:port that is.
ExecStart=/bin/sh -c 'exec /opt/evection/venv/bin/gunicorn --workers 3 --bind "$EVECTION_GUNICORN_BIND" config.wsgi:application'
ExecReload=/bin/kill -s HUP $MAINPID ExecReload=/bin/kill -s HUP $MAINPID
Restart=on-failure Restart=on-failure
+24 -5
View File
@@ -1,6 +1,12 @@
# Reverse proxy for Evection. Copy to /etc/nginx/sites-available/evection, # Reverse proxy for Evection, for the case where nginx runs on the SAME host
# adjust server_name and the certificate paths (certbot will fill those in # as Evection (see docs/install.md's "Reverse proxy" section — if your proxy
# for you if you use `certbot --nginx`), then symlink into sites-enabled. # is on a different host/VM instead, e.g. Nginx Proxy Manager, this file
# doesn't apply to you directly, but the proxy_set_header lines below still
# show what any proxy needs to forward). Copy to
# /etc/nginx/sites-available/evection, adjust server_name, the certificate
# paths (certbot will fill those in for you if you use `certbot --nginx`),
# and the port in proxy_pass if you didn't use EVECTION_GUNICORN_BIND's
# default of 127.0.0.1:8000, then symlink into sites-enabled.
server { server {
listen 80; listen 80;
@@ -24,6 +30,11 @@ server {
client_max_body_size 10M; # OPML uploads are small, but leave headroom client_max_body_size 10M; # OPML uploads are small, but leave headroom
# Optional: gunicorn already serves /static/ itself via WhiteNoise (with
# its own far-future cache headers, since filenames are content-hashed),
# so this block is purely a same-host perf optimization that skips
# gunicorn for static assets — not required, and not an option at all
# once the proxy is on a different host than $APP_DIR/staticfiles.
location /static/ { location /static/ {
alias /opt/evection/app/staticfiles/; alias /opt/evection/app/staticfiles/;
expires 30d; expires 30d;
@@ -36,10 +47,18 @@ server {
# else below, so no special-casing is needed here. # else below, so no special-casing is needed here.
location / { location / {
proxy_pass http://unix:/run/evection/gunicorn.sock; proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # $remote_addr, not $proxy_add_x_forwarded_for (S8): the latter
# APPENDS to whatever X-Forwarded-For a client already sent, so a
# client could set their own "X-Forwarded-For: 1.2.3.4" and have it
# forwarded as the first (trusted) hop — Evection's login rate
# limiter (apps.accounts.views._client_address) and anything else
# that reads this header take the first entry as the real client
# address. Overwriting it with nginx's own view of the connecting
# IP is what actually makes that trust meaningful.
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
} }
+198 -40
View File
@@ -1,9 +1,16 @@
# Installing Evection # Installing Evection
Evection is a traditional Python/Django application: `pip install`, a WSGI server Evection is a traditional Python/Django application: `pip install`, a WSGI server
(gunicorn), a Celery worker + beat scheduler for background feed polling, and an (gunicorn), and a Celery worker + beat scheduler for background feed polling.
nginx reverse proxy in front of it. There's no Docker image for v1 — everything There's no Docker image for v1 — everything below runs directly on the host.
below runs directly on the host.
**A reverse proxy in front of gunicorn is required, always** — Evection has no
TLS support of its own and assumes one is terminating HTTPS for it (see "Reverse
proxy" below). It can be nginx/Caddy/Traefik running on the same host, or
something like Nginx Proxy Manager running on an entirely different host/VM —
`scripts/install.sh` doesn't set one up for you, since which one and where it
runs is up to you; it just leaves gunicorn listening on a TCP address for that
proxy to reach.
Both SQLite and PostgreSQL are fully supported. Steps that differ are marked Both SQLite and PostgreSQL are fully supported. Steps that differ are marked
**SQLite** / **Postgres**; everything else applies to both. **SQLite** / **Postgres**; everything else applies to both.
@@ -12,7 +19,8 @@ Both SQLite and PostgreSQL are fully supported. Steps that differ are marked
Evection is deployed from git tags, never from a branch tip — the same pattern Evection is deployed from git tags, never from a branch tip — the same pattern
as the author's other self-hosted projects. A release tag looks like as the author's other self-hosted projects. A release tag looks like
`v2026.10.31` (`v<year>.<month>.<day>`); `scripts/install.sh` and `v2026.11.1` (`v<year>.<month>.<day>`, no zero padding); a second release the same
day is `v2026.11.1.2`, a third `v2026.11.1.3`, and so on. `scripts/install.sh` and
`scripts/upgrade.sh` both always deploy whatever tag is newest by creation `scripts/upgrade.sh` both always deploy whatever tag is newest by creation
time, via a detached-HEAD checkout, regardless of what's checked out or time, via a detached-HEAD checkout, regardless of what's checked out or
committed on `main`. committed on `main`.
@@ -21,7 +29,7 @@ Before your first production install, cut and push a tag from whatever commit
you want to ship: you want to ship:
```bash ```bash
git tag v$(date +%Y.%m.%d) git tag v$(date +%Y.%-m.%-d) # a later release the same day: append .2, .3, ...
git push --tags git push --tags
``` ```
@@ -32,9 +40,11 @@ run at all until a tag exists.
## Quick install ## Quick install
Once the prerequisites below are in place, `scripts/install.sh` automates Once the prerequisites below are in place, `scripts/install.sh` automates
everything from "create a system user" through "systemd units are running" — everything from "create a system user" through "systemd units are running",
and optionally nginx + a Let's Encrypt certificate too. It's interactive including asking where your reverse proxy will run so it can set up gunicorn's
(asks for your domain and database choice) and safe to re-run. listen address accordingly. It's interactive (asks for your domain, database
choice, and proxy location) and safe to re-run. It does **not** set up the
proxy itself — see "Reverse proxy" below for that, once this finishes.
```bash ```bash
sudo mkdir -p /opt/evection sudo mkdir -p /opt/evection
@@ -55,18 +65,22 @@ and 4 below first — prerequisites and what `.env` controls.
## 1. Prerequisites ## 1. Prerequisites
- Python 3.11 or 3.12 - Python 3.11 or 3.12, with its `venv` module (Debian/Ubuntu split this into a
- Redis (Celery's broker) separate `python3.X-venv` package — see below)
- Redis (Celery's broker, and the app's cache — e.g. login rate limiting)
- **Postgres only**: a PostgreSQL server (12+) and its client headers if you're not - **Postgres only**: a PostgreSQL server (12+) and its client headers if you're not
using the bundled `psycopg[binary]` wheel (most platforms are covered by the wheel, using the bundled `psycopg[binary]` wheel (most platforms are covered by the wheel,
so this is rarely needed) so this is rarely needed)
- nginx (or another reverse proxy) - A reverse proxy, already running or that you're about to set up — on this
- A domain name pointed at the host, if you want this reachable outside your LAN host or a different one (see "Reverse proxy" below)
- A domain name pointed at wherever that proxy terminates TLS, if you want
this reachable outside your LAN
Debian/Ubuntu example for the non-Python pieces: Debian/Ubuntu example for the non-Python pieces (swap `python3.13-venv` for
whatever `python3 --version` reports on your system):
```bash ```bash
sudo apt install redis-server nginx sudo apt install redis-server python3.13-venv
# Postgres only: # Postgres only:
sudo apt install postgresql sudo apt install postgresql
``` ```
@@ -92,9 +106,17 @@ than a branch.)
```bash ```bash
cd /opt/evection/app cd /opt/evection/app
sudo -u evection -H python3 -m venv /opt/evection/venv sudo -u evection -H python3 -m venv /opt/evection/venv
sudo -u evection -H /opt/evection/venv/bin/pip install . sudo -u evection -H /opt/evection/venv/bin/pip install --upgrade pip
sudo -u evection -H /opt/evection/venv/bin/pip install --require-hashes -r requirements.txt
sudo -u evection -H /opt/evection/venv/bin/pip install --no-deps .
``` ```
`requirements.txt` pins every dependency to an exact, hash-verified version, so
two installs of the same release get identical dependency code and a tampered
package fails the hash check instead of installing. The app itself goes in
with `--no-deps` afterwards. (Don't use a bare `pip install .` — it re-resolves
version ranges at install time, which is what the lock file exists to avoid.)
## 4. Configure ## 4. Configure
Copy the example environment file and edit it: Copy the example environment file and edit it:
@@ -127,11 +149,28 @@ At minimum, set the following in `.env`:
sudo -u postgres createdb evection -O evection sudo -u postgres createdb evection -O evection
``` ```
- `REDIS_URL` — `redis://localhost:6379/0` is fine for a single-host install - `REDIS_URL` — `redis://localhost:6379/0` is fine for a single-host install
- `EVECTION_GUNICORN_BIND` — the `address:port` gunicorn listens on; see
"Reverse proxy" below for how to pick this
- `EVECTION_TRUSTED_PROXIES` — the IP address (or CIDR, comma-separated for more
than one) of your reverse proxy, as gunicorn sees it connecting. The login
throttle only believes `X-Forwarded-For` from these addresses, and takes the
address your proxy actually saw rather than the first entry (which a client can
forge). Leave it unset and it falls back to trusting that header from anyone,
which is only safe while nothing but your proxy can reach gunicorn;
`manage.py check --deploy` warns until it's set. With Nginx Proxy Manager on
another host, use that host's address (check `ss -tn` on this machine while
loading the site if you're not sure what gunicorn sees).
How often every feed is polled is a site-wide setting (Django admin → Site
settings), not a `.env` value — `EVECTION_MIN_FETCH_INTERVAL_MINUTES` only sets
a floor under it, so it can't be configured lower than that without editing `.env`.
Evection reads `config.settings.prod` when run via the commands below (see each Evection reads `config.settings.prod` when run via the commands below (see each
systemd unit's `DJANGO_SETTINGS_MODULE`). That module turns on HTTPS-only cookies, systemd unit's `DJANGO_SETTINGS_MODULE`). That module turns on HTTPS-only cookies,
HSTS, and a Content-Security-Policy header on top of the base settings — see HSTS, and a Content-Security-Policy header on top of the base settings, and
`config/settings/prod.py` if you need to adjust it for your proxy setup. serves static files itself via WhiteNoise rather than relying on the proxy to
have filesystem access to this host — see `config/settings/prod.py` if you need
to adjust any of that.
## 5. Initialize the database and static files ## 5. Initialize the database and static files
@@ -190,28 +229,95 @@ sudo systemctl status evection-gunicorn evection-celery-worker evection-celery-b
sudo journalctl -u evection-gunicorn -f sudo journalctl -u evection-gunicorn -f
``` ```
## 7. nginx ## 7. Reverse proxy
Copy `docs/deploy/nginx.conf.example` to `/etc/nginx/sites-available/evection`, gunicorn only ever listens on plain HTTP, on the `EVECTION_GUNICORN_BIND`
adjust `server_name` and the certificate paths, then enable it: address you set in step 4 — it never terminates TLS itself. A reverse proxy
in front of it is not optional: it's what the setup wizard, login, and every
other page actually get reached through. Two shapes this takes:
**Proxy on the same host as Evection.** Set `EVECTION_GUNICORN_BIND=127.0.0.1:8000`
(install.sh's default when you say the proxy is local) so gunicorn isn't
reachable from outside this machine at all — only the proxy, listening on
80/443, is. `docs/deploy/nginx.conf.example` is a working nginx config for
this case; copy it into place and get a certificate the usual way:
```bash ```bash
sudo cp docs/deploy/nginx.conf.example /etc/nginx/sites-available/evection sudo cp docs/deploy/nginx.conf.example /etc/nginx/sites-available/evection
# edit it: server_name, and the proxy_pass port if you didn't use 8000
sudo ln -s /etc/nginx/sites-available/evection /etc/nginx/sites-enabled/ sudo ln -s /etc/nginx/sites-available/evection /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx sudo nginx -t && sudo systemctl reload nginx
```
Get a TLS certificate (Let's Encrypt via certbot is the easiest path):
```bash
sudo apt install certbot python3-certbot-nginx sudo apt install certbot python3-certbot-nginx
sudo certbot --nginx -d your-domain.example sudo certbot --nginx -d your-domain.example
``` ```
Evection's `prod` settings enable `SECURE_SSL_REDIRECT` and HSTS by default, so `127.0.0.1:8000` is still a TCP port any local process can connect to (any
plan to have TLS working before your first real visit — see other user or container sharing this host, not just nginx). If you want
`config/settings/prod.py` to disable the redirect temporarily if you need to debug gunicorn reachable by nothing but the proxy, bind it to a Unix socket
over plain HTTP first. instead: set `EVECTION_GUNICORN_BIND=unix:/run/evection/gunicorn.sock` in
`.env`, add `RuntimeDirectory=evection` under `[Service]` in
`evection-gunicorn.service` (creates `/run/evection`, owned by the
`evection` user, on every start), and point nginx's `proxy_pass` at
`http://unix:/run/evection/gunicorn.sock:;` instead of the TCP address —
nginx needs read/write access to that socket, so either run nginx's worker
as a user in the `evection` group or adjust the directory's permissions
accordingly. This is optional hardening, not required.
**Proxy on a different host or VM** (Nginx Proxy Manager, a Caddy box, a
container host, etc.). Set `EVECTION_GUNICORN_BIND=0.0.0.0:8000` (or to this
host's specific IP) so the proxy can actually reach gunicorn over the
network, then **firewall that port to only accept connections from the
proxy's host** — gunicorn itself has no auth in front of it, so it must
never be reachable from the open internet directly. This isn't optional
hardening: Evection trusts `X-Forwarded-Proto` and `X-Forwarded-For` from
whoever can reach this port at all (see `config/settings/prod.py` and
`apps/accounts/views.py`'s login rate limiter), so anyone who can reach it
directly can set those headers themselves and be trusted as if they were
the proxy. With ufw, that's:
```bash
sudo ufw allow from <proxy-host-ip> to any port 8000 proto tcp
sudo ufw deny 8000/tcp
```
(order matters — the `allow from` rule must be added before the blanket
`deny`). Point the proxy's upstream/forward target at `this-host:8000`,
and make sure it **overwrites** `X-Forwarded-For` with its own view of the
client's address rather than appending to whatever the client already
sent — a proxy that appends lets a client forge a trusted-looking first
hop. It must also set `X-Forwarded-Proto` (whatever sets up TLS
passthrough/termination normally handles this, but double-check — see the
next paragraph for why it matters). Whichever proxy you use,
`docs/deploy/nginx.conf.example`'s `location /` block shows the three
headers that need forwarding either way, including that overwrite-not-
append distinction for `X-Forwarded-For`.
**Nginx Proxy Manager specifically:** NPM's generated config already sets
`X-Forwarded-For` to the appending (unsafe) form by default — override it
per proxy host rather than editing NPM's own templates: open the proxy
host entry for Evection, go to its **Advanced** tab, and add this to the
**Custom Nginx Configuration** box:
```nginx
proxy_set_header X-Forwarded-For $remote_addr;
```
This lands inside NPM's own `location` block, after its default headers,
so nginx uses this one (the last `proxy_set_header` for a given name in
the same block wins) — no edit to any file NPM manages, and it survives
NPM regenerating its config. Save, and NPM reloads nginx for you. If
something else sits in front of NPM itself (a CDN, another load balancer,
Cloudflare, ...), `$remote_addr` at NPM's layer is that upstream's
address, not the original client — forward whatever *that* layer's own
trusted client-IP header says instead.
Either way, the proxy must set `X-Forwarded-Proto: https` on requests it
forwards once it's terminating TLS — Evection's `prod` settings
(`SECURE_PROXY_SSL_HEADER` in `config/settings/prod.py`) trust that header to
know the original request was HTTPS, which is what keeps `SECURE_SSL_REDIRECT`
from causing a redirect loop. Plan to have TLS working end-to-end before your
first real visit; see `config/settings/prod.py` to disable the redirect
temporarily if you need to debug over plain HTTP first.
## 8. Visit the site ## 8. Visit the site
@@ -225,8 +331,51 @@ surfaces any feeds that are failing to fetch, which is the fastest way to catch
misconfigured feed or an outage at the source. misconfigured feed or an outage at the source.
`/health/` is an unauthenticated liveness check (DB connectivity only) — `/health/` is an unauthenticated liveness check (DB connectivity only) —
`scripts/upgrade.sh` uses it to confirm a deploy actually came back up, and `scripts/upgrade.sh` uses it to confirm a deploy actually came back up.
it's a reasonable target for an external uptime monitor too. Point an external uptime monitor at `/health/ready/` instead. It returns 503
with a per-check breakdown when any of these fail:
- `database` and `redis` — reachable.
- `poll_heartbeat` — the background poller ticked in the last five minutes
(proves Celery beat is scheduling *and* a worker is consuming).
- `queue_depth` — no more than 1000 tasks waiting in the Celery queue
(workers aren't keeping up, or aren't running).
- `recent_fetch_success` — at least one feed has been fetched successfully
within three poll intervals (minimum 30 minutes), so "scheduled but every
fetch is failing" is caught. A brand-new install gets that long before it
counts as stale.
- `feed_failures` — no more than half of your active feeds are failing, once
you have at least four (a handful of dead feeds is normal; most of them
failing at once points at your server, DNS, or outbound network).
The payload also reports `queue_depth`, `active_feeds`, `failing_feeds`, and
the age of the last successful fetch, so you can graph them. The queue and feed
figures are reused for 15 seconds (`EVECTION_HEALTH_CACHE_SECONDS`, 0 disables
it) so a fast-polling monitor can't turn them into steady database load; the
database, Redis and poller checks are always live. The endpoint is
unauthenticated, so if you'd rather not expose those numbers publicly, restrict
`/health/ready/` to your monitor's address at the proxy.
**Retention.** By default every fetched article is kept forever. In the Django
admin, "Site settings" has a *Retention days* field: set it to delete articles
fetched longer ago than that, nightly (03:30 UTC, via Celery beat). Articles
anyone has starred or saved for later are always kept; read state and tags for
deleted articles go with them. A small record (feed and entry id) is kept for
each deleted article so a feed that still lists it can't bring it back as new
and unread — those records are themselves removed after a year, so an entry a
feed is *still* publishing a year after it was purged (with no publish date, or
a recent one) can come back as new. Entries whose own publish date is already
older than the window aren't fetched in the first place. A large first purge
works in time-boxed runs (about four minutes each) that queue their own
continuation, so it never monopolises a worker. The dashboard shows the article count and the
age of the oldest one, so you can see what retention is doing.
**SQLite.** SQLite installs run in WAL mode with a 20-second busy timeout (set
automatically in `config/settings/base.py`) so gunicorn, the Celery worker, and
the full-text-search triggers can share the file without "database is locked"
errors. That's plenty for a personal or small-team instance; for many users or
a very large archive, use PostgreSQL (its search uses a stored, GIN-indexed
vector kept current by a trigger, added by migration automatically).
## Backups ## Backups
@@ -255,10 +404,13 @@ Equivalent by hand:
```bash ```bash
cd /opt/evection/app cd /opt/evection/app
sudo -u evection -H git fetch --tags --force git fetch --tags # deliberately not --force: see below
LATEST_TAG="$(sudo -u evection -H git tag --sort=-creatordate | head -n1)" LATEST_TAG="$(git tag --sort=-creatordate | head -n1)"
sudo -u evection -H git checkout --detach "$LATEST_TAG" git checkout --detach "$LATEST_TAG"
sudo -u evection -H /opt/evection/venv/bin/pip install . chown -R evection:evection /opt/evection/app
sed -i "s/^EVECTION_PWA_CACHE_VERSION=.*/EVECTION_PWA_CACHE_VERSION=$LATEST_TAG/" .env
sudo -u evection -H /opt/evection/venv/bin/pip install --require-hashes -r requirements.txt
sudo -u evection -H /opt/evection/venv/bin/pip install --no-deps --upgrade .
sudo -u evection -H env DJANGO_SETTINGS_MODULE=config.settings.prod \ sudo -u evection -H env DJANGO_SETTINGS_MODULE=config.settings.prod \
/opt/evection/venv/bin/python manage.py migrate /opt/evection/venv/bin/python manage.py migrate
sudo -u evection -H env DJANGO_SETTINGS_MODULE=config.settings.prod \ sudo -u evection -H env DJANGO_SETTINGS_MODULE=config.settings.prod \
@@ -266,15 +418,21 @@ sudo -u evection -H env DJANGO_SETTINGS_MODULE=config.settings.prod \
sudo systemctl restart evection-gunicorn evection-celery-worker evection-celery-beat sudo systemctl restart evection-gunicorn evection-celery-worker evection-celery-beat
``` ```
A plain `git fetch --tags` refuses to move a tag you already have onto a
different commit — the sign of a rewritten or tampered release. If it fails
with "would clobber existing tag", stop and find out why before deploying;
don't reach for `--force`, which would silently follow the rewritten tag into
production (`scripts/upgrade.sh` stops there too).
To ship a new release, just cut and push a new tag on the machine you build To ship a new release, just cut and push a new tag on the machine you build
from — the next `upgrade.sh` run on the server picks it up: from — the next `upgrade.sh` run on the server picks it up:
```bash ```bash
git tag v$(date +%Y.%m.%d) git tag v$(date +%Y.%-m.%-d) # a later release the same day: v2026.11.1.2, .3, ...
git push --tags git push --tags
``` ```
Bump `EVECTION_PWA_CACHE_VERSION` in `.env` whenever a release changes cached `upgrade.sh` sets `EVECTION_PWA_CACHE_VERSION` in `.env` to the deployed tag on every
static assets (CSS/JS/icons) — it's baked into the service worker's cache name, so run — it's baked into the service worker's cache name, so this forces every installed
changing it forces every installed PWA to drop its old cache instead of serving PWA to drop its old cache instead of serving stale assets after the upgrade. You don't
stale assets after the upgrade. need to bump it by hand.
+2
View File
@@ -22,6 +22,7 @@ dependencies = [
"psycopg[binary]>=3.1,<4", "psycopg[binary]>=3.1,<4",
"gunicorn>=21,<24", "gunicorn>=21,<24",
"django-csp>=4.0,<5", "django-csp>=4.0,<5",
"whitenoise>=6.6,<7",
] ]
[project.optional-dependencies] [project.optional-dependencies]
@@ -33,6 +34,7 @@ dev = [
"freezegun>=1.4,<2", "freezegun>=1.4,<2",
"ruff>=0.4", "ruff>=0.4",
"black>=24", "black>=24",
"pip-tools>=7,<8",
] ]
[tool.setuptools] [tool.setuptools]
+721
View File
@@ -0,0 +1,721 @@
#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
# pip-compile --allow-unsafe --extra=dev --generate-hashes --output-file=requirements-dev.txt pyproject.toml
#
amqp==5.4.1 \
--hash=sha256:79a9c0ab70e71745667f127ff80666894a734c26236b6f33149c964b096f0b20 \
--hash=sha256:ac2b816a14a380ed10c5ebbf85a334fd68111fa476496867a5ccd2fd09926d5e
# via kombu
asgiref==3.13.0 \
--hash=sha256:034cc8cbf72d00553b4be256703ac545120b4e50af13b49dec99da899aeeecb4 \
--hash=sha256:f9596a2434512686473d50f96ff6207fbba49ae1a785dfac614df4aeb085674d
# via
# django
# django-htmx
billiard==4.3.1 \
--hash=sha256:2c7075283191d9c0add66cf8fca8e06ba599e75fe7319b67186759f8877dfdaf \
--hash=sha256:c88559b306ee5dc93f8d5f843d07da15d795d67af26720d14ee9d09f09eb0b22
# via celery
black==26.10.1 \
--hash=sha256:03c0ddd93bb392e71209903a691767eb366fe1a76deb9509ccbaae9e1f14bb52 \
--hash=sha256:0ce08b367307b0fd91c9dd1d4084e62b05b3055475f951f0f34a46b6e2393b64 \
--hash=sha256:182f6c32be38074b16d378498c498b32cb51928178ee611485344972c35ec9c6 \
--hash=sha256:1935b32f5326028019856e18cb42b4da63db23765dc84464cec723e0de478a9b \
--hash=sha256:19fa8f5beb5e77c54c9c7e21d00cc93ed6c8b6228ee385616906d6befe081143 \
--hash=sha256:2520037aa62f8a1454d0811b8f5c88b444445b03a4bfba480d8d220893b64c34 \
--hash=sha256:28842f9a8207cc1df6eb983a35a14c5a0dfcd603d214fe82d84bef552afd2e3a \
--hash=sha256:289282aa2e09d3162312a3be1788ff21b08e9ea9cc4a81e656024728b32428fb \
--hash=sha256:2ffbc023a12d0c729408823b8f10514490bd0baa301d0d4e21a7240249f9507f \
--hash=sha256:3414a0c52901964dceabd98c7c56beac0f964115a116ecedcce7247359b14017 \
--hash=sha256:4d9a90516db1d99c25dbb20cc0998e0e01531dd903466c7744e56d66f864220a \
--hash=sha256:51d5e417e700fe6ec0b0ecdc408c6f6cb5def80328f31f724993d82c6486b746 \
--hash=sha256:5cd88fd7b444ca51f3fc883b6f6657ea53a258b0b2eef6d9f2dfcfa17ce0e27b \
--hash=sha256:5f9f83beae62437e060dafd53d7f1fc327e3d3494f74d72ee5c2b73eb90fc4e7 \
--hash=sha256:70ccbd175b7f6be29d2b727ee7ca6b4c54053df59da653a6df80b175d20a94fa \
--hash=sha256:7bdade400bfe24d78a7762896acc2f9a8e1a17fb0fd0536bf6b7c7097cf3eec7 \
--hash=sha256:8375962579d537364cc0efa19b1474481915d3a793f9fc0774901814c5e5b5f4 \
--hash=sha256:978113a40223a6aaefc17364176a809a320e6b288683841427fff04c6d7b4130 \
--hash=sha256:9a0219b29cd70e49f920acb7081e6ce5025c719008447c521d0200dcad93206a \
--hash=sha256:b5347d760f0c02bb00dd249384cab71c3bf828b4f68d5b401eb116e0390f147d \
--hash=sha256:b6272cfd7e1e8e271f5b0e0207259fe2834687e5cb9b5f620b34a44db9754993 \
--hash=sha256:d42dd2fac7c342ae67e64ee99c9532e20b2a84e92c79ed3317fa2ef54c801d93 \
--hash=sha256:d5bd3518d8e97138fef295230b1e9804076d69fa4e3594071494a8c68abe6266 \
--hash=sha256:d8b3a9074a680b3c5749633714e9ae3992a1e5a23343a97ad61cd9b119b444d2 \
--hash=sha256:f6dba8138cdc99061ef07b958ac082d2aa057b6961d1936f9717c350f02bab5f \
--hash=sha256:fe85fc4019bee59bc495c0f2a8ee76c5cd02c7015508d94a967ba2376f39a52c \
--hash=sha256:ff57f63029aa1353fa8b1b0c8971fd88a6c92dc766608d2eee33ad2deb23270e
# via evection (pyproject.toml)
build==1.6.1 \
--hash=sha256:51cc11666391ab6f092070437ac747002ff46f3e4113a3622177ee6b488bfc53 \
--hash=sha256:ecd351a4be9d35a9eaaba244a7687143c9c7d4aea6ac964e7e7ddab20cbcf4e7
# via pip-tools
celery==5.6.3 \
--hash=sha256:0808f42f80909c4d5833202360ffafb2a4f83f4d8e23e1285d926610e9a7afa6 \
--hash=sha256:177006bd2054b882e9f01be59abd8529e88879ef50d7918a7050c5a9f4e12912
# via evection (pyproject.toml)
certifi==2026.10.23 \
--hash=sha256:a9895e6e2758b371e0545e321a3a8fd1fcfd9829ada8824a936dfb16aaaf36cd \
--hash=sha256:e9b54de76e530bb6cf0ccd036787103acb58fd42ba930ea8bf948ddbaa2f61d1
# via requests
charset-normalizer==3.5.2 \
--hash=sha256:01077390b03f7988f11d700a2194e69b119741a86b1a638b1db88891e3eced8e \
--hash=sha256:01b0c0d2262a9e28e8484a278c7e1b5d650e3ac8cf2683d2967e25899f208bdf \
--hash=sha256:04851f73ae72b8413dddadb16a49dfee95263553741fd42d546f7d66907e6be5 \
--hash=sha256:0521c5665880b33d603717defa76c094048900010897909952397feb3039da56 \
--hash=sha256:0774bf9bf620249fee3e0b8b9fd3065de213be30f3aa94ce2494b3b638949e26 \
--hash=sha256:0891b9d3903c5571c03771ca669a4b0ec5618ca722a5c957d3d29cd4e5062848 \
--hash=sha256:0c951d5e6dd9c2ff60609476752bee49da4206adde960ebc247766937f72e718 \
--hash=sha256:0fed1d06615f022ee3b13caf5e8b180cfea32bb2c5aded8a9d44277afc040f93 \
--hash=sha256:114e4d0c92d618409ed82a99e22b5c5e768fe995f2973f78265f4524f49d4640 \
--hash=sha256:11912e4bb14baae7c5d8791aa55ba0a3a03ec6729073307b0f57270abaa713d3 \
--hash=sha256:11a4d68a6ecda3292cb1e50239e111543ba5d709bb62a6b4ea1afcfa729d8875 \
--hash=sha256:124fbf1a8ff966d87ae05bb8bd45a71f966055ed8bba320d0c7cf450bc5f4d0e \
--hash=sha256:1461ac396c4fdb983a675f20aa555624f0ee18ac83d832b9244ffff3d8055275 \
--hash=sha256:1503bccbeb36d5527790c3930327704c39af22de3112f1b1666a9f3ce15ee204 \
--hash=sha256:15bb4005af6320d259dc7593ca84a38d7fe06a421dbcf7b910ae23979101e787 \
--hash=sha256:15c44f7edfd477b06f517a5cc317fc1707edb9de2c865f43d4b6513907473234 \
--hash=sha256:16fa0eccf81304b79c5cd87f9271c3b85dd9dd99245e4422ae9c0dd45e0f99d3 \
--hash=sha256:183b88127acdb4fabe59d951ab424faf1af7b63cdbb5f776186c1ea2ffcaed98 \
--hash=sha256:195c26fb65950f8fce54e26349852b7bdd7c5f120aeefbcc440b8a20faaed4a3 \
--hash=sha256:1afb975bd5d68d5ce9f6b6d44fdf2f7e34b895a35e95708a7a91b20a3b51d187 \
--hash=sha256:1b4cbc7c3491ccb4aa17fcd8165649d01cf39f76de1696da8631b5f71b85401d \
--hash=sha256:1bc0baf5ef96b6ede57d47f4b8fe4d9d84019c3bfcbeb20a41edc6a6ee341f1f \
--hash=sha256:1c50fe28bbc2ced33386f298650d91218076c05420e6cbd790b913adc41659e7 \
--hash=sha256:1db38f4c5496827c1a501846d64d14c3b80c7e6714e406cd7dc36a9899fa1011 \
--hash=sha256:211d5a3eb6af8f513b8d4ca19a8c1b7accab1b5f0d3175f9826b03c1a920dc1f \
--hash=sha256:23851fb4e1b85ed3f6c2a27b777cdfe2e19fb5b38429a8faf38c7542b7665869 \
--hash=sha256:254eb48b9fa5ee9898a3c445825a1f340fe53712a098904b39b0bddba8ea3cb1 \
--hash=sha256:2625388c6c754520c37abaf3b41eb34d1cc4a373f457898f08606c8e362b891d \
--hash=sha256:281cb91036248400f4cc957495cccd44c275c2e0c5854f7e45ac5cf7dc193847 \
--hash=sha256:28a15fdad492a99b6eccfaaed66ef3f74050680545ea61ec8b2f4c538f1f1320 \
--hash=sha256:28b4f0d66fb834ff90f28209ac7bce77868c45d8c93e26f906709d9b7c2e1af9 \
--hash=sha256:2a925889534b3748302dae5dead07cc13480de1dac3aea80a941b729b471ef93 \
--hash=sha256:2b7b3bbfb4fe8ef40600792d762fbaa9057559f9d3fad209525b7a22b99e91fd \
--hash=sha256:2c9ad19a6cfcd5ea5c0d41161d22f9df1dcc277e9bef2751391334546a314c00 \
--hash=sha256:2cc961b171b3f3440f410489ab3573e86aea8736134ebbb40ea1338b7f0831bc \
--hash=sha256:2ce45c6627b22c47e390bc91a41c3d13032192e699fa0bea96e9671b373d69b0 \
--hash=sha256:2e06a3a98f916dd41d27f3105e02e7a40181c98c94b9158733d03a6f80506c09 \
--hash=sha256:304d5463e65a35d7bb0850550e0780395395f6fcf452f04db7d5ca7cecc425ac \
--hash=sha256:304d8e4d493af723536393eee0c689eb7813f4a474c8b479dee63f1fdd98f621 \
--hash=sha256:30fcd120b732aa79317f08dee04d7de0847822e4cf7ee0e9f445bb958832252c \
--hash=sha256:31f3930700408d211f13378ccbe1c40845d8da54bd0681fac3a9b5aae81c7aa8 \
--hash=sha256:34276fd796040bf0993ab33a369aa572e6979c7aab225a88893667ad8eac8f7a \
--hash=sha256:355ad8011081dec5412240c087a9a0c9d4d5039f3ed11a3f13e18c2b29b56c51 \
--hash=sha256:38a873987f3be698494da8b2e3085e29da02da7b633dce73e79c699a113d7bf0 \
--hash=sha256:39de2a259fc954455c57274dc94c79d5842774e1247a016aff30bc0efed0f4ef \
--hash=sha256:3d14b50de6bf4d0edf857a9386836846f982b8f524e188e2e68b96d702bcf4aa \
--hash=sha256:3d21b8b13c7592db2ac5e544a6d83187b995257472b0c9e8351b6d507ae37ed6 \
--hash=sha256:3d31298449090ab8d47b7b1b2a555ff73cac7ed438a08b7ac160980c7ebed649 \
--hash=sha256:3ddacd27458c45bdacd6bd6db644bfb730efbf9e830310186e3045c9c5be8fb2 \
--hash=sha256:3df041de8887954562c9b261cba85ca0e9ded74048daf125f45edcfaa4832229 \
--hash=sha256:40ab6bffa02ae10a0581e6c198be7d2d8ca5c2a0c64e4ed3465d766df457573e \
--hash=sha256:4275811936e2f06feff5e598fb42a1b7ae852da8e39605211892b56b81a34efd \
--hash=sha256:443eae2bf318abeaf6f15d785138f71fd6de770e99a92158b8b814265e079115 \
--hash=sha256:447441e76ec720b15e64418d32e092297340387053047c7c694f579efb0ee1d9 \
--hash=sha256:4495c5002a7b28557e7e222e77e0b661183e432b7d6d2e788101e3f240e05b8c \
--hash=sha256:44bd4fbb29dfbeba60e7d2bd000c59e4b21ddb3cc53912b14048d37092706d7c \
--hash=sha256:4685902cf26edf013ed7a3da0f426ebba7a00ebb9541386d835afbf002c11cab \
--hash=sha256:498dc3188ca05a68231ac3fdbfc7f57eb67e1343c30e0fea17f8218c1599b253 \
--hash=sha256:4c2b5031f63e331e3839b40aed2dd6f191e9c07edbde303e7876846ea1946995 \
--hash=sha256:4d48f2d08b9de5864e2c8744d4461b862fb149a18274abc8b698c45975573438 \
--hash=sha256:4f87960d57feabfb618e4e0af6e7371645fa26a277860739d6e5d6e0012c92f0 \
--hash=sha256:50e3adfb96fc189eb27b1cf62d3b598b89b4bb0420d93a3d3e42e137409011be \
--hash=sha256:51cf45226a9b588d0d2b4880c62d686934b63ab0bd79ca23ab0e9762eb27441b \
--hash=sha256:52aa6992700996af31f375de0c6bacd402b0097fe40b53c426b9f51a90ebabc7 \
--hash=sha256:55ea99acb17b9325618de155a0cd6a2e8f5d10be008113e1d433bbb58db543b2 \
--hash=sha256:56bc200a365efb37383b7852e4cc5898d3b2da5987289b543956cf8cad71018a \
--hash=sha256:588461c2e8384d309bd63e5826019b6977bc66d629b99ac8737bb795d7b2cb5a \
--hash=sha256:58ca3755ee7ff7f59b57789ec9833c9de9ea275405cdd240eda1f193112e398a \
--hash=sha256:58f361dcbab699cf8f42db3f47c8e7fd1036f138c23a5d08de9fde5f425a730c \
--hash=sha256:598a11a2c7ebaa5334bf698bf29568c9c390abac6a154d8170fedecd1cea38c5 \
--hash=sha256:59f63901b0031c3136cf64704dcb21de0bbae62ce2c9529bc39d27665463de37 \
--hash=sha256:5cde776b7cc66e4f6c99612cea4aa7269aa65863f7a15841b2c264f103822f4e \
--hash=sha256:5e2b6b57e9733d39f0c9fd3185efa6b8e29652c4cd8fe94180272cf6ed9a78c4 \
--hash=sha256:5fb29fb8cd1a46c27a1bf9613ad5ec2599310d46b4025d9556404a6b6a292800 \
--hash=sha256:6045373d5a89a5ec71afde535db987ca28e76dfa276c2d4c818265b375d4b055 \
--hash=sha256:619799369eeef6366ed3e8755a5670f4f2f0fb6b30a0fd7264dc0fdc2357058e \
--hash=sha256:62588a277bfb59def052abd940703fa35107152bf479781a878617d60faf8fb5 \
--hash=sha256:62603db9a7caa0802eaa28c1c46fecd7b3a263a774069c24c3c28c302448721c \
--hash=sha256:65cd72beeeca9d3aaea1201e5923859f308f952f9c71de93f06063c79f0f7a3b \
--hash=sha256:68eb192d85ab8e5f6ec69c2bc6ac0179fbf04a5ac1569d12fbef74883fe102d0 \
--hash=sha256:6bd128f206a7752ae1f2ab6c61bf8a24ba28913a10df8b14c2637b973ff97a80 \
--hash=sha256:6be488a102b8cf28d0391d8c4ba7748938ae28b78ad901f8585520fca33ead1a \
--hash=sha256:7218e8f32b0956cfcd048fd42d9d5779809745ca1d86113ca56f66e7ae1549c4 \
--hash=sha256:7441d755b7ab94f8d4eb3e43ec05482d760842fd263d003a99102d742cd835e2 \
--hash=sha256:749e97e1b32313717a565abbe321bc2190bc8b35f1a67e4cdbc7c56c8d8ffe58 \
--hash=sha256:75a3ceed0724d625d64b86ca20aba182e4df462e04c2414fc941c0f523f06aac \
--hash=sha256:780fbe7cab297b81dad9fb8dc5eb003c0468ffb0d9e5f65068c53a34661a96bc \
--hash=sha256:78456a747de8dc58360ffa581f30a002baf5aa28cb262536545e91f113ed7639 \
--hash=sha256:7967d08cf06dee78443b874f98c98036f624f3a4e73e11f9f64f5be4d25393cf \
--hash=sha256:7a881931aa470808df94a8c380eed2bbbc76cd9dc622310f99665658c821eb6d \
--hash=sha256:7dcd882da75ef9adf94903b1e3b9419e8aa8fb4c7396822b834b9ef7fb96954f \
--hash=sha256:7e841fb9010836c992c9f12fcbd43a831de93a5f726fc1ccd8ca1d0268c5014c \
--hash=sha256:7fdde2c9fd9e3eca40631e024664cf2584272cc8f96308cbe5fdfc930f51d8bc \
--hash=sha256:8024d00c3faf3fc0c16e07a69f4405e8eac7cc0ab15f65fe6cf43827c4cf72b4 \
--hash=sha256:80d02b6f04e92601a081dd97b23d3128033098bff5d35d392ddcc0476ea11253 \
--hash=sha256:838dcc90063569a0448120554591a1d6c4a4ffe11babf048908793154ab86ade \
--hash=sha256:849df64e889b2e17230d58410a03dba311a65b163508fd33679b2b737d4b7858 \
--hash=sha256:87475fabc8d9996fd9c27debb395e642e8c838d78a00b6e932227a0e06b81e26 \
--hash=sha256:87e50a3e7cb90af586b6c5faf23e302a970415ac73bd7bd90a515a04b427ef96 \
--hash=sha256:89b53f3cda69831909888e0494f4fa0bcd3537e3e138dabeb620bd6ad946bae8 \
--hash=sha256:8a893cc101149f80a653f82062ebc95b34525a2614382e1da5458fe7c6997249 \
--hash=sha256:8b2bfab86aa71ae13aa41a6a26aab338e0db2b8bc75434b05aea89e011ff35a4 \
--hash=sha256:8d86d6fc60743dc916eb79e2eb1ec4818e21e427731543af40a3021851174a13 \
--hash=sha256:915563965d418f986e7e145accc592eae9e1a1be3566ff98a05d7a9ec42a76e1 \
--hash=sha256:92888bb3187c5ba50500b00b3b310c9f2c651709d28036077680cb5255450a03 \
--hash=sha256:93223adc95033dd47133a46ccfc316a0139176fd79085762e27202ec56018f03 \
--hash=sha256:9373ad13ef0d2c0fb761e04e55bfdee5a08b52cef2c882c8fbe9935b1517152e \
--hash=sha256:9409a8bf35cf78353942504b24a57de3d75b708997a1e4bd8db71ac8633ce364 \
--hash=sha256:9b7f416ff0978e2f2249330527f0ad6fa02f4932e6199692d3b52da2048c19e4 \
--hash=sha256:9bde855991b7e362c146535e3136a50bfaffc0487d38b33ca7e5edefc6e23849 \
--hash=sha256:9cae88599c7219005d879f98e5ed53341e9a122af585e1091200358a3003d2a0 \
--hash=sha256:9cf9b1a857e25c4baceeb3624e92a56df3668f398c4acba74e174d81fb4d1d3a \
--hash=sha256:9f56f72050826f63dcee7a7f55b0a77168cb3bfc553fd405e7f8f9ece75a4036 \
--hash=sha256:a090bb2c68df85450502e3e20d665e3a5af9c65a84d6508ed477badd49166fd3 \
--hash=sha256:a192e2c40070d92c3ccf777e3a5c4ff515573cd2bb7ed0c537fdadbbec5bbf21 \
--hash=sha256:a19a731138fc27d5682277d3b9df22855cea1239bce7fcec5f78f42ef2d1f3c3 \
--hash=sha256:a66c3bc5ab1f0ff2164fc9965ddd611ff0802173f4b9d24554c563f6ab7e1d6e \
--hash=sha256:a815775b6c38d4e0ff7bcffbeba67feded90202bb6a226b8dd35f1c855217413 \
--hash=sha256:a89012d6d5476ee112d20d998570ed58df2260a852afb1758809cd6900411d21 \
--hash=sha256:ae4f5fea5b8b8ccff88238cc8569303e5ee95efae67fa62922a311397a71f346 \
--hash=sha256:b6856554c4f44d79fc2307d5768854310a8f0096e501c75637542c82292b0429 \
--hash=sha256:b6b751274acb69d77b3323d6b7dbaa3c7fdfc1eb829b7eb61d262f32e1af9685 \
--hash=sha256:b736353c0a625bbd5fcec108576e2385db3496f4f771f785ff32e108d3c3bc45 \
--hash=sha256:b7fd005a73d9e657273b7a10dc71a9e03c8fb9ee6999798d6918ce095b81ac7f \
--hash=sha256:b91363207bd9dc966a691e959bb47f64b30f7ac4b072be9968b366982f7db77c \
--hash=sha256:ba0b1d2620edf869789c3879223f52bf2afc5d31b3cb47cc57b3a12c05e2aa9d \
--hash=sha256:bbbfc8e28816f19d7c0f1816664980c0a9875d01b27cdf8eedddb639d9e108ad \
--hash=sha256:bd16aabe4a02a297c23417aa17ac6299dbd8c49f673bcd645b4929b11f5a4400 \
--hash=sha256:c0afc6800ba57ccc350374c5bd6150419915d95ce93cdbab2d783d75eaf30ecb \
--hash=sha256:c6708715abcf3c73b99508253e961a9967f02fe536532834149574eda6de0d1c \
--hash=sha256:c7c9ab723cde841fefb34efbad91e87f00a674b1fe1cd0784fde742bf2c154dc \
--hash=sha256:c8f3d67aeaf55f017982b73683f0e7342ba2f6635a78f69ce89ebb26aa411e5c \
--hash=sha256:c9790464842f85f437dbbb54417eda1e0e6bfc52dd8d22d6fd1c994b73b2dc74 \
--hash=sha256:ca403d7e4798f525fdfc78e258820419cbbd0f0ecbab9de7840e3c017cf6b8cf \
--hash=sha256:d008d90a7f2471519aef0c90dfbe73b3e6e4d5e66ac48e19154c17e89e98b604 \
--hash=sha256:d19fbd981a488e22cd04883659ca6b08f50b5974f9fd7c95655ef6a043e5893f \
--hash=sha256:d1befeed746d247c81127bb14de9dc3d30edb6e5976d34f83f86ed262b1d9105 \
--hash=sha256:d2374b62878abb00cd8309b32af6c0b715cd02dec0ca74ef12e5069bdc64144a \
--hash=sha256:d376bbd28b3a8999db1a103b3b388aee6f1ddeb3e51bc2172993efdcd86e064d \
--hash=sha256:d4a7319f304a774bed22115bc891618e45f85065ab44ea6acd07d274e750519a \
--hash=sha256:d6734d2ef8a50fbf8445c139477da401f50d62a0606bf00e20ec6d87773fefb1 \
--hash=sha256:d760fe2a4d7c3b226cb9026d6a842868d52a7901bd98420e1baf14e80da85cf5 \
--hash=sha256:d913de495d90407cd859d263bee2e5d1a4ed3eb6573c04e70d9ec619a7cbed7f \
--hash=sha256:db19d07e2e0129e974a0e65d0064fc222a446cd5122c2fd4184d2af9fc734a9e \
--hash=sha256:dca9ab98072a5a54ebacebdc45f53e645336b320c667410b061be1ca588ae709 \
--hash=sha256:ddc7dacc8ece3a182e7f15cb862d1fd616b46d076cb1ae9dd232b2c38b655874 \
--hash=sha256:ddf19c062bea7a0cc80f519243d2c01dd091be0cf952a0750d4ad576709559f5 \
--hash=sha256:def79fa35ef0cef8d2accec024f4fdc7ead3012ff02f5215c783f39f03ef8cfc \
--hash=sha256:df29a0a7107f7011e77f4eebdddec4c7331e24d787a0b21a46d63bdf7445da95 \
--hash=sha256:e09a3942ecbdee5cce73ea9d42da82b81b72ac1bf031ce069b93b5adf4eac8cd \
--hash=sha256:e242bb1c5e76e97dfa9e7f209a71e93a01d7f19ffdd5cfbb2e2d55b4f08f8ab0 \
--hash=sha256:e243bd13217235fc7290c621941c3f5cc8b66e4872495be821d7436ba2fb838d \
--hash=sha256:e2af3aad578aa6bd1384bcf4750fc285e5a9de53f40b7d41e5a0bf748edeb2b3 \
--hash=sha256:e4e81e09c1578b8df602e3db08b0b3ea0a6947ad612f52bf8dc5ea8d47691f0c \
--hash=sha256:e54da4baf05720032d527874d40b65fa4d7e5c6c6a43d0c3adbeffcaf275a2b3 \
--hash=sha256:e80e6c2f55656b4824d72065abb4ddd6a525c74bd78a0aab5d9fc2cf4fb5af50 \
--hash=sha256:ed2a239c0ea213acc1908150a3037257083c7c083128f1a4cec2ec4b97dca491 \
--hash=sha256:ed905975ab14056a2e5eb1c376cb2e1ebc5396baf84163939c518556fccde9f5 \
--hash=sha256:ee21e28f0430bd6dc9086c6e525d5e818a44a5ad19720c8a0ef766792f3eb5e5 \
--hash=sha256:ee43c17b173d46a3212baa6ead3ae258eeabdae48c263a01ccf0218c366dd655 \
--hash=sha256:ef4fcbf3327382cd4c9f540babd61248208af7b93eec4de397b4d5f58a09e288 \
--hash=sha256:eff0ac9dbe711a4aee69bf04a83896aa9b85f19641264053a9f6d48573abb7dd \
--hash=sha256:f0aa869112ef88429ae17820d99c3dd9504c9e9c671d3c246f3d7442cb051084 \
--hash=sha256:f3c96f633825733f735c5a9cf21d21a257d8e1edf0b1cee0a064b9c424ca0f7d \
--hash=sha256:f5833ad231be5eb6553de524a70f48d71b2c8563101750531e0b80184e175cd4 \
--hash=sha256:f5ec61164adcec446f8969a3358ec3f9b26bbda3b9213e5586d219afa8df2915 \
--hash=sha256:f7d486c83842422badd511868fd8a9a20e9407ace71564b6af47ce7e60a336c1 \
--hash=sha256:fb9e68df06293761f9fe66ade60a9bc6d0f5e42b8acf2939a9158af86ab0e5bd \
--hash=sha256:fc14a032f813bf5fe624d991960ea83e9715adc27e4c1830a2361eb1d02ac341 \
--hash=sha256:fcff63213e8e6e47770541a4607175404f47cbb3ebea7b6058cc82d524a0e424 \
--hash=sha256:fd1fbe0f116b6e55da77aca2c6ddcddcfac2186cbf78bdebf40fc156efca389d \
--hash=sha256:fe9753dfee015c570d73df76f899f18444d41388bffcde097deba51c4fadbb9f
# via requests
click==8.5.0 \
--hash=sha256:255bc9599cf7748b4b1a446ccc735421bd08a2ae529a8b88597d3de5664ee360 \
--hash=sha256:ba0d2089de75ea0310e2dde03160e6ca10009947fb95a182f9b54021bb272e34
# via
# black
# celery
# click-didyoumean
# click-plugins
# click-repl
# pip-tools
click-didyoumean==0.3.1 \
--hash=sha256:4f82fdff0dbe64ef8ab2279bd6aa3f6a99c3b28c05aa09cbfc07c9d7fbb5a463 \
--hash=sha256:5c4bb6007cfea5f2fd6583a2fb6701a22a41eb98957e63d0fac41c10e7c3117c
# via celery
click-plugins==1.1.1.2 \
--hash=sha256:008d65743833ffc1f5417bf0e78e8d2c23aab04d9745ba817bd3e71b0feb6aa6 \
--hash=sha256:d7af3984a99d243c131aa1a828331e7630f4a88a9741fd05c927b204bcf92261
# via celery
click-repl==0.4.1 \
--hash=sha256:5cb10881d4c5ebaa8695eceb69911af3062ee78342812b713564b17aad333eb5 \
--hash=sha256:c32a1cf6f95e5bd6e92076f81ce24eafd33f2f0ffb0135887e335b8e446d1c0b
# via celery
defusedxml==0.7.1 \
--hash=sha256:1bb3032db185915b62d7c6209c5a8792be6a32ab2fedacc84e01b52c51aa3e69 \
--hash=sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61
# via evection (pyproject.toml)
django==5.2.18 \
--hash=sha256:461c5dd06d2ea16bd5ca37d3f46e4def1d6b0fe7588c6f4e2119517bb0af8b2d \
--hash=sha256:92ed81d500be6408ecd704d7bd1366c534f30427bffcc63c5fefb129561aec7c
# via
# django-csp
# django-htmx
# evection (pyproject.toml)
django-csp==4.0 \
--hash=sha256:b27010bb702eb20a3dad329178df2b61a2b82d338b70fbdc13c3a3bd28712833 \
--hash=sha256:d5a0a05463a6b75a4f1fc1828c58c89af8db9364d09fc6e12f122b4d7f3d00dc
# via evection (pyproject.toml)
django-environ==0.14.0 \
--hash=sha256:8dbe8a57f0a540ab8abd6f54f230de5e99e3a2c9d797cb9caecb037bca3d47d8 \
--hash=sha256:b6c48d93b9d2ff8a3ea14099e90c35aa4f101c1b4d5f262dfee0d27b06742ed7
# via evection (pyproject.toml)
django-htmx==1.29.0 \
--hash=sha256:0ec5be1645ed71e6787bd75e0250624f00274ac3bb1730f07b6629a95688929b \
--hash=sha256:337dfa35b8da13fd68bf968f2b9cc9a4144a4e71f06c204d7ff10f7343988102
# via evection (pyproject.toml)
factory-boy==3.3.3 \
--hash=sha256:1c39e3289f7e667c4285433f305f8d506efc2fe9c73aaea4151ebd5cdea394fc \
--hash=sha256:866862d226128dfac7f2b4160287e899daf54f2612778327dd03d0e2cb1e3d03
# via evection (pyproject.toml)
faker==40.44.0 \
--hash=sha256:97245a3756dcaf35418b89bea9f03ae6881557db0032bf048744c2aaee2a9108 \
--hash=sha256:db808a3ec33fe9ce5caf40b9fb61f85f46e429a56966f765209272adec83e4c6
# via factory-boy
feedparser==6.0.14 \
--hash=sha256:088679b0c4b543ee211a820dd544698c76a402122eae7473c04a43425f283d06 \
--hash=sha256:e35e3f760151b0c3b22cac9684155cae186a233e16c49bcbc6c49e91e3131137
# via evection (pyproject.toml)
feedparser-sgmllib==2.1.0 \
--hash=sha256:2cab2d43b95a954f920f18aebce7a4dbbb3f539780b127e2aa114f579821e01d \
--hash=sha256:61facf2918c4389b5b00714f76c5e03431ffcd94cd1f51d657edd6cd7c396579
# via feedparser
freezegun==1.5.5 \
--hash=sha256:ac7742a6cc6c25a2c35e9292dfd554b897b517d2dec26891a2e8debf205cb94a \
--hash=sha256:cd557f4a75cf074e84bc374249b9dd491eaeacd61376b9eb3c423282211619d2
# via evection (pyproject.toml)
gunicorn==23.0.0 \
--hash=sha256:ec400d38950de4dfd418cff8328b2c8faed0edb0d517d3394e457c317908ca4d \
--hash=sha256:f014447a0101dc57e294f6c18ca6b40227a4c90e9bdb586042628030cba004ec
# via evection (pyproject.toml)
idna==3.20 \
--hash=sha256:a7db850025b95ded1eae8a46181a1a6c56c92c96f0e2b005d9ff8dc0210cab44 \
--hash=sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c
# via requests
iniconfig==2.3.1 \
--hash=sha256:67f4b9c50da0dedf52af349e7749a80a9057a5031199791b906c3bb3ae878960 \
--hash=sha256:9121e2c1fdb355232495be3194c8dfe87ccc2d5dee45947b78e68f499790d7a7
# via pytest
kombu==5.6.2 \
--hash=sha256:8060497058066c6f5aed7c26d7cd0d3b574990b09de842a8c5aaed0b92cc5a55 \
--hash=sha256:efcfc559da324d41d61ca311b0c64965ea35b4c55cc04ee36e55386145dace93
# via celery
mypy-extensions==1.1.0 \
--hash=sha256:1be4cccdb0f2482337c4743e60421de3a356cd97508abadd57d47403e94f5505 \
--hash=sha256:52e68efc3284861e772bbcd66823fde5ae21fd2fdb51c62a211403730b916558
# via black
nh3==0.3.7 \
--hash=sha256:157ec1eb7a62f3d9a7badb8d82d89aa810e3e24e097eedfa481a25d0c8a99877 \
--hash=sha256:15f5fbf090f5c88d61c820e1fc1fceecb6520cca9fe85649c06b57ef9dc9ff62 \
--hash=sha256:18f4278ecd157d43cb35acd5aae9f35cfa79f546b4922bd86536adc0f6312102 \
--hash=sha256:19f288c938ec6eef1f5d2c6cab47838e71fef8097e1c1233802be5a6230ba086 \
--hash=sha256:4968fe8d2db97c6f047659bf46a449fd8ec377f44ebf3e0a1b96c0d3a333ae32 \
--hash=sha256:5ffdfcb9a686ffb12765376bcfb6b5b55728516d3c0ee317d29982381ded3df8 \
--hash=sha256:614dac4a4c36ad084e78447d16fe898dedd762e354a7ab9cda2984e82f67883d \
--hash=sha256:618e3059caf41ccdf5dcccb3fa9df4cf6e4efe23d1382a8bbfca272a8a4f8bfc \
--hash=sha256:6698a822132beedab80f131c08d8d0ac5a178ddeb488d02ca4b67716ecfac7af \
--hash=sha256:6c3aa50eb26e9228238271db9f983cbc3b006dfbfeca2d4dc34c33ddc6ac5ea5 \
--hash=sha256:6e4280115d44c3b278eef712a86748c1a723105cd79feec46952383117ab4e59 \
--hash=sha256:70f5ac8626e899a4bab0ef74ca2f5bd602f49c7b739e6e5026b4afc6d63dac42 \
--hash=sha256:71860d01c16f4d8c72e334e0674beb2b0899dbd0bf760de18932ef4390303848 \
--hash=sha256:808def0c8c07843e6e50dc84f532457bfa2cfd17417b219a5d9e7c773709331a \
--hash=sha256:874b7d67a067bd29a59223f6270fc30da4edd8e6d87fd219fc93bcbaa662c946 \
--hash=sha256:91a4dab4e94d9fc54b9f67b1adfb23e81fab7ab43f33c3b8c97be9aa38f789ba \
--hash=sha256:94fd6e59553fbb9ffd8ba71bbd5a54e3126ba01799a097ae30d5341d750bc6ac \
--hash=sha256:9b7279d43323a25225df23576af6594a16693f61431170848b8b2ac21ad4f174 \
--hash=sha256:bc42bb1193c1e28a1e74c2cabaca178e118a7103e8832699fef8a2b3e2496493 \
--hash=sha256:be53a4825585f701955cb9baf49f478f56eb81e20294329fe4bc689dd5dd81fa \
--hash=sha256:d56e76bd3cadb09b6b0cef364850811663734b348a25f5f587a2819c495367bd \
--hash=sha256:de2b2aab32ea303405debefdcfc58043d3e635fa3f67b9eb140d2b0e0c0d2563 \
--hash=sha256:e8fd1ab205258b29254f72db377d99e2c96aa7653ef3b015ccab0420b094b506 \
--hash=sha256:eae64328e46a25785535afcb6885b6f182ecaf5ee8c88f8c075422db8aacc65b \
--hash=sha256:f04b7d333b27f13ca439da3cf1c75c2fba34f104969f6ce4ac8e7079699c2f4a \
--hash=sha256:f266d3f1b3647449923a8e406524632220dd5d8b647078dfe45b885d33d10479 \
--hash=sha256:fd4a70efb45d5372174f718878eb7a35c12677626a63b2f103b23b833457dcac
# via evection (pyproject.toml)
packaging==26.3 \
--hash=sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79 \
--hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c
# via
# black
# build
# django-csp
# gunicorn
# kombu
# pytest
# wheel
pathspec==1.1.1 \
--hash=sha256:17db5ecd524104a120e173814c90367a96a98d07c45b2e10c2f3919fff91bf5a \
--hash=sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189
# via black
pip-tools==7.6.2 \
--hash=sha256:04dbe0e25f3b60fbdeee17d0590dac7fe20039876aa410a946fc54dc926c09ba \
--hash=sha256:f37eb4b736859c2c30230e50977312c61918b7ef931071ff7811fd067715c25f
# via evection (pyproject.toml)
platformdirs==4.14.2 \
--hash=sha256:6d13b4c1d66b7548dcec9e3db13c1ba9d9adb65041b5d07869313afc07e6e41b \
--hash=sha256:dc1f93390f2a9265e44e5afc8667ec7faf737739604e39bc538fec889720c36b
# via black
pluggy==1.7.0 \
--hash=sha256:7dd7b0d8832ba3cb632c306926ded123429211b83641b35dc5c41ad2d34f9bec \
--hash=sha256:d1eaa46ebb595891b860ab086b4d09c8588af65ebd4361b8e8f4bb8920b90ba8
# via pytest
prompt-toolkit==3.0.53 \
--hash=sha256:01c0891d7f9237d5e339f7d3e42cdae80b7534abb1c7c0e3352efba6231492f2 \
--hash=sha256:9ec8a0ad96d5c56148b3f914aa79c1564c3fde5d2e6b876e7bc327e353cf8fa6
# via click-repl
psycopg[binary]==3.3.6 \
--hash=sha256:a1db9f7148b06a28606767efaca51fa6f9398c5c0a3810519be69d7000bdb631 \
--hash=sha256:c081f2250df751a943036e42db6df4571c66cd0aabe8291a7a506512b12007d2
# via evection (pyproject.toml)
psycopg-binary==3.3.6 \
--hash=sha256:05a83ac9fd52b9bca7cb5ab04b3691163170bd16f53defa27216ea3aa07ee781 \
--hash=sha256:0a52991594ac4db888c7d39bccef331797e30cb31a95cae02cf2607f83a42dc2 \
--hash=sha256:0bf08b749cc144f33b44a91b78e3f71c60eb07963746a0df5a100b36ce3d7475 \
--hash=sha256:0ebfad5d131de9f892ae9e70cc7616207768b6714b66a52d4612b8ceaf78b372 \
--hash=sha256:1679a1cb93fbe5a6d1fd58d82cbddcc6fcb8c61446ba7cae6eb2a7b19bc585de \
--hash=sha256:198a48e68cc99ccac03ba95ac857e73aa66f3bf6be77019fafb0832a05f7ad03 \
--hash=sha256:1fbd30e537dab22cafdf080608f10148fe2a5f3a61294ddb5113caac8a623840 \
--hash=sha256:289aadd6a00e151203c081f708348ec89f1e483c9b510ef4ac3981f847f01f79 \
--hash=sha256:2f122603f36050937982abf9668d8bc4769a79f7c93a65013b1c49f1cab7b56b \
--hash=sha256:303732e798fe6729f8e12021b9c96107df8e95ecec4dd487c67b98ec2a59435e \
--hash=sha256:31cd942c23f613276b81a6e6598cefa12960058b0f46e1e874b540c793f6aca5 \
--hash=sha256:366db6e97e66b37211475f20c4c1324a2dc0dd825e46d4e87f9d599304d276f9 \
--hash=sha256:373704aea331d3f3e3402c125a1543f5875e2986ebb54f97d1647942161f803f \
--hash=sha256:37d40450659401600e6d043ff586c89a71a69f33cbb8bcdba6cdb2569beecdbe \
--hash=sha256:37e517c146b185f9c0c6e8d0a0ebbdeeeb67896af28466e032bc810d0c7dc7a7 \
--hash=sha256:3af90f92769d8cc10f94515ee7a0aef36ea85ca733a0ce22858f6e0953f41138 \
--hash=sha256:3c9e663b2e800e3218994cf948c11bcc2844e6491b34aa80d089baf6531827bf \
--hash=sha256:3f84dab25e0385692ee13274c68678377e0b1a70ab9d14e56264cbf61f60c62d \
--hash=sha256:4690cf67738f0e0e49a32aeec99bf0e4595cc2b4f1af984a4345394b1dcff91a \
--hash=sha256:566dd827f17728efdf7d88a5b066f815170f6fdad13967ae952842d90e6aaa9f \
--hash=sha256:5927b7ba63153cd8e9862987290a2b783a5c590daf2a4ef981700cc3569166d4 \
--hash=sha256:5ad8f35e67cc16d1fad1fa8c88972dc9b3a3141ea67897399904edab96a301b6 \
--hash=sha256:5ea8beeb5541780b4b50b462eeacbc4f594ce3b911dc20c81c75f267876f71d2 \
--hash=sha256:5f598f19fa9a91540b5cee17932ffd227b7b53a481605bcc4573c0eafa647300 \
--hash=sha256:612382ac3ed13651c7fa44b5fee9fbf7baaa2ddbc6f500391672682c5f1df9e0 \
--hash=sha256:6ff05561e4a067d35507dc5c90f1deb2ec1c9703ac5cccc1bc26e08a197f9c5a \
--hash=sha256:7308c93cf0b19bbaf8e6ff0a6ad50d3c442385739245fe15a8d593bf841734a6 \
--hash=sha256:79a2a1c3449f6c3409427078ed1cec10de79f3023cb5f2504f0597d350ad46c7 \
--hash=sha256:7beb3e41c9a1e509f3ed85263386588cbe3e975aa67be21f79f44fd35ffaeefc \
--hash=sha256:86147cb5d140341c3363fb5bacce31f8d5543902a46699d3c536b101bbceaf9e \
--hash=sha256:889e42acec10450185e0cdfb396f375e2c1a8d7737c114830a7fde4654f59e30 \
--hash=sha256:910ace140e3e7b7596898d083f37a8fe90c5c40684252ad4e682364b2cd3deba \
--hash=sha256:955e3dd94da361e052d2e49acf591017158dc8f8ed2c8a42c2e3943403c39dc2 \
--hash=sha256:9892188bb15e5803beb51afe8a25add6b56be391a53058e8bca03b74e1e6bf22 \
--hash=sha256:98c02090d88f2ebc0ec1e8da538f77d225ce0fffecf372aa39262e62a1b054ef \
--hash=sha256:9b2f11794e017ce340934e35de46181c46ef71ec75ea3d85dd75cd836761c01e \
--hash=sha256:a2e44a342d2aee40508e28a563d8961c39d9bbd8cae36d8578f0a3c6658aab0f \
--hash=sha256:a4ee3bdd5468a725f2a4d9aab8a74b6d0279f768c8b5d3aeb102c5307ff3d59c \
--hash=sha256:a5165300324efd5a772c48a88ab3a928513ab3979fca76553e62ee815f7b2b9c \
--hash=sha256:a9348c5b43a3bb5ef8c2e89d5237c9c87eeafb01d338c84a7aebbc5cd0313299 \
--hash=sha256:aa73160077345ec21b3f51e8e24b3de2e99586217e497629326eb9b2ea88c52e \
--hash=sha256:ad1c785e784cfd87e8436c6b7702f2d321fc39601bbaf29bc63a41a867091638 \
--hash=sha256:b3f75dee0f9afafabe4edc52c4842f1e1878ed2069bd05b22d6fe961e97e4dba \
--hash=sha256:b599defe9190b17e9907c8b4d114c181e702c87efcd1b8a0ad40971cdcc4634a \
--hash=sha256:b82491019b884d62318b5f30706c3d7e6d4e5a6cb7eabcb3edc0c1b0fdaceae9 \
--hash=sha256:b8ece331509f7a975b90501f41e83ad905e4141753fedf3f2711b2bc70a8efbc \
--hash=sha256:b979a42815410432420275412633960807178b1ce26591a16ce06e78a5bd4bb2 \
--hash=sha256:be4f9b3c9338ac5dd217c5847e21521b396c8117f78dc420d495a5c49bbef874 \
--hash=sha256:bf8c8481d026b85dd70c5fa7dde85b2333aed0b32a2602bcd38a900cbd78a49c \
--hash=sha256:c61617eaae0112ca154da87ffb99b73af2c74067acac28dfb9a4455b019dff2e \
--hash=sha256:c6d19cb4999d03231e8730a5f66c8f5068bc3b532677eb39dab0f600bff3e312 \
--hash=sha256:c7753871eb57e6a5f4646f6168590c6653073dea5e9e720b201c8875332df4c8 \
--hash=sha256:c7f92daa0d2a1c76f07264abddf8cbabd30152a2f09c3270e50f0c7efdf5dcac \
--hash=sha256:cbd5f73073ed19c378d4c35499db1e3e703a5b1a324e521204065967bfaa7a18 \
--hash=sha256:cec5ea900390897d0b46130f60bc2883bf19c314f9044235217c8be88b0ef269 \
--hash=sha256:d636338c8f21b0df2f84657b00bc34f9313f826ef93f1155bc743607e4a0c5eb \
--hash=sha256:dc75da5a20951049f7b773145f998f69d181adad9c58a0ff36e0cf1d73c10e10 \
--hash=sha256:e23a66a763fbe83fcc210bc77c27e5a5ea380ebf091c06f34d8561b695e5a40f \
--hash=sha256:e8cbb54454dbf1bbf2ff08dd7693e8d94ac94b1a20f70f4b3b813d52ecb5cbc1 \
--hash=sha256:ee2c4728c691245e24501fcd7a97b5b381236b9985bc445bba88cdce7d1b5784 \
--hash=sha256:f0535693ce476a722b718b002d5d2c27d47e71ca945276ac194409c98e74c492 \
--hash=sha256:f19cc87343eaa55255e76b31259a570072ac95d6ae82c92dd34b97691f5e49dc \
--hash=sha256:f21d057f3e5f5491067e5b292498073b73847d48799b099803fef100775fcc52 \
--hash=sha256:f87dbdc42e78ee0f7ea180c03f8c78e80a949e373066629bd90fefff10552dff \
--hash=sha256:fa34eb47969297471db7b7f193622c7e3ee839ec05abd05f1fe104d5b1b1dcf4 \
--hash=sha256:fdccb3a0e184b03e9baa673b15a809cf36c339c85dbda0ebc25a698846dfbee8
# via psycopg
pygments==2.21.0 \
--hash=sha256:2363c69b61c4a97c838da3b130dcd6468f4848992b21a82f2a63ec34377137d9 \
--hash=sha256:610ca751c9bc2492b38eb9a38a7fbc93edbbb2d7182edaf34e66ae493dee5c8c
# via pytest
pyjwt==2.15.1 \
--hash=sha256:42d59d631f7768a1028a64c7ff581a9bf7519804daf91fc5b6c56e30eec5e193 \
--hash=sha256:4f259e80cdfb6b3fc18a7de51fd1ef9ec79652f25019bae68975ca2468a34df8
# via redis
pyproject-hooks==1.3.3 \
--hash=sha256:5fc53fdac9f7bd63fbcdc868fb5f90b4784d78a53a3d3388cd738b807441a20b \
--hash=sha256:defda19b854fa0d3bd4f76ea4ddcba8abd7dcfcdd585a6690ade050744fc5f43
# via
# build
# pip-tools
pytest==8.4.2 \
--hash=sha256:86c0d0b93306b961d58d62a4db4879f27fe25513d4b969df351abdddb3c30e01 \
--hash=sha256:872f880de3fc3a5bdc88a11b39c9710c3497a547cfa9320bc3c5e62fbf272e79
# via
# evection (pyproject.toml)
# pytest-django
pytest-django==4.14.0 \
--hash=sha256:26787dd3f422cfbab8f55b80a776e2edea7a11092cb74e960bef1312515708ef \
--hash=sha256:c533b08d89cc675efcd5398eea270b34547e35f9a3608e2c9748dd88428ea187
# via evection (pyproject.toml)
python-dateutil==3.0.0 \
--hash=sha256:3c79630202e4152c6add678446984236efe4f57e84d8e892aab0637bab82e9fd \
--hash=sha256:7ebcb3c29a92f3ceafe79a7a48a5c45677e9e004bbeec5742ac0bbb00560b7a2
# via
# celery
# freezegun
pytokens==0.4.1 \
--hash=sha256:0fc71786e629cef478cbf29d7ea1923299181d0699dbe7c3c0f4a583811d9fc1 \
--hash=sha256:11edda0942da80ff58c4408407616a310adecae1ddd22eef8c692fe266fa5009 \
--hash=sha256:140709331e846b728475786df8aeb27d24f48cbcf7bcd449f8de75cae7a45083 \
--hash=sha256:24afde1f53d95348b5a0eb19488661147285ca4dd7ed752bbc3e1c6242a304d1 \
--hash=sha256:26cef14744a8385f35d0e095dc8b3a7583f6c953c2e3d269c7f82484bf5ad2de \
--hash=sha256:27b83ad28825978742beef057bfe406ad6ed524b2d28c252c5de7b4a6dd48fa2 \
--hash=sha256:292052fe80923aae2260c073f822ceba21f3872ced9a68bb7953b348e561179a \
--hash=sha256:29d1d8fb1030af4d231789959f21821ab6325e463f0503a61d204343c9b355d1 \
--hash=sha256:2a44ed93ea23415c54f3face3b65ef2b844d96aeb3455b8a69b3df6beab6acc5 \
--hash=sha256:30f51edd9bb7f85c748979384165601d028b84f7bd13fe14d3e065304093916a \
--hash=sha256:34bcc734bd2f2d5fe3b34e7b3c0116bfb2397f2d9666139988e7a3eb5f7400e3 \
--hash=sha256:3ad72b851e781478366288743198101e5eb34a414f1d5627cdd585ca3b25f1db \
--hash=sha256:3f901fe783e06e48e8cbdc82d631fca8f118333798193e026a50ce1b3757ea68 \
--hash=sha256:42f144f3aafa5d92bad964d471a581651e28b24434d184871bd02e3a0d956037 \
--hash=sha256:4a14d5f5fc78ce85e426aa159489e2d5961acf0e47575e08f35584009178e321 \
--hash=sha256:4a58d057208cb9075c144950d789511220b07636dd2e4708d5645d24de666bdc \
--hash=sha256:4e691d7f5186bd2842c14813f79f8884bb03f5995f0575272009982c5ac6c0f7 \
--hash=sha256:5502408cab1cb18e128570f8d598981c68a50d0cbd7c61312a90507cd3a1276f \
--hash=sha256:584c80c24b078eec1e227079d56dc22ff755e0ba8654d8383b2c549107528918 \
--hash=sha256:5ad948d085ed6c16413eb5fec6b3e02fa00dc29a2534f088d3302c47eb59adf9 \
--hash=sha256:670d286910b531c7b7e3c0b453fd8156f250adb140146d234a82219459b9640c \
--hash=sha256:682fa37ff4d8e95f7df6fe6fe6a431e8ed8e788023c6bcc0f0880a12eab80ad1 \
--hash=sha256:6d6c4268598f762bc8e91f5dbf2ab2f61f7b95bdc07953b602db879b3c8c18e1 \
--hash=sha256:79fc6b8699564e1f9b521582c35435f1bd32dd06822322ec44afdeba666d8cb3 \
--hash=sha256:8bdb9d0ce90cbf99c525e75a2fa415144fd570a1ba987380190e8b786bc6ef9b \
--hash=sha256:8fcb9ba3709ff77e77f1c7022ff11d13553f3c30299a9fe246a166903e9091eb \
--hash=sha256:941d4343bf27b605e9213b26bfa1c4bf197c9c599a9627eb7305b0defcfe40c1 \
--hash=sha256:967cf6e3fd4adf7de8fc73cd3043754ae79c36475c1c11d514fc72cf5490094a \
--hash=sha256:970b08dd6b86058b6dc07efe9e98414f5102974716232d10f32ff39701e841c4 \
--hash=sha256:97f50fd18543be72da51dd505e2ed20d2228c74e0464e4262e4899797803d7fa \
--hash=sha256:9bd7d7f544d362576be74f9d5901a22f317efc20046efe2034dced238cbbfe78 \
--hash=sha256:add8bf86b71a5d9fb5b89f023a80b791e04fba57960aa790cc6125f7f1d39dfe \
--hash=sha256:b35d7e5ad269804f6697727702da3c517bb8a5228afa450ab0fa787732055fc9 \
--hash=sha256:b49750419d300e2b5a3813cf229d4e5a4c728dae470bcc89867a9ad6f25a722d \
--hash=sha256:d31b97b3de0f61571a124a00ffe9a81fb9939146c122c11060725bd5aea79975 \
--hash=sha256:d70e77c55ae8380c91c0c18dea05951482e263982911fc7410b1ffd1dadd3440 \
--hash=sha256:d9907d61f15bf7261d7e775bd5d7ee4d2930e04424bab1972591918497623a16 \
--hash=sha256:da5baeaf7116dced9c6bb76dc31ba04a2dc3695f3d9f74741d7910122b456edc \
--hash=sha256:dc74c035f9bfca0255c1af77ddd2d6ae8419012805453e4b0e7513e17904545d \
--hash=sha256:dcafc12c30dbaf1e2af0490978352e0c4041a7cde31f4f81435c2a5e8b9cabb6 \
--hash=sha256:ee44d0f85b803321710f9239f335aafe16553b39106384cef8e6de40cb4ef2f6 \
--hash=sha256:f66a6bbe741bd431f6d741e617e0f39ec7257ca1f89089593479347cc4d13324
# via black
pyyaml==6.0.3 \
--hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \
--hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \
--hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \
--hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \
--hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \
--hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \
--hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \
--hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \
--hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \
--hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \
--hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \
--hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \
--hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \
--hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \
--hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \
--hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \
--hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \
--hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \
--hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \
--hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \
--hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \
--hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \
--hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \
--hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \
--hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \
--hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \
--hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \
--hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \
--hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \
--hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \
--hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \
--hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \
--hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \
--hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \
--hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \
--hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \
--hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \
--hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \
--hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \
--hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \
--hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \
--hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \
--hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \
--hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \
--hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \
--hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \
--hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \
--hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \
--hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \
--hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \
--hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \
--hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \
--hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \
--hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \
--hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \
--hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \
--hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \
--hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \
--hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \
--hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \
--hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \
--hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \
--hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \
--hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \
--hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \
--hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \
--hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \
--hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \
--hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \
--hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \
--hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \
--hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \
--hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0
# via responses
redis==5.3.1 \
--hash=sha256:ca49577a531ea64039b5a36db3d6cd1a0c7a60c34124d46924a45b956e8cf14c \
--hash=sha256:dc1909bd24669cc31b5f67a039700b16ec30571096c5f1f0d9d2324bff31af97
# via evection (pyproject.toml)
requests==2.34.2 \
--hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \
--hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed
# via
# evection (pyproject.toml)
# responses
responses==0.26.3 \
--hash=sha256:74474f799334ac4f37d93b6437ecc3bb1bb5c77a8d31780a338643be2dce0af8 \
--hash=sha256:b0c11ca8131b8b227b8d5108e6ed39772222bd5aab030ed430e8f99057c4c409
# via evection (pyproject.toml)
ruff==0.17.3 \
--hash=sha256:029c37cbd3de48ca37aa12117bead2d065f15da17d66bf84e0d4274013705db2 \
--hash=sha256:27bb571ac78fef178fe5118d2827df95c2aae74a48fff2b15f6acd03af5ff5bb \
--hash=sha256:39405f341deec934b4289634324ad695f72df2970f9121582872eaf5b722f0f5 \
--hash=sha256:4590b9d86c0b1543ebb09094254e191cd43fe8edaf330baa035aab03da251532 \
--hash=sha256:46539cb031b01fd29ae0a02142483c46640b233ee12bb1c1b690bffa1555d5ca \
--hash=sha256:56b8ffb3f3b3cbccb8b76b71b38261b7ed982ef5d20f5b447b1b61a7f4949b24 \
--hash=sha256:68074094f55ca3d29ab19491f640282f7a84061c4c7977aa5094b968ca457e9a \
--hash=sha256:692fdb46d0fd9c71e380947dc5fc72f2c3f881073bc1f739e4579376231affbe \
--hash=sha256:6a754c0341bc9ca151b7b335988b3263921c08859f84ca219b70cee825571b6f \
--hash=sha256:7fa4b0f931cdbcb8a52044f370a3eb429fb42eb88807b0e800fcaa98dd42b24e \
--hash=sha256:81fce74bcd6be0e9da7c67c7553fd0f1f12964a2052cd728ef917b937a5ca1d9 \
--hash=sha256:a57049c8addb3662073859bb55a7ad35aa54f8a7c9ed268911d1333c5aadaaa9 \
--hash=sha256:ab5fa7ed5ce18554ea69d850ad162cbaa0c85fc47c08d83a569f806681c9a6ad \
--hash=sha256:ab86b84cd9d05e56d329742c0cabdcbe2d1d56c98f4a04aab9b5b66196febfb4 \
--hash=sha256:b75cc113b3aca388933d8296b667b1ac401a80317ea3074c9b0a66541199da29 \
--hash=sha256:bbde18beccd98c29a227c63c99022e3443b99258e08326df6425dfe0443705ef \
--hash=sha256:c673c170e27834d1262e3a9c1e97bdc9b9c3b180c63d89173ce69747232474c1 \
--hash=sha256:e407b4a6852853a9a2cdce96963e94ad9775578bb44e7561a0f2829f3cc04b11
# via evection (pyproject.toml)
six==1.17.0 \
--hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \
--hash=sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81
# via python-dateutil
sqlparse==0.6.0 \
--hash=sha256:113c35c75365ab9cc9c7231d68c6428fb11c085fc8e9eb1ad659b7ddbf6cd2b9 \
--hash=sha256:b861c0288ce2fa56209a9a6412d2e066ac664b3873b89c26c9d8415e8e32996f
# via django
typing-extensions==4.16.0 \
--hash=sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8 \
--hash=sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5
# via
# click-repl
# psycopg
tzdata==2026.6 \
--hash=sha256:0a10eec6e13ef16b627fabed9a5f2ba61a62b1c67f327305f4e7e412a70b4aec \
--hash=sha256:85d2cd9c09f81f0bb29db27759fc83af4f1e620efb814f8c5bf8b4cc4feece6d
# via
# kombu
# python-dateutil
tzlocal==5.4.5 \
--hash=sha256:ba976898bad27945178c9ca25c222c19e322b30c10342b9f40b46714f70d2d5f \
--hash=sha256:ecab534f6d65d9deb4c4ea52c121ab6dd5958e9b878e71dcb102578269752454
# via celery
urllib3==2.8.0 \
--hash=sha256:0cf3cae568d36aa9576b28dfb35f11328f1cb974ca7647d9475ebb86c75ac6e3 \
--hash=sha256:63bf2ead4c879426ebf22ef2a781eeb4aa3b4ae798a0435506f8687fd5bb9b63
# via
# requests
# responses
vine==5.1.0 \
--hash=sha256:40fdf3c48b2cfe1c38a49e9ae2da6fda88e4794c810050a728bd7413811fb1dc \
--hash=sha256:8b62e981d35c41049211cf62a0a1242d8c1ee9bd15bb196ce38aefd6799e61e0
# via
# amqp
# celery
# kombu
wcwidth==0.9.4 \
--hash=sha256:093469e01d881bff3db93ff3f56c364dc87c441ea01f9f41c22de0c0bdf5c6ef \
--hash=sha256:12786620375dc83fabf4b2b2aa52416fef2bc6094522ea29c8a53c75b0dfe9b6 \
--hash=sha256:14410eb73178cc492aa9bf22f7c12e3bf930e5470b7220cf6f530efbfa2e03f3 \
--hash=sha256:17619066f95c10f44d7ae08ee456602456bc66cc9d5e91966ec73d799acebc5f \
--hash=sha256:29c0685f5f5f8815b88308fdcbe2cd12a92996a21a5b1e18352d165174ef7a6d \
--hash=sha256:2b58fc8cf2de1bf9cdb68b8beaa5be198742414a7c8cd97aa51ec039783ffa49 \
--hash=sha256:2cca7b5d0b9d0ae49068d3ce806e965326a987f555b62eaa322223396e10d01b \
--hash=sha256:4831662c396dd4c59a016ec53c0ac96738006d653a00482777d9a2e2ea323d62 \
--hash=sha256:79e09e9824c88f5e275ca99e921685567a5c5c1bb6165c16e27ddd1adab33a95 \
--hash=sha256:87e4101789415a6cf55ec4c53eceeb4782c2654c88692941cb588f19dca1959f \
--hash=sha256:8f892713ab5795d3abae181e9c02db0eb24de4084f9ee080535369d3d98d7bb5 \
--hash=sha256:901eace6d2dd25ebb917d0ce5db18e5c537e1504c905cf32f0d006c12098b05d \
--hash=sha256:9c4b0dd04cca520792100f6110d10be262c94b285a281d1759c551f159214633 \
--hash=sha256:9e5090d253f5df99659cddfd3ed90dd21f6ff51989b0421331a63fb2681cbe27 \
--hash=sha256:a1526cd25c6420e5153bb7d764c7509e692215ee3ad9b725db1ad665d47eae09 \
--hash=sha256:a8c9ce14c6b4a499e61de6de7e9e456de11be8bc991554e2ab2ed4feb40b9664 \
--hash=sha256:ab1e5190fb82476e75fbf963e9d59b4fb1c1e1f4ced82069a0c989016aa959f7 \
--hash=sha256:b185c8b927b23af2721be1783499d1cfa5c028b098b0858951b4ae224477f1c2 \
--hash=sha256:b52f265a694dda6b3ad9050c8bd89158ad519109ef7c5eb110b906cc49f25763 \
--hash=sha256:b7969c9aa258d536df54c8ac8af5cad0c74a6268c9972e1fb60a5f29feffb66d \
--hash=sha256:befbf46467100e43fd8c8cc0c031a0105f13f5418c3be95fdcf0c47214487967 \
--hash=sha256:c36954849aafae1092d6f9de9548c5280c1d6841637a27ca288b2241c802c4ab \
--hash=sha256:d68f09093c9bec9226a52edc47487042ed4f64c5df91d06f00419495071c0279 \
--hash=sha256:d76b7c715c5ca0103b28915f699f1020ae21f06f1d22cdd85a7a97f7b1536eed \
--hash=sha256:d8905f1390d828d745fdd1904374c90386d902d3cfa9ee88e9875099f75a4e17 \
--hash=sha256:e05458491759aa42b7b85f36345b3e6df082d6d2d9bf616299668fd598e0209d \
--hash=sha256:f279f32c8e206a9092fa350e53ca0c2bd8072e45b1c9fd31bb6d476b59b591c4 \
--hash=sha256:f28d77153303450ee13ebbe888297e75cf532c754555b6cabafc9bdda16c421c \
--hash=sha256:fd8d57d3cdfe570e3265c8fab1885c24e34ef40889c0759c0c304a19e9835954
# via prompt-toolkit
wheel==0.48.0 \
--hash=sha256:3217dcc807155e45db462d7ef2431f5ddda0d7273b700d05a67b271ceb1287ab \
--hash=sha256:94800765601e9171bf5d58d066e640662842bcedcbab982b2c90787a2c987322
# via pip-tools
whitenoise==6.12.0 \
--hash=sha256:f723ebb76a112e98816ff80fcea0a6c9b8ecde835f8ddda25df7a30a3c2db6ad \
--hash=sha256:fc5e8c572e33ebf24795b47b6a7da8da3c00cff2349f5b04c02f28d0cc5a3cc2
# via evection (pyproject.toml)
# The following packages are considered to be unsafe in a requirements file:
pip==26.2.2 \
--hash=sha256:3a8e5c800492787f33a4114ef54444546c340df428974ebba17e0fd47e11af2f \
--hash=sha256:fd7f4dfa8190e014013b9f08f3e973f4e4c26ffd179a204895533a7254813a70
# via pip-tools
setuptools==84.2.1 \
--hash=sha256:3ff8b3b5492a0aa66960f4419365a86b84e013070d4313e4d4a6894f5d64d6bc \
--hash=sha256:ca140b45d5325a9478f880e351484f62fef15f336462e449ca0c03c873d589c2
# via pip-tools
+463
View File
@@ -0,0 +1,463 @@
#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
# pip-compile --allow-unsafe --generate-hashes --output-file=requirements.txt pyproject.toml
#
amqp==5.4.1 \
--hash=sha256:79a9c0ab70e71745667f127ff80666894a734c26236b6f33149c964b096f0b20 \
--hash=sha256:ac2b816a14a380ed10c5ebbf85a334fd68111fa476496867a5ccd2fd09926d5e
# via kombu
asgiref==3.13.0 \
--hash=sha256:034cc8cbf72d00553b4be256703ac545120b4e50af13b49dec99da899aeeecb4 \
--hash=sha256:f9596a2434512686473d50f96ff6207fbba49ae1a785dfac614df4aeb085674d
# via
# django
# django-htmx
billiard==4.3.1 \
--hash=sha256:2c7075283191d9c0add66cf8fca8e06ba599e75fe7319b67186759f8877dfdaf \
--hash=sha256:c88559b306ee5dc93f8d5f843d07da15d795d67af26720d14ee9d09f09eb0b22
# via celery
celery==5.6.3 \
--hash=sha256:0808f42f80909c4d5833202360ffafb2a4f83f4d8e23e1285d926610e9a7afa6 \
--hash=sha256:177006bd2054b882e9f01be59abd8529e88879ef50d7918a7050c5a9f4e12912
# via evection (pyproject.toml)
certifi==2026.10.23 \
--hash=sha256:a9895e6e2758b371e0545e321a3a8fd1fcfd9829ada8824a936dfb16aaaf36cd \
--hash=sha256:e9b54de76e530bb6cf0ccd036787103acb58fd42ba930ea8bf948ddbaa2f61d1
# via requests
charset-normalizer==3.5.2 \
--hash=sha256:01077390b03f7988f11d700a2194e69b119741a86b1a638b1db88891e3eced8e \
--hash=sha256:01b0c0d2262a9e28e8484a278c7e1b5d650e3ac8cf2683d2967e25899f208bdf \
--hash=sha256:04851f73ae72b8413dddadb16a49dfee95263553741fd42d546f7d66907e6be5 \
--hash=sha256:0521c5665880b33d603717defa76c094048900010897909952397feb3039da56 \
--hash=sha256:0774bf9bf620249fee3e0b8b9fd3065de213be30f3aa94ce2494b3b638949e26 \
--hash=sha256:0891b9d3903c5571c03771ca669a4b0ec5618ca722a5c957d3d29cd4e5062848 \
--hash=sha256:0c951d5e6dd9c2ff60609476752bee49da4206adde960ebc247766937f72e718 \
--hash=sha256:0fed1d06615f022ee3b13caf5e8b180cfea32bb2c5aded8a9d44277afc040f93 \
--hash=sha256:114e4d0c92d618409ed82a99e22b5c5e768fe995f2973f78265f4524f49d4640 \
--hash=sha256:11912e4bb14baae7c5d8791aa55ba0a3a03ec6729073307b0f57270abaa713d3 \
--hash=sha256:11a4d68a6ecda3292cb1e50239e111543ba5d709bb62a6b4ea1afcfa729d8875 \
--hash=sha256:124fbf1a8ff966d87ae05bb8bd45a71f966055ed8bba320d0c7cf450bc5f4d0e \
--hash=sha256:1461ac396c4fdb983a675f20aa555624f0ee18ac83d832b9244ffff3d8055275 \
--hash=sha256:1503bccbeb36d5527790c3930327704c39af22de3112f1b1666a9f3ce15ee204 \
--hash=sha256:15bb4005af6320d259dc7593ca84a38d7fe06a421dbcf7b910ae23979101e787 \
--hash=sha256:15c44f7edfd477b06f517a5cc317fc1707edb9de2c865f43d4b6513907473234 \
--hash=sha256:16fa0eccf81304b79c5cd87f9271c3b85dd9dd99245e4422ae9c0dd45e0f99d3 \
--hash=sha256:183b88127acdb4fabe59d951ab424faf1af7b63cdbb5f776186c1ea2ffcaed98 \
--hash=sha256:195c26fb65950f8fce54e26349852b7bdd7c5f120aeefbcc440b8a20faaed4a3 \
--hash=sha256:1afb975bd5d68d5ce9f6b6d44fdf2f7e34b895a35e95708a7a91b20a3b51d187 \
--hash=sha256:1b4cbc7c3491ccb4aa17fcd8165649d01cf39f76de1696da8631b5f71b85401d \
--hash=sha256:1bc0baf5ef96b6ede57d47f4b8fe4d9d84019c3bfcbeb20a41edc6a6ee341f1f \
--hash=sha256:1c50fe28bbc2ced33386f298650d91218076c05420e6cbd790b913adc41659e7 \
--hash=sha256:1db38f4c5496827c1a501846d64d14c3b80c7e6714e406cd7dc36a9899fa1011 \
--hash=sha256:211d5a3eb6af8f513b8d4ca19a8c1b7accab1b5f0d3175f9826b03c1a920dc1f \
--hash=sha256:23851fb4e1b85ed3f6c2a27b777cdfe2e19fb5b38429a8faf38c7542b7665869 \
--hash=sha256:254eb48b9fa5ee9898a3c445825a1f340fe53712a098904b39b0bddba8ea3cb1 \
--hash=sha256:2625388c6c754520c37abaf3b41eb34d1cc4a373f457898f08606c8e362b891d \
--hash=sha256:281cb91036248400f4cc957495cccd44c275c2e0c5854f7e45ac5cf7dc193847 \
--hash=sha256:28a15fdad492a99b6eccfaaed66ef3f74050680545ea61ec8b2f4c538f1f1320 \
--hash=sha256:28b4f0d66fb834ff90f28209ac7bce77868c45d8c93e26f906709d9b7c2e1af9 \
--hash=sha256:2a925889534b3748302dae5dead07cc13480de1dac3aea80a941b729b471ef93 \
--hash=sha256:2b7b3bbfb4fe8ef40600792d762fbaa9057559f9d3fad209525b7a22b99e91fd \
--hash=sha256:2c9ad19a6cfcd5ea5c0d41161d22f9df1dcc277e9bef2751391334546a314c00 \
--hash=sha256:2cc961b171b3f3440f410489ab3573e86aea8736134ebbb40ea1338b7f0831bc \
--hash=sha256:2ce45c6627b22c47e390bc91a41c3d13032192e699fa0bea96e9671b373d69b0 \
--hash=sha256:2e06a3a98f916dd41d27f3105e02e7a40181c98c94b9158733d03a6f80506c09 \
--hash=sha256:304d5463e65a35d7bb0850550e0780395395f6fcf452f04db7d5ca7cecc425ac \
--hash=sha256:304d8e4d493af723536393eee0c689eb7813f4a474c8b479dee63f1fdd98f621 \
--hash=sha256:30fcd120b732aa79317f08dee04d7de0847822e4cf7ee0e9f445bb958832252c \
--hash=sha256:31f3930700408d211f13378ccbe1c40845d8da54bd0681fac3a9b5aae81c7aa8 \
--hash=sha256:34276fd796040bf0993ab33a369aa572e6979c7aab225a88893667ad8eac8f7a \
--hash=sha256:355ad8011081dec5412240c087a9a0c9d4d5039f3ed11a3f13e18c2b29b56c51 \
--hash=sha256:38a873987f3be698494da8b2e3085e29da02da7b633dce73e79c699a113d7bf0 \
--hash=sha256:39de2a259fc954455c57274dc94c79d5842774e1247a016aff30bc0efed0f4ef \
--hash=sha256:3d14b50de6bf4d0edf857a9386836846f982b8f524e188e2e68b96d702bcf4aa \
--hash=sha256:3d21b8b13c7592db2ac5e544a6d83187b995257472b0c9e8351b6d507ae37ed6 \
--hash=sha256:3d31298449090ab8d47b7b1b2a555ff73cac7ed438a08b7ac160980c7ebed649 \
--hash=sha256:3ddacd27458c45bdacd6bd6db644bfb730efbf9e830310186e3045c9c5be8fb2 \
--hash=sha256:3df041de8887954562c9b261cba85ca0e9ded74048daf125f45edcfaa4832229 \
--hash=sha256:40ab6bffa02ae10a0581e6c198be7d2d8ca5c2a0c64e4ed3465d766df457573e \
--hash=sha256:4275811936e2f06feff5e598fb42a1b7ae852da8e39605211892b56b81a34efd \
--hash=sha256:443eae2bf318abeaf6f15d785138f71fd6de770e99a92158b8b814265e079115 \
--hash=sha256:447441e76ec720b15e64418d32e092297340387053047c7c694f579efb0ee1d9 \
--hash=sha256:4495c5002a7b28557e7e222e77e0b661183e432b7d6d2e788101e3f240e05b8c \
--hash=sha256:44bd4fbb29dfbeba60e7d2bd000c59e4b21ddb3cc53912b14048d37092706d7c \
--hash=sha256:4685902cf26edf013ed7a3da0f426ebba7a00ebb9541386d835afbf002c11cab \
--hash=sha256:498dc3188ca05a68231ac3fdbfc7f57eb67e1343c30e0fea17f8218c1599b253 \
--hash=sha256:4c2b5031f63e331e3839b40aed2dd6f191e9c07edbde303e7876846ea1946995 \
--hash=sha256:4d48f2d08b9de5864e2c8744d4461b862fb149a18274abc8b698c45975573438 \
--hash=sha256:4f87960d57feabfb618e4e0af6e7371645fa26a277860739d6e5d6e0012c92f0 \
--hash=sha256:50e3adfb96fc189eb27b1cf62d3b598b89b4bb0420d93a3d3e42e137409011be \
--hash=sha256:51cf45226a9b588d0d2b4880c62d686934b63ab0bd79ca23ab0e9762eb27441b \
--hash=sha256:52aa6992700996af31f375de0c6bacd402b0097fe40b53c426b9f51a90ebabc7 \
--hash=sha256:55ea99acb17b9325618de155a0cd6a2e8f5d10be008113e1d433bbb58db543b2 \
--hash=sha256:56bc200a365efb37383b7852e4cc5898d3b2da5987289b543956cf8cad71018a \
--hash=sha256:588461c2e8384d309bd63e5826019b6977bc66d629b99ac8737bb795d7b2cb5a \
--hash=sha256:58ca3755ee7ff7f59b57789ec9833c9de9ea275405cdd240eda1f193112e398a \
--hash=sha256:58f361dcbab699cf8f42db3f47c8e7fd1036f138c23a5d08de9fde5f425a730c \
--hash=sha256:598a11a2c7ebaa5334bf698bf29568c9c390abac6a154d8170fedecd1cea38c5 \
--hash=sha256:59f63901b0031c3136cf64704dcb21de0bbae62ce2c9529bc39d27665463de37 \
--hash=sha256:5cde776b7cc66e4f6c99612cea4aa7269aa65863f7a15841b2c264f103822f4e \
--hash=sha256:5e2b6b57e9733d39f0c9fd3185efa6b8e29652c4cd8fe94180272cf6ed9a78c4 \
--hash=sha256:5fb29fb8cd1a46c27a1bf9613ad5ec2599310d46b4025d9556404a6b6a292800 \
--hash=sha256:6045373d5a89a5ec71afde535db987ca28e76dfa276c2d4c818265b375d4b055 \
--hash=sha256:619799369eeef6366ed3e8755a5670f4f2f0fb6b30a0fd7264dc0fdc2357058e \
--hash=sha256:62588a277bfb59def052abd940703fa35107152bf479781a878617d60faf8fb5 \
--hash=sha256:62603db9a7caa0802eaa28c1c46fecd7b3a263a774069c24c3c28c302448721c \
--hash=sha256:65cd72beeeca9d3aaea1201e5923859f308f952f9c71de93f06063c79f0f7a3b \
--hash=sha256:68eb192d85ab8e5f6ec69c2bc6ac0179fbf04a5ac1569d12fbef74883fe102d0 \
--hash=sha256:6bd128f206a7752ae1f2ab6c61bf8a24ba28913a10df8b14c2637b973ff97a80 \
--hash=sha256:6be488a102b8cf28d0391d8c4ba7748938ae28b78ad901f8585520fca33ead1a \
--hash=sha256:7218e8f32b0956cfcd048fd42d9d5779809745ca1d86113ca56f66e7ae1549c4 \
--hash=sha256:7441d755b7ab94f8d4eb3e43ec05482d760842fd263d003a99102d742cd835e2 \
--hash=sha256:749e97e1b32313717a565abbe321bc2190bc8b35f1a67e4cdbc7c56c8d8ffe58 \
--hash=sha256:75a3ceed0724d625d64b86ca20aba182e4df462e04c2414fc941c0f523f06aac \
--hash=sha256:780fbe7cab297b81dad9fb8dc5eb003c0468ffb0d9e5f65068c53a34661a96bc \
--hash=sha256:78456a747de8dc58360ffa581f30a002baf5aa28cb262536545e91f113ed7639 \
--hash=sha256:7967d08cf06dee78443b874f98c98036f624f3a4e73e11f9f64f5be4d25393cf \
--hash=sha256:7a881931aa470808df94a8c380eed2bbbc76cd9dc622310f99665658c821eb6d \
--hash=sha256:7dcd882da75ef9adf94903b1e3b9419e8aa8fb4c7396822b834b9ef7fb96954f \
--hash=sha256:7e841fb9010836c992c9f12fcbd43a831de93a5f726fc1ccd8ca1d0268c5014c \
--hash=sha256:7fdde2c9fd9e3eca40631e024664cf2584272cc8f96308cbe5fdfc930f51d8bc \
--hash=sha256:8024d00c3faf3fc0c16e07a69f4405e8eac7cc0ab15f65fe6cf43827c4cf72b4 \
--hash=sha256:80d02b6f04e92601a081dd97b23d3128033098bff5d35d392ddcc0476ea11253 \
--hash=sha256:838dcc90063569a0448120554591a1d6c4a4ffe11babf048908793154ab86ade \
--hash=sha256:849df64e889b2e17230d58410a03dba311a65b163508fd33679b2b737d4b7858 \
--hash=sha256:87475fabc8d9996fd9c27debb395e642e8c838d78a00b6e932227a0e06b81e26 \
--hash=sha256:87e50a3e7cb90af586b6c5faf23e302a970415ac73bd7bd90a515a04b427ef96 \
--hash=sha256:89b53f3cda69831909888e0494f4fa0bcd3537e3e138dabeb620bd6ad946bae8 \
--hash=sha256:8a893cc101149f80a653f82062ebc95b34525a2614382e1da5458fe7c6997249 \
--hash=sha256:8b2bfab86aa71ae13aa41a6a26aab338e0db2b8bc75434b05aea89e011ff35a4 \
--hash=sha256:8d86d6fc60743dc916eb79e2eb1ec4818e21e427731543af40a3021851174a13 \
--hash=sha256:915563965d418f986e7e145accc592eae9e1a1be3566ff98a05d7a9ec42a76e1 \
--hash=sha256:92888bb3187c5ba50500b00b3b310c9f2c651709d28036077680cb5255450a03 \
--hash=sha256:93223adc95033dd47133a46ccfc316a0139176fd79085762e27202ec56018f03 \
--hash=sha256:9373ad13ef0d2c0fb761e04e55bfdee5a08b52cef2c882c8fbe9935b1517152e \
--hash=sha256:9409a8bf35cf78353942504b24a57de3d75b708997a1e4bd8db71ac8633ce364 \
--hash=sha256:9b7f416ff0978e2f2249330527f0ad6fa02f4932e6199692d3b52da2048c19e4 \
--hash=sha256:9bde855991b7e362c146535e3136a50bfaffc0487d38b33ca7e5edefc6e23849 \
--hash=sha256:9cae88599c7219005d879f98e5ed53341e9a122af585e1091200358a3003d2a0 \
--hash=sha256:9cf9b1a857e25c4baceeb3624e92a56df3668f398c4acba74e174d81fb4d1d3a \
--hash=sha256:9f56f72050826f63dcee7a7f55b0a77168cb3bfc553fd405e7f8f9ece75a4036 \
--hash=sha256:a090bb2c68df85450502e3e20d665e3a5af9c65a84d6508ed477badd49166fd3 \
--hash=sha256:a192e2c40070d92c3ccf777e3a5c4ff515573cd2bb7ed0c537fdadbbec5bbf21 \
--hash=sha256:a19a731138fc27d5682277d3b9df22855cea1239bce7fcec5f78f42ef2d1f3c3 \
--hash=sha256:a66c3bc5ab1f0ff2164fc9965ddd611ff0802173f4b9d24554c563f6ab7e1d6e \
--hash=sha256:a815775b6c38d4e0ff7bcffbeba67feded90202bb6a226b8dd35f1c855217413 \
--hash=sha256:a89012d6d5476ee112d20d998570ed58df2260a852afb1758809cd6900411d21 \
--hash=sha256:ae4f5fea5b8b8ccff88238cc8569303e5ee95efae67fa62922a311397a71f346 \
--hash=sha256:b6856554c4f44d79fc2307d5768854310a8f0096e501c75637542c82292b0429 \
--hash=sha256:b6b751274acb69d77b3323d6b7dbaa3c7fdfc1eb829b7eb61d262f32e1af9685 \
--hash=sha256:b736353c0a625bbd5fcec108576e2385db3496f4f771f785ff32e108d3c3bc45 \
--hash=sha256:b7fd005a73d9e657273b7a10dc71a9e03c8fb9ee6999798d6918ce095b81ac7f \
--hash=sha256:b91363207bd9dc966a691e959bb47f64b30f7ac4b072be9968b366982f7db77c \
--hash=sha256:ba0b1d2620edf869789c3879223f52bf2afc5d31b3cb47cc57b3a12c05e2aa9d \
--hash=sha256:bbbfc8e28816f19d7c0f1816664980c0a9875d01b27cdf8eedddb639d9e108ad \
--hash=sha256:bd16aabe4a02a297c23417aa17ac6299dbd8c49f673bcd645b4929b11f5a4400 \
--hash=sha256:c0afc6800ba57ccc350374c5bd6150419915d95ce93cdbab2d783d75eaf30ecb \
--hash=sha256:c6708715abcf3c73b99508253e961a9967f02fe536532834149574eda6de0d1c \
--hash=sha256:c7c9ab723cde841fefb34efbad91e87f00a674b1fe1cd0784fde742bf2c154dc \
--hash=sha256:c8f3d67aeaf55f017982b73683f0e7342ba2f6635a78f69ce89ebb26aa411e5c \
--hash=sha256:c9790464842f85f437dbbb54417eda1e0e6bfc52dd8d22d6fd1c994b73b2dc74 \
--hash=sha256:ca403d7e4798f525fdfc78e258820419cbbd0f0ecbab9de7840e3c017cf6b8cf \
--hash=sha256:d008d90a7f2471519aef0c90dfbe73b3e6e4d5e66ac48e19154c17e89e98b604 \
--hash=sha256:d19fbd981a488e22cd04883659ca6b08f50b5974f9fd7c95655ef6a043e5893f \
--hash=sha256:d1befeed746d247c81127bb14de9dc3d30edb6e5976d34f83f86ed262b1d9105 \
--hash=sha256:d2374b62878abb00cd8309b32af6c0b715cd02dec0ca74ef12e5069bdc64144a \
--hash=sha256:d376bbd28b3a8999db1a103b3b388aee6f1ddeb3e51bc2172993efdcd86e064d \
--hash=sha256:d4a7319f304a774bed22115bc891618e45f85065ab44ea6acd07d274e750519a \
--hash=sha256:d6734d2ef8a50fbf8445c139477da401f50d62a0606bf00e20ec6d87773fefb1 \
--hash=sha256:d760fe2a4d7c3b226cb9026d6a842868d52a7901bd98420e1baf14e80da85cf5 \
--hash=sha256:d913de495d90407cd859d263bee2e5d1a4ed3eb6573c04e70d9ec619a7cbed7f \
--hash=sha256:db19d07e2e0129e974a0e65d0064fc222a446cd5122c2fd4184d2af9fc734a9e \
--hash=sha256:dca9ab98072a5a54ebacebdc45f53e645336b320c667410b061be1ca588ae709 \
--hash=sha256:ddc7dacc8ece3a182e7f15cb862d1fd616b46d076cb1ae9dd232b2c38b655874 \
--hash=sha256:ddf19c062bea7a0cc80f519243d2c01dd091be0cf952a0750d4ad576709559f5 \
--hash=sha256:def79fa35ef0cef8d2accec024f4fdc7ead3012ff02f5215c783f39f03ef8cfc \
--hash=sha256:df29a0a7107f7011e77f4eebdddec4c7331e24d787a0b21a46d63bdf7445da95 \
--hash=sha256:e09a3942ecbdee5cce73ea9d42da82b81b72ac1bf031ce069b93b5adf4eac8cd \
--hash=sha256:e242bb1c5e76e97dfa9e7f209a71e93a01d7f19ffdd5cfbb2e2d55b4f08f8ab0 \
--hash=sha256:e243bd13217235fc7290c621941c3f5cc8b66e4872495be821d7436ba2fb838d \
--hash=sha256:e2af3aad578aa6bd1384bcf4750fc285e5a9de53f40b7d41e5a0bf748edeb2b3 \
--hash=sha256:e4e81e09c1578b8df602e3db08b0b3ea0a6947ad612f52bf8dc5ea8d47691f0c \
--hash=sha256:e54da4baf05720032d527874d40b65fa4d7e5c6c6a43d0c3adbeffcaf275a2b3 \
--hash=sha256:e80e6c2f55656b4824d72065abb4ddd6a525c74bd78a0aab5d9fc2cf4fb5af50 \
--hash=sha256:ed2a239c0ea213acc1908150a3037257083c7c083128f1a4cec2ec4b97dca491 \
--hash=sha256:ed905975ab14056a2e5eb1c376cb2e1ebc5396baf84163939c518556fccde9f5 \
--hash=sha256:ee21e28f0430bd6dc9086c6e525d5e818a44a5ad19720c8a0ef766792f3eb5e5 \
--hash=sha256:ee43c17b173d46a3212baa6ead3ae258eeabdae48c263a01ccf0218c366dd655 \
--hash=sha256:ef4fcbf3327382cd4c9f540babd61248208af7b93eec4de397b4d5f58a09e288 \
--hash=sha256:eff0ac9dbe711a4aee69bf04a83896aa9b85f19641264053a9f6d48573abb7dd \
--hash=sha256:f0aa869112ef88429ae17820d99c3dd9504c9e9c671d3c246f3d7442cb051084 \
--hash=sha256:f3c96f633825733f735c5a9cf21d21a257d8e1edf0b1cee0a064b9c424ca0f7d \
--hash=sha256:f5833ad231be5eb6553de524a70f48d71b2c8563101750531e0b80184e175cd4 \
--hash=sha256:f5ec61164adcec446f8969a3358ec3f9b26bbda3b9213e5586d219afa8df2915 \
--hash=sha256:f7d486c83842422badd511868fd8a9a20e9407ace71564b6af47ce7e60a336c1 \
--hash=sha256:fb9e68df06293761f9fe66ade60a9bc6d0f5e42b8acf2939a9158af86ab0e5bd \
--hash=sha256:fc14a032f813bf5fe624d991960ea83e9715adc27e4c1830a2361eb1d02ac341 \
--hash=sha256:fcff63213e8e6e47770541a4607175404f47cbb3ebea7b6058cc82d524a0e424 \
--hash=sha256:fd1fbe0f116b6e55da77aca2c6ddcddcfac2186cbf78bdebf40fc156efca389d \
--hash=sha256:fe9753dfee015c570d73df76f899f18444d41388bffcde097deba51c4fadbb9f
# via requests
click==8.5.0 \
--hash=sha256:255bc9599cf7748b4b1a446ccc735421bd08a2ae529a8b88597d3de5664ee360 \
--hash=sha256:ba0d2089de75ea0310e2dde03160e6ca10009947fb95a182f9b54021bb272e34
# via
# celery
# click-didyoumean
# click-plugins
# click-repl
click-didyoumean==0.3.1 \
--hash=sha256:4f82fdff0dbe64ef8ab2279bd6aa3f6a99c3b28c05aa09cbfc07c9d7fbb5a463 \
--hash=sha256:5c4bb6007cfea5f2fd6583a2fb6701a22a41eb98957e63d0fac41c10e7c3117c
# via celery
click-plugins==1.1.1.2 \
--hash=sha256:008d65743833ffc1f5417bf0e78e8d2c23aab04d9745ba817bd3e71b0feb6aa6 \
--hash=sha256:d7af3984a99d243c131aa1a828331e7630f4a88a9741fd05c927b204bcf92261
# via celery
click-repl==0.4.1 \
--hash=sha256:5cb10881d4c5ebaa8695eceb69911af3062ee78342812b713564b17aad333eb5 \
--hash=sha256:c32a1cf6f95e5bd6e92076f81ce24eafd33f2f0ffb0135887e335b8e446d1c0b
# via celery
defusedxml==0.7.1 \
--hash=sha256:1bb3032db185915b62d7c6209c5a8792be6a32ab2fedacc84e01b52c51aa3e69 \
--hash=sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61
# via evection (pyproject.toml)
django==5.2.18 \
--hash=sha256:461c5dd06d2ea16bd5ca37d3f46e4def1d6b0fe7588c6f4e2119517bb0af8b2d \
--hash=sha256:92ed81d500be6408ecd704d7bd1366c534f30427bffcc63c5fefb129561aec7c
# via
# django-csp
# django-htmx
# evection (pyproject.toml)
django-csp==4.0 \
--hash=sha256:b27010bb702eb20a3dad329178df2b61a2b82d338b70fbdc13c3a3bd28712833 \
--hash=sha256:d5a0a05463a6b75a4f1fc1828c58c89af8db9364d09fc6e12f122b4d7f3d00dc
# via evection (pyproject.toml)
django-environ==0.14.0 \
--hash=sha256:8dbe8a57f0a540ab8abd6f54f230de5e99e3a2c9d797cb9caecb037bca3d47d8 \
--hash=sha256:b6c48d93b9d2ff8a3ea14099e90c35aa4f101c1b4d5f262dfee0d27b06742ed7
# via evection (pyproject.toml)
django-htmx==1.29.0 \
--hash=sha256:0ec5be1645ed71e6787bd75e0250624f00274ac3bb1730f07b6629a95688929b \
--hash=sha256:337dfa35b8da13fd68bf968f2b9cc9a4144a4e71f06c204d7ff10f7343988102
# via evection (pyproject.toml)
feedparser==6.0.14 \
--hash=sha256:088679b0c4b543ee211a820dd544698c76a402122eae7473c04a43425f283d06 \
--hash=sha256:e35e3f760151b0c3b22cac9684155cae186a233e16c49bcbc6c49e91e3131137
# via evection (pyproject.toml)
feedparser-sgmllib==2.1.0 \
--hash=sha256:2cab2d43b95a954f920f18aebce7a4dbbb3f539780b127e2aa114f579821e01d \
--hash=sha256:61facf2918c4389b5b00714f76c5e03431ffcd94cd1f51d657edd6cd7c396579
# via feedparser
gunicorn==23.0.0 \
--hash=sha256:ec400d38950de4dfd418cff8328b2c8faed0edb0d517d3394e457c317908ca4d \
--hash=sha256:f014447a0101dc57e294f6c18ca6b40227a4c90e9bdb586042628030cba004ec
# via evection (pyproject.toml)
idna==3.20 \
--hash=sha256:a7db850025b95ded1eae8a46181a1a6c56c92c96f0e2b005d9ff8dc0210cab44 \
--hash=sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c
# via requests
kombu==5.6.2 \
--hash=sha256:8060497058066c6f5aed7c26d7cd0d3b574990b09de842a8c5aaed0b92cc5a55 \
--hash=sha256:efcfc559da324d41d61ca311b0c64965ea35b4c55cc04ee36e55386145dace93
# via celery
nh3==0.3.7 \
--hash=sha256:157ec1eb7a62f3d9a7badb8d82d89aa810e3e24e097eedfa481a25d0c8a99877 \
--hash=sha256:15f5fbf090f5c88d61c820e1fc1fceecb6520cca9fe85649c06b57ef9dc9ff62 \
--hash=sha256:18f4278ecd157d43cb35acd5aae9f35cfa79f546b4922bd86536adc0f6312102 \
--hash=sha256:19f288c938ec6eef1f5d2c6cab47838e71fef8097e1c1233802be5a6230ba086 \
--hash=sha256:4968fe8d2db97c6f047659bf46a449fd8ec377f44ebf3e0a1b96c0d3a333ae32 \
--hash=sha256:5ffdfcb9a686ffb12765376bcfb6b5b55728516d3c0ee317d29982381ded3df8 \
--hash=sha256:614dac4a4c36ad084e78447d16fe898dedd762e354a7ab9cda2984e82f67883d \
--hash=sha256:618e3059caf41ccdf5dcccb3fa9df4cf6e4efe23d1382a8bbfca272a8a4f8bfc \
--hash=sha256:6698a822132beedab80f131c08d8d0ac5a178ddeb488d02ca4b67716ecfac7af \
--hash=sha256:6c3aa50eb26e9228238271db9f983cbc3b006dfbfeca2d4dc34c33ddc6ac5ea5 \
--hash=sha256:6e4280115d44c3b278eef712a86748c1a723105cd79feec46952383117ab4e59 \
--hash=sha256:70f5ac8626e899a4bab0ef74ca2f5bd602f49c7b739e6e5026b4afc6d63dac42 \
--hash=sha256:71860d01c16f4d8c72e334e0674beb2b0899dbd0bf760de18932ef4390303848 \
--hash=sha256:808def0c8c07843e6e50dc84f532457bfa2cfd17417b219a5d9e7c773709331a \
--hash=sha256:874b7d67a067bd29a59223f6270fc30da4edd8e6d87fd219fc93bcbaa662c946 \
--hash=sha256:91a4dab4e94d9fc54b9f67b1adfb23e81fab7ab43f33c3b8c97be9aa38f789ba \
--hash=sha256:94fd6e59553fbb9ffd8ba71bbd5a54e3126ba01799a097ae30d5341d750bc6ac \
--hash=sha256:9b7279d43323a25225df23576af6594a16693f61431170848b8b2ac21ad4f174 \
--hash=sha256:bc42bb1193c1e28a1e74c2cabaca178e118a7103e8832699fef8a2b3e2496493 \
--hash=sha256:be53a4825585f701955cb9baf49f478f56eb81e20294329fe4bc689dd5dd81fa \
--hash=sha256:d56e76bd3cadb09b6b0cef364850811663734b348a25f5f587a2819c495367bd \
--hash=sha256:de2b2aab32ea303405debefdcfc58043d3e635fa3f67b9eb140d2b0e0c0d2563 \
--hash=sha256:e8fd1ab205258b29254f72db377d99e2c96aa7653ef3b015ccab0420b094b506 \
--hash=sha256:eae64328e46a25785535afcb6885b6f182ecaf5ee8c88f8c075422db8aacc65b \
--hash=sha256:f04b7d333b27f13ca439da3cf1c75c2fba34f104969f6ce4ac8e7079699c2f4a \
--hash=sha256:f266d3f1b3647449923a8e406524632220dd5d8b647078dfe45b885d33d10479 \
--hash=sha256:fd4a70efb45d5372174f718878eb7a35c12677626a63b2f103b23b833457dcac
# via evection (pyproject.toml)
packaging==26.3 \
--hash=sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79 \
--hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c
# via
# django-csp
# gunicorn
# kombu
prompt-toolkit==3.0.53 \
--hash=sha256:01c0891d7f9237d5e339f7d3e42cdae80b7534abb1c7c0e3352efba6231492f2 \
--hash=sha256:9ec8a0ad96d5c56148b3f914aa79c1564c3fde5d2e6b876e7bc327e353cf8fa6
# via click-repl
psycopg[binary]==3.3.6 \
--hash=sha256:a1db9f7148b06a28606767efaca51fa6f9398c5c0a3810519be69d7000bdb631 \
--hash=sha256:c081f2250df751a943036e42db6df4571c66cd0aabe8291a7a506512b12007d2
# via evection (pyproject.toml)
psycopg-binary==3.3.6 \
--hash=sha256:05a83ac9fd52b9bca7cb5ab04b3691163170bd16f53defa27216ea3aa07ee781 \
--hash=sha256:0a52991594ac4db888c7d39bccef331797e30cb31a95cae02cf2607f83a42dc2 \
--hash=sha256:0bf08b749cc144f33b44a91b78e3f71c60eb07963746a0df5a100b36ce3d7475 \
--hash=sha256:0ebfad5d131de9f892ae9e70cc7616207768b6714b66a52d4612b8ceaf78b372 \
--hash=sha256:1679a1cb93fbe5a6d1fd58d82cbddcc6fcb8c61446ba7cae6eb2a7b19bc585de \
--hash=sha256:198a48e68cc99ccac03ba95ac857e73aa66f3bf6be77019fafb0832a05f7ad03 \
--hash=sha256:1fbd30e537dab22cafdf080608f10148fe2a5f3a61294ddb5113caac8a623840 \
--hash=sha256:289aadd6a00e151203c081f708348ec89f1e483c9b510ef4ac3981f847f01f79 \
--hash=sha256:2f122603f36050937982abf9668d8bc4769a79f7c93a65013b1c49f1cab7b56b \
--hash=sha256:303732e798fe6729f8e12021b9c96107df8e95ecec4dd487c67b98ec2a59435e \
--hash=sha256:31cd942c23f613276b81a6e6598cefa12960058b0f46e1e874b540c793f6aca5 \
--hash=sha256:366db6e97e66b37211475f20c4c1324a2dc0dd825e46d4e87f9d599304d276f9 \
--hash=sha256:373704aea331d3f3e3402c125a1543f5875e2986ebb54f97d1647942161f803f \
--hash=sha256:37d40450659401600e6d043ff586c89a71a69f33cbb8bcdba6cdb2569beecdbe \
--hash=sha256:37e517c146b185f9c0c6e8d0a0ebbdeeeb67896af28466e032bc810d0c7dc7a7 \
--hash=sha256:3af90f92769d8cc10f94515ee7a0aef36ea85ca733a0ce22858f6e0953f41138 \
--hash=sha256:3c9e663b2e800e3218994cf948c11bcc2844e6491b34aa80d089baf6531827bf \
--hash=sha256:3f84dab25e0385692ee13274c68678377e0b1a70ab9d14e56264cbf61f60c62d \
--hash=sha256:4690cf67738f0e0e49a32aeec99bf0e4595cc2b4f1af984a4345394b1dcff91a \
--hash=sha256:566dd827f17728efdf7d88a5b066f815170f6fdad13967ae952842d90e6aaa9f \
--hash=sha256:5927b7ba63153cd8e9862987290a2b783a5c590daf2a4ef981700cc3569166d4 \
--hash=sha256:5ad8f35e67cc16d1fad1fa8c88972dc9b3a3141ea67897399904edab96a301b6 \
--hash=sha256:5ea8beeb5541780b4b50b462eeacbc4f594ce3b911dc20c81c75f267876f71d2 \
--hash=sha256:5f598f19fa9a91540b5cee17932ffd227b7b53a481605bcc4573c0eafa647300 \
--hash=sha256:612382ac3ed13651c7fa44b5fee9fbf7baaa2ddbc6f500391672682c5f1df9e0 \
--hash=sha256:6ff05561e4a067d35507dc5c90f1deb2ec1c9703ac5cccc1bc26e08a197f9c5a \
--hash=sha256:7308c93cf0b19bbaf8e6ff0a6ad50d3c442385739245fe15a8d593bf841734a6 \
--hash=sha256:79a2a1c3449f6c3409427078ed1cec10de79f3023cb5f2504f0597d350ad46c7 \
--hash=sha256:7beb3e41c9a1e509f3ed85263386588cbe3e975aa67be21f79f44fd35ffaeefc \
--hash=sha256:86147cb5d140341c3363fb5bacce31f8d5543902a46699d3c536b101bbceaf9e \
--hash=sha256:889e42acec10450185e0cdfb396f375e2c1a8d7737c114830a7fde4654f59e30 \
--hash=sha256:910ace140e3e7b7596898d083f37a8fe90c5c40684252ad4e682364b2cd3deba \
--hash=sha256:955e3dd94da361e052d2e49acf591017158dc8f8ed2c8a42c2e3943403c39dc2 \
--hash=sha256:9892188bb15e5803beb51afe8a25add6b56be391a53058e8bca03b74e1e6bf22 \
--hash=sha256:98c02090d88f2ebc0ec1e8da538f77d225ce0fffecf372aa39262e62a1b054ef \
--hash=sha256:9b2f11794e017ce340934e35de46181c46ef71ec75ea3d85dd75cd836761c01e \
--hash=sha256:a2e44a342d2aee40508e28a563d8961c39d9bbd8cae36d8578f0a3c6658aab0f \
--hash=sha256:a4ee3bdd5468a725f2a4d9aab8a74b6d0279f768c8b5d3aeb102c5307ff3d59c \
--hash=sha256:a5165300324efd5a772c48a88ab3a928513ab3979fca76553e62ee815f7b2b9c \
--hash=sha256:a9348c5b43a3bb5ef8c2e89d5237c9c87eeafb01d338c84a7aebbc5cd0313299 \
--hash=sha256:aa73160077345ec21b3f51e8e24b3de2e99586217e497629326eb9b2ea88c52e \
--hash=sha256:ad1c785e784cfd87e8436c6b7702f2d321fc39601bbaf29bc63a41a867091638 \
--hash=sha256:b3f75dee0f9afafabe4edc52c4842f1e1878ed2069bd05b22d6fe961e97e4dba \
--hash=sha256:b599defe9190b17e9907c8b4d114c181e702c87efcd1b8a0ad40971cdcc4634a \
--hash=sha256:b82491019b884d62318b5f30706c3d7e6d4e5a6cb7eabcb3edc0c1b0fdaceae9 \
--hash=sha256:b8ece331509f7a975b90501f41e83ad905e4141753fedf3f2711b2bc70a8efbc \
--hash=sha256:b979a42815410432420275412633960807178b1ce26591a16ce06e78a5bd4bb2 \
--hash=sha256:be4f9b3c9338ac5dd217c5847e21521b396c8117f78dc420d495a5c49bbef874 \
--hash=sha256:bf8c8481d026b85dd70c5fa7dde85b2333aed0b32a2602bcd38a900cbd78a49c \
--hash=sha256:c61617eaae0112ca154da87ffb99b73af2c74067acac28dfb9a4455b019dff2e \
--hash=sha256:c6d19cb4999d03231e8730a5f66c8f5068bc3b532677eb39dab0f600bff3e312 \
--hash=sha256:c7753871eb57e6a5f4646f6168590c6653073dea5e9e720b201c8875332df4c8 \
--hash=sha256:c7f92daa0d2a1c76f07264abddf8cbabd30152a2f09c3270e50f0c7efdf5dcac \
--hash=sha256:cbd5f73073ed19c378d4c35499db1e3e703a5b1a324e521204065967bfaa7a18 \
--hash=sha256:cec5ea900390897d0b46130f60bc2883bf19c314f9044235217c8be88b0ef269 \
--hash=sha256:d636338c8f21b0df2f84657b00bc34f9313f826ef93f1155bc743607e4a0c5eb \
--hash=sha256:dc75da5a20951049f7b773145f998f69d181adad9c58a0ff36e0cf1d73c10e10 \
--hash=sha256:e23a66a763fbe83fcc210bc77c27e5a5ea380ebf091c06f34d8561b695e5a40f \
--hash=sha256:e8cbb54454dbf1bbf2ff08dd7693e8d94ac94b1a20f70f4b3b813d52ecb5cbc1 \
--hash=sha256:ee2c4728c691245e24501fcd7a97b5b381236b9985bc445bba88cdce7d1b5784 \
--hash=sha256:f0535693ce476a722b718b002d5d2c27d47e71ca945276ac194409c98e74c492 \
--hash=sha256:f19cc87343eaa55255e76b31259a570072ac95d6ae82c92dd34b97691f5e49dc \
--hash=sha256:f21d057f3e5f5491067e5b292498073b73847d48799b099803fef100775fcc52 \
--hash=sha256:f87dbdc42e78ee0f7ea180c03f8c78e80a949e373066629bd90fefff10552dff \
--hash=sha256:fa34eb47969297471db7b7f193622c7e3ee839ec05abd05f1fe104d5b1b1dcf4 \
--hash=sha256:fdccb3a0e184b03e9baa673b15a809cf36c339c85dbda0ebc25a698846dfbee8
# via psycopg
pyjwt==2.15.1 \
--hash=sha256:42d59d631f7768a1028a64c7ff581a9bf7519804daf91fc5b6c56e30eec5e193 \
--hash=sha256:4f259e80cdfb6b3fc18a7de51fd1ef9ec79652f25019bae68975ca2468a34df8
# via redis
python-dateutil==3.0.0 \
--hash=sha256:3c79630202e4152c6add678446984236efe4f57e84d8e892aab0637bab82e9fd \
--hash=sha256:7ebcb3c29a92f3ceafe79a7a48a5c45677e9e004bbeec5742ac0bbb00560b7a2
# via celery
redis==5.3.1 \
--hash=sha256:ca49577a531ea64039b5a36db3d6cd1a0c7a60c34124d46924a45b956e8cf14c \
--hash=sha256:dc1909bd24669cc31b5f67a039700b16ec30571096c5f1f0d9d2324bff31af97
# via evection (pyproject.toml)
requests==2.34.2 \
--hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \
--hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed
# via evection (pyproject.toml)
six==1.17.0 \
--hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \
--hash=sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81
# via python-dateutil
sqlparse==0.6.0 \
--hash=sha256:113c35c75365ab9cc9c7231d68c6428fb11c085fc8e9eb1ad659b7ddbf6cd2b9 \
--hash=sha256:b861c0288ce2fa56209a9a6412d2e066ac664b3873b89c26c9d8415e8e32996f
# via django
typing-extensions==4.16.0 \
--hash=sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8 \
--hash=sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5
# via
# click-repl
# psycopg
tzdata==2026.6 \
--hash=sha256:0a10eec6e13ef16b627fabed9a5f2ba61a62b1c67f327305f4e7e412a70b4aec \
--hash=sha256:85d2cd9c09f81f0bb29db27759fc83af4f1e620efb814f8c5bf8b4cc4feece6d
# via
# kombu
# python-dateutil
tzlocal==5.4.5 \
--hash=sha256:ba976898bad27945178c9ca25c222c19e322b30c10342b9f40b46714f70d2d5f \
--hash=sha256:ecab534f6d65d9deb4c4ea52c121ab6dd5958e9b878e71dcb102578269752454
# via celery
urllib3==2.8.0 \
--hash=sha256:0cf3cae568d36aa9576b28dfb35f11328f1cb974ca7647d9475ebb86c75ac6e3 \
--hash=sha256:63bf2ead4c879426ebf22ef2a781eeb4aa3b4ae798a0435506f8687fd5bb9b63
# via requests
vine==5.1.0 \
--hash=sha256:40fdf3c48b2cfe1c38a49e9ae2da6fda88e4794c810050a728bd7413811fb1dc \
--hash=sha256:8b62e981d35c41049211cf62a0a1242d8c1ee9bd15bb196ce38aefd6799e61e0
# via
# amqp
# celery
# kombu
wcwidth==0.9.4 \
--hash=sha256:093469e01d881bff3db93ff3f56c364dc87c441ea01f9f41c22de0c0bdf5c6ef \
--hash=sha256:12786620375dc83fabf4b2b2aa52416fef2bc6094522ea29c8a53c75b0dfe9b6 \
--hash=sha256:14410eb73178cc492aa9bf22f7c12e3bf930e5470b7220cf6f530efbfa2e03f3 \
--hash=sha256:17619066f95c10f44d7ae08ee456602456bc66cc9d5e91966ec73d799acebc5f \
--hash=sha256:29c0685f5f5f8815b88308fdcbe2cd12a92996a21a5b1e18352d165174ef7a6d \
--hash=sha256:2b58fc8cf2de1bf9cdb68b8beaa5be198742414a7c8cd97aa51ec039783ffa49 \
--hash=sha256:2cca7b5d0b9d0ae49068d3ce806e965326a987f555b62eaa322223396e10d01b \
--hash=sha256:4831662c396dd4c59a016ec53c0ac96738006d653a00482777d9a2e2ea323d62 \
--hash=sha256:79e09e9824c88f5e275ca99e921685567a5c5c1bb6165c16e27ddd1adab33a95 \
--hash=sha256:87e4101789415a6cf55ec4c53eceeb4782c2654c88692941cb588f19dca1959f \
--hash=sha256:8f892713ab5795d3abae181e9c02db0eb24de4084f9ee080535369d3d98d7bb5 \
--hash=sha256:901eace6d2dd25ebb917d0ce5db18e5c537e1504c905cf32f0d006c12098b05d \
--hash=sha256:9c4b0dd04cca520792100f6110d10be262c94b285a281d1759c551f159214633 \
--hash=sha256:9e5090d253f5df99659cddfd3ed90dd21f6ff51989b0421331a63fb2681cbe27 \
--hash=sha256:a1526cd25c6420e5153bb7d764c7509e692215ee3ad9b725db1ad665d47eae09 \
--hash=sha256:a8c9ce14c6b4a499e61de6de7e9e456de11be8bc991554e2ab2ed4feb40b9664 \
--hash=sha256:ab1e5190fb82476e75fbf963e9d59b4fb1c1e1f4ced82069a0c989016aa959f7 \
--hash=sha256:b185c8b927b23af2721be1783499d1cfa5c028b098b0858951b4ae224477f1c2 \
--hash=sha256:b52f265a694dda6b3ad9050c8bd89158ad519109ef7c5eb110b906cc49f25763 \
--hash=sha256:b7969c9aa258d536df54c8ac8af5cad0c74a6268c9972e1fb60a5f29feffb66d \
--hash=sha256:befbf46467100e43fd8c8cc0c031a0105f13f5418c3be95fdcf0c47214487967 \
--hash=sha256:c36954849aafae1092d6f9de9548c5280c1d6841637a27ca288b2241c802c4ab \
--hash=sha256:d68f09093c9bec9226a52edc47487042ed4f64c5df91d06f00419495071c0279 \
--hash=sha256:d76b7c715c5ca0103b28915f699f1020ae21f06f1d22cdd85a7a97f7b1536eed \
--hash=sha256:d8905f1390d828d745fdd1904374c90386d902d3cfa9ee88e9875099f75a4e17 \
--hash=sha256:e05458491759aa42b7b85f36345b3e6df082d6d2d9bf616299668fd598e0209d \
--hash=sha256:f279f32c8e206a9092fa350e53ca0c2bd8072e45b1c9fd31bb6d476b59b591c4 \
--hash=sha256:f28d77153303450ee13ebbe888297e75cf532c754555b6cabafc9bdda16c421c \
--hash=sha256:fd8d57d3cdfe570e3265c8fab1885c24e34ef40889c0759c0c304a19e9835954
# via prompt-toolkit
whitenoise==6.12.0 \
--hash=sha256:f723ebb76a112e98816ff80fcea0a6c9b8ecde835f8ddda25df7a30a3c2db6ad \
--hash=sha256:fc5e8c572e33ebf24795b47b6a7da8da3c00cff2349f5b04c02f28d0cc5a3cc2
# via evection (pyproject.toml)
+123 -54
View File
@@ -1,6 +1,11 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Automates docs/install.md's manual steps: system user, virtualenv, .env, # Automates docs/install.md's manual steps: system user, virtualenv, .env,
# migrate/collectstatic, systemd units, and (optionally) nginx + TLS. # migrate/collectstatic, systemd units. Does NOT set up a reverse proxy —
# Evection always requires one in front of gunicorn (see docs/install.md),
# but which one (local nginx/Caddy/Traefik, or something like Nginx Proxy
# Manager on a different host entirely) is up to you, so this script just
# leaves gunicorn listening on EVECTION_GUNICORN_BIND for you to point it
# at.
# #
# Run as root, from inside the checkout you want to serve: # Run as root, from inside the checkout you want to serve:
# sudo ./scripts/install.sh # sudo ./scripts/install.sh
@@ -11,7 +16,7 @@
# Non-interactive use (e.g. scripted provisioning) — set these before # Non-interactive use (e.g. scripted provisioning) — set these before
# running, and the script won't prompt: # running, and the script won't prompt:
# EVECTION_ALLOWED_HOSTS, EVECTION_DATABASE_URL, EVECTION_REDIS_URL, # EVECTION_ALLOWED_HOSTS, EVECTION_DATABASE_URL, EVECTION_REDIS_URL,
# EVECTION_USER (defaults to "evection") # EVECTION_GUNICORN_BIND, EVECTION_USER (defaults to "evection")
set -euo pipefail set -euo pipefail
@@ -26,6 +31,13 @@ VENV_DIR="$EVECTION_HOME/venv"
EVECTION_USER="${EVECTION_USER:-evection}" EVECTION_USER="${EVECTION_USER:-evection}"
PYTHON_BIN="${PYTHON_BIN:-python3}" PYTHON_BIN="${PYTHON_BIN:-python3}"
# $APP_DIR ends up owned by $EVECTION_USER (see below), but the git steps
# in this script run as root — without this, git refuses to touch a repo
# it doesn't own ("detected dubious ownership"), which is exactly what
# happens re-running this script after a previous run got far enough to
# chown the checkout.
git config --global --add safe.directory "$APP_DIR"
echo "==> Installing Evection" echo "==> Installing Evection"
echo " App directory: $APP_DIR" echo " App directory: $APP_DIR"
echo " Service user: $EVECTION_USER" echo " Service user: $EVECTION_USER"
@@ -45,13 +57,52 @@ if [ "$PY_MAJOR" -lt 3 ] || { [ "$PY_MAJOR" -eq 3 ] && [ "$PY_MINOR" -lt 11 ]; }
echo "Python $PY_VERSION found, but Evection needs 3.11 or newer." >&2 echo "Python $PY_VERSION found, but Evection needs 3.11 or newer." >&2
exit 1 exit 1
fi fi
if ! "$PYTHON_BIN" -c 'import ensurepip' >/dev/null 2>&1; then
echo "'$PYTHON_BIN' can't create virtualenvs (no ensurepip) — on" \
"Debian/Ubuntu, install it with: apt install python${PY_VERSION}-venv" >&2
exit 1
fi
for bin in redis-cli nginx; do # Celery (and therefore all feed fetching) is entirely dependent on Redis
if ! command -v "$bin" >/dev/null 2>&1; then # being reachable — it fails silently from the installer's point of view
echo "Warning: '$bin' not found on PATH. Install it before going live" \ # (the worker/beat processes just retry forever in the background), so this
"(see docs/install.md's prerequisites section: apt install redis-server nginx)." >&2 # has to be a hard failure here, not a warning that's easy to miss while
# every later step "succeeds" at installing an app that can never fetch a
# single feed. Check the actual REDIS_URL that will end up in .env (an
# explicit EVECTION_REDIS_URL override, if set — the default local Redis
# otherwise), not just whether some redis-cli happens to be on PATH.
REDIS_URL_TO_CHECK="${EVECTION_REDIS_URL:-redis://localhost:6379/0}"
check_redis() {
"$PYTHON_BIN" -c "
import sys, socket, urllib.parse
u = urllib.parse.urlparse('$REDIS_URL_TO_CHECK')
try:
with socket.create_connection((u.hostname or 'localhost', u.port or 6379), timeout=3):
pass
except OSError as e:
print(e, file=sys.stderr)
sys.exit(1)
"
}
REDIS_HOST="$("$PYTHON_BIN" -c "import urllib.parse; print(urllib.parse.urlparse('$REDIS_URL_TO_CHECK').hostname or 'localhost')")"
if ! check_redis; then
if [[ "$REDIS_HOST" =~ ^(localhost|127\.0\.0\.1|::1)$ ]] && command -v apt-get >/dev/null 2>&1; then
echo "==> Redis isn't reachable at $REDIS_URL_TO_CHECK — installing redis-server"
apt-get install -y redis-server
systemctl enable --now redis-server
if ! check_redis; then
echo "Redis still isn't reachable after installing redis-server —" \
"check: systemctl status redis-server" >&2
exit 1
fi fi
done else
echo "Redis isn't reachable at $REDIS_URL_TO_CHECK. Evection's background" \
"feed-fetching depends on it entirely, so this has to be fixed before" \
"continuing — either start/install Redis there, or set" \
"EVECTION_REDIS_URL to point at wherever it actually runs." >&2
exit 1
fi
fi
# --- 2. System user ----------------------------------------------------- # --- 2. System user -----------------------------------------------------
@@ -71,7 +122,22 @@ fi
# fast instead of hanging the install on a silent username/password # fast instead of hanging the install on a silent username/password
# prompt. # prompt.
if [ -d "$APP_DIR/.git" ]; then if [ -d "$APP_DIR/.git" ]; then
GIT_TERMINAL_PROMPT=0 git -C "$APP_DIR" fetch --tags --force || true # Deliberately NOT --force (S7, matches scripts/upgrade.sh): a plain
# `git fetch --tags` refuses to move a tag ref this checkout already
# has onto a different commit, so a rewritten/tampered release tag
# gets reported here instead of silently followed. A fresh checkout
# with no tags at all, or an unreachable remote, both legitimately
# fail this fetch too — this is a first install, not an upgrade, so
# either case still falls through to "stay on whatever's checked out"
# below rather than blocking the install.
FETCH_OUTPUT="$(GIT_TERMINAL_PROMPT=0 git -C "$APP_DIR" fetch --tags 2>&1)" || {
echo "$FETCH_OUTPUT" >&2
if echo "$FETCH_OUTPUT" | grep -q "would clobber existing tag"; then
echo "==> A release tag points at a different commit than expected —" \
"leaving it alone rather than force-following it. Investigate" \
"with 'git fetch --tags --dry-run -v' before deploying." >&2
fi
}
LATEST_TAG="$(git -C "$APP_DIR" tag --sort=-creatordate | head -n1)" LATEST_TAG="$(git -C "$APP_DIR" tag --sort=-creatordate | head -n1)"
if [ -n "$LATEST_TAG" ]; then if [ -n "$LATEST_TAG" ]; then
echo "==> Checking out latest release: $LATEST_TAG" echo "==> Checking out latest release: $LATEST_TAG"
@@ -92,13 +158,27 @@ chown -R "$EVECTION_USER:$EVECTION_USER" "$APP_DIR"
# --- 3. Virtualenv + install --------------------------------------------- # --- 3. Virtualenv + install ---------------------------------------------
if [ -d "$VENV_DIR" ] && [ ! -x "$VENV_DIR/bin/pip" ]; then
echo "==> $VENV_DIR exists but looks incomplete (no bin/pip) — recreating it"
rm -rf "$VENV_DIR"
fi
if [ ! -d "$VENV_DIR" ]; then if [ ! -d "$VENV_DIR" ]; then
echo "==> Creating virtualenv at $VENV_DIR" echo "==> Creating virtualenv at $VENV_DIR"
sudo -u "$EVECTION_USER" -H "$PYTHON_BIN" -m venv "$VENV_DIR" sudo -u "$EVECTION_USER" -H "$PYTHON_BIN" -m venv "$VENV_DIR"
fi fi
echo "==> Installing Evection and its dependencies" echo "==> Installing Evection and its dependencies"
sudo -u "$EVECTION_USER" -H "$VENV_DIR/bin/pip" install --upgrade pip sudo -u "$EVECTION_USER" -H "$VENV_DIR/bin/pip" install --upgrade pip
sudo -u "$EVECTION_USER" -H "$VENV_DIR/bin/pip" install "$APP_DIR" # requirements.txt (generated by `pip-compile --generate-hashes`, committed
# alongside pyproject.toml) pins every dependency to an exact, hash-
# verified version (S7) — two installs of the same release get identical
# dependency code, and a tampered or substituted package on the index
# fails the hash check instead of installing silently. --require-hashes
# also means every entry in the file must carry a hash, so an
# unreviewed/unpinned addition fails closed rather than installing
# unverified. The app itself is installed separately with --no-deps: its
# own version is pinned by which git tag is checked out, not by this file.
sudo -u "$EVECTION_USER" -H "$VENV_DIR/bin/pip" install --require-hashes -r "$APP_DIR/requirements.txt"
sudo -u "$EVECTION_USER" -H "$VENV_DIR/bin/pip" install --no-deps "$APP_DIR"
# --- 4. .env -------------------------------------------------------------- # --- 4. .env --------------------------------------------------------------
@@ -122,15 +202,34 @@ if [ ! -f "$ENV_FILE" ]; then
fi fi
DATABASE_URL="${DATABASE_URL:-sqlite:///$APP_DIR/db.sqlite3}" DATABASE_URL="${DATABASE_URL:-sqlite:///$APP_DIR/db.sqlite3}"
GUNICORN_BIND="${EVECTION_GUNICORN_BIND:-}"
if [ -z "$GUNICORN_BIND" ] && [ -t 0 ]; then
echo
echo "Evection must run behind a reverse proxy — gunicorn needs an" \
"address:port that proxy can reach."
read -rp "Is the reverse proxy on this same machine, or a different one? [same/different] [same]: " PROXY_LOCATION
if [[ "${PROXY_LOCATION:-}" =~ ^[Dd] ]]; then
read -rp "Address:port for gunicorn to listen on (e.g. 0.0.0.0:8000, or this host's own IP:8000) [0.0.0.0:8000]: " GUNICORN_BIND
GUNICORN_BIND="${GUNICORN_BIND:-0.0.0.0:8000}"
echo " Remember to firewall this port to only accept connections" \
"from the reverse proxy's host."
else
read -rp "Address:port for gunicorn to listen on [127.0.0.1:8000]: " GUNICORN_BIND
GUNICORN_BIND="${GUNICORN_BIND:-127.0.0.1:8000}"
fi
fi
GUNICORN_BIND="${GUNICORN_BIND:-127.0.0.1:8000}"
cat > "$ENV_FILE" <<EOF cat > "$ENV_FILE" <<EOF
SECRET_KEY=$SECRET_KEY SECRET_KEY=$SECRET_KEY
DEBUG=False DEBUG=False
ALLOWED_HOSTS=$ALLOWED_HOSTS ALLOWED_HOSTS=$ALLOWED_HOSTS
DATABASE_URL=$DATABASE_URL DATABASE_URL=$DATABASE_URL
REDIS_URL=${EVECTION_REDIS_URL:-redis://localhost:6379/0} REDIS_URL=${EVECTION_REDIS_URL:-redis://localhost:6379/0}
EVECTION_GUNICORN_BIND=$GUNICORN_BIND
EVECTION_SIGNUP_ENABLED=False EVECTION_SIGNUP_ENABLED=False
EVECTION_MIN_FETCH_INTERVAL_MINUTES=15 EVECTION_MIN_FETCH_INTERVAL_MINUTES=15
EVECTION_PWA_CACHE_VERSION=v1 EVECTION_PWA_CACHE_VERSION=${LATEST_TAG:-v1}
EOF EOF
chown "$EVECTION_USER:$EVECTION_USER" "$ENV_FILE" chown "$EVECTION_USER:$EVECTION_USER" "$ENV_FILE"
chmod 600 "$ENV_FILE" chmod 600 "$ENV_FILE"
@@ -138,6 +237,11 @@ else
echo "==> $ENV_FILE already exists — leaving it alone" echo "==> $ENV_FILE already exists — leaving it alone"
fi fi
# Needed below for the final "gunicorn is listening on..." message even
# when $ENV_FILE already existed and the block above was skipped.
GUNICORN_BIND="$(grep -m1 '^EVECTION_GUNICORN_BIND=' "$ENV_FILE" | cut -d= -f2-)"
GUNICORN_BIND="${GUNICORN_BIND:-127.0.0.1:8000}"
# --- 5. Migrate + static files -------------------------------------------- # --- 5. Migrate + static files --------------------------------------------
echo "==> Running migrations" echo "==> Running migrations"
@@ -158,56 +262,21 @@ for unit in evection-gunicorn evection-celery-worker evection-celery-beat; do
"$APP_DIR/docs/deploy/${unit}.service" > "/etc/systemd/system/${unit}.service" "$APP_DIR/docs/deploy/${unit}.service" > "/etc/systemd/system/${unit}.service"
done done
systemctl daemon-reload systemctl daemon-reload
systemctl enable --now evection-gunicorn evection-celery-worker evection-celery-beat systemctl enable evection-gunicorn evection-celery-worker evection-celery-beat
# Not "enable --now": that only starts a service if it's stopped, so a
# re-run after a unit file changed (e.g. a new EVECTION_GUNICORN_BIND)
# would reload the daemon but leave the already-running process on its
# stale old ExecStart indefinitely. Always restart explicitly instead.
systemctl restart evection-gunicorn evection-celery-worker evection-celery-beat
echo echo
echo "==> Services installed. Checking status..." echo "==> Services installed. Checking status..."
systemctl --no-pager status evection-gunicorn evection-celery-worker evection-celery-beat \ systemctl --no-pager status evection-gunicorn evection-celery-worker evection-celery-beat \
|| echo "One or more services didn't start cleanly — see 'journalctl -u evection-gunicorn' etc." >&2 || echo "One or more services didn't start cleanly — see 'journalctl -u evection-gunicorn' etc." >&2
# --- 7. nginx (optional) ---------------------------------------------------
SET_UP_NGINX="${EVECTION_SETUP_NGINX:-}"
if [ -z "$SET_UP_NGINX" ] && [ -t 0 ]; then
echo
read -rp "Set up nginx and an HTTPS certificate now? [y/N]: " SET_UP_NGINX
fi
if [[ "${SET_UP_NGINX:-}" =~ ^[Yy]$ ]]; then
if ! command -v nginx >/dev/null 2>&1; then
echo "nginx isn't installed — install it first (apt install nginx), then" \
"run docs/install.md's nginx section manually." >&2
else
DOMAIN="${EVECTION_DOMAIN:-}"
if [ -z "$DOMAIN" ] && [ -t 0 ]; then
read -rp "Domain name for nginx/TLS (must match a hostname in ALLOWED_HOSTS): " DOMAIN
fi
if [ -z "$DOMAIN" ]; then
echo "No domain given — skipping nginx setup. Run docs/install.md's nginx section manually." >&2
else
sed -e "s/your-domain.example/$DOMAIN/g" \
-e "s#/opt/evection#$EVECTION_HOME#g" \
"$APP_DIR/docs/deploy/nginx.conf.example" > "/etc/nginx/sites-available/evection"
ln -sf /etc/nginx/sites-available/evection /etc/nginx/sites-enabled/evection
if nginx -t; then
systemctl reload nginx
echo "==> nginx configured for $DOMAIN (HTTP only so far)"
if command -v certbot >/dev/null 2>&1; then
certbot --nginx -d "$DOMAIN" || echo "certbot failed — run it manually: certbot --nginx -d $DOMAIN" >&2
else
echo "certbot isn't installed. Install it and run:" \
"apt install certbot python3-certbot-nginx && certbot --nginx -d $DOMAIN" >&2
fi
else
echo "nginx config test failed — not enabling it. Check /etc/nginx/sites-available/evection." >&2
fi
fi
fi
else
echo "==> Skipping nginx — see docs/install.md step 7 to set it up yourself."
fi
echo echo
echo "==> Done." echo "==> Done."
echo " Visit the site (once nginx/TLS is in place) — the setup wizard creates your admin account." echo " gunicorn is listening on $GUNICORN_BIND — point your reverse proxy at it"
echo " (see docs/install.md's \"Reverse proxy\" section) before visiting the site."
echo " Once that's in place, the setup wizard creates your admin account on first visit."
echo " To upgrade later: sudo ./scripts/upgrade.sh" echo " To upgrade later: sudo ./scripts/upgrade.sh"
+74 -8
View File
@@ -26,7 +26,11 @@ APP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
EVECTION_HOME="$(dirname "$APP_DIR")" EVECTION_HOME="$(dirname "$APP_DIR")"
VENV_DIR="$EVECTION_HOME/venv" VENV_DIR="$EVECTION_HOME/venv"
EVECTION_USER="${EVECTION_USER:-evection}" EVECTION_USER="${EVECTION_USER:-evection}"
SOCKET="/run/evection/gunicorn.sock"
# $APP_DIR is owned by $EVECTION_USER, but the git steps below run as
# root — without this, git refuses to touch a repo it doesn't own
# ("detected dubious ownership").
git config --global --add safe.directory "$APP_DIR"
if ! id -u "$EVECTION_USER" >/dev/null 2>&1; then if ! id -u "$EVECTION_USER" >/dev/null 2>&1; then
echo "System user '$EVECTION_USER' doesn't exist — is Evection installed?" \ echo "System user '$EVECTION_USER' doesn't exist — is Evection installed?" \
@@ -50,11 +54,34 @@ cd "$APP_DIR"
# the service user has no reason to have its own git credentials set up, # the service user has no reason to have its own git credentials set up,
# and GIT_TERMINAL_PROMPT=0 means a credential problem fails fast instead # and GIT_TERMINAL_PROMPT=0 means a credential problem fails fast instead
# of hanging the upgrade on a silent username/password prompt. # of hanging the upgrade on a silent username/password prompt.
GIT_TERMINAL_PROMPT=0 git fetch --tags --force #
# Deliberately NOT --force (S7): a plain `git fetch --tags` refuses to
# move a tag ref this checkout already has onto a different commit — the
# exact situation if a release tag were rewritten (rolled back, re-cut
# against a different commit, or tampered with) rather than a new tag
# being added. --force would silently follow that rewritten tag into
# production; refusing and stopping here means a human has to look at
# what actually changed before this script will deploy it.
set +e
FETCH_OUTPUT="$(GIT_TERMINAL_PROMPT=0 git fetch --tags 2>&1)"
FETCH_STATUS=$?
set -e
if [ "$FETCH_STATUS" -ne 0 ]; then
echo "$FETCH_OUTPUT" >&2
if echo "$FETCH_OUTPUT" | grep -q "would clobber existing tag"; then
echo
echo "A release tag points at a different commit than this checkout" \
"already has for it, rather than only new tags having been added." \
"Refusing to force-follow a rewritten tag (S7) — review what" \
"actually changed before deciding to update it yourself, e.g.:" \
" git fetch --tags --dry-run -v" >&2
fi
exit 1
fi
LATEST_TAG="$(git tag --sort=-creatordate | head -n1)" LATEST_TAG="$(git tag --sort=-creatordate | head -n1)"
if [ -z "$LATEST_TAG" ]; then if [ -z "$LATEST_TAG" ]; then
echo "No tags found in this repo — nothing to deploy." \ echo "No tags found in this repo — nothing to deploy." \
"Cut and push a release tag first (e.g. git tag v$(date +%Y.%m.%d) && git push --tags)." >&2 "Cut and push a release tag first (e.g. git tag v$(date +%Y.%-m.%-d) && git push --tags)." >&2
exit 1 exit 1
fi fi
CURRENT_REF="$(git describe --tags --exact-match 2>/dev/null || echo "none")" CURRENT_REF="$(git describe --tags --exact-match 2>/dev/null || echo "none")"
@@ -68,8 +95,27 @@ echo "Deploying $LATEST_TAG (currently on: $CURRENT_REF)"
git checkout --quiet --detach "$LATEST_TAG" git checkout --quiet --detach "$LATEST_TAG"
chown -R "$EVECTION_USER:$EVECTION_USER" "$APP_DIR" chown -R "$EVECTION_USER:$EVECTION_USER" "$APP_DIR"
# Bump the PWA's cache-busting version to this release's tag on every
# deploy, not just ones that happened to touch static assets — a missed
# manual bump otherwise leaves installed PWAs serving a stale cached
# shell/pages indefinitely. Cheap even when nothing cacheable changed:
# it just makes the service worker open a fresh cache name next time it
# activates.
if grep -q '^EVECTION_PWA_CACHE_VERSION=' "$APP_DIR/.env"; then
sed -i "s/^EVECTION_PWA_CACHE_VERSION=.*/EVECTION_PWA_CACHE_VERSION=$LATEST_TAG/" "$APP_DIR/.env"
else
echo "EVECTION_PWA_CACHE_VERSION=$LATEST_TAG" >> "$APP_DIR/.env"
fi
echo "==> Installing updated dependencies" echo "==> Installing updated dependencies"
sudo -u "$EVECTION_USER" -H "$VENV_DIR/bin/pip" install --upgrade "$APP_DIR" # requirements.txt (generated by `pip-compile --generate-hashes`, committed
# alongside pyproject.toml, and checked out as part of $LATEST_TAG above)
# pins every dependency to an exact, hash-verified version (S7) — this
# deploy gets exactly the dependency code that release was tested
# against, and a tampered or substituted package on the index fails the
# hash check instead of installing silently.
sudo -u "$EVECTION_USER" -H "$VENV_DIR/bin/pip" install --require-hashes -r "$APP_DIR/requirements.txt"
sudo -u "$EVECTION_USER" -H "$VENV_DIR/bin/pip" install --no-deps --upgrade "$APP_DIR"
echo "==> Running migrations" echo "==> Running migrations"
sudo -u "$EVECTION_USER" -H env DJANGO_SETTINGS_MODULE=config.settings.prod \ sudo -u "$EVECTION_USER" -H env DJANGO_SETTINGS_MODULE=config.settings.prod \
@@ -94,7 +140,30 @@ fi
echo "==> Verifying" echo "==> Verifying"
sleep 2 sleep 2
if curl -sf --unix-socket "$SOCKET" http://localhost/health/ >/dev/null; then # gunicorn always listens on a plain TCP address (EVECTION_GUNICORN_BIND in
# .env, set by scripts/install.sh) rather than a Unix socket, since the
# reverse proxy in front of it may be on a different host entirely — see
# docs/install.md. 0.0.0.0 isn't a valid address to connect *to*, so check
# against localhost in that case; this health check only needs to prove
# gunicorn itself is answering on this host, not reach it the way the
# proxy does.
HEALTH_BIND="$(grep -m1 '^EVECTION_GUNICORN_BIND=' "$APP_DIR/.env" | cut -d= -f2-)"
HEALTH_BIND="${HEALTH_BIND:-127.0.0.1:8000}"
HEALTH_HOST="${HEALTH_BIND%:*}"
HEALTH_PORT="${HEALTH_BIND##*:}"
if [ "$HEALTH_HOST" = "0.0.0.0" ]; then
HEALTH_HOST="127.0.0.1"
fi
# Use the venv's own Python rather than curl — curl isn't guaranteed to be
# installed on a minimal host, but the venv obviously already is.
if "$VENV_DIR/bin/python" -c "
import sys, urllib.request
try:
urllib.request.urlopen('http://$HEALTH_HOST:$HEALTH_PORT/health/', timeout=5)
except Exception as e:
print(e, file=sys.stderr)
sys.exit(1)
"; then
echo "Health check OK" echo "Health check OK"
else else
echo "Health check FAILED — check: journalctl -u evection-gunicorn -n 50" >&2 echo "Health check FAILED — check: journalctl -u evection-gunicorn -n 50" >&2
@@ -104,7 +173,4 @@ systemctl status evection-gunicorn --no-pager -l | head -10
echo echo
echo "==> Deployed $LATEST_TAG." echo "==> Deployed $LATEST_TAG."
echo " If this release changed cached static assets (CSS/JS/icons), bump"
echo " EVECTION_PWA_CACHE_VERSION in $APP_DIR/.env and restart evection-gunicorn"
echo " again so installed PWAs drop their old cache instead of serving stale assets."
echo " journalctl -u evection-gunicorn -f to watch logs." echo " journalctl -u evection-gunicorn -f to watch logs."
+56 -1
View File
@@ -113,6 +113,19 @@ form p {
display: inline; display: inline;
} }
.new-articles-banner {
display: block;
width: 100%;
margin: 0 0 0.75rem;
padding: 0.5rem 0.75rem;
border: none;
border-radius: 0.375rem;
background: var(--accent);
color: #fff;
font-weight: 600;
text-align: center;
}
.empty-state { .empty-state {
opacity: 0.6; opacity: 0.6;
padding: 2rem 0; padding: 2rem 0;
@@ -290,22 +303,63 @@ form p {
padding: 0 0.5rem; padding: 0 0.5rem;
} }
/* `summary`'s native disclosure triangle is a ::marker, positioned for
list-item/inline content — once the tag-name + unread-count row below
needed to be a flex box (block-level), the marker got pushed onto its
own line above instead of sitting beside it. Replacing it with a
plain ::before triangle lets `summary` itself be a flex row with that
triangle as a normal flex item, so it stays inline no matter what the
rest of the row contains. */
.sidebar-section summary { .sidebar-section summary {
display: flex;
align-items: center;
gap: 0.4rem;
padding: 0.4rem 0.75rem; padding: 0.4rem 0.75rem;
cursor: pointer; cursor: pointer;
color: var(--text-dim); color: var(--text-dim);
font-size: 0.75rem; font-size: 0.75rem;
text-transform: uppercase; text-transform: uppercase;
letter-spacing: 0.04em; letter-spacing: 0.04em;
list-style: none;
}
.sidebar-section summary::-webkit-details-marker {
display: none;
}
.sidebar-section summary::before {
content: "▶";
flex: 0 0 auto;
font-size: 0.6rem;
transition: transform 0.1s ease;
}
.sidebar-section[open] > summary::before {
transform: rotate(90deg);
}
.sidebar-tag-row {
display: flex;
align-items: center;
justify-content: space-between;
gap: 0.4rem;
flex: 1 1 auto;
min-width: 0;
} }
.sidebar-tag-link { .sidebar-tag-link {
display: inline-block;
padding: 0.15rem 0.4rem;
margin: -0.15rem -0.4rem;
border-radius: 0.375rem;
color: inherit; color: inherit;
text-decoration: none; text-decoration: none;
} }
.sidebar-tag-link.active { .sidebar-tag-link.active {
color: var(--accent); background: var(--bg-row-selected);
color: var(--text);
font-weight: 600;
} }
.sidebar-item-list { .sidebar-item-list {
@@ -505,6 +559,7 @@ form p {
font-size: 0.8rem; font-size: 0.8rem;
color: var(--text-dim); color: var(--text-dim);
flex: 0 0 auto; flex: 0 0 auto;
white-space: nowrap;
} }
.article-actions { .article-actions {
+61
View File
@@ -0,0 +1,61 @@
(function () {
// The server groups articles into "Received today/yesterday/<date>"
// headers using UTC (config.settings.base TIME_ZONE), so an article
// published near UTC midnight can land in the wrong day group from the
// viewer's own perspective. Each article row carries its UTC instant in
// data-published-utc; this throws away the server's date-header <li>s
// and rebuilds them using the browser's local calendar day instead.
var MONTHS = [
"January", "February", "March", "April", "May", "June",
"July", "August", "September", "October", "November", "December",
];
function localDateOf(iso) {
if (!iso) return null;
var date = new Date(iso);
if (isNaN(date.getTime())) return null;
date.setHours(0, 0, 0, 0);
return date;
}
function labelFor(day, today) {
if (day === null) return "Undated";
var formatted = day.getDate() + " " + MONTHS[day.getMonth()] + " " + day.getFullYear();
var diffDays = Math.round((today - day) / 86400000);
if (diffDays === 0) return "Received today — " + formatted;
if (diffDays === 1) return "Received yesterday — " + formatted;
return "Received " + formatted;
}
function regroupDateHeaders(root) {
if (!root) return;
var list = root.matches && root.matches("ul.article-list") ? root : root.querySelector("ul.article-list");
if (!list || !list.querySelector("li.article-row[data-published-utc], li.article-row")) return;
list.querySelectorAll("li.date-header").forEach(function (el) {
el.parentNode.removeChild(el);
});
var today = new Date();
today.setHours(0, 0, 0, 0);
var currentKey;
list.querySelectorAll("li.article-row").forEach(function (row) {
var day = localDateOf(row.getAttribute("data-published-utc"));
var key = day === null ? "undated" : day.getTime();
if (key !== currentKey) {
var header = document.createElement("li");
header.className = "date-header";
header.textContent = labelFor(day, today);
list.insertBefore(header, row);
currentKey = key;
}
});
}
regroupDateHeaders(document);
document.body.addEventListener("htmx:afterSwap", function (event) {
regroupDateHeaders(event.target);
});
})();
+45
View File
@@ -0,0 +1,45 @@
(function () {
// Everything is stored and server-rendered in UTC (config.settings.base
// TIME_ZONE) — these <time datetime="..."> elements carry that UTC
// instant plus a UTC-formatted fallback as their initial text, and this
// reformats them using the browser's own timezone, client-side, so the
// server never needs to know (or guess) where the viewer actually is.
var SHORT_OPTIONS = { month: "short", day: "numeric", hour: "numeric", minute: "2-digit" };
var LONG_OPTIONS = {
year: "numeric",
month: "short",
day: "numeric",
hour: "2-digit",
minute: "2-digit",
};
function localizeTimesWithin(root) {
if (!root || typeof root.querySelectorAll !== "function") return;
root.querySelectorAll("time[data-local-format]").forEach(function (el) {
var iso = el.getAttribute("datetime");
if (!iso) return;
var date = new Date(iso);
if (isNaN(date.getTime())) return;
var options = el.dataset.localFormat === "short" ? SHORT_OPTIONS : LONG_OPTIONS;
if (options === SHORT_OPTIONS && date.getFullYear() !== new Date().getFullYear()) {
// Date-only rows from a previous year would otherwise be ambiguous.
options = Object.assign({ year: "numeric" }, SHORT_OPTIONS);
}
try {
el.textContent = date.toLocaleString(undefined, options);
} catch (e) {
/* Intl unavailable or threw — leave the server-rendered UTC text as-is */
}
});
}
localizeTimesWithin(document);
// Article rows/content are frequently swapped in by htmx (pagination,
// the inline expand-article view, the dashboard's retry action) without
// a full page load, so every swap needs the same pass over just the
// new content.
document.body.addEventListener("htmx:afterSwap", function (event) {
localizeTimesWithin(event.target);
});
})();
+46
View File
@@ -0,0 +1,46 @@
(function () {
// Remembers which sidebar <details> sections (one per tag, plus
// "Untagged" — see templates/core/_sidebar.html's data-sidebar-key)
// the user has collapsed, so a full page navigation (this isn't an
// SPA — every click re-renders the sidebar server-side) doesn't keep
// snapping them back open. Per-browser via localStorage, same pattern
// as theme.js/theme-init.js — not synced across devices/accounts.
var STORAGE_KEY = "evection-sidebar-collapsed";
function readCollapsed() {
try {
var raw = localStorage.getItem(STORAGE_KEY);
return raw ? JSON.parse(raw) : [];
} catch (e) {
return [];
}
}
function writeCollapsed(keys) {
try {
localStorage.setItem(STORAGE_KEY, JSON.stringify(keys));
} catch (e) {
/* localStorage unavailable (private mode, etc.) — state just won't persist */
}
}
var collapsed = readCollapsed();
document.querySelectorAll(".sidebar-section[data-sidebar-key]").forEach(function (details) {
var key = details.dataset.sidebarKey;
if (collapsed.indexOf(key) !== -1) {
details.open = false;
}
details.addEventListener("toggle", function () {
var current = readCollapsed();
var index = current.indexOf(key);
if (details.open) {
if (index !== -1) current.splice(index, 1);
} else if (index === -1) {
current.push(key);
}
writeCollapsed(current);
});
});
})();
+23 -5
View File
@@ -1,8 +1,11 @@
function toggleArticle(event, id) { // Delegated listeners, not inline onclick="" attributes: production's
event.preventDefault(); // Content Security Policy (config/settings/prod.py, script-src 'self')
var link = event.currentTarget; // blocks inline event handlers, which silently turned the title click into
// a plain link to the detail page and the new-articles banner into a no-op.
function toggleArticle(link) {
var row = link.closest("li.article-row"); var row = link.closest("li.article-row");
var body = document.getElementById("article-body-" + id); var body = row.querySelector(".article-inline-body");
var wasOpen = row.classList.contains("is-expanded"); var wasOpen = row.classList.contains("is-expanded");
document.querySelectorAll("li.article-row.is-expanded").forEach(function (openRow) { document.querySelectorAll("li.article-row.is-expanded").forEach(function (openRow) {
@@ -25,5 +28,20 @@ function toggleArticle(event, id) {
); );
} }
} }
return false;
} }
document.addEventListener("click", function (event) {
var link = event.target.closest("a.article-title[data-inline-url]");
if (link) {
// Leave modified clicks alone so "open in new tab" still works.
if (event.button !== 0 || event.ctrlKey || event.metaKey || event.shiftKey || event.altKey) {
return;
}
event.preventDefault();
toggleArticle(link);
return;
}
if (event.target.closest(".new-articles-banner")) {
location.reload();
}
});
+85
View File
@@ -0,0 +1,85 @@
// Marks the browser tab with a red dot on its favicon when new unread
// articles have arrived since this page loaded — visible from a background
// tab, unlike the in-page "new articles" banner. Runs on every
// authenticated page; a full page load resets the baseline (and the icon),
// which is exactly when the user has seen the new articles.
(function () {
var POLL_MS = 30000;
var url = document.currentScript && document.currentScript.dataset.url;
var icon = document.querySelector('link[rel="icon"]');
if (!url || !icon) return;
var originalHref = icon.href;
var baseline = null;
var badged = null; // the dotted icon's data: URL, drawn once on first need
var showing = false;
function drawDot() {
return new Promise(function (resolve) {
var image = new Image();
image.onload = function () {
var size = 64;
var canvas = document.createElement("canvas");
canvas.width = canvas.height = size;
var ctx = canvas.getContext("2d");
ctx.drawImage(image, 0, 0, size, size);
ctx.beginPath();
ctx.arc(size * 0.74, size * 0.26, size * 0.23, 0, 2 * Math.PI);
ctx.fillStyle = "#e53935";
ctx.fill();
ctx.lineWidth = size * 0.07;
ctx.strokeStyle = "#ffffff";
ctx.stroke();
resolve(canvas.toDataURL("image/png"));
};
image.onerror = function () {
resolve(null);
};
image.src = originalHref;
});
}
// Replace the <link> rather than editing href: some browsers ignore an
// in-place change to an existing favicon element.
function setIcon(href) {
var fresh = icon.cloneNode();
fresh.href = href;
icon.parentNode.replaceChild(fresh, icon);
icon = fresh;
}
function showDot(on) {
if (on === showing) return Promise.resolve();
showing = on;
if (!on) {
setIcon(originalHref);
return Promise.resolve();
}
var ready = badged ? Promise.resolve(badged) : drawDot();
return ready.then(function (href) {
badged = href;
if (href && showing) setIcon(href);
});
}
function poll() {
var query = baseline === null ? "" : "?since=" + baseline;
fetch(url + query, { credentials: "same-origin", headers: { Accept: "application/json" } })
.then(function (response) {
// A redirect means the session ended (we'd be reading the login
// page) — stop polling instead of hammering it.
if (!response.ok || response.redirected) throw new Error("stop");
return response.json();
})
.then(function (data) {
if (baseline === null) baseline = data.latest_id;
else showDot(data.new_count > 0);
setTimeout(poll, POLL_MS);
})
.catch(function (error) {
if (error.message !== "stop") setTimeout(poll, POLL_MS);
});
}
poll();
})();
+7 -1
View File
@@ -53,7 +53,13 @@
{% endif %} {% endif %}
<script src="{% static 'js/stream.js' %}" defer></script> <script src="{% static 'js/stream.js' %}" defer></script>
{% if user.is_authenticated %}<script src="{% static 'js/theme.js' %}" defer></script>{% endif %} <script src="{% static 'js/localize-times.js' %}" defer></script>
<script src="{% static 'js/date-groups.js' %}" defer></script>
{% if user.is_authenticated %}
<script src="{% static 'js/theme.js' %}" defer></script>
<script src="{% static 'js/sidebar-state.js' %}" defer></script>
<script src="{% static 'js/tab-badge.js' %}" data-url="{% url 'reader:new_since' %}" defer></script>
{% endif %}
<script src="{% static 'js/register-sw.js' %}" defer></script> <script src="{% static 'js/register-sw.js' %}" defer></script>
</body> </body>
</html> </html>
+5 -2
View File
@@ -31,10 +31,13 @@
</ul> </ul>
{% for entry in sidebar_tag_groups %} {% for entry in sidebar_tag_groups %}
<details class="sidebar-section" open> <details class="sidebar-section" data-sidebar-key="tag-{{ entry.tag.pk }}" open>
<summary> <summary>
<span class="sidebar-tag-row">
<a href="{% url 'reader:stream' %}?tag={{ entry.tag.pk }}" <a href="{% url 'reader:stream' %}?tag={{ entry.tag.pk }}"
class="sidebar-tag-link{% if request.GET.tag == entry.tag.pk|stringformat:'s' %} active{% endif %}">{{ entry.tag.name }}</a> class="sidebar-tag-link{% if request.GET.tag == entry.tag.pk|stringformat:'s' %} active{% endif %}">{{ entry.tag.name }}</a>
{% if entry.unread_count %}<span class="unread-count">{{ entry.unread_count }}</span>{% endif %}
</span>
</summary> </summary>
{% if entry.subscriptions %} {% if entry.subscriptions %}
<ul class="sidebar-item-list"> <ul class="sidebar-item-list">
@@ -54,7 +57,7 @@
{% endfor %} {% endfor %}
{% if sidebar_untagged %} {% if sidebar_untagged %}
<details class="sidebar-section" open> <details class="sidebar-section" data-sidebar-key="untagged" open>
<summary>Untagged</summary> <summary>Untagged</summary>
<ul class="sidebar-item-list"> <ul class="sidebar-item-list">
{% for subscription in sidebar_untagged %} {% for subscription in sidebar_untagged %}
+10 -3
View File
@@ -1,4 +1,5 @@
{% extends "base.html" %} {% extends "base.html" %}
{% load safe_urls %}
{% block title %}Dashboard — Evection{% endblock %} {% block title %}Dashboard — Evection{% endblock %}
{% block content %} {% block content %}
<h1>Dashboard</h1> <h1>Dashboard</h1>
@@ -6,7 +7,7 @@
<ul class="dashboard-stats"> <ul class="dashboard-stats">
<li>{{ stats.feed_count }} feed(s) — {{ stats.active_feed_count }} active</li> <li>{{ stats.feed_count }} feed(s) — {{ stats.active_feed_count }} active</li>
<li>{{ stats.subscription_count }} subscription(s)</li> <li>{{ stats.subscription_count }} subscription(s)</li>
<li>{{ stats.article_count }} article(s)</li> <li>{{ stats.article_count }} article(s){% if stats.oldest_article_at %}, oldest fetched <time datetime="{{ stats.oldest_article_at|date:"c" }}" data-local-format="long">{{ stats.oldest_article_at|date:"M j, Y H:i" }}</time>{% endif %}</li>
<li>{{ stats.error_feed_count }} feed(s) currently erroring</li> <li>{{ stats.error_feed_count }} feed(s) currently erroring</li>
</ul> </ul>
@@ -27,11 +28,17 @@
<tbody> <tbody>
{% for feed in erroring_feeds %} {% for feed in erroring_feeds %}
<tr> <tr>
<td><a href="{{ feed.site_url|default:feed.feed_url }}">{{ feed.title|default:feed.feed_url }}</a></td> <td><a href="{{ feed.site_url|default:feed.feed_url|safe_http_url }}">{{ feed.title|default:feed.feed_url }}</a></td>
<td>{{ feed.subscriber_count }}</td> <td>{{ feed.subscriber_count }}</td>
<td>{{ feed.get_last_fetch_status_display }}</td> <td>{{ feed.get_last_fetch_status_display }}</td>
<td>{{ feed.fetch_error_count }}</td> <td>{{ feed.fetch_error_count }}</td>
<td>{{ feed.last_fetched_at|default:"never" }}</td> <td>
{% if feed.last_fetched_at %}
<time datetime="{{ feed.last_fetched_at|date:"c" }}" data-local-format="long">{{ feed.last_fetched_at|date:"M j, Y H:i" }}</time>
{% else %}
never
{% endif %}
</td>
<td>{{ feed.last_error_message|default:"—" }}</td> <td>{{ feed.last_error_message|default:"—" }}</td>
<td> <td>
<form method="post" action="{% url 'core:retry_feed' feed.id %}"> <form method="post" action="{% url 'core:retry_feed' feed.id %}">
+41 -1
View File
@@ -42,12 +42,38 @@ async function trimCache(cacheName, maxItems) {
await Promise.all(toDelete.map((key) => cache.delete(key))); await Promise.all(toDelete.map((key) => cache.delete(key)));
} }
// Every HTML response carries X-Evection-Auth: an opaque id for who it was
// rendered for ("anon" when no one is logged in — see apps.pwa.middleware).
// PAGES_CACHE holds one person's pages; the marker it belongs to is kept in
// the shell cache (a service worker's own memory doesn't survive being
// stopped). Any change — session expired, logged in, switched accounts —
// purges the pages before anything new is stored, so one session's
// authenticated HTML can never be served to the next, whether or not the
// logout request itself passed through this worker.
const AUTH_MARKER_KEY = "/__evection-auth-marker__";
async function syncAuthMarker(response) {
const marker = response.headers.get("X-Evection-Auth");
if (marker === null) return null; // not an HTML page we can attribute
const shell = await caches.open(SHELL_CACHE);
const stored = await shell.match(AUTH_MARKER_KEY);
const previous = stored ? await stored.text() : null;
if (previous !== marker) {
await caches.delete(PAGES_CACHE);
await shell.put(AUTH_MARKER_KEY, new Response(marker));
}
return marker;
}
async function networkFirstForNavigation(request) { async function networkFirstForNavigation(request) {
try { try {
const response = await fetch(request); const response = await fetch(request);
const marker = await syncAuthMarker(response);
if (marker !== null && marker !== "anon") {
const cache = await caches.open(PAGES_CACHE); const cache = await caches.open(PAGES_CACHE);
cache.put(request, response.clone()); cache.put(request, response.clone());
trimCache(PAGES_CACHE, MAX_CACHED_PAGES); trimCache(PAGES_CACHE, MAX_CACHED_PAGES);
}
return response; return response;
} catch (err) { } catch (err) {
const cached = await caches.match(request); const cached = await caches.match(request);
@@ -66,8 +92,23 @@ async function cacheFirstForAsset(request) {
return response; return response;
} }
const LOGOUT_PATH = "{% url 'accounts:logout' %}";
self.addEventListener("fetch", (event) => { self.addEventListener("fetch", (event) => {
const { request } = event; const { request } = event;
const url = new URL(request.url);
// Logging out ends this device's session, so any authenticated HTML
// sitting in PAGES_CACHE must not outlive it — otherwise a later
// offline navigation (shared device, different account logging in
// next) could still serve a previous user's cached article pages.
// Purging it synchronously with the logout request itself, rather
// than via a postMessage from the page, means there's no race with
// the navigation that follows.
if (request.method === "POST" && url.pathname === LOGOUT_PATH) {
event.respondWith(caches.delete(PAGES_CACHE).then(() => fetch(request)));
return;
}
// Mutations (mark-read, star, OPML upload, forms...) are always // Mutations (mark-read, star, OPML upload, forms...) are always
// network-only — never intercept anything but GET. // network-only — never intercept anything but GET.
@@ -78,7 +119,6 @@ self.addEventListener("fetch", (event) => {
return; return;
} }
const url = new URL(request.url);
if (url.pathname.startsWith("/static/")) { if (url.pathname.startsWith("/static/")) {
event.respondWith(cacheFirstForAsset(request)); event.respondWith(cacheFirstForAsset(request));
} }
+5 -3
View File
@@ -1,10 +1,12 @@
<div class="article-content-header"> {% load safe_urls %}<div class="article-content-header">
<h2 class="article-content-title">{{ article.title|default:"(untitled)" }}</h2> <h2 class="article-content-title">{{ article.title|default:"(untitled)" }}</h2>
<p class="article-meta"> <p class="article-meta">
{% if article.feed.favicon_url %}<img src="{{ article.feed.favicon_url }}" alt="" class="feed-favicon">{% endif %} {% if article.feed.favicon_url %}<img src="{{ article.feed.favicon_url }}" alt="" class="feed-favicon">{% endif %}
{{ article.feed.title }} &middot; {{ article.published_at|date:"M j, Y H:i" }} {{ article.feed.title }}
{# fetched_at stands in when the feed gave no publish date #}
&middot; <time datetime="{{ article.published_at|default:article.fetched_at|date:"c" }}" data-local-format="long">{{ article.published_at|default:article.fetched_at|date:"M j, Y H:i" }}</time>
{% if article.author %} &middot; {{ article.author }}{% endif %} {% if article.author %} &middot; {{ article.author }}{% endif %}
{% if article.link %} &middot; <a href="{{ article.link }}" target="_blank" rel="noopener noreferrer">Original &#8599;</a>{% endif %} {% with safe_link=article.link|safe_http_url %}{% if safe_link %} &middot; <a href="{{ safe_link }}" target="_blank" rel="noopener noreferrer">Original &#8599;</a>{% endif %}{% endwith %}
</p> </p>
<div <div
+4 -4
View File
@@ -10,10 +10,10 @@
</ul> </ul>
<nav class="pagination"> <nav class="pagination">
{% if has_previous_page %} {% if has_newer_page %}
<a href="?filter={{ active_filter }}{% if active_tag %}&tag={{ active_tag.pk }}{% endif %}&page={{ page|add:"-1" }}">&larr; Newer</a> <a href="?{{ newer_qs }}">&larr; Newer</a>
{% endif %} {% endif %}
{% if has_next_page %} {% if has_older_page %}
<a href="?filter={{ active_filter }}{% if active_tag %}&tag={{ active_tag.pk }}{% endif %}&page={{ page|add:"1" }}">Older &rarr;</a> <a href="?{{ older_qs }}">Older &rarr;</a>
{% endif %} {% endif %}
</nav> </nav>
+6 -5
View File
@@ -1,14 +1,15 @@
<li id="article-{{ article.pk }}" class="article-row{% if article.is_read %} is-read{% endif %}"> <li id="article-{{ article.pk }}" class="article-row{% if article.is_read %} is-read{% endif %}"
data-published-utc="{{ article.published_at|default:article.fetched_at|date:"c" }}">
<div class="article-row-main"> <div class="article-row-main">
{% include "reader/_article_actions.html" %} {% include "reader/_article_actions.html" %}
{% if article.feed.favicon_url %} {% if article.feed.favicon_url %}
<img src="{{ article.feed.favicon_url }}" alt="" class="feed-favicon"> <img src="{{ article.feed.favicon_url }}" alt="" class="feed-favicon">
{% endif %} {% endif %}
<span class="article-feed-name">{{ article.feed.title|default:article.feed.feed_url }}</span> <span class="article-feed-name">{{ article.feed.title|default:article.feed.feed_url }}</span>
<a href="{% url 'reader:article_detail' article.pk %}" class="article-title" <a href="{% url 'reader:article_detail' article.pk %}{% if scope_qs %}?{{ scope_qs }}{% endif %}" class="article-title"
data-inline-url="{% url 'reader:article_inline' article.pk %}" data-inline-url="{% url 'reader:article_inline' article.pk %}">{{ article.title|default:"(untitled)" }}</a>
onclick="return toggleArticle(event, {{ article.pk }})">{{ article.title|default:"(untitled)" }}</a> {# fetched_at stands in when a feed gives no publish date — same fallback the stream's sort order uses #}
<span class="article-time">{{ article.published_at|date:"g:i A" }}</span> <time class="article-time" datetime="{{ article.published_at|default:article.fetched_at|date:"c" }}" data-local-format="short">{{ article.published_at|default:article.fetched_at|date:"M j, g:i A" }}</time>
</div> </div>
<div id="article-body-{{ article.pk }}" class="article-inline-body" hidden></div> <div id="article-body-{{ article.pk }}" class="article-inline-body" hidden></div>
</li> </li>
@@ -0,0 +1,5 @@
{% if new_count %}
<button type="button" class="new-articles-banner">
{{ new_count }} new article{{ new_count|pluralize }} — click to show
</button>
{% endif %}
+1 -1
View File
@@ -1,7 +1,7 @@
{% extends "base.html" %} {% extends "base.html" %}
{% block title %}{{ article.title|default:"Article" }} — Evection{% endblock %} {% block title %}{{ article.title|default:"Article" }} — Evection{% endblock %}
{% block content %} {% block content %}
<p><a href="{% url 'reader:stream' %}">&larr; Back to stream</a></p> <p><a href="{% url 'reader:stream' %}{% if back_qs %}?{{ back_qs }}{% endif %}">&larr; Back to stream</a></p>
<article> <article>
{% include "reader/_article_actions.html" %} {% include "reader/_article_actions.html" %}
{% include "reader/_article_content.html" %} {% include "reader/_article_content.html" %}
+6
View File
@@ -23,5 +23,11 @@
</form> </form>
</div> </div>
<div id="new-articles-banner"
hx-get="{% url 'reader:check_new' %}?since={{ latest_article_id }}&{{ new_check_qs }}"
hx-trigger="every 30s"
hx-target="this"
hx-swap="innerHTML"></div>
{% include "reader/_article_list.html" %} {% include "reader/_article_list.html" %}
{% endblock %} {% endblock %}