Commit Graph
62 Commits
Author SHA1 Message Date
ksmithandClaude Sonnet 5.5 8d2900424a Use keyset pagination for the stream (C4)
The stream did a COUNT(*) and OFFSET slice on every request, ordered by
a non-unique (published_at, fetched_at) — deep pages got slower and
articles sharing a timestamp could repeat or vanish across pages.

It now orders by (Coalesce(published_at, fetched_at), id) and pages with
before/after cursors, fetching one extra row to know whether another
page exists instead of counting. Pagination links also keep the feed
scope, which the old links silently dropped.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 08:51:20 -07:00
ksmithandClaude Sonnet 5.5 70b0d4f277 Index Postgres search and harden SQLite search/concurrency (C5, C6)
Postgres recomputed a tsvector over every article's full text on every
search. A migration now adds a stored, GIN-indexed search_vector column
kept current by a trigger (so no ingest/update path can forget it), and
the backend reads that column. Untested here: no Postgres in the dev
sandbox, so the SQL is reviewed but not executed — worth one run of the
(skipped-on-SQLite) Postgres test module against a real instance.

SQLite search built one bound parameter per subscribed feed, so enough
feeds would exceed the variable limit, and the OperationalError was
swallowed into "no results". It now scopes through a subquery on
Subscription and logs failures. SQLite connections also get WAL mode,
a 20s busy timeout, and IMMEDIATE transactions so gunicorn, Celery, and
the FTS triggers stop tripping over each other.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 08:51:20 -07:00
ksmithandClaude Sonnet 5 cb0ce5a336 Batch rule/tag lookups and index FilterRule for the ingest query (C9)
FilterRule had no indexes beyond the default primary/foreign-key ones,
and apply_filter_rules_for_feed issued 2 queries per subscriber (its
enabled rules, its subscription tags) inside a loop over every
subscriber of the feed — both cost grew with subscriber/rule count on
every single feed poll.

Adds a composite index on (user, is_enabled, priority), matching the
exact filter+order shape every caller already uses (this task,
apps.rules.tasks' backlog re-apply). apply_filter_rules_for_feed now
fetches rules and subscription tags once for the whole feed's subscriber
set, grouped by user/subscription in Python, instead of once per
subscriber.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-25
2026-10-31 16:09:36 -06:00
ksmithandClaude Sonnet 5 abf4cef55b Cache sidebar unread counts for a short TTL (C3)
apps.core.context_processors.sidebar runs on every authenticated page
render and called unread_counts_by_feed, a join/group-by over the
user's whole article corpus, every single time — cost grew with the
archive's size regardless of which page was being viewed.

unread_counts_by_feed now caches its {feed_id: count} result per user
for 30 seconds (apps.core.cache_utils). The reader views that change
the current user's own read state (toggle_read, mark_all_read, viewing
an article) invalidate it immediately so the sidebar never visibly lags
behind the action that just caused it; other paths (rule-driven state
changes during background ingestion) aren't explicitly invalidated and
simply self-heal within the TTL, which is an acceptable tradeoff
already made elsewhere in this app (the stream's own "new articles"
banner polls on the same 30s cadence).

Also adds a global cache.clear() autouse fixture (conftest.py) — the
test CACHES backend (LocMemCache) lives in process memory and survives
Django's per-test DB rollback, so a cached count keyed by a user's pk
could otherwise leak into an unrelated later test reusing that pk.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 16:09:28 -06:00
ksmithandClaude Sonnet 5 b33fe905d2 Push due-feed scheduling into an indexed DB query (C1)
poll_due_feeds loaded every active feed's timing fields and recomputed
each one's error-backoff due-ness in Python on every beat tick (every
1-2 minutes) — scheduler CPU and DB reads grew linearly with total feed
count regardless of how many were actually due.

Feed.next_fetch_at (indexed together with is_active) is now set by
fetch_feed itself right after every fetch attempt, success or error,
using the same backoff formula that used to run per-feed in
poll_due_feeds. The scheduler's query is now a direct indexed range
scan (next_fetch_at <= now), ordered so the most overdue feeds claim
first, and bounded to POLL_BATCH_SIZE so a large backlog spreads across
several ticks instead of dispatching everything at once. A migration
backfills next_fetch_at for existing feeds using the same formula.
reset_feed_backoff also clears next_fetch_at so a manually-reset feed
is immediately due again, matching its prior behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 16:09:18 -06:00
ksmithandClaude Sonnet 5 b84cfe6474 Document the Nginx Proxy Manager fix for S8's X-Forwarded-For spoofing
docs/install.md covered the raw-nginx case via nginx.conf.example, but
gave no concrete guidance for Nginx Proxy Manager specifically, whose
generated config also defaults to the appending (spoofable) form of
X-Forwarded-For — the same production deployment this review is based
on uses NPM. Documents the Advanced tab / Custom Nginx Configuration
override, why it takes effect ahead of NPM's own default header, and a
note for anyone with something in front of NPM itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 15:35:07 -06:00
ksmithandClaude Sonnet 5 ca03a5994b Stop nginx from forwarding a client-spoofable X-Forwarded-For (S8)
The example nginx config set X-Forwarded-For to $proxy_add_x_forwarded_for,
which APPENDS to whatever X-Forwarded-For a client already sent rather
than replacing it. A client could send its own "X-Forwarded-For:
1.2.3.4" and have it forwarded as the first (trusted) entry — exactly
what apps.accounts.views._client_address reads to key the login rate
limiter, letting an attacker spoof a different address on every request
to dodge the per-address throttle. Changed to $remote_addr, nginx's own
view of who actually connected, which overwrites rather than appends.

docs/install.md's "different host" proxy guidance also gets a concrete
ufw example instead of just "firewall that port" prose, explains why
that firewall rule isn't optional (gunicorn trusts whoever can reach it
directly as if they were the proxy), and documents an optional Unix
socket setup for the same-host case for anyone who wants gunicorn
unreachable by anything but the proxy, not just this nginx config.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-24
2026-10-31 15:27:36 -06:00
ksmithandClaude Sonnet 5 cb5617edb8 Pin dependencies by hash and refuse to follow a rewritten release tag (S7)
pyproject.toml used broad version ranges with no lock file, so two
deployments of the same release could end up with different dependency
versions, and there was no check that a package actually matches what
was reviewed/tested — a compromised or substituted package on the index
could install silently.

requirements.txt and requirements-dev.txt (generated via `pip-compile
--generate-hashes`, see the new README section on regenerating them)
pin every dependency to an exact version with its hash. install.sh and
upgrade.sh now install from requirements.txt with --require-hashes,
then the app itself with --no-deps — a tampered or substituted package
fails the hash check instead of installing.

Both scripts also no longer `git fetch --tags --force`: a plain fetch
refuses to move a tag ref they already have onto a different commit,
so a rewritten/tampered release tag is reported and stops the
deploy/install instead of being silently followed into production.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 15:27:29 -06:00
ksmithandClaude Sonnet 5 03dd37e69d Notify the open stream when new articles arrive
The stream had no way to learn about articles the background poll
ingested while the page was open — only a manual reload or navigation
ever picked them up.

Adds a lightweight "N new articles — click to show" banner: the stream
records the highest article id in its current scope (filter/tag/feed)
when rendered, and an htmx poll every 30s checks reader:check_new for
anything newer in that same scope. It never swaps the list out from
under you mid-read — clicking the banner is what actually reloads it.
id, not published_at, is the cursor, since a feed can backdate an
entry's published date below everything currently on screen while it's
still genuinely new to this stream.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-23
2026-10-31 15:04:55 -06:00
ksmithandClaude Sonnet 5 17e7f63e61 Update link/author/summary/published_at on refetch, not just title (T6)
A changed guid's article only ever had its title/content_html/
content_hash refreshed on a subsequent poll — legitimate feed edits to
an entry's link, author, summary, or publication date were silently
ignored forever, which can affect navigation, ordering, and rules that
key off those fields.

content_hash is now computed over every mutable field feedparser gives
us for an entry (title, link, author, summary, content_html,
published_at), and all of them are written together when the hash
changes. Per-user read/star state is untouched either way, same as
before.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-22
2026-10-31 14:44:15 -06:00
ksmithandClaude Sonnet 5 0bfe9c8076 Enqueue the initial feed fetch instead of running it inline (T3)
add_feed ran fetch_and_ingest synchronously during the POST request, so
a slow or large feed tied up a Gunicorn worker for the duration of the
fetch (up to the 15s HTTP timeout, plus parse/ingest time) and a user
could repeat it with distinct URLs to reduce request capacity.

The subscription is now created synchronously as before, but the first
fetch is dispatched via fetch_feed.delay() — the same task the periodic
poll uses, so the save/error bookkeeping is unchanged. The user sees an
immediate redirect with a message that the fetch is in progress rather
than waiting on it in the request/response cycle.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 14:44:09 -06:00
ksmithandClaude Sonnet 5 7bb981b545 Rebuild stream date-group headers in the viewer's local timezone
The server still groups articles into "Received today/yesterday/<date>"
headers using UTC, so an article published near UTC midnight could land
in a mismatched day group. Each article row now ships its raw UTC
instant (data-published-utc); static/js/date-groups.js discards the
server-rendered headers on load and rebuilds them using the browser's
own local calendar day, same approach as the article-time localization
this follows up on.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-21
2026-10-31 14:36:27 -06:00
ksmithandClaude Sonnet 5 fd68400822 Render article/feed times in the viewer's local timezone, client-side
Everything is stored and server-rendered in UTC (config.settings.base's
TIME_ZONE), so article times, the article detail header, and the
dashboard's "last fetched" column were always showing UTC regardless of
where the viewer actually is.

Article/feed timestamps now render as <time datetime="..."> elements
carrying the UTC instant plus a UTC-formatted fallback as their initial
text; static/js/localize-times.js reformats them using the browser's
own timezone via toLocaleString(), both on page load and after every
htmx swap (pagination, the inline article expand, the dashboard's retry
action all inject new <time> elements without a full page load).

No server-side change needed — this doesn't touch the (currently
unused) per-user timezone field, and works correctly even if the
viewer travels or changes their system timezone, with no settings page
required. Verified live in the browser against a UTC-stored timestamp
rendering correctly shifted to the sandbox's local zone.

Known related issue, out of scope here: the stream's "Received
today/yesterday" date-group headers (apps.reader.views._group_by_date)
are still computed server-side via timezone.localdate(), which is also
effectively UTC — an article just before/after UTC midnight could land
in a date group that doesn't match its now-correctly-localized
time-of-day. Flagged for the user rather than fixed, since correcting
it properly would mean moving date-grouping to the client too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-20
2026-10-31 13:31:07 -06:00
ksmithandClaude Sonnet 5 90837b339c Make the Site settings admin page go straight to the edit form
It's a true singleton, not a collection, so the normal changelist (an
"Add" link, a list with 0 or 1 rows, an extra click to get to the one
row) was just friction — and visually inconsistent with every other
app's admin section, which do show an "Add" link. The changelist view
now redirects straight to the single row's change form (creating it
first if this is a fresh install), and "Add" is never offered at all.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-19
2026-10-31 13:23:47 -06:00
ksmithandClaude Sonnet 5 dc8b9ff83b Make the feed-poll interval site-wide instead of per-feed
Replaces Feed.fetch_interval_minutes with a single admin-editable
SiteSettings.fetch_interval_minutes (apps.core), following the same
singleton-row pattern as accounts.SetupLock. Per-feed error backoff is
unchanged — only the configured baseline interval is now shared across
every feed, loaded once per poll_due_feeds run rather than queried per
feed. EVECTION_MIN_FETCH_INTERVAL_MINUTES keeps working as the floor
underneath it.

SiteSettings is registered in Django admin as a singleton: "Add" is
only offered when no row exists yet, and delete is always refused.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-18
2026-10-31 12:59:47 -06:00
ksmithandClaude Sonnet 5 a1ea2e5662 Add login rate limiting (S6)
Django's default LoginView had no throttle — an internet-facing
instance was exposed to unlimited password-guessing/credential-
stuffing attempts. apps.accounts.views.LoginView adds two independent,
Redis-cache-backed throttles (shared across all gunicorn workers,
unlike the per-process default LocMemCache): by client address (catches
one attacker spraying many usernames) and by the submitted username
(catches a distributed attack on one specific account). Either blocks
further attempts for a 5-minute window; both are generous enough not
to lock out a user who's just mistyped their own password a few times.

Client address prefers X-Forwarded-For's first hop, falling back to
REMOTE_ADDR — the same reverse-proxy trust boundary config.settings.prod
already relies on for X-Forwarded-Proto (gunicorn is never meant to be
reachable except through that proxy).

CACHES now points at Redis (already a hard install requirement for
Celery) instead of Django's implicit per-process default, since the
whole point is a shared counter across every gunicorn worker; test
settings override it back to LocMemCache so the suite doesn't need a
real Redis instance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-17
2026-10-31 12:46:49 -06:00
ksmithandClaude Sonnet 5 9241a54958 Cap entries per poll and stored content size (T4)
ssrf_guard.MAX_RESPONSE_BYTES (S1) already bounds the raw download, but
a feed within that byte cap can still carry an unreasonable number of
entries, or one entry with an unreasonably large body — both cost real
CPU/DB time (nh3 sanitization, an Article row/UPDATE) per entry.
fetch_and_ingest now processes at most MAX_ENTRIES_PER_POLL (200)
entries per poll (logging a warning when truncated) and sanitize_html
bounds both its input (sanitizer cost) and output (what's actually
stored/rendered) length.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:46:40 -06:00
ksmithandClaude Sonnet 5 64f25789f5 Isolate unexpected fetch failures and harden task durability (T2)
fetch_feed had no time limit and no handling for anything beyond the
network/parse errors fetch_and_ingest already catches internally — an
unexpected exception (a parser/sanitizer bug, a DB error) crashed the
task with no error recorded on the Feed, the claim left held for the
full CLAIM_STALE_MINUTES, and no visibility beyond a bare traceback.
fetch_feed now catches that, records it the same way a handled failure
would be (reusing fetcher.record_fetch_error, promoted from a private
helper), clears the claim, and re-raises so Celery still marks the task
FAILED. soft_time_limit/time_limit bound a single fetch that somehow
outruns FETCH_TIMEOUT_SECONDS + ssrf_guard's response cap during
parsing/ingestion.

Also: a worker process killed mid-task (OOM, deploy restart, crash)
previously lost that task silently — Celery's default is to ack a
message the moment a worker *starts* it. CELERY_TASK_ACKS_LATE +
CELERY_TASK_REJECT_ON_WORKER_LOST (config/settings/base.py) make the
message redeliverable instead, and CELERY_WORKER_PREFETCH_MULTIPLIER=1
keeps a worker from grabbing a batch of tasks it could lose all at once.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:46:29 -06:00
ksmithandClaude Sonnet 5 0466ccc334 Fix sidebar disclosure triangle rendering above the tag name
summary's native ::marker triangle is positioned for inline content —
once the tag-name + unread-count row needed to be a flex box (for
v2026.10.31-15's tag unread counts), the marker got pushed onto its
own line above instead of sitting beside it, confirmed via a
screenshot after deploying that release.

Replaces the native marker with a plain ::before triangle (hiding
-webkit-details-marker too) so `summary` itself can safely be the flex
row, with the triangle as a normal flex item that stays inline no
matter what the rest of the row contains. Verified in the browser,
including that collapse/expand still works and the triangle still
rotates on open.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-16
2026-10-31 12:39:55 -06:00
ksmithandClaude Sonnet 5 e760eeda1d Show each tag's total unread count and remember collapsed sections
Closes gitea#24: the sidebar's per-feed unread counts were already
computed by sidebar() (apps.core.context_processors); each tag group's
total is now the sum of its own feeds' counts (a feed can carry more
than one tag, so this reuses the same per-feed data rather than a
second query) and renders next to the tag name like a feed's own
count.

Closes gitea#25: each sidebar <details> section (one per tag, plus
"Untagged") now carries a stable data-sidebar-key, and
static/js/sidebar-state.js remembers which ones are collapsed in
localStorage — this isn't a SPA, so every click re-renders the sidebar
server-side, and previously every section reset back to expanded on
the next page load. Per-browser, same pattern as theme.js/theme-init.js,
not synced across devices/accounts.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-15
2026-10-31 12:34:57 -06:00
ksmithandClaude Sonnet 5 9439f9cc59 Add SSRF protection to feed fetching: validate scheme/address, bound redirects and response size (S1)
fetch_and_ingest fetched a feed_url supplied straight from the "add
feed" form or an uploaded OPML file, with requests following redirects
on its own and no check that the target (or a redirect's target) was
actually a public, fetchable feed rather than an internal service,
localhost, or a cloud metadata endpoint — reachable from wherever this
app runs, on any authenticated user's behalf.

apps.feeds.services.ssrf_guard.safe_get() replaces the direct
requests.get() call: it resolves and validates the hostname (rejecting
anything that isn't a globally routable address) before the initial
request and before following each redirect hop manually, caps the
number of hops, and bounds the response body size by streaming it
instead of buffering it unbounded. OPML-imported feeds go through the
same check the first time poll_due_feeds fetches them, since this is
the one chokepoint every feed URL passes through regardless of which
path created the Feed row.

Known residual gap, documented in ssrf_guard.py: this validates then
connects rather than pinning the TCP connection to the exact address it
validated, so a narrow DNS-rebinding race isn't closed. Fixing that
needs a custom transport and felt like more machinery than this
self-hosted app's threat model currently justifies.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-14
2026-10-31 12:25:20 -06:00
ksmithandClaude Sonnet 5 fd8eeb605e Scope backlog re-apply per-feed and always queue it through Celery (T5, S5)
Completes the T5 fix: reapply_rules_for_user_backlog now computes each
feed's subscription-tag ids up front and passes them to
evaluate_rules_for_article per article, so a feed/tag-scoped rule from
a different feed can't fire on this backlog (see the previous commit's
rule_in_scope() in apps.rules.engine).

Also completes S5's second half: the "Re-apply" view now always
dispatches reapply_rules_for_user.delay() instead of calling the
backlog function inline from the request — a large backlog or a
regex that outruns the new timeout guard no longer risks tying up a
web worker.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:20:25 -06:00
ksmithandClaude Sonnet 5 91c0b1dd26 Centralize rule feed/tag scoping in the engine and fix backlog re-apply (T5)
reapply_rules_for_user_backlog narrowed its *backlog* to the triggering
rule's feed/tag scope, but then loaded every enabled rule for the user
and ran all of them against that backlog with no per-rule scope check
— so re-applying one feed- or tag-scoped rule could also fire an
unrelated rule scoped to a different feed/tag, if it happened to match.

rule_in_scope() moves the feed/tag scope check into the engine itself
(apply_filter_rules_for_feed already computed this per-subscription via
query filters; it now just passes the tag ids and lets the engine
decide), so backlog re-apply and ingest-time dispatch can't drift into
scoping rules differently again.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:19:56 -06:00
ksmithandClaude Sonnet 5 f2ee793cf8 upgrade.sh: auto-bump EVECTION_PWA_CACHE_VERSION on every deploy
The release-notes reminder to manually bump this and restart gunicorn
was easy to miss or forget, leaving installed PWAs on a stale cached
shell/pages after an upgrade that changed static assets. upgrade.sh
now sets it to the deployed tag itself before restarting services, so
there's no manual step and no cache-invalidation window to miss.
install.sh does the same on first install for consistency.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-13
2026-10-31 12:08:05 -06:00
ksmithandClaude Sonnet 5 b7d9ee672c Add an atomic per-feed claim to poll_due_feeds (T1)
poll_due_feeds scanned for due feeds and dispatched fetch_feed without
marking them claimed, so overlapping beat ticks or a slow-running
fetch could dispatch the same feed twice, racing on ETag/status/error-
count writes. Feed.claimed_at is now set via an UPDATE ... WHERE
claimed_at IS NULL OR claimed_at < stale-cutoff, so only one caller's
UPDATE can ever match a given row; fetch_feed clears it when done.
A claim older than CLAIM_STALE_MINUTES is treated as abandoned so a
worker dying mid-fetch doesn't wedge the feed forever.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-12
2026-10-31 12:03:18 -06:00
ksmithandClaude Sonnet 5 d7b433e580 Purge the service worker's authenticated-page cache on logout (S3)
PAGES_CACHE held every cached navigation response (including
authenticated article pages) with no logout-time purge, so a later
offline navigation on a shared or account-switched device could still
serve another user's cached pages. Logout is a plain POST, so the
fetch handler now special-cases that request: it deletes PAGES_CACHE
and lets the real logout request through, with no race against the
navigation that follows since it's synchronous with the same request.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:03:10 -06:00
ksmithandClaude Sonnet 5 b52ba2be8a Make first-run setup race-safe with an atomic SetupLock claim (S4)
The setup view's User.objects.exists() check and the admin account's
save() weren't in the same transaction, so two concurrent requests
during first deployment could both pass the check and both create an
admin account. SetupLock is a singleton row (pk=1); claiming it and
creating the admin now happen in one atomic block, so only the first
of any racing requests can succeed — the rest hit IntegrityError and
are sent to the login page instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:03:05 -06:00
ksmithandClaude Sonnet 5 56eb2bc498 Fail closed in production if SECRET_KEY is missing or the placeholder default (S2)
base.py's env.str() fallback is a publicly known string in this repo's
history. Running prod settings with it (an unset or forgotten .env
value) would sign sessions/CSRF tokens with a key anyone can read.
install.sh already generates a real random key for fresh installs, so
this only blocks the broken case.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 12:02:58 -06:00
ksmithandClaude Sonnet 5 9d91da6fc6 Fix /health/ 400ing when hit directly over loopback (as upgrade.sh does)
scripts/upgrade.sh's health check connects straight to gunicorn's bind
address, bypassing the reverse proxy entirely, so the request's Host
header is "127.0.0.1:<port>" and it has no X-Forwarded-Proto header.
Against prod settings that's exactly a DisallowedHost (ALLOWED_HOSTS only
has the public domain) — which Django turns into a 400 — and would have
then hit a redirect-to-HTTPS loop from SECURE_SSL_REDIRECT right after,
since there's no TLS listener on that port to redirect to. Allow the
loopback addresses in ALLOWED_HOSTS (harmless: gunicorn's bind address is
never reachable from outside this host per docs/install.md) and exempt
/health/ from the HTTPS redirect specifically.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-11
2026-10-31 09:19:27 -06:00
ksmithandClaude Sonnet 5 c316f411f8 Make the active-tag sidebar highlight as visible as the active-feed one
.sidebar-tag-link.active only changed text color, which is easy to miss
next to .sidebar-feed-link.active's background pill + bold weight. Give
the tag link the same background-pill treatment (padding/margin offset
so the box doesn't shift layout when inactive).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-10
2026-10-31 09:17:45 -06:00
ksmithandClaude Sonnet 5 e1907ff587 install.sh: fail hard (or auto-install) when Redis isn't actually reachable
The old check only looked for redis-cli on PATH and printed a warning if
missing — it never tested an actual connection, so a host with no Redis
installed at all still sailed through the entire install looking
successful. The real failure only showed up later as every feed stuck on
"pending" forever, with the worker/beat units quietly retrying their
connection in the background rather than surfacing anything to whoever
ran the script. Actually connect to the REDIS_URL that will end up in
.env; on Debian/Ubuntu with the default local address, install and start
redis-server automatically; otherwise fail the install outright with a
clear message instead of a warning that's easy to miss.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-9
2026-10-31 09:12:41 -06:00
ksmithandClaude Sonnet 5 b0d62c91f8 upgrade.sh: use the venv's Python for the health check, not curl
curl isn't installed by default on this install's host, and nothing else
in install.sh/upgrade.sh actually requires it — the venv's own Python
(which obviously exists by the time this check runs) can make the same
HTTP request via urllib, so drop the curl dependency entirely rather than
adding it to the prerequisites.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-8
2026-10-31 09:08:48 -06:00
ksmithandClaude Sonnet 5 2068df471f install.sh: actually restart services when their unit files change
systemctl enable --now only starts a service if it's stopped — on a
re-run where a unit file's ExecStart changed (e.g. picking up a new
EVECTION_GUNICORN_BIND), the already-running process just kept running
on its old command line indefinitely. This is exactly what happened
switching gunicorn from a Unix socket to TCP: the new unit file was
written and daemon-reload'd, but gunicorn itself never restarted, so it
kept listening on the old socket. Restart explicitly instead of relying
on --now.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-7
2026-10-31 09:05:46 -06:00
ksmithandClaude Sonnet 5 21c33e8188 Make the reverse proxy a required, pluggable part of every deployment
gunicorn previously only ever bound to a Unix socket, which assumes the
reverse proxy is nginx running on this same host — it can't be reached by
a proxy on a different machine at all (e.g. Nginx Proxy Manager on its own
box), which is this project's actual deployment target. Switch gunicorn to
listening on a plain TCP address instead, controlled by a new
EVECTION_GUNICORN_BIND setting in .env.

install.sh now asks whether the proxy is local or remote and picks a
sensible bind default either way (127.0.0.1:8000 vs 0.0.0.0:8000, with a
reminder to firewall the port in the remote case), and no longer tries to
automate nginx/certbot itself — docs/install.md's new "Reverse proxy"
section covers both shapes instead, since which proxy and where it runs is
the operator's choice, not something this script should assume or set up.
upgrade.sh's health check now curls the TCP bind instead of the old socket
path.

Also add WhiteNoise so gunicorn serves /static/ itself: the old approach
(nginx aliasing the staticfiles directory on disk) silently breaks the
moment the proxy isn't on this host and can't see this filesystem.
nginx.conf.example's own /static/ alias becomes an optional same-host perf
optimization rather than a requirement.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-6
2026-10-31 09:01:46 -06:00
ksmithandClaude Sonnet 5 62702e13e3 Detect and recreate a half-created venv, not just a missing one
The idempotency check only looked for the venv directory's existence, so
a venv left behind by an earlier failed run (e.g. missing ensurepip, as
just hit) — present but with no bin/pip — made every later re-run skip
venv creation entirely and then fail on a pip that was never created.
Check for bin/pip specifically and wipe+recreate if it's missing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-5
2026-10-31 08:52:36 -06:00
ksmithandClaude Sonnet 5 78611c0eae Check for a working venv module before install.sh does anything else
On Debian/Ubuntu, python3-venv (ensurepip support) is a separate package
from python3 itself, and apt installing just python3 leaves venv creation
failing partway through install.sh — after the system user, chown, and
git checkout have all already happened. Check for it up front alongside
the Python version check, and document the apt package in install.md's
prerequisites.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-4
2026-10-31 08:51:28 -06:00
ksmithandClaude Sonnet 5 50d2ba9404 Mark the app checkout as a safe.directory for root's git commands
install.sh chowns $APP_DIR to the service user, but its own (and
upgrade.sh's) git fetch/checkout steps run as root per the previous fix —
git refuses that combination by default ("detected dubious ownership"),
which is exactly what happens re-running install.sh after an earlier run
got far enough to chown the checkout. Add the directory to root's
safe.directory list before touching it with git.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-3
2026-10-31 08:49:37 -06:00
ksmithandClaude Sonnet 5 8c23cc631c Fix install.sh failing with permission denied creating the venv
EVECTION_HOME (the venv's parent directory, e.g. /opt/evection) was never
chowned to the service user — only the app checkout inside it was — so
venv creation failed with Permission denied whenever EVECTION_HOME was
created by root before running the script (the documented quick-install
steps do exactly that: mkdir + git clone as root, then run install.sh).

Also run the release-tag git fetch/checkout as root instead of
sudo -u $EVECTION_USER in both install.sh and upgrade.sh: the service
user has no reason to have its own git credentials configured, so running
as it just means the private repo's auth prompt hangs the script (as seen
on this install) or silently fails. Root already has working credentials
from the initial clone. Added GIT_TERMINAL_PROMPT=0 so a credential
problem fails fast instead of hanging on a prompt either way.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31-2
2026-10-31 08:39:09 -06:00
ksmithandClaude Sonnet 5 3e627b1ef8 Document the tag-based release/deploy flow in docs/install.md
install.sh and upgrade.sh deploy git tags, not branch tips, but the docs
still described git pull/clone-and-run. Add a "Releases" section explaining
the v<year>.<month>.<day> tagging convention and that a tag must exist
before the first production install, update the manual upgrade steps to
match what upgrade.sh actually does (fetch tags, checkout latest), and
mention the /health/ endpoint upgrade.sh verifies against.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v2026.10.31
2026-10-31 08:29:21 -06:00
ksmithandClaude Sonnet 5 3ba46f9d49 Match upgrade.sh's example tag to the actual v<year>.<month>.<day> format
The "no tags found" error suggested git tag v1.0.0, which doesn't match
how this project's tags are actually named — generates today's date into
the example instead. No behavior change: git tag --sort=-creatordate
already sorts by actual tag creation time regardless of the tag name's
format, so the date-based scheme worked with the existing logic already.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 01:52:09 -06:00
ksmithandClaude Sonnet 5 2e4674a18f Make the updater deploy tagged releases, like ds-chat's upgrade.sh
scripts/upgrade.sh now fetches tags and does a detached-HEAD checkout of
the latest one (git tag --sort=-creatordate | head -n1), matching the
ds-chat project's deploy/upgrade.sh exactly rather than a plain git pull
on whatever branch happened to be checked out. Errors clearly if the repo
has no tags yet (true for Evection right now — nothing to deploy until a
release is tagged and pushed), and no-ops cleanly if already on the
latest tag. scripts/install.sh does the same tag pin on first install
(falling back to whatever's checked out if no tags exist yet), so the
very first deploy is consistent with every upgrade after it.

Added a GET /health/ endpoint (apps.core.views.health) that checks DB
connectivity and is exempt from the first-run setup redirect, for the
upgrade script's post-restart verification (curled over the gunicorn
unix socket, same as ds-chat's curl-based check) and for any external
uptime monitor.

While wiring the setup-wizard exemption for /health/, found that
SetupWizardMiddleware._a_user_exists's reset fixture was scoped only to
apps/accounts/tests.py, so a test anywhere else that creates a user and
makes a request could leak a stale "a user exists" into later test files
with no reset — confirmed this concretely by temporarily breaking the new
exemption and watching the "works before setup" test still pass purely
from accounts/tests.py leaving the module-order-adjacent apps/articles
tests to poison the shared class attribute. Moved the fixture to a new
top-level conftest.py so it's autouse across every test file, which also
makes the new health-check test meaningful regardless of run order.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 22:15:37 -06:00
ksmithandClaude Sonnet 5 a89f651a71 Add install.sh / upgrade.sh to automate the documented deployment steps
Keeps the traditional pip+systemd+nginx deployment (no Docker, per the
original decision) but automates the manual steps already in
docs/install.md instead of leaving them as copy-paste instructions:

- scripts/install.sh: creates the system user, virtualenv, generates
  .env (auto SECRET_KEY, prompts for domain/SQLite-vs-Postgres —
  non-interactive via EVECTION_ALLOWED_HOSTS/EVECTION_DATABASE_URL/etc
  for scripted provisioning), migrates, collects static files, installs
  the systemd units (sed-substituted for the actual install path/user
  rather than hardcoded to /opt/evection), and optionally sets up nginx +
  certbot. Safe to re-run — skips what's already done rather than
  redoing it, and never overwrites an existing .env.
- scripts/upgrade.sh: git pull, reinstall, migrate, collectstatic,
  restart whichever of the three services are actually present (handles
  the combined worker+beat deployment variant too).

docs/install.md now leads with the script as the recommended path, with
the existing step-by-step instructions kept below for anyone who wants
to understand or customize what it's doing.

Verified: shellcheck clean on both scripts (via a throwaway shellcheck-py
venv, since neither apt nor pip --user were available non-interactively);
the systemd-unit and nginx-config sed substitutions produce correct
output against the real template files; the generated .env format
round-trips through `manage.py check` under config.settings.prod without
error. Did not actually run install.sh against a real system — that
creates a system user and installs systemd units, which needs the
person deploying it to run (and review) themselves.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 22:11:18 -06:00
ksmithandClaude Sonnet 5 20dd61e938 Add the logo to login/setup pages and raise light-mode contrast
The brand logo only appeared in the sidebar, so unauthenticated pages
(login, first-run setup, create-user) had no branding at all. Added an
.auth-brand block to base.html's shared unauthenticated shell so all
three get it automatically, and generalized the theme-swapped logo
show/hide rules (previously scoped to .sidebar-brand) so the same
.brand-wordmark-dark/.brand-wordmark-light classes work in both places —
safe to de-scope since neither .sidebar-brand img nor .auth-brand img
touches `display`, so there's no repeat of the earlier specificity bug.

Raised light-mode contrast across the board (WCAG contrast vs white,
computed): --text-dim 3.64:1 -> 5.80:1 (was failing AA for normal text —
this is the timestamp/feed-name color, used constantly), --star 3.65:1 ->
5.0:1, --border 1.25:1 -> 1.58:1 (still a hairline by design, just more
visible), and widened the --bg/--bg-sidebar/--bg-row-hover/--bg-row-
selected tiers, which were previously all within a few points of white
and made the whole theme look flat.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 22:03:13 -06:00
ksmithandClaude Sonnet 5 1a8ae95db0 Fix logout 405 — Django's LogoutView only accepts POST
The sidebar's "Log out" was a plain <a href> GET link. Django 5's
LogoutView stopped accepting GET (CSRF hardening — a bare link or even an
<img> tag could otherwise log someone out), so clicking it 405'd instead
of logging out. Changed it to a POST form, matching every other
state-changing action in the sidebar/app (unsubscribe, delete tag, etc.).

Extended .sidebar-top-nav's styling to cover a <button> the same as an
<a>, so the form-submit button looks identical to the other nav items.

No existing test exercised logout at all — added one for the POST case
and one confirming GET now correctly 405s, then verified end-to-end
against the running dev server (login, POST /logout/, confirm the session
is gone).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 22:01:06 -06:00
ksmithandClaude Sonnet 5 b08a78e747 Fix double-logo bug and size up the sidebar brand image
.sidebar-brand img { display: block } (0,1,1 specificity) was beating
.brand-wordmark-light { display: none } (0,1,0), so both the light and
dark logo variants rendered stacked regardless of theme. Scoped the
show/hide rules to .sidebar-brand .brand-wordmark-{dark,light} (0,2,0) so
they correctly win, and moved display:block onto the dark variant's own
rule instead of the generic img selector.

Also sized the logo up 50% (64px -> 96px) per feedback, and regenerated
the source PNGs at 400px tall (previously 160px) so they stay crisp at
the larger size instead of upscaling a smaller source.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 21:59:29 -06:00
ksmithandClaude Sonnet 5 4af7860acd Apply the real brand logo and add a light/dark theme toggle
Replaces the generic placeholder PWA icons and plain "Evection" text in
the sidebar with the actual logo kit (static/img/brand-full-{dark,light}
.png — the icon+wordmark lockup, swapped by theme via CSS). The PWA icon
set (icon-192, icon-512, maskable, apple-touch-icon) is regenerated from
the icon artwork: cropped to its content bounding box, padded to a square,
and flattened onto a solid brand-navy (#0f2038) background — maskable
specifically uses a tighter ~62% safe-zone scale so OS masking never clips
the art. manifest.webmanifest and the theme-color meta now use that same
navy instead of the old placeholder #111111.

Refined the light/dark CSS palette (static/css/base.css) using colors
sampled directly from the logo: brand navy for light-mode text, a blue
pulled from the orbit ring for --accent, amber for --star (already close
to brand, left mostly as-is). Dark mode shifted from a neutral charcoal
background toward the same navy family for consistency with the mark.

Added an explicit Light/Dark/System toggle in the sidebar footer, on top
of the existing prefers-color-scheme support rather than replacing it:
static/js/theme-init.js runs undeferred in <head> (before first paint, to
avoid a flash of the wrong theme) and applies any saved localStorage
choice via a data-theme attribute; static/js/theme.js (deferred) wires the
three buttons. CSS resolves theme in this priority: explicit
[data-theme] attribute, else prefers-color-scheme, else dark (the
unconditional :root default). Both scripts are external files/CSP-safe —
verified no inline <script> crept in and the CSP header is still present
under config.settings.prod.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 21:55:35 -06:00
ksmithandClaude Sonnet 5 f818b11041 Accept a FreshRSS "Export" .zip directly for OPML import
FreshRSS's Export feature produces a .zip (an .opml.xml plus a starred-
articles .json this app doesn't import) rather than a bare .opml file —
uploading that zip as-is failed with "That file doesn't look like a valid
OPML export" since defusedxml can't parse a zip as XML.

apps.feeds.opml.extract_opml_bytes detects a zip upload and pulls the
.opml/.opml.xml member out of it before parsing; a bare .opml upload is
returned unchanged, so existing behavior is untouched. Verified against
the reporter's actual FreshRSS export (18 feeds across 6 categories) both
directly through extract_opml_bytes/parse_opml and as a real upload
through the running app.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 21:45:16 -06:00
ksmithandClaude Sonnet 5 753ae9efc7 Make Starred and Saved-for-later pre-configured system tags
UserArticleState.is_starred/is_saved_for_later/starred_at are gone —
starring and saving now add/remove an ArticleTag against a per-user
"Starred" / "Saved for later" Tag, created lazily on first use and marked
Tag.is_system=True. is_read/is_hidden stay on UserArticleState since
they're genuine reading state, not categorization, which is what made
these two candidates for unifying into the general tag mechanism.

System tags are excluded from the Manage Tags page and from delete_tag
(get_object_or_404 now requires is_system=False, so attempting to delete
one 404s) and from the subscription-tag autocomplete, since they
categorize articles, not feeds. The star/save icons are still exactly
where they were — this only changes what powers them.

apps.rules.engine's star action now does get_system_tag + ArticleTag.
get_or_create instead of setting a boolean field. A data migration
converts every existing is_starred=True/is_saved_for_later=True row into
its ArticleTag equivalent before the fields are dropped — verified by
seeding the pre-removal schema directly (including a row with both flags
set) and confirming the migration produces the right tags.

apps.articles.queries.with_user_state gained one more query (tag lookup
for the current page's articles) to stay in the same "fixed number of
queries regardless of article count" shape it already had.

Verified end to end against a running dev server: toggling star/saved,
the starred/saved filters, the sidebar's Favourites count, and that the
Starred tag doesn't appear on the Manage Tags page.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 21:36:44 -06:00
ksmithandClaude Sonnet 5 0e132e9ab9 Remove folders; feeds are organized by tags now
Folders are gone — organization is tags-only, via the SubscriptionTag
model added last commit. A migration converts every existing Folder into
a same-named Tag applied to its feeds, and every folder-scoped FilterRule
into an equivalent tag-scoped one, before the Folder table is dropped, so
upgrading an existing instance loses nothing (verified by seeding the
pre-removal schema directly and running the migration chain against it).

Unlike folders, a feed can carry more than one tag, so:
- apps.articles.tasks.apply_filter_rules_for_feed and apps.rules.tasks's
  backlog re-apply now scope by "does this feed carry tag X" rather than
  "does this feed belong to folder X".
- apps.feeds.opml: a feed's tags export as a comma-separated "category"
  attribute (the standard OPML way to represent multiple categories)
  instead of nesting under one folder outline. Import reads both that
  attribute and nested category outlines, turning each enclosing level
  into its own tag rather than collapsing to the nearest one.
- The sidebar groups feeds by tag instead of folder, so a feed with two
  tags now appears in both groups; untagged feeds fall into "Untagged".

AddFeedForm gained a comma-separated tag_names field for parity with the
old add-time folder picker. EditSubscriptionForm dropped its folder
field entirely — tagging a subscription happens through the tag editor
already on that page.

Verified end to end: the migration chain from empty, the data migration
against a seeded pre-removal schema (folder/subscription/rule all convert
correctly), and the running app's sidebar grouping, OPML export category
attributes, and tag editor.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 21:30:06 -06:00
ksmithandClaude Sonnet 5 3b58fdcb7e Add subscription editing, feed-level tags, and a remembered stream filter
Subscription editing was a real gap: there was no way to rename a feed's
display title, move it between folders, or create a folder at all outside
OPML import. Added an edit page (apps/feeds/views.py:edit_subscription,
EditSubscriptionForm) reachable from the subscriptions list.

Added feed-level tagging via a new SubscriptionTag model (apps/articles/
models.py), mirroring the existing ArticleTag — tag a whole feed on its
edit page, and every one of its articles surfaces under that tag in the
stream alongside individually-tagged articles (apps.reader.views._apply_
scope now matches either). Renamed AddTagToArticleForm to AddTagForm
since it's shared by both now.

The stream also now remembers the last filter (unread/all/starred/saved)
in the session instead of always defaulting to "unread" when a sidebar
feed/folder/tag link omits ?filter= — sidebar links no longer force
?filter=all, letting the remembered choice apply.

Verified end to end against a running dev server: editing a subscription's
title/folder, creating a folder inline, tagging a feed and seeing its
articles appear under that tag, and the remembered-filter behavior across
requests with no ?filter= param.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-30 21:20:37 -06:00