An OPML upload was read fully into memory, parsed into a whole tree,
walked recursively, and (for large imports) embedded in a Celery
message. Nothing bounded raw size, ZIP entries, nesting depth, outline
count or feed count.
Uploads are capped at 5 MB (checked on the reported size before reading)
and ZIPs at 50 entries, with the decompressed member bounded by what it
actually yields rather than only its header. Parsing now streams with
iterparse and aborts the moment a limit is crossed — 24 levels deep,
20,000 outlines, 5,000 feeds — so a hostile file never builds a huge or
deep tree; a 50,000-level nesting bomb is rejected cleanly. Limit errors
are shown to the user. Large imports go to the worker by cache reference
(deleted once read, expires after an hour) instead of as the whole
document inside the broker message.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A feed's channel link, an entry's link and an OPML htmlUrl were stored
as given and rendered in href attributes (bulk writes skip model
validation), so a malicious feed could supply a javascript: or data:
link. CSP is a second layer, not the control.
apps.core.urlutils accepts only absolute http(s) URLs. Fetching stores
channel/entry links through it — relative links now resolve against the
feed's own URL instead of being stored as dead relative paths — OPML
import skips non-http(s) feed URLs and blanks unsafe htmlUrl, and the
add-feed form rejects non-http(s) (URLField also allows ftp). A
safe_http_url template filter guards the two places a feed-supplied URL
becomes a link (the article's "Original" link and the dashboard), so
values already in the database from before this fix are neutralized too.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Folders are gone — organization is tags-only, via the SubscriptionTag
model added last commit. A migration converts every existing Folder into
a same-named Tag applied to its feeds, and every folder-scoped FilterRule
into an equivalent tag-scoped one, before the Folder table is dropped, so
upgrading an existing instance loses nothing (verified by seeding the
pre-removal schema directly and running the migration chain against it).
Unlike folders, a feed can carry more than one tag, so:
- apps.articles.tasks.apply_filter_rules_for_feed and apps.rules.tasks's
backlog re-apply now scope by "does this feed carry tag X" rather than
"does this feed belong to folder X".
- apps.feeds.opml: a feed's tags export as a comma-separated "category"
attribute (the standard OPML way to represent multiple categories)
instead of nesting under one folder outline. Import reads both that
attribute and nested category outlines, turning each enclosing level
into its own tag rather than collapsing to the nearest one.
- The sidebar groups feeds by tag instead of folder, so a feed with two
tags now appears in both groups; untagged feeds fall into "Untagged".
AddFeedForm gained a comma-separated tag_names field for parity with the
old add-time folder picker. EditSubscriptionForm dropped its folder
field entirely — tagging a subscription happens through the tag editor
already on that page.
Verified end to end: the migration chain from empty, the data migration
against a seeded pre-removal schema (folder/subscription/rule all convert
correctly), and the running app's sidebar grouping, OPML export category
attributes, and tag editor.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Subscription editing was a real gap: there was no way to rename a feed's
display title, move it between folders, or create a folder at all outside
OPML import. Added an edit page (apps/feeds/views.py:edit_subscription,
EditSubscriptionForm) reachable from the subscriptions list.
Added feed-level tagging via a new SubscriptionTag model (apps/articles/
models.py), mirroring the existing ArticleTag — tag a whole feed on its
edit page, and every one of its articles surfaces under that tag in the
stream alongside individually-tagged articles (apps.reader.views._apply_
scope now matches either). Renamed AddTagToArticleForm to AddTagForm
since it's shared by both now.
The stream also now remembers the last filter (unread/all/starred/saved)
in the session instead of always defaulting to "unread" when a sidebar
feed/folder/tag link omits ?filter= — sidebar links no longer force
?filter=all, letting the remembered choice apply.
Verified end to end against a running dev server: editing a subscription's
title/folder, creating a folder inline, tagging a feed and seeing its
articles appear under that tag, and the remembered-filter behavior across
requests with no ?filter= param.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Implements OPML parsing with defusedxml (untrusted upload, XXE risk)
and export with the stdlib's plain ElementTree (trusted, app-generated
output). Nesting deeper than one level flattens into its nearest
enclosing folder, matching the app's flat-folder model. Import reuses
existing Feed/Subscription rows via get_or_create, so re-importing the
same file is idempotent and cross-user Feed dedup is preserved.
Large imports (>50 entries) dispatch to a Celery task instead of
blocking the request; newly imported feeds aren't fetched immediately
since they have no last_fetched_at, so the next poll_due_feeds beat
cycle picks them up naturally.
Adds a Subscriptions page (replacing the standalone "Add feed" link in
nav) listing current subscriptions with unsubscribe, plus Import/
Export OPML actions.
Verified live: imported a 2-feed/1-folder OPML file against the
running dev server (via an authenticated session, since this browser
tool has no file-picker automation), confirmed the resulting
subscriptions and folder in the UI, exported and inspected the
resulting OPML structure, and unsubscribed from one feed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Implements Folder/Feed/Subscription (feeds app) and Article/
UserArticleState/Tag/ArticleTag (articles app), preserving the
shared-Feed/shared-Article dedup design: a feed is fetched once
regardless of subscriber count, and per-user read/star/save/hide state
is created lazily rather than eagerly per subscriber.
Adds the fetch pipeline: feedparser-based parsing with conditional GET
(ETag/Last-Modified), nh3 HTML sanitization, guid-based dedup with
content-hash change detection that never resets a user's read/star
state, and exponential backoff on fetch errors respecting a configured
minimum interval floor. Wired into Celery via poll_due_feeds/fetch_feed
tasks, plus a minimal "add feed by URL" view that fetches synchronously
so new subscribers see results immediately.
Verified live against a real feed (hnrss.org) for ingestion, 304
dedup, and error backoff, in addition to the unit/integration test
suite (28 tests, all passing).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>