Bound OPML uploads in size, depth, count and broker payload (R2-S5, #37)

An OPML upload was read fully into memory, parsed into a whole tree,
walked recursively, and (for large imports) embedded in a Celery
message. Nothing bounded raw size, ZIP entries, nesting depth, outline
count or feed count.

Uploads are capped at 5 MB (checked on the reported size before reading)
and ZIPs at 50 entries, with the decompressed member bounded by what it
actually yields rather than only its header. Parsing now streams with
iterparse and aborts the moment a limit is crossed — 24 levels deep,
20,000 outlines, 5,000 feeds — so a hostile file never builds a huge or
deep tree; a 50,000-level nesting bomb is rejected cleanly. Limit errors
are shown to the user. Large imports go to the worker by cache reference
(deleted once read, expires after an hour) instead of as the whole
document inside the broker message.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
ksmithandClaude Sonnet 5.5 committed 2026-11-01 11:58:49 -07:00
1 parent 41f99e9405
commit 698940763f
6 files changed
+240 -40

No files matched your search

+19 -3
View File
@@ -1,8 +1,10 @@
import logging
import uuid
from io import BytesIO
from django.contrib import messages
from django.contrib.auth.decorators import login_required
from django.core.cache import cache
from django.http import HttpResponse
from django.shortcuts import get_object_or_404, redirect, render
from django.views.decorators.http import require_POST
@@ -12,12 +14,13 @@ from apps.articles.models import SubscriptionTag, Tag
from .forms import AddFeedForm, EditSubscriptionForm, OPMLImportForm
from .models import Feed, Subscription
from .opml import export_opml, extract_opml_bytes, import_opml, parse_opml
from .opml import MAX_UPLOAD_BYTES, export_opml, extract_opml_bytes, import_opml, parse_opml
from .tasks import enqueue_fetch, import_opml_task
logger = logging.getLogger(__name__)
OPML_ASYNC_THRESHOLD = 50
OPML_UPLOAD_TTL_SECONDS = 60 * 60
@login_required
@@ -149,16 +152,29 @@ def unsubscribe(request, subscription_id):
def import_opml_view(request):
form = OPMLImportForm(request.POST or None, request.FILES or None)
if request.method == "POST" and form.is_valid():
uploaded = form.cleaned_data["opml_file"].read()
uploaded = form.cleaned_data["opml_file"].read(MAX_UPLOAD_BYTES + 1)
try:
content = extract_opml_bytes(uploaded)
entry_count = len(parse_opml(BytesIO(content)))
except ValueError as exc:
# Our own limit/format errors carry a user-presentable message.
messages.error(request, str(exc))
return redirect("feeds:import_opml")
except Exception:
messages.error(request, "That file doesn't look like a valid OPML export.")
return redirect("feeds:import_opml")
if entry_count > OPML_ASYNC_THRESHOLD:
import_opml_task.delay(request.user.id, content.decode("utf-8"))
# Hand the worker a reference, not the document: a multi-MB
# file doesn't belong inside a broker message.
upload_key = f"opml-import:{uuid.uuid4().hex}"
try:
cache.set(upload_key, content, timeout=OPML_UPLOAD_TTL_SECONDS)
import_opml_task.delay(request.user.id, upload_key)
except Exception:
logger.exception("Could not queue the OPML import")
messages.error(request, "Couldn't queue that import right now — try again shortly.")
return redirect("feeds:import_opml")
messages.success(request, f"Importing {entry_count} feeds in the background.")
else:
result = import_opml(request.user, BytesIO(content))