Compare commits

...
Author SHA1 Message Date
ksmithandClaude Sonnet 5 17e7f63e61 Update link/author/summary/published_at on refetch, not just title (T6)
A changed guid's article only ever had its title/content_html/
content_hash refreshed on a subsequent poll — legitimate feed edits to
an entry's link, author, summary, or publication date were silently
ignored forever, which can affect navigation, ordering, and rules that
key off those fields.

content_hash is now computed over every mutable field feedparser gives
us for an entry (title, link, author, summary, content_html,
published_at), and all of them are written together when the hash
changes. Per-user read/star state is untouched either way, same as
before.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 14:44:15 -06:00
ksmithandClaude Sonnet 5 0bfe9c8076 Enqueue the initial feed fetch instead of running it inline (T3)
add_feed ran fetch_and_ingest synchronously during the POST request, so
a slow or large feed tied up a Gunicorn worker for the duration of the
fetch (up to the 15s HTTP timeout, plus parse/ingest time) and a user
could repeat it with distinct URLs to reduce request capacity.

The subscription is now created synchronously as before, but the first
fetch is dispatched via fetch_feed.delay() — the same task the periodic
poll uses, so the save/error bookkeeping is unchanged. The user sees an
immediate redirect with a message that the fetch is in progress rather
than waiting on it in the request/response cycle.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 14:44:09 -06:00
ksmithandClaude Sonnet 5 7bb981b545 Rebuild stream date-group headers in the viewer's local timezone
The server still groups articles into "Received today/yesterday/<date>"
headers using UTC, so an article published near UTC midnight could land
in a mismatched day group. Each article row now ships its raw UTC
instant (data-published-utc); static/js/date-groups.js discards the
server-rendered headers on load and rebuilds them using the browser's
own local calendar day, same approach as the article-time localization
this follows up on.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 14:36:27 -06:00
ksmithandClaude Sonnet 5 fd68400822 Render article/feed times in the viewer's local timezone, client-side
Everything is stored and server-rendered in UTC (config.settings.base's
TIME_ZONE), so article times, the article detail header, and the
dashboard's "last fetched" column were always showing UTC regardless of
where the viewer actually is.

Article/feed timestamps now render as <time datetime="..."> elements
carrying the UTC instant plus a UTC-formatted fallback as their initial
text; static/js/localize-times.js reformats them using the browser's
own timezone via toLocaleString(), both on page load and after every
htmx swap (pagination, the inline article expand, the dashboard's retry
action all inject new <time> elements without a full page load).

No server-side change needed — this doesn't touch the (currently
unused) per-user timezone field, and works correctly even if the
viewer travels or changes their system timezone, with no settings page
required. Verified live in the browser against a UTC-stored timestamp
rendering correctly shifted to the sandbox's local zone.

Known related issue, out of scope here: the stream's "Received
today/yesterday" date-group headers (apps.reader.views._group_by_date)
are still computed server-side via timezone.localdate(), which is also
effectively UTC — an article just before/after UTC midnight could land
in a date group that doesn't match its now-correctly-localized
time-of-day. Flagged for the user rather than fixed, since correcting
it properly would mean moving date-grouping to the client too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-31 13:31:07 -06:00
11 changed files with 237 additions and 34 deletions

No files matched your search

+33 -8
View File
@@ -97,8 +97,15 @@ def sanitize_html(raw_html: str) -> str:
return cleaned[:MAX_CONTENT_HTML_CHARS]
def _content_hash(title: str, content_html: str) -> str:
return hashlib.sha256(f"{title}\n{content_html}".encode("utf-8")).hexdigest()
def _content_hash(
title: str, link: str, author: str, summary: str, content_html: str, published_at
) -> str:
"""Covers every mutable Article field an entry carries (T6) — a feed
editing its link/author/summary/published date, not just title or
body, must still be detected as a change worth re-saving."""
published_key = published_at.isoformat() if published_at else ""
payload = "\n".join([title, link, author, summary, content_html, published_key])
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
def _entry_guid(entry) -> str:
@@ -213,20 +220,24 @@ def fetch_and_ingest(feed: Feed) -> FetchResult:
for entry in entries:
guid = _entry_guid(entry)
title = (getattr(entry, "title", "") or "")[:1000]
link = (getattr(entry, "link", "") or "")[:2000]
author = (getattr(entry, "author", "") or "")[:500]
summary = (getattr(entry, "summary", "") or "")[:5000]
content_html = sanitize_html(_entry_content_html(entry))
content_hash = _content_hash(title, content_html)
published_at = _entry_published_at(entry)
content_hash = _content_hash(title, link, author, summary, content_html, published_at)
article, created = Article.objects.get_or_create(
feed=feed,
guid=guid,
defaults={
"link": (getattr(entry, "link", "") or "")[:2000],
"link": link,
"title": title,
"author": (getattr(entry, "author", "") or "")[:500],
"author": author,
"content_html": content_html,
"summary": (getattr(entry, "summary", "") or "")[:5000],
"summary": summary,
"content_hash": content_hash,
"published_at": _entry_published_at(entry),
"published_at": published_at,
},
)
@@ -234,9 +245,23 @@ def fetch_and_ingest(feed: Feed) -> FetchResult:
new_ids.append(article.pk)
elif article.content_hash != content_hash:
article.title = title
article.link = link
article.author = author
article.summary = summary
article.content_html = content_html
article.published_at = published_at
article.content_hash = content_hash
article.save(update_fields=["title", "content_html", "content_hash"])
article.save(
update_fields=[
"title",
"link",
"author",
"summary",
"content_html",
"published_at",
"content_hash",
]
)
updated_ids.append(article.pk)
return FetchResult(
+36
View File
@@ -1,3 +1,5 @@
import datetime as dt
import pytest
import requests
import responses
@@ -204,6 +206,40 @@ def test_refetching_updated_content_does_not_reset_user_state():
assert ArticleTag.objects.filter(user=user, article=article, tag=starred_tag).exists()
@pytest.mark.django_db
@responses.activate
def test_refetching_picks_up_link_author_summary_and_published_at_changes():
"""Previously only title/content_html/content_hash were updated on
refetch (T6) — a feed correcting its link, author, summary, or
publication date without touching the title went unnoticed forever."""
feed = Feed.objects.create(feed_url=FEED_URL)
responses.add(responses.GET, FEED_URL, body=GOOD_FEED_XML, status=200)
fetch_and_ingest(feed)
feed.save()
article = Article.objects.get(feed=feed)
assert article.link == "https://example.com/1"
assert article.author == ""
edited_xml = GOOD_FEED_XML.replace(
"<link>https://example.com/1</link>",
"<link>https://example.com/1-moved</link>",
).replace(
"<pubDate>Mon, 01 Jan 2024 00:00:00 GMT</pubDate>",
"<pubDate>Tue, 02 Jan 2024 00:00:00 GMT</pubDate>\n <author>jane@example.com</author>",
)
responses.add(responses.GET, FEED_URL, body=edited_xml, status=200)
result = fetch_and_ingest(feed)
feed.save()
assert result.updated_article_ids == [article.pk]
article.refresh_from_db()
assert article.link == "https://example.com/1-moved"
assert article.author == "jane@example.com"
assert article.published_at == dt.datetime(2024, 1, 2, tzinfo=dt.timezone.utc)
def test_sanitize_html_caps_stored_output_length():
huge = "<p>" + ("a" * (MAX_CONTENT_HTML_CHARS + 10_000)) + "</p>"
cleaned = sanitize_html(huge)
+8 -3
View File
@@ -54,7 +54,11 @@ GOOD_FEED_XML = """<?xml version="1.0"?>
@pytest.mark.django_db
@responses.activate
def test_add_feed_view_subscribes_and_fetches_immediately(client, two_users):
def test_add_feed_view_subscribes_and_enqueues_the_initial_fetch(client, two_users):
"""The subscription is created synchronously; the first fetch is
dispatched via fetch_feed.delay() (T3) rather than run inline — in
tests CELERY_TASK_ALWAYS_EAGER still runs it immediately, so the
article it ingests is visible by the time the response returns."""
alice, _ = two_users
client.login(username="alice", password="pw-alice-123")
responses.add(responses.GET, "https://example.com/feed.xml", body=GOOD_FEED_XML, status=200)
@@ -62,9 +66,10 @@ def test_add_feed_view_subscribes_and_fetches_immediately(client, two_users):
response = client.post(reverse("feeds:add_feed"), {"feed_url": "https://example.com/feed.xml"})
assert response.status_code == 302
assert Subscription.objects.filter(
subscription = Subscription.objects.get(
user=alice, feed__feed_url="https://example.com/feed.xml"
).exists()
)
assert subscription.feed.articles.filter(guid="guid-1").exists()
@pytest.mark.django_db
+11 -19
View File
@@ -12,8 +12,7 @@ from apps.articles.models import SubscriptionTag, Tag
from .forms import AddFeedForm, EditSubscriptionForm, OPMLImportForm
from .models import Feed, Subscription
from .opml import export_opml, extract_opml_bytes, import_opml, parse_opml
from .services.fetcher import fetch_and_ingest
from .tasks import import_opml_task
from .tasks import fetch_feed, import_opml_task
OPML_ASYNC_THRESHOLD = 50
@@ -39,23 +38,16 @@ def add_feed(request):
user=request.user, subscription=subscription, tag=tag
)
# Fetch synchronously so the user sees results immediately; the
# periodic Celery poll takes over from here for subsequent fetches.
result = fetch_and_ingest(feed)
feed.save()
if result.status == Feed.STATUS_ERROR:
messages.warning(
request,
f"Subscribed to {feed_url}, but the initial fetch failed: {result.error_message}",
)
else:
article_count = len(result.new_article_ids)
messages.success(
request,
f"Subscribed to '{subscription.display_title}' — "
f"{article_count} article(s) fetched.",
)
# Enqueue the initial fetch instead of running it inline (T3) — a
# slow or large feed would otherwise tie up a Gunicorn worker for
# the duration of the request; fetch_feed handles the save/error
# bookkeeping the same way the periodic poll's dispatch does.
fetch_feed.delay(feed.pk)
messages.success(
request,
f"Subscribed to {feed_url}. Fetching its articles now — "
"refresh in a few seconds if you don't see them yet.",
)
return redirect("reader:stream")
return render(request, "feeds/add_feed.html", {"form": form})
+29
View File
@@ -1,3 +1,5 @@
import re
import pytest
from django.contrib.auth import get_user_model
from django.urls import reverse
@@ -188,3 +190,30 @@ def test_sidebar_shows_a_tags_total_unread_count(client, alice):
# The only "7" on the page — the individual feeds show 3 and 4.
content = response.content.decode()
assert '<span class="unread-count">7</span>' in content
@pytest.mark.django_db
def test_article_rows_carry_their_utc_instant_for_client_side_date_grouping(client, alice):
"""The server still groups date headers in UTC, but ships each row's
raw UTC instant so date-groups.js can rebuild the headers using the
viewer's local calendar day instead."""
from django.utils import timezone
feed = Feed.objects.create(feed_url="https://example.com/feed.xml")
Subscription.objects.create(user=alice, feed=feed)
dated = Article.objects.create(
feed=feed, guid="dated", title="Dated", published_at=timezone.now()
)
undated = Article.objects.create(feed=feed, guid="undated", title="Undated")
client.login(username="alice", password="pw-alice-123")
response = client.get(reverse("reader:stream"), {"filter": "all"})
content = response.content.decode()
expected_iso = dated.published_at.isoformat()
assert f'id="article-{dated.pk}"' in content
assert f'data-published-utc="{expected_iso}"' in content
undated_row = re.search(rf'<li id="article-{undated.pk}".*?>', content, flags=re.DOTALL)
assert undated_row is not None
assert 'data-published-utc=""' in undated_row.group(0)
+61
View File
@@ -0,0 +1,61 @@
(function () {
// The server groups articles into "Received today/yesterday/<date>"
// headers using UTC (config.settings.base TIME_ZONE), so an article
// published near UTC midnight can land in the wrong day group from the
// viewer's own perspective. Each article row carries its UTC instant in
// data-published-utc; this throws away the server's date-header <li>s
// and rebuilds them using the browser's local calendar day instead.
var MONTHS = [
"January", "February", "March", "April", "May", "June",
"July", "August", "September", "October", "November", "December",
];
function localDateOf(iso) {
if (!iso) return null;
var date = new Date(iso);
if (isNaN(date.getTime())) return null;
date.setHours(0, 0, 0, 0);
return date;
}
function labelFor(day, today) {
if (day === null) return "Undated";
var formatted = day.getDate() + " " + MONTHS[day.getMonth()] + " " + day.getFullYear();
var diffDays = Math.round((today - day) / 86400000);
if (diffDays === 0) return "Received today — " + formatted;
if (diffDays === 1) return "Received yesterday — " + formatted;
return "Received " + formatted;
}
function regroupDateHeaders(root) {
if (!root) return;
var list = root.matches && root.matches("ul.article-list") ? root : root.querySelector("ul.article-list");
if (!list || !list.querySelector("li.article-row[data-published-utc], li.article-row")) return;
list.querySelectorAll("li.date-header").forEach(function (el) {
el.parentNode.removeChild(el);
});
var today = new Date();
today.setHours(0, 0, 0, 0);
var currentKey;
list.querySelectorAll("li.article-row").forEach(function (row) {
var day = localDateOf(row.getAttribute("data-published-utc"));
var key = day === null ? "undated" : day.getTime();
if (key !== currentKey) {
var header = document.createElement("li");
header.className = "date-header";
header.textContent = labelFor(day, today);
list.insertBefore(header, row);
currentKey = key;
}
});
}
regroupDateHeaders(document);
document.body.addEventListener("htmx:afterSwap", function (event) {
regroupDateHeaders(event.target);
});
})();
+41
View File
@@ -0,0 +1,41 @@
(function () {
// Everything is stored and server-rendered in UTC (config.settings.base
// TIME_ZONE) — these <time datetime="..."> elements carry that UTC
// instant plus a UTC-formatted fallback as their initial text, and this
// reformats them using the browser's own timezone, client-side, so the
// server never needs to know (or guess) where the viewer actually is.
var SHORT_OPTIONS = { hour: "numeric", minute: "2-digit" };
var LONG_OPTIONS = {
year: "numeric",
month: "short",
day: "numeric",
hour: "2-digit",
minute: "2-digit",
};
function localizeTimesWithin(root) {
if (!root || typeof root.querySelectorAll !== "function") return;
root.querySelectorAll("time[data-local-format]").forEach(function (el) {
var iso = el.getAttribute("datetime");
if (!iso) return;
var date = new Date(iso);
if (isNaN(date.getTime())) return;
var options = el.dataset.localFormat === "short" ? SHORT_OPTIONS : LONG_OPTIONS;
try {
el.textContent = date.toLocaleString(undefined, options);
} catch (e) {
/* Intl unavailable or threw — leave the server-rendered UTC text as-is */
}
});
}
localizeTimesWithin(document);
// Article rows/content are frequently swapped in by htmx (pagination,
// the inline expand-article view, the dashboard's retry action) without
// a full page load, so every swap needs the same pass over just the
// new content.
document.body.addEventListener("htmx:afterSwap", function (event) {
localizeTimesWithin(event.target);
});
})();
+2
View File
@@ -53,6 +53,8 @@
{% endif %}
<script src="{% static 'js/stream.js' %}" defer></script>
<script src="{% static 'js/localize-times.js' %}" defer></script>
<script src="{% static 'js/date-groups.js' %}" defer></script>
{% if user.is_authenticated %}
<script src="{% static 'js/theme.js' %}" defer></script>
<script src="{% static 'js/sidebar-state.js' %}" defer></script>
+7 -1
View File
@@ -31,7 +31,13 @@
<td>{{ feed.subscriber_count }}</td>
<td>{{ feed.get_last_fetch_status_display }}</td>
<td>{{ feed.fetch_error_count }}</td>
<td>{{ feed.last_fetched_at|default:"never" }}</td>
<td>
{% if feed.last_fetched_at %}
<time datetime="{{ feed.last_fetched_at|date:"c" }}" data-local-format="long">{{ feed.last_fetched_at|date:"M j, Y H:i" }}</time>
{% else %}
never
{% endif %}
</td>
<td>{{ feed.last_error_message|default:"—" }}</td>
<td>
<form method="post" action="{% url 'core:retry_feed' feed.id %}">
+4 -1
View File
@@ -2,7 +2,10 @@
<h2 class="article-content-title">{{ article.title|default:"(untitled)" }}</h2>
<p class="article-meta">
{% if article.feed.favicon_url %}<img src="{{ article.feed.favicon_url }}" alt="" class="feed-favicon">{% endif %}
{{ article.feed.title }} &middot; {{ article.published_at|date:"M j, Y H:i" }}
{{ article.feed.title }}
{% if article.published_at %}
&middot; <time datetime="{{ article.published_at|date:"c" }}" data-local-format="long">{{ article.published_at|date:"M j, Y H:i" }}</time>
{% endif %}
{% if article.author %} &middot; {{ article.author }}{% endif %}
{% if article.link %} &middot; <a href="{{ article.link }}" target="_blank" rel="noopener noreferrer">Original &#8599;</a>{% endif %}
</p>
+5 -2
View File
@@ -1,4 +1,5 @@
<li id="article-{{ article.pk }}" class="article-row{% if article.is_read %} is-read{% endif %}">
<li id="article-{{ article.pk }}" class="article-row{% if article.is_read %} is-read{% endif %}"
data-published-utc="{% if article.published_at %}{{ article.published_at|date:"c" }}{% endif %}">
<div class="article-row-main">
{% include "reader/_article_actions.html" %}
{% if article.feed.favicon_url %}
@@ -8,7 +9,9 @@
<a href="{% url 'reader:article_detail' article.pk %}" class="article-title"
data-inline-url="{% url 'reader:article_inline' article.pk %}"
onclick="return toggleArticle(event, {{ article.pk }})">{{ article.title|default:"(untitled)" }}</a>
<span class="article-time">{{ article.published_at|date:"g:i A" }}</span>
{% if article.published_at %}
<time class="article-time" datetime="{{ article.published_at|date:"c" }}" data-local-format="short">{{ article.published_at|date:"g:i A" }}</time>
{% endif %}
</div>
<div id="article-body-{{ article.pk }}" class="article-inline-body" hidden></div>
</li>