Skip to content

Replace ROME Propono AtomPub server with self-contained StAX implementation - #161

Closed
snoopdave wants to merge 5 commits into
apache:masterfrom
snoopdave:replace-propono-atompub
Closed

snoopdave wants to merge 5 commits into
apache:masterfrom
snoopdave:replace-propono-atompub

Conversation

@snoopdave

Copy link
Copy Markdown
Contributor

What

Reimplements the Atom Publishing Protocol (RFC 5023) server using only JDK StAX (javax.xml.stream) and plain DTOs — no ROME, no Propono.

Why

The AtomPub server was built on ROME's rome-propono, which is only available up to ROME 1.19.0 (the last release that ships Propono). That pin held the entire ROME stack at 1.19.0 for the whole app, even though feed rendering doesn't use Propono. Dropping rome-propono frees ROME-for-feeds to be upgraded independently in a later, deliberate step.

All Propono usage was confined to webservices/atomprotocol/; feeds and the Planet aggregator are untouched.

Changes

  • RollerAtomServlet (new) replaces Propono's AtomServlet — method dispatch, 201 Created + Location/Content-Location, and media streaming ported over. Wired in web.xml; propono.properties and RollerAtomHandlerFactory removed.
  • New wire model (AtomEntry, AtomFeed, AtomContent, AtomLink, AtomPerson, AtomCategory, plus the service-doc DTOs and AtomMediaResource) with StAX AtomWriter / AtomReader. AtomReader disables DTDs and external entities (XXE-safe).
  • RollerAtomHandler / RollerAtomService / EntryCollection / MediaCollection keep their Roller domain-mapping logic; only the ROME types they touched changed. Fixes a latent BASIC-auth bug that compared against the null instance field instead of the looked-up user's password.
  • Auth: keep BASIC + OAuth, drop WSSE (removes WSSEUtilities and the wsse choice from the admin config labels, en/ja/zh_CN).

Design decisions

  • XML: JDK StAX only — no JDOM/JAXB. javax.xml.stream is not part of the javax→jakarta migration, so it's safe.
  • Scope: server only (the AtomPub server receives requests; there is no outbound HTTP in this path).

Testing

  • Unit tests for the reader, writer, DTOs, and request wrapper.
  • Integration test (RollerAtomProtocolTest) driving the full create / retrieve / update / delete lifecycle plus service-doc and media upload against in-memory Derby.
  • Schema-validation tests (AtomSchemaValidationTest) that validate AtomWriter output against the RFC 4287 (Atom) and RFC 5023 (AtomPub) RELAX NG schemas using Jing.

All 34 new tests pass (mvn -pl app test -Dtest='...atomprotocol.*').

Note: the HTTP transport and BASIC auth over the wire need the Spring web context and aren't exercised by the JUnit reactor. Recommend confirming wire-format interop with an over-the-wire exerciser (e.g. APE) against a deployed instance, since the previous format was ROME-generated.

…tation

Reimplements the Atom Publishing Protocol (RFC 5023) server using only
JDK StAX (javax.xml.stream) and plain DTOs -- no ROME, no Propono. This
removes the rome-propono dependency that pinned the entire ROME stack to
1.19.0, freeing ROME-for-feeds to be upgraded independently.

- New Roller-owned servlet (RollerAtomServlet) replaces Propono's
  AtomServlet; method dispatch, 201 Created/Location handling and media
  streaming are ported over.
- New wire model (AtomEntry/AtomFeed/AtomContent/AtomLink/AtomPerson/
  AtomCategory and the service-doc DTOs) with StAX AtomWriter/AtomReader.
  AtomReader disables DTDs and external entities (XXE-safe).
- RollerAtomHandler/RollerAtomService/EntryCollection/MediaCollection
  keep their domain-mapping logic; only the ROME types they touched
  changed. Fixes a latent BASIC-auth bug that compared against the null
  instance field instead of the looked-up user's password.
- Auth: keep BASIC + OAuth, drop WSSE (removes WSSEUtilities and the
  wsse choice from the admin config labels).
- Adds unit tests for the reader/writer/DTOs/request, an integration
  test driving the create/retrieve/update/delete lifecycle against
  in-memory Derby, and schema-validation tests that check AtomWriter
  output against the RFC 4287/5023 RELAX NG schemas via Jing.

The HTTP transport and BASIC auth over the wire require the Spring web
context and are not exercised by the JUnit reactor; verify those with an
over-the-wire exerciser (e.g. APE) against a deployed instance.

@mraible mraible left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran the same multi-agent review process over this PR that we've been using on the ROL-2183 stack (find in parallel, then adversarially verify every candidate; one finding below was verified against the rome-propono 1.19.0 bytecode). Nine verified findings and one lower-confidence one are inline on the relevant lines.

Two coordination notes that aren't tied to a single file:

The WSSE removal overlaps with #154, which keeps WSSE and currently tells admins AtomPub auth is "basic or wsse" (globalConfig text and ROL-2183 say the same). Happy to go either way, but the two PRs should agree, and whichever merges second inherits the reconciliation.

The Playwright branch of the stack (#157) has a WebServicesIT that exercises AtomPub end to end over Basic auth in CI, and it would have caught several of the inline findings here. Porting or extending it against this implementation is probably the highest-leverage follow-up, and I'm glad to do that once this lands.

Really glad to see AtomPub get a self-contained implementation, the Propono dependency has been a millstone for the Jakarta work.

Comment thread app/src/main/resources/propono.properties
Port master's Propono-based AtomPub changes to the StAX implementation:

- RollerAtomServlet answers 404 while webservices.enableAtomPub is off,
  authenticates before reading the body, caps Atom entry bodies at
  webservices.atomPubMaxEntrySize (413), and refuses malformed entries
  or entries with a DOCTYPE (400).
- AtomReader reports parse errors and DOCTYPEs as HTTP 400.
- MediaCollection takes master's media type policy, directory lookup and
  file naming fixes; createMediaResource builds the immutable resource.
- RollerAtomHandler takes master's strict basic/oauth auth selection and
  the shared isEntryPath() helper.
- Port RollerAtomServletTest, MediaCollectionTest and
  MediaCollectionPathTest off Propono types; drop the factory and SAX
  parser cases that no longer apply. Drop propono.properties.
- Stream media bodies to MediaCollection instead of buffering them;
  only Atom entries are read into memory, under the entry size limit.
- AtomReader reads only direct children of atom:entry (ignores
  atom:source) and accepts type="xhtml" text constructs.
- Media DELETE through the edit URI looks up the file without the
  .media-link suffix; missing media on GET, PUT and DELETE is a 404.
- POST without Content-Type returns 415.
- Write app:edited as a child of atom:entry (RFC 5023 10.2) and declare
  charset=utf-8 on the entry media type.
- Factor AtomWriter document setup into one helper.
- Note the removed AtomHandlerFactory extension point in CHANGES.md.

@snoopdave snoopdave left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR-Review: 2 inline comments posted.

Comment thread app/pom.xml Outdated
</dependency>

<!-- todo: remove/replace propono -->
<!-- Used by Trackback; previously reached the classpath transitively

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐞Claude Issue: Blocking: This adds commons-httpclient:3.1 as a direct compile dependency "for Trackback", but Trackback was removed on master (#178/#163) and nothing in app/src imports org.apache.commons.httpclient any more. mvn dependency:tree shows nothing else needs it. The library is EOL and has known CVEs (CVE-2012-5783, CVE-2014-3577, CVE-2015-5262), so the WAR would ship a vulnerable, unused jar that dependency scanners will flag. Remove the whole <dependency> block, including its exclusions.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖Claude: fixed by removing the commons-httpclient 3.1 dependency block (and its stale Trackback comment) from app/pom.xml. Nothing in app/src uses it; the full app test suite passes without it.

public Entry getEntry(AtomRequest areq) throws AtomException {


public AtomEntry getEntry(AtomRequest areq) throws AtomException {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐞Claude Issue: Important: getEntry() (GET on a .media-link URI) has no permission check. Any authenticated user can read the media-link entry (name, URLs, content type) of a file in any weblog, although getCollection() in this class requires canView and getMediaResource() requires canEdit. An unknown handle also gives website == null, which reaches getMediaFileByPath(null, ...) and returns a 500. This was carried over from the Propono version, but the rewrite is the moment to make it consistent: return AtomNotFoundException when the weblog is missing and AtomNotAuthorizedException when !RollerAtomHandler.canView(user, website), plus a test for each.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖Claude: fixed by making MediaCollection.getEntry() return AtomNotFoundException for an unknown weblog and AtomNotAuthorizedException when !canView(user, website), matching getCollection(). Tests: mediaLinkEntryOfAnUnknownWeblogIsNotFound, mediaLinkEntryNeedsPermissionOnTheWeblog.

- Send only the status code for 5xx AtomPub errors and unexpected
  exceptions; log the details server-side.
- Set Location/Content-Location only when the edit URI is a valid URI
  under this server's AtomPub URL (no CR/LF, no foreign host).
- Require view permission for media-link entries and return 404 for an
  unknown weblog, matching the media collection.
- Remove the unused commons-httpclient 3.1 dependency.
} else {
// Client errors carry a message written by Roller for the client
log.debug("Returning error to client: " + ae.getMessage(), ae);
response.sendError(ae.getStatus(), ae.getMessage());
AtomEntry entry) throws AtomException {
String location = safeLocation(entry.getLinkHref("edit"), handler.getAtomURL());
if (location != null) {
response.setHeader("Location", location);
AtomEntry entry) throws AtomException {
String location = safeLocation(entry.getLinkHref("edit"), handler.getAtomURL());
if (location != null) {
response.setHeader("Location", location);
String location = safeLocation(entry.getLinkHref("edit"), handler.getAtomURL());
if (location != null) {
response.setHeader("Location", location);
response.setHeader("Content-Location", location);
- Exclude xercesImpl and xml-apis from the test-scoped Jing dependency.
  Jing declares them with runtime scope, which replaced the WAR's
  Xerces 2.11.0 and xml-apis 1.4.01 with 2.9.1 and 1.0.b2.
- Add source and copyright headers to the RFC 4287 and RFC 5023 RELAX NG
  schemas used by AtomSchemaValidationTest.
@snoopdave

Copy link
Copy Markdown
Contributor Author

Replaced by #210, which has the same code from a branch in apache/roller instead of a fork. The review history stays here.

@snoopdave snoopdave closed this Oct 10, 2026
snoopdave added a commit that referenced this pull request Oct 10, 2026
…tation (#210)

* Replace ROME Propono AtomPub server with self-contained StAX implementation

Reimplements the Atom Publishing Protocol (RFC 5023) server using only
JDK StAX (javax.xml.stream) and plain DTOs -- no ROME, no Propono. This
removes the rome-propono dependency that pinned the entire ROME stack to
1.19.0, freeing ROME-for-feeds to be upgraded independently.

- New Roller-owned servlet (RollerAtomServlet) replaces Propono's
  AtomServlet; method dispatch, 201 Created/Location handling and media
  streaming are ported over.
- New wire model (AtomEntry/AtomFeed/AtomContent/AtomLink/AtomPerson/
  AtomCategory and the service-doc DTOs) with StAX AtomWriter/AtomReader.
  AtomReader disables DTDs and external entities (XXE-safe).
- RollerAtomHandler/RollerAtomService/EntryCollection/MediaCollection
  keep their domain-mapping logic; only the ROME types they touched
  changed. Fixes a latent BASIC-auth bug that compared against the null
  instance field instead of the looked-up user's password.
- Auth: keep BASIC + OAuth, drop WSSE (removes WSSEUtilities and the
  wsse choice from the admin config labels).
- Adds unit tests for the reader/writer/DTOs/request, an integration
  test driving the create/retrieve/update/delete lifecycle against
  in-memory Derby, and schema-validation tests that check AtomWriter
  output against the RFC 4287/5023 RELAX NG schemas via Jing.

The HTTP transport and BASIC auth over the wire require the Spring web
context and are not exercised by the JUnit reactor; verify those with an
over-the-wire exerciser (e.g. APE) against a deployed instance.

* Address PR #161 review findings

- Stream media bodies to MediaCollection instead of buffering them;
  only Atom entries are read into memory, under the entry size limit.
- AtomReader reads only direct children of atom:entry (ignores
  atom:source) and accepts type="xhtml" text constructs.
- Media DELETE through the edit URI looks up the file without the
  .media-link suffix; missing media on GET, PUT and DELETE is a 404.
- POST without Content-Type returns 415.
- Write app:edited as a child of atom:entry (RFC 5023 10.2) and declare
  charset=utf-8 on the entry media type.
- Factor AtomWriter document setup into one helper.
- Note the removed AtomHandlerFactory extension point in CHANGES.md.

* Address CodeQL alerts and review findings on PR #161

- Send only the status code for 5xx AtomPub errors and unexpected
  exceptions; log the details server-side.
- Set Location/Content-Location only when the edit URI is a valid URI
  under this server's AtomPub URL (no CR/LF, no foreign host).
- Require view permission for media-link entries and return 404 for an
  unknown weblog, matching the media collection.
- Remove the unused commons-httpclient 3.1 dependency.

* Keep Jing from downgrading Xerces; credit the RFC schemas

- Exclude xercesImpl and xml-apis from the test-scoped Jing dependency.
  Jing declares them with runtime scope, which replaced the WAR's
  Xerces 2.11.0 and xml-apis 1.4.01 with 2.9.1 and 1.0.b2.
- Add source and copyright headers to the RFC 4287 and RFC 5023 RELAX NG
  schemas used by AtomSchemaValidationTest.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants