You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Jackson 3: FAIL_ON_TRAILING_TOKENS default flip not compensated in main-source parsers (JSONataUtils.loadJSONFile + CLI) #435
Jackson 3 flips the FAIL_ON_TRAILING_TOKENS deserialization default from false (Jackson 2) to true. During the migration (#430, PR #431) this leniency was restored on exactly one test mapper (src/test/java/.../expressions/utils/Utils.java:83) but was not restored on the main‑source parse paths, so those became stricter than under Jackson 2. Content after the first parsed value (trailing tokens / trailing typos / concatenated values) that Jackson 2 silently ignored now throws at runtime.
The core public evaluate path is unaffected (it takes a pre‑parsed JsonNode), which is why the test suite stayed green. The exposure is in the file/string loaders below.
Where
Main source (production / public API)
src/main/java/com/api/jsonata4java/JSONataUtils.java:374 — loadJSONFile uses new ObjectMapper().readValue(br, Object.class). This is a public utility used to load .json files. A file that loaded fine under 2.x (trailing whitespace‑separated token, two concatenated values, etc.) now throws a trailing‑token error. Highest impact.
src/main/java/com/api/jsonata4java/Tester.java:123 — the mapper was faithfully rebuilt for ESCAPE_NON_ASCII but does not disable FAIL_ON_TRAILING_TOKENS. Clearest evidence the flag flip was known (fixed in test Utils) but applied inconsistently. CLI tool, lower impact.
src/main/java/com/api/jsonata4java/TesterTimeBox.java:121 — new ObjectMapper() for readTree. CLI tool.
src/main/java/com/api/jsonata4java/testerui/TesterUI.java and Test.java — same pattern; demo/UI, low impact.
Test source (latent inconsistency)
src/test/java/.../expressions/utils/Utils.java:76 (getJson) and :247 (toJson) still use bare new ObjectMapper() while the sibling mapper field (line 83) disables FAIL_ON_TRAILING_TOKENS. Same file, divergent parsing. Currently latent (both helpers are unreferenced), but it undermines the "parse identically to Jackson 2" intent stated in the line‑83 comment.
Verified
FAIL_ON_TRAILING_TOKENS defaults to true in jackson‑databind 3.2.0 (bytecode: iconst_1 in the enum init) vs false in Jackson 2. Only Utils.java:83 compensates.
Suggested fix
Introduce a single shared "Jackson‑2‑compatible" mapper factory (disabling FAIL_ON_TRAILING_TOKENS, and centralizing ESCAPE_NON_ASCII etc.) and route all parse sites through it — at minimum JSONataUtils.loadJSONFile, and the Utils.javagetJson/toJson helpers, so parsing behavior no longer diverges by call site. This also addresses the duplicated builder configuration noted during review (identical JsonMapper.builder()... blocks now appear in AgnosticTestSuite and Tester).
Resolved by design: keep the public loadJSONFile (and the other main-source parsers) strict.
Rationale — this project is a functional port of jsonata.js, and the JS reference parses input with JSON.parse, which rejects trailing tokens after the first value. So Jackson 3's stricter FAIL_ON_TRAILING_TOKENS=true default is the functionally-compatible behavior; restoring Jackson 2's leniency on the public API would make the port more permissive than the reference. No compat shim will be added.
The one lenient mapper (src/test/java/.../expressions/utils/Utils.java:83, which disables FAIL_ON_TRAILING_TOKENS) is retained deliberately: it exists only to keep a long-standing test-data typo ("[\"h11\", \"h21\"]]" in BasicExpressionsTests) parsing byte-for-byte, so no test assertion had to change during the migration. That is an intentional test-only exception, not an inconsistency to reconcile.
Activity
wnm3 commentedon Jul 7, 2026
Resolved by design: keep the public
loadJSONFile(and the other main-source parsers) strict.Rationale — this project is a functional port of jsonata.js, and the JS reference parses input with
JSON.parse, which rejects trailing tokens after the first value. So Jackson 3's stricterFAIL_ON_TRAILING_TOKENS=truedefault is the functionally-compatible behavior; restoring Jackson 2's leniency on the public API would make the port more permissive than the reference. No compat shim will be added.The one lenient mapper (
src/test/java/.../expressions/utils/Utils.java:83, which disablesFAIL_ON_TRAILING_TOKENS) is retained deliberately: it exists only to keep a long-standing test-data typo ("[\"h11\", \"h21\"]]"inBasicExpressionsTests) parsing byte-for-byte, so no test assertion had to change during the migration. That is an intentional test-only exception, not an inconsistency to reconcile.Closing as won't-fix (working as intended).