Repository navigation
feat(scenarios): scenario-card schema + loader (private packages/qa package) and nine open seed cards - #30340
Conversation
Claude-Session: https://claude.ai/code/session_01Q4EjjPXwX1YfHyKJUh4agu Co-authored-by: Claude <[email protected]>
… changeset Claude-Session: https://claude.ai/code/session_01Q4EjjPXwX1YfHyKJUh4agu Co-authored-by: Claude <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q4EjjPXwX1YfHyKJUh4agu Co-authored-by: Claude <[email protected]>
…spkg rollback pin Claude-Session: https://claude.ai/code/session_01Q4EjjPXwX1YfHyKJUh4agu Co-authored-by: Claude <[email protected]>
📓 Docs Drift CheckThis PR changes 1 package(s): 21 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 5 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 81e722ef4db8370682478a0e106f8189915e6a65 && git checkout 81e722ef4db8370682478a0e106f8189915e6a65
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 0c7e95aff5996de2313b4668760eafee2e50172d c1affd3f516a0afb92ef645bf45a53548a488f1e && git checkout -B drift-repro 0c7e95aff5996de2313b4668760eafee2e50172d && git merge --no-ff c1affd3f516a0afb92ef645bf45a53548a488f1e
node scripts/docs-audit/affected-docs.mjs --json 0c7e95aff5996de2313b4668760eafee2e50172d
|
|
epic:#3600#1 · hotlong · session_01Q4EjjPXwX1YfHyKJUh4agu 席位核对Verdict: ACCEPT at head
|
…enario-card-schema
… row scope, unanswerable, write outcomes, agent actions; controls upgraded; six retired cards removed Claude-Session: https://claude.ai/code/session_01Q4EjjPXwX1YfHyKJUh4agu Co-authored-by: Claude <[email protected]>
…n; README for the v3 library shape Claude-Session: https://claude.ai/code/session_01Q4EjjPXwX1YfHyKJUh4agu Co-authored-by: Claude <[email protected]>
… ErrorCode catalogue The control cards' validation refusals now expect VALIDATION_FAILED, the code the engine's record validator answers for a per-field refusal. Claude-Session: https://claude.ai/code/session_01Q4EjjPXwX1YfHyKJUh4agu Co-authored-by: Claude <[email protected]>
|
epic:#3600#1 · hotlong · session_01Q4EjjPXwX1YfHyKJUh4agu 席位核对 (patch round 1, option A: foundation)Verdict: ACCEPT at head
|
Fixes #30289
Clause-②: no
Metadata-protocol: none
维护者速读
@objectstack/scenarios(packages/qa/scenarios,不发布、无构建步骤、不调用任何模型),内含场景卡的 zod schema 与加载器。按 v3 场景库的地基改造:五项 schema 扩展(跨对象关系路径、卡级asOf加相对时间窗、按角色的行范围与逐角色真值、"种子答不了"的真值类型、写操作声明预期结果),外加可选的run.agentActions。三张简单对照卡(发票、项目任务、CRM)升级到新格式,每张 8–9 个问数、5 个写操作。删除 6 张退役卡。README 写明 v3 场景库形态和中档卡的能力底线。packages/spec、不改公开面、包为private: true、expect形状未动(与已合并的评分器共用)。仓级改动只有已登记的三处(cross-package 声明、turbo 输入、ci-filter-parity 自测按名记录)。回滚 = revert 本 PR。packages/qa/scenarios/src/loader.ts的checkPath/resolverFor(关系路径)和逐角色真值那一段;②packages/qa/scenarios/src/schema.ts的OutcomeSchema与 guardrail 对 outcome 的要求(在loader.ts);③docs/qa/scenarios/invoice_with_line_items.json(新格式的完整样子)。pnpm --filter @objectstack/scenarios test,预期Test Files 3 passed、Tests 59 passed;把invoice_with_line_items.json里 sales_rep 的scope删掉再跑,library 用例转红,amount_per_invoice的逐角色真值对不上。What changes
Rescope (seat decision, option A). This PR is now the v3 library's foundation. The six mid-tier cards (travel expense, order-to-cash, service desk, leave and balances, requisition-to-receipt, asset lifecycle) are sub-issues #30476, #30477, #30478, #30479, #30480 and #30481. Each is Blocked-by this PR and each lands as its own PR. No mid-tier card content is written here.
New private package
@objectstack/scenarios(packages/qa/scenarios), named after its siblings (private: true,exportsatsrc, no build step,zodits only runtime dependency). It has three files:src/schema.ts: the card shape, with strict objects throughout.src/truth.ts: the declarative truth evaluator.src/loader.ts:loadScenarios(dir, { areas })returns typed cards withtruthsandpersonaTruths, or{ file, path, message }refusals. A card with any refusal is not returned.The five schema extensions. Each one extends a closed enum or shape; nothing is loosened.
Relation paths.
filter[].fieldandgroupByacceptlookup.fieldand deeper paths, such asinvoice.statusoraccount.industry. Every hop except the last must be a lookup declared inexpect.lookups, and the path resolves through the seed rows' lookup ids. A hop that is not a declared lookup is refused at the exact path, and so is a final field that no seed row of the reached object carries.Time windows.
asOfis a card-level fixed date, validated as an ISO date. The new filter opin_windowtakes a value of{ kind: last_days, days }or{ kind: calendar, period: this_month | previous_month | this_year | previous_year }, and the window is inclusive and computed againstasOf. The loader refuses three things: a window when the card has noasOf, a window over a field that holds no date, and a window value given to any op other thanin_window.Persona row scope. A persona can now declare
scope: [{ object, field, value }], wherefieldmay be a path (invoice.ownerscopes invoice lines through their invoice). A question can listpersonas, and the loader then also computes that question's truth as each listed persona. Before evaluating, it applies the persona's scope rules for the question's object. If the question reads an object or field the persona hides, the persona's truth is{ kind: withheld }, meaning a correct reply declines.Unanswerable. A new truth kind
{ kind: unanswerable, reason }says the seed cannot answer, so a correct reply says so instead of giving a figure. It joins the closed enum.Declared write outcomes. Every operation now names the persona it acts for (
actingAs) and declares anoutcome, so it can be graded by code alone. The outcome is one of:applied, with the post-state rows: each named by a seededid, or bywherefor a row the write creates, and carrying either thevaluesthe data API must read back orabsent. It can also name anauditActor.refused, with a code from the spec's closedErrorCodecatalogue (@objectstack/spec/api: StandardErrorCode plus the registered ledger, ADR-0112). The catalogue is read from the spec export and never restated here, so the package now depends on@objectstack/specand its vitest config aliases that import to spec source.pending_confirmation.The guardrail checks are now held to the outcome:
write_validationneeds a refused create or update,confirm_irreversibleneedspending_confirmation, andaudit_actorneedsappliedwith anauditActor.Two smaller additions:
run.agentActionslists at least 3 agent-callable business actions,{ name, object, summary, confirm }, with at least one confirm-gated.run.countsReason, or it is refused.The three controls, upgraded. The
expectblocks are untouched: the corpus derivation still equals the cards.invoice_with_line_itemsinvoice.status;issued_onthis month (asOf2026-10-31); 2 per-persona (sales_rep row scope, warehouse_staff withheld); 1 unanswerableproject_with_tasksproject.status; 2 per-persona (contractor scope by assignee, client_viewer and contractor withheld); 1 unanswerablecrm_account_with_contactsaccount.industry; 2 per-persona (sdr scope viaaccount.owner, partner_user withheld); 1 unanswerableEach control's five operations are the same set of outcomes: one applied create, one applied update with an audit actor, one confirm-gated delete, one refused validation violation (
VALIDATION_FAILED, the code the engine's record validator answers for a per-field refusal), and one refused overstep by a persona without the right (PERMISSION_DENIED). Each control also declares 3 agent actions, one of them confirm-gated.Retired: the six earlier generic cards are removed with their test pins.
docs/qa/scenarios/README.mdnow describes the v3 shape, the new arms, and the mid-tier capability floor, including the declarative floor: flows, formulas and validation rules are preferred over code hooks, so a graduated app can install unmanaged.Repo wiring forced by existing gates, unchanged from the first round:
scripts/cross-package-test-inputs.mjsandturbo.jsondeclare that this suite readsdocs/qa/scenarios/*.jsonand the checklist area files.scripts/check-ci-filter-parity.mjsnames that glob in its rollback pin, which moves from 20 to 21.Tests
test/library.test.ts: the three controls load with zero refusals. All 25 truths are asserted against hand-worked values, and so are the 10 persona truths.test/refusals.test.ts: 38 refusal cases in a table, each asserting the single refusal'sfile, exactpathand message class. There are also 2 controls (the unmutated card loads clean, and a below-band card loads once it statescountsReason), plus id/basename and invalid-JSON cases. The new arms are covered by:test/truth.test.ts: the evaluator, now including unanswerable, the resolver for paths, and window ranges across month and year edges (including a leap February).Acceptance notes
withheldfor that persona. That includes a hidden field reached through a relation path, and the lookup field used as a hop counts as read.expectshape is untouched.Verification
Package suite and typecheck (head c1affd3)
Ablations: one per new arm, through scripts/ablation-replace.mjs, each restored byte-identical (git hash-object == HEAD blob), git diff HEAD empty after
relation_paths: mutation landed (blob 73dcb24d->3fb8af58), 32 test(s) red, e.g. ['loads every card with no refusal', 'holds exactly the three control cards (the mid-tier cards land on their own)']; restored: Truetime_windows: mutation landed (blob d562f0d3->9254d982), 2 test(s) red, e.g. ["invoice_with_line_items: every question's computed truth is the hand-worked one", 'computes windows against asOf, inclusive, across month and year edges']; restored: Truepersona_row_scope: mutation landed (blob 73dcb24d->c801f0c3), 3 test(s) red, e.g. ["invoice_with_line_items: every question's computed truth is the hand-worked one", "project_with_tasks: every question's computed truth is the hand-worked one"]; restored: Truepersona_withheld: mutation landed (blob 73dcb24d->39a838a3), 3 test(s) red, e.g. ["invoice_with_line_items: every question's computed truth is the hand-worked one", "project_with_tasks: every question's computed truth is the hand-worked one"]; restored: Trueunanswerable: mutation landed (blob d562f0d3->f9a664ff), 4 test(s) red, e.g. ["invoice_with_line_items: every question's computed truth is the hand-worked one", "project_with_tasks: every question's computed truth is the hand-worked one"]; restored: Trueoutcome_guardrail_match: mutation landed (blob 73dcb24d->d93bd217), 1 test(s) red, e.g. ['refuses a confirmation check on an operation that does not wait']; restored: Trueoutcome_post_state_id: mutation landed (blob 73dcb24d->f1d67cba), 1 test(s) red, e.g. ['refuses a post-state naming a created row by an id the seed lacks']; restored: Trueagent_actions_confirm: mutation landed (blob 73dcb24d->58d94865), 1 test(s) red, e.g. ['refuses agent actions with none confirm-gated']; restored: Truecounts_reason: mutation landed (blob 73dcb24d->313cb397), 1 test(s) red, e.g. ['refuses fewer than 8 questions without a stated reason']; restored: Truerefusal_code_catalogue: mutation landed (blob 47f63c7d->163daac4), 1 test(s) red, e.g. ['refuses a refusal code outside the platform error-code catalogue']; restored: TrueDerived gates (dispatch-gates --commands, reconciled with --ran)
Tree: objectstack-ai/objectstack at c1affd3 (merge base f506b79).
dispatch-gates --ranreported92 derived famil(ies) accounted for — 88 run, 4 NOT-MEASURED (3 DERIVED from a recorded exit 3, 1 claimed)and exited 0.check:cross-package-test-inputs,check:ci-filter-parity,check:pm-dispatch-gates,check:test-source-alias,check:type-check-coverage,check:undeclared-dep-imports,check:doc-authoring,check:issue-citations,check:nul-bytes,check:changeset-fixed,check-changeset-no-majorandcheck-empty-changeset.check:error-code-casing(ADR-0112 D1) was red in CI on c3a257d because the refusal test used a lowercase literal. Reproduced locally withpnpm check:error-code-casing: exit 1,packages/qa/scenarios/test/refusals.test.ts:83 'validation_error' (assignment). On c1affd3 it exits 0 (no unlisted lowercase error codes in 11258 scanned file(s)). The fix is a closed vocabulary rather than an exemption: the refusal-code field is now the spec'sErrorCode, and the test probes a SCREAMING code outside the catalogue (VALIDATION_BROKEN).dist/of packages this diff does not build):check:dual-build-cjs-loads,check:lean-entry-closure,@objectstack/lint check:doc-formula-expressions.check:type-check-debt.Generated by Claude Code