Repository navigation
Reset phase 2A: triage the 144 quarantined Bun files - #149
Merged
Merged
Conversation
…ives later pushes Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…d failed test:prepare bun run test:prepare (and so the nightly job and the SDK source build) stopped at 'Original action-plan producer identity is stale'. Regenerated with bun scripts/actions-plan-identity.ts. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
#143 moved the loader schema generator to the Rust-backed root and left src/domain-packs/loader.ts as declarations only, so the static validators build failed with 'Export named generateDomainPackLoaderSchema not found' (tests/domain-packs/schema-browser-embedding.test.ts). The same import scripts/build-domain-pack-loaders.ts uses. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…d-ideals/nullability-ideals already do 2b32fe1 ('Assert accessor refusal in both validation entry points', in #144/#145) established that the Rust admission may either throw AdmissionError('Accessor properties are not invoked during admission') or return an invalid result. Apply the same helper to the other tests that pass getters. policy-v2 passes fully; the others move on to later, unrelated assertions and stay quarantined with the new reason. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…int list
tests/actions-manifest.test.ts read src/extensions/actions/{manifests,framed-json}.ts, which #143 deleted
when the Rust port took over; the lint now covers the two modules that remain.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…sible regressions, with reasons - 130 files stay quarantined, each with a one-line reason (a failing assertion or error, tagged with a shared cause). - 8 files move to slow-tests.txt: no assertion failure of their own, they need more time (relationship projection matrices, Arrow IPC, DDD PostgreSQL) or a prepared environment (WASM package, action prerequisites). - 6 files were fixed or already pass (see the previous commits). - Remove the temporary triage step from the ts job. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… skip the known rdf_parity case; drop four dead evidence allocations
The nightly job stopped at test:prepare ('Could not resolve bindings/core/wasm/pkg/umf_core_wasm.js').
The same job then runs build:reference, which needed Go and failed on four evidence-map allocations whose
records phase 1 deleted (uuid, typed-reporting, typed-sdk, actions-certificate: those operations now have no
recorded evidence). cargo test --workspace skips the single known-failing rdf_parity case.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The nightly job's build:docs step refuses stale excerpts; refreshed with bun scripts/actions-docs/build.ts --refresh (the example imports from src/index-node-v4, the V4 root since #143; the public-types list follows the current exports). Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…e: no links to deleted evidence - build:docs failed on 'Missing repository link ../../evidence/...' in support-and-evidence.md. The page cited qualification counts from the deleted evidence; it now states what CI tests and points at known-failures.txt. Dead links to removed files in the guide pages became plain text. - The nightly job's suites (Bun, quarantined Bun, browsers, SDK package, cargo test) run with !cancelled(), so one failing suite no longer hides the others. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… addon (120 s limit) Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…orkflow_call guard The last nightly run got through setup, WASM, test:prepare, typecheck and the reference/guide build (first time), then hit the 150 min limit inside the full Bun inventory. The inventory keeps its single job with a 6 h limit; the quarantined files (informational) run as four parallel shards from the cached addon. The workflow_call trigger and the github.workflow guard are the same lines release.yml's pull request adds, so the two merge cleanly. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Phase 2A of the reset: triage the 144 Bun test files that #143 (the Rust port, merged with CI skipped) left failing on a clean Linux run and quarantined in
scripts/known-failures.txt. Every file was run, locally on macOS (all 144) and on Linux in CI (78 files before the run was cancelled to free runners). The two agree on every one of those 78 files, so the macOS run is used for the rest.Result:
known-failures.txtgoes from 144 entries to 130, and every remaining entry now has a one-line reason (a failing assertion or error, tagged with the likely shared cause). The 130 are possible regressions of public behaviour (see "Findings" below), not stale or environment-bound tests.Category table
tests/actions-manifest.test.ts"portable modules contain no host-only imports" iterated oversrc/extensions/actions/{manifests,framed-json}.ts, which #143 deleted. Its list now names the two modules that remain (that case passes). No test file was deleted: nothing else was obsolete with confidence, and the brief says not to delete behaviour tests to get green.scripts/slow-tests.txt(nightly runs them aftertest:prepare/build:reference):arrow/ipc,core-ideals/relationship-{extras,postgresql,tablespec-projection},projections/ddd-postgresql(file killed after 5-10 min, no assertion failure of their own),binding/postgresql-indexes(only failure is its 60 s timeout),domain-packs/download(needs the browser WASM package),actions-reference/process-recovery(needs the action test prerequisites). Whether they pass in nightly is reported below.core/field-ideals,core/nullability-ideals: stale quarantine, they pass (accessor refusal was fixed by 2b32fe1 in #144/#145 after they were listed).domain-packs/schema-browser-embedding: real source bug, fixed.binding/stable-ids,security/evaluate,security/protection/policy-v2: pass after the accessor-refusal helper (below).Fixes (each cites why the new behaviour is the intended one)
docs/helix/05-deploy/schema-browser/static-validators.tsimportedgenerateDomainPackLoaderSchemafromsrc/domain-packs/loader, which Complete the Rust port and prepare UMF 0.10.11 [skip ci] #143 turned into declarations only (moved to the Rust-backed root). Now imports fromsrc/index-node-v4, likescripts/build-domain-pack-loaders.ts. This broke the schema-browser build, not only a test.AdmissionError('Accessor properties are not invoked during admission')or return an invalid result, and addedexpectAccessorRefusalforfield-ideals/nullability-ideals. Applied the same helper (plusexpectAccessorRefusalByfor APIs whose refusal is a status) inbinding-uuid,binding/stable-ids,core/key-ideals,core/relationship-ideal,security/evaluate,security/protection/{domains,policy-v2}. Three files pass; the others move on to later, unrelated assertions and keep their quarantine with the new reason.src/extensions/actions/plan-identity.tswas stale on main (Original action-plan producer identity is stale), sobun run test:prepare, the nightly job and the SDK source build all stopped there. Regenerated withbun scripts/actions-plan-identity.ts..github/workflows/ci.yml: separate concurrency group per trigger, so a nightly/manual run is not cancelled by a later push to the branch.Findings: possible real regressions (please read)
Roughly by shared cause (counts are files, approximate where a file has several causes):
tests/core-ideals/cardinality-parquet-projection.test.ts:crates/umf-core/src/cardinality_parquet_projection.rs:610callsOption::unwrap()onNone; a panic across the addon boundary aborts Bun (exit 134), taking the caller's whole process down.core/core.test.ts: the YAML writer emits a block scalar without an indentation indicator, so strings with leading spaces read back changed (writeJsonValue({c:' lead\n second\n '},'yaml')reads back aslead\n second). The same cluster shows aswrongly indented line in block scalarwhen the PostgreSQL catalog tests read their own YAML (3 files), pluscore/native-json(YAML 1.1 underscored integers come back quoted) andjson-schema/native.binding-uuid(an unknown logical facet no longer makesinspectBinding(...).completefalse),actions-manifest(validateBulkOutcomeaccepts a 129-emoji key module),security/protection/domains(accepts a 4 MB closure document instead ofPROTECTION_DOMAIN_BOUND),binding/table-operations(noTABLES_OPERATION_GENERATEDetc. diagnostics for invalid declarations),actions/interpreter(strict reference admission no longer throws).umf.avroextension reportsSEMANTICS_UNCHECKED(warning), which makesinspectAvro(...).completefalse for ordinary schemas and makes tests that treat warnings as errors throw (8 files incl.avro/bundle,avro/corpus,core-ideals/relationship-avro,facets-avro).Number outside the core interoperable numeric profile/Parsing would change the decimal value(7 files:core/javascript-numeric,core/facet-ideals,jsonld/from-rdf,openapi/extraction,core-ideals/key-sqlserver, ...).JSON nesting exceeds the admission limit,Source structural limit exceeded,Serialized text limit exceeded,literal text budget exceededthrown where tests expect a result (about 13 files).Expected "blocked"/"refused", got "projected"/"resolved"/"evaluated") in about 9 files (key-avro-projection,tablespec/avro-integers,linkml/slot-*,shacl/lists,security/protection/{disclosure,query-properties},security/write); acceptance became refusal (Expected "candidate"/"compiled"/"transformed", got "blocked"/"refused") in about 5 (jsonld/legacy,generalized-rdf/dataset,lifecycle/g01-fill,lifecycle/g11-field-replacement,security/protection/functions).tests/sqlserver/indexes.test.tsrewritesfixtures/sqlserver/index-projection.json; the new output has a stray newline inside a diagnostic string (Key "PK_Items"\n (PK) is not enforced...) and reordersnativeSchema.fixtures/postgresql/ddl-declarations.jsonis likewise rewritten with different content bypostgresql/declarations.test.ts.g_15vsg_14) are not reset per case (owl/list-axioms,owl/declarations,rdf/turtle); this is the same cause as the Rustrdf_paritycase already excluded from nextest.Also found: 50 test files overwrite tracked fixtures (
Bun.write('fixtures/...')) every time they run, whether or not they pass; running a quarantined file dirties the checkout. Not changed here.CI and timing
PR CI on the final commit (run 43157274708): all 37 jobs green (size-guard, native-addon, rust-build, rust 1-8, ts 1-24, python 3.11/3.14; nightly skipped by design). An earlier run on the previous commit failed only
ts (7):security/protection/policy-v2passes but one of its tests takes 132 s on the PR job's addon (120 s limit), so it moved toslow-tests.txt.PR job: the 24
tsshards ran 13.4 min median, 15.2 min max in the final run (about 335 job-minutes); python 10.0 / 13.0 min; rust partitions 1.9-8.5 min after a 5.6 min build; about 4-5 min of each is setup (apt,playwright install). I did not change the shard count: more shards only lengthen the runner queue, which dominates wall time on this org (the nightly dispatch sat queued for hours). Nothing was moved out of the PR job to hit a target except the 8 files above.Nightly (
gh workflow run ci.yml --ref reset/phase2-tests;ci.ymlexists on main, so dispatch is allowed). It had never completed. Dispatches so far, each fixing what the previous one hit: (1)Original action-plan producer identity is staleatbun run test:prepare(regenerated); (2)Could not resolve bindings/core/wasm/pkg/umf_core_wasm.js(the WASM package is now built first, Chromium and Go installed early); (3)build:referencefailed on four evidence-map allocations whose records phase 1 deleted, thenbuild:docson stale Actions-guide excerpts and a guide page linking to deleted evidence (fixed here); (4) setup, WASM,test:prepare(the SDK source build, PostgreSQL WASM, smithy),typecheck,build:referenceandbuild:docsall passed for the first time, then the full Bun inventory hit the 150 minute job limit. Now: the limit is 360 minutes, the quarantined files run as four parallel informational shards (quarantine), every nightly suite runs with!cancelled()so one failure does not hide the rest, andcargo test --workspaceskips the single knownrdf_paritycase. Dispatch 5 (run 43166980553) was still running when this was written: the quarantine shards had mostly finished, the Bun inventory, browser suites, SDK package test andcargo test --workspacehad not. Its result is not known yet; whether the 8 files moved toslow-tests.txtpass there is part of that result.Not in this PR
Fixing the 130 behaviours; deleting any behaviour test; changing shard counts; removing the fixture-writing from the tests; history rewrite, tags, releases.
🤖 Generated with Claude Code