Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .changeset/30429-seed-receipt-landed-on-throw.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
'@objectstack/spec': minor
'@objectstack/metadata-protocol': minor
'@objectstack/runtime': patch
---

fix(metadata-protocol,runtime): a seed load that throws part-way still reports `seedApplied.landed` for the datasets that finished before the throw

Clause-②: yes (widening: a thrown load's receipt carries `landed` for the datasets that finished, where it carried none)
Metadata-protocol: semantic

- **`@objectstack/spec`: `seedApplied.landed`** (both publish responses) now says what it covers on a load that threw part-way (`success: false` with `error`): only the datasets that FINISHED before the throw. The dataset that threw is not listed (its rows are in `inserted` / `updated`, its object in `error`). Absent `landed` now means the apply failed before any load ran, or the producer reports no ids; a throw before any dataset finished reports `landed: []`.
- **`@objectstack/metadata-protocol`**: `SeedLoadPartialWrites` gains `finished`, the finished datasets' `object` / `landedIds` / `insertedIds` in load order, and `applySeedBodies`' catch folds it through the same `seedAppliedLanded` projection (same two lists, same 200 cap) the returned path uses. `seedLoadPartialWrites` and `SeedLoadPartialWrites` are exported from the package root.
- **`@objectstack/runtime`**: the package-publish door's fallback seed apply reads the same `seedLoadPartialWrites` off a thrown load and reports the same `landed`, plus the `inserted` / `updated` counts the response schema requires (it reported neither before) and the object whose seed stopped the load in `error`, as `metadata-protocol`'s apply does.
- Why the dataset that threw is left out: its ids are only partly known when it throws. A batched insert whose response was lost may have committed rows whose ids never came back, so a list of the ids the loader did see would read as complete while missing rows.

Supersedes, in the same release: `.changeset/30348-seed-receipt-landed-ids.md` says absent `landed` means "not reported (a load that threw, or a producer that reports no ids)". After this change a load that threw part-way reports `landed` for its finished datasets; absent now means no load ran, or a producer that reports no ids.
4 changes: 2 additions & 2 deletions content/docs/references/api/protocol.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2546,7 +2546,7 @@ The authoring surface of a metadata write. Closed: `copilot` is an AI assistant'
| **updated** | `integer` | ✅ | Rows updated by the externalId-keyed upsert. |
| **error** | `string` | optional | Single failure message, present when the seed apply threw before the loader ran (including "no readable seed bodies"). |
| **errors** | `any[]` | optional | Per-record failures reported by the seed loader. Present only when the loader ran and returned a non-empty error list. |
| **landed** | `{ object: string; inserted: object; matched: object }[]` | optional | Per object, the stored rows the seed landed on: `inserted` (created by this publish) and `matched` (pre-existing rows a seed key matched), each capped at `SEED_APPLIED_LANDED_IDS_CAP` (200) ids with `total` and `truncated` stating any cut. One entry per object the loader ran, empty lists included. ABSENT means "not reported" — the apply failed before or while loading (see `error`), or the producer reports no ids — never "no rows landed". |
| **landed** | `{ object: string; inserted: object; matched: object }[]` | optional | Per object, the stored rows the seed landed on: `inserted` (created by this publish) and `matched` (pre-existing rows a seed key matched), each capped at `SEED_APPLIED_LANDED_IDS_CAP` (200) ids with `total` and `truncated` stating any cut. One entry per object the loader ran, empty lists included. When the load threw part-way (`success: false` with `error`), it lists only the datasets that FINISHED before the throw, and is EMPTY when none had; the dataset that threw is not listed (its rows are in the counts, its object in `error`). ABSENT means "not reported" — the apply failed before any load ran (see `error`), or the producer reports no ids — never "no rows landed". |

### Nested Shape: `PublishMetaItemResponse.materializeApplied`

Expand Down Expand Up @@ -2647,7 +2647,7 @@ The authoring surface of a metadata write. Closed: `copilot` is an AI assistant'
| **error** | `string` | optional | Single failure message, present when the apply failed before the loader ran (including "no readable seed bodies"). When the failure is the seed bodies' own schema refusal, this is a one-sentence headline and the per-path detail rides `issues[]`. |
| **errors** | `any[]` | optional | Per-record failures reported by the seed loader, plus any seed-body read failures. May be present and empty on a clean load. |
| **issues** | `{ path: string; message: string; code?: string }[]` | optional | Structured spec-validation findings behind `error`, present when the apply was refused by the seed bodies' own schema — the declared 422 `seedRequestValidationError` mints. The per-path detail lives HERE, once; `error` stays a one-sentence headline. Absent on non-validation failures (driver faults, unreadable bodies), whose whole story is `error` / `errors[]`. |
| **landed** | `{ object: string; inserted: object; matched: object }[]` | optional | Per object, the stored rows the seed landed on: `inserted` (created by this publish) and `matched` (pre-existing rows a seed key matched), each capped at `SEED_APPLIED_LANDED_IDS_CAP` (200) ids with `total` and `truncated` stating any cut. One entry per object the loader ran, empty lists included. ABSENT means "not reported" — the apply failed before or while loading (see `error`), or the producer reports no ids — never "no rows landed". |
| **landed** | `{ object: string; inserted: object; matched: object }[]` | optional | Per object, the stored rows the seed landed on: `inserted` (created by this publish) and `matched` (pre-existing rows a seed key matched), each capped at `SEED_APPLIED_LANDED_IDS_CAP` (200) ids with `total` and `truncated` stating any cut. One entry per object the loader ran, empty lists included. When the load threw part-way (`success: false` with `error`), it lists only the datasets that FINISHED before the throw, and is EMPTY when none had; the dataset that threw is not listed (its rows are in the counts, its object in `error`). ABSENT means "not reported" — the apply failed before any load ran (see `error`), or the producer reports no ids — never "no rows landed". |

### Nested Shape: `PublishPackageDraftsResponse.materializeApplied`

Expand Down
9 changes: 8 additions & 1 deletion packages/metadata-protocol/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,14 @@ export {
} from './migrations/recorded-by-sentinel.js';
export type { SentinelHistoryRow } from './migrations/recorded-by-sentinel.js';

export { SeedLoaderService, type SeedLoaderServiceOptions } from './seed-loader.js';
export {
SeedLoaderService,
type SeedLoaderServiceOptions,
// [#30429] The runtime's fallback seed apply reads a thrown load's finished
// datasets through it, so its receipt reports `landed` as this package's does.
seedLoadPartialWrites,
type SeedLoadPartialWrites,
} from './seed-loader.js';
export { runBuildProbes } from './build-probes.js';
export type {
RuntimeBuildIssue,
Expand Down
22 changes: 16 additions & 6 deletions packages/metadata-protocol/src/protocol-publish-batch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -134,9 +134,8 @@ export async function applySeedBodies(
// [#30348] Which rows landed, per object: created by this publish
// vs pre-existing rows a seed key matched — the loader's own ids,
// capped and folded by the one projection both receipt producers
// share. Only on this path: a load that THREW (the catch below)
// reports counts but no ids, so `landed` stays absent there —
// "not reported", never "nothing landed".
// share. [#30429] A load that THREW reports the same key from the
// catch below, for the datasets that finished.
landed: seedAppliedLanded(r.results),
};
} catch (e: any) {
Expand All @@ -156,9 +155,11 @@ export async function applySeedBodies(
// The counters describe what was WRITTEN. The loader writes dataset
// by dataset and stops at the one that throws; the rows of the
// datasets ahead of it stay written (nothing rolls them back), so the
// receipt carries the loader's own count of them and names the seed
// that stopped it. No reading means the throw came before the loader
// wrote anything — the request parse, or the loader's import.
// receipt carries the loader's own count of them (plus any rows the
// dataset that threw wrote first) and names the seed that stopped
// it. No reading means no load ran — the request parse refused, or
// the loader's import failed — so nothing was written; a reading
// with zero counts is a load that threw before writing a row.
const written = partialWritesOf?.(e);
const failure = clientFacingFailureText(e, 'seed apply failed');
return {
Expand All @@ -168,6 +169,15 @@ export async function applySeedBodies(
error: written?.failedObject
? `seed apply stopped at the seed for object '${written.failedObject}': ${failure}`
: failure,
// [#30429] The rows of the datasets that FINISHED before the
// throw, through the same projection as the returned path — the
// ids a reader clearing a partial publish's sample data needs. The
// dataset that threw is not listed (its ids are only partly known,
// see `SeedLoadPartialWrites.finished`); its rows are in the
// counts above and its object in `error`. Empty when the throw
// came before any dataset finished; absent when there is no
// reading (no load ran, see above): "not reported".
...(written ? { landed: seedAppliedLanded(written.finished) } : {}),
// [#10524] A DECLARED refusal's structured findings ride the
// receipt beside the headline `error` — the message is a
// one-sentence headline now, so this is where the per-key
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.

import { describe, it, expect, vi } from 'vitest';
import { SeedLoaderService } from './seed-loader.js';
import { SeedLoaderService, seedLoadPartialWrites } from './seed-loader.js';
import { seedAppliedLanded } from './seed-applied-landed.js';
import type { IDataEngine, IMetadataService } from '@objectstack/spec/contracts';
import { assertEngineDeleteDispatch, assertEngineUpdateDispatch, assertEngineFindOnePredicate } from '@objectstack/metadata-core';

Expand Down Expand Up @@ -300,3 +301,58 @@ describe('seed deferred back-fill failure is reported, not swallowed (framework#
expect(result.summary.totalErrored).toBe(0);
});
});

/**
* [#30429] A load that throws in PASS 2 — after every dataset finished — is
* still a thrown load, and its partial-writes reading lists every dataset as
* finished: no `failedObject`, and the receipt's `landed` (the same
* `seedAppliedLanded` projection the publish catch uses) names every row the
* load wrote.
*
* Pass 2 guards its own driver traffic (a failed back-fill write is counted,
* a failed lookup probe is a miss), so the throw is staged at the one call in
* it that is not guarded: the `logger.error` its back-fill failure reports
* through — a log transport that fails while pass 2 reports. What is pinned is
* the loader's state at that point, not the transport.
*/
describe('a load that throws in pass 2 reports every dataset as finished (#30429)', () => {
it('finished lists both datasets with their store ids; no failed object; landed names every row', async () => {
const { engine, store } = createFaithfulEngine();
const metadata = createMetadata();
const realUpdate = (engine.update as any).getMockImplementation();
(engine.update as any).mockImplementation(async (obj: string, data: any, opts: any) => {
if (obj === 'audit_department') throw new Error('fetch failed');
return realUpdate(obj, data, opts);
});
const logger = createLogger();
logger.error.mockImplementation(() => { throw new Error('the log transport is down'); });

const err = await new SeedLoaderService(engine, metadata, logger)
.load({ seeds: SEEDS, config: CONFIG })
.then(() => undefined, (e: unknown) => e);

expect((err as Error)?.message).toBe('the log transport is down');
// Positive control: the throw came from pass 2's back-fill report, after
// both datasets wrote their row.
expect((engine.update as any).mock.calls.some(([obj]: [string]) => obj === 'audit_department')).toBe(true);
const deptIds = (store.audit_department ?? []).map((r) => String(r.id));
const workerIds = (store.audit_worker ?? []).map((r) => String(r.id));
expect(deptIds).toHaveLength(1);
expect(workerIds).toHaveLength(1);
const partial = seedLoadPartialWrites(err)!;
expect(partial).toEqual({
inserted: 2,
updated: 0,
finished: [
{ object: 'audit_department', landedIds: deptIds, insertedIds: deptIds },
{ object: 'audit_worker', landedIds: workerIds, insertedIds: workerIds },
],
});
expect('failedObject' in partial).toBe(false);
const list = (ids: string[]) => ({ ids, total: ids.length, truncated: false });
expect(seedAppliedLanded(partial.finished)).toEqual([
{ object: 'audit_department', inserted: list(deptIds), matched: list([]) },
{ object: 'audit_worker', inserted: list(workerIds), matched: list([]) },
]);
});
});
12 changes: 10 additions & 2 deletions packages/metadata-protocol/src/seed-loader-seed-files.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -558,7 +558,15 @@ describe('seedLoadPartialWrites: the rows a thrown load had written', () => {
expect((err as Error).message).toBe('the storage service is down');
expect((store.pw_tag ?? []).map((r) => r.name).sort()).toEqual(['T1', 'T2']);
expect((store.pw_doc ?? []).map((r) => r.code)).toEqual(['D1']);
expect(seedLoadPartialWrites(err)).toEqual({ inserted: 3, updated: 0, failedObject: 'pw_doc' });
// [#30429] The finished dataset (`pw_tag`) carries its landed ids — the
// store's own; the dataset that threw (`pw_doc`) is in the counts but not
// in `finished`, although its row D1 was written before the throw.
const tagIds = (store.pw_tag ?? []).map((r) => String(r.id));
expect(tagIds).toHaveLength(2);
expect(seedLoadPartialWrites(err)).toEqual({
inserted: 3, updated: 0, failedObject: 'pw_doc',
finished: [{ object: 'pw_tag', landedIds: tagIds, insertedIds: tagIds }],
});
});

it('a load that throws before any dataset wrote: zero, and no object named', async () => {
Expand All @@ -569,7 +577,7 @@ describe('seedLoadPartialWrites: the rows a thrown load had written', () => {

expect((err as Error).message).toBe('metadata store unavailable');
expect(store).toEqual({});
expect(seedLoadPartialWrites(err)).toEqual({ inserted: 0, updated: 0 });
expect(seedLoadPartialWrites(err)).toEqual({ inserted: 0, updated: 0, finished: [] });
});

it('an error that did not come out of a load carries no reading', () => {
Expand Down
19 changes: 19 additions & 0 deletions packages/metadata-protocol/src/seed-loader.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ import { isMissingTableError } from '@objectstack/metadata/errors';
// [#25779] `seedFile(path)` values: stored, kept across replays, and owned by
// the row that holds them; an id-shaped file value that names no file refused.
import { SeedDatasetFiles, type SeedFileClaims } from './seed-files.js';
import type { SeedLandingResult } from './seed-applied-landed.js';

interface Logger {
info(message: string, meta?: Record<string, any>): void;
Expand Down Expand Up @@ -581,6 +582,21 @@ export interface SeedLoadPartialWrites {
readonly updated: number;
/** The object of the dataset that was loading when the load threw; absent when it threw outside any dataset. */
readonly failedObject?: string;
/**
* [#30429] The datasets that FINISHED before the throw, in load order, each
* with the row ids it landed on (`landedIds` / `insertedIds`, exactly as its
* returned result carries them) — what the publish receipt projects into
* `seedApplied.landed` with the same `seedAppliedLanded` a returned load
* uses. Empty when the throw came before any dataset finished.
*
* The dataset that threw is NOT here, although its rows are in the counts
* above: its ids are only partly known when it throws. A batched insert
* whose response was lost may have committed rows whose ids never came back
* (the loader re-reads by key on its retry for exactly that reason), so a
* list of the ids it did see would read as complete while missing rows. A
* finished dataset's list is complete; that is the line this field draws.
*/
readonly finished: readonly SeedLandingResult[];
}

/** The writes a thrown load had made, keyed by the error it threw. */
Expand Down Expand Up @@ -815,6 +831,9 @@ export class SeedLoaderService implements ISeedLoaderService {
inserted: finished.reduce((sum, r) => sum + r.inserted, 0) + (loading?.written.inserted ?? 0),
updated: finished.reduce((sum, r) => sum + r.updated, 0) + (loading?.written.updated ?? 0),
...(loading ? { failedObject: loading.object } : {}),
// A snapshot of the ids alone: the results array is the live one pass 2
// writes its counters into.
finished: finished.map((r) => ({ object: r.object, landedIds: r.landedIds, insertedIds: r.insertedIds })),
};
const carrier = err !== null && typeof err === 'object' ? err : new Error(String(err));
partialWritesByError.set(carrier, partial);
Expand Down
Loading
Loading