diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cf43e38..9f955dc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -26,6 +26,7 @@ that depends on another. Build first and it goes away — nothing is broken. | Path | What | How it's tested | | --- | --- | --- | | `packages/validate` | Event/Feed validation against the OTE JSON Schema | vitest + fixtures | +| `packages/migrate` | Moves a document to a newer spec version + D032 warnings | vitest + fixtures | | `packages/build-feed` | Assembles `events/*.json` + `ote.config.json` → `feed.json` | vitest + fixtures | | `packages/export-ics` / `export-rss` / `export-jsonld` | `feed.json` → ICS / RSS / schema.org JSON-LD | vitest + fixtures | | `apps/editor` | Static web editor (form → event JSON → issue/PR links) | vitest for `src/lib/`; UI by hand | diff --git a/README.md b/README.md index 4e847ff..0109103 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,8 @@ Design rationale lives in [DESIGN.md](DESIGN.md); the spec lives in | Package | What it does | | --- | --- | -| [`@opentechevents/validate`](packages/validate/) | Validates OTE Event/Feed documents against the vendored v0.4 JSON Schema. | +| [`@opentechevents/validate`](packages/validate/) | Validates OTE Event/Feed documents against the version each one declares. Every published spec version is embedded. | +| [`@opentechevents/migrate`](packages/migrate/) | Moves a document to a newer spec version, and warns where D032's multi-part pricing change silently altered what it means. | | [`@opentechevents/export-ics`](packages/export-ics/) | Valid OTE Feed → iCalendar (RFC 5545). | | [`@opentechevents/export-rss`](packages/export-rss/) | Valid OTE Feed → RSS 2.0. | | [`@opentechevents/export-jsonld`](packages/export-jsonld/) | Valid OTE Feed/Event → schema.org `Event` JSON-LD (SEO structured data). | diff --git a/packages/migrate/CHANGELOG.md b/packages/migrate/CHANGELOG.md new file mode 100644 index 0000000..b3f307f --- /dev/null +++ b/packages/migrate/CHANGELOG.md @@ -0,0 +1,35 @@ +# Changelog + +All notable changes to `@opentechevents/migrate` are documented here. + +## 0.1.0 + +First release. Closes the last open part of #97. + +- **`migrateDocument(json, options?)` and `ote-migrate`** move an OTE feed or + standalone event to a newer spec version. The JSON change is `specVersion` + and nothing else — every published release so far has been additive — but + the result is validated against the target version rather than assumed, so + a future release that breaks that pattern fails here instead of shipping. +- **D032 warnings, which are the reason this is a package and not a `sed`.** + Before 0.5 a part of a multi-part event carried its own complete `offers`; + from 0.5 an unscoped offer is read as `admission` and a surcharge needs + `"scope": "add-on"` plus `partOf.offers`. The old document stays valid + while quietly under-reporting what attending costs, so the migration says + which events need a human and what the question is. It never rewrites + pricing: whether a number is the whole price or an extra is a fact about + the event, not about the JSON. +- Warnings never change the exit code. They are decisions, not faults. +- **Documents outside the support window migrate.** Validation here is + against the version the document declares, deliberately bypassing the + support policy that `validateDocument` applies — a feed on an unsupported + release is precisely the one that has to run this, and a migrator that + refused it would be useless to the only people who need it. A registered + adopter was on 0.3 when 0.3 left the window. +- Refuses, rather than guessing: a missing or never-published `specVersion`, + an unpublished target, a **downgrade** (dropping fields a newer version + added is silent data loss), and input that is already invalid at the + version it claims. +- Output is written with two-space indent and a trailing newline — what OTE + documents in this ecosystem already use, so migrating one does not show up + as a whole-file reformat in the next diff. diff --git a/packages/migrate/README.md b/packages/migrate/README.md new file mode 100644 index 0000000..6593d52 --- /dev/null +++ b/packages/migrate/README.md @@ -0,0 +1,123 @@ +# @opentechevents/migrate + +Moves an OTE document to a newer spec version — and tells you the one thing a +find-and-replace gets wrong. + +```sh +npx @opentechevents/migrate feed.json --write +``` + +## Why this is not `sed` + +What it does to your JSON really is small: it rewrites `specVersion`. Every +published OTE release so far has been additive, so no valid document has ever +needed a structural change to move up. `migrateDocument` checks that rather +than assuming it — the output is validated against the target version before +you get it back. + +The reason this is a package is **D032**, the multi-part pricing change in +0.5. + +Before 0.5, a part of a multi-part event carried its own complete `offers`. +That was the only way to price it; there was no way to say *"this workshop +costs 60 on top of the conference ticket"*. From 0.5 there is: the set's +shared admission goes in `partOf.offers`, and a surcharge is an offer with +`"scope": "add-on"`. + +The trap is that the old document stays **valid**. An offer with no `scope` is +read as `admission` — a complete way to attend that part. So a workshop that +really costs 240 + 60 goes on publishing, validly and machine-readably, that +it costs 60. Nothing can flag that automatically: whether the 60 is the whole +price or an extra is a fact about your event, not about the JSON. + +So `ote-migrate` asks you: + +```text +✓ feed.json — OTE 0.4.0 → 0.6.0 + + ! events[1].offers (D032) + This event ("Taller de testing en Python") is part of a multi-part set and + carries its own offers. From 0.5 an offer without a "scope" is read as + "admission" — a complete way to attend this part. If any of these prices is + really a surcharge on top of the set's ticket, mark it "scope": "add-on" + and put the shared admission in partOf.offers, or the feed will + under-report what attending costs. + + 3 things to check by hand. The document is valid either way — these are + about what it means. +``` + +Warnings never change the exit code. They are decisions, not faults. + +## It works on versions that are out of support + +The support window covers the last three minors. A document older than that +gets an error from `ote-validate` — correctly, because a consumer built for +the window cannot read it: + +```sh +$ ote-validate feed-0.3.json +✗ feed-0.3.json (feed) — OTE 0.3.0 + specVersion: OTE Spec 0.3.0 is outside the support window … migrate to 0.6.0. +``` + +That document is exactly the one that needs migrating, so this tool validates +against the version the document declares and ignores the window: + +```sh +$ ote-migrate feed-0.3.json --write +✓ feed-0.3.json — OTE 0.3.0 → 0.6.0 +``` + +## CLI + +```text +Usage: ote-migrate [output.json] [options] + + --to Target spec version (default: the newest published one) + --kind feed | event (default: inferred from the document) + --write Rewrite the input file in place + --versions List the published versions and exit +``` + +With no output path and no `--write`, the document goes to stdout and +everything else to stderr, so it pipes cleanly: + +```sh +ote-migrate feed.json | jq .specVersion +``` + +Exit codes: `0` migrated · `1` cannot migrate (unplaceable version, a +downgrade, or input that is invalid at the version it claims) · `2` usage or +I/O error. + +## Library + +```ts +import { migrateDocument } from "@opentechevents/migrate"; + +const { document, from, to, warnings } = await migrateDocument(feed, { + to: "0.6.0", +}); +``` + +`migrateDocument(json, options?)` returns `{ kind, from, to, changed, +document, warnings }`. The input is never mutated. It throws `MigrationError` +when the migration cannot be attempted: + +- the document declares no `specVersion`, or one that was never published — + there are no rules to migrate it from; +- the target is not a published version; +- the target is **older** than the declared version. Migrating downwards means + dropping fields a newer version added, and silently losing data somebody + wrote on purpose is worse than refusing; +- the input is not valid at the version it claims. Migrating an invalid + document produces an invalid one wearing a newer label, and a later reader + has no way to tell the damage came from here. + +## What it will not do + +It does not rewrite your pricing, invent `partOf.offers`, or guess a `scope`. +The repo-wide rule that connectors never invent data applies here too, and +D032 is the case where guessing would be most tempting and most expensive — +a wrong price is worse than an absent one. diff --git a/packages/migrate/fixtures/feed-0.3.json b/packages/migrate/fixtures/feed-0.3.json new file mode 100644 index 0000000..edd5c74 --- /dev/null +++ b/packages/migrate/fixtures/feed-0.3.json @@ -0,0 +1,305 @@ +{ + "specVersion": "0.3.0", + "title": "Eventos de CoruñaJUG", + "description": "Charlas y meetups de la comunidad Coruña Java User Group.", + "url": "https://www.corunajug.org", + "textLanguage": "es", + "organizers": [ + { + "name": "Coruña JUG", + "url": "https://corunajug.org" + } + ], + "license": "CC-BY-4.0", + "licenseUrl": "https://creativecommons.org/licenses/by/4.0/", + "updatedAt": "2026-08-28T00:00:00Z", + "events": [ + { + "id": "https://www.corunajug.org/events/ddd-jvm-e-ia-modelizando-value-objects-con-el-minimo-esfuerzo-y-el-maximo-2025-09-24", + "url": "https://www.eventbrite.es/e/entradas-ddd-jvm-e-ia-modelizando-value-objects-con-el-minimo-esfuerzo-y-el-maximo-1684478674059", + "name": "DDD, JVM e IA: modelizando Value Objects con el mínimo esfuerzo y el máximo", + "image": [ + "https://corunajug.org/img/talks/ddd-vo.jpg" + ], + "startDate": "2025-09-24T19:30", + "endDate": "2025-09-24T21:30", + "timezone": "Europe/Madrid", + "attendanceMode": "in-person", + "location": { + "venue": "Sngular", + "address": { + "street": "Rúa Sor Joaquina, 6", + "locality": "A Coruña", + "region": "Galicia", + "postalCode": "15011", + "country": "ES" + } + }, + "tags": [ + "ddd", + "java", + "ia" + ], + "languages": [ + "es" + ], + "updatedAt": "2026-08-28T00:00:00Z" + }, + { + "id": "https://www.corunajug.org/events/mama-quiero-ser-ingeniero-agentico-estrategias-atemporales-para-sobrevivir-en-un-mundo-ia-centrico-2025-10-16", + "url": "https://www.eventbrite.es/e/mama-quiero-ser-ingeniero-agentico-tickets-1766491978059", + "name": "Mamá, quiero ser ingeniero agéntico, estrategias atemporales para sobrevivir en un mundo IA-céntrico", + "image": [ + "https://corunajug.org/img/talks/ing-ai.png" + ], + "startDate": "2025-10-16T19:30", + "endDate": "2025-10-16T21:30", + "timezone": "Europe/Madrid", + "attendanceMode": "in-person", + "location": { + "venue": "Cidade das TIC", + "address": { + "street": "Av. de Pedralonga", + "locality": "A Coruña", + "region": "Galicia", + "postalCode": "15009", + "country": "ES" + } + }, + "tags": [ + "ia", + "agentes" + ], + "languages": [ + "es" + ], + "updatedAt": "2026-08-22T00:00:00Z" + }, + { + "id": "https://www.corunajug.org/events/full-stack-open-space-2025-12-11", + "url": "https://www.eventbrite.es/e/entradas-full-stack-open-space-1974513421797", + "name": "Full Stack Open Space", + "image": [ + "https://corunajug.org/img/talks/full-stack-open-space.jpg" + ], + "startDate": "2025-12-11T19:30", + "endDate": "2025-12-11T21:30", + "timezone": "Europe/Madrid", + "attendanceMode": "in-person", + "location": { + "venue": "Cidade das TIC", + "address": { + "street": "Av. de Pedralonga", + "locality": "A Coruña", + "region": "Galicia", + "postalCode": "15009", + "country": "ES" + } + }, + "tags": [ + "full-stack", + "open-space" + ], + "languages": [ + "es" + ], + "updatedAt": "2026-08-22T00:00:00Z" + }, + { + "id": "https://www.corunajug.org/events/ddd-by-a-dummy-2026-01-29", + "url": "https://www.eventbrite.es/e/dddd-ddd-by-a-dummy-tickets-1980026497555", + "name": "DDD by a Dummy", + "image": [ + "https://corunajug.org/img/talks/DDD-Dummy.jpg" + ], + "startDate": "2026-01-29T19:30", + "endDate": "2026-01-29T21:30", + "timezone": "Europe/Madrid", + "attendanceMode": "in-person", + "location": { + "venue": "Cidade das TIC", + "address": { + "street": "Av. de Pedralonga", + "locality": "A Coruña", + "region": "Galicia", + "postalCode": "15009", + "country": "ES" + } + }, + "tags": [ + "ddd" + ], + "languages": [ + "es" + ], + "updatedAt": "2026-08-22T00:00:00Z" + }, + { + "id": "https://www.corunajug.org/events/novedades-en-java-26-2026-03-25", + "url": "https://www.eventbrite.es/e/novedades-en-java-26-tickets-1984159965870", + "name": "Novedades en java 26", + "image": [ + "https://corunajug.org/img/talks/java26.png" + ], + "startDate": "2026-03-25T19:30", + "endDate": "2026-03-25T21:30", + "timezone": "Europe/Madrid", + "attendanceMode": "in-person", + "location": { + "venue": "Cidade das TIC", + "address": { + "street": "Av. de Pedralonga", + "locality": "A Coruña", + "region": "Galicia", + "postalCode": "15009", + "country": "ES" + } + }, + "tags": [ + "java" + ], + "languages": [ + "es" + ], + "updatedAt": "2026-08-22T00:00:00Z" + }, + { + "id": "https://www.corunajug.org/events/kotlin-para-desarrolladores-java-lo-que-te-estas-perdiendo-2026-04-22", + "url": "https://www.eventbrite.es/e/novedades-en-java-26-tickets-1984159965870", + "name": "Kotlin para desarrolladores Java: lo que te estás perdiendo", + "image": [ + "https://corunajug.org/img/talks/kotlin-para-java.jpeg" + ], + "startDate": "2026-04-22T19:30", + "endDate": "2026-04-22T21:30", + "timezone": "Europe/Madrid", + "attendanceMode": "in-person", + "location": { + "venue": "Cidade das TIC", + "address": { + "street": "Av. de Pedralonga", + "locality": "A Coruña", + "region": "Galicia", + "postalCode": "15009", + "country": "ES" + } + }, + "tags": [ + "kotlin", + "java" + ], + "languages": [ + "es" + ], + "updatedAt": "2026-08-22T00:00:00Z" + }, + { + "id": "https://www.corunajug.org/events/evolucion-scala-2026-05-21", + "url": "https://www.eventbrite.es/e/entradas-evolucion-de-scala-java-haskel-y-rust-1988869469121", + "name": "Evolución de Scala: Java, Haskel y Rust", + "image": [ + "https://corunajug.org/img/talks/evolucion-scala.png" + ], + "startDate": "2026-05-21T19:30", + "endDate": "2026-05-21T21:30", + "timezone": "Europe/Madrid", + "attendanceMode": "in-person", + "location": { + "venue": "Cidade das TIC", + "address": { + "street": "Av. de Pedralonga", + "locality": "A Coruña", + "region": "Galicia", + "postalCode": "15009", + "country": "ES" + } + }, + "tags": [ + "scala", + "java", + "rust", + "haskell" + ], + "languages": [ + "es" + ], + "updatedAt": "2026-08-22T00:00:00Z" + }, + { + "id": "https://www.corunajug.org/events/mesa-final-temporada", + "url": "https://www.eventbrite.es/e/mesa-redonda-de-comunidades-tickets-1991005340570", + "name": "Mesa redonda de comunidades", + "image": [ + "https://corunajug.org/img/talks/mesa-final-temporada.png" + ], + "organizers": [ + { + "name": "Comunidades tech de A Coruña" + } + ], + "startDate": "2026-07-02T19:00", + "endDate": "2026-07-02T21:30", + "timezone": "Europe/Madrid", + "attendanceMode": "in-person", + "location": { + "venue": "Cidade das TIC. Oficinas Odeene", + "address": { + "street": "Av. de Pedralonga", + "locality": "A Coruña", + "region": "Galicia", + "postalCode": "15009", + "country": "ES" + } + }, + "tags": [ + "Coruna_tech" + ], + "languages": [ + "es" + ], + "updatedAt": "2026-08-22T00:00:00Z" + }, + { + "id": "https://www.corunajug.org/events/bienvenida-curso-26-27-2026-09-17", + "url": "https://www.eventbrite.es/e/comezamos-o-curso-2627-tickets-1998924147936", + "name": "Comezamos o curso 26/27", + "description": "Arrincamos o novo curso do Coruña JUG cunha xuntanza de benvida, xunto con Cascarilla Tech.", + "image": [ + "https://corunajug.org/img/talks/comezamos-o-curso-26-27.png" + ], + "organizers": [ + { + "name": "Coruña JUG", + "url": "https://www.corunajug.org" + }, + { + "name": "Cascarilla Tech", + "url": "https://www.cascarillatech.org" + } + ], + "startDate": "2026-09-24T19:30", + "endDate": "2026-09-24T21:30", + "timezone": "Europe/Madrid", + "attendanceMode": "in-person", + "location": { + "venue": "Cidade das TIC", + "address": { + "street": "Av. de Pedralonga", + "locality": "A Coruña", + "region": "Galicia", + "postalCode": "15009", + "country": "ES" + } + }, + "tags": [ + "java", + "coruña jug", + "cascarilla tech" + ], + "languages": [ + "gl" + ], + "updatedAt": "2026-08-28T00:00:00Z" + } + ] +} diff --git a/packages/migrate/fixtures/feed-0.4-multipart.json b/packages/migrate/fixtures/feed-0.4-multipart.json new file mode 100644 index 0000000..04e1cc0 --- /dev/null +++ b/packages/migrate/fixtures/feed-0.4-multipart.json @@ -0,0 +1,89 @@ +{ + "specVersion": "0.4.0", + "title": "TRG Conf 2026", + "description": "Las charlas y talleres de TRG Conf 2026, como documentos independientes.", + "url": "https://trgcon.example", + "textLanguage": "es", + "organizers": [ + { + "name": "TRG Conf", + "url": "https://trgcon.example" + } + ], + "license": "CC-BY-4.0", + "updatedAt": "2026-09-01T10:00:00Z", + "events": [ + { + "id": "https://trgcon.example/2026/talks/keynote", + "url": "https://trgcon.example/2026/talks/keynote", + "name": "Keynote de apertura", + "description": "Charla inaugural.", + "image": [ + "https://trgcon.example/img/keynote.png" + ], + "startDate": "2026-10-14T09:00", + "endDate": "2026-10-14T09:45", + "timezone": "Europe/Madrid", + "attendanceMode": "in-person", + "location": { + "venue": "La Nave, Madrid" + }, + "tags": [ + "tech" + ], + "languages": [ + "es" + ], + "partOf": { + "id": "https://trgcon.example/2026", + "name": "TRG Conf 2026", + "type": "multipart" + }, + "updatedAt": "2026-09-01T09:00:00Z", + "offers": [ + { + "name": "General", + "price": 240, + "currency": "EUR", + "url": "https://trgcon.example/tickets" + } + ] + }, + { + "id": "https://trgcon.example/2026/workshops/testing", + "url": "https://trgcon.example/2026/workshops/testing", + "name": "Taller de testing en Python", + "description": "Taller práctico de tres horas.", + "image": [ + "https://trgcon.example/img/workshop.png" + ], + "startDate": "2026-10-14T09:30", + "endDate": "2026-10-14T12:30", + "timezone": "Europe/Madrid", + "attendanceMode": "in-person", + "location": { + "venue": "La Nave, Madrid" + }, + "tags": [ + "python" + ], + "languages": [ + "es" + ], + "offers": [ + { + "name": "Taller de testing", + "price": 60, + "currency": "EUR", + "url": "https://trgcon.example/tickets#workshops" + } + ], + "partOf": { + "id": "https://trgcon.example/2026", + "name": "TRG Conf 2026", + "type": "multipart" + }, + "updatedAt": "2026-09-01T09:00:00Z" + } + ] +} diff --git a/packages/migrate/package.json b/packages/migrate/package.json new file mode 100644 index 0000000..a86c055 --- /dev/null +++ b/packages/migrate/package.json @@ -0,0 +1,42 @@ +{ + "name": "@opentechevents/migrate", + "version": "0.1.0", + "description": "Moves an OTE document to a newer spec version, and says what a find-and-replace would have got wrong", + "license": "MIT", + "homepage": "https://github.com/OpenTechEvents/ote-tools/tree/main/packages/migrate", + "repository": { + "type": "git", + "url": "git+https://github.com/OpenTechEvents/ote-tools.git", + "directory": "packages/migrate" + }, + "bugs": "https://github.com/OpenTechEvents/ote-tools/issues", + "type": "module", + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "bin": { + "ote-migrate": "./dist/bin.js" + }, + "files": [ + "dist" + ], + "scripts": { + "build": "tsc -p tsconfig.build.json", + "test": "vitest run", + "typecheck": "tsc -p tsconfig.json --noEmit" + }, + "dependencies": { + "@opentechevents/validate": "workspace:*" + }, + "devDependencies": { + "@types/node": "^22.0.0", + "typescript": "^6.0.3", + "vitest": "^5.1.0" + }, + "publishConfig": { + "access": "public" + } +} diff --git a/packages/migrate/src/bin.ts b/packages/migrate/src/bin.ts new file mode 100644 index 0000000..5004af2 --- /dev/null +++ b/packages/migrate/src/bin.ts @@ -0,0 +1,4 @@ +#!/usr/bin/env node +import { runCli } from "./cli.js"; + +process.exitCode = await runCli(process.argv.slice(2)); diff --git a/packages/migrate/src/cli.ts b/packages/migrate/src/cli.ts new file mode 100644 index 0000000..39301c0 --- /dev/null +++ b/packages/migrate/src/cli.ts @@ -0,0 +1,166 @@ +import { readFileSync, writeFileSync } from "node:fs"; +import { resolve } from "node:path"; + +import { LATEST_VERSION, PUBLISHED_VERSIONS } from "@opentechevents/validate"; + +import { migrateDocument, MigrationError, type DocumentKind } from "./index.js"; + +export interface CliIO { + out: (text: string) => void; + err: (line: string) => void; +} + +const USAGE = [ + "Usage: ote-migrate [output.json] [options]", + "", + " --to Target spec version (default: the newest published one)", + " --kind feed | event (default: inferred from the document)", + " --write Rewrite the input file in place", + " --versions List the published versions and exit", + "", + "With no output path and no --write, the migrated document goes to stdout,", + "so warnings and notes are on stderr and the JSON can be piped.", +].join("\n"); + +interface Args { + input?: string; + output?: string; + to?: string; + kind?: DocumentKind; + write: boolean; + versions: boolean; +} + +/** Parses argv, or returns null when it does not make sense. */ +function parseArgs(argv: string[]): Args | null { + const args: Args = { write: false, versions: false }; + const positional: string[] = []; + + for (let i = 0; i < argv.length; i += 1) { + const arg = argv[i]!; + if (arg === "--write") { + args.write = true; + } else if (arg === "--versions") { + args.versions = true; + } else if (arg === "--to") { + const value = argv[++i]; + if (value === undefined) return null; + args.to = value; + } else if (arg === "--kind") { + const value = argv[++i]; + if (value !== "feed" && value !== "event") return null; + args.kind = value; + } else if (arg.startsWith("-")) { + return null; + } else { + positional.push(arg); + } + } + + if (args.versions) return args; + if (positional.length === 0 || positional.length > 2) return null; + args.input = positional[0]; + args.output = positional[1]; + // Writing in place and writing somewhere else are different instructions, + // and obeying one of them silently would be a coin flip over the user's + // file. Refuse instead. + if (args.write && args.output !== undefined) return null; + return args; +} + +/** + * Runs the CLI. Exit codes: + * 0 = migrated (warnings do not change this — they are decisions, not faults) + * 1 = cannot migrate: unplaceable version, a downgrade, or invalid input + * 2 = usage or I/O error. + */ +export async function runCli( + argv: string[], + io: CliIO = { + out: (text) => console.log(text), + err: (line) => console.error(line), + }, +): Promise { + const args = parseArgs(argv); + if (args === null) { + io.err(USAGE); + return 2; + } + if (args.versions) { + io.out(`published: ${PUBLISHED_VERSIONS.join(", ")}`); + io.out(`latest: ${LATEST_VERSION}`); + return 0; + } + + const input = args.input!; + let raw: string; + try { + raw = readFileSync(resolve(input), "utf8"); + } catch { + io.err(`ote-migrate: cannot read "${input}"`); + return 2; + } + + let json: unknown; + try { + json = JSON.parse(raw); + } catch (e) { + io.err(`✗ ${input} — invalid JSON: ${(e as Error).message}`); + return 1; + } + + let result; + try { + result = await migrateDocument(json, { kind: args.kind, to: args.to }); + } catch (e) { + if (e instanceof MigrationError) { + io.err(`✗ ${input} — ${e.message}`); + return 1; + } + throw e; + } + + // Two spaces of indent and a trailing newline: what every OTE document in + // this ecosystem is written with, so migrating one does not show up as a + // whole-file reformat in the publisher's next diff. + const text = `${JSON.stringify(result.document, null, 2)}\n`; + const destination = args.write ? input : args.output; + + if (destination !== undefined) { + try { + writeFileSync(resolve(destination), text, "utf8"); + } catch { + io.err(`ote-migrate: cannot write "${destination}"`); + return 2; + } + io.out( + result.changed + ? `✓ ${destination} — OTE ${result.from} → ${result.to}` + : `✓ ${destination} — already OTE ${result.to}, nothing to change`, + ); + } else { + io.out(text.trimEnd()); + io.err( + result.changed + ? `✓ ${input} — OTE ${result.from} → ${result.to}` + : `✓ ${input} — already OTE ${result.to}, nothing to change`, + ); + } + + // On stderr even when the document went to a file: these are for the person + // running the command, and they are the reason this is not a `sed`. + for (const warning of result.warnings) { + io.err(""); + io.err(` ! ${warning.path} (${warning.decision})`); + io.err(` ${warning.message}`); + } + if (result.warnings.length > 0) { + io.err(""); + io.err( + ` ${result.warnings.length} thing${result.warnings.length === 1 ? "" : "s"} to check by ` + + "hand. The document is valid either way — these are about what it means.", + ); + } + + return 0; +} diff --git a/packages/migrate/src/index.ts b/packages/migrate/src/index.ts new file mode 100644 index 0000000..4de4360 --- /dev/null +++ b/packages/migrate/src/index.ts @@ -0,0 +1,293 @@ +// Moves an OTE document from the version it declares to a newer one. +// +// The honest summary of what this does to the JSON: it rewrites +// `specVersion`. Every published OTE release so far has been additive — +// 0.1→0.2→0.3→0.4 added optional fields, 0.5 added six more, and 0.6 changed +// no schema at all — so no valid document has ever needed a structural +// rewrite to move up. That is checked rather than assumed: `migrateDocument` +// validates the result against the target version before returning it. +// +// So why a package instead of `sed -i s/0.4.0/0.6.0/`? Because 0.5 carries +// D032, and D032 changes what pricing a document already contains *means* +// without changing whether it is valid. A `sed` produces a document that +// passes every check and quietly under-reports the cost of attending. The +// warnings below are the part that cannot be automated, and they are the +// reason this exists. + +import { + declaredSpecVersion, + loadValidators, + LATEST_VERSION, + PUBLISHED_VERSIONS, + type ValidationError, +} from "@opentechevents/validate"; + +/** Feed or event: the two document shapes the spec defines. */ +export type DocumentKind = "feed" | "event"; + +/** + * Something the migration could not decide on the publisher's behalf. + * + * Never a defect in the input and never a reason to stop: a warning means + * the document is valid at the target version but may no longer mean what it + * meant, and only the organizer knows which. `path` points into the document + * so a CLI can say where to look. + */ +export interface MigrationWarning { + /** Where in the document, in the same dotted style validation errors use. */ + path: string; + /** What changed, and what the publisher has to decide. */ + message: string; + /** The spec decision this comes from, for anyone who wants the prose. */ + decision: string; +} + +export interface MigrateOptions { + /** Feed or event. Inferred from the document's shape when omitted. */ + kind?: DocumentKind; + /** Target version. Defaults to the newest published release. */ + to?: string; +} + +export interface MigrationResult { + kind: DocumentKind; + /** The version the input declared. */ + from: string; + /** The version the output declares. */ + to: string; + /** False when the document already declared the target version. */ + changed: boolean; + /** The migrated document. A new object; the input is never mutated. */ + document: unknown; + /** Decisions left to the publisher. Never empty for nothing. */ + warnings: MigrationWarning[]; +} + +/** A migration that could not be attempted at all, with the reason. */ +export class MigrationError extends Error { + constructor(message: string) { + super(message); + this.name = "MigrationError"; + } +} + +// `PUBLISHED_VERSIONS` is a `const` tuple, so its `includes` only accepts a +// member of that tuple — which is useless for checking a version that came +// out of somebody's document. Widened once, here. +const PUBLISHED: readonly string[] = PUBLISHED_VERSIONS; + +/** `"0.4.0"` → `[0, 4, 0]`, for comparing releases rather than strings. */ +function parts(version: string): number[] { + return version.split(".").map((n) => Number.parseInt(n, 10)); +} + +/** True when `a` is strictly older than `b`. */ +function isOlder(a: string, b: string): boolean { + const [x, y] = [parts(a), parts(b)]; + for (let i = 0; i < 3; i += 1) { + if ((x[i] ?? 0) !== (y[i] ?? 0)) return (x[i] ?? 0) < (y[i] ?? 0); + } + return false; +} + +/** + * Feed or event, by shape. + * + * A feed is the one with `events`. Deliberately the only signal used: a + * standalone event and a feed differ in several ways, but this is the one + * the spec guarantees, and guessing from anything softer risks migrating a + * document against the wrong schema. + */ +function inferKind(json: unknown): DocumentKind { + const record = json as Record | null; + return record !== null && typeof record === "object" && Array.isArray(record.events) + ? "feed" + : "event"; +} + +/** Every event in the document, whichever shape it is. */ +function eventsOf(document: Record, kind: DocumentKind): Record[] { + if (kind === "event") return [document]; + const events = document.events; + return Array.isArray(events) ? (events as Record[]) : []; +} + +/** + * D032, the one change in OTE's history that a find-and-replace gets wrong. + * + * Before 0.5 a part of a multi-part event carried its own complete `offers`, + * and that was the only way to price it — there was no way to say "this + * costs 60 *on top of* the conference ticket". From 0.5 there is: the set's + * shared admission goes in `partOf.offers`, and a surcharge is an offer with + * `scope: "add-on"`. + * + * The trap is that the old document stays valid. An offer with no `scope` is + * read as `admission` — "a complete way to attend this part" — so a workshop + * that really costs 240 + 60 now publishes, validly and machine-readably, + * that it costs 60. Nothing flags it, because nothing can: whether that 60 + * is the whole price or an extra is a fact about the event, not about the + * JSON. Hence a warning aimed at a human, and no attempt to rewrite. + * + * Only parts of a multi-part set are worth asking about. A standalone event's + * `offers` never had a shared admission to sit on top of, so its meaning is + * unchanged and warning about it would be noise — and a warning that + * misfires teaches people to stop reading warnings. + */ +function multipartPricingWarnings( + document: Record, + kind: DocumentKind, +): MigrationWarning[] { + const warnings: MigrationWarning[] = []; + const events = eventsOf(document, kind); + + events.forEach((event, index) => { + const partOf = event.partOf as Record | undefined; + const offers = event.offers; + if (partOf === undefined || !Array.isArray(offers) || offers.length === 0) return; + + // Already answered: a document that says `scope` has been through this. + const unscoped = (offers as Record[]).filter( + (offer) => offer !== null && typeof offer === "object" && offer.scope === undefined, + ); + if (unscoped.length === 0) return; + + const where = kind === "feed" ? `events[${index}]` : "(document)"; + const name = typeof event.name === "string" ? ` ("${event.name}")` : ""; + warnings.push({ + path: `${where}.offers`, + decision: "D032", + message: + `This event${name} is part of a multi-part set and carries its own offers. ` + + `From 0.5 an offer without a "scope" is read as "admission" — a complete way to ` + + `attend this part. If any of these prices is really a surcharge on top of the ` + + `set's ticket, mark it "scope": "add-on" and put the shared admission in ` + + `partOf.offers, or the feed will under-report what attending costs.`, + }); + }); + + // Several parts of one set repeating the same price is the other half of + // D032: it is still valid, but `partOf.offers` is now where that belongs, + // and `sharedOffersConsistent` keeps the copies honest from there on. + const bySet = new Map(); + for (const event of events) { + const partOf = event.partOf as Record | undefined; + const id = partOf?.id; + if (typeof id !== "string") continue; + if (partOf?.offers !== undefined) continue; + bySet.set(id, (bySet.get(id) ?? 0) + 1); + } + for (const [id, count] of bySet) { + if (count < 2) continue; + warnings.push({ + path: "events[].partOf.offers", + decision: "D032", + message: + `${count} events share partOf.id "${id}" and none declares partOf.offers. ` + + `From 0.5 the set's shared admission belongs there, written identically on every ` + + `part — a consumer that needs the price of a multi-part event reads partOf.offers. ` + + `Leaving it out is valid; it just means the set has no published price.`, + }); + } + + return warnings; +} + +/** The errors a document has against one specific version's schemas. */ +async function errorsAgainst( + json: unknown, + kind: DocumentKind, + version: string, +): Promise { + const validators = await loadValidators(version); + const result = kind === "feed" ? validators.validateFeed(json) : validators.validateEvent(json); + return result.errors; +} + +/** + * Migrates a document to a newer spec version. + * + * Validation happens against the declared version directly rather than + * through `validateDocument`, and that is deliberate: `validateDocument` + * applies the support policy, which calls an out-of-window version an error. + * A document on an unsupported release is precisely the one that needs + * migrating, and a migrator that refuses it would be useless to the only + * people who have to run it. + * + * Throws `MigrationError` when the migration cannot be attempted — an + * unplaceable version, a target that does not exist, a downgrade, or input + * that is not valid at the version it claims. Migrating an invalid document + * would produce an invalid one wearing a newer label. + */ +export async function migrateDocument( + json: unknown, + options: MigrateOptions = {}, +): Promise { + const kind = options.kind ?? inferKind(json); + const to = options.to ?? LATEST_VERSION; + const from = declaredSpecVersion(json); + + if (from === null) { + throw new MigrationError( + "This document declares no usable specVersion, so there is no version to migrate it " + + `from. Published versions: ${PUBLISHED_VERSIONS.join(", ")}.`, + ); + } + if (!PUBLISHED.includes(from)) { + throw new MigrationError( + `This document declares OTE Spec ${from}, which was never published, so there are no ` + + `rules to migrate it from. Published versions: ${PUBLISHED_VERSIONS.join(", ")}.`, + ); + } + if (!PUBLISHED.includes(to)) { + throw new MigrationError( + `OTE Spec ${to} is not a published version. Published versions: ` + + `${PUBLISHED_VERSIONS.join(", ")}.`, + ); + } + if (isOlder(to, from)) { + throw new MigrationError( + `This document declares OTE Spec ${from}; ${to} is older. Migrating downwards would ` + + "mean dropping fields the newer version added, and silently losing data a publisher " + + "wrote on purpose is worse than refusing.", + ); + } + + // Invalid in, invalid out — but now mislabelled, and a later reader has no + // way to tell the damage was ours. The errors belong to the version the + // document claims, so that is what they are measured against. + const before = await errorsAgainst(json, kind, from); + if (before.length > 0) { + throw new MigrationError( + `This document is not valid OTE ${from}, so it cannot be migrated — fix it first, ` + + `then migrate. ${before.length} error${before.length === 1 ? "" : "s"}, starting with: ` + + `${before[0]!.path}: ${before[0]!.message}`, + ); + } + + const document = structuredClone(json) as Record; + const changed = from !== to; + document.specVersion = to; + + // Only warn about a boundary this migration actually crosses. Going 0.5→0.6 + // cannot have changed what the pricing means, because 0.5 is where it + // changed and the document is already there. + const warnings = + isOlder(from, "0.5.0") && !isOlder(to, "0.5.0") + ? multipartPricingWarnings(document, kind) + : []; + + // The claim this package makes is that the output is valid at the target. + // Every release so far has been additive, so this has never yet failed — + // which is exactly why it is checked rather than trusted. A future release + // that breaks the pattern fails loudly here instead of shipping bad feeds. + const after = await errorsAgainst(document, kind, to); + if (after.length > 0) { + throw new MigrationError( + `Migrating from ${from} to ${to} produced a document that is not valid at ${to}: ` + + `${after[0]!.path}: ${after[0]!.message}. This is a bug in @opentechevents/migrate — ` + + "please report it rather than hand-editing around it.", + ); + } + + return { kind, from, to, changed, document, warnings }; +} diff --git a/packages/migrate/test/cli.test.ts b/packages/migrate/test/cli.test.ts new file mode 100644 index 0000000..4b63565 --- /dev/null +++ b/packages/migrate/test/cli.test.ts @@ -0,0 +1,120 @@ +import { mkdtempSync, readFileSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { LATEST_VERSION } from "@opentechevents/validate"; +import { describe, expect, it } from "vitest"; + +import { runCli } from "../src/cli.js"; + +const fixturePath = (name: string): string => + fileURLToPath(new URL(`../fixtures/${name}`, import.meta.url)); + +interface Run { + code: number; + out: string; + err: string; +} + +async function run(...argv: string[]): Promise { + const out: string[] = []; + const err: string[] = []; + const code = await runCli(argv, { + out: (text) => out.push(text), + err: (line) => err.push(line), + }); + return { code, out: out.join("\n"), err: err.join("\n") }; +} + +/** A throwaway copy, so a test that writes never touches the fixture. */ +function tempCopy(name: string): string { + const dir = mkdtempSync(join(tmpdir(), "ote-migrate-")); + const path = join(dir, name); + writeFileSync(path, readFileSync(fixturePath(name), "utf8")); + return path; +} + +describe("ote-migrate CLI", () => { + it("prints usage and exits 2 with no arguments", async () => { + const result = await run(); + expect(result.code).toBe(2); + expect(result.err).toMatch(/Usage: ote-migrate/); + }); + + it("lists the published versions", async () => { + const result = await run("--versions"); + expect(result.code).toBe(0); + expect(result.out).toMatch(/published: /); + expect(result.out).toMatch(new RegExp(`latest: ${LATEST_VERSION}`)); + }); + + it("exits 2 for a file it cannot read", async () => { + const result = await run("no-such-file.json"); + expect(result.code).toBe(2); + expect(result.err).toMatch(/cannot read/); + }); + + it("exits 1 for invalid JSON", async () => { + const dir = mkdtempSync(join(tmpdir(), "ote-migrate-")); + const path = join(dir, "bad.json"); + writeFileSync(path, "{ not json"); + const result = await run(path); + expect(result.code).toBe(1); + expect(result.err).toMatch(/invalid JSON/); + }); + + // The document goes to stdout and everything else to stderr, so the JSON + // can be piped without the notes landing in it. + it("writes the document to stdout and the note to stderr", async () => { + const result = await run(fixturePath("feed-0.3.json"), "--to", "0.6.0"); + expect(result.code).toBe(0); + expect(JSON.parse(result.out).specVersion).toBe("0.6.0"); + expect(result.err).toMatch(/OTE 0\.3\.0 → 0\.6\.0/); + }); + + it("writes to an output path when given one", async () => { + const dir = mkdtempSync(join(tmpdir(), "ote-migrate-")); + const output = join(dir, "out.json"); + const result = await run(fixturePath("feed-0.3.json"), output, "--to", "0.6.0"); + expect(result.code).toBe(0); + expect(JSON.parse(readFileSync(output, "utf8")).specVersion).toBe("0.6.0"); + }); + + it("rewrites in place with --write", async () => { + const path = tempCopy("feed-0.3.json"); + const result = await run(path, "--write", "--to", "0.6.0"); + expect(result.code).toBe(0); + expect(JSON.parse(readFileSync(path, "utf8")).specVersion).toBe("0.6.0"); + }); + + // Two different instructions about where the output goes; guessing would + // be a coin flip over somebody's file. + it("refuses --write together with an output path", async () => { + const result = await run(fixturePath("feed-0.3.json"), "out.json", "--write"); + expect(result.code).toBe(2); + }); + + it("reports the D032 warnings on stderr and still exits 0", async () => { + const result = await run(fixturePath("feed-0.4-multipart.json"), "--to", "0.6.0"); + expect(result.code).toBe(0); + expect(result.err).toMatch(/D032/); + expect(result.err).toMatch(/to check by hand/); + // A warning is a decision, not a fault: the document still migrated. + expect(JSON.parse(result.out).specVersion).toBe("0.6.0"); + }); + + it("exits 1 when the document cannot be migrated", async () => { + const dir = mkdtempSync(join(tmpdir(), "ote-migrate-")); + const path = join(dir, "noversion.json"); + writeFileSync(path, JSON.stringify({ title: "x", events: [] })); + const result = await run(path); + expect(result.code).toBe(1); + expect(result.err).toMatch(/no usable specVersion/); + }); + + it("rejects an unknown flag", async () => { + const result = await run(fixturePath("feed-0.3.json"), "--nope"); + expect(result.code).toBe(2); + }); +}); diff --git a/packages/migrate/test/migrate.test.ts b/packages/migrate/test/migrate.test.ts new file mode 100644 index 0000000..a5f2b84 --- /dev/null +++ b/packages/migrate/test/migrate.test.ts @@ -0,0 +1,145 @@ +import { readFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; + +import { LATEST_VERSION, loadValidators } from "@opentechevents/validate"; +import { describe, expect, it } from "vitest"; + +import { migrateDocument, MigrationError } from "../src/index.js"; + +const fixture = (name: string): unknown => + JSON.parse(readFileSync(fileURLToPath(new URL(`../fixtures/${name}`, import.meta.url)), "utf8")); + +const multipart04 = (): Record => + fixture("feed-0.4-multipart.json") as Record; +const feed03 = (): Record => fixture("feed-0.3.json") as Record; + +describe("migrateDocument", () => { + it("moves a feed to the latest version by default", async () => { + const result = await migrateDocument(feed03()); + expect(result.from).toBe("0.3.0"); + expect(result.to).toBe(LATEST_VERSION); + expect(result.changed).toBe(true); + expect((result.document as Record).specVersion).toBe(LATEST_VERSION); + }); + + // The whole reason this package exists rather than a `sed`: the adopter + // who most needs to migrate is the one whose version left the support + // window, and `validateDocument` calls that an error. Migrating must still + // work for them. + it("migrates a document whose version is outside the support window", async () => { + const result = await migrateDocument(feed03(), { to: "0.6.0" }); + expect(result.to).toBe("0.6.0"); + const validators = await loadValidators("0.6.0"); + expect(validators.validateFeed(result.document).errors).toEqual([]); + }); + + it("leaves the input untouched", async () => { + const input = feed03(); + await migrateDocument(input, { to: "0.6.0" }); + expect(input.specVersion).toBe("0.3.0"); + }); + + it("reports no change when the document already declares the target", async () => { + const result = await migrateDocument(feed03(), { to: "0.3.0" }); + expect(result.changed).toBe(false); + expect(result.warnings).toEqual([]); + }); + + it("infers a standalone event from the absence of events[]", async () => { + const event = fixture("../../validate/fixtures/versioned/event-0.4.0.json"); + const result = await migrateDocument(event, { to: "0.6.0" }); + expect(result.kind).toBe("event"); + expect(result.from).toBe("0.4.0"); + }); + + describe("refusals", () => { + it("refuses a document with no specVersion", async () => { + await expect(migrateDocument({ title: "x", events: [] })).rejects.toThrow(MigrationError); + }); + + it("refuses a version that was never published", async () => { + await expect(migrateDocument({ specVersion: "9.9.9", events: [] })).rejects.toThrow( + /never published/, + ); + }); + + it("refuses an unpublished target", async () => { + await expect(migrateDocument(feed03(), { to: "9.9.9" })).rejects.toThrow( + /not a published version/, + ); + }); + + // Dropping the fields a newer version added is data loss, and the + // publisher wrote them on purpose. + it("refuses a downgrade", async () => { + await expect(migrateDocument(feed03(), { to: "0.2.0" })).rejects.toThrow(/older/); + }); + + it("refuses input that is not valid at the version it claims", async () => { + const broken = feed03(); + delete broken.title; + await expect(migrateDocument(broken, { to: "0.6.0" })).rejects.toThrow(/not valid OTE 0.3.0/); + }); + }); + + describe("D032 — multi-part pricing", () => { + /** The per-event "this offer now reads as admission" warnings only. */ + const scopeOnly = (warnings: { path: string }[]): { path: string }[] => + warnings.filter((w) => w.path.endsWith(".offers") && !w.path.includes("partOf")); + + it("warns that an unscoped offer on a part now reads as admission", async () => { + const result = await migrateDocument(multipart04(), { to: "0.6.0" }); + const scopeWarnings = result.warnings.filter( + (w) => w.path.endsWith(".offers") && !w.path.includes("partOf"), + ); + expect(scopeWarnings.length).toBeGreaterThan(0); + expect(scopeWarnings[0]!.decision).toBe("D032"); + expect(scopeWarnings[0]!.message).toMatch(/add-on/); + }); + + it("warns that a shared set has no partOf.offers", async () => { + const result = await migrateDocument(multipart04(), { to: "0.6.0" }); + const shared = result.warnings.find((w) => w.path === "events[].partOf.offers"); + expect(shared).toBeDefined(); + expect(shared!.message).toMatch(/share partOf.id/); + }); + + it("stays quiet for a document with no multi-part events", async () => { + const result = await migrateDocument(feed03(), { to: "0.6.0" }); + expect(result.warnings).toEqual([]); + }); + + // 0.5 is where the meaning changed. A document already on 0.5 has had + // that conversation; repeating it on every 0.6 bump is noise. + it("does not warn when the migration does not cross 0.5", async () => { + const already05 = multipart04(); + already05.specVersion = "0.5.0"; + const result = await migrateDocument(already05, { to: "0.6.0" }); + expect(result.warnings).toEqual([]); + }); + + // A publisher who has answered the question says so with `scope`. + it("does not warn about offers that already declare a scope", async () => { + const answered = multipart04(); + const events = answered.events as Record[]; + for (const event of events) { + for (const offer of (event.offers ?? []) as Record[]) { + offer.scope = "admission"; + } + } + const result = await migrateDocument(answered, { to: "0.6.0" }); + expect(scopeOnly(result.warnings)).toEqual([]); + // The set still has no partOf.offers, so that warning is untouched — + // answering one question does not answer the other. + expect(result.warnings.map((w) => w.path)).toEqual(["events[].partOf.offers"]); + }); + }); + + // Every published release so far has been additive, which is the claim the + // package makes. If a future one is not, this is where it surfaces. + it("produces a valid document for every upward hop between published versions", async () => { + const result = await migrateDocument(fixture("feed-0.3.json"), { to: "0.4.0" }); + const validators = await loadValidators("0.4.0"); + expect(validators.validateFeed(result.document).errors).toEqual([]); + }); +}); diff --git a/packages/migrate/tsconfig.build.json b/packages/migrate/tsconfig.build.json new file mode 100644 index 0000000..a89af8a --- /dev/null +++ b/packages/migrate/tsconfig.build.json @@ -0,0 +1,11 @@ +// The emitting config, used by `pnpm build`: src/ only, so tests never reach +// dist/. Its sibling tsconfig.json covers src + test and only checks types. +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "noEmit": false, + "outDir": "dist", + "rootDir": "src" + }, + "include": ["src"] +} diff --git a/packages/migrate/tsconfig.json b/packages/migrate/tsconfig.json new file mode 100644 index 0000000..ca97e4e --- /dev/null +++ b/packages/migrate/tsconfig.json @@ -0,0 +1,7 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "noEmit": true + }, + "include": ["src", "test"] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 5587648..2c7ee27 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -341,6 +341,22 @@ importers: specifier: ^5.1.0 version: 5.1.0(@types/node@22.20.5)(jsdom@30.1.3)(vite@7.3.6(@types/node@22.20.5)) + packages/migrate: + dependencies: + '@opentechevents/validate': + specifier: workspace:* + version: link:../validate + devDependencies: + '@types/node': + specifier: ^22.0.0 + version: 22.20.5 + typescript: + specifier: ^6.0.3 + version: 6.0.3 + vitest: + specifier: ^5.1.0 + version: 5.1.0(@types/node@22.20.5)(jsdom@30.1.3)(vite@7.3.6(@types/node@22.20.5)) + packages/preview-feed: dependencies: '@opentechevents/export-rss':