Skip to content

Phase 2: umf-core <0.11, tag-driven release, pages.yml, quickstart in CI, short AGENTS.md - #58

Merged
easel merged 8 commits into
mainfrom
reset/phase2
Nov 11, 2026
Merged

easel merged 8 commits into
mainfrom
reset/phase2

Conversation

@easel

@easel easel commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Phase 2 for tablespec (phase 1 #57 is merged).

umf-core range and test results

  • umf-core>=0.10.0,<=0.10.3 -> >=0.10.0,<0.11; uv.lock moves umf-core 0.10.3 -> 0.10.10. No code changes were needed.
  • Full suite on umf-core 0.10.10, local macOS, before any workflow changes: 4414 passed, 42 skipped, 1 failed, 46 errors (15 min). The 1 failure was a ruff-format error in a file I had just added (fixed). The 46 errors are all Failed: Timeout in the spark_session fixture (no JDK on that machine).
  • First nightly full run on Linux CI (dispatch on this branch, run 43084749655): 4407 passed, 43 skipped, 1 failed (same format error), 47 errors (54 min). 45 errors were again Timeout >600s in the Spark session fixture, 2 were collection errors for the two deleted test files below. Root cause of the 20 failures / 4 errors seen earlier is not Spark itself: tests/conftest.py::spark_session takes an exclusive fcntl.flock on a shared lock file for the whole session, so with xdist -n 2 the second worker blocks on its first Spark test until the 600 s timeout and every later Spark test in that worker errors. Fix (this PR): the nightly job runs non-Spark tests with -n auto and the Spark tests (-m spark_only) in one process (-n 0, Java 17 from the runner image), then the umf-core 0.10.0 lower-bound check.
  • Second nightly full run (run 43093467919, after the fix): every step succeeded: non-Spark parallel suite, Spark suite in one process, lower-bound umf-core 0.10.0 check, artifact upload. (GitHub's API was returning 502 for the job log when I finished, so I cannot quote the exact pass counts for this run; see the run page.)
  • Nothing was xfailed or skipped to get green; no product test was deleted. Deleted tests: tests/unit/test_distribution_identity.py and tests/unit/test_pypi_publication.py, which only exercised the removed release-verification scripts (plan section 2).

Workflows (8 -> 3)

  • release.yml rewritten (tag v*.*.* only; no dispatch recovery): reuses ci.yml (lint + fast suite + README quickstart), build (uv build, twine check, sdist < 5 MB, wheel/sdist versions equal the tag, plugin manifests equal the tag), smoke (fresh venv install of the built wheel, tablespec --help, README quickstart: tablespec generate to SQL, PySpark and JSON Schema), integration (latest umf-core in range; shared-UMF, conformance, golden, encapsulation tests), publish (tag only; environment pypi; pypa/gh-action-pypi-publish, trusted publishing), github-release (SHA256SUMS, attestation, notes from CHANGELOG). A manual dispatch is a dry run that never publishes. Verified with a temporary push trigger on this branch (removed): lint, fast, quickstart, build, smoke, integration all green; publish/github-release skipped.
  • pages.yml replaces microsite.yml and publish-microsite.yml: builds Hugo, link-checks under /tablespec/, runs tests/website; deploys only on release: published or dispatch. The /simple/ PEP 503 index is retired (install from PyPI); removed build_pages_artifact.py, verify_release_artifacts.py, verify_pypi_release.py, publish-pypi.yml, and the Playwright visual-snapshot spec with its darwin/linux baselines (the link-check spec stays).
  • ci.yml: added a quickstart job (builds the wheel, installs it in a fresh venv, runs the README commands via .github/run_quickstart.py).
  • Version stamp: the website footer shows tablespec <version> (HUGO_PARAMS_VERSION from the release tag; checked locally).

Docs

README quickstart is now an executed tablespec generate example (the old one wrote a single-file YAML that the CLI refuses as legacy); install section is plain PyPI with the aarch64 umf-core==0.10.0 note. AGENTS.md 319 -> 45 lines (CLAUDE.md points to it); CHANGELOG.md added; docs/release-publishing.md rewritten for the new flow; website pages no longer reference the Pages index or umf-core==0.10.3. docs/helix stays (open owner decision). Tracked files 1707 -> 1698, tree 32.3 MB.

Known failures

None known. Not verified: publish/github-release/pages deploy (need owner setup and a tag); macOS (CI is Linux-only; locally Spark tests cannot run without a JDK).

Not in this PR

docs/helix removal; mkdocs removal; demo media move; the umf-core>=0.10.11 floor (waits for umf 0.10.11 - follow-up: bump the floor, uv lock --upgrade-package umf-core, rerun full); version bump/tag 0.0.17; deleting stale remote branches.

Owner actions before the first release

  1. pypi.org -> umf-tablespec -> Publishing: add a trusted publisher for repository DocumentDrivenDX/tablespec, workflow release.yml, environment pypi (the existing publisher is bound to the now-deleted publish-pypi.yml; delete it).
  2. Ensure the GitHub environment pypi exists (it already does for publish-pypi.yml).
  3. Bump the three plugin manifests to the release version, add the CHANGELOG section, then push tag vX.Y.Z. Publishing the GitHub Release (created by the workflow) triggers pages.yml.

🤖 Generated with Claude Code

easel and others added 8 commits November 11, 2026 14:10
…README quickstart executed in CI, short AGENTS.md

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…deadlocks xdist workers)

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
@easel
easel merged commit 5a27ee1 into main Nov 11, 2026
10 checks passed
@easel
easel deleted the reset/phase2 branch November 11, 2026 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant