Skip to content

feat: [6/6] document package and validate large payload usage - #393

Open
wangbill (YunchuWang) wants to merge 6 commits into
yunchuwang-payload-phase-5-workerfrom
yunchuwang-payload-phase-6-release
Open

wangbill (YunchuWang) wants to merge 6 commits into
yunchuwang-payload-phase-5-workerfrom
yunchuwang-payload-phase-6-release

Conversation

@YunchuWang

@YunchuWang wangbill (YunchuWang) commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Summary

What changed?

  • Wire the optional package into existing prepare-release and official build/release packaging, without broadening CI triggers or publishing anything.
  • Add the 300 KiB runnable example, package/root configuration and cleanup documentation, changelogs and ordered migration/release guidance.
  • Add real packed-consumer checks that reject npm-published core 0.4.0, accept the completed core >=0.4.1, load runtime APIs and type-check ES2020 declarations without global ErrorOptions.
  • Remove phase-only interim documentation and unrelated README/code-block/line-ending formatting changes. Keep the original oversized PR branch/history intact.

Why is this change needed?

  • Phase 6/6 of the requested large-payload implementation split. This PR's base is phase 5's branch, so its diff excludes previously reviewed phases.
  • Behavioral reference: durabletask-dotnet 91739687. The stack also inherits the reviewed JavaScript carryover and whole-message size-preflight repairs described below; it does not claim unchanged .NET failure-priority semantics.

Issues / work items

  • Resolves: N/A (user-requested phased feature review).
  • Related: original oversized PR #387. It is closed without merging; its original branch/history remain preserved.

Project checklist

  • Release notes are not required for the next release
    • Otherwise: Notes added to CHANGELOG.md
  • Backport is not required
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • All required tests have been added/updated (unit tests, E2E tests)
  • Breaking change?
    • If yes:
      • Impact: No intentional API break in this phase; the payload feature remains opt-in.
      • Migration guidance: Do not release partial phases. Merge the stack in order and publish the completed core before dependent packages.

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

  • No
  • Yes, AI helped write parts of this PR (e.g., GitHub Copilot)
  • Yes, an AI agent generated most of this PR

If AI was used:

  • Tool(s): GitHub Copilot.
  • AI-assisted areas/files: This phase's source, regression tests, and directly related metadata/documentation.
  • What you changed after AI output: Split the implementation along independently buildable boundaries; retained focused regression evidence and removed unrelated formatting-only changes. Human review is still required.

AI verification (required if AI was used):

  • I understand the code and can explain it
  • I verified referenced APIs/types exist and are correct
  • I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
  • I reviewed concurrency/async behavior
  • I checked for unintended breaking or behavior changes

These human attestations are intentionally left unchecked.


Testing

Automated tests

  • Current review head: ae63cab8ecb6ec6b2b3e0e15067091a5b8a6b24e.
  • git diff --check origin/yunchuwang-payload-phase-5-worker...HEAD — passed. Prior heads remain ancestors; predecessor bases, phase-local review scopes, release/cleanup/version notes and ready-for-review state are preserved.
  • Combined final on exact ae63cab8ecb6ec6b2b3e0e15067091a5b8a6b24e: npm run test:unit — 3,180 tests / 133 suites passed (core 2,028; Functions 226; Blob 39; Azure-managed 821; export-history 66). Ten opt-in service tests are skipped by this unit command.
  • On the same final head, npm run build passed all five workspaces and npm run lint passed. This is the true final aggregate containing both approved commits; the earlier stopped pre-lifecycle run is not a final pass.
  • Final structural/blob audit passed: one unary transport site and one upload/resolve site in _call; all five semantic callers delegate; phase4–6 client/lifecycle fixtures match the approved runner fix, and PR1 processor plus existing public-loopback fixtures are unchanged.
  • Final DTS live run: 39837123528 on exact ae63cab8ecb6ec6b2b3e0e15067091a5b8a6b24e — PENDING (in_progress) at 2026-10-22T19:07:38.5343134Z. The link is authoritative for later completion; no early pass is claimed.
  • PR388 source/body remain unchanged at 137bb547e8781fd84c2738c3e13ae0f65cd66174; its separately verified hosted evidence is 30 SUCCESS / 1 SKIPPED. Review approval is still required. Main-base-only PR filters are unchanged; absent stacked-PR jobs are not passes.
  • Historical unchanged-provider receipt on 39cebd7651a669b50cdd73fdcc5e394344e08fa4: npm run test:azurite -w @microsoft/durabletask-js-azureblob — all 10 real local-Azurite/.NET-gzip fixture checks passed. This was not rerun or relabeled as current-head service evidence.
  • Historical consumers on 39cebd7651a669b50cdd73fdcc5e394344e08fa4: npm --registry=https://packagefeedproxy.microsoft.io/npm run test:payload-packages -- $oldCoreArchive passed published-core-0.4.0 rejection, packed runtime/native causes and strict ES2020 declarations.
  • Historical npm --prefix test\e2e-functions\test-app run build and npm pack --dry-run --json --workspace @microsoft/durabletask-js --workspace @microsoft/durabletask-js-azureblob passed on that same head (219/8 files). Public packaging/provider/declaration surfaces are unchanged; no fresh package/consumer run is claimed.
  • Scope limits remain: no live storage-outage or mixed .NET/JavaScript execution, no transactional rollback or generic orphan collector. Tombstone cleanup cannot discover blobs without a backend-recorded reference.

Manual validation (only if runtime/behavior changed)

  • Environment (OS, Node.js version, components): Windows, Node.js v24.14.0; current all-workspace units/build/lint. Historical 39cebd7 receipts additionally used local Azurite 3.37.0, actual .NET 10 gzip vectors, packed npm consumers and strict ES2020 TypeScript.
  • Steps + observed results:
    1. Materialized this phase on its declared predecessor, with no future-phase imports.
    2. Ran the exact owning checks above and inspected the incremental patch.
    3. Verified the full dependency stack, clean package graph and final semantic delta against the preserved original implementation; no cloud resources, PR merges or npm publications.
  • Evidence (optional): Full local logs are session artifacts, not repository changes.

Notes for reviewers

Shared unary boundary correction

  • f3ea69fc79eb594961ab4d57651f5e1360fb9127 restores one private _call pipeline: combine cancellation once; externalize once; generate fresh metadata outside each RPC catch; invoke only the operation's RPC policy; hydrate once after success. true retries the RPC, false rethrows unchanged, and throwing maps a genuine RPC error with its cause. The no-metadata path preserves its original dispatch timing.
  • Scheduling and completion waits no longer inline separate pipelines; rewind/restart mappings use the same RPC-only boundary. Errors are not mutated or tagged and no public API, helper signature, dependency or middleware framework was added.
  • 7c9d67f97c961c148a5fdd4809e7443c268d7162 handles runner dedupe through a per-invocation BUSINESS runnerAlreadyExists outcome set only inside that RPC policy. It is not an error tag or global marker; matching storage/metadata codes or semantic classes remain original failures, and cancellation takes precedence.
  • All earlier reviewed carryover, UTF-8 preflight, token/cancellation, rewind mapping and worker-release repairs remain ancestors. No PR merge, package publication or history rewrite was performed.
Merge order Phase PR Base branch
1 Core contracts #388 main
2 SDK integration #389 yunchuwang-payload-phase-1-core
3 Azure Blob provider #390 yunchuwang-payload-phase-2-sdk
4 Purge protocol/client #391 yunchuwang-payload-phase-3-blob
5 Purge worker #392 yunchuwang-payload-phase-4-client
6 Release/usage/consumers #393 yunchuwang-payload-phase-5-worker

Merge 1 → 2 → 3 → 4 → 5 → 6. Each PR targets its predecessor; retarget the next PR only after its prerequisite merges. Do not publish packages or describe a partial phase as the complete feature.

  • This is the final review phase, not a release operation. Merge all six phases before preparing core 0.4.1, then publish dependent packages in the documented order.
  • The reviewed SDK worker-release/client-deadline behavior and unchanged payload provider are preserved. DTO/runner and client/worker tests were separated for review; the management-only purge kind fix has dedicated enabled/disabled ordinary/entity regressions.
  • Main-base-only PR workflows intentionally do not run on these stacked bases. The explicitly dispatched combined-final DTS run is linked above with its recorded status; this is aggregate evidence, not proof that omitted PR jobs ran.
  • Original PR387 is closed without merging; its original branch and commits remain untouched.
  • Full live DTS/mixed-.NET execution has not been run locally (Docker daemon unavailable). The final phase records aggregate local Blob/Azurite, source/wire interoperability and packed-consumer evidence without claiming absent hosted checks.

Phase 6 of 6: add release packaging, the large-payload example, complete interoperability/cleanup documentation and real tarball/ES2020 consumer checks. Keep phase release gates explicit and remove unrelated README formatting changes. The completed stack preserves c9b53fc behavior except the tested management-client purge kind correction.

Co-authored-by: Copilot App <[email protected]>
Copilot-Session: 81e20512-d3a2-4f8e-a574-f396140ec9c7
@YunchuWang

Copy link
Copy Markdown
Member Author

Final cumulative-stack integration evidence for 72c74983b940ec1a4c829f3e4f90e53e73efebc5:

The existing DTS Emulator E2E workflow was dispatched on yunchuwang-payload-phase-6-release and passed all 10 jobs across Node.js 22 and 24. The six incremental bases/commit ancestors and the final runtime delta versus the original implementation were also independently checked.

The usual PR unit/sample/DTS workflows filter their PR base to main, so this manual run does not mean those workflows automatically ran on every stacked PR. Per-phase local build/test evidence is recorded in each description; phase 1's main-based hosted suite passed separately. This run rechecks the existing DTS suites on the final combined source, not a live mixed-.NET/JavaScript large-payload replay scenario.

wangbill (YunchuWang) and others added 5 commits October 19, 2026 13:05
Retain the six inherited carryover/preflight release-note lines and provider orphan-data limitations alongside the final phase's routing and ES2020 notes. Preserve original phase commits and release wiring; no rollback collector or package publication is introduced.

Co-authored-by: Copilot App <[email protected]>
Copilot-Session: 81e20512-d3a2-4f8e-a574-f396140ec9c7
Preserve finalized packaging, ES2020 consumers, cleanup documentation and existing core repair notes while carrying reviewed work-item release and completion-wait deadline isolation through the full stack.

Co-authored-by: Copilot App <[email protected]>
Copilot-Session: 81e20512-d3a2-4f8e-a574-f396140ec9c7
Preserve the completed SDK repair sweep and final release/consumer behavior while inheriting the exact approved core guards and permanent regressions. Combined workspace units, all builds, lint and scoped formatting passed before this commit.

Co-authored-by: Copilot App <[email protected]>
Copilot-Session: 81e20512-d3a2-4f8e-a574-f396140ec9c7
Preserve existing release/consumer and cleanup behavior while inheriting the two exact approved mappings/private-flow repairs and permanent regressions. The single final full-workspace unit/build/lint pass, scoped formatting and patch identity checks passed before this commit.

Co-authored-by: Copilot App <[email protected]>
Copilot-Session: 81e20512-d3a2-4f8e-a574-f396140ec9c7
Preserve the original final head as first parent and inherit both exact reviewed f3ea69f shared-call and 7c9d67f runner business-deduplication fixes. The true final all-workspace unit/build/lint pass, exact source/test blobs, single-pipeline audit and scoped formatting passed before this commit.

Co-authored-by: Copilot App <[email protected]>
Copilot-Session: 81e20512-d3a2-4f8e-a574-f396140ec9c7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant