diff --git a/reference/mcp/overview.md b/reference/mcp/overview.md index a342716eb..1fc82dca4 100644 --- a/reference/mcp/overview.md +++ b/reference/mcp/overview.md @@ -64,6 +64,6 @@ See [MCP Tools and Resources](./tools-and-resources.md) for the full generation The following items are explicitly deferred to a follow-on release: - OAuth 2.1 PRM (Protected Resource Metadata) authorization. -- Cross-worker session sharing (each MCP session is bound to the worker that accepted the GET stream). +- Sharing notification replay history between workers. The live GET stream belongs to one worker, but the session accepts subscribe POSTs on other workers and [persisted resource subscriptions](./subscriptions.md#scope-and-durability) can restore on a new worker. - TypeScript type reflection into JSON Schema for custom `mcpTools` entries (schemas are hand-authored). - Global REST/operations rate limiting — the MCP rate limits (per-session, per-tool, per-client, and `initialize`) apply to the MCP surface only. diff --git a/reference/mcp/subscriptions.md b/reference/mcp/subscriptions.md index 477074191..fdb5b68d4 100644 --- a/reference/mcp/subscriptions.md +++ b/reference/mcp/subscriptions.md @@ -32,7 +32,7 @@ A subscription delivers its updates over the session's GET-SSE channel, so the c { "error": { "code": -32602, "message": "open the GET SSE stream before subscribing to resources" } } ``` -The open stream is also the subscription's lifecycle boundary: when it closes (by any of the paths in [Unsubscribing](#unsubscribing)), the subscription's underlying change iterator is released. This is why Harper rejects a stream-less subscribe rather than letting the iterator leak with nowhere to deliver. +The open stream is also a subscription lifecycle boundary: when it closes (by any of the paths in [Unsubscribing](#unsubscribing)), the subscription's underlying change iterator is released. Credential expiry also stops delivery; see [Subscription lifetime and credential rotation](#subscription-lifetime-and-credential-rotation). ## Subscribing @@ -47,7 +47,7 @@ Send `resources/subscribe` with the resource URI: } ``` -A successful subscribe returns an empty result: +A successful subscribe means the current GET worker has installed the subscription, and returns an empty result: ```json { "jsonrpc": "2.0", "id": 7, "result": {} } @@ -74,11 +74,15 @@ Synthetic `harper://*` metadata URIs (`harper://about`, `harper://openapi`, `har A refused subscribe, row-backed or custom, answers a JSON-RPC error rather than an HTTP 500 (Changed in: v5.4.0): -| Answer | When | -| -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | -| `-32602 permission denied: ` | The session user may not read the resource. | -| `-32603 failed to subscribe: ` | The subscribe failed for another reason; the raw error is in the server log. | -| `-32602 resources/subscribe for was cancelled: ` | The GET stream closed, or an unsubscribe or another subscribe for the URI arrived, during setup. | +| Answer | When | +| ----------------------------------------------------------- | ----------------------------------------------------------------------------------------- | +| `-32602 permission denied: ` | The credentials used for this subscription do not permit reading the resource. | +| `-32603 resource subscription timed out; retry the request` | The GET worker did not acknowledge installation within the 30-second acknowledgment wait. | +| `-32603 resource subscription failed; retry the request` | Installation failed, or another request for the same URI replaced this request. | + +A missing or expired session returns HTTP `404`. If requests for the same URI overlap, let them settle and retry with the current credential only if that subscription is still wanted. For other `-32603` failures, retry only transient causes, with backoff. If failures repeat, have the server operator inspect the logged error and fix its cause before retrying. An invalid custom stream or a consistently failing custom gate will not be repaired by retries. + +A failed or timed-out subscribe attempts to remove the state written by that request without removing a later request for the same URI. The error response does not report whether cleanup succeeded. If you no longer want that URI subscribed after any failed or timed-out request, send `resources/unsubscribe` to confirm removal, following the transient-error retry guidance above. If the GET closes during setup, a waiting POST can complete after reconnect, or return the timeout error if no reconnect acknowledges it. Open a replacement GET before retrying. `resources/list` advertised these URIs under `http(s)://` before 5.1.18 . A legacy `http://` or `https://` URI is still accepted by `resources/subscribe` and `resources/unsubscribe`, so a client holding an older listing keeps working; new clients should use the `harper+rest://` form the current listing returns. @@ -125,11 +129,29 @@ Send `resources/unsubscribe` with the same URI: - `params.uri` is required and must be a string — otherwise `-32602` (`resources/unsubscribe requires params.uri`). - Unsubscribing a URI that was never subscribed is a no-op and still returns `{}`. -Subscriptions are also torn down automatically when the session's GET-SSE stream closes — via explicit `DELETE /mcp`, idle-timeout eviction from `system.mcp_session`, a dropped TCP connection, or an idle-prune sweep. +Subscriptions are also torn down automatically when the session's GET-SSE stream closes — via explicit `DELETE /mcp`, idle-timeout eviction from `system.mcp_session`, a dropped TCP connection, or an idle-prune sweep. Credential expiry stops delivery too; see [Subscription lifetime and credential rotation](#subscription-lifetime-and-credential-rotation). ## Scope and durability -Subscriptions are **per-worker**: a subscription is bound to the worker that holds the session's SSE stream, and the audit-log change feed delivers commits locally on that worker. A custom resource's stream also runs on that worker, and must itself deliver changes made on other workers, as Harper table subscriptions do. The live subscription objects can't be persisted, but the durable list of subscribed URIs lives on the `mcp_session` record, which is how they survive a reconnect. +A live subscription runs on the worker holding the session's GET-SSE stream. The subscribe POST may reach another HTTP worker: it updates the durable desired URI set, and succeeds after the current GET worker acknowledges installation. The durable state survives worker changes and a server restart. A custom resource's stream still needs to deliver changes originating on other workers, as Harper table subscriptions do. + +## Subscription lifetime and credential rotation + + + +Resource updates stop when either the subscribing POST's bearer credential or the receiving GET's bearer credential expires, whichever comes first. Harper keeps the GET credential's expiry when it re-reads the user's permissions after a role change. Reconnecting can restore an unexpired subscription under the current GET permissions, but cannot extend the original POST credential's expiry. A reconnect after a network interruption does not require re-subscribing while the original subscribing POST credential is still valid. Component bearer validators that omit `authExpiresAt` default to five minutes, capped by a JWT's `exp`. Return the verified token expiry as described in [Bearer validators](../http/api.md#serverregisterbearervalidatorvalidator-options). + +MCP does not return the effective subscription expiry. Obtain credential lifetime from the issuer, such as `expires_in` or a JWT's `exp`, and renew before the earlier POST/GET expiry. For component bearer validators, the service must also communicate any shorter `authExpiresAt` lifetime, including the five-minute default; clients cannot discover that server-side cap through MCP. + +A subscription created with a scoped token restores only when the GET presents a scoped token with the same username and role permissions. A newly issued scoped token may have a later expiry, but restoring does not extend the original subscription expiry. An ordinary credential with the same username, such as a user created after the scoped token was issued, cannot inherit that scoped subscription; reconnecting with it removes the stored scoped entry while retaining valid ordinary subscriptions. An explicit new `resources/subscribe` POST can use its own scoped credential. + +To renew subscriptions after rotating credentials: + +1. Close the old GET stream and open `GET /mcp` for the same session with the replacement credential. +2. Send `resources/subscribe` for each URI you want to keep watching, using the replacement credential on each POST too. +3. Re-read each resource to catch changes made while its subscription was inactive; re-subscribing does not replay those changes. + +An open GET alone does not mean that updates are still being delivered. A table or custom iterator can end at expiry while the GET stays open. Harper can also close the GET when it detects an expired credential during delivery. Renew before expiry rather than waiting for the connection to close. Re-subscribing with the replacement credential establishes a new subscription lifetime. ## Resuming a dropped stream @@ -141,13 +163,13 @@ Mcp-Session-Id: 0c8f... Last-Event-ID: 137 ``` -Harper replays every buffered frame with a higher id (exclusive) before live frames resume, so a brief network blip doesn't drop `notifications/resources/updated` or `list_changed` events. On reconnect, Harper also restores the session's persisted subscriptions from the `mcp_session` record (best-effort — a URI that is no longer subscribable is dropped; one whose subscribe failed, or that nothing on this worker matches yet, stays on the record and is retried on the next reconnect). The GET response does not wait for the restore (Changed in: v5.4.0): subscriptions are restored after the stream opens, at most eight at a time, and their notifications follow on the stream. +Harper replays buffered frames with a higher id (exclusive) before live frames resume, subject to the per-worker limits below. On reconnect, Harper restores persisted subscriptions under the current GET permissions, bounded by the original subscribing POST expiry and the current GET expiry. The GET response waits for restoration before returning HTTP `200`. Expired, denied, or no-longer-subscribable entries are removed. A transient restoration failure retains the URI and returns HTTP `503` with `Retry-After: 1` (Changed in: v5.4.0). Resend `GET /mcp` with the same `Mcp-Session-Id`, `Mcp-Protocol-Version` and, if used, `Last-Event-ID` headers, using valid authorization. A `503` alone does not require repeating `initialize` or the subscribe POSTs. An absent session returns HTTP `404`. Caveats, all best-effort by design: - **Buffer bound.** Only the last 100 frames are retained; a client that was disconnected long enough to miss more than 100 frames will not get the older ones. A non-numeric `Last-Event-ID` replays the entire buffer. -- **Per-worker.** The replay buffer and event-id sequence live on the worker that served the original stream. If the reconnecting `GET` lands on a different worker (one that never held the prior stream), its buffer is empty and there is nothing to replay. This is the same per-worker binding that scopes sessions and subscriptions. +- **Per-worker.** The replay buffer and event-id sequence live on the worker that served the original stream. If the reconnecting `GET` lands on a different worker (one that never held the prior stream), its buffer is empty and there is nothing to replay. Persisted subscription state can restore on the new worker, but the old notification history cannot. ## Not yet supported -See [the v1 out-of-scope list](./overview.md#out-of-scope-for-v1). Notably, cross-worker session sharing is deferred, so subscription delivery and `Last-Event-ID` replay are both best-effort across a reconnect that changes workers. +See [the v1 out-of-scope list](./overview.md#out-of-scope-for-v1). Notification replay history is not shared between workers; restoring a subscription on another worker does not recover updates missed during the disconnection. diff --git a/reference/mcp/tools-and-resources.md b/reference/mcp/tools-and-resources.md index e12a5aa63..70647b73e 100644 --- a/reference/mcp/tools-and-resources.md +++ b/reference/mcp/tools-and-resources.md @@ -242,9 +242,9 @@ The contract: - **Authorize and acquire before returning.** The returned iterable is consumed after the transaction ends, so a check made while iterating no longer runs as the session user. A generator method (`async *watch()`) would run its whole body that way, so a `subscribe` that names one is ignored with a warning, and the entry stays readable. - **Release.** Harper releases the stream on `resources/unsubscribe`, when the session's GET stream closes, on `DELETE /mcp`, and when the stream ends or throws: it calls the stream's `end()` if it has one, otherwise its iterator's `return()`. Return a stream with `end()` — a Harper subscription such as `Table.subscribe(...)`, or your own queue. Without `end()`, the iterator must have a `return()` that releases its source at once. A stream with neither, or an async generator with no `end()`, fails the subscribe with `-32603`: there is nothing to call, or `return()` cannot interrupt the generator while it waits for its next value. -- **Access after subscribe.** Harper ends the stream when the session's bearer token expires, or when its user is dropped or no longer has a role. A [scoped token](/reference/v5/security/jwt-authentication#scoped-tokens-inline-role) belongs to no user, so only its expiry ends the stream, and an anonymous session has no identity to lapse, so only the release paths above end its stream. Changes to resource-specific access after subscribe are yours to enforce, for example by checking each change before yielding it. +- **Access after subscribe.** Harper stops updates when either the subscribing POST's or the receiving GET's bearer credential expires, whichever comes first, or when the user is dropped or no longer has a role. See [Subscription lifetime and credential rotation](./subscriptions.md#subscription-lifetime-and-credential-rotation). A [scoped token](/reference/v5/security/jwt-authentication#scoped-tokens-inline-role) belongs to no user, so only its expiry ends the stream, and an anonymous session has no identity to lapse, so only the release paths above end its stream. Changes to resource-specific access after subscribe are yours to enforce, for example by checking each change before yielding it. - **Workers.** A subscription runs on the worker holding the session's GET stream. If changes can originate on other workers, your stream must deliver them; Harper table subscriptions do. -- **Errors.** An `AccessViolation`, which Harper's own permission checks throw (for example, a table read with `target.checkPermission` that the user's role does not allow), answers `-32602 permission denied: `. Any other error, including one your own gate throws, answers `-32603 failed to subscribe: `; the raw error goes to the server log. +- **Errors.** An `AccessViolation`, which Harper's own permission checks throw (for example, a table read with `target.checkPermission` that the user's role does not allow), answers `-32602 permission denied: `. Other installation failures, including an error your own gate throws, now answer `-32603 resource subscription failed; retry the request`. See [subscription error responses](./subscriptions.md#which-uris-are-subscribable). The raw error goes to the server log. #### Template parameters are client-controlled @@ -338,6 +338,8 @@ This is independent of the `http` exportType — the only switch that operators Both profiles serve `resources/list`, `resources/read`, and `resources/templates/list`. Row-backed application resources, and custom resources that declare `subscribe`, can additionally be watched with `resources/subscribe` — see [MCP Resource Subscriptions](./subscriptions.md). +For the GET-stream prerequisite, credential expiry, and renewal steps, see [Subscription lifetime and credential rotation](./subscriptions.md#subscription-lifetime-and-credential-rotation). + ### `harper://` URIs | URI | Profile | Content | diff --git a/release-notes/v5-lincoln/5.4.md b/release-notes/v5-lincoln/5.4.md index 84dcb9450..a6773cd11 100644 --- a/release-notes/v5-lincoln/5.4.md +++ b/release-notes/v5-lincoln/5.4.md @@ -10,6 +10,8 @@ All patch release notes for 5.4.x are available on the [releases page](https://g ## Upgrade Notes +MCP clients using expiring credentials must renew their subscriptions before expiry. Harper preserves the original subscribing POST expiry across workers and reconnects; an open GET does not guarantee that updates are still being delivered. Open a replacement GET and re-subscribe each URI with the new credential. See [Subscription lifetime and credential rotation](/reference/v5/mcp/subscriptions#subscription-lifetime-and-credential-rotation). + Full-text indexes using fulltext 0.5.0 with current field-based declarations, or prerelease 0.6, rebuild locally after this upgrade. Plan for search unavailability and review the [Tantivy upgrade requirements](#tantivy-027-and-existing-index-rebuilds) before deploying; direct downgrade to fulltext 0.5.x is unsupported. An OIDC exchange now refuses a token that two enabled trust policies give a different user or different operations, where before the first policy by `id` answered (see [Conflicting Trust Policies Are Refused](#conflicting-trust-policies-are-refused)). Before upgrading, run `list_oidc_trust` and look for two enabled policies, with the same issuer and audience, whose claims can match the same runs but which name different users or `operations`, such as the policies of two `harper deploy setup=true provider=github-actions` runs that share a workflow, branch and environment. Drop, disable or narrow one of each pair, or the runs both match stop authenticating. A 5.4 node's `list_oidc_trust` names each pair in `invalid_reason`. Upgrade every node: a node still running 5.3 keeps answering with the first policy by `id`. @@ -73,7 +75,7 @@ A durable MQTT session now resumes through the same checked replay. When a recon ### MCP Clients Can Subscribe to Custom Resources -A static `mcpResources` entry can now name a `subscribe` method, so MCP clients can watch a custom URI such as `memory:///{id}` with `resources/subscribe`. The method is called like the entry's read method, as the session's user, and returns an async iterable; each value it yields sends `notifications/resources/updated` with the URI. Harper releases the stream on unsubscribe, when the GET stream closes, on `DELETE /mcp`, and when the session's token expires or its user is dropped. A subscribe that fails now answers a JSON-RPC error instead of an HTTP 500, for row-backed resources too, and a reconnecting GET stream opens without waiting for its subscriptions to be restored. See [Subscribing to custom resources](/reference/v5/mcp/tools-and-resources#subscribing-to-custom-resources). +A static `mcpResources` entry can now name a `subscribe` method, so MCP clients can watch a custom URI such as `memory:///{id}` with `resources/subscribe`. The method is called like the entry's read method, as the session's user, and returns an async iterable. Each value it yields sends `notifications/resources/updated` with the URI. Harper releases the stream on unsubscribe, when the GET stream closes, and on `DELETE /mcp`. For subscriptions tied to a stored user, delivery also stops if that user is dropped or no longer has a role. Updates also stop at the earlier POST/GET bearer expiry. A subscribe that fails now answers a JSON-RPC error instead of an HTTP 500, for row-backed resources too. See [Subscribing to custom resources](/reference/v5/mcp/tools-and-resources#subscribing-to-custom-resources). ## Built-in Agent @@ -206,6 +208,12 @@ When two enabled OIDC trust policies match one identity token and give it a diff ## MCP +### Resource Subscriptions Preserve Credential Expiry Across Workers + +MCP resource subscriptions now work when the subscribe POST and GET-SSE stream reach different HTTP workers. Reconnecting GETs continue to wait for restoration before HTTP `200`; a transient restoration failure now returns HTTP `503` with `Retry-After: 1`. The original subscribing POST expiry is preserved across workers and reconnects. Delivery is also bounded by the current GET credential's expiry. Re-reading a user's permissions after a role change also retains the GET credential's expiry. A network reconnect with matching credentials restores unexpired subscriptions without new subscribe calls. Stored scoped subscriptions require a scoped GET with the same username and role permissions; an ordinary credential with that username cannot inherit them. To renew after rotating credentials, close the old GET, open a replacement GET with the new credential, and send each `resources/subscribe` POST with that credential too. Subscriptions can end at expiry without closing the GET stream. See [Subscription lifetime and credential rotation](/reference/v5/mcp/subscriptions#subscription-lifetime-and-credential-rotation) ([Make MCP resource subscriptions reliable across HTTP workers](https://github.com/HarperFast/harper/pull/2436); [Fix scoped MCP reconnects and concurrent subscribe failures](https://github.com/HarperFast/harper/pull/4029)). + +Clients matching the formerly documented `-32603 failed to subscribe: ` or `-32602 resources/subscribe for was cancelled: ` responses should handle the current `-32603 resource subscription failed; retry the request` response and, when the GET closes during setup without a successful reconnect, `-32603 resource subscription timed out; retry the request`. Let overlapping requests settle before retrying a transient failure; do not retry solely because the message says to. + ### Session Creation Is Rate-Limited by Default MCP `initialize` is now limited per client identity, on both profiles, by the new `mcp..rateLimit.initializePerSecond` (default `10`) and `initializeBurst` (default `50`). Each session is a stored row kept for the idle timeout, and the application profile accepts anonymous `initialize`, so this limit is on by default; set `initializePerSecond: 0` to turn it off. A denied `initialize` gets HTTP `429` with a `Retry-After` header and a JSON-RPC error, and no session is created. Clients that open sessions in bursts, such as a test suite or a pool of agents behind one IP address, may now see that `429` after upgrading. `initialize` also stores only the names of the client capabilities Harper checks (`sampling`, `elicitation`, `roots`), not the capabilities object the client sent. See [`initializePerSecond`](/reference/v5/mcp/configuration#mcpprofileratelimitinitializepersecond) ([harper#3537](https://github.com/HarperFast/harper/pull/3537)).