Repository navigation
Closed
Conversation
… once the reactor has stopped, also one the stop found queued, instead of throwing (#585) Called off the reactor with nothing decrypted left to take, TlsDecryptingPipeReader.ReadAsync (BoringSSL's read half) switches to the reactor to decrypt. Once the reactor has stopped, that switch resumes on the thread pool and throws ObjectDisposedException, for a read made after the stop and for one whose switch the stop found queued alike. The kTLS RX half, TcpConnectionPipeReader behind TlsProloguePipeReader, returns a completed read there instead, since the stop closed the connection. The handshake picks the half, not the application, so the two should agree. FillOnReactor now catches the switch's ObjectDisposedException and ends the buffer as a closed snapshot does, through EndClosed(), which Ingest's closed path now shares: the read completes with IsCompleted and whatever was decrypted but not consumed, or, after a receive-queue overflow before the stop, fails with the IOException a read on the reactor and the kTLS half report. What is still queued on the connection is not decrypted: in the default shared-buffer mode it points into the receive buffers the stop freed. Tests (Tls, OffReactorFlushTests): the test that asserted the ObjectDisposedException now asserts a completed read, and a kTLS RX control gives the same answer. A read whose switch is posted while the handler holds the reactor, in the pass it then stops, returns completed without having reached the connection. A read after the stop reports an overflow before it as an IOException, and returns completed with nothing decrypted when ciphertext was queued before it. On main the four BoringSSL tests fail with ObjectDisposedException and the control passes.
This was referenced Nov 14, 2026
This was referenced Nov 14, 2026
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #585.
Called from another thread with nothing decrypted left to take,
TlsDecryptingPipeReader.ReadAsync(BoringSSL's read half) awaits aReactorSwitchso it can decrypt on the reactor. Once the reactor has stopped, the switch's continuation runs on the thread pool, whereGetResultthrowsObjectDisposedException. That happens to a read made after the stop, and to one whose switch was still queued when the stop came, since the stop's last drain hands it to the pool. Under kTLS RX,TlsConnectionDualPipehands outTcpConnectionPipeReaderbehindTlsProloguePipeReaderinstead, and the same read returnsIsCompleted: trueat once, because the stop closed the connection. The handshake picks the half, not the application, so a caller that ends on a completed read got an exception, but only on BoringSSL connections. #580 made the two write halves agree in the same situation.A read parked on the connection before the stop returned completed on both halves before this too, as the stop's close completes it.
What changes. In
FillOnReactor, the coldNoInliningmethod an off-reactor read goes through:ObjectDisposedExceptionis caught, and the buffer ends the way a closed snapshot ends it, so the read completes withIsCompleted: trueand whatever was decrypted but not consumed. Catching, rather than checkingPostsClosedbefore the switch, also covers the read the stop found queued: it got past any such check while the reactor still ran, as tls: a flush from another thread returns completed once the reactor has stopped, also one the stop found queued, instead of waiting forever (#568) #580's flush did.Ingestthrough a one-lineEndClosed(), so a receive-queue overflow before the stop still ends the read with theIOExceptionit ends with on the reactor. The kTLS half reports that overflow the same way (TcpConnectionPipeReader.LostToOverflow, checked below). Ending cleanly there would report data the overflow dropped as the peer's clean end.TlsSession.Closedstill false, as for a peer that closed without close_notify.No new state, nothing the reactor's path does changes, and no native change.
Why core.
TlsDecryptingPipeReaderlives insrc/ioxide/Tls/Pipes, and the core'sTlsConnectionDualPipeconstructs it. The fix is in the reader's own off-reactor read, so there is no satellite it could live in. It is 19 lines added and 4 removed in that one file, the class remarks included.What a caller notices. A read from another thread on a BoringSSL TLS pipe that has to decrypt, made once the reactor has stopped or queued when it stopped, returns a completed
ReadResultinstead of throwingObjectDisposedException, or throwsIOExceptionif the receive queue overflowed before the stop. Nothing in this repo depends on the old exception.Http2ConnectionandNghttp2Connectionread their pipe and then switch to the reactor themselves, so off the reactor after a stop they end on their own switch either way, and Kestrel'sHopDuplexPipedecrypts in its own pump. The change shows for direct users ofTlsConnectionDualPipethat read off the reactor, which its remarks allow ("Usable from any thread").Tests. In
OffReactorFlushTests(Tls), beside the stopped-reactor flush tests:ObjectDisposedException;_pendingthe stop's close sets is still set), so a read the reactor ran before its stop, which the stop's close completes too, can't pass for it;IOException, on the existing overflow test's setup (64-byte buffers, a 16-entry queue);On main, 1, 3, 4 and 5 fail with
ObjectDisposedExceptionand the control passes. Taking out each part of the fix fails its own test:TlsDecryptingPipeReader.cs)the read after the stop: ObjectDisposedException; 3:expected [completed], got [ObjectDisposedException]; 4:expected [IOException], got [ObjectDisposedException]EndClosed()kept inIngestEndClosed()in the catch (a plainEnd(null))expected [IOException], got [completed]Fill()in the catch, decrypting as on the reactor)the read after the stop: completed, with 1000 bytes, decrypted out of the freed receive buffers; 3:expected [completed], got [completed, by a read that reached the connection]With the fix all pass. Looping the file's tests in a scratch runner, all 50 rounds passed with the fix (750 tests), and on main the four failed in each of 10 rounds. The same runner, with two throwaway tests, showed the kTLS RX answers this matches: an overflow before the stop gives
IOException, 3 of 3.Suites. On the head: Unit 263/0, E2E 730/0, Tls 273/0 (269 before: one test replaced, four added). The
tlsmodule is loaded on this box, so the kTLS tests ran and nothing was skipped. Chaos, Http, File, Pg and Redis weren't run, since this change doesn't reach them. No benchmark: the reactor's read path is unchanged, and onlyFillOnReactor, already an async method, gains an exception handler.For the reviewer.
TlsSession.Closedfalse is what a peer that closes without close_notify gets too.completed at once, with 1000 bytes, 3 of 3, and those bytes can fault once read. The plain connection's reader does the same. That is the teardown's, which frees the shared buffers without draining the connections their handlers still hold, not this reader's, so it is left for its own issue, and this PR doesn't copy it.