Repository navigation
test(tenants/pg): the cached-tenant test stops the invalidation listener before the database goes - #337
Merged
Conversation
…ner before the database goes The listener evicts the cache on its first connect and again on the Put's NOTIFY, both after Open returns and on their own schedule. The test read the cache as populated, closed the pool, and then one of those evictions landed, so Resolve went to the closed pool. The database-down case it proves has no live listener either, so the test stops it first; the old version failed 2 of 40 runs locally with the CI message.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TestACachedTenantOutlivesTheDatabasefailed on #336's first CI run witha cached tenant with the database gone: <nil> tenant store down: closed pool, and passed on the re-run.The invalidation listener evicts the cache on its first connect (
synced) and again when the test'sPutcommits with itsNOTIFY. Both run afterOpenreturns, on the listener's schedule. The test read the cache as populated, closed the pool, and one of those evictions landed in between, so the nextResolvemissed the cache and reached the closed pool.The case the test proves, a cached tenant outliving a database outage, has no live listener either: while the database is down the listener cannot reconnect, and a reconnect is what flushes the cache. The test now stops the listener before it fills the cache and closes the pool, so nothing can evict between the two steps.
Evidence against a throwaway postgres:17: the old test fails 2 of 40 runs with the CI message; the new one passes 20 of 20 under
-race, andtenants/pg+poolset/pgpass 3 times with no skip. Lint, fmt and asl clean on the package.