Repository navigation
Add redisPersistentConnection setting to reuse the Redis connection per PHP process - #369
Conversation
…er PHP process Every tracking request opened a new connection to Redis (Queue\Backend\Redis::connect). With the setting on, the standalone Redis backend uses pconnect, with one persistent id per database so a client sharing the connection cannot select() another database. Off by default; configurable in config.ini.php only, like redisTimeout. Closes matomo-org#55 Co-Authored-By: Claude Opus 5.5 <[email protected]>
snake14
left a comment
There was a problem hiding this comment.
Thank you for taking the time to create this PR.
The persistent connection works as described: 100 tracking requests reused one connection per FPM worker, and the workers reconnected cleanly after Redis restarted. A couple things to check:
plugin.jsonis still6.0.4, which is already released, and the changelog uses anUnreleasedheading, so the weekly release won't pick this up. Bumping to6.0.5with a plain6.0.5changelog heading would fix it.- The new tests would still pass without the feature:
FactoryTest::test_makeBackend_shouldConnectWithAPersistentConnection_IfConfiguredonly checkstestConnection(), andtest_persistentConnection_shouldKeepEachDatabaseSeparatepasses even without the database in the persistent ID, sinceconnect()always callsselect(). AssertinggetConnection()->getPersistentID()equalsqueuedtracking-<db>would pin both.
The Factory and database-separation tests passed without the feature. They now check getPersistentID(), so they fail if the setting is not wired or the database is missing from the id. Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
Thanks for the review! Both done in b15118f: bumped to 6.0.5 with a plain changelog heading, and the Factory and database-separation tests now assert |
|
Thank you for the changes @psykologist90. AI found few more things to check. Please let me know what you think. They're low severity and could possibly be deferred:
|
…hout pooling phpredis reconnects a dropped persistent connection by itself, so the FAQ caveat about a failed first request after a Redis restart did not hold. With connection pooling on, two clients never share a socket, so the separation test passed even with a constant persistent id; it now runs with pooling off. queuedtracking:test marks the setting as standalone Redis only. Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
Thanks, all three are right. Done in f25cb06:
I also updated the PR description to match. |
snake14
left a comment
There was a problem hiding this comment.
Thank you for the changes. Everything appears to work as described. 👍
If you would like to create a backport PR for 5.x-dev, I would be happy to review it.
|
Thank you! I've just opened the backport PR. |
…er PHP process (#369) (#371) * Add redisPersistentConnection setting to reuse the Redis connection per PHP process Every tracking request opened a new connection to Redis (Queue\Backend\Redis::connect). With the setting on, the standalone Redis backend uses pconnect, with one persistent id per database so a client sharing the connection cannot select() another database. Off by default; configurable in config.ini.php only, like redisTimeout. Closes #55 * Bump to 6.0.5 and assert the persistent connection id in tests The Factory and database-separation tests passed without the feature. They now check getPersistentID(), so they fail if the setting is not wired or the database is missing from the id. * Address review: drop the restart caveat, test database separation without pooling phpredis reconnects a dropped persistent connection by itself, so the FAQ caveat about a failed first request after a Redis restart did not hold. With connection pooling on, two clients never share a socket, so the separation test passed even with a constant persistent id; it now runs with pooling off. queuedtracking:test marks the setting as standalone Redis only. --------- (cherry picked from commit 8e5dc00) Co-authored-by: Eric Cobos <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]>
Description
Adds an opt-in
redisPersistentConnectionsetting. When it is on, the standalone Redis backend connects with phpredispconnect()instead ofconnect(), so each PHP-FPM worker keeps one connection open instead of opening a new one on every tracking request.Why.
Queue\Backend\Redis::connect()always callsconnect(), so every tracking request opens and closes its own Redis connection. The number of new connections therefore grows with tracking traffic: one per request. Each one costs a TCP handshake, plus a TLS handshake when Redis is reached over TLS (as on most managed Redis services), and that work lands on the web server and on Redis. phpredis already supports reusing a connection per PHP process throughpconnect(); this exposes that as an option.How.
redisPersistentConnection(bool, defaultfalse). LikeredisTimeout, it is not exposed in the UI and is set inconfig.ini.php:Queue\Backend\RedisgetssetUsePersistentConnection().Factorywires it up for the standalone backend only. Sentinel and Cluster are unchanged.queuedtracking-<database>. phpredis shares a persistent connection between clients with the same host, port and id, so a per-database id stops another client in the same process fromselect()-ing a different database on a shared socket. This matters whenredis.pconnect.pooling_enabledis off; with pooling on (the default) two live clients never share a socket.close()is still only called fromsetConfig()before a connection exists, so the persistent connection is not torn down per request.queuedtracking:testprints the new setting. FAQ entry and changelog added.Trade-off (also in the FAQ): Redis sees up to
pm.max_childrenconnections per web server.Matomo 5 is still widely used. Would you take a backport to
5.x-dev? The touched files are nearly the same there, and I'm happy to open that PR.Issue No
Closes #55
Steps to Replicate the Issue
INFO statstotal_connections_receivedgrows by N.Measured with phpredis 6.3.0 / PHP 8.4 against Redis 7, calling the backend 1000 times in one process (each call simulating one tracking request): 1000 connections without the setting, 1 with it. Two persistent backends on databases 14 and 15 in the same process kept their keys apart.
Checklist
PluginTestssuite against Matomo6.x-devon PHP 8.5, MySQL 8.0, Redis 7 + Sentinel: 337 tests, no failures, the existingat()deprecation warning only; phpcs and phpstan clean)SettingsTest,FactoryTest,RedisTest, the last two assertinggetPersistentID();SentinelTestskips the inherited persistent-connection test since the setting applies to the standalone backend only)6.0.5inplugin.jsonand the changelog)🤖 Generated with Claude Code