Skip to content

Add redisPersistentConnection setting to reuse the Redis connection per PHP process - #369

Merged
snake14 merged 3 commits into
matomo-org:6.x-devfrom
psykologist90:redis-persistent-connection
Nov 16, 2026
Merged

snake14 merged 3 commits into
matomo-org:6.x-devfrom
psykologist90:redis-persistent-connection

Conversation

@psykologist90

@psykologist90 psykologist90 commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds an opt-in redisPersistentConnection setting. When it is on, the standalone Redis backend connects with phpredis pconnect() instead of connect(), so each PHP-FPM worker keeps one connection open instead of opening a new one on every tracking request.

Why. Queue\Backend\Redis::connect() always calls connect(), so every tracking request opens and closes its own Redis connection. The number of new connections therefore grows with tracking traffic: one per request. Each one costs a TCP handshake, plus a TLS handshake when Redis is reached over TLS (as on most managed Redis services), and that work lands on the web server and on Redis. phpredis already supports reusing a connection per PHP process through pconnect(); this exposes that as an option.

How.

  • New system setting redisPersistentConnection (bool, default false). Like redisTimeout, it is not exposed in the UI and is set in config.ini.php:
    [QueuedTracking]
    redisPersistentConnection = 1
    
  • Queue\Backend\Redis gets setUsePersistentConnection(). Factory wires it up for the standalone backend only. Sentinel and Cluster are unchanged.
  • The persistent id is queuedtracking-<database>. phpredis shares a persistent connection between clients with the same host, port and id, so a per-database id stops another client in the same process from select()-ing a different database on a shared socket. This matters when redis.pconnect.pooling_enabled is off; with pooling on (the default) two live clients never share a socket.
  • close() is still only called from setConfig() before a connection exists, so the persistent connection is not torn down per request.
  • queuedtracking:test prints the new setting. FAQ entry and changelog added.

Trade-off (also in the FAQ): Redis sees up to pm.max_children connections per web server.

Matomo 5 is still widely used. Would you take a backport to 5.x-dev? The touched files are nearly the same there, and I'm happy to open that PR.

Issue No

Closes #55

Steps to Replicate the Issue

  1. Enable the queue with the Redis backend and send N tracking requests.
  2. Expected: Redis connections are reused across requests handled by the same PHP worker.
  3. Actual: Redis INFO stats total_connections_received grows by N.

Measured with phpredis 6.3.0 / PHP 8.4 against Redis 7, calling the backend 1000 times in one process (each call simulating one tracking request): 1000 connections without the setting, 1 with it. Two persistent backends on databases 14 and 15 in the same process kept their keys apart.

Checklist

  • [✔] Tested locally or on demo2/demo3? (full PluginTests suite against Matomo 6.x-dev on PHP 8.5, MySQL 8.0, Redis 7 + Sentinel: 337 tests, no failures, the existing at() deprecation warning only; phpcs and phpstan clean)
  • [✔] New test case added/updated? (SettingsTest, FactoryTest, RedisTest, the last two asserting getPersistentID(); SentinelTest skips the inherited persistent-connection test since the setting applies to the standalone backend only)
  • [✔] Are all newly added texts included via translation?
  • [NA] Are text sanitized properly?
  • [✔] Version bumped? (6.0.5 in plugin.json and the changelog)
  • [✔] I have understood, reviewed, and tested all AI outputs before use
  • [✔] All AI instructions respect security, IP, and privacy rules
  • [✔] Documentation updated?

🤖 Generated with Claude Code

…er PHP process

Every tracking request opened a new connection to Redis (Queue\Backend\Redis::connect).
With the setting on, the standalone Redis backend uses pconnect, with one persistent id
per database so a client sharing the connection cannot select() another database.
Off by default; configurable in config.ini.php only, like redisTimeout.

Closes matomo-org#55

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@psykologist90
psykologist90 marked this pull request as ready for review November 11, 2026 20:47

@snake14 snake14 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for taking the time to create this PR.

The persistent connection works as described: 100 tracking requests reused one connection per FPM worker, and the workers reconnected cleanly after Redis restarted. A couple things to check:

  1. plugin.json is still 6.0.4, which is already released, and the changelog uses an Unreleased heading, so the weekly release won't pick this up. Bumping to 6.0.5 with a plain 6.0.5 changelog heading would fix it.
  2. The new tests would still pass without the feature: FactoryTest::test_makeBackend_shouldConnectWithAPersistentConnection_IfConfigured only checks testConnection(), and test_persistentConnection_shouldKeepEachDatabaseSeparate passes even without the database in the persistent ID, since connect() always calls select(). Asserting getConnection()->getPersistentID() equals queuedtracking-<db> would pin both.

The Factory and database-separation tests passed without the feature. They now
check getPersistentID(), so they fail if the setting is not wired or the database
is missing from the id.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
@psykologist90

Copy link
Copy Markdown
Contributor Author

Thanks for the review! Both done in b15118f: bumped to 6.0.5 with a plain changelog heading, and the Factory and database-separation tests now assert getPersistentID() is queuedtracking-<db> (they fail if the wiring or the database in the ID is removed). Also fixed the checklist mark that failed the gate. CI needs approval to run on the new commit.

@snake14

snake14 commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Thank you for the changes @psykologist90.

AI found few more things to check. Please let me know what you think. They're low severity and could possibly be deferred:

Low

  1. The FAQ's caveat that a worker may fail its first request after a Redis restart doesn't apply with phpredis defaults, since redis.pconnect.echo_check_liveness reconnects a dead pooled socket (tracking kept working through a restart here). Either qualify it as applying only when that check is off, or drop it.
  2. With redis.pconnect.pooling_enabled=1 (the default) two live clients never share a socket, so test_persistentConnection_shouldKeepEachDatabaseSeparate passes even with a constant persistent id. Running it with pooling disabled would make the separation assertions meaningful.
  3. queuedtracking:test prints PersistentConnection: 1 for Sentinel and Cluster too, where Factory ignores the setting. Printing it only for standalone Redis, or noting "standalone only", would avoid misleading someone debugging a Sentinel setup.

…hout pooling

phpredis reconnects a dropped persistent connection by itself, so the FAQ caveat
about a failed first request after a Redis restart did not hold.
With connection pooling on, two clients never share a socket, so the separation
test passed even with a constant persistent id; it now runs with pooling off.
queuedtracking:test marks the setting as standalone Redis only.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
@psykologist90

Copy link
Copy Markdown
Contributor Author

Thanks, all three are right. Done in f25cb06:

  1. Dropped the restart caveat from the FAQ. I couldn't reproduce a failed request after a Redis restart, with redis.pconnect.echo_check_liveness on or off.
  2. The separation test now runs with redis.pconnect.pooling_enabled=0, where clients with the same persistent id really do share the socket, so it fails with a constant id.
  3. queuedtracking:test now prints the setting as PersistentConnection (standalone Redis only).

I also updated the PR description to match.

@snake14 snake14 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the changes. Everything appears to work as described. 👍

If you would like to create a backport PR for 5.x-dev, I would be happy to review it.

@snake14
snake14 merged commit 8e5dc00 into matomo-org:6.x-dev Nov 16, 2026
25 of 37 checks passed
@psykologist90

Copy link
Copy Markdown
Contributor Author

Thank you! I've just opened the backport PR.

snake14 pushed a commit that referenced this pull request Nov 16, 2026
…er PHP process (#369) (#371)

* Add redisPersistentConnection setting to reuse the Redis connection per PHP process

Every tracking request opened a new connection to Redis (Queue\Backend\Redis::connect).
With the setting on, the standalone Redis backend uses pconnect, with one persistent id
per database so a client sharing the connection cannot select() another database.
Off by default; configurable in config.ini.php only, like redisTimeout.

Closes #55



* Bump to 6.0.5 and assert the persistent connection id in tests

The Factory and database-separation tests passed without the feature. They now
check getPersistentID(), so they fail if the setting is not wired or the database
is missing from the id.



* Address review: drop the restart caveat, test database separation without pooling

phpredis reconnects a dropped persistent connection by itself, so the FAQ caveat
about a failed first request after a Redis restart did not hold.
With connection pooling on, two clients never share a socket, so the separation
test passed even with a constant persistent id; it now runs with pooling off.
queuedtracking:test marks the setting as standalone Redis only.



---------



(cherry picked from commit 8e5dc00)

Co-authored-by: Eric Cobos <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
@psykologist90
psykologist90 deleted the redis-persistent-connection branch November 17, 2026 06:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use persistent connections to redis instance

2 participants