Skip to content

Add pluggable WAL storage providers - #6598

Draft
dlmarion wants to merge 1 commit into
apache:mainfrom
dlmarion:wal-spi
Draft

dlmarion wants to merge 1 commit into
apache:mainfrom
dlmarion:wal-spi

Conversation

@dlmarion

Copy link
Copy Markdown
Contributor

Introduce the WalProvider SPI and properties to configure the available set of WAL providers and the current provider.

This commit converts the current DFS implementation to a DfsWalProvider and the upgrade code converts all existing wal entries in ZooKeeper and the root and metadata tables to use the 'dfs' provider id.

Post-upgrade and on instance restart the
`instance.wal.provider' property can be set to the class name of a WALProvider SPI implementation. The
'instance.wal.providers' property must contain the class name and configuration properties for the current and all in-use providers.

An InMemoryWalProvider class in the test module provides an example implementation and is used in various ITs.

Assisted-By: OpenAI GPT-6 Luna

Introduce the WalProvider SPI and properties to configure
the available set of WAL providers and the current provider.

This commit converts the current DFS implementation to a
DfsWalProvider and the upgrade code converts all existing
wal entries in ZooKeeper and the root and metadata tables
to use the 'dfs' provider id.

Post-upgrade and on instance restart the
`instance.wal.provider' property can be set to the class
name of a WALProvider SPI implementation. The
'instance.wal.providers' property must contain the
class name and configuration properties for the current
and all in-use providers.

An InMemoryWalProvider class in the test module provides
an example implementation and is used in various ITs.

Assisted-By: OpenAI GPT-6 Luna
@dlmarion dlmarion added this to the 4.0.0 milestone Oct 16, 2026
@dlmarion dlmarion self-assigned this Oct 16, 2026
Comment thread WAL-SPI.md

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file will be removed before merge and moved over to the website repo

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant