Repository navigation
perf: post codec and JDBC benchmark results on pull requests - #292
Merged
Merged
Conversation
Add a driver-side benchmark module and a workflow that posts its result as a sticky comment on every pull request. The benchmark measures the direct and Jackson codec decode/encode paths and the JDBC result-set getters on fixed payloads. It starts no server, so server and network time cannot bury codec changes in shared-runner noise. The workflow measures the base commit and the PR merge commit, three interleaved rounds each, and reports medians. Throughput drops and allocation increases over 10% are flagged; the check stays green so reviewers judge the numbers. The base side is skipped until the module exists on the base branch, so this PR's first comment reports absolute head numbers. Refs #291.
Performance benchmarkHead Driver-side microbenchmarks only; server and network time are excluded. Shared-runner noise is about ±10%, so only larger regressions are flagged. JDK 25.0.5, Linux amd64, 4 CPUs. Throughput
Allocation
Workloads and thresholds live in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
libsql-perf, a driver-side benchmark for the JSON codec and the JDBC result-set getters, and.github/workflows/perf.yml, which builds the base and PR merge commits, runs three interleaved rounds, and upserts a sticky comment with the comparison.This is the harness that produced #291.
Why
#291 found
DirectHranaJsonCodecallocating ~55% more than the Jackson baseline per scan and encoding ~19% slower. Those numbers came from a throwaway harness. Landing the harness gives every PR the same metrics, so regressions show up in review instead of in a later investigation.Details
getObject, plus Jackson baselines for the codec rows.libsql-itests.<!-- libsql-perf -->, updated on each push. The rawperf-resultsJSON is uploaded as an artifact.maven.deploy.skipkeeps the module out of publishing; it is in the reactor so it compiles and passes Spotless in normal CI.First run
The base commit has no
libsql-perfmodule yet, so this PR's own comment reports head-only numbers. Deltas start with the next PR.Test plan
./mvnw -pl libsql-perf -am -DskipTests packagejava -cp 'libsql-perf/target/classes:libsql-perf/target/lib/*' io.github.magnusp.libsql.perf.PerfBenchmark --quick./mvnw spotless:checkcomment.shsyntax-checked withbash -n.Refs #291.