Skip to content

TIKA-4997: tika-eval: index container_id on pipes_class* and extract_… - #3410

Merged
tballison merged 2 commits into
apache:mainfrom
tballison:TIKA-4997
Oct 26, 2026
Merged

tballison merged 2 commits into
apache:mainfrom
tballison:TIKA-4997

Conversation

@tballison

Copy link
Copy Markdown
Contributor

…exceptions*

The crash-ledger reports and summary.md join extract_exceptions to pipes_class,
which was built by create-table-as with no index, so H2 scanned pipes_class once
per exception row: 9 h on a compare with 339k extract exceptions.

Thanks for your contribution to Apache Tika! Your help is appreciated!

Before opening the pull request, please verify that

  • there is an open issue on the Tika issue tracker which describes the problem or the improvement. We cannot accept pull requests without an issue because the change wouldn't be listed in the release notes.
  • the issue ID (TIKA-XXXX)
    • is referenced in the title of the pull request
    • and placed in front of your commit messages surrounded by square brackets ([TIKA-XXXX] Issue or pull request title)
  • commits are squashed into a single one (or few commits for larger changes)
  • Tika builds and unit tests pass with ./mvnw clean install (clean test alone cannot resolve the pipes plugin zips)
  • if you used a generative AI tool: follow the ASF Generative Tooling Guidance (Generated-by: <tool> in the commit message), and consider running the pre-flight in .skills/devs/pr-review/SKILL.md — fix what it finds; don't paste its report here
  • there should be no conflicts when merging the pull request branch into the recent main branch. If there are conflicts, please try to rebase the pull request branch on top of a freshly pulled main branch
  • if you add new module that downstream users will depend upon add it to relevant group in tika-bom/pom.xml.

We will be able to faster integrate your pull request if these conditions are met. If you have any questions how to fix your problem or about using Tika in general, please sign up for the Tika mailing list. Thanks!

…exceptions*

  The crash-ledger reports and summary.md join extract_exceptions to pipes_class,
  which was built by create-table-as with no index, so H2 scanned pipes_class once
  per exception row: 9 h on a compare with 339k extract exceptions.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@THausherr
THausherr requested a balanced review from Copilot October 23, 2026 19:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

5 open issues
What changed in this PR

Improves tika-eval report generation performance by adding missing indexes used by crash-ledger/summary joins and adds regression tests to ensure those indexes are present/used.

Changes:

  • Add indexes on container_id for pipes_class* tables generated via CTAS and for extract_exceptions* tables.
  • Add test assertions that EXPLAIN output references the expected index names for key joins.
File Description
tika-eval/​tika-eval-app/​src/​test/​java/​org/​apache/​tika/​eval/​app/​ProfilerBatchTest.java Adds EXPLAIN-based assertions to validate index usage in profiler reports.
tika-eval/​tika-eval-app/​src/​test/​java/​org/​apache/​tika/​eval/​app/​ComparerBatchTest.java Adds EXPLAIN-based assertions for both compare sides to validate index usage and imports Locale.
tika-eval/​tika-eval-app/​src/​main/​resources/​profile-reports.xml Creates new container_id indexes for pipes_class and extract_exceptions during report setup.
tika-eval/​tika-eval-app/​src/​main/​resources/​comparison-reports.xml Creates new container_id indexes for pipes_class_a/b and extract_exceptions_a/b during report setup.

Review effort: Lite

Comment thread tika-eval/tika-eval-app/src/main/resources/comparison-reports.xml
Comment thread tika-eval/tika-eval-app/src/main/resources/comparison-reports.xml
Comment thread tika-eval/tika-eval-app/src/main/resources/profile-reports.xml
@tballison
tballison merged commit 3a381e3 into apache:main Oct 26, 2026
4 checks passed
@tballison
tballison deleted the TIKA-4997 branch October 26, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants