Skip to content

[CALCITE-7844] Add an opt-in type system for decimal division - #5316

Open
alexandrefimov wants to merge 5 commits into
apache:mainfrom
alexandrefimov:CALCITE-7844
Open

alexandrefimov wants to merge 5 commits into
apache:mainfrom
alexandrefimov:CALCITE-7844

Conversation

@alexandrefimov

@alexandrefimov alexandrefimov commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Add DecimalDivisionTypeSystem to reduce decimal division scale using integral digits instead of total precision. DECIMAL(6,2) / DECIMAL(6,2) derives DECIMAL(15,9); the default remains DECIMAL(15,6).

Enable it through the typeSystem connection property. A JDBC test shows how to use it with default SQL conformance. The JDBC wrapper is unchanged.

Validation on Java 21: all 5 DecimalDivisionTypeSystemTest tests, main/test Checkstyle and git diff --check passed. The full test suite was not run.

Jira: https://issues.apache.org/jira/browse/CALCITE-7844

@alexandrefimov alexandrefimov changed the title [CALCITE-7844] Fix decimal division scale reduction to use integral digits [CALCITE-7844] Incorrect scale reduction for DECIMAL division Oct 9, 2026
@mihaibudiu

Copy link
Copy Markdown
Contributor

As you can see from the test results, this is a breaking change, which will potentially affect all projects that depend on Calcite.
Unless this is a clear bug, I don't think it can be merged as it is.

@alexandrefimov

Copy link
Copy Markdown
Contributor Author

Agreed, changing the default has a compatibility cost. The bug is that the condition checks total precision instead of integral digits. For example, DECIMAL(6,2) / DECIMAL(6,2) loses three fractional digits even though DECIMAL(15,9) fits within the default precision limit of 19.

Would an opt-in fix through a type system override be a reasonable way forward?

@mihaibudiu

Copy link
Copy Markdown
Contributor

Yes, I think the right way to do this is to use a different type system.
You can contribute a new type system and a test, but probably the default has to be left as it is.

Restore the default decimal division rules and existing SQL expectations. Add DecimalDivisionTypeSystem for callers that opt in through the typeSystem connection property, with regression coverage for type inference and JDBC metadata.
Forward decimal division derivation to the configured type system when JDBC applies the conformance wrapper. Cover default and opt-in division metadata in DEFAULT, PRAGMATIC_2003 and ORACLE_12 modes.
@alexandrefimov alexandrefimov changed the title [CALCITE-7844] Incorrect scale reduction for DECIMAL division [CALCITE-7844] Add an opt-in type system for decimal division Oct 16, 2026
final RelDataTypeSystem originalTypeSystem = typeSystem;
typeSystem =
new DelegatingTypeSystem(typeSystem) {
new DelegatingTypeSystem(originalTypeSystem) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the connection between shouldConvertRaggedUnionTypesToVarying and this overload.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conformance flag installs a DelegatingTypeSystem wrapper. That class inherits the default deriveDecimalDivideType, so it loses the configured division rule. This override preserves that rule. Without it, DECIMAL(6,2) / DECIMAL(6,2) returns DECIMAL(15,6) instead of DECIMAL(15,9) in these conformance modes.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't understand why the ragged union rule is related to division.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ragged-union flag causes JDBC to wrap the configured type system in DelegatingTypeSystem. That wrapper uses the default deriveDecimalDivideType instead of the custom override. I added the forwarding here so that enabling this conformance setting would preserve the custom division rule.

This is an existing problem with how JDBC wraps custom type systems. I suggest fixing it in a separate PR with a regression test, then keeping this PR focused on opt-in decimal division. Would that make sense?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are misusing this flag. This flag controls type inference for CASE statements, not for division.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the flag is for character and binary types. I added the override because the wrapper created here drops the configured division rule, but I see why including that fix here mixes two concerns.

I reproduced the same problem with an independent custom type system on current main. Would you prefer a separate PR for preserving custom overrides in the JDBC wrapper, keeping this PR focused on opt-in decimal division?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure Calcite needs any changes.
Can't you just use the same trick to install a modified type system in your own system?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may be misunderstanding the scope of your suggestion. Would you prefer the custom division type system to live downstream, or is adding it to Calcite still an option if this PR leaves the JDBC wrapper unchanged?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about this: you should add the type system as a toplevel, class, and create a test which shows how to use this modified type system? This will be a guide for everyone who needs to use it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed the JDBC wrapper change. The PR now contains the top-level DecimalDivisionTypeSystem class and tests, including an example that enables it through the typeSystem connection property with default SQL conformance.

@sonarqubecloud

Copy link
Copy Markdown

@mihaibudiu mihaibudiu added the LGTM-will-merge-soon Overall PR looks OK. Only minor things left. label Oct 26, 2026
@mihaibudiu

Copy link
Copy Markdown
Contributor

If no one objects I plan to merge this. This change does not affect any existing behavior of Calcite, it just shows how one can use a new type system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

LGTM-will-merge-soon Overall PR looks OK. Only minor things left.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants