Skip to content

fix: preserve numeric comparison precision - #993

Open
kaptinlin wants to merge 1 commit into
expr-lang:masterfrom
kaptinlin:codex/numeric-comparisons
Open

kaptinlin wants to merge 1 commit into
expr-lang:masterfrom
kaptinlin:codex/numeric-comparisons

Conversation

@kaptinlin

Copy link
Copy Markdown

Numeric comparisons currently convert integer operands to int, so uint64(math.MaxUint64) > uint64(0) evaluates to false and uint64(math.MaxUint64) == int64(-1) evaluates to true. Converting integers to float64 also makes distinct values beyond 2^53 compare equal.

Preserve signedness and width in the comparison generator. Mixed integer/float comparisons check the integer range and compare the integral part before the fractional part, preserving IEEE NaN ordering. Regenerate the five comparison helpers and add public Compile/Run regressions for typed and dynamic environments.

Validation on Go 1.27:

  • go test -p 1 ./...
  • go test -race -p 1 .
  • go test -p 1 -tags=expr_debug -run=TestDebugger ./vm
  • go vet ./...
  • go test -p 1 -ldflags='-s -w' -run '^$' -fuzz '^FuzzNumericComparison$' -fuzztime=20s -parallel=1 . passed 392,471 inputs against a math/big.Rat oracle.
  • Formatting, regeneration consistency and clean-base patch application passed.

A temporary go-valueexpr workspace also reproduces the corrected comparisons while preserving uint64 identity and its existing Shape type checks. Arithmetic helpers are byte-identical to the base.

@kaptinlin
kaptinlin marked this pull request as ready for review October 3, 2026 17:07

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant