Skip to content

Make typed reads statically compilable under juliac --trim - #26

Open
quinnj wants to merge 4 commits into
mainfrom
jq/trim-safe-typed-reads
Open

quinnj wants to merge 4 commits into
mainfrom
jq/trim-safe-typed-reads

Conversation

@quinnj

@quinnj quinnj commented Oct 5, 2026

Copy link
Copy Markdown
Member

Problem

juliac --trim=safe rejected typed queries. An application with a few hundred row structs hit 1,455 verifier errors, all in Postgres.jl:

  • 1,443: typed fields that aren't timestamps or decimals were decoded by the column's OID decoder, which returns Any, so every field's StructUtils.make call was a runtime dispatch.
  • 12: in protocol and parameter code.

Separately, reading a json/jsonb column (or any non-text column) into a String field threw MethodError: Cannot convert JSON.LazyValue{String} to String.

There was no failing test for the trim problem: the 1.0 release removed test/trim_compile_tests.jl.

Change

Typed reads (default PostgresStyle).

  • Fields whose type Postgres.jl parses from text now decode from the column text by their declared type. That covers strings, symbols, enums, integers, floats, Bool, Char, Date, Time, DateTime, UUID, bytea, lazy JSON, timestamps, decimals, vectors of these, and each wrapped in Union{Nothing,…}/Union{Missing,…}. The source stays a String, so each read compiles to static calls.
  • A String field receives any column's text.
  • Conversions that worked before are kept:
    • Integer fields accept integral numeric text such as 12.00 and throw InexactError when out of range.
    • A Date field reads the UTC date of a timestamptz.
  • A parser registered for the column still applies when its declared type fits the field.
  • Custom styles keep the OID decoder, so their lift methods still receive decoded values. Other field types keep it too.

Other trim fixes.

  • Protocol strings containing a NUL byte threw UndefVarError instead of PostgresInterfaceError. The check referred to Postgres.PostgresInterfaceError from inside API.
  • writestartupmessage took an unused statement_timeout. With options passed as a String, the call no longer needs runtime dispatch.
  • writemessages and tuple parameters to execute iterated heterogeneous tuples in a loop. Both now unroll; the parameter case reuses bind_tuple_params!.
  • Date/DateTime parameters used string(x), whose Dates printing is not trim-safe. They now use a static formatter that produces the same text.

Restored trim test.

  • test/trim_compile_tests.jl compiles test/postgres_trim_queries.jl with JuliaC and requires zero verifier errors. It then runs the executable against the test database under trust auth.
  • The workload covers nullable strings and timestamps, numerics into Float64/Int, enums, arrays, and jsonb into String. It also covers prepared statements, transactions, and parameters of each bound type.
  • JuliaC needs Julia 1.12, so it lives in test/trim/Project.toml, as in JSON.jl.
  • Released Julia's Base64 pipes and Dates.CompoundPeriod constructor are not trim-safe; SASLAuth's SCRAM exchange and interval decoding reach them. Errors raised inside those stdlib files are reported but not counted. The compile uses --trim=unsafe-warn so the executable still builds.

Version bumped to 2.3.0 for the behavior changes below.

Behavior changes (default style, typed fields)

  • An Int/Float field over a bool column now errors; it previously read 1/0.
  • A registered parser whose declared type doesn't fit the field is no longer applied and then converted. The field parses the column text instead.

Verification

  • Full test suite locally, including the live-database tests and the restored trim test: 0 Postgres.jl verifier errors (30 stdlib-gap errors reported separately), and the compiled executable ran its workload successfully. The only other error was the Kerberos fixture, whose Docker image failed to install packages (Could not resolve deb.debian.org); it passed on the previous run.
  • In a downstream application (about 1,400 typed queries) built with juliac --trim=safe and stdlib patches, verifier errors went from 1,472 to 17. The remaining 17 are pre-existing and in the application's own code. Its 8,751-test integration suite passes.

🤖 Generated with Claude Code

quinnj and others added 4 commits October 5, 2026 16:22
`juliac --trim=safe` rejected typed queries: an application with a few
hundred row structs hit 1,455 verifier errors, all in Postgres.jl. Typed
fields that are not timestamps or decimals were decoded by the column's OID
decoder, which returns `Any`, so every field's `StructUtils.make` call was a
runtime dispatch. Reading a json/jsonb (or any non-text) column into a
`String` field also failed with a MethodError, because the OID decoder
produced lazy JSON, integers, UUIDs and so on.

With the default `PostgresStyle`, fields whose type Postgres.jl parses from
text (strings, symbols, enums, integers, floats, Bool, Char, Date, Time,
DateTime, UUID, bytea, lazy JSON, timestamps, decimals, and vectors of these,
optionally unioned with Nothing or Missing) now decode from the column text
by their declared type. The source stays a `String`, so each read compiles
to static calls. A `String` field receives any column's text. Decoders keep
the conversions that worked before: integer fields accept integral numeric
text such as "12.00" and throw InexactError when out of range, and a Date
field reads the UTC date of a timestamptz. A parser registered for the
column still applies when its declared type fits the field. Custom styles
keep the OID decoder, so their lifts still receive decoded values; other
field types keep it too.

Also fixed while verifying under `--trim`:
- Protocol strings containing a NUL byte threw UndefVarError (the check
  referred to `Postgres.PostgresInterfaceError` from inside the API module)
  instead of PostgresInterfaceError.
- The startup message took an unused `statement_timeout` argument; with
  `options` passed as a String the call no longer needs runtime dispatch.
- `writemessages` and tuple parameters to `execute` iterated heterogeneous
  tuples in a loop; both now unroll.
- Date and DateTime parameters were formatted with `string(x)`, whose Dates
  printing is not trim-safe; they use an equivalent static formatter.

Behavior changes for typed fields read with the default style: an Int or
Float field over a bool column now errors (it previously read 1 or 0), and
a registered parser whose declared type does not fit the field is no longer
applied before converting; the field parses the column text instead.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The 1.0 release removed test/trim_compile_tests.jl, so later changes to
typed decoding regressed `--trim` builds without a failing test. The
restored test compiles test/postgres_trim_queries.jl with JuliaC, requires
zero verifier errors, and runs the executable against the test database
(trust auth). The workload covers typed reads of nullable strings, dates and
timestamps, numerics into Float64 and Int fields, enums, arrays, and jsonb
into a String field, plus prepared statements, transactions and parameters
of each bound type.

JuliaC needs Julia 1.12, so it lives in test/trim/Project.toml and the test
builds that environment only on released 1.12+ (64-bit, non-Windows), as
JSON.jl does. Released Julia's Base64 pipes and Dates.CompoundPeriod
constructor are not trim-safe; SASLAuth's SCRAM exchange and interval
decoding reach them. Errors raised inside those stdlib files are reported
and not counted, and the test compiles with `--trim=unsafe-warn` so the
executable still builds.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The Kerberos fixture's cleanup ran `DROP ROLE krbuser` right after closing
the last krbuser connection. That connection created a temp table, and
PostgreSQL drops a session's temp tables only as its backend exits, which
can trail `close!`. On a fast runner the DROP ROLE won the race and failed
with 2BP01 ("owner of table pg_temp_4.bulk_gss"). Poll pg_stat_activity
until the role has no sessions (up to 10s) before dropping it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant