Skip to content

feat(compilerlib): compile to an llvm::Module in the caller's context - #265

Draft
SizzleUnrlsd wants to merge 10 commits into
mainfrom
feat/compile-to-module
Draft

SizzleUnrlsd wants to merge 10 commits into
mainfrom
feat/compile-to-module

Conversation

@SizzleUnrlsd

@SizzleUnrlsd SizzleUnrlsd commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The CoreTrace analyzers compile each source to textual IR or bitcode, then parse it into a module of their own (#263). compileToModule(args, context, instrument), in a new public header compilerlib/compile_module.hpp, gives the module itself, created in an llvm::LLVMContext of the host. The output file is not written; the other files the arguments ask for, such as a dependency file, are.

Closes #263

Change

  • refactor(compilerlib): run compile() through runCompilation: the compilation runs from the context it is given. compile does not change.
  • feat(compilerlib): compile to an llvm::Module in the caller's context:
    • compileToModule takes the arguments of an LLVM IR compilation, -S -emit-llvm or -c -emit-llvm, and fails on any other;
    • code generation creates the module in the host's context, through the action that compile already uses for in-memory output; with instrument, the module is instrumented first;
    • whether the context discards value names is restored after the compilation, since it applies to the modules read into the context later. Before LLVM 22, the optimization record that Clang's action leaves on the context is removed too.
  • test(compilerlib): check the modules compileToModule gives: the module prints as the text compile gives, at -O0 and -O2, instrumented or not; a bitcode compilation writes its dependency file and no output; a compilation to an object and a broken source fail; the context keeps its setting of value names.
  • docs(readme): describe compiling to a module.
  • fix(compilerlib): keep value names in an LLVM 16 context given some, with its README and changelog notes. The Linux / LLVM 16 job of this pull request aborted in ModuleLeavesTheValueNamesSettingOfTheContext: double free or corruption. With LLVM 16, Value::setName("") does nothing in a context that discards names, so destroying a module whose values have names, after the context was set back to discarding them, left each argument's name in its function's symbol table, which freed it again. LLVM 17 removes the name (16.0.6 against 17.0.6, Value::setNameImpl). With LLVM 16, a context that discards names now keeps them after a compilation that names values; the test expects that with LLVM 16, and the restored setting from LLVM 17 on.

This branch comes from commit 2c1c83b of the experimental branch, without what it took from the working directory change: #262 brings that separately. The two change compile in compiler.cpp, so the second to merge needs the other one merged in first.

Measurements

coretrace-stack-analyzer db7fb25, with coretrace-compiler built from main (f0a22d9), against the same analyzer taking each module from compileToModule in its bitcode mode, instead of compiling to a temporary .bc file, reading it and parsing it. That analyzer change is not published yet: one of its regression tests looks for the log line of the bitcode parse. macOS arm64 (M3, 8 cores), LLVM 20.1.2, --format=json --timing, no IR cache, an empty resource summary cache. Medians of 5 runs, 3 for the 53 sources; the builds alternate within each repetition.

Sources --jobs main this branch Loading: main, this branch
Lua 5.4.6, 34 C files 1 7.70 s 7.72 s 1.14 s, 0.97 s
2 7.68 s 7.69 s 1.12 s, 0.97 s
8 C++ files, each including 11 standard headers 1 4.27 s 3.90 s 3.81 s, 3.49 s
2 4.39 s 4.02 s 4.03 s, 3.64 s
coretrace-stack-analyzer's 53 C++ sources 1 55.0 s 51.9 s 44.1 s, 40.7 s
2 52.3 s 50.8 s 42.2 s, 40.3 s
  • Loading, at --jobs=1, takes 8 % less time on both sets of C++ sources and 15 % less on Lua's. The analyzer still compiles one source at a time here, whatever --jobs says; feat(compilerlib): give each compilation a working directory of its own #262 is the change that lets it compile in parallel.
  • Reports: the same diagnostics and functions with both builds. The order of Lua's diagnostics, and so their ids, varies from run to run with either build.
  • Peak memory does not grow: the medians stay within the range of main's runs, or below it on Lua (166 MB against 177 to 189 MB at --jobs=1).

Tests

Local runs, on macOS arm64 with AppleClang:

Run Result
LLVM 20.1.2, ctest -j2 265/265, the 257 of main plus 8 new tests.
LLVM 16.0.6, Debug, before the LLVM 16 fix ModuleLeavesTheValueNamesSettingOfTheContext aborts: pointer being freed was not allocated, in ~ValueSymbolTable from ~Module, as on the CI's Linux / LLVM 16 job.
LLVM 16.0.6, after the fix all 171 unit tests pass.

Each commit builds and passes the unit tests with LLVM 20. With LLVM 16, ModuleLeavesTheValueNamesSettingOfTheContext fails from the commit that adds it until the fix. clang-format 18, scripts/check-license-compliance.sh and the commit checker pass.

CI at 3c715b2

  • LLVM versions run 43170150694: 15/15 jobs pass. Every test passes in each job: 268 on Windows, 265 on Linux and macOS. ModuleLeavesTheValueNamesSettingOfTheContext passes in all 15, Linux / LLVM 16 included, where it aborted at 3dd3255.
  • Build run 43170150808: passes. The Docker job is skipped, as on every pull request.
  • Other checks: clang-format, license-compliance and the commit conventions pass, and so do the runs of the push to this branch.

compile() built its context and ran the whole compilation itself. The
compilation now runs in runCompilation, from the context it is given,
which compileToModule will share.

Refs #263
The analyzers compile each source to textual IR or bitcode, then read
it back into a module of their own: the module was written and parsed
again for nothing. compileToModule(args, context, instrument) gives
the module itself, created in the caller's context, with the arguments
of an LLVM IR compilation (-S -emit-llvm or -c -emit-llvm). The output
file is not written; the other files the arguments ask for, such as a
dependency file, are.

The module is the one compile() writes, instrumented or not. Code
generation sets whether the context discards value names; the
context's own setting is restored, since it applies to the modules
read into it later, and so is the optimization record that Clang's
action leaves on it before LLVM 22.

Refs #263
compileToModule gives the module that compile() writes as text, at -O0
and -O2, instrumented or not, and the module of a bitcode compilation,
whose output file is not written and whose dependency file is. It
fails on a compilation that is not to LLVM IR, reports the errors of a
broken source, and leaves the context's setting of value names.

Refs #263
@SizzleUnrlsd SizzleUnrlsd added the enhancement New feature or request label Nov 12, 2026
@SizzleUnrlsd SizzleUnrlsd self-assigned this Nov 12, 2026
With LLVM 16, destroying a module of compileToModule freed memory twice
when the compilation named values and the context, restored after it,
discarded names: LLVM 16's Value::setName("") does nothing in such a
context, so Function::clearArguments leaves each argument's name in
the function's symbol table, which frees it again. LLVM 17 removes the
name. With LLVM 16, a context that discards names now keeps them after
a compilation that names values. The Linux / LLVM 16 job of the CI
aborted in ModuleLeavesTheValueNamesSettingOfTheContext.

Refs #263
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hosts get each module as text or bitcode, then parse it back

1 participant