PALM

Rust tests, guided by program analysis.

Website & interactive demo · Quick start · Paper · 中文技术指南

PALM generates Rust tests by combining program analysis with large language models. It extracts path constraints and code context, generates candidates, repairs compilation errors, and reports execution results and coverage with test code excluded.

This is the maintained implementation of the ASE 2025 paper, with changes made after the paper's experiments.

Illustration of the minimal offline fixture, using fixed local responses. It does not measure model quality. Explore both paths on the website.

Quick start

First complete the prerequisites: Git, Rust, a C linker, Python 3.9+, and the pinned analysis and coverage tools. Then run the minimal pipeline without API keys:

git clone https://github.com/cppbear/PALM.git
cd PALM
cargo build --workspace --locked
python3 scripts/check_minimal.py

The script works in temporary copies, prints their location, and finishes with:

Minimal analysis/generation/repair/coverage checks passed.

See minimal pipeline validation for the checked scenarios. For a real model, install PALM, select a runtime configuration, and follow a bounded example.

Prerequisites

Use a native build environment with Git, rustup, stable Rust, a C linker, and Python 3.9 or later for the validation scripts. Tool and dependency downloads may require network access.

1. Install the pinned analysis toolchain and components:

   rustup toolchain install nightly-2025-03-19 --profile minimal \
     --component rust-src --component rustc-dev --component llvm-tools-preview --component rust-analyzer

2. Install cargo-llvm-cov:

   cargo +stable install cargo-llvm-cov --version 0.6.16 --locked

Use the same nightly for the target package. rust-analyzer provides editor support; compiler analysis uses components such as rustc-dev. Building, installation, and the offline checks need no model credentials.

Continue with the quick start once the prerequisites are installed.

What PALM does

• Extracts representative path constraints and the focal function's dependencies for model prompts.
• Generates unit tests or integration tests, with explicit function selection and request budgets.
• Uses compiler diagnostics to repair candidates that do not compile.
• Reports compilation, execution, and coverage results for each focal function, preserving library/binary ownership.

How it works

flowchart TD
    A["Rust package"] --> P["Representative<br/>path constraints"]
    A --> X["Focal-function<br/>context"]
    P --> C["Model-generated<br/>test candidates"]
    X --> C
    C --> D["Compilation check"]
    D -->|Compiles| F["Test execution<br/>and coverage"]
    D -->|Compiler errors| E["Model repair"]
    E -.->|Recheck| D
    classDef source fill:#f7f7fc,stroke:#8774ab,color:#222133
    classDef analysis fill:#eeebf8,stroke:#8774ab,color:#222133
    classDef generate fill:#5941bc,stroke:#5941bc,color:#ffffff
    classDef repair fill:#f4eaf2,stroke:#854d79,color:#64375b
    classDef result fill:#e8f2eb,stroke:#276047,color:#214f3a
    class A,D source
    class P,X analysis
    class C generate
    class E repair
    class F result

Loading

Repair addresses compilation errors. Runtime assertion failures and candidate timeouts remain failed test outcomes.

A small example

The minimal fixture includes this function, whose support module checks whether the input is positive:

pub fn classify(value: i32) -> i32 {
    if support::positive(value) {
        1
    } else {
        0
    }
}

The offline check supplies fixed local model responses with assertions for both paths. These excerpts omit the test wrappers and helper assertions:

assert_eq!(classify(2), 1);
assert_eq!(classify(-1), 0);

In a separate case, nested::double takes an i32. The check supplies an intentionally invalid candidate and a fixed repair response:

- assert_eq!(double("bad"), 4);
+ assert_eq!(double(2), 4);

The check validates compilation repair, execution, coverage, and source restoration with real Rust tools. Its fixed local responses do not measure model quality. Explore the illustrated examples, run the offline check, then use the validation guide to interpret its output.

Installation

From the repository root:

./install.sh

This installs brinfo, call_chain, focxt, and utgen. Ensure Cargo's binary directory, typically $HOME/.cargo/bin, is in PATH. The installer also works from another directory; ./install.sh --help lists tool selection options. See build and installation checks for verification and the component READMEs below for individual installation commands.

Model configuration is loaded at runtime by gen and fix. Changing the API address, key, or model takes effect on the next command without rebuilding.

Docker

Run docker/docker-build, then docker/docker-run to open a shell with the repository mounted at /home/palm/palm. Follow Installation inside the container. The image includes the pinned compiler and coverage tools; files written in the mounted repository remain on the host after the container exits.

Both scripts locate the repository from their own paths. See Docker validation and CI scope for the checked environment and optional container pipeline check.

Workflow

Use a fresh working copy of the target package: preprocessing changes its source and test layout, and generation may add a test dependency.

1. Run utgen pre-process, then utgen analyze on that copy.
2. Generate candidates with utgen gen --requirement --context, using a function list and request cap for an initial trial.
3. Run utgen fix with the same function list and mode, then inspect request reports and per-function statistics.

Use --integration on both generation and repair commands for integration tests. The examples provide complete commands for small unit, integration, and mixed-target trials. A completed command does not imply every candidate passed; result fields report the outcomes.

To measure existing tests, run utgen coverage -p <original-crate-copy> before preprocessing, on a separate copy. It needs no model configuration and retains coverage.xml and coverage.json.

Supported scope

Area	Current scope
Target package	One standalone Cargo package with ordinary lib/bin targets, including custom names and entry files. The complete workflow does not support multi-package workspaces.
Test modes	Unit tests use the recorded lib/bin target. Function-level integration tests require a library and compile-check external access.
Shared definitions	A definition compiled into both lib and bin is represented by lib. This does not test every binary variant.
Rust features	Analysis follows the active build configuration. There is no general macro expansion or automatic feature-matrix exploration.
Toolchain	nightly-2025-03-19 and cargo-llvm-cov 0.6.16.
Platforms	Full offline suite on Linux CI; exercised locally on macOS Apple Silicon. Windows has not been validated.

See the CLI reference for source-processing boundaries, cache compatibility, timeouts, and coverage interpretation.

Project Structure

Directory	Purpose
brinfo	Extract condition chains from HIR and MIR.
focxt	Construct context for each focal function.
focxt/call_chain	Extract calls and type dependencies.
utgen	Generate, compile-check, repair, and evaluate tests.
build-utils	Configure compiler library search paths during builds.
examples	Minimal and mixed-target fixtures, plus the bundled bytes target.
docs	Technical, usage, validation, and research documentation.
docker/	Container build and run scripts.

Development checks

cargo build --workspace --locked
cargo test --workspace --locked

Default checks use local model responses. See CONTRIBUTING.md for the regression check relevant to a change, and build validation for CI scope. The real-model request test is explicitly opt-in.

Contributing

Bug reports and focused pull requests are welcome. Include a reproducer and relevant validation as described in CONTRIBUTING.md.

Documentation

• Examples: bounded unit, integration, and mixed-target trials.
• Technical guide in Chinese: architecture, data flow, and implementation.
• CLI reference: configuration, commands, and outputs.
• Research materials: conference poster and presentation, with revision notes.
• Documentation index: all guides and validation entry points.

Citation

If you use PALM in your research, please cite PALM: Synergizing Program Analysis and LLMs to Enhance Rust Unit Test Coverage (ASE 2025). Published paper · Open-access preprint. CITATION.cff provides the machine-readable citation.

@inproceedings{Chu2025PALM,
  author    = {Bei Chu and Yang Feng and Kui Liu and Hange Shi and Zifan Nan and Zhaoqiang Guo and Baowen Xu},
  title     = {{PALM}: Synergizing Program Analysis and {LLMs} to Enhance {Rust} Unit Test Coverage},
  booktitle = {2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE)},
  year      = {2025},
  pages     = {2720--2732},
  doi       = {10.1109/ASE63991.2025.00223}
}

License

PALM is licensed under the MIT License. The bundled bytes example retains its original MIT license and copyright notice. Existing third-party notices and rights in included material remain applicable.