Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@
target/
*.txt
!/examples/bytes-smoke-*.txt
!/examples/mixed-smoke-*.txt
api.json
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ Generation or repair completing does not mean every candidate passes. Inspect bo
| [focxt/call_chain](focxt/call_chain/README.md) | Extract calls and type dependencies through `cargo call-chain`. |
| [utgen](utgen/README.md) | Generate tests, check compilation, repair tests, and collect coverage. |
| [build-utils](build-utils/README.md) | Configure the compiler library search path during builds. |
| [examples](examples/README.md) | The minimal fixture, bundled bytes target, and usage instructions. |
| [examples](examples/README.md) | Minimal and mixed-target fixtures, the bundled bytes target, and scoped trial instructions. |
| `docker/` | Container build and run scripts. |
| [docs](docs/README.md) | Technical guide, validation guides, and ASE 2025 materials. |

Expand Down
22 changes: 19 additions & 3 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

Start with the [minimal pipeline](../docs/minimal-pipeline.md) for an automated, no-credentials validation. The commands below use a working copy of the larger bytes target and limit model generation and repair to selected functions.

| Example | Purpose |
| --- | --- |
| [minimal](minimal/README.md) | Offline generation, compilation repair, execution, and coverage with fixed local model responses. |
| [mixed-targets](mixed-targets/README.md) | Library/binary ownership checks and a four-function model trial. |
| [bytes](#bytes) | Larger analysis target, with limited unit and integration model trials below. |

The [bytes analysis check](../docs/bytes-analysis.md) validates all 663 exported functions and their context artifacts on two fresh copies. Run `cargo build --workspace --release --locked` and `python3 scripts/check_bytes.py` from the repository root. This check does not call a model service or measure generated-test quality.

## bytes
Expand Down Expand Up @@ -68,15 +74,25 @@ Both commands default to four active function tasks and a 180-second deadline pe

### Integration Test Mode

Use a fresh working copy and repeat preprocessing and analysis before choosing this mode, since existing generation results are reused independently of these flags:
Use a fresh working copy, since unit and integration candidates cannot share a generation cache. After setting `palm_repo` and `PALM_CONFIG` as above, this trial selects only `Bytes::len` and `BytesMut::truncate` from the [integration function list](bytes-smoke-integration-2.txt):

```sh
utgen gen -p <target-crate-path> --integration --requirement --context
utgen fix -p <target-crate-path> --integration
palm_integration_dir="$(mktemp -d "${TMPDIR:-/tmp}/palm-bytes-integration.XXXXXX")"
cp -R "$palm_repo/examples/bytes/." "$palm_integration_dir/"
utgen pre-process -p "$palm_integration_dir"
mv "$palm_integration_dir/tests.bak" "${palm_integration_dir}-original-tests"
utgen analyze -p "$palm_integration_dir"

utgen gen -p "$palm_integration_dir" --integration --requirement --context \
--functions-file "$palm_repo/examples/bytes-smoke-integration-2.txt" --max-requests 8
utgen fix -p "$palm_integration_dir" --integration \
--functions-file "$palm_repo/examples/bytes-smoke-integration-2.txt" --max-requests 8
```

Integration mode generates files under the target's `tests/` directory after filtering functions using the analysis visibility flag and checking compilation. Pass `--integration` to `fix` as well to repair and evaluate candidates in integration-test targets. Imports and helpers stay with each candidate, and both combined and per-candidate coverage retain integration scope. Keep the same function selection for both commands; mixing cached generation modes is rejected.

The limits above allow up to eight request attempts for generation and eight for repair, including retries. Inspect a failed command before continuing; rerunning a command starts a new request budget. The original integration tests are preserved beside the working copy so this trial evaluates only the selected candidates. If `tests.bak/` remains inside the crate instead, PALM also evaluates the original integration tests for comparison, independently of the function list.

### Results

Read the outputs in this order:
Expand Down
2 changes: 2 additions & 0 deletions examples/bytes-smoke-integration-2.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
bytes::bytes::bytes::Bytes::len
bytes::bytes_mut::bytes_mut::BytesMut::truncate
4 changes: 4 additions & 0 deletions examples/mixed-smoke-4.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
lib:mixed_fixture::mixed_fixture::shared::classify
lib:mixed_fixture::mixed_fixture::helper
bin:mixed_fixture::mixed_fixture::helper
bin:mixed_fixture::mixed_fixture::bin_only
24 changes: 24 additions & 0 deletions examples/mixed-targets/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,27 @@ python3 scripts/check_mixed.py
```

The script works in temporary copies with fixed local model responses. It checks analysis ownership, dependency context, unit and integration statistics, compilation repair, and source restoration. Additional temporary variants cover custom Cargo targets and definitions shared between binaries. The fixture's `main` panics so accidental execution is visible; the generated unit-test pipeline must use the test harness.

## Limited model trial

After [installing PALM](../../README.md#installation) and [configuring a model](../../utgen/README.md#prerequisites), run from the repository root:

```sh
palm_repo="$(pwd)"
export PALM_CONFIG="$palm_repo/utgen/res/api.json"
palm_mixed_dir="$(mktemp -d "${TMPDIR:-/tmp}/palm-mixed.XXXXXX")"
cp -R "$palm_repo/examples/mixed-targets/." "$palm_mixed_dir/"
utgen pre-process -p "$palm_mixed_dir"
utgen analyze -p "$palm_mixed_dir"

utgen gen -p "$palm_mixed_dir" --requirement --context \
--functions-file "$palm_repo/examples/mixed-smoke-4.txt" --max-requests 12
utgen fix -p "$palm_mixed_dir" \
--functions-file "$palm_repo/examples/mixed-smoke-4.txt" --max-requests 8
```

The [four-function list](../mixed-smoke-4.txt) selects the shared `classify` definition through the library, both distinct `helper` definitions, and the binary's `bin_only` function that calls the library. Its target-qualified names are exact keys from `brinfo/name_map.json`; the ambiguous name `mixed_fixture::helper` alone cannot select one helper.

Both commands use unit mode and default to four active function tasks. The request caps include retries; a new invocation starts a new budget. Check each command's status and [result files](../README.md#results) before proceeding. Repair addresses compilation errors; runtime assertion failures remain failed outcomes. Keep the working copy and model responses outside the repository.

These selections include binary functions, so they cannot be passed unchanged with `--integration`. Integration mode requires library functions; see the [two-function integration trial](../README.md#integration-test-mode). Shared lib/bin definitions are represented by their library compilation, while binary-specific functions use the binary test harness.
2 changes: 1 addition & 1 deletion utgen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ The following is a command reference rather than captured help output. Use `utge
| `analyze` | Clear Cargo's check cache, run brinfo and focxt, and validate their function indices and artifacts. Currently supports one standalone crate passed with `-p`. |
| `coverage` | Run existing tests once and export XML/JSON with test code excluded. Supports one standalone crate passed with `-p`. |
| `gen` | Generate candidates, check compilation, and collect pre-repair statistics. |
| `fix` | Attempt to repair candidates with compilation errors and collect post-repair statistics using unit-test insertion. |
| `fix` | Attempt to repair compilation errors and collect post-repair statistics, using unit-test insertion or integration-test targets with `--integration`. |

All commands take `-p, --project-dir`. Except for `coverage`, commands also accept `-w, --work-dir` for individual crates in a larger project. Both relative paths are resolved against the current shell directory; work directories do not resolve against `--project-dir`. Work directories may be repeated or comma-separated and default to the project directory. The current `analyze` command requires a single standalone crate; use that crate as `-p`, with no separate work-directory selection.

Expand Down
Loading