Skip to content

fix(flow): verifier-audit minor follow-ups (keyless-table diff, seed-cache version, tz-aware stamps) - #1807

Open
abrichr wants to merge 1 commit into
feat/verifier-audit-kitfrom
fix/verifier-audit-minor
Open

abrichr wants to merge 1 commit into
feat/verifier-audit-kitfrom
fix/verifier-audit-minor

Conversation

@abrichr

@abrichr abrichr commented Nov 4, 2026

Copy link
Copy Markdown
Member

Three minor follow-ups from the independent review of #1783. This PR is based on feat/verifier-audit-kit; it retargets to main when #1783 merges.

  • AppWorld row diff: EXCEPT is a set difference. A table without a primary key is now compared with its rowid included, so two identical inserts count as two rows. Before this, a duplicate insert into such a table would have been classed as DEGENERATE.
  • AppWorld seed cache: the cache is now keyed by AppWorld version as well as task, so a cache from another release is not reused.
  • AgentDojo clock normalization: a timezone-aware timestamp is converted to local time before the comparison with the run window. Before this, it raised a TypeError.

Tests: two new unit tests (a keyless table with a duplicate insert; an aware timestamp in and out of the window). 19 pass. ruff and mypy are clean.

…eed cache by version, accept tz-aware stamps
@abrichr

abrichr commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

From: Grok, To: coordinator — review of #1807 at 0686b93 (base feat/verifier-audit-kit): APPROVE

What holds:

  • In _delta, a table without a key is compared with its rowid included, so two identical inserts count as two added rows. _has_primary_key reads table_info from the schema the table exists in.
  • AppWorld's save_local_dbs, as _dump calls it, does not VACUUM (vaccum defaults to False), so rowids are not renumbered between the seed dump and the end dump.
  • from appworld import __version__ exists upstream (src/appworld/init.py sets it from the package metadata), so _seed_dir does not raise.
  • In _clock_free, a stamp with a timezone is converted to naive local time before it is compared with the naive local run window. A naive stamp is handled as before.

Notes, not blocking:

  1. With rowids included, a keyless row that is deleted and then re-inserted with identical content but a new rowid shows as one removed and one added row, even though the multiset did not change. If a true multiset is wanted, a count per distinct row (GROUP BY the columns with COUNT(*)) would give it without depending on rowids. Upstream AppWorld tables appear to have keys, so this is defensive either way.
  2. The seed cache is keyed by the version of the appworld package. Seed data downloaded separately into appworld_root can change without a package bump. Adding a hash of the data directory's version file, if one exists, would close that gap.

Testing:

  • tests/test_verifier_audit.py at the head: 19 passed.
  • Control with the base branch's openadapt_flow: the 2 new tests fail and 17 pass.
  • ruff check and ruff format --check are clean on the 3 files.
  • I did not run a live AppWorld or AgentDojo replay.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant