Repository navigation
Conversation
Reference counting checks whether each relationship row comes from a dirty or deleted file. Both checks can rediscover the repository and read HEAD, so a kernel-scale survey repeatedly performs the same work. Snapshot the excluded paths and overlay counts together before scanning. Stream the LanceDB batches so the query can consume each batch as it arrives. Exercise both clean and edited worktrees in the reference-count test. Co-authored-by: Chris J Arges <[email protected]>
The working-copy overlay is one slot per process and only describes the checked-out commit when git-only mode is off. ensure_workdir_index_built already declines to build or refresh it otherwise, but every workdir_* helper, the file_survey reference counts and generate_git_manifest read the slot regardless. An overlay filled by an earlier query at HEAD was therefore applied to queries naming another commit, to the same commit after HEAD moved, and to git-only queries: indexed rows in files modified in the working copy were dropped and working-copy-only functions added. ensure_workdir_index_built now reports whether the overlay may answer for git_sha, and all reads go through with_workdir, which returns an empty result when it may not. generate_git_manifest checks eligibility before merging modified and deleted paths, still without building the overlay. The reference counts also refresh the overlay before generating the manifest. Previously a file modified in an earlier REPL command and since restored kept its stale working-copy hash in the manifest, so its indexed callers were skipped.
12458
marked this pull request as draft
November 20, 2026 05:40
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #140. This branch is built on
file-survey-reference-counts, so until #140 merges this PR also shows its commit. Review only the last commit (workdir: only read the overlay for the revision it describes).Problem
DatabaseManagerkeeps one working-copy overlay per process. That's a parsed snapshot of the uncommitted edits in the checkout. It describes only the commit that's checked out, and only when the caller hasn't asked for git-only results.workdir_overlay_usable(git_sha)encodes that rule, andensure_workdir_index_builtuses it to decide whether to build or refresh the overlay.The rule decides when the overlay gets built, but nothing checks it when the overlay is read. Each
workdir_*helper callsensure_workdir_index_builtand then readsself.workdir_indexregardless of the result. If the overlay isn't usable,ensure_workdir_index_builtreturns early without clearing it, so whatever an earlier query built is still there. Three things read the overlay this way:workdir_*lookup helpers: function and type lookup, callers, callees, grep, regex, and the deleted/dirty-count checks;get_distinct_reference_counts_git_aware, the working-copy half of file_survey's reference counts;generate_git_manifest, which merged the overlay's dirty and deleted paths into every manifest, for any commit.A query at the checked-out commit fills the overlay. Later in the same process, the overlay is read for something it doesn't describe:
set_git_only(true).That query then mixes in edits made on top of a different revision. Indexed rows in files that are modified in the working copy get dropped, and functions that exist only in the working copy get added.
Example: HEAD is A, and
a.chas an uncommitted edit adding a call tooverlay_only(). A file_survey at A builds the overlay. A git-only query at A, or a query at A after HEAD has moved past it, still reports one caller ofoverlay_onlyand skips the indexed caller ina.c.There's a related ordering bug in the reference counts: the manifest was generated before the overlay was refreshed for the current command. If a file was modified in an earlier REPL command and has since been restored, the manifest still carries its old working-copy hash, so its indexed callers were skipped.
Fix
ensure_workdir_index_builtnow returns whether the overlay may answer forgit_sha.with_workdir(git_sha, f)helper, which returns an empty result when it may not, so the check and the read can't get out of step again.generate_git_manifestchecksworkdir_overlay_usablebefore merging the overlay's paths. It still doesn't build the overlay itself, so generating a manifest doesn't walk the tree.get_distinct_reference_counts_git_awarerefreshes the overlay before generating the manifest, and reads the overlay whenever there are targets, not only when the index is scanned.Testing
an_overlay_built_at_head_does_not_answer_for_another_revisionbuilds the overlay at HEAD, then checks three cases:Each case fails without the corresponding change.
cargo fmt --check,cargo clippy --all-targets -- -D warningsandcargo testall pass.