Repository navigation
Add bounded native MLX evaluation for saved adapters - #278
Merged
Merged
Conversation
ProfSynapse
marked this pull request as ready for review
October 27, 2026 18:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add bounded native MLX generation/evaluation for a pinned base model and existing LoRA adapters, without submitting training. The source-checkout host reuses captured isolated imports, the native owned-process guardian and SQLite records, with distinct signed single-use admissions and finite time/token/file/output bounds. Complete private exports score through the existing evaluator; metadata, correctness trees and inline schemas validate before dispatch.
Actual Apple M4 acceptance on execution source
edc4c7aa: five real numerical checks passed; both tokenizer profiles passed all120original rows; base and two saved SFT adapters each completed the same40held-out cases with clean termination. All120generations stopped naturally with zero token-limit finishes, and each prompt matched original tokenizer IDs. Complete exports/signatures/hashes were verified and scored on CPU. Allthree runs passed original-authority cold readback after their separate start expiries, without model reloads or renewed authority.Quality result: every model scored 0/20 historical command checks and emitted no parsed tool calls on those prompts. Reference loss improved on each adapter's own dataset (conversation2.183849→1.908499; Nexus1.609131→1.250325). Conversation assertions test nonempty output only. These results do not establish semantic quality, current Nexus API compatibility or useful sampled GRPO rewards. The evidence report records exact identities and limits. Public native lifecycle/ordinary qualification gates remain unchanged.
Validation:303focused CPU tests pass; independent source R3 ACCEPT and final evidence/package/claim R4 ACCEPT. The final reviewer independently verified three signatures, complete exports,120prompt/mask joins, loss arithmetic, post-expiry receipts,751captured source members and package equivalence, plus17focused CPU tests. Seven closure/lock checks CURRENT; Ruff/diff/skills pass. Documentation head
0f024b29changes no executable code from the Mac execution source. Fresh wheel/sdist verify all923wheel Pythonmembers against final Git and exact execution source; other runtime assets equal the original candidate. Neutral base-only installed consumer passes. Source-only host examples require the reviewed Git checkout.Ready for review; merge waits on full current-head CI. No additional model training, installer, application packaging, deployment or capability promotion is included.