Skip to content

Add bounded native MLX evaluation for saved adapters - #278

Merged
ProfSynapse merged 5 commits into
mainfrom
feat/native-mlx-evaluation-20261027
Oct 27, 2026
Merged

ProfSynapse merged 5 commits into
mainfrom
feat/native-mlx-evaluation-20261027

Conversation

@ProfSynapse

@ProfSynapse ProfSynapse commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Add bounded native MLX generation/evaluation for a pinned base model and existing LoRA adapters, without submitting training. The source-checkout host reuses captured isolated imports, the native owned-process guardian and SQLite records, with distinct signed single-use admissions and finite time/token/file/output bounds. Complete private exports score through the existing evaluator; metadata, correctness trees and inline schemas validate before dispatch.

Actual Apple M4 acceptance on execution source edc4c7aa: five real numerical checks passed; both tokenizer profiles passed all120original rows; base and two saved SFT adapters each completed the same40held-out cases with clean termination. All120generations stopped naturally with zero token-limit finishes, and each prompt matched original tokenizer IDs. Complete exports/signatures/hashes were verified and scored on CPU. Allthree runs passed original-authority cold readback after their separate start expiries, without model reloads or renewed authority.

Quality result: every model scored 0/20 historical command checks and emitted no parsed tool calls on those prompts. Reference loss improved on each adapter's own dataset (conversation2.183849→1.908499; Nexus1.609131→1.250325). Conversation assertions test nonempty output only. These results do not establish semantic quality, current Nexus API compatibility or useful sampled GRPO rewards. The evidence report records exact identities and limits. Public native lifecycle/ordinary qualification gates remain unchanged.

Validation:303focused CPU tests pass; independent source R3 ACCEPT and final evidence/package/claim R4 ACCEPT. The final reviewer independently verified three signatures, complete exports,120prompt/mask joins, loss arithmetic, post-expiry receipts,751captured source members and package equivalence, plus17focused CPU tests. Seven closure/lock checks CURRENT; Ruff/diff/skills pass. Documentation head 0f024b29 changes no executable code from the Mac execution source. Fresh wheel/sdist verify all923wheel Pythonmembers against final Git and exact execution source; other runtime assets equal the original candidate. Neutral base-only installed consumer passes. Source-only host examples require the reviewed Git checkout.

Ready for review; merge waits on full current-head CI. No additional model training, installer, application packaging, deployment or capability promotion is included.

@ProfSynapse
ProfSynapse marked this pull request as ready for review October 27, 2026 18:47
@ProfSynapse
ProfSynapse merged commit b997bf9 into main Oct 27, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant