Repository navigation
Add restricted native MLX GRPO implementation candidate - #276
Closed
ProfSynapse wants to merge 5 commits into
Closed
ProfSynapse wants to merge 5 commits into
ProfSynapse wants to merge 5 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a native MLX GRPO implementation candidate through the existing public TrainingAPI/native family. The host owns exact source/runtime/model/input/reward selection, grants, catalogs and durable stores; submission uses the existing synchronous owned process and guardian. Independent review and fresh Apple qualification remain required before any hardware claim.
The source audit selects an attributed MIT objective subset from
inductiveML/mlxrl@5ee8a6bd7764b0585094d78c530ac87d26364034. A separate named profile and 54-package closure preserve the qualified SFT environment. Native GRPO recompiles or rejects public controls, retains sampled IDs and generated-only loss, bounds rollout/reward/evidence work, and requires grouped reward variation plus an independently recomputed saved adapter delta for five-role verification.Evidence above 1 MiB uses the exact admitted profile limit for exclusive publication and same-tree readback. The generic writer retains its 1 MiB default; larger writes require an explicit validated finite limit. Other SFT/control writes keep their default, and profile/artifact/guardian bounds remain in force. Evidence over the admitted limit rejects before any control document is created. The existing restricted host consumer admits only the exact reviewed GRPO selection; no new public provider verb or permissive qualification issuer is added.
Validation:
Mac execution, GPU fit, real grouped updates, fresh runtime/source capture and public artifact readback still require root-owned review and fresh finite admissions. Public logs/cancel/reconcile remain unavailable. See
docs/architecture/native-mlx-grpo.mdfor upstream pins, supported controls, the bounded-publication correction and qualification proposal.The new source-only example under
examples/training_qualification/native_grpo_preparation/provides native-only Linux input preparation through the existing service/normalizer/verifier, without a Modal profile or ML effects. Its reviewed JSON configuration binds exact seed bytes, prompt column, canonical public request and fresh private output. It retains failures and emits actual prepared identity/publication/request outputs. The two color-choice rows and optimizer/layer/budget proposal are explicitly controlled mechanics data for root review, with no guaranteed reward variation or admission authority. The wrapper plus existing preparation-service regression passes 44 controls, including private collisions, retained partial stages and ML/network guards. Existing runtime/worker/native_host/P2/SFT code and locks are unchanged by this addition.Closed native GRPO phase diagnosis
The first retained source197 model attempt failed at the old unclassified run_request boundary (122), before an initial output file. No stderr text was retained; neither byte counts nor the later ordinary tokenizer-only pass establish a cause. Source/CPU audit found no established corrective config change.
Commit
24d94530225a057825a0c0d52a479555b75ae232adds trusted child-local GRPO phase exits130–148 for compilation/tokenizer/import/model/adapter/rollout/objective/update/save/reload/publication/custody. Request/schema fields stay closed, public failure stays native_worker_failed, SFT120/121/122 and the success inventory remain unchanged. No raw diagnostic content or new output file. Historical attempts cannot be reclassified or replayed. See the canonical native-local-training reference for the exact static map.Validation:289 native CPU tests passed,4 Apple-only skips;26new diagnostics controls inject actual operation failures and round-trip synthetic success through five-role verification. SFT raw/messages canonical hashes unchanged; all7maintainer invocations CURRENT; Ruff/skills/diff pass;15native/public modules match both fresh source archives, with no package install. No Mac/MLX/model/training/provider/admission effect. CPU doubles do not qualify hardware. Root owns integration, fresh independent review, final package/capture refresh and any separately admitted new attempt.
Explicit prompt-ID contract and reusable tokenizer-only profile
Commit
983069ad34dff1ac4a96a054fa447f55906ae294fixes a concrete pinned Transformers5.20.2 API mismatch: apply_chat_template defaults to a mapping, so the old tuple conversion iterates string keys. GRPO now explicitly requests return_dict=False through a shared prompt-tokenization helper, preserving generation scaffold/configured kwargs and rejecting reserved-control collisions or malformed IDs before ML imports (phase135). SFT, runtime pins, diagnostics and public failure/success inventories remain unchanged. This static defect does not establish historical122's cause or permit replay.The existing config-first preparation example gains profile-tokenizer for Linux/Darwin, reusing that exact helper with an actual recompiled workload, verified GRPO publication and private complete tokenizer snapshot, exact installed version and finite byte budgets. It retains create-only private token evidence and closed counts/hash summaries, no model/MLX/training/admission. See the checked-in tokenizer-profile.example.json and README. Root owns actual paths/source review and any Mac action.
Validation:304nativeCPU pass/4Apple-only skips,29wrapper pass;25new mapping-default/scaffold/collision/malformed-ID/private-publication/custody/version/bound/CLI controls. All7maintainer invocations CURRENT without pin/inventory changes; SFTcanonical hashes unchanged; Ruff/diff/skill sync pass;15native/public modules match fresh wheel/sdist without installation. Root reports diagnostic-only d88 R4accepted; this new change needs final integrated R5 and fresh source/capture review before any separately admitted model attempt.
Integrated into merged PR #277, merge
5c9a5cf9a4b19e5ce193c3c09a46df22c86296a7. Its reviewed/tested tree includes this work and the final Mac qualification fixes. This draft is superseded; branch and retained evidence are preserved.