Repository navigation
fix(slurm): honor slurm_oversubscribe in the batch partition - #1445
Conversation
The slurm.conf template reassigned slurm_oversubscribe from slurm_manage_gpus right before the PartitionName line, so a value set in the inventory was never used: OverSubscribe was always NO with slurm_manage_gpus true and EXCLUSIVE otherwise. Move that expression into the role default and drop the reassignment. The rendered default is unchanged for both slurm_manage_gpus values, and an explicit slurm_oversubscribe now reaches slurm.conf. Add tests/slurm-conf/test_partition.py, which renders the template in memory for the default and for explicit settings. Signed-off-by: Dezhen Lu <[email protected]>
dholt
left a comment
There was a problem hiding this comment.
Code review approved; please hold merging until the day after the 26.11 tag. This keeps the release tree unchanged after QA; this approval is not a request to merge before then.
Thanks @0z5a for the focused fix and regression test. Reviewed commit 4bea04d43659ac545de6163e20a81a316b32f6bf.
Correct
No blocking findings. Moving the expression into role defaults preserves Ansible variable precedence. Full rendered configurations are byte-identical to the parent with slurm_manage_gpus: true (OverSubscribe=NO) and false (OverSubscribe=EXCLUSIVE). Explicit NO, EXCLUSIVE, YES and FORCE settings now reach the partition in both modes. I also checked FORCE:4 and the adjacent MaxTime/DefaultTime fields. Repository-wide search found no other OverSubscribe assignment or separate documentation requiring an update.
Secure / Readable / Elegant
No findings. The change removes the template-local override without adding execution paths or dependencies.
Altruistic
- nit / non-blocking suggestion:
tests/slurm-conf/test_partition.py:51— consider wiring this test into CI in a follow-up. None of the current workflows invokes or discoverstests/slurm-conf; the existing Slurm workflow runs only the epilog suite, and the Molecule matrix excludes Slurm. Runningpython tests/slurm-conf/test_partition.py -vwith the CI-pinned Ansible environment would preserve this regression coverage automatically.
Local validation: both test methods passed (10 parameter cases), role lint passed, all 12 maintenance playbook syntax checks passed after initializing the pinned Kubespray submodule, and git diff --check passed. These are offline checks, not live scheduler validation.
roles/slurm/defaults/main.ymldocumentsslurm_oversubscribe("Sets partition OverSubscribe. To avoid sharing nodes, set to EXCLUSIVE"), but theslurm.conftemplate reassigns it fromslurm_manage_gpuson the line before thePartitionNameline. A value set inconfig/group_vars/was never used:OverSubscribewas alwaysNOwithslurm_manage_gpus: trueandEXCLUSIVEotherwise.This PR moves that expression into the role default and removes the
setline from the template. The rendered default is the same as before for both values ofslurm_manage_gpus; an explicitslurm_oversubscribenow reachesslurm.conf.Fixes #1444
Testing
tests/slurm-conf/test_partition.pyrenders the template in memory with Ansible'sTemplar(same approach astests/slurm-lua/test_lua.py). Onmaster@4eb47c33it fails in the 6 subtests where the setting differs from the automatic value; with this change both tests pass (python3 tests/slurm-conf/test_partition.py). Run on macOS arm64, Python 3.12, ansible 10.7.0 (ansible-core 2.17.14, the versionscripts/setup.shpins).git diff --check,scripts/deepops/check-template-syntax.py,scripts/deepops/ansible-lint-roles.sh(ansible-lint 26.1.1, ansible 10.7.0) andansible-playbook --syntax-check playbooks/slurm-cluster/slurm.ymlpass.AI disclosure: found and drafted with an AI coding assistant (Claude Code, claude-fable-5-2), including the default change, the new test and this description; the tests were run as described above.