Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions docs/model-workflows/finetune-readiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,9 @@ uses Python's standard library and the existing local cache validator.
an image or verify its framework compatibility, contents, trust or registry
accessibility. The operator must choose a reviewed fine-tuning image.
- Supported cache layout: `config.json`, `tokenizer_config.json`,
`tokenizer.json`, and either `model.safetensors` or
`tokenizer.json` (the vLLM quickstart's serving example lists only
`config.json`, `tokenizer.json` and the weights; this check is stricter and
requires `tokenizer_config.json` too), and either `model.safetensors` or
`model.safetensors.index.json` with a nonempty `weight_map`. Every indexed
shard must exist, be nonempty, and resolve within the cache. Weight indexes
larger than 8 MiB fail closed. The explicit required-file list must include
Expand Down Expand Up @@ -74,7 +76,11 @@ srun --nodes=1 --ntasks=1 --gpus=1 --immediate=5 --time=1 \

This may pull the container image. The allocation has a one-minute scheduler
time limit and a five-second immediate-allocation limit; the local command has
the configured timeout. A slow image pull can make the probe fail. Confirm job
the configured timeout. A slow image pull can make the probe fail, because the
five-second window is too short to pull an image: **pre-pull or pre-import the
pinned image on the target node** (for example with the site's normal
Enroot/Pyxis import step) before using `--gpu-smoke`, and treat a failure on a
cold node as inconclusive. Confirm job
termination through the site's normal procedure after a timeout; do not blindly
retry. Require a zero exit and exactly one GPU row. No training, dataset loading,
model loading or shared-cache mounting is performed by this probe. In particular,
Expand Down
4 changes: 4 additions & 0 deletions docs/model-workflows/shared-cache.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,10 @@ export MODEL_REVISION=607a30d783dfa663caf39e06633721c8d4cfcd7e
--revision "$MODEL_REVISION" --cache-dir "$HF_HUB_CACHE"
```

The example list is what the vLLM quickstart needs. The fine-tuning readiness
check also requires `tokenizer_config.json`, so download it too if the same
snapshot will be used for fine-tuning.

For another model, explicitly list all needed configuration, tokenizer and weight
files. For sharded weights, include the index **and every referenced shard**.
A successful download of `config.json` alone does not make a usable model.
Expand Down
11 changes: 8 additions & 3 deletions docs/model-workflows/vllm-quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ python3.12 -m venv "$HOME/.venvs/deepops-vllm"
"$HOME/.venvs/deepops-vllm/bin/vllm" --version
```

The example pins vLLM **0.31.0**, licensed under **Apache-2.0** (verified
The example pins vLLM **0.31.0**, licensed under **Apache-2.0** (verified on
2026-10-07 against the [tagged license](https://github.com/vllm-project/vllm/blob/v0.31.0/LICENSE)
and [package metadata](https://pypi.org/project/vllm/0.31.0/)). This is the serving
package's license, not the model's or every dependency's. Transitive packages
Expand All @@ -67,7 +67,7 @@ package problem. vLLM loads a local snapshot, not a mutable Hub repository name.
The example model, `openai-community/gpt2` at
`607a30d783dfa663caf39e06633721c8d4cfcd7e`, is listed as **MIT** in its
[pinned model card](https://huggingface.co/openai-community/gpt2/blob/607a30d783dfa663caf39e06633721c8d4cfcd7e/README.md)
(verified 2026-10-07). Review its license terms before use; re-check the model's
(verified on 2026-10-07). Review its license terms before use; re-check the model's
license and access requirements when selecting another model or revision.

```bash
Expand All @@ -80,7 +80,12 @@ python3 scripts/validation/validate_model_cache.py --json \
--require-file config.json --require-file tokenizer.json --require-file model.safetensors
```

Stop unless exit status is `0` and `ok` is `true`. For a different model, review
Stop unless exit status is `0` and `ok` is `true`. These three files are the
set this example was validated with. The fine-tuning readiness check
([finetune-readiness.md](finetune-readiness.md)) additionally requires
`tokenizer_config.json` because its validator insists on that layout; add it
here too if your model's tokenizer needs it, or if one cache snapshot will serve
both workflows. For a different model, review
its complete tokenizer/configuration/weight file list, including every indexed
shard. The validator only checks files you name. Never enable remote model code
as a workaround; this example does not use `--trust-remote-code`.
Expand Down
2 changes: 2 additions & 0 deletions scripts/validation/validate_finetune.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,8 @@ def parse_record(output):


def gpu_count(tres):
# Fails closed: if Slurm changes the TRES format, no match returns None and
# the node is reported not ready instead of guessed.
matches = re.findall(r"(?:^|,)gres/gpu=(\d+)(?:,|$)", tres)
return int(matches[0]) if len(matches) == 1 else None

Expand Down
5 changes: 3 additions & 2 deletions skills/finetune-model/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,9 @@ python3 scripts/validation/validate_finetune.py --json \

Off by default. Only with explicit authorization to allocate a GPU and run the
reviewed image, repeat the command with `--gpu-smoke`. This can download the
container image and submits one bounded, single-node, single-GPU `nvidia-smi`
job after all readiness gates pass. It does not load the model or train.
container image, but its scheduler window is only five seconds, so pre-pull the
pinned image on the node first. It submits one bounded, single-node, single-GPU
`nvidia-smi` job after all readiness gates pass. It does not load the model or train.
`gpu_smoke_ok: true` proves only the container GPU probe, not cache access
inside the container, framework compatibility, learning or checkpoint recovery.

Expand Down
5 changes: 4 additions & 1 deletion skills/serve-model/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,10 @@ or authorize a workload.

1. Follow quickstart steps 1–2. Require cache validator exit `0` **and** `ok: true`
as the actual reader before startup. Config alone is not a complete model;
name every required weight shard and tokenizer file.
name every required weight shard and tokenizer file. The three files in the
example are the set it was validated with; `finetune-model` also requires
`tokenizer_config.json`. Add it for any model whose tokenizer needs it, or
when one cache snapshot serves both workflows.
2. Use quickstart step 3 unchanged for the public example: local snapshot and
tokenizer, offline loading, one GPU, foreground process, loopback only. Never
enable remote model code. `HF_HUB_OFFLINE=1` is not a firewall and belongs only
Expand Down
Loading