[Wave 3] Medium — Module sweep: Post-training: inference, chat, GGUF export, deploy, merging (part 2/2) #568

Wave 3 · 1 finding(s) in 1 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

• CR-A13-033 (code, Medium) — generate/generate_stream override the model's generation_config EOS list with the single tokenizer.eos_token_id, so forgelm chat runs past end-of-turn into fabricated turns
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator fine-tunes, or chats with, a model whose end-of-turn id exists only in generation_config: per the task premise, Gemma-it ([, <end_of_turn>]) and Llama-3-Instruct's original config ([<|end_of_text|>, <|eot_id|>]), with tokenizer eos = document-end. The model correctly emits end-of-turn after its answer, but forgelm chat does not stop. It prints invented 'user ... assistant ...' exchanges with the role markers stripped, until 512 tokens, and stores them in history as the assistan…
  Where. forgelm/inference.py:256, forgelm/inference.py:294, forgelm/chat.py:299-300, forgelm/chat.py:279
  Recommended fix. Do not pass eos_token_id at all, so that model.generation_config applies. Alternatively, pass the union of generation_config.eos_token_id (normalised to a list) and tokenizer.eos_token_id. Add a test with a tiny model whose generation_config eos list differs from the tokenizer eos, and assert the EosTokenCriteria ids (or that a forced end-of-turn stops generation) for both generate and generate_s…

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}