Repository navigation
fix: let OpenAI-compatible models use their real context window - #441
Merged
Merged
Conversation
OpenAI-compatible models were fixed at a 4,096-token context window, and the chat slider could only lower it. Each saved model now takes its window from the user's value, then the server's report, then 4,096. - Connect reads max_model_len, context_length, context_window, max_context_length and top_provider.context_length, and asks a single-model llama.cpp server's /props for its allocated n_ctx. - Endpoint settings show each model's window and where it came from, with a field to set it. - Check sends one prompt of that size with max_tokens 1 and reports fits, too large (offering the limit the server names), or silently shortened. - Raising a window removes chat and agent overrides pinned at the old maximum, which chat settings saved even when the slider was never moved. Verified against a real llama.cpp server started with -c 2048 (tests/debug/openai-compatible-context-live.test.ts). Co-Authored-By: Claude Opus 5.5 <[email protected]>
Found verifying #439 in a running Obsidian against llama.cpp and Ollama. - A chat committed its budget when the model was picked, so a chat that sat at the 4,096 fallback stayed there after the window was raised. Chats now record the model's whole window beside their budget; a budget at the whole window follows it when it grows, on reopen and in an open chat. A smaller budget the user chose is kept. Older chats count as "whole window" only at the OpenAI-compatible 4,096 fallback. - A check that the server shortened no longer offers "Use N": Ollama keeps about half its num_ctx (16,386 of 32,768), so N understated it. - "Use N" equal to the server's own report clears the user value, so the model keeps following the server instead of pinning a copy. - The live lane takes COMPATIBLE_CONTEXT_REPORTED=none for servers whose model list reports no context (Ollama's /v1). - Model rows keep their controls whole at narrow widths, and the window line and check note stay stacked when another plugin restyles setting descriptions. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #439. OpenAI-compatible models were stuck at a 4,096-token context window, and the chat slider could only lower it.
What changes
max_model_len,context_length,context_window,max_context_lengthortop_provider.context_lengthin/v1/models. A single-model llama.cpp server is asked for/props→default_generation_settings.n_ctx; its trained maximum (meta.n_ctx_train) is ignored.max_tokens: 1and reports one of three results:num_ctx.Verification
npm run buildpasses (lint, mobile reachability, type check). Full Jest suite: 7,183 passed, 119 skipped, 0 failed.-c 8192: 3 of 3 pass. The oversized check returnstoo-largewithserverLimit: 8192.num_ctx32,768): the Check cases pass. The oversized check returnstruncatedwithpromptTokens: 16386. Its discovery case originally failed because Ollama reports no size; it passes with the newCOMPATIBLE_CONTEXT_REPORTED=none.dev:errorsstayed empty.Notes
num_ctx. It read 49,140 tokens withnum_ctx32,768, and Check reports what the server actually did.🤖 Generated with Claude Code