Repository navigation
test: make the ML funcall provisioning spec offline-deterministic - #835
Merged
Merged
Conversation
…fixture resolve() fetches the remote index over HTTP before validating the model id, so on a network-constrained CI runner the raw Net::ReadTimeout surfaced instead of the named RegistryError the contract ruling requires. The unknown-id rejection does not need the network: point INTERSCRIPT_ML_INDEX at a local fixture and the spec becomes deterministic (with_env restores the process env). The three flake failures today (PR runs and main, 3.4/ubuntu) were all this spec hitting Net::ReadTimeout / 502 on the registry fetch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the ML funcall provisioning contract spec deterministic: the unknown-model rejection is now pinned against a local index fixture (
INTERSCRIPT_ML_INDEX→ temp file), not the live registry index.Why
resolve()fetchesDEFAULT_INDEX_URLover HTTP before validating the model id, so on a network-constrained CI runner the rawNet::ReadTimeout(or a 502) surfaces instead of the namedRegistryErrorthe contract ruling requires. Today this flake failed three CI jobs across PR runs and main (Ruby 3.4 / ubuntu, 3.3 / windows). The contract under test — unknown ids fail loudly with a named provisioning error enumerating known ids — does not need the network:load_indexreads afile:-path index withFile.read.Verification
RegistryErrorwith/unknown model id "no-such-model" (known:/— deterministic, no network touched (the env var makesload_indextake theFile.readbranch).