Skip to content

ci: add CTL diagnostics to the Windows runner trust probe - #3323

Draft
juenglin wants to merge 4 commits into
NVIDIA:mainfrom
juenglin:windows-trust-probe-ctl-state
Draft

juenglin wants to merge 4 commits into
NVIDIA:mainfrom
juenglin:windows-trust-probe-ctl-state

Conversation

@juenglin

Copy link
Copy Markdown
Contributor

Record the CTL auto-update policy and registry state, plus the
SequenceNumber of the local and Windows Update Disallowed CTLs, and
whether each includes Microsoft's 2026-10-20 eMudhra update. The new
rows are informational and run through Invoke-Diagnostic, so a failure
there cannot skip the gating checks. Report the sequence number even
when certutil fails (NTE_BAD_ALGID on Windows without ML-DSA support).

Record the CTL auto-update policy and registry state, plus the
SequenceNumber of the local and Windows Update Disallowed CTLs, and
whether each includes Microsoft's 2026-10-20 eMudhra update. The new
rows are informational and run through Invoke-Diagnostic, so a failure
there cannot skip the gating checks. Report the sequence number even
when certutil fails (NTE_BAD_ALGID on Windows without ML-DSA support).
@copy-pr-bot

copy-pr-bot Bot commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the CI/CD CI/CD infrastructure label Oct 31, 2026
@juenglin

Copy link
Copy Markdown
Contributor Author

/ok to test 054d973

@github-actions

Copy link
Copy Markdown
Contributor

@juenglin

Copy link
Copy Markdown
Contributor Author

/ok to test cf8eddf

Record the Windows build, and ask certutil whether the local Disallowed
CTL matches each compromised certificate. On GitHub-hosted runners with
DisableRootAutoUpdate=1, finish with an informational experiment: set
EnableDisallowedCertAutoUpdate=1, restart CryptSvc, then recount explicit
distrust and repeat the Invoke-WebRequest connections to the test
endpoints in fresh pwsh processes; if that does not help, also remove
DisableRootAutoUpdate and repeat. The experiment changes machine policy,
so it runs after all gating checks and never on self-hosted runners.

Factor the Invoke-WebRequest result classification into Get-TlsOutcome so
the gating check and the experiment share it.
@juenglin
juenglin force-pushed the windows-trust-probe-ctl-state branch from cf8eddf to 1d23828 Compare October 31, 2026 19:59
@juenglin

Copy link
Copy Markdown
Contributor Author

/ok to test 1d23828

…ust probe

The previous experiment showed EnableDisallowedCertAutoUpdate=1 makes chain
building distrust all 16 certificates while Invoke-WebRequest still trusts
the test endpoints. Extend the experiment to narrow that down:

- Log the chain .NET builds for each TLS connection (SslStream validation
  callback; policy errors, chain status and per-element status flags), at
  baseline and after each experiment step.
- Before the policy changes, add the 16 certificates to
  LocalMachine\Disallowed as the upgrade-ca-certificates action does,
  measure, then remove them again and measure.
- Remove DisableRootAutoUpdate when chain building or TLS still trusts the
  certificates after EnableDisallowedCertAutoUpdate=1, not only when the
  explicit-distrust count is short.

Every step restarts CryptSvc and measures in fresh pwsh processes.
@juenglin

Copy link
Copy Markdown
Contributor Author

/ok to test 1c25840

… in the trust probe

On GitHub-hosted runners the chain .NET builds during TLS shows the eMudhra
intermediate as NoError even when the same certificate, checked on its own,
is explicitly distrusted; on self-hosted runners the TLS chain flags it as
Revoked,ExplicitDistrust. Add diagnostics to find out why:

- TLS chain rows now give the SHA-256 of each certificate Windows used and
  of each certificate the server sent, marked when it is in the impacted
  list.
- Search the certificate stores and the CryptnetUrlCache (certificates
  downloaded through AIA) for certificates that carry a compromised key or
  an eMudhra G3 intermediate name, and report any that are not in the
  impacted list. Runs at baseline and after the experiment.
- Connect with the inbox curl.exe (Schannel, --ssl-no-revoke) and with gh
  (Go crypto/x509 over Windows chain building), at baseline and after each
  experiment step, so the result can be compared with a client that does
  not use .NET. gh runs with a placeholder token and an empty config
  directory.
- Record pwsh, .NET, crypt32.dll, curl, gh and gh's Go toolchain versions;
  Go releases with the GO-2026-6809 fix reject the intermediates on their
  own.
@juenglin

Copy link
Copy Markdown
Contributor Author

/ok to test e634a6a

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant