Repository navigation
fix(secrets): check URL secrets for placeholders and host:port addresses - #215
Merged
Merged
Conversation
The url provider pattern reported help text such as ?token=... and proxy mode specifications such as udp://127.0.0.1:[email protected]:0 as credentials. A provider pattern may now name its secret group, which goes through the placeholder check credentials already get; a query value ends at a character no URL holds unescaped, and a host:port pair before @ is an address. Closes #209
healthchecks' gotify tests use ?token=abc, a three-character placeholder, and
full-stack-fastapi-template's .env names its database password through
${POSTGRES_PASSWORD}. Neither is a hardcoded secret since the url pattern
checks its secret for placeholders. Exactly these four entries are removed.
Refs #209
Review of the url pattern change: a password holding an unescaped @ was cut at it and dropped as a placeholder, and a dotted user name with a numeric password was taken for a host and port. The password now runs to the last @, only an IP address or localhost before :port@ is an address, and sentence punctuation after a placeholder is ignored. Document the secret group for plugin authors. Refs #209
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #209.
The
urlprovider pattern ofhardcoded-secretreported, athighseverity and confidence, help text holding a placeholder (?token=...) and proxy-mode specifications (reverse:udp://127.0.0.1:[email protected]:0), whosehost:port@it read as a user and a password.Change
All in
plugins/secrets.py, with no URL-specific code in the detector:SecretPatternmay name asecretgroup.matchesthen checks every match and applies to that group theis_placeholdercheck credentials already get; a match whose secret is a placeholder is not a secret. Patterns without the group behave as before, so plugin-defined patterns are unaffected.urlpattern is split in two entries (user information, query parameter), since Python forbids two groups of one name in a regex. Both keep providerurl, so findings and metadata are unchanged.`,<,>,{,},|,\,^). Prose around a URL, such as Markdown backticks, is no longer part of the value;?token=<token>and?token={token}no longer match at all.@before the host, since a password may hold one (mongodb://admin:P@ssw0rd123@mongo).localhostand a port before@are an address: a negative lookahead skips them. A dotted name is not, since user names such asjohn.doehave dots.(see ?token=...)) is ignored by the placeholder check.docs/plugins.mddocuments thesecretgroup for plugin authors.The first measurement on mitmproxy showed that
`?token=...`in a docstring was still reported, because the captured value included the closing backtick; the URL character set above is the fix for it, and replaced a broader change to the shared placeholder regex that would also have changed the credential rule.Validation
tests/test_url_secret_placeholders.py(no existing test changed): 9 placeholder and address cases are not reported; 9 real credentials are still reported, including a password holding@, a dotted user name with a numeric password,https://admin:[email protected]/, a real value after a placeholder in the same literal, and a real token inside Markdown backticks. The existingtest_credentials_in_urls_are_secrets_including_parameter_defaultsstill passes.test_packaging::test_the_version_has_one_source_of_truth, comes from a stale local editable install (metadata 0.8.0 vs 0.21.0) and fails identically onmain.ruff check,ruff format --checkon the new file andmypypass;secrets.pyalready differs fromruff formatonmain, not on the changed lines.be75825, enterprise plugins atac35460,[email protected], no baseline, compared with the 87 results of the 2026-10-02 reference: exactly F027, F028 (gen_sample_flows.py:2,:271), F046 and F047 (test_mode_servers.py:352,:356) disappear, nothing else changes. 87 → 83 results,high18 → 14. Plugin digests are identical.Review
The review found that the first version cut a password holding
@at it and dropped it as a placeholder (mongodb://admin:P@ssw0rd123@mongowas no longer reported), and took a dotted user name with a numeric password (john.doe:123456@, Supabase pooler URLs) for an address. Both are fixed and covered by tests; the new regexes stay linear on 8 MB adversarial strings.Regression snapshots
The first CI run failed the
regressionjob: four snapshot findings were no longer reported. Both are placeholders the change is meant to drop, and exactly these four entries are removed from the expected files (no bulk regeneration):healthchecks:hc/integrations/gotify/tests/test_notify.py:50,:93,:105,?token=abcin test assertions, a three-character value that the credential rule already treats as a placeholder (len < 4).full-stack-fastapi-template:.env:18,postgresql://postgres:${POSTGRES_PASSWORD}@localhost:5432/app, where the password is an environment-variable reference.Every other finding and the coverage of all regression repositories are unchanged; this was checked locally with a read-only diff of each snapshot (the only local difference, a
syntax-erroronbackend/app/api/deps.py, also appears onmainwith the local Python 3.14 and not in CI).Limits
reverse:tcp://example.com:[email protected]:80) is reported as a credential: it cannot be told fromjohn.doe:123456@.@; only the query value is bounded by the RFC 3986 set.