Skip to content

fix(secrets): check URL secrets for placeholders and host:port addresses - #215

Merged
SizzleUnrlsd merged 3 commits into
mainfrom
fix/url-secret-placeholders
Oct 4, 2026
Merged

SizzleUnrlsd merged 3 commits into
mainfrom
fix/url-secret-placeholders

Conversation

@SizzleUnrlsd

@SizzleUnrlsd SizzleUnrlsd commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Closes #209.

The url provider pattern of hardcoded-secret reported, at high severity and confidence, help text holding a placeholder (?token=...) and proxy-mode specifications (reverse:udp://127.0.0.1:[email protected]:0), whose host:port@ it read as a user and a password.

Change

All in plugins/secrets.py, with no URL-specific code in the detector:

  • SecretPattern may name a secret group. matches then checks every match and applies to that group the is_placeholder check credentials already get; a match whose secret is a placeholder is not a secret. Patterns without the group behave as before, so plugin-defined patterns are unaffected.
  • The url pattern is split in two entries (user information, query parameter), since Python forbids two groups of one name in a regex. Both keep provider url, so findings and metadata are unchanged.
  • A query value ends at a character no URL holds unescaped (RFC 3986: whitespace, quotes, `, <, >, {, }, |, \, ^). Prose around a URL, such as Markdown backticks, is no longer part of the value; ?token=<token> and ?token={token} no longer match at all.
  • The password runs to the last @ before the host, since a password may hold one (mongodb://admin:P@ssw0rd123@mongo).
  • An IP address or localhost and a port before @ are an address: a negative lookahead skips them. A dotted name is not, since user names such as john.doe have dots.
  • Sentence punctuation after a secret ((see ?token=...)) is ignored by the placeholder check.
  • docs/plugins.md documents the secret group for plugin authors.

The first measurement on mitmproxy showed that `?token=...` in a docstring was still reported, because the captured value included the closing backtick; the URL character set above is the fix for it, and replaced a broader change to the shared placeholder regex that would also have changed the credential rule.

Validation

  • New tests/test_url_secret_placeholders.py (no existing test changed): 9 placeholder and address cases are not reported; 9 real credentials are still reported, including a password holding @, a dotted user name with a numeric password, https://admin:[email protected]/, a real value after a placeholder in the same literal, and a real token inside Markdown backticks. The existing test_credentials_in_urls_are_secrets_including_parameter_defaults still passes.
  • Full suite: 1661 passed, 1 failed. The failure, test_packaging::test_the_version_has_one_source_of_truth, comes from a stale local editable install (metadata 0.8.0 vs 0.21.0) and fails identically on main.
  • ruff check, ruff format --check on the new file and mypy pass; secrets.py already differs from ruff format on main, not on the changed lines.
  • mitmproxy at be75825, enterprise plugins at ac35460, [email protected], no baseline, compared with the 87 results of the 2026-10-02 reference: exactly F027, F028 (gen_sample_flows.py:2, :271), F046 and F047 (test_mode_servers.py:352, :356) disappear, nothing else changes. 87 → 83 results, high 18 → 14. Plugin digests are identical.

Review

The review found that the first version cut a password holding @ at it and dropped it as a placeholder (mongodb://admin:P@ssw0rd123@mongo was no longer reported), and took a dotted user name with a numeric password (john.doe:123456@, Supabase pooler URLs) for an address. Both are fixed and covered by tests; the new regexes stay linear on 8 MB adversarial strings.

Regression snapshots

The first CI run failed the regression job: four snapshot findings were no longer reported. Both are placeholders the change is meant to drop, and exactly these four entries are removed from the expected files (no bulk regeneration):

  • healthchecks: hc/integrations/gotify/tests/test_notify.py:50, :93, :105, ?token=abc in test assertions, a three-character value that the credential rule already treats as a placeholder (len < 4).
  • full-stack-fastapi-template: .env:18, postgresql://postgres:${POSTGRES_PASSWORD}@localhost:5432/app, where the password is an environment-variable reference.

Every other finding and the coverage of all regression repositories are unchanged; this was checked locally with a read-only diff of each snapshot (the only local difference, a syntax-error on backend/app/api/deps.py, also appears on main with the local Python 3.14 and not in CI).

Limits

  • A user part that is a dotted host name with a port (reverse:tcp://example.com:[email protected]:80) is reported as a credential: it cannot be told from john.doe:123456@.
  • The user-information password keeps the URL's character set up to the last @; only the query value is bounded by the RFC 3986 set.

The url provider pattern reported help text such as ?token=... and proxy
mode specifications such as udp://127.0.0.1:[email protected]:0 as credentials.
A provider pattern may now name its secret group, which goes through the
placeholder check credentials already get; a query value ends at a character
no URL holds unescaped, and a host:port pair before @ is an address.

Closes #209
@SizzleUnrlsd SizzleUnrlsd added the bug Something isn't working label Oct 4, 2026
@SizzleUnrlsd SizzleUnrlsd self-assigned this Oct 4, 2026
healthchecks' gotify tests use ?token=abc, a three-character placeholder, and
full-stack-fastapi-template's .env names its database password through
${POSTGRES_PASSWORD}. Neither is a hardcoded secret since the url pattern
checks its secret for placeholders. Exactly these four entries are removed.

Refs #209
Review of the url pattern change: a password holding an unescaped @ was cut at
it and dropped as a placeholder, and a dotted user name with a numeric
password was taken for a host and port. The password now runs to the last @,
only an IP address or localhost before :port@ is an address, and sentence
punctuation after a placeholder is ignored. Document the secret group for
plugin authors.

Refs #209
@SizzleUnrlsd
SizzleUnrlsd merged commit d6e3fa1 into main Oct 4, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

hardcoded-secret: the url pattern reports placeholders and host:port pairs as credentials

1 participant