Skip to content

feat: extend the Go re-check to +dirty builds, other module paths and the project's own advisory #208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
github-actions merged 2 commits into from
Oct 19, 2026
Merged

feat: extend the Go re-check to +dirty builds, other module paths and the project's own advisory #208

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cac1851
feat(sbom): extend the Go re-check to +dirty builds, other module pat…
Eandalf-Bigstack Oct 17, 2026
8611c97
fix(rootfs): read the kernel's ixgbe file directly in the E610 check
Eandalf-Bigstack Oct 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions rootfs/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,8 +110,11 @@ base_rootfs_install:
$(Q)$(INSTALL_MODULES) $(ROOTDIR) $(HEX_DATADIR)/os/base.modules
@# The Intel E610 (8086:57AE-57B2) is known to the kernel's own ixgbe from Linux 6.14. Every
@# image that carries NIC drivers takes ixgbe from the kernel's module tree, so check that tree:
@# fail the build on a kernel whose ixgbe lacks the E610 10GBASE-T id.
$(Q)modinfo -k $(KERNEL_VERS) -F alias ixgbe | grep -qi 'v00008086d000057B0'
@# fail the build on a kernel whose ixgbe lacks the E610 10GBASE-T id, or that has no ixgbe. The
@# module file is read directly: the jail installs its kernel packages without running depmod,
@# so its tree has no index to look ixgbe up by name in.
$(Q)f=$$(find /lib/modules/$(KERNEL_VERS)/kernel -name 'ixgbe.ko*' | head -1) && [ -n "$$f" ] && \
modinfo -F alias $$f | grep -qi 'v00008086d000057B0'
$(Q)$(INSTALL_DATA) $(ROOTDIR) $(HEX_DATADIR)/os/systemd-hexctl-sys.service ./lib/systemd/system
$(Q)$(INSTALL_DATA) $(ROOTDIR) $(HEX_DATADIR)/os/passwd ./etc
$(Q)$(INSTALL_DATA) $(ROOTDIR) $(HEX_DATADIR)/os/shadow ./etc
Expand Down
115 changes: 95 additions & 20 deletions scripts/recheckvulns.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,15 @@
from is outside every affected range, at or above every fixed release, and was released after all
of them, the fixed releases being tags of the repository. That last condition keeps a renumbered
project's old commits: prometheus's 2021 commit descends from v2.26.0, which sorts above its 3.x
module versions (v0.311.3) but came long before them. A fixed pseudo-version whose commit it does
not contain keeps it. The clones hold commits only and are brought up to date on later scans, the
report's descriptor lists them, and GOMODULE_GIT in the environment points at a mirror.
module versions (v0.311.3) but came long before them. The ranges GitHub gives under the
repository's other module paths count too (github.com/elastic/beats for github.com/elastic/beats/v7).
A fixed pseudo-version whose commit it does not contain keeps the match, unless the commit is itself
a release, which the pseudo-version then only names. The clones hold commits only and are brought
up to date on later scans, the report's descriptor lists them, and GOMODULE_GIT in the environment
points at a mirror. With the same option, a match from the Go vulnerability database that names a
module with no affected range at all, every version and no fix, moves to "ignoredMatches" when the
project's own advisory for the flaw (a GitHub advisory the repository published) names only the
repository's other module paths: containerd v1 for a flaw in a CRI feature v2 added.
"""

import argparse
Expand Down Expand Up @@ -718,8 +724,9 @@ def recheck_ossa(rep, path):


# A Go pseudo-version names a commit: v0.0.0-20250227174106-7829dbe62d7f,
# v1.5.1-0.20260427112133-525d1bab07e0, v1.2.3-pre.0.20200101000000-0123456789ab.
PSEUDO = re.compile(r'^v?\d+\.\d+\.\d+-(?:[0-9A-Za-z.]+\.)?(\d{14})-([0-9a-f]{12})$')
# v1.5.1-0.20260427112133-525d1bab07e0, v1.2.3-pre.0.20200101000000-0123456789ab. A build from a
# modified tree carries build metadata after it: v7.0.0-20260930214605-b80bf8464fba+dirty.
PSEUDO = re.compile(r'^v?\d+\.\d+\.\d+-(?:[0-9A-Za-z.]+\.)?(\d{14})-([0-9a-f]{12})(?:\+[0-9A-Za-z.-]+)?$')
RELEASE_TAG = re.compile(r'^v(\d+)\.(\d+)\.(\d+)$')
GO_COND = re.compile(r'^\s*(<=|>=|<|>|=)\s*v?(\S+?)\s*$')

Expand All @@ -746,9 +753,12 @@ def go_in_range(ver, conds):
_ADV = {}


def advisory(vid, module, repo):
"""The affected ranges [(conditions, fix or None)] and fix commits grype's own database holds
for vid on module, from GitHub's advisory; None if it has none for that module."""
def same_repo(name, repo):
"""True if name is repo's module path, or the same path with a major-version suffix."""
return re.fullmatch(rf'github\.com/{re.escape(repo)}(/v\d+)?', name) is not None


def db_search(vid):
if vid not in _ADV:
out = subprocess.run(
[GRYPE, 'db', 'search', '--vuln', vid, '-o', 'json'],
Expand All @@ -757,13 +767,23 @@ def advisory(vid, module, repo):
if out.returncode:
sys.exit(f'recheckvulns: cannot read {vid} from grype\'s database')
_ADV[vid] = json.loads(out.stdout or '[]')
ranges, commits = [], set()
for rec in _ADV[vid]:
return _ADV[vid]


def advisory(vid, module, repo):
"""The affected ranges [(conditions, fix or None)] and fix commits grype's own database holds
for vid, from GitHub's advisory, on module and on the repository's other module paths (the
same project, released under another major version: github.com/elastic/beats for
github.com/elastic/beats/v7); None if it names the module under neither."""
ranges, commits, named = [], set(), False
for rec in db_search(vid):
if rec['vulnerability'].get('provider') != 'github':
continue
for p in rec.get('packages') or []:
if p['package'].get('ecosystem') != 'go-module' or p['package'].get('name') != module:
pkg = p.get('package') or {}
if pkg.get('ecosystem') != 'go-module' or not same_repo(pkg.get('name', ''), repo):
continue
named |= pkg['name'] == module
for r in p['detail'].get('ranges') or []:
conds = [GO_COND.match(c) for c in r['version']['constraint'].split(',')]
if conds and all(conds):
Expand All @@ -772,7 +792,7 @@ def advisory(vid, module, repo):
m = re.match(rf'https://github\.com/{re.escape(repo)}/commit/([0-9a-f]{{7,40}})\b', ref['url'])
if m:
commits.add(m.group(1))
return (ranges, commits) if ranges else None
return (ranges, commits) if ranges and named else None


def fetch_repo(url, path):
Expand All @@ -796,6 +816,29 @@ def ancestor(path, a, b):
stderr=subprocess.DEVNULL).returncode == 0


def project_advisory(m, repo):
"""(GHSA, module paths) when the Go vulnerability database's record behind m names its module
with no affected range at all, while the project's own GitHub advisory for the same flaw names
only the repository's other module paths; else None."""
vul, art = m['vulnerability'], m['artifact']
rec = next((r for r in db_search(vul['id']) if r['vulnerability'].get('provider') == 'govulndb'), None)
pkgs = [p for p in (rec or {}).get('packages') or [] if (p.get('package') or {}).get('name') == art['name']]
if not pkgs or any(p['detail'].get('ranges') for p in pkgs):
return None
for g in [x['id'] for x in m.get('relatedVulnerabilities') or [] if x['id'].startswith('GHSA-')]:
for r in db_search(g):
v = r['vulnerability']
if v.get('provider') != 'github' or not any(
u['url'].startswith(f'https://github.com/{repo}/security/advisories/') for u in v.get('refs') or []
):
continue
names = {p['package']['name'] for p in r.get('packages') or []
if (p.get('package') or {}).get('ecosystem') == 'go-module'}
if names and art['name'] not in names and all(same_repo(n, repo) for n in names):
return g, sorted(names)
return None


def recheck_gopseudo(rep, path):
"""Re-judge matches on Go modules at a pseudo-version by the module's own history in place; a
summary, or None if the report has no such match."""
Expand All @@ -806,7 +849,18 @@ def recheck_gopseudo(rep, path):
parts = art['name'].split('/')
if pm and len(parts) >= 3 and parts[0] == 'github.com':
todo.append((m, f'{parts[1]}/{parts[2]}', pm.group(2)))
if not todo:
# The Go vulnerability database can list a project's older module path with no range at all
# (every version, no fix) where the project's own advisory names only its newer one: containerd
# v1 for a flaw in a CRI feature v2 added. grype reports such a match without a constraint.
blanket = [
(m, '/'.join(m['artifact']['name'].split('/')[1:3])) for m in rep['matches']
if m['artifact']['type'] == 'go-module' and m['artifact']['name'].startswith('github.com/')
Comment thread
Eandalf-Bigstack marked this conversation as resolved.
Dismissed
and len(m['artifact']['name'].split('/')) >= 3 and not PSEUDO.match(m['artifact']['version'])
and m['vulnerability'].get('namespace', '').startswith('govulndb')
and any((d.get('found') or {}).get('versionConstraint', '').startswith('none')
for d in m.get('matchDetails') or [])
]
if not todo and not blanket:
return None

repos = {}
Expand All @@ -823,8 +877,18 @@ def recheck_gopseudo(rep, path):
tags[name] = int(date or 0)
repos[repo] = (clone, tags)

ignored = {'fix commit': [], 'release': []}
ignored = {'fix commit': [], 'release': [], 'project advisory': []}
kept = 0
deferred = 0
for m, repo in blanket:
said = project_advisory(m, repo)
if not said:
continue
deferred += 1
g, names = said
ignore(m, f'Go vulnerability database: {m["vulnerability"]["id"]} names {m["artifact"]["name"]} '
f'with no affected range; the project\'s own advisory {g} names only {", ".join(names)}')
ignored['project advisory'].append(m)
for m, repo, short in todo:
vul, art = m['vulnerability'], m['artifact']
clone, tags = repos[repo]
Expand All @@ -843,14 +907,22 @@ def recheck_gopseudo(rep, path):
# The module path has no major-version suffix, so the project's releases cannot be
# named: every version a Go build can link is a pseudo-version, which sorts below
# every fixed release. Judge by the newest release the commit descends from instead,
# when the fixes are releases of this repository and that one came after them all.
# when the fixes are releases of this repository and that one came after them all. A
# fixed pseudo-version the commit does not contain keeps the match, unless the commit
# is itself a release: the pseudo-version then only names that release, and a fix on
# another branch says nothing about it (beats/v7 at v9.5.5, its fix commit on 9.2).
fixes = [f for _, f in ranges if f and not PSEUDO.match(f)]
merged = [t for t in (git_out(clone, 'tag', '--merged', sha, '--list', 'v*') or '').split()
if RELEASE_TAG.match(t)]
if pseudo_fixes or not fixes or not merged or any('v' + f not in tags for f in fixes):
exact = [t for t in merged
if (git_out(clone, 'rev-parse', '-q', '--verify', t + '^{commit}') or '').strip() == sha]
if (
(pseudo_fixes and not exact) or not fixes or not merged
or any('v' + f not in tags for f in fixes)
):
kept += 1
continue
base = max(merged, key=semkey)
base = max(exact or merged, key=semkey)
if (
any(go_in_range(base, conds) for conds, _ in ranges)
or semkey(base) < max(semkey(f) for f in fixes)
Expand All @@ -859,8 +931,8 @@ def recheck_gopseudo(rep, path):
kept += 1
continue
why = (
f'descends from {base}, released after every fixed release '
f'({", ".join(sorted(set(fixes), key=semkey))}) and outside every affected range'
f'{"is release" if exact else "descends from"} {base}, released after every fixed '
f'release ({", ".join(sorted(set(fixes), key=semkey))}) and outside every affected range'
)
by = 'release'
ignore(m, f'go pseudo-version: {art["name"]} {art["version"]} is commit {sha[:12]}, which {why}')
Expand All @@ -870,17 +942,20 @@ def recheck_gopseudo(rep, path):
rep['matches'] = [m for m in rep['matches'] if id(m) not in drop]
rep['ignoredMatches'] = (
(rep.get('ignoredMatches') or []) + ignored['fix commit'] + ignored['release']
+ ignored['project advisory']
)
rep.setdefault('descriptor', {})['goPseudoRecheck'] = {
'source': GOMODULE_GIT,
'repositories': sorted(repos),
'judged': len(todo),
'ignored': {by: len(ms) for by, ms in ignored.items()},
'kept': kept,
'blanketJudged': len(blanket),
}
return (
f'Go pseudo-version matches: {len(todo)} judged; {len(ignored["fix commit"])} ignored by a '
f'fix commit, {len(ignored["release"])} by a release; {kept} kept'
f'fix commit, {len(ignored["release"])} by a release; {kept} kept. Go vulnerability database '
f'matches with no range: {len(blanket)} judged; {deferred} ignored by the project\'s own advisory'
)


Expand Down
42 changes: 40 additions & 2 deletions scripts/tests/test_recheckvulns.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -339,10 +339,48 @@ chk " kept" "$(q "$gkept")" "GHSA-k001@${LINKED:0:12} GHSA-k002@${LINKED:0:12}
chk " a fix commit the advisory names is an ancestor" "$(gwhy GHSA-f001)" "go pseudo-version: github.com/acme/widget $PL is commit ${LINKED:0:12}, which contains the fix commit ${FIX:0:12}"
chk " so is a fixed pseudo-version's commit" "$(gwhy GHSA-f002)" "go pseudo-version: github.com/acme/widget $PL is commit ${LINKED:0:12}, which contains the fix commit ${FIX:0:12}"
chk " descends from a release after every fix" "$(gwhy GHSA-r001)" "go pseudo-version: github.com/acme/widget $PL is commit ${LINKED:0:12}, which descends from v2.6.3, released after every fixed release (1.6.27, 2.2.4) and outside every affected range"
chk " descriptor" "$(q "r['descriptor']['goPseudoRecheck']['repositories'], r['descriptor']['goPseudoRecheck']['judged'], r['descriptor']['goPseudoRecheck']['ignored'], r['descriptor']['goPseudoRecheck']['kept']")" "['acme/widget'] 10 {'fix commit': 2, 'release': 1} 7"
chk " summary" "$OUT" "Go pseudo-version matches: 10 judged; 2 ignored by a fix commit, 1 by a release; 7 kept"
chk " descriptor" "$(q "r['descriptor']['goPseudoRecheck']['repositories'], r['descriptor']['goPseudoRecheck']['judged'], r['descriptor']['goPseudoRecheck']['ignored'], r['descriptor']['goPseudoRecheck']['kept']")" "['acme/widget'] 10 {'fix commit': 2, 'release': 1, 'project advisory': 0} 7"
chk " summary" "$OUT" "Go pseudo-version matches: 10 judged; 2 ignored by a fix commit, 1 by a release; 7 kept. Go vulnerability database matches with no range: 0 judged; 0 ignored by the project's own advisory"
chk " the clone holds no trees" "$(git -C "$T/gomodules/acme/widget.git" rev-parse --is-bare-repository)" "true"

# A build from a modified tree (+dirty); a pseudo-version that names a release, under the module path
# with a major-version suffix, whose fix commit is on another branch while the repository's other
# module path gives semantic ranges; the same advisory on a commit that is no release; an advisory
# naming only the other path. And the Go vulnerability database naming the old module path with no
# range, where the project's own advisory names only the new one.
PT=$(pseudo $(git -C "$G" rev-parse v2.6.3))
graw(){ echo "$2" > "$T/gadv/$1.json"; }
pk(){ echo "{\"package\": {\"name\": \"$1\", \"ecosystem\": \"go-module\"}, \"detail\": {\"ranges\": [$2]}}"; }
gadv GHSA-r002 "$(rg '<1.6.27' '"1.6.27"'), $(rg '>=2.0.0,<2.2.4' '"2.2.4"')" ""
graw GHSA-s001 "[{\"vulnerability\": {\"id\": \"GHSA-s001\", \"provider\": \"github\", \"refs\": []}, \"packages\": [$(pk github.com/acme/widget "$(rg '<1.6.27' '"1.6.27"'), $(rg '>=2.0.0,<2.2.4' '"2.2.4"')"), $(pk github.com/acme/widget/v7 "$(rg "<0.0.0-20260129092249-${SIDE:0:12}" "\"0.0.0-20260129092249-${SIDE:0:12}\"")")]}]"
graw GHSA-s002 "[{\"vulnerability\": {\"id\": \"GHSA-s002\", \"provider\": \"github\", \"refs\": []}, \"packages\": [$(pk github.com/acme/widget "$(rg '<2.2.4' '"2.2.4"')")]}]"
own='{"url": "https://github.com/acme/widget/security/advisories/GHSA-p00X"}'
govdb(){ graw $1 "[{\"vulnerability\": {\"id\": \"$1\", \"provider\": \"govulndb\"}, \"packages\": [$(pk github.com/acme/widget "$2")]}]"; }
ghsa(){ graw $1 "[{\"vulnerability\": {\"id\": \"$1\", \"provider\": \"github\", \"refs\": [$3]}, \"packages\": [$2]}]"; }
govdb GO-2099-0001 ""; ghsa GHSA-p001 "$(pk github.com/acme/widget/v2 "$(rg '>=2.1.0,<2.1.9' '"2.1.9"')")" "${own/p00X/p001}"
govdb GO-2099-0002 ""; ghsa GHSA-p002 "$(pk github.com/acme/widget/v2 "$(rg '>=2.1.0,<2.1.9' '"2.1.9"')")" ""
govdb GO-2099-0003 "$(rg '<1.7.40' '"1.7.40"')"; ghsa GHSA-p003 "$(pk github.com/acme/widget/v2 "$(rg '>=2.1.0,<2.1.9' '"2.1.9"')")" "${own/p00X/p003}"
govdb GO-2099-0004 ""; ghsa GHSA-p004 "$(pk github.com/acme/widget "$(rg '<1.7.40' '"1.7.40"')"), $(pk github.com/acme/widget/v2 "$(rg '>=2.1.0,<2.1.9' '"2.1.9"')")" "${own/p00X/p004}"
govdb GO-2099-0005 ""; ghsa GHSA-p005 "$(pk github.com/acme/widget/v2 "$(rg '>=2.1.0,<2.1.9' '"2.1.9"')")" "${own/p00X/p005}"
gv(){ echo '{"vulnerability": {"id": "'$1'", "namespace": "govulndb:language:go", "severity": "Critical", "fix": {"versions": [], "state": "not-fixed"}}, "relatedVulnerabilities": [{"id": "CVE-2099-'${1: -4}'"}, {"id": "'$2'"}], "matchDetails": [{"type": "exact-direct-match", "found": {"versionConstraint": "'$3'"}}], "artifact": {"name": "github.com/acme/widget", "version": "v1.7.36", "type": "go-module"}}'; }
goreport2(){ cat > "$T/vuln.json" <<EOF
{"descriptor": {"name": "grype"}, "matches": [
$(gm GHSA-r002 github.com/acme/widget ${PL}+dirty), $(gm GHSA-s001 github.com/acme/widget/v7 $PT), $(gm GHSA-s001 github.com/acme/widget/v7 $PL),
$(gm GHSA-s002 github.com/acme/widget/v7 $PT),
$(gv GO-2099-0001 GHSA-p001 'none (unknown)'), $(gv GO-2099-0002 GHSA-p002 'none (unknown)'), $(gv GO-2099-0003 GHSA-p003 'none (unknown)'),
$(gv GO-2099-0004 GHSA-p004 'none (unknown)'), $(gv GO-2099-0005 GHSA-p005 '<1.7.40 (go)')]}
EOF
}
goreport2; KERNEL_VULNS_GIT="file://$T/nowhere" GOMODULE_GIT="file://$T/gomods" GRYPE="$T/bin/grype" run "$T/vuln.json" "$T/checkout6" --gomodules "$T/gomodules"
g2kept="' '.join(m['vulnerability']['id'] + '@' + m['artifact']['name'].split('/')[-1] for m in r['matches'])"
gwhy2(){ q "next(m['appliedIgnoreRules'][0]['reason'] for m in r['ignoredMatches'] if m['vulnerability']['id'] == '$1' and m['artifact']['version'] == '$2')"; }
chk "Go pass, second report, exits 0" "$RC" "0"
chk " kept" "$(q "$g2kept")" "GHSA-s001@v7 GHSA-s002@v7 GO-2099-0002@widget GO-2099-0003@widget GO-2099-0004@widget GO-2099-0005@widget"
chk " a +dirty build is judged by its commit" "$(gwhy2 GHSA-r002 ${PL}+dirty)" "go pseudo-version: github.com/acme/widget ${PL}+dirty is commit ${LINKED:0:12}, which descends from v2.6.3, released after every fixed release (1.6.27, 2.2.4) and outside every affected range"
chk " a release, by the other module path's ranges" "$(gwhy2 GHSA-s001 $PT)" "go pseudo-version: github.com/acme/widget/v7 $PT is commit $(git -C "$G" rev-parse v2.6.3 | cut -c1-12), which is release v2.6.3, released after every fixed release (1.6.27, 2.2.4) and outside every affected range"
chk " the project's own advisory names only the new module path" "$(gwhy2 GO-2099-0001 v1.7.36)" "Go vulnerability database: GO-2099-0001 names github.com/acme/widget with no affected range; the project's own advisory GHSA-p001 names only github.com/acme/widget/v2"
chk " summary" "$OUT" "Go pseudo-version matches: 4 judged; 0 ignored by a fix commit, 2 by a release; 2 kept. Go vulnerability database matches with no range: 4 judged; 1 ignored by the project's own advisory"

# The next scan fetches what changed: a new release
git -C "$G" checkout -q main && gc 2026-03-01T00:00:00 four && gt v2.7.0 && git -C "$T/gomods/acme/widget" fetch -q "$G" '+refs/heads/*:refs/heads/*' '+refs/tags/*:refs/tags/*'
goreport; KERNEL_VULNS_GIT="file://$T/nowhere" GOMODULE_GIT="file://$T/gomods" GRYPE="$T/bin/grype" run "$T/vuln.json" "$T/checkout6" --gomodules "$T/gomodules"
Expand Down
Loading