fix(vmware-esxi): one alert per host for three ESXi rules

[kryonsx]

Why

Three ESXi rules break the limit of 10 alerts per hour on the one v11 deployment that sends ESXi logs (three hosts). Between 2026-09-28 00:00 and 2026-09-29 18:00 UTC:

Rule	Alerts	Worst hour
Virtual Machine Escape Detection	140	140
ESXi Syslog Forwarding Disruption Detection	271	40
ESXi Firewall Rule Modification Detection	20	12

All three group by adversary.hostname, which ESXi alerts never carry (the ESXi filter writes no origin.hostname). Grouping keys that do not resolve are skipped, so the syslog and firewall rules opened a new top-level alert for every matching record, and the VM escape rule, left with lastEvent.log.process, stored a child alert for every record under one parent per process name.

Noise is reduced here only by de-duplication. The conditions are unchanged, and nothing is excluded by name.

What changes

Rule	Before	After
Virtual Machine Escape Detection (1.0.0 to 1.1.0)	groupBy lastEvent.log.process, adversary.hostname	deduplicateBy dataSource, lastEvent.log.process
ESXi Syslog Forwarding Disruption Detection (1.0.0 to 1.1.0)	groupBy adversary.hostname	deduplicateBy dataSource
ESXi Firewall Rule Modification Detection (1.0.0 to 1.1.0)	groupBy adversary.hostname	deduplicateBy dataSource

dataSource is the ESXi host. One alert is raised per host (and process, for the VM escape rule); repeats are dropped for seven days.

Expected volume

Replaying 30 days of real records from the deployment's three ESXi hosts with the conditions as they are and seven-day de-duplication:

Rule	Matching records in 30 days	Alerts	Worst hour	Worst day
Virtual Machine Escape Detection	12,008	5	1	1
ESXi Syslog Forwarding Disruption Detection	7,270	9	1	1
ESXi Firewall Rule Modification Detection	458	10	2	2

ESXi writes some of these records two to four at a time in the same second, and the engine stores alerts that arrive together in parallel, so each alert can come with up to three copies (at most 4 in any hour or day); that engine limit is separate from this change.

What the conditions matched is routine: guest file operations from vCenter's agent that returned FileNotFound or InvalidArgument (VM escape), the remote syslog host becoming unreachable (syslog disruption), and the configuration store processing its nfs_firewall_rulesets plug-in file (firewall modification). Narrowing those conditions is left for a separate review.

Tests

• plugins/alerts/vmware_esxi_alert_volume_test.go (new) evaluates parsed ESXi records (process and message) with the pinned go-sdk v1.1.36 CEL. It pins the de-duplication keys, checks that they resolve, and checks one record each rule must match and one it must not.
• go test ./... in plugins/alerts passes on this branch, which is based on current v11 (89cd26c4).
• Local EventProcessor lab (engine 8a3ade7, go-sdk v1.1.36, the production events and alerts plugins, OpenSearch 2.19.1, this branch's ESXi filter from current v11) with fabricated ESXi syslog lines in two runs: one matching line per rule on one host plus two lines that must not match, then the same three lines again on that host and on a second host. Both runs processed all of their events, and the real parser filled dataSource and log.process on every line.
  • With this branch: each rule produced 3 alerts and stored 2 (one per host); the repeat on the same host was dropped as a duplicate. The two lines that must not match produced none.
  • With the rules as shipped: the syslog and firewall rules stored every alert as a new top-level alert (adversary.hostname resolved on none of them), and the VM escape rule stored one parent and two children, grouping both hosts under the process name.

🤖 Generated with Claude Code