Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 69 additions & 15 deletions go/test/site/runner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,30 @@ the result as a GitHub commit status on the exact commit it tested:
| Status | When | What runs |
|---|---|---|
| `siterig/full` | each new commit on `master` | every scenario |
| `siterig/short` | the head of each open pull request from this repository that touches `go/internal/{control,loadpoint,ocpp,telemetry}` or `go/test/site` | the short set |
| `siterig/short` | the head of each open pull request from this repository that touches a watched path (below) | the short set |
| `siterig/random` | once a night on `master` | that night's random seeds |

The watched paths are the control packages, every internal package they
import, and the rig itself:

- `go/cmd/ftw` (control tick wiring)
- `go/internal/{control,loadpoint,ocpp,telemetry}`
- `go/internal/{drivers,mdnsresolve}` (driver host, command deadlines)
- `go/internal/{mpc,optimizercontract}` (plans)
- `go/internal/{gridcost,forecasting,sunpos}` (plan inputs)
- `go/internal/{config,configreload}` (limits)
- `go/internal/units` (charge steps)
- `go/internal/{cars,sitepower,events,state}`
- `drivers/` and `optimizer/native/bundle/` (the bundled drivers and solver)
- `go/test/site`

`go/cmd/ftw` imports every package, the API and UI among them; only its own
files are watched. A change to the API or UI alone does not run the rig.

`watchedPaths` in `runner.go` is the list; `TestTouchesWatched` pins it.
`TestWatchedPathsCoverImports` fails when a control package starts to
import an internal package that is not on the list.

A commit that already has the status is not run again. To run one again,
post a new `pending` status or remove the result and wait a day. No GitHub
Actions workflow runs the rig; `native-release.yml` only reads
Expand All @@ -25,43 +46,76 @@ rig's timing, and the rig then fails the run as "host paused". Not the home
energy box: the rig needs the CPU, and the box is production.

The rig uses 100 ports per lane from `-port-base` (default 18700) on
127.0.0.1. Leave them free.
127.0.0.1: lane k uses `port-base + 100k` up to 99 above it, so three lanes
take 18700 to 18999. Leave them free. `make siterig` uses the same default,
so on the runner's host run it with `FTW_SITE_PORT_BASE=19700`. A second
runner on one host needs its own `-port-base` and `-work`.

Pull requests from forks are skipped unless `-allow-forks` is set: the rig
builds and runs the pull request's code on this machine.

## Install

The runner keeps its clone in `~/siterig/ftw` (`-work` plus `ftw`) and
clones it there on first start if it is missing. Build the binary from the
same clone:

```bash
gh auth login # an account that may write commit statuses on srcfl/ftw
mkdir -p ~/siterig/bin
git clone https://github.com/srcfl/ftw ~/siterig/src
cd ~/siterig/src/go && go build -o ~/siterig/bin/siterig-runner ./test/site/runner
git clone https://github.com/srcfl/ftw ~/siterig/ftw
cd ~/siterig/ftw/go && go build -o ~/siterig/bin/siterig-runner ./test/site/runner
~/siterig/bin/siterig-runner -once # one pass in the foreground
```

To rebuild after a change to this directory:

```bash
cd ~/siterig/ftw && git pull --quiet origin master
cd go && go build -o ~/siterig/bin/siterig-runner ./test/site/runner
```

then restart the service. The scenarios themselves always come from the
commit under test.

macOS (launchd):

```bash
sed "s/USER/$USER/g" ~/siterig/src/go/test/site/runner/energy.ftw.siterig-runner.plist \
sed "s/USER/$USER/g" ~/siterig/ftw/go/test/site/runner/energy.ftw.siterig-runner.plist \
> ~/Library/LaunchAgents/energy.ftw.siterig-runner.plist
launchctl load ~/Library/LaunchAgents/energy.ftw.siterig-runner.plist
```

Linux (systemd): edit `siterig-runner.service`, copy it to
`/etc/systemd/system/` and run `sudo systemctl enable --now siterig-runner`.
The log is `~/siterig/runner.log`.

Linux, as a system service: replace `USER` in `siterig-runner.service`,
copy it to `/etc/systemd/system/` and run
`sudo systemctl enable --now siterig-runner`. The log is
`journalctl -u siterig-runner`.

Linux, as a user service. Root is needed once, for
`loginctl enable-linger`, which keeps the service running after you log out
and starts it at boot:

```bash
mkdir -p ~/.config/systemd/user
sed -e '/^User=/d' -e "s/USER/$USER/g" -e 's/multi-user.target/default.target/' \
~/siterig/ftw/go/test/site/runner/siterig-runner.service \
> ~/.config/systemd/user/siterig-runner.service
systemctl --user daemon-reload
systemctl --user enable --now siterig-runner
sudo loginctl enable-linger "$USER"
```

The log is `~/siterig/runner.log` (launchd) or `journalctl -u
siterig-runner` (systemd). Rebuild the binary after a change to this
directory; the scenarios themselves always come from the commit under test.
The log is `journalctl --user -u siterig-runner`. Check `HOME` and `PATH`
in the unit if your home is not `/home/$USER` or Go lives elsewhere.

## How long it takes

On an M5 MacBook (10 cores), 22 October 2026: the 23 scenarios of that
morning took 72 minutes in three lanes. With the one-phase charger cases the
set adds up to about 270 minutes, so three lanes need about 90 and four
about 70; `s12-20` alone takes up to 56, so more than four do not help.
Each lane is one `ftw` and one `simulator run`, both mostly idle.
On an M5 MacBook (10 cores), 27 October 2026: the full set took about
2 h 20 min in three lanes. `s12-20` alone took up to 56 minutes on
22 October, so more than four lanes do not help. Each lane is one `ftw` and
one `simulator run`, both mostly idle.

## Flags

Expand Down
2 changes: 1 addition & 1 deletion go/test/site/runner/main.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// siterig-runner runs FTW's site test (go/test/site) on a machine of its own
// and publishes the result as a GitHub commit status on the exact commit:
// siterig/full on each new master commit, siterig/short on the head of each
// open pull request that touches control, loadpoint, OCPP or telemetry code,
// open pull request that touches code on the control path (watchedPaths),
// and siterig/random for a night's random seeds. native-release.yml reads
// siterig/full before it publishes a beta. See README.md.
package main
Expand Down
20 changes: 19 additions & 1 deletion go/test/site/runner/runner.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,30 @@ const (
)

// watchedPaths are the parts of the tree whose changes get the short set on
// a pull request.
// a pull request: the control packages, every internal package they import,
// and the rig itself. TestWatchedPathsCoverImports keeps the imports in.
var watchedPaths = []string{
"go/cmd/ftw/", // control tick wiring
"go/internal/control/", "go/internal/loadpoint/", "go/internal/ocpp/", "go/internal/telemetry/",
"go/internal/drivers/", "go/internal/mdnsresolve/", // driver host, command deadlines
"go/internal/mpc/", "go/internal/optimizercontract/", // plans
"go/internal/gridcost/", "go/internal/forecasting/", "go/internal/sunpos/", // plan inputs
"go/internal/config/", "go/internal/configreload/", // limits
"go/internal/units/", // charge steps
"go/internal/cars/", "go/internal/sitepower/",
"go/internal/events/", "go/internal/state/",
"drivers/", "optimizer/native/bundle/", // bundled drivers and solver
"go/test/site/",
}

// controlPackages are the watched packages whose imports must be watched
// too. go/cmd/ftw is left out: it imports every package, the API and UI
// among them.
var controlPackages = []string{
"control", "loadpoint", "ocpp", "telemetry", "drivers", "mpc",
"optimizercontract", "config", "configreload", "cars", "sitepower",
}

// job is one run of the rig on one commit.
type job struct {
Context string // which status it publishes
Expand Down
76 changes: 76 additions & 0 deletions go/test/site/runner/runner_test.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package main

import (
"os/exec"
"strings"
"testing"
"time"
Expand Down Expand Up @@ -75,6 +76,81 @@ func TestPlanRunsWhatIsDue(t *testing.T) {
}
}

func TestTouchesWatched(t *testing.T) {
for _, tc := range []struct {
file string
want bool
}{
{"go/cmd/ftw/main.go", true},
{"go/internal/control/dispatch.go", true},
{"go/internal/loadpoint/loadpoint.go", true},
{"go/internal/ocpp/handlers.go", true},
{"go/internal/telemetry/store.go", true},
{"go/internal/drivers/lua.go", true},
{"go/internal/mpc/mpc.go", true},
{"go/internal/optimizercontract/contract.go", true},
{"go/internal/config/config.go", true},
{"go/internal/configreload/apply.go", true},
{"go/internal/cars/cars.go", true},
{"go/internal/sitepower/sitepower.go", true},
{"go/internal/units/charge_steps.go", true},
{"go/internal/gridcost/gridcost.go", true},
{"go/internal/state/store.go", true},
{"go/internal/events/bus.go", true},
{"drivers/sungrow.lua", true},
{"drivers/BUNDLED_SOURCE.json", true},
{"optimizer/native/bundle/manifest.json", true},
{"go/test/site/rig_test.go", true},
{"go/test/site/runner/runner.go", true},
{"docs/architecture.md", false},
{"web/components/theme.css", false},
{"go/internal/api/api.go", false},
{"go/internal/driverrepo/repo.go", false}, // shares a prefix with drivers/, not a path
{"go/internal/configx/x.go", false}, // shares a prefix with config/
{"optimizer/native/src/main.rs", false}, // only the bundle ships
{"go/cmd/ftwctl/main.go", false}, // shares a prefix with cmd/ftw/
{"web/drivers/list.js", false}, // drivers/ only at the root
{".changeset/a.md", false},
} {
if got := touchesWatched([]string{tc.file}); got != tc.want {
t.Errorf("%s: %v, want %v", tc.file, got, tc.want)
}
}
if touchesWatched(nil) {
t.Error("no files: watched")
}
if !touchesWatched([]string{"README.md", "go/internal/mpc/plan.go"}) {
t.Error("one watched file among others: not watched")
}
}

// TestWatchedPathsCoverImports fails when a control package starts to
// import an internal package that is not watched, so a change there would
// skip the rig.
func TestWatchedPathsCoverImports(t *testing.T) {
const module = "github.com/srcfl/ftw/go/internal/"
args := []string{"list", "-deps"}
for _, p := range controlPackages {
if !touchesWatched([]string{"go/internal/" + p + "/x.go"}) {
t.Errorf("control package %s is not watched", p)
}
args = append(args, module+p)
}
out, err := exec.Command("go", args...).Output()
if err != nil {
t.Fatalf("go list: %v", err)
}
for _, dep := range strings.Fields(string(out)) {
rest, ok := strings.CutPrefix(dep, module)
if !ok {
continue
}
if !touchesWatched([]string{"go/internal/" + rest + "/x.go"}) {
t.Errorf("%s imports %s, which is not watched", strings.Join(controlPackages, ", "), rest)
}
}
}

func TestDescribe(t *testing.T) {
state, desc := describe([]byte(`[{"name":"s1-25","verdict":"pass"},{"name":"s2-16","verdict":"known"},{"name":"s9-25","verdict":"fail"}]`), 24*time.Minute)
if state != "failure" || desc != "1 pass, 1 known, 1 fail of 3 in 24m0s: s9-25" {
Expand Down
1 change: 1 addition & 0 deletions go/test/site/runner/siterig-runner.service
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# systemd unit for siterig-runner (README.md). Replace USER and the paths,
# then: sudo cp siterig-runner.service /etc/systemd/system/ &&
# sudo systemctl enable --now siterig-runner
# README.md also shows how to run it as a user service.
[Unit]
Description=FTW site test runner (siterig/short, siterig/full commit statuses)
After=network-online.target
Expand Down
Loading