Skip to content

feat(thermaltwin): learn each declared room's thermal twin in shadow - #1996

Merged
frahlg merged 2 commits into
srcfl:masterfrom
hannesb90:feat/thermal-twin-shadow
Nov 15, 2026
Merged

frahlg merged 2 commits into
srcfl:masterfrom
hannesb90:feat/thermal-twin-shadow

Conversation

@hannesb90

@hannesb90 hannesb90 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #1995. This branch contains #1995's commits (071ccfd, f066969), which declare sensors. Review only the last commit, 5150469. It rebases cleanly once #1995 merges.

Problem and result

Step 3 of ADR 0012 (#1994), following the owner's answer there: "Go ahead, starting with the twin in shadow." Planning heat needs to know how long a room coasts and how hard the heat source moves it. FTW measures neither today. This learns both, per declared room sensor, and commands nothing.

Model. One first-order twin per zone, fitted online by recursive least squares over 30-minute windows:

dT_in/dt = a·(T_out − T_in) + b·P + c
Term Reads as
a 1/τ: inertia (tau_h)
b 1/C: K per kWh of heat input (capacity_kwh_per_k)
a/b heat loss (ua_w_per_k)
c other gains: sun, people, other heat (other_gain_k_per_h)

Inputs.

  • T_in is a sensor declared with asset = "space_heat" (feat(drivers): let a control declare what it does to heat #1995).
  • T_out comes from the same forecast cache loadmodel learns from.
  • P is hp_power_w from the sensor's own driver, or from the single other driver that reports it. With two candidates the twin does not guess: it learns without heat input and says so.

Judgement. The twin is never judged by its own fit. Each three-hour stretch is simulated from the parameters it had at the start, using the inputs that really happened, and compared with the measured temperature. Status is learning, then trusted once there is a week of windows, a day of judgements, physical parameters and a mean error ≤ 0.3 K. It becomes drifting above 0.5 K, or no_heat_input when no heat input is available.

GET /api/thermal/twins returns every twin with shadow: true. The route is also listed for analysis agents.

Scope and safety

  • Commands nothing. No new path to Registry. Per the owner's answer 3, an untrusted twin gives no planned control, and today no twin gives any.

  • Layer, not core (docs(vision): a hard, simple core with layers on top #1772). The twin runs in its own goroutine and reads only LatestMetric (no RawW/SmoothedW/ReadingsBy*) and the forecast cache. It never holds up the control tick. It fails soft: a missing catalog, sensor or outdoor temperature just skips windows.

  • Bad data is rejected, not learned:

    • a reading older than 5 min, or fewer than 25 of 30 minutes in a window;
    • more than 3 K/h of change, or an implausible temperature or power;
    • the heat input source changing mid-window.

    A changed source restarts the twin, because a fit to one source means nothing against another.

  • Persistence goes through modelstate with a feature fingerprint, so state fitted to another model form cold-starts.

  • Learning steps: none, per the owner's answer 4. The twin learns only from ordinary operation and owner holds.

  • Only space_heat zones. A tank or buffer has the same form, but it loses heat to the room it stands in, which no sensor here gives yet.

Known limits, measured in tests rather than assumed:

  • After a step change in the house (insulation, for example), the 3 h prediction recovers within a week, but τ needs about three weeks. Until then the bias term absorbs the difference. A later planner must therefore not use τ from a twin whose house recently changed. TestTwinFollowsAChangedHouse pins this.
  • b is learned against electrical input, so it includes COP, which varies with outdoor temperature. That is good enough for a shadow twin, and it is a known refinement before planning.

Verification

Raspberry Pi 4 (linux/arm64), Go 1.26.10, pinned drivers fetched:

go test ./internal/thermaltwin ./internal/api ./internal/drivers ./internal/thermal -count=1   ok
go test ./cmd/ftw -run ThermalTwin -count=1                                                 ok
go vet ./internal/thermaltwin ./internal/api ./cmd/ftw                                       clean
  • Model tests use a synthetic house with known physics (τ 40 h, 15 kWh/K):
    • it learns τ, C and UA within 10–15 %;
    • it earns trust through 0.1 K sensor quantisation;
    • it is not trusted after one day;
    • it reports drift the same day for an unexplained heat source;
    • it relearns a changed house.
  • Service tests cover persistence and restore, a cold start on a foreign fingerprint, a stale sensor, an ambiguous power source, preferring the sensor's own driver, zone removal and missing outdoor temperature.
  • Not run:
    • The race detector: ThreadSanitizer does not support this Pi kernel's 39-bit VMA, so I rely on CI for it.
    • make verify locally.
    • Real-site data: there is none yet. That is what shadow mode is for, and real-site behaviour is unverified until it has run on a real house.

Checklist

  • The change follows VISION.md and one selected scope.
  • I checked overlapping PRs and coordinated shared files/contracts.
  • Relevant checks cover the changed behaviour and failure paths.
  • A human reviewed changed UI in a browser, or no UI changed.
  • A Changeset is included, or the change is exempt.
  • Every commit has a DCO sign-off.

🤖 Generated with Claude Code

@frahlg frahlg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and tested: the shadow property holds. The service reads only LatestMetric and the outdoor temperature, has no path to the registry or a driver default, runs in its own goroutine, and the stack passes make verify on today's master. One thing before merge:

  1. "Trusted" does not show skill yet (thermaltwin/model.go, Status() and physical()). A twin is trusted on 3 h mean error ≤ 0.3 K, a physical range for tau and capacity, and some power samples. It is never compared with "the temperature stays as it is". In a thermostat-held house (indoor 21 ± 0.15 K, heat following the cold, three weeks of windows) a probe reached trusted at 0.08 K MAE, although (Tout−Tin) and P move together and the room never moves, so tau and C cannot be learned there. Nothing reads the flag today, but ADR 0012 makes it the gate for planned control, and #2013 would be its first reader. Please judge against persistence (the error must beat "no change") and require enough variation in heat input in the windows judged; until then, call the state something other than trusted.

Lower:

  • The heat input is hp_power_w, the heat pump's total electrical power, which includes hot-water charging and defrost, so b is biased. A space-heat power metric declared by the driver would fix it; worth a line in the PR or the ADR.
  • thermalTwins.Zones re-reads and parses the whole driver catalog every 30 minutes on every site, heat pump or not, while Tick holds mu. Cheap to skip when no enabled driver declares sensors.
  • The new thermaltwin/<driver>/<metric> rows stay in the config store when a zone goes away; delete them with the zone.

hannesb90 and others added 2 commits November 15, 2026 08:04
Step 3 of ADR 0012 (srcfl#1994). For every room sensor a driver declares,
FTW fits dT/dt = a(Tout - Tin) + b*P + c by recursive least squares over
30-minute windows: inertia, capacity, heat loss and other gains. The
twin is judged by simulating each three-hour stretch from the parameters
it had at the start with the inputs that really happened, and reports
learning, trusted, drifting or no_heat_input.

It reads LatestMetric and the load model's outdoor temperature, persists
through modelstate, and is shown at GET /api/thermal/twins with
shadow: true. It commands nothing and nothing plans from it.

Refs srcfl#520

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: hannesb90 <[email protected]>
Review of srcfl#1996: a twin reached "trusted" in a house a thermostat holds
still, at 0.08 K error, although outdoor temperature and heat input move
together there and the room never moves, so inertia and capacity cannot
be learned. The error was never compared with "the temperature stays as
it is".

A twin is now trusted only when, over a week of three-hour judgements,
its error is at most 0.8 of the error of assuming no change, and heat
input varied within at least a fifth of the judged stretches. A twin
whose error is within the limit without that reads "unproven". The API
shows both errors and the share. Stored twins of the earlier form cold
start.

Also from the review:

- The service no longer reads the driver catalog. main hands it the
  zones from the catalog it already loads at start and on each applied
  configuration, so nothing is parsed every 30 minutes or under the
  service's lock.
- A zone that is no longer declared has its stored twin deleted, also
  when it went away while FTW was not running. An incomplete zone list
  (a driver with no catalog entry) forgets nothing.
- hp_power_w includes hot-water charging and defrost, which biases b;
  said where the metric is named and in the changeset.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Signed-off-by: hannesb90 <[email protected]>
@hannesb90
hannesb90 force-pushed the feat/thermal-twin-shadow branch from 1f1e0be to 5ea4f36 Compare November 15, 2026 07:20
@hannesb90

Copy link
Copy Markdown
Contributor Author

Thanks for the probe. Rebased onto master and addressed in 5ea4f36.

1. Trusted now needs skill. Each three-hour judgement is also scored for "the temperature stays as it is", and for whether heat input varied within the stretch (lowest window at most 70 % of the highest, and the highest at least 0.1 kW). Over a week of judgements a twin is trusted only when its error is at most 0.8 of the no-change error and heat input varied in at least a fifth of the stretches. A twin within the 0.3 K limit without that reads unproven. GET /api/thermal/twins shows both errors and the share.

TestTwinDoesNotTrustAThermostatHeldHouse is your case: indoor 21 ± 0.15 K, heat following the cold, three weeks. It ends at 0.10 K error against 0.09 K for no change, heat varied in 0 % of stretches, physical parameters in range, so the old rule trusted it. It now reads unproven. The 0.8, 0.7 and one-fifth are proposals; say if you want them elsewhere.

Stored twins of the earlier form cold start, since the fingerprint changed.

Lower:

  • hp_power_w: said where the metric is named and in the changeset. It includes hot-water charging and defrost, so b is biased low until a driver can declare space-heat power.
  • Catalog: the service no longer reads it. main hands it the zones from the catalog it already loads at start and on each applied configuration, so nothing is parsed every 30 minutes or under mu. The cost is that a hot-edited driver's sensors are picked up at the next applied configuration or restart, not within a window.
  • Stored rows: a zone that is no longer declared has its thermaltwin/... row deleted, also when it went away while FTW was not running. If an enabled driver has no catalog entry the list counts as incomplete and nothing is deleted, so an unreadable driver file does not cost weeks of learning. This adds Store.DeleteConfig.

Not run here: the race detector (unsupported on the Pi I tested on); CI ran it green.

@miravoss26 miravoss26 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thermal twin in shadow mode: an online RLS fit per declared room sensor, an /api/thermal/twins read endpoint, persistence in the state store, and a changeset. It commands nothing. Stacked on #1995, so this review only covers the surface a reviewer will hit first, not the full ~1400 added lines.

What I checked:

  • Read-only HTTP endpoint (GET /api/thermal/twins). No new deps, no new hosts, no secrets, no shell-outs. The state write goes through the existing durableConfigWrite transaction with a parameterised DELETE.
  • Test files exist for the model, the service, the API and the store, including tolerance checks on recovered tau / capacity / UA.

What I did not do: verify the RLS maths in thermaltwin/model.go or the windowing in service.go. That is the part that needs a human with the ADR 0012 context. Also, this PR can't merge before #1995, so merge order matters.

Needs a human decision on the model review. Nothing in the security screen blocks it.

@frahlg frahlg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, all four points are addressed. At 5ea4f36, merged with today's master (including #2009, which also touches state/store.go): go build ./... and go vet are clean, and go test passes for state, thermaltwin, cmd/ftw and api.

A probe outside the PR (not committed) confirms the new gate. A thermostat-held house stays unproven: MAE 0.19 K against 0.18 K for no change, with no heat variation. Pump cycling and a nightly setback reach trusted. The shadow property holds: only main.go and the read-only GET /api/thermal/twins reach thermaltwin.

Notes for later, not blocking:

  • A twin that was trusted and then fails the skill test reads unproven, not drifting.
  • The setback case had a heat-varied share of 0.23, close to the 0.2 gate, so real houses may sit near it. Worth watching once sites report.

@frahlg
frahlg merged commit 4d704b9 into srcfl:master Nov 15, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants