diff --git a/.changeset/app-history-measured-power.md b/.changeset/app-history-measured-power.md deleted file mode 100644 index af11336df..000000000 --- a/.changeset/app-history-measured-power.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"ftw": patch ---- - -The app's history for solar, grid and battery now shows the measured power, the same as the box's own charts. Before, it came from the energy counters, which on a Sungrow move in 0.1 kWh steps, so each five minutes read 0 or 1.2 kW while the sun gave 0.4 kW. Home use is unchanged. Five minutes in which a flow went unmeasured for more than half the time is now a gap for that flow, not a lower value. diff --git a/.changeset/ios-palette-web.md b/.changeset/ios-palette-web.md deleted file mode 100644 index f4a751306..000000000 --- a/.changeset/ios-palette-web.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"ftw": patch ---- - -The box's dashboard now uses the phone app's colours. The grid is one pink colour both ways, and the sign and the word say whether you buy or sell. The battery is cyan when it stores power and orange when it gives power, and the car is violet. Green now only means live or confirmed, and red only means refused, failed or blocked. Prices are shades of ink, not red and green. diff --git a/.changeset/pilot-evidence.md b/.changeset/pilot-evidence.md new file mode 100644 index 000000000..124de0a9f --- /dev/null +++ b/.changeset/pilot-evidence.md @@ -0,0 +1,5 @@ +--- +"ftw": patch +--- + +Expose fresh effective EV pilot current separately from the dynamic current setting. A confirmed lower pilot that explains measured delivery is an informational charger limitation even when its internal cause is unknown. Stale or future pilot evidence cannot explain an EV shortfall. A pilot that is only past its 90 s window reads as unknown, and the driver's reported limit, such as Easee's load balancing, still stands. diff --git a/AGENTS.md b/AGENTS.md index 110dbadd9..6c7f3839f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -200,13 +200,21 @@ Native 0.x path: week, with no open `release-blocker`. 4. Dispatch the same workflow for `v0.X.Y` stable, naming the tested beta. The workflow checks the source commit and release assets, and keeps GitHub - `releases/latest` on the old 2.x line. + `releases/latest` on the old 2.x line. Newer betas may ship while the + sites test the candidate; the workflow refuses a stable only when a newer + stable exists. Beta boxes still take the highest release, so a late stable + does not hide a newer beta. 5. Promote the signed driver channel for installs that read `drivers-stable`: `gh workflow run ftw-drivers-release.yml -R srcfl/device-drivers --ref main -f channel=stable`. It promotes the driver commit already published as `drivers-beta`. If it refuses because `main` has moved past `drivers-beta`, publish beta first (`-f channel=beta`), then promote. +The workflow runs only from `master`, and a beta tag must match +`package.json` there. So there is no patch branch for an older line: fix a +stable on `master`, publish the next beta and promote it by the steps above. +The site check starts again with that beta. + The old lines are retired: no further 2.x or 3.x releases, including safety hotfixes or betas. All new Core releases use the new 0.x path. Do not recommend 3.x beta as an install or intermediate upgrade. Old clients and aliases may diff --git a/CHANGELOG.md b/CHANGELOG.md index a021316eb..415572bbe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,12 +1,199 @@ # Changelog +## 0.159.1 + +### Patch Changes + +- fd3349a: Update the bundled Energyplan planner to 0.7.0. With a peak-power tariff it now plans with the hour already in progress, so each day's peak counts once. A new plan is never worse for the car goals or the bill than the plan the planner already holds, a site whose limits cannot all be met gets a clear reason instead of a failed plan, and the planner needs less memory. The bundle now carries the Linux workers only; a Mac keeps Core's own planner. Core still validates every plan. +- ec8ce36: `make native-solver-check` no longer fails on a host the Energyplan bundle has + no worker for, such as a Mac when the bundle ships only the Linux workers. The + verifier still checks every bundled file and skips only the handshake there, + as it already did on other unsupported hosts. + +## 0.159.0 + +### Minor Changes + +- f0a7202: A phone now hears when a device does not do what FTW asked. The new + `control.not_following` notification fires when the box's answer to "Are we + in control?" has said a device is not following for ten minutes, as when a + battery charges while FTW asks it to discharge. It fires once until the + device follows again, and at most once an hour. It is on by default for + owners who have notifications on, and each owner can turn it off with the + other notification choices. + +## 0.158.5 + +### Patch Changes + +- 2e9c780: Bundle growatt 2.1.3. A Growatt inverter with no battery no longer shows a + second battery at 0 W and 0 % beside the site's real one: the driver now + reports a battery only when its voltage, charge level or power is above 0. +- 3e0a9cb: The forecast caution plan is no longer lost to a late call. Core fixed each + worker call's time limit when planning started, so after a point plan the + caution call could ask the worker for more time than the call had left. A + worker that used its full time limit then answered after Core had given up, + and Core published the point plan instead. Each call's time limit now fits + the time left before that call's deadline, and Core skips a call that has too + little time left, as it already does near the end of a slot. +- 7c8430c: Price the clock hour in progress when a plan starts part way through it. With 15-minute prices Core replans at :15, :30 and :45, and the peak-power tariff it sends the planner left out the hour already running, so a car plugged in at 17:20 could set a new monthly peak in that hour. Core now sends that hour with the import it has measured so far (from the site meter's hourly counter, else the history; time neither covers counts at no less than the house's current import), and today's completed peak goes in today's group, so each day is one sample. Only a planner that announces `demand_elapsed_hours` gets this shape; a planner without it, such as the bundled one, gets exactly the request it got before. + +## 0.158.4 + +### Patch Changes + +- a71fb80: A battery that may not sell to the grid exports far less while it covers the + house. In planner_self, planner_cheap and passive arbitrage, a discharging + battery now holds the grid a dead band above 0 W instead of around it, so a + drop in house load no longer sells battery energy about half the time. The + site imports a little more instead; that energy stays in the battery. + +## 0.158.3 + +### Patch Changes + +- fc4895a: A battery whose Max SoC was saved as 0 charges again. Settings used to save a + blank number field as 0, and since 0.157.1 that 0 was the battery's charge + ceiling, so charging stopped at any level, manual Charge included. Core now + treats a battery soc_max of 0 or less as unset and uses the planner's ceiling, + and Settings saves a cleared number field as unset. + +## 0.158.2 + +### Patch Changes + +- e6e389c: Bundle zap 3.1.3. `read_pv` and `read_battery` work again on a Sourceful Zap with firmware 7.x, which renamed the PV and battery fields in its local API. A site that reads PV or battery through its Zap gets those readings back. Sites with a native inverter driver are not affected. + +## 0.158.1 + +### Patch Changes + +- be11e36: The energy tax refund figure for battery storage now counts in proportion. Whatever leaves a battery, to the house or the grid, carries the grid's share of what the battery held, after the battery's efficiency over the year. The old order counted the house's use as grid energy first and the export as other energy first, so a battery with solar almost never got a figure. The home box's 2026 hours now give 42 kWh instead of 0. +- 7210076: Bundle zap 3.1.2. A Sourceful Zap works as the site meter again after it updates to firmware 7.x. That firmware renamed the fields in its local API, so the driver found no power reading and FTW lost its site meter and stopped dispatching. The driver now reads both the old and the new names. `read_pv` and `read_battery` on a Zap with firmware 7.x are not covered yet. + +## 0.158.0 + +### Minor Changes + +- 875240e: History shows the figure a Swedish household gives Skatteverket to get back the energy tax on grid electricity it stored in a battery and later sent back to the grid. Pick a calendar year to see the kWh, month by month, and download the hourly basis as CSV to keep with the application. Where the readings cannot tell where energy went, FTW counts low: solar counts first in every hour, what the house takes from the battery counts as grid energy first and what goes to the grid as other energy first, losses count as other energy, and an hour counts only when every meter read all of it. The figure needs each battery's size set, and it assumes FTW reads every solar inverter, battery and car charger on the site. The same figure is at `GET /api/energy/battery-return?year=` and the hours at `/api/energy/battery-return.csv`. +- b322f55: Store the home's time zone on the site. Car goals, the import hour, daily history and savings now read the site's `site.time_zone` instead of the host's clock, so a Docker box on UTC meets a 17:00 goal at 17:00 at home. The app, the web client and the on-box setup offer their own zone to a box that keeps none, and the box restarts once to run on it. A box that already keeps a zone, stored or from its host, keeps it; change it under Settings → Site. + +### Patch Changes + +- 9724190: The figure for Skatteverket's energy tax refund after battery storage now works for sites with more than one battery. Each battery keeps its own account of grid and other energy, with its own size and efficiency, so one battery's exports never carry another's grid energy, and for each battery solar, a car and the other batteries count first in every hour. Each battery takes its size from the driver that reads it, and the hourly CSV has one row per battery and hour. +- fdc2741: Bundle tesla_wall_connector 0.1.2. An idle Tesla Wall Connector Gen 3 no longer shows charging power. With the contactor open, the unit still reads 0.5–1 A of sensor offset on its legs, and the driver multiplied that by the grid voltage, so an idle charger showed about 180–280 W in the flow and in history. The driver now reports 0 W while the contactor is open, and a leg that reads 0 V carries no power. +- cd1ce6e: The energy ledger keeps hourly totals for four years instead of two, then daily totals. Skatteverket's refund of energy tax after battery storage is worked out hour by hour for a calendar year and may be claimed until three years after the year ends; with two years of hours, the start of an older year had only daily totals and counted as incomplete. +- c6f757b: Name the box's time zone by the rules its clock runs on. A Docker box that mounts the host's `/etc/localtime` used to tell the app and web client it was on UTC while it planned on the host's clock, so the car's ready time and the plan's days showed an hour or two off. Core now matches a copied zone file to the zone with the same rules, and says UTC only when the clock is UTC. + +## 0.157.3 + +### Patch Changes + +- a217d77: Stop a frozen site meter from driving the home battery. When the meter has no reading newer than the one the last battery order used, for example while its reads fail inside the watchdog window, the battery keeps its last order and may only move toward 0 W. Before, a held +2 kW import reading wound the battery to 10 kW discharge in ten ticks while the site exported 8 kW. +- 3ef88a3: A slow planner solve now finishes after a restart. The missing-plan retry no longer starts a new solve every 5 s while one is still running. Each new request cancelled the one before it, so on a slow box no plan landed. +- edaf041: Command the battery on every control tick at the default settings. The dispatch holdoff no longer skips about every other tick: it is off when `min_dispatch_interval_s` is not longer than `control_interval_s`, and otherwise allows 0.25 s of tick jitter. +- 7840c33: Stop asking a home battery to discharge when its charge is below 5 % or it reports that it cannot discharge. The slew limit used to ramp such a battery back toward the discharge it was already doing. If stopping it at once would put the site or one phase over the fuse or import limit, other batteries take that load first, and the blocked battery keeps only the discharge still needed. +- 7747b9a: Keep the battery above its minimum level when no plan exists, during fuse protection and under the hourly import limit. The floor is the highest of 5 %, the battery's `soc_min` and the planner's `soc_min` (10 % by default). Below it, the car is the lever for the fuse. The hourly import limit also no longer discharges below a battery-boost reserve. +- 1cb59ba: The home battery now protects each fuse phase in the direction its current flows. It no longer discharges into a phase that already exports past the fuse, and no longer cuts a discharge that keeps an importing phase under it. It counts its own share of each phase, so it holds a steady relief instead of switching between full relief and none on every dispatch, also when its own power reading lags the meter. When one phase imports past the line and another exports past it, it takes the move that leaves the worse phase least over instead of holding its last command. When the meter does not show its moves, because the meter repeats one reading or the battery sits on a phase other than the one over the line, it stops adding relief once it has moved one ampere per phase, instead of running out to its discharge limit. A battery on another phase then goes back to its power from before that relief, so it does not hold its own phase past the line. A load that cancels its move on a live meter does not stop it, nor do phase currents that a whole-ampere meter rounds to the same value while its total moves. It relieves to the fuse less `safety_margin_a`, the same line the car uses. +- ed66473: Keep home battery charging out of the room a slow charger's raise still needs. When a charger's driver declares its response time, as Easee Cloud declares 180 s, the battery now counts a car's offer as not yet in the meter for that time, as the car's fuse limit already did, instead of 20 s. A charger that takes two minutes to start no longer pushes a 25 A phase to 32.6 A. A charger that declares no response time keeps the 20 s, so the battery and a car on a local charger still charge together within a minute of the car's start. +- c51f64b: Watch the control loop from outside. When no control tick has finished for 10 seconds (or three intervals, if longer), Core sends every driver its default mode, so a battery leaves a forced setpoint. After 60 seconds Core exits and the service restarts it. An update now becomes current only after its first control tick, and a crash in dispatch math no longer keeps the control lock held. +- 10de589: Keep the control loop free of database reads. The PV curtailment check now + skips device lookups when nothing is curtailed, and device lookups read the + in-memory device list, so a busy state database can no longer stall a tick. +- 60303cc: Keep the control loop on time while the forecast saves its device identity. The save no longer holds the lock the control loop reads, so a busy database can no longer delay control by up to 10 seconds. After a failed save, wait 30 seconds before trying again instead of retrying every second. +- 1cb59ba: The car now charges up to the fuse rating less `fuse.safety_margin_a`, not a fixed 1 A below it. A charging car rises one step once that step fits under the line, instead of needing two steps of room and keeping one free; an idle car starts with all the room it has. Home battery charge beside a car uses the same line, with no 1 A floor. On a three-phase site with a 0.5 A margin, a car near the fuse gets 0.5 A to 1.5 A more on each phase. Cuts above the line still wait for the fuse's heat budget, and the hour's import limit keeps its own margins. A car never rises or starts into battery discharge that the fuse guard added. When a battery goes offline, its share of that discharge counts for 20 s, the time Core gives the battery's default mode to reach the meter, and then stops holding the car back, whether or not another battery is online. +- 61a0804: A device command that succeeds after the control tick stops waiting now counts as a success. Before, a driver that took longer than half the control interval to answer (1 s by default) was put back in its own default mode after every accepted command, so a slow battery or cloud charger switched between forced and autonomous mode on every tick. A started command now gets at least 5 s to finish; it still falls back to the default mode if it fails, runs out of that time or is cancelled. +- 573d24f: Stop a driver whose start-up runs past 30 seconds, so it can no longer block boot or a settings save. A saved fuse, export limit or battery power limit now reaches control before any driver restarts. +- 47b54d4: Retry a driver that fails to start, for example a USB P1 meter that is not + ready at boot. FTW tries again after 10 s and doubles the wait up to 5 min + until the driver starts, leaves the config or is removed. Before, the site + stayed without control until a restart or a settings save. +- c352465: Bundle sungrow 1.6.1. After the inverter drops off the network and comes back, the driver writes the release instead of reporting it held. A forced charge or discharge left over from before the outage now usually ends within about a minute of the return. Before, it ran for three to eight minutes, or did not end. The driver also keeps reading the battery's direction and PV, instead of dropping them for the session. +- 1cb59ba: When the home battery cuts its charge or discharge to protect the fuse, it now cuts only by the overage, so the grid settles on the limit instead of below it. It used to cut half of `safety_margin_a` more on top. Sites without a safety margin see no change. +- 7f10c78: Let a battery back into control as soon as it reads again after an outage. + A device that refused commands because it was unreachable, even for less + than a minute, no longer stays out of dispatch for 5 min. On its first good + reading FTW sends its default mode again and takes it back into dispatch, + so FTW's next command reaches it at once. If FTW has no new command for it, + the device keeps whatever it last ran until the default takes hold. On a + Sungrow the resent default does nothing yet, so an old forced command can + still run until FTW sends a new one; that needs a driver fix. A device that + reads fine and keeps refusing still waits 5 min between tries. +- 77b15c2: Send a driver's autonomous default on the device's first good poll after an + outage, when a default failed while the device was away. Before, the retry + waited for its backoff timer (up to 30 s), so the device could keep running + FTW's last forced battery command after it came back. FTW retries once per + return, not on every poll. +- e948e36: Treat a site meter that repeats one reading as stale. When FTW moves its own batteries and chargers by more than 500 W in sum, the devices' own readings follow, and the meter's total and phase values still stay the same for 30 s and 3 polls after that, FTW stops dispatch, returns devices to their own control and raises no EV, as for a silent meter. From the first such repeat, battery dispatch treats the reading as held, as it does for a reading with an old timestamp: it holds the battery instead of acting on the same error again. This also holds for a meter frozen since FTW started and for a car plugged in during the freeze. A quiet site is never marked stale, nor is a meter that sends a new reading only every 10 s and loses one, nor are moves that cancel out at the meter, and the first changed reading reopens dispatch. Before, such a meter counted as fresh because each poll brought a new timestamp, and in a rig run the battery swung from +5 kW to -5 kW on 40 s of repeated data. An EV order counts only once the charger reports the car drawing it, so at a site with no battery a frozen meter is found 30 s after that, not sooner. +- cf6a07d: Start Charge now on one phase once there is room for 6 A on that phase. An idle charger set to `1p` or `auto` on a three-phase site waited for room for the three-phase minimum (4,140 W) on a single phase, so it never started on a 16 A fuse and rarely on 25 A. It now waits for room for its own least order, 1,380 W. +- 8a7138d: A battery at or under its minimum level no longer discharges. Before, a battery at exactly 5.0 % (or exactly at `soc_min`) still counted as above the floor. Planner modes now keep `soc_min` with a fresh plan too, not only when the plan is missing. A discharge already running stops at the floor even when the grid sits inside the deadband; before, the battery kept its last setpoint and drained on. Plain `self_consumption` and manual holds still use Core's 5 % floor. +- 22d0bd3: After a restart, one high site reading no longer holds an idle car back for close to a minute. The fuse heat model used to start at the first reading's current, as if that overload had lasted. In seed 207539 one reading of 27.94 A on a 20 A fuse delayed the car's start by about 43 s. The model now starts at the first reading's current, at most the working limit. A sustained overload after a restart is still cut on the next reading, and readings above 1.2 × the fuse rating are still cut at once. +- 7a60827: An OCPP charger that a loadpoint names now starts at 0 A when it boots and reports no car. Until now it ran at its own default, usually its full rating, until FTW's first order landed, so a car plugged in during that gap could draw 16 A per phase on a fuse FTW had not yet shared out. FTW waits until the connector statuses after the boot have been quiet for one second, and any socket that reports a car, or meter values that name a transaction or show power, cancels the limit, so a charger that reconnects to a restarted Core with a car charging on any socket is not paused. FTW's first order replaces the 0 A limit. A charger that refuses it is logged and not asked again until its next boot. A charger listed only under `ocpp.chargers`, which FTW meters but never orders, is left alone. +- 1cb59ba: `fuse.safety_margin_a` now defaults to 0. On a site with no saved margin, FTW works to the fuse rating: the car, the home battery's charge and fuse relief, and the fuse's total import and export ceilings each move up 0.5 A per phase, to the rating. A margin is opt-in, for an inverter whose own phase limiter trips at the rating. FTW keeps any margin you have saved. If yours is exactly 0.5 A, FTW's old default, the log says so once at startup; set 0 to drop it. +- 09ebd42: Charge now works on a solar-only charger when a second solar-only charger is plugged in. Before, the idle charger kept the fuse share and Charge now sent 0 A with the reason fuse_limit. +- c0d1447: Charge now no longer ends on a single "not connected" reading. The cable must read out for 30 seconds, the same grace the car sessions use, before Charge now or Pause is cleared. A different session or charger within that time still ends it. +- c33e87d: Keep a one-off charging goal and its ready-by time when the car plugs in. A vehicle profile's `target_soc` now applies to a session only when the car has no goal of its own. +- 1fd653e: Stop a heavy load on one phase from marking a three-phase car as one-phase. FTW now marks a one-phase car only when its own order rises and one phase follows. A three-phase car no longer gets triple its planned order or swings between 6 A and 16 A. After a restart, a one-phase car is marked again at its next order rise. +- 930a102: A saved car goal no longer starts the car at full power when no plan is in force, such as just after a restart or after one failed replan. Without a plan the car now waits at 0 W until its latest start: the time the rest of its goal, and of every other car goal due by the same time, takes at full power, plus one hour. A loadpoint whose minimum charge power is 0 W now also gets this reserve. +- 0d54327: Keep OCPP charger commands flowing when a charger's boot or session message triggers slow work, such as a device registry write. That work now runs after the charger's lock is released, and a charging limit command always returns by its deadline, so one stalled write can no longer block fuse cuts to every charger and then the control loop. Each charger also holds at most one unanswered charging limit, so a stall no longer leaves a backlog of old limits to replay ahead of the current one. +- fd928c6: Start Core when history.db is damaged, empty or missing, including on sites that converted beta history. Core moves the old file aside as `history.db.corrupt-`, starts an empty history and shows the loss in the storage banner and on `/api/health`. Before, Core exited and restarted every 5 seconds, so no driver ran. +- 2be6210: Start again after state.db is damaged. When settings cannot be read because state.db is corrupt, FTW restores it from its last snapshot before it reads settings, instead of exiting on every start. When a restore brings back older settings, FTW runs them and reports the step back on /api/health instead of exiting. +- 3448637: Start FTW when one old history file cannot be read. FTW renames the file with an `.unreadable` suffix and keeps it, imports the other days, keeps that day's older copy for history views, and lists the file under `unreadable_cold_files` in the history health report. +- d35748e: Start the previous release once when the current release on disk is damaged and both read the same data. Before, Core failed every start until someone edited the install by hand. This lives in `ftw-launcher`, which `ftw update` does not replace: refresh it with the installer from this release, as described under "Launcher and command updates" in docs/native-beta.md. +- 8c66c4a: A control interval or watchdog timeout of 0 in Settings now means "use the default" (2 s and 60 s). Before, a save with an empty or zero control interval made the next start crash in a loop. Boxes that already stored a 0 start again with the default. +- 4e22b10: Save the update channel on the first boot that finds none. A box installed from a beta tag now stays on beta after it installs the stable that beta becomes, instead of moving to stable because its version string changed. +- 676085a: A box on the beta channel now updates to the highest release, even when a stable is promoted after newer betas have shipped. Before, it took whichever release GitHub listed first, so a late stable hid the newest beta until the next beta shipped. +- 168b78c: A Modbus device that comes back after an outage is reached within 10 s. Core used to wait up to 60 s between reconnect attempts, so an inverter could hold its last forced battery mode for a minute after it returned. +- f427f21: An OCPP 1.6 charger that keeps charging while Core restarts now has its session energy counted again. Before, Core forgot the running transaction, so session energy stayed at 0 until the car was unplugged, and a stop then reported the charger's lifetime meter total as the session. Core now takes up the transaction from the first meter value the charger sends for it and counts energy from that reading. Energy from before the restart is still lost. Transaction ids now start from the clock, so a restart no longer hands out an id that a charger may still hold. A stop that names another transaction than the running one no longer ends the running one, and a NaN or Inf meter reading is now ignored. +- 1cb59ba: The car comes before home battery charging, also when the battery does not cover the car. The battery now charges only from the solar surplus the car leaves. Before, with battery cover off, FTW took the car out of the meter reading, so the battery charged from solar the car was using while the site imported up to 10 kW. Battery cover off still means the battery never discharges into the car. A planned grid purchase keeps its charge. In manual peak shaving, a battery charge that was running when the car started now comes off while the site imports, instead of holding under the peak limit. +- 1cb59ba: The car's fuse cut now acts on what the site meter shows. A cut the meter already shows no longer counts as still on its way, so when the house adds load after it, FTW cuts again once the fuse's heat budget is spent instead of waiting up to three minutes. A house load that moves on one phase alone is not taken for the car's cut landing, so it no longer stacks a second cut on a slow charger. A raise that is on its way but no longer fits now comes off before it lands, when it would take a phase above 1.2 times the fuse rating, or over the line on a phase whose heat budget is spent. For a car the site meter has shown on three phases, this also holds when the house steps up on one phase alone. Until then, a rise on any phase counts as the raise landing, so a car on one phase is not cut while every phase is under the line. +- 1cb59ba: A battery charge rise now fits each phase before FTW sends it. The room is the fuse's working limit less the phase's reading and less FTW's own rises the meter does not show yet: the car's raises and the battery's earlier orders. The car's raise comes first. A battery order stays counted until the meter has had time to show the battery's own reading of it, so a battery that ramps slowly does not get a second rise on top. On a meter that reports phase currents without phase power, a phase counts as export only when the other phases cannot carry the exported total alone; any other phase counts as import. Near the line this keeps the battery from charging into a phase the meter shows full, and from stacking a rise on one the meter has not shown yet, which pushed a phase over the fuse and made the charge rise and fall around the limit. Holding a charge is never a rise, so this check never cuts a charge. A frozen meter is not covered here. +- 1cb59ba: After a restart, the home battery no longer stays in discharge for a car the meter already shows charging. Until the charger's first order of the new run can have landed, the battery counts the car's measured draw as part of the meter reading, never above FTW's latest order to the car and never above the consumption the site meter shows. FTW does not know the charger's offer in force then: one from before the restart, under an older max_charge_w, or the charger's own default. So the battery still keeps room for the car to rise from its draw to the larger of the charger's max_charge_w and the site fuse's rating. On a site whose fuse allows more than the car's draw plus FTW's latest order, that leaves no credit; a charger with no max_charge_w gets none either. The draw counts only when the sample reached FTW at least 1 s, and at most one control interval more, before the site reading did, and the charger's last poll succeeded. FTW judges a sample by when it arrived, not by the charger's own stamp, so a charger clock running behind FTW's, as OCPP chargers stamp their own samples, does not matter. A sample already there when FTW started does not count. The lowest sample since then counts, so a rise the meter does not show yet never does. A charger that samples every 30 s or more rarely gets the credit. Before, the 0 A orders FTW sent while the meter was still starting counted as the car's draw, so a lower order looked like a rise and the battery kept discharging toward its floor. +- 6ab2f94: Add `price.base_url`, for tests and development only. It points the elprisetjustnu price source at another server and turns off the Nord Pool fallback. The site rig uses it to pin prices. Real sites leave it unset and see no change. + +## 0.157.2 + +### Patch Changes + +- e293e2a: Give the car priority over home battery charging at the main fuse. Reduce battery charging before cutting an existing car offer, with a fixed response timeout and the rated fuse limit still enforced. Restart the car when its minimum fits the measured room, and let it reach its requested power while the battery uses the space left over. + +## 0.157.1 + +### Patch Changes + +- 34f425e: End manual battery Charge when FTW restarts. Return to automatic planning + with the saved battery export permission, or Idle when the planner is disabled. + Ignore old retained MQTT Charge commands so reconnecting cannot resume the + override. A new Charge command still works. +- 34f425e: Stop battery charging at the configured maximum charge level in every EMS mode, + including manual holds and self-tuning. Use each battery's soc_max, or the + planner's limit when the battery has none. Charging resumes one percentage + point below that limit; discharge remains available. + +## 0.157.0 + +### Minor Changes + +- c56fccf: Connecting an assistant no longer needs a terminal. When the box has no house password, Settings → Access asks you to choose one right above the assistant's code, and choosing it signs that browser in. Any browser that reaches the box may set the first one, also through a Docker port or on the box itself. A new box asks for it on the setup wizard's last step. Any password will do: a short one gets a warning, not a refusal. The Home Assistant app still needs none. + + The password, not the address, now proves the owner: a browser on the box itself can let an assistant in once it is signed in. Settings → System calls it the house password and explains it without developer words. If you forget it, or someone else set it first, run `ftw password reset` on the box (`docker compose exec ftw /app/ftw password reset` on Docker), then choose a new one on the box's page. On a native box whose `ftw` command predates this release, run `/opt/ftw/releases//ftw password reset` instead. + +### Patch Changes + +- e39e7b8: The app's history for solar, grid and battery now shows the measured power, the same as the box's own charts. Before, it came from the energy counters, which on a Sungrow move in 0.1 kWh steps, so each five minutes read 0 or 1.2 kW while the sun gave 0.4 kW. Home use is unchanged. Five minutes in which a flow went unmeasured for more than half the time is now a gap for that flow, not a lower value. +- e5edf34: Update the bundled Energyplan planner to 0.6.0. With an hourly import limit set, Energyplan keeps every clock hour under it, lets the battery cover the house so the car fits, and shows a car goal it cannot reach as short instead of promising it. Plans without a limit are unchanged. Core still validates every plan. +- 792ef29: A car whose own charger takes less than the charger's three-phase levels now charges toward its goal when no plan is available. On a step table with both one-phase and three-phase levels, the reserve FTW falls back to without a plan gave such a car a one-phase level, which no three-phase order has, and the charger was ordered 0 W. An example is a 3.7 kW plug-in hybrid on a charger whose table runs 1.4-3.7 kW on one phase and 4.1-11 kW on three. The reserve now holds to the same limits as the plan: such a car is ordered the lowest three-phase level, and once FTW sees it draw on one phase, three times the level it takes. +- 286ac14: The box's dashboard now uses the phone app's colours. The grid is one pink colour both ways, and the sign and the word say whether you buy or sell. The battery is cyan when it stores power and orange when it gives power, and the car is violet. Green now only means live or confirmed, and red only means refused, failed or blocked. Prices are shades of ink, not red and green. +- b0ce810: The plan now keeps each clock hour under the hourly import limit. The planner used to see only the fuse. With a 4.5 kW limit it could plan the car at 4.8 kW for four hours, import 5.5 kWh in an hour and show a charging goal as met that FTW could not deliver. Each plan now gets every clock hour in the box's local time and what the hour in progress has used, and FTW rejects a plan that imports more in an hour than it may. A planned hour keeps a little under the limit, because the car charges in whole amps and FTW raises it only with a step to spare; the plan then promises about what the car gets. When the limit puts a goal out of reach, the plan says how much is missing. The home battery covers the house so the car gets more of the hour. If the Energyplan worker's plan breaks an hour, FTW uses its own plan. Where FTW cannot plan the site itself, it keeps the worker's plan, says the plan may break the limit, and still holds each hour to it live. A new limit replans at once. + ## 0.156.0 ### Minor Changes - 5cf4a80: Statistics are one answer on the box, for two anonymous daily reports. With FTW Cloud they are part of Cloud. Without Cloud they are on until you turn them off under Settings → FTW app → Anonymous statistics, which shows both messages; nobody is asked any more. A No, a fleet report switched off, or Cloud turned off on an earlier release stays off. - + Beside the fleet report, the box can now send one energy report a day: the date and that UTC day's solar generation, grid import and export, battery charge and discharge and vehicle charge, in whole kWh, never in the same message as the fleet report and at its own time. It skips any day with a gap in the energy record and any site a partner controls. Once a release names an Oblivious HTTP relay, both reports go through it, so FTW Cloud never sees the box's address. This release names none yet: the energy report is not sent, a box that said yes sends the fleet report straight to FTW Cloud as before, and a box that shares only by default sends nothing until a relay is named. + - 2d7c8eb: The anonymous daily fleet report also says, coarsely, how the box ran: how slow its slowest plan of the last day was, how many plans fell back, how often its drivers failed to read, and whether its newest update of the last week failed. Each is a bucket of a few words, under the same switch as the rest of the report. Settings shows them in the message preview. - d68dedc: An Apple Watch can charge the car without its iPhone near. Its commands reach the box through FTW Cloud as short encrypted exchanges the box answers itself, and an owner's phone adds the watch by its key, with a passkey step-up. The box lists `cmd.exchange` while FTW Cloud carries them. @@ -21,8 +208,9 @@ - 21e3d16: FTW Cloud's protocol gains `x`, which carries a watch's exchange with its box through a one-shot request, so a watch can charge the car without its iPhone near. - 7b99353: Batteries now take the whole charge FTW asks for when another battery is down. On a site with inverter groups, the PV of an inverter whose battery is offline, faulted or cannot charge was still counted as charge for that battery, and that part of the target went to no battery. With 3.3 kW of PV on the down battery's inverter and a 9.6 kW charge target, the other two batteries got 6.3 kW in total; when the target was smaller than that PV, they got nothing and the surplus went to the grid. That PV now counts as power on the AC bus, so the other batteries share the whole target by capacity, within their limits. - 41f3908: A cloud driver can name the WebSocket hosts it streams from in its `DRIVER` table as `ws_hosts`, the way it names `http_hosts`. When the driver's config already grants HTTP, FTW grants the WebSocket for those hosts, so a saved driver needs no `capabilities.websocket` entry; hosts the config lists are kept. A driver without HTTP gets no network access this way. - + Opening a WebSocket now gives up after 5 s instead of 15 s, ends at once when the driver call is cancelled (a stand-down or a stop), and names the HTTP status when the server refuses the upgrade. The driver's poll and its commands wait for that dial. + - 0c8eaef: A driver's rotated sign-in token, a settings save and a vehicle wake reservation no longer fail at once with "database is locked (SQLITE_BUSY)" while another part of FTW writes to the database. They now wait up to five seconds for it, as other writes already did. A MyUplink token lost this way could stop the driver signing in after the next restart. - 3aad0d8: The Charge now slider in the EV charger dialog now stops at the most FTW can order. It took its top from the charger's maximum, so a 32 A one-phase charger whose levels stop at 3.7 kW read 6–32 A, "32 A · 7.4 kW" and "FTW received 32 A" while FTW charged at 16 A. When a charger has a list of levels, the top is now the highest level at or below its maximum: 16 A · 3.7 kW for that charger. - 0cb9bf2: Each charging order now tells the charger's driver how much current per phase FTW allows that charger right now, in a new `cap_amps_per_phase` field. It is the charger's share of the room below the fuse and the import limit, after the split between chargers, within the order's `max_amps_per_phase` and the charger's maximum. Until now a driver saw only the fuse rating. The Easee Cloud driver needs this to start a 6 A charge at 7 A only when 7 A fits, so a car on a tight phase is not stopped and restarted. The field adds no new limit, and FTW sends the same orders as before. @@ -82,7 +270,7 @@ - 7ad6948: Core now removes what a stopped full backup left behind before it accepts a new one. A backup cut short when Core stops, for example by an update, left its raw copy in /dev/shm until the next reboot. On a Raspberry Pi that held more than a gigabyte of RAM and sent the next backup's copy to the SD card, where it ran past its two-hour limit. At start Core removes such raw copies, the stage and verification folders and any unfinished archive in the backups folder, and logs each path and its size. It leaves them when another backup is running, and it removes nothing else. - aa8bc17: Report the longest step and step count beside each total phase time when a history write or forecast read fails. Archive failures also report connection, preparation, locks, write and commit times before cleanup. This separates one slow operation from many short operations without logging sample values or changing storage behavior. - 18f75ac: Home use on the energy flow no longer shows "≈" when every device is read, and "% self-powered now" is back. The page marked Home use as an estimate whenever /api/status carried a second, energy-averaged value, which it always did. It now follows Core's `load_quality` alone, and /api/status no longer sends `load_estimate`. - + Home use has one value. 0.154.3 was never published as a beta; its change also ships here: Home use is the site meter minus every device that is read, shown when one device is offline (marked "≈"), and blank only when the site meter is missing or stale. The Home Assistant sensor "Load Power (energy average)" announced for 0.154.3 is removed before release; "Load Power" stays. ## 0.154.3 @@ -90,7 +278,7 @@ ### Patch Changes - 01ab443: Show Home use again when one device is offline. Live Home use is the energy balance as is: the site meter minus every device that is read, as before 0.141. A device that is offline, or a setup problem, no longer blanks it; the dashboard marks it "estimated" with "≈". A negative value is shown, since it points at production FTW cannot see. After sunset at the site's map pin an offline inverter counts as 0 W. Home use is blank only when the site meter is missing or stale. `/api/status` adds `load_quality` and `load_reason` as hints. - + Home use now has two series. The live display value is logged as the `load_display_w` metric. The energy-based Home use from closed history intervals, with its quality flags, stays what load models train on. Home Assistant gets a second sensor, "Load Power (energy average)", next to "Load Power". ## 0.154.2 @@ -106,8 +294,9 @@ - 4d4ce53: Keep the day's Home use when a device drops out or Core restarts. An unknown PV, battery or EV source holds its last known value, and the meter or a restart holds for up to two minutes, marked as estimated. A day's total is unknown only when more than a tenth of it is missing. Days cached under the old rule are recomputed once. - 9bd114e: Keep a rejected held value unknown until its source reports again. A charger that finished while silent no longer comes back as a phantom charge when the house later uses more power. - 85fa9e1: Train both load models on the closed quarter-hour household energy that daily Home use is built from. The built-in model no longer learns from one instant reading per minute, and the Energyplan forecast no longer keeps its own load collector, so forecasts learn the Home use shown in History. - + Keep cold-day counters in their stored minute units when training moves to quarter-hours, preserving learned profiles and partial-day evidence. + - f805c6d: Remove the timing note under the Overview energy flow. Home use still shows current power, and the diagram still marks missing or paused data. - bbaf0b3: Keep load-model training strict on hybrid inverters whose PV reading is optional. While such a PV source has not reported, Home use still shows the house, but the interval does not train the forecast; after dark the PV counts as 0 W and the night trains as before. - 266b292: Keep the day's Home use when a PV inverter goes silent at night. While the sun is below the horizon at the site's map pin, an unknown PV source counts as 0 W: past civil dusk as a measurement, so the night also trains the load models, and in twilight as a marked estimate. Without a pin, or by day, the gap stays unknown as before. @@ -134,11 +323,13 @@ ### Minor Changes - 8e604db: Add a complete configuration schema and durable setup drafts. Draft edits preserve unedited settings and credentials; validation and a confirmed commit use the current Settings revision and save path. This prepares setup through MCP; first-run pairing, MCP writes and code sharing remain separate work. - + Validation returns normalized settings as a new draft revision for review. The final save checks those exact portable settings before writing to SQLite, including credential bindings. Drafts, Settings saves and reloads preserve exact integer driver identifiers; numbers that would change are rejected with a field path. + - 30ce5ad: Owners can grant an assistant setup access as a separate choice. MCP reads the box's complete setup schema, resumes a draft, edits one field or list entry, validates and saves the exact draft the user confirms. The box retains its Settings checks, reports pending restarts and preserves a save receipt. A separate confirmed call can restart to apply the exact saved setup, with a live grant and revision check before the restart. Saving does not prove physical measurement or control. - + Every draft write and final Settings transaction rechecks the live grant and request deadline after preparation. Revoked access and ended requests leave both settings and commit receipts unchanged. + - 499d5de: Share MCP access with a single-use code. Choose rights and expiry, let the recipient connect with its own key, and cancel invitations or remove access from the box page or app. ### Patch Changes @@ -157,8 +348,9 @@ ### Minor Changes - ccdc740: The box's own page shows the owner's cars. The overview has a card for each car, plugged in or not, with its level, where the level came from and how old it is ("72% · From the car · 1 h ago", or "≈" once it is an estimate), its charge limit, its goal and the charger it is on. When a charger is in use and the box cannot tell which car it is, the charger's notice asks "Which car is on garage?" with one button per car and "None of these"; a car the box named itself can be corrected the same way. Settings has a Cars tab that lists each car with its VIN, its level sources and the tags a charger knows it by, and lets the owner rename a car, set its battery size, add one and remove one. - + New routes carry the car ops for the page only: `POST /api/cars`, `PATCH` and `DELETE /api/cars/{id}`, and `POST /api/loadpoints/{id}/car`, the tap. A phone keeps sending the ops. `GET /api/cars` now leaves out each car's VIN, charge tags and sources for a request that has not proved itself as a protected one must, such as one through a public name without the API token, and says so with `redacted: true`. + - cec3408: Open Battery history on a car to see its saved sensor levels for 24 hours, 7 days or 30 days. Keep reported levels separate from charging estimates and show gaps and failed reads. - 14f6c89: Give authorized MCP sessions the full diagnostic read API, including the energy ledger, raw series, logs, configuration, driver source and support exports. Keep technical names and addresses, list the connected box's routes, and page large answers without a total size limit. Stop reads when access ends. - bb6d649: Keep house-password browser sessions for 30 days across Core updates and restarts. Save only token digests in the database, and keep logout, expiry and password changes effective after a restart. @@ -169,10 +361,11 @@ - 513fc60: Clear car-choice feedback when the plug-in changes, and ignore replies for an earlier plug-in. - ecf51ef: The Access page states that technical diagnostics include source, logs and full backups, which can contain saved credentials, before the owner grants access. - 8a45e06: Show Core's current power balance as Home use in Power now. Show the energy mean separately with its own period and age, so a missing or old mean cannot hide current load or replace it with a past zero. Keep missing or invalid current load unknown. - + Use measured AC power for the live site balance when a driver such as Pixii reports both battery DC and site-side AC power. Do not substitute DC when the AC reading is unavailable. - + Let a confirmed disconnected charger contribute a separate zero-load estimate between polls, even when it retains old power and session timestamps. Break that interval on reconnect, missing evidence, changed state or a gap. + - 0922fcc: Keep live household Load visible when current meters report at different times. Use the same Core power balance and each source's freshness; keep source-alignment requirements for history and forecasts. - a8b6d64: Keep URL-encoded driver names intact in MCP diagnostic reads and allow large exports five minutes to finish. - 4ade7f4: Connect full MCP diagnostics to the running box API. The startup wrapper now @@ -261,8 +454,9 @@ ### Minor Changes - e3c0bac: Settings has an Access tab, where an owner with no phone app lets an AI assistant such as Claude or ChatGPT into the home. The owner types the code from FTW Cloud's page, sees what the box read in the signed offer, and grants it or less, never more. Reads are the default; control is something the owner turns on, with caps no higher than the assistant asked for. Granting, renewing, narrowing and approving a request need the house password, and the page says so before it asks for a code. The tab lists who has access, with expiry and last use, and lets the owner renew, narrow or remove each one and answer waiting requests. - + The box now says why it will not take an offer: unsigned, expired, for another home, unreadable, or a build that pins no offer key. A box with only an assistant, and no phone, now links to FTW Cloud, which learns of the assistant from the box's roster. + - f17cb05: Include the Cloud home id in a linked box's pairing QR so the web app can pair through FTW Cloud. ### Patch Changes @@ -334,8 +528,9 @@ - da2aa98: Discard missing forecast measurement intervals when the system clock changes after startup. Use wall time consistently so a clock correction cannot create false high household-load observations. - 2b0cf6f: Accept idle solver rounding within 0.00001 W when checking EV charging during battery export. Keep rejecting real discharge and include exact powers in rejection diagnostics. - 022eb41: Keep a safe plan when a later forecast cannot meet the site limits. The reserve planner prepares the battery for future peaks and reports why it shortened a plan, while Core still checks every planned action. - + Retain valid intervals before corrupt future forecast values, use the reserve when the DP grid exceeds its memory budget, and require fresh battery charge readings before planning from live telemetry. + - d7cb09d: Keep complete planner worker replies readable when the worker exits before Core reads them. Retain bounded cancellation and worker restart behavior. - 5636993: Include the latest failed planner inputs, worker request and response, and current charging state in support packages. Retain the latest failure across restarts with a durable SQLite commit and report storage errors. Keep failure evidence distinct from accepted plans and redact credentials before export. @@ -360,7 +555,7 @@ - 4d9018a: Bundle sungrow 1.6.0, which reads grid, PV and battery power, SoC and per-phase values once a second, so live figures on the phone move every second instead of every five. Counters, temperature, frequency and fault bits keep the five-second rate. - 5bc76c9: Honour the interval a driver returns from `driver_poll`, as the driver contract says. Core ignored it before and polled every five seconds. The returned wait now sets the next poll, between one second and thirty seconds. A driver that calls `host.set_poll_interval` keeps that interval, as before, whatever it returns. - + Bundled drivers that return one second now read once a second: ambibox_v2x, ferroamp, ferroamp_dc2_v2x, pixii_pv, sdm630, sonnen and zap. myuplink reads every thirty seconds instead of every five. easee_cloud, tesla_wall_connector and zaptec_cloud wait ten seconds after a failed read, and solaredge_legacy thirty seconds after a failed SunSpec probe. ## 0.142.1 @@ -507,7 +702,7 @@ - 4000391: A planned EV charge now gets its share of the fuse on the worst phase, not only in the site total. When house load sits mostly on one phase, the battery charges less so the car is not cut by its charger's load balancer, and the next hour of the plan leaves the same headroom. - e0a57fc: Show an EV departure miss only when it is more than half of one minimum charging run, about 155 Wh for a 4.14 kW charger. Energyplan treats a smaller need as met instead of starting the charger for it. A car that is charging when the plan starts still counts a miss above 1 Wh. - 6526638: Keep one Modbus writer per device. The proxy defaults to read only. When external writes are enabled, FTW keeps reading but stops all device writes until the owner changes the setting and restarts Core. Restrict proxy requests to configured unit IDs, give driver traffic priority, and close idle clients on shutdown. - + Rename the Home Assistant settings tab to Integrations. Home Assistant MQTT and the Modbus proxy each keep their own section. ## 0.140.1 @@ -576,10 +771,11 @@ ### Minor Changes - 8386721: Show whether each device FTW controls does what FTW asked. A tap on the battery, charger or solar bubble answers "Are we in control?" first: Following FTW, Waiting, Limited, Not following, No contact or Not controlled, with one sentence and, when you can act, the next step. Manual override moves below it. - + "How FTW knows" lists the evidence: sent, accepted, measured and confirmed by a separate grid meter, with numbers and response curves for experts. Support reports and the API carry the same status and evidence. - + The overview stays quiet while FTW is in control and marks only devices that need a look (amber) or attention now (red). Proof follows a device through targets retuned every tick and slow cloud sources, a confirmed step survives later household loads, and a battery that takes less power as it fills is explained instead of flagged. + - bf85123: The Plan card's forecast slider is now five planning styles, from Very careful to Very bold. A line under them says how much of the spare sun the plan counts on, and another says whether extra sun goes into the battery or to the grid. The default style, Balanced, holds back a smaller forecast margin than before (k 0.3 instead of 1), and a box that still runs the old default moves to it once. Settings → Planner fine-tunes the margin and saves at once. Changing the style or battery sales on one device no longer undoes a change made on another. ### Patch Changes @@ -601,8 +797,9 @@ - 8bb3d81: Fix forecast evaluation stalling after restart. Give score writes and retention separate time budgets, skip retention sorting when the archive is within its limits, and resume after the last committed page when a step fails. Refresh calibration from saved scores even when later work needs a retry. - 46ae403: Installer messages now direct users to the new 0.x setup paths and explain that 2.x and 3.x receive no more updates. The scripts still exit without changing the site. Docker's missing-version message asks for an exact published new 0.x tag instead of suggesting an old example. - + Block retired Docker release workflow dispatches before checkout or registry writes, while keeping native Changesets version PRs running. + - f46deb1: Display cached vehicle SoC after restart with its source age, without using it for control. - 9c26d0e: Make electricity setup clearer: choose three-phase or single-phase, enter the confirmed main fuse rating, and keep voltage under Advanced. Standard connections use 230 V per phase. Show saved custom connections without changing their values. Make the country and price-zone list available during a fresh install. - 3cdeb81: Keep private FTW data out of the release Docker build context. Install commands stop on download errors and refuse to overwrite an earlier Docker test. The English and Swedish guides put the physical SD-card swap before installation and treat Docker as a separate path. @@ -948,10 +1145,11 @@ as the pin drags, so a site outside Europe learns up front that price-driven planning has no source there instead of getting an empty price curve with no explanation. - + The registry's European price-country list is held in lockstep with `prices/zones.go` by a test, so a bidding zone added there cannot silently go missing from the coverage answer. + - 6af12e8: Let Cores before v3.6.0-beta.1 update without the full history copy that could not finish on a Raspberry Pi. Release notes now carry a fixed legacy state-schema marker for those Cores and a second marker with the real schema, which newer Cores read to refuse downgrades. The stable release guard checks both. - 191dd88: Remove replaced Core, updater and optimizer images after a verified Core update, keeping every image a container uses and the rollback image. Beta boxes stop filling their disk with one image per update. - 646a52e: The Settings location picker moves from Leaflet to MapLibre GL JS 6.9.0, @@ -961,7 +1159,7 @@ policy as `/vendor/three` and `/vendor/ace`. Leaflet's now-unused copy is removed. If WebGL is unavailable the numeric latitude/longitude fields stay authoritative, exactly as before. - + Static assets are also served with pinned Content-Types instead of whatever the host OS's MIME table says: on a Windows host whose registry maps `.mjs` to text/plain, the browser (correctly, under `nosniff`) refuses the vendored @@ -984,7 +1182,7 @@ - 792cd21: Show simple forecast status and recorded learning days in More. Put model diagnostics and relearning behind Details, remove misleading training-quality bars, and require an explicit health check before showing Healthy. - 19ab0ea: Keep live storage responsive during archive and history maintenance. Bound write transactions, retain completed forecast observations for retry, and expose current storage and forecast health with failure history for diagnostics. - 2390e61: Let users choose the car's charge limit in their charging goal while keeping existing percent targets unchanged. Show pending saves separately from saved state and write failures. - + Show the current battery level and car limit with their sources, add readable charging windows, and include all cars in the household plan and overview. ## 3.5.3 @@ -1000,8 +1198,9 @@ - a3e3921: Build the offline converter's sample time index after copying and verifying the rows to reduce SD-card writes. Keep the primary key throughout the copy and require the time index before publishing the converted history. - 4675122: Stop history admission and background work before draining the accepted queue. Give the whole queue a separate shutdown budget, finish deferred cleanup on restart, and report an incomplete drain through a failed process exit. Allow 60 seconds for container shutdown during updates and restarts. - 6e9cb45: Copy numeric DuckDB history in larger bounded batches to reduce migration time on Raspberry Pi storage. Keep full readback checks and resume interrupted conversions from the saved row cursor. - + Keep state read-only until the verified history is selected, so an interrupted converter cannot change its own source through a SQLite checkpoint. Treat absent and empty SQLite WAL files alike while still checking every nonempty WAL. + - 84778bc: Keep forecast and model IDs valid when a software update changes gzip encoding. Compare bounded, verified record contents and retain the original archive bytes. Changed or corrupt records still fail validation. - ed17cf6: Read hourly history summaries before taking SQLite's write lock. Bound each read and write so a slow backfill cannot monopolize live telemetry commits; incomplete backfills keep using raw history. - 51fbcf7: Build hourly history in small batches that follow the primary index. Save progress with each committed batch, resume after timeouts and restarts, and expose unfinished work separately from raw-history migration. Keep live writes and late samples ahead of background work. @@ -1011,14 +1210,15 @@ ### Patch Changes - 18c8022: Keep charger measurement times when estimating EV energy. Use fresh power between delayed session-counter updates, reconcile overlapping energy once, and retain the estimate through a verified session restart. Missing or older counters no longer reset a confirmed battery level. Expose the estimate source and measurement ages. - + Match Easee pauses to the current vendor session even when sessionEnd is populated. Bound power estimates to its reporting cadence. Replan when a restored EV level differs from the active plan. - + Show when a stopped charge has reached its target, and distinguish an estimated battery level from one reported by the car. - + Pause dispatch when charger power is unavailable and retain spent pulse energy across recovery. Compare EV progress with the allowed duty curve. Bound progress checkpoints to 30 seconds or 30 Wh, with immediate saves for stops and user corrections. + - 76bf6c4: Return history storage to SQLite and Parquet. Store goals and charging state in a separate database, preserve older summaries, verify archives before pruning, and bound history reads and backup writes. DuckDB beta installations use a separate offline converter that keeps their original history files. Block image-only rollback to an older history format. Send known EV safety stops before session persistence. - + Resume the first SQLite history binding after an interrupted save, while rejecting unrelated history files. Read retained Parquet pages before marking a full backup verified, including files whose hashes match an already damaged source. ## 3.5.0 @@ -1026,8 +1226,9 @@ ### Minor Changes - fb6b118: Adopt AGPLv3 with a narrow Energyplan combination permission and expose source access. Commercial use of the AGPL-covered code remains allowed under its source-sharing terms. New Energyplan workers allow unmodified use with FTW only for a person's own private household. Free, noncommercial redistribution with FTW is allowed only for that household use, with the binary license and third-party notices. Other use or distribution requires a separate written Sourceful license, including commercial use, resale, paid installation/support and hosted or managed operation for others. Earlier grants remain unchanged. - + Commercial installations must arrange Energyplan rights or omit the restricted worker before upgrading, including through automatic updates. + - e5f281f: Optional Modbus TCP proxy. Other LAN integrations can talk to the device's one shared Modbus TCP session through FTW. Off by default; writes stay blocked unless you opt in. ### Patch Changes @@ -1050,12 +1251,14 @@ hidden, and stop Settings EV/System timers when the modal closes. Returning to the tab refreshes once, then resumes a single timer. - 79d180b: Lua driver host: a missing `driver_command` is an error, a battery/PV/EV/V2X/heat-pump driver that can be commanded must implement `driver_default_mode`, and fingerprint probes cannot write hardware. A read-only declaration blocks dispatch before the command hook runs. The catalog now reads `auth_post_path`. - + Update the recovery bundle to the companion driver audit, including corrected telemetry freshness, read-only declarations, and the Easee safe default. Include the read-only Zaptec Cloud and Tesla Wall Connector drivers so their setup paths also resolve offline. + - e5f281f: Fresh site-meter data now makes Core trim curtail-capable solar when solar alone pushes export above the configured site limit. Core reports any overage it cannot remove without an unsafe zero-watt cap or control of unsupported solar. - f2874ff: Preserve valid forecast learning across the beta.3 hash-policy upgrade and keep known driver battery limits when both battery overrides are zero. Use fresh, complete telemetry for the current household-load estimate. - + Bundle Energyplan 0.4.5. EV goals share site capacity across deadlines, retain safe partial plans when goals cannot be met, and reserve the charger's real pulse power. Report missing charge and executable pulse cost without a false optimality claim. + - bd7d76b: Add `sim-ocpp`, an OCPP 1.6J / 2.0.1 charge-point simulator covering the recorded Evify charger range. Tesla Wall Connector is listed but skipped: it has no OCPP. - 42da9f2: Add an updater-then-Core upgrade script so a 2.x Compose site can move to a published 3.x pair without the orange Update hop. @@ -1132,8 +1335,9 @@ the locator YAML. An edited leftover seed or a wizard document cannot replace Settings already stored in SQLite; only an old-Core rollback save is imported. - 83412d1: Release retained DuckDB buffers after a failed telemetry commit runs out of memory, then retry the same queued tick. This lets live collection recover before the normal maintenance threshold and lets background history import resume. - + Increase the primary DuckDB memory budget to 256 MB so a full telemetry tick can reconcile the observed startup gap while history queries run. + - 3e2198c: Accept `FTW_API_TOKEN` on the LAN as the API token. With the house password on, a matching Bearer is no longer hashed as a password guess, and a mismatch no longer locks Settings for the household. - 234cb3a: Bound Lua driver safety so a stuck poll, a huge watchdog override, or a redirected POST cannot leave hardware on its last setpoint. Every driver VM @@ -1161,9 +1365,9 @@ - 715ec0f: Show startup and background history import progress, keep incomplete history visible, and resume the update view when another client starts the work. - 715ec0f: Start live collection after importing the catalog and energy accounting, then move large sample archives into DuckDB in the background. Resume verified progress after an interruption, preserve live writes, and report incomplete history until every source is checked. Release native database buffers only between active SQL connections. - + Give Core startup more time and leave its image and data in place if readiness fails. Preserve the previous image ID before replacement. Check the updater before opening data, and stop with a clear message when an older installation needs to update its updater first. - + Keep catalog ID allocation safe after an abrupt process exit, and defer raw-history retention until import finishes. ## 3.2.0 @@ -1171,7 +1375,7 @@ ### Minor Changes - a62b082: Use embedded DuckDB for all time-series reads and writes, including the energy ledger. Keep SQLite for configuration and learned state, and retire the FTWDB shadow process. - + Core verifies the import of existing SQLite and Parquet history before starting control. Health separates queued ticks from durable commits. State schema 3 requires a full backup; returning to an older Core requires a verified full restore with the matching version. ### Patch Changes @@ -1243,8 +1447,9 @@ ### Patch Changes - 48013fd: Restore and save rotated OAuth tokens before drivers start so myUplink can stay connected across Core restarts and updates. - + Apply signed OAuth rules to managed drivers, including official beta installs. Allow token exchange only at the declared path, block redirects, and let each driver save only its declared secret keys with bounded keys and values. + - d6da557: Close the FTWDB shadow connection after each durable batch so the next batch does not reuse a socket closed by the sidecar's idle timeout. This avoids repeated transport errors and delayed shadow copies during normal beta collection. - 76f811b: Keep unchanged form defaults out of saved settings. Opening Planner and saving now preserves an unset SoC limit, including when saving from another tab. @@ -1268,14 +1473,14 @@ ### Major Changes - faad896: BREAKING CHANGE: Remove the built-in CalDAV server, its API endpoints, calendar settings and calendar-driven charging and away events. - + Use loadpoint targets and ready-by schedules for future charging. Existing goals, calendar data, learned models and forecast archives remain in place. Older config files still load and warn when calendar support was enabled. ### Minor Changes - 92ff96c: Add an optional FTWDB beta sidecar that copies committed live site history through a bounded memory queue. Keep SQLite and Parquet authoritative and report candidate gaps, errors and durable acknowledgements in health. - ab1431f: Store settings and credentials together in SQLite, with durable commits before applying changes. Import YAML once and retain it as a database locator and recovery export. Remove background YAML reloads. Reject stale Settings forms and preserve the previous live settings on a failed write. Capture current settings in backups and keep forecast learning state unchanged. - + Mark the migration as state schema 2 so upgrades take a full backup. Returning to a YAML-only Core requires a matching backup restore. ### Patch Changes @@ -1293,7 +1498,7 @@ ### Minor Changes - 74b5cb0: Correct weather interval timing, panel direction and forecast energy. Train household and PV models only on fresh, complete measurements. Keep grid limits separate from household demand and let PV learn its scale without a battery-based guess. - + Use the local Energyplan PV and load models as the planner's first forecast source, with no required panel geometry. Keep the previous forecast as a shadow and use it when the new model lacks a valid prediction or its worker fails. Save issued forecasts, per-signal sources and model state for causal comparison, report uncertainty by horizon, and add a read-only forecast evaluation command. ### Patch Changes @@ -1342,7 +1547,7 @@ same downside PV input. Core DP remains the validated fallback, with a visible reason when it takes over. The worker and its license ship and update with Core; source stays private. Explicit core and python settings keep their roles. - + Reject EV plans above the battery limit and clip DP power at the operating band so fallback energy matches the power it schedules. @@ -1359,10 +1564,11 @@ - 5801b4d: Keep unchanged drivers running when settings pass through the web API. Compare numeric values from JSON and YAML equally, including nested driver settings. Bound charger settings requests so a missing reply leads to a visible retry state. - 8241ca1: Use Easee session evidence to retain a confirmed battery level across a restart only when the same active session is verified. - 9bbc69d: Keep a confirmed EV battery level across restart only when fresh charger telemetry identifies the same hardware and charging session. Show when the level cannot be retained or the disk write failed. Changing battery capacity preserves the current level and its confidence. - + Treat a car declining current as a separate charging status. It no longer changes the estimated battery level to the target or sends a completed notification. Completion needs a fresh matched vehicle battery reading. - + A higher goal, an explicit retry or measured charging lets the planner resume after a prior refusal. + - 2e01f7f: Let the charging view save the usual car’s usable battery size without replacing other settings. Apply a saved size through the shared config path and keep the current charge level steady. A failed save leaves the previous size in use. - 008ba8c: Explain when the charger’s own current limit reduces a request, without blaming the main fuse. - 5e01da8: Add Pause charging and Resume plan beside Charge now, with status that waits for the charger to stop. Let users change battery size beside the current level. Explain whether that level survives a restart, and never call a car that declined charge full. Keep the request deadline active until the response body arrives. @@ -1372,17 +1578,19 @@ - 87a7e80: Show when the current car uses a different battery size from the usual size just saved. - a4979e5: Stop dispatching an old plan when its replacement fails, so a removed charging goal cannot keep charging the car. Keep the current plan during normal recalculation, require a successful new plan after failure, and preserve manual Start and Pause with the usual safety limits. - aea554e: Add a charger without inventing an ID or pressing Save again. Charger settings apply on change, with errors and a retry beside the form. OCPP setup stays separate from cloud chargers. - + Charging feedback separates the FTW request, the charger's reported limit and measured power. Old charger readings cannot claim current charging. Manual current changes apply on release; Return to plan names the action that ends a manual hold. Charge level and schedule writes run in order, and failed requests stay visible. - + Keep the charging goal and solar rule together, with no mode tabs. Show when Charge now overrides them. Show the current slider only while manual charging is active. Opening goal settings does not send a command. + - 7e6dc2e: Guide the first charger connection from Chargers into the charger catalog and back after saving the connection. Hide the unrelated global Save button for charger autosave; keep explicit saves beside OCPP and shared-car settings. - 7a31411: Keep the battery level entered while waiting when the same charger first verifies its session at charging start. Count newly delivered energy from that level and save it once the session is verified. - 8f3b6b5: Keep EV power below the fuse budget when the budget falls between charging steps or below the minimum. - cca809c: Save charging goals before applying them. If storage fails, keep the previous goal and return a clear error for both goal edits and removals. A successful retry applies and saves the new goal together. - 27bf915: Bind a saved manual charging request to its charger hardware and verified charging session. Keep a saved pause on the same charger. When a prior positive request cannot be verified, pause and ask the owner to confirm instead of resuming automatic charging. Preserve explicit Start or Clear actions that arrive before the first charger reading. - + A running request also stops if the charger, session or loadpoint binding changes. A clear issued before telemetry survives another immediate restart, and concurrent Set/Clear writes preserve the order shown by the controller. + - 7085724: Show when a charging choice applies now but could not be saved for restart. Retry its save when fresh charger data arrives and clear the message only after storage confirms it. - c0c5bfb: Preserve manual charging while the charger driver starts or has no reading. An absent reading no longer counts as an unplug. A confirmed unplug still ends the manual session. - cf41766: Ask OCPP chargers for fresh hardware identity after reconnecting, with bounded retries for missing replies and a safe fallback when the request is unsupported. Pause an older manual Start when hardware identity is lost, keep an explicit Pause, and allow a new Start to bind to the next verified identity. Keep cable status unknown during a network interruption, and preserve the reconnect boundary even when it falls between control ticks. @@ -1398,15 +1606,16 @@ - 08e117d: Keep an explicit charging pause across restart even before the charger reports hardware identity. A prior Start without matching session proof restores only a pause that needs confirmation. Use the current OCPP connection's boot identity, and retry failed saves when charger data returns. - 7a7024b: Pause a prior manual request when a session counter resets, including when the previous session had no verified ID. Keep an explicit Start when the same uninterrupted session gains its first verified ID. - cd07ddc: EV modal, Manual tab: after Charge now the line under the button follows the charger instead of repeating the request. It says that the amps were sent and the box is waiting for the charger to confirm, that the charger has taken the limit and the car has not started drawing, that the car is charging, that the charger offers the current but the car is not drawing it (with the charger's own reason, such as "EV not accepting current"), that the command stalled, or that the main fuse limits the charge right now — each with the time elapsed. The plan strip above the tabs says the same while a manual charge runs, so the charger's reason is no longer hidden behind the manual sentence. A refused Start (403, 404, 409) now reads as a failure with the server's reason instead of "Charging at 16 A". - + `GET /api/loadpoints` carries this as `manual` per loadpoint: `state` (`sent`, `accepted`, `charging`, `not_drawing`, `stalled`, `limited`), `started_at_ms`, `since_ms`, requested and commanded watts and amps, the charger's reported limit and reason. `POST …/manual_hold` answers with `started_at_ms`, and an Update of the amps keeps the first press as the start. `commanded_since_ms` says when the box's current order was first given. + - 2391dd1: Full backups now include managed drivers whose active links use absolute paths inside the data directory. The archive stores relative links so restore works at a new path. Links that escape the data directory or form cycles remain blocked. - 3b66faf: Keep the selected optimizer image after updates and rollbacks by saving and checking its Compose pin. Report a failed pin write instead of a successful update, preserve other host settings and file permissions, and keep shell payloads containing credentials out of updater logs. - 2391dd1: Restart restarts the existing container and keeps its exact image, including local test builds. It never pulls or recreates from a stale Compose tag. Core refuses the unsafe restart path on older updaters and explains how to update the updater. - 7806f79: Stop savings database reads when an app request times out or is canceled, so the request releases its place for later app reads. Keep canceled calculations out of the daily savings cache. - 99371b8: Keep driver paths consistent when settings are saved. Adding a charger no longer restarts other drivers with paths that fail to load. - ab8f5ac: A surplus-only EV can take leftover PV while the home battery buys from the grid. Surplus-only is an EV policy, not a site-wide import ban: the car still cannot import, and the home battery still cannot feed the car. - + Core DP includes this change. Sites using the optional Python/HiGHS planner need an updated optimizer to produce the same allocation. Core DP does not require a new optimizer image. ## 2.14.0 @@ -1415,8 +1624,9 @@ - 50a0992: EV modal: one way to let the home battery help the car. "Boost from home battery" now lives in the EV modal, below the charging tabs: pick how much the home battery keeps in reserve and for how long, start it, and see why it stopped when the box ends it. The "Legacy site-wide battery cover" toggle is gone from the modal; a site that still has that older setting on sees one line about it there with a button to turn it off. The Devices card no longer carries a second copy of the boost form, only a line about an active boost. - 173e32e: EV modal: the plan is visible at plug-in. Above the tabs the modal now draws the planned charge windows for the car on a 24 h track, with the energy in them, under the usual one-line status. Right below sits the car's current charge, which the plan is built from: drag it to the real value and let go, and the box replans and redraws the plan. The "Set current charge" button and the SoC editor in the Scheduled tab are gone. `GET /api/loadpoints` carries the windows as `plan_windows`, and `POST /api/loadpoints/{id}/soc` replans before it answers. - + Two fixes underneath, both seen on a real site: setting the car's charge level now clears the "session complete" latch instead of snapping back to the target on the next tick, and ten minutes of steady charging after that latch releases it, so the estimate follows the energy going in instead of sitting at the target while kilowatt-hours flow. `soc_source` reports `completed` when the latch is what pinned the value. + - 43e6723: EV modal, Scheduled tab: the schedule saves as you change it. Move the target, pick a time, tick Repeat daily or choose weekdays, and the box saves, replans and redraws the plan above; the status line says so. The "Set schedule" button is gone; "Remove schedule" is the one button left. Weekday chips are new on the box (the wire and the phone client already had them). ### Patch Changes @@ -1469,7 +1679,7 @@ records the terminal-corrected cost difference — the field evidence for its scheduled removal. Set planner.engine: python to keep the old arrangement during the transition. - + A battery that has drifted outside soc_min…soc_max no longer stops Core from planning. The planner starts from the nearest band edge, warns with the real reading and the difference in Wh, and records the @@ -1481,7 +1691,7 @@ ### Patch Changes - aa8bf39: Keep Manual… strategy buttons on the Plan card in simple view, open them when the live mode changes to a manual fallback, and mark a tap before the server confirms. Hide manual now stays hidden — the status poll no longer reopens the drawer — and a tap no longer flickers back to the previous strategy. - + A house left in a manual mode can start planning again: the Plan card shows "Use the plan" whenever the planner is not driving. It hands the battery to the planner mode this household's own prefs imply — the passive one unless battery export is allowed — and never grants export rights on its own. ## 2.10.0 @@ -1566,10 +1776,11 @@ identified vehicle profile. The report is visible on `GET /api/ocpp/chargers` as `charging_needs`, and quarantine still applies — a pending charge point's needs are shown but never reach a loadpoint. - + An AC session states energy without a battery size, so no target fraction is derived from it; guessing one would feed the planner a number the car never claimed. + - b824000: An adopted OCPP charger is now a device like any other. It gets a row in `/api/devices` and under Settings → Devices, keyed on the vendor and serial from its `BootNotification` rather than on the name it dialled with — that @@ -1579,32 +1790,33 @@ the dialled name, recorded as an endpoint so it reads as stable-until-changed. Pending chargers get no row: a device row says this hardware is part of the site, and quarantine says an unadopted charge point is not. - + `GET /api/ocpp/chargers` now also reports each charger's `serial` and `firmware`, and OCPP 1.6's deprecated `chargeBoxSerialNumber` is read when the current field is empty — shipped firmware disagrees about which to fill, and losing it loses the only stable identity some chargers ever report. - + The OCPP server's own settings — on/off, bind address, both ports, path, username and password — are editable under Settings → Chargers instead of only in `config.yaml`. TLS paths and per-charger credentials stay in the file: they are host filesystem paths and one secret per charger, set once at commissioning. + - b824000: The OCPP listener can now be pinned to one interface, served over TLS, and given a credential per charger. - + `ocpp.bind` finally does something. The library builds its listen address from the port alone, so the socket is unavoidably open on every interface; FTW now refuses the WebSocket handshake for a connection that arrived on any other address. That is an access control rather than a smaller attack surface — the port still answers a scan — and the docs say so. - + `ocpp.tls` serves `wss://` instead of `ws://`, ending the plaintext basic auth anyone on the LAN could sniff. `client_ca_file` additionally requires every charge point to present a certificate signed by that CA (OCPP 2.0.1 security profile 3). Half a TLS section is refused at startup rather than quietly serving plaintext. - + `ocpp.chargers` gives a named charge point a password of its own. On OCPP the basic-auth username is the charge point identity, so a listed charger must present both, and the shared password stops being enough to connect under its @@ -1618,7 +1830,7 @@ - b824000: Three fixes from running the OCPP central system against Sourceful's device simulator. Each one let FTW report a limit it had not actually imposed, or refuse a control that should have worked. - + **Charging profiles are sent as Relative, not Absolute.** FTW's schedule is a single period at second 0 with no end — "hold this limit until I say otherwise". Absolute expresses that only with a `startSchedule` timestamp, and @@ -1628,19 +1840,19 @@ answers **Accepted** while charging on at full rate. Relative carries no timestamp, so there is nothing to misparse — and nothing that depends on the charger's clock agreeing with ours. - - **A charger that refuses a charge-point-wide profile is retried on connector - 1.** OCPP 1.6 permits a `TxDefaultProfile` on connector 0 — it is how a profile + + **A charger that refuses a charge-point-wide profile is retried on connector 1.** OCPP 1.6 permits a `TxDefaultProfile` on connector 0 — it is how a profile applies to every connector — but some chargers read the connector-0 rule as `ChargePointMaxProfile`-only and reject it. Rejecting means no limit at all, so one retry on the first connector is the difference between a charger FTW steers and one it can only meter. - + **Manual EV controls reach an OCPP charger.** Pause, Resume, Force start and set-current posted to `/api/ev/command` went straight to the Lua driver registry, which an OCPP charge point is not in — it dialled us rather than being dialled. They failed with `driver "" not found` while automatic dispatch steered the same charger correctly. + - 4f6060d: Enabling the OCPP server from the Chargers panel now works on the first try: the username field carries the real default ("ftw") instead of a placeholder that validation then rejected, and saving an OCPP change @@ -1724,18 +1936,18 @@ - bf2c142: Add OCPP 1.6J support so EV chargers connect to FTW directly instead of through a vendor cloud. An OCPP charger needs no driver: the protocol is vendor-neutral, so one server in core handles every charger that speaks it. - + Chargers dial FTW rather than the other way round, so there is nothing to add under `drivers:`. A charge point becomes a device on its first BootNotification, keyed by the last segment of the URL it connected to, and dispatch treats it like any other EV reading. - + This reinstates `go/internal/ocpp`, retired as unused in #578, and wires it into the process behind a new `ocpp` config section. It matters because Charge Amps has no FTW driver at all and every current model speaks OCPP, while Easee and Zaptec can be commissioned once through their vendor portal and then run with no cloud in the runtime path. - + FTW throttles, pauses and resumes an OCPP charger like any other EV charger. Every command is a current limit rather than a remote start or stop, because `RemoteStopTransaction` is unreliable on Charge Amps hardware — units @@ -1743,11 +1955,12 @@ profile is honoured consistently and keeps the session meter intact across a pause. Below the IEC 61851 minimum of 6 A the charger is told 0 A rather than being rounded up to current the site fuse was not asked to carry. - + The server is off by default. Enabling it requires a username and password, and FTW refuses to start without them: the OCPP library builds its listen address from the port alone, so the socket is reachable on every interface and basic auth is the only gate. Keep the port closed at your router. + - bf2c142: OCPP charge points now start quarantined. A charge point that no charger entry (loadpoint) names connects as "pending": it shows on Settings → Chargers with its vendor, dialect and live state so it can be adopted, but its telemetry is @@ -1758,27 +1971,28 @@ entry with its id as the charger driver and saving — charger entries hot-reload, so adoption and un-adoption take effect on the save. - bf2c142: Serve OCPP 2.0.1 alongside 1.6J, so newer chargers connect without a driver too. - + Each version listens on its own port. A charger picks its dialect during the WebSocket handshake, before any message is sent, and the underlying library keeps one message handler per listener — so a single port cannot serve both. Set `ocpp.port_v201` to enable 2.0.1; leaving it unset keeps 1.6J only. - + Only the message encoding differs. Both dialects share one charger map, one telemetry path and one control path, so a 2.0.1 charger is metered, throttled and paused exactly like a 1.6 one, and dispatch cannot tell them apart. - + 2.0.1 restructures the messages more than the names suggest: StartTransaction and StopTransaction collapse into a single TransactionEvent, transaction ids become strings, connector status loses its charging meaning, and meter samples arrive inside transaction events as well as on their own. The new handler normalises all of that back to the same charger state. - + OCPP 2.1 is not supported. No production-grade Go implementation of it exists: the library FTW uses covers 1.6 and 2.0.1 and has no 2.1 support, and the Go projects that do claim 2.1 are early-stage validators and emulators rather than servers. Adding it later is one more handler and one more listener; the version-neutral core does not change. + - a95a971: Tesla Wall Connector Gen 3 can be added as a local EV charger. The setup wizard and Settings → EV take the box's LAN address; FTW reads plug state, power and session energy over HTTP with no Tesla account. The wall connector cannot take a current setpoint — steer a Tesla through tesla_vehicle. Experimental until a live Gen 3 has been exercised. - bf2c142: Vehicle profiles for chargers shared by several cars. A new `vehicles:` config list (also editable under Settings → Chargers → Vehicles) holds each car's diff --git a/Makefile b/Makefile index c24d24bb0..d92419048 100644 --- a/Makefile +++ b/Makefile @@ -37,6 +37,7 @@ help: @echo " sim-ocpp dial Evify OCPP chargers at a running FTW" @echo " dev start sims + main app against config.local.yaml" @echo " e2e run the full-stack e2e test" + @echo " update-chain install, update, crash and roll back real builds" @echo " verify fast pre-commit: test + compose + vet + build" @echo " verify-all pre-push: verify + Linux builds" @echo " install-hooks install git pre-commit + pre-push hooks (opt-in)" @@ -121,6 +122,18 @@ release-workflow-test: e2e: drivers-present cd go && FTW_E2E=1 go test ./test/e2e -v -timeout 180s +# The native update chain with real builds: install, update through a release +# mirror, three kills in probation, then a binary rollback. Run it before +# every stable. UPDATE_CHAIN_FROM (or FTW_UPDATE_CHAIN_FROM) lists the +# releases to update from; by default the last three betas before HEAD. With +# no beta tags the test fails; UPDATE_CHAIN_FROM=tree builds the previous +# release from this tree instead. +UPDATE_CHAIN_FROM ?= $(or $(FTW_UPDATE_CHAIN_FROM),$(shell git tag --list 'v0.*-beta.*' --merged HEAD --no-contains HEAD --sort=-v:refname 2>/dev/null | head -3)) +.PHONY: update-chain +update-chain: + cd go && FTW_UPDATE_CHAIN=1 FTW_UPDATE_CHAIN_FROM="$(UPDATE_CHAIN_FROM)" \ + go test -count=1 -timeout 60m -run 'TestUpdateChain$$' -v ./cmd/ftw-launcher/ + # FTW against device-simulator's house with a real fuse # (go/test/site/README.md). SITERIG_SET is short, full or scenario names; # SITERIG_LANES runs that many at once on ports 100 apart. diff --git a/README.md b/README.md index 3a794b9a3..a26871631 100644 --- a/README.md +++ b/README.md @@ -246,6 +246,24 @@ metadata are the detailed reference. Other files under [`docs/`](docs/) are focused installation or external-integration guides. +## Related repositories + +FTW spans several repositories. This one is Core; the others are: + +| Repository | Role | +|---|---| +| [`srcfl/device-drivers`](https://github.com/srcfl/device-drivers) | Lua driver source, contracts, tests and the signed driver bundle Core pins. Change drivers there. | +| [`srcfl/device-simulator`](https://github.com/srcfl/device-simulator) | Simulated house: inverters, batteries, chargers and meters over Modbus TCP, MQTT and OCPP. | +| [`srcfl/ftw-webapp`](https://github.com/srcfl/ftw-webapp) | Installable everyday client. | +| [`srcfl/ftw-app`](https://github.com/srcfl/ftw-app) | Native iOS and Android app. | +| [`srcfl/ftw-cloud`](https://github.com/srcfl/ftw-cloud) | Opt-in, box-signed, read-only copy of each home's state for the apps. | +| [`srcfl/home-assistant-addons`](https://github.com/srcfl/home-assistant-addons) | Home Assistant app packaging of each FTW release. | +| [`srcfl/ftw-web`](https://github.com/srcfl/ftw-web) | The project website. | +| [`srcfl/energyplan-demo`](https://github.com/srcfl/energyplan-demo) | Browser demo running the Energyplan worker in WebAssembly against a simulated home. | + +The Energyplan solver source is private; Core receives only its compiled +worker and public integration metadata. + ## Contributing Read [CONTRIBUTING.md](CONTRIBUTING.md). User-visible changes need a Changeset. diff --git a/config.example.yaml b/config.example.yaml index c8e9d86a7..c9073eea6 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -5,6 +5,7 @@ site: name: "Home" + # time_zone: Europe/Stockholm # the home's clock; empty uses the host's zone (UTC in Docker) control_interval_s: 2 # 2 s matches Ferroamp ehub ~1 Hz; drop to 1 for snappier response grid_target_w: 0 # 0 = self-consumption grid_tolerance_w: 42 # deadband (The Answer) @@ -12,7 +13,7 @@ site: smoothing_alpha: 0.3 slew_rate_w: 3000 # W per cycle. Both Ferroamp and Sungrow ramp internally — this is a soft ceiling, not the primary protection # slew_enabled: false # disable the external slew limiter entirely and trust the inverter's internal ramp control (experimental) - min_dispatch_interval_s: 2 # debounce; keep == control_interval_s + min_dispatch_interval_s: 2 # least time between battery commands; at or below control_interval_s it never skips a tick fuse: max_amps: 16 @@ -284,3 +285,11 @@ fleet_ping: # price: # demand_price_per_kw: 7000 # demand_top_n: 3 + +# Tests and development only: read elprisetjustnu-format prices from another +# server, with no Nord Pool fallback. The site rig serves pinned prices this +# way (go/test/site/prices_test.go). Leave it unset on a real site. +# Only the elprisetjustnu provider reads base_url; other providers ignore it. +# price: +# provider: elprisetjustnu +# base_url: http://127.0.0.1:8099 diff --git a/contract/push-catalogue.yaml b/contract/push-catalogue.yaml index 3a563679d..e428f57d1 100644 --- a/contract/push-catalogue.yaml +++ b/contract/push-catalogue.yaml @@ -44,6 +44,13 @@ events: # kettle must not page anyone. title: The house is drawing too much body: "{phase} is over the fuse rating." + - kind: control.not_following + # After the box's own threshold: ten minutes in which the box's answer + # to "Are we in control?" said a device does not do what FTW asked, as + # a battery that charges when told to discharge. A slow battery must not + # page anyone. Once until the device follows again. + title: A device is not following FTW + body: "{name} does not do what FTW asks. Open FTW to see why." - kind: box.unreachable # The one sentence the box cannot send about itself. The relay holds # this pre-encrypted and posts it only when the box has missed its diff --git a/contract/registry.yaml b/contract/registry.yaml index a874b6af9..94d7f805e 100644 --- a/contract/registry.yaml +++ b/contract/registry.yaml @@ -223,6 +223,7 @@ cloud_purposes: [serve, stats] ops: - { name: site.mode.set, scope: ftw.mode.write, agent: ftw.goals.request, desc: Change the site operating mode } - { name: planner.prefs.set, scope: ftw.mode.write, agent: ftw.goals.request, desc: Set household planner preferences } + - { name: site.time_zone.propose, scope: ftw.setup.write, desc: Offer the phone's time zone to a box that keeps none } - { name: battery.hold, scope: ftw.dispatch.write, desc: Hold the battery at a fixed setpoint } - { name: loadpoint.hold, scope: ftw.dispatch.write, agent: ftw.dispatch.hold, desc: Charge the car now at a fixed current } - { name: loadpoint.boost, scope: ftw.dispatch.write, desc: Boost the car from the house battery } diff --git a/docker-compose.macos.yml b/docker-compose.macos.yml index ff75041d9..370a66d4b 100644 --- a/docker-compose.macos.yml +++ b/docker-compose.macos.yml @@ -22,10 +22,10 @@ # In config.yaml set the MQTT host to `mosquitto` (port 1883), NOT # `localhost` / `127.0.0.1`. There is no host networking to make # localhost mean "the broker". -# - First enable of the house password (api.lan_auth) is loopback-only -# inside the container. A Mac curl to http://127.0.0.1:8080 arrives as -# the bridge gateway. Use `docker compose -f docker-compose.macos.yml -# exec ftw` and then curl http://127.0.0.1:8080 from there. +# - A Mac browser on http://127.0.0.1:8080 arrives as the bridge +# gateway, not loopback. Set the house password in the page as usual. +# To reset a forgotten one: `docker compose -f docker-compose.macos.yml +# exec ftw /app/ftw password reset`. # - mDNS (`zap.local`) and UDP broadcast device discovery do NOT cross # the Docker Desktop VM boundary. Configure every driver with an # EXPLICIT IP address. Outbound unicast TCP (Modbus TCP to a known diff --git a/docs/agents.md b/docs/agents.md index 37d982f9d..dfeb0e005 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -628,7 +628,7 @@ above and its approval key, and an agent can never reach them. | `GET /api/agents` | A client on the LAN. Rows, requests and keys, the box's facts and the registry's agent ops | | `GET /api/agents/audit?before=&n=` | A client on the LAN | | `POST /api/agents/offer {offer}` | A client on the LAN. The offer as the box reads it; grants nothing | -| `POST /api/agents {offer, scopes, ops, caps, ttlS}` | A LAN client signed in with the house password | +| `POST /api/agents {offer, scopes, ops, caps, ttlS}` | A browser signed in with the house password, on the LAN or on the box | | `POST /api/agents/{id}/widen {epoch, scopes, ops, caps, ttlS}` | The same | | `POST /api/agent-requests/{reqId} {approve: true}` | The same | | `POST /api/agents/approval-key {deviceId, pub}` | The same: a phone that lost its key | @@ -672,13 +672,14 @@ above and its approval key, and an agent can never reach them. replace the offer's when given. Without `ttlS` the grant lasts 30 days, or the offer's length if shorter. A `ttlS` past the role's longest grant is `E_CAP_EXCEEDED {cap: "expiresAtMs", max}`. -- **Every route that widens access needs another device on the home - network, signed in with the house password.** The box never reads the - sign-in from loopback, so a browser on the box, or one through an SSH - tunnel to it, cannot widen access with or without the password: a - driver's Lua and the session passthrough can both reach the box from - inside it. `box.widen` in `GET /api/agents` says whether this request - would pass, so the page stops before a code is spent. +- **Every route that widens access needs a browser signed in with the + house password,** on another device or on the box itself. The password, + not the address, proves the owner. Loopback stays an owner for settings + without it, so it could replace the password and then sign in: it is not + a wall against programs on the box. A driver cannot reach the API + (below), and the session passthrough is refused. `box.widen` in + `GET /api/agents` says whether this request would pass, so the page stops + before a code is spent. - **Drivers never reach the box's API.** A driver's HTTP, TCP or WebSocket connection to a port the API listens on is refused, whether it goes to a loopback address or to an address of the box. The check runs on the @@ -691,16 +692,25 @@ above and its approval key, and an agent can never reach them. What the sign-in is worth today: -- **Off by default.** With `api.lan_auth` off, which is the default, every - client on the home network is an owner with no sign-in. Granting a remote - agent from there would let any device on the Wi-Fi give the outside world - a way in, so those routes refuse with 403 until the house password is on. -- **With the house password on.** The password has at least 10 characters - and is stored as argon2id (64 MiB, 3 passes). Five wrong guesses lock every - sign-in for 30 seconds, box-wide. A sign-in sets a random 32-byte cookie, - `HttpOnly` and `SameSite=Strict`, that lasts 12 hours and dies with a - restart. Browser requests from other sites are refused by Origin and - `Sec-Fetch-Site` checks. +- **Off until someone chooses it.** A new box asks for it during setup; an + older box may still have it off. With `api.lan_auth` off every client on + the home network is an owner with no sign-in. Granting a remote agent + from there would let any device on the Wi-Fi give the outside world a way + in, so those routes refuse with 403 until the house password is on. The + Access page asks for it right above the code: any browser that reaches + the box may choose the first one, which signs that browser in. While the + lock is off that browser is already an owner, so choosing it grants + nothing new (Fredrik, 25 October 2026). An owner locked out by someone + else's password, or who forgot it, runs `ftw password reset` on the box; + see [operations.md](operations.md#house-password). +- **With the house password on.** Its length is the owner's choice: the + page warns below 8 characters but takes any password that is not empty + (Fredrik, 25 October 2026). It is stored as argon2id (64 MiB, 3 + passes). Five wrong guesses lock every sign-in for 30 seconds, box-wide. + A sign-in sets a random 32-byte cookie, + `HttpOnly` and `SameSite=Strict`, that lasts 30 days, survives a restart + and ends when the password changes. Browser requests from other sites are + refused by Origin and `Sec-Fetch-Site` checks. - **What it lacks.** The page is plain HTTP, so anyone who can read the home network's traffic can read the password and the cookie. There is one shared password and no person behind it, so the audit row names the @@ -756,8 +766,8 @@ receipts for current agents, plus 32 old receipts, with a 128-row storage bound. No expired or cancelled record can reopen a valid code after pruning. The local routes are `GET/POST /api/agent-invites` and -`DELETE /api/agent-invites/{id}`. POST uses the same house-password and -non-loopback LAN gate as an ordinary grant. Signed app messages allow an +`DELETE /api/agent-invites/{id}`. POST uses the same house-password gate +as an ordinary grant. Signed app messages allow an owner to share access away from home. The recipient connects FTW from its own assistant, selects **I have an invitation code** on the consent page and pastes the code there. Core must be online. The normal signed-roster, diff --git a/docs/architecture.md b/docs/architecture.md index 8b7e1f327..1d7d8dee3 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -205,8 +205,13 @@ have no DuckDB dependency. The [FTWDB experiment is retired](ftwdb-shadow.md). State schema 5 added aggregate history and binds state.db to a specific history.db generation. Portable backups export a SQLite read snapshot with row counts and hashes checked, plus any Parquet files still on the box. A -config-only snapshot cannot recover missing history. To return to an older -Core across a state-schema change, stop Core and restore a verified full backup +config-only snapshot cannot recover missing history. If history.db is damaged, +empty or missing at startup, Core moves it aside as `history.db.corrupt-`, +starts an empty history bound to a new generation and reports the loss on +`/api/health`. On a site that converted beta history this also clears the beta +generation key; the beta originals stay on disk. A readable history.db from +another generation is still refused. +To return to an older Core across a state-schema change, stop Core and restore a verified full backup with its matching version; rolling back the program alone across that boundary is refused. diff --git a/docs/cars.md b/docs/cars.md index aaa49c50a..7a087afa8 100644 --- a/docs/cars.md +++ b/docs/cars.md @@ -181,7 +181,12 @@ charger's phase mode. A change of car or goal rolls the charger's deadline before the replan, so that plan has the car's goal in it. - **One goal per car**, used wherever the car is plugged in. `readyByMin` is - in the box's time zone. + in the box's time zone: the site's `site.time_zone`, or the host's when the + site stores none. A Docker host runs on UTC, so the app, the web client + and the on-box setup offer their own zone (`site.time_zone.propose`). The + box takes an offer only while it keeps no zone, and restarts to run on it. + A recurring goal keeps its clock time in the new zone. A one-off goal that + finishes at the car's limit keeps the instant it was saved for. - **Surplus-only is the car's choice.** The site's battery rule (`surplus_unlock_bat_soc`) is the charger's: it rides `PUT /api/loadpoints/{id}/schedule` as before, and no car op carries it. diff --git a/docs/energyplan-contract.md b/docs/energyplan-contract.md index d15d238e7..4b56584b0 100644 --- a/docs/energyplan-contract.md +++ b/docs/energyplan-contract.md @@ -36,10 +36,13 @@ whose I/O exceeds the call deadline; a later call may start a fresh process. The handshake identifies the worker, its version, protocol range and features. Protocol compatibility and supported features govern use, not matching Core and Energyplan release numbers. Current features are `champion`, -`forecast_reset`, `partial_slots`, `demand_charges`, `ev_duty` and -`charging_periods` and `published_prices`; the worker also reports -`forecast_protocol_version`. -Core negotiates `charging_periods` before sending its optional fields. +`forecast_reset`, `partial_slots`, `demand_charges`, `ev_duty`, +`charging_periods`, `published_prices`, `hourly_import_caps` and +`demand_elapsed_hours`; the worker also reports `forecast_protocol_version`. +Core negotiates `charging_periods` before sending its optional fields, and +`hourly_import_caps` before sending `hourly_import_caps`. +Core negotiates `demand_elapsed_hours` before sending a demand hour that +starts before the first slot or ended by its execution start. New optional request fields need capability negotiation or a paired update that preserves deployed clients. Unknown optimizer request fields are errors. @@ -83,15 +86,63 @@ Worker model support does not prove Core can execute every supported case. Physical capacity, device power, grid limits, allowed EV steps and mode rules are hard limits. Measured battery energy outside an operating band may recover -without worsening the existing violation. After recovery it must stay inside -the band. Physical zero and capacity remain hard limits throughout. Default -wear cost is zero; charge/discharge efficiency remains in the physical model. +without worsening the existing violation, for each battery and for the +batteries' total, whose band is the sum of their bands (Core applies the same +SoC limits to every battery). After recovery it must stay inside the band. +Physical zero and capacity remain hard limits throughout. Default wear cost +is zero; charge/discharge efficiency remains in the physical model. Multiple batteries keep their own limits. Current execution requires a common charge/discharge direction in each slot. Demand-charge windows use explicit UTC intervals and elapsed import from Core; Core owns local calendar and daylight-saving conversion. +`hourly_import_caps` carries the owner's hourly import limit as a hard +limit. Each entry is `{start_ms, end_ms, max_mean_w, elapsed_import_wh}`: +one local clock hour as UTC milliseconds, the mean import the hour may hold +in W, and the import in Wh the hour has already used before the plan starts +(0 for later hours). Core sends one entry per clock hour that overlaps the +plan, from the first slot's execution start to the horizon end; a +daylight-saving day has 23 or 25. The hour in progress may start before the +first slot and the last may end after the horizon; an entry outside the plan +is ignored. Entries must not overlap, last 1 minute to 25 hours, and number +at most 512. Import is `max(0, grid)` per pulse state: a duty slot imports +`duty * max(0, grid on)` plus `(1 - duty) * max(0, grid off)`, at the cut +points between the cars' duties. In every scenario, the plan's import inside +an hour from execution start must not exceed +`max_mean_w * hours - elapsed_import_wh`. In an hour where every car is off +it may instead reach what the house alone imports there, +`max(0, load_w + pv_w)` per slot with the battery idle, if that is more: Core +cannot cut the house, but its live limit stops a car before the house. +`max_mean_w` may be 0: the hour then has room for the house alone. The worker +is stricter in one case: while a car pulses into solar surplus and the +battery charges, it counts the car's on-step import plus the whole charge. +The cap wins over EV goals, whose miss shows in +`flex_shortfall_wh`. The plan reports each hour in +`plan.hourly_import_caps` as `{start_ms, end_ms, import_wh, max_wh}` on the +published forecast, `import_wh` including the elapsed part. + +`demand_charges` carries peak-power tariffs as +`{id, price_per_kw, top_n, already_kw, floor_kw, hours}`, each hour +`{start_ms, end_ms, elapsed_import_kwh, weight, group}` with weight 1 by +default. An hour's sample is its weight times its mean import in kW: the +import before the first slot's execution start, `elapsed_import_kwh`, plus the +plan's import per pulse state from then or `start_ms`, whichever is later, to +`end_ms`, over the hour's length. A group is one sample, the largest of its +hours'; an ungrouped hour and each `already_kw` value are one sample each. The +bill is `price_per_kw * max(floor_kw, mean of the top_n largest samples)`, or +of all samples if there are fewer. Hours must end inside the horizon, not +overlap, last 1 minute to 25 hours and number at most 512 per charge. An hour +that ended by execution start must share its group with an hour that overlaps +the plan. Core makes each local day a group. With `demand_elapsed_hours` it +sends the hour in progress from its start, puts the completed peak of each day +the plan reaches in that day's group and sends each other completed day as one +`already_kw` value, so each local day is one sample. Without the feature Core +sends only hours from the first slot on and every completed day in +`already_kw`. The plan reports `plan.demand_cost` and, per charge in +`plan.demand_charges`, `billed_kw`, `cost` and each hour's unweighted +`mean_kw`, in ascending start order. + Thermal, commercial and recourse inputs are unsupported. Protocol v1 has no per-phase electrical model, future connection windows or phase-switch schedule. Total watts cannot certify phase safety. Those extensions need an explicit @@ -167,13 +218,25 @@ For minimization, a reported bound must satisfy `lower_bound <= optimum <= candidate objective` for the same feasible set and objective. Unknown bounds and gaps remain null. When executed pulse cost and the optimized cost model differ, report `feasible` and omit the cost bound. -Core validates physical execution; it does not independently prove optimality. - -Current Core allows 500 ms for small plans and 5 s for larger plans or demand -charges, with a 7 s transport timeout. It reduces the budget near the first -slot's end and leaves 50 ms for validation/publication. These are compute and -call budgets, not a proven wall-time SLA. Measure queueing, startup, decoding, -solving, replay, encoding and Core validation separately when changing them. +A plan from a search that left out part of the model reports `feasible` with +a null bound. Core validates physical execution; it does not independently +prove optimality. + +Core makes no worker call when less than 500 ms of the first slot remain. It +allows 500 ms for small plans, 1.5 s with an active car, and 5 s when +slots × (3 × batteries + 2 × active cars) is at least 965, or when the request +carries PV curtailment, demand charges or hourly import caps. Each call's +`time_limit_s` is the smallest of that budget, 5.6 s and the time left before +that call's deadline once its request is prepared, less a reply margin of +max(5% of that time, 30 ms). A later call of the same planning attempt, such as +the forecast-caution call after the point call, can get less, and Core sends +no request with less than 450 ms before its deadline. Core starts the worker +with `--time-limit=5s`. The transport timeout is 7 s. Near the first slot's end +Core keeps 50 ms for validation and publication and a reply margin of +max(5% of the remaining time after that, 30 ms). +These are compute and call budgets, not a proven wall-time SLA. Measure +queueing, startup, decoding, solving, replay, encoding and Core validation +separately when changing them. A warm worker may keep one previous plan as a candidate. Before reuse it must match asset identities, advance time, rebuild energy from current measurements, @@ -184,6 +247,12 @@ preferences and cost. `solver.warm_start_valid`, when present, means a cached candidate passed this check; it does not mean the final plan used it. Restart may discard this optional cache without losing stored user goals. +Replies may carry optional diagnostics. `solver.source`, `solver.fallbacks`, +`error.fallbacks` and `error.model_variables` are worker-defined, opaque +numeric codes; each `fallbacks` is an array of them. `solver.model_variables` +is an integer; Core rejects a reply whose value is not one. Core never +accepts, rejects or ranks a plan by the values of these diagnostics. + On failure Core may use its validated fallback only where it can represent the site. Otherwise an old plan may remain as diagnostic history; retention does not grant permission to execute stale or unvalidated actions. Restart diff --git a/docs/operations.md b/docs/operations.md index 338969ceb..0668fa010 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -185,31 +185,62 @@ mutations remain locked. remote access; remote access is the FTW app. See [architecture.md](architecture.md#remote-access-boundary). -`api.lan_auth` is off by default. Turn it on from loopback inside the -process: `http://127.0.0.1:8080` Settings → System on the machine, or `curl` -to `127.0.0.1`. When on, protected LAN routes need the house password or -`FTW_API_TOKEN`. `curl` sends `Authorization: Bearer ` or -`Authorization: Bearer `. The browser login form sets a -session cookie (`ftw_lan`, 12 hours). Loopback (`127.0.0.1` / `::1`) never -asks. Live status stays readable without the password; a viewer caller is -minted for those reads. The two secrets do not share a lockout: retrying the -API token cannot lock the household out of Settings. A first enable from -another LAN address is refused, so a visitor cannot set the password. -`POST /api/config` cannot flip the flag. - -On an older Docker install under Docker Desktop (`docker-compose.macos.yml`), -a host curl to localhost arrives as the bridge gateway, not loopback, so -Settings and host curl get 403. Desktop SNAT uses that same peer for every -published-port client, including LAN visitors, so it must not count as -loopback. Enable from inside the container: +### House password + +The house password (`api.lan_auth`) keeps other devices on the home network +out of Settings and stops them letting an assistant in. Live status stays +readable without it; a viewer caller is minted for those reads. + +- **A new box** asks for it on the setup wizard's last step and on the + assistant setup path. The Home Assistant app asks for none: Home + Assistant signs its users in. A script that posts first-run settings + without `house_password` leaves the lock off. +- **A running box without one** asks for it where it is needed: Settings → + System → House password, or Settings → Access, right above the + assistant's code. Any browser that reaches the box may set the first + one. While the lock is off that browser is already an owner, so setting + it grants nothing new; it closes the door behind it. Choosing it signs + that browser in. `POST /api/config` cannot flip the flag. +- **Any password** is accepted, however short. The page warns below 8 + characters; the choice is the owner's. +- **When it is on**, protected LAN routes need the password, a browser + session, or `FTW_API_TOKEN`. Scripts send `Authorization: Bearer + ` or `Authorization: Bearer `. The browser + sign-in sets an `HttpOnly` session cookie (`ftw_lan`, 30 days). Five + wrong passwords lock every sign-in for 30 seconds; retrying the API + token never counts. From another device, changing or turning off the + password needs it. +- **The box itself** (`127.0.0.1` / `::1`) needs no password for settings, + so `ftw` commands keep working and `ftw password reset` can turn it off. + A browser on the box signs in like any other before it lets an + assistant in. Drivers cannot reach the API at all; anything else that + runs on the box can already change every setting. + +**Forgot it**, or someone else set it first: turn it off on the box, then +choose a new one on the box's page. ```bash -docker compose -f docker-compose.macos.yml exec ftw \ - wget -qO- --header='Content-Type: application/json' \ - --post-data='{"enabled":true,"password":"ETT-LANGT-LOSEN"}' \ - http://127.0.0.1:8080/api/auth/password +ftw password reset ``` +The `ftw` command moves with `install.sh --refresh`, not with `ftw update`. +If it answers `unknown command password`, run the same step with Core's own +binary for the release `ftw status` names: + +```bash +/opt/ftw/releases/v0.X.Y/ftw password reset +``` + +On Docker, run the box's own binary in its container (add +`-f docker-compose.macos.yml` on Docker Desktop): + +```bash +docker compose exec ftw /app/ftw password reset +``` + +Either signs every browser out. Without shell access to the box, the +password cannot be reset. + An owner pairing QR is minted only from loopback or with the house password. Promoting a paired phone to owner uses the same gate. The first pairing on an empty box is an owner whatever the code said, so @@ -218,9 +249,6 @@ LAN once an owner exists. The FTW app and Home Assistant MQTT are unchanged. -Recovery: `curl` to `127.0.0.1`, or set `api.lan_auth: false` in -`config.yaml` and restart Core. - ### "remote access to protected API routes is disabled" The full message is `remote access to protected API routes is disabled; diff --git a/docs/safety.md b/docs/safety.md index c53c4ae6b..6229cb646 100644 --- a/docs/safety.md +++ b/docs/safety.md @@ -51,17 +51,25 @@ available battery energy and hardware power capability. The EV hard limit treats the main fuse as heat, not as one reading. It low-passes each phase's I² from the site meter with a 60 s time constant. -A phase over the fuse less 1 A cuts the EVs once that filtered current -passes the same limit, and at once above 1.2 × the fuse. A running car's +Each phase's working limit is the fuse rating less `fuse.safety_margin_a`; +no amps are held back beyond that setting. A phase over the limit cuts the +EVs once that filtered current passes the same limit, and at once above +1.2 × the fuse. A running car rises one step, or the rest of its demand, +only when that step fits under the limit, and not within 20 s of a cut. Its rise reads the higher of the reading and that filtered current. An idle car's start reads each phase as it is, plus its new current; the filtered -current holds it back only while it is over the limit. Where the meter +current holds it back only while it is over the limit. Neither counts +battery discharge that the fuse guard added as room: that relief ends once +the site fits without it. Where the meter reports phase currents, they replace total import as the EVs' fuse check; a tariff import ceiling still applies. A phase's current takes its direction from the same phase's power. It counts as export only when that power shows it clearly and the phases' -powers add up to the meter's total; otherwise it counts as import. An +powers add up to the meter's total; otherwise it counts as import. The +battery's fuse checks make one exception for a meter with no phase power +while the total exports: a phase counts as export when the other phases' +currents cannot carry that export alone. An exporting phase is never cut for: less EV load would raise its export. A rise or start on it reads its current as it is, without its heat. diff --git a/docs/self-update.md b/docs/self-update.md index a698bfcea..5565a2978 100644 --- a/docs/self-update.md +++ b/docs/self-update.md @@ -128,7 +128,7 @@ duration; in a log or script it is one line per step. Core records every finished step with its own timing, so a step that ends between two reads is still shown. `ftw update` waits through the restart, including a history migration, and reports the version and health that result. A trial only -becomes current after readiness; +becomes current after its first control tick; a failed trial falls back to the previous Core, and `ftw update` names the Core that runs. Already current exits 0, so a script can run the step unattended; a failed step exits 1. The same steps are Core API calls. A diff --git a/drivers/BUNDLED_SOURCE.json b/drivers/BUNDLED_SOURCE.json index 5dfa246e7..073081cdb 100644 --- a/drivers/BUNDLED_SOURCE.json +++ b/drivers/BUNDLED_SOURCE.json @@ -16,7 +16,7 @@ "from the signed channel. Run scripts/sync-bundled-drivers.sh to update." ], "repository": "srcfl/device-drivers", - "commit": "57984984a9bf6939d5206c5a05138296d9c6d940", + "commit": "e07d27038e83e431cc01c1cd988ffcb1997355c0", "source_dir": "drivers/lua", "drivers": [ "ambibox_v2x", "ctek", "ctek_hybrid", "ctek_v2", "deye", "easee_cloud", diff --git a/go/cmd/ftw-launcher/main.go b/go/cmd/ftw-launcher/main.go index a7390472c..4d58cbfad 100644 --- a/go/cmd/ftw-launcher/main.go +++ b/go/cmd/ftw-launcher/main.go @@ -122,10 +122,15 @@ func launch(root, config, userDrivers string, extra []string, execFn execFunc) e if err != nil { return err } - if trial { - return fmt.Errorf("fallback selected another trial %s", tag) + // With current damaged, the fallback is the previous release as a + // trial. It gets one attempt; a failure marks it so it is not retried. + err = execRelease(path, tag, trial, root, config, userDrivers, extra, execFn) + if err != nil && trial { + if failErr := manager.FailTrial(tag); failErr != nil { + return fmt.Errorf("fallback trial exec failed (%v) and could not be recorded: %w", err, failErr) + } } - return execRelease(path, tag, false, root, config, userDrivers, extra, execFn) + return err } func execRelease(path, tag string, trial bool, root, config, userDrivers string, extra []string, execFn execFunc) error { diff --git a/go/cmd/ftw-launcher/main_test.go b/go/cmd/ftw-launcher/main_test.go index 7da3c0683..80d98ba8d 100644 --- a/go/cmd/ftw-launcher/main_test.go +++ b/go/cmd/ftw-launcher/main_test.go @@ -70,6 +70,84 @@ func TestTrialExecFailureRunsCurrentRelease(t *testing.T) { } } +// damagedCurrentWithPendingUpdate leaves v0.132.0 current but damaged, +// v0.131.0 previous and healthy, and v0.133.0 waiting to start. +func damagedCurrentWithPendingUpdate(t *testing.T) (string, nativeupdate.Manager) { + t.Helper() + root := t.TempDir() + manager := nativeupdate.Manager{Root: root} + for _, tag := range []string{"v0.131.0", "v0.132.0", "v0.133.0"} { + testRelease(t, root, tag) + } + if err := manager.Init("v0.131.0"); err != nil { + t.Fatal(err) + } + if err := manager.Prepare("v0.132.0"); err != nil { + t.Fatal(err) + } + if _, _, _, err := manager.Select(); err != nil { + t.Fatal(err) + } + if err := manager.Commit("v0.132.0"); err != nil { + t.Fatal(err) + } + if err := manager.Prepare("v0.133.0"); err != nil { + t.Fatal(err) + } + if err := os.Remove(filepath.Join(root, "releases", "v0.132.0", "web", "index.html")); err != nil { + t.Fatal(err) + } + return root, manager +} + +func TestTrialExecFailureWithDamagedCurrentRunsPrevious(t *testing.T) { + root, manager := damagedCurrentWithPendingUpdate(t) + var launched []string + err := launch(root, "/data/config.yaml", "/data/drivers", nil, func(binary string, _ []string, env []string) error { + launched = append(launched, binary) + if strings.HasSuffix(binary, "v0.133.0/ftw") { + return errors.New("cannot exec") + } + if !containsEnv(env, "FTW_NATIVE_TRIAL_TAG=v0.131.0") { + t.Fatal("the fallback must start as a trial") + } + return nil + }) + if err != nil { + t.Fatal(err) + } + if len(launched) != 2 || !strings.HasSuffix(launched[0], "v0.133.0/ftw") || !strings.HasSuffix(launched[1], "v0.131.0/ftw") { + t.Fatalf("launch order = %v", launched) + } + state, err := manager.Read() + if err != nil || state.Current != "v0.132.0" || state.Trial != "v0.131.0" || state.LastFailed != "v0.133.0" { + t.Fatalf("slots: %+v %v", state, err) + } +} + +func TestFallbackExecFailureIsRecordedAndNotRetried(t *testing.T) { + root, manager := damagedCurrentWithPendingUpdate(t) + var launched []string + failAll := func(binary string, _ []string, _ []string) error { + launched = append(launched, binary) + return errors.New("cannot exec") + } + if err := launch(root, "/data/config.yaml", "/data/drivers", nil, failAll); err == nil { + t.Fatal("want the fallback exec error") + } + if len(launched) != 2 || !strings.HasSuffix(launched[1], "v0.131.0/ftw") { + t.Fatalf("launch order = %v", launched) + } + state, err := manager.Read() + if err != nil || state.Trial != "" || state.LastFailed != "v0.131.0" { + t.Fatalf("slots: %+v %v", state, err) + } + launched = nil + if err := launch(root, "/data/config.yaml", "/data/drivers", nil, failAll); err == nil || len(launched) != 0 { + t.Fatalf("second start err=%v launched=%v; want the damaged-release error", err, launched) + } +} + func containsEnv(env []string, value string) bool { for _, item := range env { if item == value { diff --git a/go/cmd/ftw-launcher/update_chain_test.go b/go/cmd/ftw-launcher/update_chain_test.go new file mode 100644 index 000000000..f49460923 --- /dev/null +++ b/go/cmd/ftw-launcher/update_chain_test.go @@ -0,0 +1,706 @@ +package main + +import ( + "archive/tar" + "bytes" + "compress/gzip" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "fmt" + "io" + "io/fs" + "net" + "net/http" + "net/http/httptest" + "os" + "os/exec" + "path/filepath" + "runtime" + "strings" + "sync" + "syscall" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/nativeupdate" +) + +// TestUpdateChain runs the native update chain with real binaries: the +// launcher installs a previous release, Core updates itself to the candidate +// through the release mirror, the candidate is killed three times inside its +// probation and the launcher falls back, and then Core takes a binary +// rollback. Slots, the running version, settings, a charging goal and the +// history count are checked at every step. +// +// It builds Core, so it only runs on request: +// +// make update-chain # previous = the last three betas +// FTW_UPDATE_CHAIN=1 FTW_UPDATE_CHAIN_FROM=tree go test -run TestUpdateChain ./cmd/ftw-launcher/ +// +// FTW_UPDATE_CHAIN_FROM lists git refs to update from, separated by spaces. +// Each one is built from `git archive`; "tree" builds the previous release +// from this tree as well. FTW_UPDATE_CHAIN_TAG names the candidate. +func TestUpdateChain(t *testing.T) { + if os.Getenv("FTW_UPDATE_CHAIN") != "1" { + t.Skip("set FTW_UPDATE_CHAIN=1 (make update-chain) to build and run the update chain") + } + if runtime.GOARCH != "arm64" && runtime.GOARCH != "amd64" { + t.Skipf("native releases exist for arm64 and amd64, not %s", runtime.GOARCH) + } + refs, err := chainRefs(os.Getenv("FTW_UPDATE_CHAIN_FROM")) + if err != nil { + t.Fatal(err) + } + _, file, _, _ := runtime.Caller(0) + repo := filepath.Clean(filepath.Join(filepath.Dir(file), "..", "..", "..")) + work := t.TempDir() + candidateTag := envOr("FTW_UPDATE_CHAIN_TAG", "v0.999.0-beta.1") + launcher := chainBuild(t, filepath.Join(repo, "go"), "./cmd/ftw-launcher", "", filepath.Join(work, "ftw-launcher")) + candidate := chainBuild(t, filepath.Join(repo, "go"), "./cmd/ftw", candidateTag, filepath.Join(work, "candidate", "ftw")) + mirror := newReleaseMirror(t) + mirror.add(t, repo, repo, candidateTag, candidate, launcher) + + for _, ref := range refs { + t.Run(ref, func(t *testing.T) { + previousTag, tree := "v0.998.0-beta.1", repo + if ref != chainTreeRef { + if nativeupdate.ValidTag(ref) { + previousTag = ref + } + tree = filepath.Join(t.TempDir(), "src") + exportRef(t, repo, ref, tree) + } + previous := chainBuild(t, filepath.Join(tree, "go"), "./cmd/ftw", previousTag, filepath.Join(t.TempDir(), "ftw")) + mirror.add(t, repo, tree, previousTag, previous, launcher) + runUpdateChain(t, launcher, mirror, previousTag, candidateTag) + }) + } +} + +// chainTreeRef in FTW_UPDATE_CHAIN_FROM builds the previous release from this +// tree instead of a git ref. +const chainTreeRef = "tree" + +// chainRefs reads FTW_UPDATE_CHAIN_FROM. An empty list is an error, not a +// quiet fall back to "tree": a clone without beta tags would otherwise pass +// by updating the candidate from its own source. +func chainRefs(from string) ([]string, error) { + refs := strings.Fields(from) + if len(refs) == 0 { + return nil, fmt.Errorf("no previous release to update from: FTW_UPDATE_CHAIN_FROM is empty " + + "(make update-chain found no beta tags before HEAD; shallow clone or fork?). " + + "Run git fetch --tags, name refs in FTW_UPDATE_CHAIN_FROM, " + + "or set FTW_UPDATE_CHAIN_FROM=" + chainTreeRef + " to build the previous release from this tree") + } + return refs, nil +} + +func TestChainRefs(t *testing.T) { + for _, from := range []string{"", " ", "\n"} { + if refs, err := chainRefs(from); err == nil || !strings.Contains(err.Error(), "FTW_UPDATE_CHAIN_FROM="+chainTreeRef) { + t.Fatalf("chainRefs(%q) = %v, %v; want an error naming the tree opt-in", from, refs, err) + } + } + for from, want := range map[string]string{ + "tree": "tree", + "v0.157.1-beta.1 v0.157.0-beta.1\n": "v0.157.1-beta.1 v0.157.0-beta.1", + } { + if refs, err := chainRefs(from); err != nil || strings.Join(refs, " ") != want { + t.Fatalf("chainRefs(%q) = %v, %v; want %s", from, refs, err, want) + } + } +} + +func runUpdateChain(t *testing.T, launcher string, mirror *releaseMirror, previous, candidate string) { + b := newChainBox(t, launcher, mirror) + mirror.publish(previous) + + // Install the previous release as a fresh box does. + archive, checksum := mirror.files(previous) + b.launcherCmd(t, "install", previous, archive, checksum) + b.launcherCmd(t, "init", previous) + b.wantSlots(t, "after init", nativeupdate.Slots{Current: previous}) + b.start(t) + b.waitVersion(t, previous) + want := b.seed(t) + b.stop(t, syscall.SIGTERM) + + // The previous Core sees the candidate on its channel and updates. + b.start(t) + b.waitVersion(t, previous) + want = b.check(t, "previous restarted", want) + mirror.publish(candidate) + b.update(t, candidate, false) + b.wantSlots(t, "update staged", nativeupdate.Slots{Current: previous, Next: candidate}) + b.start(t) + b.waitVersion(t, candidate) + b.waitCommitted(t, candidate, previous) + want = b.check(t, "candidate committed", want) + + // A clean stop inside probation does not count as a crash. + b.stop(t, syscall.SIGTERM) + b.start(t) + b.waitVersion(t, candidate) + if s := b.slots(t); s.Probation == nil || len(s.Probation.Crashes) != 0 { + t.Fatalf("a clean stop counted as a crash: %+v", s.Probation) + } + + // Three kills inside probation send the box back to the previous release. + for kill := 1; kill <= 3; kill++ { + b.stop(t, syscall.SIGKILL) + b.start(t) + if kill < 3 { + b.waitVersion(t, candidate) + if s := b.slots(t); s.Current != candidate || s.Probation == nil || len(s.Probation.Crashes) != kill { + t.Fatalf("after kill %d: slots %+v probation %+v", kill, s, s.Probation) + } + continue + } + b.waitVersion(t, previous) + } + b.wantSlots(t, "probation fallback", nativeupdate.Slots{Current: previous, LastFailed: candidate}) + want = b.check(t, "probation fallback", want) + + // Retry the failed candidate on request, then take a binary rollback. + b.update(t, candidate, true) + b.wantSlots(t, "retry staged", nativeupdate.Slots{Current: previous, Next: candidate}) + b.start(t) + b.waitVersion(t, candidate) + b.waitCommitted(t, candidate, previous) + want = b.check(t, "retry committed", want) + var rollback map[string]any + if status := b.call(t, "POST", "/api/version/binary-rollback", nil, &rollback); status != http.StatusAccepted { + t.Fatalf("binary rollback = %d %v", status, rollback) + } + b.waitExit(t, 90*time.Second) + b.wantSlots(t, "rollback staged", nativeupdate.Slots{Current: candidate, Previous: previous, Next: previous}) + b.start(t) + b.waitVersion(t, previous) + b.waitCommitted(t, previous, candidate) + b.check(t, "binary rollback", want) + b.stop(t, syscall.SIGTERM) +} + +// chainSnapshot is what an update must keep: two settings, a saved charging +// goal and the recorded history. +type chainSnapshot struct { + Mode string + GridTargetW float64 + GoalSoC float64 + GoalClock int // minutes after local midnight + GoalDays uint8 + History int +} + +func (b *chainBox) seed(t *testing.T) chainSnapshot { + t.Helper() + for _, req := range []struct { + method, path string + body any + }{ + {"POST", "/api/mode", map[string]any{"mode": "peak_shaving"}}, + {"POST", "/api/target", map[string]any{"grid_target_w": -125}}, + {"PUT", "/api/loadpoints/garage/schedule", map[string]any{"soc": 0.8, "time_of_day_min": 420, "recurring": true, "days": 31}}, + } { + var out map[string]any + if status := b.call(t, req.method, req.path, req.body, &out); status != http.StatusOK { + t.Fatalf("%s %s = %d %v", req.method, req.path, status, out) + } + } + if !chainWait(60*time.Second, func() bool { return b.history(t) >= 5 }) { + t.Fatalf("history has %d rows after a minute", b.history(t)) + } + snap := b.snapshot(t) + want := chainSnapshot{Mode: "peak_shaving", GridTargetW: -125, GoalSoC: 0.8, GoalClock: 420, GoalDays: 31, History: snap.History} + if snap != want { + t.Fatalf("seeded %+v, read back %+v", want, snap) + } + return snap +} + +// check compares the stored state with want. History only grows. +func (b *chainBox) check(t *testing.T, step string, want chainSnapshot) chainSnapshot { + t.Helper() + got := b.snapshot(t) + if got.History < want.History { + t.Fatalf("%s: history fell from %d to %d rows", step, want.History, got.History) + } + want.History = got.History + if got != want { + t.Fatalf("%s: stored state changed\n got %+v\nwant %+v", step, got, want) + } + return got +} + +func (b *chainBox) snapshot(t *testing.T) chainSnapshot { + t.Helper() + var mode struct { + Mode string `json:"mode"` + GridTargetW float64 `json:"grid_target_w"` + } + if status := b.call(t, "GET", "/api/mode", nil, &mode); status != http.StatusOK { + t.Fatalf("GET /api/mode = %d", status) + } + var lps struct { + Loadpoints []struct { + Schedule struct { + SoC float64 `json:"soc"` + Clock int `json:"time_of_day_min"` + Days uint8 `json:"days"` + } `json:"schedule"` + } `json:"loadpoints"` + } + if status := b.call(t, "GET", "/api/loadpoints", nil, &lps); status != http.StatusOK || len(lps.Loadpoints) != 1 { + t.Fatalf("GET /api/loadpoints = %d %+v", status, lps) + } + goal := lps.Loadpoints[0].Schedule + return chainSnapshot{Mode: mode.Mode, GridTargetW: mode.GridTargetW, GoalSoC: goal.SoC, + GoalClock: goal.Clock, GoalDays: goal.Days, History: b.history(t)} +} + +// history counts the samples recorded in the last hour; a chain run is shorter. +func (b *chainBox) history(t *testing.T) int { + t.Helper() + var out struct { + Items []struct { + N float64 `json:"n"` + } `json:"items"` + } + if status := b.call(t, "GET", "/api/history?range=1h&points=5000", nil, &out); status != http.StatusOK { + t.Fatalf("GET /api/history = %d %s", status, b.last) + } + n := 0 + for _, item := range out.Items { + n += max(1, int(item.N)) + } + return n +} + +// update asks the running Core to install target, as the Settings page does, +// and waits for Core to stop so the launcher can start it. +func (b *chainBox) update(t *testing.T, target string, retry bool) { + t.Helper() + var info struct { + Latest string `json:"latest"` + UpdateAvailable bool `json:"update_available"` + Err string `json:"err"` + } + if status := b.call(t, "GET", "/api/version/check?force=1", nil, &info); status != http.StatusOK || info.Latest != target || !info.UpdateAvailable { + t.Fatalf("version check = %d %+v, want %s available", status, info, target) + } + var out map[string]any + if status := b.call(t, "POST", "/api/version/update", map[string]any{"retry": retry}, &out); status != http.StatusAccepted { + t.Fatalf("POST /api/version/update = %d %v", status, out) + } + b.waitExit(t, 3*time.Minute) +} + +// chainBox plays systemd for one install root: it starts the launcher, which +// execs Core in the same process, and stops or kills it. +type chainBox struct { + launcher, root, data, config, drivers, api string + mirror *releaseMirror + log *os.File + cmd *exec.Cmd + done chan struct{} + last string // the last reply body +} + +func newChainBox(t *testing.T, launcher string, mirror *releaseMirror) *chainBox { + t.Helper() + dir := t.TempDir() + b := &chainBox{launcher: launcher, root: filepath.Join(dir, "opt"), data: filepath.Join(dir, "var"), mirror: mirror} + b.config = filepath.Join(b.data, "config.yaml") + b.drivers = filepath.Join(b.data, "drivers") + for _, d := range []string{b.root, b.drivers} { + if err := os.MkdirAll(d, 0o755); err != nil { + t.Fatal(err) + } + } + port := chainFreePort(t) + b.api = fmt.Sprintf("http://127.0.0.1:%d", port) + cfg := map[string]any{ + "site": map[string]any{"name": "update-chain", "control_interval_s": 2, "grid_target_w": 0, + "grid_tolerance_w": 42, "watchdog_timeout_s": 60}, + "fuse": map[string]any{"max_amps": 16, "phases": 3, "voltage": 230}, + "drivers": []map[string]any{{"name": "meter", "lua": "drivers/chain_meter.lua", "is_site_meter": true, + "capabilities": map[string]any{"standalone": true}}}, + "loadpoints": []map[string]any{{"id": "garage", "driver_name": "chain-charger", "min_charge_w": 4140, "max_charge_w": 11040, "vehicle_capacity_wh": 60000}}, + "api": map[string]any{"port": port}, + "ocpp": map[string]any{"enabled": false}, + "state": map[string]any{"path": filepath.Join(b.data, "state.db")}, + "app_link": map[string]any{"enabled": false}, + "fleet_ping": map[string]any{"enabled": false}, + "device_repository": map[string]any{"enabled": false}, + } + raw, err := json.MarshalIndent(cfg, "", " ") // JSON is valid YAML + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(b.config, raw, 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(b.drivers, "chain_meter.lua"), []byte(chainMeterLua), 0o644); err != nil { + t.Fatal(err) + } + if b.log, err = os.Create(filepath.Join(dir, "box.log")); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { + if b.cmd != nil { + _ = b.cmd.Process.Kill() + <-b.done + } + b.log.Close() + if t.Failed() { + if tail, err := os.ReadFile(b.log.Name()); err == nil { + if len(tail) > 8000 { + tail = tail[len(tail)-8000:] + tail = tail[bytes.IndexByte(tail, '\n')+1:] + } + t.Logf("box log tail:\n%s", tail) + } + } + }) + return b +} + +// chainMeterLua is a site meter that needs no device: a steady 1.2 kW import. +const chainMeterLua = `DRIVER = { host_api_min = 1, host_api_max = 1, id = "chain_meter", name = "Update chain meter", + version = "1.0.0", protocols = { "standalone" }, capabilities = { "meter" }, read_only = true } +function driver_init(config) host.set_make("FTW test"); host.set_sn("chain-meter-1"); return true end +function driver_poll() + host.emit("meter", { w = 1200, hz = 50, l1_v = 230, l2_v = 230, l3_v = 230, l1_a = 2, l2_a = 2, l3_a = 1.2, + l1_w = 460, l2_w = 460, l3_w = 280, import_wh = 1000, export_wh = 0 }) + return 1000 +end +function driver_command(action, value, context) return action == "init" or action == "deinit" end +function driver_default_mode() end +` + +func (b *chainBox) launcherCmd(t *testing.T, args ...string) { + t.Helper() + cmd := exec.Command(b.launcher, append([]string{"-root", b.root}, args...)...) + if out, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("ftw-launcher %v: %v\n%s", args, err, out) + } +} + +func (b *chainBox) start(t *testing.T) { + t.Helper() + if b.cmd != nil { + t.Fatal("start while Core runs") + } + cmd := exec.Command(b.launcher, "-root", b.root, "-config", b.config, "-user-drivers", b.drivers) + cmd.Dir = b.data + cmd.Env = append(os.Environ(), "FTW_RELEASE_MIRROR="+b.mirror.srv.URL) + cmd.Stdout, cmd.Stderr = b.log, b.log + if err := cmd.Start(); err != nil { + t.Fatal(err) + } + done := make(chan struct{}) + go func() { _ = cmd.Wait(); close(done) }() + b.cmd, b.done = cmd, done +} + +func (b *chainBox) stop(t *testing.T, sig syscall.Signal) { + t.Helper() + if err := b.cmd.Process.Signal(sig); err != nil { + t.Fatal(err) + } + b.waitExit(t, 60*time.Second) +} + +func (b *chainBox) waitExit(t *testing.T, timeout time.Duration) { + t.Helper() + select { + case <-b.done: + b.cmd = nil + case <-time.After(timeout): + t.Fatalf("Core did not stop within %s", timeout) + } +} + +// waitVersion waits until the full API of tag answers. +func (b *chainBox) waitVersion(t *testing.T, tag string) { + t.Helper() + var last string + ok := chainWait(3*time.Minute, func() bool { + select { + case <-b.done: + t.Fatalf("Core exited while waiting for %s", tag) + default: + } + var status struct { + Version string `json:"version"` + } + if b.call(t, "GET", "/api/status", nil, &status) == http.StatusOK { + last = status.Version + } + return last == tag + }) + if !ok { + t.Fatalf("Core never served %s (last %q)", tag, last) + } +} + +func (b *chainBox) waitCommitted(t *testing.T, current, previous string) { + t.Helper() + var s nativeupdate.Slots + if !chainWait(time.Minute, func() bool { + s = b.slots(t) + return s.Current == current && s.Trial == "" + }) { + t.Fatalf("trial %s never committed: %+v", current, s) + } + if s.Previous != previous || s.Next != "" || s.Probation == nil || s.Probation.Tag != current { + t.Fatalf("committed %s: slots %+v probation %+v", current, s, s.Probation) + } +} + +func (b *chainBox) slots(t *testing.T) nativeupdate.Slots { + t.Helper() + s, err := (nativeupdate.Manager{Root: b.root}).Read() + if err != nil { + t.Fatal(err) + } + return s +} + +// wantSlots compares the release slots, leaving probation to the caller. +func (b *chainBox) wantSlots(t *testing.T, step string, want nativeupdate.Slots) { + t.Helper() + got := b.slots(t) + got.Probation = nil + if got != want { + t.Fatalf("%s: slots %+v, want %+v", step, got, want) + } +} + +func (b *chainBox) call(t *testing.T, method, path string, body, out any) int { + t.Helper() + var reader io.Reader + if body != nil { + raw, err := json.Marshal(body) + if err != nil { + t.Fatal(err) + } + reader = bytes.NewReader(raw) + } + req, err := http.NewRequest(method, b.api+path, reader) + if err != nil { + t.Fatal(err) + } + req.Header.Set("Content-Type", "application/json") + resp, err := (&http.Client{Timeout: 30 * time.Second}).Do(req) + if err != nil { + return 0 + } + defer resp.Body.Close() + raw, _ := io.ReadAll(resp.Body) + b.last = string(raw) + if out != nil && json.Unmarshal(raw, out) != nil && resp.StatusCode == http.StatusOK { + t.Fatalf("%s %s: unreadable reply %q", method, path, raw) + } + return resp.StatusCode +} + +// releaseMirror serves releases the way GitHub does for FTW_RELEASE_MIRROR: +// the list at /releases and the assets at /download//. The list holds +// only the newest release, which is all a channel check reads. +type releaseMirror struct { + srv *httptest.Server + dir string + mu sync.Mutex + newest string +} + +func newReleaseMirror(t *testing.T) *releaseMirror { + m := &releaseMirror{dir: t.TempDir()} + mux := http.NewServeMux() + mux.HandleFunc("GET /releases", func(w http.ResponseWriter, _ *http.Request) { + m.mu.Lock() + tag := m.newest + m.mu.Unlock() + archive := "ftw-linux-" + runtime.GOARCH + ".tar.gz" + list := []map[string]any{{"tag_name": tag, "prerelease": strings.Contains(tag, "-beta."), + "published_at": time.Now().UTC(), "body": "", + "assets": []map[string]string{{"name": archive}, {"name": archive + ".sha256"}}}} + _ = json.NewEncoder(w).Encode(list) + }) + mux.Handle("GET /download/", http.StripPrefix("/download/", http.FileServer(http.Dir(m.dir)))) + m.srv = httptest.NewServer(mux) + t.Cleanup(m.srv.Close) + return m +} + +func (m *releaseMirror) publish(tag string) { + m.mu.Lock() + defer m.mu.Unlock() + m.newest = tag +} + +func (m *releaseMirror) files(tag string) (archive, checksum string) { + archive = filepath.Join(m.dir, tag, "ftw-linux-"+runtime.GOARCH+".tar.gz") + return archive, archive + ".sha256" +} + +func (m *releaseMirror) schema(tag string) string { + raw, _ := os.ReadFile(filepath.Join(m.dir, tag, "schema")) + return string(raw) +} + +// add packs a release as scripts/package-linux.py does, with this host's +// binaries. tree supplies web/ and state-schema.json; drivers come from repo. +func (m *releaseMirror) add(t *testing.T, repo, tree, tag, ftw, launcher string) { + t.Helper() + var schema struct { + Version int `json:"version"` + } + raw, err := os.ReadFile(filepath.Join(tree, "state-schema.json")) + if err == nil { + err = json.Unmarshal(raw, &schema) + } + if err != nil || schema.Version <= 0 { + t.Fatalf("%s: state-schema.json: %v", tag, err) + } + dir := filepath.Join(m.dir, tag) + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "schema"), []byte(fmt.Sprint(schema.Version)), 0o644); err != nil { + t.Fatal(err) + } + var buf bytes.Buffer + gz := gzip.NewWriter(&buf) + tw := tar.NewWriter(gz) + add := func(name string, mode int64, data []byte) { + if err := tw.WriteHeader(&tar.Header{Name: name, Mode: mode, Size: int64(len(data)), Typeflag: tar.TypeReg}); err != nil { + t.Fatal(err) + } + if _, err := tw.Write(data); err != nil { + t.Fatal(err) + } + } + addFile := func(name, path string, mode int64) { + data, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + add(name, mode, data) + } + addFile("ftw", ftw, 0o755) + addFile("ftw-launcher", launcher, 0o755) + addFile("drivers/BUNDLED_SOURCE.json", filepath.Join(repo, "drivers", "BUNDLED_SOURCE.json"), 0o644) + addFile("state-schema.json", filepath.Join(tree, "state-schema.json"), 0o644) + add("optimizer/native/bundle/manifest.json", 0o644, []byte("{}\n")) + version, _ := json.Marshal(map[string]any{"version": tag, "arch": runtime.GOARCH, "state_schema": schema.Version}) + add("release-version.json", 0o644, append(version, '\n')) + web := filepath.Join(tree, "web") + if err := filepath.WalkDir(web, func(path string, d fs.DirEntry, err error) error { + if err != nil || d.IsDir() { + return err + } + rel, err := filepath.Rel(web, path) + if err == nil && d.Type().IsRegular() { + addFile("web/"+filepath.ToSlash(rel), path, 0o644) + } + return err + }); err != nil { + t.Fatal(err) + } + if err := tw.Close(); err != nil { + t.Fatal(err) + } + if err := gz.Close(); err != nil { + t.Fatal(err) + } + archive, checksum := m.files(tag) + if err := os.WriteFile(archive, buf.Bytes(), 0o644); err != nil { + t.Fatal(err) + } + sum := sha256.Sum256(buf.Bytes()) + line := hex.EncodeToString(sum[:]) + " " + filepath.Base(archive) + "\n" + if err := os.WriteFile(checksum, []byte(line), 0o644); err != nil { + t.Fatal(err) + } +} + +func chainBuild(t *testing.T, module, pkg, version, out string) string { + t.Helper() + ldflags := "-s -w" + if version != "" { + ldflags += " -X main.Version=" + version + } + cmd := exec.Command("go", "build", "-trimpath", "-tags=netgo,osusergo", "-ldflags", ldflags, "-o", out, pkg) + cmd.Dir = module + cmd.Env = append(os.Environ(), "CGO_ENABLED=0") + if raw, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("build %s %s: %v\n%s", pkg, version, err, raw) + } + return out +} + +// exportRef writes the parts of ref that a release needs into dest. +func exportRef(t *testing.T, repo, ref, dest string) { + t.Helper() + cmd := exec.Command("git", "-C", repo, "archive", "--format=tar", ref, "go", "web", "state-schema.json") + raw, err := cmd.Output() + if err != nil { + t.Fatalf("git archive %s: %v", ref, err) + } + tr := tar.NewReader(bytes.NewReader(raw)) + for { + h, err := tr.Next() + if err == io.EOF { + return + } + if err != nil { + t.Fatal(err) + } + path := filepath.Join(dest, filepath.FromSlash(h.Name)) + switch h.Typeflag { + case tar.TypeDir: + err = os.MkdirAll(path, 0o755) + case tar.TypeReg: + if err = os.MkdirAll(filepath.Dir(path), 0o755); err == nil { + var data []byte + if data, err = io.ReadAll(tr); err == nil { + err = os.WriteFile(path, data, fs.FileMode(h.Mode)&0o755|0o644) + } + } + } + if err != nil { + t.Fatal(err) + } + } +} + +func chainFreePort(t *testing.T) int { + t.Helper() + l, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + defer l.Close() + return l.Addr().(*net.TCPAddr).Port +} + +func chainWait(timeout time.Duration, ok func() bool) bool { + deadline := time.Now().Add(timeout) + for time.Now().Before(deadline) { + if ok() { + return true + } + time.Sleep(250 * time.Millisecond) + } + return ok() +} + +func envOr(key, fallback string) string { + if v := os.Getenv(key); v != "" { + return v + } + return fallback +} diff --git a/go/cmd/ftw/app_link.go b/go/cmd/ftw/app_link.go index 2b9e58206..61e3cf1d6 100644 --- a/go/cmd/ftw/app_link.go +++ b/go/cmd/ftw/app_link.go @@ -822,6 +822,7 @@ func startAppLink( Modes: modes, Plans: plans, PlannerPrefs: gateway, + TimeZones: gateway, History: history, Prices: priceReader, Loadpoints: loadpoints, @@ -936,6 +937,16 @@ func (l *lateAPI) Apply(safetyK *float64, export *string) (appproto.PlannerPrefs return srv.ApplyPlannerPrefs(safetyK, export) } +// ProposeTimeZone stores a client's time zone through the API server, once it +// exists, for the reason Apply waits. +func (l *lateAPI) ProposeTimeZone(zone string) (appproto.TimeZoneAnswer, error) { + srv := l.srv.Load() + if srv == nil { + return appproto.TimeZoneAnswer{}, errors.New("the box is still starting") + } + return srv.ProposeTimeZone(zone) +} + func (l *lateAPI) Route(r *http.Request) apiauth.RouteFacts { srv := l.srv.Load() if srv == nil { diff --git a/go/cmd/ftw/battery_soc_max_test.go b/go/cmd/ftw/battery_soc_max_test.go new file mode 100644 index 000000000..b6d5243fa --- /dev/null +++ b/go/cmd/ftw/battery_soc_max_test.go @@ -0,0 +1,122 @@ +package main + +import ( + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/control" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// The planner's earlier charge budget cannot override the owner's SoC limit. +// No solar or export is needed to reproduce the 10 kW order at 90%. +func TestConfiguredSoCMaxStopsDispatch(t *testing.T) { + for _, mode := range []control.Mode{control.ModeCharge, control.ModePlannerArbitrage} { + t.Run(string(mode), func(t *testing.T) { + cfg := parseBatteryLimitConfig(t, "", " soc_max: 0.9\n max_charge_w: 10000\n") + ctrl := newControlStateFromConfig(cfg) + ctrl.Mode = mode + ctrl.SlewEnabled = false + ctrl.UseEnergyDispatch = true + ctrl.SlotDirective = func(now time.Time) (control.SlotDirective, bool) { + return control.SlotDirective{ + SlotStart: now, SlotEnd: now.Add(15 * time.Minute), + BatteryEnergyWh: 2500, PlannedGridW: 11000, HasPlannedGridW: true, + }, true + } + tel := batteryLimitStore(1000) + soc := 0.9 + tel.Update("battery", telemetry.DerBattery, 0, &soc, nil) + got := control.ComputeDispatch(tel, ctrl, map[string]float64{"battery": 15000}, 40000) + if len(got) != 1 || got[0].TargetW != 0 { + t.Fatalf("soc_max=0.9 at 90%% sent %+v, want one 0 W target", got) + } + }) + } +} + +func TestSoCMaxConfigPrecedenceAndReload(t *testing.T) { + cfg := parseBatteryLimitConfig(t, "", " soc_max: 0.9\n") + cfg.Planner = &config.Planner{SoCMax: .8} + ctrl := newControlStateFromConfig(cfg) + ctrl.Mode, ctrl.SlewEnabled = control.ModeCharge, false + tel := batteryLimitStore(1000) + capacities := map[string]float64{"battery": 10000} + check := func(soc float64, charging bool) { + t.Helper() + tel.Update("battery", telemetry.DerBattery, 0, &soc, nil) + got := control.ComputeDispatch(tel, ctrl, capacities, 40000) + if len(got) != 1 || (got[0].TargetW > 0) != charging { + t.Fatalf("SoC %.2f: got %+v, want charging=%v", soc, got, charging) + } + } + check(.85, true) // The battery's 90% overrides the planner's 80%. + check(.9, false) + // The config watcher replaces the limits through this same function. + cfg.Batteries = nil + ctrl.DriverLimits = driverLimitsFrom(cfg.ModbusControlDrivers(), cfg.Batteries, cfg.Planner) + check(.85, false) // With no battery override, the planner's 80% applies. + cfg.Planner = nil + ctrl.DriverLimits = driverLimitsFrom(cfg.ModbusControlDrivers(), cfg.Batteries, cfg.Planner) + check(.99, true) // No configured ceiling keeps the 100% default. + check(1, false) + zero := 0.0 + cfg.Batteries = map[string]config.Battery{"battery": {SoCMax: &zero}} + ctrl.DriverLimits = driverLimitsFrom(cfg.ModbusControlDrivers(), cfg.Batteries, cfg.Planner) + check(.5, false) // Validate drops a configured zero; one that bypasses it still blocks charging. +} + +// Settings saved a blank Max SoC as 0. Loading that config must not leave +// a 0 % ceiling that blocks every charge order, manual Charge included. +func TestSavedZeroSoCMaxDoesNotBlockCharge(t *testing.T) { + cfg := parseBatteryLimitConfig(t, "", " soc_min: 0\n soc_max: 0\n max_charge_w: 0\n max_discharge_w: 0\n") + if cfg.Batteries["battery"].SoCMax != nil { + t.Fatalf("soc_max = %v after load, want unset", *cfg.Batteries["battery"].SoCMax) + } + cfg.Planner = &config.Planner{Enabled: true, SoCMax: .9} + ctrl := newControlStateFromConfig(cfg) + ctrl.Mode, ctrl.SlewEnabled = control.ModeCharge, false + tel := batteryLimitStore(1000) + capacities := map[string]float64{"battery": 10000} + for _, tc := range []struct { + soc float64 + charging bool + }{{.58, true}, {.9, false}} { + soc := tc.soc + tel.Update("battery", telemetry.DerBattery, 0, &soc, nil) + got := control.ComputeDispatch(tel, ctrl, capacities, 40000) + if len(got) != 1 || (got[0].TargetW > 0) != tc.charging { + t.Fatalf("SoC %.2f: got %+v, want charging=%v under the planner's 90%%", soc, got, tc.charging) + } + } +} + +// Dispatch keeps the higher of the battery's and the running planner's +// soc_min. The planner's unset soc_min means its 10 % default. +func TestSoCMinReachesDispatch(t *testing.T) { + f := func(v float64) *float64 { return &v } + for _, tc := range []struct { + name string + battery *float64 + planner *config.Planner + want float64 + }{ + {"no planner, no battery floor", nil, nil, 0}, + {"planner off", nil, &config.Planner{SoCMin: 0.2}, 0}, + {"planner default", nil, &config.Planner{Enabled: true}, 0.10}, + {"planner set", nil, &config.Planner{Enabled: true, SoCMin: 0.15}, 0.15}, + {"battery higher", f(0.25), &config.Planner{Enabled: true, SoCMin: 0.15}, 0.25}, + {"planner higher", f(0.05), &config.Planner{Enabled: true, SoCMin: 0.15}, 0.15}, + {"battery only", f(0.2), nil, 0.2}, + } { + t.Run(tc.name, func(t *testing.T) { + cfg := parseBatteryLimitConfig(t, "", " max_charge_w: 5000\n") + cfg.Batteries = map[string]config.Battery{"battery": {SoCMin: tc.battery}} + got := driverLimitsFrom(cfg.ModbusControlDrivers(), cfg.Batteries, tc.planner)["battery"].SoCMin + if got != tc.want { + t.Fatalf("SoCMin = %v, want %v", got, tc.want) + } + }) + } +} diff --git a/go/cmd/ftw/bootstrap.go b/go/cmd/ftw/bootstrap.go index 3b7e90702..6e46540d6 100644 --- a/go/cmd/ftw/bootstrap.go +++ b/go/cmd/ftw/bootstrap.go @@ -102,7 +102,19 @@ func runBootstrap(configPath, webDir, driverDir string) { } writeBootstrapJSON(w, http.StatusOK, chargers) }) - mux.HandleFunc("POST /api/config", func(w http.ResponseWriter, r *http.Request) { + mux.HandleFunc("POST /api/config", bootstrapConfig(configPath, saveInitial, func() error { return execCurrentProcess(nil) })) + + srv := newHTTPServer(":8080", secureBootstrapMutations(mux)) + if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed { + slog.Error("bootstrap server", "err", err) + os.Exit(1) + } +} + +// bootstrapConfig saves the wizard's settings, with the house password it +// asked for, then restarts into the normal server. +func bootstrapConfig(configPath string, save func(*config.Config) error, restart func() error) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { body, err := io.ReadAll(io.LimitReader(r.Body, 1<<20)) r.Body.Close() if err != nil { @@ -118,23 +130,21 @@ func runBootstrap(configPath, webDir, driverDir string) { writeBootstrapJSON(w, http.StatusUnprocessableEntity, map[string]string{"error": err.Error()}) return } - if err := saveInitial(&cfg); err != nil { + if err := applyInitialHousePassword(r, body, &cfg); err != nil { + writeBootstrapJSON(w, http.StatusBadRequest, map[string]string{"error": err.Error()}) + return + } + if err := save(&cfg); err != nil { writeBootstrapJSON(w, http.StatusInternalServerError, map[string]string{"error": "write config: " + err.Error()}) return } slog.Info("config written by setup wizard — restarting", "path", configPath) writeBootstrapJSON(w, http.StatusOK, map[string]string{"status": "ok", "restart": "true"}) go func() { - if err := execCurrentProcess(nil); err != nil { + if err := restart(); err != nil { slog.Error("bootstrap re-exec failed", "err", err) } }() - }) - - srv := newHTTPServer(":8080", secureBootstrapMutations(mux)) - if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed { - slog.Error("bootstrap server", "err", err) - os.Exit(1) } } @@ -165,7 +175,7 @@ func bootstrapAssistant(save func(*config.Config) error, restart func() error) h Confirmed bool `json:"confirmed"` Password string `json:"password"` } - dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1024)) + dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)) dec.DisallowUnknownFields() if err := dec.Decode(&q); err != nil || !q.Confirmed { writeBootstrapJSON(w, http.StatusBadRequest, map[string]string{"error": "confirm starting an empty measurement installation"}) @@ -199,6 +209,25 @@ func bootstrapAssistant(save func(*config.Config) error, restart func() error) h } } +// applyInitialHousePassword locks a new box with the house password the +// wizard asked for, beside its settings. Home Assistant signs its users in +// itself and sends none. A script that posts settings without one leaves the +// lock off, as before; the page asks for a password later. +func applyInitialHousePassword(r *http.Request, body []byte, cfg *config.Config) error { + var q struct { + HousePassword string `json:"house_password"` + } + if err := json.Unmarshal(body, &q); err != nil || q.HousePassword == "" || api.HomeAssistantIngress(r) { + return nil + } + hash, err := api.HashInitialLANPassword(q.HousePassword) + if err != nil { + return err + } + cfg.API.LANAuth, cfg.LANPasswordHash = true, hash + return nil +} + // Bootstrap needs the same country/zone choices before any config exists. func bootstrapPriceZones(w http.ResponseWriter, _ *http.Request) { zones := prices.Zones() diff --git a/go/cmd/ftw/bootstrap_assistant_test.go b/go/cmd/ftw/bootstrap_assistant_test.go index 2d0ebf776..67708a812 100644 --- a/go/cmd/ftw/bootstrap_assistant_test.go +++ b/go/cmd/ftw/bootstrap_assistant_test.go @@ -1,6 +1,7 @@ package main import ( + "encoding/json" "net/http/httptest" "os" "path/filepath" @@ -133,3 +134,80 @@ func TestHomeAssistantBootstrapNeedsNoSecondPassword(t *testing.T) { t.Fatal("HA setup invented a password or equipment") } } + +// The wizard's own settings save carries the house password it asked for, +// so a new box starts locked. Home Assistant signs its users in itself, and +// a script that sends none keeps the lock off as before. +func TestBootstrapWizardSavesTheHousePassword(t *testing.T) { + t.Setenv("FTW_API_TOKEN", "") + const settings = `"site":{"name":"Home","control_interval_s":5,"grid_tolerance_w":42,"watchdog_timeout_s":60,"smoothing_alpha":0.3,"gain":0.5,"slew_rate_w":500,"min_dispatch_interval_s":5},"fuse":{"max_amps":16,"phases":3,"voltage":230},"drivers":[],"api":{"port":8080}` + for _, tc := range []struct { + name, body string + want int + locked bool + }{ + {"with a password", `{` + settings + `,"house_password":"first-owner-password"}`, 200, true}, + {"a short one, the owner's choice", `{` + settings + `,"house_password":"1234"}`, 200, true}, + {"too long", `{` + settings + `,"house_password":"` + strings.Repeat("x", 513) + `"}`, 400, false}, + {"from a script without one", `{` + settings + `}`, 200, false}, + {"through Home Assistant", `{` + settings + `,"house_password":"first-owner-password"}`, 200, false}, + } { + t.Run(tc.name, func(t *testing.T) { + path := filepath.Join(t.TempDir(), "config.yaml") + restarted := make(chan struct{}, 1) + handler := secureBootstrapMutations(bootstrapConfig(path, func(cfg *config.Config) error { return saveInitialConfig(path, cfg) }, + func() error { restarted <- struct{}{}; return nil })) + req := httptest.NewRequest("POST", "http://box.local/api/config", strings.NewReader(tc.body)) + req.RemoteAddr = "192.168.1.12:1234" + req.Header.Set("Content-Type", "application/json") + req.Header.Set("Origin", "http://box.local") + if tc.name == "through Home Assistant" { + handler = api.WithHomeAssistantIngress(handler) + req.RemoteAddr = "172.30.32.2:1234" + req.Header.Del("Origin") + req.Header.Set("X-Ingress-Path", "/api/hassio_ingress/test") + req.Header.Set("X-Remote-User-Id", "owner") + } + w := httptest.NewRecorder() + handler.ServeHTTP(w, req) + if w.Code != tc.want { + t.Fatalf("%d %s", w.Code, w.Body.String()) + } + if tc.want != 200 { + if _, err := os.Stat(path); !os.IsNotExist(err) { + t.Fatal("a refused save wrote settings") + } + return + } + <-restarted + cfg, err := config.Load(path) + if err != nil { + t.Fatal(err) + } + if cfg.API.LANAuth != tc.locked || (cfg.LANPasswordHash != "") != tc.locked { + t.Fatalf("lan_auth %v, hash set %v; want %v", cfg.API.LANAuth, cfg.LANPasswordHash != "", tc.locked) + } + raw, _ := os.ReadFile(path) + if strings.Contains(string(raw), "first-owner-password") { + t.Fatal("the locator holds the house password") + } + var sent struct { + HousePassword string `json:"house_password"` + } + _ = json.Unmarshal([]byte(tc.body), &sent) + if tc.locked && !api.VerifyStoredLANSecret(mustOpenState(t, cfg.ConfigDatabase), sent.HousePassword) { + t.Fatal("the stored hash does not match the password the wizard sent") + } + }) + } +} + +func mustOpenState(t *testing.T, path string) *state.Store { + t.Helper() + st, err := state.Open(path) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = st.Close() }) + return st +} diff --git a/go/cmd/ftw/charge_restart_test.go b/go/cmd/ftw/charge_restart_test.go new file mode 100644 index 000000000..c5feb3fb4 --- /dev/null +++ b/go/cmd/ftw/charge_restart_test.go @@ -0,0 +1,83 @@ +package main + +import ( + "context" + "path/filepath" + "sync" + "testing" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/control" + "github.com/srcfl/ftw/go/internal/state" +) + +func TestRestartEndsManualCharge(t *testing.T) { + for _, tc := range []struct { + name string + planner *config.Planner + storedExport string + want control.Mode + }{ + {"planner enabled", &config.Planner{Enabled: true}, "", control.ModePlannerPassiveArbitrage}, + {"export allowed", &config.Planner{Enabled: true}, "allowed", control.ModePlannerArbitrage}, + {"export refused", &config.Planner{Enabled: true, BatteryExport: "allowed"}, "not_allowed", control.ModePlannerPassiveArbitrage}, + {"export unknown", &config.Planner{Enabled: true, BatteryExport: "allowed"}, "unknown", control.ModePlannerPassiveArbitrage}, + {"export from config", &config.Planner{Enabled: true, BatteryExport: "allowed"}, "", control.ModePlannerArbitrage}, + {"invalid stored permission", &config.Planner{Enabled: true, BatteryExport: "allowed"}, "invalid", control.ModePlannerArbitrage}, + {"no export permission", &config.Planner{Enabled: true, BatteryExport: "invalid"}, "invalid", control.ModePlannerPassiveArbitrage}, + {"planner disabled", &config.Planner{Enabled: false}, "allowed", control.ModeIdle}, + {"no planner", nil, "", control.ModeIdle}, + } { + t.Run(tc.name, func(t *testing.T) { + cfg := parseBatteryLimitConfig(t, "", " max_charge_w: 10000\n soc_max: 0.9\n") + cfg.Planner = tc.planner + path := filepath.Join(t.TempDir(), "state.db") + st, err := state.Open(path) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = st.Close() }) + if err := st.SaveConfig(config.StateKeyBatteryExport, tc.storedExport); err != nil { + t.Fatal(err) + } + // A new, explicit Charge still works during this process. + ctrl := newControlStateFromConfig(cfg) + modes := appModes{ctrl: ctrl, ctrlMu: &sync.Mutex{}, state: st} + if err := modes.SetMode(context.Background(), control.ModeCharge); err != nil { + t.Fatal(err) + } + if targets := control.ComputeDispatch(batteryLimitStore(1000), ctrl, map[string]float64{"battery": 15000}, 40000); len(targets) != 1 || targets[0].TargetW != 10000 { + t.Fatalf("explicit Charge: %+v, want 10000 W", targets) + } + for boot := 1; boot <= 2; boot++ { + if err := st.Close(); err != nil { + t.Fatal(err) + } + st, err = state.Open(path) + if err != nil { + t.Fatal(err) + } + ctrl = newControlStateFromConfig(cfg) + restoreStoredMode(ctrl, st, cfg) + if ctrl.Mode != tc.want { + t.Fatalf("boot %d: mode = %q, want %q", boot, ctrl.Mode, tc.want) + } + if saved, _ := st.LoadConfig("mode"); saved != string(tc.want) { + t.Fatalf("boot %d: stored mode = %q, want %q", boot, saved, tc.want) + } + if tc.name == "planner enabled" || tc.name == "export unknown" { + if export, _ := st.LoadConfig(config.StateKeyBatteryExport); export != "unknown" { + t.Fatalf("boot %d: unknown export permission became %q", boot, export) + } + } + // Before the fresh plan arrives, import without solar must + // not resume the earlier full-power charging order. + for _, target := range control.ComputeDispatch(batteryLimitStore(1000), ctrl, map[string]float64{"battery": 15000}, 40000) { + if target.TargetW > 0 { + t.Fatalf("boot %d: charging before a fresh plan: %+v", boot, target) + } + } + } + }) + } +} diff --git a/go/cmd/ftw/control_state.go b/go/cmd/ftw/control_state.go index f97581f66..7dc634de4 100644 --- a/go/cmd/ftw/control_state.go +++ b/go/cmd/ftw/control_state.go @@ -21,19 +21,19 @@ func newControlStateFromConfig(cfg *config.Config) *control.State { ctrl.SlewEnabled = *cfg.Site.SlewEnabled } ctrl.MinDispatchIntervalS = cfg.Site.MinDispatchIntervalS + ctrl.ControlIntervalS = cfg.Site.ControlIntervalS ctrl.InverterGroups = inverterGroupsFrom(cfg.Drivers) ctrl.SupportsPVCurtail = supportsPVCurtailFrom(cfg.Drivers) ctrl.SolarFeedDrivers = solarFeedDriversFrom(cfg.Drivers) - ctrl.DriverLimits = driverLimitsFrom(cfg.ModbusControlDrivers(), cfg.Batteries) + ctrl.DriverLimits = driverLimitsFrom(cfg.ModbusControlDrivers(), cfg.Batteries, cfg.Planner) // Per-phase fuse params for the per-phase clamp inside applyFuseGuard // + forceFuseDischarge. Reads l1_a/l2_a/l3_a from the meter driver // when SiteFuseAmps > 0; otherwise the per-phase clamp is disabled. ctrl.SiteFuseAmps = cfg.Fuse.MaxAmps ctrl.SiteFuseVoltage = cfg.Fuse.Voltage ctrl.SiteFusePhases = cfg.Fuse.Phases - // EffectiveSafetyMarginA distinguishes nil ("unset, use default") - // from explicit 0 ("operator chose to disable"). The earlier - // `<= 0 -> default` shortcut clobbered the disable case. + // EffectiveSafetyMarginA resolves nil to the default (0: work to the + // fuse rating) and keeps any explicit value, 0 included. ctrl.SiteFuseSafetyA = cfg.Fuse.EffectiveSafetyMarginA() // PV surplus absorber underlay (opt-in). cap == 0 keeps it off. ctrl.PVSurplusAbsorbSoCCap = cfg.Site.PVSurplusAbsorbSoCCap @@ -77,14 +77,39 @@ func energyDispatchEnabled(cfg *config.Config) bool { // and weighted split equally because nothing set its weights. var removedModes = map[string]bool{"priority": true, "weighted": true} -// restoreStoredMode applies the mode saved in state.db. A removed mode becomes -// manual self-consumption and is saved back, so the stored mode matches what -// the site runs. Any other unknown value keeps the default mode, as before. -func restoreStoredMode(ctrl *control.State, st *state.Store) { +// restoreStoredMode keeps durable modes, but never resumes forced Charge. +// Charge returns to the planner, or Idle if planning is disabled. Removed +// modes become self-consumption; unknown values keep the default mode. +func restoreStoredMode(ctrl *control.State, st *state.Store, cfg *config.Config) { v, ok := st.LoadConfig("mode") if !ok { return } + if control.Mode(v) == control.ModeCharge { + next := control.ModeIdle + values := map[string]string{} + if cfg != nil && cfg.Planner != nil && cfg.Planner.Enabled { + export := config.BatteryExportUnknown + if e, valid := config.ParseBatteryExport(cfg.Planner.BatteryExport); valid { + export = e + } + storedExport, _ := st.LoadConfig(config.StateKeyBatteryExport) + if e, valid := config.ParseBatteryExport(storedExport); valid { + export = e + } + next = control.Mode(export.PlannerModeKey()) + // Keep unknown as unknown. Otherwise the later preference + // migration would infer a refusal from the new passive mode. + values[config.StateKeyBatteryExport] = string(export) + } + ctrl.Mode = next + values["mode"] = string(next) + if err := st.SaveConfigValues(values); err != nil { + slog.Warn("could not save the mode after ending manual Charge at restart", "mode", next, "err", err) + } + slog.Info("manual Charge ended at restart", "previous_mode", v, "mode", next) + return + } if m := control.Mode(v); control.IsValidMode(m) { ctrl.Mode = m return diff --git a/go/cmd/ftw/control_state_test.go b/go/cmd/ftw/control_state_test.go index e0f650357..5f4fad63d 100644 --- a/go/cmd/ftw/control_state_test.go +++ b/go/cmd/ftw/control_state_test.go @@ -167,6 +167,29 @@ func TestBatteryLimitConfigBothZeroUsesDefault(t *testing.T) { } } +// A config with no fuse.safety_margin_a reaches Core at the fuse rating, and +// an explicit value still wins. +func TestUnsetFuseMarginReachesCoreAsZero(t *testing.T) { + cfg := parseBatteryLimitConfig(t, "", " max_charge_w: 10000\n") + ctrl := newControlStateFromConfig(cfg) + if ctrl.SiteFuseSafetyA != 0 { + t.Fatalf("unset margin reached Core as %v A, want 0", ctrl.SiteFuseSafetyA) + } + // 63 A × 230 V × 3 = 43470 W is the ceiling with no margin. + if err := ctrl.SetPeakLimit(43470); err != nil { + t.Fatalf("peak limit at the fuse: %v", err) + } + if err := ctrl.SetPeakLimit(43471); err == nil { + t.Fatal("a peak limit 1 W over the fuse was accepted") + } + + half := 0.5 + cfg.Fuse.SafetyMarginA = &half + if got := newControlStateFromConfig(cfg).SiteFuseSafetyA; got != 0.5 { + t.Fatalf("explicit 0.5 A reached Core as %v A", got) + } +} + // A "NaN" saved from an MQTT command must not come back as the PI setpoint // on every boot. func TestRestoredGridTargetIgnoresNonFinite(t *testing.T) { @@ -184,10 +207,15 @@ func TestRestoredGridTargetIgnoresNonFinite(t *testing.T) { // self-consumption and store it, not keep a mode no door accepts any more. func TestRestoreStoredModeMapsRemovedModesToSelfConsumption(t *testing.T) { for stored, want := range map[string]control.Mode{ - "priority": control.ModeSelfConsumption, - "weighted": control.ModeSelfConsumption, - "planner_arbitrage": control.ModePlannerArbitrage, - "peak_shaving": control.ModePeakShaving, + "priority": control.ModeSelfConsumption, + "weighted": control.ModeSelfConsumption, + "idle": control.ModeIdle, + "self_consumption": control.ModeSelfConsumption, + "peak_shaving": control.ModePeakShaving, + "planner_self": control.ModePlannerSelf, + "planner_cheap": control.ModePlannerCheap, + "planner_passive_arbitrage": control.ModePlannerPassiveArbitrage, + "planner_arbitrage": control.ModePlannerArbitrage, } { t.Run(stored, func(t *testing.T) { st, err := state.Open(filepath.Join(t.TempDir(), "state.db")) @@ -201,7 +229,7 @@ func TestRestoreStoredModeMapsRemovedModesToSelfConsumption(t *testing.T) { ctrl := control.NewState(0, 50, "meter") ctrl.Mode = control.ModeIdle - restoreStoredMode(ctrl, st) + restoreStoredMode(ctrl, st, nil) if ctrl.Mode != want { t.Fatalf("mode = %q, want %q", ctrl.Mode, want) @@ -225,7 +253,7 @@ func TestRestoreStoredModeLeavesAnUnknownValueAlone(t *testing.T) { } ctrl := control.NewState(0, 50, "meter") - restoreStoredMode(ctrl, st) + restoreStoredMode(ctrl, st, nil) if ctrl.Mode != control.ModeSelfConsumption { t.Fatalf("mode = %q, want the default", ctrl.Mode) diff --git a/go/cmd/ftw/driver_failure_default.go b/go/cmd/ftw/driver_failure_default.go index 07e965515..a7761def8 100644 --- a/go/cmd/ftw/driver_failure_default.go +++ b/go/cmd/ftw/driver_failure_default.go @@ -5,6 +5,7 @@ import ( "encoding/json" "errors" "log/slog" + "slices" "sort" "sync" "time" @@ -105,6 +106,28 @@ type refusalState struct { // excludedAt is when the driver was last put out of dispatch, and // zero while it is still in. It dates the retry window. excludedAt time.Time + // outage marks an exclusion that the device's own outage explains: its + // reads were failing when it was excluded, a read failed before any + // good read after it, or the watchdog took it offline while it was + // out. Such a device kept the last command it accepted and every + // default write in between failed. When it reads again, update resends + // the default and lets one command through at once instead of waiting + // out driverRefusalRetryInterval. + outage bool + // readAtExclusion is the device's LastSuccess when it was excluded. A + // command can reach a dead link before the next poll does, so the + // first failed read can come just after the exclusion. A failed read + // with no good read since this time is that same outage. + readAtExclusion time.Time + // retriedAt is when update last let the driver back in for a trial. + // A trial refused while reads were fine is the device's answer, so for + // one retry interval after it a failed read does not count as an + // outage. On a flaky link that refuses for real, it would otherwise + // send a default and a trial on every good read after a failed one. + // It ends after that interval, or when the excluded action is + // accepted, so a later outage counts again even if dispatch sent the + // driver nothing in between. + retriedAt time.Time } func newDriverActuationTracker(tel *telemetry.Store) *driverActuationTracker { @@ -174,6 +197,9 @@ func (t *driverActuationTracker) recordCommandOutcomeFor(name, action string, er if st.excludedAt.IsZero() || action != st.excludedAction { delete(st.consecutive, action) } + if st.excludedAt.IsZero() && action == st.excludedAction { + st.retriedAt = time.Time{} + } if len(st.consecutive) == 0 { delete(t.refusals, name) } else { @@ -205,6 +231,8 @@ func (t *driverActuationTracker) recordCommandOutcomeFor(name, action string, er st.excludedAction = action if st.excludedAt.IsZero() { st.excludedAt = now + st.outage = t.unreachable(name) + st.readAtExclusion = t.lastRead(name) } t.refusals[name] = st t.tel.SetDriverCommandFault(name, true, st.reason) @@ -225,11 +253,31 @@ func (t *driverActuationTracker) recordCommandOutcomeFor(name, action string, er if st.consecutive[action] >= driverRefusalLimit && st.excludedAt.IsZero() { st.excludedAction = action st.excludedAt = now + st.outage = t.unreachable(name) + st.readAtExclusion = t.lastRead(name) t.tel.SetDriverCommandFault(name, true, st.reason) } t.refusals[name] = st } +// unreachable reports whether the device is out of reach right now: its +// last poll failed or the watchdog has it offline. A refusal then says the +// wire is down, not that the device turns commands away. +func (t *driverActuationTracker) unreachable(name string) bool { + h := t.tel.DriverHealth(name) + return h != nil && (h.Status == telemetry.StatusOffline || h.LastReadFailed()) +} + +// lastRead is when the device last delivered fresh telemetry, or zero if it +// never has. +func (t *driverActuationTracker) lastRead(name string) time.Time { + h := t.tel.DriverHealth(name) + if h == nil || h.LastSuccess == nil { + return time.Time{} + } + return *h.LastSuccess +} + func dispatchAction(payload []byte) string { var command struct { Action string `json:"action"` @@ -245,7 +293,9 @@ func dispatchAction(payload []byte) string { // update walks every driver that cannot actuate to its autonomous default, // once per transition, and returns the names to send it to. It also ends the -// retry window for drivers excluded long enough to deserve another try. +// retry window for drivers excluded long enough to deserve another try, and +// at once for a driver excluded during an outage that reads again; that +// driver gets its default again too. // // observeOnly drivers are never returned. A telemetry-only driver must // receive no command at all, and that includes the safe one: core has no @@ -259,33 +309,61 @@ func (t *driverActuationTracker) update(now time.Time, observeOnly map[string]bo defer t.mu.Unlock() health := t.tel.AllHealth() + // returned holds drivers back from an outage. Their default went out + // while the device could not take it, so it is sent again this tick. + var returned []string for name, st := range t.refusals { - if _, known := health[name]; !known { + h, known := health[name] + if !known { // Driver removed or restarted: its health record went with // it, and so did the fault. Start it over clean. delete(t.refusals, name) delete(t.defaulted, name) continue } - if st.excludedAt.IsZero() || now.Sub(st.excludedAt) < driverRefusalRetryInterval { + if st.excludedAt.IsZero() { + continue + } + // After the exclusion, an outage is the watchdog taking the driver + // offline, or a failed read before any good one. The second covers + // a command that hit the dead link before a poll did, in an outage + // shorter than the watchdog timeout. A device that has read fine + // since it refused keeps the full window, even if a later read + // fails, and so does one that refused a trial with good reads + // less than one retry interval after the trial began. + noReadSince := h.LastSuccess == nil || !h.LastSuccess.After(st.readAtExclusion) + recentTrial := !st.retriedAt.IsZero() && now.Sub(st.retriedAt) < driverRefusalRetryInterval + if !st.outage && (h.Status == telemetry.StatusOffline || (!recentTrial && h.LastReadFailed() && noReadSince)) { + st.outage = true + t.refusals[name] = st + } + back := st.outage && h.Status != telemetry.StatusOffline && !h.LastReadFailed() + if !back && now.Sub(st.excludedAt) < driverRefusalRetryInterval { continue } - // Retry window is up. Let one command through: a single fresh - // refusal puts the driver straight back out, an accepted one - // clears the record entirely. + // Retry window is up, or the device is back. Let one command + // through: a single fresh refusal puts the driver straight back + // out, an accepted one clears the record entirely. if st.consecutive == nil { st.consecutive = map[string]int{} } st.consecutive[st.excludedAction] = driverRefusalLimit - 1 st.excludedAt = time.Time{} + st.outage = false + st.retriedAt = now t.refusals[name] = st t.tel.SetDriverCommandFault(name, false, "") // health holds copies, so apply the release to this tick's // snapshot too — otherwise the latch below reads the driver as // still faulted and defaults a driver we just let back in. - if h, ok := health[name]; ok { - h.SetCommandFault(false, "") - health[name] = h + h.SetCommandFault(false, "") + health[name] = h + if back { + delete(t.defaulted, name) + returned = append(returned, name) + slog.Info("excluded driver reads again after an outage — resending its default and letting one command through", + "driver", name) + continue } slog.Info("driver exclusion retry window elapsed — letting one command through", "driver", name, "window", driverRefusalRetryInterval) @@ -327,6 +405,14 @@ func (t *driverActuationTracker) update(now time.Time, observeOnly map[string]bo } } + // A driver that also flags its own fault was already added above. + for _, name := range returned { + if observeOnly[name] || slices.Contains(pending, name) { + continue + } + pending = append(pending, name) + } + sort.Strings(pending) return pending } diff --git a/go/cmd/ftw/driver_failure_default_test.go b/go/cmd/ftw/driver_failure_default_test.go index 296821484..9b47b3e2e 100644 --- a/go/cmd/ftw/driver_failure_default_test.go +++ b/go/cmd/ftw/driver_failure_default_test.go @@ -294,3 +294,344 @@ func TestRemovedDriverIsForgotten(t *testing.T) { t.Error("removed driver left bookkeeping behind") } } + +// excludeDuringOutage refuses driverRefusalLimit battery commands while the +// device's reads are failing, the way a Sungrow refuses every write while it +// is unreachable, and lets the tracker send its one default. +func excludeDuringOutage(t *testing.T, tel *telemetry.Store, tracker *driverActuationTracker, sender driverCommandSender, now time.Time) { + t.Helper() + payload := []byte(`{"action":"battery","power_w":-3000}`) + tel.RecordDriverError("sungrow", "driver_poll: none of 17 modbus reads returned data") + for i := 0; i < driverRefusalLimit; i++ { + tracker.dispatchCommand(context.Background(), sender, "driver send", "sungrow", payload, time.Second, now) + } + if tel.DriverHealth("sungrow").IsOnline() { + t.Fatal("driver was not excluded") + } + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) +} + +// Random rig seeds 207532–207539 (27 Oct 2026): an inverter outage made the +// Sungrow refuse its battery command and its default, so it was excluded. +// When it came back it still ran its last forced command (187 / 3854 W in +// 207537, from 213 to 510 s) because Core neither resent the default nor +// let a command through until driverRefusalRetryInterval had passed. +func TestDriverExcludedDuringOutageGetsDefaultWhenItReadsAgain(t *testing.T) { + for _, tc := range []struct { + name string + watchdog bool + }{ + {"short outage, reads failing", false}, + {"long outage, watchdog offline", true}, + } { + t.Run(tc.name, func(t *testing.T) { + tel := pollingStore(t, "sungrow") + tracker := newDriverActuationTracker(tel) + now := time.Now() + excludeDuringOutage(t, tel, tracker, refusingSender("modbus reconnect backoff active"), now) + + now = now.Add(10 * time.Second) + if tc.watchdog { + tel.WatchdogScan(time.Nanosecond) + } + if pending := tracker.update(now, nil); len(pending) != 0 { + t.Fatalf("unreachable driver got a second default: %v", pending) + } + + // First fresh reading after the outage. + now = now.Add(20 * time.Second) + tel.RecordDriverSuccess("sungrow") + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + if !tel.DriverHealth("sungrow").IsOnline() { + t.Fatal("driver back from an outage still waits out the retry window") + } + if pending := tracker.update(now.Add(time.Second), nil); len(pending) != 0 { + t.Fatalf("returned driver got its default twice: %v", pending) + } + }) + } +} + +// A device that keeps refusing while it keeps dropping out is retried each +// time it reads again. Once it refuses while reading fine, it waits the full +// retry window like any device that answers and refuses. +func TestDriverRefusingAfterOutageKeepsRetryWindowWhileOnline(t *testing.T) { + tel := pollingStore(t, "sungrow") + tracker := newDriverActuationTracker(tel) + sender := refusingSender("modbus write refused") + payload := []byte(`{"action":"battery","power_w":-3000}`) + now := time.Now() + excludeDuringOutage(t, tel, tracker, sender, now) + + // Back, but the trial command is refused while reads fail again. + now = now.Add(5 * time.Second) + tel.RecordDriverSuccess("sungrow") + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + tel.RecordDriverError("sungrow", "driver_poll: none of 17 modbus reads returned data") + tracker.dispatchCommand(context.Background(), sender, "driver send", "sungrow", payload, time.Second, now) + if tel.DriverHealth("sungrow").IsOnline() { + t.Fatal("refused trial command left the driver in dispatch") + } + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + + // Back again: retried at once, not after the window. + now = now.Add(5 * time.Second) + tel.RecordDriverSuccess("sungrow") + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + if !tel.DriverHealth("sungrow").IsOnline() { + t.Fatal("driver back from a second outage waits out the retry window") + } + + // Now it reads fine and refuses: that is the device's answer. + tracker.dispatchCommand(context.Background(), sender, "driver send", "sungrow", payload, time.Second, now) + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + excludedAt := now + for i := 0; i < 5; i++ { + now = now.Add(time.Minute - time.Second) + // One failed read in between is not an outage. + if i == 2 { + tel.RecordDriverError("sungrow", "one modbus read timed out") + if pending := tracker.update(now, nil); len(pending) != 0 { + t.Fatalf("one failed read sent a default: %v", pending) + } + } + tel.RecordDriverSuccess("sungrow") + if pending := tracker.update(now, nil); len(pending) != 0 { + t.Fatalf("online refusing driver was defaulted again at %v: %v", now.Sub(excludedAt), pending) + } + if tel.DriverHealth("sungrow").IsOnline() { + t.Fatalf("online refusing driver let back in after %v", now.Sub(excludedAt)) + } + } + tracker.update(excludedAt.Add(driverRefusalRetryInterval), nil) + if !tel.DriverHealth("sungrow").IsOnline() { + t.Fatal("retry window elapsed and the driver was not let back in") + } +} + +// Rig seeds 207533 and 207536 (27 Oct 2026): the command hit the dead link +// before a poll did, so the last read had succeeded when the driver was +// excluded. In 207536 the outage lasted 29 s, under the 60 s watchdog, and +// the inverter held 187 / 3000 W for the full retry window after it came +// back. A failed read before any good one since the exclusion is the same +// outage. +func TestDriverExcludedJustBeforeShortOutageGetsDefaultWhenItReadsAgain(t *testing.T) { + blocked := errors.Join( + errors.New("driver command may have run: force command write failed: connection refused"), + errors.Join(drivers.ErrControlBlocked, errors.New("driver_default_mode returned false")), + ) + payload := []byte(`{"action":"battery","power_w":-3000}`) + for _, tc := range []struct { + name string + exclude func(*driverActuationTracker, time.Time) + }{ + {"control blocked", func(tracker *driverActuationTracker, now time.Time) { + tracker.recordCommandOutcomeFor("sungrow", "battery", blocked, now) + }}, + {"refused three times", func(tracker *driverActuationTracker, now time.Time) { + sender := refusingSender("force command write failed: connection refused") + for i := 0; i < driverRefusalLimit; i++ { + tracker.dispatchCommand(context.Background(), sender, "driver send", "sungrow", payload, time.Second, now) + } + }}, + } { + t.Run(tc.name, func(t *testing.T) { + tel := pollingStore(t, "sungrow") + tracker := newDriverActuationTracker(tel) + now := time.Now() + tc.exclude(tracker, now) + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + + // Polls fail for 30 s, under the watchdog timeout. + for i := 1; i <= 6; i++ { + tel.RecordDriverError("sungrow", "driver_poll: none of 17 modbus reads returned data") + tel.WatchdogScan(time.Minute) + if pending := tracker.update(now.Add(time.Duration(i)*5*time.Second), nil); len(pending) != 0 { + t.Fatalf("default sent while reads fail: %v", pending) + } + if tel.DriverHealth("sungrow").IsOnline() { + t.Fatal("driver back in dispatch while reads fail") + } + } + + now = now.Add(35 * time.Second) + tel.RecordDriverSuccess("sungrow") + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + if !tel.DriverHealth("sungrow").IsOnline() { + t.Fatalf("driver back from a short outage waits out the %v window", driverRefusalRetryInterval) + } + }) + } +} + +// A device that refuses for real on a flaky link: it is excluded with good +// reads, the next read fails, and it gets its one early trial on the good +// read after. It refuses that trial while reading fine. From then on a +// failed read between good ones is the link, not an outage, so it waits the +// full retry window instead of getting a default and a trial on every good +// read after a failed one. +func TestDriverRefusingOnFlakyLinkGetsOneEarlyTrial(t *testing.T) { + tel := pollingStore(t, "sungrow") + tracker := newDriverActuationTracker(tel) + sender := refusingSender("modbus write refused") + payload := []byte(`{"action":"battery","power_w":-3000}`) + now := time.Now() + for i := 0; i < driverRefusalLimit; i++ { + tracker.dispatchCommand(context.Background(), sender, "driver send", "sungrow", payload, time.Second, now) + } + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + + // Within one retry window of the first exclusion, the early trial + // comes 4 s in and its window has not elapsed by the end. + end := now.Add(driverRefusalRetryInterval) + trials, defaults := 0, 0 + for now.Before(end) { + now = now.Add(2 * time.Second) + tel.RecordDriverError("sungrow", "one modbus read timed out") + if pending := tracker.update(now, nil); len(pending) != 0 { + t.Fatalf("default sent while a read failed: %v", pending) + } + now = now.Add(2 * time.Second) + time.Sleep(time.Millisecond) // LastSuccess is wall time + tel.RecordDriverSuccess("sungrow") + defaults += len(tracker.update(now, nil)) + if tel.DriverHealth("sungrow").IsOnline() { + trials++ + tracker.dispatchCommand(context.Background(), sender, "driver send", "sungrow", payload, time.Second, now) + defaults += len(tracker.update(now, nil)) + } + } + if trials != 1 || defaults != 2 { + t.Fatalf("refusing driver on a flaky link got %d trials and %d defaults inside the retry window, want 1 and 2", trials, defaults) + } +} + +// A driver back from a dead-link outage may get no command for the excluded +// action for minutes (dispatch idle: 207535 showed status.dispatch empty for +// 67 s). Its next short outage, a command refused on the dead link before a +// poll fails (207536), must count as an outage again once the trial guard +// has run out, or the device holds its last command for the full window. +// Inside one retry interval of the trial the guard still holds: that is the +// flaky-link storm TestDriverRefusingOnFlakyLinkGetsOneEarlyTrial covers. +func TestSecondDeadLinkOutageAfterReturnGetsDefault(t *testing.T) { + payload := []byte(`{"action":"battery","power_w":-3000}`) + for _, tc := range []struct { + name string + accepted bool + gap time.Duration + wantBack bool + }{ + {"dispatch idle 6 min between outages", false, 6 * time.Minute, true}, + {"battery command accepted between outages", true, time.Minute, true}, + {"dispatch idle 1 min between outages", false, time.Minute, false}, + } { + t.Run(tc.name, func(t *testing.T) { + tel := pollingStore(t, "sungrow") + tracker := newDriverActuationTracker(tel) + now := time.Now() + excludeDuringOutage(t, tel, tracker, refusingSender("modbus reconnect backoff active"), now) + + now = now.Add(20 * time.Second) + time.Sleep(time.Millisecond) // LastSuccess is wall time + tel.RecordDriverSuccess("sungrow") + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + if tc.accepted { + tracker.dispatchCommand(context.Background(), &stubSender{}, "driver send", "sungrow", payload, time.Second, now) + } + for end := now.Add(tc.gap); now.Before(end); { + now = now.Add(5 * time.Second) + time.Sleep(time.Millisecond) + tel.RecordDriverSuccess("sungrow") + if p := tracker.update(now, nil); len(p) != 0 { + t.Fatalf("default while healthy: %v", p) + } + } + + dead := refusingSender("force command write failed: connection refused") + for i := 0; i < driverRefusalLimit && tel.DriverHealth("sungrow").IsOnline(); i++ { + tracker.dispatchCommand(context.Background(), dead, "driver send", "sungrow", payload, time.Second, now) + } + if tel.DriverHealth("sungrow").IsOnline() { + t.Fatal("dead-link refusals did not exclude the driver") + } + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + for i := 1; i <= 6; i++ { + tel.RecordDriverError("sungrow", "driver_poll: none of 17 modbus reads returned data") + tel.WatchdogScan(time.Minute) + if p := tracker.update(now.Add(time.Duration(i)*5*time.Second), nil); len(p) != 0 { + t.Fatalf("default while reads fail: %v", p) + } + } + now = now.Add(35 * time.Second) + time.Sleep(time.Millisecond) + tel.RecordDriverSuccess("sungrow") + got := tracker.update(now, nil) + if !tc.wantBack { + assertStringsEqual(t, got, nil) + if tel.DriverHealth("sungrow").IsOnline() { + t.Fatal("driver let back in within one retry interval of its last trial") + } + return + } + assertStringsEqual(t, got, []string{"sungrow"}) + if !tel.DriverHealth("sungrow").IsOnline() { + t.Fatalf("driver back from a 29 s outage waits out the %v window", driverRefusalRetryInterval) + } + }) + } +} + +// An accepted command for the excluded action ends the trial guard at once. +// On a hybrid inverter a curtail streak can keep the refusal record alive +// after the battery trial was accepted. A dead-link outage a minute later is +// then a new outage, so the device gets its default when it reads again +// instead of waiting out the retry window. +func TestAcceptedTrialEndsGuardWhileOtherActionStreakRemains(t *testing.T) { + tel := pollingStore(t, "sungrow") + tracker := newDriverActuationTracker(tel) + battery := []byte(`{"action":"battery","power_w":-3000}`) + curtail := []byte(`{"action":"curtail","power_limit_w":2500}`) + now := time.Now() + excludeDuringOutage(t, tel, tracker, refusingSender("modbus reconnect backoff active"), now) + + now = now.Add(20 * time.Second) + time.Sleep(time.Millisecond) // LastSuccess is wall time + tel.RecordDriverSuccess("sungrow") + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + tracker.dispatchCommand(context.Background(), refusingSender("curtail register refused"), "pv curtail send", "sungrow", curtail, time.Second, now) + tracker.dispatchCommand(context.Background(), &stubSender{}, "driver send", "sungrow", battery, time.Second, now) + if _, kept := tracker.refusals["sungrow"]; !kept { + t.Fatal("curtail streak did not keep the refusal record") + } + for end := now.Add(time.Minute); now.Before(end); { + now = now.Add(5 * time.Second) + time.Sleep(time.Millisecond) + tel.RecordDriverSuccess("sungrow") + if p := tracker.update(now, nil); len(p) != 0 { + t.Fatalf("default while healthy: %v", p) + } + } + + dead := refusingSender("force command write failed: connection refused") + for i := 0; i < driverRefusalLimit; i++ { + tracker.dispatchCommand(context.Background(), dead, "driver send", "sungrow", battery, time.Second, now) + } + if tel.DriverHealth("sungrow").IsOnline() { + t.Fatal("dead-link refusals did not exclude the driver") + } + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + for i := 1; i <= 6; i++ { + tel.RecordDriverError("sungrow", "driver_poll: none of 17 modbus reads returned data") + tel.WatchdogScan(time.Minute) + if p := tracker.update(now.Add(time.Duration(i)*5*time.Second), nil); len(p) != 0 { + t.Fatalf("default while reads fail: %v", p) + } + } + now = now.Add(35 * time.Second) + time.Sleep(time.Millisecond) + tel.RecordDriverSuccess("sungrow") + assertStringsEqual(t, tracker.update(now, nil), []string{"sungrow"}) + if !tel.DriverHealth("sungrow").IsOnline() { + t.Fatalf("driver back from a 29 s outage waits out the %v window", driverRefusalRetryInterval) + } +} diff --git a/go/cmd/ftw/ev_ocpp_blip_test.go b/go/cmd/ftw/ev_ocpp_blip_test.go index 9653a614b..f46f4469e 100644 --- a/go/cmd/ftw/ev_ocpp_blip_test.go +++ b/go/cmd/ftw/ev_ocpp_blip_test.go @@ -131,6 +131,11 @@ func TestChargeNowSurvivesOCPPSocketBlip(t *testing.T) { } tick() tick() + if tc.unplugAfter { + // The cable stays out for UnplugGrace. + now = now.Add(loadpoint.UnplugGrace) + tick() + } after := commands[sent:] hold, held := ctrl.GetManualHold("garage", now) diff --git a/go/cmd/ftw/ev_ocpp_reconnect_test.go b/go/cmd/ftw/ev_ocpp_reconnect_test.go index bb0d7a91d..fc56e51bb 100644 --- a/go/cmd/ftw/ev_ocpp_reconnect_test.go +++ b/go/cmd/ftw/ev_ocpp_reconnect_test.go @@ -194,6 +194,9 @@ func TestOCPPReconnectIsNotAnUnplug(t *testing.T) { } status(cp, false) tick() + // The cable stays out for UnplugGrace. + now = now.Add(loadpoint.UnplugGrace) + tick() if _, ok := ctrl.GetManualHold("garage", now); ok { t.Fatal("physical unplug did not clear hold") } diff --git a/go/cmd/ftw/ev_send.go b/go/cmd/ftw/ev_send.go index 959a46f8b..29535394a 100644 --- a/go/cmd/ftw/ev_send.go +++ b/go/cmd/ftw/ev_send.go @@ -16,12 +16,6 @@ func ocppApprovedIDs(cfg *config.Config) []string { if cfg == nil { return nil } - lua := make(map[string]struct{}, len(cfg.Drivers)) - for _, d := range cfg.Drivers { - if d.Name != "" { - lua[d.Name] = struct{}{} - } - } seen := make(map[string]struct{}) var out []string add := func(id string) { @@ -39,11 +33,41 @@ func ocppApprovedIDs(cfg *config.Config) []string { add(c.ID) } } + for _, id := range ocppManagedIDs(cfg) { + add(id) + } + return out +} + +// ocppStartLimitIDs is the chargers that get the 0 A start limit on boot. +// Setup measurement mode blocks every charger write, standdowns included, so +// none does then. measurementAtBoot is the mode Core started in: leaving it +// takes a restart, so a reload that clears it must not lift the block early. +func ocppStartLimitIDs(cfg *config.Config, measurementAtBoot bool) []string { + if measurementAtBoot { + return nil + } + return ocppManagedIDs(cfg) +} + +// ocppManagedIDs is the OCPP chargers a loadpoint names: the ones FTW orders. +// An id listed only under ocpp.chargers is approved but never ordered. +func ocppManagedIDs(cfg *config.Config) []string { + if cfg == nil { + return nil + } + lua := make(map[string]struct{}, len(cfg.Drivers)) + for _, d := range cfg.Drivers { + if d.Name != "" { + lua[d.Name] = struct{}{} + } + } + var out []string for _, lp := range cfg.Loadpoints { - if _, isLua := lua[lp.DriverName]; isLua { + if _, isLua := lua[lp.DriverName]; isLua || lp.DriverName == "" { continue } - add(lp.DriverName) + out = append(out, lp.DriverName) } return out } diff --git a/go/cmd/ftw/ev_send_test.go b/go/cmd/ftw/ev_send_test.go index 0503e2b71..4165d242e 100644 --- a/go/cmd/ftw/ev_send_test.go +++ b/go/cmd/ftw/ev_send_test.go @@ -46,6 +46,42 @@ func TestOCPPApprovedIDsSkipLuaLoadpointNames(t *testing.T) { } } +// Only a loadpoint's charger gets FTW's orders, so only it may start at 0 A. +func TestOCPPManagedIDsAreLoadpointChargersOnly(t *testing.T) { + cfg := &config.Config{ + Drivers: []config.Driver{{Name: "easee", Lua: "drivers/easee.lua"}}, + Loadpoints: []config.Loadpoint{ + {ID: "cloud", DriverName: "easee"}, + {ID: "garage", DriverName: "garage"}, + }, + OCPP: &config.OCPP{Enabled: true, Chargers: []config.OCPPCharger{{ID: "wallbox", Password: "own"}}}, + } + got := ocppManagedIDs(cfg) + if len(got) != 1 || got[0] != "garage" { + t.Fatalf("managed %v, want garage only", got) + } +} + +// Setup measurement mode blocks every charger write, standdowns included, so +// no charger may start at 0 A while Core runs in it. Start passes the config +// Core booted with; a hot reload passes the saved config, which may already +// have left the mode while the block holds until restart. Both must give none. +func TestOCPPStartLimitIDsFollowMeasurementModeAtBoot(t *testing.T) { + loadpoints := []config.Loadpoint{{ID: "garage", DriverName: "garage"}} + measuring := &config.Config{Setup: &config.Setup{MeasurementOnly: true}, Loadpoints: loadpoints} + left := &config.Config{Loadpoints: loadpoints} + + if got := ocppStartLimitIDs(measuring, measuring.MeasurementOnly()); len(got) != 0 { + t.Fatalf("start in measurement mode: start limit for %v, want none", got) + } + if got := ocppStartLimitIDs(left, true); len(got) != 0 { + t.Fatalf("reload that left measurement mode before restart: start limit for %v, want none", got) + } + if got := ocppStartLimitIDs(left, false); len(got) != 1 || got[0] != "garage" { + t.Fatalf("normal operation: start limit for %v, want garage", got) + } +} + func containsID(ids []string, want string) bool { for _, id := range ids { if id == want { diff --git a/go/cmd/ftw/forecast_curtailment.go b/go/cmd/ftw/forecast_curtailment.go index 754560114..bbe6af2b8 100644 --- a/go/cmd/ftw/forecast_curtailment.go +++ b/go/cmd/ftw/forecast_curtailment.go @@ -92,7 +92,7 @@ func (f *forecastCurtailment) SetReleaseEvidence(evidence func(string) bool) { } // PendingDrivers maps pending hardware identities to current runtime names. -// Call outside control locks: identity lookup may read SQLite. The caller may +// Identity lookup reads the store's in-memory device cache. The caller may // restore these names into Core's normal curtailment state, which still decides // whether fresh telemetry and current intent permit release. Unknown identities // remain censored and never map to an arbitrary driver. @@ -101,15 +101,20 @@ func (f *forecastCurtailment) PendingDrivers(currentNames []string) []string { return nil } identities := make(map[string]string, len(currentNames)) - for _, name := range currentNames { - if f.store != nil { + f.mu.Lock() + defer f.mu.Unlock() + // With no persisted identity there is nothing to map, so keep f.mu held: + // dropping it would let persist move an active entry into f.ids after + // the lookups were skipped, and that driver would go missing. + if len(f.ids) > 0 && f.store != nil { + f.mu.Unlock() + for _, name := range currentNames { if device := f.store.LookupDeviceByDriverName(name); device != nil { identities[name] = device.DeviceID } } + f.mu.Lock() } - f.mu.Lock() - defer f.mu.Unlock() seen := make(map[string]bool, len(currentNames)) var pending []string for _, name := range currentNames { diff --git a/go/cmd/ftw/forecast_curtailment_lookup_test.go b/go/cmd/ftw/forecast_curtailment_lookup_test.go new file mode 100644 index 000000000..4a722cbb5 --- /dev/null +++ b/go/cmd/ftw/forecast_curtailment_lookup_test.go @@ -0,0 +1,44 @@ +package main + +import ( + "path/filepath" + "sync/atomic" + "testing" + + "github.com/srcfl/ftw/go/internal/state" +) + +// countingCurtailStore wraps the real *state.Store (what main.go passes) +// and counts identity lookups made from the control tick. +type countingCurtailStore struct { + *state.Store + lookups atomic.Int64 +} + +func (c *countingCurtailStore) LookupDeviceByDriverName(name string) *state.Device { + c.lookups.Add(1) + return c.Store.LookupDeviceByDriverName(name) +} + +func TestPendingDriversSkipsIdentityLookupsWithNothingCurtailed(t *testing.T) { + st, err := state.Open(filepath.Join(t.TempDir(), "state.db")) + if err != nil { + t.Fatal(err) + } + defer st.Close() + store := &countingCurtailStore{Store: st} + f := newForecastCurtailment(&stubSender{}, store) + defer f.Close() + if f.Active() { + t.Fatal("precondition: nothing curtailed, nothing persisted") + } + names := []string{"meter", "battery", "pv", "ev"} + for tick := 0; tick < 30; tick++ { // 30 ticks = 1 min at default control_interval_s=2 + if got := f.PendingDrivers(names); len(got) != 0 { + t.Fatalf("pending=%v", got) + } + } + if n := store.lookups.Load(); n != 0 { + t.Fatalf("%d identity lookups in 30 ticks with nothing curtailed or persisted", n) + } +} diff --git a/go/cmd/ftw/forecast_curtailment_race_test.go b/go/cmd/ftw/forecast_curtailment_race_test.go new file mode 100644 index 000000000..8dbb569a5 --- /dev/null +++ b/go/cmd/ftw/forecast_curtailment_race_test.go @@ -0,0 +1,32 @@ +package main + +import ( + "fmt" + "testing" +) + +// An active entry that the background worker moves into f.ids while +// PendingDrivers runs must still be reported as pending. The long name list +// widens the window; the old unlock gap missed pv in most runs. +func TestPendingDriversKeepsEntryMovedByPersist(t *testing.T) { + names := make([]string, 0, 20001) + for i := 0; i < 20000; i++ { + names = append(names, fmt.Sprintf("other-%d", i)) + } + names = append(names, "pv") + misses := 0 + const iters = 100 + for i := 0; i < iters; i++ { + store := &curtailStore{identities: map[string]string{"pv": "hardware-1"}} + f := newForecastCurtailment(&stubSender{}, store) + f.record("pv", true) // wakes the worker, which moves entry -> ids + got := f.PendingDrivers(names) + if len(got) != 1 || got[0] != "pv" { + misses++ + } + f.Close() + } + if misses > 0 { + t.Fatalf("pv curtailed but not pending in %d/%d runs", misses, iters) + } +} diff --git a/go/cmd/ftw/forecast_identity_lock_test.go b/go/cmd/ftw/forecast_identity_lock_test.go new file mode 100644 index 000000000..53f397d34 --- /dev/null +++ b/go/cmd/ftw/forecast_identity_lock_test.go @@ -0,0 +1,102 @@ +package main + +import ( + "database/sql" + "path/filepath" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/state" + _ "modernc.org/sqlite" +) + +// RefreshIdentity must not hold forecastSiteConfig.mu while SaveConfig waits +// for the SQLite write lock, or the tick's Snapshot() waits with it. +func TestForecastSnapshotDoesNotWaitForIdentitySave(t *testing.T) { + path := filepath.Join(t.TempDir(), "state.db") + st, err := state.Open(path) + if err != nil { + t.Fatal(err) + } + defer st.Close() + + s := &forecastSiteConfig{ + store: st, + baseRevision: "base", + required: map[string]bool{"meter": false}, + identity: func(string) (string, bool) { return "serial:ABC123", true }, + configuredAt: time.Now().Add(-time.Minute), + } + + // Another writer (rolloff, backup, history writer) holds the WAL write lock. + other, err := sql.Open("sqlite", path+"?_pragma=busy_timeout(5000)") + if err != nil { + t.Fatal(err) + } + defer other.Close() + other.SetMaxOpenConns(1) + tx, err := other.Begin() + if err != nil { + t.Fatal(err) + } + if _, err := tx.Exec(`UPDATE config SET value = value WHERE 0`); err != nil { + t.Fatal(err) + } + + refreshDone := make(chan time.Duration, 1) + go func() { + start := time.Now() + s.RefreshIdentity(time.Now()) + refreshDone <- time.Since(start) + }() + time.Sleep(100 * time.Millisecond) // RefreshIdentity now waits in SaveConfig + + start := time.Now() + _ = s.Snapshot() // what the control tick does before persistTelemetryTick + blocked := time.Since(start) + _ = tx.Rollback() + refresh := <-refreshDone + t.Logf("Snapshot waited %v; RefreshIdentity took %v", blocked, refresh) + if blocked > time.Second { + t.Fatalf("tick Snapshot blocked %v behind identity SaveConfig held under forecast lock", blocked) + } +} + +// A failed binding save stays dirty and waits forecastBindingRetry before the +// next attempt, instead of retrying on every 1 s identity poll. +func TestForecastIdentityBindingSaveBacksOffAfterFailure(t *testing.T) { + path := filepath.Join(t.TempDir(), "state.db") + broken, err := state.Open(path) + if err != nil { + t.Fatal(err) + } + _ = broken.Close() // every save now fails + + s := &forecastSiteConfig{ + store: broken, + baseRevision: "base", + required: map[string]bool{"meter": false}, + identity: func(string) (string, bool) { return "serial:ABC123", true }, + configuredAt: time.Now().Add(-time.Minute), + } + now := time.Now() + s.RefreshIdentity(now) + if !s.bindingDirty { + t.Fatal("failed binding save was marked clean") + } + + st, err := state.Open(path) + if err != nil { + t.Fatal(err) + } + defer st.Close() + s.store = st + s.RefreshIdentity(now.Add(time.Second)) + if _, ok := st.LoadConfig(forecastIdentityReceiptKey); ok || !s.bindingDirty { + t.Fatal("binding save retried inside the back-off window") + } + s.RefreshIdentity(now.Add(forecastBindingRetry)) + if _, ok := st.LoadConfig(forecastIdentityReceiptKey); !ok || s.bindingDirty { + t.Fatal("binding save not retried after the back-off window") + } +} diff --git a/go/cmd/ftw/forecast_site.go b/go/cmd/ftw/forecast_site.go index 06719754d..12a4274ee 100644 --- a/go/cmd/ftw/forecast_site.go +++ b/go/cmd/ftw/forecast_site.go @@ -34,6 +34,7 @@ type forecastSiteConfig struct { required map[string]bool // value identifies PV sources accepted forecastIdentityReceipt bindingDirty bool + bindingRetryAt time.Time // a failed binding save waits until then configuredAt time.Time heatingPrior, ratedPV float64 annualBase float64 @@ -49,6 +50,10 @@ type forecastIdentityReceipt struct { const forecastIdentityReceiptKey = "forecast/live_identity_v1" +// forecastBindingRetry spaces out binding saves after a failure. Each attempt +// can wait seconds on SQLite, and the identity poll runs every second. +const forecastBindingRetry = 30 * time.Second + // forecastPipelinePolicy names how Core composes the forecast it plans with. // The evaluation cohort keys on it rather than on the Core version, so error // bands and baselines survive updates that leave forecasting alone. Bump it @@ -137,6 +142,7 @@ func (s *forecastSiteConfig) Configure(cfg *config.Config, catalog []drivers.Cat } weatherData, _ := json.Marshal(weather) weatherRevision := fmt.Sprintf("%x", sha256.Sum256(weatherData)) + var weatherReceipt []byte s.mu.Lock() for _, old := range earlier { if err != nil || old == baseRevision || s.accepted.BaseRevision != old { @@ -152,13 +158,10 @@ func (s *forecastSiteConfig) Configure(cfg *config.Config, catalog []drivers.Cat if s.weatherRevision != weatherRevision || s.weatherSinceMS <= 0 { s.weatherRevision = weatherRevision s.weatherSinceMS = time.Now().UnixMilli() - receipt, _ := json.Marshal(struct { + weatherReceipt, _ = json.Marshal(struct { Revision string SinceMS int64 }{weatherRevision, s.weatherSinceMS}) - if err := s.store.SaveConfig("forecast/weather_generation", string(receipt)); err != nil { - slog.Warn("forecast weather generation not saved", "err", err) - } } v.WeatherSinceMS = s.weatherSinceMS v.SiteID = s.value.SiteID @@ -190,6 +193,12 @@ func (s *forecastSiteConfig) Configure(cfg *config.Config, catalog []drivers.Cat }{weather, s.annualBase, s.annualIncludesHeating}) s.priorRevision = fmt.Sprintf("%x", sha256.Sum256(priorData)) s.mu.Unlock() + // Save outside s.mu: the control tick reads Snapshot and must not wait on SQLite. + if weatherReceipt != nil { + if err := s.store.SaveConfig("forecast/weather_generation", string(weatherReceipt)); err != nil { + slog.Warn("forecast weather generation not saved", "err", err) + } + } s.RefreshIdentity(time.Now()) return previous != s.Snapshot().Revision } @@ -198,10 +207,10 @@ func (s *forecastSiteConfig) Configure(cfg *config.Config, catalog []drivers.Cat // Serial/MAC identities are usable immediately. Endpoint fallback retains the // existing device contract after a short init grace. A previously stronger // identity must return before it can qualify again after restart. -// Caller serializes this with model rebinding under forecastConfigMu. +// Caller serializes this with model rebinding under forecastConfigMu, so the +// saves below, made outside s.mu, keep their order. func (s *forecastSiteConfig) RefreshIdentity(now time.Time) bool { s.mu.Lock() - defer s.mu.Unlock() ids := make(map[string]string, len(s.required)) pending, pvPending := false, false for name, isPV := range s.required { @@ -250,12 +259,25 @@ func (s *forecastSiteConfig) RefreshIdentity(now time.Time) bool { s.accepted.LearningBaseRevision = learningBase } s.bindingDirty = true - if encoded, err := json.Marshal(s.accepted); err == nil { - if err = s.store.SaveConfig(forecastIdentityReceiptKey, string(encoded)); err != nil { - slog.Warn("forecast identity binding not saved", "err", err) - } else { - s.bindingDirty = false - } + } + var binding []byte + if !pending && s.bindingDirty && !now.Before(s.bindingRetryAt) { + binding, _ = json.Marshal(s.accepted) + } + s.mu.Unlock() + + // Save outside s.mu: the control tick reads Snapshot and must not wait on SQLite. + if binding != nil { + err := s.store.SaveConfig(forecastIdentityReceiptKey, string(binding)) + s.mu.Lock() + if err != nil { + s.bindingRetryAt = now.Add(forecastBindingRetry) + } else { + s.bindingDirty = false + } + s.mu.Unlock() + if err != nil { + slog.Warn("forecast identity binding not saved", "err", err, "retry_in", forecastBindingRetry) } } if changed { diff --git a/go/cmd/ftw/household_clock_test.go b/go/cmd/ftw/household_clock_test.go new file mode 100644 index 000000000..95d42a872 --- /dev/null +++ b/go/cmd/ftw/household_clock_test.go @@ -0,0 +1,34 @@ +package main + +import ( + "os" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/loadpoint" + "github.com/srcfl/ftw/go/internal/localzone" +) + +// Erik's car goal (31 October 2026): 100 % by 17:00 on Sunday, on a Docker +// box whose host clock is UTC. With the zone stored on the site, the box +// plans 17:00 in Stockholm, 16:00Z in winter, instead of 17:00Z (18:00 at +// home). +func TestABoxOnUTCPlansACarGoalOnTheStoredHouseholdClock(t *testing.T) { + local := time.Local + t.Setenv("TZ", os.Getenv("TZ")) + t.Cleanup(func() { + _ = localzone.Use("") + time.Local = local + }) + time.Local = time.UTC + + useHouseholdClock("Europe/Stockholm") + + const sunday = 1 << 6 + goal := loadpoint.NewSchedule(1, 17*60, true, sunday, false) + saturday := time.Date(2026, 10, 31, 12, 0, 0, 0, time.UTC) + got := goal.NextDeadlineUTC(saturday, nil) + if want := time.Date(2026, 11, 1, 16, 0, 0, 0, time.UTC); !got.Equal(want) { + t.Fatalf("17:00 goal due %s, want %s (17:00 in Stockholm)", got.UTC().Format(time.RFC3339), want.Format(time.RFC3339)) + } +} diff --git a/go/cmd/ftw/main.go b/go/cmd/ftw/main.go index 3c1f5de0a..27bc7a544 100644 --- a/go/cmd/ftw/main.go +++ b/go/cmd/ftw/main.go @@ -54,6 +54,7 @@ import ( "github.com/srcfl/ftw/go/internal/drivers" "github.com/srcfl/ftw/go/internal/events" "github.com/srcfl/ftw/go/internal/forecast" + "github.com/srcfl/ftw/go/internal/ftwcli" "github.com/srcfl/ftw/go/internal/gatewayidentity" "github.com/srcfl/ftw/go/internal/ha" "github.com/srcfl/ftw/go/internal/loadmodel" @@ -325,6 +326,12 @@ func main() { // Shift os.Args so the subcommand's flag.FlagSet sees its own flags. runNovaClaim(os.Args[2:]) return + case "password": + // A container has no separate ftw command, so Core's own binary + // resets the house password: docker compose exec ftw /app/ftw + // password reset. It asks the running Core on loopback. + exitCode = ftwcli.Run(os.Args[1:], os.Stdout, os.Stderr) + return } } @@ -402,6 +409,12 @@ func main() { config.ManagedDriversDirOverride = filepath.Join(filepath.Dir(*configPath), "driver-repository", "active") // ---- Load config ---- + // Load reads settings from state.db, so heal a corrupt file first. + settingsHeal, err := config.HealSettingsDatabase(*configPath) + if err != nil { + slog.Error("heal settings database", "err", err) + os.Exit(1) + } cfg, err := config.Load(*configPath) if err != nil { if isConfigMissing(*configPath) { @@ -412,6 +425,9 @@ func main() { os.Exit(1) } slog.Info("config loaded", "site", cfg.Site.Name, "drivers", len(cfg.Drivers)) + // The household clock is set before anything else reads it: every + // goroutine from here on reads time.Local. + useHouseholdClock(cfg.Site.TimeZone) // Repaired-but-wrong config: ERROR so it reaches the log ring and the // support report, without stopping a boot the repair made safe. for _, w := range cfg.LoadWarnings { @@ -502,6 +518,9 @@ func main() { slog.Error("open state", "err", err) os.Exit(1) } + if settingsHeal != nil { + st.RecordHealEvent(*settingsHeal) + } if err := st.EnableHistoryAggregation(); err != nil { slog.Error("enable history aggregation", "err", err) os.Exit(1) @@ -510,6 +529,11 @@ func main() { slog.Error("absorb cold history", "err", err) os.Exit(1) } + if cleared, err := st.KeepDailyZone(context.Background(), localzone.Name()); err != nil { + slog.Warn("could not check the zone of cached daily energy", "err", err) + } else if cleared { + slog.Info("household zone changed; daily energy is computed again on the new clock", "zone", localzone.Name()) + } defer func() { if err := st.Close(); err != nil { slog.Error("state shutdown failed", "err", err) @@ -591,11 +615,9 @@ func main() { // ---- Control state ---- ctrl := newControlStateFromConfig(cfg) - // Restore persisted mode + target if present. The planner variants - // have to be listed too — without them the strategy the user picked in - // the UI (planner_self / planner_cheap / planner_arbitrage) is silently - // dropped on restart and the dashboard appears to forget the selection. - restoreStoredMode(ctrl, st) + // Restore durable modes and targets. Forced Charge ends at restart; + // the planner makes a fresh plan instead of resuming that override. + restoreStoredMode(ctrl, st, cfg) storedTrust, _ := st.LoadConfig(config.StateKeyForecastTrust) storedExport, _ := st.LoadConfig(config.StateKeyBatteryExport) storedSafetyK, _ := st.LoadConfig(config.StateKeySafetyK) @@ -1104,6 +1126,30 @@ func main() { newCfg.ModbusProxyWriteOwners = oldCfg.ModbusProxyWriteOwners forecastConfigMu.Lock() defer forecastConfigMu.Unlock() + // Limits first: Reload can spend seconds per changed driver, and a + // lowered limit must not wait for that. configreload.Apply has + // already written the battery fuse and export limits. A driver that + // leaves the config keeps its old limits until Reload stops it. + limits := driverLimitsFrom(newCfg.ModbusControlDrivers(), newCfg.Batteries, newCfg.Planner) + ctrlMu.Lock() + reloadLimits := make(map[string]control.PowerLimits, len(ctrl.DriverLimits)+len(limits)) + for name, lim := range ctrl.DriverLimits { + reloadLimits[name] = lim + } + for name, lim := range limits { + reloadLimits[name] = lim + } + ctrl.DriverLimits = reloadLimits + ctrlMu.Unlock() + // Keep the loadpoint controller's per-phase EV fuse clamp in step + // with the battery side. SetSiteFuse takes its own lock. + if lpController != nil { + lpController.SetSiteFuse(loadpoint.SiteFuse{ + MaxAmps: newCfg.Fuse.MaxAmps, + Voltage: newCfg.Fuse.Voltage, + PhaseCnt: newCfg.Fuse.Phases, + }) + } // Driver paths are already resolved by config.Load; no extra // work needed here. Re-apply the battery SoC-window → driver // config mapping so a hot-edited soc_max reaches the driver too. @@ -1150,19 +1196,7 @@ func main() { ctrl.InverterGroups = inverterGroupsFrom(newCfg.Drivers) ctrl.SupportsPVCurtail = supportsPVCurtailFrom(newCfg.Drivers) ctrl.SolarFeedDrivers = solarFeedDriversFrom(newCfg.Drivers) - ctrl.DriverLimits = driverLimitsFrom(newCfg.ModbusControlDrivers(), newCfg.Batteries) - // Fuse params + safety margin: previously startup-only. - // Hot-reload them so operators can tune the per-phase margin - // from the UI without restarting (e.g. raising it after the - // inverter's own protection trips, lowering it to recover - // last few hundred W of arbitrage headroom). - ctrl.SiteFuseAmps = newCfg.Fuse.MaxAmps - ctrl.SiteFuseVoltage = newCfg.Fuse.Voltage - ctrl.SiteFusePhases = newCfg.Fuse.Phases - // Mirror the startup-path default semantics — nil → 0.5, - // explicit 0 → disabled. See EffectiveSafetyMarginA. - ctrl.SiteFuseSafetyA = newCfg.Fuse.EffectiveSafetyMarginA() - ctrl.MaxExportW = newCfg.Site.MaxExportW + ctrl.DriverLimits = limits // Lowering the fuse can strand a peak limit that was legal when it // was set. SetPeakLimit refuses to create that state; only a reload // can arrive at it from the other side, and it is silent — the @@ -1178,20 +1212,6 @@ func main() { "peak_limit_w", peakLimitW, "import_ceiling_w", peakCeilingW) } - // Keep the loadpoint controller's per-phase EV fuse clamp in - // sync with hot-reloaded fuse params — previously startup-only, - // so an operator tuning max_amps / margin from the UI updated - // the control-package battery lever (above) but left the EV - // clamp on the stale startup value until restart. SetSiteFuse - // takes its own lock; call it outside ctrlMu. - if lpController != nil { - lpController.SetSiteFuse(loadpoint.SiteFuse{ - MaxAmps: newCfg.Fuse.MaxAmps, - Voltage: newCfg.Fuse.Voltage, - PhaseCnt: newCfg.Fuse.Phases, - }) - } - // Site-meter swap propagation. The config apply callback // already updated ctrl.SiteMeterDriver under ctrlMu before // this applier ran, so the dispatch loop reads from the @@ -1291,6 +1311,7 @@ func main() { // and one whose entry was removed goes back to pending. if ocppSrv != nil { ocppSrv.Handler().SetApprovedIDs(ocppApprovedIDs(newCfg)) + ocppSrv.Handler().SetManagedIDs(ocppStartLimitIDs(newCfg, setupMeasurementAtBoot)) // A charger adopted by this save booted long ago and will not // boot again just because we changed our mind, so its device // row has to be written here rather than waiting for one. @@ -1461,6 +1482,7 @@ func main() { HeartbeatIntervalS: cfg.OCPP.HeartbeatIntervalS, ChargerSecrets: cfg.OCPP.ChargerSecrets(), ApprovedIDs: approved, + ManagedIDs: ocppStartLimitIDs(cfg, setupMeasurementAtBoot), } if t := cfg.OCPP.TLS; t != nil { ocppCfg.TLS = &ocpp.TLSConfig{ @@ -1663,6 +1685,20 @@ func main() { defer ctrlMu.Unlock() return control.PhaseImbalanceW(tel, ctrl, timeout, time.Now()) } + // The planner keeps each clock hour under the owner's import + // ceiling, from the budget Core's live limit counts. + mpcSvc.ImportCeiling = func() mpc.ImportCeiling { + ctrlMu.Lock() + defer ctrlMu.Unlock() + return control.PlanningImportCeiling(ctrl) + } + // The demand charges' hour in progress counts what the site meter + // measured, the same counter the import ceiling uses. + mpcSvc.HourImport = func() mpc.HourImport { + ctrlMu.Lock() + defer ctrlMu.Unlock() + return control.MeasuredHourImport(ctrl) + } // Cap planned export below the fuse when the operator set a site // export ceiling, so the DP never schedules a discharge that would // over-export and trip an inverter (the Ferroamp 0x8030 fault). @@ -1702,6 +1738,11 @@ func main() { } return states } + // The deadline reserve holds to the plan's limits. The controller + // reads them under its own locks; it is assigned further down. + mpcSvc.PlanningLimits = func(st loadpoint.State) (minW, maxW float64, steps []float64) { + return lpController.PlanningLimits(st) + } mpcSvc.Loadpoints = func(slotLenMin int) []*mpc.LoadpointSpec { specs := make([]*mpc.LoadpointSpec, 0) for _, st := range lpMgr.States() { @@ -2586,7 +2627,10 @@ func main() { if timeout <= 0 { timeout = 60 * time.Second } - if gate := evaluateSiteDispatchFreshnessAt(tel, meter, candidate.Fuse.MaxAmps, candidate.Fuse.Phases, timeout, time.Now()); !gate.Allowed() { + ctrlMu.Lock() + gate := siteMeterGateLocked(tel, ctrl, meter, candidate.Fuse.MaxAmps, candidate.Fuse.Phases, timeout, time.Now()) + ctrlMu.Unlock() + if !gate.Allowed() { return fmt.Errorf("measure before enabling control: %s", gate.Reason) } return nil @@ -2648,10 +2692,6 @@ func main() { // bound at startup stays; no port gap for healthcheck probes. apiHandler.Swap(handler) slog.Info("HTTP API ready", "addr", httpSrv.Addr) - if err := trial.complete(selfUpdater); err != nil { - slog.Error("native trial readiness commit failed", "err", err) - os.Exit(1) - } if trial == nil { reconcileNativeFallback(nativeRoot, Version, selfUpdater) } @@ -2682,6 +2722,9 @@ func main() { } return dev.DeviceID, dev.Make, dev.Serial, true }) + // The control.not_following rule reads the box's own answer to "Are we + // in control?", the one GET /api/status serves. + notifSvc.SetControlReader(srv.ControlStates) // FuseReader: on each HealthTick the fuse_over_limit rule reads // the site meter's live per-phase currents from telemetry and // compares against cfg.Fuse.MaxAmps. Closes over cfg + cfgMu so @@ -2820,10 +2863,13 @@ func main() { // ---- Control loop ---- controlInterval := time.Duration(cfg.Site.ControlIntervalS) * time.Second + if controlInterval <= 0 { + controlInterval = 2 * time.Second // same guard as the tick + } // fuseMaxW is recomputed per tick from ctrl.SiteFuse* under ctrlMu — // the config apply callback updates those fields directly, so a // startup snapshot here would go stale on the first hot-reload. - dtS := float64(cfg.Site.ControlIntervalS) + dtS := controlInterval.Seconds() // Every dispatch command carries its own deadline — see // driverCommandTimeout for the stall it bounds. driverCmdTimeout := driverCommandTimeout(controlInterval) @@ -2855,6 +2901,32 @@ func main() { const evStopReplanCooldown = 60 * time.Second var staleDefaults staleSiteDefaultTracker solarFeed := newSolarFeedSender() + // Every path out of a tick calls tickDone. The first one commits a native + // trial, so a release that cannot run its control loop is never committed. + // A tick blocked by a stale meter or by setup still counts. + tickSup := newTickSupervisor(time.Now(), controlInterval) + trialPending := true + tickDone := func() { + tickSup.beat(time.Now(), controlInterval) + if !trialPending { + return + } + trialPending = false + if err := trial.complete(selfUpdater); err != nil { + slog.Error("native trial readiness commit failed", "err", err) + os.Exit(1) + } + } + supCtx, stopSup := context.WithCancel(ctx) + defer stopSup() + go tickSup.run(supCtx, time.Second, time.Now, func() { + capMu.RLock() + observeOnlySnap := observeOnly + capMu.RUnlock() + sendDefaultsInParallel(reg.Names(), func(name string) { + sendDriverDefault(supCtx, srv, name, "control_tick_stalled", observeOnlySnap) + }) + }, func() { os.Exit(1) }) for { select { case <-sigc: @@ -2956,6 +3028,7 @@ func main() { if _, err := persistTelemetryTick(st, tel, snapshotTickPersistControl(ctrl, ctrlMu), nowMs, watchdogTimeout, energyIdentity, forecastSettings.Snapshot()); err != nil { slog.Warn("setup measurement persistence failed", "err", err) } + tickDone() continue } if watchdogTimeout <= 0 { @@ -2998,15 +3071,11 @@ func main() { // standdown/default commands remain permitted. ctrlMu.Lock() ctrl.ObserveSiteImport(tel) + freshness := siteDispatchGateLocked(tel, ctrl, watchdogTimeout, tickNow) siteMeterDriver := ctrl.SiteMeterDriver - siteFuseAmps := ctrl.SiteFuseAmps - siteFusePhases := ctrl.SiteFusePhases forecastIntent := forecastCurtailmentActive(ctrl, tickNow) ctrlMu.Unlock() forecastCurtail.ObserveIntent(forecastIntent) - freshness := evaluateSiteDispatchFreshnessAt( - tel, siteMeterDriver, siteFuseAmps, siteFusePhases, watchdogTimeout, tickNow, - ) if reason := setupRuntimeBlockReason(); reason != "" { freshness.Reason = reason } @@ -3057,6 +3126,7 @@ func main() { if err != nil { slog.Warn("tick persistence failed", "samples", sampleCount, "err", err) } + tickDone() continue } @@ -3129,19 +3199,23 @@ func main() { evFuseDemands = lpController.FuseDemands() } - ctrlMu.Lock() - ctrl.EVSurplusOnlyReserveW = evReserveW - ctrl.EVSurplusOnlyChargingW = evSurplusOnlyChargingW - ctrl.EVCurtailHeadroomW = evCurtailHeadroomW - ctrl.BatteryBoostEVChargingW = boostEVW - ctrl.BatteryBoostReserveSoC = boostReserveSoC - ctrl.SetEVCommanded(lpStatesSnapshot) - ctrl.SetEVFuseDemands(evFuseDemands) - fuseMaxW := ctrl.SiteFuseAmps * ctrl.SiteFuseVoltage * float64(ctrl.SiteFusePhases) - targets := control.ComputeDispatch(tel, ctrl, capsSnap, fuseMaxW) - planMissingNow := ctrl.Mode.IsPlannerMode() && ctrl.PlanStale - mode := ctrl.Mode - ctrlMu.Unlock() + var fuseMaxW float64 + var targets []control.DispatchTarget + var planMissingNow bool + var mode control.Mode + withCtrlLock(ctrlMu, func() { + ctrl.EVSurplusOnlyReserveW = evReserveW + ctrl.EVSurplusOnlyChargingW = evSurplusOnlyChargingW + ctrl.EVCurtailHeadroomW = evCurtailHeadroomW + ctrl.BatteryBoostEVChargingW = boostEVW + ctrl.BatteryBoostReserveSoC = boostReserveSoC + ctrl.SetEVCommanded(lpStatesSnapshot) + ctrl.SetEVFuseDemands(evFuseDemands) + fuseMaxW = ctrl.SiteFuseAmps * ctrl.SiteFuseVoltage * float64(ctrl.SiteFusePhases) + targets = control.ComputeDispatch(tel, ctrl, capsSnap, fuseMaxW) + planMissingNow = ctrl.Mode.IsPlannerMode() && ctrl.PlanStale + mode = ctrl.Mode + }) // ---- Self-tune override: step one battery, hold its siblings at 0 ---- // Only the controllable battery pool is commanded, and the step @@ -3149,9 +3223,9 @@ func main() { finalTargets := targets selfTuneName, selfTuneCmd, selfTuneActive := selfTune.CurrentCommand() if selfTuneActive { - ctrlMu.Lock() - finalTargets = control.SelfTuneDispatch(tel, ctrl, capsSnap, fuseMaxW, selfTuneName, selfTuneCmd) - ctrlMu.Unlock() + withCtrlLock(ctrlMu, func() { + finalTargets = control.SelfTuneDispatch(tel, ctrl, capsSnap, fuseMaxW, selfTuneName, selfTuneCmd) + }) } if troubleshootingMode { gridW, haveGrid := troubleshootingGridW(tel, siteMeterDriver) @@ -3246,7 +3320,7 @@ func main() { // planner state. That must not wait for the normal 15 min // interval; rebuild immediately, retrying briefly if prices / // forecasts / telemetry were not ready during startup. - if mpcSvc != nil && planMissingNow && time.Since(lastMissingPlanReplan) > missingPlanReplanCooldown { + if missingPlanRetryDue(mpcSvc, planMissingNow, lastMissingPlanReplan, missingPlanReplanCooldown) { lastMissingPlanReplan = time.Now() go mpcSvc.ReplanWithReason(ctx, "missing_plan_retry") slog.Info("missing MPC plan → replan triggered") @@ -3318,6 +3392,7 @@ func main() { } modelsMu.Unlock() } + tickDone() } } } @@ -3418,6 +3493,15 @@ func rolloffLoop(ctx context.Context, st *state.Store, coldDir string, dataMaint // seedImportHour gives the import ceiling the clock hour's import that // state.db recorded before this start, so a restart keeps it. +// useHouseholdClock runs Core on the site's stored zone, or the host's when +// none is stored. +func useHouseholdClock(stored string) { + if err := localzone.Use(stored); err != nil { + slog.Error("site.time_zone does not load; the box keeps the host's zone", "err", err) + } + slog.Info("household clock", "zone", localzone.Name(), "stored", localzone.Stored() != "") +} + func seedImportHour(st *state.Store, ctrl *control.State, now time.Time) { start := control.ImportHourStart(now) wh, covered, err := st.ImportWhIntervals(context.Background(), [][2]int64{{start.UnixMilli(), now.UnixMilli()}}) @@ -3605,12 +3689,36 @@ func driverCapacitiesFrom(drvList []config.Driver, loadpoints []config.Loadpoint return out } -// driverLimitsFrom gives dispatch the same per-driver limits as the planner. -func driverLimitsFrom(drivers []config.Driver, batteries map[string]config.Battery) map[string]control.PowerLimits { +// driverLimitsFrom gives dispatch the same per-driver power limits and charge +// ceilings as the planner. Each battery's soc_max overrides the planner's. +// The SoC floor is the higher of the battery's and the running planner's +// soc_min, with the planner's 10 % default. +func driverLimitsFrom(drivers []config.Driver, batteries map[string]config.Battery, planner *config.Planner) map[string]control.PowerLimits { + ceiling := 1.0 + if planner != nil && planner.SoCMax > 0 { + ceiling = planner.SoCMax + } + floor := 0.0 + if planner != nil && planner.Enabled { + floor = planner.SoCMin + if floor <= 0 { + floor = 0.10 + } + } out := map[string]control.PowerLimits{} for _, d := range drivers { if !d.ObserveOnly { - out[d.Name] = batteryPowerLimits(d, batteries[d.Name]) + limits := batteryPowerLimits(d, batteries[d.Name]) + limits.SoCMax = ceiling + if b := batteries[d.Name]; b.SoCMax != nil { + limits.SoCMax = *b.SoCMax + limits.SoCMaxSet = true + } + limits.SoCMin = floor + if b := batteries[d.Name]; b.SoCMin != nil && *b.SoCMin > floor { + limits.SoCMin = *b.SoCMin + } + out[d.Name] = limits } } return out diff --git a/go/cmd/ftw/missing_plan_retry.go b/go/cmd/ftw/missing_plan_retry.go new file mode 100644 index 000000000..c47b728bf --- /dev/null +++ b/go/cmd/ftw/missing_plan_retry.go @@ -0,0 +1,15 @@ +package main + +import ( + "time" + + "github.com/srcfl/ftw/go/internal/mpc" +) + +// missingPlanRetryDue reports whether the control tick should start a +// missing-plan replan. A new request cancels the one in flight, so retrying +// while a solve is pending would stop every solve slower than the cooldown +// and no plan would ever land. +func missingPlanRetryDue(svc *mpc.Service, planMissing bool, last time.Time, cooldown time.Duration) bool { + return svc != nil && planMissing && time.Since(last) > cooldown && !svc.PlanSnapshot().Pending +} diff --git a/go/cmd/ftw/missing_plan_retry_test.go b/go/cmd/ftw/missing_plan_retry_test.go new file mode 100644 index 000000000..771bd3785 --- /dev/null +++ b/go/cmd/ftw/missing_plan_retry_test.go @@ -0,0 +1,82 @@ +package main + +import ( + "context" + "path/filepath" + "sync/atomic" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/mpc" + "github.com/srcfl/ftw/go/internal/state" +) + +// slowPlanOptimizer models a solve that takes longer than the missing-plan +// retry cooldown and, like a Go DP or a busy worker, does not stop on cancel. +type slowPlanOptimizer struct { + solve time.Duration + calls atomic.Int32 +} + +func (o *slowPlanOptimizer) Optimize(_ context.Context, slots []mpc.Slot, p mpc.Params) (mpc.Plan, error) { + o.calls.Add(1) + time.Sleep(o.solve) + plan := mpc.Optimize(slots, p) + plan.Solver = &mpc.SolverInfo{Engine: "test", Backend: "slow", Status: "ok", Formulation: "deterministic"} + return plan, nil +} + +func (*slowPlanOptimizer) Close() error { return nil } + +// A solve slower than the retry cooldown must still publish a plan. Before +// the Pending guard, every retry cancelled the solve in flight, so on a slow +// box no plan ever landed. Scale: tick 1 s -> 20 ms, cooldown 5 s -> 100 ms, +// solve 6 s -> 120 ms. +func TestMissingPlanRetryLetsSlowSolvePublish(t *testing.T) { + st, err := state.Open(filepath.Join(t.TempDir(), "state.db")) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { st.Close() }) + now := time.Now().UTC().Truncate(time.Hour) + prices := make([]state.PricePoint, 4) + for i := range prices { + prices[i] = state.PricePoint{ + Zone: "SE3", SlotTsMs: now.Add(time.Duration(i) * time.Hour).UnixMilli(), + SlotLenMin: 60, SpotOreKwh: float64(40 + i*20), TotalOreKwh: float64(90 + i*20), + Source: "test", FetchedAtMs: now.UnixMilli(), + } + } + if err := st.SavePrices(prices); err != nil { + t.Fatal(err) + } + svc := mpc.New(st, nil, "SE3", mpc.Params{ + Mode: mpc.ModeSelfConsumption, SoCLevels: 11, ActionLevels: 5, + CapacityWh: 10000, InitialSoC: 0.5, SoCMin: 0.1, SoCMax: 0.95, + MaxChargeW: 3000, MaxDischargeW: 3000, + ChargeEfficiency: 0.95, DischargeEfficiency: 0.95, + }) + svc.Horizon = 4 * time.Hour + svc.BaseLoad = 500 + opt := &slowPlanOptimizer{solve: 120 * time.Millisecond} + svc.Optimizer = opt + t.Cleanup(func() { + until := time.Now().Add(5 * time.Second) + for svc.IsReplanning() && time.Now().Before(until) { + time.Sleep(10 * time.Millisecond) + } + }) + + var last time.Time + end := time.Now().Add(2 * time.Second) + for time.Now().Before(end) { + if missingPlanRetryDue(svc, svc.Latest() == nil, last, 100*time.Millisecond) { + last = time.Now() + go svc.ReplanWithReason(context.Background(), "missing_plan_retry") + } + time.Sleep(20 * time.Millisecond) + } + if svc.Latest() == nil { + t.Fatalf("no plan published after 2 s (100 s at real scale); %d solves started, each one cancelled by the next retry", opt.calls.Load()) + } +} diff --git a/go/cmd/ftw/modbus_owner_test.go b/go/cmd/ftw/modbus_owner_test.go index 1675e7934..28343c2e1 100644 --- a/go/cmd/ftw/modbus_owner_test.go +++ b/go/cmd/ftw/modbus_owner_test.go @@ -19,7 +19,7 @@ func TestExternalModbusOwnerKeepsTelemetryButLeavesPlannerAndDispatch(t *testing if got := driverCapacitiesFrom(cfg.Drivers, nil, evCatalog(), false); got["sungrow"] != 10000 { t.Fatalf("telemetry capacity lost: %v", got) } - if len(driverLimitsFrom(cfg.ModbusControlDrivers(), cfg.Batteries)) != 0 { + if len(driverLimitsFrom(cfg.ModbusControlDrivers(), cfg.Batteries, cfg.Planner)) != 0 { t.Fatal("external battery remains in dispatch limits") } if !config.ObserveOnlyDriverSet(cfg)["sungrow"] { diff --git a/go/cmd/ftw/native_update.go b/go/cmd/ftw/native_update.go index 211e74bd5..8d04099e5 100644 --- a/go/cmd/ftw/native_update.go +++ b/go/cmd/ftw/native_update.go @@ -55,8 +55,8 @@ func beginNativeTrial(root, tag, version string) (*nativeTrial, error) { return trial, nil } -// Complete only after the full API replaces the boot handler. At this point -// state and drivers are open and the HTTP listener has bound successfully. +// Complete only after the first control tick has finished. By then state, +// drivers and the full API are up, and the control loop runs. func (trial *nativeTrial) complete(checker *selfupdate.Checker) error { if trial == nil { return nil diff --git a/go/cmd/ftw/partial_config_restart_test.go b/go/cmd/ftw/partial_config_restart_test.go new file mode 100644 index 000000000..0edf7a812 --- /dev/null +++ b/go/cmd/ftw/partial_config_restart_test.go @@ -0,0 +1,88 @@ +package main + +import ( + "bytes" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "sync" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/api" + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/control" + "github.com/srcfl/ftw/go/internal/state" +) + +// A settings POST that omits site tuning must not leave a stored config the +// boot path cannot start from. +func TestPartialConfigPostSurvivesRestart(t *testing.T) { + dir := t.TempDir() + path, database := filepath.Join(dir, "config.yaml"), filepath.Join(dir, "state.db") + seed := []byte("site:\n name: Test\nfuse:\n max_amps: 16\n phases: 3\n voltage: 230\napi:\n port: 8080\ndrivers: []\n") + if err := os.WriteFile(path, seed, 0o600); err != nil { + t.Fatal(err) + } + loaded, err := config.Load(path) + if err != nil { + t.Fatal(err) + } + st, err := state.Open(database) + if err != nil { + t.Fatal(err) + } + defer st.Close() + cfg, err := config.InitializeStorage(path, database, loaded, st) + if err != nil { + t.Fatal(err) + } + if cfg.Site.ControlIntervalS != 2 { + t.Fatalf("seed interval = %d, want default 2", cfg.Site.ControlIntervalS) + } + + var cfgMu sync.RWMutex + var ctrlMu sync.Mutex + srv := api.New(&api.Deps{ + Ctrl: control.NewState(0, 42, ""), CtrlMu: &ctrlMu, + Cfg: cfg, CfgMu: &cfgMu, + ConfigPath: path, + DriverDir: dir, UserDriverDir: dir, + SaveConfig: func(p string, c *config.Config) error { return config.SaveStored(st, p, c) }, + }) + + // No If-Match, no control_interval_s / slew_rate_w / grid_tolerance_w / + // watchdog_timeout_s / min_dispatch_interval_s. smoothing_alpha is the only + // site tuning key Validate insists on. + body := []byte(`{"site":{"name":"Test","smoothing_alpha":0.3},"fuse":{"max_amps":16,"phases":3,"voltage":230},"api":{"port":8080},"drivers":[]}`) + req := httptest.NewRequest(http.MethodPost, "/api/config", bytes.NewReader(body)) + req.Header.Set("Content-Type", "application/json") + rr := httptest.NewRecorder() + srv.Handler().ServeHTTP(rr, req) + t.Logf("POST /api/config = %d %s", rr.Code, rr.Body.String()) + if rr.Code != http.StatusOK { + return // a rejected partial save is the safe outcome + } + + restarted, err := config.Load(path) + if err != nil { + t.Fatalf("reload: %v", err) + } + s := restarted.Site + t.Logf("reloaded: interval=%d slew=%v tol=%v watchdog=%d mindisp=%d", + s.ControlIntervalS, s.SlewRateW, s.GridToleranceW, s.WatchdogTimeoutS, s.MinDispatchIntervalS) + if s.ControlIntervalS != 2 || s.WatchdogTimeoutS != 60 { + t.Fatalf("reloaded interval=%d watchdog=%d, want the defaults 2 and 60", s.ControlIntervalS, s.WatchdogTimeoutS) + } + + // Same expression as go/cmd/ftw/main.go:2839 and :2854. + controlInterval := time.Duration(restarted.Site.ControlIntervalS) * time.Second + defer func() { + if p := recover(); p != nil { + t.Fatalf("boot ticker panics after a partial settings save: %v", p) + } + }() + ticker := time.NewTicker(controlInterval) + ticker.Stop() +} diff --git a/go/cmd/ftw/site_dispatch_safety.go b/go/cmd/ftw/site_dispatch_safety.go index b65583d31..d1d815270 100644 --- a/go/cmd/ftw/site_dispatch_safety.go +++ b/go/cmd/ftw/site_dispatch_safety.go @@ -24,6 +24,16 @@ type siteDispatchDecision struct { func (d siteDispatchDecision) Allowed() bool { return d.Reason == siteDispatchReady } +// frozenMeter closes the gate when the site meter repeats one reading while +// FTW's orders moved (control.State.ObserveSiteMeterFreeze). Such a meter +// answers every poll, so its UpdatedAt alone looks fresh. +func (d siteDispatchDecision) frozenMeter(frozen bool) siteDispatchDecision { + if frozen && d.Allowed() { + d.Reason = siteDispatchMeterStale + } + return d +} + // evaluateSiteDispatchFreshnessAt rejects physical dispatch when the site // meter itself is stale. Per-phase currents are optional until the configured // meter has emitted at least one of them; once available, every configured @@ -71,11 +81,38 @@ func siteDispatchNow( timeout = 60 * time.Second } ctrlMu.Lock() - siteMeter := ctrl.SiteMeterDriver - fuseAmps := ctrl.SiteFuseAmps - phases := ctrl.SiteFusePhases - ctrlMu.Unlock() - return evaluateSiteDispatchFreshnessAt(tel, siteMeter, fuseAmps, phases, timeout, now) + defer ctrlMu.Unlock() + return siteDispatchGateLocked(tel, ctrl, timeout, now) +} + +// siteDispatchGateLocked is the pre-dispatch verdict the tick and +// siteDispatchNow share: the site meter's age and phase currents, and +// whether it repeats one reading while FTW's devices move. Call it with +// ctrlMu held. +func siteDispatchGateLocked( + tel *telemetry.Store, + ctrl *control.State, + timeout time.Duration, + now time.Time, +) siteDispatchDecision { + return siteMeterGateLocked(tel, ctrl, ctrl.SiteMeterDriver, ctrl.SiteFuseAmps, ctrl.SiteFusePhases, timeout, now) +} + +// siteMeterGateLocked is siteDispatchGateLocked for a given meter and fuse. +// Setup uses it to judge a candidate's fuse before it enables control. Only +// the running site meter can be frozen. Call it with ctrlMu held. +func siteMeterGateLocked( + tel *telemetry.Store, + ctrl *control.State, + meter string, + fuseAmps float64, + phases int, + timeout time.Duration, + now time.Time, +) siteDispatchDecision { + ctrl.ObserveSiteMeterFreeze(tel, now) + frozen := meter == ctrl.SiteMeterDriver && ctrl.SiteMeterFrozen() + return evaluateSiteDispatchFreshnessAt(tel, meter, fuseAmps, phases, timeout, now).frozenMeter(frozen) } // sitePhaseCurrentsAt returns the configured phases in L1/L2/L3 order. diff --git a/go/cmd/ftw/site_meter_stale_test.go b/go/cmd/ftw/site_meter_stale_test.go index 74e8f42a4..318b7dc6e 100644 --- a/go/cmd/ftw/site_meter_stale_test.go +++ b/go/cmd/ftw/site_meter_stale_test.go @@ -92,6 +92,78 @@ func TestEvaluateSiteDispatchFreshnessAllowsMeterWithoutPhaseTelemetry(t *testin } } +// A frozen site meter (control.State.SiteMeterFrozen) closes the gate as a +// stale meter, and leaves an already closed gate's reason alone. +func TestFrozenSiteMeterClosesDispatchGate(t *testing.T) { + tel := telemetry.NewStore() + tel.Update("meter", telemetry.DerMeter, 2667, nil, nil) + open := evaluateSiteDispatchFreshnessAt(tel, "meter", 0, 3, time.Minute, time.Now()) + if !open.Allowed() { + t.Fatalf("fresh meter blocked: %q", open.Reason) + } + if got := open.frozenMeter(false); !got.Allowed() { + t.Fatalf("meter that is not frozen blocked: %q", got.Reason) + } + if got := open.frozenMeter(true); got.Reason != siteDispatchMeterStale { + t.Fatalf("frozen meter reason = %q, want %q", got.Reason, siteDispatchMeterStale) + } + phases := siteDispatchDecision{Reason: siteDispatchPhaseCurrentsStale} + if got := phases.frozenMeter(true); got.Reason != siteDispatchPhaseCurrentsStale { + t.Fatalf("frozen meter replaced reason %q with %q", siteDispatchPhaseCurrentsStale, got.Reason) + } +} + +// The gate the tick and operator setpoints share watches the meter itself: +// one repeated sample (seed 207537) while FTW winds the battery 3 kW closes +// it as site_meter_stale, which sends drivers to default and chargers to +// 0 W, before the rig's EV rise at 38 s. A changed value reopens it. +func TestSiteDispatchGateClosesOnFrozenMeter(t *testing.T) { + tel := telemetry.NewStore() + cfg := &config.Config{Site: config.Site{WatchdogTimeoutS: 120}} + ctrl := control.NewState(0, 50, "meter") + cfgMu, ctrlMu := &sync.RWMutex{}, &sync.Mutex{} + frozen := []byte(`{"w":2667,"l1_a":2.4,"l2_a":5.5,"l3_a":-0.2,"l1_v":230,"l2_v":230,"l3_v":230}`) + soc := 0.5 + tel.Update("bat", telemetry.DerBattery, 0, &soc, nil) + tel.EnsureDriverHealth("meter") + tel.EnsureDriverHealth("bat") + + start := time.Now() + staleAt := -1 + for s := 0; s <= 40; s++ { + if s == 2 { // FTW's order, and the battery follows it + ctrl.PrevTargets["bat"] = -3000 + tel.Update("bat", telemetry.DerBattery, -3000, &soc, nil) + } + tel.Update("meter", telemetry.DerMeter, 2667, nil, frozen) + got := siteDispatchNow(tel, cfg, cfgMu, ctrl, ctrlMu, start.Add(time.Duration(s)*time.Second)) + if !got.Allowed() { + if got.Reason != siteDispatchMeterStale { + t.Fatalf("frozen meter reason = %q, want %q", got.Reason, siteDispatchMeterStale) + } + staleAt = s + break + } + } + if staleAt < 0 || staleAt >= 38 { + t.Fatalf("meter repeated one sample for 40 s while the battery moved 3 kW: stale at %d s, want before 38 s", staleAt) + } + + // Setup's check before it enables control, with the candidate's own + // fuse, sees the same verdict. + ctrlMu.Lock() + setup := siteMeterGateLocked(tel, ctrl, "meter", 25, 3, 2*time.Minute, start.Add(40*time.Second)) + ctrlMu.Unlock() + if setup.Reason != siteDispatchMeterStale { + t.Fatalf("setup gate on a frozen meter = %q, want %q", setup.Reason, siteDispatchMeterStale) + } + + tel.Update("meter", telemetry.DerMeter, 2654, nil, nil) + if got := siteDispatchNow(tel, cfg, cfgMu, ctrl, ctrlMu, start.Add(41*time.Second)); !got.Allowed() { + t.Fatalf("meter moving again stayed blocked: %q", got.Reason) + } +} + func TestStaleSiteDefaultTrackerIncludesOwnerOnceAndRearmsOnRecovery(t *testing.T) { var tracker staleSiteDefaultTracker names := []string{"ferroamp", "sungrow", "easee"} diff --git a/go/cmd/ftw/tick_supervisor.go b/go/cmd/ftw/tick_supervisor.go new file mode 100644 index 000000000..943bfff40 --- /dev/null +++ b/go/cmd/ftw/tick_supervisor.go @@ -0,0 +1,121 @@ +package main + +import ( + "context" + "log/slog" + "sync" + "sync/atomic" + "time" +) + +// The control tick runs every safety check itself: the driver watchdog, the +// stale-meter stop and the fuse guard. A tick that wedges therefore leaves +// devices on their last forced setpoint, and a Sungrow leaves forced mode only +// through its default. tickSupervisor watches the tick from outside. +// +// Once the last completed tick is older than tickStallAfter, it sends every +// driver its default mode. Once it is older than tickExitAfter, it exits the +// process so the service manager restarts Core; startup sends defaults again, +// and a native probation counts the exit as a crash. +// +// Ages use the monotonic clock. A Pi without an RTC boots on a stale clock and +// NTP steps it forward, often while the loop already runs; a wall-clock age +// would then trip a healthy Core, and a backward step would hide a stall. +type tickSupervisor struct { + // base carries a monotonic reading when it comes from time.Now. + // lastNs is the time from base to the last completed tick. + base time.Time + lastNs atomic.Int64 + intervalNs atomic.Int64 + + // stalled is touched only by the supervisor goroutine. + stalled bool +} + +const tickStallFloor = 10 * time.Second +const tickExitFloor = 60 * time.Second + +func newTickSupervisor(now time.Time, interval time.Duration) *tickSupervisor { + s := &tickSupervisor{base: now} + s.beat(now, interval) + return s +} + +// beat records a completed tick and the control interval it ran at. +func (s *tickSupervisor) beat(now time.Time, interval time.Duration) { + s.intervalNs.Store(int64(interval)) + s.lastNs.Store(int64(now.Sub(s.base))) +} + +func tickStallAfter(interval time.Duration) time.Duration { + return max(3*interval, tickStallFloor) +} + +func tickExitAfter(interval time.Duration) time.Duration { + return max(tickExitFloor, 2*tickStallAfter(interval)) +} + +// check reports what the supervisor must do at now. sendDefaults is true once +// per stall; exit stays true while the stall lasts past tickExitAfter. +func (s *tickSupervisor) check(now time.Time) (sendDefaults, exit bool, age time.Duration) { + interval := time.Duration(s.intervalNs.Load()) + age = now.Sub(s.base) - time.Duration(s.lastNs.Load()) + if age < tickStallAfter(interval) { + if s.stalled { + slog.Info("control tick recovered", "age", age) + } + s.stalled = false + return false, false, age + } + sendDefaults = !s.stalled + s.stalled = true + return sendDefaults, age >= tickExitAfter(interval), age +} + +// run checks the tick every period until ctx ends. sendDefaults runs in its +// own goroutine, so a default that blocks cannot hold back the exit. +func (s *tickSupervisor) run(ctx context.Context, period time.Duration, now func() time.Time, sendDefaults func(), exit func()) { + t := time.NewTicker(period) + defer t.Stop() + for { + select { + case <-ctx.Done(): + return + case <-t.C: + } + send, stop, age := s.check(now()) + if send { + slog.Error("control tick stalled — reverting every driver to autonomous", "age", age) + go sendDefaults() + } + if stop { + slog.Error("control tick still stalled — exiting so the service restarts Core", "age", age) + exit() + return + } + } +} + +// sendDefaultsInParallel sends every named driver its default at once, so one +// slow driver does not hold back the others. +func sendDefaultsInParallel(names []string, send func(name string)) { + var wg sync.WaitGroup + for _, name := range names { + wg.Add(1) + go func() { + defer wg.Done() + send(name) + }() + } + wg.Wait() +} + +// withCtrlLock runs fn under ctrlMu with a deferred unlock. A panic in +// dispatch math then releases the lock, and the process exits through its +// deferred shutdown, which sends driver defaults. Without the defer, shutdown +// can hang on a replan or the HA bridge waiting for ctrlMu. +func withCtrlLock(mu *sync.Mutex, fn func()) { + mu.Lock() + defer mu.Unlock() + fn() +} diff --git a/go/cmd/ftw/tick_supervisor_clock_test.go b/go/cmd/ftw/tick_supervisor_clock_test.go new file mode 100644 index 000000000..779ad4cc7 --- /dev/null +++ b/go/cmd/ftw/tick_supervisor_clock_test.go @@ -0,0 +1,64 @@ +package main + +import ( + "testing" + "time" + "unsafe" +) + +// stepWall returns t with its wall clock moved by step and its monotonic +// reading kept, which is how time.Now looks after NTP steps the clock. It +// edits time.Time's private layout, so it checks the result and fails loudly +// if a Go release changes that layout. +func stepWall(tb testing.TB, t time.Time, step time.Duration) time.Time { + tb.Helper() + if step%time.Second != 0 { + tb.Fatal("stepWall takes whole seconds") + } + const hasMonotonic = 1 << 63 + const nsecShift = 30 + stepped := t + raw := (*struct { + wall uint64 + ext int64 + loc *time.Location + })(unsafe.Pointer(&stepped)) + if raw.wall&hasMonotonic == 0 { + tb.Fatal("stepWall needs a time from time.Now") + } + raw.wall = uint64(int64(raw.wall) + int64(step/time.Second)< tick { + t.Errorf("trial commits at byte %d, outside tickDone (%d..%d)", commit, done, tick) + } + + body := s[tick:] + body = body[:strings.Index(body, "\nfunc ")] + exits := strings.Count(body, "\n\t\t\t\tcontinue\n") + beats := strings.Count(body, "tickDone()\n\t\t\t\tcontinue\n") + if exits == 0 || beats != exits { + t.Errorf("%d tick exits by continue, %d of them call tickDone", exits, beats) + } + if !strings.HasSuffix(body, "\t\t\ttickDone()\n\t\t}\n\t}\n}\n") { + t.Error("the tick does not end with tickDone()") + } +} + +func expectOne(t *testing.T, ch <-chan struct{}, what string) { + t.Helper() + select { + case <-ch: + case <-time.After(2 * time.Second): + t.Fatalf("missing: %s", what) + } +} + +func expectNone(t *testing.T, ch <-chan struct{}, what string) { + t.Helper() + select { + case <-ch: + t.Fatalf("unexpected: %s", what) + case <-time.After(50 * time.Millisecond): + } +} diff --git a/go/cmd/sim-ocpp/ocppcp/boot_limit_test.go b/go/cmd/sim-ocpp/ocppcp/boot_limit_test.go new file mode 100644 index 000000000..811e8ece3 --- /dev/null +++ b/go/cmd/sim-ocpp/ocppcp/boot_limit_test.go @@ -0,0 +1,111 @@ +package ocppcp + +import ( + "context" + "encoding/json" + "fmt" + "net" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/ocpp" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +func freePort(t *testing.T) int { + t.Helper() + l, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + defer l.Close() + return l.Addr().(*net.TCPAddr).Port +} + +// startFTW runs FTW's Central System with id named by a loadpoint. +func startFTW(t *testing.T, id string) (*ocpp.Server, DialOpts) { + t.Helper() + for attempt := 1; ; attempt++ { + cfg := &ocpp.Config{ + Enabled: true, Bind: "127.0.0.1", HeartbeatIntervalS: 60, + Port: freePort(t), PortV201: freePort(t), + ApprovedIDs: []string{id}, ManagedIDs: []string{id}, + } + srv, err := ocpp.Start(context.Background(), cfg, telemetry.NewStore()) + if err != nil && attempt < 5 { + continue // another test took the port + } + if err != nil { + t.Fatal(err) + } + t.Cleanup(srv.Stop) + return srv, DialOpts{ + URL16: fmt.Sprintf("ws://127.0.0.1:%d", cfg.Port), + URL201: fmt.Sprintf("ws://127.0.0.1:%d", cfg.PortV201), + } + } +} + +func waitFor(t *testing.T, what string, ok func() bool) { + t.Helper() + deadline := time.Now().Add(3 * time.Second) + for !ok() { + if time.Now().After(deadline) { + t.Fatalf("timed out waiting for %s", what) + } + time.Sleep(5 * time.Millisecond) + } +} + +// Seed 207534: the charger booted at 5.0 s, a car plugged in at 7.3 s and +// drew the charger's 16 A rating from 10 s until FTW's first 0 A landed at +// 13.5 s, with 18.9 A on a 16 A fuse. With the start limit, the charger holds +// 0 A from boot, and FTW's order still sets the rate afterwards. +func TestChargerHoldsZeroAmpsFromBootUntilFTWOrder(t *testing.T) { + for _, modelID := range []string{"easee-charge-up", "charge-amps-aura", "defa-power"} { + t.Run(modelID, func(t *testing.T) { + m, ok := Lookup(modelID) + if !ok { + t.Fatalf("no model %s", modelID) + } + sim := New(m) + srv, opts := startFTW(t, sim.DialID()) + waitFor(t, "the charger to connect", func() bool { return sim.Dial(opts) == nil }) + t.Cleanup(sim.Close) + if got := sim.LimitA(); got != m.MaxAmps() { + t.Fatalf("unsteered limit %v A, want the rating %v A", got, m.MaxAmps()) + } + + if err := sim.Boot(); err != nil { + t.Fatal(err) + } + // After an accepted boot a charger reports its connectors. + if err := sim.statusPlugged(false); err != nil { + t.Fatal(err) + } + waitFor(t, "the 0 A start limit", func() bool { return sim.LimitA() == 0 }) + + if err := sim.Plug(); err != nil { + t.Fatal(err) + } + sim.Tick(time.Second) + if w := sim.PowerW(); w != 0 { + t.Fatalf("car drew %v W before FTW's first order, want 0", w) + } + + order, _ := json.Marshal(map[string]any{ + "action": "ev_set_current", "power_w": 6900.0, "voltage": 230.0, "site_phases": 3, + }) + if err := srv.Command(context.Background(), sim.DialID(), order); err != nil { + t.Fatalf("order: %v", err) + } + sim.Tick(time.Second) + if got := sim.LimitA(); got != 10 { + t.Fatalf("effective limit %v A after FTW's order, want 10 A", got) + } + if w := sim.PowerW(); w != 6900 { + t.Fatalf("car draws %v W after FTW's order, want 6900", w) + } + }) + } +} diff --git a/go/internal/api/api.go b/go/internal/api/api.go index ecc4b96a1..429a795c8 100644 --- a/go/internal/api/api.go +++ b/go/internal/api/api.go @@ -522,6 +522,8 @@ func (s *Server) routes() { s.handle("GET /api/energy/assets", Read, s.handleEnergyAssets) s.handle("GET /api/energy/history", Read, s.handleEnergyHistory) s.handle("GET /api/energy/history.csv", Read, s.handleEnergyHistoryCSV) + s.handle("GET /api/energy/battery-return", Read, s.handleBatteryReturn) + s.handle("GET /api/energy/battery-return.csv", Read, s.handleBatteryReturnCSV) s.handle("GET /api/savings/daily", Read, s.handleSavingsDaily) s.handle("GET /api/prices", Read, s.handlePrices) s.handle("GET /api/prices/zones", Read, s.handlePriceZones) @@ -784,7 +786,7 @@ func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) { // storage: surface DB corruption-recovery events from this boot so a // corrupt database is never a silent, blank-dashboard failure again. if s.deps.State != nil { - storage := map[string]any{"state": "ok", "cache": "ok"} + storage := map[string]any{"state": "ok", "cache": "ok", "history": "ok"} for _, ev := range s.deps.State.HealEvents() { storage[ev.Tier] = ev.Action // "rebuilt" | "restored" storage["last_event_ms"] = ev.AtMs @@ -1714,6 +1716,9 @@ func (s *Server) handlePostConfigWith(w http.ResponseWriter, r *http.Request, re // declared via DRIVER.config_secrets — keeps config-package // catalog-agnostic. restoreDriverConfigSecrets(&newCfg, s.deps.Cfg, s.driverSecretKeys()) + // A reviewed draft must save exactly as reviewed; the next load + // fills its unset timings instead. + newCfg.Site.FillUnsetTimings() } s.deps.CfgMu.RUnlock() @@ -1934,6 +1939,11 @@ func (s *Server) handleSetPeakImportCeiling(w http.ResponseWriter, r *http.Reque s.deps.CtrlMu.Lock() s.deps.Ctrl.PeakImportCeilingW = req.PeakImportCeilingW s.deps.CtrlMu.Unlock() + // The plan spreads each hour's import under the ceiling; a new ceiling + // changes what the plan may deliver. + if s.deps.MPC != nil && req.PeakImportCeilingW != prevW { + s.deps.MPC.RequestReplan("import_ceiling_changed") + } writeJSON(w, 200, map[string]any{"status": "ok", "peak_import_ceiling_w": req.PeakImportCeilingW}) } diff --git a/go/internal/api/api_agent_catalog.go b/go/internal/api/api_agent_catalog.go index 8bc6bf73f..bda1909f0 100644 --- a/go/internal/api/api_agent_catalog.go +++ b/go/internal/api/api_agent_catalog.go @@ -110,6 +110,8 @@ var agentAPIQueryHints = map[string]map[string]string{ "/api/agents/audit": {"before": "Audit sequence before which to page", "n": "Maximum audit rows"}, "/api/series": {"driver": "Configured driver name", "metric": "Metric or comma-separated metrics from /api/series/catalog", "range": "Time back, such as 6h", "since": "Unix milliseconds", "until": "Unix milliseconds", "points": "Maximum points per metric; 0 returns stored rows", "format": "json (default) or csv"}, "/api/energy/history.csv": {"scope": "system or asset", "asset_id": "Asset id from /api/energy/assets", "since": "Unix milliseconds", "until": "Unix milliseconds", "bucket": "Bucket duration, at least 5m", "limit": "Maximum buckets, up to 5000"}, + "/api/energy/battery-return": {"year": "Calendar year in Swedish time, 2018 to now; default last year"}, + "/api/energy/battery-return.csv": {"year": "Calendar year in Swedish time, 2018 to now; default last year"}, } // This context is requested for technical debugging, not for routine status, diff --git a/go/internal/api/api_agents.go b/go/internal/api/api_agents.go index 7634e13d6..eae2ca857 100644 --- a/go/internal/api/api_agents.go +++ b/go/internal/api/api_agents.go @@ -5,13 +5,15 @@ package api // An owner phone manages agents over its session, each grant signed by its // approval key. On the home network the box is itself the authority, so // these routes ask for no signature. They ask for something else: widening -// access needs another device on the home network, signed in with the house -// password. The box itself never counts, password or not: a driver's Lua and -// the session passthrough reach it from inside. With api.lan_auth off, which -// is the default, every device on the home network is an owner with no -// sign-in at all, and granting a remote agent from there would let any of -// them open the house to the outside. Narrowing stays open to any LAN -// client, as DELETE /api/cloud/grant does. +// access needs a browser signed in with the house password, on another +// device or on the box itself. What proves the owner is the password, not +// the address: a driver's Lua reaches the box from inside but does not know +// it, and the session passthrough is refused outright. With api.lan_auth +// off every device on the home network is an owner with no sign-in at all, +// and granting a remote agent from there would let any of them open the +// house to the outside; the page asks for a password first, and choosing +// one signs that browser in. Narrowing stays open to any LAN client, as +// DELETE /api/cloud/grant does. // // Every route is local: the session passthrough never carries them, so a // phone must use its signed messages and an agent can never reach them. @@ -86,9 +88,9 @@ type AgentFacts struct { NowMs int64 `json:"nowMs"` MarkMs int64 `json:"markMs"` // Widen says this request would pass the gate for widening access: a - // LAN client signed in with the house password, never the box itself. - // The list sets it per request, so the page stops before a code is - // spent on a grant the box would refuse. + // browser signed in with the house password. The list sets it per + // request, so the page stops before a code is spent on a grant the box + // would refuse. Widen bool `json:"widen"` // LongestMs is the longest grant the box takes for each role, so the // page offers no length the box would refuse. @@ -135,9 +137,8 @@ func (s *Server) handleAgentsClockReset(w http.ResponseWriter, r *http.Request) } // agentsGate admits a client on the home network, and with widen only one -// signed in with the house password. Loopback never is: the sign-in is not -// read there, since a driver's Lua and the session passthrough can both reach -// the box from inside it. It returns the caller's name for the audit log. +// signed in with the house password, the box itself included. It returns +// the caller's name for the audit log. func (s *Server) agentsGate(w http.ResponseWriter, r *http.Request, widen bool) (string, bool) { if s.deps == nil || s.deps.Agents == nil { writeAppLinkError(w, http.StatusServiceUnavailable, "agents are not available on this box") diff --git a/go/internal/api/api_agents_test.go b/go/internal/api/api_agents_test.go index df119ffe4..01441c397 100644 --- a/go/internal/api/api_agents_test.go +++ b/go/internal/api/api_agents_test.go @@ -307,16 +307,17 @@ func TestTheAgentListCarriesTheBoxFacts(t *testing.T) { } } -// On the box itself the house password does not count, so the list tells the -// page that this request cannot widen access before the owner spends a code -// on a grant the box would refuse. Another device signed in can. +// The list tells the page whether this request can widen access before the +// owner spends a code on a grant the box would refuse. The house password +// counts wherever the browser runs, the box itself included; without it, +// even the box cannot. func TestTheListSaysWhetherThisRequestCanWiden(t *testing.T) { resetLANGuesses(t) policy := lanAuthPolicy(testHousePassword) s := New(&Deps{Agents: newStubAgents("")}) - widen := func(url, remote string) (bool, int) { + widenWith := func(url, remote, auth string) (bool, int) { var after *http.Request - serveLANAuth(policy, lanAuthRequest(http.MethodGet, url, remote, "Bearer "+testHousePassword), + serveLANAuth(policy, lanAuthRequest(http.MethodGet, url, remote, auth), func(r *http.Request) { after = r }) w := httptest.NewRecorder() s.handleAgentsList(w, after) @@ -331,9 +332,16 @@ func TestTheListSaysWhetherThisRequestCanWiden(t *testing.T) { s.handleAgentsAdd(add, post) return got.Box.Widen, add.Code } - if ok, code := widen("http://127.0.0.1:8080/api/agents", "127.0.0.1:1234"); ok || code != http.StatusForbidden { + widen := func(url, remote string) (bool, int) { return widenWith(url, remote, "Bearer "+testHousePassword) } + if ok, code := widen("http://127.0.0.1:8080/api/agents", "127.0.0.1:1234"); !ok || code != http.StatusCreated { t.Errorf("on the box itself, with the house password: widen %v, grant %d", ok, code) } + if ok, code := widenWith("http://127.0.0.1:8080/api/agents", "127.0.0.1:1234", ""); ok || code != http.StatusForbidden { + t.Errorf("on the box itself, without it: widen %v, grant %d", ok, code) + } + if ok, code := widenWith("http://127.0.0.1:8080/api/agents", "127.0.0.1:1234", "Bearer not-the-password"); ok || code != http.StatusForbidden { + t.Errorf("on the box itself, with a wrong one: widen %v, grant %d", ok, code) + } if ok, code := widen("http://192.168.1.1:8080/api/agents", "192.168.1.5:1234"); !ok || code != http.StatusCreated { t.Errorf("another device, signed in: widen %v, grant %d", ok, code) } diff --git a/go/internal/api/api_battery_return.go b/go/internal/api/api_battery_return.go new file mode 100644 index 000000000..b7dab237b --- /dev/null +++ b/go/internal/api/api_battery_return.go @@ -0,0 +1,133 @@ +package api + +import ( + "encoding/csv" + "fmt" + "net/http" + "strconv" + "time" + + "github.com/srcfl/ftw/go/internal/batteryreturn" +) + +// Skatteverket refunds energy tax on battery-stored grid energy from 2018. +const batteryReturnFirstYear = 2018 + +// handleBatteryReturn answers how much grid energy the battery sent back to +// the grid in one Swedish calendar year: the figure for Skatteverket's refund +// of energy tax after battery storage. +func (s *Server) handleBatteryReturn(w http.ResponseWriter, r *http.Request) { + rep, status, err := s.batteryReturn(r) + if err != nil { + writeJSON(w, status, map[string]string{"error": err.Error()}) + return + } + months := make([]map[string]any, len(rep.Months)) + for i, m := range rep.Months { + months[i] = map[string]any{ + "month": i + 1, "returned_wh": m.Returned, "hours": m.Hours, "complete_hours": m.CompleteHours, + } + } + batteries := make([]map[string]any, len(rep.Batteries)) + for i, b := range rep.Batteries { + batteries[i] = map[string]any{ + "asset_id": b.AssetID, "label": b.Label, "sized": b.Sized, + "returned_wh": b.Returned, "efficiency": b.Efficiency, "capacity_wh": b.CapacityWh, + } + } + writeJSON(w, http.StatusOK, map[string]any{ + "year": rep.Year, + "zone": batteryreturn.Zone, + "returned_wh": rep.Returned, + "hours": len(rep.Hours), + "complete_hours": rep.CompleteHours(), + "batteries": batteries, + "unread": append([]string{}, rep.Unread...), + "problem": rep.Problem, + "months": months, + }) +} + +// handleBatteryReturnCSV is every hour behind the figure, for the owner to +// keep with the application: one row per battery and hour, so each row can +// be checked on its own and the figure is the sum of returned_wh. +func (s *Server) handleBatteryReturnCSV(w http.ResponseWriter, r *http.Request) { + rep, status, err := s.batteryReturn(r) + if err != nil { + http.Error(w, err.Error(), status) + return + } + w.Header().Set("Content-Type", "text/csv; charset=utf-8") + w.Header().Set("Content-Disposition", fmt.Sprintf(`attachment; filename="ftw-battery-return-%d.csv"`, rep.Year)) + cw := csv.NewWriter(w) + _ = cw.Write([]string{ + "hour_start", "battery", "grid_import_wh", "grid_export_wh", "battery_charge_wh", "battery_discharge_wh", + "solar_wh", "vehicle_discharge_wh", "other_batteries_discharge_wh", "grid_to_battery_wh", "battery_to_grid_wh", + "returned_wh", "grid_held_wh", "other_held_wh", "complete", "efficiency", "capacity_wh", + }) + wh := func(v float64) string { return strconv.FormatFloat(v, 'f', 1, 64) } + for _, h := range rep.Hours { + for i, b := range h.Batteries { + bat := rep.Batteries[i] + _ = cw.Write([]string{ + h.Start.Format(time.RFC3339), bat.AssetID, wh(h.GridImport), wh(h.GridExport), wh(b.Charge), + wh(b.Discharge), wh(h.Solar), wh(h.VehicleDischarge), wh(h.BatteryDischarge - b.Discharge), + wh(b.GridToBattery), wh(b.BatteryToGrid), wh(b.Returned), wh(b.GridHeld), wh(b.OtherHeld), + strconv.FormatBool(h.Complete), strconv.FormatFloat(b.Efficiency, 'f', 4, 64), wh(bat.CapacityWh), + }) + } + } + cw.Flush() +} + +func (s *Server) batteryReturn(r *http.Request) (batteryreturn.Report, int, error) { + if s.deps.State == nil { + return batteryreturn.Report{}, http.StatusServiceUnavailable, fmt.Errorf("energy history unavailable") + } + loc, err := time.LoadLocation(batteryreturn.Zone) + if err != nil { + return batteryreturn.Report{}, http.StatusInternalServerError, err + } + now := time.Now() + thisYear := now.In(loc).Year() + // The year an owner applies for is the one that has ended. + year := thisYear - 1 + if raw := r.URL.Query().Get("year"); raw != "" { + year, err = strconv.Atoi(raw) + if err != nil || year < batteryReturnFirstYear || year > thisYear { + return batteryreturn.Report{}, http.StatusBadRequest, + fmt.Errorf("year must be %d to %d", batteryReturnFirstYear, thisYear) + } + } + var site batteryreturn.Site + if s.deps.Cfg != nil && s.deps.CfgMu != nil { + s.deps.CfgMu.RLock() + // A charger's battery_capacity_wh is a car's, not a home battery's. + vehicles := map[string]bool{} + for _, lp := range s.deps.Cfg.Loadpoints { + vehicles[lp.DriverName] = true + } + for _, d := range s.deps.Cfg.Drivers { + if d.BatteryCapacityWh > 0 && !vehicles[d.Name] { + site.Batteries = append(site.Batteries, batteryreturn.SizedBattery{Driver: d.Name, CapacityWh: d.BatteryCapacityWh}) + } + } + if w := s.deps.Cfg.Weather; w != nil && placed(w.Latitude, w.Longitude) { + site.Lat, site.Lon, site.Placed = w.Latitude, w.Longitude, true + } + s.deps.CfgMu.RUnlock() + } + rep, err := batteryreturn.Build(r.Context(), s.deps.State, year, now, site) + if err != nil { + return batteryreturn.Report{}, http.StatusInternalServerError, err + } + return rep, http.StatusOK, nil +} + +// placed reports whether the owner set a position in Sweden, where the +// refund applies. A wrong position could put the sun down while panels +// still produce: setup's map starts on central Stockholm, so a site left +// there has none, and swapped coordinates land outside the country. +func placed(lat, lon float64) bool { + return lat >= 55 && lat <= 69.5 && lon >= 10.5 && lon <= 24.5 && !(lat == 59.3293 && lon == 18.0686) +} diff --git a/go/internal/api/api_battery_return_test.go b/go/internal/api/api_battery_return_test.go new file mode 100644 index 000000000..09c839893 --- /dev/null +++ b/go/internal/api/api_battery_return_test.go @@ -0,0 +1,216 @@ +package api + +import ( + "encoding/csv" + "encoding/json" + "math" + "net/http" + "net/http/httptest" + "path/filepath" + "strconv" + "strings" + "sync" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/batteryreturn" + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/state" +) + +// seedBatteryReturn records three cycles of two hours charging from the +// grid and one hour sending the battery to the grid, ending in the hour +// before now. One cycle proves no efficiency: the battery may have started +// full. Each driver named reads a battery that does the same; the first is +// "battery". +func seedBatteryReturn(t *testing.T, now time.Time, drivers ...string) *state.Store { + t.Helper() + if len(drivers) == 0 { + drivers = []string{"battery"} + } + st, err := state.Open(filepath.Join(t.TempDir(), "state.db")) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = st.Close() }) + base := now.Truncate(time.Hour).Add(-10 * time.Hour) + var obs []state.EnergyObservation + var imported, exported float64 + prev := base.Add(-10 * time.Minute) + for at := prev; at.Before(now); at = at.Add(2 * time.Minute) { + charge, discharge := 0.0, 0.0 + if h := int(at.Sub(base).Hours()); h >= 0 && h < 9 { + if h%3 < 2 { + charge = 2000 + } else { + discharge = 3600 + } + } + dt := at.Sub(prev).Hours() + imported += charge * dt * float64(len(drivers)) + exported += discharge * dt * float64(len(drivers)) + prev = at + obs = append(obs, + state.EnergyObservation{AssetID: "meter:1/grid_meter", DeviceID: "meter:1", AssetKind: state.AssetGridMeter, Flow: state.FlowGridImport, AtMs: at.UnixMilli(), CounterWh: apiEnergyPtr(imported)}, + state.EnergyObservation{AssetID: "meter:1/grid_meter", DeviceID: "meter:1", AssetKind: state.AssetGridMeter, Flow: state.FlowGridExport, AtMs: at.UnixMilli(), CounterWh: apiEnergyPtr(exported)}, + ) + for i, driver := range drivers { + device := "inverter:" + strconv.Itoa(i+1) + obs = append(obs, + state.EnergyObservation{AssetID: device + "/battery", DeviceID: device, AssetKind: state.AssetBattery, Label: driver, Flow: state.FlowBatteryCharge, AtMs: at.UnixMilli(), PowerW: apiEnergyPtr(charge)}, + state.EnergyObservation{AssetID: device + "/battery", DeviceID: device, AssetKind: state.AssetBattery, Label: driver, Flow: state.FlowBatteryDischarge, AtMs: at.UnixMilli(), PowerW: apiEnergyPtr(discharge)}, + ) + } + } + if err := st.RecordTickWithOptionalHistory(nil, nil, obs); err != nil { + t.Fatal(err) + } + return st +} + +func TestBatteryReturnAPIGivesTheYearAndEveryHour(t *testing.T) { + loc, err := time.LoadLocation(batteryreturn.Zone) + if err != nil { + t.Fatal(err) + } + now := time.Now() + year := now.In(loc).Year() + st := seedBatteryReturn(t, now) + cfg := &config.Config{Drivers: []config.Driver{{Name: "battery", BatteryCapacityWh: 4000}}} + srv := New(&Deps{State: st, Cfg: cfg, CfgMu: &sync.RWMutex{}}) + get := func(path string) *httptest.ResponseRecorder { + rr := httptest.NewRecorder() + srv.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, path, nil)) + return rr + } + + rr := get("/api/energy/battery-return?year=" + strconv.Itoa(year)) + if rr.Code != http.StatusOK { + t.Fatalf("status=%d body=%s", rr.Code, rr.Body.String()) + } + var got struct { + Year int `json:"year"` + Zone string `json:"zone"` + ReturnedWh float64 `json:"returned_wh"` + Hours int `json:"hours"` + CompleteHours int `json:"complete_hours"` + Problem string `json:"problem"` + Months []struct { + Month int `json:"month"` + } `json:"months"` + } + if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil { + t.Fatal(err) + } + if got.Year != year || got.Zone != batteryreturn.Zone || len(got.Months) != 12 || got.CompleteHours < 3 || got.Problem != "" { + t.Fatalf("response %+v", got) + } + // Early on 1 January the charging hours belong to last year. + if now.Add(-11*time.Hour).In(loc).Year() == year && got.ReturnedWh <= 0 { + t.Fatalf("returned %.1f Wh after charging from the grid and exporting", got.ReturnedWh) + } + + rr = get("/api/energy/battery-return.csv?year=" + strconv.Itoa(year)) + if rr.Code != http.StatusOK || !strings.Contains(rr.Header().Get("Content-Disposition"), "ftw-battery-return-"+strconv.Itoa(year)+".csv") { + t.Fatalf("csv status=%d headers=%v", rr.Code, rr.Header()) + } + rows, err := csv.NewReader(rr.Body).ReadAll() + if err != nil { + t.Fatal(err) + } + if len(rows) != got.Hours+1 || rows[0][0] != "hour_start" || rows[0][11] != "returned_wh" || rows[1][1] != "inverter:1/battery" || rows[1][16] != "4000.0" { + t.Fatalf("%d rows for %d hours, header %v", len(rows), got.Hours, rows[0]) + } + + // The default is the year an owner applies for. + rr = get("/api/energy/battery-return") + if rr.Code != http.StatusOK || !strings.Contains(rr.Body.String(), `"year":`+strconv.Itoa(year-1)) { + t.Fatalf("default status=%d body=%s", rr.Code, rr.Body.String()) + } + // Without a configured battery size there is no figure. + rr = httptest.NewRecorder() + New(&Deps{State: st}).Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/api/energy/battery-return?year="+strconv.Itoa(year), nil)) + if rr.Code != http.StatusOK || !strings.Contains(rr.Body.String(), `"problem":"battery_size"`) || !strings.Contains(rr.Body.String(), `"returned_wh":0`) { + t.Fatalf("no size: status=%d body=%s", rr.Code, rr.Body.String()) + } + for _, bad := range []string{"2017", strconv.Itoa(year + 1), "last"} { + if rr := get("/api/energy/battery-return?year=" + bad); rr.Code != http.StatusBadRequest { + t.Fatalf("year=%s status=%d", bad, rr.Code) + } + } +} + +func TestBatteryReturnAPICountsEachBattery(t *testing.T) { + loc, err := time.LoadLocation(batteryreturn.Zone) + if err != nil { + t.Fatal(err) + } + now := time.Now() + year := strconv.Itoa(now.In(loc).Year()) + st := seedBatteryReturn(t, now, "battery", "garage") + // The car's size does not make its charger a battery FTW must read. + cfg := &config.Config{ + Drivers: []config.Driver{{Name: "battery", BatteryCapacityWh: 4000}, {Name: "garage", BatteryCapacityWh: 4000}, {Name: "car", BatteryCapacityWh: 60_000}}, + Loadpoints: []config.Loadpoint{{ID: "drive", DriverName: "car"}}, + } + srv := New(&Deps{State: st, Cfg: cfg, CfgMu: &sync.RWMutex{}}) + rr := httptest.NewRecorder() + srv.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/api/energy/battery-return?year="+year, nil)) + var got struct { + ReturnedWh float64 `json:"returned_wh"` + Hours int `json:"hours"` + Problem string `json:"problem"` + Unread []string `json:"unread"` + Batteries []struct { + Label string `json:"label"` + Sized bool `json:"sized"` + ReturnedWh float64 `json:"returned_wh"` + } `json:"batteries"` + } + if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil { + t.Fatal(err) + } + if got.Problem != "" || len(got.Unread) != 0 || len(got.Batteries) != 2 || !got.Batteries[0].Sized || !got.Batteries[1].Sized || + got.Batteries[0].ReturnedWh != got.Batteries[1].ReturnedWh || math.Abs(got.Batteries[0].ReturnedWh+got.Batteries[1].ReturnedWh-got.ReturnedWh) > 0.01 { + t.Fatalf("response %s", rr.Body.String()) + } + + rr = httptest.NewRecorder() + srv.Handler().ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/api/energy/battery-return.csv?year="+year, nil)) + rows, err := csv.NewReader(rr.Body).ReadAll() + if err != nil { + t.Fatal(err) + } + // One row per battery and hour; the figure is the sum of returned_wh. + if len(rows) != 2*got.Hours+1 || rows[1][1] != "inverter:1/battery" || rows[2][1] != "inverter:2/battery" || rows[1][0] != rows[2][0] { + t.Fatalf("%d rows for %d hours: %v %v", len(rows), got.Hours, rows[1], rows[2]) + } + var sum float64 + for _, row := range rows[1:] { + v, err := strconv.ParseFloat(row[11], 64) + if err != nil { + t.Fatal(err) + } + sum += v + } + if math.Abs(sum-got.ReturnedWh) > 0.1*float64(len(rows)) { + t.Fatalf("csv sums to %.1f Wh, figure %.1f", sum, got.ReturnedWh) + } +} + +func TestOnlyASwedishPositionPlacesTheSite(t *testing.T) { + for _, c := range []struct { + lat, lon float64 + want bool + }{ + {59.6, 16.5, true}, {67.86, 20.23, true}, {55.6, 13.0, true}, + {16.5, 59.6, false}, // swapped + {59.6, -16.5, false}, // lost sign + {59.3293, 18.0686, false}, // setup's starting pin + {0, 0, false}, {60.17, 24.94 + 1, false}, + } { + if got := placed(c.lat, c.lon); got != c.want { + t.Fatalf("placed(%v, %v) = %v", c.lat, c.lon, got) + } + } +} diff --git a/go/internal/api/api_battery_soc_max_test.go b/go/internal/api/api_battery_soc_max_test.go new file mode 100644 index 000000000..e0a8038e9 --- /dev/null +++ b/go/internal/api/api_battery_soc_max_test.go @@ -0,0 +1,42 @@ +package api + +import ( + "encoding/json" + "sync" + "testing" + "testing/synctest" + "time" + + "github.com/srcfl/ftw/go/internal/control" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +func TestSoCMaxFeedbackUsesTheDispatchResumeLevel(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + tel := telemetry.NewStore() + soc := .9 + tel.Update("battery", telemetry.DerBattery, 0, &soc, nil) + tel.RecordDriverSuccess("battery") + tel.Update("meter", telemetry.DerMeter, 1000, nil, nil) + tel.RecordDriverSuccess("meter") + st := control.NewState(0, 50, "meter") + st.Mode = control.ModeCharge + st.DriverLimits = map[string]control.PowerLimits{"battery": {SoCMax: .9}} + got := control.ComputeDispatch(tel, st, map[string]float64{"battery": 15000}, 13800) + if len(got) != 1 || got[0].TargetW != 0 { + t.Fatalf("charge not stopped: %+v", got) + } + request, _ := json.Marshal(map[string]any{"action": "battery", "power_w": 0}) + cmd := tel.BeginCommand("battery", request, time.Now()) + tel.CompleteCommand(cmd, "accepted") + for range 16 { + tel.Update("battery", telemetry.DerBattery, 0, &soc, []byte(`{"setpoint_w":0}`)) + time.Sleep(3 * time.Second) + } + srv := New(&Deps{Tel: tel, Ctrl: st, CtrlMu: &sync.Mutex{}}) + f := srv.controlFeedback(time.Now())[0] + if f.Status != "limited" || f.Reason != "battery_full" || f.Severity != "info" || f.ChargeResumeSoC == nil || *f.ChargeResumeSoC != .89 { + t.Fatalf("wrong explanation of the 90%% stop: %+v", f) + } + }) +} diff --git a/go/internal/api/api_control_feedback.go b/go/internal/api/api_control_feedback.go index eb5abc406..bd15a1553 100644 --- a/go/internal/api/api_control_feedback.go +++ b/go/internal/api/api_control_feedback.go @@ -35,27 +35,31 @@ type ControlFeedback struct { SiteEvidence *ControlComparison `json:"site_evidence,omitempty"` ToleranceW *float64 `json:"tolerance_w"` // Internal steps of the verdict; the API exposes their result above. - State string `json:"-"` - VerificationTier *int `json:"-"` - VerificationLost bool `json:"-"` - Escalated bool `json:"-"` // not following long enough to be an alarm - Response string `json:"-"` - SiteBeforeW *float64 `json:"-"` - SiteAfterW *float64 `json:"-"` - SiteBeforeAtMs int64 `json:"-"` - SiteAfterAtMs int64 `json:"-"` - SiteDeltaW *float64 `json:"-"` - DeviceDeltaW *float64 `json:"-"` - VerifiedAtMs int64 `json:"-"` - RequestedW *float64 `json:"requested_w"` - SentW *float64 `json:"sent_w"` - ReadbackW *float64 `json:"readback_w"` - ActualW *float64 `json:"actual_w"` - BatterySoC *float64 `json:"battery_soc,omitempty"` - ChargeResumeSoC *float64 `json:"charge_resume_soc,omitempty"` - RequestedA *float64 `json:"requested_a"` - OfferedA *float64 `json:"offered_a"` - DeviceLimitA *float64 `json:"device_limit_a"` + State string `json:"-"` + VerificationTier *int `json:"-"` + VerificationLost bool `json:"-"` + Escalated bool `json:"-"` // not following long enough to be an alarm + Response string `json:"-"` + SiteBeforeW *float64 `json:"-"` + SiteAfterW *float64 `json:"-"` + SiteBeforeAtMs int64 `json:"-"` + SiteAfterAtMs int64 `json:"-"` + SiteDeltaW *float64 `json:"-"` + DeviceDeltaW *float64 `json:"-"` + VerifiedAtMs int64 `json:"-"` + RequestedW *float64 `json:"requested_w"` + SentW *float64 `json:"sent_w"` + ReadbackW *float64 `json:"readback_w"` + ActualW *float64 `json:"actual_w"` + BatterySoC *float64 `json:"battery_soc,omitempty"` + ChargeResumeSoC *float64 `json:"charge_resume_soc,omitempty"` + RequestedA *float64 `json:"requested_a"` + OfferedA *float64 `json:"offered_a"` + DeviceLimitA *float64 `json:"device_limit_a"` + PilotA *float64 `json:"pilot_a"` + PilotObservedAt string `json:"pilot_observed_at,omitempty"` + PilotConfirmedAtMs int64 `json:"pilot_confirmed_at_ms,omitempty"` + DynamicLimitA *float64 `json:"dynamic_limit_a"` // LimitA is the current a charger lets the car take now, when its driver // reports it below the order; LimitedBy names the cause when the driver // shows it, and Make the charger for the sentence. @@ -73,15 +77,19 @@ type ControlFeedback struct { } type feedbackReading struct { - SetpointW *float64 `json:"setpoint_w"` - MaxA *float64 `json:"max_a"` - DeviceLimitA *float64 `json:"device_limit_a"` - DeviceLimitAgeS *float64 `json:"device_limit_age_s"` - Reason string `json:"reason_no_current_label"` - LimitA *float64 `json:"current_limit_a"` - LimitedBy string `json:"current_limited_by"` - Connected *bool `json:"connected"` - Online *bool `json:"is_online"` + SetpointW *float64 `json:"setpoint_w"` + MaxA *float64 `json:"max_a"` + PilotA *float64 `json:"pilot_a"` + PilotObservedAt string `json:"pilot_observed_at"` + PilotConfirmedAtMs *int64 `json:"pilot_confirmed_at_ms"` + DynamicLimitA *float64 `json:"dynamic_limit_a"` + DeviceLimitA *float64 `json:"device_limit_a"` + DeviceLimitAgeS *float64 `json:"device_limit_age_s"` + Reason string `json:"reason_no_current_label"` + LimitA *float64 `json:"current_limit_a"` + LimitedBy string `json:"current_limited_by"` + Connected *bool `json:"connected"` + Online *bool `json:"is_online"` } func watts(v float64) *float64 { @@ -98,7 +106,7 @@ func (s *Server) controlFeedback(now time.Time) []ControlFeedback { } mode, meter := "", "" clamped := map[string]bool{} - full := map[string]bool{} + full := map[string]float64{} targeted := map[string]bool{} var hold control.BatteryManualHold var held bool @@ -108,7 +116,9 @@ func (s *Server) controlFeedback(now time.Time) []ControlFeedback { meter = s.deps.Ctrl.SiteMeterDriver for _, t := range s.deps.Ctrl.LastTargets { clamped[t.Driver] = t.Clamped - full[t.Driver] = s.deps.Ctrl.BatteryChargePaused(t.Driver) + if s.deps.Ctrl.BatteryChargePaused(t.Driver) { + full[t.Driver] = s.deps.Ctrl.BatteryChargeResumeSoC(t.Driver) + } targeted[t.Driver] = true } hold, held = s.deps.Ctrl.GetBatteryManualHold(now) @@ -156,6 +166,22 @@ func (s *Server) controlFeedback(now time.Time) []ControlFeedback { var d feedbackReading _ = json.Unmarshal(rd.Data, &d) f.ReadbackW, f.OfferedA, f.DeviceReason = d.SetpointW, d.MaxA, d.Reason + f.DynamicLimitA = d.DynamicLimitA + pilotAge, pilotValid := pilotReadingAge(d, now) + pilotFresh := pilotValid && pilotAge <= pilotFreshFor + if pilotFresh { + f.PilotA, f.PilotObservedAt, f.PilotConfirmedAtMs = d.PilotA, d.PilotObservedAt, *d.PilotConfirmedAtMs + } + // New pilot-aware drivers must provide usable evidence. Older + // drivers keep their existing normalized-limit contract, and so + // does a valid pilot that is only too old: easee_cloud re-reads it + // once a minute, a late read can pass 90 s, and Core keeps the + // last reading for a minute after the driver stops sending. The + // driver checked its limit against the pilot when it was fresh, + // so only the pilot becomes unknown. + if d.PilotConfirmedAtMs != nil && (!pilotValid || (pilotFresh && *d.PilotA == 0)) { + d.LimitA, d.LimitedBy = nil, "" + } if d.LimitA != nil { f.LimitA, f.LimitedBy = d.LimitA, d.LimitedBy if s.deps.Registry != nil { @@ -202,9 +228,9 @@ func (s *Server) controlFeedback(now time.Time) []ControlFeedback { if clamped[rd.Driver] && kind == telemetry.DerBattery { coreReason = "core_limit" } - if full[rd.Driver] && kind == telemetry.DerBattery && f.BatterySoC != nil { + if resume, paused := full[rd.Driver]; paused && kind == telemetry.DerBattery && f.BatterySoC != nil { coreReason = "battery_full" - f.ChargeResumeSoC = watts(control.BatteryChargeResumeSoC) + f.ChargeResumeSoC = watts(resume) } if held && kind == telemetry.DerBattery && (hold.Driver == "" || hold.Driver == rd.Driver) { f.Mode = "manual" @@ -373,7 +399,7 @@ func setControlStatus(f *ControlFeedback) { "vehicle_complete", "vehicle_limit_completion", "vehicle_not_requesting": status = "following" case "battery_full", "battery_nearly_full", "battery_nearly_empty", "core_limit", - "fuse_limit", "fuse_cooldown", "charger_limit", "load_balancer_limit", "one_phase": + "fuse_limit", "fuse_cooldown", "charger_limit", "load_balancer_limit", "charger_pilot_limit", "one_phase": status = "limited" case "import_limit": status = "limited" @@ -442,6 +468,8 @@ func classifyControlFeedback(f *ControlFeedback, cmd telemetry.CommandEvidence, set := func(state, reason, severity string) { f.State, f.Reason, f.Severity = state, reason, severity } if !fresh { f.ActualW, f.ReadbackW, f.OfferedA, f.DeviceLimitA, f.LimitA, f.LimitedBy, f.Make = nil, nil, nil, nil, nil, "", "" + f.PilotA, f.DynamicLimitA = nil, nil + f.PilotObservedAt, f.PilotConfirmedAtMs = "", 0 set("unknown", "telemetry_stale", "warning") return } @@ -476,7 +504,10 @@ func classifyControlFeedback(f *ControlFeedback, cmd telemetry.CommandEvidence, set("warning", "no_power_response", "warning") return } - if f.RequestedA != nil && f.DeviceLimitA != nil && *f.RequestedA > *f.DeviceLimitA+0.5 { + // With a fresh pilot and the current the charger lets the car take, the + // draw is judged against them below, so the charger's own limit cannot + // hide a car taking less. + if !(f.PilotA != nil && f.LimitA != nil) && f.RequestedA != nil && f.DeviceLimitA != nil && *f.RequestedA > *f.DeviceLimitA+0.5 { set("limited", "device_limit", "warning") return } @@ -512,9 +543,10 @@ func classifyControlFeedback(f *ControlFeedback, cmd telemetry.CommandEvidence, // shows it (easee_cloud 1.4.1). That explains a car taking about that // much, never more: within 10 %, or as the car follows a step, within // 2 A but at most 15 % of the limit. A car taking far less, nothing, or - // more keeps its verdict. A lower current whose cause the driver does - // not show still asks for a look. - if f.Kind == "ev" && f.SentW != nil && f.RequestedW != nil && f.RequestedA != nil && f.LimitA != nil && f.ActualW != nil && + // more keeps its verdict. A fresh pilot names the charger as the cause + // only when it is below the order and is that current. A lower current + // whose cause the driver does not show still asks for a look. + if f.Kind == "ev" && commanded && cmd.Result == "accepted" && f.SentW != nil && f.RequestedW != nil && f.RequestedA != nil && f.LimitA != nil && f.ActualW != nil && *f.SentW > 100 && *f.RequestedA > 0 && *f.LimitA > 0 && *f.LimitA+0.5 < *f.RequestedA && settled { perA := *f.RequestedW / *f.RequestedA limitW := *f.LimitA * perA @@ -522,6 +554,12 @@ func classifyControlFeedback(f *ControlFeedback, cmd telemetry.CommandEvidence, if *f.ActualW >= limitW-math.Max(tolerance, math.Min(2*perA, 0.15*limitW)) && *f.ActualW <= limitW+tolerance && *f.ActualW < *f.SentW { if f.LimitedBy == "load_balancer" { set("limited", "load_balancer_limit", "info") + } else if f.PilotA != nil && *f.PilotA+0.5 < *f.RequestedA && math.Abs(*f.PilotA-*f.LimitA) <= 0.5 { + if f.DeviceLimitA != nil && math.Abs(*f.LimitA-*f.DeviceLimitA) <= 0.5 { + set("limited", "device_limit", "info") + } else { + set("limited", "charger_pilot_limit", "info") + } } else { set("limited", "offered_current_lower", "warning") } @@ -661,3 +699,22 @@ func (s *Server) loadpointsWithFeedback(states []loadpoint.State) []loadpointFee } return out } + +// pilotFreshFor is how long a pilot's confirmation can explain a draw; it +// matches the 90 s for which easee_cloud vouches for a cloud read. +const pilotFreshFor = 90 * time.Second + +// pilotReadingAge is how long ago a pilot-aware driver confirmed its pilot. +// ok is false when the pilot is not a current, or its confirmation is +// missing or lies after now, or its source time after the confirmation. +func pilotReadingAge(d feedbackReading, now time.Time) (age time.Duration, ok bool) { + if d.PilotA == nil || d.PilotConfirmedAtMs == nil || *d.PilotConfirmedAtMs <= 0 || math.IsNaN(*d.PilotA) || math.IsInf(*d.PilotA, 0) || *d.PilotA < 0 { + return 0, false + } + confirmed := time.UnixMilli(*d.PilotConfirmedAtMs) + source, err := time.Parse(time.RFC3339Nano, d.PilotObservedAt) + if err != nil || confirmed.After(now) || source.After(confirmed) { + return 0, false + } + return now.Sub(confirmed), true +} diff --git a/go/internal/api/api_control_states.go b/go/internal/api/api_control_states.go new file mode 100644 index 000000000..564da4bdd --- /dev/null +++ b/go/internal/api/api_control_states.go @@ -0,0 +1,20 @@ +package api + +import ( + "time" + + "github.com/srcfl/ftw/go/internal/notifications" +) + +// ControlStates is the box's answer to "Are we in control?" for the +// control.not_following notification rule: the same verdicts GET +// /api/status serves in control_feedback, so a push never says what the +// box's page does not. +func (s *Server) ControlStates(now time.Time) []notifications.ControlState { + rows := s.controlFeedback(now) + out := make([]notifications.ControlState, 0, len(rows)) + for _, f := range rows { + out = append(out, notifications.ControlState{Driver: f.Driver, Kind: f.Kind, Status: f.Status}) + } + return out +} diff --git a/go/internal/api/api_control_states_test.go b/go/internal/api/api_control_states_test.go new file mode 100644 index 000000000..98ad91a5d --- /dev/null +++ b/go/internal/api/api_control_states_test.go @@ -0,0 +1,47 @@ +package api + +import ( + "encoding/json" + "sync" + "testing" + "testing/synctest" + "time" + + "github.com/srcfl/ftw/go/internal/control" + "github.com/srcfl/ftw/go/internal/notifications" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// The control.not_following rule reads the same verdict GET /api/status +// serves: a battery that charges while FTW asked it to discharge. +func TestControlStatesAreTheStatusDocumentsVerdicts(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + tel := telemetry.NewStore() + read := func(power float64) { + tel.Update("hybrid", telemetry.DerBattery, power, nil, []byte(`{}`)) + tel.Update("hybrid", telemetry.DerMeter, 800+power, nil, []byte(`{"power_origin":"external_meter"}`)) + tel.RecordDriverSuccess("hybrid") + } + for range 5 { + read(0) + time.Sleep(3 * time.Second) + } + request, _ := json.Marshal(map[string]any{"action": "battery", "power_w": -5000}) + tel.CompleteCommand(tel.BeginCommand("hybrid", request, time.Now()), "accepted") + for range 15 { + read(4400) + time.Sleep(3 * time.Second) + } + srv := New(&Deps{Tel: tel, Ctrl: &control.State{SiteMeterDriver: "hybrid"}, CtrlMu: &sync.Mutex{}}) + now := time.Now() + rows := srv.controlFeedback(now) + got := srv.ControlStates(now) + if len(got) != len(rows) || len(got) == 0 { + t.Fatalf("states %+v for rows %+v", got, rows) + } + want := notifications.ControlState{Driver: "hybrid", Kind: "battery", Status: "not_following"} + if got[0] != want || rows[0].Reason != "power_wrong_direction" { + t.Fatalf("got %+v (%s), want %+v", got[0], rows[0].Reason, want) + } + }) +} diff --git a/go/internal/api/api_lan_auth_test.go b/go/internal/api/api_lan_auth_test.go index 6a43c40c4..208d2ff65 100644 --- a/go/internal/api/api_lan_auth_test.go +++ b/go/internal/api/api_lan_auth_test.go @@ -484,46 +484,38 @@ func TestAuthPasswordEnableAndStatus(t *testing.T) { t.Fatalf("empty enable status = %d, want 400 (body=%s)", rr.Code, rr.Body.String()) } - short := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/auth/password", strings.NewReader(`{"password":"short","enabled":true}`)) - short.RemoteAddr = "127.0.0.1:43210" - short.Header.Set("Content-Type", "application/json") + long := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/auth/password", + strings.NewReader(`{"password":"`+strings.Repeat("x", maxLANPasswordLen+1)+`","enabled":true}`)) + long.RemoteAddr = "127.0.0.1:43210" + long.Header.Set("Content-Type", "application/json") rr = httptest.NewRecorder() - srv.Handler().ServeHTTP(rr, short) + srv.Handler().ServeHTTP(rr, long) if rr.Code != http.StatusBadRequest { - t.Fatalf("short enable status = %d, want 400 (body=%s)", rr.Code, rr.Body.String()) + t.Fatalf("over-long enable status = %d, want 400 (body=%s)", rr.Code, rr.Body.String()) } + // The first password may come from a LAN device: while the lock is off + // it is already an owner. Choosing it signs that browser in. body := `{"password":"` + testHousePassword + `","enabled":true}` lanEnable := httptest.NewRequest(http.MethodPost, "http://ftw.local:8080/api/auth/password", strings.NewReader(body)) lanEnable.RemoteAddr = "192.168.1.10:43210" lanEnable.Header.Set("Content-Type", "application/json") rr = httptest.NewRecorder() srv.Handler().ServeHTTP(rr, lanEnable) - if rr.Code != http.StatusForbidden { - t.Fatalf("first LAN enable status = %d, want 403 (body=%s)", rr.Code, rr.Body.String()) - } - - // Docker Desktop published ports SNAT to the bridge gateway, not - // loopback. That peer is RFC1918, same as a LAN visitor. - bridge := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/auth/password", strings.NewReader(body)) - bridge.RemoteAddr = "172.17.0.1:43210" - bridge.Header.Set("Content-Type", "application/json") - rr = httptest.NewRecorder() - srv.Handler().ServeHTTP(rr, bridge) - if rr.Code != http.StatusForbidden { - t.Fatalf("docker-bridge enable status = %d, want 403 (body=%s)", rr.Code, rr.Body.String()) + if rr.Code != http.StatusOK { + t.Fatalf("first LAN enable status = %d, want 200 (body=%s)", rr.Code, rr.Body.String()) } - if !strings.Contains(rr.Body.String(), "docker compose") { - t.Fatalf("403 body should name the Docker Desktop exec path, got %s", rr.Body.String()) + if !strings.Contains(rr.Body.String(), `"signed_in":true`) { + t.Fatalf("first LAN enable should sign the browser in: %s", rr.Body.String()) } - - post := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/auth/password", strings.NewReader(body)) - post.RemoteAddr = "127.0.0.1:43210" - post.Header.Set("Content-Type", "application/json") + cookie := lanSessionFromRecorder(t, rr) + signedIn := httptest.NewRequest(http.MethodGet, "http://ftw.local:8080/api/config", nil) + signedIn.RemoteAddr = "192.168.1.10:43210" + signedIn.AddCookie(&http.Cookie{Name: lanSessionCookieName, Value: cookie}) rr = httptest.NewRecorder() - srv.Handler().ServeHTTP(rr, post) + srv.Handler().ServeHTTP(rr, signedIn) if rr.Code != http.StatusOK { - t.Fatalf("loopback enable status = %d, want 200 (body=%s)", rr.Code, rr.Body.String()) + t.Fatalf("signed-in config read = %d, want 200 (body=%s)", rr.Code, rr.Body.String()) } req = httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/auth/status", nil) @@ -555,6 +547,179 @@ func TestAuthPasswordEnableAndStatus(t *testing.T) { } } +// Where the request comes from does not decide who may set the first +// password: a LAN device, a published Docker port and the box itself all +// may, and each is signed in. Afterwards the password is what counts. +func TestTheFirstHousePasswordFromAnyAddress(t *testing.T) { + for _, remote := range []string{"192.168.1.10:43210", "172.17.0.1:43210", "127.0.0.1:43210", "[::1]:43210"} { + t.Run(remote, func(t *testing.T) { + resetLANGuesses(t) + srv := newLANAuthServer(t) + body := `{"password":"` + testHousePassword + `","enabled":true}` + rr := postLANAuthJSON(srv, http.MethodPost, "http://ftw.local:8080/api/auth/password", remote, body) + if rr.Code != http.StatusOK { + t.Fatalf("first enable = %d, want 200 (body=%s)", rr.Code, rr.Body.String()) + } + if token := lanSessionFromRecorder(t, rr); !lanSessionValid(token) { + t.Fatal("the browser that chose the password is not signed in") + } + + // Someone else on the network cannot change it without it. + again := `{"password":"another-house-pass","enabled":true}` + rr = postLANAuthJSON(srv, http.MethodPost, "http://ftw.local:8080/api/auth/password", "192.168.1.77:5000", again) + if rr.Code != http.StatusUnauthorized { + t.Fatalf("second device changing it = %d, want 401 (body=%s)", rr.Code, rr.Body.String()) + } + if !VerifyStoredLANSecret(srv.deps.State, testHousePassword) { + t.Fatal("the first password was replaced") + } + }) + } +} + +// Two first enables race: Authenticate let both in while the lock was off, +// and the write lock serialises them. The second proved nothing once the +// lock is on, so it may not replace the first browser's password. +func TestASecondFirstEnableCannotReplaceTheFirst(t *testing.T) { + resetLANGuesses(t) + srv := newLANAuthServer(t) + first := postLANAuthJSON(srv, http.MethodPost, "http://ftw.local:8080/api/auth/password", "192.168.1.10:43210", + `{"password":"`+testHousePassword+`","enabled":true}`) + if first.Code != http.StatusOK { + t.Fatalf("first = %d (body=%s)", first.Code, first.Body.String()) + } + // The second request as the handler sees it after Authenticate ran with + // the lock still off: a LAN owner with no proof on its context. + for _, body := range []string{`{"password":"visitor-pass","enabled":true}`, `{"enabled":false}`} { + late := httptest.NewRequest(http.MethodPost, "http://ftw.local:8080/api/auth/password", strings.NewReader(body)) + late.RemoteAddr = "192.168.1.77:5000" + late.Header.Set("Content-Type", "application/json") + late = late.WithContext(apiauth.WithCaller(late.Context(), localCaller(late))) + rr := httptest.NewRecorder() + srv.handleAuthPassword(rr, late) + if rr.Code != http.StatusUnauthorized { + t.Fatalf("%s after the lock came on = %d, want 401 (body=%s)", body, rr.Code, rr.Body.String()) + } + } + if !VerifyStoredLANSecret(srv.deps.State, testHousePassword) || !srv.deps.Cfg.API.LANAuth { + t.Fatal("the late request changed the lock") + } +} + +// The box itself may replace a password it never proved, as it may change +// any setting, but that does not sign it in. Signed in, it stays signed in. +func TestTheBoxItselfChangingThePasswordEarnsNoSignIn(t *testing.T) { + resetLANGuesses(t) + srv := newLANAuthServer(t) + enableStoredLANAuth(t, srv) + rr := postLANAuthJSON(srv, http.MethodPost, "http://127.0.0.1:8080/api/auth/password", "127.0.0.1:43210", + `{"password":"replaced-on-the-box","enabled":true}`) + if rr.Code != http.StatusOK || strings.Contains(rr.Body.String(), `"signed_in":true`) { + t.Fatalf("unproved change on the box = %d %s, want 200 without a sign-in", rr.Code, rr.Body.String()) + } + for _, c := range rr.Result().Cookies() { + if c.Name == lanSessionCookieName && c.Value != "" { + t.Fatal("an unproved change set a session cookie") + } + } + + req := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/auth/password", + strings.NewReader(`{"password":"changed-while-signed-in","enabled":true}`)) + req.RemoteAddr = "127.0.0.1:43210" + req.Header.Set("Content-Type", "application/json") + req.Header.Set("Authorization", "Bearer replaced-on-the-box") + rr = httptest.NewRecorder() + srv.Handler().ServeHTTP(rr, req) + if rr.Code != http.StatusOK || !strings.Contains(rr.Body.String(), `"signed_in":true`) { + t.Fatalf("proved change on the box = %d %s, want a sign-in", rr.Code, rr.Body.String()) + } +} + +// The browser on the box itself signs in with the cookie, and that cookie +// lets it widen an assistant's access. +func TestACookieOnTheBoxItselfCanWiden(t *testing.T) { + resetLANGuesses(t) + srv := newLANAuthServer(t) + srv.deps.Agents = newStubAgents("") + rr := postLANAuthJSON(srv, http.MethodPost, "http://127.0.0.1:8080/api/auth/password", "127.0.0.1:43210", + `{"password":"`+testHousePassword+`","enabled":true}`) + token := lanSessionFromRecorder(t, rr) + list := func(cookie string) bool { + req := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/agents", nil) + req.RemoteAddr = "127.0.0.1:43210" + if cookie != "" { + req.AddCookie(&http.Cookie{Name: lanSessionCookieName, Value: cookie}) + } + rr := httptest.NewRecorder() + srv.Handler().ServeHTTP(rr, req) + var got struct { + Box AgentFacts `json:"box"` + } + _ = json.Unmarshal(rr.Body.Bytes(), &got) + return got.Box.Widen + } + if !list(token) { + t.Fatal("the box's own browser, signed in, cannot widen") + } + if list("") { + t.Fatal("the box itself widens without signing in") + } +} + +// An app session reaches the box looking like loopback. Its own key proves +// it; a house password in its headers is never read. +func TestAnAppSessionNeverPresentsTheHousePassword(t *testing.T) { + resetLANGuesses(t) + req := sessionRequest(http.MethodGet, "/api/agents", "") + req.Header.Set("Authorization", "Bearer "+testHousePassword) + ok, after := resolveLANSecret(req, lanAuthPolicy(testHousePassword)) + if _, checked := lanSecretFrom(after.Context()); ok || checked { + t.Fatalf("a session's header was read as the house password: ok %v checked %v", ok, checked) + } +} + +// How long the password is, is the owner's choice: the page warns about a +// short one, and the box takes it. +func TestAShortHousePasswordIsTheOwnersChoice(t *testing.T) { + resetLANGuesses(t) + srv := newLANAuthServer(t) + rr := postLANAuthJSON(srv, http.MethodPost, "http://ftw.local:8080/api/auth/password", "192.168.1.10:43210", `{"password":"1234","enabled":true}`) + if rr.Code != http.StatusOK { + t.Fatalf("short password = %d, want 200 (body=%s)", rr.Code, rr.Body.String()) + } + if !VerifyStoredLANSecret(srv.deps.State, "1234") { + t.Fatal("the short password does not sign in") + } + if _, err := HashInitialLANPassword("1234"); err != nil { + t.Fatalf("a new box refused a short password: %v", err) + } + if _, err := HashInitialLANPassword(""); err == nil { + t.Fatal("a new box took an empty password") + } +} + +// Turning the lock off, as ftw password reset does from the box, signs +// nobody in and ends every session. +func TestTurningTheHousePasswordOffSignsNobodyIn(t *testing.T) { + resetLANGuesses(t) + srv := newLANAuthServer(t) + body := `{"password":"` + testHousePassword + `","enabled":true}` + rr := postLANAuthJSON(srv, http.MethodPost, "http://ftw.local:8080/api/auth/password", "192.168.1.10:43210", body) + token := lanSessionFromRecorder(t, rr) + rr = postLANAuthJSON(srv, http.MethodPost, "http://127.0.0.1:8080/api/auth/password", "127.0.0.1:43210", `{"enabled":false}`) + if rr.Code != http.StatusOK { + t.Fatalf("disable = %d (body=%s)", rr.Code, rr.Body.String()) + } + for _, c := range rr.Result().Cookies() { + if c.Name == lanSessionCookieName && c.Value != "" { + t.Fatal("turning the lock off set a session cookie") + } + } + if lanSessionValid(token) { + t.Fatal("a session outlived turning the lock off") + } +} + func TestPostConfigCannotEnableLANAuth(t *testing.T) { srv, _, cfg := postConfigServer(t, nil) body := strings.Replace(firstSiteMeterConfig, `"api": {"port": 8080}`, `"api": {"port": 8080, "lan_auth": true}`, 1) diff --git a/go/internal/api/api_notifications_push_test.go b/go/internal/api/api_notifications_push_test.go index c0510440c..985905224 100644 --- a/go/internal/api/api_notifications_push_test.go +++ b/go/internal/api/api_notifications_push_test.go @@ -387,14 +387,16 @@ func TestRulesPutSeedsDefaultsOnFirstTouch(t *testing.T) { t.Fatalf("defaults not seeded: %+v", notif) } // Everything is opt-in except the FTW Cloud and agent notices, which - // every owner should hear without having asked, and car.which: with + // every owner should hear without having asked, car.which: with // several cars and no answer, the box plans for a visitor until someone - // taps, and only this asks (docs/cars.md). + // taps, and only this asks (docs/cars.md), and control.not_following: a + // device that does not do what FTW asked (Fredrik, 5 Nov 2026). onByDefault := map[string]bool{ notifications.PushCloudGranted: true, notifications.PushCloudRevoked: true, notifications.PushAgentAdded: true, notifications.PushAgentRequest: true, notifications.PushAgentExpiring: true, notifications.PushApprovalKeySet: true, notifications.PushOwnerAdded: true, notifications.PushCarWhich: true, + notifications.PushControlNotFollowing: true, } enabled := 0 for _, e := range notif.Events { diff --git a/go/internal/api/api_peak_import_ceiling_test.go b/go/internal/api/api_peak_import_ceiling_test.go index a956628d3..540f4c580 100644 --- a/go/internal/api/api_peak_import_ceiling_test.go +++ b/go/internal/api/api_peak_import_ceiling_test.go @@ -131,3 +131,22 @@ func TestPeakImportCeilingNotAppliedWhenTheSaveFails(t *testing.T) { t.Fatalf("ceiling = %.0f W after a failed save, want 6,000", ctrl.PeakImportCeilingW) } } + +// A new ceiling changes what the plan may deliver, so the plan follows it at +// once instead of at the next quarter. +func TestPeakImportCeilingChangeReplans(t *testing.T) { + _, _, svc := newScheduleServer(t) + st, err := state.Open(filepath.Join(t.TempDir(), "state.db")) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = st.Close() }) + srv := newCeilingServer(t, st, &control.State{}) + srv.deps.MPC = svc + + postCeiling(t, srv, "4500") + waitForSchedulePlan(t, svc) + if _, reason := svc.LastReplanInfo(); reason != "import_ceiling_changed" { + t.Fatalf("replan reason = %q, want import_ceiling_changed", reason) + } +} diff --git a/go/internal/api/api_pilot_evidence_test.go b/go/internal/api/api_pilot_evidence_test.go new file mode 100644 index 000000000..35d76c3fe --- /dev/null +++ b/go/internal/api/api_pilot_evidence_test.go @@ -0,0 +1,128 @@ +package api + +import ( + "encoding/json" + "fmt" + "github.com/srcfl/ftw/go/internal/telemetry" + "testing" + "testing/synctest" + "time" +) + +func TestPilotFreshnessRequiresSpecificConfirmation(t *testing.T) { + now := time.Now().Truncate(time.Millisecond) + ms := now.UnixMilli() + base := feedbackReading{PilotA: watts(13), PilotObservedAt: now.Add(-24 * time.Hour).Format(time.RFC3339Nano), PilotConfirmedAtMs: &ms} + if age, ok := pilotReadingAge(base, now); !ok || age != 0 { + t.Fatal("unchanged old source value with current confirmation rejected") + } + // A confirmation past the window is valid but too old to explain a draw. + old := now.Add(-91 * time.Second).UnixMilli() + stale := base + stale.PilotConfirmedAtMs = &old + if age, ok := pilotReadingAge(stale, now); !ok || age <= pilotFreshFor { + t.Fatalf("stale confirmation: age %v ok %v", age, ok) + } + for _, tc := range []struct { + name string + change func(*feedbackReading) + }{ + {"missing confirmation", func(d *feedbackReading) { d.PilotConfirmedAtMs = nil }}, + {"zero confirmation", func(d *feedbackReading) { v := int64(0); d.PilotConfirmedAtMs = &v }}, + {"missing source", func(d *feedbackReading) { d.PilotObservedAt = "" }}, + {"future confirmation", func(d *feedbackReading) { v := now.Add(time.Second).UnixMilli(); d.PilotConfirmedAtMs = &v }}, + {"future source", func(d *feedbackReading) { d.PilotObservedAt = now.Add(time.Second).Format(time.RFC3339Nano) }}, + {"negative pilot", func(d *feedbackReading) { d.PilotA = watts(-1) }}, + } { + t.Run(tc.name, func(t *testing.T) { + d := base + tc.change(&d) + if _, ok := pilotReadingAge(d, now); ok { + t.Fatal("invalid pilot accepted") + } + }) + } +} + +func TestPilotAwareDriverFeedback(t *testing.T) { + // The charger holds FTW's 15 A order; limit is the driver's current the + // car may take, and age the pilot confirmation's age at the verdict. + const at13 = `,"current_limit_a":13` + const balanced = at13 + `,"current_limited_by":"load_balancer"` + for _, tc := range []struct { + name string + draw, pilot float64 + limit string + device bool // the charger's own 13 A limit + age time.Duration + result, reason, severity string + }{ + {"fresh lower pilot, internal cause unknown", 8850, 13, at13, false, 5 * time.Second, "limited", "charger_pilot_limit", "info"}, + {"confirmed static limit", 8850, 13, at13, true, 5 * time.Second, "limited", "device_limit", "warning"}, + {"static limit cannot hide low draw", 4000, 13, at13, true, 5 * time.Second, "not_following", "power_below_target", "warning"}, + {"static limit without a current limit", 8850, 13, "", true, 5 * time.Second, "limited", "device_limit", "warning"}, + {"pilot at the end of its window", 8850, 13, at13, false, pilotFreshFor, "limited", "charger_pilot_limit", "info"}, + {"stale pilot cannot explain power", 8850, 13, at13, false, pilotFreshFor + time.Second, "limited", "offered_current_lower", "warning"}, + {"stale pilot keeps the load balancing", 8850, 13, balanced, false, pilotFreshFor + time.Second, "limited", "load_balancer_limit", "info"}, + {"future confirmation cannot explain power", 8850, 13, at13, false, -time.Minute, "not_following", "power_below_target", "warning"}, + {"zero delivery stays visible", 0, 13, at13, false, 5 * time.Second, "not_following", "no_power_response", "warning"}, + {"zero pilot cannot explain ongoing draw", 8850, 0, at13, false, 5 * time.Second, "not_following", "power_below_target", "warning"}, + {"delivery below effective pilot stays visible", 4000, 13, at13, false, 5 * time.Second, "not_following", "power_below_target", "warning"}, + {"pilot above the order", 8850, 20, at13, false, 5 * time.Second, "limited", "offered_current_lower", "warning"}, + {"pilot above the limit", 8850, 14.2, at13, false, 5 * time.Second, "limited", "offered_current_lower", "warning"}, + {"pilot within 0.5 A of the order", 9900, 14.8, `,"current_limit_a":14.4`, false, 5 * time.Second, "limited", "offered_current_lower", "warning"}, + } { + t.Run(tc.name, func(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + e := newEVTrace(tc.draw) + for range 6 * 60 / 5 { + // tick waits five seconds before the verdict. + confirmed := time.Now().Add(5*time.Second - tc.age) + e.extra = fmt.Sprintf(`,"max_a":15,"dynamic_limit_a":15%s,"pilot_a":%g,"pilot_observed_at":%q,"pilot_confirmed_at_ms":%d`, tc.limit, tc.pilot, time.Now().Add(-time.Hour).Format(time.RFC3339Nano), confirmed.UnixMilli()) + if tc.device { + e.extra += `,"device_limit_a":13,"device_limit_age_s":0` + } + e.tick(10350, "plan") + } + f := e.verdict(t) + if f.Status != tc.result || f.Reason != tc.reason || f.Severity != tc.severity { + t.Fatalf("got %s/%s/%s want %s/%s/%s", f.Status, f.Reason, f.Severity, tc.result, tc.reason, tc.severity) + } + // A pilot past its window is unknown, not shown. + if tc.age > pilotFreshFor && (f.PilotA != nil || f.PilotConfirmedAtMs != 0) { + t.Fatalf("stale pilot shown: %+v", f) + } + if tc.reason == "charger_pilot_limit" { + if f.Severity != "info" || f.PilotA == nil || *f.PilotA != 13 || f.DynamicLimitA == nil || *f.DynamicLimitA != 15 { + t.Fatalf("pilot evidence: %+v", f) + } + raw, err := json.Marshal(f) + if err != nil { + t.Fatal(err) + } + var out map[string]any + if err = json.Unmarshal(raw, &out); err != nil { + t.Fatal(err) + } + if out["pilot_a"] != float64(13) || out["dynamic_limit_a"] != float64(15) { + t.Fatal(string(raw)) + } + } + }) + }) + } +} + +func TestPilotLimitNeverHidesCommandFailureOrPendingCommand(t *testing.T) { + now := time.Now() + for _, tc := range []struct{ result, reason string }{{"failed", "command_failed"}, {"unconfirmed", "command_unconfirmed"}, {"pending", "waiting_response"}} { + t.Run(tc.result, func(t *testing.T) { + f := ControlFeedback{Kind: "ev", SentW: watts(10350), RequestedW: watts(10350), RequestedA: watts(15), OfferedA: watts(15), PilotA: watts(13), LimitA: watts(13), ActualW: watts(8850), ObservedAtMs: now.UnixMilli()} + cmd := telemetry.CommandEvidence{Result: tc.result, Since: now.Add(-3 * time.Minute)} + classifyControlFeedback(&f, cmd, true, true, "", nil, now) + if f.Reason != tc.reason { + t.Fatalf("got %s want %s", f.Reason, tc.reason) + } + }) + } +} diff --git a/go/internal/api/api_time_zone.go b/go/internal/api/api_time_zone.go new file mode 100644 index 000000000..9786d7ab3 --- /dev/null +++ b/go/internal/api/api_time_zone.go @@ -0,0 +1,55 @@ +package api + +import ( + "errors" + "log/slog" + "time" + + "github.com/srcfl/ftw/go/internal/appproto" + "github.com/srcfl/ftw/go/internal/configreload" + "github.com/srcfl/ftw/go/internal/localzone" +) + +// householdZoneKnown is localzone.Known; tests stand in a host on UTC. +var householdZoneKnown = localzone.Known + +// ProposeTimeZone stores a client's time zone as the site's household clock, +// when the box keeps none: no zone stored, and a host on UTC. A box that +// keeps one answers with it, so a phone abroad never moves the house's clock; +// the owner changes a kept zone in setup. +// +// Core sets the clock once at start, so a zone with other rules than the +// running clock restarts Core. That happens once per box, on its first +// offer. +func (s *Server) ProposeTimeZone(zone string) (appproto.TimeZoneAnswer, error) { + loc, err := localzone.Load(zone) + if err != nil { + return appproto.TimeZoneAnswer{}, err + } + if s.deps.Cfg == nil || s.deps.CfgMu == nil || s.deps.SaveConfig == nil { + return appproto.TimeZoneAnswer{}, errors.New("settings are not available yet") + } + s.configWriteMu.Lock() + defer s.configWriteMu.Unlock() + + s.deps.CfgMu.RLock() + next := *s.deps.Cfg + s.deps.CfgMu.RUnlock() + if next.Site.TimeZone != "" { + return appproto.TimeZoneAnswer{Zone: next.Site.TimeZone}, nil + } + if householdZoneKnown() { + return appproto.TimeZoneAnswer{Zone: localzone.Name()}, nil + } + next.Site.TimeZone = zone + if err := s.deps.SaveConfig(s.deps.ConfigPath, &next); err != nil { + return appproto.TimeZoneAnswer{}, err + } + configreload.Apply(s.deps.CfgMu, s.deps.Cfg, s.deps.CtrlMu, s.deps.Ctrl, &next, s.deps.ConfigApplier) + restart := !localzone.SameRules(loc, time.Local) + slog.Info("household time zone stored from a client", "tz", zone, "restart", restart) + if restart && s.deps.Restart != nil { + s.queueRestart(nil, nil) + } + return appproto.TimeZoneAnswer{Zone: zone, Taken: true}, nil +} diff --git a/go/internal/api/api_time_zone_test.go b/go/internal/api/api_time_zone_test.go new file mode 100644 index 000000000..0f1cad2d0 --- /dev/null +++ b/go/internal/api/api_time_zone_test.go @@ -0,0 +1,115 @@ +package api + +import ( + "context" + "sync" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/control" +) + +// zoneServer is a box whose host clock is UTC unless known says otherwise. +// It counts saves and restarts. +func zoneServer(t *testing.T, stored string, known bool) (srv *Server, cfg *config.Config, saves *int, restarts chan struct{}) { + t.Helper() + local := time.Local + time.Local = time.UTC + wasKnown := householdZoneKnown + householdZoneKnown = func() bool { return known } + t.Cleanup(func() { time.Local, householdZoneKnown = local, wasKnown }) + + var cfgMu sync.RWMutex + var ctrlMu sync.Mutex + cfg = &config.Config{Site: config.Site{TimeZone: stored}} + saves = new(int) + restarts = make(chan struct{}, 1) + srv = New(&Deps{ + Ctrl: control.NewState(0, 42, ""), CtrlMu: &ctrlMu, + Cfg: cfg, CfgMu: &cfgMu, + ConfigPath: t.TempDir() + "/config.yaml", + SaveConfig: func(_ string, c *config.Config) error { + *saves++ + return nil + }, + Restart: func(context.Context) error { + restarts <- struct{}{} + return nil + }, + }) + return srv, cfg, saves, restarts +} + +// Erik's box: Docker on UTC, nothing stored. The phone's zone becomes the +// household clock, and Core restarts to run on it. +func TestABoxOnUTCTakesThePhonesZoneAndRestarts(t *testing.T) { + srv, cfg, saves, restarts := zoneServer(t, "", false) + answer, err := srv.ProposeTimeZone("Europe/Stockholm") + if err != nil { + t.Fatal(err) + } + if !answer.Taken || answer.Zone != "Europe/Stockholm" { + t.Fatalf("answer = %+v, want Europe/Stockholm taken", answer) + } + if *saves != 1 || cfg.Site.TimeZone != "Europe/Stockholm" { + t.Fatalf("saves=%d live zone %q, want one save of Europe/Stockholm", *saves, cfg.Site.TimeZone) + } + select { + case <-restarts: + case <-time.After(2 * time.Second): + t.Fatal("Core did not restart onto the household clock") + } +} + +// A zone with UTC's rules needs no restart: the clock already reads it. +func TestAZoneWithUTCRulesIsStoredWithoutARestart(t *testing.T) { + srv, _, saves, restarts := zoneServer(t, "", false) + if answer, err := srv.ProposeTimeZone("Atlantic/Reykjavik"); err != nil || !answer.Taken { + t.Fatalf("answer %+v err %v", answer, err) + } + if *saves != 1 { + t.Fatalf("saves = %d, want 1", *saves) + } + select { + case <-restarts: + t.Fatal("restarted for a zone on the clock it already runs") + case <-time.After(400 * time.Millisecond): + } +} + +// A phone abroad never moves a house's clock the site keeps. +func TestAStoredZoneIsKept(t *testing.T) { + srv, cfg, saves, _ := zoneServer(t, "Europe/Stockholm", false) + answer, err := srv.ProposeTimeZone("America/New_York") + if err != nil { + t.Fatal(err) + } + if answer.Taken || answer.Zone != "Europe/Stockholm" || *saves != 0 || cfg.Site.TimeZone != "Europe/Stockholm" { + t.Fatalf("answer %+v saves %d zone %q, want Europe/Stockholm kept", answer, *saves, cfg.Site.TimeZone) + } +} + +// A native install keeps the host zone it plans on today. +func TestAHostZoneIsKept(t *testing.T) { + srv, cfg, saves, _ := zoneServer(t, "", true) + answer, err := srv.ProposeTimeZone("America/New_York") + if err != nil { + t.Fatal(err) + } + if answer.Taken || *saves != 0 || cfg.Site.TimeZone != "" { + t.Fatalf("answer %+v saves %d zone %q, want the host zone kept", answer, *saves, cfg.Site.TimeZone) + } +} + +func TestAZoneNoClientCanLoadIsRefused(t *testing.T) { + srv, _, saves, _ := zoneServer(t, "", false) + for _, zone := range []string{"", "Local", "Mars/Olympus"} { + if _, err := srv.ProposeTimeZone(zone); err == nil { + t.Fatalf("took %q", zone) + } + } + if *saves != 0 { + t.Fatalf("saves = %d, want none", *saves) + } +} diff --git a/go/internal/api/lan_auth.go b/go/internal/api/lan_auth.go index 18cb830df..107bb0b9e 100644 --- a/go/internal/api/lan_auth.go +++ b/go/internal/api/lan_auth.go @@ -15,6 +15,7 @@ import ( "strings" "sync" "time" + "unicode/utf8" "golang.org/x/crypto/argon2" @@ -23,7 +24,10 @@ import ( const ( lanAuthPasswordKey = "lan_auth_password" - minLANPasswordLen = 10 + // Any password the owner chooses is accepted, however short: the page + // warns about a short one but does not refuse it (Fredrik, 25 October + // 2026). The upper bound only keeps the hashed input small. + maxLANPasswordLen = 512 lanGuessLimit = 5 lanGuessCooldown = 30 * time.Second @@ -229,8 +233,8 @@ func hashLANPassword(password string) (string, error) { // HashInitialLANPassword lets the first-run server establish the owner's // lock before the normal API starts. It grants no password-change authority. func HashInitialLANPassword(password string) (string, error) { - if len(password) < minLANPasswordLen || len(password) > 512 { - return "", fmt.Errorf("house password must be between %d and 512 characters", minLANPasswordLen) + if password == "" || utf8.RuneCountInString(password) > maxLANPasswordLen { + return "", fmt.Errorf("house password must be between 1 and %d characters", maxLANPasswordLen) } return hashLANPassword(password) } @@ -364,10 +368,20 @@ func (s *Server) handleAuthLogin(w http.ResponseWriter, r *http.Request) { writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "invalid password"}) return } + if status, msg := startLANSession(w, st, encoded); status != 0 { + writeJSON(w, status, map[string]string{"error": msg}) + return + } + writeJSON(w, http.StatusOK, map[string]string{"status": "ok"}) +} + +// startLANSession signs this browser in after it proved the house password, +// whose stored hash is encoded. It sets the session cookie, or returns the +// status and reason to refuse with. +func startLANSession(w http.ResponseWriter, st *state.Store, encoded string) (int, string) { token, err := issueLANSession() if err != nil { - writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "could not create session"}) - return + return http.StatusInternalServerError, "could not create session" } lanSessionMu.Lock() now := lanSessionNow() @@ -375,11 +389,10 @@ func (s *Server) handleAuthLogin(w http.ResponseWriter, r *http.Request) { if err := st.SaveLANSession(lanSessionDigest(token), now.Add(lanSessionTTL).UnixMilli(), now.UnixMilli(), encoded); err != nil { dropLANSession(token) slog.Warn("LAN session could not be saved", "category", lanSessionStoreErrorCategory(err)) - writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "could not save session"}) - return + return http.StatusServiceUnavailable, "could not save session" } setLANSessionCookie(w, token, int(lanSessionTTL/time.Second)) - writeJSON(w, http.StatusOK, map[string]string{"status": "ok"}) + return 0, "" } func (s *Server) handleAuthLogout(w http.ResponseWriter, r *http.Request) { @@ -400,6 +413,15 @@ type lanAuthPasswordRequest struct { Enabled *bool `json:"enabled"` } +// localBrowser is a direct request on the home network or the box itself: +// no app session, no proxy, and a local host name. Only such a request gets +// the sign-in cookie. +func localBrowser(r *http.Request) bool { + auth, err := parseAuthority(r.Host) + return err == nil && isLocalAuthority(auth) && isLocalClient(r.RemoteAddr) && + !hasForwardingHeader(r.Header) && !appLinkOverSession(r) +} + func (s *Server) handleAuthPassword(w http.ResponseWriter, r *http.Request) { s.configWriteMu.Lock() defer s.configWriteMu.Unlock() @@ -419,26 +441,33 @@ func (s *Server) handleAuthPassword(w http.ResponseWriter, r *http.Request) { s.deps.CfgMu.RLock() cfgCopy := *s.deps.Cfg + currentlyOn := s.deps.Cfg.API.LANAuth s.deps.CfgMu.RUnlock() + proved, _ := lanSecretFrom(r.Context()) + // Authenticate let this request in before the write lock: if the lock + // came on in between, a LAN request that proved nothing would replace + // the password the first browser just chose. The box itself, an app + // session and a remote token caller have their own proof. + if currentlyOn && !proved && isLocalClient(r.RemoteAddr) && !isLoopbackClient(r.RemoteAddr) && !appLinkOverSession(r) { + w.Header().Set("WWW-Authenticate", `Bearer realm="ftw-lan"`) + writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "valid LAN password required"}) + return + } enabled := *req.Enabled if enabled { - s.deps.CfgMu.RLock() - currentlyOn := s.deps.Cfg.API.LANAuth - s.deps.CfgMu.RUnlock() - // While the lock is off every LAN peer is an owner. First enable - // from the LAN is how a visitor sets their own password and - // locks Settings. The box itself (loopback) is the only door - // that may turn the lock on. An app session request is built to - // look like loopback; it is not the box (appLinkOwnerMintAllowed). + // While the lock is off, whoever reaches this route is already an + // owner: they may change any setting and pair a phone as an owner + // (appLinkOwnerMintAllowed). Setting the first password gives them + // nothing new and closes the door behind them, so the address they + // call from does not matter: the box itself, a LAN device, or a + // published Docker port. A visitor who gets there first locks the + // owner out of Settings, which the owner sees at once and undoes on + // the box with `ftw password reset` (Fredrik, 25 October 2026). + // The app does not turn it on: the password is typed on the box's + // own page. if !currentlyOn && appLinkOverSession(r) { writeJSON(w, http.StatusForbidden, map[string]string{ - "error": "turn the house password on at the box, not through the app", - }) - return - } - if !currentlyOn && !isLoopbackClient(r.RemoteAddr) { - writeJSON(w, http.StatusForbidden, map[string]string{ - "error": "enable the house password from loopback inside the process. A published Docker port is not loopback — on Docker Desktop use docker compose -f docker-compose.macos.yml exec ftw, then curl http://127.0.0.1:8080", + "error": "turn the house password on from the box's own page, not through the app", }) return } @@ -449,9 +478,9 @@ func (s *Server) handleAuthPassword(w http.ResponseWriter, r *http.Request) { writeJSON(w, http.StatusBadRequest, map[string]string{"error": "password required to enable LAN auth"}) return } - case len(req.Password) < minLANPasswordLen: + case utf8.RuneCountInString(req.Password) > maxLANPasswordLen: writeJSON(w, http.StatusBadRequest, map[string]string{ - "error": fmt.Sprintf("password must be at least %d characters", minLANPasswordLen), + "error": fmt.Sprintf("password must be at most %d characters", maxLANPasswordLen), }) return default: @@ -488,10 +517,29 @@ func (s *Server) handleAuthPassword(w http.ResponseWriter, r *http.Request) { writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "could not revoke sessions"}) return } + switch { + case !enabled: + slog.Info("audit: house password turned off", "from", remoteHost(r)) + case req.Password != "" && currentlyOn: + slog.Info("audit: house password changed", "from", remoteHost(r)) + case req.Password != "": + slog.Info("audit: house password turned on", "from", remoteHost(r)) + } + // A browser that chose the first password, or changed it while signed + // in, stays signed in and goes on without typing it again. The box + // itself may replace a password it never proved, as it may change any + // setting, but that earns no sign-in. A failed sign-in only means the + // page asks for the password; the lock is already on. + signedIn := false + if enabled && req.Password != "" && (!currentlyOn || proved) && localBrowser(r) { + status, _ := startLANSession(w, s.deps.State, cfgCopy.LANPasswordHash) + signedIn = status == 0 + } writeJSON(w, http.StatusOK, map[string]any{ "status": "ok", "lan_auth": enabled, "configured": lanPasswordConfigured(s.deps.State), + "signed_in": signedIn, }) } diff --git a/go/internal/api/security.go b/go/internal/api/security.go index 0c45e0d2a..508861193 100644 --- a/go/internal/api/security.go +++ b/go/internal/api/security.go @@ -171,11 +171,18 @@ func decideLANCaller(r *http.Request, policy MutationPolicy, houseOK bool) apiau // counted as a house-password guess. The outer listener and Server.Handler // both wrap Authenticate; a context flag stops the inner wrap from hashing // or counting twice. +// +// The proof counts from the box itself too, so a browser on the box can let +// an assistant in once it signs in. Loopback stays an owner for settings +// without it, so it is not a wall against programs on the box: drivers +// cannot reach the API at all (drivers/selfguard.go), and anything else +// running there can already change every setting. An app session is proved +// by its own key and never presents this one. func resolveLANSecret(r *http.Request, policy MutationPolicy) (bool, *http.Request) { if ok, checked := lanSecretFrom(r.Context()); checked { return ok, r } - if !lanAuthOn(policy) || isLoopbackClient(r.RemoteAddr) || !isLocalClient(r.RemoteAddr) { + if !lanAuthOn(policy) || !isLocalClient(r.RemoteAddr) || appLinkOverSession(r) { return false, r } if secret, ok := parseBearer(r.Header.Get("Authorization")); ok { diff --git a/go/internal/appproto/command.go b/go/internal/appproto/command.go index 82410204c..8617ea4b9 100644 --- a/go/internal/appproto/command.go +++ b/go/internal/appproto/command.go @@ -42,6 +42,9 @@ func defaultOps() map[string]opSpec { // meets the dispatch gate before anything moves, so a sick meter must // not lock the household out of holding more reserve. OpPlannerPrefsSet: {scope: ScopeModeWrite, dispatchWrite: false}, + // The household clock is setup the box holds, and a goal read on the + // wrong clock is wrong whatever the meter says. + OpSiteTimeZonePropose: {scope: ScopeSetupWrite, dispatchWrite: false}, // The loadpoint pair moves energy — a manual hold commands the // charger and a boost opens the battery-to-EV path — so both sit // behind the dispatch gate. A box with no loadpoint controller diff --git a/go/internal/appproto/handler.go b/go/internal/appproto/handler.go index 0c895bcde..5984f8c37 100644 --- a/go/internal/appproto/handler.go +++ b/go/internal/appproto/handler.go @@ -61,6 +61,10 @@ type Config struct { // window. PlannerPrefs PlannerPrefs + // TimeZones stores a client's time zone on a box that keeps none, or nil + // before the API server is bound. + TimeZones TimeZones + // PlanChargers puts each charger's share on the plan, or nil on a box with // no chargers. PlanFeed pushes the plan when the box replans, or nil for // a box that pushes it only after a command. @@ -1145,6 +1149,8 @@ func (h *Handler) runCmd(ctx context.Context, cmd Cmd, by string) error { return h.setMode(ctx, cmd, uptimeMs) case OpPlannerPrefsSet: return h.setPlannerPrefs(cmd, uptimeMs) + case OpSiteTimeZonePropose: + return h.proposeTimeZone(cmd, uptimeMs) case OpLoadpointHold: return h.loadpointHold(cmd, uptimeMs) case OpLoadpointBoost: diff --git a/go/internal/appproto/ports.go b/go/internal/appproto/ports.go index aac78eb20..2fa92bb9b 100644 --- a/go/internal/appproto/ports.go +++ b/go/internal/appproto/ports.go @@ -257,6 +257,20 @@ type PlannerPrefs interface { Apply(safetyK *float64, export *string) (PlannerPrefsSnapshot, error) } +// TimeZoneAnswer is what the box did with a client's time zone. +type TimeZoneAnswer struct { + // Zone is the household zone the site keeps. + Zone string + // Taken is true when the offer became the site's zone. + Taken bool +} + +// TimeZones takes a client's time zone as the household clock while the box +// keeps none of its own. Nil answers E_UNAVAILABLE. +type TimeZones interface { + ProposeTimeZone(zone string) (TimeZoneAnswer, error) +} + // PlanReader hands over the planner's current output. // // Nil means the planner has produced nothing at all, which the wire reports as diff --git a/go/internal/appproto/protocol.go b/go/internal/appproto/protocol.go index 0a4010bee..728cdd8a2 100644 --- a/go/internal/appproto/protocol.go +++ b/go/internal/appproto/protocol.go @@ -142,6 +142,7 @@ const ( MsgNow = protocol.MsgNow OpSetMode = protocol.OpSetMode OpPlannerPrefsSet = protocol.OpPlannerPrefsSet + OpSiteTimeZonePropose = protocol.OpSiteTimeZonePropose OpLoadpointHold = protocol.OpLoadpointHold OpLoadpointBoost = protocol.OpLoadpointBoost OpLoadpointSoCSet = protocol.OpLoadpointSoCSet @@ -246,6 +247,7 @@ const ( ScopeMembersRead = protocol.ScopeMembersRead ScopeMembersWrite = protocol.ScopeMembersWrite ScopeCarsWrite = protocol.ScopeCarsWrite + ScopeSetupWrite = protocol.ScopeSetupWrite CapDerCar = protocol.CapDerCar ErrBooting = protocol.ErrBooting ErrUnknownOp = protocol.ErrUnknownOp diff --git a/go/internal/appproto/time_zone.go b/go/internal/appproto/time_zone.go new file mode 100644 index 000000000..d0f63c789 --- /dev/null +++ b/go/internal/appproto/time_zone.go @@ -0,0 +1,54 @@ +package appproto + +import "github.com/srcfl/ftw/go/internal/localzone" + +// proposeTimeZone offers the client's time zone as the household clock. The +// box decides: it takes the zone only while it keeps none of its own, and +// otherwise answers E_PRECONDITION with the zone it keeps, so a phone abroad +// never moves the house's clock. +func (h *Handler) proposeTimeZone(cmd Cmd, uptimeMs int64) error { + zones := h.cfg.TimeZones + if zones == nil { + return h.rejectWith(cmd, &ErrorBody{ + Code: ErrUnavailable, + Retryable: ErrorRetryable[ErrUnavailable], + Args: map[string]any{"op": cmd.Op}, + }) + } + zone, _ := cmd.Args["tz"].(string) + if _, err := localzone.Load(zone); err != nil { + return h.rejectWith(cmd, badArg(cmd.Op, "tz", cmd.Args["tz"])) + } + + if took, err := h.acceptCmd(cmd, uptimeMs); !took || err != nil { + return err + } + + answer, err := zones.ProposeTimeZone(zone) + if err != nil { + h.log.Warn("time zone not stored", "tz", zone, "err", err) + return h.settleAndReport(cmd.CmdID, CmdResult{ + CmdID: cmd.CmdID, + State: CmdRejected, + Error: &ErrorBody{ + Code: ErrUnavailable, + Retryable: ErrorRetryable[ErrUnavailable], + Args: map[string]any{"op": cmd.Op}, + }, + }) + } + if !answer.Taken { + return h.settleAndReport(cmd.CmdID, CmdResult{ + CmdID: cmd.CmdID, + State: CmdRejected, + Error: &ErrorBody{ + Code: ErrPrecondition, + Retryable: ErrorRetryable[ErrPrecondition], + Args: map[string]any{"op": cmd.Op, "tz": answer.Zone}, + }, + }) + } + // Applied: the site stores the zone. Core restarts to run on it, and the + // next session's hello names it. + return h.settleAndReport(cmd.CmdID, CmdResult{CmdID: cmd.CmdID, State: CmdApplied}) +} diff --git a/go/internal/appproto/time_zone_test.go b/go/internal/appproto/time_zone_test.go new file mode 100644 index 000000000..83db46ec7 --- /dev/null +++ b/go/internal/appproto/time_zone_test.go @@ -0,0 +1,86 @@ +package appproto + +import "testing" + +// memZones keeps a site zone the way the API server does: an offer is taken +// only while none is kept. +type memZones struct { + kept string + calls int +} + +func (m *memZones) ProposeTimeZone(zone string) (TimeZoneAnswer, error) { + m.calls++ + if m.kept != "" { + return TimeZoneAnswer{Zone: m.kept}, nil + } + m.kept = zone + return TimeZoneAnswer{Zone: zone, Taken: true}, nil +} + +func cmdProposeTimeZone(id string, zone any) Cmd { + return Cmd{ + CmdID: id, + Op: OpSiteTimeZonePropose, + Args: map[string]any{"tz": zone}, + NotValidAfterMs: 200_000, + Expect: Expect{Rev: 7}, + } +} + +func zoneRig(t *testing.T, zones TimeZones) (*Handler, *recorder) { + t.Helper() + h, _, rec, _ := newRig(t) + h.cfg.TimeZones = zones + subscribe(t, h, rec) + rec.reset() + return h, rec +} + +func TestATimeZoneOfferIsTakenByABoxWithoutOne(t *testing.T) { + zones := &memZones{} + h, rec := zoneRig(t, zones) + + deliver(t, h, MsgCmd, nil, cmdProposeTimeZone("0192f2a0-7c1e-7000-8000-0123456789b0", "Europe/Stockholm")) + + res := body[CmdResult](t, rec.only(t, MsgCmdResult)) + if res.State != CmdApplied || zones.kept != "Europe/Stockholm" { + t.Fatalf("result %+v kept %q, want applied Europe/Stockholm", res, zones.kept) + } +} + +// The box decides: a kept zone stays, and the answer names it. +func TestATimeZoneOfferMeetsTheZoneTheBoxKeeps(t *testing.T) { + zones := &memZones{kept: "Europe/Stockholm"} + h, rec := zoneRig(t, zones) + + deliver(t, h, MsgCmd, nil, cmdProposeTimeZone("0192f2a0-7c1e-7000-8000-0123456789b1", "America/New_York")) + + res := body[CmdResult](t, rec.only(t, MsgCmdResult)) + if res.State != CmdRejected || res.Error == nil || res.Error.Code != ErrPrecondition || res.Error.Args["tz"] != "Europe/Stockholm" { + t.Fatalf("result %+v, want E_PRECONDITION naming Europe/Stockholm", res) + } +} + +func TestATimeZoneNoClientCanLoadIsRefusedBeforeTheBoxSeesIt(t *testing.T) { + zones := &memZones{} + h, rec := zoneRig(t, zones) + + deliver(t, h, MsgCmd, nil, cmdProposeTimeZone("0192f2a0-7c1e-7000-8000-0123456789b2", "Local")) + + res := body[CmdResult](t, rec.only(t, MsgCmdResult)) + if res.State != CmdRejected || res.Error == nil || res.Error.Code != ErrUnknownOp || zones.calls != 0 { + t.Fatalf("result %+v calls %d, want a refused arg and no offer", res, zones.calls) + } +} + +func TestATimeZoneOfferWithoutAPortIsUnavailable(t *testing.T) { + h, rec := zoneRig(t, nil) + + deliver(t, h, MsgCmd, nil, cmdProposeTimeZone("0192f2a0-7c1e-7000-8000-0123456789b3", "Europe/Stockholm")) + + res := body[CmdResult](t, rec.only(t, MsgCmdResult)) + if res.State != CmdRejected || res.Error == nil || res.Error.Code != ErrUnavailable { + t.Fatalf("result %+v, want E_UNAVAILABLE", res) + } +} diff --git a/go/internal/batteryreturn/batteryreturn.go b/go/internal/batteryreturn/batteryreturn.go new file mode 100644 index 000000000..18ea943ff --- /dev/null +++ b/go/internal/batteryreturn/batteryreturn.go @@ -0,0 +1,632 @@ +// Package batteryreturn works out the figure a Swedish household reports to +// Skatteverket to get back the energy tax on electricity it bought from the +// grid, stored in a battery and fed back to the same grid (11 kap. 13 § lagen +// (1994:1776) om skatt på energi). The owner applies once per calendar year +// and must be able to show how the figure was reached, so the report keeps +// every hour. +// +// The ledger says how much each meter counted, not where each kWh went. The +// report counts low where the readings leave room, and shares in proportion +// where nothing can say: +// +// - In each hour a battery took from the grid only what solar, a +// discharging car and the other batteries could not have supplied, and +// gave to the grid only what they could not have exported. Every source +// on the site must be one FTW reads. +// - Each battery keeps an account of its own, so one battery's exports +// never carry another's grid energy. The account follows what the +// battery holds and how much of it came from the grid. Energy enters +// after the battery's round-trip efficiency over the year, so losses +// are never refunded. Whatever leaves, to the house or the grid, carries +// the grid's share of what the battery held. The ledger does not say +// what came first within an hour, so the hour's charge mixes in before +// its discharge leaves. +// - An hour counts only when every meter that reported during the year +// read all of it. An hour that does not count adds nothing, and what a +// battery took in it counts as energy from elsewhere. Without a +// battery's own readings that battery is taken to be full of energy from +// elsewhere after the hour. The year starts the same way, so every +// battery needs its configured size. +package batteryreturn + +import ( + "context" + "fmt" + "math" + "sort" + "time" + + "github.com/srcfl/ftw/go/internal/state" + "github.com/srcfl/ftw/go/internal/sunpos" +) + +// Zone is the clock Skatteverket's calendar year runs on. +const Zone = "Europe/Stockholm" + +const ( + hourMS = int64(time.Hour / time.Millisecond) + // settle leaves the ledger time to write the hour that just ended. + settle = 5 * time.Minute + // civilDuskZenithDeg: with the sun 6° below the horizon a panel makes + // nothing, so a sleeping solar inverter reads 0. + civilDuskZenithDeg = 96 +) + +// Ledger is the part of the store the report reads. +type Ledger interface { + EnergyAssets() ([]state.EnergyAsset, error) + LoadEnergyHistoryContext(context.Context, state.EnergyHistoryQuery) ([]state.EnergyLedgerPoint, bool, error) +} + +// Site is what the report needs beyond the ledger. +type Site struct { + // Batteries are the drivers with a battery size in Settings. Each + // account needs an upper bound on what its battery holds. + Batteries []SizedBattery + // Lat and Lon place the site when Placed. Then a solar inverter that + // sleeps after dusk reads 0 instead of leaving the night incomplete. + Lat, Lon float64 + Placed bool +} + +// SizedBattery is a driver with a battery size in Settings. The ledger names +// each asset after the driver that read it last, so the size belongs to the +// battery meter with an asset of that name. +type SizedBattery struct { + Driver string + CapacityWh float64 +} + +// Hour is one clock hour of the basis. Energies are Wh. +type Hour struct { + Start time.Time + GridImport float64 + GridExport float64 + // BatteryCharge and BatteryDischarge are the batteries together. + BatteryCharge float64 + BatteryDischarge float64 + Solar float64 + VehicleDischarge float64 + // GridToBattery, BatteryToGrid, Returned, GridHeld and OtherHeld are the + // sums over Batteries. + GridToBattery float64 + BatteryToGrid float64 + Returned float64 + GridHeld float64 + OtherHeld float64 + Complete bool + // Batteries is each battery's part, in the order of Report.Batteries. + Batteries []BatteryHour +} + +// BatteryHour is one battery's part of an hour. Energies are Wh. +type BatteryHour struct { + Charge float64 + Discharge float64 + // GridToBattery is the least grid energy the battery can have taken. + GridToBattery float64 + // BatteryToGrid is the least of its energy that can have gone out to + // the grid. + BatteryToGrid float64 + // Returned is the part of BatteryToGrid that was grid energy stored in + // this battery earlier: the refundable energy. + Returned float64 + // GridHeld and OtherHeld are what it holds after the hour. + GridHeld float64 + OtherHeld float64 + // Efficiency is the battery's round-trip efficiency over the year. + Efficiency float64 +} + +// Month sums the hours of one calendar month. +type Month struct { + Returned float64 + Hours int + CompleteHours int +} + +// Problem says why a year has no figure. +type Problem string + +const ( + // NoSize: a battery the ledger read in the year has no size in Settings. + NoSize Problem = "battery_size" + // Unread: the ledger did not read a battery that has a size in Settings. + // What it did could have been counted for the others. + Unread Problem = "battery_unread" +) + +// Report is one calendar year, or the part of it that has passed. +type Report struct { + Year int + Returned float64 + Hours []Hour + Months [12]Month + // Batteries are the batteries the ledger read in the year. + Batteries []Battery + // Unread names the drivers with a battery size that the ledger did not + // read in the year. + Unread []string + // Problem is set when nothing is returned for a reason other than the + // readings. + Problem Problem +} + +// Battery is one battery meter the ledger read in the year, with its own +// account. +type Battery struct { + // AssetID is its latest identity, and Label the driver that read it + // last. + AssetID string + Label string + // Sized: Settings holds its size. + Sized bool + // Efficiency is what came out over what went in, over its read hours. + Efficiency float64 + // CapacityWh caps what it holds: the configured size, or the most it + // gained in a day if that is more. + CapacityWh float64 + Returned float64 +} + +// CompleteHours counts the hours with complete readings. +func (r Report) CompleteHours() int { + n := 0 + for _, m := range r.Months { + n += m.CompleteHours + } + return n +} + +type role int + +const ( + roleNone role = iota + roleGrid + roleBattery + roleSolar + roleVehicle +) + +func roleOf(kind state.EnergyAssetKind) role { + switch kind { + case state.AssetGridMeter: + return roleGrid + case state.AssetBattery: + return roleBattery + case state.AssetPV: + return roleSolar + case state.AssetVehicleCharger: + return roleVehicle + } + return roleNone +} + +// reads reports whether the report reads flow from an asset with this role. +func (r role) reads(flow state.EnergyFlow) bool { + switch r { + case roleGrid: + return flow == state.FlowGridImport || flow == state.FlowGridExport + case roleBattery: + return flow == state.FlowBatteryCharge || flow == state.FlowBatteryDischarge + case roleSolar: + return flow == state.FlowPVGeneration + case roleVehicle: + return flow == state.FlowVehicleCharge || flow == state.FlowVehicleDischarge + } + return false +} + +// readings is what one asset recorded in each hour. +type readings struct { + seen []bool // a measured or integrated reading + // resumed marks the hour a reading came back after an outage: a gap + // marker, or a counter delta the ledger spread over the outage. The + // outage's hours lack their share, so neither counts. + resumed []bool + bad []bool // anything else that is not a reading +} + +func (r readings) ok(i int) bool { return r.seen[i] && !r.resumed[i] && !r.bad[i] } + +// Build reads the year from the ledger. Hours from now on are left out. +func Build(ctx context.Context, l Ledger, year int, now time.Time, site Site) (Report, error) { + loc, err := time.LoadLocation(Zone) + if err != nil { + return Report{}, err + } + start := time.Date(year, 1, 1, 0, 0, 0, 0, loc) + end := time.Date(year+1, 1, 1, 0, 0, 0, 0, loc) + if cut := now.Add(-settle).Truncate(time.Hour); cut.Before(end) { + end = cut + } + r := Report{Year: year} + if !end.After(start) { + return r, nil + } + from, to := start.UnixMilli(), end.UnixMilli() + n := int((to - from) / hourMS) + r.Hours = make([]Hour, n) + for i := range r.Hours { + r.Hours[i].Start = start.Add(time.Duration(i) * time.Hour) + r.Hours[i].Complete = true + } + // dark[i]: the whole hour is past civil dusk at the site. + dark := make([]bool, n+1) + for i := range dark { + if !site.Placed { + break + } + at := start.Add(time.Duration(i) * time.Hour) + dark[i] = true + for _, t := range []time.Time{at, at.Add(30 * time.Minute), at.Add(time.Hour)} { + dark[i] = dark[i] && sunpos.At(t, site.Lat, site.Lon).ZenithDeg > civilDuskZenithDeg + } + } + + assets, err := l.EnergyAssets() + if err != nil { + return Report{}, err + } + owed, meters := spans(assets, from, to) + r.sizeBatteries(assets, meters, site.Batteries, to) + for i := range r.Hours { + r.Hours[i].Batteries = make([]BatteryHour, len(r.Batteries)) + } + metered := false + unread := make([][]bool, len(r.Batteries)) // a battery's readings are missing + for b := range unread { + unread[b] = make([]bool, n) + } + for _, a := range assets { + role := roleOf(a.Kind) + sp, found := owed[a.AssetID] + if role == roleNone || !found { + continue + } + // owes reports whether the asset answers for hour i. + owes := func(i int) bool { + at := from + int64(i)*hourMS + return at < sp.to && at+hourMS > sp.from + } + b := sp.meter // the battery meter, for a battery + // One hour past the end shows whether the last hour's readings ran + // to its end. + rd := readings{make([]bool, n+1), make([]bool, n+1), make([]bool, n+1)} + vehicle := false + err := loadAsset(ctx, l, a.AssetID, start, end.Add(time.Hour), func(p state.EnergyLedgerPoint) { + i := int((p.BucketStartMS - from) / hourMS) + if p.BucketLenMS != hourMS { + // A daily total cannot be split into hours. + for j := max(i, 0); j <= n && int64(j) < (p.BucketStartMS+p.BucketLenMS-from+hourMS-1)/hourMS; j++ { + rd.bad[j] = true + } + return + } + if i < 0 || i > n || !role.reads(p.Flow) { + return + } + vehicle = vehicle || p.Flow == state.FlowVehicleDischarge + switch p.Quality { + case "measured", "integrated": + rd.seen[i] = true + case "gap", "recovered": + rd.resumed[i] = true + return + default: + rd.bad[i] = true + return + } + if i == n { + return + } + h := &r.Hours[i] + switch p.Flow { + case state.FlowGridImport: + h.GridImport += p.EnergyWh + case state.FlowGridExport: + h.GridExport += p.EnergyWh + case state.FlowBatteryCharge: + h.BatteryCharge += p.EnergyWh + h.Batteries[b].Charge += p.EnergyWh + case state.FlowBatteryDischarge: + h.BatteryDischarge += p.EnergyWh + h.Batteries[b].Discharge += p.EnergyWh + case state.FlowPVGeneration: + h.Solar += p.EnergyWh + case state.FlowVehicleDischarge: + h.VehicleDischarge += p.EnergyWh + } + }) + if err != nil { + return Report{}, err + } + ok := rd.ok + switch role { + case roleVehicle: + if !vehicle { + continue // a charger that never gives energy back + } + case roleSolar: + // Past dusk solar is 0 whether or not the inverter answers, and + // the morning's first reading only ends a night without sun. + ok = func(i int) bool { + return dark[i] || (rd.seen[i] && !rd.bad[i] && (!rd.resumed[i] || i > 0 && dark[i-1])) + } + case roleGrid: + metered = true + } + // A battery that takes over from another asset may be new hardware: + // nothing of the old account carries over. + if role == roleBattery && sp.from != math.MinInt64 && sp.from >= from && sp.from < to { + i := int((sp.from - from) / hourMS) + unread[b][i], r.Hours[i].Complete = true, false + } + // A meter that reported during the year must read every hour it + // answers for. Silence, before or after, is not a quiet hour. + for i := range r.Hours { + if owes(i) && (!ok(i) || owes(i+1) && !ok(i+1)) { + r.Hours[i].Complete = false + if role == roleBattery { + unread[b][i] = true + } + } + } + } + + for b := range r.Batteries { + r.Batteries[b].fit(r.Hours, b, unread[b], loc) + } + for i := range r.Hours { + h := &r.Hours[i] + // Without a grid meter nothing says what crossed the connection. + h.Complete = h.Complete && metered + m := &r.Months[h.Start.In(loc).Month()-1] + m.Hours++ + if !h.Complete { + continue + } + if r.Problem == "" { + m.CompleteHours++ + } + for b := range h.Batteries { + // Solar, a discharging car and the other batteries may each have + // supplied this battery's charge or the export. + bh := &h.Batteries[b] + others := h.BatteryDischarge - bh.Discharge + bh.GridToBattery = clamp(bh.Charge-h.Solar-h.VehicleDischarge-others, h.GridImport) + bh.BatteryToGrid = clamp(h.GridExport-h.Solar-h.VehicleDischarge-others, bh.Discharge) + h.GridToBattery += bh.GridToBattery + h.BatteryToGrid += bh.BatteryToGrid + } + } + for b := range r.Batteries { + r.account(b, unread[b], loc) + } + return r, nil +} + +// sizeBatteries lists the year's battery meters with their sizes and sets +// the problem, if any. A driver's size belongs to the meter with an asset of +// its name, so a new identity behind the driver keeps it. +func (r *Report) sizeBatteries(all []state.EnergyAsset, meters [][]state.EnergyAsset, sized []SizedBattery, to int64) { + sizes := map[string]float64{} + for _, s := range sized { + sizes[s.Driver] = s.CapacityWh + } + read := map[string]bool{} // drivers named by a meter read in the year + unsized := false + for _, meter := range meters { + bat := Battery{AssetID: meter[len(meter)-1].AssetID, Efficiency: 1} + for _, a := range meter { + if a.Label != "" { + bat.Label = a.Label + } + // A meter named by two drivers takes the larger size, which + // counts lower. + if wh, ok := sizes[a.Label]; ok { + bat.Sized, bat.CapacityWh = true, max(bat.CapacityWh, wh) + read[a.Label] = true + } + } + unsized = unsized || !bat.Sized + r.Batteries = append(r.Batteries, bat) + } + for _, s := range sized { + // A battery first read after the year was not there to read, like + // solar added later. + later := false + for _, a := range all { + later = later || a.Kind == state.AssetBattery && a.Label == s.Driver && a.FirstSeenMS >= to + } + if !read[s.Driver] && !later { + r.Unread = append(r.Unread, s.Driver) + } + } + switch { + case unsized: + r.Problem = NoSize + case len(r.Unread) > 0: + r.Problem = Unread + } +} + +// fit sets the battery's efficiency and size from its readings. +func (bat *Battery) fit(hours []Hour, b int, unread []bool, loc *time.Location) { + var in, out float64 + for i, h := range hours { + if !unread[i] { + in += h.Batteries[b].Charge + out += h.Batteries[b].Discharge + } + } + bat.Efficiency = 0 + if in > 0 { + bat.Efficiency = min(out/in, 1) + } + for i := range hours { + hours[i].Batteries[b].Efficiency = bat.Efficiency + } + // A battery that holds what it gained in a day is at least that large, + // whatever the configuration says. + var held, low, high float64 + for i, h := range hours { + if unread[i] || i > 0 && h.Start.In(loc).YearDay() != hours[i-1].Start.In(loc).YearDay() { + held, low, high = 0, 0, 0 + } + if unread[i] { + continue + } + held += bat.Efficiency*h.Batteries[b].Charge - h.Batteries[b].Discharge + low, high = min(low, held), max(high, held) + bat.CapacityWh = max(bat.CapacityWh, high-low) + } +} + +// account follows battery b's grid and other energy through the year. +func (r *Report) account(b int, unread []bool, loc *time.Location) { + bat := &r.Batteries[b] + k := bat.CapacityWh + grid, other := 0.0, k // unknown content at the start + for i := range r.Hours { + h := &r.Hours[i] + bh := &h.Batteries[b] + if unread[i] || r.Problem != "" { + grid, other = 0, k + } else { + // An incomplete hour has no bounds: all its discharge counts as + // used at home and all its charge as energy from elsewhere. + grid += bh.Efficiency * bh.GridToBattery + other += bh.Efficiency * (bh.Charge - bh.GridToBattery) + // The account never holds more than a full battery. + if held := grid + other; held > k { + grid, other = grid*k/held, other*k/held + } + // What leaves carries the grid's share of what the battery held. + // More out than the account holds means the account ran low; + // the rest carries nothing from the grid. + held := grid + other + out := min(bh.Discharge, held) + if held > 0 { + share := grid / held + bh.Returned = min(bh.BatteryToGrid, out) * share + grid -= out * share + other -= out * (1 - share) + } + } + bh.GridHeld, bh.OtherHeld = grid, other + h.GridHeld += grid + h.OtherHeld += other + h.Returned += bh.Returned + bat.Returned += bh.Returned + r.Months[h.Start.In(loc).Month()-1].Returned += bh.Returned + r.Returned += bh.Returned + } +} + +// span is the time [from, to) an asset answers for, in Unix ms. A battery's +// meter is its index among the batteries. +type span struct { + from, to int64 + meter int +} + +// handoverMS allows for the old and new identity reading in the same tick +// in either order. +const handoverMS = int64(2 * time.Minute / time.Millisecond) + +// spans gives each asset that matters for [from, to) the time it answers +// for, and lists the battery meters. +// +// A device whose identity changes, a driver that learns the serial for +// instance, shows up as a new asset of the same kind once the old one +// stops. Assets of one kind that never read at the same time form one +// meter: each answers from its first reading until the next one's first +// reading, so the hours between them are owed by a meter that did not read +// them. The first answers from the beginning and the last to the end, so a +// meter that appears late or falls silent leaves those hours incomplete. A +// meter whose last reading came before the period is gone; one whose first +// came after it has not arrived. +func spans(assets []state.EnergyAsset, from, to int64) (map[string]span, [][]state.EnergyAsset) { + byRole := map[role][]state.EnergyAsset{} + for _, a := range assets { + if r := roleOf(a.Kind); r != roleNone { + byRole[r] = append(byRole[r], a) + } + } + out := map[string]span{} + var batteries [][]state.EnergyAsset + for role, list := range byRole { + sort.Slice(list, func(i, j int) bool { return list[i].FirstSeenMS < list[j].FirstSeenMS }) + var meters [][]state.EnergyAsset + for _, a := range list { + // Continue the meter that stopped last before this one began. + best := -1 + for m, meter := range meters { + last := meter[len(meter)-1].LastSeenMS + if last <= a.FirstSeenMS+handoverMS && (best < 0 || last > meters[best][len(meters[best])-1].LastSeenMS) { + best = m + } + } + if best < 0 { + meters = append(meters, []state.EnergyAsset{a}) + } else { + meters[best] = append(meters[best], a) + } + } + for _, meter := range meters { + if meter[len(meter)-1].LastSeenMS < from || meter[0].FirstSeenMS >= to { + continue + } + index := -1 + if role == roleBattery { + index = len(batteries) + batteries = append(batteries, meter) + } + for k, a := range meter { + sp := span{math.MinInt64, math.MaxInt64, index} + if k > 0 { + sp.from = a.FirstSeenMS + } + if k+1 < len(meter) { + sp.to = meter[k+1].FirstSeenMS + } + if sp.to > from && sp.from < to+hourMS { + out[a.AssetID] = sp + } + } + } + } + return out, batteries +} + +// clamp keeps v within [0, hi]. +func clamp(v, hi float64) float64 { + return math.Max(0, math.Min(v, hi)) +} + +// loadAsset reads one asset month by month, so no single query scans a year. +func loadAsset(ctx context.Context, l Ledger, assetID string, start, end time.Time, each func(state.EnergyLedgerPoint)) error { + for from := start; from.Before(end); { + to := from.AddDate(0, 1, 0) + if to.After(end) { + to = end + } + buckets := int(to.Sub(from) / time.Hour) + points, truncated, err := l.LoadEnergyHistoryContext(ctx, state.EnergyHistoryQuery{ + AssetID: assetID, SinceMS: from.UnixMilli(), UntilMS: to.UnixMilli(), + BucketMS: hourMS, Limit: buckets + 1, + }) + if err != nil { + return err + } + if truncated { + return fmt.Errorf("energy ledger for %s truncated", assetID) + } + for _, p := range points { + each(p) + } + from = to + } + return nil +} diff --git a/go/internal/batteryreturn/batteryreturn_test.go b/go/internal/batteryreturn/batteryreturn_test.go new file mode 100644 index 000000000..6bfef7564 --- /dev/null +++ b/go/internal/batteryreturn/batteryreturn_test.go @@ -0,0 +1,814 @@ +package batteryreturn + +import ( + "context" + "math" + "path/filepath" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/state" +) + +var stockholm = func() *time.Location { + loc, err := time.LoadLocation(Zone) + if err != nil { + panic(err) + } + return loc +}() + +// The test year starts at local midnight; the report runs two days in. +var ( + yearStart = time.Date(2025, 1, 1, 0, 0, 0, 0, stockholm) + twoDays = yearStart.Add(48 * time.Hour) + reportNow = twoDays.Add(settle) +) + +const ( + grid = "meter:1/grid_meter" + battery = "inverter:1/battery" + battery2 = "inverter:2/battery" + solar = "inverter:1/pv" + charger = "charger:1/vehicle_charger" +) + +// labels name each asset after the driver that read it. +var labels = map[string]string{grid: "meter", battery: "battery", battery2: "battery 2", solar: "solar", charger: "charger"} + +// sized is a site with batteries of these sizes in Wh, read by "battery" +// and "battery 2". +func sized(wh ...float64) Site { + var s Site + for i, w := range wh { + s.Batteries = append(s.Batteries, SizedBattery{Driver: []string{"battery", "battery 2"}[i], CapacityWh: w}) + } + return s +} + +// oneBattery is 10 kWh, configured, at an unplaced site. +var oneBattery = sized(10_000) + +// flows is one hour of a site, in kWh. +type flows struct{ imp, exp, chg, dis, pv float64 } + +// fakeLedger holds hourly points as the store returns them for 1 h buckets. +type fakeLedger struct { + assets []state.EnergyAsset + points map[string][]state.EnergyLedgerPoint +} + +func (f *fakeLedger) EnergyAssets() ([]state.EnergyAsset, error) { return f.assets, nil } + +func (f *fakeLedger) LoadEnergyHistoryContext(_ context.Context, q state.EnergyHistoryQuery) ([]state.EnergyLedgerPoint, bool, error) { + var out []state.EnergyLedgerPoint + for _, p := range f.points[q.AssetID] { + if p.BucketStartMS < q.UntilMS && p.BucketStartMS+p.BucketLenMS > q.SinceMS { + out = append(out, p) + } + } + return out, false, nil +} + +func (f *fakeLedger) add(asset string, flow state.EnergyFlow, at time.Time, length time.Duration, wh float64, quality string) { + f.points[asset] = append(f.points[asset], state.EnergyLedgerPoint{ + AssetID: asset, Flow: flow, BucketStartMS: at.UnixMilli(), BucketLenMS: length.Milliseconds(), + EnergyWh: wh, Quality: quality, + }) +} + +func (f *fakeLedger) install(id string, kind state.EnergyAssetKind) { + f.assets = append(f.assets, state.EnergyAsset{AssetID: id, Kind: kind, Label: labels[id], + FirstSeenMS: yearStart.Add(-24 * time.Hour).UnixMilli(), LastSeenMS: reportNow.UnixMilli()}) +} + +// drop removes an asset's points for the hours from..to-1. +func (f *fakeLedger) drop(asset string, from, to int) { + var kept []state.EnergyLedgerPoint + for _, p := range f.points[asset] { + i := int((p.BucketStartMS - yearStart.UnixMilli()) / hourMS) + if i < from || i >= to { + kept = append(kept, p) + } + } + f.points[asset] = kept +} + +// mark sets the quality of an asset's points in hour i. +func (f *fakeLedger) mark(asset string, i int, quality string) { + for j, p := range f.points[asset] { + if p.BucketStartMS == yearStart.Add(time.Duration(i)*time.Hour).UnixMilli() { + f.points[asset][j].Quality = quality + } + } +} + +// site builds a ledger in which every meter reads every hour of the two +// days and the hour after them; an hour missing from the map is quiet. +func site(hours map[int]flows) *fakeLedger { + f := &fakeLedger{points: map[string][]state.EnergyLedgerPoint{}} + f.install(grid, state.AssetGridMeter) + f.install(battery, state.AssetBattery) + f.install(solar, state.AssetPV) + for i := range 49 { + at, h := yearStart.Add(time.Duration(i)*time.Hour), hours[i] + f.add(grid, state.FlowGridImport, at, time.Hour, h.imp*1000, "measured") + f.add(grid, state.FlowGridExport, at, time.Hour, h.exp*1000, "measured") + f.add(battery, state.FlowBatteryCharge, at, time.Hour, h.chg*1000, "integrated") + f.add(battery, state.FlowBatteryDischarge, at, time.Hour, h.dis*1000, "integrated") + f.add(solar, state.FlowPVGeneration, at, time.Hour, h.pv*1000, "integrated") + } + return f +} + +// another adds a battery with its own driver that charges and discharges as +// given in kWh. The site's grid and solar stay as they are. +func (f *fakeLedger) another(id string, hours map[int]flows) *fakeLedger { + f.install(id, state.AssetBattery) + for i := range 49 { + at, h := yearStart.Add(time.Duration(i)*time.Hour), hours[i] + f.add(id, state.FlowBatteryCharge, at, time.Hour, h.chg*1000, "integrated") + f.add(id, state.FlowBatteryDischarge, at, time.Hour, h.dis*1000, "integrated") + } + return f +} + +// arbitrage charges 10 kWh from the grid at night and sends 9 kWh to the +// grid in the evening, on each day listed. +func arbitrage(days ...int) map[int]flows { + hours := map[int]flows{} + for _, d := range days { + for h := 1; h <= 4; h++ { + hours[d*24+h] = flows{imp: 2.5, chg: 2.5} + } + for h := 18; h <= 20; h++ { + hours[d*24+h] = flows{exp: 3, dis: 3} + } + } + return hours +} + +// Two days of arbitrage charge 20 kWh and discharge 18: an efficiency of +// 0.9. + +func build(t *testing.T, l Ledger, s Site) Report { + t.Helper() + r, err := Build(context.Background(), l, 2025, reportNow, s) + if err != nil { + t.Fatal(err) + } + return r +} + +func near(t *testing.T, what string, got, want float64) { + t.Helper() + if math.Abs(got-want) > 1 { + t.Fatalf("%s = %.1f Wh, want %.1f", what, got, want) + } +} + +func returnedOn(r Report, day int) float64 { + var wh float64 + for _, h := range r.Hours[day*24 : day*24+24] { + wh += h.Returned + } + return wh +} + +func TestArbitrageReturnsGridEnergyAfterLoss(t *testing.T) { + r := build(t, site(arbitrage(0, 1)), oneBattery) + if b := r.Batteries; len(b) != 1 || math.Abs(b[0].Efficiency-0.9) > 1e-9 || b[0].CapacityWh != 10_000 || r.CompleteHours() != 48 || r.Problem != "" { + t.Fatalf("batteries %+v complete %d problem %q", b, r.CompleteHours(), r.Problem) + } + // The year starts with a battery full of energy from elsewhere. Each + // night thins it out, so the second evening returns more than the + // first, and never more than the grid energy left after the loss. + first, second := returnedOn(r, 0), returnedOn(r, 1) + if first <= 0 || second <= first || r.Returned > 0.9*20_000 { + t.Fatalf("returned %.1f then %.1f Wh", first, second) + } + near(t, "year", r.Returned, first+second) + near(t, "january", r.Months[0].Returned, r.Returned) + if r.Months[0].Hours != 48 || r.Months[1].Hours != 0 { + t.Fatalf("months %+v", r.Months[:2]) + } + if h := r.Hours[2]; h.GridToBattery != 2500 || h.BatteryToGrid != 0 { + t.Fatalf("charging hour %+v", h) + } + if h := r.Hours[19]; h.GridToBattery != 0 || h.BatteryToGrid != 3000 { + t.Fatalf("export hour %+v", h) + } + for i, h := range r.Hours { + if h.GridHeld+h.OtherHeld > 10_000+1e-6 { + t.Fatalf("hour %d holds %.1f Wh", i, h.GridHeld+h.OtherHeld) + } + } +} + +func TestAnEmptyBatteryFilledFromTheGridReturnsAllItSends(t *testing.T) { + // The house empties the battery first. More came out than went in, so + // the efficiency is 1. Each night brings 10 kWh of grid energy and each + // evening sends 9 to the grid; the second night tops up the 1 kWh left. + hours := arbitrage(0, 1) + hours[0] = flows{dis: 10} + r := build(t, site(hours), oneBattery) + near(t, "first day", returnedOn(r, 0), 9000) + near(t, "second day", returnedOn(r, 1), 9000) +} + +func TestWhatLeavesCarriesTheGridShare(t *testing.T) { + // The house empties the battery, the grid puts in 5 kWh at night, the + // house takes 2.5 in the morning, the sun puts in 5 at midday and 5 go + // to the grid in the evening. Then a third of the battery is grid + // energy: 2.5 of 7.5 kWh. + r := build(t, site(map[int]flows{ + 0: {dis: 10}, + 2: {imp: 5, chg: 5}, + 7: {dis: 2.5}, + 12: {pv: 5, chg: 5}, + 19: {dis: 5, exp: 5}, + }), oneBattery) + near(t, "returned", r.Returned, 5000.0/3) +} + +func TestWithinAnHourTheChargeMixesInFirst(t *testing.T) { + // The house empties the battery and the sun puts in 5 kWh. One hour the + // grid puts in 4 while the house takes 4, so the house takes 4/9 grid + // energy and leaves 2.22 kWh of it among 5. The next hour 5 kWh go to + // the grid with that share. + r := build(t, site(map[int]flows{ + 8: {dis: 10}, + 36: {pv: 5, chg: 5}, + 44: {imp: 4, chg: 4, dis: 4}, + 45: {dis: 5, exp: 5}, + }), oneBattery) + near(t, "returned", r.Returned, 4.0*5/9*1000) +} + +func TestStoredSolarSentToTheGridIsNotReturned(t *testing.T) { + hours := map[int]flows{} + for d := range 2 { + for h := 10; h <= 13; h++ { + hours[d*24+h] = flows{pv: 4, chg: 2.5, exp: 1.5} + } + for h := 18; h <= 20; h++ { + hours[d*24+h] = flows{exp: 3, dis: 3} + } + } + if r := build(t, site(hours), oneBattery); r.Returned != 0 { + t.Fatalf("returned %.1f Wh of solar", r.Returned) + } +} + +func TestGridEnergyUsedAtHomeIsNotReturned(t *testing.T) { + hours := map[int]flows{} + for d := range 2 { + for h := 1; h <= 4; h++ { + hours[d*24+h] = flows{imp: 2.5, chg: 2.5} + } + for h := 7; h <= 9; h++ { + hours[d*24+h] = flows{dis: 3} + } + for h := 11; h <= 15; h++ { + hours[d*24+h] = flows{pv: 1, exp: 1} + } + } + if r := build(t, site(hours), oneBattery); r.Returned != 0 { + t.Fatalf("returned %.1f Wh used at home", r.Returned) + } +} + +func TestOnlyTheShareThatLeftTheSiteIsReturned(t *testing.T) { + hours := arbitrage(0, 1) + for h := 18; h <= 20; h++ { + hours[24+h] = flows{exp: 1, dis: 3} // 2 kWh of each hour stays at home + } + r := build(t, site(hours), oneBattery) + if h := r.Hours[24+19]; h.BatteryToGrid != 1000 { + t.Fatalf("export hour %+v", h) + } + // The battery holds the same as in plain arbitrage and only a third of + // each hour's discharge leaves the site. + plain := build(t, site(arbitrage(0, 1)), oneBattery) + near(t, "second day", returnedOn(r, 1), returnedOn(plain, 1)/3) +} + +func TestSolarInTheHourCountsFirst(t *testing.T) { + // Charging 3 kWh while the sun gives 2 takes at least 1 from the grid; + // sending 3 out while the sun gives 2 takes at least 1 from the battery. + r := build(t, site(map[int]flows{ + 10: {pv: 2, chg: 3, imp: 1.5}, + 18: {pv: 2, dis: 3, exp: 3}, + }), oneBattery) + if r.Hours[10].GridToBattery != 1000 || r.Hours[18].BatteryToGrid != 1000 { + t.Fatalf("charge %+v export %+v", r.Hours[10], r.Hours[18]) + } +} + +func TestAMissingHourForgetsWhatTheBatteryHeld(t *testing.T) { + l := site(arbitrage(0, 1)) + // The battery's reading for noon on the second day has a gap. + l.mark(battery, 36, "gap") + r := build(t, l, oneBattery) + // The hour before it may have ended early, so it does not count either. + if r.Hours[35].Complete || r.Hours[36].Complete || r.CompleteHours() != 46 { + t.Fatalf("hours 35/36 complete %v/%v, %d complete hours", r.Hours[35].Complete, r.Hours[36].Complete, r.CompleteHours()) + } + // Without the gap the second day returns 2 kWh. + if got := returnedOn(r, 1); got != 0 { + t.Fatalf("second day returned %.1f Wh after the gap", got) + } +} + +func TestOnlyMeasuredReadingsCount(t *testing.T) { + // A recovered counter delta is spread evenly over an outage, so it can + // move midday export into an evening hour. + for _, quality := range []string{"recovered", "gap", "estimated", "invalid", "reset"} { + // Without the grid meter the hour attributes nothing, but the + // battery's readings keep the mix. + // Without the grid meter's reading the hour credits nothing, but + // the battery's readings keep the account. + l := site(arbitrage(0, 1)) + l.mark(grid, 44, quality) + r := build(t, l, oneBattery) + if h := r.Hours[44]; h.Complete || r.Hours[43].Complete || h.Returned != 0 || r.Hours[43].Returned != 0 || h.GridHeld+h.OtherHeld == 10_000 { + t.Fatalf("%s grid reading counted: %+v", quality, r.Hours[42:45]) + } + // Without the battery's, the account starts over: full of other + // energy. + l = site(arbitrage(0, 1)) + l.mark(battery, 43, quality) + if r := build(t, l, oneBattery); r.Hours[43].Complete || r.Hours[43].GridHeld != 0 || r.Hours[43].OtherHeld != 10_000 || returnedOn(r, 1) != 0 { + t.Fatalf("%s battery reading counted", quality) + } + } +} + +func TestAMeterThatFellSilentLeavesItsHoursIncomplete(t *testing.T) { + // The solar driver stopped after the first day. Its last reading is then. + l := site(arbitrage(0, 1)) + l.drop(solar, 24, 49) + l.assets[2].LastSeenMS = yearStart.Add(24 * time.Hour).UnixMilli() + if r := build(t, l, oneBattery); r.CompleteHours() != 23 || r.Hours[30].Complete || returnedOn(r, 1) != 0 { + t.Fatalf("%d complete hours without solar", r.CompleteHours()) + } + // Solar added in the middle of the year cannot vouch for the hours + // before it. + l = site(arbitrage(0, 1)) + l.drop(solar, 0, 24) + l.assets[2].FirstSeenMS = yearStart.Add(24 * time.Hour).UnixMilli() + if r := build(t, l, oneBattery); r.CompleteHours() != 24 || r.Hours[10].Complete || !r.Hours[30].Complete { + t.Fatalf("%d complete hours before solar", r.CompleteHours()) + } +} + +// renew moves an asset to a new identity: the old one reads until 10 +// minutes into hour from, the new one starts there with the ledger's +// baseline marker and reads from hour from+1 on. With silent > 0 the old +// one stops that many hours earlier. +func (f *fakeLedger) renew(id string, from, silent int) string { + next := id + "+serial" + handover := yearStart.Add(time.Duration(from)*time.Hour + 10*time.Minute).UnixMilli() + for i, a := range f.assets { + if a.AssetID != id { + continue + } + f.assets[i].LastSeenMS = handover - int64(silent)*hourMS + f.assets = append(f.assets, state.EnergyAsset{AssetID: next, Kind: a.Kind, FirstSeenMS: handover, LastSeenMS: a.LastSeenMS}) + break + } + var kept []state.EnergyLedgerPoint + for _, p := range f.points[id] { + switch i := int((p.BucketStartMS - yearStart.UnixMilli()) / hourMS); { + case i > from: + p.AssetID = next + f.points[next] = append(f.points[next], p) + case i <= from-silent: + kept = append(kept, p) + } + } + f.points[id] = kept + f.add(next, state.FlowGridImport, yearStart.Add(time.Duration(from)*time.Hour), time.Hour, 0, "gap") + return next +} + +func TestANewIdentityContinuesTheMeter(t *testing.T) { + // The inverter's driver learns its serial at 06:10 on the second day, + // and the grid meter, battery and solar get new asset ids. + l := site(arbitrage(0, 1)) + for _, id := range []string{grid, battery, solar} { + l.renew(id, 30, 0) + } + r := build(t, l, oneBattery) + if r.Problem != "" || len(r.Batteries) != 1 { + t.Fatalf("problem %q, %d batteries", r.Problem, len(r.Batteries)) + } + // Only the handover hour is incomplete; it resets the mix. + if r.CompleteHours() != 47 || r.Hours[30].Complete || !r.Hours[29].Complete || !r.Hours[31].Complete { + t.Fatalf("%d complete hours; 29/30/31 %v/%v/%v", r.CompleteHours(), r.Hours[29].Complete, r.Hours[30].Complete, r.Hours[31].Complete) + } + + // A battery that fell silent at 22:10 on the first day and came back + // under a new id at 06:10 owes the night it did not read. + l = site(arbitrage(0, 1)) + l.renew(battery, 30, 8) + r = build(t, l, oneBattery) + if len(r.Batteries) != 1 || r.Problem != "" { + t.Fatalf("problem %q, %d batteries", r.Problem, len(r.Batteries)) + } + for i := 22; i <= 30; i++ { + if r.Hours[i].Complete { + t.Fatalf("hour %d counted while the battery was silent", i) + } + } + if got := returnedOn(r, 1); got != 0 { + t.Fatalf("second day returned %.1f Wh after the silent night", got) + } +} + +func TestANewBatteryStartsItsOwnAccount(t *testing.T) { + // A new battery asset takes over at 06:10 on the second day, and both + // read that hour without a gap marker. It may be new hardware, so the + // grid energy the old battery took that night does not carry over. + l := site(arbitrage(0, 1)) + const next = battery + "+new" + handover := yearStart.Add(30*time.Hour + 10*time.Minute).UnixMilli() + l.assets[1].LastSeenMS = handover + l.assets = append(l.assets, state.EnergyAsset{AssetID: next, Kind: state.AssetBattery, FirstSeenMS: handover, LastSeenMS: reportNow.UnixMilli()}) + var kept []state.EnergyLedgerPoint + for _, p := range l.points[battery] { + i := int((p.BucketStartMS - yearStart.UnixMilli()) / hourMS) + if i >= 30 { + q := p + q.AssetID = next + l.points[next] = append(l.points[next], q) + } + if i <= 30 { + kept = append(kept, p) + } + } + l.points[battery] = kept + r := build(t, l, oneBattery) + if len(r.Batteries) != 1 || r.Problem != "" || r.Hours[30].Complete { + t.Fatalf("problem %q, %d batteries, hour 30 complete %v", r.Problem, len(r.Batteries), r.Hours[30].Complete) + } + if got := returnedOn(r, 1); got != 0 { + t.Fatalf("second day returned %.1f Wh across the swap", got) + } +} + +func TestDailyTotalsCannotBeSplitIntoHours(t *testing.T) { + l := site(arbitrage(0, 1)) + l.drop(battery, 0, 24) + l.add(battery, state.FlowBatteryCharge, yearStart, 24*time.Hour, 10_000, "measured") + r := build(t, l, oneBattery) + if r.CompleteHours() != 24 || r.Hours[0].Complete || r.Hours[23].Complete || !r.Hours[24].Complete { + t.Fatalf("%d complete hours", r.CompleteHours()) + } +} + +func TestNoGridMeterCountsNothing(t *testing.T) { + l := site(arbitrage(0, 1)) + l.assets = l.assets[1:] + if r := build(t, l, oneBattery); r.CompleteHours() != 0 || r.Returned != 0 { + t.Fatalf("%d complete hours, %.1f Wh", r.CompleteHours(), r.Returned) + } +} + +func TestEveryBatteryNeedsItsSize(t *testing.T) { + if r := build(t, site(arbitrage(0, 1)), Site{}); r.Problem != NoSize || r.Returned != 0 || r.CompleteHours() != 0 { + t.Fatalf("no size: problem %q, %.1f Wh", r.Problem, r.Returned) + } + // A second battery without a size. + if r := build(t, site(arbitrage(0, 1)).another(battery2, arbitrage(0, 1)), oneBattery); r.Problem != NoSize || r.Returned != 0 || len(r.Batteries) != 2 || r.Batteries[1].Sized { + t.Fatalf("second battery unsized: problem %q, %.1f Wh, %+v", r.Problem, r.Returned, r.Batteries) + } + // A size for a battery the ledger never read, or did not read in the + // year: what it did could have been counted for the first. + r := build(t, site(arbitrage(0, 1)), sized(10_000, 5000)) + if r.Problem != Unread || r.Returned != 0 || len(r.Unread) != 1 || r.Unread[0] != "battery 2" { + t.Fatalf("never read: problem %q, unread %v", r.Problem, r.Unread) + } + l := site(arbitrage(0, 1)) + l.install(battery2, state.AssetBattery) + l.assets[3].FirstSeenMS, l.assets[3].LastSeenMS = yearStart.AddDate(-1, 0, 0).UnixMilli(), yearStart.Add(-time.Hour).UnixMilli() + if r := build(t, l, sized(10_000, 5000)); r.Problem != Unread || len(r.Batteries) != 1 { + t.Fatalf("read before the year: problem %q, %+v", r.Problem, r.Batteries) + } + // One first read after the year was not there to read. + l.assets[3].FirstSeenMS, l.assets[3].LastSeenMS = reportNow.Add(time.Hour).UnixMilli(), reportNow.Add(2*time.Hour).UnixMilli() + if r := build(t, l, sized(10_000, 5000)); r.Problem != "" || len(r.Batteries) != 1 || r.Returned != build(t, site(arbitrage(0, 1)), oneBattery).Returned { + t.Fatalf("read after the year: problem %q, %.1f Wh", r.Problem, r.Returned) + } + // No battery at all is no problem, just nothing. + l = site(nil) + l.assets = append(l.assets[:1], l.assets[2:]...) + if r := build(t, l, Site{}); r.Problem != "" || r.Returned != 0 || r.CompleteHours() != 48 { + t.Fatalf("no battery: problem %q, %d complete hours", r.Problem, r.CompleteHours()) + } +} + +// TestOneBatterysExportsDoNotCarryAnothersGridEnergy is the second review's +// case: battery A charges only from solar and sends 9 kWh to the grid each +// evening, while battery B charges 10 kWh from the grid each night and +// never discharges. One mix for both would count A's exports as B's grid +// energy. +func TestOneBatterysExportsDoNotCarryAnothersGridEnergy(t *testing.T) { + a, b := map[int]flows{}, map[int]flows{} + for d := range 2 { + for h := 1; h <= 4; h++ { + a[d*24+h] = flows{imp: 2.5} + b[d*24+h] = flows{chg: 2.5} + } + for h := 10; h <= 13; h++ { + a[d*24+h] = flows{pv: 2.5, chg: 2.5} + } + for h := 18; h <= 20; h++ { + a[d*24+h] = flows{exp: 3, dis: 3} + } + } + r := build(t, site(a).another(battery2, b), sized(10_000, 20_000)) + if r.Problem != "" || r.CompleteHours() != 48 || r.Returned != 0 { + t.Fatalf("problem %q, %d complete hours, returned %.1f Wh", r.Problem, r.CompleteHours(), r.Returned) + } + // The bounds still give B the grid energy and A the export. + if h := r.Hours[2].Batteries; h[0].GridToBattery != 0 || h[1].GridToBattery != 2500 { + t.Fatalf("night %+v", h) + } + if h := r.Hours[19].Batteries; h[0].BatteryToGrid != 3000 || h[1].BatteryToGrid != 0 { + t.Fatalf("evening %+v", h) + } +} + +// bothFromTheGrid is A doing arbitrage as above with 10 kWh, and B the same +// at half the power with 5 kWh: each night hour charges A 2.5 kWh and B +// 1.25, each evening hour sends 3 and 1.5 to the grid. It returns the site +// with A, and B. +func bothFromTheGrid() (map[int]flows, map[int]flows) { + a, b := arbitrage(0, 1), map[int]flows{} + for i, h := range a { + b[i] = flows{chg: h.chg / 2, dis: h.dis / 2} + a[i] = flows{imp: h.imp * 1.5, exp: h.exp * 1.5, chg: h.chg, dis: h.dis} + } + return a, b +} + +func TestTwoBatteriesFromTheGridEachReturnTheirOwn(t *testing.T) { + a, b := bothFromTheGrid() + r := build(t, site(a).another(battery2, b), sized(10_000, 5000)) + if r.Problem != "" || r.CompleteHours() != 48 || len(r.Batteries) != 2 { + t.Fatalf("problem %q, %d complete hours", r.Problem, r.CompleteHours()) + } + // In the evening the export is both batteries' discharge, so each sent + // all of its own: 4.5 kWh out less the other's 1.5 or 3. + if h := r.Hours[19].Batteries; h[0].BatteryToGrid != 3000 || h[1].BatteryToGrid != 1500 { + t.Fatalf("evening %+v", h) + } + // A is the one-battery arbitrage. B is the same at half the size and + // half the flows, so it returns half as much. + if bat := r.Batteries; math.Abs(bat[0].Efficiency-0.9) > 1e-9 || math.Abs(bat[1].Efficiency-0.9) > 1e-9 || bat[1].CapacityWh != 5000 { + t.Fatalf("batteries %+v", bat) + } + plain := build(t, site(arbitrage(0, 1)), oneBattery) + near(t, "A", r.Batteries[0].Returned, plain.Returned) + near(t, "B", r.Batteries[1].Returned, plain.Returned/2) + near(t, "year", r.Returned, 1.5*plain.Returned) + if a, b := r.Hours[24+4].Batteries[0], r.Hours[24+4].Batteries[1]; math.Abs(b.GridHeld-a.GridHeld/2) > 1 || math.Abs(b.OtherHeld-a.OtherHeld/2) > 1 { + t.Fatalf("after the second night A %+v B %+v", a, b) + } +} + +func TestExportEitherBatteryCouldHaveSentCountsLow(t *testing.T) { + // On the second evening the batteries discharge 4.5 kWh an hour but only + // 3 go to the grid. B's 1.5 could be part of it, so A sent at least 1.5; + // A's 3 could be all of it, so B sent at least nothing. + a, b := bothFromTheGrid() + for h := 18; h <= 20; h++ { + a[24+h] = flows{exp: 3, dis: 3} + } + r := build(t, site(a).another(battery2, b), sized(10_000, 5000)) + if h := r.Hours[24+19].Batteries; h[0].BatteryToGrid != 1500 || h[1].BatteryToGrid != 0 { + t.Fatalf("evening %+v", h) + } + // Up to the second evening both hold what they did in the plain case. + // Then half of A's discharge counts as sent out and none of B's. + plain := build(t, site(arbitrage(0, 1)), oneBattery) + near(t, "A", r.Batteries[0].Returned, returnedOn(plain, 0)+returnedOn(plain, 1)/2) + near(t, "B", r.Batteries[1].Returned, returnedOn(plain, 0)/2) +} + +func TestABatterysMissingHourForgetsOnlyItsAccount(t *testing.T) { + // B's reading at 10:00 on the first day has a gap. Hours 9 and 10 count + // for neither battery, but only B starts over full of other energy. + a, b := bothFromTheGrid() + l := site(a).another(battery2, b) + l.mark(battery2, 10, "gap") + r := build(t, l, sized(10_000, 5000)) + if r.CompleteHours() != 46 || r.Hours[9].Complete || r.Hours[10].Complete { + t.Fatalf("%d complete hours", r.CompleteHours()) + } + plain := build(t, site(arbitrage(0, 1)), oneBattery) + near(t, "A", r.Batteries[0].Returned, plain.Returned) + // B starts over after the gap full of other energy, so its first night + // is forgotten: the first evening returns nothing and the second less + // than without the gap. + first := 0.0 + for _, h := range r.Hours[:24] { + first += h.Batteries[1].Returned + } + if first != 0 || r.Batteries[1].Returned <= 0 || r.Batteries[1].Returned >= plain.Returned/2 { + t.Fatalf("B returned %.1f on the first day, %.1f Wh in all", first, r.Batteries[1].Returned) + } +} + +func TestTheSizeIsAtLeastWhatTheBatteryGainedInADay(t *testing.T) { + // Configured as 5 kWh, the battery gains 9 kWh a night after loss. + r := build(t, site(arbitrage(0, 1)), sized(5000)) + if r.Batteries[0].CapacityWh != 9000 { + t.Fatalf("capacity %v", r.Batteries[0].CapacityWh) + } + // A larger size only counts lower. + small, large := build(t, site(arbitrage(0, 1)), oneBattery), build(t, site(arbitrage(0, 1)), sized(15_000)) + if large.Returned >= small.Returned { + t.Fatalf("15 kWh returned %.1f, 10 kWh %.1f", large.Returned, small.Returned) + } +} + +func TestACarGivingEnergyBackMustBeRead(t *testing.T) { + // A charger that never gives energy back is not needed. + l := site(arbitrage(0, 1)) + l.install(charger, state.AssetVehicleCharger) + l.add(charger, state.FlowVehicleCharge, yearStart.Add(3*time.Hour), time.Hour, 7000, "integrated") + if r := build(t, l, oneBattery); r.CompleteHours() != 48 { + t.Fatalf("%d complete hours with a plain charger", r.CompleteHours()) + } + // One that does must read every hour, and what it gave counts like solar. + l = site(arbitrage(0, 1)) + l.install(charger, state.AssetVehicleCharger) + for i := range 49 { + if i == 30 { + continue // offline + } + wh := 0.0 + if i == 19 { + wh = 2000 + } + l.add(charger, state.FlowVehicleDischarge, yearStart.Add(time.Duration(i)*time.Hour), time.Hour, wh, "integrated") + } + r := build(t, l, oneBattery) + if r.Hours[30].Complete || r.Hours[29].Complete || r.CompleteHours() != 46 { + t.Fatalf("%d complete hours with the car charger offline", r.CompleteHours()) + } + if h := r.Hours[19]; h.VehicleDischarge != 2000 || h.BatteryToGrid != 1000 || r.Problem != "" { + t.Fatalf("export hour %+v", h) + } +} + +func TestSolarIsZeroAfterDusk(t *testing.T) { + // A solar inverter sleeps from 16:00 in a Stockholm January. Its first + // reading after waking ends the outage. + sleeping := func(wake int) *fakeLedger { + l := site(arbitrage(0, 1)) + for d := range 3 { + l.drop(solar, d*24-8, d*24+wake) + l.mark(solar, d*24+wake, "gap") + } + return l + } + placed := Site{Batteries: oneBattery.Batteries, Lat: 59.33, Lon: 18.07, Placed: true} + full := build(t, site(arbitrage(0, 1)), placed) + // Waking at 06:00, before dawn, loses nothing. + if r := build(t, sleeping(6), placed); r.CompleteHours() != 48 || r.Returned != full.Returned { + t.Fatalf("%d complete hours, %.1f Wh", r.CompleteHours(), r.Returned) + } + // Waking at 10:00 leaves the morning without solar readings. Those + // hours attribute nothing, and the battery's readings keep the mix. + r := build(t, sleeping(10), placed) + for _, i := range []int{8, 9, 10, 32, 33, 34} { + if r.Hours[i].Complete { + t.Fatalf("hour %d counted without solar", i) + } + } + for _, i := range []int{2, 19, 26, 43} { + if !r.Hours[i].Complete { + t.Fatalf("dark hour %d not counted", i) + } + } + near(t, "late wake", r.Returned, full.Returned) + // Unplaced, nothing says the sun was down. + if r := build(t, sleeping(6), oneBattery); r.Hours[2].Complete || r.Hours[19].Complete || r.Returned != 0 { + t.Fatalf("night hours counted at an unplaced site") + } +} + +func TestTheYearRunsOnSwedishTime(t *testing.T) { + l := site(arbitrage(0, 1)) + // 23:00 UTC on 31 December is the first hour of the Swedish year. + if got := l.points[grid][0].BucketStartMS; got != time.Date(2024, 12, 31, 23, 0, 0, 0, time.UTC).UnixMilli() { + t.Fatalf("first hour %v", time.UnixMilli(got).UTC()) + } + r := build(t, l, oneBattery) + if len(r.Hours) != 48 || !r.Hours[0].Start.Equal(yearStart) { + t.Fatalf("%d hours from %v", len(r.Hours), r.Hours[0].Start) + } + // The hour that just ended waits for its last readings. + r, err := Build(context.Background(), l, 2025, twoDays.Add(2*time.Minute), oneBattery) + if err != nil || len(r.Hours) != 47 { + t.Fatalf("%d hours, err %v", len(r.Hours), err) + } + // A year that has not started has no hours. + if r, err := Build(context.Background(), l, 2026, reportNow, oneBattery); err != nil || len(r.Hours) != 0 || r.Returned != 0 { + t.Fatalf("2026: %d hours, err %v", len(r.Hours), err) + } +} + +// TestTheStoreHoursAreTheReportHours reads hours the real ledger wrote: +// a grid meter by its counters and a battery by power, every two minutes. +func TestTheStoreHoursAreTheReportHours(t *testing.T) { + st, err := state.Open(filepath.Join(t.TempDir(), "state.db")) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = st.Close() }) + from := yearStart.Add(-10 * time.Minute) + until := yearStart.Add(3*time.Hour + 10*time.Minute) + var obs []state.EnergyObservation + for at := from; !at.After(until); at = at.Add(2 * time.Minute) { + imported, exported := 2000*at.Sub(from).Hours(), 0.0 + charging, discharging := 2000.0, 0.0 + obs = append(obs, + state.EnergyObservation{AssetID: grid, DeviceID: "meter:1", AssetKind: state.AssetGridMeter, Flow: state.FlowGridImport, AtMs: at.UnixMilli(), CounterWh: &imported}, + state.EnergyObservation{AssetID: grid, DeviceID: "meter:1", AssetKind: state.AssetGridMeter, Flow: state.FlowGridExport, AtMs: at.UnixMilli(), CounterWh: &exported}, + state.EnergyObservation{AssetID: battery, DeviceID: "inverter:1", AssetKind: state.AssetBattery, Label: "battery", Flow: state.FlowBatteryCharge, AtMs: at.UnixMilli(), PowerW: &charging}, + state.EnergyObservation{AssetID: battery, DeviceID: "inverter:1", AssetKind: state.AssetBattery, Label: "battery", Flow: state.FlowBatteryDischarge, AtMs: at.UnixMilli(), PowerW: &discharging}, + ) + } + if err := st.RecordTickWithOptionalHistory(nil, nil, obs); err != nil { + t.Fatal(err) + } + r, err := Build(context.Background(), st, 2025, yearStart.Add(3*time.Hour+settle), oneBattery) + if err != nil { + t.Fatal(err) + } + if len(r.Hours) != 3 || r.CompleteHours() != 3 { + t.Fatalf("%d hours, %d complete", len(r.Hours), r.CompleteHours()) + } + for _, h := range r.Hours { + near(t, "import", h.GridImport, 2000) + near(t, "charge", h.BatteryCharge, 2000) + near(t, "grid to battery", h.GridToBattery, 2000) + } +} + +// TestASpreadOutageIsNotAnHour is the review's case: the grid meter's +// counter is offline from 09:30 to 22:30 while midday solar is exported and +// the battery feeds the house in the evening. The ledger spreads the export +// over the outage; none of it may count as battery energy sent out. +func TestASpreadOutageIsNotAnHour(t *testing.T) { + st, err := state.Open(filepath.Join(t.TempDir(), "state.db")) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = st.Close() }) + var obs []state.EnergyObservation + var imported, exported float64 + from := yearStart.Add(-10 * time.Minute) + for at := from; at.Before(yearStart.Add(25 * time.Hour)); at = at.Add(2 * time.Minute) { + h := at.In(stockholm).Hour() + pv, charge, discharge := 0.0, 0.0, 0.0 + switch { + case h >= 1 && h < 5: + charge = 2500 + imported += 2500.0 / 30 + case h >= 10 && h < 15: + pv = 4000 + exported += 4000.0 / 30 + case h >= 18 && h < 21: + discharge = 2500 + } + p, c, d := pv, charge, discharge + obs = append(obs, + state.EnergyObservation{AssetID: solar, DeviceID: "inverter:1", AssetKind: state.AssetPV, Flow: state.FlowPVGeneration, AtMs: at.UnixMilli(), PowerW: &p}, + state.EnergyObservation{AssetID: battery, DeviceID: "inverter:1", AssetKind: state.AssetBattery, Label: "battery", Flow: state.FlowBatteryCharge, AtMs: at.UnixMilli(), PowerW: &c}, + state.EnergyObservation{AssetID: battery, DeviceID: "inverter:1", AssetKind: state.AssetBattery, Label: "battery", Flow: state.FlowBatteryDischarge, AtMs: at.UnixMilli(), PowerW: &d}, + ) + if local := at.In(stockholm); local.Day() == 1 && (local.Hour()*60+local.Minute() >= 9*60+30 && local.Hour()*60+local.Minute() < 22*60+30) { + continue // the meter is offline + } + i, e := imported, exported + obs = append(obs, + state.EnergyObservation{AssetID: grid, DeviceID: "meter:1", AssetKind: state.AssetGridMeter, Flow: state.FlowGridImport, AtMs: at.UnixMilli(), CounterWh: &i}, + state.EnergyObservation{AssetID: grid, DeviceID: "meter:1", AssetKind: state.AssetGridMeter, Flow: state.FlowGridExport, AtMs: at.UnixMilli(), CounterWh: &e}, + ) + } + if err := st.RecordTickWithOptionalHistory(nil, nil, obs); err != nil { + t.Fatal(err) + } + r, err := Build(context.Background(), st, 2025, yearStart.Add(24*time.Hour+settle), oneBattery) + if err != nil { + t.Fatal(err) + } + if r.Returned != 0 || r.Hours[19].Complete { + t.Fatalf("returned %.1f Wh; hour 19 %+v", r.Returned, r.Hours[19]) + } +} diff --git a/go/internal/cars/profile_target_test.go b/go/internal/cars/profile_target_test.go new file mode 100644 index 000000000..9376ad505 --- /dev/null +++ b/go/internal/cars/profile_target_test.go @@ -0,0 +1,48 @@ +package cars + +import ( + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/loadpoint" +) + +// A car merged from a vehicles[] profile with target_soc keeps a one-off +// goal's deadline across a plug-in. +func TestProfileTargetKeepsOneOffDeadline(t *testing.T) { + h := newHarness(t, homeConfig(), nil, nil) + h.start(Profile{ID: "ioniq", Name: "Ioniq", CapacityWh: 58000, Identifiers: []string{"RFID-9"}, TargetSoC: .9}) + if c := h.only(); c.Name != "Ioniq" { + t.Fatalf("not merged: %+v", c) + } + // One-off goal: 80 % by 07:00 tomorrow. + if ok, err := h.mgr.SetScheduleChecked("easee", loadpoint.NewSchedule(.8, 420, false, 0, false)); !ok || err != nil { + t.Fatal(ok, err) + } + h.tick(time.Second) + before := h.state("easee") + if before.TargetTime.IsZero() || before.TargetSoC != .8 { + t.Fatalf("goal not applied before plug-in: %.2f by %s", before.TargetSoC, before.TargetTime) + } + h.plug("easee", "s1") + for i := 0; i < 3; i++ { + h.tick(5 * time.Second) + } + after := h.state("easee") + if after.TargetTime.IsZero() || after.TargetSoC != .8 { + t.Fatalf("plug-in erased the one-off goal: before %.2f by %s, after %.2f by %s", + before.TargetSoC, before.TargetTime, after.TargetSoC, after.TargetTime) + } +} + +// Without a goal, the profile target still applies to the session. +func TestProfileTargetAppliesWithoutGoal(t *testing.T) { + h := newHarness(t, homeConfig(), nil, nil) + h.start(Profile{ID: "ioniq", Name: "Ioniq", CapacityWh: 58000, Identifiers: []string{"RFID-9"}, TargetSoC: .9}) + h.tick(time.Second) + h.plug("easee", "s1") + h.tick(5 * time.Second) + if st := h.state("easee"); st.TargetSoC != .9 || !st.TargetTime.IsZero() { + t.Fatalf("profile target not applied: %.2f by %s", st.TargetSoC, st.TargetTime) + } +} diff --git a/go/internal/cars/reserve_test.go b/go/internal/cars/reserve_test.go index 44e697860..623a608d9 100644 --- a/go/internal/cars/reserve_test.go +++ b/go/internal/cars/reserve_test.go @@ -16,10 +16,23 @@ import ( func fallback(t *testing.T, h *harness) (ok bool, maxW float64) { t.Helper() svc := &mpc.Service{LoadpointStates: h.mgr.States} - d, _ := svc.SlotDirectiveAt(h.now) + // Ask at the goal's deadline: before its latest start a goal with + // slack holds 0 W (#1718). + at := h.now + if st, ok := h.mgr.State("easee"); ok && st.TargetTime.After(at) { + at = st.TargetTime + } + d, _ := svc.SlotDirectiveAt(at) return d.LoadpointFallback["easee"], d.LoadpointMaxPowerW["easee"] } +// reserveNeed is the reserve's need for a session on the charger's own +// table, as fallback's service holds it. +func reserveNeed(st loadpoint.State) (energyWh, powerW float64, ok bool) { + minW, maxW, steps := loadpoint.LimitPower(st, st.AllowedStepsW) + return loadpoint.DeadlineNeed(st, minW, maxW, steps) +} + // A failed replan keeps the session car's goal charging, held to that car's // own charger and level, the moment the box knows the car; never a guess for // a car it does not know. @@ -91,7 +104,7 @@ func TestAFailedReplanKeepsTheSessionCarsGoal(t *testing.T) { t.Fatal(err) } st := h.state("easee") - need, peak, ok := loadpoint.DeadlineNeed(st) + need, peak, ok := reserveNeed(st) want := (0.6 - 0.3) * 40000 / loadpoint.DefaultChargeEfficiency if !ok || st.VehicleCapacityWh != 40000 || math.Abs(st.CurrentSoC-.3) > 1e-9 || math.Abs(need-want) > 1 || peak > 7400 { t.Fatalf("the Volvo's need %v (want %v) at %v W, capacity %v, level %v", need, want, peak, st.VehicleCapacityWh, st.CurrentSoC) @@ -117,7 +130,7 @@ func TestAFailedReplanKeepsTheSessionCarsGoal(t *testing.T) { if ok, _ := fallback(t, h); !ok { t.Fatal("a visitor's goal got no reserve") } - if need, _, _ := loadpoint.DeadlineNeed(st); math.Abs(need-0.5*76000/loadpoint.DefaultChargeEfficiency) > 1 { + if need, _, _ := reserveNeed(st); math.Abs(need-0.5*76000/loadpoint.DefaultChargeEfficiency) > 1 { t.Fatalf("a visitor's need %v, want it from empty", need) } h.tick(time.Second) @@ -145,7 +158,7 @@ func TestTheReserveStopsAtTheCarsChargeLimit(t *testing.T) { h.obs["tesla-ha"] = telemetry.VehicleObservation{Driver: "tesla-ha", SoC: .5, ChargeLimit: .7, At: h.now, ReportedAt: h.now} h.tick(time.Second) st := h.state("easee") - need, _, ok := loadpoint.DeadlineNeed(st) + need, _, ok := reserveNeed(st) if want := (0.7 - 0.5) * 76000 / loadpoint.DefaultChargeEfficiency; !ok || st.CarChargeLimit != .7 || math.Abs(need-want) > 50 { t.Fatalf("need %v (want about %v), limit %v", need, want, st.CarChargeLimit) } diff --git a/go/internal/cars/tick.go b/go/internal/cars/tick.go index 8c9abe825..15fec65bb 100644 --- a/go/internal/cars/tick.go +++ b/go/internal/cars/tick.go @@ -177,8 +177,8 @@ func (r *Runtime) resumeSessionLocked(st loadpoint.State, s *session) { const bootWindow = 30 * time.Second // UnplugGrace is how long a charger must say the cable is out before the -// session ends. -const UnplugGrace = 30 * time.Second +// session ends. The loadpoint clears Charge now on the same grace. +const UnplugGrace = loadpoint.UnplugGrace // PrePlugMaxAge is how much older than the plug-in a level may be and still // start the session. An older one may predate a drive. @@ -801,7 +801,9 @@ func (r *Runtime) syncLocked(now time.Time) []func() { s.seeded = true seed := r.seedLocked(car, s, st) acts = append(acts, func() { lps.SeedSessionLevel(id, seed) }) - if car != nil && car.ProfileTarget > 0 { + // The profile target fills in only for a car without a goal; + // SetTarget clears the deadline of the goal it would replace. + if car != nil && car.ProfileTarget > 0 && !goal.HasTarget() { target := car.ProfileTarget acts = append(acts, func() { lps.SetTarget(id, target, time.Time{}) }) } diff --git a/go/internal/config/config.go b/go/internal/config/config.go index 731455fdc..0b8e473f0 100644 --- a/go/internal/config/config.go +++ b/go/internal/config/config.go @@ -17,6 +17,7 @@ import ( "sync" "time" + "github.com/srcfl/ftw/go/internal/localzone" "gopkg.in/yaml.v3" ) @@ -54,8 +55,9 @@ type Config struct { LoadWarnings []string `yaml:"-" json:"-"` LoadpointWarnings []string `yaml:"-" json:"-"` // Retired names settings of removed features that loading dropped so - // they cannot steer the site. Startup warns once for each; - // DropRetiredSettings deletes them from stored settings. + // they cannot steer the site, and stored values that match an old + // default. Startup warns once for each; DropRetiredSettings deletes the + // removed settings from stored settings and leaves the values alone. Retired []string `yaml:"-" json:"-"` // Used once at startup to end an old calendar's persisted away selection. RetiredCalendarEnabled bool `yaml:"-" json:"-"` @@ -800,6 +802,11 @@ type Site struct { SlewRateW float64 `yaml:"slew_rate_w" json:"slew_rate_w"` MinDispatchIntervalS int `yaml:"min_dispatch_interval_s" json:"min_dispatch_interval_s"` + // TimeZone is the household's IANA zone: the clock car goals, the import + // hour, daily history and savings read. Empty keeps the host's zone. + // Core reads it once at start (localzone.Use). + TimeZone string `yaml:"time_zone,omitempty" json:"time_zone,omitempty"` + // SlewEnabled gates the external per-cycle ramp limiter. Both // supported inverter families (Ferroamp, Sungrow) have their own // internal power-ramp control loops; the external slew was @@ -868,11 +875,16 @@ type Site struct { MaxExportW float64 `yaml:"max_export_w,omitempty" json:"max_export_w,omitempty"` } -// DefaultFuseSafetyMarginA is the fall-back per-phase amp headroom -// applied when fuse.safety_margin_a is unset (nil) in the YAML. -// Single source of truth — main.go routes through Fuse.Effective- -// SafetyMarginA() rather than re-declaring it. -const DefaultFuseSafetyMarginA = 0.5 +// DefaultFuseSafetyMarginA is the per-phase margin applied when +// fuse.safety_margin_a is unset (nil). Core works to the fuse rating by +// default; a margin is opt-in. Single source of truth: main.go routes +// through Fuse.EffectiveSafetyMarginA() rather than re-declaring it. +const DefaultFuseSafetyMarginA = 0.0 + +// oldDefaultFuseSafetyMarginA was the default until FTW worked to the fuse +// rating. A stored value equal to it may be the settings form's old prefill +// rather than a choice, so loading names it once and keeps it. +const oldDefaultFuseSafetyMarginA = 0.5 // Fuse describes the shared breaker limit used by the fuse guard. type Fuse struct { @@ -880,15 +892,11 @@ type Fuse struct { Phases int `yaml:"phases" json:"phases"` Voltage float64 `yaml:"voltage" json:"voltage"` - // SafetyMarginA reserves headroom (per-phase amps) below MaxAmps - // inside the dispatch fuse guard. Pointer so we can distinguish - // "unset" (nil → DefaultFuseSafetyMarginA) from "explicitly - // disabled" (non-nil 0.0). Inverters often have their own per- - // phase current protection that trips before the breaker; without - // a margin the dispatch can ride right up to MaxAmps and the - // inverter cuts to 0 W in one tick, then dispatch ramps back up — - // visible as a flap. 0.5 A × 230 V × 3 phases ≈ 345 W of aggregate - // headroom. + // SafetyMarginA is an opt-in per-phase margin below MaxAmps: each + // phase's working limit is MaxAmps − SafetyMarginA. Pointer so an + // explicit value, 0 included, always wins over the default (nil → + // DefaultFuseSafetyMarginA, which is 0). Set it for an inverter whose + // own per-phase limiter trips at the rating and cuts to 0 W. SafetyMarginA *float64 `yaml:"safety_margin_a,omitempty" json:"safety_margin_a,omitempty"` } @@ -897,10 +905,9 @@ func (f Fuse) MaxPowerW() float64 { return f.MaxAmps * f.Voltage * float64(f.Phases) } -// EffectiveSafetyMarginA returns the per-phase amp headroom to apply, -// resolving nil ("unset → use default") vs an explicit value (including -// 0.0 to disable the margin entirely). Single read site so the default -// can never drift across consumers. +// EffectiveSafetyMarginA returns the per-phase margin to apply: the +// explicit value when set, otherwise DefaultFuseSafetyMarginA. Single read +// site so the default can never drift across consumers. func (f Fuse) EffectiveSafetyMarginA() float64 { if f.SafetyMarginA == nil { return DefaultFuseSafetyMarginA @@ -908,6 +915,17 @@ func (f Fuse) EffectiveSafetyMarginA() float64 { return *f.SafetyMarginA } +// noteOldFuseMargin adds one notice when fuse.safety_margin_a holds FTW's +// old default. The value stays: from April to September 2026 the settings +// form saved its 0.5 A prefill on any save, but a value saved later is the +// user's choice, and the stored document cannot tell the two apart. +func (c *Config) noteOldFuseMargin() { + if m := c.Fuse.SafetyMarginA; m != nil && *m == oldDefaultFuseSafetyMarginA { + c.Retired = append(c.Retired, + "fuse.safety_margin_a is 0.5 A, FTW's old default. FTW now works to the fuse rating. Set 0 to use it.") + } +} + // Driver is one driver entry. Each driver is a Lua script loaded by // the driver host at startup (or on hot-reload via the file watcher). type Driver struct { @@ -1132,6 +1150,11 @@ type Price struct { GridTariffOreKwh float64 `yaml:"grid_tariff_ore_kwh,omitempty" json:"grid_tariff_ore_kwh,omitempty"` VATPercent float64 `yaml:"vat_percent,omitempty" json:"vat_percent,omitempty"` APIKey string `yaml:"api_key,omitempty" json:"api_key,omitempty"` + // BaseURL is for tests and development only. It points the + // elprisetjustnu provider at another server with the same API and + // turns off the Nord Pool fallback, so the site rig can pin prices. + // Only the elprisetjustnu provider reads it; other providers ignore it. + BaseURL string `yaml:"base_url,omitempty" json:"base_url,omitempty"` // Currency is the ISO code for pricing (default "SEK"). ENTSOE // returns EUR/MWh; we convert using ECB daily FX rates. @@ -1491,6 +1514,7 @@ func Parse(data []byte, baseDir string) (*Config, error) { var retiredControl retiredControlSettings _ = doc.Decode(&retiredControl) c.applyRetiredControl(retiredControl) + c.noteOldFuseMargin() // An omitted app_link section follows the new default. An explicit YAML // null was a valid opt-out before that default changed, so retain it as an // explicit disabled section instead of letting applyDefaults turn it on. @@ -1683,6 +1707,23 @@ func relToBaseDir(baseDir, p string) string { return rel } +// FillUnsetTimings treats a zero control interval or watchdog timeout as +// unset. Neither has a meaning at zero: a 0 s interval panics the boot ticker +// and a 0 s watchdog marks every driver stale. Every decoded settings +// document runs this, not only YAML, so a partial save cannot store a zero +// that the next boot cannot start from. +func (s *Site) FillUnsetTimings() { + if s.ControlIntervalS == 0 { + // 2 s matches Ferroamp's ehub MQTT cadence (~1 Hz) without + // dispatching twice on the same telemetry sample, and halves + // the perceived response lag operators saw at the original 5 s. + s.ControlIntervalS = 2 + } + if s.WatchdogTimeoutS == 0 { + s.WatchdogTimeoutS = 60 + } +} + // applyDefaults fills in sensible zero-value defaults. func applyDefaults(c *Config) { if c.DeviceRepository == nil { @@ -1723,18 +1764,10 @@ func applyDefaults(c *Config) { } // Endpoint is deliberately left blank: Resolved() supplies the default at // use, so the collector's address is never written into a household's YAML. - if c.Site.ControlIntervalS == 0 { - // 2 s matches Ferroamp's ehub MQTT cadence (~1 Hz) without - // dispatching twice on the same telemetry sample, and halves - // the perceived response lag operators saw at the original 5 s. - c.Site.ControlIntervalS = 2 - } + c.Site.FillUnsetTimings() if c.Site.GridToleranceW == 0 { c.Site.GridToleranceW = 42 // The Answer } - if c.Site.WatchdogTimeoutS == 0 { - c.Site.WatchdogTimeoutS = 60 - } if c.Site.SmoothingAlpha == 0 { c.Site.SmoothingAlpha = 0.3 } @@ -1920,6 +1953,7 @@ func isLegacyDefaultDriverRepository(repo DriverRepositorySource) bool { // Validate ensures the config is internally consistent and safe to run with. func (c *Config) Validate() error { c.NormalizeUnits() + c.dropZeroBatteryCeilings() if err := c.validateSetup(); err != nil { return err } @@ -1950,6 +1984,11 @@ func (c *Config) Validate() error { if err := c.Price.Validate(); err != nil { return err } + if c.Site.TimeZone != "" { + if _, err := localzone.Load(c.Site.TimeZone); err != nil { + return fmt.Errorf("site.time_zone: %w", err) + } + } // Empty drivers list is a valid shape — e.g. an EV-only site that // configured a cloud EV charger in the setup wizard and doesn't @@ -2060,7 +2099,7 @@ func (c *Config) Validate() error { } // safety_margin_a must be in [0, max_amps) when explicitly set. // Negative would *raise* the per-phase threshold above the breaker - // rating (defeating the guard); >= max_amps zeroes the headroom + // rating (defeating the guard); >= max_amps leaves no working limit // and silently disables the per-phase clamp — both are real safety // holes if reached through a typo'd config. nil (unset) is OK and // resolves to DefaultFuseSafetyMarginA at the consumer. diff --git a/go/internal/config/config_test.go b/go/internal/config/config_test.go index 86bb85cb9..d628ee75a 100644 --- a/go/internal/config/config_test.go +++ b/go/internal/config/config_test.go @@ -1260,12 +1260,43 @@ func TestNotificationsMaskSecrets(t *testing.T) { } } +// An unset margin works to the fuse rating: the default is 0 and a margin +// is opt-in (owner rule, 27 Oct 2026). func TestFuseSafetyMarginNilUsesDefault(t *testing.T) { - // Field omitted in YAML → nil pointer → default 0.5 A. f := Fuse{MaxAmps: 16, Voltage: 230, Phases: 3} - if got := f.EffectiveSafetyMarginA(); got != DefaultFuseSafetyMarginA { - t.Errorf("nil margin: got %v, want %v (DefaultFuseSafetyMarginA)", - got, DefaultFuseSafetyMarginA) + if got := f.EffectiveSafetyMarginA(); got != 0 || DefaultFuseSafetyMarginA != 0 { + t.Errorf("nil margin: got %v (default %v), want 0", got, DefaultFuseSafetyMarginA) + } +} + +// Every explicit margin is kept. Exactly 0.5 A was the old default, which +// the settings form once saved on any save, so loading adds one notice for +// it and changes nothing. +func TestFuseSafetyMarginLoadKeepsExplicitValues(t *testing.T) { + for _, tc := range []struct { + line string + want float64 + wantNotice bool + }{ + {"", 0, false}, + {" safety_margin_a: 0\n", 0, false}, + {" safety_margin_a: 0.5\n", 0.5, true}, + {" safety_margin_a: 0.4\n", 0.4, false}, + {" safety_margin_a: 1.2\n", 1.2, false}, + } { + yaml := strings.Replace(minimalYAML, " max_amps: 16\n", " max_amps: 16\n"+tc.line, 1) + c, err := Parse([]byte(yaml), "/tmp") + if err != nil { + t.Fatalf("%q: %v", tc.line, err) + } + if got := c.Fuse.EffectiveSafetyMarginA(); got != tc.want { + t.Errorf("%q: margin %v, want %v", tc.line, got, tc.want) + } + notice := len(c.Retired) == 1 && strings.Contains(c.Retired[0], "old default") && + strings.Contains(c.Retired[0], "Set 0") + if notice != tc.wantNotice || (!tc.wantNotice && len(c.Retired) != 0) { + t.Errorf("%q: notices %q, want the old-default notice: %v", tc.line, c.Retired, tc.wantNotice) + } } } diff --git a/go/internal/config/heal_before_load_test.go b/go/internal/config/heal_before_load_test.go new file mode 100644 index 000000000..61b6bc800 --- /dev/null +++ b/go/internal/config/heal_before_load_test.go @@ -0,0 +1,139 @@ +package config + +import ( + "os" + "strings" + "testing" + + "github.com/srcfl/ftw/go/internal/state" +) + +func damageHeader(t *testing.T, database string) { + t.Helper() + f, err := os.OpenFile(database, os.O_WRONLY, 0) + if err != nil { + t.Fatal(err) + } + defer f.Close() + junk := make([]byte, 4096) + for i := range junk { + junk[i] = 0xAB + } + if _, err := f.WriteAt(junk, 0); err != nil { + t.Fatal(err) + } +} + +// A damaged state.db with a good snapshot. Boot reads settings (Load) before +// state.Open, so the heal must run before Load, with or without the +// verified-good marker that a power loss leaves in place. +func TestCorruptStateDBWithGoodSnapshotHealsBeforeLoad(t *testing.T) { + for _, keepMarker := range []bool{false, true} { + name := "marker-removed-by-background-verify" + if keepMarker { + name = "marker-present-default" + } + t.Run(name, func(t *testing.T) { + path, database, st := importedLiveSettings(t) + if err := st.SnapshotState(); err != nil { + t.Fatal(err) + } + if err := st.Close(); err != nil { + t.Fatal(err) + } + if !keepMarker { + if err := os.Remove(database + ".clean"); err != nil { + t.Fatal(err) + } + } + damageHeader(t, database) + + // Same order as main.go. + ev, err := HealSettingsDatabase(path) + if err != nil { + t.Fatal(err) + } + if ev == nil || ev.Tier != "state" || ev.Action != "restored" { + t.Fatalf("heal event = %+v", ev) + } + cfg, err := Load(path) + if err != nil { + t.Fatalf("config.Load after heal: %v", err) + } + if cfg.Site.Name != "Live Settings" { + t.Fatalf("site = %q", cfg.Site.Name) + } + healed, err := state.Open(database) + if err != nil { + t.Fatal(err) + } + defer healed.Close() + if _, err := InitializeStorage(path, database, cfg, healed); err != nil { + t.Fatal(err) + } + }) + } +} + +func TestHealthyStateDBIsNotHealedBeforeLoad(t *testing.T) { + path, database, st := importedLiveSettings(t) + if err := st.SnapshotState(); err != nil { + t.Fatal(err) + } + if err := st.Close(); err != nil { + t.Fatal(err) + } + ev, err := HealSettingsDatabase(path) + if err != nil || ev != nil { + t.Fatalf("heal = %+v, %v", ev, err) + } + if _, err := os.Stat(database + ".clean"); err != nil { + t.Fatalf("verified-good marker removed: %v", err) + } +} + +// Load read newer settings from a damaged but readable state.db, then +// state.Open restored an older snapshot. Core runs the snapshot's settings +// and reports the step back instead of exiting. +func TestOlderSnapshotSettingsRunAndAreReported(t *testing.T) { + path, database, st := importedLiveSettings(t) + if err := st.SnapshotState(); err != nil { + t.Fatal(err) + } + newer, err := Load(path) + if err != nil { + t.Fatal(err) + } + newer.Site.Name = "Newer Settings" + if err := SaveStored(st, path, newer); err != nil { + t.Fatal(err) + } + loaded, err := Load(path) + if err != nil { + t.Fatal(err) + } + if err := st.Close(); err != nil { + t.Fatal(err) + } + if err := os.Remove(database + ".clean"); err != nil { + t.Fatal(err) + } + damageHeader(t, database) + + healed, err := state.Open(database) + if err != nil { + t.Fatal(err) + } + defer healed.Close() + got, err := InitializeStorage(path, database, loaded, healed) + if err != nil { + t.Fatal(err) + } + if got.Site.Name != "Live Settings" || got.Revision >= loaded.Revision { + t.Fatalf("settings = %q revision %d, loaded revision %d", got.Site.Name, got.Revision, loaded.Revision) + } + events := healed.HealEvents() + if len(events) == 0 || !strings.Contains(events[len(events)-1].Detail, "settings went back from revision") { + t.Fatalf("heal events = %+v", events) + } +} diff --git a/go/internal/config/restart_required.go b/go/internal/config/restart_required.go index b3aeaa4f5..ecaccb0f2 100644 --- a/go/internal/config/restart_required.go +++ b/go/internal/config/restart_required.go @@ -25,6 +25,9 @@ func RestartRequiredFor(oldCfg, newCfg *Config) []string { reasons = append(reasons, "setup.measurement_only — restart to change equipment control") } + if oldCfg.Site.TimeZone != newCfg.Site.TimeZone { + reasons = append(reasons, "site.time_zone — the household clock is set at startup") + } if oldCfg.API.Port != newCfg.API.Port { reasons = append(reasons, "api.port — HTTP server binds the port at startup") } @@ -33,7 +36,7 @@ func RestartRequiredFor(oldCfg, newCfg *Config) []string { reasons = append(reasons, "state.path / state.cold_dir — database files are opened at startup") } if priceNeedsRestart(oldCfg.Price, newCfg.Price) { - reasons = append(reasons, "price.provider / zone / API key — spot-price client is constructed at startup") + reasons = append(reasons, "price.provider / zone / API key / base_url — spot-price client is constructed at startup") } if plannerNeedsRestart(oldCfg.Planner, newCfg.Planner) { reasons = append(reasons, "planner.enabled / engine — planner process is constructed at startup") @@ -84,7 +87,7 @@ func priceNeedsRestart(oldP, newP *Price) bool { if !oldOn { return false } - return oldP.Provider != newP.Provider || oldP.Zone != newP.Zone || oldP.APIKey != newP.APIKey || oldP.Currency != newP.Currency + return oldP.Provider != newP.Provider || oldP.Zone != newP.Zone || oldP.APIKey != newP.APIKey || oldP.Currency != newP.Currency || oldP.BaseURL != newP.BaseURL } func plannerNeedsRestart(oldP, newP *Planner) bool { diff --git a/go/internal/config/restart_required_test.go b/go/internal/config/restart_required_test.go index 6fceb818b..9d7223d85 100644 --- a/go/internal/config/restart_required_test.go +++ b/go/internal/config/restart_required_test.go @@ -174,6 +174,16 @@ func TestRestartRequiredFor_PriceZoneSwapNeedsRestart(t *testing.T) { } } +func TestRestartRequiredFor_PriceBaseURLChangeNeedsRestart(t *testing.T) { + old := baseCfg() + old.Price = &Price{Provider: "elprisetjustnu", Zone: "SE3", BaseURL: "http://127.0.0.1:8099"} + n := baseCfg() + n.Price = &Price{Provider: "elprisetjustnu", Zone: "SE3", BaseURL: "http://127.0.0.1:8100"} + if !containsSubstring(RestartRequiredFor(old, n), "price") { + t.Fatal("changing price.base_url should require restart") + } +} + func TestRestartRequiredFor_NilInputs(t *testing.T) { if r := RestartRequiredFor(nil, baseCfg()); r != nil { t.Fatalf("expected nil reasons for nil old, got %v", r) diff --git a/go/internal/config/retired_control_test.go b/go/internal/config/retired_control_test.go index 0eb9f03f8..ca0897e03 100644 --- a/go/internal/config/retired_control_test.go +++ b/go/internal/config/retired_control_test.go @@ -132,3 +132,51 @@ func TestStoredRetiredControlSettingsMigrateAndAreDeleted(t *testing.T) { t.Fatalf("second DropRetiredSettings = %v, %v; want nothing", removed, err) } } + +// A stored 0.5 A margin keeps steering the site and stays in state.db; the +// notice comes from loading it, so dropping retired settings leaves it alone. +func TestStoredOldDefaultFuseMarginIsKeptWithANotice(t *testing.T) { + dir := t.TempDir() + path, database := filepath.Join(dir, "config.yaml"), filepath.Join(dir, "state.db") + yaml := strings.Replace(minimalYAML, " max_amps: 16\n", " max_amps: 16\n safety_margin_a: 0.5\n", 1) + if err := os.WriteFile(path, []byte(yaml), 0600); err != nil { + t.Fatal(err) + } + cfg, err := Load(path) + if err != nil { + t.Fatal(err) + } + st, err := state.Open(database) + if err != nil { + t.Fatal(err) + } + defer st.Close() + if _, err := InitializeStorage(path, database, cfg, st); err != nil { + t.Fatal(err) + } + for range 2 { + loaded, err := Load(path) + if err != nil { + t.Fatal(err) + } + if loaded.ConfigDatabase == "" { + t.Fatal("loaded from the YAML file, not from stored settings") + } + if got := loaded.Fuse.EffectiveSafetyMarginA(); got != 0.5 { + t.Fatalf("stored margin = %v, want 0.5", got) + } + if len(loaded.Retired) != 1 || !strings.Contains(loaded.Retired[0], "safety_margin_a is 0.5 A") { + t.Fatalf("notices = %q, want one naming the old 0.5 A default", loaded.Retired) + } + if removed, err := DropRetiredSettings(st, path, loaded); err != nil || len(removed) != 0 { + t.Fatalf("DropRetiredSettings = %v, %v; the margin is a setting, not a retired one", removed, err) + } + } + after, _, err := st.Configuration() + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(after.Document), `"safety_margin_a":0.5`) { + t.Fatalf("stored settings lost the margin: %s", after.Document) + } +} diff --git a/go/internal/config/soc_bounds.go b/go/internal/config/soc_bounds.go index c0da7f092..dbeb45d8d 100644 --- a/go/internal/config/soc_bounds.go +++ b/go/internal/config/soc_bounds.go @@ -1,5 +1,10 @@ package config +import ( + "log/slog" + "maps" +) + // WithBatterySoCBounds returns a copy of drivers in which every driver that has // a matching batteries[] entry gets charge_ceil_soc / discharge_floor_soc // defaulted into its free-form Config from the battery's soc_max / soc_min. @@ -51,3 +56,26 @@ func WithBatterySoCBounds(drivers []Driver, batteries map[string]Battery) []Driv } return out } + +// dropZeroBatteryCeilings treats a battery soc_max of 0 or less as unset. +// Settings saved a cleared number field as 0, and an older Settings saved +// every field on the tab. Since soc_max became the dispatch charge ceiling, +// that 0 blocked all charging, manual Charge included. A battery that must +// not charge sets max_charge_w: 0. +func (c *Config) dropZeroBatteryCeilings() { + cloned := false + for name, b := range c.Batteries { + if b.SoCMax == nil || *b.SoCMax > 0 { + continue + } + slog.Warn("config: ignoring battery soc_max of 0 or less; the planner's ceiling applies", + "battery", name, "soc_max", *b.SoCMax) + // Callers validate shallow copies of the live config; never write + // into the map they share. + if !cloned { + c.Batteries, cloned = maps.Clone(c.Batteries), true + } + b.SoCMax = nil + c.Batteries[name] = b + } +} diff --git a/go/internal/config/soc_bounds_test.go b/go/internal/config/soc_bounds_test.go index 630acf1fb..58c116cb2 100644 --- a/go/internal/config/soc_bounds_test.go +++ b/go/internal/config/soc_bounds_test.go @@ -1,6 +1,9 @@ package config -import "testing" +import ( + "encoding/json" + "testing" +) func f64(v float64) *float64 { return &v } @@ -96,3 +99,67 @@ func TestWithBatterySoCBoundsDoesNotMutateInput(t *testing.T) { t.Errorf("merged copy lost existing key foo: got %v", got) } } + +// Settings saved a blank Max SoC as 0. Only soc_max loses it: a soc_min of 0 +// and the power limits are real choices. +func TestDropZeroBatteryCeilings(t *testing.T) { + c := &Config{Batteries: map[string]Battery{ + "blank": {SoCMin: f64(0), SoCMax: f64(0), MaxChargeW: f64(7500)}, + "negative": {SoCMax: f64(-0.1)}, + "set": {SoCMax: f64(0.9)}, + "unset": {SoCMin: f64(0.1)}, + }} + + c.dropZeroBatteryCeilings() + + for name, b := range c.Batteries { + if name != "set" && b.SoCMax != nil { + t.Errorf("%s: soc_max = %v, want unset", name, *b.SoCMax) + } + } + if b := c.Batteries["set"]; b.SoCMax == nil || *b.SoCMax != 0.9 { + t.Errorf("set: soc_max = %v, want 0.9", b.SoCMax) + } + if b := c.Batteries["blank"]; b.SoCMin == nil || *b.SoCMin != 0 || b.MaxChargeW == nil || *b.MaxChargeW != 7500 { + t.Errorf("blank: other limits changed: %+v", b) + } + if out := WithBatterySoCBounds([]Driver{{Name: "blank"}}, c.Batteries); out[0].Config["charge_ceil_soc"] != nil { + t.Errorf("a dropped soc_max still reached the driver: %v", out[0].Config) + } +} + +// A box's settings live in the config database. A zero an older Core stored +// there must load as unset, and the copy being validated must not write into +// the map it shares with the live config. +func TestStoredZeroSoCMaxLoadsAsUnset(t *testing.T) { + path, _, st := importedLiveSettings(t) + doc, found, err := st.Configuration() + if err != nil || !found { + t.Fatalf("stored configuration: found=%v err=%v", found, err) + } + var stored map[string]any + if err := json.Unmarshal(doc.Document, &stored); err != nil { + t.Fatal(err) + } + stored["config"].(map[string]any)["batteries"] = map[string]any{ + "pixii": map[string]any{"soc_min": 0, "soc_max": 0, "max_charge_w": 0, "max_discharge_w": 0}, + } + raw, _ := json.Marshal(stored) + if _, err := st.SaveConfiguration(raw, doc.Revision, nil); err != nil { + t.Fatal(err) + } + got, err := Load(path) + if err != nil { + t.Fatal(err) + } + if b := got.Batteries["pixii"]; b.SoCMax != nil || b.SoCMin == nil || b.MaxChargeW == nil { + t.Fatalf("loaded %+v, want soc_max unset and the other limits kept", b) + } + + live := Config{Batteries: map[string]Battery{"pixii": {SoCMax: f64(0)}}} + next := live + next.dropZeroBatteryCeilings() + if live.Batteries["pixii"].SoCMax == nil || next.Batteries["pixii"].SoCMax != nil { + t.Fatalf("live %+v, copy %+v: want only the copy changed", live.Batteries, next.Batteries) + } +} diff --git a/go/internal/config/storage.go b/go/internal/config/storage.go index 501b03310..5b002ef49 100644 --- a/go/internal/config/storage.go +++ b/go/internal/config/storage.go @@ -9,6 +9,7 @@ import ( "os" "path/filepath" "reflect" + "time" "github.com/srcfl/ftw/go/internal/state" "gopkg.in/yaml.v3" @@ -64,8 +65,11 @@ func decodeStored(c state.Configuration, database, baseDir string) (*Config, err } _ = json.Unmarshal(c.Document, &legacyControl) cfg.applyRetiredControl(legacyControl.Config) + cfg.noteOldFuseMargin() // This is the same typed config that was validated on save. Do not apply new // defaults during a storage-only reload: nil, false and zero stay distinct. + // The only exception is the site timings, which have no meaning at zero. + cfg.Site.FillUnsetTimings() if err := cfg.Validate(); err != nil { return nil, fmt.Errorf("stored settings: %w", err) } @@ -73,6 +77,41 @@ func decodeStored(c state.Configuration, database, baseDir string) (*Config, err return cfg, nil } +func restoredFromSnapshot(st *state.Store) bool { + for _, ev := range st.HealEvents() { + if ev.Tier == "state" && ev.Action == "restored" { + return true + } + } + return false +} + +// HealSettingsDatabase runs state.HealBeforeLoad on the database a seed file +// names. Boot calls it before Load, which reads settings before state.Open. +func HealSettingsDatabase(path string) (*state.HealEvent, error) { + database := filepath.Join(filepath.Dir(path), "state.db") + data, err := os.ReadFile(path) + if err == nil { + var source struct { + Database string `yaml:"config_database"` + } + if yaml.Unmarshal(data, &source) != nil || source.Database == "" { + return nil, nil // the seed holds the settings + } + database = source.Database + if !filepath.IsAbs(database) { + database = filepath.Join(filepath.Dir(path), database) + } + } else if !errors.Is(err, os.ErrNotExist) { + return nil, nil // Load reports it + } + database, err = filepath.Abs(database) + if err != nil { + return nil, err + } + return state.HealBeforeLoad(database) +} + func loadStored(database, baseDir string) (*Config, error) { doc, err := state.ReadConfiguration(database) if err != nil { @@ -98,7 +137,9 @@ func loadSettingsBesideSeed(seedPath string) (*Config, bool) { // InitializeStorage imports YAML once, then records the database location in // the seed file. If a crash interrupted this last step, reuse the committed // document instead of importing the old YAML again. A recovered database must -// still contain the exact settings revision loaded before state.Open. +// still contain the exact settings revision loaded before state.Open, unless +// state.Open restored it from an older snapshot: then the snapshot's settings +// run and /api/health reports the step back. func InitializeStorage(path, database string, cfg *Config, st *state.Store) (*Config, error) { database, err := filepath.Abs(database) if err != nil { @@ -112,13 +153,21 @@ func InitializeStorage(path, database string, cfg *Config, st *state.Store) (*Co return nil, err } if cfg.ConfigDatabase != "" { - if !found || doc.Revision != cfg.Revision { + rolledBack := found && doc.Revision < cfg.Revision && restoredFromSnapshot(st) + if !found || (doc.Revision != cfg.Revision && !rolledBack) { return nil, errors.New("database recovery lost current settings; restore a full backup") } current, err := decodeStored(doc, database, filepath.Dir(path)) if err != nil { return nil, err } + if rolledBack { + // The next boot would load the snapshot's settings anyway; run them + // now instead of exiting, and say so. + st.RecordHealEvent(state.HealEvent{Tier: "state", Action: "restored", AtMs: time.Now().UnixMilli(), + Detail: fmt.Sprintf("state.db was corrupt — restored from last snapshot; settings went back from revision %d to %d", cfg.Revision, doc.Revision)}) + cfg = current + } if !reflect.DeepEqual(cfg, current) { return nil, errors.New("database recovery changed current settings; restore a full backup") } diff --git a/go/internal/config/storage_test.go b/go/internal/config/storage_test.go index 5339d591e..8a0100760 100644 --- a/go/internal/config/storage_test.go +++ b/go/internal/config/storage_test.go @@ -492,3 +492,25 @@ func TestDropRetiredSettingsDeletesTheAskWhyKey(t *testing.T) { t.Fatalf("second DropRetiredSettings = %v, %v; want nothing", removed, err) } } + +// A box that stored zero timings before they counted as unset must still boot. +// Slew stays 0, because zero slew means "no limit". +func TestStoredZeroTimingsLoadAsDefaults(t *testing.T) { + path, _, st := importedLiveSettings(t) + cfg, err := Load(path) + if err != nil { + t.Fatal(err) + } + cfg.Site.ControlIntervalS, cfg.Site.WatchdogTimeoutS, cfg.Site.SlewRateW = 0, 0, 0 + if err := SaveStored(st, path, cfg); err != nil { + t.Fatal(err) + } + got, err := Load(path) + if err != nil { + t.Fatal(err) + } + if got.Site.ControlIntervalS != 2 || got.Site.WatchdogTimeoutS != 60 || got.Site.SlewRateW != 0 { + t.Fatalf("reloaded interval=%d watchdog=%d slew=%v, want 2, 60, 0", + got.Site.ControlIntervalS, got.Site.WatchdogTimeoutS, got.Site.SlewRateW) + } +} diff --git a/go/internal/configreload/apply.go b/go/internal/configreload/apply.go index 62f412a30..fa1d418c6 100644 --- a/go/internal/configreload/apply.go +++ b/go/internal/configreload/apply.go @@ -50,6 +50,7 @@ func Apply( if newCfg.Site.MinDispatchIntervalS != oldCfg.Site.MinDispatchIntervalS { ctrl.MinDispatchIntervalS = newCfg.Site.MinDispatchIntervalS } + ctrl.ControlIntervalS = newCfg.Site.ControlIntervalS gain := newCfg.Site.Gain if gain == 0 { gain = 0.5 @@ -79,6 +80,15 @@ func Apply( if newCfg.Site.DCLinkProtectionMarginW != oldCfg.Site.DCLinkProtectionMarginW { ctrl.DCLinkProtectionMarginW = newCfg.Site.DCLinkProtectionMarginW } + // Fuse and export limits reach dispatch before the applier restarts any + // driver, so a save that lowers them cannot wait behind driver I/O. + ctrl.SiteFuseAmps = newCfg.Fuse.MaxAmps + ctrl.SiteFuseVoltage = newCfg.Fuse.Voltage + ctrl.SiteFusePhases = newCfg.Fuse.Phases + // nil → 0 (work to the fuse rating), an explicit value wins. See + // EffectiveSafetyMarginA. + ctrl.SiteFuseSafetyA = newCfg.Fuse.EffectiveSafetyMarginA() + ctrl.MaxExportW = newCfg.Site.MaxExportW // Site-meter swap (operator moved `is_site_meter: true` from one // driver to another, or set it for the first time). Without this // the dispatcher keeps reading the old driver's meter telemetry — diff --git a/go/internal/configreload/apply_fuse_order_test.go b/go/internal/configreload/apply_fuse_order_test.go new file mode 100644 index 000000000..473987c6d --- /dev/null +++ b/go/internal/configreload/apply_fuse_order_test.go @@ -0,0 +1,54 @@ +package configreload + +import ( + "sync" + "testing" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/control" +) + +// The runtime applier in main.go starts with reg.Reload, which can spend +// seconds in driver shutdown, init and startup defaults. Dispatch must already +// use the committed fuse and export limits when the applier starts. +func TestApplyFuseReachesControlBeforeRuntimeApplier(t *testing.T) { + var cfgMu sync.RWMutex + var ctrlMu sync.Mutex + cfg := &config.Config{} + cfg.Fuse.MaxAmps = 25 + cfg.Fuse.Voltage = 230 + cfg.Fuse.Phases = 3 + cfg.Site.MaxExportW = 11000 + + ctrl := control.NewState(0, 50, "meter") + ctrl.SiteFuseAmps = 25 + ctrl.SiteFuseVoltage = 230 + ctrl.SiteFusePhases = 3 + ctrl.MaxExportW = 11000 + + newCfg := *cfg + newCfg.Fuse.MaxAmps = 16 + newCfg.Site.MaxExportW = 3000 + + var cfgAmps, ctrlAmps, ctrlExport float64 + Apply(&cfgMu, cfg, &ctrlMu, ctrl, &newCfg, func(n, o *config.Config) { + // This is where main.go calls reg.Reload. + cfgMu.RLock() + cfgAmps = cfg.Fuse.MaxAmps + cfgMu.RUnlock() + ctrlMu.Lock() + ctrlAmps = ctrl.SiteFuseAmps + ctrlExport = ctrl.MaxExportW + ctrlMu.Unlock() + }) + + if cfgAmps != 16 { + t.Fatalf("cfg fuse at applier start = %v, want 16", cfgAmps) + } + if ctrlAmps != cfgAmps { + t.Errorf("dispatch fuse at applier start = %v A while committed config says %v A", ctrlAmps, cfgAmps) + } + if ctrlExport != 3000 { + t.Errorf("dispatch MaxExportW at applier start = %v, want 3000", ctrlExport) + } +} diff --git a/go/internal/control/battery_full.go b/go/internal/control/battery_full.go index b7c358404..f36dc75de 100644 --- a/go/internal/control/battery_full.go +++ b/go/internal/control/battery_full.go @@ -6,9 +6,14 @@ import ( "github.com/srcfl/ftw/go/internal/telemetry" ) -// A one percentage point gap prevents repeated starts at a rounded 100%. -// This limits charging only; the plan can discharge a full battery at once. -const BatteryChargeResumeSoC = 0.99 +// A one percentage point gap prevents repeated starts at the charge ceiling. +const batteryChargeHysteresisSoC = 0.01 + +// BatteryChargeResumeSoC uses the same ceiling as dispatch. Callers hold the +// control lock, as they do for BatteryChargePaused. +func (s *State) BatteryChargeResumeSoC(driver string) float64 { + return math.Max(0, s.DriverLimits[driver].socCeiling()-batteryChargeHysteresisSoC) +} // BatteryChargePaused reports the dispatcher's full-battery stop. Callers use // the same lock as ComputeDispatch; this state is local to the running device. @@ -17,6 +22,9 @@ func (s *State) BatteryChargePaused(driver string) bool { } func (s *State) updateBatteryChargeFull(store *telemetry.Store, capacities map[string]float64) { + if s == nil || store == nil { + return + } if s.batteryChargeFull == nil { s.batteryChargeFull = map[string]bool{} } @@ -36,9 +44,9 @@ func (s *State) updateBatteryChargeFull(store *telemetry.Store, capacities map[s if age < 0 || age > telemetry.BatterySoCMaxAge || math.IsNaN(soc) || math.IsInf(soc, 0) || soc < 0 || soc > 1 { continue } - if soc == 1 { + if soc >= s.DriverLimits[name].socCeiling() { s.batteryChargeFull[name] = true - } else if soc <= BatteryChargeResumeSoC { + } else if soc <= s.BatteryChargeResumeSoC(name) { delete(s.batteryChargeFull, name) } } diff --git a/go/internal/control/battery_full_test.go b/go/internal/control/battery_full_test.go index 585b1f934..165c59b61 100644 --- a/go/internal/control/battery_full_test.go +++ b/go/internal/control/battery_full_test.go @@ -60,6 +60,7 @@ func TestFullBatteryManualStopDischargeAndSibling(t *testing.T) { } } st.SetBatteryManualHold(BatteryManualHold{PowerW: -2000, ExpiresAt: time.Now().Add(time.Hour)}) + tel.Update("ferroamp", telemetry.DerMeter, -1000, nil, nil) // a new sample allows the reversal got = ComputeDispatch(tel, st, capacities, 13800) for _, target := range got { if target.TargetW >= 0 { diff --git a/go/internal/control/battery_relief_pending_test.go b/go/internal/control/battery_relief_pending_test.go new file mode 100644 index 000000000..dfbce2fe4 --- /dev/null +++ b/go/internal/control/battery_relief_pending_test.go @@ -0,0 +1,156 @@ +package control + +import ( + "math" + "testing" +) + +// Reproductions ported from srcfl/ftw#1816 (segran2) to master's +// capBatteryCharge and clampEVBatteryHeadroom. Master's keepEndingDischarge +// (#1847) holds them, but only while a car charges, and only on import. +// Nothing on master holds the same case with no car, on import or export, +// and these tests do not cover those gaps. +// +// Site: 25 A, three phases, 230 V, 1 A margin. Phases read 22.3/12.3/12.3 A, +// which leaves 1.7 A on L1, or 1,173 W of balanced battery change. +const reliefRoomW = (24 - 22.3) * 690 + +func reliefFixturePhases() []float64 { return []float64{22.3, 12.3, 12.3} } + +// l1AfterA is L1 once the batteries move from what they measure to their +// targets, all else unchanged. +func l1AfterA(out []DispatchTarget, measured map[string]float64) float64 { + a := 22.3 + for _, target := range out { + a += (target.TargetW - measured[target.Driver]) / 690 + } + return a +} + +// Ending a discharge is our raise. The battery measures -2,000 W and our +// newer -1,000 W order may not show yet, so 1,000 W of the raise is already +// on its way. Ending the rest at once spends the same room twice: L1 would +// reach 25.2 A. The battery keeps -827 W and L1 lands on 24 A. +func TestEndingReliefReservesPendingRise(t *testing.T) { + store, state := evSplitFixture(t, 10787, reliefFixturePhases(), 4140, "3p") + sent(state, "ev", 4140, 4140) + sent(state, "first", -2000, -1000) + batteryReading(store, "first", -2000, .5) + out := clampEVBatteryHeadroom([]DispatchTarget{{Driver: "first", TargetW: 0}}, store, state, 17250) + got := targetW(out, "first") + if math.Abs(got-(-2000+reliefRoomW)) > .01 { + t.Fatalf("battery %.1f W, want %.1f W", got, -2000+reliefRoomW) + } + if l1 := l1AfterA(out, map[string]float64{"first": -2000}); l1 > 24.001 { + t.Fatalf("L1 %.3f A, over the 24 A line", l1) + } +} + +// The relief kept never goes past the room the phase lacks, the battery's +// latest order or its command bounds. +func TestEndingReliefPendingRiseBounds(t *testing.T) { + for _, tc := range []struct { + name string + pending, room, last, lowest float64 + want float64 + }{ + {"pending", 1000, 1173, -1000, -5000, -827}, + {"settled", 0, 1173, -1000, -5000, 0}, + {"room for both", 1000, 2500, -1000, -5000, 0}, + {"discharge blocked", 1000, 1173, -1000, 0, 0}, + {"bounds limit the relief", 1000, 1173, -1000, -500, -500}, + {"no relief to keep", 2000, 1173, 0, -5000, 0}, + } { + t.Run(tc.name, func(t *testing.T) { + out := capBatteryCharge([]DispatchTarget{{Driver: "first", TargetW: 0}}, + map[string]float64{"first": tc.last}, map[string]float64{"first": tc.lowest}, + tc.pending, tc.room, tc.room) + if got := out[0].TargetW; math.Abs(got-tc.want) > 1e-6 { + t.Fatalf("battery %.3f W, want %.3f W", got, tc.want) + } + }) + } +} + +// Two batteries end their relief: first at -1,000 W, second at -2,000 W. +// The phase takes 1,173 W of the 3,000 W raise, so 1,827 W must stay. A +// battery that cannot keep its share must not take it from one that can: +// in #1816's old head, first got 609 W it could not keep and second kept +// 1,218 W, so L1 reached 24.88 A. +func TestEndingReliefSharesOnlyAvailableSupply(t *testing.T) { + for _, tc := range []struct { + name string + firstSoC float64 + firstMaxDisW float64 + wantFirst float64 // NaN: only the total and bounds are checked. + wantSecond float64 + }{ + {"first at its SoC floor", .01, 5000, 0, -1827}, + {"first bounded to 300 W", .5, 300, math.NaN(), math.NaN()}, + } { + for _, reverse := range []bool{false, true} { + name := tc.name + if reverse { + name += " reversed" + } + t.Run(name, func(t *testing.T) { + store, state := evSplitFixture(t, 10787, reliefFixturePhases(), 4140, "3p") + state.DriverLimits["first"] = PowerLimits{MaxChargeW: 10000, MaxDischargeW: tc.firstMaxDisW} + sent(state, "ev", 4140, 4140) + sent(state, "first", -1000) + sent(state, "second", -2000) + batteryReading(store, "first", -1000, tc.firstSoC) + batteryReading(store, "second", -2000, .5) + targets := []DispatchTarget{{Driver: "first", TargetW: 0}, {Driver: "second", TargetW: 0}} + if reverse { + targets[0], targets[1] = targets[1], targets[0] + } + out := clampEVBatteryHeadroom(targets, store, state, 17250) + first, second := targetW(out, "first"), targetW(out, "second") + if !math.IsNaN(tc.wantFirst) && (math.Abs(first-tc.wantFirst) > .01 || math.Abs(second-tc.wantSecond) > .01) { + t.Fatalf("first %.1f W, second %.1f W; want %.1f W, %.1f W", first, second, tc.wantFirst, tc.wantSecond) + } + if first > 0 || first < -tc.firstMaxDisW-1e-6 || second > 0 || second < -2000-1e-6 { + t.Fatalf("relief past what each battery can keep: first %.1f W, second %.1f W", first, second) + } + if l1 := l1AfterA(out, map[string]float64{"first": -1000, "second": -2000}); math.Abs(l1-24) > .001 { + t.Fatalf("L1 %.3f A, want 24 A", l1) + } + }) + } + } +} + +// The same sharing at capBatteryCharge, with bounds as lowest orders. +func TestEndingReliefSharingBounds(t *testing.T) { + for _, tc := range []struct { + name string + lastFirst, lowestFirst, lowestSecond, pending float64 + wantFirst, wantSecond float64 + }{ + {"first blocked", -1000, 0, -5000, 0, 0, -1827}, + {"too little supply", -1000, -100, -700, 0, -100, -700}, + {"first had no relief", 0, -5000, -5000, 0, 0, -827}, + {"pending past supply", -1000, 0, -5000, 1000, 0, -2000}, + } { + for _, reverse := range []bool{false, true} { + name := tc.name + if reverse { + name += " reversed" + } + t.Run(name, func(t *testing.T) { + targets := []DispatchTarget{{Driver: "first", TargetW: 0}, {Driver: "second", TargetW: 0}} + if reverse { + targets[0], targets[1] = targets[1], targets[0] + } + out := capBatteryCharge(targets, + map[string]float64{"first": tc.lastFirst, "second": -2000}, + map[string]float64{"first": tc.lowestFirst, "second": tc.lowestSecond}, + tc.pending, reliefRoomW, reliefRoomW) + if first, second := targetW(out, "first"), targetW(out, "second"); math.Abs(first-tc.wantFirst) > 1e-6 || math.Abs(second-tc.wantSecond) > 1e-6 { + t.Fatalf("first %.3f W, second %.3f W; want %.3f W, %.3f W", first, second, tc.wantFirst, tc.wantSecond) + } + }) + } + } +} diff --git a/go/internal/control/battery_rise_phase_test.go b/go/internal/control/battery_rise_phase_test.go new file mode 100644 index 000000000..522880bd9 --- /dev/null +++ b/go/internal/control/battery_rise_phase_test.go @@ -0,0 +1,236 @@ +package control + +import ( + "fmt" + "math" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// C2 of the 27 Oct rig triage: a battery charge rise fits each phase's room, +// (fuseLineA − measured phase − our unshown rises) × V × phases, before it is +// sent. 16 A fuse, 230 V, 3 phases. + +// riseStore holds one meter reading with these phase currents, all import, +// and the battery's reading. +func riseStore(t *testing.T, phaseA [3]float64, batW float64) *telemetry.Store { + t.Helper() + data := map[string]any{} + var totalW float64 + for i, a := range phaseA { + data[fmt.Sprintf("l%d_a", i+1)] = math.Abs(a) + data[fmt.Sprintf("l%d_w", i+1)] = a * 230 + totalW += a * 230 + } + return phaseDirectionStore(t, totalW, batW, data) +} + +// riseState asks the battery for askW of charge (Charge mode at a charge +// limit of askW), so the fuse path alone shapes the order. +func riseState(askW float64) *State { + st := phaseDirectionState(0) + st.Mode = ModeCharge + now := time.Date(2026, 10, 27, 12, 0, 0, 0, time.UTC) + st.clock = func() time.Time { return now } + st.batteryCommands = map[string]commandLog{} + ask(st, askW) + return st +} + +func ask(st *State, w float64) { + st.DriverLimits = map[string]PowerLimits{"bat": {MaxChargeW: w, MaxDischargeW: 10000}} +} + +func batteryOrder(t *testing.T, store *telemetry.Store, st *State) float64 { + t.Helper() + for _, o := range ComputeDispatch(store, st, map[string]float64{"bat": 15200}, 16*230*3) { + if o.Driver == "bat" { + return o.TargetW + } + } + t.Fatal("no battery target") + return 0 +} + +// Seed 207530 at 897 s: L3 reads 16.04 A, over the 16 A line, with the +// battery at 491 W after a cut to 491 W. A plan target of 3350 W is no rise +// room: the order stays at or below the battery's reading. +func TestBatteryRiseIntoPhaseOverTheLine(t *testing.T) { + store := riseStore(t, [3]float64{8.5, 10.5, 16.04}, 491) + st := riseState(3350) + sent(st, "bat", 3349, 491) + if got := batteryOrder(t, store, st); got > 491+1e-6 { + t.Fatalf("order %.0f W with L3 at 16.04 A and the battery at 491 W; want <= 491 W", got) + } +} + +// Seed 207535 at 37-41 s, with no margin. L3 reads 14.27 A with the battery +// at 12 W: 871 W fits under the line and goes out whole. Two seconds later +// the battery reads 871 W, but the meter's reading is from before the +// battery moved and still shows L3 at 14.27 A. The 871 W rise is ours and +// not shown, so a further ask may add only what is left above it: the order +// stays at or below 12 W + (16 − 14.27) × 690 W. +func TestBatteryRiseCountsOwnUnshownRise(t *testing.T) { + phases := [3]float64{13.77, 13.97, 14.27} + limitW := 12 + (16-14.27)*690 + + st := riseState(871) + sent(st, "bat", 12) + if got := batteryOrder(t, riseStore(t, phases, 12), st); math.Abs(got-871) > 1e-6 { + t.Fatalf("first ask: order %.1f W, want the whole 871 W (it fits under %.1f W)", got, limitW) + } + + st = riseState(1500) + sent(st, "bat", 12, 871) + got := batteryOrder(t, riseStore(t, phases, 871), st) + if got > limitW+1e-6 { + t.Fatalf("order %.1f W while the meter does not show our 871 W yet: L3 would reach %.2f A; want <= %.1f W", + got, 14.27+(got-12)/690, limitW) + } + if got < 871-1e-6 { + t.Fatalf("order %.1f W cut below our 871 W hold; a rise check never cuts", got) + } +} + +// Closed loop, battery only, no margin: house loads 13.77/13.97/14.27 A and +// a charge ask that grows 300 W a tick from 600 W to 3000 W, as a PV +// surplus does. The battery reaches each order one tick later; the meter +// shows the battery one tick after the battery's own reading does. L3 +// never passes the 16 A line, the order never moves down (no hunting), and +// it ends at the line: 1193.7 W. +func TestBatteryRiseClosedLoopStopsAtTheLine(t *testing.T) { + house := [3]float64{13.77, 13.97, 14.27} + st := riseState(600) + now := st.now() + st.clock = func() time.Time { return now } + var orders, l3 []float64 + batW, meterBatW := 0.0, 0.0 + for tick := 0; tick < 40; tick++ { + ask(st, math.Min(3000, 600+300*float64(tick))) + var a [3]float64 + for i := range a { + a[i] = house[i] + meterBatW/690 + } + order := batteryOrder(t, riseStore(t, a, batW), st) + orders = append(orders, math.Round(order*10)/10) + l3 = append(l3, math.Round(a[2]*1000)/1000) + meterBatW, batW = batW, order + now = now.Add(2 * time.Second) + } + for i := range orders { + if l3[i] > 16+1e-3 { + t.Errorf("tick %d: L3 %.3f A over the 16 A line", i, l3[i]) + } + if i > 0 && orders[i] < orders[i-1]-0.5 { + t.Errorf("tick %d: order fell %.1f -> %.1f W", i, orders[i-1], orders[i]) + } + } + if want := (16 - 14.27) * 690; math.Abs(orders[len(orders)-1]-want) > 1 { + t.Errorf("final order %.1f W, want %.1f W: L3 at the line", orders[len(orders)-1], want) + } + if t.Failed() { + t.Logf("orders %v", orders) + t.Logf("L3 %v", l3) + } +} + +// Car first: the car's raise from 6 A to 10 A per phase went out 2 s ago +// and the meter does not show it yet: L3 reads 11 A with the car at 6 A. +// The raise takes L3 to 15 A, so the battery's 2000 W charge ask gets the +// 1 A left, 690 W. Once the meter shows the car at 10 A and the battery at +// 690 W, L3 sits at the 16 A line and the battery holds. With the car's demand +// and without it (a raise still on its way after the demand left). +func TestBatteryRiseLeavesRoomForUnshownCarRaise(t *testing.T) { + for _, demand := range []bool{true, false} { + t.Run(fmt.Sprintf("demand=%v", demand), func(t *testing.T) { + st := riseState(2000) + now := st.now() + st.clock = func() time.Time { return now } + sent(st, "bat", 0) + var log commandLog + log = log.keep(sentCommand{at: now.Add(-evSettle - time.Second), w: 4140, a: 6, n: 3}, now.Add(-evSettle)) + log = log.keep(sentCommand{at: now.Add(-2 * time.Second), w: 6900, a: 10, n: 3}, now.Add(-evSettle)) + st.evLimit.offers = map[string]chargerOffers{"car": {log: log, settle: evSettle}} + st.EVCommandedW = 6900 + if demand { + st.evFuseDemands = []evFuseDemand{{"car", 6900, "3p"}} + } + got := batteryOrder(t, riseStore(t, [3]float64{9, 9, 11}, 0), st) + if got > 690+1e-6 { + t.Fatalf("battery %.1f W beside the car's unshown 4 A: L3 would reach %.2f A", got, 11+4+got/690) + } + + now = now.Add(evSettle) + got = batteryOrder(t, riseStore(t, [3]float64{13 + got/690, 13 + got/690, 15 + got/690}, got), st) + if math.Abs(got-690) > 1 { + t.Fatalf("battery %.1f W once the meter shows the car, want 690 W: L3 at the 16 A line", got) + } + }) + } +} + +// A meter that reports current magnitudes and no phase power, as Tibber +// Pulse does (l1_a..l3_a only). L1 reads 15.5 A, L2 and L3 8.7 A, and the +// total exports 1300 W. L2 and L3 alone can carry that export, so L1 may be +// a load importing 15.5 A beside them, and a three-phase charge rise adds to +// it: the order stays at or below (16 − 15.5) × 690 W. When the currents add +// up to the export, every phase must export, and charge may soak it up. +func TestBatteryRiseMagnitudeOnlyMeter(t *testing.T) { + st := riseState(3000) + sent(st, "bat", 0) + store := phaseDirectionStore(t, -1300, 0, map[string]any{"l1_a": 15.5, "l2_a": 8.7, "l3_a": 8.7}) + if got, limit := batteryOrder(t, store, st), (16-15.5)*690; got > limit+1e-6 { + t.Fatalf("order %.0f W: L1 would reach %.2f A if it imports; want <= %.0f W", got, 15.5+got/690, limit) + } + + st = riseState(3000) + sent(st, "bat", 0) + store = phaseDirectionStore(t, -12075, 0, map[string]any{"l1_a": 17.5, "l2_a": 17.5, "l3_a": 17.5}) + if got := batteryOrder(t, store, st); math.Abs(got-3000) > 1e-6 { + t.Fatalf("order %.0f W with every phase exporting 17.5 A; want the whole 3000 W ask", got) + } +} + +// A battery that ramps 50 W a tick toward its order, so it has not reached +// an order older than evSettle, and a meter that shows the battery two ticks +// after the battery reads it. The order counts as unshown until evSettle +// after the battery reads it: L3 never passes the 16 A line and the order +// never falls. +func TestBatteryRiseSlowBatteryLaggingMeter(t *testing.T) { + house := [3]float64{13.77, 13.97, 14.27} + st := riseState(3000) + now := st.now() + st.clock = func() time.Time { return now } + var hist, orders []float64 + batW, maxL3 := 0.0, 0.0 + for tick := 0; tick < 120; tick++ { + meterBatW := 0.0 + if n := len(hist); n >= 2 { + meterBatW = hist[n-2] + } + var a [3]float64 + for i := range a { + a[i] = house[i] + meterBatW/690 + } + order := batteryOrder(t, riseStore(t, a, batW), st) + orders = append(orders, math.Round(order)) + maxL3 = math.Max(maxL3, house[2]+batW/690) + hist = append(hist, batW) + batW = math.Min(order, batW+50) + now = now.Add(2 * time.Second) + } + if maxL3 > 16+1e-3 { + t.Errorf("L3 reached %.3f A, over the 16 A line", maxL3) + } + for i := 1; i < len(orders); i++ { + if orders[i] < orders[i-1]-0.5 { + t.Errorf("tick %d: order fell %.0f -> %.0f W", i, orders[i-1], orders[i]) + break + } + } + if t.Failed() { + t.Logf("orders %v", orders) + } +} diff --git a/go/internal/control/battery_soc_max_test.go b/go/internal/control/battery_soc_max_test.go new file mode 100644 index 000000000..fb7763ab0 --- /dev/null +++ b/go/internal/control/battery_soc_max_test.go @@ -0,0 +1,154 @@ +package control + +import ( + "testing" + "testing/synctest" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +func cappedBatterySite(soc, power float64) (*telemetry.Store, *State, map[string]float64) { + tel, st, capacities := fullBatterySite(soc, power) + st.DriverLimits = map[string]PowerLimits{"battery": {SoCMax: .9, MaxChargeW: 10000}} + return tel, st, capacities +} + +func TestSoCMaxStopsChargingInEveryMode(t *testing.T) { + for _, mode := range AllModes() { + for _, energy := range []bool{false, true} { + t.Run(string(mode)+"/energy="+map[bool]string{false: "off", true: "on"}[energy], func(t *testing.T) { + tel, st, capacities := cappedBatterySite(.9, 2000) + st.Mode, st.UseEnergyDispatch = mode, energy + st.SlewEnabled, st.SlewRateW = true, 10 + st.SlotDirective = func(now time.Time) (SlotDirective, bool) { + return SlotDirective{SlotStart: now, SlotEnd: now.Add(15 * time.Minute), BatteryEnergyWh: 500}, true + } + st.PlanTarget = func(time.Time) (string, float64, string, bool) { + return string(ModeCharge), 2000, "charge", true + } + got := ComputeDispatch(tel, st, capacities, 13800) + if len(got) != 1 || got[0].TargetW > 0 || !st.BatteryChargePaused("battery") { + t.Fatalf("90%% ceiling did not stop charging: %+v", got) + } + }) + } + } +} + +func TestSoCMaxManualHoldAndSibling(t *testing.T) { + tel, st, capacities := cappedBatterySite(.9, 0) + tel.Update("other", telemetry.DerBattery, 0, ptrF64(.8), nil) + tel.RecordDriverSuccess("other") + capacities["other"] = 9600 + st.DriverLimits["other"] = PowerLimits{SoCMax: .95} + st.SlewEnabled = false + st.SetBatteryManualHold(BatteryManualHold{PowerW: 3000, ExpiresAt: time.Now().Add(time.Hour)}) + got := ComputeDispatch(tel, st, capacities, 13800) + if targetFor(t, got, "battery").TargetW != 0 || targetFor(t, got, "other").TargetW < 2900 { + t.Fatalf("charge must go to the battery below its own ceiling: %+v", got) + } + st.SetBatteryManualHold(BatteryManualHold{PowerW: -2000, ExpiresAt: time.Now().Add(time.Hour)}) + tel.Update("ferroamp", telemetry.DerMeter, -1000, nil, nil) // a new sample allows the reversal + got = ComputeDispatch(tel, st, capacities, 13800) + if targetFor(t, got, "battery").TargetW >= 0 { + t.Fatalf("charge ceiling blocked discharge: %+v", got) + } + st.BatteryHoldTargetValid = func(driver, id string) bool { return driver == "battery" && id == "hardware" } + st.SetBatteryManualHold(BatteryManualHold{Driver: "battery", DeviceID: "hardware", PowerW: 3000, ExpiresAt: time.Now().Add(time.Hour)}) + got = ComputeDispatch(tel, st, capacities, 13800) + if targetFor(t, got, "battery").TargetW != 0 || targetFor(t, got, "other").TargetW != 0 { + t.Fatalf("scoped hold bypassed its ceiling or charged another battery: %+v", got) + } +} + +func TestSoCMaxStopsSelfTuneWithoutPriorControlTick(t *testing.T) { + tel, st, capacities := cappedBatterySite(.9, 1000) + got := SelfTuneDispatch(tel, st, capacities, 13800, "battery", 3000) + if len(got) != 1 || got[0].TargetW != 0 || !got[0].Clamped { + t.Fatalf("self-tune crossed the charge ceiling: %+v", got) + } +} + +func TestSoCMaxStopsPreviousOrderDuringQuietOrHeldTick(t *testing.T) { + for _, holdoff := range []bool{false, true} { + tel, st, capacities := cappedBatterySite(.9, 0) + st.Mode = ModeSelfConsumption + tel.Update("ferroamp", telemetry.DerMeter, 0, nil, nil) + st.PrevTargets["battery"] = 10000 + st.LastTargets = []DispatchTarget{{Driver: "battery", TargetW: 10000}} + if holdoff { + now := time.Now() + st.LastDispatch, st.MinDispatchIntervalS = &now, 60 + } + got := ComputeDispatch(tel, st, capacities, 13800) + if len(got) != 1 || got[0].TargetW != 0 || st.LastTargets[0].TargetW != 0 || st.PrevTargets["battery"] != 0 { + t.Fatalf("holdoff=%v left the prior charge active: %+v", holdoff, got) + } + } +} + +func TestSoCMaxHysteresisAndMissingLevel(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + tel, st, capacities := cappedBatterySite(.899, 0) + check := func(paused bool) { + t.Helper() + got := ComputeDispatch(tel, st, capacities, 13800) + if len(got) != 1 || st.BatteryChargePaused("battery") != paused || (got[0].TargetW == 0) != paused { + t.Fatalf("paused=%v targets=%+v, want paused=%v", st.BatteryChargePaused("battery"), got, paused) + } + } + check(false) + for _, level := range []float64{.9, .92, .899, .891} { + tel.Update("battery", telemetry.DerBattery, 0, &level, nil) + check(true) + } + time.Sleep(6 * time.Minute) + tel.Update("battery", telemetry.DerBattery, 0, nil, nil) + tel.RecordDriverSuccess("battery") + check(true) + tel.Update("battery", telemetry.DerBattery, 0, ptrF64(.89), nil) + check(false) + }) +} + +func TestSoCMaxReloadTakesEffectOnNextTick(t *testing.T) { + tel, st, capacities := cappedBatterySite(.9, 0) + ComputeDispatch(tel, st, capacities, 13800) + st.DriverLimits["battery"] = PowerLimits{SoCMax: .95} + got := ComputeDispatch(tel, st, capacities, 13800) + if len(got) != 1 || got[0].TargetW <= 0 { + t.Fatalf("raising the ceiling did not release the stop: %+v", got) + } + st.DriverLimits["battery"] = PowerLimits{SoCMax: .85} + got = ComputeDispatch(tel, st, capacities, 13800) + if len(got) != 1 || got[0].TargetW != 0 { + t.Fatalf("lowering the ceiling did not stop charging: %+v", got) + } +} + +func TestSoCMaxStillAllowsFuseRelief(t *testing.T) { + tel, st, capacities := setupFuseSaver(16000, 0, .9, 10000) + st.DriverLimits["bat"] = PowerLimits{SoCMax: .9, MaxChargeW: 10000, MaxDischargeW: 10000} + st.Mode = ModeIdle + got := ComputeDispatch(tel, st, capacities, 13800) + if len(got) != 1 || got[0].TargetW >= 0 { + t.Fatalf("charge ceiling blocked fuse relief: %+v", got) + } +} + +func TestSoCMaxExportGuardCannotRestorePausedCharge(t *testing.T) { + tel, st, capacities := cappedBatterySite(.9, 2000) + st.Mode = ModeIdle + st.SiteFuseAmps, st.SiteFuseVoltage, st.SiteFusePhases = 20, 230, 3 + st.batteryCommands = map[string]commandLog{"battery": commandLog{}.add(st.now(), 2000)} + st.updateBatteryChargeFull(tel, capacities) + // The stop remains active below the ceiling until the resume level. Even + // an export overage must not restore the charge that the stop removed. + tel.Update("battery", telemetry.DerBattery, 2000, ptrF64(.895), nil) + tel.Update("ferroamp", telemetry.DerMeter, -20000, nil, nil) + got := ComputeDispatch(tel, st, capacities, 13800) + if len(got) != 1 || got[0].TargetW != 0 { + t.Fatalf("export guard restored paused charging: %+v", got) + } +} diff --git a/go/internal/control/control_test.go b/go/internal/control/control_test.go index 44ab142ed..70bf04a0c 100644 --- a/go/internal/control/control_test.go +++ b/go/internal/control/control_test.go @@ -143,6 +143,7 @@ func TestDispatchWrongDirectionWindupDecaysPIIntegral(t *testing.T) { // every subsequent cycle (that's the 2026-05-25 sunrise regression). var lastTarget float64 for i := 0; i < 5; i++ { + store.Update("ferroamp", telemetry.DerMeter, -1000, nil, nil) // the PI acts once per new sample targets := ComputeDispatch(store, st, caps(map[string]float64{"ferroamp": 15200}), 11040) if len(targets) == 1 { lastTarget = targets[0].TargetW @@ -371,8 +372,10 @@ func TestPlannerSelfIdleGateLeavesExistingDischargeAloneInsideDeadband(t *testin Strategy: "self_consumption", } // Grid balanced — battery is covering load fine. Deadband → no - // dispatch issued, but the running battery state is preserved. - store := seedStore(0, []struct { + // dispatch issued, but the running battery state is preserved. A + // battery that may not export holds the band above 0 W, so the + // balanced grid imports a little. + store := seedStore(20, []struct { name string currentW, soc float64 }{ @@ -4062,12 +4065,23 @@ func TestPlannerSelfIdleGateLeavesSurplusOnlyEVReserve(t *testing.T) { st.MinDispatchIntervalS = 0 st.SlotDirective = func(time.Time) (SlotDirective, bool) { return dir, true } - targets := ComputeDispatch(store, st, caps(map[string]float64{"ferroamp": 15200}), 11040) - if len(targets) != 1 { - t.Fatalf("want 1 target, got %d", len(targets)) + // The PI may take a few ticks to get there; it must never pass it. + var last float64 + for i := 0; i < 10; i++ { + targets := ComputeDispatch(store, st, caps(map[string]float64{"ferroamp": 15200}), 11040) + if len(targets) != 1 { + t.Fatalf("tick %d: want 1 target, got %d", i, len(targets)) + } + last = targets[0].TargetW + if last > 1000+1 { + t.Fatalf("tick %d: TargetW = %f — battery took the surplus-only EV reserve", i, last) + } + store.Update("ferroamp", telemetry.DerBattery, last, ptrF64(0.5), nil) + store.Update("ferroamp", telemetry.DerMeter, -3000+last, nil, nil) + store.DriverHealthMut("ferroamp").RecordSuccess() } - if math.Abs(targets[0].TargetW-1000) > 1 { - t.Errorf("TargetW = %f — idle gate must leave surplus-only EV reserve; want ~1000", targets[0].TargetW) + if math.Abs(last-1000) > 1 { + t.Errorf("TargetW = %f — idle gate must leave surplus-only EV reserve; want ~1000", last) } } @@ -5496,20 +5510,19 @@ func TestEnergyDispatchClampsDischargeWhenEVActuallyCharging(t *testing.T) { // charge battery, EV is on surplus_only at 2.5 kW with 11 kW max, // and 3 kW is currently exporting at the real meter. // -// dispatch.go:685 subtracts EVChargingW from gridW before the home- -// battery decision so the battery sees "what export would exist -// without the EV", which is 3000 + 2500 = 5500 W in this scenario. -// The reserve cap then carves out reserveRemaining for the EV to -// step up into. +// The car comes before battery charge, so the battery sees the +// export the car leaves: the real 3000 W, not the 5500 W there would +// be without the car. The reserve cap then carves out +// reserveRemaining for the EV to step up into. // // Pre-fix (reserve = full MaxChargeW = 11000): reserveRemaining -// = 11000 - 2500 = 8500. ceiling = 5500 - 8500 = -3000 → 0. Battery +// = 11000 - 2500 = 8500. ceiling = 3000 - 8500 → 0. Battery // idles; surplus dumps to grid. // // Post-fix (reserve = CurrentPowerW + EVRampHeadroomW = 4500): -// reserveRemaining = 4500 - 2500 = 2000. ceiling = 5500 - 2000 = -// 3500. Battery picks up the bulk of the surplus while leaving 2 kW -// of headroom for the EV to ramp up. +// reserveRemaining = 4500 - 2500 = 2000. ceiling = 3000 - 2000 = +// 1000. Battery takes the surplus beyond 2 kW of headroom for the +// EV to ramp up, and none of the car's own 2.5 kW. // // The test sets EVSurplusOnlyReserveW directly to the value // loadpoint.SurplusReserveW would produce; the helper itself is @@ -5541,12 +5554,12 @@ func TestEnergyDispatchAbsorbsSurplusBeyondEVReserveActualDraw(t *testing.T) { t.Fatalf("want 1 target, got %d", len(targets)) } got := targets[0].TargetW - // Want ≈ 3500 W. Allow ±400 W slack for time drift / formula edge. - if got < 3100 { - t.Errorf("TargetW = %.0f W — battery should absorb the surplus that the EV's adaptive reserve no longer hoards (want ≈ 3500 W, was 0 pre-fix)", got) + // Want ≈ 1000 W. Allow ±100 W slack for time drift / formula edge. + if got < 900 { + t.Errorf("TargetW = %.0f W — battery should absorb the surplus that the EV's adaptive reserve no longer hoards (want ≈ 1000 W, was 0 pre-fix)", got) } - if got > 3900 { - t.Errorf("TargetW = %.0f W — battery overshoot, must leave EVRampHeadroomW (2 kW) of headroom for EV ramp-up (want ≈ 3500 W)", got) + if got > 1100 { + t.Errorf("TargetW = %.0f W — battery overshoot, must leave the car's draw and EVRampHeadroomW (2 kW) of headroom for EV ramp-up (want ≈ 1000 W)", got) } } diff --git a/go/internal/control/dispatch.go b/go/internal/control/dispatch.go index 388e73afc..37a3899e5 100644 --- a/go/internal/control/dispatch.go +++ b/go/internal/control/dispatch.go @@ -5,7 +5,9 @@ import ( "encoding/json" "fmt" "log/slog" + "maps" "math" + "slices" "sort" "time" @@ -79,6 +81,14 @@ func (m Mode) IsPlannerMode() bool { m == ModePlannerArbitrage } +// batteryMayNotExport reports the planner modes whose battery never sells to +// the grid. Only planner_arbitrage may. +func (m Mode) batteryMayNotExport() bool { + return m == ModePlannerSelf || + m == ModePlannerCheap || + m == ModePlannerPassiveArbitrage +} + // PlanTargetFunc returns the current slot as mode, grid target W, // decision id, and ok. ok false means the plan is missing. The decision // id is report metadata. New callers use SlotDirectiveFunc. @@ -154,6 +164,10 @@ const MaxCommandW = 5000 // elsewhere in this package for sign decisions. const evActiveThresholdW = 100.0 +// holdoffToleranceS is how much earlier than MinDispatchIntervalS a +// dispatch may run. It absorbs jitter in the work before each tick. +const holdoffToleranceS = 0.25 + type vehiclePowerFlow struct { Live bool EVChargeW float64 @@ -227,6 +241,28 @@ type PowerLimits struct { MaxDischargeW float64 MaxChargeWSet bool MaxDischargeWSet bool + // SoCMax is the battery's charge ceiling, 0..1. An unset limit uses 100%. + SoCMax float64 + SoCMaxSet bool + // SoCMin is the owner's discharge floor, 0..1: the higher of the + // battery's and the planner's soc_min. Core never goes below 5 %. + SoCMin float64 +} + +// socFloor is the lowest SoC that planner modes, fuse relief and the hourly +// import limit may discharge to. +func (l PowerLimits) socFloor() float64 { + if l.SoCMin > dischargeFloorSoC && l.SoCMin <= 1 { + return l.SoCMin + } + return dischargeFloorSoC +} + +func (l PowerLimits) socCeiling() float64 { + if (l.SoCMaxSet || l.SoCMax > 0) && l.SoCMax >= 0 && l.SoCMax <= 1 { + return l.SoCMax + } + return 1 } func effectivePowerLimit(value float64, set bool) float64 { @@ -311,9 +347,10 @@ type State struct { SiteFuseVoltage float64 SiteFusePhases int - // SiteFuseSafetyA is amps held back from the breaker. Without it the - // inverter's own limiter trips first, output falls to 0, and dispatch - // ramps from idle. Zero disables the margin. + // SiteFuseSafetyA is an opt-in margin in amps per phase below the + // breaker; zero, the default, works to the rating. Set it for an + // inverter whose own limiter trips at the rating: output falls to 0 + // and dispatch ramps from idle. SiteFuseSafetyA float64 // Peak limit — enforced only in PeakShaving mode @@ -356,7 +393,8 @@ type State struct { EVCurtailHeadroomW float64 // BatteryCoversEV lets the battery discharge into the EV. Off, EV // load is taken out of the meter before the PI so the battery does - // not feed the car. + // not feed the car. Either way the car comes before battery charge: + // the battery charges only from the surplus the car leaves. BatteryCoversEV bool // BatteryBoostEVChargingW is the live charging draw belonging to @@ -381,16 +419,35 @@ type State struct { evLimit evMeterLimit // fuseHeat is the main fuse's heat per phase, from site readings only. fuseHeat fuseHeat - // evOffers and batteryCommands are our own recent commands, the only - // device input the EV hard limit and the EV/battery split use. - evOffers commandLog + // batteryCommands and evLimit.offers are our own recent commands, the + // only device input the EV hard limit and the EV/battery split use. batteryCommands map[string]commandLog + // batteryReadings is each battery's own reading at our last commands, + // for the power the site meter may not show yet (unshownBatteryW). + batteryReadings map[string]commandLog + // batteryFuseRelief is the discharge the fuse guard added to each + // battery's last dispatched command (forceFuseDischarge). It ends once + // the site fits without it, so an EV rise or start never counts it as + // room (evFuseReliefW). + batteryFuseRelief map[string]float64 + // batteriesOfflineSince is, per battery whose relief still counts, + // when dispatch first found it offline (expireOfflineFuseRelief). + batteriesOfflineSince map[string]time.Time + // phaseResponse holds the readings the batteries' phase relief checks + // the meter against (phaseUnresponsive). + phaseResponse phaseResponse // PI controller (outer, site-level) PI *PIController // meterClampLogged records that the live-meter clamp was already // reported, so it is logged when it engages rather than every tick. meterClampLogged bool + // dispatchMeterAt is the UpdatedAt of the last site-meter sample the + // battery dispatch acted on. See siteMeterHeld. + dispatchMeterAt time.Time + // meterFreeze finds a site meter that repeats one reading while FTW's + // orders move. See ObserveSiteMeterFreeze. + meterFreeze meterFreeze // Slew + holdoff SlewRateW float64 @@ -400,6 +457,7 @@ type State struct { // the operator-facing rationale. SlewEnabled bool MinDispatchIntervalS int + ControlIntervalS int // tick period; no holdoff when MinDispatchIntervalS <= it LastDispatch *time.Time PrevTargets map[string]float64 batteryChargeFull map[string]bool @@ -1073,11 +1131,15 @@ func (s *State) SetPeakLimit(w float64) error { return fmt.Errorf("peak_limit_w must be ≥ 0, got %.0f W", w) } if ceiling := s.peakLimitCeilingW(); ceiling > 0 && w > ceiling { + margin := "" + if s.SiteFuseSafetyA > 0 { + margin = fmt.Sprintf(", less a %.1f A safety margin", s.SiteFuseSafetyA) + } return fmt.Errorf( "peak_limit_w %.0f W is above the site's import ceiling %.0f W "+ - "(fuse %.0f A × %.0f V × %d phases, less a %.1f A safety margin) "+ + "(fuse %.0f A × %.0f V × %d phases%s) "+ "— a peak limit above the fuse can never bind", - w, ceiling, s.SiteFuseAmps, s.SiteFuseVoltage, s.SiteFusePhases, s.SiteFuseSafetyA) + w, ceiling, s.SiteFuseAmps, s.SiteFuseVoltage, s.SiteFusePhases, margin) } s.PeakLimitW = w return nil @@ -1144,6 +1206,10 @@ type batteryInfo struct { dischargeBlocked bool chargeBlocked bool socMissing bool // no fresh SoC reading; soc is 0 + // belowOwnerFloor: a planner mode, with or without a plan, and SoC at + // or under the owner's floor. Unlike a driver's transient report, this + // must also stop a discharge already running, past the deadband and slew. + belowOwnerFloor bool } // chargeCap returns the effective per-battery charge ceiling, falling @@ -1416,9 +1482,12 @@ func ComputeDispatch( // Manual holds bypass the holdoff: the operator just installed a // setpoint and expects immediate effect, not a 5 s wait. The fuse // guard at the end of the cycle still protects the site. - if !manualHoldActive && state.LastDispatch != nil { + // The ticker is fixed, but the work before each dispatch varies, so + // two ticks can land just under the interval apart. The 0.25 s + // tolerance keeps that jitter from skipping every other tick. + if !manualHoldActive && state.LastDispatch != nil && state.MinDispatchIntervalS > state.ControlIntervalS { elapsed := state.now().Sub(*state.LastDispatch).Seconds() - if elapsed < float64(state.MinDispatchIntervalS) { + if elapsed < float64(state.MinDispatchIntervalS)-holdoffToleranceS { // Holdoff suppresses normal re-dispatch, but the // fuse-saver overrides — an overflow can't wait 5 s for // the next eligible tick. @@ -1438,6 +1507,9 @@ func ComputeDispatch( if siteMeter != nil { rawGridW = siteMeter.SmoothedW } + meterHeld := state.siteMeterHeld(siteMeter) + // A repeat that ignores FTW's own move is no new sample either. + meterHeld = meterHeld || state.siteMeterRepeatsAfterMove(siteMeter) // Live EV/V2X charger readings override the manual slider on each tick — // hardware truth beats guesses. Positive V2X is a vehicle charging load; // negative V2X is kept separate so the default controller does not @@ -1460,14 +1532,20 @@ func ComputeDispatch( // discharge keeps its old opt-in guard so Self does not fill the home // battery from a car by default. gridW := rawGridW - if state.Mode != ModeSelfConsumption && !state.BatteryCoversEV { - gridW -= state.uncoveredEVChargingW() - } if !state.BatteryCoversEV { gridW -= vehicleFlow.V2XDischargeW } + // The car comes before battery charge: charge comes only from what the + // meter shows left after the car. The car leaves the signal below so + // the battery never discharges for it. + chargeGridW := gridW + if state.Mode != ModeSelfConsumption && !state.BatteryCoversEV { + gridW -= state.uncoveredEVChargingW() + } // ---- Gather online batteries ---- + // A planner mode keeps the owner's reserve, with or without a plan. + ownerFloor := !manualHoldActive && state.Mode.IsPlannerMode() batteries := make([]batteryInfo, 0, len(driverCapacities)) for name, cap := range driverCapacities { r := store.Get(name, telemetry.DerBattery) @@ -1489,7 +1567,8 @@ func ComputeDispatch( // A configured zero is a hard direction block, not the legacy // "use MaxCommandW" sentinel. Feed it into the allocator as well as // the clamps so capable siblings receive the blocked battery's share. - dischargeBlocked = dischargeBlocked || lim.dischargeCap() == 0 + belowOwnerFloor := ownerFloor && soc <= lim.socFloor() + dischargeBlocked = dischargeBlocked || lim.dischargeCap() == 0 || belowOwnerFloor chargeBlocked = chargeBlocked || lim.chargeCap() == 0 || state.BatteryChargePaused(name) batteries = append(batteries, batteryInfo{ driver: name, @@ -1505,6 +1584,7 @@ func ComputeDispatch( dischargeBlocked: dischargeBlocked, chargeBlocked: chargeBlocked, socMissing: socMissing, + belowOwnerFloor: belowOwnerFloor, }) } onlineBats := make([]batteryInfo, 0, len(batteries)) @@ -1513,6 +1593,7 @@ func ComputeDispatch( onlineBats = append(onlineBats, b) } } + state.expireOfflineFuseRelief(onlineBats) if len(onlineBats) == 0 { publishBatterylessEVFuseBudget(store, state, fuseMaxW) state.LastTargets = nil @@ -1538,6 +1619,14 @@ func ComputeDispatch( for _, b := range onlineBats { currentTotal += b.currentW } + // Without the car the meter may show export the car is using. Where + // the battery would charge, it sees the meter with the car instead: + // it charges only from the surplus after the car, and never turns to + // discharge for it. + noCarGridW := gridW + if gridW < chargeGridW { + gridW = math.Max(gridW, math.Min(chargeGridW, currentTotal)) + } surplus := newSurplusAccounting(rawGridW, gridW, currentTotal, state) // passive_arbitrage idle slot + live PV surplus: don't absorb. Forecasts @@ -1720,15 +1809,19 @@ func ComputeDispatch( // Leave PV for a surplus-only EV. A planned import of at least // 100 W keeps its charge. Discharge is left alone. - if state.EVSurplusOnlyReserveW > 0 && targetTotalW > 0 { - deliberateGridCharge := currentDirective.HasPlannedGridW && currentDirective.PlannedGridW >= 100 - if !deliberateGridCharge { - ceiling := surplus.chargeCeilingAfterEVReserveW() - if targetTotalW > ceiling { - targetTotalW = ceiling - } + deliberateGridCharge := currentDirective.HasPlannedGridW && currentDirective.PlannedGridW >= 100 + if state.EVSurplusOnlyReserveW > 0 && targetTotalW > 0 && !deliberateGridCharge { + ceiling := surplus.chargeCeilingAfterEVReserveW() + if targetTotalW > ceiling { + targetTotalW = ceiling } } + // The car before battery charge: a charge the plan makes from PV + // takes only the surplus the car leaves. + pvCharge := currentDirective.HasPlannedGridW && !deliberateGridCharge + if targetTotalW > 0 && pvCharge && gridW > noCarGridW { + targetTotalW = math.Min(targetTotalW, currentTotal-gridW) + } // Charge slots only. If PV drops, the energy budget would hold the // planned charge on grid import until the replan, about 10 minutes @@ -1736,7 +1829,11 @@ func ComputeDispatch( // Below 100 W is meter noise. if currentDirective.HasPlannedGridW && targetTotalW > 0 { const planGridDeadband = 100.0 - projectedGridW := gridW + (targetTotalW - currentTotal) + // As before the car came first, the projection leaves out the + // car the battery does not cover, while the plan's grid counts + // the car it planned. The car's draw never starts this + // back-off. + projectedGridW := noCarGridW + (targetTotalW - currentTotal) gridErr := projectedGridW - currentDirective.PlannedGridW if gridErr > planGridDeadband { adjusted := targetTotalW - gridErr @@ -1757,6 +1854,26 @@ func ComputeDispatch( // already handled above) // - planner_cheap / planner_arbitrage when UseEnergyDispatch=false + // A battery doing what a limit forbids. The deadband exit below + // and the held-meter exit both fall through to cut it. + batteryBreaksLimit := (noSelfDischarge && anyBatteryDischarging(onlineBats)) || + anyExplicitZeroDischargeViolation(onlineBats) || + anyBelowOwnerFloorDischarging(onlineBats, state.PrevTargets) || + anyAtHardFloorDischarging(onlineBats, state.PrevTargets) || + anyBlockedBatteryCharging(onlineBats, noSelfCharge) + + // The PI acts on the meter alone. A held sample carries no new + // error, so hold the targets and do not integrate it again. When + // a battery breaks a limit, skip the PI with no correction; the + // clamps, holdTowardZero and the safety pipeline then cut it. + if meterHeld { + if !batteryBreaksLimit { + return fuseSaverEarlyExit(store, state, driverCapacities, fuseMaxW) + } + totalCorrection = 0 + break + } + // Surplus-only EV reserve (legacy/reactive path): when an EV // under a surplus_only loadpoint is connected, bias the grid // signal so the PI leaves `reserveRemaining` of export untouched @@ -1802,6 +1919,18 @@ func ComputeDispatch( // import/export energy is observed separately by the API; accumulated // import must never lower this target and cause battery-fed export. activeGridTargetW := state.GridTargetW + // A battery that may not export holds the whole dead band at or + // above 0 W when the plain target asks it to discharge. Aimed at + // 0 W, each drop in house load sold battery energy until the next + // correction, about as often as the site imported, and a negative + // planned grid had the battery export outright. A little import + // costs less: that energy stays in the battery for later. A + // discharge inside the dead band, such as a standby reading, keeps + // the plain target. + if state.Mode.batteryMayNotExport() && + currentTotal-(biasedGridW-activeGridTargetW) < -state.GridToleranceW { + activeGridTargetW = math.Max(activeGridTargetW, state.GridToleranceW) + } var errW float64 switch effectiveMode { @@ -1812,6 +1941,12 @@ func ComputeDispatch( errW = gridW - state.PeakLimitW } else if gridW < 0 { errW = gridW // exporting → charge with surplus + } else if chargeGridW > 0 && currentTotal > 0 && state.EVChargingW > evActiveThresholdW { + // The car before battery charge: the site imports while + // the car and the battery both charge. Take the charge + // down by the import, to 0 at most; under the peak limit + // the battery does not discharge. + errW = math.Min(chargeGridW, currentTotal) } else { errW = 0 } @@ -1845,10 +1980,7 @@ func ComputeDispatch( // full pipeline ends in the same forceFuseDischarge the // early exit's fuse-saver would have run. surplusActive := state.EVSurplusOnlyReserveW > 0 && effectiveMode == ModeSelfConsumption - if !surplusActive && math.Abs(errW) < state.GridToleranceW && - !(noSelfDischarge && anyBatteryDischarging(onlineBats)) && - !anyExplicitZeroDischargeViolation(onlineBats) && - !anyBlockedBatteryCharging(onlineBats, noSelfCharge) { + if !surplusActive && math.Abs(errW) < state.GridToleranceW && !batteryBreaksLimit { // A small grid error is not a statement that the site is // safe. errW compares one aggregate number against one // aggregate target; two protections bind on numbers this @@ -1945,6 +2077,11 @@ func ComputeDispatch( "mode", string(effectiveMode)) } totalCorrection = allowed - currentTotal + // The house alone has surplus, so a discharge would feed the + // car. Stop it now rather than over several PI steps. + if gridW > noCarGridW && currentTotal+totalCorrection < 0 { + totalCorrection = -currentTotal + } // Surplus-only EV reserve cap: stack on top of the meter clamp. // Mirror the energy-path cap so the battery target on the @@ -2218,6 +2355,9 @@ func ComputeDispatch( // per-driver cap (DriverLimits, falling back to MaxCommandW) so we // never issue a command outside safe bounds. raw = clampTargetsToPowerLimits(raw, state.DriverLimits) + if meterHeld { + raw = holdTowardZero(raw, state.PrevTargets) + } // ---- Fuse guard (bidirectional, #145) ---- return applyDispatchSafetyPipeline(raw, store, state, driverCapacities, fuseMaxW, dispatchSafetyOptions{ @@ -2225,6 +2365,7 @@ func ComputeDispatch( noSelfDischarge: noSelfDischarge, noSelfCharge: noSelfCharge, chargeBlocked: chargeBlockedDrivers(onlineBats), + floorBlocked: belowOwnerFloorDrivers(onlineBats), updatePrevTargets: true, recordDispatch: true, }) @@ -2319,11 +2460,81 @@ func chargeBlockedDrivers(bats []batteryInfo) map[string]bool { return out } +// belowOwnerFloorDrivers is the set of online batteries at or under the +// owner SoC floor. Slew anchors on measured power and would hand back part +// of a discharge the allocator already withdrew. +func belowOwnerFloorDrivers(bats []batteryInfo) map[string]bool { + var out map[string]bool + for _, b := range bats { + if !b.belowOwnerFloor { + continue + } + if out == nil { + out = make(map[string]bool, len(bats)) + } + out[b.driver] = true + } + return out +} + +// floorStandbyW is the largest draw a pack parked at its floor may read +// without counting as a discharge. Packs report tens of watts of standby +// draw there; treating that as a discharge sent an order every tick. +const floorStandbyW = 50.0 + +// dischargingAtFloor reports a real discharge on a pack at its floor: past +// standby draw, or any measured discharge after FTW's own last order to it +// was a discharge, which the driver would otherwise hold. +func dischargingAtFloor(b batteryInfo, prev map[string]float64) bool { + return b.currentW < -floorStandbyW || b.currentW < -1 && prev[b.driver] < 0 +} + +// anyBelowOwnerFloorDischarging reports a battery at or under the owner +// floor that is discharging. The deadband exit returns no targets, and a +// driver keeps its last setpoint, so such a battery would drain on. +func anyBelowOwnerFloorDischarging(bats []batteryInfo, prev map[string]float64) bool { + for _, b := range bats { + if b.belowOwnerFloor && dischargingAtFloor(b, prev) { + return true + } + } + return false +} + +// anyAtHardFloorDischarging reports a battery at or under Core's 5 % floor +// that is discharging, in any mode. The deadband exit would leave the +// driver on its last discharge setpoint. +func anyAtHardFloorDischarging(bats []batteryInfo, prev map[string]float64) bool { + for _, b := range bats { + if !b.socMissing && b.soc <= dischargeFloorSoC && dischargingAtFloor(b, prev) { + return true + } + } + return false +} + +// floorBelowOwnerFloorDischarge moves a discharge target to 0 W for each +// battery at or under the owner floor. Fuse relief runs later and stops at +// the same floor, so it cannot reopen the discharge. +func floorBelowOwnerFloorDischarge(targets []DispatchTarget, blocked map[string]bool) []DispatchTarget { + if len(blocked) == 0 { + return targets + } + for i := range targets { + if targets[i].TargetW < 0 && blocked[targets[i].Driver] { + targets[i].TargetW = 0 + targets[i].Clamped = true + } + } + return targets +} + type dispatchSafetyOptions struct { manualHoldActive bool noSelfDischarge bool noSelfCharge bool chargeBlocked map[string]bool + floorBlocked map[string]bool updatePrevTargets bool recordDispatch bool } @@ -2336,8 +2547,12 @@ func applyDispatchSafetyPipeline( fuseMaxW float64, opts dispatchSafetyOptions, ) []DispatchTarget { + if state != nil { + state.updateBatteryChargeFull(store, driverCapacities) + } // ---- Fuse guard (bidirectional, #145) ---- targets = applyFuseGuard(targets, store, state, fuseMaxW) + targets = fitBatteryRisesToFuse(targets, store, state) // ---- Plan/exec sign-mismatch floor (energy planner modes only) ---- // Operator-report 2026-04-28 (08:00-08:15 CEST): planner_arbitrage @@ -2360,16 +2575,20 @@ func applyDispatchSafetyPipeline( targets = floorNegativeTargets(targets) } targets = floorBlockedCharge(targets, opts.noSelfCharge, opts.chargeBlocked) + targets = floorBelowOwnerFloorDischarge(targets, opts.floorBlocked) + targets = floorBlockedDischarge(targets, store, state, driverCapacities, fuseMaxW) // A battery-boost lease may never draw the stationary fleet below its // explicit reserve. Apply this immediately before forceFuseDischarge: // normal planning/operator intent is constrained, while the physical fuse // emergency remains superior and may still discharge to prevent a trip. + // The hour's import limit is not a fuse and keeps the reserve. targets = applyBatteryBoostReserve(targets, store, state, driverCapacities) // forceFuseDischarge runs after the normal policy rails. A fuse overflow can // demand a battery target far beyond what slew would allow in one tick; // slew-limiting that response would leave the fuse violated for multiple ticks. + beforeRelief := append([]DispatchTarget(nil), targets...) targets = forceFuseDischarge(targets, store, state, driverCapacities, fuseMaxW) // Keep the hardware direction contract last. forceFuseDischarge already // allocates within the discharge cap, but this final check also covers any @@ -2378,13 +2597,49 @@ func applyDispatchSafetyPipeline( targets = clampTargetsToPowerLimits(targets, state.DriverLimits) } targets = floorMissingSoCDischarge(targets, store) - targets = state.floorFullBatteryCharge(targets) targets = clampEVBatteryHeadroom(targets, store, state, fuseMaxW) + targets = state.applyEVBatteryYield(targets) + targets = state.floorFullBatteryCharge(targets) republishFuseEVCapAfterFuseDischarge(targets, store, state, fuseMaxW) recordDispatchTargets(targets, state, opts.updatePrevTargets, opts.recordDispatch) + if opts.recordDispatch { + state.recordFuseRelief(beforeRelief, targets) + state.recordBatteryReadings(targets, store) + } return targets } +// recordFuseRelief keeps, per battery, the discharge in the final command +// that the fuse guard added: below both the command before forceFuseDischarge +// and 0 W. Planner or operator discharge is not relief. An offline battery +// gets no command, so its relief carries over until +// expireOfflineFuseRelief ends it. +func (s *State) recordFuseRelief(before, after []DispatchTarget) { + if s == nil { + return + } + pre := make(map[string]float64, len(before)) + for _, t := range before { + pre[t.Driver] += t.TargetW + } + post := make(map[string]float64, len(after)) + for _, t := range after { + post[t.Driver] += t.TargetW + } + relief := make(map[string]float64) + for driver, w := range post { + if r := math.Min(pre[driver]-w, -w); r > 0 { + relief[driver] = r + } + } + for driver := range s.batteriesOfflineSince { + if _, ok := post[driver]; !ok && s.batteryFuseRelief[driver] > 0 { + relief[driver] = s.batteryFuseRelief[driver] + } + } + s.batteryFuseRelief = relief +} + // Unknown battery energy cannot support a discharge command, even during // fuse relief. Check the final targets so slew cannot restore live discharge // and EV headroom reflects only the battery commands we can actually send. @@ -2461,7 +2716,6 @@ func recordDispatchTargets(targets []DispatchTarget, state *State, updatePrevTar for _, t := range targets { state.batteryCommands[t.Driver] = state.batteryCommands[t.Driver].add(now, t.TargetW) } - state.evOffers = state.evOffers.add(now, math.Max(0, state.EVCommandedW)) } if updatePrevTargets { for _, t := range targets { @@ -2514,31 +2768,36 @@ func applyBatteryBoostReserve(targets []DispatchTarget, store *telemetry.Store, continue } r := store.Get(out[i].Driver, telemetry.DerBattery) - capacityWh := driverCapacities[out[i].Driver] - if r == nil || r.SoC == nil || capacityWh <= 0 || *r.SoC <= state.BatteryBoostReserveSoC { - out[i].TargetW = 0 - out[i].Clamped = true - continue - } - // Bound the next control interval's discharge energy to the SoC - // headroom above reserve. A 0.9 discharge-efficiency factor is - // deliberately conservative: delivering 1 Wh to the AC bus costs - // more than 1 Wh inside the pack. This closes the between-ticks - // overshoot that a threshold-only clamp would permit. - dtS := float64(state.MinDispatchIntervalS) - if dtS <= 0 { - dtS = 5 - } - headroomWh := (*r.SoC - state.BatteryBoostReserveSoC) * capacityWh - maxDischargeW := headroomWh * 0.9 * 3600 / dtS - if -out[i].TargetW > maxDischargeW { - out[i].TargetW = -maxDischargeW + if lower := boostReserveLowerW(r, driverCapacities[out[i].Driver], state); out[i].TargetW < lower { + out[i].TargetW = lower out[i].Clamped = true } } return out } +// boostReserveLowerW is the lowest target that keeps a battery above the +// battery-boost reserve until the next tick. Without a reserve it is -Inf. +func boostReserveLowerW(r *telemetry.DerReading, capacityWh float64, state *State) float64 { + if state == nil || state.BatteryBoostReserveSoC <= 0 { + return math.Inf(-1) + } + if r == nil || r.SoC == nil || capacityWh <= 0 || *r.SoC <= state.BatteryBoostReserveSoC { + return 0 + } + // Bound the next control interval's discharge energy to the SoC + // headroom above reserve. A 0.9 discharge-efficiency factor is + // deliberately conservative: delivering 1 Wh to the AC bus costs + // more than 1 Wh inside the pack. This closes the between-ticks + // overshoot that a threshold-only clamp would permit. + dtS := float64(state.MinDispatchIntervalS) + if dtS <= 0 { + dtS = 5 + } + headroomWh := (*r.SoC - state.BatteryBoostReserveSoC) * capacityWh + return -headroomWh * 0.9 * 3600 / dtS +} + // protectiveCurtailLimitW caps PV at live load plus margin when SoC is // high and surplus exceeds that margin. (0, false) means leave PV alone. func protectiveCurtailLimitW(state *State, store *telemetry.Store) (float64, bool) { @@ -3380,7 +3639,7 @@ func floorNegativeTargets(targets []DispatchTarget) []DispatchTarget { // toward charge after noSelfCharge set 0. The snap-to-zero list inside // slew never grew the later charge gates, which is how an idle slot // commanded +1500 W. This only moves a target toward zero. Discharge -// stays out: the fuse saver may need it. +// has its own twin, floorBlockedDischarge, which yields to the fuse. func floorBlockedCharge(targets []DispatchTarget, noSelfCharge bool, chargeBlocked map[string]bool) []DispatchTarget { if !noSelfCharge && len(chargeBlocked) == 0 { return targets @@ -3397,6 +3656,106 @@ func floorBlockedCharge(targets []DispatchTarget, noSelfCharge bool, chargeBlock return targets } +// floorBlockedDischarge is the discharge twin of floorBlockedCharge. Slew +// anchors on measured watts, so a pack the distributor parked at 0 W (SoC +// below the floor, discharge_capable=false) is walked back toward its live +// discharge. Stop it at 0 W. If the stop would put the site or one phase +// over the fuse or import limit, the blocked packs keep only the discharge +// the capable packs cannot cover, and never more than their slewed ramp. +// forceFuseDischarge then takes the rest from the capable packs. +func floorBlockedDischarge( + targets []DispatchTarget, + store *telemetry.Store, + state *State, + driverCapacities map[string]float64, + fuseMaxW float64, +) []DispatchTarget { + if store == nil || state == nil { + return targets + } + var out []DispatchTarget + var rampW float64 // discharge the slewed ramp still asks of blocked packs + blocked := make([]bool, len(targets)) + readings := make(map[string]*telemetry.DerReading, len(targets)) + for i, t := range targets { + r := store.Get(t.Driver, telemetry.DerBattery) + readings[t.Driver] = r + if t.TargetW >= 0 || r == nil { + continue + } + if !dischargeParked(r, state.DriverLimits[t.Driver]) { + continue + } + if out == nil { + out = append([]DispatchTarget(nil), targets...) + } + blocked[i] = true + rampW -= t.TargetW + out[i].TargetW = 0 + out[i].Clamped = true + } + if out == nil { + return targets + } + if fuseMaxW <= 0 { + return out + } + meter := store.Get(state.SiteMeterDriver, telemetry.DerMeter) + keepW := blockedDischargeKeepW(store, out, blocked, readings, meter, state, driverCapacities, fuseMaxW) + if keepW <= 0 { + return out + } + if keepW >= rampW { + return targets + } + for i, t := range targets { + if blocked[i] { + out[i].TargetW = t.TargetW * keepW / rampW + } + } + return out +} + +// dischargeParked reports whether the distributor parks a pack's discharge +// at 0 W: no SoC, SoC at or under Core's 5 % floor, discharge_capable=false +// or a zero discharge limit. The owner's soc_min is a planner rule; the +// owner floor applies it separately (floorBelowOwnerFloorDischarge). +func dischargeParked(r *telemetry.DerReading, lim PowerLimits) bool { + dischargeBlocked, _ := batteryDirectionBlocks(r.Data) + return r.SoC == nil || *r.SoC <= dischargeFloorSoC || dischargeBlocked || lim.dischargeCap() == 0 +} + +// blockedDischargeKeepW is the discharge the blocked packs must keep after +// floorBlockedDischarge set them to 0 W. forceFuseDischarge covers the +// aggregate overage from capable packs, so only the part beyond their room +// stays with the blocked packs. Both read the phases as the stop leaves +// them (phaseReliefForTargets), so a phase that only the stop puts over the +// line is covered the same way. +func blockedDischargeKeepW( + store *telemetry.Store, + out []DispatchTarget, + blocked []bool, + readings map[string]*telemetry.DerReading, + meter *telemetry.DerReading, + state *State, + driverCapacities map[string]float64, + fuseMaxW float64, +) float64 { + reliefW := math.Max(0, fuseReliefFromSnapshot(out, readings, meter, livePhaseSample(store, state.SiteMeterDriver, meter), state, fuseMaxW)) + var roomW float64 + for i, t := range out { + r := readings[t.Driver] + if blocked[i] || r == nil || driverCapacities[t.Driver] <= 0 { + continue + } + lower, _ := fuseTargetBounds(r, state.DriverLimits[t.Driver]) + roomW += math.Max(0, t.TargetW-lower) + } + _, _, predicted := predictedGridFromSnapshot(out, readings, meter) + roomW = math.Min(roomW, math.Max(0, predicted+state.effectiveExportCeilingW(fuseMaxW))) + return reliefW - roomW +} + // coverLoadChargeSlot reports whether the current plan slot is a charge-from- // PV-surplus slot: the DP meant to soak surplus (PlannedGridW below the // grid-charge import band), NOT buy from the grid. Such a slot carries no hard @@ -3537,7 +3896,7 @@ func clampTargetsToPowerLimits(targets []DispatchTarget, limits map[string]Power const dischargeFloorSoC = 0.05 // clampWithSoC applies the hard safety clamps for one battery command: -// - don't discharge below SoC 5 % (site: don't make target < 0 when SoC < 0.05); +// - don't discharge at or below SoC 5 % (no target < 0 when SoC <= 0.05); // BMS handles fine-grained SoC but we never ask it to pull an empty pack. // - cap charge at the battery's MaxChargeW (falls back to MaxCommandW default). // - cap discharge at the battery's MaxDischargeW (same fallback). @@ -3549,7 +3908,7 @@ func clampWithSoC(target float64, b batteryInfo) (float64, bool) { clamped := target wasClamped := false // Block discharge when the battery is empty. - if b.soc < dischargeFloorSoC && target < 0 { + if b.soc <= dischargeFloorSoC && target < 0 { clamped = 0 wasClamped = true } @@ -3577,70 +3936,26 @@ func applyFuseGuard(targets []DispatchTarget, store *telemetry.Store, state *Sta // reflected in currentGrid; subtracting them here without adding a // replacement target would double-remove their contribution and miss // fuse overages. - seenBat := make(map[string]struct{}, len(targets)) + batteries := make(map[string]*telemetry.DerReading, len(targets)) var currentBat float64 for _, t := range targets { - if _, seen := seenBat[t.Driver]; seen { + if _, seen := batteries[t.Driver]; seen { continue } - seenBat[t.Driver] = struct{}{} - if r := store.Get(t.Driver, telemetry.DerBattery); r != nil { + r := store.Get(t.Driver, telemetry.DerBattery) + batteries[t.Driver] = r + if r != nil { currentBat += r.SmoothedW } } var currentGrid float64 + var phases phaseSample if siteMeter != "" { - if r := store.Get(siteMeter, telemetry.DerMeter); r != nil { - currentGrid = r.SmoothedW - } - } - var sumTarget float64 - for _, t := range targets { - sumTarget += t.TargetW - } - predicted := currentGrid - currentBat + sumTarget - - // Per-phase relief: the worst single-phase amperage above the fuse - // trip threshold (less the safety margin), converted by - // perPhaseReliefW into the AGGREGATE battery action needed to bring - // it back — the site's configured phase count does that conversion. - // - // This guard uses relief in BOTH directions. `perPhaseOverageW` is - // direction-agnostic — phase amps from the meter are absolute - // magnitudes. Attribute to whichever side the AGGREGATE METER is - // currently flowing (currentGrid), not to the post-target - // `predicted`. Per-phase amps and currentGrid are read from the - // same DerMeter sample, so they're internally consistent; - // `predicted` mixes in `sumTarget` (a hypothetical future state) - // and can swing across 0 on a large planner request, attributing - // a current-export overage to the import path (or vice versa) and - // pushing the grid further into the violating direction. - perPhase := perPhaseReliefW(store, state) - // Aggregate budget honours the safety margin too — keep dispatch - // commands strictly inside the breaker envelope so the inverter's - // own per-phase limiter doesn't fire first and cause a flap. - // - // Import ceiling is the tighter of (fuse − safety margin) and the - // clock hour's import limit; export ceiling is the tighter of (fuse − - // safety margin) and the operator's max_export_w protection limit — - // set when an inverter trips on sustained export below the breaker - // (recurring Ferroamp 0x8030 fault). Both share one enforcement surface. - effImportW := state.effectiveImportCeilingW(fuseMaxW) - effExportW := state.effectiveExportCeilingW(fuseMaxW) - importOverage := predicted - effImportW - exportOverage := -effExportW - predicted - if perPhase > 0 { - if currentGrid >= 0 { - if perPhase > importOverage { - importOverage = perPhase - } - } else { - if perPhase > exportOverage { - exportOverage = perPhase - } + if meter := store.Get(siteMeter, telemetry.DerMeter); meter != nil { + currentGrid = meter.SmoothedW + phases = livePhaseSample(store, siteMeter, meter) } } - out := make([]DispatchTarget, len(targets)) copy(out, targets) @@ -3693,24 +4008,41 @@ func applyFuseGuard(targets []DispatchTarget, store *telemetry.Store, state *Sta state.FuseHoldUntil = time.Time{} } - if importOverage <= 0 && exportOverage <= 0 { - return out + // Predict from the held targets. Relief taken from the request would + // count the hold's cut a second time, and a charge limited by one phase + // would switch between the cap and 0 on every dispatch. + var sumTarget float64 + for _, t := range out { + sumTarget += t.TargetW } + predicted := currentGrid - currentBat + sumTarget - // Headroom buffer: shrink targets by `overage + half the configured - // safety margin` so post-clamp the grid sits *below* the threshold - // instead of riding right at it. Without this, the next dispatch - // tick lets the planner re-ramp into the threshold and the system - // oscillates at the boundary — the operator's "safety margin" - // becomes the active steady-state instead of a buffer below it. + // Per phase, each way, once the targets replace the batteries' power + // in the same raw reading. + phaseImportW, phaseExportW := phaseReliefForTargets(phases, state, out, batteries) + // Aggregate budget honours the safety margin too — keep dispatch + // commands strictly inside the breaker envelope so the inverter's + // own per-phase limiter doesn't fire first and cause a flap. // - // Half-margin chosen so the post-clamp phase amps still cluster - // near the threshold (operator wants the fuse used efficiently) - // but with enough buffer to absorb load fluctuations and per-phase - // imbalance on the next tick. Operators wanting tighter or looser - // buffer just adjust safety_margin_a. - buffer := state.fuseSafetyMarginW() * 0.5 + // Import ceiling is the tighter of (fuse − safety margin) and the + // clock hour's import limit; export ceiling is the tighter of (fuse − + // safety margin) and the operator's max_export_w protection limit — + // set when an inverter trips on sustained export below the breaker + // (recurring Ferroamp 0x8030 fault). Both share one enforcement surface. + effImportW := state.effectiveImportCeilingW(fuseMaxW) + effExportW := state.effectiveExportCeilingW(fuseMaxW) + importOverage := math.Max(predicted-effImportW, phaseImportW) + exportOverage := math.Max(-effExportW-predicted, phaseExportW) + + if importOverage <= 0 && exportOverage <= 0 { + return out + } + // Cut by the overage only, so the grid lands on the line, not below it. + // The prediction from the held targets above keeps the next dispatch + // from re-ramping through the line, and the 30 s hold below keeps the + // planner at the clamped total. A site that wants headroom sets + // safety_margin_a, which lowers the line itself. switch { case importOverage > 0: // Too much import (aggregate or per-phase) → shrink charging. @@ -3727,7 +4059,7 @@ func applyFuseGuard(targets []DispatchTarget, store *telemetry.Store, state *Sta // still flip idle batteries to discharge. return out } - newTotal := totalCharge - importOverage - buffer + newTotal := totalCharge - importOverage if newTotal < 0 { newTotal = 0 } @@ -3755,7 +4087,7 @@ func applyFuseGuard(targets []DispatchTarget, store *telemetry.Store, state *Sta // is the lever in that scenario (annotateCurtailment). return out } - newTotal := totalDischarge - exportOverage - buffer + newTotal := totalDischarge - exportOverage if newTotal < 0 { newTotal = 0 } @@ -3830,106 +4162,459 @@ func (s *State) effectiveExportCeilingW(fuseMaxW float64) float64 { return eff } -// perPhaseOverageW returns the wattage by which the worst single phase -// exceeds the per-phase fuse amperage (less the safety margin). 0 when -// within limits, when per-phase data isn't available, or when the -// per-phase clamp is disabled (state.SiteFuseAmps == 0). The meter -// driver must emit l1_a / l2_a / l3_a in DerReading.Data — Pixii, -// Ferroamp, and Sungrow all do this today. -// -// Only the configured phases are read, in L1/L2/L3 order, matching -// sitePhaseCurrentsAt in cmd/ftw: a 1Φ site has no L2, so a stray -// l2_a in the meter payload is not a breaker this site owns. -// -// Same law as fuseSafetyMarginW: amps become watts only through the -// configured per-phase voltage. No hardcoded 230 V here — a site whose -// fuse description is incomplete gets no per-phase clamp rather than an -// invented one. Config fills both fields (fuse.voltage defaults to 230, -// fuse.phases to 3, and validation rejects <= 0), so this only affects -// harnesses that wire SiteFuseAmps alone. -// -// Direction-agnostic: phase amps from the meter are absolute magnitudes, -// so this function reports overage regardless of whether the breaker is -// being approached on the import or export side. The caller attributes -// the overage to whichever direction the aggregate grid is flowing. -func perPhaseOverageW(store *telemetry.Store, state *State) float64 { - if state == nil || state.SiteFuseAmps <= 0 || state.SiteMeterDriver == "" { - return 0 +// phaseSample is one meter reading's phase currents with the raw total of +// the same reading. The zero value has no phases. +type phaseSample struct { + meter *telemetry.DerReading + totalW float64 + store *telemetry.Store +} + +// livePhaseSample pairs meter with its raw total while the meter still +// reports. A watchdog-offline meter gives no phases, since its cached +// currents are no longer the site. So does a store that has moved on to a +// newer reading, since its total no longer matches meter's currents. +func livePhaseSample(store *telemetry.Store, driver string, meter *telemetry.DerReading) phaseSample { + if store == nil || meter == nil { + return phaseSample{} + } + totalW, at, ok := store.OnlineMeterW(driver) + if !ok || !at.Equal(meter.UpdatedAt) { + return phaseSample{} } - r := store.Get(state.SiteMeterDriver, telemetry.DerMeter) - return perPhaseOverageReadingW(r, state) + return phaseSample{meter: meter, totalW: totalW, store: store} } -// perPhaseOverageReadingW is the snapshot form used when one control decision -// must keep the aggregate reading and its phase data from the same meter sample. -func perPhaseOverageReadingW(r *telemetry.DerReading, state *State) float64 { - if state == nil || state.SiteFuseAmps <= 0 || state.SiteFuseVoltage <= 0 || state.SiteFusePhases <= 0 { - return 0 +// signedPhasesA is the sample's phase currents, import positive +// (phaseCurrentsA). When no phase reads as export while the total does, the +// meter gives no direction per phase. A phase then counts as export only +// when it must: when the other phases' currents, at most V × A of real power +// each, cannot carry the exported total on their own (tolerance as in +// phasePowersAddUp). A phase that may import counts as import, +// phaseCurrentsA's safe side, since a reactive load can draw its whole +// current beside exporting phases. false without usable phase currents. +func (p phaseSample) signedPhasesA(state *State) ([]float64, bool) { + if p.meter == nil || state == nil { + return nil, false } - if r == nil || len(r.Data) == 0 { - return 0 + amps, _, valid := phaseCurrentsA(p.meter, state, p.totalW) + if !valid || len(amps) == 0 { + return nil, false } - var d struct { - L1A *float64 `json:"l1_a"` - L2A *float64 `json:"l2_a"` - L3A *float64 `json:"l3_a"` + exports := p.totalW < 0 + var sumW float64 + for _, a := range amps { + exports = exports && a >= 0 + sumW += a * state.SiteFuseVoltage } - if err := json.Unmarshal(r.Data, &d); err != nil { - return 0 + if !exports { + return amps, true + } + tolW := math.Max(250, sumW/10) + for i, a := range amps { + if sumW-a*state.SiteFuseVoltage < -p.totalW-tolW { + amps[i] = -a + } } - // Use absolute magnitude — drivers like Pixii decode per-phase amps - // as signed i16, so export shows up as negative numbers and a naive - // `*p > maxA` walk starting from 0 leaves maxA = 0 (no fire) even - // when a phase is at -17 A. The fuse trips on current magnitude - // regardless of direction, and the caller attributes the overage - // to the active aggregate-grid direction. - phaseAmps := [...]*float64{d.L1A, d.L2A, d.L3A} - phases := state.SiteFusePhases - if phases > len(phaseAmps) { - phases = len(phaseAmps) - } - maxA := 0.0 - for i := 0; i < phases; i++ { - p := phaseAmps[i] - if p == nil { + return amps, true +} + +// unshownBatteryW is the batteries' increase the site meter may not show +// yet: each battery's latest order less the least the meter may show of it, +// the lowest order of the last evSettle (as pendingBatteryCommandsW) or the +// lowest of the battery's own readings over that time, the current one +// included. A battery that ramps slower than evSettle has not reached an +// order older than evSettle, so the order stays unshown until evSettle after +// the battery reads it. +func (s *State) unshownBatteryW(now time.Time, store *telemetry.Store) float64 { + var unshown float64 + for driver, log := range s.batteryCommands { + low, _, latest, ok := log.span(now) + if !ok { continue } - a := math.Abs(*p) - if a > maxA { - maxA = a + if read, _, _, seen := s.batteryReadings[driver].span(now); seen { + low = math.Min(low, read) } + if w, _, live := store.OnlineRawW(driver, telemetry.DerBattery); live { + low = math.Min(low, w) + } + unshown += math.Max(0, latest-low) } - threshold := state.SiteFuseAmps - state.SiteFuseSafetyA - if threshold < 0 { - threshold = 0 + return unshown +} + +// recordBatteryReadings logs each dispatched battery's reading for +// unshownBatteryW. +func (s *State) recordBatteryReadings(targets []DispatchTarget, store *telemetry.Store) { + if store == nil { + return } - if maxA <= threshold { - return 0 + now := s.now() + for _, t := range targets { + w, _, live := store.OnlineRawW(t.Driver, telemetry.DerBattery) + if !live { + continue + } + if s.batteryReadings == nil { + s.batteryReadings = map[string]commandLog{} + } + s.batteryReadings[t.Driver] = s.batteryReadings[t.Driver].add(now, w) } - return (maxA - threshold) * state.SiteFuseVoltage } -// perPhaseReliefW converts a worst-phase overage into the AGGREGATE -// battery action needed to clear it. One conversion, one law: a battery -// spreads its action evenly across the site's phases, so each watt of -// aggregate action relieves 1/SiteFusePhases watts on the worst phase, -// and an overage of N watts needs N × SiteFusePhases watts of action. +// fitBatteryRisesToFuse keeps battery charge rises inside the room the site +// meter shows on its busiest phase: the working limit (fuseLineA) less the +// phase's reading and less our own rises that reading may not show yet, the +// chargers' offers (counted as evShownW does) and the batteries' orders +// (unshownBatteryW). A three-phase battery puts its power evenly on +// every phase. A rise is charge above the battery's latest order, or above +// its reading before our first order. Holding an order is never a rise, and +// a discharge we end uses room first. Nothing here cuts below the hold: an +// overage is the relief's (applyFuseGuard, forceFuseDischarge), which reads +// the battery's own power. Without usable phase currents the aggregate guard +// is the check. +func fitBatteryRisesToFuse(targets []DispatchTarget, store *telemetry.Store, state *State) []DispatchTarget { + if store == nil || state == nil || len(targets) == 0 || state.SiteFuseVoltage <= 0 || state.SiteFusePhases <= 0 { + return targets + } + meter := store.Get(state.SiteMeterDriver, telemetry.DerMeter) + if meter == nil { + return targets + } + amps, ok := livePhaseSample(store, state.SiteMeterDriver, meter).signedPhasesA(state) + if !ok { + return targets + } + now := state.now() + perA := state.SiteFuseVoltage * float64(state.SiteFusePhases) + unshownA := state.unshownBatteryW(now, store) / perA + for _, c := range state.evLimit.offers { + _, a := c.log.unshown(now, c.batterySettle()) + unshownA += a // A one-phase offer may land on any phase. + } + worstA := math.Inf(-1) + for _, a := range amps { + worstA = math.Max(worstA, a) + } + roomW := (state.fuseLineA() - worstA - unshownA) * perA + holds := make([]float64, len(targets)) + var riseW float64 + for i, t := range targets { + var base float64 + if _, _, latest, sent := state.batteryCommands[t.Driver].span(now); sent { + base = latest + } else if w, _, live := store.OnlineRawW(t.Driver, telemetry.DerBattery); live { + base = w + } + roomW -= math.Max(0, math.Min(0, t.TargetW)-math.Min(0, base)) + holds[i] = math.Min(t.TargetW, math.Max(0, base)) + riseW += t.TargetW - holds[i] + } + if riseW <= 0 || roomW >= riseW { + return targets + } + scale := math.Max(0, roomW) / riseW + out := append([]DispatchTarget(nil), targets...) + for i := range out { + if out[i].TargetW > holds[i] { + out[i].TargetW, out[i].Clamped = holds[i]+(out[i].TargetW-holds[i])*scale, true + } + } + return out +} + +// moveW is the batteries' change from the raw power the phases hold to +// the targets. The phase currents are raw, so the move is too. Each +// battery's raw power comes live from the store and must be the reading in +// batteries. false when a targeted battery has no reading, has gone +// offline or has moved on to a newer reading: its power in the phases is +// then unknown. // -// The phase count is the site's, from config — not the constant 3 the -// two call sites used to carry separately. On a 1Φ site the aggregate -// meter and the single phase are the same wire, so demanding 3 × N -// there overshoots by 2 × N and can push the grid through zero into a -// violation on the opposite side. +// sentW is the same change from the command each battery got in the last +// dispatch, for a battery that got one, and from its reading otherwise. // -// Conservative for a 1Φ battery on a 3Φ site (Pixii Home etc.): it -// over-corrects on the phases it does not sit on — less import/export -// there, still safe. -func perPhaseReliefW(store *telemetry.Store, state *State) float64 { - overage := perPhaseOverageW(store, state) - if overage <= 0 { - return 0 +// rawW is each battery's raw power in this reading. +func (p phaseSample) moveW(state *State, targets []DispatchTarget, batteries map[string]*telemetry.DerReading) (moveW, sentW float64, rawW map[string]float64, ok bool) { + if p.store == nil { + return 0, 0, nil, false + } + rawW = make(map[string]float64, len(targets)) + for _, t := range targets { + moveW += t.TargetW + sentW += t.TargetW + if _, seen := rawW[t.Driver]; seen { + continue + } + r := batteries[t.Driver] + if r == nil { + return 0, 0, nil, false + } + w, at, ok := p.store.OnlineRawW(t.Driver, telemetry.DerBattery) + if !ok || !at.Equal(r.UpdatedAt) { + return 0, 0, nil, false + } + rawW[t.Driver] = w + moveW -= w + if last, sent := state.lastDispatchedW(t.Driver); sent { + sentW -= last + } else { + sentW -= w + } + } + return moveW, sentW, rawW, true +} + +// lastDispatchedW is the command driver got in the last dispatch, if it +// got one. +func (s *State) lastDispatchedW(driver string) (float64, bool) { + if s == nil { + return 0, false + } + log := s.batteryCommands[driver] + if len(log) == 0 || s.LastDispatch == nil || !log[len(log)-1].at.Equal(*s.LastDispatch) { + return 0, false + } + return log[len(log)-1].w, true +} + +// phaseReliefForTargets is the relief that keeps every phase inside +// fuseLineA once the targets replace the batteries' power. 0, 0 when that +// power is unknown. +// +// The relief counts the batteries' move from their readings +// (phaseFuseReliefW). A reading can lag the meter: the meter already shows +// the battery at its last command while the reading still holds the power +// before it. The phase then sits at the line, the reading shows no relief +// in force, and the relief is taken back; on the next tick the reading +// shows it and the relief goes on top of it. The order runs further out +// each time. So a move from the last command may not pass the room the +// meter shows either (phaseSentReliefW). +// +// The relief also counts on the meter showing the batteries' move. When it +// does not, because the meter repeats one reading or the worst phase is one +// the batteries do not reach, each tick finds the same overage and adds it +// to a battery already moved for it: the order runs out to the battery's +// limit. Once the meter fails to show a move it should +// (phaseUnresponsive), the relief toward the worst phase stops. On a +// meter that repeats its reading it stops at the last command. On a +// battery off the worst phase it goes back to the batteries' power when +// that phase last moved: what they moved since did not reach it, and it +// pushes their own phase toward its line. +func phaseReliefForTargets(p phaseSample, state *State, targets []DispatchTarget, batteries map[string]*telemetry.DerReading) (importW, exportW float64) { + moveW, sentW, rawW, ok := p.moveW(state, targets, batteries) + if !ok { + return 0, 0 + } + importW, exportW = phaseFuseReliefW(p, state, moveW) + sentImportW, sentExportW := phaseSentReliefW(p, state, sentW) + importW, exportW = math.Max(importW, sentImportW), math.Max(exportW, sentExportW) + if amps, ok := p.signedPhasesA(state); ok { + if worst, stuck, backTo := state.phaseUnresponsive(amps, p.totalW, rawW); stuck { + capW := sentW + if backTo != nil { + capW = targetsMoveFromW(targets, backTo, rawW) + } + if worst > 0 { + importW = math.Min(importW, math.Max(0, capW)) + } else { + exportW = math.Min(exportW, math.Max(0, -capW)) + } + } } - return overage * float64(state.SiteFusePhases) + return importW, exportW +} + +// targetsMoveFromW is the targets' change from the batteries' power in +// from. A battery that from has not seen counts at its power in rawW. +func targetsMoveFromW(targets []DispatchTarget, from, rawW map[string]float64) (moveW float64) { + seen := make(map[string]bool, len(targets)) + for _, t := range targets { + moveW += t.TargetW + if seen[t.Driver] { + continue + } + seen[t.Driver] = true + if w, ok := from[t.Driver]; ok { + moveW -= w + } else { + moveW -= rawW[t.Driver] + } + } + return moveW +} + +// phaseResponseA is the current a battery move must change on every phase +// before any meter is sure to show it: one ampere, the step in which P1 +// meters report phase current. A phase that changed by less than half of +// it has not moved. +const phaseResponseA = 1.0 + +// phaseResponse holds the readings phaseUnresponsive measures from, each +// with the batteries' raw power at that reading: the one in which the +// worst phase last moved (amps), and the one in which the meter's phases +// or total last changed (changed: the phases, then the total). offPhase +// is set once the batteries are found off the worst phase, phase +// offPhaseAt, and holds until that phase's reference ends. +type phaseResponse struct { + amps, changed []float64 + rawW, changedRawW map[string]float64 + offPhase bool + offPhaseAt int +} + +// phaseUnresponsive reports whether the meter fails to show the batteries +// on the worst phase. worstA is the worst phase's signed current, + +// import. It is stuck when the batteries' raw power has changed by +// phaseResponseA or more per phase since either +// +// - the meter last changed at all, phases or total: the meter repeats +// one reading (C5 makes such a meter stale). A P1 meter reports phase +// current in whole amperes, so a live one can repeat its phases while +// its total moves. backTo is nil: the relief holds the last command. +// - the worst phase last moved, while another phase moved by twice the +// even share of that change or more. A balanced battery moves each +// phase by that share, so this battery sits off the worst phase. +// backTo is the batteries' power when the worst phase last moved. The +// verdict holds until that reference ends, so the relief does not go +// out again once the batteries are back. +// +// A live meter on which the worst phase does not move is not enough: a +// load on that phase can cancel the move, at once or step by step, and +// there the relief must go on. The worst phase's reference ends when that +// phase moves, when it is at or inside the line (fuseLineA), or when +// another phase becomes the worst. Without the second, a reference kept +// while the relief holds the phase at the line would count a load that +// later changes on another phase against all the batteries moved since. +func (s *State) phaseUnresponsive(amps []float64, totalW float64, rawW map[string]float64) (worstA float64, stuck bool, backTo map[string]float64) { + if s == nil || len(amps) == 0 { + return 0, false, nil + } + worst := 0 + for i, a := range amps { + if math.Abs(a) > math.Abs(amps[worst]) { + worst = i + } + } + ref := &s.phaseResponse + perA := s.SiteFuseVoltage * float64(s.SiteFusePhases) + if changed := append(slices.Clone(amps), totalW); !slices.Equal(changed, ref.changed) { + ref.changed, ref.changedRawW = changed, maps.Clone(rawW) + } + if math.Abs(batteriesMovedW(&ref.changedRawW, rawW)) >= phaseResponseA*perA { + return amps[worst], true, nil + } + if len(ref.amps) != len(amps) || math.Abs(amps[worst]-ref.amps[worst]) >= phaseResponseA/2 || + math.Abs(amps[worst]) <= s.fuseLineA() || (ref.offPhase && worst != ref.offPhaseAt) { + ref.amps, ref.rawW, ref.offPhase = slices.Clone(amps), maps.Clone(rawW), false + return amps[worst], false, nil + } + movedW := batteriesMovedW(&ref.rawW, rawW) + if ref.offPhase { + return amps[worst], true, ref.rawW + } + if math.Abs(movedW) < phaseResponseA*perA { + return amps[worst], false, nil + } + for i, a := range amps { + if i != worst && (a-ref.amps[i])/(movedW/perA) >= 2 { + ref.offPhase, ref.offPhaseAt = true, worst + return amps[worst], true, ref.rawW + } + } + return amps[worst], false, nil +} + +// batteriesMovedW is the batteries' raw power change from ref. A battery +// that ref has not seen joins it at its present power. +func batteriesMovedW(ref *map[string]float64, rawW map[string]float64) (movedW float64) { + for driver, w := range rawW { + was, ok := (*ref)[driver] + if !ok { + if *ref == nil { + *ref = make(map[string]float64, len(rawW)) + } + (*ref)[driver] = w + continue + } + movedW += w - was + } + return movedW +} + +// phaseSentReliefW is the relief that keeps sentW, the batteries' move from +// their last commands, inside the room the meter shows on each phase. A +// phase already over the line has no room; its overage is +// phaseFuseReliefW's. So this relief never asks for more than to stay at +// the last command, and a battery that cannot reach its command does not +// wind the relief up. With a phase over the export line, holding the last +// command against the import line would pin that overage where it is, so +// the import side gives way to phaseFuseReliefW's least-over move; the +// same holds the other way round. +func phaseSentReliefW(p phaseSample, state *State, sentW float64) (importW, exportW float64) { + amps, ok := p.signedPhasesA(state) + if !ok { + return 0, 0 + } + perA := state.SiteFuseVoltage * float64(state.SiteFusePhases) + limitA := state.fuseLineA() + hi, lo := math.Inf(-1), math.Inf(1) + overImport, overExport := false, false + for _, a := range amps { + overImport, overExport = overImport || a > limitA, overExport || a < -limitA + a = math.Max(-limitA, math.Min(limitA, a)) + sentW/perA + hi, lo = math.Max(hi, a), math.Min(lo, a) + } + importW, exportW = math.Max(0, hi-limitA)*perA, math.Max(0, -limitA-lo)*perA + if importW < 1 || overExport { + importW = 0 + } + if exportW < 1 || overImport { + exportW = 0 + } + return importW, exportW +} + +// phaseFuseReliefW is the aggregate battery move that brings every phase +// inside the working limit, fuseLineA: the fuse less safety_margin_a, the +// same line the car uses. importW asks for less charge or more +// discharge, exportW for the reverse. moveW is the batteries' change from +// the power this reading already holds (phaseSample.moveW); like the +// relief, it spreads evenly over the site's phases. +// +// The phases are the meter's signed currents (signedPhasesA). When no move +// fits every phase, the move leaves the worst phase least over, so on a +// meter that gives each phase's direction a move never raises the current +// on the phase it protects. Two limits remain: a meter without phase power +// under an exporting total, where a phase the total does not force to +// export is read as import, and a battery on one phase, which is read as +// balanced. 0, 0 without usable phase currents. +func phaseFuseReliefW(p phaseSample, state *State, moveW float64) (importW, exportW float64) { + amps, ok := p.signedPhasesA(state) + if !ok { + return 0, 0 + } + perA := state.SiteFuseVoltage * float64(state.SiteFusePhases) // aggregate W per phase ampere + limitA := state.fuseLineA() + hi, lo := math.Inf(-1), math.Inf(1) + for _, a := range amps { + a += moveW / perA + hi, lo = math.Max(hi, a), math.Min(lo, a) + } + // The smallest shift that keeps every phase inside, or the middle one. + shiftA := math.Max(-limitA-lo, math.Min(0, limitA-hi)) + if -limitA-lo > limitA-hi { + shiftA = -(hi + lo) / 2 + } + importW, exportW = math.Max(0, -shiftA)*perA, math.Max(0, shiftA)*perA + // A target that fills the limit exactly can land a rounding error past it. + if importW < 1 { + importW = 0 + } + if exportW < 1 { + exportW = 0 + } + return importW, exportW } // applyPlanSignFloor enforces "executed battery total must agree in sign @@ -4050,17 +4735,55 @@ func fuseSaverEarlyExit( driverCapacities map[string]float64, fuseMaxW float64, ) []DispatchTarget { - out := fuseSaverFromLivePower(store, state, driverCapacities, fuseMaxW) + out, live := fuseSaverAtLivePower(store, state, driverCapacities, fuseMaxW) out = state.stopFullBatteries(out, store, driverCapacities) if out == nil || state == nil { return out } + // The early path starts from live power, so relief is discharge below + // the live power the decision read. Without a command the last one + // stands, and so does its relief. + state.recordFuseRelief(live, out) state.LastTargets = out now := state.now() state.LastDispatch = &now return out } +// siteMeterHeld reports whether the site meter has no sample newer than the +// one the last dispatch acted on. A failed poll keeps the old reading and its +// UpdatedAt, and the dispatch gate passes it until the watchdog expires. Acting +// on it again moves the battery one more held error each tick: a held +2 kW +// import wound a battery to -10 kW in ten ticks while the site exported 8 kW. +// A failed read does not count by itself: a poll that emits and then errors +// still delivers a new sample. +// The EV fuse limit follows the same rule (ev_fuse_budget.go). +func (s *State) siteMeterHeld(meter *telemetry.DerReading) bool { + if meter == nil { + return false + } + if !meter.UpdatedAt.After(s.dispatchMeterAt) { + return true + } + s.dispatchMeterAt = meter.UpdatedAt + return false +} + +// holdTowardZero keeps each target between 0 and the last command sent to +// that battery. Without a new meter sample, dispatch may stop or reduce a +// battery but must not raise one. A battery with no earlier command gets 0. +func holdTowardZero(targets []DispatchTarget, prev map[string]float64) []DispatchTarget { + for i := range targets { + p := prev[targets[i].Driver] + w := math.Max(math.Min(targets[i].TargetW, math.Max(p, 0)), math.Min(p, 0)) + if w != targets[i].TargetW { + targets[i].TargetW = w + targets[i].Clamped = true + } + } + return targets +} + // holdFleetAtZero builds the forced-idle command set: 0 W for every battery // this dispatcher is allowed to command. Eligibility is the same predicate the // main path's "gather online batteries" applies — a health record that reports @@ -4100,10 +4823,10 @@ func fuseTargetBounds(r *telemetry.DerReading, lim PowerLimits) (lower, upper fl lower = -lim.dischargeCap() upper = lim.chargeCap() dischargeBlocked, chargeBlocked := batteryDirectionBlocks(r.Data) - if r.SoC == nil || *r.SoC < dischargeFloorSoC || dischargeBlocked { + if r.SoC == nil || *r.SoC <= lim.socFloor() || dischargeBlocked { lower = 0 } - if chargeBlocked { + if chargeBlocked || r.SoC != nil && *r.SoC >= lim.socCeiling() { upper = 0 } return lower, upper @@ -4402,24 +5125,38 @@ func fuseSaverFromLivePower( driverCapacities map[string]float64, fuseMaxW float64, ) []DispatchTarget { + out, _ := fuseSaverAtLivePower(store, state, driverCapacities, fuseMaxW) + return out +} + +// fuseSaverAtLivePower is fuseSaverFromLivePower that also returns the live +// power of each online battery it started from, so the caller records relief +// against the same reading the decision used. +func fuseSaverAtLivePower( + store *telemetry.Store, + state *State, + driverCapacities map[string]float64, + fuseMaxW float64, +) (out, rawLive []DispatchTarget) { if fuseMaxW <= 0 || len(driverCapacities) == 0 || store == nil || state == nil { - return nil + return nil, nil } meter := store.Get(state.SiteMeterDriver, telemetry.DerMeter) if meter == nil { - return nil + return nil, nil } rawLive, safeBaseline, batteryReadings := fleetAtLivePower(store, driverCapacities, state.DriverLimits) if len(rawLive) == 0 { - return nil + return nil, nil } // A command-safe baseline may differ from the live snapshot even when the // site is safe. Preserve the deadband in that case: this early-return path // exists to answer a fuse condition, not to take control of an externally // moved battery merely because its reading sits outside a command cap. - requiredRelief := fuseReliefFromSnapshot(rawLive, batteryReadings, meter, state, fuseMaxW, false) + phases := livePhaseSample(store, state.SiteMeterDriver, meter) + requiredRelief := fuseReliefFromSnapshot(rawLive, batteryReadings, meter, phases, state, fuseMaxW) if requiredRelief <= 0 { - return nil + return nil, nil } _, _, predictedRaw := predictedGridFromSnapshot(rawLive, batteryReadings, meter) maxRelief := predictedRaw + state.effectiveExportCeilingW(fuseMaxW) @@ -4433,9 +5170,9 @@ func fuseSaverFromLivePower( maxRelief, ) if len(safeBaseline) == 0 { - return nil + return nil, nil } - out := forceFuseDischargeWithSnapshot(safeBaseline, batteryReadings, meter, state, driverCapacities, fuseMaxW, true) + out = forceFuseDischargeWithSnapshot(safeBaseline, batteryReadings, meter, phases, state, driverCapacities, fuseMaxW) rawByDriver := make(map[string]DispatchTarget, len(rawLive)) for _, t := range rawLive { rawByDriver[t.Driver] = t @@ -4444,12 +5181,12 @@ func fuseSaverFromLivePower( rawTarget, ok := rawByDriver[t.Driver] if !ok || t.TargetW != rawTarget.TargetW || t.Clamped != rawTarget.Clamped { if !fuseSnapshotStillCurrent(store, state, meter.UpdatedAt, batteryReadings, out) { - return nil + return nil, nil } - return out + return out, rawLive } } - return nil + return nil, nil } // fuseReliefFromSnapshot returns the import-side relief required for one @@ -4458,47 +5195,49 @@ func fuseReliefFromSnapshot( targets []DispatchTarget, batteryReadings map[string]*telemetry.DerReading, meter *telemetry.DerReading, + phases phaseSample, state *State, fuseMaxW float64, - accountPhaseTargetDelta bool, ) float64 { + overage, _ := fuseReliefPartsFromSnapshot(targets, batteryReadings, meter, phases, state, fuseMaxW) + return overage +} + +// fuseReliefPartsFromSnapshot also returns limitOnlyW, the part of the relief +// that only the hour's import limit asks for. +func fuseReliefPartsFromSnapshot( + targets []DispatchTarget, + batteryReadings map[string]*telemetry.DerReading, + meter *telemetry.DerReading, + phases phaseSample, + state *State, + fuseMaxW float64, +) (overage, limitOnlyW float64) { if fuseMaxW <= 0 || len(targets) == 0 || state == nil { - return 0 + return 0, 0 } - currentBat, sumTarget, predicted := predictedGridFromSnapshot(targets, batteryReadings, meter) - currentGrid := 0.0 - if meter != nil { - currentGrid = meter.SmoothedW - } + _, _, predicted := predictedGridFromSnapshot(targets, batteryReadings, meter) // The fuse acts on every reading. The hour's import limit is a budget: // the part of its overage that an EV cut we sent will remove waits for // that cut while the meter may not show it yet. Relief for the same watts // would discharge the battery into export once the cut lands (#1674). - // Per-phase relief remains fuse-only and may act only while the matching - // aggregate meter sample is on the import side. + // Per-phase relief is fuse-only. fuseW := math.Max(0, fuseMaxW-state.fuseSafetyMarginW()) - overage := predicted - fuseW + overage = predicted - fuseW + limitOverage := math.Inf(-1) if ceilingW := state.effectiveImportCeilingW(fuseMaxW); ceilingW < fuseW { - overage = math.Max(overage, predicted-ceilingW-state.pendingEVCutW()) - } - if currentGrid >= 0 { - relief := perPhaseOverageReadingW(meter, state) * float64(state.SiteFusePhases) - if relief > 0 && accountPhaseTargetDelta { - // The early fuse baseline can only hold or reduce each included - // target. Count that import reduction against the raw phase relief so - // a command-cap clamp is not followed by the same correction twice. - relief += sumTarget - currentBat - if relief < 0 { - relief = 0 - } - } - if relief > overage { - overage = relief - } + limitOverage = predicted - ceilingW - state.pendingEVCutW() } - return overage + if phaseW, _ := phaseReliefForTargets(phases, state, targets, batteryReadings); phaseW > overage { + overage = phaseW + } + if limitOverage > overage { + limitOnlyW = math.Max(0, limitOverage-math.Max(0, overage)) + overage = limitOverage + } + return overage, limitOnlyW } // predictedGridFromSnapshot holds non-targeted site power constant and replaces @@ -4550,7 +5289,7 @@ func forceFuseDischarge( readings[target.Driver] = store.Get(target.Driver, telemetry.DerBattery) } meter := store.Get(state.SiteMeterDriver, telemetry.DerMeter) - return forceFuseDischargeWithSnapshot(targets, readings, meter, state, driverCapacities, fuseMaxW, false) + return forceFuseDischargeWithSnapshot(targets, readings, meter, livePhaseSample(store, state.SiteMeterDriver, meter), state, driverCapacities, fuseMaxW) } // forceFuseDischargeWithSnapshot keeps all math for one decision on the same @@ -4560,15 +5299,15 @@ func forceFuseDischargeWithSnapshot( targets []DispatchTarget, batteryReadings map[string]*telemetry.DerReading, meter *telemetry.DerReading, + phases phaseSample, state *State, driverCapacities map[string]float64, fuseMaxW float64, - accountPhaseTargetDelta bool, ) []DispatchTarget { if fuseMaxW <= 0 || len(targets) == 0 || state == nil { return targets } - overage := fuseReliefFromSnapshot(targets, batteryReadings, meter, state, fuseMaxW, accountPhaseTargetDelta) + overage, limitOnlyW := fuseReliefPartsFromSnapshot(targets, batteryReadings, meter, phases, state, fuseMaxW) if overage <= 0 { return targets } @@ -4581,13 +5320,12 @@ func forceFuseDischargeWithSnapshot( overage = exportRoom } - type slot struct { - idx int - headroom float64 - } - slots := make([]slot, 0, len(targets)) - var totalHeadroom float64 + // The fuse may discharge down to the SoC floor. The hour's import limit + // is a bill, not a breaker: its extra part also keeps the boost reserve. + lowers := make([]float64, len(targets)) + limitLowers := make([]float64, len(targets)) for i, t := range targets { + lowers[i], limitLowers[i] = math.Inf(1), math.Inf(1) cap, ok := driverCapacities[t.Driver] if !ok || cap <= 0 { continue @@ -4596,46 +5334,56 @@ func forceFuseDischargeWithSnapshot( if r == nil { continue } - lower, _ := fuseTargetBounds(r, state.DriverLimits[t.Driver]) + lowers[i], _ = fuseTargetBounds(r, state.DriverLimits[t.Driver]) + limitLowers[i] = math.Max(lowers[i], boostReserveLowerW(r, cap, state)) + } + // Give the limit-only part first to the batteries above the reserve. + // The fuse part then takes what is left down to the SoC floor, so a + // battery under the reserve still serves the fuse and the total is + // never less than in the other order. + fusePartW := math.Max(0, overage-limitOnlyW) + out := allocateFuseRelief(targets, overage-fusePartW, limitLowers) + return allocateFuseRelief(out, fusePartW, lowers) +} + +// allocateFuseRelief subtracts reliefW from the targets in proportion to +// each one's room above its lower bound. +func allocateFuseRelief(targets []DispatchTarget, reliefW float64, lowers []float64) []DispatchTarget { + if reliefW <= 0 { + return targets + } + var totalHeadroom float64 + for i, t := range targets { // Relief subtracts from the current target. Its full safe travel is // the distance to the current lower command bound. A live positive // target contributes charge cancellation even when SoC or the driver // blocks discharge, but the target cannot cross below 0 W in that case. - room := t.TargetW - lower - if room <= 0 { - continue + if room := t.TargetW - lowers[i]; room > 0 { + totalHeadroom += room } - slots = append(slots, slot{idx: i, headroom: room}) - totalHeadroom += room } if totalHeadroom <= 0 { return targets } - - allocate := overage - if allocate > totalHeadroom { - allocate = totalHeadroom - } - + allocate := math.Min(reliefW, totalHeadroom) out := make([]DispatchTarget, len(targets)) copy(out, targets) - for _, s := range slots { - share := allocate * (s.headroom / totalHeadroom) + for i, t := range targets { + room := t.TargetW - lowers[i] + if room <= 0 { + continue + } // Subtract from the current target. Replacing a charge with -share // removed the charge twice and discharged more than the overage. - out[s.idx].TargetW -= share - out[s.idx].Clamped = true + out[i].TargetW -= allocate * (room / totalHeadroom) + out[i].Clamped = true } return out } // evInMeterW is EV load the meter is sure to contain: the operator's declared -// load plus the lowest managed offer of the last evSettle. This tick's new -// offer is not in it yet. Charger readings take no part in the fuse share. +// load plus the managed offers already shown (evShownW). Charger readings +// take no part in the fuse share. func (s *State) evInMeterW() float64 { - low, _, _, ok := s.evOffers.span(s.now()) - if !ok { - low = 0 - } - return s.ManualEVChargingW + math.Max(0, low) + return s.ManualEVChargingW + s.evShownW(s.now()) } diff --git a/go/internal/control/ev_base_after_restart_test.go b/go/internal/control/ev_base_after_restart_test.go new file mode 100644 index 000000000..830eb906d --- /dev/null +++ b/go/internal/control/ev_base_after_restart_test.go @@ -0,0 +1,552 @@ +package control + +import ( + "encoding/json" + "maps" + "math" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/loadpoint" + "github.com/srcfl/ftw/go/internal/sitesim" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// restartFixture is seed 207532 just after Core restarted: a 16 A site with +// no margin, a battery FTW found discharging 5 kW, and a charger whose first +// logged offers of this run are 0 A, sent while the meter was stale, then +// 6,210 W (9 A on three phases) 2 s ago. The charger takes 40 s to apply an +// order, so the car still draws what it drew before the restart. Its +// max_charge_w is 11,040 W (16 A). evW is what the charger measures, +// arriving evMeterDelay before the site reading; the site meter reads +// gridW with the phase currents phaseA, and the PV inverter pvW. +func restartFixture(t *testing.T, gridW float64, phaseA [3]float64, batteryW, pvW, evW float64) (*telemetry.Store, *State) { + t.Helper() + store := telemetry.NewStore() + raw, _ := json.Marshal(map[string]any{"l1_a": phaseA[0], "l2_a": phaseA[1], "l3_a": phaseA[2]}) + store.Update("meter", telemetry.DerMeter, gridW, nil, raw) + store.RecordDriverSuccess("meter") + store.Update("sungrow", telemetry.DerBattery, batteryW, ptrF64(.3), nil) + store.RecordDriverSuccess("sungrow") + if pvW != 0 { + store.Update("pv", telemetry.DerPV, pvW, nil, nil) + store.RecordDriverSuccess("pv") + } + setChargerSample(store, evW, nil) + + state := NewState(0, 0, "meter") + now := time.Now() + state.clock = func() time.Time { return now } + state.SiteFuseAmps, state.SiteFuseVoltage, state.SiteFusePhases, state.SiteFuseSafetyA = 16, 230, 3, 0 + state.DriverLimits = map[string]PowerLimits{"sungrow": {MaxChargeW: 5000, MaxDischargeW: 5000}} + state.batteryCommands = map[string]commandLog{"sungrow": commandLog{}.add(now.Add(-2*time.Second), batteryW)} + state.evFuseDemands = []evFuseDemand{{"garage", 6210, "3p"}} + state.EVCommandedW = 6210 + found := sentCommand{at: now.Add(-6 * time.Second), w: 0, a: 0, n: 3} + state.evLimit.set = true + state.evLimit.offers = map[string]chargerOffers{"garage": { + log: commandLog{found, {at: now.Add(-4 * time.Second), w: 0, a: 0, n: 3}, {at: now.Add(-2 * time.Second), w: 6210, a: 9, n: 3}}, + settle: evUnknownSettle, found: found, driver: "charger", maxW: 11040, + watchFrom: found.at, // Core saw the sample arrive + }} + return store, state +} + +// C10 (27 Oct triage, seed 207532): after a restart, the 0 A offers sent +// while the meter was stale counted as the EV base the meter shows, while +// the meter showed the car at 8.2 kW. The new 6.2 kW order then counted as +// a 9 A rise to come, and the battery was held in discharge for the car +// with battery_covers_ev off, down toward its floor (-3,599 W here). The +// car's measured draw is in the reading: the 6.2 kW order is a cut. Until +// it lands the car may still rise to the 16 A of an offer Core does not +// know, so the battery keeps only the discharge that rise needs on L2. +func TestEVBaseAfterRestartCreditsMeasuredDraw(t *testing.T) { + store, state := restartFixture(t, 780, [3]float64{2.03, 4.97, .47}, -5000, -3000, 8222) + out := clampEVBatteryHeadroom([]DispatchTarget{{Driver: "sungrow", TargetW: 0}}, store, state, 16*230*3) + riseA := (11040 - 8222) / 690.0 + want := -(5000 - (16-4.97-riseA)*690) + if got := targetW(out, "sungrow"); math.Abs(got-want) > 1 { + t.Fatalf("battery %.0f W, want %.0f W: only the car's rise to its 16 A max needs room", got, want) + } +} + +// Review of #1926: a car below the offer in force. The car draws 8.2 kW, +// fresh and inside what the meter shows, but the charger's offer from +// before the restart, or its own default, is 16 A. Until Core's 6.2 kW +// order lands the car may rise to 11 kW (a ramp, a heater). The battery +// keeps the discharge that rise needs. +func TestEVBaseAfterRestartCarBelowOfferInForce(t *testing.T) { + houseA := 3.0 + evW, batW := 8222.0, -5000.0 + gridW := houseA*690 + evW + batW + phaseA := gridW / 690 + store, state := restartFixture(t, gridW, [3]float64{phaseA, phaseA, phaseA}, batW, 0, evW) + out := clampEVBatteryHeadroom([]DispatchTarget{{Driver: "sungrow", TargetW: 0}}, store, state, 16*230*3) + got := targetW(out, "sungrow") + if afterA := phaseA + (got-batW)/690 + (11040-evW)/690; afterA > 16+1e-6 { + t.Fatalf("battery ends discharge to %.0f W; a car rising to its offer in force puts %.2f A on each phase", got, afterA) + } +} + +// The credit stays on the safe side. A charger reading the meter cannot +// contain counts only up to the consumption the meter shows: here the +// charger claims 8.2 kW, but the site consumes 680 W, so 5,530 W of the +// 6,210 W order is still a rise to come, and the battery keeps the +// discharge that the busiest phase has no room to end. +func TestEVBaseAfterRestartCreditStaysInsideTheMeter(t *testing.T) { + store, state := restartFixture(t, 680, [3]float64{4.97, -1, -1}, 0, 0, 8222) + state.batteryCommands["sungrow"] = commandLog{}.add(state.now().Add(-time.Second), -5000) + out := clampEVBatteryHeadroom([]DispatchTarget{{Driver: "sungrow", TargetW: 0}}, store, state, 16*230*3) + riseA := (6210 - 680) / 690.0 + want := -(5000 - (16-4.97-riseA)*690) + if got := targetW(out, "sungrow"); math.Abs(got-want) > 1 { + t.Fatalf("battery %.0f W, want %.0f W: the charger's reading counts only up to the meter's 680 W", got, want) + } +} + +// chargerData is the fixture charger's data: its sample stamped on the +// charger's clock 5 s before the site reading arrived, with extra fields. +func chargerData(store *telemetry.Store, extra map[string]any) json.RawMessage { + at := store.Get("meter", telemetry.DerMeter).UpdatedAt.Add(-5 * time.Second) + d := map[string]any{"control_power_observed_at": at.Format(time.RFC3339Nano)} + maps.Copy(d, extra) + raw, _ := json.Marshal(d) + return raw +} + +// setChargerSample makes w the fixture charger's reading (chargerData), +// arriving evMeterDelay before the site reading. +func setChargerSample(store *telemetry.Store, w float64, extra map[string]any) { + store.Update("charger", telemetry.DerEV, w, nil, chargerData(store, extra)) + store.RecordDriverSuccess("charger") + store.Get("charger", telemetry.DerEV).UpdatedAt = store.Get("meter", telemetry.DerMeter).UpdatedAt.Add(-evMeterDelay) +} + +// setMaxW sets the fixture charger's max_charge_w. +func setMaxW(state *State, w float64) { + c := state.evLimit.offers["garage"] + c.maxW = w + state.evLimit.offers["garage"] = c +} + +// setRiseCapW puts both max_charge_w and the site fuse at w, the most an +// offer in force may be, so a case can test the other bounds alone. +func setRiseCapW(state *State, w float64) { + setMaxW(state, w) + state.SiteFuseAmps = w / (state.SiteFuseVoltage * float64(state.SiteFusePhases)) +} + +// The credit never exceeds the latest offer, keeps room for the car's rise +// to its max, takes the lowest of the sample, raw and smoothed draw and of +// every sample since the site reading, needs a live charger reading that +// arrived evMeterDelay before the site reading, and ends once the +// charger's first offer of this run has had its settle time: then the +// offers alone say what the meter shows. +func TestEVFoundDrawWBounds(t *testing.T) { + for _, tc := range []struct { + name string + edit func(*telemetry.Store, *State) + want float64 + }{ + {"the car may still rise to its max", func(*telemetry.Store, *State) {}, 8222 - (11040 - 6210)}, + {"no max_charge_w", func(_ *telemetry.Store, state *State) { setMaxW(state, 0) }, 0}, + {"max_charge_w below the fuse: the offer in force may still be the fuse's", func(_ *telemetry.Store, state *State) { + setMaxW(state, 7360) + }, 8222 - (11040 - 6210)}, + {"a charger sample measured after the site reading", func(store *telemetry.Store, state *State) { + store.Get("meter", telemetry.DerMeter).UpdatedAt = state.now().Add(-time.Second) + store.Get("charger", telemetry.DerEV).UpdatedAt = state.now() + store.Get("charger", telemetry.DerEV).Data = chargerData(store, map[string]any{"control_power_observed_at": state.now().Format(time.RFC3339Nano)}) + }, 0}, + // Final review of #1926: the charger's stamp does not say when the + // sample was measured on Core's clock; its arrival bounds that. + {"a sample that arrived with the site reading, however early its stamp", func(store *telemetry.Store, _ *State) { + store.Get("charger", telemetry.DerEV).UpdatedAt = store.Get("meter", telemetry.DerMeter).UpdatedAt + }, 0}, + {"only a sample older than one control interval before the reading", func(store *telemetry.Store, state *State) { + // The car may have stopped and started again since that sample; + // the current one, 1 s before the reading, may be ahead of it. + risenSince(store, state, 8222, 11040) + c := state.evLimit.offers["garage"] + c.draws[0].at = c.draws[0].at.Add(-10 * time.Second) + c.draws[0].got = c.draws[0].got.Add(-10 * time.Second) + c.draws[0].seen = c.draws[0].got + state.evLimit.offers["garage"] = c + }, 0}, + {"a sample since the site reading reads less", func(store *telemetry.Store, state *State) { + risenSince(store, state, 8222, 6000) + }, 6000 - (11040 - 6210)}, + // Review of #1926: OCPP stamps MeterValues with the charger's own + // clock. One running 1 s behind Core's stamps a rise sampled after + // the site reading as 1 s before it. The draw stays the one sampled + // before the reading. + {"a rise stamped before the reading by a charger clock behind Core's", func(store *telemetry.Store, state *State) { + risenSince(store, state, 6900, 11040) + }, 6900 - (11040 - 6210)}, + {"latest offer bounds the draw", func(_ *telemetry.Store, state *State) { setRiseCapW(state, 6210) }, 6210}, + {"measured draw below the offer", func(store *telemetry.Store, state *State) { + setRiseCapW(state, 6210) + setChargerSample(store, 4000, nil) + }, 4000}, + {"smoothed draw below the sample", func(store *telemetry.Store, state *State) { + setRiseCapW(state, 6210) + store.Get("charger", telemetry.DerEV).SmoothedW = 5000 + }, 5000}, + {"raw draw below the sample", func(store *telemetry.Store, state *State) { + setRiseCapW(state, 6210) + r := store.Get("charger", telemetry.DerEV) + r.Data = chargerData(store, map[string]any{"control_power_w": 8222}) + r.RawW = 5000 + }, 5000}, + {"control sample below raw and smoothed", func(store *telemetry.Store, state *State) { + setRiseCapW(state, 6210) + store.Get("charger", telemetry.DerEV).Data = chargerData(store, map[string]any{"control_power_w": 5000}) + }, 5000}, + {"charger offline", func(store *telemetry.Store, _ *State) { + store.DriverHealthMut("charger").SetOffline() + }, 0}, + {"no restart", func(_ *telemetry.Store, state *State) { + c := state.evLimit.offers["garage"] + c.found = sentCommand{} + state.evLimit.offers["garage"] = c + }, 0}, + {"first offer settled", func(_ *telemetry.Store, state *State) { + c := state.evLimit.offers["garage"] + c.found.at = state.now().Add(-c.settle) + state.evLimit.offers["garage"] = c + }, 0}, + {"a 40 s charger's window outlasts evSettle", func(_ *telemetry.Store, state *State) { + c := state.evLimit.offers["garage"] + c.settle, c.declared = 40*time.Second, true + c.found.at = state.now().Add(-30 * time.Second) + c.log[0].at = c.found.at + state.evLimit.offers["garage"] = c + }, 8222 - (11040 - 6210)}, + {"the shown part of the latest offer is not counted twice", func(_ *telemetry.Store, state *State) { + c := state.evLimit.offers["garage"] + c.found.at = state.now().Add(-60 * time.Second) + c.log = commandLog{c.found, {at: state.now().Add(-25 * time.Second), w: 6210, a: 9, n: 3}} + state.evLimit.offers["garage"] = c + }, 0}, + } { + t.Run(tc.name, func(t *testing.T) { + store, state := restartFixture(t, 780, [3]float64{2.03, 4.97, .47}, -5000, -3000, 8222) + tc.edit(store, state) + shownW := state.evShownW(state.now()) + meter := store.Get("meter", telemetry.DerMeter) + if got := state.evFoundDrawW(store, meter.RawW, meter.UpdatedAt, state.now(), shownW); math.Abs(got-tc.want) > 1e-6 { + t.Fatalf("credit %.0f W beside %.0f W shown, want %.0f W", got, shownW, tc.want) + } + }) + } +} + +// risenSince keeps the fixture charger's sample of fromW, arrived +// evMeterDelay before the site reading, as noteDraw does, and makes its +// current reading toW, stamped 1 s before the site reading but arriving +// with it. +func risenSince(store *telemetry.Store, state *State, fromW, toW float64) { + setChargerSample(store, fromW, nil) + c := state.evLimit.offers["garage"] + c.noteDraw(store, state.now()) + state.evLimit.offers["garage"] = c + meterAt := store.Get("meter", telemetry.DerMeter).UpdatedAt + setChargerSample(store, toW, map[string]any{"control_power_observed_at": meterAt.Add(-time.Second).Format(time.RFC3339Nano)}) + store.Get("charger", telemetry.DerEV).UpdatedAt = meterAt + // The filter's smoothed value trails the rise; this case reads the sample. + store.Get("charger", telemetry.DerEV).SmoothedW = toW +} + +// Offers the meter already shows come off the consumption the credit may +// fill: here another charger's 5 kW is shown, so of the 8,780 W the meter +// shows as consumption only 3,780 W is left for this car. +func TestEVFoundDrawWLeavesOutShownOffers(t *testing.T) { + store, state := restartFixture(t, 780, [3]float64{2.03, 4.97, .47}, -5000, -3000, 8222) + setRiseCapW(state, 6210) + meter := store.Get("meter", telemetry.DerMeter) + if got := state.evFoundDrawW(store, meter.RawW, meter.UpdatedAt, state.now(), 5000); math.Abs(got-3780) > 1e-6 { + t.Fatalf("credit %.0f W, want 3780 W", got) + } +} + +// A charger reading the meter may not show gives no credit. A fast charger +// applied the 0 A sent while the meter was silent, so the car draws +// nothing, but its reading still shows the 8.2 kW from before: held after a +// failed poll, or a sample older than the site reading (an OCPP +// MeterValues interval). The house draws 7 kW on balanced phases, the +// battery runs the -5 kW discharge Core ordered in this run, and the meter +// shows 2.9 A per phase. The 9 A order still to come leaves room to end +// only (16 - 2.9 - 9) × 690 W of the discharge. +func TestEVBaseAfterRestartNeedsACurrentChargerSample(t *testing.T) { + for _, tc := range []struct { + name string + edit func(*telemetry.Store, *State) + }{ + {"last poll failed", func(store *telemetry.Store, _ *State) { + store.RecordDriverError("charger", "timeout") + }}, + {"sample older than the site reading", func(store *telemetry.Store, state *State) { + data, _ := json.Marshal(map[string]any{"power_observed_at": state.now().Add(-10 * time.Second).Format(time.RFC3339Nano)}) + store.Update("charger", telemetry.DerEV, 8222, nil, data) + store.Get("charger", telemetry.DerEV).UpdatedAt = state.now() + store.RecordDriverSuccess("charger") + }}, + {"site reading older than the sample", func(store *telemetry.Store, state *State) { + store.Get("meter", telemetry.DerMeter).UpdatedAt = state.now().Add(-10 * time.Second) + }}, + } { + t.Run(tc.name, func(t *testing.T) { + store, state := restartFixture(t, 2000, [3]float64{2.9, 2.9, 2.9}, -5000, 0, 8222) + tc.edit(store, state) + out := clampEVBatteryHeadroom([]DispatchTarget{{Driver: "sungrow", TargetW: 0}}, store, state, 16*230*3) + got := targetW(out, "sungrow") + if afterA := 2.9 + (got+5000)/690 + 9; afterA > 16+1e-6 { + t.Fatalf("battery ends its discharge to %.0f W on a charger reading the meter may not show: %.2f A per phase once the car's order lands", got, afterA) + } + }) + } +} + +// The consumption bound reads the devices as the meter does. A battery +// that just ended a discharge, or PV that just dropped, still shows its old +// output in the smoothed value; counting that would raise the consumption +// the meter shows, and the credit with it, by the lag. +func TestEVBaseAfterRestartConsumptionReadsCurrentOutput(t *testing.T) { + for _, tc := range []struct { + name string + pvW float64 + driver string + kind telemetry.DerType + consumptionW float64 + }{ + {"battery ended its discharge", 0, "sungrow", telemetry.DerBattery, 680}, + {"PV dropped", -500, "pv", telemetry.DerPV, 1180}, + } { + t.Run(tc.name, func(t *testing.T) { + store, state := restartFixture(t, 680, [3]float64{4.97, -1, -1}, 0, tc.pvW, 8222) + store.Get(tc.driver, tc.kind).SmoothedW = -3000 + state.batteryCommands["sungrow"] = commandLog{}.add(state.now().Add(-time.Second), -5000) + out := clampEVBatteryHeadroom([]DispatchTarget{{Driver: "sungrow", TargetW: 0}}, store, state, 16*230*3) + riseA := (6210 - tc.consumptionW) / 690 + want := -(5000 - (16-4.97-riseA)*690) + if got := targetW(out, "sungrow"); math.Abs(got-want) > 1 { + t.Fatalf("battery %.0f W, want %.0f W: the meter shows %.0f W of consumption", got, want, tc.consumptionW) + } + }) + } +} + +// The consumption bound subtracts each output at the higher of raw and +// smoothed. A battery whose reading already shows a new 3 kW discharge, +// smoothed still at 1 kW, may be ahead of the site reading; so may PV. +func TestSiteConsumptionWTakesTheSmallerOutput(t *testing.T) { + store, _ := restartFixture(t, 680, [3]float64{4.97, -1, -1}, -3000, -2000, 8222) + store.Get("sungrow", telemetry.DerBattery).SmoothedW = -1000 + store.Get("pv", telemetry.DerPV).SmoothedW = -500 + if got := siteConsumptionW(store, 680); math.Abs(got-2180) > 1e-6 { + t.Fatalf("consumption %.0f W, want 2180 W: 680 W less -1,000 W battery and -500 W PV", got) + } +} + +// restartSim is seed 207532's shape on the plant: Core restarts beside a +// car drawing 12 A from before, on a charger that takes 40 s to apply an +// order. For its first 6 s the meter is silent and the inverter +// unreachable, holding the 5 kW discharge it had (C1), so Core logs 0 A +// offers. Then Core asks 9 A for the car. PV covers the house total, the +// plan is stale and battery_covers_ev is off. +func restartSim(house [3]float64) *simSite { + p := simPlant(simStart) + p.Fuse.RatingA, p.HouseW, p.PVW = 16, house, -6000 + p.EVSE.Delay, p.EVSE.ReadbackLag = 40*time.Second, 0 + p.EVSE.OfferA, p.EVSE.OfferPhases = 12, 3 + p.Battery = &sitesim.Battery{Phases: 3, MaxChargeW: 5000, MaxDischargeW: 5000, CapacityWh: 10000, SoC: .3, Latency: 2 * time.Second, W: -5000} + p.Events = []sitesim.Event{ + {Kind: sitesim.MeterSilent, For: 6 * time.Second}, + {Kind: sitesim.DriverOffline, Device: "battery", For: 6 * time.Second}, + } + s := &simSite{p: p, lp: simLoadpoint()} + s.setup = func(s *simSite) { + s.state.Mode = ModePlannerSelf + s.state.SiteFuseSafetyA = 0 + s.ctrl.SetManualHold(s.lp.ID, loadpoint.ManualHold{PowerW: 6210, PhaseMode: "3p", Persistent: true, StartedAt: s.p.Now, UpdatedAt: s.p.Now}) + } + return s +} + +// The seed on the plant: the battery ends its discharge as fast as its +// slew allows, down to what the car's rise to its 16 A max still needs on +// L2 (the 9 A of house load PV does not cover there), and no phase passes +// the fuse line on the way. +func TestSiteSimRestartEndsDischargeForACarAlreadyDrawing(t *testing.T) { + s := restartSim(houseA(4, 9, 2)) + s.boot() + s.run(14 * time.Second) + if s.offerW != 6210 || s.p.EVSE.DrawW != 8280 { + t.Fatalf("not the seed's state: offer %.0f W, car %.0f W", s.offerW, s.p.EVSE.DrawW) + } + if want := -(9*230 - 2000.0) * 3; s.batteryCmdW < want-1 { + t.Fatalf("battery held at %.0f W 8 s after the meter returned, with the car already in the reading; want %.0f W", s.batteryCmdW, want) + } + s.run(2 * time.Minute) + if s.failure != "" { + t.Fatal(s.failure) + } +} + +// The seed's charger is OCPP and samples every 10 s (MeterValues). Its +// sample counts only on the ticks just after it, so the battery holds its +// discharge between samples. It still ends it before the 9 A order counts +// as shown (evSettle after it went out, about 28 s in), and no phase +// passes the fuse line. A charger sampling every 30 s or more gets no +// credit in that time: the fix is partial for it. +func TestSiteSimRestartTenSecondSamples(t *testing.T) { + s := restartSim(houseA(4, 9, 2)) + s.evEvery = 10 * time.Second + s.boot() + s.run(20 * time.Second) + if s.offerW != 6210 || s.p.EVSE.DrawW != 8280 { + t.Fatalf("not the seed's state: offer %.0f W, car %.0f W", s.offerW, s.p.EVSE.DrawW) + } + if want := -(9*230 - 2000.0) * 3; s.batteryCmdW < want-1 { + t.Fatalf("battery held at %.0f W 14 s after the meter returned; want %.0f W", s.batteryCmdW, want) + } + s.run(2 * time.Minute) + if s.failure != "" { + t.Fatal(s.failure) + } +} + +// Review of #1926 on the plant: the charger's offer in force is 16 A, and +// the car limits itself to 12 A at the restart, then lifts that 14 s +// later, before the 40 s charger has applied Core's order. L2 carries 12 A +// of house. The battery keeps the discharge the car's rise needs: no phase +// reaches 1.2 x In, and time over In stays within the heat model. +func TestSiteSimRestartCarRisesUnderOfferInForce(t *testing.T) { + s := restartSim(houseA(4, 12, 2)) + carRisesUnderOldOffer(t, s, 8280, 14*time.Second) +} + +// carRisesUnderOldOffer runs s with the charger still on the 16 A offer it +// had before the restart, and a car that draws fromW until it lifts its own +// limit at riseAt. No phase may reach 1.2 x In, and the sim's fuse checks +// must pass. +func carRisesUnderOldOffer(t *testing.T, s *simSite, fromW float64, riseAt time.Duration) { + t.Helper() + s.p.EVSE.OfferA, s.p.Car.MaxW = 16, fromW + s.p.Events = append(s.p.Events, sitesim.Event{At: riseAt, Kind: sitesim.Change, Do: func(p *sitesim.Plant) { p.Car.MaxW = 11040 }}) + s.boot() + s.run(2 * time.Minute) + if s.failure != "" { + t.Fatal(s.failure) + } + if s.peakA >= fuseInstantFactor*16 { + t.Fatalf("a phase reached %.2f A", s.peakA) + } +} + +// Third review of #1926: a site meter that samples every 3 s. The car draws +// 10 A under the 16 A offer in force and lifts its own limit 11 s in. The +// charger reads the rise before the next meter sample, so its sample is +// newer than the site reading, which still shows the old draw. That sample +// does not count as in the reading, and the battery keeps the discharge +// the rise needs. Before: L2 reached 19.30 A. +func TestSiteSimRestartCarRisesBetweenMeterSamples(t *testing.T) { + s := restartSim(houseA(4, 12, 2)) + s.p.Meter.Every = 3 * time.Second + carRisesUnderOldOffer(t, s, 6900, 11*time.Second) +} + +// Third review of #1926: max_charge_w is not the most an offer in force +// may be. The owner lowered it to 7,360 W (10.67 A) and restarted; the +// charger still runs the 16 A Core sent before. The car draws 7 kW and may +// rise to 11,040 W before the 6.2 kW order lands. The rise reserve runs to +// the site fuse, so the battery keeps the discharge that rise needs. +func TestEVBaseAfterRestartMaxChargeLoweredAtRestart(t *testing.T) { + house := 3.0 + evW, batW := 7000.0, -5000.0 + gridW := house*690 + evW + batW + phaseA := gridW / 690 + store, state := restartFixture(t, gridW, [3]float64{phaseA, phaseA, phaseA}, batW, 0, evW) + setMaxW(state, 7360) + out := clampEVBatteryHeadroom([]DispatchTarget{{Driver: "sungrow", TargetW: 0}}, store, state, 16*230*3) + got := targetW(out, "sungrow") + if afterA := phaseA + (got-batW)/690 + (11040-evW)/690; afterA > 16+1e-6 { + t.Fatalf("battery ends discharge to %.0f W; the car rising to the old 16 A offer puts %.2f A on each phase", got, afterA) + } +} + +// The plant form: max_charge_w is now 8,280 W (12 A), and the car lifts +// its own 12 A limit 14 s in, before the 40 s charger applies Core's order. +// Before: L2 reached 19.30 A. +func TestSiteSimRestartMaxChargeLoweredAtRestart(t *testing.T) { + s := restartSim(houseA(4, 12, 2)) + s.lp.MaxChargeW, s.lp.AllowedStepsW = 8280, []float64{0} + for a := 6.0; a <= 12; a++ { + s.lp.AllowedStepsW = append(s.lp.AllowedStepsW, a*690) + } + carRisesUnderOldOffer(t, s, 8280, 14*time.Second) +} + +// Review of #1926: OCPP stamps MeterValues with the charger's own clock, +// often cut to the whole second. With the charger's clock 1-3 s behind +// Core's, the third review's case (3 s meter, the car at 10 A under a 16 A +// offer in force lifting its own limit 11 s in) stamps the rise before the +// site reading that does not show it yet. The draw stays the one sampled +// before that reading. Before: L2 reached 19.30 A. +func TestSiteSimRestartChargerClockBehind(t *testing.T) { + for _, lag := range []time.Duration{time.Second, 2 * time.Second, 3 * time.Second} { + t.Run(lag.String(), func(t *testing.T) { + s := restartSim(houseA(4, 12, 2)) + s.p.Meter.Every = 3 * time.Second + s.evClockLag = lag + carRisesUnderOldOffer(t, s, 6900, 11*time.Second) + }) + } +} + +// Final review of #1926: a sample from before the car stopped. A fast +// charger (2 s) applies the 0 A sent while the meter was silent at about +// 3 s, and the car resumes only at 11 s, after Core's 9 A order has +// landed. The charger samples every 10 s (OCPP MeterValues), at 2 s and +// 12 s. The readings from 6 s on show the car at 0 W. Core began to watch +// the charger at 7 s and does not know when the 8,280 W sample arrived, so +// it gives no credit, and the battery keeps the discharge the car's return +// needs. Judged by the charger's stamp against a 5 s allowance, that +// sample counted at 9 s, the battery ended its discharge, and L2 reached +// 21 A when the car resumed. With no credit at all, the peak is 18.10 A. +func TestSiteSimRestartOldSampleOfACarThatStopped(t *testing.T) { + s := restartSim(houseA(4, 12, 2)) + s.p.PVW = 0 + s.p.EVSE.Delay = 2 * time.Second + s.evEvery = 10 * time.Second + s.p.Events = append(s.p.Events, + sitesim.Event{At: 3 * time.Second, Kind: sitesim.Change, Do: func(p *sitesim.Plant) { p.Car.MaxW = 0 }}, + sitesim.Event{At: 11 * time.Second, Kind: sitesim.Change, Do: func(p *sitesim.Plant) { p.Car.MaxW = 11040 }}) + s.boot() + s.evAt = s.p.Now.Add(2*time.Second - s.evEvery) // samples at 2 s and 12 s + s.run(time.Minute) + if s.failure != "" { + t.Fatal(s.failure) + } + if s.peakA > 18.11 { + t.Fatalf("a phase reached %.2f A; with no credit it peaks at 18.10 A", s.peakA) + } +} + +// Review of #1926: a 22 kW charger (32 A) on a 3 x 25 A main fuse. The +// offer in force may be the charger's 32 A, so the rise reserve runs to +// max_charge_w, not to the fuse's rating: of the 14 kW the car draws, +// 22,080 - 11,000 W may still come on top of Core's latest 11 kW offer. +func TestEVFoundDrawWMaxChargeAboveFuse(t *testing.T) { + evW := 14000.0 + store, state := restartFixture(t, 9000, [3]float64{13, 13, 13}, -5000, 0, evW) + state.SiteFuseAmps = 25 + setMaxW(state, 22080) + c := state.evLimit.offers["garage"] + c.log[len(c.log)-1].w = 11000 + state.evLimit.offers["garage"] = c + meter := store.Get("meter", telemetry.DerMeter) + if got, want := state.evFoundDrawW(store, meter.RawW, meter.UpdatedAt, state.now(), 0), evW-(22080-11000); math.Abs(got-want) > 1e-6 { + t.Fatalf("credit %.0f W, want %.0f W", got, want) + } +} diff --git a/go/internal/control/ev_battery_fuse_hunt_sitesim_test.go b/go/internal/control/ev_battery_fuse_hunt_sitesim_test.go new file mode 100644 index 000000000..049d0a0a6 --- /dev/null +++ b/go/internal/control/ev_battery_fuse_hunt_sitesim_test.go @@ -0,0 +1,83 @@ +package control + +import ( + "fmt" + "math" + "math/rand/v2" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/sitesim" +) + +// The site rig's s18 at 16 A: a 3 A house, Charge now for a car whose only +// step is 6 A on three phases, and a 6 kW battery charge hold beside it. The +// battery may charge up to the fuse's working limit (fuseLineA) beside the +// car. Its fuse guard must then land it on that line and keep it there; a +// guard that cuts below the line and lets it rise again after its hold makes +// the battery hunt, and where it stands when the house moves is chance. The +// rig judges that move: 0.1 A more house must come off the battery. +func TestBatteryBesideMinimumCarHoldsTheFuseLine(t *testing.T) { + const carW = 6 * 690.0 + start := simStart + p := simPlant(start) + p.Fuse.RatingA, p.HouseW = 16, houseA(3, 3, 3) + p.EVSE.Delay, p.EVSE.ReadbackLag = 2*time.Second, 0 + p.Car.MaxW = carW + p.Battery = &sitesim.Battery{Phases: 3, MaxChargeW: 6000, MaxDischargeW: 6000, CapacityWh: 10000, SoC: .4, Latency: 2 * time.Second} + p.Events = []sitesim.Event{{At: 8 * time.Minute, Kind: sitesim.Change, Do: func(p *sitesim.Plant) { p.HouseW = houseA(3.1, 3.1, 3.1) }}} + lp := simLoadpoint() + lp.MinChargeW, lp.MaxChargeW, lp.AllowedStepsW = carW, carW, []float64{0, carW} + // The rig's car draws a few watts either side of its offer, so the meter + // reads the phases a hundredth of an amp either side of the line. + jitter := rand.New(rand.NewPCG(18, 16)) + s := &simSite{p: p, lp: lp, meterNoise: func() float64 { return (jitter.Float64() - .5) * .02 }, + setup: func(s *simSite) { + s.state.SetBatteryManualHold(BatteryManualHold{PowerW: 6000, ExpiresAt: start.Add(time.Hour)}) + }} + s.boot() + + // The battery each second, from 3 min, once car and battery run. + var battery []float64 + s.run(3 * time.Minute) + for s.p.Now.Before(start.Add(11*time.Minute)) && s.failure == "" { + s.step() + battery = append(battery, s.p.Battery.W) + } + if s.failure != "" { + t.Fatal(s.failure) + } + if s.p.EVSE.DrawW < carW-.1 { + t.Fatalf("car draws %.0f W, want its %.0f W step", s.p.EVSE.DrawW, carW) + } + // Five minutes of a steady house: the battery stays where it is. + steady := battery[:5*60] + lo, hi := math.Inf(1), math.Inf(-1) + for _, w := range steady { + lo, hi = math.Min(lo, w), math.Max(hi, w) + } + if hi-lo > 60 { + var moves []string + last := math.NaN() + for _, c := range s.commands { + if c.device == "battery" && c.at.After(start.Add(3*time.Minute)) && c.at.Before(start.Add(8*time.Minute)) && !(math.Abs(c.w-last) < 50) { + moves, last = append(moves, fmt.Sprintf("%s %.0f", c.at.Sub(start), c.w)), c.w + } + } + t.Errorf("battery hunts under a steady house: %.0f..%.0f W; commands %v", lo, hi, moves) + } + // The rig's judge: the 20 s before the step against the 20 s before 11 min. + mean := func(w []float64) float64 { + var sum float64 + for _, x := range w { + sum += x + } + return sum / float64(len(w)) + } + before, after := mean(battery[5*60-20:5*60]), mean(battery[len(battery)-20:]) + if after > before-40 { + t.Errorf("battery %.0f W before 0.1 A more house, %.0f W after; want at least 40 W less", before, after) + } + t.Logf("battery %.0f..%.0f W steady, %.0f -> %.0f W across the step; peak phase %.2f A", + lo, hi, before, after, s.peakA) +} diff --git a/go/internal/control/ev_battery_headroom.go b/go/internal/control/ev_battery_headroom.go index 86c5cb597..7098a2c74 100644 --- a/go/internal/control/ev_battery_headroom.go +++ b/go/internal/control/ev_battery_headroom.go @@ -7,20 +7,17 @@ import ( ) // clampEVBatteryHeadroom keeps optional battery charge out of room an EV -// still needs. It reads only the site meter and our own commands; no charger -// reading takes part, and a battery's own reading only bounds how much of a -// discharge we ordered it may keep. The EV still needs its demand less the -// lowest offer of the last evSettle, which the meter surely shows. +// still needs. It reads the site meter and our own commands. A battery's own +// reading only bounds how much of a discharge we ordered it may keep. A +// charger's reading counts only before any of its offers of this run has +// surely landed, and only up to what the meter shows (evFoundDrawW). The EV +// still needs its demand less the offers the meter surely shows (evShownW) +// and that draw. func clampEVBatteryHeadroom(targets []DispatchTarget, store *telemetry.Store, state *State, fuseMaxW float64) []DispatchTarget { if store == nil || state == nil || len(targets) == 0 || fuseMaxW <= 0 { return targets } now := state.now() - // This tick's new offer is not in the meter yet; the lowest logged one is. - offeredW, _, _, ok := state.evOffers.span(now) - if !ok { - offeredW = 0 - } var demandW float64 phases := 3.0 // No phase placement promise for 1p or auto chargers. for _, demand := range state.evFuseDemands { @@ -32,7 +29,6 @@ func clampEVBatteryHeadroom(targets []DispatchTarget, store *telemetry.Store, st if len(state.evFuseDemands) == 0 { return targets } - needW := math.Max(0, demandW-offeredW) meter := store.Get(state.SiteMeterDriver, telemetry.DerMeter) if meter == nil { return floorPositiveEVBatteryTargets(targets) @@ -41,6 +37,9 @@ func clampEVBatteryHeadroom(targets []DispatchTarget, store *telemetry.Store, st if !finiteFuseValue(gridW) || !finiteFuseValue(fuseMaxW) { return floorPositiveEVBatteryTargets(targets) } + offeredW := state.evShownW(now) + offeredW += state.evFoundDrawW(store, gridW, meter.UpdatedAt, now, offeredW) + needW := math.Max(0, demandW-offeredW) latest := make(map[string]float64, len(targets)) lowest := make(map[string]float64, len(targets)) targeted := make(map[string]bool, len(targets)) @@ -175,8 +174,8 @@ func evBatteryPhaseSpareW(meter *telemetry.DerReading, state *State, gridW, evIn if !valid { return math.Inf(-1), true } - margin := math.Max(1, state.SiteFuseSafetyA) // Same working margin as the EV phase guard. - return (state.SiteFuseAmps - margin - worst - evIncreaseA) * state.SiteFuseVoltage * float64(state.SiteFusePhases), true + // The same working limit as the EV phase guard, with no floor under it. + return (state.fuseLineA() - worst - evIncreaseA) * state.SiteFuseVoltage * float64(state.SiteFusePhases), true } func floorPositiveEVBatteryTargets(targets []DispatchTarget) []DispatchTarget { diff --git a/go/internal/control/ev_battery_headroom_test.go b/go/internal/control/ev_battery_headroom_test.go index 9fafb4872..32b4af518 100644 --- a/go/internal/control/ev_battery_headroom_test.go +++ b/go/internal/control/ev_battery_headroom_test.go @@ -47,7 +47,8 @@ func sent(state *State, driver string, watts ...float64) { log = log.add(at, w) } if driver == "ev" { - state.evOffers, state.EVCommandedW = log, watts[len(watts)-1] + state.evLimit.offers = map[string]chargerOffers{"car": {log: log, settle: evSettle}} + state.EVCommandedW = watts[len(watts)-1] return } state.batteryCommands[driver] = log diff --git a/go/internal/control/ev_battery_priority.go b/go/internal/control/ev_battery_priority.go new file mode 100644 index 000000000..c585c92e4 --- /dev/null +++ b/go/internal/control/ev_battery_priority.go @@ -0,0 +1,101 @@ +package control + +import ( + "log/slog" + "math" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// evBatteryYield lets an existing car offer wait for a battery charge cut. +// It never grants a car rise, never runs above the fuse's working limit +// (fuseLineA), and cannot renew while the site remains over its working limit. +type evBatteryYield struct { + tried bool + until time.Time + active bool + maxChargeW float64 +} + +func (s *State) hasBatteryCharge() bool { + for _, log := range s.batteryCommands { + if _, _, latest, ok := log.span(s.now()); ok && latest > 0 { + return true + } + } + return false +} + +func (s *State) yieldBatteryBeforeEVCut(store *telemetry.Store, meter *telemetry.DerReading, r evMeterRoom, pool evPool, requiredW, gridW, pendingBatteryW float64, rise pendingLoad, fresh bool) bool { + y := &s.evLimit.batteryYield + if !fresh || !r.perPhase || pool.offerW <= 0 || requiredW <= 0 || s.SiteFuseVoltage <= 0 || s.SiteFusePhases <= 0 { + return false + } + // This hold uses only the installation's existing current allowance. + // Include offers still on their way; a cut never funds a car increase. + amps, present, valid := phaseCurrentsA(meter, s, gridW) + lineA := s.fuseLineA() + if !present || !valid || gridW+rise.w+pendingBatteryW > lineA*s.SiteFuseVoltage*float64(s.SiteFusePhases) { + return false + } + pendingA := pendingBatteryW / (s.SiteFuseVoltage * float64(s.SiteFusePhases)) + for i, a := range amps { + if math.Max(a, math.Sqrt(s.fuseHeat.sq[i]))+rise.phaseA[i]+pendingA > lineA { + return false + } + } + var available, commanded float64 + for driver, log := range s.batteryCommands { + h := store.DriverHealth(driver) + if h == nil || !h.IsOnline() { + continue + } + _, high, latest, ok := log.span(s.now()) + if ok && finiteFuseValue(high) && finiteFuseValue(latest) { + available += math.Max(0, high) + commanded += math.Max(0, latest) + } + } + if available+1e-6 < requiredW { + return false + } + // Removing balanced charge must not overload an exporting phase. + cutA := requiredW / (s.SiteFuseVoltage * float64(s.SiteFusePhases)) + for _, a := range amps { + if a-cutA < -s.SiteFuseAmps { + return false + } + } + if !y.tried { + y.tried, y.until = true, s.now().Add(evSettle) + slog.Info("EV priority: battery charging yields before the car", + "battery_reduction_w", requiredW, "car_offer_w", pool.offerW, "until", y.until) + } + if !s.now().Before(y.until) { + return false + } + y.active, y.maxChargeW = true, math.Max(0, commanded-requiredW) + return true +} + +func (s *State) applyEVBatteryYield(targets []DispatchTarget) []DispatchTarget { + if !s.evLimit.batteryYield.active || !s.now().Before(s.evLimit.batteryYield.until) { + return targets + } + var chargeW float64 + for _, target := range targets { + chargeW += math.Max(0, target.TargetW) + } + capW := s.evLimit.batteryYield.maxChargeW + if chargeW <= capW || chargeW <= 0 { + return targets + } + for i := range targets { + if targets[i].TargetW > 0 { + targets[i].TargetW *= capW / chargeW + targets[i].Clamped = true + } + } + return targets +} diff --git a/go/internal/control/ev_battery_priority_test.go b/go/internal/control/ev_battery_priority_test.go new file mode 100644 index 000000000..6564016ea --- /dev/null +++ b/go/internal/control/ev_battery_priority_test.go @@ -0,0 +1,156 @@ +package control + +import ( + "fmt" + "testing" + "testing/synctest" + "time" + + "github.com/srcfl/ftw/go/internal/loadpoint" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// #1868: 16 A, 6 A on all three car phases, and 15.1 A on L1, then 0.1 A +// over the fuse less 1 A. The phase line is now the fuse less +// safety_margin_a, 15.5 A here, so the same readings leave 0.4 A of room +// (TestIssue1868ReadingsSitUnderTheLine). The car may wait for the battery +// only while every phase stays under that line, so the wait now serves the +// hour's import limit: the fixture is 69 W over a 9,015 W limit. The 4 kW +// battery is the continuous load that can yield first. +const issue1868CeilingW = 9015 + +func issue1868Site(t *testing.T) (*telemetry.Store, *State, []loadpoint.State, []loadpoint.Config) { + t.Helper() + tel, st := evSplitFixture(t, 9084, []float64{15.1, 12.5, 11.9}, 4140, "3p") + st.SiteFuseAmps, st.SiteFuseSafetyA = 16, .5 + st.PeakImportCeilingW = issue1868CeilingW + st.Mode = ModeCharge + st.DriverLimits = map[string]PowerLimits{"battery": {MaxChargeW: 6000, MaxDischargeW: 6000}} + soc := .4 + tel.Update("battery", telemetry.DerBattery, 4000, &soc, nil) + tel.RecordDriverSuccess("battery") + sent(st, "ev", 4140) + // The car's 4140 W offer has run for longer than any charger's response time. + a, n := st.offerPhaseA(4140, 3) + st.evLimit.offers = map[string]chargerOffers{"rav4": {log: commandLog{{at: st.now().Add(-evUnknownSettle - time.Second), w: 4140, a: a, n: n}}}} + sent(st, "battery", 4000) + lastingFuseHeat(tel, st) // The overage has lasted, as in the field. + points := []loadpoint.State{{ID: "rav4", DriverName: "car", PluggedIn: true, CommandedKnown: true, CommandedW: 4140, CommandedPhases: 3}} + configs := []loadpoint.Config{{ID: "rav4", DriverName: "car", PhaseMode: "3p", MinChargeW: 4140, MaxChargeW: 4140, AllowedStepsW: []float64{0, 4140}}} + return tel, st, points, configs +} + +func TestIssue1868BatteryChargeYieldsBeforeTheCar(t *testing.T) { + for _, mode := range AllModes() { + t.Run(string(mode), func(t *testing.T) { + tel, st, points, configs := issue1868Site(t) + cap, known := EVFuseBudget(tel, st, points, configs, 11040) + if !known || cap != 4140 { + t.Fatalf("71 W relief shed the 4140 W car: cap=%v known=%v", cap, known) + } + st.SetEVCommanded(points) + st.Mode = mode + st.SetEVFuseDemands([]loadpoint.FuseDemand{{DriverName: "car", PowerW: 4140, PhaseMode: "3p"}}) + targets := ComputeDispatch(tel, st, map[string]float64{"battery": 12200}, 11040) + if len(targets) != 1 { + t.Fatalf("missing battery command: %+v", targets) + } + for _, target := range targets { + if target.TargetW > 3931.001 { + t.Fatalf("%s left the car's relief in the battery: %+v", mode, target) + } + // Charge mode asks for a raise. No phase reaches the 15.5 A + // line, so only the hour's limit binds: the guard cuts the + // 69 W overage to 3931 W, and the car's headroom then takes + // the same 69 W again (3862 W). + if mode == ModeCharge && target.TargetW < 3861 { + t.Fatalf("small overage needlessly stopped battery charging: %+v", target) + } + } + for _, target := range SelfTuneDispatch(tel, st, map[string]float64{"battery": 12200}, 11040, "battery", 6000) { + if target.TargetW > 3931.001 { + t.Fatalf("self-tune took the car's relief: %+v", target) + } + } + }) + } +} + +// With no import limit, #1868's readings sit 0.4 A under the 15.5 A line: +// nothing cuts the car and nothing waits for the battery. +func TestIssue1868ReadingsSitUnderTheLine(t *testing.T) { + tel, st, points, configs := issue1868Site(t) + st.PeakImportCeilingW = 0 + cap, known := EVFuseBudget(tel, st, points, configs, 11040) + if !known || cap < 4140 || st.evLimit.batteryYield.active { + t.Fatalf("cap=%v known=%v yield=%+v, want the 4140 W offer kept with no wait", cap, known, st.evLimit.batteryYield) + } +} + +func TestEVPriorityCannotSpendUnmeasuredBatteryRelief(t *testing.T) { + tel, st, points, configs := issue1868Site(t) + st.SetEVFuseDemands([]loadpoint.FuseDemand{{DriverName: "car", PowerW: 10350, PhaseMode: "3p"}}) + cap, _ := EVFuseBudget(tel, st, points, configs, 11040) + if cap > points[0].CommandedW { + t.Fatalf("battery cut became an unmeasured car rise: %v W", cap) + } +} + +func TestEVPriorityFallsBackWhenTheBatteryCannotYield(t *testing.T) { + for _, tc := range []struct { + name string + set func(*telemetry.Store, *State) + }{ + {"no command", func(_ *telemetry.Store, st *State) { st.batteryCommands = nil }}, + {"not charging", func(_ *telemetry.Store, st *State) { sent(st, "battery", 0) }}, + {"insufficient charge", func(_ *telemetry.Store, st *State) { sent(st, "battery", 10) }}, + {"offline", func(tel *telemetry.Store, _ *State) { tel.Remove("battery") }}, + {"stale meter", func(tel *telemetry.Store, _ *State) { + tel.Get("meter", telemetry.DerMeter).UpdatedAt = time.Now().Add(-time.Minute) + }}, + {"over the working limit", func(tel *telemetry.Store, st *State) { + tel.Update("meter", telemetry.DerMeter, 9000, nil, []byte(`{"l1_a":15.6,"l2_a":12.5,"l3_a":11.9}`)) + lastingFuseHeat(tel, st) // the overage has lasted (I²t) + }}, + {"unseen battery rise", func(_ *telemetry.Store, st *State) { sent(st, "battery", 0, 4000) }}, + } { + t.Run(tc.name, func(t *testing.T) { + tel, st, points, configs := issue1868Site(t) + tc.set(tel, st) + cap, _ := EVFuseBudget(tel, st, points, configs, 11040) + if cap >= 4140 || st.evLimit.batteryYield.active { + t.Fatalf("unsafe battery priority: cap=%v yield=%+v", cap, st.evLimit.batteryYield) + } + }) + } +} + +func TestEVPriorityWaitForBatteryHasAFixedDeadline(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + tel, st, points, configs := issue1868Site(t) + var deadline time.Time + for tick := 0; tick <= 6; tick++ { + // The battery accepts lower commands, but the site meter never + // shows the reduction. Repeated commands cannot renew the wait. + cap, _ := EVFuseBudget(tel, st, points, configs, 11040) + if tick == 0 { + deadline = st.evLimit.batteryYield.until + } + st.SetEVCommanded(points) + if tick < 4 && cap != 4140 { + t.Fatalf("tick %d: car cut before the response wait: %v", tick, cap) + } + if tick >= 4 && cap >= 4140 { + t.Fatalf("tick %d: ignored battery cut holds the car forever: %v", tick, cap) + } + if st.evLimit.batteryYield.until != deadline { + t.Fatalf("tick %d renewed the deadline", tick) + } + _ = ComputeDispatch(tel, st, map[string]float64{"battery": 12200}, 11040) + time.Sleep(5 * time.Second) + tel.Update("meter", telemetry.DerMeter, 9084, nil, []byte(`{"l1_a":15.1,"l2_a":12.5,"l3_a":11.9}`)) + tel.RecordDriverSuccess("meter") + } + t.Logf("car protected while battery responded; fallback deadline %s", fmt.Sprint(evSettle)) + }) +} diff --git a/go/internal/control/ev_battery_sequence_test.go b/go/internal/control/ev_battery_sequence_test.go index 3d3379b58..823538df4 100644 --- a/go/internal/control/ev_battery_sequence_test.go +++ b/go/internal/control/ev_battery_sequence_test.go @@ -3,6 +3,7 @@ package control import ( "context" "encoding/json" + "fmt" "math" "math/rand" "testing" @@ -13,94 +14,98 @@ import ( ) func TestJoinedEVWaitsForMeasuredBatteryBackoffThenResumes(t *testing.T) { - now := time.Now().Truncate(time.Second) - const houseW, desiredEVW = 5000.0, 10350.0 - actualBattery, actualEV := 8000.0, 0.0 - config := loadpoint.Config{ID: "garage", DriverName: "charger", MinChargeW: 4140, MaxChargeW: desiredEVW, - PhaseMode: "3p", AllowedStepsW: []float64{0, 4140, 4830, 5520, 6210, 6900, 7590, 8280, 8970, 9660, 10350}} - mgr := loadpoint.NewManager() - mgr.Load([]loadpoint.Config{config}) - store := telemetry.NewStore() - dir := SlotDirective{SlotStart: now, SlotEnd: now.Add(15 * time.Minute), BatteryEnergyWh: 2500, Strategy: "arbitrage"} - st := newStateWithEnergyDispatch(dir, "meter") - st.clock = func() time.Time { return now } - st.SiteFuseAmps, st.SiteFuseVoltage, st.SiteFusePhases, st.SiteFuseSafetyA = 25, 230, 3, 1 - st.SlewEnabled = false - st.DriverLimits = map[string]PowerLimits{"pixii": {MaxChargeW: 10000, MaxDischargeW: 10000}} - var sampledPhaseA float64 - controller := loadpoint.NewController(mgr, - func(time.Time) (loadpoint.Directive, bool) { return loadpoint.Directive{}, false }, - func(string) (loadpoint.EVSample, bool) { - return loadpoint.EVSample{Connected: true, RequestActive: true, PowerW: actualEV, PowerAt: now, SessionWhUnavailable: true}, true - }, - func(_ context.Context, _ string, payload []byte) error { - var cmd struct { - Action string `json:"action"` - PowerW float64 `json:"power_w"` - } - if err := json.Unmarshal(payload, &cmd); err != nil { - return err + for _, desiredEVW := range []float64{4140, 10350} { + t.Run(fmt.Sprintf("%.0fW", desiredEVW), func(t *testing.T) { + now := time.Now().Truncate(time.Second) + const houseW = 5000.0 + actualBattery, actualEV := 8000.0, 0.0 + config := loadpoint.Config{ID: "garage", DriverName: "charger", MinChargeW: 4140, MaxChargeW: desiredEVW, + PhaseMode: "3p", AllowedStepsW: []float64{0, 4140, 4830, 5520, 6210, 6900, 7590, 8280, 8970, 9660, 10350}} + mgr := loadpoint.NewManager() + mgr.Load([]loadpoint.Config{config}) + store := telemetry.NewStore() + dir := SlotDirective{SlotStart: now, SlotEnd: now.Add(15 * time.Minute), BatteryEnergyWh: 2500, Strategy: "arbitrage"} + st := newStateWithEnergyDispatch(dir, "meter") + st.clock = func() time.Time { return now } + st.SiteFuseAmps, st.SiteFuseVoltage, st.SiteFusePhases, st.SiteFuseSafetyA = 25, 230, 3, 1 + st.SlewEnabled = false + st.DriverLimits = map[string]PowerLimits{"pixii": {MaxChargeW: 10000, MaxDischargeW: 10000}} + var sampledPhaseA float64 + controller := loadpoint.NewController(mgr, + func(time.Time) (loadpoint.Directive, bool) { return loadpoint.Directive{}, false }, + func(string) (loadpoint.EVSample, bool) { + return loadpoint.EVSample{Connected: true, RequestActive: true, PowerW: actualEV, PowerAt: now, SessionWhUnavailable: true}, true + }, + func(_ context.Context, _ string, payload []byte) error { + var cmd struct { + Action string `json:"action"` + PowerW float64 `json:"power_w"` + } + if err := json.Unmarshal(payload, &cmd); err != nil { + return err + } + if cmd.Action == "ev_set_current" { + actualEV = cmd.PowerW + // The physical battery may ignore a reduction. Check this at + // the EV send, before a later battery command can conceal it. + if phase := (houseW + actualBattery + actualEV) / 690; phase > 24.001 { + t.Fatalf("EV spent unmeasured battery relief: %.1f A", phase) + } + } + return nil + }) + controller.SetSiteFuse(loadpoint.SiteFuse{MaxAmps: 25, Voltage: 230, PhaseCnt: 3}) + controller.SetManualHold(config.ID, loadpoint.ManualHold{PowerW: desiredEVW, PhaseMode: "3p", Persistent: true}) + controller.SetFuseEVMax(func() (float64, bool) { return EVFuseBudget(store, st, mgr.States(), controller.FuseConfigs(), 17250) }) + controller.SetFusePhaseRoomA(func() (float64, bool) { return EVFusePhaseRoomA(st) }) + emit := func(driver string, kind telemetry.DerType, watts float64) { + d := map[string]any{"control_power_available": true, "control_power_observed_at": now.Format(time.RFC3339Nano)} + if kind == telemetry.DerMeter { + d["l1_a"], d["l2_a"], d["l3_a"] = sampledPhaseA, sampledPhaseA, sampledPhaseA + } + if kind == telemetry.DerEV { + d["power_max_age_s"] = 5 // A local charger that answers within seconds. + } + data, _ := json.Marshal(d) + soc := .5 + store.Update(driver, kind, watts, &soc, data) + store.Get(driver, kind).UpdatedAt = now + store.RecordDriverSuccess(driver) } - if cmd.Action == "ev_set_current" { - actualEV = cmd.PowerW - // The physical battery may ignore a reduction. Check this at - // the EV send, before a later battery command can conceal it. - if phase := (houseW + actualBattery + actualEV) / 690; phase > 24.001 { - t.Fatalf("EV spent unmeasured battery relief: %.1f A", phase) + var obeyedAt, resumedAt time.Time + for tick := 0; tick < 30; tick++ { + now = now.Add(5 * time.Second) + if tick == 3 { + actualBattery, obeyedAt = 0, now // Only now does the battery obey. + } + sampledPhaseA = (houseW + actualBattery + actualEV) / 690 + emit("pixii", telemetry.DerBattery, actualBattery) + emit(config.DriverName, telemetry.DerEV, actualEV) + emit("meter", telemetry.DerMeter, houseW+actualBattery+actualEV) + controller.TickWithDispatch(context.Background(), now, true) + st.SetEVCommanded(mgr.States()) + st.SetEVFuseDemands(controller.FuseDemands()) + targets := ComputeDispatch(store, st, caps(map[string]float64{"pixii": 15000}), 17250) + if len(targets) != 1 || targets[0].TargetW > 16560-houseW-desiredEVW+.001 { + t.Fatalf("tick %d: battery did not yield to the EV: %+v", tick, targets) + } + if actualEV > 0 && resumedAt.IsZero() { + resumedAt = now + } + if !obeyedAt.IsZero() { + actualBattery = math.Max(0, targets[0].TargetW) + } + if (houseW+actualBattery+actualEV)/690 > 24.001 { + t.Fatalf("tick %d: battery took the EV's headroom: EV %.0f battery %.0f W", tick, actualEV, actualBattery) } } - return nil + if resumedAt.IsZero() || resumedAt.Sub(obeyedAt) < evRestartDwell { + t.Fatalf("EV resumed at %v, battery obeyed at %v: want a %v dwell on measured room", resumedAt, obeyedAt, evRestartDwell) + } + if actualEV < desiredEVW { + t.Fatalf("EV stuck at %.0f W after recovery", actualEV) + } }) - controller.SetSiteFuse(loadpoint.SiteFuse{MaxAmps: 25, Voltage: 230, PhaseCnt: 3}) - controller.SetManualHold(config.ID, loadpoint.ManualHold{PowerW: desiredEVW, PhaseMode: "3p", Persistent: true}) - controller.SetFuseEVMax(func() (float64, bool) { return EVFuseBudget(store, st, mgr.States(), controller.FuseConfigs(), 17250) }) - controller.SetFusePhaseRoomA(func() (float64, bool) { return EVFusePhaseRoomA(st) }) - emit := func(driver string, kind telemetry.DerType, watts float64) { - d := map[string]any{"control_power_available": true, "control_power_observed_at": now.Format(time.RFC3339Nano)} - if kind == telemetry.DerMeter { - d["l1_a"], d["l2_a"], d["l3_a"] = sampledPhaseA, sampledPhaseA, sampledPhaseA - } - if kind == telemetry.DerEV { - d["power_max_age_s"] = 5 // A local charger that answers within seconds. - } - data, _ := json.Marshal(d) - soc := .5 - store.Update(driver, kind, watts, &soc, data) - store.Get(driver, kind).UpdatedAt = now - store.RecordDriverSuccess(driver) - } - var obeyedAt, resumedAt time.Time - for tick := 0; tick < 30; tick++ { - now = now.Add(5 * time.Second) - if tick == 3 { - actualBattery, obeyedAt = 0, now // Only now does the battery obey. - } - sampledPhaseA = (houseW + actualBattery + actualEV) / 690 - emit("pixii", telemetry.DerBattery, actualBattery) - emit(config.DriverName, telemetry.DerEV, actualEV) - emit("meter", telemetry.DerMeter, houseW+actualBattery+actualEV) - controller.TickWithDispatch(context.Background(), now, true) - st.SetEVCommanded(mgr.States()) - st.SetEVFuseDemands(controller.FuseDemands()) - targets := ComputeDispatch(store, st, caps(map[string]float64{"pixii": 15000}), 17250) - if len(targets) != 1 || targets[0].TargetW > 1210.001 { - t.Fatalf("tick %d: battery did not yield to the EV: %+v", tick, targets) - } - if actualEV > 0 && resumedAt.IsZero() { - resumedAt = now - } - if !obeyedAt.IsZero() { - actualBattery = math.Max(0, targets[0].TargetW) - } - if (houseW+actualBattery+actualEV)/690 > 24.001 { - t.Fatalf("tick %d: battery took the EV's headroom: EV %.0f battery %.0f W", tick, actualEV, actualBattery) - } - } - if resumedAt.IsZero() || resumedAt.Sub(obeyedAt) < evRestartDwell { - t.Fatalf("EV resumed at %v, battery obeyed at %v: want a %v dwell on measured room", resumedAt, obeyedAt, evRestartDwell) - } - if actualEV < 9660 { - t.Fatalf("EV stuck at %.0f W after recovery", actualEV) } } @@ -178,6 +183,7 @@ type evBatterySite struct { batteryDelay time.Duration // 0: the battery follows within a tick batteryCommands []evTimed meterW, targetW []evTimed // every site reading and battery target + batteryOffline bool // the battery reports nothing and gets no target } func newEVBatterySite(t *testing.T, houseW float64) *evBatterySite { @@ -254,7 +260,9 @@ func (s *evBatterySite) step() { } s.maxGridW = math.Max(s.maxGridW, s.gridW()) s.meterW = append(s.meterW, evTimed{s.now, s.gridW()}) - s.emit("pixii", telemetry.DerBattery, s.batteryW) + if !s.batteryOffline { + s.emit("pixii", telemetry.DerBattery, s.batteryW) + } s.emit("charger", telemetry.DerEV, s.evW) s.emit("meter", telemetry.DerMeter, s.gridW()) s.state.ObserveSiteImport(s.store) @@ -262,6 +270,12 @@ func (s *evBatterySite) step() { s.state.SetEVCommanded(s.mgr.States()) s.state.SetEVFuseDemands(s.ctrl.FuseDemands()) targets := ComputeDispatch(s.store, s.state, caps(map[string]float64{"pixii": 15000}), 17250) + if s.batteryOffline { + if len(targets) != 0 { + s.t.Fatalf("offline battery got targets %+v", targets) + } + return + } if len(targets) != 1 { s.t.Fatalf("targets %+v", targets) } @@ -516,6 +530,180 @@ func TestFuseOverageGetsBatteryReliefUntilEVCutShows(t *testing.T) { if s.offer() >= 10350 || s.batteryW != 0 { t.Fatalf("after the cut: offer %.0f W battery %.0f W, want a cut car and an idle battery", s.offer(), s.batteryW) } + // The car's rise never counts the battery's relief as room. A car that + // rose into it kept the battery feeding it 650 W at the 24 A line for + // as long as the house load stayed. + for range 10 { + s.run(time.Minute) + if s.batteryW != 0 || s.gridW()/690 > 24+1e-6 { + t.Fatalf("%v: offer %.0f W battery %.0f W, %.2f A on each phase; want an idle battery under the 24 A line", + s.now.Sub(stepAt), s.offer(), s.batteryW, s.gridW()/690) + } + } + if s.offer() != 5520 { + t.Fatalf("offer %.0f W, want 5,520 W: 8 A beside 16 A of house under the 24 A line", s.offer()) + } +} + +// The only battery goes offline in the middle of fuse relief and drops to +// 0 W; then the house load falls. Its relief counts whole until the default +// mode Core sent it can have reached the meter, and then it ends: the car +// gets back up to the 24 A line instead of being held below it for as long +// as the battery stays offline. +func TestOfflineBatteryReliefEndsAfterSettle(t *testing.T) { + s := newIdleBatteryEVSite(t, 3000, 0) + s.run(5 * time.Minute) + stepAt := s.now + s.houseW += 8000 + s.run(3 * time.Second) + reliefW := -s.batteryW + if reliefW <= 0 || s.state.batteryFuseRelief["pixii"] <= 0 { + t.Fatalf("setup: battery %.0f W relief %v", s.batteryW, s.state.batteryFuseRelief) + } + s.store.DriverHealthMut("pixii").SetOffline() + s.batteryOffline, s.batteryW = true, 0 + offlineAt := s.now + for s.now.Sub(offlineAt) < evSettle-time.Second { + s.step() + if got := s.state.evFuseReliefW(s.store); math.Abs(got-reliefW) > 1e-6 { + t.Fatalf("%v after offline: relief %.0f W, want the whole %.0f W", s.now.Sub(offlineAt), got, reliefW) + } + } + s.run(time.Minute - s.now.Sub(stepAt)) + s.houseW -= 4000 // 7,000 W of house. + s.run(9 * time.Minute) + t.Logf("offer %.0f W, %.2f A on each phase, relief %v", s.offer(), s.gridW()/690, s.state.batteryFuseRelief) + if got := s.state.evFuseReliefW(s.store); got != 0 { + t.Fatalf("relief %.0f W still counted %v after offline", got, s.now.Sub(offlineAt)) + } + if s.offer() != 8970 || s.gridW()/690 > 24+1e-6 { + t.Fatalf("offer %.0f W at %.2f A, want 8,970 W: 13 A beside 10.1 A of house under the 24 A line", s.offer(), s.gridW()/690) + } +} + +// Two batteries: "b2" goes offline holding 1500 W of fuse relief while +// "bat" stays online. Its relief counts whole for evSettle and then ends, +// whether dispatch sits in the deadband and sends nothing or sends "bat" a +// command every tick. Before, a quiet dispatch kept it for as long as the site +// stayed quiet, and a command dropped it on the first tick. +func TestOfflineBatteryReliefEndsBesideAnOnlineBattery(t *testing.T) { + for _, tc := range []struct { + name string + gridW float64 + }{ + {"deadband", 0}, + {"commanded", 3000}, + } { + t.Run(tc.name, func(t *testing.T) { + store, state, caps := setupPerPhase(tc.gridW, 6, 6, 6, 0.6, 10000) + soc := 0.6 + store.Update("b2", telemetry.DerBattery, -1500, &soc, nil) + caps["b2"] = 15200 + state.DriverLimits["b2"] = PowerLimits{MaxChargeW: 10000, MaxDischargeW: 10000} + state.Mode = ModeSelfConsumption + now := time.Now() + state.clock = func() time.Time { return now } + state.batteryFuseRelief = map[string]float64{"b2": 1500} + store.DriverHealthMut("b2").SetOffline() + commands := 0 + for start := now; now.Sub(start) < 2*time.Minute; { + now = now.Add(time.Second) + store.Update("meter", telemetry.DerMeter, tc.gridW, nil, []byte(`{"l1_a":6,"l2_a":6,"l3_a":6}`)) + store.Get("meter", telemetry.DerMeter).UpdatedAt = now + store.DriverHealthMut("meter").RecordSuccess() + store.Update("bat", telemetry.DerBattery, 0, &soc, nil) + store.Get("bat", telemetry.DerBattery).UpdatedAt = now + store.DriverHealthMut("bat").RecordSuccess() + if len(ComputeDispatch(store, state, caps, 11040)) > 0 { + commands++ + } + // The first dispatch, a second in, finds b2 offline. + want := 0.0 + if now.Sub(start) <= evSettle { + want = 1500 + } + if got := state.evFuseReliefW(store); got != want { + t.Fatalf("%v offline, %d commands: relief %.0f W, want %.0f W (%v)", now.Sub(start), commands, got, want, state.batteryFuseRelief) + } + } + if (commands > 0) != (tc.gridW != 0) { + t.Fatalf("setup: %d dispatches with a command", commands) + } + }) + } +} + +// The battery goes offline in relief, its relief ends, it comes back and +// takes new relief, and it goes offline again. The second offline period +// counts the new relief whole for evSettle: Core ordered that discharge a +// second ago, and the old offline clock must not end it at once. +func TestReturningBatteryReliefCountsWholeAgain(t *testing.T) { + s := newIdleBatteryEVSite(t, 3000, 0) + s.run(5 * time.Minute) + s.houseW += 8000 + s.run(3 * time.Second) + if s.state.batteryFuseRelief["pixii"] <= 0 { + t.Fatalf("setup: no relief %v", s.state.batteryFuseRelief) + } + s.store.DriverHealthMut("pixii").SetOffline() + s.batteryOffline, s.batteryW = true, 0 + s.run(2 * time.Minute) + if got := s.state.evFuseReliefW(s.store); got != 0 { + t.Fatalf("first offline period: relief %.0f W still counted", got) + } + s.batteryOffline = false + s.run(2 * time.Minute) + s.houseW += 2000 + s.run(3 * time.Second) + reliefW := s.state.batteryFuseRelief["pixii"] + if reliefW <= 0 { + t.Fatalf("setup: no new relief, battery %.0f W relief %v", s.batteryW, s.state.batteryFuseRelief) + } + s.store.DriverHealthMut("pixii").SetOffline() + s.batteryOffline = true + offlineAt := s.now + for s.now.Sub(offlineAt) < evSettle-time.Second { + s.step() + if got := s.state.evFuseReliefW(s.store); math.Abs(got-reliefW) > 1e-6 { + t.Fatalf("%v into the second offline period: relief %.0f W, want the whole %.0f W", s.now.Sub(offlineAt), got, reliefW) + } + } +} + +// Relief is only the discharge the fuse guard added: a cancelled charge or +// the planner's own discharge stays room for the car. +func TestFuseReliefIsOnlyAddedDischarge(t *testing.T) { + s := &State{} + s.recordFuseRelief( + []DispatchTarget{{Driver: "idle", TargetW: 0}, {Driver: "planned", TargetW: -1000}, {Driver: "charge", TargetW: 2000}, {Driver: "flip", TargetW: 300}}, + []DispatchTarget{{Driver: "idle", TargetW: -650}, {Driver: "planned", TargetW: -1650}, {Driver: "charge", TargetW: 500}, {Driver: "flip", TargetW: -200}}, + ) + want := map[string]float64{"idle": 650, "planned": 650, "flip": 200} + if len(s.batteryFuseRelief) != len(want) { + t.Fatalf("relief %v, want %v", s.batteryFuseRelief, want) + } + for d, w := range want { + if got := s.batteryFuseRelief[d]; math.Abs(got-w) > 1e-9 { + t.Fatalf("relief %v, want %v", s.batteryFuseRelief, want) + } + } +} + +// The car's room counts a battery's relief only as far as a live reading +// shows it. An offline battery's cached discharge is no reading: its whole +// relief counts, since Core can no longer hold that discharge. +func TestEVFuseReliefReadsOnlyLiveBatteries(t *testing.T) { + store := telemetry.NewStore() + store.Update("live", telemetry.DerBattery, -500, nil, nil) + store.DriverHealthMut("live").RecordSuccess() + store.Update("offline", telemetry.DerBattery, -500, nil, nil) + store.DriverHealthMut("offline").RecordSuccess() + store.DriverHealthMut("offline").SetOffline() + s := &State{batteryFuseRelief: map[string]float64{"live": 2000, "offline": 2000, "unread": 300}} + // live: min(2000, 500); offline and unread: the whole relief. + if got := s.evFuseReliefW(store); math.Abs(got-(500+2000+300)) > 1e-9 { + t.Fatalf("relief %.0f W, want 2,800 W", got) + } } // Only the hour's import limit leaves watts to an EV cut, only those the cut @@ -549,9 +737,13 @@ func TestFuseReliefLeavesOnlyCeilingWattsToCommandedEVCut(t *testing.T) { } st.SetEVCommanded([]loadpoint.State{{ID: "car", DriverName: "easee", PluggedIn: true, CommandedKnown: true, CommandedW: tc.commandedW}}) targets := []DispatchTarget{{Driver: "pixii"}} - batteries := map[string]*telemetry.DerReading{"pixii": {}} + store := telemetry.NewStore() + soc := 0.5 + store.Update("pixii", telemetry.DerBattery, 0, &soc, nil) + store.DriverHealthMut("pixii").RecordSuccess() + batteries := map[string]*telemetry.DerReading{"pixii": store.Get("pixii", telemetry.DerBattery)} meter := &telemetry.DerReading{SmoothedW: tc.gridW, Data: tc.data} - got := fuseReliefFromSnapshot(targets, batteries, meter, st, 17250, false) + got := fuseReliefFromSnapshot(targets, batteries, meter, phaseSample{meter: meter, totalW: meter.RawW, store: store}, st, 17250) if math.Abs(math.Max(0, got)-tc.wantW) > 1e-6 { t.Errorf("%s: relief %.0f W, want %.0f W", tc.name, got, tc.wantW) } @@ -592,7 +784,7 @@ func TestUnpluggedCarIsNoCutTheBatteryWaitsFor(t *testing.T) { st.SetEVCommanded(points) targets := []DispatchTarget{{Driver: "pixii"}} batteries := map[string]*telemetry.DerReading{"pixii": {}} - return math.Max(0, fuseReliefFromSnapshot(targets, batteries, &telemetry.DerReading{SmoothedW: gridW}, st, 17250, false)) + return math.Max(0, fuseReliefFromSnapshot(targets, batteries, &telemetry.DerReading{SmoothedW: gridW}, phaseSample{}, st, 17250)) } // 400 W over the ceiling, less than a step: the guard waits. if got := tick(7900, true, 5520); math.Abs(got-400) > 1e-6 { @@ -656,7 +848,7 @@ func TestCutOnStayingCarStillWaitsWhenAnotherLeaves(t *testing.T) { st.SetEVCommanded(points) targets := []DispatchTarget{{Driver: "pixii"}} batteries := map[string]*telemetry.DerReading{"pixii": {SmoothedW: batteryW}} - relief := fuseReliefFromSnapshot(targets, batteries, &telemetry.DerReading{SmoothedW: gridW}, st, 17250, false) + relief := fuseReliefFromSnapshot(targets, batteries, &telemetry.DerReading{SmoothedW: gridW}, phaseSample{}, st, 17250) batteryW = -math.Max(0, relief) // The battery follows within a tick. if batteryW < 0 && tick < 10 { t.Fatalf("%ds: battery %.0f W for watts a's stop removes; pending cut %.0f W", tick, batteryW, st.pendingEVCutW()) diff --git a/go/internal/control/ev_car_before_battery_test.go b/go/internal/control/ev_car_before_battery_test.go new file mode 100644 index 000000000..65e899c62 --- /dev/null +++ b/go/internal/control/ev_car_before_battery_test.go @@ -0,0 +1,266 @@ +package control + +import ( + "math" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// runCarBeforeBattery drives planner_self in a charge slot against a plant +// whose meter is the site without the battery plus the battery's last +// target. siteW is house plus car minus PV, site-signed. It returns every +// target the battery received. +func runCarBeforeBattery(t *testing.T, coversEV bool, siteW, carW, startBatteryW float64) []float64 { + t.Helper() + now := time.Now() + dir := SlotDirective{ + SlotStart: now, + SlotEnd: now.Add(15 * time.Minute), + BatteryEnergyWh: 350, // a PV charge slot: 1.4 kW for 15 minutes + Strategy: "self_consumption", + } + store := seedStore(siteW+startBatteryW, []struct { + name string + currentW, soc float64 + }{ + {"ferroamp", startBatteryW, 0.5}, + }) + st := NewState(0, 50, "ferroamp") + st.Mode = ModePlannerSelf + st.UseEnergyDispatch = true + st.SlotDirective = func(time.Time) (SlotDirective, bool) { return dir, true } + st.EVChargingW = carW + st.BatteryCoversEV = coversEV + st.SlewRateW = 100000 + st.MinDispatchIntervalS = 0 + const fuseW = 3 * 25 * 230 // keep the fuse out of this policy check + batteryW := startBatteryW + var got []float64 + for i := 0; i < 20; i++ { + targets := ComputeDispatch(store, st, caps(map[string]float64{"ferroamp": 15200}), fuseW) + if len(targets) == 1 { + batteryW = targets[0].TargetW + got = append(got, batteryW) + } + store.Update("ferroamp", telemetry.DerBattery, batteryW, ptrF64(0.5), nil) + store.Update("ferroamp", telemetry.DerMeter, siteW+batteryW, nil, nil) + store.DriverHealthMut("ferroamp").RecordSuccess() + } + return got +} + +func lastOr(v []float64, def float64) float64 { + if len(v) == 0 { + return def + } + return v[len(v)-1] +} + +// Seeds 207535, 207539 and 207530: PV 5 kW, house 3.6 kW, car 11 kW. The +// site imports about 10 kW, yet with battery_covers_ev off the battery +// charged from the PV the car was using. The car comes first: the battery +// charges only from what is left after the car, and is never discharged +// for the car. +func TestCarBeforeBatteryChargeWhenCoverIsOff(t *testing.T) { + const pvW, houseW, carW = 5000.0, 3611.0, 11040.0 + got := runCarBeforeBattery(t, false, houseW+carW-pvW, carW, 632) + for i, w := range got { + if w > 632+1 { + t.Fatalf("tick %d: battery charge rose to %.0f W while the site imports for the car", i, w) + } + if w < -1 { + t.Fatalf("tick %d: battery discharged %.0f W into the car with battery_covers_ev off", i, w) + } + } + if final := lastOr(got, 632); math.Abs(final) > 1 { + t.Errorf("battery settled at %.0f W, want 0 W: the car uses all PV", final) + } +} + +// With PV beyond house and car, the battery takes exactly the surplus the +// car leaves. +func TestCarBeforeBatteryChargesOnlyTheSurplusAfterTheCar(t *testing.T) { + for _, coversEV := range []bool{false, true} { + const pvW, houseW, carW = 16000.0, 3000.0, 11040.0 + got := runCarBeforeBattery(t, coversEV, houseW+carW-pvW, carW, 0) + const wantW = pvW - houseW - carW + for i, w := range got { + if w > wantW+60 { + t.Fatalf("covers_ev=%v tick %d: battery charge %.0f W takes PV the car uses (surplus after car %.0f W)", coversEV, i, w, wantW) + } + } + if final := lastOr(got, 0); math.Abs(final-wantW) > 60 { + t.Errorf("covers_ev=%v: battery settled at %.0f W, want about %.0f W", coversEV, final, wantW) + } + } +} + +// battery_covers_ev off still covers the house, never the car. +func TestCarBeforeBatteryCoverOffDischargesForHouseOnly(t *testing.T) { + const pvW, houseW, carW = 1000.0, 3000.0, 11040.0 + got := runCarBeforeBattery(t, false, houseW+carW-pvW, carW, 0) + const wantW = pvW - houseW // -2000 W: the house, not the car + for i, w := range got { + if w < wantW-60 { + t.Fatalf("tick %d: battery discharged %.0f W, past the house's %.0f W", i, w, wantW) + } + } + if final := lastOr(got, 0); math.Abs(final-wantW) > 60 { + t.Errorf("battery settled at %.0f W, want about %.0f W", final, wantW) + } +} + +// battery_covers_ev on lets the battery feed the car instead of charging. +func TestCarBeforeBatteryCoverOnDischargesForTheCar(t *testing.T) { + const pvW, houseW, carW = 5000.0, 3611.0, 11040.0 + got := runCarBeforeBattery(t, true, houseW+carW-pvW, carW, 632) + for i, w := range got { + if w > 632+1 { + t.Fatalf("tick %d: battery charge rose to %.0f W while the site imports", i, w) + } + } + if final := lastOr(got, 632); final > -1000 { + t.Errorf("battery settled at %.0f W, want a discharge that covers the car", final) + } +} + +// The energy path follows the same rule: a planned PV charge takes only +// the surplus the car leaves, while a planned purchase keeps its charge. +func TestCarBeforeBatteryEnergyPathPVChargeOnly(t *testing.T) { + const pvW, houseW, carW = 8000.0, 500.0, 6000.0 + for _, tc := range []struct { + name string + plannedW float64 + wantW float64 + }{ + {"pv charge", -1000, pvW - houseW - carW}, + {"planned purchase", 2000, 4000}, + } { + for _, coversEV := range []bool{false, true} { + now := time.Now() + dir := SlotDirective{ + SlotStart: now, + SlotEnd: now.Add(15 * time.Minute), + BatteryEnergyWh: 1000, // 4 kW for 15 minutes + Strategy: "cheap", + PlannedGridW: tc.plannedW, + HasPlannedGridW: true, + } + store := seedStore(houseW+carW-pvW, []struct { + name string + currentW, soc float64 + }{ + {"ferroamp", 0, 0.5}, + }) + st := NewState(0, 50, "ferroamp") + st.Mode = ModePlannerCheap + st.UseEnergyDispatch = true + st.SlotDirective = func(time.Time) (SlotDirective, bool) { return dir, true } + st.EVChargingW = carW + st.BatteryCoversEV = coversEV + st.SlewRateW = 100000 + targets := ComputeDispatch(store, st, caps(map[string]float64{"ferroamp": 15200}), 3*25*230) + if len(targets) != 1 { + t.Fatalf("%s covers_ev=%v: want 1 target, got %+v", tc.name, coversEV, targets) + } + got := targets[0].TargetW + if coversEV { + // The plan's grid check sees the car on the meter and + // backs the charge off further; only the car-first bound + // matters here. + if tc.plannedW < 100 && got > tc.wantW+1 { + t.Errorf("%s covers_ev=true: battery target %.0f W takes PV the car uses (want at most %.0f W)", tc.name, got, tc.wantW) + } + continue + } + if math.Abs(got-tc.wantW) > 1 { + t.Errorf("%s covers_ev=false: battery target %.0f W, want %.0f W", tc.name, got, tc.wantW) + } + } + } +} + +// runPeakShavingCar drives manual peak_shaving against the same plant as +// runCarBeforeBattery. The peak limit sits far above the site, so the +// import the car causes lands in peak_shaving's dead zone. +func runPeakShavingCar(t *testing.T, coversEV bool, siteW, carW, startBatteryW float64) []float64 { + t.Helper() + store := seedStore(siteW+startBatteryW, []struct { + name string + currentW, soc float64 + }{ + {"ferroamp", startBatteryW, 0.5}, + }) + st := NewState(0, 50, "ferroamp") + st.Mode = ModePeakShaving + st.PeakLimitW = 30000 + st.EVChargingW = carW + st.BatteryCoversEV = coversEV + st.SlewRateW = 100000 + st.MinDispatchIntervalS = 0 + const fuseW = 3 * 25 * 230 + batteryW := startBatteryW + var got []float64 + for i := 0; i < 20; i++ { + targets := ComputeDispatch(store, st, caps(map[string]float64{"ferroamp": 15200}), fuseW) + if len(targets) == 1 { + batteryW = targets[0].TargetW + } + got = append(got, batteryW) + store.Update("ferroamp", telemetry.DerBattery, batteryW, ptrF64(0.5), nil) + store.Update("ferroamp", telemetry.DerMeter, siteW+batteryW, nil, nil) + store.DriverHealthMut("ferroamp").RecordSuccess() + } + return got +} + +// The 207535 shape in manual peak_shaving. The battery was charging 871 W +// from the house surplus when the car started. The site now imports for +// the car, inside peak_shaving's dead zone, so nothing used to cut the +// charge. The car comes first: the charge goes, and the battery never +// discharges, because the import is under the peak limit. +func TestCarBeforeBatteryPeakShavingDropsARunningCharge(t *testing.T) { + const pv, house, car = 5000.0, 3611.0, 11040.0 + for _, cover := range []bool{false, true} { + got := runPeakShavingCar(t, cover, house+car-pv, car, 871) + for i, w := range got { + if w < -1 { + t.Fatalf("cover=%v tick %d: battery discharges %.0f W under the peak limit: %v", cover, i, w, got) + } + } + if final := lastOr(got, 871); final > 1 { + t.Fatalf("cover=%v: battery still charges %.0f W while the site imports %.0f W for the car: %v", + cover, final, house+car-pv+final, got) + } + } +} + +// With more PV than house and car together, peak_shaving still charges +// from what the car leaves: 18000 − 3611 − 11040 = 3349 W. +func TestCarBeforeBatteryPeakShavingChargesTheSurplusAfterTheCar(t *testing.T) { + const pv, house, car = 18000.0, 3611.0, 11040.0 + want := pv - house - car + for _, cover := range []bool{false, true} { + got := runPeakShavingCar(t, cover, house+car-pv, car, 0) + if final := lastOr(got, 0); math.Abs(final-want) > 60 { + t.Fatalf("cover=%v: battery settled at %.0f W, want the %.0f W left after the car: %v", cover, final, want, got) + } + } +} + +// On the PI path with battery_covers_ev off, a discharge stops at once +// when the house alone has surplus: what it gives now goes to the car. The +// battery was discharging 2,900 W; 5 kW of PV beats the 3.6 kW house, and +// the car draws 11 kW. No order may discharge, not even as a PI step on +// the way to 0 W. +func TestCarBeforeBatteryCoverOffStopsADischargeForTheCarAtOnce(t *testing.T) { + const pv, house, car = 5000.0, 3611.0, 11040.0 + got := runPeakShavingCar(t, false, house+car-pv, car, -2900) + for i, w := range got { + if w < -1 { + t.Fatalf("tick %d: battery discharges %.0f W into the car with battery_covers_ev off: %v", i, w, got) + } + } +} diff --git a/go/internal/control/ev_deadline_runtime_test.go b/go/internal/control/ev_deadline_runtime_test.go index 6919b6f49..4a2fdcd7b 100644 --- a/go/internal/control/ev_deadline_runtime_test.go +++ b/go/internal/control/ev_deadline_runtime_test.go @@ -88,7 +88,7 @@ func TestJoinedNoBatteryNoPlanGoalContinuity(t *testing.T) { legal = true } } - if !legal || 6000+got > 17250.001 || phaseA+got/(3*230) > 24.001 { + if !legal || 6000+got > 17250.001 || phaseA+got/(3*230) > st.fuseLineA()+.001 { t.Fatalf("unsafe or illegal EV offer=%v, grid=%v phaseA=%v", got, 6000+got, phaseA+got/(3*230)) } if svc.Latest() != nil { @@ -206,16 +206,40 @@ func TestJoinedMixedPhaseNoCreditBeforeMeasuredReduction(t *testing.T) { // Without a plan, a car whose own charger takes less than a stepless // charger's minimum (a 3.7 kW plug-in hybrid on an 11 or 22 kW charger) is // ordered that minimum, as the plan orders it: never 0 W because the car's -// own maximum is lower. +// own maximum is lower. On a table with one-phase and three-phase levels it +// is ordered the lowest three-phase level, not a one-phase level the +// three-phase order cannot run (review of #1807). func TestJoinedNoPlanGoalSmallCarGetsChargerMinimum(t *testing.T) { ctx := context.Background() - for _, maxW := range []float64{11040, 22080} { + mixed := []float64{0} + for _, phases := range []float64{1, 3} { + for a := 6.0; a <= 16; a++ { + mixed = append(mixed, a*230*phases) + } + } + for _, ch := range []struct { + name string + mode string + minW, maxW float64 + steps []float64 + carMaxW float64 + }{ + {"11 kW", "", 4140, 11040, nil, 3680}, + {"22 kW", "", 4140, 22080, nil, 3680}, + {"mixed table", "", 1380, 11040, mixed, 3680}, + {"mixed table, auto", "auto", 1380, 11040, mixed, 3680}, + {"mixed table, 3p", "3p", 1380, 11040, mixed, 3680}, + // The charger's minimum above the car's maximum, and that above + // the lowest one-phase level: the car's own limits leave no level. + {"mixed table, minimum 2,000 W, a 1.5 kW car", "", 2000, 11040, mixed, 1500}, + } { now := time.Now() - cfg := loadpoint.Config{ID: "garage", DriverName: "charger", VehicleCapacityWh: 15000, PluginSoC: .3, MinChargeW: 4140, MaxChargeW: maxW} + cfg := loadpoint.Config{ID: "garage", DriverName: "charger", VehicleCapacityWh: 15000, PluginSoC: .3, PhaseMode: ch.mode, + MinChargeW: ch.minW, MaxChargeW: ch.maxW, AllowedStepsW: ch.steps} mgr := loadpoint.NewManager() mgr.Load([]loadpoint.Config{cfg}) mgr.Observe(cfg.ID, true, 0, 0, true) - if !mgr.SetCarBinding(cfg.ID, loadpoint.CarBinding{ID: "phev", MaxW: 3680}) || + if !mgr.SetCarBinding(cfg.ID, loadpoint.CarBinding{ID: "phev", MaxW: ch.carMaxW}) || !mgr.SetCurrentSoC(cfg.ID, .3) || !mgr.SetTarget(cfg.ID, .8, now.Add(3*time.Hour)) { t.Fatal("goal fixture rejected") } @@ -256,11 +280,12 @@ func TestJoinedNoPlanGoalSmallCarGetsChargerMinimum(t *testing.T) { return nil }) controller.SetSiteFuse(loadpoint.SiteFuse{MaxAmps: 25, Voltage: 230, PhaseCnt: 3}) + svc.PlanningLimits = controller.PlanningLimits // as main.go wires it controller.SetFuseEVMax(func() (float64, bool) { return EVFuseBudget(tel, st, mgr.States(), controller.FuseConfigs(), 17250) }) controller.SetFusePhaseRoomA(func() (float64, bool) { return EVFusePhaseRoomA(st) }) controller.TickWithDispatch(ctx, now, true) if len(commands) == 0 || commands[len(commands)-1] != 4140 { - t.Fatalf("%.0f W charger: orders %v, want the charger's minimum 4140 W", maxW, commands) + t.Errorf("%s: orders %v, want 4140 W", ch.name, commands) } } } diff --git a/go/internal/control/ev_fuse_budget.go b/go/internal/control/ev_fuse_budget.go index 51e492c4f..c782d3819 100644 --- a/go/internal/control/ev_fuse_budget.go +++ b/go/internal/control/ev_fuse_budget.go @@ -3,6 +3,7 @@ package control import ( "log/slog" "math" + "slices" "time" "github.com/srcfl/ftw/go/internal/loadpoint" @@ -32,10 +33,10 @@ const evUnknownSettle = 3 * time.Minute // pool starts again. It stays above Easee's ~30 s contactor minimum. const evRestartDwell = 35 * time.Second -// evPhaseMarginA is the working margin below the breaker on each phase. One -// amp on the pool's phases is also the rise step and the gap between the cut -// and rise thresholds. -const evPhaseMarginA = 1.0 +// evStepA is the EV rise step: one amp on each of the pool's phases. It is +// the size of the next move, never current held back below the fuse's +// working limit (fuseLineA). +const evStepA = 1.0 // fuseHourFactor is the phase current, as a multiple of the fuse rating, // that a gG fuse of 16-63 A carries for an hour. A cut on its way covers the @@ -170,11 +171,189 @@ func (l commandLog) toCome(from, at time.Time, before sentCommand) (w, a float64 type chargerOffers struct { log commandLog settle time.Duration + // declared is true when the driver declares that time; settle is then + // not evUnknownSettle's guess (evShownW). + declared bool // found is the offer in force before the limit's first site reading, // after a restart: Core did not send it in this run. A rise counts the // charger as 0 before it; a cut takes it as shown, since the reading is // all Core knows of it. found sentCommand + // driver is the charger's driver, whose power reading only the battery + // side reads, and only before any offer of this run has surely landed + // (evFoundDrawW). maxW is the loadpoint's max_charge_w; zero when not + // configured. + driver string + maxW float64 + // draws are the charger's samples while that window is open (noteDraw), + // and watchFrom the receipt of the first reading noteDraw took there: + // a sample already there then may have arrived at any earlier time. + draws []evDraw + watchFrom time.Time + // carPhases is how many phases the car draws on, as the loadpoint saw + // it on the site meter: 1 or 3, or 0 when unknown. carPhase is the phase + // a car on one phase draws on, 1-3. + carPhases, carPhase int + // shownA is the current on each phase of the charger's offers that the + // site meter shows, as far as it can tell (seeOffers); unsure while it + // cannot. + shownA [3]float64 + unsure bool + // riseBaseA is the site meter's phase currents when a raise went out + // with no other raise still to show, and riseFromA the most of the + // charger's offers that reading may have shown. riseAheadA measures the + // rise still to come against them; riseSet is false while no raise is + // on its way. + riseBaseA [3]float64 + riseFromA float64 + riseSet bool +} + +// evMoveA is half a charger step. An order landing shows on the site meter +// as every phase moving the same way, by evMoveA or more on their mean and +// no more than a step apart, or as one phase alone moving a whole step +// while the others hold within evMoveA (a car on one phase). Meter noise +// rarely moves that far. +const evMoveA = evStepA / 2 + +// chargerMoveA is the move per phase from one reading to the next when it +// has the shape of an order landing (evMoveA), signed: alikeA when every +// phase moved alike, oneA and its phase when one phase moved alone. +func chargerMoveA(prev, now []float64) (alikeA, oneA float64, phase int) { + var sum, held, moved, d1 float64 + lo, hi, p1 := math.Inf(1), math.Inf(-1), 0 + for i := range now { + d := now[i] - prev[i] + sum, lo, hi = sum+d, math.Min(lo, d), math.Max(hi, d) + switch { + case math.Abs(d) >= 2*evMoveA: + moved, d1, p1 = moved+1, d, i + case math.Abs(d) < evMoveA: + held++ + } + } + n := float64(len(now)) + switch { + case hi-lo <= 2*evMoveA && (lo > 0 && sum/n >= evMoveA || hi < 0 && sum/n <= -evMoveA): + return sum / n, 0, 0 + case moved == 1 && held == n-1: + return 0, d1, p1 + 1 + } + return 0, 0, 0 +} + +// seeOffers reads a new site reading against the last one. With one charger +// in the pool and no change of phases in its log, a move shaped like an +// order landing (chargerMoveA) moves the current the meter shows of the +// charger on each phase (shownA), in a shape the offers can take there. A +// car the loadpoint has seen draw on one phase, or a one-phase order, is +// one level on every phase: every phase dropping alike moves it, and so +// does a move on the car's phase alone where that phase is known. A move on +// one phase while the car's phase is not known may be a house load, so it +// does not count. A three-phase order moves only on every phase alike: +// a phase that moves alone is a house load, or a car not yet seen that +// draws on one phase, and Core cannot tell which. The shown current moves +// only toward the latest offer, and stays within the offers the reading +// may show. While our own battery order moves, a move of every phase alike +// may be the battery's, so no reading counts; after a reading that did not +// count, the shown current is known again once only one offer may show. A +// house load that moves in a charger's shape reads as ours. It can only +// take cover off a cut. The rise a raise has still to show reads the meter +// differently (riseAheadA). +func (s *State) seeOffers(meter *telemetry.DerReading, gridW float64) { + lim := &s.evLimit + amps, _, valid := phaseCurrentsA(meter, s, gridW) + var now [3]float64 + copy(now[:], amps) + prev, read := lim.phasesA, valid && lim.phasesSeen && len(lim.offers) == 1 && len(amps) >= 2 && !s.batteryOrderMoves() + lim.phasesA, lim.phasesSeen = now, valid + var alikeA, oneA float64 + var phase int + if read { + alikeA, oneA, phase = chargerMoveA(prev[:len(amps)], now[:len(amps)]) + } + for id, c := range lim.offers { + if read && c.readable() { + var moveA [3]float64 + switch { + case c.carPhase > 0 || c.log[0].n == 1: + // One car on one phase: one level, moved by a drop of + // every phase alike or by its known phase alone. + m := math.Min(0, alikeA) + if c.carPhase == phase { + m += oneA + } + moveA = [3]float64{m, m, m} + default: + // A three-phase order moves every phase alike. Read as the + // car, a load cycling on the phase over the line took the + // cut's cover off there on each drop and never gave it + // back, and Core cut again and again. + moveA = [3]float64{alikeA, alikeA, alikeA} + } + // The charger moves toward its latest offer, never past it. + latestA := c.log[len(c.log)-1].a + for i, m := range moveA { + if m < 0 && c.shownA[i] > latestA { + c.shownA[i] = math.Max(latestA, c.shownA[i]+m) + } else if m > 0 && c.shownA[i] < latestA { + c.shownA[i] = math.Min(latestA, c.shownA[i]+m) + } + } + } else { + c.unsure = true + } + low, high := c.offerRangeA(lim.readingAt) + for i := range c.shownA { + c.shownA[i] = math.Min(high, math.Max(low, c.shownA[i])) + } + c.unsure = c.unsure && low < high + if ahead := s.riseAheadA(c); low >= high || ahead == [3]float64{} { + c.riseSet = false + } + lim.offers[id] = c + } +} + +// batteryOrderMoves reports whether one of our battery orders of the last +// evSettle moves the site's phases by evMoveA or more. +func (s *State) batteryOrderMoves() bool { + for _, log := range s.batteryCommands { + if low, high, _, ok := log.span(s.now()); ok && high-low >= evMoveA*s.SiteFuseVoltage*float64(s.SiteFusePhases) { + return true + } + } + return false +} + +// readable reports whether the site meter can tell what it shows of the +// charger's offers: every offer the log holds lands on the same phases. +func (c chargerOffers) readable() bool { + for _, o := range c.log { + if o.n != c.log[0].n { + return false + } + } + return len(c.log) > 0 +} + +// offerRangeA is the lowest and highest current per phase among the +// offers a reading taken at `at` may show: the one in force settle before +// it and each one sent since. Before the first logged offer the charger +// counts as before. +func (c chargerOffers) offerRangeA(at time.Time) (low, high float64) { + from := at.Add(-c.settle) + low, high = c.found.a, c.found.a + for _, o := range c.log { + switch { + case o.at.After(at): + case !o.at.After(from): + low, high = o.a, o.a + default: + low, high = math.Min(low, o.a), math.Max(high, o.a) + } + } + return low, high } // pendingLoad is load we ordered that a site reading may not show yet: @@ -187,6 +366,214 @@ type pendingLoad struct { phaseA [3]float64 // on each phase } +// evShownW is the EV load a reading taken at `at` surely shows: each +// plugged charger's latest offer less what it may not show yet. The battery +// side reads this. A raise stays unshown for the response time the +// charger's driver declares, so the battery waits for a slow charger's raise +// as long as the EV limit does. A charger that declares none gets evSettle, +// not evUnknownSettle: the battery would otherwise reserve a local charger's +// raise on top of the reading that already shows it, and stop charging for +// three minutes after each one. Easee Cloud, the slow charger we know, +// declares its 180 s. +func (s *State) evShownW(at time.Time) float64 { + var w float64 + for _, c := range s.evLimit.offers { + if n := len(c.log); n > 0 { + unshown, _ := c.log.unshown(at, c.batterySettle()) + w += math.Max(0, c.log[n-1].w-unshown) + } + } + return w +} + +// evFoundDrawW is EV draw the battery side may count as shown beyond +// evShownW (shownW) while a charger has no order of this run that has +// surely landed: from the first offer Core logs for it with no fuse limit +// set (at Core's start, or at a plug-in once no other managed charger was +// plugged), until that offer is older than the charger's settle time. Core +// then does not know which offer the charger runs on, and its first offers +// (often 0 A) say nothing of what the meter shows. +// +// A car seen drawing in that window is in the site reading when the +// charger sampled that draw before the reading, and no sample since reads +// less. The charger stamps its samples with its own clock (OCPP +// MeterValues), which may run behind Core's by any amount, so Core judges +// a sample by when it first arrived on Core's clock (noteDraw): a sample +// arrives after it was measured. It counts as before the reading only +// when Core saw it arrive evMeterDelay before the reading did (arrivedBy), +// and no more than one control interval before that: a fast charger may +// have applied a 0 A offer, or the car paused, since an older sample. The +// draw is the lowest of that sample and every sample since (noteDraw, each +// the lowest of the sample, raw and smoothed), up to the charger's current +// reading. So a rise sampled after the reading never counts, whatever the +// charger's clock. A held reading after a failed poll says nothing of now. +// A charger that samples less often (OCPP MeterValues every 10 s or more) counts +// only on the ticks from evMeterDelay to one control interval more after a +// sample arrives, and holds its battery's discharge between them. +// +// Until the latest offer lands, the car may still rise from that draw to +// the offer in force. Core does not know that offer: one from before a +// restart, under an older max_charge_w, or the charger's own default. The +// reserve takes it as the larger of the loadpoint's max_charge_w and the +// site fuse's rating, and keeps the rise past the latest offer; a charger +// with no max_charge_w counts nothing. Each charger counts no more than its +// latest offer, and in all no more than the consumption the site meter +// shows (siteConsumptionW) less shownW. The car's own limit never reads +// this; only how much of our latest offers the battery reserves as a rise +// to come. +// gridW is the site reading, which arrived at meterAt. +func (s *State) evFoundDrawW(store *telemetry.Store, gridW float64, meterAt, at time.Time, shownW float64) float64 { + if store == nil { + return 0 + } + consumptionW := siteConsumptionW(store, gridW) + if !finiteFuseValue(consumptionW) { + return 0 + } + skew := time.Duration(s.ControlIntervalS) * time.Second + if skew <= 0 { + skew = 2 * time.Second // config's default control_interval_s + } + edge := meterAt.Add(-evMeterDelay) + from := edge.Add(-skew) + var w float64 + for _, c := range s.evLimit.offers { + n := len(c.log) + if n == 0 || !c.foundOpen(at) || c.driver == "" || !(c.maxW > 0) || !finiteFuseValue(c.maxW) { + continue + } + now, ok := readEVDraw(store, c.driver, at) + if !ok { + continue + } + for _, d := range c.draws { + if d.at.Equal(now.at) { + d.merge(now) + now = d + } + } + if now.seen.Before(from) { + continue + } + drawW, before := now.w, c.arrivedBy(now, edge) + for _, d := range c.draws { + if d.seen.Before(from) { + continue + } + drawW, before = math.Min(drawW, d.w), before || c.arrivedBy(d, edge) + } + if !before { + continue + } + unshown, _ := c.log.unshown(at, c.batterySettle()) + latest := c.log[n-1].w + capW := math.Max(c.maxW, s.siteFuseMaxW()) + inLatestW := math.Min(latest, drawW-math.Max(0, capW-latest)) + w += math.Max(0, inLatestW-math.Max(0, latest-unshown)) + } + return math.Max(0, math.Min(w, consumptionW-shownW)) +} + +// evMeterDelay is how much earlier than the site reading a charger's +// sample must arrive to count as in that reading (evFoundDrawW): the site +// meter's own delay from measurement to arrival. +const evMeterDelay = time.Second + +// evDraw is one charger sample: its stamp on the charger's own clock, +// which only tells samples apart; when it first arrived on Core's clock; +// when the charger last vouched for it, on Core's clock (its arrival, or a +// later confirmed reading); and the lowest of the sample, raw and smoothed. +type evDraw struct { + at, got, seen time.Time + confirmed bool + w float64 +} + +// merge folds a repeat of the same sample into e: the lowest draw, the +// first arrival, and a later receipt only where the charger confirms it. +func (e *evDraw) merge(d evDraw) { + e.w = math.Min(e.w, d.w) + if d.got.Before(e.got) { + e.got = d.got + } + if d.confirmed && d.seen.After(e.seen) { + e.seen, e.confirmed = d.seen, true + } +} + +// readEVDraw is the charger's current sample, while its telemetry is live, +// its last poll succeeded and the sample passes qualifiedFuseObservation, +// taken as arriving with the charger's current reading. +func readEVDraw(store *telemetry.Store, driver string, at time.Time) (evDraw, bool) { + lowerW, live := store.OnlineLowerW(driver, telemetry.DerEV) + if !live { + return evDraw{}, false + } + r := store.Get(driver, telemetry.DerEV) + p, ok := qualifiedFuseObservation(r, at) + if !ok { + return evDraw{}, false + } + w := math.Min(p.PowerW, lowerW) + return evDraw{at: p.At, got: r.UpdatedAt, seen: r.UpdatedAt, confirmed: !p.ConfirmedAt.IsZero(), w: w}, finiteFuseValue(w) +} + +// arrivedBy reports whether Core saw sample d arrive by edge. A sample +// already there when Core began to watch the charger arrived at a time +// Core does not know, so it never counts as arrived. +func (c chargerOffers) arrivedBy(d evDraw, edge time.Time) bool { + return !c.watchFrom.IsZero() && d.got.After(c.watchFrom) && !d.got.After(edge) +} + +// foundOpen is true while the charger has no order of this run that has +// surely landed (evFoundDrawW). +func (c chargerOffers) foundOpen(at time.Time) bool { + return !c.found.at.IsZero() && at.Sub(c.found.at) < c.settle +} + +// noteDraw keeps the charger's samples while foundOpen, for evFoundDrawW, +// over telemetry.ControlWindowDuration, each with its first arrival. +func (c *chargerOffers) noteDraw(store *telemetry.Store, at time.Time) { + if !c.foundOpen(at) || c.driver == "" { + c.draws, c.watchFrom = nil, time.Time{} + return + } + d, ok := readEVDraw(store, c.driver, at) + if !ok { + return + } + if c.watchFrom.IsZero() { + c.watchFrom = d.got + } + if n := len(c.draws); n > 0 && !d.at.After(c.draws[n-1].at) { + if last := &c.draws[n-1]; d.at.Equal(last.at) { + last.merge(d) + } + return + } + c.draws = append(slices.DeleteFunc(c.draws, func(e evDraw) bool { + return at.Sub(e.seen) > telemetry.ControlWindowDuration + }), d) +} + +// siteConsumptionW is the consumption a site reading of gridW shows: grid +// less each live battery, PV and V2X reading, each at the higher of raw and +// smoothed. A smoothed value lags an output that just fell; the higher one +// keeps the estimate at or below what the meter shows. +func siteConsumptionW(store *telemetry.Store, gridW float64) float64 { + return gridW - store.SumOnlineHigherW(telemetry.DerBattery, telemetry.DerPV, telemetry.DerV2X) +} + +// batterySettle is how long the battery side counts a raise of this +// charger's as unshown: its declared response time, else evSettle +// (evShownW). +func (c chargerOffers) batterySettle() time.Duration { + if c.declared { + return c.settle + } + return evSettle +} + func (s *State) pendingLoad(at time.Time) pendingLoad { var p pendingLoad for _, c := range s.evLimit.offers { @@ -210,7 +597,14 @@ func (s *State) pendingLoad(at time.Time) pendingLoad { // cannot see the hottest phase. A single-phase cut covers on the phase // it lands on, which Core does not know, so with other chargers in the pool // it covers no phase. -func (s *State) unshownOrders(at time.Time, perPhase bool) (rise, cover pendingLoad, latestW float64) { +// +// Where the meter tells what it shows of the charger (seeOffers), a cut +// covers no more than the meter has still to show on each phase: a cut the +// meter shows covers no overage that comes after it. aheadA is then the +// rise on each phase the meter has still to show of the latest offers +// (riseAheadA), which a raise's own trigger reads (stepEVMeterLimit), and +// rise counts at least that much. Elsewhere aheadA is 0. +func (s *State) unshownOrders(at time.Time, perPhase bool) (rise, cover pendingLoad, aheadA [3]float64, latestW float64) { lim := &s.evLimit for _, c := range lim.offers { w, a := c.log.toCome(at.Add(-c.settle), at, c.found) @@ -231,21 +625,71 @@ func (s *State) unshownOrders(at time.Time, perPhase bool) (rise, cover pendingL if latest.n == 1 && len(lim.offers) > 1 { coverA = 0 } + riseA := math.Max(0, a) + phaseCoverA, phaseRiseA, ahead := [3]float64{coverA, coverA, coverA}, [3]float64{riseA, riseA, riseA}, [3]float64{} + if perPhase && lim.phasesSeen && len(lim.offers) == 1 && s.SiteFusePhases >= 2 && c.readable() && !c.unsure { + low, high := c.offerRangeA(at) + ahead = s.riseAheadA(c) + for i := range min(s.SiteFusePhases, len(ahead)) { + toComeA := latest.a - math.Min(high, math.Max(low, c.shownA[i])) + phaseCoverA[i] = math.Min(coverA, math.Max(0, -toComeA)) + phaseRiseA[i] = math.Max(riseA, ahead[i]) + } + } rise.w += math.Max(0, w) cover.w += math.Max(0, math.Min(-w, -runW)) for i := range rise.phaseA { - rise.phaseA[i] += math.Max(0, a) - cover.phaseA[i] += coverA + rise.phaseA[i] += phaseRiseA[i] + cover.phaseA[i] += phaseCoverA[i] + aheadA[i] += ahead[i] } latestW += latest.w } - return rise, cover, latestW + return rise, cover, aheadA, latestW +} + +// riseAheadA is the rise on each phase the site meter has still to show +// of the charger's latest offer: the raise since riseFromA less the meter's +// own rise since the raise went out, within the offers the reading may +// show. A car the loadpoint has seen draw on three phases takes the +// smallest rise across the phases: a house step on some phases does not +// show the raise on the rest. A car on one phase takes its own phase's +// rise. Any other car takes the largest rise: until the loadpoint has seen +// it, it may draw on one phase Core does not know. A landing counts in +// whatever shape it comes, a slow ramp or beside a house load that moves +// in the same reading. A house load that drops while the raise travels +// still reads as raise not shown yet, by as much as it dropped. +func (s *State) riseAheadA(c chargerOffers) (ahead [3]float64) { + lim := &s.evLimit + n := min(s.SiteFusePhases, len(ahead)) + if !c.riseSet || !lim.phasesSeen || n < 2 || len(c.log) == 0 { + return ahead + } + latest := c.log[len(c.log)-1] + rises := make([]float64, n) + for i := range rises { + rises[i] = lim.phasesA[i] - c.riseBaseA[i] + } + riseA, phases := slices.Min(rises), ahead[:n] + switch { + case c.carPhase > 0 && c.carPhase <= n: + riseA, phases = rises[c.carPhase-1], ahead[c.carPhase-1:c.carPhase] + case c.carPhases != 3 || latest.n == 1: + riseA = slices.Max(rises) + } + low, high := c.offerRangeA(lim.readingAt) + toComeA := math.Max(0, latest.a-math.Min(high, math.Max(low, c.riseFromA+riseA))) + for i := range phases { + phases[i] = toComeA + } + return ahead } // recordEVOffers keeps each plugged charger's offers, with the current each // puts on the phases its command names, for the longer of evSettle and the // power_max_age_s its driver declares, or evUnknownSettle when it declares -// none. Our own commands only: a charger's power reading never counts. +// none. Our own commands only: a charger's power reading never counts. The +// samples kept for the battery side (noteDraw) take no part in the limit. func (s *State) recordEVOffers(store *telemetry.Store, points []loadpoint.State) { now := s.now() read := now @@ -258,10 +702,10 @@ func (s *State) recordEVOffers(store *telemetry.Store, points []loadpoint.State) continue } c := s.evLimit.offers[point.ID] - c.settle = evUnknownSettle + c.settle, c.declared, c.carPhases, c.carPhase, c.driver, c.maxW = evUnknownSettle, false, point.CarPhases, point.CarPhase, point.DriverName, point.MaxChargeW if r := store.Get(point.DriverName, telemetry.DerEV); r != nil { if declared := telemetry.DeclaredPowerMaxAge(r.Data); declared > 0 { - c.settle = max(evSettle, declared) + c.settle, c.declared = max(evSettle, declared), true } } w := math.Max(0, point.CommandedW) @@ -269,10 +713,13 @@ func (s *State) recordEVOffers(store *telemetry.Store, points []loadpoint.State) sent := sentCommand{at: now, w: w, a: a, n: phases} if n := len(c.log); n == 0 || c.log[n-1].w != sent.w || c.log[n-1].a != sent.a || c.log[n-1].n != sent.n { if n == 0 && !s.evLimit.set { - c.found = sent + c.found, c.shownA = sent, [3]float64{sent.a, sent.a, sent.a} + } else if _, high := c.offerRangeA(now); sent.a > high && !c.riseSet && s.evLimit.phasesSeen { + c.riseBaseA, c.riseFromA, c.riseSet = s.evLimit.phasesA, high, true } c.log = c.log.keep(sent, read.Add(-c.settle)) } + c.noteDraw(store, now) offers[point.ID] = c } s.evLimit.offers = offers @@ -292,7 +739,7 @@ func (s *State) offerPhaseA(w float64, phases int) (float64, int) { } // EVFusePhaseRoomA is the current the chargers may still add on the -// busiest phase: below the fuse less its margin, the last reading's +// busiest phase: below the fuse's working limit (fuseLineA), the last reading's // current or heat, and every charger's offers that reading may not show. // For an idle pool the heat counts only over the working limit, as the // start reads it. false when the site meter reports no phase currents. @@ -316,13 +763,18 @@ type evMeterLimit struct { offers map[string]chargerOffers // each plugged charger's, by loadpoint phaseRoomA float64 // below the working limit on the busiest phase at readingAt; an idle pool's as a start reads it phaseRoomKnown bool // the meter reports phase currents - overSince time.Time // first reading of a run over the hour's import limit - fuseOverAt time.Time // first reading of a run with a phase, or the total, over the fuse's working limit - fuseCutAt time.Time // the first cut of that run - fuseUnderAt time.Time // first reading of a stretch under that limit - overOffers map[string]float64 // each charger's offer when that run began - idleWh float64 // import since roomSince, for the ceiling's start test - byImport bool // the hour's import limit, not the fuse, left less room at readingAt + phasesA [3]float64 // the phase currents at readingAt, when phasesSeen + phasesSeen bool + cutForA [3]float64 // each phase's highest reading that a cut still on its way was sent on, when cutForSet + cutForSet bool + overSince time.Time // first reading of a run over the hour's import limit + fuseOverAt time.Time // first reading of a run with a phase, or the total, over the fuse's working limit + fuseCutAt time.Time // the first cut of that run + fuseUnderAt time.Time // first reading of a stretch under that limit + overOffers map[string]float64 // each charger's offer when that run began + idleWh float64 // import since roomSince, for the ceiling's start test + byImport bool // the hour's import limit, not the fuse, left less room at readingAt + batteryYield evBatteryYield } type evPool struct { @@ -336,17 +788,21 @@ type evPool struct { // shared freshness gate must already allow dispatch. A fuse overage in total // import cuts at once; a phase overage cuts once that phase's heat budget is // spent, or at once above fuseInstantFactor. Otherwise the cap holds; it -// rises one step on a fresh reading with room for two, at least evSettle +// rises one step, or the rest of the demand when that is less, on a fresh +// reading with room for that rise below the working limit, at least evSettle // after the last cut. Room counts every // offer sent above what the reading may show as flowing until its charger // can have applied it; a cut never frees room before the meter shows it. // A cut counts our orders both ways: a raise on its way adds to the overage, // and a cut on its way covers the overage it was sent for until its charger // can have applied it, within its run over the limit and while the phases -// stay at or below fuseHourFactor (unshownOrders). +// stay at or below fuseHourFactor (unshownOrders), but never once the meter +// shows it. Our own rise on its way cuts before the meter shows it where it +// would read above fuseInstantFactor, or over the working limit on a phase +// whose heat budget is spent. // EVFusePhaseRoomA gives the same room per phase to the split between cars. // An idle pool starts only after the meter has kept showing -// room for its minimum plus a step. Each reading acts once, so a silent +// room for its minimum below the working limit. Each reading acts once, so a silent // meter can never raise or restart a charge. func EVFuseBudget(store *telemetry.Store, state *State, points []loadpoint.State, configs []loadpoint.Config, fuseMaxW float64) (float64, bool) { if store == nil || state == nil || !finiteFuseValue(fuseMaxW) || fuseMaxW <= 0 { @@ -367,36 +823,42 @@ func EVFuseBudget(store *telemetry.Store, state *State, points []loadpoint.State *lim = evMeterLimit{} return 0, true } - if pool.offerKnown { - state.evOffers = state.evOffers.add(state.now(), pool.offerW) - } state.recordEVOffers(store, points) if !lim.set || meter.UpdatedAt.After(lim.readingAt) { - state.stepEVMeterLimit(meter, gridW, pool, fuseMaxW) + state.stepEVMeterLimit(store, meter, gridW, pool, fuseMaxW) } return lim.capW, true } -func (s *State) stepEVMeterLimit(meter *telemetry.DerReading, gridW float64, pool evPool, fuseMaxW float64) { +func (s *State) stepEVMeterLimit(store *telemetry.Store, meter *telemetry.DerReading, gridW float64, pool evPool, fuseMaxW float64) { lim, now, at := &s.evLimit, s.now(), meter.UpdatedAt first, prevAt := !lim.set, lim.readingAt gap := first || at.Sub(prevAt) > evSettle lim.readingAt = at + s.seeOffers(meter, gridW) prev := math.Inf(1) if !first { prev = lim.capW } lim.set = true peakW := pool.offerW - if _, high, _, ok := s.evOffers.span(now); ok { - peakW = math.Max(peakW, high) + var logPeakW float64 + for _, c := range s.evLimit.offers { + if _, high, _, ok := c.log.span(now); ok { + logPeakW += high + } } + peakW = math.Max(peakW, logPeakW) // Our own battery orders the meter may not show yet count beside the // chargers' offers, in total import and on each phase. pending := s.pendingBatteryCommandsW(now) r := s.evMeterRoom(meter, gridW, pool.phases, fuseMaxW, s.withBatteryOrders(s.pendingLoad(at), pending)) _, fresh := qualifiedFuseObservation(meter, now) fresh = fresh && r.measured + lim.batteryYield.active = false + if fresh && !r.overLimit && r.limitW >= 0 { + lim.batteryYield = evBatteryYield{} + } lim.phaseRoomA, lim.phaseRoomKnown = 0, r.phaseRoomKnown if fresh { lim.phaseRoomA = r.phaseRoomA @@ -404,7 +866,7 @@ func (s *State) stepEVMeterLimit(meter *telemetry.DerReading, gridW float64, poo lim.phaseRoomA = r.startRoomA // The split between idle cars reads the start's room. } } - step := evPhaseMarginA * s.SiteFuseVoltage * pool.phases + step := evStepA * s.SiteFuseVoltage * pool.phases // The label follows whichever limit leaves less room. lim.byImport = math.Min(r.limitW, r.riseW) < r.fuseW // The fuse cuts on every reading over it, by its heat budget where the @@ -426,22 +888,65 @@ func (s *State) stepEVMeterLimit(meter *telemetry.DerReading, gridW float64, poo case at.Sub(lim.fuseUnderAt) >= evSettle: lim.fuseOverAt, lim.fuseCutAt = time.Time{}, time.Time{} } - rise, cover, latestW := s.unshownOrders(at, r.perPhase) - fuseCutW := r.fuseOverW + rise.w - cover.w - if r.perPhase { + rise, cover, aheadA, latestW := s.unshownOrders(at, r.perPhase) + fuseCutW, over := r.fuseOverW+rise.w-cover.w, r.fuseOverW > 0 + if r.perPhase && lim.phasesSeen { // A single-phase order may land on any phase, so the cut takes the - // most a rise adds to a phase and the least a cut takes off one. - // Past the current a gG fuse carries for an hour, nothing on its - // way holds back a cut. - coverA := min(cover.phaseA[0], cover.phaseA[1], cover.phaseA[2]) + // most a rise adds to a phase. A cut covers each phase by what it + // takes off there, the same on every phase unless the meter tells + // the phases apart (unshownOrders). Past the current a gG fuse + // carries for an hour, nothing on its way holds back a cut. + coverA := cover.phaseA if s.fuseHeat.worstA() > fuseHourFactor*s.SiteFuseAmps { - coverA = 0 + coverA = [3]float64{} + } + if slices.Max(coverA[:s.SiteFusePhases]) <= 0 { + lim.cutForSet = false } - fuseCutW = (r.fuseOverA + max(rise.phaseA[0], rise.phaseA[1], rise.phaseA[2]) - coverA) * s.SiteFuseVoltage * pool.phases + riseA := max(rise.phaseA[0], rise.phaseA[1], rise.phaseA[2]) + lineA, instantA, cutA := s.fuseLineA(), fuseInstantFactor*s.SiteFuseAmps, 0.0 + for i, a := range lim.phasesA[:s.SiteFusePhases] { + // Cuts on their way cover a phase up to the reading they were + // sent for. A phase that has risen since by more than noise + // carries an overage that came later: they cover only the + // part of it that phase had then. + phaseCoverA, aheadA := coverA[i], aheadA[i] + if lim.cutForSet && a > lim.cutForA[i]+evMoveA { + phaseCoverA = math.Min(phaseCoverA, math.Max(0, lim.cutForA[i]-lineA)) + } + // Where the heat budget is spent, the overage the meter shows + // comes off with our rise on its way. Elsewhere our own rise on + // its way is judged as the reading it will make: above + // fuseInstantFactor, or over the line on a phase whose heat + // budget is spent, it comes off now, as far as it takes the + // phase over the line. Under that, the heat budget judges it + // once the meter shows it, as any other reading. + hot, needA := s.fuseHeat.sq[i] > lineA*lineA, 0.0 + if landA := a + aheadA - phaseCoverA; landA > instantA || hot && landA > lineA { + needA = math.Min(aheadA-phaseCoverA, landA-lineA) + } + if a > lineA && (hot || a > instantA) { + needA = math.Max(needA, a+riseA-phaseCoverA-lineA) + } + cutA = math.Max(cutA, needA) + } + fuseCutW, over = cutA*s.SiteFuseVoltage*pool.phases, over || cutA > 0 } - over, capW, byImport := r.fuseOverW > 0, math.Inf(1), false + capW, byImport := math.Inf(1), false if over { - capW = latestW - math.Max(0, fuseCutW) + capW = latestW + if fuseCutW > 0 { + capW -= fuseCutW + if r.perPhase { + // A charger rounds to the nearest amp, so a cut that leaves + // half an amp would round back up and stay over the line. + // Only a new cut rounds. When cuts on their way cover the + // overage, the offer stays as it is, whole amps or not (a + // step table, a PV surplus); rounding it would lower it + // again on every reading. + capW = wholeStepsW(capW, step) + } + } } if r.limitW >= 0 { lim.overSince = time.Time{} @@ -457,27 +962,49 @@ func (s *State) stepEVMeterLimit(meter *telemetry.DerReading, gridW float64, poo } } if over { + requiredW := math.Max(0, latestW-capW) + if !byImport && pool.phases > 0 { + requiredW *= float64(s.SiteFusePhases) / pool.phases + } + if s.yieldBatteryBeforeEVCut(store, meter, r, pool, requiredW, gridW, pending, rise, fresh) { + capW = latestW // Keep the existing car order while the battery yields. + } lim.byImport = byImport lim.capW, lim.roomSince = math.Min(prev, math.Max(0, capW)), time.Time{} if lim.capW < prev { lim.cutAt = now + if lim.phasesSeen { + for i, a := range lim.phasesA { + if !lim.cutForSet || a > lim.cutForA[i] { + lim.cutForA[i] = a + } + } + lim.cutForSet = true + } if r.fuseOverW > 0 && lim.fuseCutAt.IsZero() { lim.fuseCutAt = now } slog.Info("EV hard limit: site meter over limit", "grid_w", gridW, "worst_phase_a", r.worstA, "fuse_heat_a", s.fuseHeat.worstA(), "import_limit_w", s.effectiveImportCeilingW(fuseMaxW), "overage_w", r.fuseOverW, "import_overage_w", math.Max(0, -r.limitW), - "unshown_rise_w", rise.w, "unshown_cut_w", cover.w, "offer_w", latestW, "cap_w", lim.capW) + "unshown_rise_w", rise.w, "unshown_cut_w", cover.w, "rise_on_its_way_a", aheadA, "offer_w", latestW, "cap_w", lim.capW) } return } // An increase needs a fresh reading and room below the level increases // may reach, left after our own battery commands that the meter may not // show yet (in r). Between that level and the import limit the cap holds. - room := 0.0 + // Battery discharge the fuse guard added ends once the site fits without + // it, so it is no room: a car that took it would keep the battery + // feeding the car at the line. + if reliefW := s.evFuseReliefW(store); reliefW > 0 { + r = s.evMeterRoom(meter, gridW, pool.phases, fuseMaxW, s.withBatteryOrders(s.pendingLoad(at), pending+reliefW)) + } + fuseRoom, importRoom := 0.0, 0.0 if fresh { - room = math.Max(0, math.Min(r.fuseW, r.riseW)) + fuseRoom, importRoom = math.Max(0, r.fuseW), math.Max(0, r.riseW) } + room := math.Min(fuseRoom, importRoom) if pool.offerW <= 0 { s.stepEVIdleStart(r, gridW, pool, first, gap, fresh, at, prevAt, pending, step) return @@ -486,15 +1013,100 @@ func (s *State) stepEVMeterLimit(meter *telemetry.DerReading, gridW float64, poo // within what this reading measured. lim.capW, lim.roomSince = math.Min(prev, peakW+room), time.Time{} // Right after a cut the meter may show battery relief that covers it. - if lim.capW < pool.wantW && room >= 2*step && now.Sub(lim.cutAt) >= evSettle { - // One step at most, and a step of room stays free. - lim.capW = math.Max(lim.capW, math.Min(lim.capW+step, peakW+room-step)) + remaining := pool.wantW - peakW + if lim.capW < pool.wantW && remaining > 0 && now.Sub(lim.cutAt) >= evSettle { + // The fuse: a rise moves one step, or the rest of the demand when + // that is less, once that move fits under the working limit. No + // room is held back below the line; evSettle after a cut and the + // heat the rise reads keep it from undoing a cut. + fits, riseW := fuseRoom+1e-6 >= math.Min(step, remaining), math.Min(lim.capW+step, peakW+fuseRoom) + // The hour's import limit keeps its own margin: a rise needs room + // for two steps, and one stays free. + if !math.IsInf(importRoom, 1) { + fits, riseW = fits && importRoom >= 2*step, math.Min(riseW, peakW+importRoom-step) + } + if fits { + lim.capW = math.Max(lim.capW, riseW) + } + } + // The import limit's rise keeps a step free, but the battery leaves room + // for the car's demand, not a further unused step. Let the last step use + // that measured room, or concurrent battery charge can keep the car + // short forever. Under the fuse alone the rise above already does this. + if s.hasBatteryCharge() && remaining > 0 && remaining <= step && room+1e-6 >= remaining && now.Sub(lim.cutAt) >= evSettle { + lim.capW = math.Max(lim.capW, pool.wantW) + } +} + +// evFuseReliefW is the fuse guard's battery relief the meter may show now: +// each battery's recorded relief, at most the discharge it reads. A battery +// with no live reading (watchdog-offline, or none yet) counts its whole +// relief: Core can no longer hold that discharge, and the car must not +// build on it. expireOfflineFuseRelief ends it evSettle after it went offline. +func (s *State) evFuseReliefW(store *telemetry.Store) float64 { + var w float64 + for driver, relief := range s.batteryFuseRelief { + if raw, _, ok := store.OnlineRawW(driver, telemetry.DerBattery); ok { + relief = math.Min(relief, math.Max(0, -raw)) + } + w += relief } + return w +} + +// expireOfflineFuseRelief ends each offline battery's recorded relief +// evSettle after dispatch first finds that battery offline. Core sent it its +// default mode then, and that command reaches the meter within evSettle, as +// ours do. Until then the car must not build on a discharge that may still +// stand; after it, holding the car below the line for relief Core no longer +// orders would be a hidden margin. It runs on every dispatch, whatever the +// other batteries do, so a dispatch that sends no command still ends it. A +// battery back online drops out of the clock: the next time it goes offline +// its relief counts whole again. +func (s *State) expireOfflineFuseRelief(online []batteryInfo) { + now := s.now() + live := make(map[string]bool, len(online)) + for _, b := range online { + live[b.driver] = true + } + var relief map[string]float64 + var since map[string]time.Time + for driver, w := range s.batteryFuseRelief { + if !live[driver] { + at, ok := s.batteriesOfflineSince[driver] + if !ok { + at = now + } + if now.Sub(at) >= evSettle { + continue + } + if since == nil { + since = make(map[string]time.Time) + } + since[driver] = at + } + if relief == nil { + relief = make(map[string]float64) + } + relief[driver] = w + } + s.batteryFuseRelief, s.batteriesOfflineSince = relief, since +} + +// wholeStepsW rounds w down to whole charger amps on the pool's phases, one +// step each. Chargers round an offer to the nearest amp, so a cap at the +// line's exact room could land half an amp over it. This is step +// quantisation, not a margin: a cap that is already whole amps stays. +func wholeStepsW(w, step float64) float64 { + if !(step > 0) || math.IsInf(w, 0) { + return w + } + return math.Floor(w/step+1e-9) * step } // stepEVIdleStart starts an idle pool once the meter has kept showing room -// for evRestartDwell: room on every reading for its minimum plus a step -// below the fuse. Under a measured hour, the ceiling's part is judged on the +// for evRestartDwell: room on every reading for its minimum below the +// fuse's working limit. Under a measured hour, the ceiling's part is judged on the // dwell's mean import, the house's load rather than one reading of it, and // the minimum needs no step of margin there: a short overage waits. func (s *State) stepEVIdleStart(r evMeterRoom, gridW float64, pool evPool, first, gap, fresh bool, at, prevAt time.Time, pending, step float64) { @@ -513,7 +1125,14 @@ func (s *State) stepEVIdleStart(r evMeterRoom, gridW float64, pool evPool, first if fresh { room = math.Max(0, perReadingW) // r holds our battery orders. } - if room < pool.minW+step { + // The fuse holds back nothing below its working limit: the charger + // rounds the cap down to a step it can draw. The hour's import limit + // keeps its separate rise margin. + keepW := 0.0 + if !r.hourly && r.riseW < r.startW { + keepW = step + } + if room+1e-6 < pool.minW+keepW { lim.roomSince = time.Time{} return } @@ -530,7 +1149,7 @@ func (s *State) stepEVIdleStart(r evMeterRoom, gridW float64, pool evPool, first if dwell < evRestartDwell { return } - lim.capW = room - step + lim.capW = math.Max(pool.minW, wholeStepsW(room-keepW, step)) if !r.hourly { return } @@ -555,7 +1174,7 @@ func EVLimitReason(state *State) string { // evMeterRoom is the measured room in EV watts. The fuse's part holds on // every reading. The import ceiling's parts are +Inf while it is off. type evMeterRoom struct { - fuseW float64 // below the fuse less its margin and the worst phase, read or heat, and what we sent; rises + fuseW float64 // below the working limit (fuseLineA) and the worst phase, read or heat, and what we sent; rises fuseOverW float64 // the fuse's measured overage that cuts now, 0 under it; see fuseHeat fuseOverA float64 // the same on the worst phase, with perPhase perPhase bool // the phase currents measure the fuse @@ -593,7 +1212,7 @@ func (s *State) evMeterRoom(meter *telemetry.DerReading, gridW, phases, fuseMaxW } r.measured = true if present { - limitA := s.SiteFuseAmps - evPhaseMarginA + limitA := s.fuseLineA() for _, a := range amps { r.worstA = math.Max(r.worstA, a) } diff --git a/go/internal/control/ev_fuse_budget_test.go b/go/internal/control/ev_fuse_budget_test.go index add26c76a..997bdd838 100644 --- a/go/internal/control/ev_fuse_budget_test.go +++ b/go/internal/control/ev_fuse_budget_test.go @@ -457,7 +457,7 @@ func TestEVHardLimitSlowChargerRisesDoNotStack(t *testing.T) { t.Run(fmt.Sprintf("declared_%.0fs/delay_%v", declaredS, delay), func(t *testing.T) { s := newEVMeterSite(t, 0, 15*690) s.chargerDelay, s.chargerMaxAgeS = delay, declaredS - limitA := s.state.SiteFuseAmps - evPhaseMarginA + limitA := s.state.fuseLineA() var maxA float64 var overSince time.Time watch := func(d time.Duration) { @@ -478,15 +478,15 @@ func TestEVHardLimitSlowChargerRisesDoNotStack(t *testing.T) { } } watch(10 * time.Minute) - // 24 - 15 A leaves 9 A; one step stays free. - if s.offer() != 5520 || s.evW != 5520 { - t.Fatalf("offer %.0f W draw %.0f W, want 5,520 W", s.offer(), s.evW) + // 24.5 - 15 A leaves 9.5 A: 9 A fits under the line. + if s.offer() != 6210 || s.evW != 6210 { + t.Fatalf("offer %.0f W draw %.0f W, want 6,210 W", s.offer(), s.evW) } s.setHouse(10 * 690) watch(15 * time.Minute) - // 24 - 10 A leaves 14 A; one step stays free. - if s.offer() != 8970 || s.evW != 8970 { - t.Fatalf("after the house fell: offer %.0f W draw %.0f W, want 8,970 W", s.offer(), s.evW) + // 24.5 - 10 A leaves 14.5 A: 14 A fits under the line. + if s.offer() != 9660 || s.evW != 9660 { + t.Fatalf("after the house fell: offer %.0f W draw %.0f W, want 9,660 W", s.offer(), s.evW) } t.Logf("worst phase peaked at %.1f A", maxA) }) @@ -505,15 +505,15 @@ func TestEVHardLimitUndeclaredFastChargerTakesRoomAtOnce(t *testing.T) { } startedAt := s.now s.run(s.chargerDelay + 2*time.Second) - // 24 - 15 A leaves 9 A; one step stays free. - if s.offer() != 5520 || s.evW != 5520 { - t.Fatalf("started at %.0f W, drawing %.0f W; want the 5,520 W room in one move", s.offer(), s.evW) + // 24.5 - 15 A leaves 9.5 A: 9 A fits under the line. + if s.offer() != 6210 || s.evW != 6210 { + t.Fatalf("started at %.0f W, drawing %.0f W; want the 6,210 W room in one move", s.offer(), s.evW) } - // 23.5 A measured, under the 24 A working limit, while the start still - // counts as flowing. - s.setHouse(15.5 * 690) + // 24.25 A measured, under the 24.5 A working limit, while the start + // still counts as flowing. + s.setHouse(15.25 * 690) s.run(time.Minute) - if got := offerWatts(s.offersSince(startedAt)); slices.Min(got) != 5520 { + if got := offerWatts(s.offersSince(startedAt)); slices.Min(got) != 6210 { t.Fatalf("the pending start cut the car: %v", got) } @@ -703,14 +703,15 @@ func evMeterBudgetFixture(t *testing.T, gridW float64, phaseA []float64, offerW return store, state, points, configs } -// A 3p car offered 15 A beside 11 A of house load on L1: 26 A on a 24 A -// working limit. With no earlier reading the fuse heat starts at 26 A, so -// two amps on three phases come off the offer at once. +// A 3p car offered 15 A beside 11 A of house load on L1: 26 A on the 25 A +// working limit, long enough to spend the heat budget, so one amp on three +// phases comes off the offer. func TestEVHardLimitTrimsPhaseOverageFromOffer(t *testing.T) { store, state, points, configs := evMeterBudgetFixture(t, 8280, []float64{26, 20, 20}, 10350, "3p") + lastingFuseHeat(store, state) got, known := EVFuseBudget(store, state, points, configs, 17250) - if !known || math.Abs(got-8970) > .01 { - t.Fatalf("cap %.1f W, want 10,350 - 2 A x 690 = 8,970 W", got) + if !known || math.Abs(got-9660) > .01 { + t.Fatalf("cap %.1f W, want 10,350 - 1 A x 690 = 9,660 W", got) } } @@ -727,19 +728,22 @@ func TestEVHardLimitStopsOnlyWhenOverageExceedsOfferLessMinimum(t *testing.T) { } } -// An idle car starts with a step of margin under the measured room, and a -// single-phase or auto car may land its whole offer on the worst phase. +// An idle car's minimum must fit below the working limit, the 25 A fuse +// with no safety margin, and a single-phase or auto car may land its whole +// offer on the worst phase. The start takes the whole room: the charger +// rounds it down to a step. func TestEVHardLimitIdleStartNeedsRoomOnDeclaredPhases(t *testing.T) { for _, tc := range []struct { name, phase string worstA float64 want float64 }{ - {"3p_room", "3p", 5, (24 - 5 - 1) * 690}, - {"legacy_default", "", 5, (24 - 5 - 1) * 690}, - {"1p_room", "1p", 5, (24 - 5 - 1) * 230}, - {"auto_room", "auto", 5, (24 - 5 - 1) * 230}, - {"3p_below_minimum", "3p", 21.5, 0}, // 1,725 W < 1,380 + 690 W + {"3p_room", "3p", 5, (25 - 5) * 690}, + {"legacy_default", "", 5, (25 - 5) * 690}, + {"1p_room", "1p", 5, (25 - 5) * 230}, + {"auto_room", "auto", 5, (25 - 5) * 230}, + {"3p_minimum_fits", "3p", 23, 1380}, // exactly the minimum + {"3p_below_minimum", "3p", 23.5, 0}, // 1,035 W < 1,380 W } { t.Run(tc.name, func(t *testing.T) { store, state, points, configs := evMeterBudgetFixture(t, tc.worstA*690, []float64{tc.worstA, tc.worstA, tc.worstA}, 0, tc.phase) @@ -764,9 +768,9 @@ func TestEVHardLimitBatteryOrderCountsOnEachPhase(t *testing.T) { phase string want float64 }{ - {"1p", (24 - 0.68 - 4703.0/690 - 1) * 230}, - {"auto", (24 - 0.68 - 4703.0/690 - 1) * 230}, - {"3p", (24 - 0.68 - 4703.0/690 - 1) * 690}, + {"1p", math.Floor(25-0.68-4703.0/690) * 230}, + {"auto", math.Floor(25-0.68-4703.0/690) * 230}, + {"3p", math.Floor(25-0.68-4703.0/690) * 690}, } { t.Run(tc.phase, func(t *testing.T) { store, state, points, configs := evMeterBudgetFixture(t, -466, []float64{-0.68, -0.68, -0.68}, 0, tc.phase) @@ -827,7 +831,7 @@ func TestEVHardLimitIgnoresChargerReadings(t *testing.T) { points[0].CommandedKnown = false store.Update("charger", telemetry.DerEV, 10242, nil, nil) store.RecordDriverSuccess("charger") - if got, _ := EVFuseBudget(store, state, points, configs, 17250); got > (24-20.45-1)*690+.01 { + if got, _ := EVFuseBudget(store, state, points, configs, 17250); got > (25-20.45)*690+.01 { t.Fatalf("a car reading earned %.1f W before our first command", got) } } diff --git a/go/internal/control/ev_fuse_cut_netting_test.go b/go/internal/control/ev_fuse_cut_netting_test.go index 575c8ad5b..8cbab6d6e 100644 --- a/go/internal/control/ev_fuse_cut_netting_test.go +++ b/go/internal/control/ev_fuse_cut_netting_test.go @@ -60,14 +60,19 @@ func TestEVFuseSlowChargerCutCoversItsOverage(t *testing.T) { } // cutRig feeds EVFuseBudget site readings one second apart and applies its -// cap to the charger's offer, in 1 A steps, as the controller does. +// cap to the charger's offer, in 1 A steps, as the controller does, or on +// the charger's step table when snap is set. type cutRig struct { t *testing.T store *telemetry.Store state *State now time.Time offerW float64 - phases int // the phases the offer lands on + phases int // the phases the offer lands on + snap func(w float64) float64 // nil: whole amps on three phases + // carPhases is the loadpoint's verdict on the car: 0 unknown, 1 one + // phase, 3 three phases. + carPhases int } func newCutRig(t *testing.T, offerW, declaredS float64) *cutRig { @@ -98,20 +103,27 @@ func (r *cutRig) deliver(gridW float64, data []byte) float64 { r.now = r.now.Add(time.Second) r.store.Update("meter", telemetry.DerMeter, gridW, nil, data) r.store.Get("meter", telemetry.DerMeter).UpdatedAt = r.now - _, _, points, configs := evMeterBudgetFixture(r.t, 0, nil, r.offerW, "3p") - points[0].CommandedPhases = r.phases + mode := "3p" + if r.phases == 1 { + mode = "1p" + } + _, _, points, configs := evMeterBudgetFixture(r.t, 0, nil, r.offerW, mode) + points[0].CommandedPhases, points[0].CarPhases = r.phases, r.carPhases capW, _ := EVFuseBudget(r.store, r.state, points, configs, 17250) if capW < r.offerW { r.offerW = math.Floor(capW/690) * 690 + if r.snap != nil { + r.offerW = r.snap(capW) + } } return capW } // A raise on its way adds to the cut. The car draws 10 A while its 14 A -// raise travels, and the house adds 8 A on L1: 26 A on the 24 A working -// limit. Once the heat budget is spent, the cut takes the 2 A the meter -// shows and the 4 A it does not show yet, so L1 ends at 24 A when the raise -// lands. Taking the overage from the raise alone left L1 at 28 A. +// raise travels, and the house adds 8 A on L1: 26 A on the 25 A working +// limit. Once the heat budget is spent, the cut takes the 1 A the meter +// shows and the 4 A it does not show yet, so L1 ends at 25 A when the raise +// lands. Taking the overage from the raise alone left L1 at 29 A. func TestEVFuseRaiseOnItsWayAddsToCut(t *testing.T) { r := newCutRig(t, 6900, 180) for range 300 { @@ -124,11 +136,11 @@ func TestEVFuseRaiseOnItsWayAddsToCut(t *testing.T) { r.read([3]float64{26, 15, 15}) cutAt = i } - if r.offerW != 5520 { - t.Fatalf("offer %.0f W after %d s over, want 9,660 W less 2 A measured and 4 A on its way = 5,520 W", r.offerW, cutAt) + if r.offerW != 6210 { + t.Fatalf("offer %.0f W after %d s over, want 9,660 W less 1 A measured and 4 A on its way = 6,210 W", r.offerW, cutAt) } for range 60 { - if r.read([3]float64{26, 15, 15}); r.offerW != 5520 { + if r.read([3]float64{26, 15, 15}); r.offerW != 6210 { t.Fatalf("cut again to %.0f W while the first cut travels", r.offerW) } } @@ -189,19 +201,20 @@ func TestEVFuseSevereOverageCutsWhateverIsOnItsWay(t *testing.T) { } } -// A cut covers only within the run over the limit it was sent in. A fast +// A cut the meter shows covers no overage that comes after it. A fast // charger that declares nothing applies a cut in 5 s, but counts as slow -// for 180 s. Once the meter has shown L1 under the limit for evSettle, a -// second step on L1 is a new run, and the landed cut covers none of it. -// Within evSettle of the landing it is the same run, and the landed cut -// covers the step until the run's first cut has had 180 s: that is the -// cost of trusting a cut on its way, at under 1.25 x In. +// for 180 s. The meter shows the cut land as every phase falling 7 A. A +// second step on L1, 30 s later in a new run or 10 s later in the same +// one, then cuts once L1's heat budget is spent, about 22-28 s at 29 A on +// the 24 A line. Before, a landed cut still covered the step in the same +// run until the run's first cut had had 180 s: L1 sat at 29 A for 2 min +// 55 s (#1914, TestSiteSimRandomHouse seed 1 at margin 0). func TestEVFuseLandedCutCoversOnlyItsRun(t *testing.T) { for _, tc := range []struct { after time.Duration // the second step after the first cut maxOver time.Duration }{ - {30 * time.Second, 30 * time.Second}, {10 * time.Second, evUnknownSettle}, + {30 * time.Second, 30 * time.Second}, {10 * time.Second, 30 * time.Second}, } { t.Run(fmt.Sprintf("step_after_%v", tc.after), func(t *testing.T) { s := fuseHeatSite(t) @@ -279,6 +292,40 @@ func TestEVFuseWithoutPhaseCurrentsCoversForSettle(t *testing.T) { } } +// The 22 Oct field site's Easee offers on a step table that is not whole +// amps (fieldNightLoadpoint). With L1 0.6 A over a hot 25 A line, the first +// cut takes 6,200 W to the 5,500 W step. While that cut travels to a slow +// charger it covers the overage, so the offer it left stays: rounding to +// whole amps applies to a new cut, not to an offer kept. Rounding the kept +// offer down took the car 6,200 -> 5,500 -> 4,800 -> 0 W in 3 s. +func TestEVFuseCoveredCutKeepsAStepTableOffer(t *testing.T) { + table := []float64{4800, 5500, 6200, 6900, 7600, 8300, 9000, 9700, 10400, 11100} + r := newCutRig(t, 6200, 180) + r.state.SiteFuseSafetyA = 0 + r.snap = func(w float64) float64 { + best := 0.0 + for _, step := range table { + if step <= w+1e-9 { + best = step + } + } + return best + } + for range 300 { + r.read([3]float64{25, 20, 20}) // heat at the line + } + var offers []float64 + for range 90 { + r.read([3]float64{25.6, 20, 20}) + if len(offers) == 0 || offers[len(offers)-1] != r.offerW { + offers = append(offers, r.offerW) + } + } + if len(offers) != 2 || offers[1] != 5500 { + t.Fatalf("offers %v, want one cut to 5,500 W held while it travels", offers) + } +} + // A change of phases is no cut. A 15 A single-phase order replaced by a 6 A // three-phase one lowers the current on the order's phase but adds 6 A to // the other two, so it covers nothing: with L2 at 32 A on the 25 A fuse the @@ -297,9 +344,9 @@ func TestEVFusePhaseChangeCoversNothing(t *testing.T) { } // After a restart, the offer Core finds in force is one it did not send in -// this run. A cut takes it as shown: the first reading over the limit trims -// the overage from it, and while that cut travels to a slow charger no -// further reading counts the found offer as a raise on its way. +// this run. A cut takes it as shown: the first reading that spends the heat +// trims the overage from it, and while that cut travels to a slow charger +// no further reading counts the found offer as a raise on its way. func TestEVFuseRestartTakesFoundOfferAsShown(t *testing.T) { s := fuseHeatSite(t) s.chargerDelay, s.chargerMaxAgeS = time.Minute, 180 @@ -307,11 +354,323 @@ func TestEVFuseRestartTakesFoundOfferAsShown(t *testing.T) { s.run(30 * time.Second) s.boot() bootAt := s.now - if took, offer := s.firstCut(5 * time.Second); took != 0 || offer != 8970 { - t.Fatalf("restart at 26 A cut after %v to %.0f W, want the first reading to 8,970 W", took, offer) + if took, offer := s.firstCut(5 * time.Second); took != time.Second || offer != 8970 { + t.Fatalf("restart at 26 A cut after %v to %.0f W, want the second reading to 8,970 W", took, offer) } s.run(50 * time.Second) if cuts := offerWatts(s.cutsSince(bootAt)); len(cuts) != 1 { t.Fatalf("cuts %v while the first travels", cuts) } } + +// A cut whose landing the meter cannot tell from an outside step covers a +// phase only up to the reading it was sent on. A house step takes L2 to +// 26 A; once its heat budget is spent, Core cuts 2 A. The second the cut +// lands, the house adds 8 A on L3, so the meter shows L1 and L2 falling +// 2 A and L3 rising 6 A: no charger's shape. L3 had no overage when the +// cut went out, so the cut covers none of L3's 2 A over the 24 A line, and +// L3 cuts once its own heat budget is spent (about 61 s from 20 A). Before, +// the landed cut covered L3 until the charger's 180 s had passed +// (TestSiteSimRandomHouse seed 68 at margin 0). +func TestEVFuseCutCoversOnlyThePhaseOverageItWasSentFor(t *testing.T) { + s := fuseHeatSite(t) + s.run(time.Minute) + s.addHouseA(0, 6, 0) // L2 26 A + if took, offer := s.firstCut(2 * time.Minute); took < 0 || offer != 8970 { + t.Fatalf("L2 cut after %v to %.0f W, want 8,970 W", took, offer) + } + s.run(s.chargerDelay - time.Second) + s.addHouseA(0, 0, 8) // lands with the cut: L3 26 A + var over time.Duration + for range 4 * 60 { + s.step() + if s.phaseA()[2] > 24 { + over += time.Second + } + } + if a := s.phaseA(); over > 70*time.Second || a[2] > 24 { + t.Fatalf("L3 over the 24 A line for %v, phases %.1f A after; cuts %v", over, a, offerWatts(s.cutsSince(time.Time{}))) + } + t.Logf("L3 over the working limit for %v", over) +} + +// C4 (27 Oct triage, seeds 207537 and 207539): our own rise on its way that +// no longer fits cuts before the meter shows it. On a 16 A fuse with no +// margin, a charger that declares 40 s takes the car from 6 A to 16 A with +// L1 at 6 A: the step fits under the line. 30 s later the house adds 9.7 A +// on L1 and 3 A on L2 before the raise lands. For a car the loadpoint has +// seen draw on three phases, L1 reads 15.7 A, under the line, but the raise +// would take it to 25.7 A, over 1.2 x In: on that reading the raise comes +// off as far as it takes L1 over the line, to 6 A in whole amps. Before, +// Core waited for the meter to show it. +func TestEVFuseRaiseOnItsWayThatNoLongerFitsCutsAtOnce(t *testing.T) { + r := newCutRig(t, 4140, 40) + r.state.SiteFuseAmps, r.carPhases = 16, 3 + for range 60 { + r.read([3]float64{6, 6, 6}) + } + r.offerW = 11040 // 6 -> 16 A + for i := range 30 { + if r.read([3]float64{6, 6, 6}); r.offerW != 11040 { + t.Fatalf("raise cut to %.0f W after %d s with L1 at 6 A", r.offerW, i+1) + } + } + if r.read([3]float64{15.7, 9, 6}); r.offerW != 4140 { + t.Fatalf("offer %.0f W with L1 at 15.7 A and 10 A on its way, want 4,140 W", r.offerW) + } +} + +// The C4 case as the triage wrote it, with only L1 moving, for a car the +// loadpoint has seen draw on three phases. A house step on L1 alone is then +// not the raise landing: the raise would take L1 to 25.7 A, over 1.2 x In, +// so it comes off on that reading. For a car not yet seen, a step on one +// phase may be a one-phase car taking the raise; the meter judges it then. +func TestEVFuseRaiseOnItsWayCutsForACarSeenOnThreePhases(t *testing.T) { + r := newCutRig(t, 4140, 40) + r.state.SiteFuseAmps, r.carPhases = 16, 3 + for range 60 { + r.read([3]float64{6, 6, 6}) + } + r.offerW = 11040 // 6 -> 16 A + for range 30 { + r.read([3]float64{6, 6, 6}) + } + if r.read([3]float64{15.7, 6, 6}); r.offerW > 4140 { + t.Fatalf("offer %.0f W with L1 at 15.7 A and 10 A on its way, want 4,140 W or less", r.offerW) + } +} + +// A house load that moves on one phase alone is not a three-phase cut +// landing. While a cut travels to a charger that declares 40 s, such a move +// must not take the cut's cover off the phase it was sent for. Before, a +// 3 A load switching off on L2 read as the cut landing: Core cut again to +// 7,590 W and left L1 2 A under the line for over 2 min. A 1.5 A load +// cycling on L2 gave three cuts and two rises; one cut is enough. +func TestEVFuseOnePhaseHouseMoveIsNotTheCutLanding(t *testing.T) { + t.Run("drop_on_L2", func(t *testing.T) { + s := fuseHeatSite(t) + s.chargerDelay, s.chargerMaxAgeS = 40*time.Second, 40 + s.addHouseA(0, 3, 0) // L2 23 A + s.run(2 * time.Minute) + stepAt := s.now + s.addHouseA(6, 0, 0) // L1 26 A + if took, offer := s.firstCut(2 * time.Minute); took < 0 || offer != 8970 { + t.Fatalf("first cut after %v to %.0f W, want 8,970 W", took, offer) + } + s.run(10 * time.Second) + s.addHouseA(0, -3, 0) + s.run(5 * time.Minute) + if cuts := offerWatts(s.cutsSince(stepAt)); len(cuts) != 1 || s.offer() != 8970 { + t.Fatalf("cuts %v, offer %.0f W after; want one cut to 8,970 W", cuts, s.offer()) + } + }) + t.Run("cycling_on_L2", func(t *testing.T) { + s := fuseHeatSite(t) + s.chargerDelay, s.chargerMaxAgeS = 40*time.Second, 40 + s.run(time.Minute) + stepAt := s.now + s.addHouseA(5, 0, 0) // L1 25 A on the 24 A line + for i := range 20 * 60 { + if i%7 == 0 { + s.addHouseA(0, []float64{1.5, -1.5}[i/7%2], 0) + } + s.step() + } + var offers []float64 + for _, o := range s.offersSince(stepAt) { + if n := len(offers); n == 0 || o.w != offers[n-1] { + offers = append(offers, o.w) + } + } + if len(offers) != 2 || offers[1] != 9660 { + t.Fatalf("offers %v, want one cut to 9,660 W and no rise", offers) + } + }) +} + +// While our own battery order moves the phases, a move of every phase +// alike may be the battery's, so it is not read as the car's cut landing. +// A 3 A cut travels to a charger that declares 40 s. A battery order of +// 1 A a phase discharge lands and goes back: L1 reads 28, 27, then 28 A +// again. Read as the cut landing, the 1 A drop took a third of the cut's +// cover, and Core cut a second time, to 8,280 W, for an overage the first +// cut already covers. +func TestEVFuseBatteryOrderMoveIsNotTheCutLanding(t *testing.T) { + r := newCutRig(t, 11040, 40) // 16 A + for range 120 { + r.read([3]float64{20, 15, 15}) + } + for i := 0; i < 120 && r.offerW == 11040; i++ { + r.read([3]float64{28, 15, 15}) // 3 A over the 25 A line + } + if r.offerW != 8970 { + t.Fatalf("heat cut to %.0f W, want 8,970 W", r.offerW) + } + r.state.batteryCommands = map[string]commandLog{"bat": commandLog{}.add(r.now.Add(-time.Minute), 0).add(r.now, -690)} + for range 3 { + r.read([3]float64{27, 14, 14}) + } + r.state.batteryCommands["bat"] = r.state.batteryCommands["bat"].add(r.now, 0) + for range 8 { + if r.read([3]float64{28, 15, 15}); r.offerW != 8970 { + t.Fatalf("offer %.0f W after our battery order came and went, want the one cut to 8,970 W", r.offerW) + } + } +} + +// A raise that fits lands, and no phase passes the line, so nothing cuts. +// 16 A fuse, charger that declares 40 s, car from 6 A to 14 A while every +// phase has 8 A of room. Before, C4 counted a landing only when the move +// had a charger's shape. A house load on L2 that switched off in the +// landing reading, or a car that ramped 0.4 A a reading, left the whole +// raise counted as on its way on top of a reading that already held it: +// 14 + 8 A read as over 1.2 x In, and Core cut to 5,520 or 6,900 W. +func TestEVFuseRaiseThatLandedIsNotCountedTwice(t *testing.T) { + for _, tc := range []struct { + name string + l2 float64 // L2 before the landing; it reads 14 A after + carPhases int + }{ + {"clean", 6, 0}, + {"house_off_on_L2_1.2A", 7.2, 0}, + {"house_off_on_L2_2A", 8, 0}, + {"house_off_on_L2_2A_car_seen_on_three_phases", 8, 3}, + } { + t.Run(tc.name, func(t *testing.T) { + r := newCutRig(t, 4140, 40) + r.state.SiteFuseAmps, r.carPhases = 16, tc.carPhases + for range 60 { + r.read([3]float64{6, tc.l2, 6}) + } + r.offerW = 9660 // 6 -> 14 A + for range 10 { + r.read([3]float64{6, tc.l2, 6}) + } + for i := range 120 { + if r.read([3]float64{14, 14, 14}); r.offerW != 9660 { + t.Fatalf("raise that landed at 14/14/14 A cut to %.0f W after %d s", r.offerW, i+1) + } + } + }) + } + for _, carPhases := range []int{0, 3} { + t.Run(fmt.Sprintf("slow_ramp_car_phases_%d", carPhases), func(t *testing.T) { + r := newCutRig(t, 4140, 40) + r.state.SiteFuseAmps, r.carPhases = 16, carPhases + for range 60 { + r.read([3]float64{6, 6, 6}) + } + r.offerW = 9660 + r.read([3]float64{6, 6, 6}) + a := 6.0 + for i := range 120 { + a = math.Min(14, a+0.4) + if r.read([3]float64{a, a, a}); r.offerW != 9660 { + t.Fatalf("raise ramping to 14 A cut to %.0f W at %.1f A after %d s", r.offerW, a, i+1) + } + } + }) + } +} + +// A one-phase car the loadpoint has not seen yet takes a three-phase raise +// on L1 while the house drifts on L2. 25 A fuse, 40 s charger: the car goes +// from 6 to 16 A, L1 lands at 24 A, and no phase is ever over the line. +// Before, the one-phase landing counted only while L2 stayed within 0.5 A +// of its reading when the raise went out, over the whole response time: +// 0.6 A of drift made the raise count as still to come, L1 read 34 A, and +// Core cut to 4,830 W one second after the landing (field matrix with +// 0.7 A on L2 every 17 s: car1p cut 11,100 -> 0 W). A house load of a +// step or more on L2 in the landing reading hid it too. Until the loadpoint +// has seen the car on three phases, the largest rise counts as the landing. +func TestEVFuseUnseenOnePhaseCarLandsBesideDrift(t *testing.T) { + for _, tc := range []struct { + name string + before, landA float64 // L2 drift before the landing, and in its reading + }{ + {"no_drift", 0, 0}, + {"drift_0.6A_before", 0.6, 0}, + {"drift_1A_before", 1, 0}, + {"drift_-0.6A_before", -0.6, 0}, + {"drift_0.7A_in_the_landing_reading", 0, 0.7}, + {"drift_-0.7A_in_the_landing_reading", 0, -0.7}, + {"house_1.2A_on_L2_in_the_landing_reading", 0, 1.2}, + {"house_-1.2A_on_L2_in_the_landing_reading", 0, -1.2}, + } { + t.Run(tc.name, func(t *testing.T) { + r := newCutRig(t, 4140, 40) // 6 A a phase ordered + for range 120 { + r.read([3]float64{14, 5, 5}) // car 6 A on L1, house 8 A + } + r.offerW = 11040 // 16 A + r.read([3]float64{14, 5, 5}) + for range 10 { + r.read([3]float64{14, 5 + tc.before, 5}) + } + l2 := 5 + tc.before + tc.landA + for i := range 120 { + if r.read([3]float64{24, l2, 5}); r.offerW != 11040 { + t.Fatalf("one-phase car landed at 24 A on a 25 A fuse, L2 %.1f A: cut to %.0f W after %d s", l2, r.offerW, i+1) + } + } + }) + } +} + +// A one-phase load that cycles on a phase is not a three-phase cut landing, +// even for a car not yet seen on three phases. The house falls from 18 to +// 10 A a phase, so the car rises toward the line, while 1.5 A on L2 +// toggles. Before, each drop of the load read as the cut landing on L2 and +// took its cover off there; the load's return could not give it back, so +// Core cut again and again: 9,660 down to 4,830 W and up, 21 cuts in 20 min +// with a 40 s charger. #1913 cut once. +func TestEVFuseLoadCyclingOnTheOverPhaseDoesNotHunt(t *testing.T) { + for _, period := range []int{3, 11} { + t.Run(fmt.Sprintf("every_%ds", period), func(t *testing.T) { + s := newEVMeterSite(t, 0, 18*690) + s.chargerDelay, s.chargerMaxAgeS = 40*time.Second, 40 + s.run(5 * time.Minute) + s.setHouse(10 * 690) + stepAt := s.now + loadA := 1.5 + for i := range 20 * 60 { + if i%period == 0 { + s.addHouseA(0, loadA, 0) + loadA = -loadA + } + s.step() + } + if cuts := offerWatts(s.cutsSince(stepAt)); len(cuts) > 1 { + t.Fatalf("cuts %v; one is enough", cuts) + } + }) + } +} + +// A one-phase order's car phase is not known, so a house load that moves +// alone on another phase is not its cut landing. 25 A fuse, a charger that +// declares 40 s, 16 A ordered on one phase: L1 reads 28 A and the heat cut +// takes it to 13 A. While that cut travels, a 3 A load on L2 switches off. +// Before, that move took 3 A of cover off the cut still on its way, and +// Core cut again for an overage the first cut already covered. +func TestEVFuseOnePhaseOrderHouseMoveIsNotTheCutLanding(t *testing.T) { + r := newCutRig(t, 3680, 40) // 16 A on one phase + r.phases = 1 + r.snap = func(w float64) float64 { return math.Floor(w/230) * 230 } + for range 120 { + r.read([3]float64{20, 15, 12}) + } + for i := 0; i < 120 && r.offerW == 3680; i++ { + r.read([3]float64{28, 15, 12}) // 3 A over the 25 A line + } + first := r.offerW + if first >= 3680 { + t.Fatalf("no heat cut with L1 3 A over the line") + } + r.read([3]float64{28, 15, 12}) + for i := range 20 { + if r.read([3]float64{28, 12, 12}); r.offerW != first { + t.Fatalf("offer %.0f W %d s after a 3 A house load on L2 switched off, want the one cut to %.0f W", r.offerW, i+1, first) + } + } +} diff --git a/go/internal/control/ev_fuse_idle_start_test.go b/go/internal/control/ev_fuse_idle_start_test.go index b8f8ffff6..33cc2fb70 100644 --- a/go/internal/control/ev_fuse_idle_start_test.go +++ b/go/internal/control/ev_fuse_idle_start_test.go @@ -23,10 +23,11 @@ func (s *evMeterSite) firstOfferSince(at time.Time) evTimed { // stopped. Its next start read the heat its own current had left on top of // the start it asked for: the cap stayed at 0 for 84 s, then gave 9 A. A // start reads the phases now plus the car's new current; the heat, under the -// working limit, does not hold it back. +// working limit, does not hold it back. Here the car runs at 14 A beside +// 10.5 A on L1, exactly the 24.5 A working limit. func TestEVIdleStartReadsThePhasesNotTheCarsOwnHeat(t *testing.T) { s := newEVMeterSite(t, 0, 0) - s.housePhaseA, s.houseW = [3]float64{8.9, 2, 2}, (8.9+2+2)*230 + s.housePhaseA, s.houseW = [3]float64{10.5, 2, 2}, (10.5+2+2)*230 s.run(10 * time.Minute) if s.offer() != 9660 || s.evW != 9660 { t.Fatalf("setup: offer %.0f, draw %.0f W, want 14 A on L1's working limit", s.offer(), s.evW) @@ -45,7 +46,7 @@ func TestEVIdleStartReadsThePhasesNotTheCarsOwnHeat(t *testing.T) { } // The 35 s restart dwell runs from the car's stop, 15 s before the ask. if wait := start.at.Sub(askedAt); wait > 25*time.Second || start.w < 8970 { - t.Fatalf("start %.0f W after %v with L1 at 8.9 A and 24 A to the working limit", start.w, wait) + t.Fatalf("start %.0f W after %v with L1 at 10.5 A and 24.5 A to the working limit", start.w, wait) } else { t.Logf("start %.0f W after %v", start.w, wait) } @@ -68,9 +69,9 @@ func TestEVIdleStartWaitsForTheHeatOfARealOverage(t *testing.T) { if start.at.IsZero() { t.Fatalf("no start within 3 min of the drop: offers %v", offerWatts(s.offersSince(droppedAt))) } - // L1's filtered current stays over 24 A for about 11 s after the drop; - // the dwell starts only then. - if wait := start.at.Sub(droppedAt); wait < evRestartDwell+10*time.Second { + // L1's filtered current stays over the 24.5 A line for about 8 s after + // the drop: 60 s × ln((26² - 8²)/(24.5² - 8²)). The dwell starts only then. + if wait := start.at.Sub(droppedAt); wait < evRestartDwell+7*time.Second { t.Fatalf("started %.0f W %v after the drop, before the heat of a 26 A overage had cooled", start.w, wait) } else { t.Logf("start %.0f W after %v", start.w, wait) diff --git a/go/internal/control/ev_fuse_offer_logged_late_test.go b/go/internal/control/ev_fuse_offer_logged_late_test.go new file mode 100644 index 000000000..d8674a3b2 --- /dev/null +++ b/go/internal/control/ev_fuse_offer_logged_late_test.go @@ -0,0 +1,60 @@ +package control + +import ( + "encoding/json" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// Site rig s1 and s15 (28 Oct 2026, c9b52c36): Charge now from idle on a +// 25 A or 16 A fuse, 3 A of house per phase covered by the battery, so every phase +// reads 0 A. The charger answers in about a second. Core logs an offer when +// the next control tick reads the loadpoint's command, 2 s after it went +// out, and that tick's meter reading came in 7 ms before it. The reading +// already shows the car at 15.9 A, but it is older than the logged offer. +// The landing then read as a raise not shown yet: 15.9 + 16 A passed +// 1.2 x In, and Core cut 11,040 W to 5,520 W with 9 A of room on every +// phase ("EV hard limit: site meter over limit ... worst_phase_a=15.9 +// rise_on_its_way_a=[16 16 16] cap_w=5520"). The car then crept back 1 A +// at a time and held 9,660 W on fuse_limit. Since #1923's ab027102 the +// rise still to come reads the meter's own rise since the raise went out. +func TestEVFuseRaiseShownBeforeItsOfferIsLogged(t *testing.T) { + for _, tc := range []struct { + name string + fuseA float64 + phases int + offerW float64 + landedA [3]float64 + }{ + {"s1_25A_fuse", 25, 3, 11040, [3]float64{15.9, 15.9, 15.9}}, + {"s1_16A_fuse", 16, 3, 10350, [3]float64{15, 15, 15}}, + {"s15_one_phase_charger_on_L1", 25, 1, 3680, [3]float64{16, 0, 0}}, + } { + t.Run(tc.name, func(t *testing.T) { + r := newCutRig(t, 0, 0) + r.phases, r.state.SiteFuseAmps = tc.phases, tc.fuseA + // Each reading comes in 7 ms before the tick that reads it. + read := func(phaseA [3]float64) { + t.Helper() + data, _ := json.Marshal(map[string]any{"l1_a": phaseA[0], "l2_a": phaseA[1], "l3_a": phaseA[2]}) + r.now = r.now.Add(2 * time.Second) + r.store.Update("meter", telemetry.DerMeter, (phaseA[0]+phaseA[1]+phaseA[2])*230, nil, data) + r.store.Get("meter", telemetry.DerMeter).UpdatedAt = r.now.Add(-7 * time.Millisecond) + _, _, points, configs := evMeterBudgetFixture(t, 0, nil, r.offerW, "3p") + points[0].CommandedPhases = r.phases + if capW, _ := EVFuseBudget(r.store, r.state, points, configs, tc.fuseA*690); capW < r.offerW { + t.Fatalf("offer %.0f W cut to %.0f W with the car at %v A on a %.0f A fuse", r.offerW, capW, phaseA, tc.fuseA) + } + } + for range 15 { + read([3]float64{}) + } + r.offerW = tc.offerW // sent at the last tick, logged at this one + for range 60 { + read(tc.landedA) + } + }) + } +} diff --git a/go/internal/control/ev_fuse_one_phase_start_test.go b/go/internal/control/ev_fuse_one_phase_start_test.go new file mode 100644 index 000000000..49a39ab6b --- /dev/null +++ b/go/internal/control/ev_fuse_one_phase_start_test.go @@ -0,0 +1,123 @@ +package control + +import ( + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/loadpoint" +) + +// C6 (rig seeds 207534, 207536, 207537, 207531): an idle pool whose order +// lands on one phase needs room for 6 A on that phase, not for the +// configured three-phase minimum of 4,140 W. Without that, Charge now on 1p +// or auto never starts on a 16 A fuse ("below minimum step … fuse_cap_w=0 +// min_step_w=1380"). +func TestIdleOnePhasePoolStartsOnSixAmpsOfRoom(t *testing.T) { + steps := []float64{0, 1380, 1610, 1840, 2070, 2300, 2530, 2760, 2990, 3220, 3450, 3680, + 4140, 4830, 5520, 6210, 6900, 7590, 8280, 8970, 9660, 10350, 11040} + for _, tc := range []struct { + name string + fuseA float64 + houseA float64 + mode string + holdW float64 + }{ + {"16A 1p", 16, 4.5, "1p", 3680}, + {"25A 1p", 25, 13.7, "1p", 3680}, + {"16A auto", 16, 4.5, "auto", 3680}, + {"25A auto", 25, 13.7, "auto", 3680}, + } { + t.Run(tc.name, func(t *testing.T) { + cfg := loadpoint.Config{ID: "garage", DriverName: "easee", MinChargeW: 4140, MaxChargeW: 11040, + PhaseMode: "auto", AllowedStepsW: steps} + s := newPhaseSite(t, cfg, tc.houseA) + fuseW := tc.fuseA * 230 * 3 + s.state.SiteFuseAmps = tc.fuseA + s.ctrl.SetSiteFuse(loadpoint.SiteFuse{MaxAmps: tc.fuseA, Voltage: 230, PhaseCnt: 3}) + s.ctrl.SetFuseEVMax(func() (float64, bool) { + return EVFuseBudget(s.store, s.state, s.mgr.States(), s.ctrl.FuseConfigs(), fuseW) + }) + s.ctrl.SetManualHold(cfg.ID, loadpoint.ManualHold{PowerW: tc.holdW, PhaseMode: tc.mode, Persistent: true}) + start := s.now + deadline := start.Add(evRestartDwell + 5*time.Second) // the dwell plus one 5 s tick + for s.now.Before(deadline) && s.offerW < 1380 { + s.tick() + } + st, _ := s.mgr.State(cfg.ID) + if s.offerW < 1380 { + t.Fatalf("after %s: offer %.0f W, EV cap %.0f W, reason %q; want at least 1380 W", + s.now.Sub(start), s.offerW, s.state.evLimit.capW, st.CommandedReason) + } + t.Logf("started after %s at %.0f W on one phase", s.now.Sub(start), s.offerW) + if !s.offer1p { + t.Fatalf("offer %.0f W runs on three phases; want one", s.offerW) + } + for i := 0; i < 12; i++ { + s.tick() + if s.worstA() > tc.fuseA { + t.Fatalf("offer %.0f W puts %.1f A on a %.0f A fuse", s.offerW, s.worstA(), tc.fuseA) + } + } + }) + } +} + +// The start point is exact: on a 16 A fuse an idle 1p or auto pool starts at +// 1,380 W, its least order, once the worst phase leaves 6 A of room under the +// working line, and waits while it leaves 5.7 A. A hidden margin on the +// one-phase minimum would move the start and fail here. The cases are set +// from the working line as EVFuseBudget reports it, not from the constant +// that sets it, so they follow the line if it moves. +func TestIdleOnePhasePoolStartBoundary(t *testing.T) { + steps := []float64{0, 1380, 1610, 1840, 2070, 2300, 2530, 2760, 2990, 3220, 3450, 3680, + 4140, 4830, 5520, 6210, 6900, 7590, 8280, 8970, 9660, 10350, 11040} + const fuseA = 16.0 + site := func(t *testing.T, houseA float64, mode string) *phaseSite { + cfg := loadpoint.Config{ID: "garage", DriverName: "easee", MinChargeW: 4140, MaxChargeW: 11040, + PhaseMode: "auto", AllowedStepsW: steps} + s := newPhaseSite(t, cfg, houseA) + s.state.SiteFuseAmps = fuseA + s.ctrl.SetSiteFuse(loadpoint.SiteFuse{MaxAmps: fuseA, Voltage: 230, PhaseCnt: 3}) + s.ctrl.SetFuseEVMax(func() (float64, bool) { + return EVFuseBudget(s.store, s.state, s.mgr.States(), s.ctrl.FuseConfigs(), fuseA*230*3) + }) + s.ctrl.SetManualHold(cfg.ID, loadpoint.ManualHold{PowerW: 3680, PhaseMode: mode, Persistent: true}) + return s + } + // With no house load, an idle pool's room on its busiest phase is the + // whole working line. + probe := site(t, 0, "1p") + probe.tick() + lineA := probe.state.evLimit.phaseRoomA + if !probe.state.evLimit.phaseRoomKnown || lineA <= 6 || lineA > fuseA { + t.Fatalf("working line %.2f A (known %v) on a %.0f A fuse", lineA, probe.state.evLimit.phaseRoomKnown, fuseA) + } + for _, mode := range []string{"1p", "auto"} { + t.Run(mode+" 6.0 A of room starts", func(t *testing.T) { + s := site(t, lineA-6, mode) + start := s.now + for s.now.Sub(start) < evRestartDwell+5*time.Second && s.offerW == 0 { + s.tick() + } + if s.offerW != 1380 || !s.offer1p { + t.Fatalf("worst phase %.1f A: offer %.0f W, one phase %v; want 1380 W on one phase", + lineA-6, s.offerW, s.offer1p) + } + for i := 0; i < 60; i++ { + s.tick() + if s.worstA() > fuseA { + t.Fatalf("tick %d: %.2f A on a %.0f A fuse", i, s.worstA(), fuseA) + } + } + }) + t.Run(mode+" 5.7 A of room waits", func(t *testing.T) { + s := site(t, lineA-5.7, mode) + for i := 0; i < 40; i++ { + s.tick() + if s.offerW != 0 { + t.Fatalf("tick %d: offer %.0f W with 5.7 A of room; want 0", i, s.offerW) + } + } + }) + } +} diff --git a/go/internal/control/ev_fuse_phase_room_test.go b/go/internal/control/ev_fuse_phase_room_test.go index c2347ac39..f35cb207a 100644 --- a/go/internal/control/ev_fuse_phase_room_test.go +++ b/go/internal/control/ev_fuse_phase_room_test.go @@ -137,11 +137,11 @@ func (s *roomSite) run(d time.Duration) { } } -// requireWithinLimit fails when a phase passed the 24 A working limit. +// requireWithinLimit fails when a phase passed the working limit (fuseLineA). func (s *roomSite) requireWithinLimit(after string) { s.t.Helper() - if limitA := s.state.SiteFuseAmps - evPhaseMarginA; s.worstA > limitA+1e-6 { - s.t.Fatalf("%s: a phase reached %.1f A, over the %.0f A working limit", after, s.worstA, limitA) + if limitA := s.state.fuseLineA(); s.worstA > limitA+1e-6 { + s.t.Fatalf("%s: a phase reached %.1f A, over the %.1f A working limit", after, s.worstA, limitA) } } @@ -244,16 +244,16 @@ func TestEVFusePhaseRoomHoldsOnSilentMeter(t *testing.T) { } return roomA } - if got := room(); math.Abs(got-9) > 1e-9 { - t.Fatalf("room %.2f A beside 15 A, want 9 A", got) + if got := room(); math.Abs(got-10) > 1e-9 { + t.Fatalf("room %.2f A beside 15 A, want 10 A", got) } points[0].CommandedW = 5520 now = now.Add(time.Second) - if got := room(); math.Abs(got-1) > 1e-9 { - t.Fatalf("room %.2f A after an 8 A offer, want 1 A", got) + if got := room(); math.Abs(got-2) > 1e-9 { + t.Fatalf("room %.2f A after an 8 A offer, want 2 A", got) } now = now.Add(10 * time.Minute) - if got := room(); math.Abs(got-1) > 1e-9 { - t.Fatalf("room %.2f A after ten silent minutes, want 1 A", got) + if got := room(); math.Abs(got-2) > 1e-9 { + t.Fatalf("room %.2f A after ten silent minutes, want 2 A", got) } } diff --git a/go/internal/control/ev_priority_sitesim_test.go b/go/internal/control/ev_priority_sitesim_test.go new file mode 100644 index 000000000..028bed013 --- /dev/null +++ b/go/internal/control/ev_priority_sitesim_test.go @@ -0,0 +1,84 @@ +package control + +import ( + "fmt" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/loadpoint" + "github.com/srcfl/ftw/go/internal/sitesim" +) + +// Exercise the whole loop, including delayed physical responses and fuse +// heating. A small house step must take battery charge first; a house +// that cannot fit the car must stop it, then let it restart without input. +func TestIssue1868ClosedLoopPriorityAndRecovery(t *testing.T) { + for _, planned := range []bool{false, true} { + for _, phases := range []int{1, 3} { + for _, delay := range []time.Duration{5 * time.Second, 40 * time.Second, 120 * time.Second} { + t.Run(fmt.Sprintf("plan=%v/phases=%d/charger=%s", planned, phases, delay), func(t *testing.T) { + minimum := 6 * 230 * float64(phases) + sc := simScenario{name: "issue1868", run: 20 * time.Minute, + plant: func(p *sitesim.Plant) { + p.Fuse.RatingA, p.HouseW = 16, houseA(3, 2, 2) + p.EVSE.Phases, p.EVSE.Delay = phases, delay + p.Car.Phases, p.Car.MaxW = phases, minimum + p.Battery = &sitesim.Battery{Phases: 3, MaxChargeW: 6000, MaxDischargeW: 6000, CapacityWh: 12200, SoC: .3, Latency: 2 * time.Second} + p.Events = []sitesim.Event{ + {At: 5 * time.Minute, Kind: sitesim.Change, Do: func(p *sitesim.Plant) { p.HouseW = houseA(3.1, 2, 2) }}, + {At: 10 * time.Minute, Kind: sitesim.Change, Do: func(p *sitesim.Plant) { p.HouseW = houseA(16, 16, 16) }}, + {At: 14 * time.Minute, Kind: sitesim.Change, Do: func(p *sitesim.Plant) { p.HouseW = houseA(3, 2, 2) }}, + } + }, + lp: func(c *loadpoint.Config) { + c.MinChargeW, c.MaxChargeW = minimum, minimum + c.PhaseMode = fmt.Sprintf("%dp", phases) + c.AllowedStepsW = []float64{0, minimum} + }, + setup: planBatteryCharge(6000), + } + if planned { + sc.plan = planEV(minimum) + } + sc.expect = []simExpect{ + + {9 * time.Minute, func(s *simSite) error { + if s.p.EVSE.DrawW < minimum-.1 { + return fmt.Errorf("car yielded to battery: car %.0f, battery %.0f W", s.p.EVSE.DrawW, s.batteryW()) + } + for _, cmd := range s.commands { + if cmd.device == "evse" && cmd.at.After(simStart.Add(5*time.Minute)) && cmd.w < minimum-.1 { + return fmt.Errorf("small step cut the car: %v", cmd) + } + } + return nil + }}, + {14*time.Minute - time.Second, func(s *simSite) error { + if s.p.EVSE.DrawW > 0 { + return fmt.Errorf("car did not stop when house used the fuse: %.0f W", s.p.EVSE.DrawW) + } + return nil + }}, + {20*time.Minute - time.Second, func(s *simSite) error { + if s.p.EVSE.DrawW < minimum-.1 { + return fmt.Errorf("car failed to restart: %.0f W; battery %.0f W", s.p.EVSE.DrawW, s.batteryW()) + } + return nil + }}, + } + s := runSimScenario(sc, simStart) + + if s.p.Fuse.Blown != [3]bool{} { + t.Fatalf("fuse opened: %+v", s.p.Fuse) + } + // Outside house steps may exceed the working limit during device + // response. The fixture checks unsafe raises and sustained overages. + if s.failure != "" { + t.Fatal(s.failure) + } + t.Logf("car %.0f W, residual battery %.0f W, peak phase %.2f A", s.p.EVSE.DrawW, s.batteryW(), s.peakA) + }) + } + } + } +} diff --git a/go/internal/control/ev_site_test.go b/go/internal/control/ev_site_test.go index 6a079b0d8..8458c3998 100644 --- a/go/internal/control/ev_site_test.go +++ b/go/internal/control/ev_site_test.go @@ -178,10 +178,12 @@ func TestEVSiteScheduledEVMayImportOnCheapNight(t *testing.T) { } } -func TestEVSiteSurplusOnlyDoesNotClaimPVSoakWhenTogetherTheyWouldImport(t *testing.T) { +func TestEVSiteSurplusOnlyTakesPVSoakWithoutImport(t *testing.T) { // Battery plan 4 kW of 7.5 kW leftover — soak, not Pixii buying. - // The old live reader treated any meter import as grid-charge and - // offered the full leftover, so a 3Φ snap held while soak+EV imported. + // The car comes before battery charge (owner decision P1, 27 Oct + // 2026), so the car may take the soak; the battery takes what the + // car leaves. Together they never import, and the battery never + // discharges for the car. start := evComboSiteStart() site := newSiteClock(t, evSiteConfig{ Start: start, @@ -191,14 +193,17 @@ func TestEVSiteSurplusOnlyDoesNotClaimPVSoakWhenTogetherTheyWouldImport(t *testi PVW: evComboPVW, }) site.run(12) - const soakHeadroom = 3500.0 // leftover 7500 − soak 4000 for _, rec := range site.ticks { if rec.N < 4 { continue } - if rec.EVW > soakHeadroom+loadpoint.SitePowerEpsW { - t.Fatalf("tick %d: surplus-only EV %.0f W claimed PV-soak (headroom %.0f); ticks=%s", - rec.N, rec.EVW, soakHeadroom, site.dumpTicks()) + if rec.GridW > loadpoint.SitePowerEpsW { + t.Fatalf("tick %d: surplus-only EV %.0f W and battery %.0f W import %.0f W; ticks=%s", + rec.N, rec.EVW, rec.BatW, rec.GridW, site.dumpTicks()) + } + if rec.BatW < -loadpoint.SitePowerEpsW { + t.Fatalf("tick %d: battery discharged %.0f W beside surplus-only EV %.0f W; ticks=%s", + rec.N, rec.BatW, rec.EVW, site.dumpTicks()) } } } diff --git a/go/internal/control/ev_slow_charger_battery_test.go b/go/internal/control/ev_slow_charger_battery_test.go new file mode 100644 index 000000000..dfef0c33c --- /dev/null +++ b/go/internal/control/ev_slow_charger_battery_test.go @@ -0,0 +1,86 @@ +package control + +import ( + "fmt" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/loadpoint" + "github.com/srcfl/ftw/go/internal/sitesim" +) + +// A charger that declares a 180 s response time, as Easee Cloud does, and +// takes delay to start, beside a planned 9.4 kW battery charge on a 25 A +// site with 3 A of house load per phase. The car's 11 kW offer stays +// unshown until the charger applies it, and the battery must not take that +// room: when the car starts, no phase may pass the instant limit, and the +// car must not sit idle with room. +func TestSlowChargerUnshownOfferKeepsBatteryOut(t *testing.T) { + for _, delay := range []time.Duration{5 * time.Second, 30 * time.Second, 40 * time.Second, 2 * time.Minute} { + sc := simScenario{name: "slow charger", run: 15 * time.Minute, + plant: func(p *sitesim.Plant) { + p.HouseW, p.EVSE.Delay = houseA(3, 3, 3), delay + p.Battery = &sitesim.Battery{Phases: 3, MaxChargeW: 9400, MaxDischargeW: 9400, CapacityWh: 20000, SoC: .3, Latency: 3 * time.Second} + }, + setup: func(s *simSite) { + s.evMaxAge = evUnknownSettle + planBatteryCharge(9400)(s) + }} + s := runSimScenario(sc, simStart) + t.Logf("delay %v: peak phase %.1f A, %v over %.1f A, idle with room %v, car %.0f W, battery %.0f W", + delay, s.peakA, s.overLimit, s.state.fuseLineA(), s.idleWithRoom, s.p.EVSE.DrawW, s.p.Battery.W) + if s.failure != "" { + t.Errorf("delay %v: %s", delay, s.failure) + } + if limitA := fuseInstantFactor * s.p.Fuse.RatingA; s.peakA > limitA { + t.Errorf("delay %v: peak phase %.1f A above the instant limit %.1f A", delay, s.peakA, limitA) + } + } +} + +// #1868's rig shape (s18): a 16 A site with 3 A of house load, a local +// charger that declares no response time and draws within 2 s, a car that +// takes only [0, 4140] W, and a 6000 W battery charge. Once the car draws, +// the reading shows it, and the battery shares the rest of the fuse well +// before evUnknownSettle; s18's judge needs it above 1000 W 100-120 s after +// Charge now. +func TestFastChargerYoungOfferLetsBatteryShare(t *testing.T) { + for _, tc := range []struct { + name string + setup func(*simSite) + }{ + {"manual hold", func(s *simSite) { + s.state.SetBatteryManualHold(BatteryManualHold{PowerW: 6000, ExpiresAt: s.p.Now.Add(time.Hour)}) + }}, + {"plan", planBatteryCharge(6000)}, + } { + t.Run(tc.name, func(t *testing.T) { + var expect []simExpect + for _, at := range []time.Duration{60 * time.Second, 100 * time.Second, 120 * time.Second, 170 * time.Second} { + expect = append(expect, simExpect{at, func(s *simSite) error { + if s.p.EVSE.DrawW < 4100 || s.p.Battery.W < 3000 { + return fmt.Errorf("car %.0f W and battery %.0f W at %v; both should charge", s.p.EVSE.DrawW, s.p.Battery.W, at) + } + return nil + }}) + } + sc := simScenario{name: "s18 " + tc.name, run: 4 * time.Minute, + plant: func(p *sitesim.Plant) { + p.Fuse.RatingA, p.HouseW, p.EVSE.Delay, p.EVSE.ReadbackLag = 16, houseA(3, 3, 3), 2*time.Second, 0 + p.Battery = &sitesim.Battery{Phases: 3, MaxChargeW: 6000, MaxDischargeW: 6000, CapacityWh: 20000, SoC: .4, Latency: 2 * time.Second} + }, + lp: func(c *loadpoint.Config) { + c.MinChargeW, c.MaxChargeW, c.AllowedStepsW = 4140, 4140, []float64{0, 4140} + }, + setup: tc.setup, + expect: expect} + s := runSimScenario(sc, simStart) + if s.failure != "" { + t.Error(s.failure) + } + if limitA := fuseInstantFactor * s.p.Fuse.RatingA; s.peakA > limitA { + t.Errorf("peak phase %.1f A above the instant limit %.1f A", s.peakA, limitA) + } + }) + } +} diff --git a/go/internal/control/frozen_meter_test.go b/go/internal/control/frozen_meter_test.go new file mode 100644 index 000000000..4d1cdbd77 --- /dev/null +++ b/go/internal/control/frozen_meter_test.go @@ -0,0 +1,197 @@ +package control + +import ( + "encoding/json" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// The site meter's reads fail (held reading, no new UpdatedAt) while the +// battery keeps reporting fresh telemetry. Within the 60 s watchdog window +// dispatch must not act on the held meter value again and again. +func runFrozenMeter(t *testing.T, heldGridW, realHouseW float64) (lastCmd, realGrid float64) { + t.Helper() + store := telemetry.NewStore() + store.Update("meter", telemetry.DerMeter, heldGridW, nil, nil) + store.DriverHealthMut("meter").RecordSuccess() + soc := 0.5 + store.Update("bat", telemetry.DerBattery, 0, &soc, nil) + store.DriverHealthMut("bat").RecordSuccess() + + st := NewState(0, 50, "meter") + st.Mode = ModeSelfConsumption + clk := time.Now() + st.clock = func() time.Time { return clk } + capsMap := caps(map[string]float64{"bat": 10000}) + st.DriverLimits = map[string]PowerLimits{"bat": {MaxChargeW: 10000, MaxDischargeW: 10000}} + frozenAt := store.Get("meter", telemetry.DerMeter).UpdatedAt + + batW := 0.0 + for tick := 0; tick < 10; tick++ { // 10 ticks x 5 s = 50 s < 60 s watchdog + // Meter poll fails: RecordError, reading held. + store.DriverHealthMut("meter").RecordError("modbus timeout") + targets := ComputeDispatch(store, st, capsMap, 3*25*230) + for _, tg := range targets { + if tg.Driver == "bat" { + batW = tg.TargetW + } + } + // Battery follows its command and reports it (fresh). + for i := 0; i < 30; i++ { + store.Update("bat", telemetry.DerBattery, batW, &soc, nil) + } + store.DriverHealthMut("bat").RecordSuccess() + t.Logf("tick %d: bat cmd %.0f W, real grid %.0f W, meter says %.0f W", + tick, batW, realHouseW+batW, store.Get("meter", telemetry.DerMeter).SmoothedW) + clk = clk.Add(5 * time.Second) + } + if !store.Get("meter", telemetry.DerMeter).UpdatedAt.Equal(frozenAt) { + t.Fatal("meter reading moved") + } + return batW, realHouseW + batW +} + +func TestFrozenImportMeterDoesNotDischargeToExport(t *testing.T) { + cmd, grid := runFrozenMeter(t, 2000, 2000) + if grid < -500 { + t.Fatalf("frozen +2000 W meter drove battery to %.0f W; real grid %.0f W (export)", cmd, grid) + } +} + +func TestFrozenExportMeterDoesNotChargeFromGrid(t *testing.T) { + cmd, grid := runFrozenMeter(t, -3000, 1500) + if grid > 3000 { + t.Fatalf("frozen -3000 W meter drove battery to %.0f W; real grid %.0f W (import)", cmd, grid) + } +} + +// A meter that stops updating after a good sample holds the battery at its +// last order and lets a manual hold only cut toward 0. The next new sample +// releases the hold. +func TestHeldMeterOnlyCutsTowardZero(t *testing.T) { + store := seedStore(2000, []struct { + name string + currentW, soc float64 + }{{"bat", 0, 0.5}}) + st := NewState(0, 50, "ferroamp") + st.Mode = ModeSelfConsumption + st.MinDispatchIntervalS = 0 + st.SlewEnabled = false + capsMap := caps(map[string]float64{"bat": 10000}) + st.DriverLimits = map[string]PowerLimits{"bat": {MaxChargeW: 10000, MaxDischargeW: 10000}} + + first := targetFor(t, ComputeDispatch(store, st, capsMap, 3*25*230), "bat").TargetW + if first >= 0 { + t.Fatalf("fresh +2000 W import did not discharge: %.0f W", first) + } + store.Update("bat", telemetry.DerBattery, first, ptrF64(0.5), nil) + if got := ComputeDispatch(store, st, capsMap, 3*25*230); len(got) != 0 { + t.Fatalf("held meter sample produced new orders: %+v", got) + } + + st.SetBatteryManualHold(BatteryManualHold{PowerW: first / 2, ExpiresAt: time.Now().Add(time.Hour)}) + if got := targetFor(t, ComputeDispatch(store, st, capsMap, 3*25*230), "bat").TargetW; got != first/2 { + t.Fatalf("cut toward 0 on a held sample: got %.0f W, want %.0f W", got, first/2) + } + st.SetBatteryManualHold(BatteryManualHold{PowerW: 3000, ExpiresAt: time.Now().Add(time.Hour)}) + if got := targetFor(t, ComputeDispatch(store, st, capsMap, 3*25*230), "bat").TargetW; got != 0 { + t.Fatalf("held sample allowed a reversal to charge: %.0f W", got) + } + + store.Update("ferroamp", telemetry.DerMeter, 2000, nil, nil) + if got := targetFor(t, ComputeDispatch(store, st, capsMap, 3*25*230), "bat").TargetW; got != 3000 { + t.Fatalf("new sample did not release the hold: %.0f W", got) + } +} + +func heldMeterState() (*State, map[string]float64) { + st := NewState(0, 50, "ferroamp") + st.Mode = ModeSelfConsumption + st.MinDispatchIntervalS = 0 + st.SlewEnabled = false + st.DriverLimits = map[string]PowerLimits{"bat": {MaxChargeW: 10000, MaxDischargeW: 10000}} + return st, caps(map[string]float64{"bat": 10000}) +} + +func reportBattery(store *telemetry.Store, w float64, data json.RawMessage) { + soc := 0.5 + for i := 0; i < 40; i++ { + store.Update("bat", telemetry.DerBattery, w, &soc, data) + } + store.DriverHealthMut("bat").RecordSuccess() +} + +func sentTo(targets []DispatchTarget, driver string) *DispatchTarget { + for i := range targets { + if targets[i].Driver == driver { + return &targets[i] + } + } + return nil +} + +// A held meter must not keep a battery discharging after max_discharge_w +// becomes an explicit 0. The deadband exit falls through for this; so must +// the held exit. +func TestHeldMeterStillSendsExplicitZeroDischarge(t *testing.T) { + store := seedStore(2000, []struct { + name string + currentW, soc float64 + }{{"bat", 0, 0.5}}) + st, capsMap := heldMeterState() + first := targetFor(t, ComputeDispatch(store, st, capsMap, 3*25*230), "bat").TargetW + if first >= 0 { + t.Fatalf("fresh +2000 W import did not discharge: %.0f W", first) + } + reportBattery(store, first, nil) + st.DriverLimits = map[string]PowerLimits{"bat": {MaxChargeW: 10000, MaxDischargeW: 0, MaxDischargeWSet: true}} + sent := sentTo(ComputeDispatch(store, st, capsMap, 3*25*230), "bat") + if sent == nil || sent.TargetW < 0 { + t.Fatalf("held meter left battery discharging at %.0f W under max_discharge_w=0: %+v", first, sent) + } +} + +// A held meter must not keep a battery charging after its driver reports +// charge_capable=false. +func TestHeldMeterStillStopsBlockedCharge(t *testing.T) { + store := seedStore(-2000, []struct { + name string + currentW, soc float64 + }{{"bat", 0, 0.5}}) + st, capsMap := heldMeterState() + first := targetFor(t, ComputeDispatch(store, st, capsMap, 3*25*230), "bat").TargetW + if first <= 0 { + t.Fatalf("fresh -2000 W export did not charge: %.0f W", first) + } + reportBattery(store, first, json.RawMessage(`{"charge_capable":false}`)) + sent := sentTo(ComputeDispatch(store, st, capsMap, 3*25*230), "bat") + if sent == nil || sent.TargetW > 0 { + t.Fatalf("held meter left battery charging at %.0f W after charge_capable=false: %+v", first, sent) + } +} + +// A meter poll that emits and then returns an error leaves the latest read +// marked failed on every tick, but each tick still brings a new sample. +// Dispatch must act on it from a fresh start. +func TestEmitThenErrorMeterIsNotHeld(t *testing.T) { + store := seedStore(2000, []struct { + name string + currentW, soc float64 + }{{"bat", 0, 0.5}}) + st, capsMap := heldMeterState() + acted := false + for tick := 0; tick < 10; tick++ { + time.Sleep(time.Millisecond) // UpdatedAt is wall-clock time + store.Update("ferroamp", telemetry.DerMeter, 2000, nil, nil) + store.DriverHealthMut("ferroamp").RecordSuccess() + store.DriverHealthMut("ferroamp").RecordError("secondary register timeout") + if sent := sentTo(ComputeDispatch(store, st, capsMap, 3*25*230), "bat"); sent != nil && sent.TargetW < 0 { + acted = true + } + } + if !acted { + t.Fatal("ten new +2000 W meter samples, battery never dispatched") + } +} diff --git a/go/internal/control/fuse_battery_phase_direction_test.go b/go/internal/control/fuse_battery_phase_direction_test.go new file mode 100644 index 000000000..199b43585 --- /dev/null +++ b/go/internal/control/fuse_battery_phase_direction_test.go @@ -0,0 +1,987 @@ +package control + +import ( + "encoding/json" + "fmt" + "math" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// The battery fuse paths read each phase with its direction and count the +// batteries' own move. 16 A fuse, 230 V, 3 phases; the working limit is +// the fuse less safety_margin_a (fuseLineA). + +func phaseDirectionStore(t *testing.T, gridW, batW float64, data map[string]any) *telemetry.Store { + t.Helper() + s := telemetry.NewStore() + raw, err := json.Marshal(data) + if err != nil { + t.Fatal(err) + } + s.Update("meter", telemetry.DerMeter, gridW, nil, raw) + s.DriverHealthMut("meter").RecordSuccess() + soc := 0.6 + s.Update("bat", telemetry.DerBattery, batW, &soc, nil) + s.DriverHealthMut("bat").RecordSuccess() + return s +} + +func phaseDirectionState(safetyA float64) *State { + st := NewState(0, 50, "meter") + st.SiteFuseAmps, st.SiteFuseVoltage, st.SiteFusePhases, st.SiteFuseSafetyA = 16, 230, 3, safetyA + st.DriverLimits = map[string]PowerLimits{"bat": {MaxChargeW: 10000, MaxDischargeW: 10000}} + return st +} + +// L1 exports 20 A, L2/L3 import 12 A, the total imports 920 W. No phase +// imports over the limit, and a discharge adds export on L1. +var exportingL1 = map[string]any{"l1_a": 20.0, "l2_a": 12.0, "l3_a": 12.0, + "l1_w": -4600.0, "l2_w": 2760.0, "l3_w": 2760.0} + +func TestFuseSaverDoesNotDischargeIntoExportingPhase(t *testing.T) { + store := phaseDirectionStore(t, 920, 0, exportingL1) + out := fuseSaverFromLivePower(store, phaseDirectionState(0.5), map[string]float64{"bat": 15200}, 16*230*3) + for _, o := range out { + if o.TargetW < 0 { + t.Fatalf("fuse saver ordered %.0f W discharge; L1 export 20.0 A -> %.1f A on a 16 A fuse", o.TargetW, 20+(-o.TargetW)/690) + } + } +} + +func TestIdleDispatchDoesNotDischargeIntoExportingPhase(t *testing.T) { + store := phaseDirectionStore(t, 920, 0, exportingL1) + st := phaseDirectionState(0.5) + st.Mode = ModeIdle + out := ComputeDispatch(store, st, map[string]float64{"bat": 15200}, 16*230*3) + for _, o := range out { + if o.TargetW < 0 { + t.Fatalf("Idle ComputeDispatch ordered %.0f W discharge; L1 export 20.0 A -> %.1f A", o.TargetW, 20+(-o.TargetW)/690) + } + } +} + +// L1 imports 22 A, L2/L3 export 15 A, the battery discharges 3 kW and the +// total exports. Cutting the discharge raises L1's import. +func TestFuseGuardKeepsDischargeThatRelievesImportingPhase(t *testing.T) { + store := phaseDirectionStore(t, (22-30)*230, -3000, map[string]any{ + "l1_a": 22.0, "l2_a": 15.0, "l3_a": 15.0, + "l1_w": 5060.0, "l2_w": -3450.0, "l3_w": -3450.0}) + st := phaseDirectionState(0.5) + out := applyFuseGuard([]DispatchTarget{{Driver: "bat", TargetW: -3000}}, store, st, 16*230*3) + if out[0].TargetW > -3000+1 { + t.Fatalf("fuse guard cut discharge -3000 -> %.0f W; L1 import 22.0 -> %.1f A; hold max discharge %.0f W until +%v", + out[0].TargetW, 22+(out[0].TargetW+3000)/690, st.FuseHoldMaxDischargeW, st.FuseHoldUntil.Sub(st.now())) + } +} + +// L1 exports 18 A (one-phase PV), L2/L3 import 14 A, the total imports +// 2300 W. Charging relieves L1 and loads L2/L3. +var exportingL1Charge = map[string]any{"l1_a": 18.0, "l2_a": 14.0, "l3_a": 14.0, + "l1_w": -4140.0, "l2_w": 3220.0, "l3_w": 3220.0} + +func TestFuseSaverIgnoresExportingPhaseUnderImportTotal(t *testing.T) { + store := phaseDirectionStore(t, 2300, 0, exportingL1Charge) + if out := fuseSaverFromLivePower(store, phaseDirectionState(0), map[string]float64{"bat": 15200}, 11040); out != nil { + t.Errorf("fuseSaverFromLivePower = %.0f W; discharging pushes exporting L1 further past the fuse", out[0].TargetW) + } +} + +// No charge fits both sides: 3000 W takes L2/L3 to 18.3 A, none leaves L1 +// at 18 A of export. The guard leaves the charge where the worst phase is +// least over: 1380 W, with every phase at 16 A. +func TestFuseGuardChargeBalancesExportingAndImportingPhases(t *testing.T) { + store := phaseDirectionStore(t, 2300, 0, exportingL1Charge) + got := applyFuseGuard([]DispatchTarget{{Driver: "bat", TargetW: 3000}}, store, phaseDirectionState(0), 11040) + l1, l2 := -18+got[0].TargetW/690, 14+got[0].TargetW/690 + if worst := math.Max(math.Abs(l1), math.Abs(l2)); worst > 16.01 { + t.Errorf("applyFuseGuard charge 3000 -> %.0f W leaves L1 %.2f A, L2/L3 %.2f A; want every phase at or below 16 A", got[0].TargetW, l1, l2) + } +} + +// The relief counts the battery's own share of the phase it reads. A +// battery that follows its order keeps L1 at the working limit, 15.5 A on +// a 0.5 A margin, on every tick instead of returning to 0 W when the +// overage stops showing. +func TestPerPhaseFuseReliefHoldsWhileOverloadPersists(t *testing.T) { + st := phaseDirectionState(0.5) + st.Mode = ModeIdle + now := time.Date(2026, 10, 27, 12, 0, 0, 0, time.UTC) + st.clock = func() time.Time { return now } + + store := telemetry.NewStore() + house := [3]float64{20, 5, 5} // A per phase, import + batW := 0.0 // battery follows its last order within a tick + caps := map[string]float64{"bat": 15200} + var targets, l1 []float64 + for tick := 0; tick < 10; tick++ { + a := house + for i := range a { + a[i] += batW / (230 * 3) // a three-phase battery spreads evenly + } + raw, _ := json.Marshal(map[string]any{"l1_a": a[0], "l2_a": a[1], "l3_a": a[2]}) + store.Update("meter", telemetry.DerMeter, (a[0]+a[1]+a[2])*230, nil, raw) + store.DriverHealthMut("meter").RecordSuccess() + soc := 0.6 + store.Update("bat", telemetry.DerBattery, batW, &soc, nil) + store.DriverHealthMut("bat").RecordSuccess() + + out := ComputeDispatch(store, st, caps, 16*230*3) + w := 0.0 + for _, o := range out { + if o.Driver == "bat" { + w = o.TargetW + } + } + targets = append(targets, math.Round(w)) + l1 = append(l1, math.Round(a[0]*10)/10) + batW = w + now = now.Add(5 * time.Second) + } + line := st.fuseLineA() + for i := 1; i < len(l1); i++ { + if l1[i] > line || l1[i] < line-0.05 { + t.Fatalf("tick %d: L1 %.1f A, want the %.1f A working limit while the overload persists; targets %v, L1 %v", i, l1[i], line, targets, l1) + } + } +} + +// chargeLoop repeats a 4000 W charge request against house loads of +// 13/7/5 A on a 0 A margin, with a three-phase battery that follows each +// order. dispatch returns the battery's next target. +func chargeLoop(t *testing.T, ticks int, dispatch func(*telemetry.Store, *State) float64) (targets, l1 []float64) { + t.Helper() + st := phaseDirectionState(0) + now := time.Date(2026, 10, 27, 12, 0, 0, 0, time.UTC) + st.clock = func() time.Time { return now } + store := telemetry.NewStore() + batW := 0.0 + for tick := 0; tick < ticks; tick++ { + a := [3]float64{13, 7, 5} + for i := range a { + a[i] += batW / 690 + } + raw, _ := json.Marshal(map[string]any{"l1_a": a[0], "l2_a": a[1], "l3_a": a[2], + "l1_w": a[0] * 230, "l2_w": a[1] * 230, "l3_w": a[2] * 230}) + store.Update("meter", telemetry.DerMeter, (a[0]+a[1]+a[2])*230, nil, raw) + store.DriverHealthMut("meter").RecordSuccess() + soc := 0.5 + store.Update("bat", telemetry.DerBattery, batW, &soc, nil) + store.DriverHealthMut("bat").RecordSuccess() + batW = dispatch(store, st) + targets = append(targets, math.Round(batW)) + l1 = append(l1, math.Round(a[0]*100)/100) + now = now.Add(5 * time.Second) + } + return targets, l1 +} + +// The fuse hold caps the request before the guard reads the phases, so the +// hold's cut is not taken off again. On a 0 A margin the charge holds +// 2070 W with L1 at the 16 A rating on every repeat, instead of switching +// between 2070 W and 0 W. +func TestFuseGuardHeldChargeHoldsSteady(t *testing.T) { + assertSteady := func(name string, targets, l1 []float64) { + t.Helper() + for i := range targets { + if math.Abs(targets[i]-2070) > 1 || (i > 0 && math.Abs(l1[i]-16) > 0.01) { + t.Fatalf("%s: tick %d charge %v W, L1 %v A; want 2070 W and 16 A; targets %v, L1 %v", name, i, targets[i], l1[i], targets, l1) + } + } + } + targets, l1 := chargeLoop(t, 10, func(store *telemetry.Store, st *State) float64 { + return applyFuseGuard([]DispatchTarget{{Driver: "bat", TargetW: 4000}}, store, st, 16*230*3)[0].TargetW + }) + assertSteady("applyFuseGuard", targets, l1) + + targets, l1 = chargeLoop(t, 12, func(store *telemetry.Store, st *State) float64 { + st.Mode = ModeCharge + st.DriverLimits = map[string]PowerLimits{"bat": {MaxChargeW: 4000, MaxDischargeW: 4000}} + for _, o := range ComputeDispatch(store, st, map[string]float64{"bat": 15200}, 16*230*3) { + if o.Driver == "bat" { + return o.TargetW + } + } + return 0 + }) + assertSteady("Charge mode", targets, l1) +} + +// A watchdog-offline meter's cached phases are not the site: they drive no +// phase relief. +func TestPhaseReliefSkipsOfflineMeter(t *testing.T) { + store := phaseDirectionStore(t, 20*230, 0, map[string]any{"l1_a": 20.0, "l2_a": 0.0, "l3_a": 0.0, + "l1_w": 4600.0, "l2_w": 0.0, "l3_w": 0.0}) + meter := store.Get("meter", telemetry.DerMeter) + if imp, _ := phaseFuseReliefW(livePhaseSample(store, "meter", meter), phaseDirectionState(0), 0); imp <= 0 { + t.Fatalf("online meter: import relief %.0f W, want > 0 for L1 at 20 A", imp) + } + store.DriverHealthMut("meter").SetOffline() + if imp, exp := phaseFuseReliefW(livePhaseSample(store, "meter", meter), phaseDirectionState(0), 0); imp != 0 || exp != 0 { + t.Fatalf("offline meter: relief %.0f/%.0f W, want none", imp, exp) + } +} + +// The batteries' move counts only power read live from the store, in the +// same reading the rest of the decision used. A battery that went offline, +// moved on to a newer reading or has no reading leaves its power in the +// phases unknown: no phase relief, as #1876's stop relief did. Before, +// the cached reading's RawW drove the relief. +func TestPhaseReliefNeedsTheBatteryReadingItUses(t *testing.T) { + data := map[string]any{"l1_a": 15.0, "l2_a": 6.0, "l3_a": 5.1, "l1_w": 3450.0, "l2_w": 1380.0, "l3_w": 1173.0} + st := phaseDirectionState(0) + stop := []DispatchTarget{{Driver: "bat", TargetW: 0}} + seed := func() (*telemetry.Store, map[string]*telemetry.DerReading, phaseSample) { + store := phaseDirectionStore(t, 6003, -3000, data) + meter := store.Get("meter", telemetry.DerMeter) + return store, map[string]*telemetry.DerReading{"bat": store.Get("bat", telemetry.DerBattery)}, livePhaseSample(store, "meter", meter) + } + + _, readings, phases := seed() + // Stopping 3000 W puts L1 at 15 + 4.35 A; 3.35 A over 16 A is 2310 W. + if imp, _ := phaseReliefForTargets(phases, st, stop, readings); math.Abs(imp-2310) > 1 { + t.Fatalf("live readings: relief %.0f W, want 2310 W", imp) + } + + store, readings, phases := seed() + store.DriverHealthMut("bat").SetOffline() + if imp, exp := phaseReliefForTargets(phases, st, stop, readings); imp != 0 || exp != 0 { + t.Errorf("offline battery: relief %.0f/%.0f W, want none", imp, exp) + } + + store, readings, phases = seed() + time.Sleep(time.Millisecond) + soc := 0.6 + store.Update("bat", telemetry.DerBattery, -3000, &soc, nil) + if imp, exp := phaseReliefForTargets(phases, st, stop, readings); imp != 0 || exp != 0 { + t.Errorf("newer battery reading: relief %.0f/%.0f W, want none", imp, exp) + } + + // Counted from 0 W, a 2000 W charge would read as L1 at 17.9 A. + _, _, phases = seed() + charge := []DispatchTarget{{Driver: "bat", TargetW: 2000}} + if imp, exp := phaseReliefForTargets(phases, st, charge, map[string]*telemetry.DerReading{}); imp != 0 || exp != 0 { + t.Errorf("no battery reading: relief %.0f/%.0f W, want none", imp, exp) + } +} + +// The battery's working limit on each phase is L = In − safety_margin_a, +// in both directions, with no floor under the margin. A phase at L needs no +// relief, 0.1 A past it needs exactly 0.1 A, and a target that lands a +// phase exactly at L passes the guard unchanged. +func TestBatteryFuseLineProperty(t *testing.T) { + const v = 230.0 + for _, in := range []float64{16, 20, 25, 35} { + for _, phases := range []int{1, 3} { + for _, margin := range []float64{0, 0.5, 1.2} { + for _, dir := range []float64{1, -1} { // 1 import, -1 export + name := fmt.Sprintf("%gA/%dp/margin%g/dir%+g", in, phases, margin, dir) + st := NewState(0, 50, "meter") + st.SiteFuseAmps, st.SiteFuseVoltage, st.SiteFusePhases, st.SiteFuseSafetyA = in, v, phases, margin + st.DriverLimits = map[string]PowerLimits{"bat": {MaxChargeW: 50000, MaxDischargeW: 50000}} + line := in - margin + if got := st.fuseLineA(); got != line { + t.Fatalf("%s: fuseLineA %.2f A, want %.2f A", name, got, line) + } + perA := v * float64(phases) + fuseMaxW := in * perA + // store reads the worst phase at worstA and 2 A on the + // others, all flowing in dir, with phase power. + store := func(worstA float64) *telemetry.Store { + amps := []float64{worstA, 2, 2}[:phases] + data := map[string]any{} + var totalW float64 + for i, a := range amps { + w := dir * a * v + data[fmt.Sprintf("l%d_a", i+1)] = a + data[fmt.Sprintf("l%d_w", i+1)] = w + totalW += w + } + return phaseDirectionStore(t, totalW, 0, data) + } + relief := func(worstA float64) (float64, float64) { + s := store(worstA) + meter := s.Get("meter", telemetry.DerMeter) + return phaseFuseReliefW(livePhaseSample(s, "meter", meter), st, 0) + } + + if imp, exp := relief(line); imp != 0 || exp != 0 { + t.Errorf("%s: phase at the %.1f A line: relief %.1f/%.1f W, want none", name, line, imp, exp) + } + + imp, exp := relief(line + 0.1) + want := 0.1 * perA + gotW, otherW := imp, exp + if dir < 0 { + gotW, otherW = exp, imp + } + if math.Abs(gotW-want) > 1 || otherW != 0 { + t.Errorf("%s: 0.1 A past the line: relief %.1f/%.1f W, want %.1f W on the flowing side", name, imp, exp, want) + } + + // The battery moves the phase from line − 2 A to the line. + targetW := dir * 2 * perA + out := applyFuseGuard([]DispatchTarget{{Driver: "bat", TargetW: targetW}}, store(line-2), st, fuseMaxW) + if out[0].Clamped || math.Abs(out[0].TargetW-targetW) > 1e-6 || !st.FuseHoldUntil.IsZero() { + t.Errorf("%s: target %.0f W that lands at the %.1f A line: guard gave %+v, hold until %v", name, targetW, line, out[0], st.FuseHoldUntil) + } + } + } + } + } +} + +// C3, rig seed 207533, on a 16 A fuse with no margin: the battery holds a +// 3 kW discharge with L1 at 14.4 A, then PV drops 1280 W and L1 reads +// 16.3 A. The relief brings L1 to the line. On the next ticks the reading +// sits at the line with the battery at its relief power, and the relief +// stays: L1 crosses the rating once, on the PV step, and never again while +// PV is low. Each tick the battery reaches the order it was sent the tick +// before, and the meter reads it there. +func TestFuseReliefStaysAtTheLineAfterPVDrop(t *testing.T) { + targets, l1, line := pvDropLoop(0, 0) + assertHeldAtTheLine(t, targets, l1, line) +} + +// pvDropLoop runs the 207533 shape: a -3005 W hold, L1 at 14.45 A, and PV +// down 1280 W from tick 5 to tick 29, with 5 s ticks. The meter sees the +// battery's actual power. The battery's own reading reports the power of +// readLag ticks earlier. maxDischargeW > 0 caps what the battery delivers +// without Core knowing it. +func pvDropLoop(readLag int, maxDischargeW float64) (targets, l1 []float64, line float64) { + r := pvDropRun(pvDropCase{readLag: readLag, maxDischargeW: maxDischargeW}) + return r.targets, r.l1, r.line +} + +// pvDropCase varies pvDropLoop. freezeFrom > 0 makes the meter repeat its +// reading from that tick on. onePhase puts the battery on that phase only +// (1-3), which Core reads as balanced. load replaces the house and car +// current per phase. ticks > 0 replaces the 40 ticks, pvW the PV step, and +// extra adds import current per phase at a tick. resA > 0 makes the meter +// report phase current in steps of resA, as a P1 meter does in whole +// amperes. +type pvDropCase struct { + readLag int + maxDischargeW float64 + freezeFrom int + onePhase int + load [3]float64 + ticks int + pvW func(tick int) float64 + extra func(tick int) [3]float64 + resA float64 +} + +type pvDropResult struct { + targets, l1 []float64 + // batPhase is the true current on the battery's phase when onePhase > 0. + batPhase []float64 + line float64 +} + +func pvDropRun(c pvDropCase) (r pvDropResult) { + st := phaseDirectionState(0) + now := time.Date(2026, 10, 27, 12, 0, 0, 0, time.UTC) + st.clock = func() time.Time { return now } + st.SetBatteryManualHold(BatteryManualHold{PowerW: -3005, ExpiresAt: now.Add(time.Hour)}) + + store := telemetry.NewStore() + load := [3]float64{23.44, 11.04, 11.54} // house and car, A import per phase + if c.load != ([3]float64{}) { + load = c.load + } + pvW := -3200.0 + actual := []float64{-3005} + r.line = st.fuseLineA() + var frozen json.RawMessage + var frozenW float64 + ticks := 40 + if c.ticks > 0 { + ticks = c.ticks + } + for tick := 0; tick < ticks; tick++ { + switch tick { + case 5: + pvW = -1920 + case 30: + pvW = -3200 + } + if c.pvW != nil { + pvW = c.pvW(tick) + } + var extra [3]float64 + if c.extra != nil { + extra = c.extra(tick) + } + batW := actual[len(actual)-1] + var a [3]float64 + data := map[string]any{} + var totalW float64 + for i := range a { + a[i] = load[i] + extra[i] + (pvW+batW)/690 + if c.onePhase > 0 { + a[i] = load[i] + extra[i] + pvW/690 + if i == c.onePhase-1 { + a[i] += batW / 230 + } + } + data[fmt.Sprintf("l%d_a", i+1)] = math.Abs(a[i]) + if c.resA > 0 { + data[fmt.Sprintf("l%d_a", i+1)] = math.Round(math.Abs(a[i])/c.resA) * c.resA + } + data[fmt.Sprintf("l%d_w", i+1)] = a[i] * 230 + totalW += a[i] * 230 + } + raw, _ := json.Marshal(data) + if c.freezeFrom > 0 && tick >= c.freezeFrom { + if frozen == nil { + frozen, frozenW = raw, totalW + } + raw, totalW = frozen, frozenW + } + store.Update("meter", telemetry.DerMeter, totalW, nil, raw) + store.DriverHealthMut("meter").RecordSuccess() + soc := 0.6 + store.Update("bat", telemetry.DerBattery, actual[max(0, len(actual)-1-c.readLag)], &soc, nil) + store.DriverHealthMut("bat").RecordSuccess() + + order := batW + for _, o := range ComputeDispatch(store, st, map[string]float64{"bat": 15200}, 16*230*3) { + if o.Driver == "bat" { + order = o.TargetW + } + } + batW = order + if c.maxDischargeW > 0 { + batW = math.Max(order, -c.maxDischargeW) + } + actual = append(actual, batW) + r.targets = append(r.targets, math.Round(order)) + r.l1 = append(r.l1, math.Round(a[0]*100)/100) + if c.onePhase > 0 { + r.batPhase = append(r.batPhase, math.Round(a[c.onePhase-1]*100)/100) + } + now = now.Add(5 * time.Second) + } + return r +} + +// assertHeldAtTheLine wants L1 at the line while PV is low, and one crossing +// of it, on the PV step. +func assertHeldAtTheLine(t *testing.T, targets, l1 []float64, line float64) { + t.Helper() + crossings := 0 + for i := range l1 { + if l1[i] > line+0.01 && (i == 0 || l1[i-1] <= line+0.01) { + crossings++ + } + if i > 5 && i < 30 && (l1[i] > line+0.01 || l1[i] < line-0.05) { + t.Errorf("tick %d: L1 %.2f A, want the %.1f A line while PV is low", i, l1[i], line) + } + } + if crossings != 1 { + t.Errorf("L1 crossed the %.1f A line %d times, want once (the PV step)", line, crossings) + } + if t.Failed() { + t.Logf("targets %v", targets) + t.Logf("L1 %v", l1) + } +} + +// The battery's reading lags the meter by one or two ticks: the meter +// already shows the battery at its relief while the reading still holds +// the power before it. Counted from that reading, the relief looked like a +// move still to make, so every other tick took it back, and the order ran +// out further each time (-3214, -3005, -3422, -3005, ... -5717 W) with L1 +// back at 16.3 A on every other tick. The last command bounds the move, so +// the relief holds -3214 W with L1 at the line. +func TestFuseReliefHoldsWhenTheBatteryReadingLags(t *testing.T) { + for _, lag := range []int{1, 2} { + t.Run(fmt.Sprintf("lag %d", lag), func(t *testing.T) { + targets, l1, line := pvDropLoop(lag, 0) + assertHeldAtTheLine(t, targets, l1, line) + for i := 6; i < 30; i++ { + if targets[i] != targets[5] { + t.Errorf("tick %d: order %.0f W, want the %.0f W relief held", i, targets[i], targets[5]) + } + } + }) + } +} + +// A battery that cannot reach its order, with no limit Core knows, must +// not wind the relief up through the last-command bound: the order settles, +// and it returns to the hold once PV comes back. +func TestFuseReliefSettlesWhenTheBatteryFallsShort(t *testing.T) { + targets, l1, _ := pvDropLoop(0, 3100) + for i := 8; i < 30; i++ { + if targets[i] != targets[i-1] || targets[i] < -3300 { + t.Errorf("tick %d: order %.0f W after %.0f W, want it settled near -3214 W", i, targets[i], targets[i-1]) + } + } + if targets[31] != -3005 { + t.Errorf("tick 31: order %.0f W, want the -3005 W hold back once PV returns", targets[31]) + } + if t.Failed() { + t.Logf("targets %v", targets) + t.Logf("L1 %v", l1) + } +} + +// Mixed directions, three-phase battery holding -2000 W: L1 imports at the +// 16 A line, and a one-phase PV inverter on L3 steps from -15 A to -18 A +// of export. No move fits both phases. The least-over move cuts the +// discharge by 690 W and leaves L1 at 17 A and L3 at -17 A. Before, the +// last-command bound read L1 at the import line as a reason to keep +// -2000 W, put back every cut the export side made, and left L3 at -18 A. +func TestFuseReliefLeavesMixedPhasesLeastOver(t *testing.T) { + st := phaseDirectionState(0) + now := time.Date(2026, 10, 27, 12, 0, 0, 0, time.UTC) + st.clock = func() time.Time { return now } + st.SetBatteryManualHold(BatteryManualHold{PowerW: -2000, ExpiresAt: now.Add(time.Hour)}) + store := telemetry.NewStore() + base := [3]float64{18.9, 8, -12.1} // A without the battery, + import + batW := -2000.0 + var orders, worst []float64 + for tick := 0; tick < 20; tick++ { + if tick == 3 { + base[2] = -15.1 + } + data := map[string]any{} + var totalW, w float64 + for i := range base { + a := base[i] + batW/690 + data[fmt.Sprintf("l%d_a", i+1)] = math.Abs(a) + data[fmt.Sprintf("l%d_w", i+1)] = a * 230 + totalW += a * 230 + w = math.Max(w, math.Abs(a)) + } + raw, _ := json.Marshal(data) + store.Update("meter", telemetry.DerMeter, totalW, nil, raw) + store.DriverHealthMut("meter").RecordSuccess() + soc := 0.6 + store.Update("bat", telemetry.DerBattery, batW, &soc, nil) + store.DriverHealthMut("bat").RecordSuccess() + for _, o := range ComputeDispatch(store, st, map[string]float64{"bat": 15200}, 16*230*3) { + if o.Driver == "bat" { + batW = o.TargetW + } + } + orders = append(orders, math.Round(batW)) + worst = append(worst, math.Round(w*100)/100) + now = now.Add(5 * time.Second) + } + for i := 8; i < 20; i++ { + if worst[i] > 17.05 { + t.Errorf("tick %d: worst phase %.2f A, want at most 17 A (L1 17, L3 -17)", i, worst[i]) + } + } + if t.Failed() { + t.Logf("orders %v", orders) + t.Logf("worst phase |A| %v", worst) + } +} + +// A one-phase battery on L2, which Core reads as balanced, with L1 at 18 A +// while PV is low. The first relief step, -1378 W, lands on L2 alone and +// takes it to -17.97 A of export. That step is what shows the battery off +// L1. Before, the relief then held that step: L2 stayed at -17.97 A for +// all 24 PV-low ticks and reached -19.83 A, past 1.2 x In, on the PV +// return tick. Now the battery goes back to its power before the step, +// and L2 is inside the line from the next tick until PV returns. +// +// Once PV returns, L1 still sits at 16.14 A, which the battery cannot +// reach, and L2 has less than 3 A of room (690 W on one phase). The relief +// creeps out to L2's line before the battery has moved the one ampere per +// phase the rule needs, and the least-over move then leaves L2 about as +// far past the export line as L1 is past the import line. That is the +// known limit of a one-phase battery read as balanced. +func TestFuseReliefLetsAOnePhaseBatteryOffItsOwnPhase(t *testing.T) { + r := pvDropRun(pvDropCase{onePhase: 2, load: [3]float64{20.78, 6, 11.54}}) + for i := 6; i < 40; i++ { + if math.Abs(r.batPhase[i]) > fuseInstantFactor*16 { + t.Errorf("tick %d: L2 %.2f A, past %.1f x In", i, r.batPhase[i], fuseInstantFactor) + } + if i >= 8 && i < 30 && math.Abs(r.batPhase[i]) > r.line+0.01 { + t.Errorf("tick %d: L2 %.2f A, want inside the %.1f A line while PV is low", i, r.batPhase[i], r.line) + } + if i >= 9 && i < 30 && r.targets[i] != r.targets[4] { + t.Errorf("tick %d: order %.0f W, want the %.0f W order from before the step", i, r.targets[i], r.targets[4]) + } + if i >= 30 && math.Abs(r.batPhase[i]) > r.line+0.25 { + t.Errorf("tick %d: L2 %.2f A, more than 0.25 A past the %.1f A line", i, r.batPhase[i], r.line) + } + } + if t.Failed() { + t.Logf("targets %v", r.targets) + t.Logf("L1 %v", r.l1) + t.Logf("L2 %v", r.batPhase) + } +} + +// The meter repeats its 16.3 A reading from the PV step on (frozen@5) +// while the battery follows its orders. Before, every tick found the same +// 0.3 A overage and added it to a battery already moved for it: -3214, +// -3422, ... -10000 W, with the true L1 down at 4.3 A. Now the relief stops +// once the battery has moved one ampere per phase (690 W) and L1 has not +// moved: at most that ampere and one step past the first relief. +func TestFuseReliefStopsWhenTheMeterIsFrozen(t *testing.T) { + r := pvDropRun(pvDropCase{freezeFrom: 5}) + assertReliefStops(t, r) +} + +// The meter freezes on a reading that is not the worst phase's reference: +// L1 reads 16.3 A on the PV step, the relief brings it to 16.0 A, and a +// 0.2 A load on L1 starts at tick 8, the reading the meter then repeats. +// L1 moved only 0.1 A from 16.3 A, so the stop counts the battery's move +// from the reading the meter repeats, not from that reference: the order +// stops once the battery has moved one ampere per phase since the freeze. +func TestFuseReliefStopsWhenTheMeterFreezesOffTheReference(t *testing.T) { + r := pvDropRun(pvDropCase{ + freezeFrom: 8, + extra: func(tick int) [3]float64 { + if tick >= 8 { + return [3]float64{0.2, 0, 0} + } + return [3]float64{} + }, + }) + const floorW = -3214 - 0.2*690 - phaseResponseA*690 - 0.2*690 + for i := 8; i < 30; i++ { + if r.targets[i] < floorW { + t.Errorf("tick %d: order %.0f W, past %.0f W", i, r.targets[i], floorW) + } + if i >= 15 && r.targets[i] != r.targets[15] { + t.Errorf("tick %d: order %.0f W, want the %.0f W order held", i, r.targets[i], r.targets[15]) + } + } + if t.Failed() { + t.Logf("targets %v", r.targets) + t.Logf("L1 %v", r.l1) + } +} + +// A one-phase battery on L2 and L1 at 16.3 A while PV is low. The battery +// cannot reach L1, so L1 never shows its move. Before, the relief grew by +// 0.3 A of overage a tick to -4512 W and drove L2 to -16.4 A of export, +// then -18.25 A on the PV return tick. Now it stops once the battery has +// moved one ampere per phase, and goes back to the -3005 W hold: what it +// moved did not reach L1. L2 stays inside the line. +func TestFuseReliefStopsWhenTheBatteryCannotReachTheWorstPhase(t *testing.T) { + r := pvDropRun(pvDropCase{onePhase: 2, load: [3]float64{19.08, 6, 11.54}}) + const floorW = -3214 - phaseResponseA*690 - 0.3*690 + for i := range r.targets { + if r.targets[i] < floorW { + t.Errorf("tick %d: order %.0f W, past %.0f W", i, r.targets[i], floorW) + } + if i >= 10 && r.targets[i] != -3005 { + t.Errorf("tick %d: order %.0f W, want the -3005 W hold back", i, r.targets[i]) + } + } + for i, a := range r.batPhase { + if math.Abs(a) > r.line+0.01 { + t.Errorf("tick %d: L2 %.2f A, want inside the %.1f A line", i, a, r.line) + } + } + if t.Failed() { + t.Logf("targets %v", r.targets) + t.Logf("L2 %v", r.batPhase) + } +} + +// assertReliefStops wants the order no further out than one ampere per +// phase and one 0.3 A step past the first relief (-3214 W), and held from +// tick 9 until PV returns. +func assertReliefStops(t *testing.T, r pvDropResult) { + t.Helper() + const floorW = -3214 - phaseResponseA*690 - 0.3*690 + for i := 5; i < 30; i++ { + if r.targets[i] < floorW { + t.Errorf("tick %d: order %.0f W, past %.0f W", i, r.targets[i], floorW) + } + if i >= 9 && r.targets[i] != r.targets[9] { + t.Errorf("tick %d: order %.0f W, want the %.0f W order held", i, r.targets[i], r.targets[9]) + } + } + if t.Failed() { + t.Logf("targets %v", r.targets) + t.Logf("L1 %v", r.l1) + if r.batPhase != nil { + t.Logf("battery phase %v", r.batPhase) + } + } +} + +// The meter fails to show the batteries only once they moved one ampere +// per phase (phaseResponseA, 690 W here) and either the meter repeated its +// reading all the while, or the worst phase moved less than half an ampere +// while another phase carried the move. A phase that moves becomes the new +// reference. +func TestPhaseUnresponsiveNeedsOneAmpereOfBatteryMove(t *testing.T) { + st := phaseDirectionState(0) + for _, c := range []struct { + name string + amps []float64 + batW float64 + stuck bool + }{ + {"first reading: a reference", []float64{16.3, 9, 9.5}, -3000, false}, + {"689 W moved, same reading", []float64{16.3, 9, 9.5}, -3689, false}, + {"690 W moved, same reading", []float64{16.3, 9, 9.5}, -3690, true}, + {"L1 0.49 A, L2 and L3 still: a load cancels the move", []float64{16.79, 9, 9.5}, -3690, false}, + {"L1 0.49 A, L2 and L3 one share each: a load on L1", []float64{16.79, 8, 8.5}, -3690, false}, + {"L1 0.49 A, L2 1.99 shares", []float64{16.79, 7.01, 9.5}, -3690, false}, + {"L1 0.49 A, L2 two shares: a battery off L1", []float64{16.79, 7, 9.5}, -3690, true}, + {"L1 moved 0.5 A: a new reference", []float64{15.8, 9, 9.5}, -3690, false}, + {"same reading again", []float64{15.8, 9, 9.5}, -3690, false}, + {"690 W back, same reading", []float64{15.8, 9, 9.5}, -3000, true}, + {"L1 0.1 A, L2 and L3 still", []float64{15.9, 9, 9.5}, -3000, false}, + {"690 W back again, the meter repeats L1 0.1 A off the reference", []float64{15.9, 9, 9.5}, -2310, true}, + } { + if _, got, _ := st.phaseUnresponsive(c.amps, 0, map[string]float64{"bat": c.batW}); got != c.stuck { + t.Errorf("%s: stuck %v, want %v", c.name, got, c.stuck) + } + } +} + +// pvLowAfter5 drops PV to -440 W from tick 5 on, which puts L1 at 18.45 A +// under the -3005 W hold. +func pvLowAfter5(tick int) float64 { + if tick >= 5 { + return -440 + } + return -3200 +} + +// The battery's reading lags the meter by a tick (207537). L1 reads +// 18.45 A at tick 5, and one relief step of -4694 W brings it to the line. +// The reading then catches up with that order while the meter stays put. A +// 0.4 A load on L1 starts at tick 15 and stays. Before, the catch-up +// counted as a move the meter did not show: the order stayed at -4694 W +// and L1 at 16.4 A to the end, and the heat model trips that after about +// three minutes. The catch-up is no new move, so the relief follows the +// load. +func TestFuseReliefFollowsALoadAfterTheReadingCatchesUp(t *testing.T) { + r := pvDropRun(pvDropCase{ + ticks: 60, + readLag: 1, + pvW: pvLowAfter5, + extra: func(tick int) [3]float64 { + if tick >= 15 { + return [3]float64{0.4, 0, 0} + } + return [3]float64{} + }, + }) + assertL1AtTheLine(t, r, 17, 60) +} + +// L1 reads 18.45 A at tick 5 and the relief step is -4694 W. A balanced +// 2 A load per phase (a three-phase heat pump) starts in the reading that +// shows the step, so L1 reads 18.0 A, a move of 0.45 A. Before, that +// counted as a meter that does not show the battery: the order stayed at +// -4694 W with L1 at 18 A (1.125 x In) to the end, and the heat model +// trips that after about 94 s. The meter is live, so the relief goes on. +func TestFuseReliefGoesOnWhenALoadMasksTheStep(t *testing.T) { + r := pvDropRun(pvDropCase{ + pvW: pvLowAfter5, + extra: func(tick int) [3]float64 { + if tick >= 6 { + return [3]float64{2, 2, 2} + } + return [3]float64{} + }, + }) + assertL1AtTheLine(t, r, 7, 40) +} + +// A one-phase load on L1 rises 0.15 A a tick from tick 10 to tick 39, +// after the PV step has put L1 at the line. The battery follows, so L1 +// never moves half an ampere, while L2 and L3 move with the battery. +// Before, after 690 W of following, the relief stopped until L1 had risen +// half an ampere past its 16.3 A reference: L1 climbed to 16.9 A, fell +// back and climbed again. Now L1 stays within one tick's rise of the line, +// and at the line once the rise stops. +func TestFuseReliefFollowsASlowLoadRise(t *testing.T) { + r := pvDropRun(pvDropCase{ + ticks: 50, + pvW: func(tick int) float64 { + if tick >= 5 { + return -1920 + } + return -3200 + }, + extra: func(tick int) [3]float64 { + return [3]float64{0.15 * float64(min(max(tick-9, 0), 30)), 0, 0} + }, + }) + for i := 10; i < 40; i++ { + if r.l1[i] > r.line+0.16 { + t.Errorf("tick %d: L1 %.2f A, more than one 0.15 A rise over the %.1f A line", i, r.l1[i], r.line) + } + } + assertL1AtTheLine(t, r, 41, 50) +} + +// assertL1AtTheLine wants L1 at or under the line on ticks [from, to). +func assertL1AtTheLine(t *testing.T, r pvDropResult, from, to int) { + t.Helper() + for i := from; i < to; i++ { + if r.l1[i] > r.line+0.01 { + t.Errorf("tick %d: L1 %.2f A, over the %.1f A line with a battery that has room", i, r.l1[i], r.line) + } + } + if t.Failed() { + t.Logf("targets %v", r.targets) + t.Logf("L1 %v", r.l1) + } +} + +// A P1 meter reports phase current in whole amperes. L1 reads 18 A after +// a 2070 W PV drop, and the relief steps -1380 W. A balanced 1.8 A load +// per phase starts in the reading that shows the step, so every phase +// rounds to the same ampere as before while the meter's total moves. +// Before, that counted as a frozen meter: the order stayed at -4385 W and +// the true L1 at 17.8 A (1.11 x In) to the end. A frozen meter repeats its +// total too, so the relief goes on. +func TestFuseReliefGoesOnWhenAWholeAmpereMeterRepeatsItsPhases(t *testing.T) { + r := pvDropRun(pvDropCase{ + resA: 1, + load: [3]float64{15 + 3005.0/690, 5 + 3005.0/690, 5 + 3005.0/690}, + pvW: func(tick int) float64 { + if tick >= 5 { + return 2070 + } + return 0 + }, + extra: func(tick int) [3]float64 { + if tick >= 6 { + return [3]float64{1.8, 1.8, 1.8} + } + return [3]float64{} + }, + }) + assertL1AtTheLine(t, r, 8, 40) +} + +// The relief holds L1 at the line through a slow PV fall, 69 W a tick. +// L1 stays within 0.1 A of its 16.3 A reading from the PV step, so that +// reading stayed the reference while the battery moved 899 W. A 2 kW +// one-phase load on L3 then switches off, which moves L3 by 6.9 balanced +// shares of that move, and a 0.4 A load starts on L1. Before, that read as +// a battery off L1: the order stayed at -3904 W and L1 at 16.4 A to the +// end. L1 at or inside the line ends the reference, so no verdict comes: +// L1 is over the line only on the tick the 0.4 A load starts. (Without +// that end, the step back of the next test ends the hold too, at the cost +// of two ticks with L1 at 16.8 and 17.7 A.) +func TestFuseReliefFollowsALoadAfterADriftAtTheLine(t *testing.T) { + r := pvDropRun(pvDropCase{ + ticks: 50, + pvW: func(tick int) float64 { + switch { + case tick < 5: + return -3200 + case tick < 10: + return -1920 + case tick < 20: + return -1920 + 69*float64(tick-9) + } + return -1920 + 690 + }, + extra: func(tick int) [3]float64 { + var e [3]float64 + if tick >= 20 { + e[2] = -2000.0 / 230 + } + if tick >= 22 { + e[0] = 0.4 + } + return e + }, + }) + assertL1AtTheLine(t, r, 20, 22) + assertL1AtTheLine(t, r, 23, 50) +} + +// The same with a PV fall of 23 W a tick that goes on through the load +// change, so L1 reads 16.03 A, over the line, on every tick and the +// reference stays. A 1 kW load on L3 switches off at tick 30, 3 balanced +// shares of the battery's move, and a 0.4 A load starts on L1 at tick 32. +// Before, the order stayed at -3697 W and L1 climbed to 16.7 A, where it +// stayed. Now the battery goes back to its power when L1 last moved, which +// moves L1 by more than half an ampere on the next tick and ends the +// reference: L1 is at 16.9 A for that one tick, then at the line again. +func TestFuseReliefRecoversFromAFalseOffPhaseVerdict(t *testing.T) { + r := pvDropRun(pvDropCase{ + ticks: 60, + pvW: func(tick int) float64 { + switch { + case tick < 5: + return -3200 + case tick < 10: + return -1920 + case tick < 40: + return -1920 + 23*float64(tick-9) + } + return -1920 + 23*30 + }, + extra: func(tick int) [3]float64 { + var e [3]float64 + if tick >= 30 { + e[2] = -1000.0 / 230 + } + if tick >= 32 { + e[0] = 0.4 + } + return e + }, + }) + over := 0 + for i := 30; i < 60; i++ { + if r.l1[i] > r.line+0.05 { + over++ + } + if r.l1[i] > r.line+1 { + t.Errorf("tick %d: L1 %.2f A, more than 1 A over the %.1f A line", i, r.l1[i], r.line) + } + } + if over > 2 { + t.Errorf("L1 over the %.1f A line on %d ticks after the load change, want 2 at most", r.line, over) + } + assertL1AtTheLine(t, r, 40, 60) +} + +// The off-phase verdict holds until the worst phase's reference ends, and +// gives the batteries' power at that reference. The reference ends when +// the worst phase moves half an ampere, comes inside the line, or another +// phase becomes the worst. A meter whose phases repeat while its total +// moves is not frozen. +func TestPhaseUnresponsiveOffPhaseVerdictHoldsUntilTheReferenceEnds(t *testing.T) { + st := phaseDirectionState(0) + for _, c := range []struct { + name string + amps []float64 + totalW float64 + batW float64 + stuck bool + backTo float64 + }{ + {"first reading: a reference", []float64{16.3, -16.2, 9.5}, 1000, -3000, false, 0}, + {"L3 two shares: a battery off L1", []float64{16.3, -16.2, 7.5}, 300, -3690, true, -3000}, + {"battery back, L3 back: the verdict holds", []float64{16.3, -16.2, 9.5}, 1000, -3000, true, -3000}, + {"L1 0.15 A down, L2 the worst: a new reference", []float64{16.15, -16.2, 9.5}, 950, -3000, false, 0}, + {"L1 the worst again, no move", []float64{16.3, -15, 9.5}, 1300, -3000, false, 0}, + {"L3 two shares again", []float64{16.3, -15, 7.5}, 600, -3690, true, -3000}, + {"L1 at the line: a new reference", []float64{16, -15, 7.5}, 500, -3690, false, 0}, + {"L1 over, no move", []float64{16.3, -15, 7.5}, 600, -3690, false, 0}, + {"same phases, total moved: not frozen", []float64{16.3, -15, 7.5}, 650, -4380, false, 0}, + {"same phases, same total: frozen", []float64{16.3, -15, 7.5}, 650, -5070, true, 0}, + } { + _, stuck, backTo := st.phaseUnresponsive(c.amps, c.totalW, map[string]float64{"bat": c.batW}) + if stuck != c.stuck || backTo["bat"] != c.backTo { + t.Errorf("%s: stuck %v back to %v, want %v back to %v", c.name, stuck, backTo, c.stuck, c.backTo) + } + } +} diff --git a/go/internal/control/fuse_heat.go b/go/internal/control/fuse_heat.go index 80db61c75..0ac83b55e 100644 --- a/go/internal/control/fuse_heat.go +++ b/go/internal/control/fuse_heat.go @@ -20,16 +20,26 @@ const fuseInstantFactor = 1.2 // fuseHeat models each phase of the main fuse as a first-order low-pass on // I², fed only by the site meter's phase currents. A breaker and a fuse both // trip on heat, not on one reading. FTW does not know which one a site has, -// so the EV hard limit holds the filtered current under the working limit, -// below both types' one-hour ratings (a miniature circuit breaker does not -// trip at 1.13 × In, a gG fuse does not blow at 1.25 × In), and gives no -// budget above fuseInstantFactor. +// so the EV hard limit holds the filtered current at or under the working +// limit, the rating less the user's optional margin (fuseLineA). That is at +// or below both types' one-hour ratings (a miniature circuit breaker does not +// trip at 1.13 × In, a gG fuse does not blow at 1.25 × In). A reading above +// the line spends heat budget first; above fuseInstantFactor there is none. // // A reading stands for the time until the next one. A meter gap counts at // the last current seen: the reading after it neither fills the gap with its -// own current nor erases an overload from before it. A Core restart starts -// the model at the first reading's current, with no memory of the minutes -// before. +// own current nor erases an overload from before it. +// +// A Core restart loses the model, but not the fuse's heat: a fuse cools over +// minutes and a restart takes seconds. The model starts at the first +// reading's current, at most the working limit (start). Starting cold would +// grant a full fresh budget to an overload already under way: from cold, +// 1.15 × In takes about 85 s to reach In. Starting at the line, the first +// reading over it spends the budget at once, and a sustained overload is cut +// on the next reading. One reading cannot show that an overage has lasted, +// so it is not charged as if it had; its current still counts for the time +// until the next reading, and the instant cut above fuseInstantFactor +// applies to it as to any other. type fuseHeat struct { set bool at time.Time // the last reading @@ -37,6 +47,15 @@ type fuseHeat struct { sq [3]float64 // filtered I² per phase up to at, A² } +// start seeds the model from the first reading after Core starts: each +// phase at its current, at most lineA. +func (h *fuseHeat) start(at time.Time, amps []float64, lineA float64) { + h.step(at, amps) + for i, sq := range h.sq { + h.sq[i] = math.Min(sq, lineA*lineA) + } +} + // step folds the time since the last reading into the model at that // reading's currents, then holds the new ones. func (h *fuseHeat) step(at time.Time, amps []float64) { @@ -117,9 +136,15 @@ func (s *State) stepFuseHeat(meter *telemetry.DerReading, totalW float64) { if s.fuseHeat.set && !meter.UpdatedAt.After(s.fuseHeat.at) { return } - if amps, _, valid := phaseCurrentsA(meter, s, totalW); valid { - s.fuseHeat.step(meter.UpdatedAt, amps) + amps, _, valid := phaseCurrentsA(meter, s, totalW) + if !valid { + return + } + if !s.fuseHeat.set { + s.fuseHeat.start(meter.UpdatedAt, amps, s.fuseLineA()) // evMeterRoom's limitA + return } + s.fuseHeat.step(meter.UpdatedAt, amps) } // phaseCurrentsA reads the site's configured phases, import positive. A diff --git a/go/internal/control/fuse_heat_test.go b/go/internal/control/fuse_heat_test.go index 1264c2c17..0dc8ef8af 100644 --- a/go/internal/control/fuse_heat_test.go +++ b/go/internal/control/fuse_heat_test.go @@ -107,21 +107,73 @@ func TestFuseHeatCountsGapAtLastCurrentSeen(t *testing.T) { } } -// A Core restart has no memory: the first reading sets the heat. +// A Core restart has no memory: the first reading sets the heat, at most +// the working limit. One reading cannot show that an overage has lasted. func TestFuseHeatStartsAtFirstReadingAfterRestart(t *testing.T) { store, state := fuseHeatState() state.stepFuseHeat0(meterAt(store, time.Unix(1_800_000_000, 0), [3]float64{26, 3, 0})) - if state.fuseHeat.sq != [3]float64{676, 9, 0} { - t.Fatalf("first reading set %v", state.fuseHeat.sq) + if state.fuseHeat.sq != [3]float64{625, 9, 0} { + t.Fatalf("first reading set %v, want L1 at the 25 A working limit", state.fuseHeat.sq) + } + if state.fuseHeat.amps != [3]float64{26, 3, 0} { + t.Fatalf("first reading held %v, want its own currents", state.fuseHeat.amps) + } +} + +// firstStart runs until the first offer above 0 W and returns how long +// after the first reading of the run that was. +func (s *evMeterSite) firstStart(limit time.Duration) time.Duration { + s.t.Helper() + first := s.now.Add(time.Second) + for s.now.Sub(first) < limit { + s.step() + if s.offer() > 0 { + return s.now.Sub(first) + } + } + return -1 +} + +// Seed 207539: Core starts while one reading shows 1.4 × In on L1, then the +// house is back at 10 A per phase. A reading without room starts the idle +// car's dwell, as it always does; the heat that one second added is gone +// within a couple of readings. The car must not also wait the ~52 s a fuse +// already at 35 A would need to cool under the working limit. +func TestEVStartAfterRestartDoesNotWaitOutOneHighReading(t *testing.T) { + s := newEVMeterSite(t, 0, 6900) + s.addHouseA(.4*25+25-10, 0, 0) // 35 A on L1 + s.step() // Core's first reading + s.addHouseA(-(.4*25 + 25 - 10), 0, 0) + got := s.firstStart(3 * time.Minute) + if got < 0 || got > evRestartDwell+3*time.Second { + t.Fatalf("after one 35 A reading the car started after %v, want the %v dwell and at most 3 s of heat", got, evRestartDwell) + } +} + +// A sustained 1.15 × In on L1 right after a restart gets no fresh budget: +// the heat starts at the working limit and the second reading cuts. A cold +// model would let it run for about 72 s before it reached the 24 A line. +func TestEVHardLimitSustainedOverageAfterRestartCuts(t *testing.T) { + if cold := secondsToSpend(0, 1.15*25, 24); cold < 60 { + t.Fatalf("setup: a cold model spends the budget after %d s", cold) + } + s := fuseHeatSite(t) + s.boot() + s.addHouseA(1.15*25-20, 0, 0) // 28.75 A, under the instant cut + took, offer := s.firstCut(2 * time.Minute) + if took < 0 || took > time.Second || offer >= 10350 { + t.Fatalf("1.15 × In after a restart cut after %v to %.0f W, want the second reading", took, offer) } } // fuseHeatSite charges at 15 A beside 5 A of house load on each phase of a // 25 A fuse with no import ceiling: 20 A per phase, long enough for the heat -// to settle there. +// to settle there. The site opts into a 1 A safety margin, so the working +// limit is 24 A. func fuseHeatSite(t *testing.T) *evMeterSite { t.Helper() s := newEVMeterSite(t, 0, 3450) + s.state.SiteFuseSafetyA = 1 s.run(10 * time.Minute) if s.offer() != 10350 || s.evW != 10350 || math.Abs(s.state.fuseHeat.worstA()-20) > .01 { t.Fatalf("setup: offer %.0f draw %.0f heat %.2f A", s.offer(), s.evW, s.state.fuseHeat.worstA()) @@ -273,9 +325,10 @@ func TestEVHardLimitHeatBudgetAcrossMeterGap(t *testing.T) { } } -// After a Core restart the heat starts at the first reading. Restarting -// 30 s into 26 A on L1 counts the fuse as already at 26 A and cuts at once, -// which is conservative only because that reading is high. +// After a Core restart the heat starts at the first reading, at most the +// working limit. Restarting 30 s into 26 A on L1, with about 31 s of budget +// left, counts the fuse as at the line: the second reading spends the +// budget and cuts. func TestEVHardLimitRestartStartsHeatAtFirstReading(t *testing.T) { s := fuseHeatSite(t) s.addHouseA(6, 0, 0) @@ -285,8 +338,8 @@ func TestEVHardLimitRestartStartsHeatAtFirstReading(t *testing.T) { } s.boot() took, offer := s.firstCut(5 * time.Second) - if took != 0 || offer != 8970 { - t.Fatalf("restart at 26 A cut after %v to %.0f W, want the first reading to 8,970 W", took, offer) + if took != time.Second || offer != 8970 { + t.Fatalf("restart at 26 A cut after %v to %.0f W, want the second reading to 8,970 W", took, offer) } } @@ -339,11 +392,14 @@ func TestFuseHeatSixteenAmpPeaks(t *testing.T) { } // fuse16Site is Fredrik's case end to end: a 16 A fuse, 4 A of house load -// and the car at 10 A on each phase, 14 A in all, settled. +// and the car at 10 A on each phase, 14 A in all, settled. The site opts +// into a 1 A safety margin, so the working limit is 15 A, and the car asks +// for 10 A: a rise that fits under the line would otherwise fill it. func fuse16Site(t *testing.T) *evMeterSite { t.Helper() s := newEVMeterSite(t, 0, 2760) - s.state.SiteFuseAmps = 16 + s.state.SiteFuseAmps, s.state.SiteFuseSafetyA = 16, 1 + s.ctrl.SetManualHold("garage", loadpoint.ManualHold{PowerW: 6900, PhaseMode: "3p", Persistent: true}) s.ctrl.SetSiteFuse(loadpoint.SiteFuse{MaxAmps: 16, Voltage: 230, PhaseCnt: 3}) s.ctrl.SetFuseEVMax(func() (float64, bool) { return EVFuseBudget(s.store, s.state, s.mgr.States(), s.mgr.Configs(), 16*690) @@ -357,22 +413,27 @@ func fuse16Site(t *testing.T) *evMeterSite { // The same peaks on L1 with the car charging: patterns whose heat stays // under 15 A while the meter shows a peak never move the offer; the others -// cut once when the budget is spent and settle where the peaks fit. A rise reads the heat as well as -// the reading, so the car does not climb back between peaks and cut again. +// cut when the budget is spent. A rise reads the heat as well as the +// reading, and needs its whole step under the line, so the car climbs back +// only once the heat has cooled a step under it. Where the pattern's heat +// sits right at the line (17 A for 10 s, 14 A for 20 s: 15.07 A rms at +// 10 A) the car steps up and down slowly, about once in five minutes, and +// never stops. func TestEVHardLimitSixteenAmpPeaksCutOnceAndSettle(t *testing.T) { for _, tc := range []struct { name string gap time.Duration // 0: the 17 A stays cutAfter time.Duration // 0: never - settleW float64 + maxCuts int + settleW float64 // 0: not judged }{ - {"14A_50s", 50 * time.Second, 0, 6900}, + {"14A_50s", 50 * time.Second, 0, 0, 6900}, // The heat passes 15 A only as the peak ends, when the meter // already shows 14 A: nothing is left to cut. - {"14A_30s", 30 * time.Second, 0, 6900}, - {"14A_20s", 20 * time.Second, 98 * time.Second, 6210}, - {"14A_10s", 10 * time.Second, 48 * time.Second, 5520}, - {"steady_17A", 0, 23 * time.Second, 5520}, + {"14A_30s", 30 * time.Second, 0, 0, 6900}, + {"14A_20s", 20 * time.Second, 98 * time.Second, 3, 0}, + {"14A_10s", 10 * time.Second, 48 * time.Second, 1, 6210}, + {"steady_17A", 0, 23 * time.Second, 1, 5520}, } { t.Run(tc.name, func(t *testing.T) { s := fuse16Site(t) @@ -401,12 +462,13 @@ func TestEVHardLimitSixteenAmpPeaksCutOnceAndSettle(t *testing.T) { cuts = append(cuts, offers[i]) } } - want := 0 - if tc.cutAfter > 0 { - want = 1 + if len(cuts) > tc.maxCuts || tc.cutAfter > 0 && len(cuts) == 0 { + t.Fatalf("cuts %v, want 1 to %d", offerWatts(cuts), tc.maxCuts) } - if len(cuts) != want { - t.Fatalf("cuts %v, want %d", offerWatts(cuts), want) + for _, o := range offers { + if o.w <= 0 { + t.Fatalf("the peaks stopped the car: %v", offerWatts(offers)) + } } if tc.cutAfter > 0 { // From the first reading of the first peak. @@ -414,7 +476,7 @@ func TestEVHardLimitSixteenAmpPeaksCutOnceAndSettle(t *testing.T) { t.Errorf("cut after %v, want ~%v", took, tc.cutAfter) } } - if s.offer() != tc.settleW { + if tc.settleW > 0 && s.offer() != tc.settleW { t.Errorf("settled at %.0f W, want %.0f W", s.offer(), tc.settleW) } if maxHeat > 15.2 { @@ -556,6 +618,20 @@ func TestExportingPhaseLevelIsItsReading(t *testing.T) { } } +// lastingFuseHeat sets the heat model as if the site meter's last reading +// had held for an hour: the case under test has lasted. +func lastingFuseHeat(store *telemetry.Store, state *State) { + meter := store.Get(state.SiteMeterDriver, telemetry.DerMeter) + amps, _, valid := phaseCurrentsA(meter, state, meter.RawW) + if !valid { + panic("lastingFuseHeat: no usable phase currents") + } + state.fuseHeat = fuseHeat{set: true, at: meter.UpdatedAt.Add(-time.Hour)} + for i, a := range amps { + state.fuseHeat.amps[i], state.fuseHeat.sq[i] = a, a*a + } +} + // stepFuseHeat0 folds a reading with phase currents but no phase power, so // its total does not matter. func (s *State) stepFuseHeat0(meter *telemetry.DerReading) { s.stepFuseHeat(meter, 0) } diff --git a/go/internal/control/fuse_line_property_test.go b/go/internal/control/fuse_line_property_test.go new file mode 100644 index 000000000..a4aa32241 --- /dev/null +++ b/go/internal/control/fuse_line_property_test.go @@ -0,0 +1,347 @@ +package control + +import ( + "encoding/json" + "fmt" + "math" + "math/rand/v2" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/loadpoint" + "github.com/srcfl/ftw/go/internal/sitesim" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// lineRig feeds EVFuseBudget one site reading a second, every configured +// phase at the same current, with one car whose offer the test sets. +type lineRig struct { + t *testing.T + store *telemetry.Store + state *State + now time.Time + phases int + stepW, fuseMaxW float64 + offerW float64 + configs []loadpoint.Config + lastRoom evMeterRoom + lastGridW, lastCapW float64 +} + +func newLineRig(t *testing.T, inA float64, phases int, marginA, offerW float64) *lineRig { + t.Helper() + r := &lineRig{t: t, store: telemetry.NewStore(), state: NewState(0, 0, "meter"), now: time.Unix(1_800_000_000, 0), + phases: phases, stepW: evStepA * 230 * float64(phases), fuseMaxW: inA * 230 * float64(phases), offerW: offerW} + r.state.clock = func() time.Time { return r.now } + r.state.SiteFuseAmps, r.state.SiteFuseVoltage, r.state.SiteFusePhases, r.state.SiteFuseSafetyA = inA, 230, phases, marginA + // The charger declares a response time under evSettle, so an offer is + // shown evSettle after it is sent. + data, _ := json.Marshal(map[string]any{"power_max_age_s": 1}) + r.store.Update("charger", telemetry.DerEV, 0, nil, data) + mode := "3p" + if phases == 1 { + mode = "1p" + } + r.configs = []loadpoint.Config{{ID: "car", DriverName: "charger", PhaseMode: mode, + MinChargeW: 6 * 230 * float64(phases), MaxChargeW: 32 * 230 * float64(phases)}} + return r +} + +// heat sets the fuse model as if each phase had run at levelA, and holds +// heldA until the next reading. +func (r *lineRig) heat(levelA, heldA float64) { + r.state.fuseHeat = fuseHeat{set: true, at: r.now} + for i := range r.phases { + r.state.fuseHeat.sq[i], r.state.fuseHeat.amps[i] = levelA*levelA, heldA + } +} + +// read delivers one reading a second after the last, with every phase at +// a, and returns the cap. +func (r *lineRig) read(a float64) float64 { + r.t.Helper() + r.now = r.now.Add(time.Second) + data := map[string]any{"control_power_available": true, "control_power_observed_at": r.now.Format(time.RFC3339Nano)} + for _, key := range []string{"l1_a", "l2_a", "l3_a"}[:r.phases] { + data[key] = a + } + raw, _ := json.Marshal(data) + r.lastGridW = a * 230 * float64(r.phases) + r.store.Update("meter", telemetry.DerMeter, r.lastGridW, nil, raw) + r.store.Get("meter", telemetry.DerMeter).UpdatedAt = r.now + r.store.RecordDriverSuccess("meter") + points := []loadpoint.State{{ID: "car", DriverName: "charger", PluggedIn: true, CommandedKnown: true, + CommandedW: r.offerW, CommandedPhases: r.phases}} + r.lastCapW, _ = EVFuseBudget(r.store, r.state, points, r.configs, r.fuseMaxW) + r.lastRoom = r.state.evMeterRoom(r.store.Get("meter", telemetry.DerMeter), r.lastGridW, float64(r.phases), r.fuseMaxW, pendingLoad{}) + return r.lastCapW +} + +// The owner's rule of 27 Oct 2026: each phase's working limit is the fuse +// rating less safety_margin_a, with nothing held back below it. Core cuts +// above the line by the heat model and raises only when the next step fits +// under it. +func TestFuseLineProperty(t *testing.T) { + for _, inA := range []float64{16, 20, 25, 35} { + for _, phases := range []int{1, 3} { + for _, marginA := range []float64{0, .5, 1.2} { + lineA := inA - marginA + t.Run(fmt.Sprintf("%gA/%dp/margin_%gA", inA, phases, marginA), func(t *testing.T) { + testFuseLineAtTheLimit(t, inA, phases, marginA, lineA) + testFuseLineJustOver(t, inA, phases, marginA, lineA) + testFuseLineExactFit(t, inA, phases, marginA, lineA) + testFuseLineIdleStart(t, inA, phases, marginA, lineA) + }) + } + } + } +} + +// At the line, with its heat at the line, nothing is over and the cap +// holds for ten minutes. +func testFuseLineAtTheLimit(t *testing.T, inA float64, phases int, marginA, lineA float64) { + t.Helper() + r := newLineRig(t, inA, phases, marginA, 0) + r.offerW = 6 * r.stepW + r.heat(lineA, lineA) + for i := range 600 { + if capW := r.read(lineA); capW != r.offerW { + t.Fatalf("at the line: reading %d cut the cap to %.1f W from %.1f W", i+1, capW, r.offerW) + } + if r.lastRoom.fuseOverA != 0 || r.lastRoom.overLimit { + t.Fatalf("at the line: reading %d over by %.3f A (overLimit %v)", i+1, r.lastRoom.fuseOverA, r.lastRoom.overLimit) + } + } +} + +// 0.1 A over the line: from a cold fuse the cut waits for the heat budget +// and lands on the first reading after it is spent; from a hot fuse it +// lands on the next reading; above 1.2 × In it lands on the same reading. +// Each cut takes at least the 0.1 A and at most one step. +func testFuseLineJustOver(t *testing.T, inA float64, phases int, marginA, lineA float64) { + t.Helper() + overA := lineA + .1 + offerW := 10 * evStepA * 230 * float64(phases) + checkCut := func(name string, r *lineRig, capW float64) { + t.Helper() + if cutW := offerW - capW; cutW < .1*r.stepW-1e-6 || cutW > r.stepW+1e-6 { + t.Fatalf("%s: cut %.1f W, want 0.1 A to one step (%.0f W)", name, cutW, r.stepW) + } + } + + r := newLineRig(t, inA, phases, marginA, offerW) + coldA := lineA - 2 + r.heat(coldA, overA) + tStar := -fuseHeatTau.Seconds() * math.Log(1-(lineA*lineA-coldA*coldA)/(overA*overA-coldA*coldA)) + for k := 1; ; k++ { + capW := r.read(overA) + if float64(k) < tStar { + if capW != offerW { + t.Fatalf("cold: cut to %.1f W at %d s, before the budget is spent at %.1f s", capW, k, tStar) + } + continue + } + if capW >= offerW { + t.Fatalf("cold: no cut at %d s, the first reading after the budget (%.1f s)", k, tStar) + } + checkCut("cold", r, capW) + break + } + + r = newLineRig(t, inA, phases, marginA, offerW) + r.heat(lineA, overA) + if capW := r.read(overA); capW >= offerW { + t.Fatal("hot: no cut on the next reading") + } else { + checkCut("hot", r, capW) + } + + r = newLineRig(t, inA, phases, marginA, offerW) + r.heat(coldA, coldA) + if capW := r.read(fuseInstantFactor*inA + .1); capW >= offerW { + t.Fatalf("above %.1f × In: no cut on the same reading", fuseInstantFactor) + } +} + +// A rise moves one step when level + step is exactly the line, and not +// when it is 0.01 A over. A demand less than a step rises when the room +// equals it, with or without battery charge. +func testFuseLineExactFit(t *testing.T, inA float64, phases int, marginA, lineA float64) { + t.Helper() + settled := func(houseA float64) *lineRig { + t.Helper() + r := newLineRig(t, inA, phases, marginA, 0) + r.offerW = 6 * r.stepW + r.heat(lineA, lineA) + for range 30 { // The offer shows; no room. + r.read(lineA) + } + r.heat(houseA, houseA) // The house fell, and its heat has cooled. + return r + } + r := settled(lineA - evStepA) + if capW := r.read(lineA - evStepA); math.Abs(capW-(r.offerW+r.stepW)) > 1e-6 { + t.Fatalf("level + step = line: cap %.1f W, want one step over the %.0f W offer", capW, r.offerW) + } + r = settled(lineA - evStepA + .01) + if capW := r.read(lineA - evStepA + .01); capW != r.offerW { + t.Fatalf("level + step = line + 0.01 A: cap %.1f W, want the %.0f W offer", capW, r.offerW) + } + for _, charging := range []bool{false, true} { + r = settled(lineA - evStepA/2) + wantW := r.offerW + r.stepW/2 + r.state.SetEVFuseDemands([]loadpoint.FuseDemand{{DriverName: "charger", PowerW: wantW, PhaseMode: r.configs[0].PhaseMode}}) + if charging { + r.state.batteryCommands = map[string]commandLog{"battery": commandLog{}.add(r.now.Add(-time.Minute), 2000)} + } + if capW := r.read(lineA - evStepA/2); math.Abs(capW-wantW) > 1e-6 { + t.Fatalf("half a step to go, room for it, battery charging %v: cap %.1f W, want %.1f W", charging, capW, wantW) + } + } +} + +// An idle car with room for exactly its minimum starts at the minimum +// once the meter has shown that room for evRestartDwell. +func testFuseLineIdleStart(t *testing.T, inA float64, phases int, marginA, lineA float64) { + t.Helper() + r := newLineRig(t, inA, phases, marginA, 0) + minW := r.configs[0].MinChargeW + r.heat(lineA, lineA) + if capW := r.read(lineA); capW != 0 { + t.Fatalf("idle start with no room: cap %.1f W", capW) + } + levelA := lineA - minW/(230*float64(phases)) + for k := 1; k <= 60; k++ { + capW := r.read(levelA) + switch at := time.Duration(k-1) * time.Second; { + case at < evRestartDwell && capW != 0: + t.Fatalf("idle start: %.1f W after %v of room, before the %v dwell", capW, at, evRestartDwell) + case at >= evRestartDwell: + if capW != minW { + t.Fatalf("idle start: cap %.1f W after %v of room for exactly %.0f W", capW, at, minW) + } + return + } + } +} + +// fitScenario runs a 3 × 16 A site with the default safety margin and +// enough house on each phase to leave roomA under the line. With 12 A of +// room, twelve steps of the car fit exactly. The car asks for 16 A. +func fitScenario(roomA float64, run time.Duration, noise func() float64) *simSite { + houseLoadA := 16 - simFuseSafetyA - roomA + sc := simScenario{name: "exact fit", run: run, + plant: func(p *sitesim.Plant) { + p.Fuse.RatingA = 16 + p.HouseW = houseA(houseLoadA, houseLoadA, houseLoadA) + }, + setup: func(s *simSite) { s.meterNoise = noise }} + return runSimScenario(sc, simStart) +} + +// Closed loop: the car climbs to the line and stays there for 30 min. +func TestSiteSimCarEndsAtTheLine(t *testing.T) { + s := fitScenario(12, 30*time.Minute, nil) + if s.failure != "" { + t.Fatal(s.failure) + } + lineA := s.state.fuseLineA() + if want := 12.0 * 690; s.p.EVSE.DrawW != want || s.peakA > lineA+1e-6 { + t.Fatalf("car %.0f W, peak phase %.3f A; want %.0f W with every phase at most %.1f A", s.p.EVSE.DrawW, s.peakA, want, lineA) + } + var last float64 + for _, c := range s.commands { + if c.device == "evse" { + if c.w < last { + t.Fatalf("cut to %v with every phase at or under the line", c) + } + last = c.w + } + } +} + +// Closed loop with meter noise, two hours a case, house load constant. +// Every case: no phase passes the line or 1.2 × In, the plant's gG fuse +// holds, the car never stops, every rise is one step at least evSettle +// after a cut, and the car stays within dropSteps of the top it climbed to. +// +// With part of a step left under the line, the car holds still. With +// exactly whole steps left, the car sits at the line and zero-mean noise +// reads over it about half the time. The heat over the line then cuts a +// step, and the car rises back once the heat has cooled to a step of room: +// a slow cycle, three to five cuts an hour. A reading 1 A over the line, +// as 0.2 A of noise gives now and then, cuts two steps. That cycle is the +// cost of holding no amps back below the line (owner rule of 27 Oct); +// timers and the heat model set its pace. This test pins that pace, so a +// faster hunt fails. +func TestSiteSimNoisyLineCyclesSlowly(t *testing.T) { + for _, tc := range []struct { + roomA, sigma float64 + cutsPerHour float64 // the most allowed after the climb + dropSteps float64 // the furthest below the top + }{ + {12.25, .1, 0, 0}, {11.75, .2, 0, 0}, // part of a step left + {12, .03, 5, 1}, {12, .1, 5, 1}, {12, .2, 5, 2}, // exact fit + } { + t.Run(fmt.Sprintf("room_%.2fA/noise_%.2fA", tc.roomA, tc.sigma), func(t *testing.T) { + rng := rand.New(rand.NewPCG(1, 0)) + s := fitScenario(tc.roomA, 2*time.Hour, func() float64 { return tc.sigma * rng.NormFloat64() }) + if s.failure != "" { + t.Fatal(s.failure) + } + lineA := s.state.fuseLineA() + if s.peakA > lineA+1e-6 || s.peakA > fuseInstantFactor*s.p.Fuse.RatingA || s.p.Fuse.AnyBlown() { + t.Fatalf("peak phase %.2f A on the %.1f A line, fuse %+v", s.peakA, lineA, s.p.Fuse) + } + var moves []simCommand + last := 0.0 + for _, c := range s.commands { + if c.device == "evse" && c.w != last { + moves, last = append(moves, c), c.w + } + } + // The start, then the climb to the line: one step per reading. + climb := 0 + for climb+1 < len(moves) && moves[climb+1].w > moves[climb].w { + climb++ + } + top, after := moves[climb].w, moves[climb+1:] + cuts := 0 + for i, c := range after { + prev := moves[climb] + if i > 0 { + prev = after[i-1] + } + switch { + case c.w < top-tc.dropSteps*690: + t.Fatalf("cut to %.0f W, more than %.0f steps below %.0f W: %v", c.w, tc.dropSteps, top, after) + case c.w < prev.w: + cuts++ + case c.w-prev.w != 690: + t.Fatalf("rose from %.0f W to %.0f W, want one step: %v", prev.w, c.w, after) + case c.at.Sub(prev.at) < evSettle: + t.Fatalf("rose %v after a cut: %v then %v", c.at.Sub(prev.at), prev, c) + } + } + perHour := float64(cuts) / s.p.Now.Sub(moves[climb].at).Hours() + if perHour > tc.cutsPerHour { + t.Fatalf("%d cuts after the climb (%.1f an hour, limit %.0f): %v", cuts, perHour, tc.cutsPerHour, after) + } + t.Logf("car %.0f W, peak phase %.2f A, %d cuts in %v after the climb", + s.p.EVSE.DrawW, s.peakA, cuts, s.p.Now.Sub(moves[climb].at).Round(time.Minute)) + }) + } +} + +// With no safety margin, battery charge may fill a phase to exactly the +// rating beside a car that wants nothing more. +func TestEVBatteryChargeFillsPhaseToTheRating(t *testing.T) { + store, state := evSplitFixture(t, (20+18+18)*230, []float64{20, 18, 18}, 4140, "3p") + state.SiteFuseSafetyA = 0 + sent(state, "ev", 4140) + sent(state, "first", 0) + out := clampEVBatteryHeadroom([]DispatchTarget{{Driver: "first", TargetW: 8000}}, store, state, 17250) + if got, want := targetW(out, "first"), (25.0-20)*690; math.Abs(got-want) > .01 { + t.Fatalf("battery %.1f W, want %.1f W: L1 from 20 A to exactly 25 A", got, want) + } +} diff --git a/go/internal/control/fuse_phase.go b/go/internal/control/fuse_phase.go index da041b83e..0375ea16c 100644 --- a/go/internal/control/fuse_phase.go +++ b/go/internal/control/fuse_phase.go @@ -8,6 +8,14 @@ import ( "github.com/srcfl/ftw/go/internal/telemetry" ) +// fuseLineA is each phase's working limit: the fuse rating less the user's +// optional safety margin, never below 0. Every per-phase fuse decision reads +// it. Anti-hunting lives in timers and in the size of the next step, never in +// amps held back below this line. +func (s *State) fuseLineA() float64 { + return math.Max(0, s.SiteFuseAmps-math.Max(0, s.SiteFuseSafetyA)) +} + // phaseImbalanceA is how far the worst phase sits above the mean phase // current of one meter sample. The aggregate fuse budget assumes balanced // phases, but the breaker, and a charger's own load balancer, act on the @@ -17,7 +25,7 @@ import ( // Meters report phase magnitudes, so the mean uses |aggregate|. Phases // flowing in opposite directions make the gap look smaller than it is; the // per-phase fuse guards still act on a real overage. A single-phase battery -// or charger is outside this model, as it is for perPhaseReliefW. +// or charger is outside this model, as it is for phaseFuseReliefW. func phaseImbalanceA(r *telemetry.DerReading, state *State) (float64, bool) { if state == nil || state.SiteFuseAmps <= 0 || state.SiteFuseVoltage <= 0 || state.SiteFusePhases <= 1 { return 0, false diff --git a/go/internal/control/fuse_saver_test.go b/go/internal/control/fuse_saver_test.go index 0e2f48476..fac8238ae 100644 --- a/go/internal/control/fuse_saver_test.go +++ b/go/internal/control/fuse_saver_test.go @@ -300,19 +300,18 @@ func setupPerPhaseSite(phases int, voltage float64, aggGridW float64, l1A, l2A, func TestPerPhaseClampScalesChargingOnImbalance(t *testing.T) { // Isolate the per-phase case: aggregate predicted (7000 + 4000 = // 11000 W) is under the 11040 W fuse, so the legacy aggregate - // branch wouldn't fire. But L1 alone is at 18 A × 230 V = 4140 W - // (over the 16 A fuse on that phase) — only the per-phase clamp - // can catch it. - store, state, _ := setupPerPhase(7000, 18, 12, 8, 0.6, 10000) + // branch wouldn't fire. But the 4000 W charge adds 4000 / 690 = + // 5.8 A to each phase, which takes L1 from 13 A to 18.8 A, over the + // 16 A fuse (margin 0) — only the per-phase clamp can catch it. + store, state, _ := setupPerPhase(7000, 13, 12, 8, 0.6, 10000) targets := []DispatchTarget{{Driver: "bat", TargetW: 4000}} guarded := applyFuseGuard(targets, store, state, 11040) if !guarded[0].Clamped { t.Errorf("Clamped flag must mark per-phase response") } - // Per-phase overage = (18 − 16) × 230 = 460 W on the worst phase. - // Balanced-3Φ assumption ⇒ 460 × 3 = 1380 W of total battery - // reduction needed. Charge 4000 → 4000 − 1380 = 2620 W. - expected := 2620.0 + // L1 has (16 − 13) × 690 = 2070 W of room for a balanced-3Φ + // charge. Charge 4000 → 2070 W. + expected := 2070.0 if math.Abs(guarded[0].TargetW-expected) > 1 { t.Errorf("charge after per-phase clamp = %.0f W, want %.0f W", guarded[0].TargetW, expected) @@ -384,8 +383,9 @@ func TestPerPhaseClampDominatesAggregateWhenLarger(t *testing.T) { // trigger forceFuseDischarge, which would push more power out and // worsen the over-current phase. Regression for PR #219 review S1. func TestPerPhaseClampFiresOnExportImbalance(t *testing.T) { - // Aggregate gridW = -7000 W (heavy export). Phases (magnitude): - // L2 = 18 A (over the 16 A fuse), L1 = 12, L3 = 8. Per-phase + // Aggregate gridW = -7000 W (heavy export). Phases (magnitude, no + // phase power, so they take the total's direction): L2 = 18 A (over + // the 16 A fuse), L1 = 12, L3 = 8. Per-phase // overage = (18 − 16) × 230 × 3 = 1380 W of discharge to shrink. // Battery is currently discharging −5000 W; the target reduces // magnitude to bring L2 back under fuse. @@ -460,43 +460,40 @@ func TestForceFuseDischargeIgnoresExportSidePerPhase(t *testing.T) { // inverter's own per-phase limiter trips first and causes a flap). // Regression for PR #219 — the bug class motivating the safety margin. func TestPerPhaseClampHonorsSafetyMargin(t *testing.T) { - // Phases (15.5, 12, 8). MaxAmps = 16, SafetyA = 1.0 → effective - // threshold = 15 A. L1 at 15.5 A is OVER the effective threshold - // (under the raw 16 A). Per-phase overage = (15.5 − 15) × 230 × 3 - // = 345 W of charge to shrink. - store, state, _ := setupPerPhase(7000, 15.5, 12, 8, 0.6, 10000) + // Phases (10, 7, 5). MaxAmps = 16, SafetyA = 1.0 → effective + // threshold = 15 A. The 4000 W charge adds 5.8 A per phase, so L1 + // reaches 15.8 A: over 15 A by 0.8 A, 550 W of charge to shrink. + // Without the margin it is under the 16 A fuse and nothing clamps. + store, state, _ := setupPerPhase(5000, 10, 7, 5, 0.6, 10000) state.SiteFuseSafetyA = 1.0 state.SiteFusePhases = 3 targets := []DispatchTarget{{Driver: "bat", TargetW: 4000}} guarded := applyFuseGuard(targets, store, state, 11040) if !guarded[0].Clamped { - t.Errorf("safety margin should pull the threshold under L1 = 15.5 A") + t.Errorf("safety margin should pull the threshold under L1") } // Aggregate effFuseW = 11040 − 1.0×230×3 = 10350. Predicted = - // 7000 + 4000 − 0 = 11000 → aggregate overage = 11000 − 10350 = 650. - // Per-phase overage × 3 = 345. Aggregate wins (650 > 345). - // Buffer = half-margin = 0.5×1×230×3 = 345 W (post-fix headroom). - // Charge 4000 − 650 − 345 = 3005. - expected := 3005.0 + // 5000 + 4000 − 0 = 9000 → no aggregate overage. Per-phase wins. + // Charge 4000 − 550 = 3450, which puts L1 on the 15 A line. + expected := 3450.0 if math.Abs(guarded[0].TargetW-expected) > 1 { t.Errorf("charge after safety-margin clamp = %.0f W, want %.0f W", guarded[0].TargetW, expected) } } -// SafetyMarginA = 0 must produce identical output to the pre-PR -// behaviour (locks the back-compat contract every existing per-phase -// test relies on by leaving the field unset). +// SafetyMarginA = 0 works to the fuse rating (every existing per-phase +// test relies on it by leaving the field unset). func TestPerPhaseClampSafetyMarginZeroIsBackCompat(t *testing.T) { - store, state, _ := setupPerPhase(7000, 18, 12, 8, 0.6, 10000) + store, state, _ := setupPerPhase(7000, 13, 12, 8, 0.6, 10000) state.SiteFuseSafetyA = 0 state.SiteFusePhases = 3 targets := []DispatchTarget{{Driver: "bat", TargetW: 4000}} guarded := applyFuseGuard(targets, store, state, 11040) // Same math as TestPerPhaseClampScalesChargingOnImbalance. - expected := 2620.0 + expected := 2070.0 if math.Abs(guarded[0].TargetW-expected) > 1 { - t.Errorf("safety_margin_a=0 must match pre-PR behaviour: got %.0f W, want %.0f W", + t.Errorf("safety_margin_a=0 must work to the 16 A rating: got %.0f W, want %.0f W", guarded[0].TargetW, expected) } } @@ -560,10 +557,10 @@ func TestPerPhaseReliefMatchesSinglePhaseSiteOnExport(t *testing.T) { if !guarded[0].Clamped { t.Fatalf("export-side per-phase overage must clamp the discharge target") } - // Discharge 5000 − overage 1955 − buffer (half the 115 W margin) - // = 2987.5 W. Demanding 1955 × 3 leaves nothing: the discharge is - // zeroed and the site swings to 520 W of import. - expected := -2987.5 + // Discharge 5000 − overage 1955 = 3045 W. Demanding 1955 × 3 leaves + // nothing: the discharge is zeroed and the site swings to 520 W of + // import. + expected := -3045.0 if math.Abs(guarded[0].TargetW-expected) > 1 { t.Errorf("discharge after 1Φ export clamp = %.1f W, want %.1f W", guarded[0].TargetW, expected) @@ -599,15 +596,15 @@ func TestPerPhaseClampReadsConfiguredPhasesOnly(t *testing.T) { // Per-phase watts come from the configured voltage, the same source // fuseSafetyMarginW uses. A 240 V site converts its amps at 240 V. func TestPerPhaseClampUsesConfiguredVoltage(t *testing.T) { - store, state, _ := setupPerPhaseSite(3, 240, 7000, 18, 12, 8, 0.6, 10000) + store, state, _ := setupPerPhaseSite(3, 240, 7000, 13, 12, 8, 0.6, 10000) targets := []DispatchTarget{{Driver: "bat", TargetW: 4000}} guarded := applyFuseGuard(targets, store, state, 16*240.0*3) if !guarded[0].Clamped { - t.Fatalf("L1 at 18 A on a 16 A fuse must clamp") + t.Fatalf("a 4000 W charge on L1 at 13 A on a 16 A fuse must clamp") } - // (18 − 16) × 240 = 480 W per phase × 3 = 1440 W of relief. - // Charge 4000 → 2560. At the old hardcoded 230 V it would be 2620. - expected := 2560.0 + // L1 has (16 − 13) A of room below the fuse: 3 × 240 × 3 = 2160 W + // of charge. At the old hardcoded 230 V it would be 2070. + expected := 2160.0 if math.Abs(guarded[0].TargetW-expected) > 1 { t.Errorf("charge after per-phase clamp on a 240 V site = %.0f W, "+ "want %.0f W", guarded[0].TargetW, expected) @@ -655,33 +652,63 @@ func TestFuseSaverFiresInIdleMode(t *testing.T) { } } -// Operator-report 2026-04-30: with safety_margin_a=1 (threshold = 15 A) -// the system stabilised with phase amps oscillating between 15.0 and -// 15.8 A — fuse guard reactively scaled targets just enough to clear -// the trip, then the planner re-ramped, then it re-tripped. Half-margin -// buffer keeps post-clamp aggregate below the threshold by an explicit -// margin so the next tick has room before re-tripping. -func TestFuseGuardLeavesHeadroomBelowThreshold(t *testing.T) { - store, state, _ := setupPerPhase(-7000, 12, 18, 8, 0.6, 10000) - state.SiteFuseSafetyA = 1.0 - state.SiteFusePhases = 3 - soc := 0.6 - store.Update("bat", telemetry.DerBattery, -5000, &soc, nil) - store.DriverHealthMut("bat").RecordSuccess() +// A guard cut lands the worst phase on the line, In − safety_margin_a, and +// stays there. It takes no extra headroom below the line: the prediction +// from held targets and the 30 s hold keep the next dispatch from +// re-ramping through it. 16 A fuse, margin 1.2 A, so the line is 14.8 A. +// The battery follows each order; the loop runs 60 ticks of 5 s, so the +// hold expires and fires again several times. +func TestFuseGuardCutLandsOnTheLine(t *testing.T) { + const marginA, lineA = 1.2, 14.8 + for _, tc := range []struct { + name string + houseA [3]float64 // signed: + import, − export + requestW float64 + roomA float64 // worst phase's room to the line before the battery moves + }{ + {"charge on import", [3]float64{13, 7, 5}, 4000, lineA - 13}, + {"discharge on export", [3]float64{-8, -12, -5}, -5000, lineA - 12}, + } { + t.Run(tc.name, func(t *testing.T) { + st := NewState(0, 50, "meter") + st.SiteFuseAmps, st.SiteFuseVoltage, st.SiteFusePhases, st.SiteFuseSafetyA = 16, 230, 3, marginA + st.DriverLimits = map[string]PowerLimits{"bat": {MaxChargeW: 10000, MaxDischargeW: 10000}} + now := time.Date(2026, 10, 27, 12, 0, 0, 0, time.UTC) + st.clock = func() time.Time { return now } + store := telemetry.NewStore() + wantW := math.Copysign(tc.roomA*690, tc.requestW) + batW := 0.0 + for tick := 0; tick < 60; tick++ { + a := tc.houseA + data := map[string]any{} + for i := range a { + a[i] += batW / 690 + data[fmt.Sprintf("l%d_a", i+1)] = a[i] + data[fmt.Sprintf("l%d_w", i+1)] = a[i] * 230 + } + raw, _ := json.Marshal(data) + store.Update("meter", telemetry.DerMeter, (a[0]+a[1]+a[2])*230, nil, raw) + store.DriverHealthMut("meter").RecordSuccess() + soc := 0.5 + store.Update("bat", telemetry.DerBattery, batW, &soc, nil) + store.DriverHealthMut("bat").RecordSuccess() + + worstA := 0.0 + for _, x := range a { + worstA = math.Max(worstA, math.Abs(x)) + } + if tick > 0 && math.Abs(worstA-lineA) > 0.01 { + t.Fatalf("tick %d: worst phase %.3f A, want %.1f A (the line, not below it)", tick, worstA, lineA) + } - out := applyFuseGuard( - []DispatchTarget{{Driver: "bat", TargetW: -5000}}, - store, state, 11040) - if !out[0].Clamped { - t.Fatalf("expected per-phase export clamp") - } - // perPhase = (18−15) × 230 × 3 = 2070 W - // half-margin buffer = 0.5 × 1A × 230V × 3p = 345 W - // totalDischarge = 5000 → newTotal = 5000 − 2070 − 345 = 2585 - expected := -2585.0 - if math.Abs(out[0].TargetW-expected) > 1 { - t.Errorf("post-clamp target = %.0f W, want %.0f W (overage + half-margin)", - out[0].TargetW, expected) + out := applyFuseGuard([]DispatchTarget{{Driver: "bat", TargetW: tc.requestW}}, store, st, 16*230*3) + if math.Abs(out[0].TargetW-wantW) > 1 { + t.Fatalf("tick %d: target %.1f W, want %.1f W (cut by the overage only)", tick, out[0].TargetW, wantW) + } + batW = out[0].TargetW + now = now.Add(5 * time.Second) + } + }) } } diff --git a/go/internal/control/golden_dump_test.go b/go/internal/control/golden_dump_test.go index ae96c9960..7bd618d44 100644 --- a/go/internal/control/golden_dump_test.go +++ b/go/internal/control/golden_dump_test.go @@ -331,7 +331,12 @@ func runGoldenScenarioAt(sc goldenScenario, scenarioNow time.Time, slotDirective // Pre-dispatch attribution reads (pure). intent := planSignIntent(st) - perPhase := perPhaseOverageW(store, st) + // The worst phase's overage either way, in watts on that phase. + var perPhase float64 + importW, exportW := phaseFuseReliefW(livePhaseSample(store, st.SiteMeterDriver, store.Get(st.SiteMeterDriver, telemetry.DerMeter)), st, 0) + if st.SiteFusePhases > 0 { + perPhase = math.Max(importW, exportW) / float64(st.SiteFusePhases) + } effImp := st.effectiveImportCeilingW(sc.Inputs.FuseMaxW) effExp := st.effectiveExportCeilingW(sc.Inputs.FuseMaxW) @@ -1316,8 +1321,9 @@ func slewScenarios() []goldenScenario { State: plan(ModePlannerSelf, 500), Slot: staleSlot(), })) - // A discharge-blocked battery is parked at 0 W by the distributor and - // then walked back toward its live discharge by the same mechanism. + // A discharge-blocked battery is parked at 0 W by the distributor. Slew + // used to walk it back toward its live discharge; floorBlockedDischarge + // now holds it at 0 W while the fuse does not need the ramp. blk := gb("ferroamp", 15200, -2000, 0.50) blk.DischargeBlocked = true out = append(out, mk("blocked_sibling_slew_reopens_parked_target", goldenInputs{ diff --git a/go/internal/control/holdoff_cadence_test.go b/go/internal/control/holdoff_cadence_test.go new file mode 100644 index 000000000..a3a5c8602 --- /dev/null +++ b/go/internal/control/holdoff_cadence_test.go @@ -0,0 +1,74 @@ +package control + +import ( + "testing" + "time" +) + +// runHoldoffTicks models main.go: ComputeDispatch runs once per fixed 2 s +// ticker tick, after a variable amount of pre-dispatch work. Each clock read +// inside ComputeDispatch advances the fake clock by 50 µs to model the +// dispatch call's own run time. It returns how many ticks sent no battery +// command. +func runHoldoffTicks(t *testing.T, minDispatchS, controlS int, preWorkMs []int) (skipped int) { + t.Helper() + t0 := time.Now() + st := NewState(0, 42, "meter") + st.MinDispatchIntervalS = minDispatchS + st.ControlIntervalS = controlS + st.SlewRateW = 3000 + var now time.Time + st.clock = func() time.Time { + now = now.Add(50 * time.Microsecond) + return now + } + capsMap := caps(map[string]float64{"bat": 10000}) + for k, w := range preWorkMs { + // Ticker fires at t0 + 2k s; pre-dispatch work takes w ms. + now = t0.Add(time.Duration(2*k)*time.Second + time.Duration(w)*time.Millisecond) + // House imports 1500 W, battery idle: every normal tick must discharge. + store := seedDeadbandSite(1500, []deadbandBattery{{"bat", 0, 0.6, ""}}) + if out := ComputeDispatch(store, st, capsMap, 0); len(out) == 0 { + skipped++ + t.Logf("tick %d (+%d ms): holdoff exit, no battery command", k, w) + } + } + return skipped +} + +// Constant pre-dispatch work: no jitter at all. Before the tolerance, the +// dispatch call's own run time alone pulled elapsed just below 2 s on every +// tick after a dispatch, so half of all ticks sent no battery command. +func TestHoldoffDefaultCadenceConstantWork(t *testing.T) { + work := []int{100, 100, 100, 100, 100, 100, 100, 100, 100, 100} + if n := runHoldoffTicks(t, 2, 0, work); n > 0 { + t.Errorf("%d of %d ticks issued no battery command at the default 2 s/2 s cadence", n, len(work)) + } +} + +// Jittered pre-dispatch work, as from loadpoint commands and SQL writes. +func TestHoldoffDefaultCadenceJitteredWork(t *testing.T) { + work := []int{120, 80, 150, 90, 140, 60, 110, 100, 130, 70} + if n := runHoldoffTicks(t, 2, 0, work); n > 0 { + t.Errorf("%d of %d ticks issued no battery command at the default 2 s/2 s cadence", n, len(work)) + } +} + +// A charger command can hold one tick's pre-work for up to half the control +// interval. With the holdoff no longer than one tick, no tick is skipped, +// even when the jitter is far beyond the tolerance. +func TestHoldoffOffWhenNotLongerThanControlInterval(t *testing.T) { + work := []int{1000, 50, 900, 20, 1000, 10, 800, 30, 1000, 0} + if n := runHoldoffTicks(t, 2, 2, work); n > 0 { + t.Errorf("%d of %d ticks issued no battery command with min_dispatch_interval_s == control_interval_s", n, len(work)) + } +} + +// A holdoff longer than the tick still spaces the commands: 5 s on 2 s +// ticks dispatches at 0, 6, 12 and 18 s and holds the six ticks between. +func TestHoldoffLongerThanControlIntervalStillHolds(t *testing.T) { + work := []int{100, 100, 100, 100, 100, 100, 100, 100, 100, 100} + if n := runHoldoffTicks(t, 5, 2, work); n != 6 { + t.Errorf("got %d held ticks, want 6 with a 5 s holdoff on 2 s ticks", n) + } +} diff --git a/go/internal/control/import_hour.go b/go/internal/control/import_hour.go index bb69f7823..21be1f2ca 100644 --- a/go/internal/control/import_hour.go +++ b/go/internal/control/import_hour.go @@ -5,6 +5,7 @@ import ( "math" "time" + "github.com/srcfl/ftw/go/internal/mpc" "github.com/srcfl/ftw/go/internal/telemetry" ) @@ -177,7 +178,7 @@ func (s *State) stepHourImportLimits(at time.Time, gap bool) { "imported_wh", h.importedWh, "limit_w", rateW, "was_w", h.limitW) } h.limitW, h.riseW, h.changedAt = rateW, math.Min(h.riseW, rateW), at - step := evPhaseMarginA * s.SiteFuseVoltage * float64(s.SiteFusePhases) + step := evStepA * s.SiteFuseVoltage * float64(s.SiteFusePhases) left := h.start.Add(time.Hour).Sub(at) if gap || !(step > 0) || at.Sub(h.cutAt) < importHourRiseGap || left < importHourRiseMinLeft || rateW < h.riseW+2*step { return @@ -218,6 +219,50 @@ func (s *State) importRiseCeilingW(fuseMaxW float64) float64 { return eff } +// PlanningImportCeiling is the ceiling as the planner gets it, from the hour +// the live limit counts: the clock hour in progress and what it has used. +// Time with no record counts at the ceiling. Until a reading measures the +// hour against this ceiling, the rest of the hour may hold the ceiling, so +// the time before counts as used. A planned hour holds the ceiling less half +// a rise step: the car runs in whole steps and a rise keeps a step in hand, +// so a car under the ceiling ends each hour 0.4-0.6 kWh short of it. Planned +// at the ceiling the plan promised the car 8-15 % more than it got; at half +// a step less the car gets as much, within 1 % of the promise with the +// battery covering the house and 6 % without +// (TestSiteSimPlanUnderImportCeiling). Zero while the ceiling is off. +// Callers hold the control lock. +func PlanningImportCeiling(s *State) mpc.ImportCeiling { + if s == nil || !(s.PeakImportCeilingW > 0) { + return mpc.ImportCeiling{} + } + now, ceiling := s.now(), s.PeakImportCeilingW + start := s.importHourStart(now) + elapsedWh := ceiling * now.Sub(start).Hours() + if h := &s.importHour; s.importHourMeasured() { + elapsedWh = h.importedWh + ceiling*h.unknown.Hours() + } + step := evStepA * s.SiteFuseVoltage * float64(s.SiteFusePhases) + return mpc.ImportCeiling{W: ceiling, MeanW: max(0, ceiling-step/2), HourStart: start, ElapsedWh: elapsedWh} +} + +// MeasuredHourImport is the clock hour in progress as the site meter +// measured it, with or without a ceiling: the import counted so far and the +// time its readings cover (time with no record is left out, not counted at +// the ceiling). The planner's demand charges price that hour from it. Zero +// until a reading or the boot seed counts this hour. Callers hold the +// control lock. +func MeasuredHourImport(s *State) mpc.HourImport { + if s == nil { + return mpc.HourImport{} + } + h := &s.importHour + if h.lastAt.IsZero() || !s.importHourStart(s.now()).Equal(h.start) { + return mpc.HourImport{} + } + covered := max(0, h.lastAt.Sub(h.start)-h.unknown) + return mpc.HourImport{Start: h.start, ImportedWh: h.importedWh, Covered: covered} +} + // ImportHourStatus is the clock hour's import budget left, negative once it // is overrun, and the import limit in force. ok is false while the ceiling is // off. Callers hold the control lock or a copy of State. diff --git a/go/internal/control/import_hour_planning_test.go b/go/internal/control/import_hour_planning_test.go new file mode 100644 index 000000000..63f76d518 --- /dev/null +++ b/go/internal/control/import_hour_planning_test.go @@ -0,0 +1,111 @@ +package control + +import ( + "math" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/mpc" +) + +// The planner gets the clock hour Core's live limit counts and what it has +// used. On 25 October 2026 02:00 comes twice in Stockholm, and each of the +// two hours counts its own import. +func TestPlanningImportCeilingOnTheLongDay(t *testing.T) { + zone, err := time.LoadLocation("Europe/Stockholm") + if err != nil { + t.Fatal(err) + } + var now time.Time + st, store := newImportHourState(&now) + st.PeakImportCeilingW, st.importHourZone = 4500, zone + first := time.Date(2026, 10, 25, 0, 0, 0, 0, time.UTC) // 02:00 CEST + for _, hour := range []time.Time{first, first.Add(time.Hour)} { + for at := hour; !at.After(hour.Add(20 * time.Minute)); at = at.Add(2 * time.Second) { + feedMeter(st, store, &now, at, 3000) + } + got := PlanningImportCeiling(st) + if !got.HourStart.Equal(hour) || got.HourStart.In(zone).Hour() != 2 || got.W != 4500 { + t.Fatalf("at %s: hour from %s, ceiling %.0f W", now.In(zone).Format("15:04 MST"), got.HourStart.In(zone).Format("15:04 MST"), got.W) + } + if want := 3000 * 20 / 60.; math.Abs(got.ElapsedWh-want) > 1 { + t.Fatalf("at %s: %.1f Wh used, want %.1f Wh", now.In(zone).Format("15:04 MST"), got.ElapsedWh, want) + } + } + st.PeakImportCeilingW = 0 + if got := PlanningImportCeiling(st); got != (mpc.ImportCeiling{}) { + t.Fatalf("ceiling off gave %+v", got) + } +} + +// A planned hour holds the ceiling less half a rise step, the margin +// TestSiteSimPlanUnderImportCeiling measures. +func TestPlanningImportCeilingKeepsHalfARiseStep(t *testing.T) { + now := time.Date(2026, 10, 26, 9, 0, 0, 0, hourTestZone) + st, _ := newImportHourState(&now) + st.PeakImportCeilingW = 4500 + // Half of 1 A on each of three 230 V phases. + if got := PlanningImportCeiling(st); got.MeanW != 4155 { + t.Fatalf("planned mean %.0f W, want 4155 W", got.MeanW) + } + st.PeakImportCeilingW = 300 + if got := PlanningImportCeiling(st); got.W != 300 || got.MeanW != 0 { + t.Fatalf("a ceiling under half a step: %+v, want it on with no room", got) + } +} + +// Before a reading measures the hour, as after a restart or a new ceiling, +// the live limit lets the rest of the hour hold the ceiling: the time before +// counts as used. A restart that seeds the hour from state.db counts what +// the record covers, and the time it misses at the ceiling. +func TestPlanningImportCeilingAfterARestart(t *testing.T) { + now := time.Date(2026, 10, 26, 9, 40, 0, 0, hourTestZone) + st, store := newImportHourState(&now) + st.PeakImportCeilingW = 4500 + if got := PlanningImportCeiling(st); math.Abs(got.ElapsedWh-3000) > 1e-6 || got.HourStart.Minute() != 0 { + t.Fatalf("at boot: %.1f Wh used in the hour from %s; want 3000 Wh from 09:00", got.ElapsedWh, got.HourStart.Format("15:04")) + } + // state.db covered 30 of the 40 minutes with 1.2 kWh. + st.SeedHourImport(now, 1200, 30*time.Minute) + feedMeter(st, store, &now, now.Add(time.Second), 2000) + if got := PlanningImportCeiling(st); math.Abs(got.ElapsedWh-(1200+4500/6.+2000/3600.)) > 1 { + t.Fatalf("seeded: %.1f Wh used, want 1200 Wh plus 10 unseen minutes at the ceiling", got.ElapsedWh) + } + now = now.Add(10 * time.Minute) + feedMeter(st, store, &now, now, 2000) + st.PeakImportCeilingW = 3000 // set during the hour, not yet seen by a reading + if got := PlanningImportCeiling(st); math.Abs(got.ElapsedWh-2500) > 1 { + t.Fatalf("new ceiling: %.1f Wh used, want 50 minutes at 3 kW", got.ElapsedWh) + } +} + +// The demand charges' hour in progress counts what the site meter measured, +// with or without an import ceiling, and only the time its readings cover: +// the planner fills the rest itself. +func TestMeasuredHourImportCountsTheMeteredHour(t *testing.T) { + var now time.Time + st, store := newImportHourState(&now) + st.PeakImportCeilingW = 0 + hour := time.Date(2026, 9, 9, 17, 0, 0, 0, hourTestZone) + if got := MeasuredHourImport(st); got != (mpc.HourImport{}) { + t.Fatalf("before a reading %+v", got) + } + for at := hour.Add(5 * time.Minute); !at.After(hour.Add(20 * time.Minute)); at = at.Add(2 * time.Second) { + feedMeter(st, store, &now, at, 3000) + } + got := MeasuredHourImport(st) + if !got.Start.Equal(hour) || got.Covered != 15*time.Minute || math.Abs(got.ImportedWh-3000*15/60.) > 1 { + t.Fatalf("got %+v; want 750 Wh over the 15 minutes from 17:05", got) + } + // After a restart the record restores the hour; what it misses is not covered. + seeded, _ := newImportHourState(&now) + seeded.SeedHourImport(now, 600, 12*time.Minute) + if got := MeasuredHourImport(seeded); !got.Start.Equal(hour) || got.Covered != 12*time.Minute || got.ImportedWh != 600 { + t.Fatalf("seeded %+v", got) + } + // In the next hour, before its first reading, the last hour's import is gone. + now = hour.Add(time.Hour + time.Minute) + if got := MeasuredHourImport(st); got != (mpc.HourImport{}) { + t.Fatalf("next hour %+v", got) + } +} diff --git a/go/internal/control/meter_freeze.go b/go/internal/control/meter_freeze.go new file mode 100644 index 000000000..913f7a5da --- /dev/null +++ b/go/internal/control/meter_freeze.go @@ -0,0 +1,211 @@ +package control + +import ( + "encoding/json" + "log/slog" + "math" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// A site meter can answer every poll with the same numbers. Each answer is +// a new sample, so every check that keys on UpdatedAt calls it fresh. In rig +// seed 207537 a meter repeated one sample for 39.9 s; dispatch wound the +// battery from +4 kW to -5 kW on it and raised an EV. +// +// A live meter must show a large move of FTW's own devices. When FTW's +// orders move, the devices' own readings follow, and the meter still repeats +// the same total and phase values long after that, the meter is frozen and +// counts as stale. An order a device does not follow (a full battery, a car +// that does not draw, a charger that has not applied it yet) does not count, +// and moves that cancel each other out do not count, since the meter sees +// only their sum. +const ( + // meterFreezeMoveW is how far FTW's devices must move in sum, by order + // and by their own readings, from where they stood when the meter last + // changed. Site meters resolve the total to 1-10 W and phase currents + // to 0.01-1 A (1 A is 230 W). Batteries follow an order to within tens + // of watts, and inverters ignore steps below about 100-200 W. 500 W is + // 50 times the coarsest total step and above every inverter dead band. + // Each EV step is at least 1380 W. + meterFreezeMoveW = 500.0 + // meterFreezeAfter is how long a live meter may take to show that move. + // Swedish HAN and DSMR 4 meters send a reading every 10 s, so a driver + // polling every second sees up to 10 identical samples by design. A + // telegram lost to a CRC error leaves the last values in place for 20 s, + // and FTW makes its large moves right after a telegram that shows a load + // step. 30 s is three telegram periods, so one lost telegram cannot trip + // it. Seed 207537's freeze is still caught before its EV rise at 38 s. + meterFreezeAfter = 30 * time.Second + // meterFreezeSamples is how many identical samples after the move the + // verdict needs, so a slow poller is judged on several answers. + meterFreezeSamples = 3 +) + +// meterSample is the total and per-phase values a site meter reports. +// Absent fields compare as absent. +type meterSample struct { + vals [10]float64 + has [10]bool +} + +type meterFreeze struct { + driver string + at time.Time // UpdatedAt of the last sample seen + sample meterSample + // devices are FTW's devices when the sample last changed. + devices map[ownDeviceKey]ownDevice + movedAt time.Time // when the devices first moved past meterFreezeMoveW + repeats int // identical samples since movedAt + // repeatAfterMove: the sample at `at` repeats the values from before + // the devices moved past meterFreezeMoveW. See siteMeterRepeatsAfterMove. + repeatAfterMove bool + frozen bool +} + +// ObserveSiteMeterFreeze compares the site meter's newest sample with the +// one before it, and FTW's own devices with where they stood when the meter +// last changed. Call it every control tick under the control lock, before +// the dispatch gate reads SiteMeterFrozen. A quiet site whose devices do not +// move is never frozen, and any change in the meter's values clears it. +func (s *State) ObserveSiteMeterFreeze(store *telemetry.Store, now time.Time) { + if s == nil || store == nil { + return + } + f := &s.meterFreeze + w, data, at, ok := store.OnlineMeterSample(s.SiteMeterDriver) + if !ok || f.driver != s.SiteMeterDriver { + // An offline meter is stale on its own; start over once it returns. + *f = meterFreeze{driver: s.SiteMeterDriver} + if !ok { + return + } + } + // No new sample: siteMeterHeld and the watchdog handle that. + if !at.After(f.at) { + return + } + sample := readMeterSample(w, data) + devices := s.ownDevices(store) + if f.at.IsZero() || sample != f.sample { + if f.frozen { + slog.Info("site meter values move again", "site_meter", f.driver) + } + *f = meterFreeze{driver: f.driver, at: at, sample: sample, devices: devices} + return + } + f.at = at + if f.frozen { + return // only a change in the meter's values clears it + } + moved := devicesMovedW(f.devices, devices) + if math.Abs(moved) <= meterFreezeMoveW { + // Not moved, or moved back: a live meter may repeat itself here. + f.movedAt, f.repeats, f.repeatAfterMove = time.Time{}, 0, false + return + } + f.repeatAfterMove = true + if f.movedAt.IsZero() { + f.movedAt = now + return + } + f.repeats++ + if f.repeats >= meterFreezeSamples && now.Sub(f.movedAt) >= meterFreezeAfter { + f.frozen = true + slog.Warn("site meter repeats one reading while FTW's devices moved; treating it as stale", + "site_meter", f.driver, "devices_moved_w", math.Round(moved), + "since", now.Sub(f.movedAt).Round(time.Second), "samples", f.repeats) + } +} + +// SiteMeterFrozen reports whether ObserveSiteMeterFreeze found the site +// meter repeating one reading while FTW's devices moved. +func (s *State) SiteMeterFrozen() bool { + return s != nil && s.meterFreeze.frozen +} + +// siteMeterRepeatsAfterMove reports whether meter is the sample +// ObserveSiteMeterFreeze judged last, and it repeats the values from before +// FTW's devices moved by more than meterFreezeMoveW. A live meter shows +// such a move, so the repeat carries no new error. Battery dispatch treats +// it as a held sample (siteMeterHeld) from the first repeat, not only once +// the meter counts as stale. On a meter that sends a reading every 10 s, +// this waits for the next reading instead of acting on the old one again. +func (s *State) siteMeterRepeatsAfterMove(meter *telemetry.DerReading) bool { + f := &s.meterFreeze + return meter != nil && f.driver == s.SiteMeterDriver && f.repeatAfterMove && meter.UpdatedAt.Equal(f.at) +} + +type ownDeviceKey struct { + t telemetry.DerType + name string +} + +// ownDevice is one battery or charger: FTW's last order to it and its own +// latest reading. +type ownDevice struct { + orderW, readW float64 +} + +// ownDevices are every battery and charger that reports a reading. One FTW +// has no order for counts as ordered to 0 W: a battery before FTW's first +// order after a start, a charger with no offer (unplugged, or its offer not +// yet known). So a meter frozen from the start, or a car plugged in during +// a freeze, is judged like any other move. +func (s *State) ownDevices(store *telemetry.Store) map[ownDeviceKey]ownDevice { + out := make(map[ownDeviceKey]ownDevice) + for driver, w := range store.OnlineRawWByType(telemetry.DerBattery) { + out[ownDeviceKey{telemetry.DerBattery, driver}] = ownDevice{orderW: s.PrevTargets[driver], readW: w} + } + for driver, w := range store.OnlineRawWByType(telemetry.DerEV) { + out[ownDeviceKey{telemetry.DerEV, driver}] = ownDevice{orderW: s.evCommanded[driver], readW: w} + } + return out +} + +// devicesMovedW is the net power FTW moved its devices by between a and b, +// in the site's sign, which is what the meter should show. Each device +// counts the smaller of its order's move and its reading's move, in the +// reading's direction, so only an order the device followed counts. A +// device that moved against its order, or was absent from a, counts 0. +func devicesMovedW(a, b map[ownDeviceKey]ownDevice) float64 { + moved := 0.0 + for k, now := range b { + then, ok := a[k] + if !ok { + continue + } + order, read := now.orderW-then.orderW, now.readW-then.readW + if order*read <= 0 { + continue + } + moved += math.Copysign(min(math.Abs(order), math.Abs(read)), read) + } + return moved +} + +func readMeterSample(w float64, data json.RawMessage) meterSample { + var d struct { + L1A *float64 `json:"l1_a"` + L2A *float64 `json:"l2_a"` + L3A *float64 `json:"l3_a"` + L1W *float64 `json:"l1_w"` + L2W *float64 `json:"l2_w"` + L3W *float64 `json:"l3_w"` + L1V *float64 `json:"l1_v"` + L2V *float64 `json:"l2_v"` + L3V *float64 `json:"l3_v"` + } + if len(data) > 0 { + _ = json.Unmarshal(data, &d) + } + var out meterSample + out.vals[0], out.has[0] = w, true + for i, p := range [...]*float64{d.L1A, d.L2A, d.L3A, d.L1W, d.L2W, d.L3W, d.L1V, d.L2V, d.L3V} { + if p != nil { + out.vals[i+1], out.has[i+1] = *p, true + } + } + return out +} diff --git a/go/internal/control/meter_freeze_test.go b/go/internal/control/meter_freeze_test.go new file mode 100644 index 000000000..6ed8a0ca4 --- /dev/null +++ b/go/internal/control/meter_freeze_test.go @@ -0,0 +1,448 @@ +package control + +import ( + "encoding/json" + "math" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// frozenRigMeter is the sample the rig's site meter repeated in seed 207537. +var frozenRigMeter = json.RawMessage(`{"w":2667,"l1_a":2.4,"l2_a":5.5,"l3_a":-0.2,` + + `"l1_w":552,"l2_w":1265,"l3_w":-46,"l1_v":230,"l2_v":230,"l3_v":230}`) + +func freezeState(clk *time.Time) *State { + st := NewState(0, 50, "meter") + st.clock = func() time.Time { return *clk } + return st +} + +// setBattery sends an order and lets the battery follow it. +func setBattery(st *State, store *telemetry.Store, w float64) { + st.PrevTargets["bat"] = w + store.Update("bat", telemetry.DerBattery, w, ptrF64(0.5), nil) +} + +// freezeStore is a store whose test drivers all report: the check reads +// telemetry through health-gated accessors. +func freezeStore() *telemetry.Store { + store := telemetry.NewStore() + for _, name := range []string{"meter", "bat", "own", "charger", "a", "b"} { + store.EnsureDriverHealth(name) + } + return store +} + +// Seed 207537: the meter answers every poll with one sample while the +// battery PI winds on it. The freeze must count as stale well before the +// rig's EV rise went out (38 s into the freeze), and the battery must stop +// moving once it does. +func TestFrozenMeterWithMovingOrdersIsStale(t *testing.T) { + store := freezeStore() + soc := 0.5 + store.Update("bat", telemetry.DerBattery, 4075, &soc, nil) + store.DriverHealthMut("bat").RecordSuccess() + + clk := time.Now() + st := freezeState(&clk) + st.Mode = ModeSelfConsumption + st.MinDispatchIntervalS = 0 + st.DriverLimits = map[string]PowerLimits{"bat": {MaxChargeW: 5000, MaxDischargeW: 5000}} + st.PrevTargets["bat"] = 4075 + capsMap := caps(map[string]float64{"bat": 10000}) + + frozenAfter := time.Duration(-1) + minW, maxW := 4075.0, 4075.0 + ordered := 4075.0 + start := clk + for clk.Sub(start) < 38*time.Second { + // The rig's battery follows an order about 2 s later: one tick. + store.Update("bat", telemetry.DerBattery, ordered, &soc, nil) + store.Update("meter", telemetry.DerMeter, 2667, nil, frozenRigMeter) + store.DriverHealthMut("meter").RecordSuccess() + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + if frozenAfter < 0 { + frozenAfter = clk.Sub(start) + } + } else { + // The gate is open: dispatch acts on the frozen meter. + for _, tg := range ComputeDispatch(store, st, capsMap, 3*16*230) { + if tg.Driver == "bat" { + ordered = tg.TargetW + minW, maxW = min(minW, tg.TargetW), max(maxW, tg.TargetW) + } + } + } + clk = clk.Add(2 * time.Second) + } + if frozenAfter < 0 { + t.Fatalf("meter repeated one sample for 38 s while battery orders moved %.0f..%.0f W; not stale", minW, maxW) + } + if frozenAfter >= 38*time.Second { + t.Fatalf("stale only after %v; the rig's EV rise went out at 38 s", frozenAfter) + } + t.Logf("stale after %v; battery orders %.0f..%.0f W", frozenAfter, minW, maxW) + // In the rig the battery swung from +5 kW to -5 kW before the verdict. + // Once the battery has moved past meterFreezeMoveW and the meter still + // repeats itself, dispatch holds it. + if maxW-minW > 2500 { + t.Fatalf("battery orders swung %.0f..%.0f W on a repeated sample before the verdict", minW, maxW) + } + + // Any change in the meter's values ends the freeze. + store.Update("meter", telemetry.DerMeter, 2654, nil, nil) + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + t.Fatal("a moving meter stayed stale") + } +} + +// Holding the battery on a repeat must not stall a live meter's loop. A +// 10 s HAN meter, polled on every 2 s tick, and the battery following an +// order one tick later: dispatch reaches the import target, does not +// overshoot into export by acting on one reading five times, and the meter +// never counts as stale. A noise-free meter that sends a new reading on +// every tick does the same. +func TestFrozenMeterHoldLetsLiveMeterLoopSettle(t *testing.T) { + // The 2 s case overshoots about 1.4 kW with or without the hold: the PI + // acting on a meter one tick late. Only the 10 s case is bounded here. + for _, tc := range []struct { + telegram time.Duration + maxExport float64 + }{{2 * time.Second, math.Inf(1)}, {10 * time.Second, 1000}} { + telegram := tc.telegram + store := freezeStore() + soc := 0.5 + clk := time.Now() + st := freezeState(&clk) + st.Mode = ModeSelfConsumption + st.MinDispatchIntervalS = 0 + st.DriverLimits = map[string]PowerLimits{"bat": {MaxChargeW: 5000, MaxDischargeW: 5000}} + capsMap := caps(map[string]float64{"bat": 10000}) + start := clk + houseW, ordered, batW, gridW := 3000.0, 0.0, 0.0, 3000.0 + for clk.Sub(start) < 3*time.Minute { + if clk.Sub(start)%telegram == 0 { + gridW = houseW + batW // the meter shows the battery as it is now + if -gridW > tc.maxExport { + t.Fatalf("%v telegrams: the battery overshot to %.0f W export at %v", telegram, -gridW, clk.Sub(start)) + } + } + batW = ordered // the battery takes the last order + store.Update("bat", telemetry.DerBattery, batW, &soc, nil) + store.Update("meter", telemetry.DerMeter, gridW, nil, nil) + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + t.Fatalf("%v telegrams: live meter marked stale at %v", telegram, clk.Sub(start)) + } + for _, tg := range ComputeDispatch(store, st, capsMap, 3*16*230) { + if tg.Driver == "bat" { + ordered = tg.TargetW + } + } + clk = clk.Add(2 * time.Second) + } + if err := houseW + batW; err > 300 || err < -300 { + t.Errorf("%v telegrams: the site still imports %.0f W after 3 min (battery %.0f W)", telegram, err, batW) + } + } +} + +// A quiet site whose devices do not move keeps a meter that repeats itself. +func TestFrozenMeterQuietSiteStaysFresh(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) + for i := 0; i < 150; i++ { // 5 min at 2 s + setBattery(st, store, -300+float64(i%2)*400) // small PI moves under the threshold + store.Update("meter", telemetry.DerMeter, 12, nil, frozenRigMeter) + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + t.Fatalf("quiet site marked stale after %d identical samples", i+1) + } + clk = clk.Add(2 * time.Second) + } +} + +// A HAN or DSMR 4 meter sends a new reading every 10 s; a driver polling it +// every second repeats each reading ten times. Large battery moves inside +// that window must not mark it stale, or it would flap. +func TestFrozenMeterTenSecondTelegramsDoNotFlap(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) + for s := 0; s < 300; s++ { + setBattery(st, store, float64((s%4)-2)*2000) // the battery swings +-4 kW + gridW := 800 + float64((s/10)%7) // one new value per 10 s telegram + store.Update("meter", telemetry.DerMeter, gridW, nil, nil) + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + t.Fatalf("10 s telegram meter marked stale at %d s", s) + } + clk = clk.Add(time.Second) + } +} + +// Orders a device does not follow do not count: a car that does not draw a +// raised offer (a slow charger, or a full car), a full battery that stays +// at 0 W, and a battery that charges on its own against an order to +// discharge. The rig's meter has no noise, so it repeats itself here. +func TestFrozenMeterOrdersNotFollowedDoNotCount(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) + st.PrevTargets["bat"] = 0 + store.Update("bat", telemetry.DerBattery, 0, ptrF64(1), nil) + store.Update("own", telemetry.DerBattery, 0, ptrF64(0.1), nil) + store.Update("charger", telemetry.DerEV, 0, nil, nil) + for i := 0; i < 60; i++ { // 2 min at 2 s + st.PrevTargets["bat"] = 3000 * float64(i%2) + st.evCommanded = map[string]float64{"charger": 11040} + if i == 1 { + st.PrevTargets["own"] = -3000 + store.Update("own", telemetry.DerBattery, 1000, ptrF64(0.1), nil) + } + store.Update("meter", telemetry.DerMeter, -2332, nil, nil) + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + t.Fatalf("orders the devices ignored marked the meter stale after %d samples", i+1) + } + clk = clk.Add(2 * time.Second) + } +} + +// The verdict waits for the slowest legitimate refresh after the move, and +// for several samples, before it calls the meter stale. +func TestFrozenMeterWaitsForTheMoveToLand(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) + poll := func() bool { + store.Update("meter", telemetry.DerMeter, 500, nil, nil) + st.ObserveSiteMeterFreeze(store, clk) + return st.SiteMeterFrozen() + } + setBattery(st, store, 0) + poll() + setBattery(st, store, -2000) + poll() // the move is seen here + moved := clk + for clk.Sub(moved) < meterFreezeAfter-time.Second { + clk = clk.Add(time.Second) + if poll() { + t.Fatalf("stale %v after the move", clk.Sub(moved)) + } + } + clk = clk.Add(time.Second) + if !poll() { + t.Fatalf("not stale %v after a 2 kW move", clk.Sub(moved)) + } +} + +// A 10 s HAN or DSMR 4 meter behind a bridge polled every second that loses +// one telegram (a CRC error) serves the old values for 20 s. A battery move +// right after the last good telegram must not mark this live meter stale. +func TestFrozenMeterLostTelegramDoesNotTrip(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) + tripped := -1 + gridW := 800.0 + for s := 0; s < 60; s++ { + // the battery steps 1 kW further one second after each telegram + if s%10 == 1 { + setBattery(st, store, -float64(s/10)*1000) + } + // telegrams every 10 s, except the one at s=20 is lost + if s%10 == 0 && s != 20 { + gridW++ + } + store.Update("meter", telemetry.DerMeter, gridW, nil, nil) + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() && tripped < 0 { + tripped = s + } + clk = clk.Add(time.Second) + } + if tripped >= 0 { + t.Errorf("live 10 s meter with one lost telegram marked stale at %d s", tripped) + } +} + +// Two batteries trade 1.5 kW each way. The meter sees only the sum, which +// does not change, so the repeat is no evidence. +func TestFrozenMeterOppositeMovesCancel(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) + set := func(name string, w float64) { + st.PrevTargets[name] = w + store.Update(name, telemetry.DerBattery, w, ptrF64(0.5), nil) + } + set("a", 2000) + set("b", 0) + for s := 0; s < 60; s++ { + if s == 2 { + set("a", 500) + set("b", 1500) + } + store.Update("meter", telemetry.DerMeter, 300, nil, nil) // net unchanged + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + t.Errorf("opposite moves that cancel at the meter marked it stale at %d s", s) + return + } + clk = clk.Add(time.Second) + } +} + +// A car plugged in during a freeze, at a site with no battery: its charger +// had no offer when the meter last changed. FTW's raise, which the car +// draws, must still count, or an EV rise on a frozen meter goes unseen. +func TestFrozenMeterNewlyPluggedEVCounts(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) + store.Update("charger", telemetry.DerEV, 0, nil, nil) + for s := 0; s < 60; s++ { + switch { + case s == 5: // car plugged: first offer, 0 W + st.evCommanded = map[string]float64{"charger": 0} + case s >= 10: // FTW raises to 6 A x 3 and the car draws it + st.evCommanded = map[string]float64{"charger": 4140} + store.Update("charger", telemetry.DerEV, 4140, nil, nil) + } + store.Update("meter", telemetry.DerMeter, 2667, nil, frozenRigMeter) + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + return + } + clk = clk.Add(time.Second) + } + t.Error("frozen meter while FTW raised a newly plugged EV 4.1 kW for 50 s: never stale") +} + +// A battery that moves 2 kW and back before the meter's next telegram +// leaves nothing for the meter to show, so a repeat is no evidence. +func TestFrozenMeterMoveAndBackDoesNotCount(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) + setBattery(st, store, 0) + for s := 0; s < 90; s++ { + switch s { + case 1: + setBattery(st, store, 2000) + case 5: + setBattery(st, store, 0) + } + store.Update("meter", telemetry.DerMeter, 300, nil, nil) + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + t.Fatalf("a move that came back marked the meter stale at %d s", s) + } + clk = clk.Add(time.Second) + } +} + +// A meter already frozen when FTW starts: FTW has no battery orders yet +// when the meter's only sample is taken. Winding the battery 3 kW on it must +// still count. +func TestFrozenMeterFromStartIsCaught(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) // PrevTargets empty, as after a restart + store.Update("bat", telemetry.DerBattery, 0, ptrF64(0.5), nil) + for s := 0; s < 60; s++ { + if s >= 3 { + setBattery(st, store, -3000) + } + store.Update("meter", telemetry.DerMeter, 2667, nil, frozenRigMeter) + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + return + } + clk = clk.Add(time.Second) + } + t.Error("meter frozen from boot while FTW wound the battery to -3 kW: never stale") +} + +// After a start, FTW's first order to a battery that already runs on its +// own counts only by how far the battery's reading moves. Ordering what it +// already does is no move. +func TestFrozenMeterFirstOrderMatchingAutonomousBatteryDoesNotCount(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) + store.Update("bat", telemetry.DerBattery, 1500, ptrF64(0.5), nil) + for s := 0; s < 60; s++ { + if s >= 3 { + setBattery(st, store, 1500) + } + store.Update("meter", telemetry.DerMeter, 2667, nil, frozenRigMeter) + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + t.Fatalf("first order matching the battery's own 1500 W marked the meter stale at %d s", s) + } + clk = clk.Add(time.Second) + } +} + +// A slow poller (one answer every 20 s) after a 3 kW move: the verdict +// needs three repeated answers after the move, not just 30 s. +func TestFrozenMeterSlowPollerNeedsThreeRepeats(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) + start := clk + poll := func(s int) bool { + store.Update("meter", telemetry.DerMeter, 900, nil, nil) + st.ObserveSiteMeterFreeze(store, start.Add(time.Duration(s)*time.Second)) + return st.SiteMeterFrozen() + } + setBattery(st, store, 0) + poll(0) + setBattery(st, store, -3000) + poll(20) // the move is seen + if poll(40) { + t.Fatal("stale after one repeat") + } + if poll(60) { + t.Fatal("stale after two repeats, 40 s after the move") + } + if !poll(80) { + t.Fatal("not stale after three repeats, 60 s after a 3 kW move") + } +} + +// The rig's OCPP charger accepts a new limit at once, and the car draws it +// about 30 s later (seed 207537: offer at 157 s, draw at 188 s). A live +// meter with no noise repeats itself until then. Counting the accepted +// offer as a move would mark this meter stale, so an EV order counts only +// once the charger's reading shows the car drawing it. +func TestFrozenMeterAcceptedOfferWaitsForTheCar(t *testing.T) { + store := freezeStore() + clk := time.Now() + st := freezeState(&clk) + store.Update("charger", telemetry.DerEV, 0, nil, nil) + st.evCommanded = map[string]float64{"charger": 0} + gridW := 208.0 + for s := 0; s < 120; s++ { + if s == 1 { // FTW raises to 6 A x 3; the charger accepts at once + st.evCommanded = map[string]float64{"charger": 4140} + } + if s == 32 { // the car draws; the meter and the charger show it + gridW += 4140 + store.Update("charger", telemetry.DerEV, 4140, nil, nil) + } + store.Update("meter", telemetry.DerMeter, gridW, nil, nil) + st.ObserveSiteMeterFreeze(store, clk) + if st.SiteMeterFrozen() { + t.Fatalf("live meter marked stale at %d s; the car started 31 s after the offer", s) + } + clk = clk.Add(time.Second) + } +} diff --git a/go/internal/control/no_export_band_test.go b/go/internal/control/no_export_band_test.go new file mode 100644 index 000000000..335d8e47d --- /dev/null +++ b/go/internal/control/no_export_band_test.go @@ -0,0 +1,161 @@ +package control + +import ( + "math/rand" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// coverLoadSite is a site whose only battery covers the house load in a +// discharge slot planned to land the grid near 0 W, with no PV. +func coverLoadSite(mode Mode, toleranceW, gridW, batteryW float64) (*telemetry.Store, *State) { + now := time.Now() + dir := SlotDirective{ + SlotStart: now, SlotEnd: now.Add(15 * time.Minute), + BatteryEnergyWh: -600, PlannedGridW: 0, HasPlannedGridW: true, + } + store := seedStore(gridW, []struct { + name string + currentW, soc float64 + }{{"ferroamp", batteryW, 0.6}}) + st := NewState(0, toleranceW, "ferroamp") + st.Mode = mode + st.UseEnergyDispatch = true + st.SlewRateW = 100_000 + st.MinDispatchIntervalS = 0 + st.SlotDirective = func(time.Time) (SlotDirective, bool) { return dir, true } + return store, st +} + +func batteryTarget(t *testing.T, store *telemetry.Store, st *State, holdW float64) float64 { + t.Helper() + targets := ComputeDispatch(store, st, caps(map[string]float64{"ferroamp": 20000}), 40000) + if len(targets) == 0 { + return holdW // the dead band keeps the current command + } + if len(targets) != 1 { + t.Fatalf("want 1 target, got %+v", targets) + } + return targets[0].TargetW +} + +// A field report on passive arbitrage: the battery covered 2.4 kW of house +// load and the meter showed 100–270 W of export about half the time. A +// battery that may not export now trims its discharge until the site +// imports, and leaves a small import alone. +func TestNoExportDischargeHoldsTheBandAboveZero(t *testing.T) { + for _, tc := range []struct { + name string + mode Mode + gridW float64 + wantUp bool // the discharge is trimmed + }{ + {"passive, report snapshot", ModePlannerPassiveArbitrage, -164, true}, + {"passive, export inside the old band", ModePlannerPassiveArbitrage, -20, true}, + {"passive, small import", ModePlannerPassiveArbitrage, 30, false}, + {"planner_self, export inside the old band", ModePlannerSelf, -20, true}, + {"arbitrage may export, band unchanged", ModePlannerArbitrage, -20, false}, + } { + t.Run(tc.name, func(t *testing.T) { + const batteryW = -2571.0 + store, st := coverLoadSite(tc.mode, 42, tc.gridW, batteryW) + load, battery := tc.gridW-batteryW, batteryW + for range 20 { // the house load holds still + grid, soc := load+battery, 0.6 + store.Update("ferroamp", telemetry.DerMeter, grid, nil, nil) + store.Update("ferroamp", telemetry.DerBattery, battery, &soc, nil) + battery = batteryTarget(t, store, st, battery) + if !tc.wantUp && battery != batteryW { + t.Fatalf("target %.0f W, want the %.0f W discharge kept", battery, batteryW) + } + } + if grid := load + battery; tc.wantUp && (grid < 0 || grid > 2*42) { + t.Fatalf("settled at %.0f W discharge with the grid at %.0f W, want 0–84 W import", battery, grid) + } + }) + } +} + +// The legacy plan path can carry a negative grid target, meant for PV +// export. An idle battery under such a plan must not discharge toward it, +// not even on the first tick. +func TestNoExportIgnoresANegativePlannedGridForTheBattery(t *testing.T) { + for _, mode := range []Mode{ModePlannerCheap, ModePlannerPassiveArbitrage} { + t.Run(string(mode), func(t *testing.T) { + store, st := coverLoadSite(mode, 42, 800, 0) + st.SlotDirective = nil + st.PlanTarget = func(time.Time) (string, float64, string, bool) { + return string(ModeSelfConsumption), -1365, "plan", true + } + load, battery := 800.0, 0.0 + for i := range 10 { + grid, soc := load+battery, 0.6 + store.Update("ferroamp", telemetry.DerMeter, grid, nil, nil) + store.Update("ferroamp", telemetry.DerBattery, battery, &soc, nil) + battery = batteryTarget(t, store, st, battery) + if grid := load + battery; grid < 0 { + t.Fatalf("tick %d: discharging %.0f W exports %.0f W", i, -battery, -grid) + } + } + if battery > -700 { + t.Fatalf("settled at %.0f W, want the 800 W load covered", battery) + } + }) + } +} + +// Surplus that appears while the battery discharges is charged at once. +// The bias may not turn that into grid import beyond the dead band. +func TestNoExportBiasChargesSurplusWithoutImport(t *testing.T) { + store, st := coverLoadSite(ModePlannerSelf, 42, -1500, -100) + load, battery := -1500.0+100, -100.0 + for i := range 20 { + grid, soc := load+battery, 0.6 + store.Update("ferroamp", telemetry.DerMeter, grid, nil, nil) + store.Update("ferroamp", telemetry.DerBattery, battery, &soc, nil) + battery = batteryTarget(t, store, st, battery) + if grid := load + battery; grid > 42 { + t.Fatalf("tick %d: charging %.0f W imports %.0f W", i, battery, grid) + } + } + if battery < 1300 { + t.Fatalf("settled at %.0f W, want the 1.4 kW surplus charged", battery) + } +} + +// Closed loop: a battery that follows its command within one tick covers a +// noisy house load with occasional steps. Aimed at 0 W, the site exported +// about as much as it imported; held above 0 W, it exports far less. +func TestNoExportCoverLoadExportsLessThanItImports(t *testing.T) { + rng := rand.New(rand.NewSource(1)) + store, st := coverLoadSite(ModePlannerPassiveArbitrage, 42, 0, -2400) + load, battery := 2400.0, -2400.0 + var exportTicks int + var exportWh, importWh float64 + const ticks, tickS = 3000, 5.0 + for range ticks { + load += rng.NormFloat64() * 60 + if rng.Float64() < 0.01 { + load += (rng.Float64() - 0.5) * 1500 + } + load = min(max(load, 600), 4500) + grid, soc := load+battery, 0.6 + store.Update("ferroamp", telemetry.DerMeter, grid, nil, nil) + store.Update("ferroamp", telemetry.DerBattery, battery, &soc, nil) + battery = batteryTarget(t, store, st, battery) + if grid < -50 { + exportTicks++ + } + if grid < 0 { + exportWh -= grid * tickS / 3600 + } else { + importWh += grid * tickS / 3600 + } + } + t.Logf("export below -50 W in %d of %d ticks; exported %.0f Wh, imported %.0f Wh", exportTicks, ticks, exportWh, importWh) + if exportWh > importWh/2 || exportTicks > ticks*15/100 { + t.Fatalf("exported %.0f Wh against %.0f Wh imported, %d of %d ticks below -50 W", exportWh, importWh, exportTicks, ticks) + } +} diff --git a/go/internal/control/peak_limit_test.go b/go/internal/control/peak_limit_test.go index 596212a4e..b1c7f7861 100644 --- a/go/internal/control/peak_limit_test.go +++ b/go/internal/control/peak_limit_test.go @@ -7,8 +7,8 @@ import ( ) // SetPeakLimit is the one place the peak-shaving threshold is validated -// against the site's fuse. A 16 A / 230 V / 3-phase site is 11040 W; the -// default 0.5 A safety margin costs 345 W, so the highest limit that can +// against the site's fuse. A 16 A / 230 V / 3-phase site is 11040 W; an +// opt-in 0.5 A safety margin costs 345 W, so the highest limit that can // still be the first thing to bind is 10695 W. func fusedState() *State { st := NewState(0, 50, "ferroamp") @@ -63,6 +63,23 @@ func TestSetPeakLimitRejectsValueInsideTheSafetyMargin(t *testing.T) { } } +// The message names the margin only when the site has one. With the +// default 0 the ceiling is the fuse itself. +func TestSetPeakLimitMessageNamesOnlyARealMargin(t *testing.T) { + st := fusedState() + if err := st.SetPeakLimit(20000); err == nil || !strings.Contains(err.Error(), "less a 0.5 A safety margin") { + t.Fatalf("with a 0.5 A margin the message must name it, got %v", err) + } + st.SiteFuseSafetyA = 0 + err := st.SetPeakLimit(20000) + if err == nil || strings.Contains(err.Error(), "margin") || !strings.Contains(err.Error(), "11040 W") { + t.Fatalf("with no margin the message must give the fuse as the ceiling and no margin, got %v", err) + } + if err := st.SetPeakLimit(11040); err != nil { + t.Fatalf("with no margin the fuse itself is the ceiling, got %v", err) + } +} + // Zero is a real threshold in peak shaving — "correct everything above // 0 W of import" — not the zero-means-disabled convention that // PeakImportCeilingW and MaxExportW use. Dispatch has always read it that @@ -122,7 +139,7 @@ func TestSetPeakLimitRejectsNonFinite(t *testing.T) { } // An undescribed fuse yields no ceiling rather than an invented one — -// the same back-compat rule fuseSafetyMarginW and perPhaseOverageW keep +// the same back-compat rule fuseSafetyMarginW and phaseFuseReliefW keep // for harnesses that wire only some of the fuse fields. func TestSetPeakLimitWithoutFuseChecksOnlyTheSign(t *testing.T) { st := NewState(0, 50, "ferroamp") // no SiteFuse* wired diff --git a/go/internal/control/sitesim_field_test.go b/go/internal/control/sitesim_field_test.go index 5b0c8d30a..094df99be 100644 --- a/go/internal/control/sitesim_field_test.go +++ b/go/internal/control/sitesim_field_test.go @@ -305,6 +305,20 @@ func probeCases() []fieldCase { c.carPhases = 1 out = append(out, c) } + for _, fuse := range []float64{16, 25} { + for _, ms := range []struct{ mode, steps string }{{"", "mixed"}, {"auto", "mixed"}, {"3p", "mixed"}, {"", "3p"}, {"auto", "3p"}} { + // The same car on a table with three-phase levels: the reserve + // orders one of them, as the plan does. + out = append(out, fieldCase{report: "probe", fuseA: fuse, mode: ms.mode, steps: ms.steps, timing: timingEasee, carPhases: 1, use: useNoPlan, + house: houseBalanced, houseName: "balanced"}) + } + } + for _, steps := range []string{"mixed", "3p"} { + // A 2.3 kW one-phase car, below the charger's three-phase levels + // over three: marked, it is ordered three times its own level. + out = append(out, fieldCase{report: "car2300W", fuseA: 25, steps: steps, timing: timingFast, carPhases: 1, use: useNoPlan, + house: houseBalanced, houseName: "balanced", car: func(car *sitesim.Car) { car.MaxW = 2300 }}) + } for _, use := range []fieldUse{usePlanned, useNow} { // The charger in srcfl/ftw#1763's trace: Easee's own load balancing // keeps each phase under 18 A after the house, so the car gets 12 A @@ -459,7 +473,7 @@ func (s *simSite) coreEvery() time.Duration { func (s *simSite) declaredMaxAge() time.Duration { if s.field == nil { - return 0 + return s.evMaxAge } return s.field.c.timing.declared } @@ -467,6 +481,9 @@ func (s *simSite) declaredMaxAge() time.Duration { // evData is what a charger driver adds to its reading. func (s *simSite) evData() map[string]any { if s.field == nil { + if s.evMaxAge > 0 { + return map[string]any{"power_max_age_s": s.evMaxAge.Seconds()} + } return nil } d := map[string]any{"connected": s.ev.Plugged, "control_power_confirmed": true} @@ -516,7 +533,7 @@ func (f *fieldRun) harvest(s *simSite, kind string) { } } -func (f *fieldRun) limitA(s *simSite) float64 { return s.p.Fuse.RatingA - evPhaseMarginA } +func (f *fieldRun) limitA(s *simSite) float64 { return s.state.fuseLineA() } // send records the command as the driver registry does and checks a cut. func (f *fieldRun) send(ctx context.Context, s *simSite, driver string, payload []byte) error { @@ -569,7 +586,7 @@ func (f *fieldRun) checkCut(s *simSite, w float64) { over = over || math.Abs(a) >= limit || s.heat[i] >= limit*limit } if !over { - f.fail(s, failCut, "cut the car %.0f -> %.0f W for the fuse with every phase under %.0f A (meter %.1f A)", + f.fail(s, failCut, "cut the car %.0f -> %.0f W for the fuse with every phase under %.1f A (meter %.1f A)", s.offerW, w, limit, s.meter.PhaseA) return } @@ -577,6 +594,11 @@ func (f *fieldRun) checkCut(s *simSite, w float64) { if !queued || e.DrawW <= 0 { return } + // Past the current a gG fuse carries for an hour, no cut on its way + // holds back the next one (fuseHourFactor). + if hourA := fuseHourFactor * s.p.Fuse.RatingA; max(s.heat[0], s.heat[1], s.heat[2]) > hourA*hourA { + return + } drawA := e.DrawW / s.p.Voltage / float64(e.DrawPhases) carA := s.p.Car.MaxW / s.p.Voltage / float64(min(max(queuedPhases, 1), e.Phases, s.p.Car.Phases)) covered := true @@ -594,7 +616,7 @@ func (f *fieldRun) checkCut(s *simSite, w float64) { } } if covered { - f.fail(s, failOverCut, "cut the car %.0f -> %.0f W while a cut to %.0f A it has not applied covers the overage (meter %.1f A, limit %.0f A)", + f.fail(s, failOverCut, "cut the car %.0f -> %.0f W while a cut to %.0f A it has not applied covers the overage (meter %.1f A, limit %.1f A)", s.offerW, w, queuedA, s.meter.PhaseA, limit) } } @@ -612,7 +634,7 @@ func (f *fieldRun) wantsCharge(s *simSite, baseW float64) bool { if level > 0 { level = math.Max(level, math.Sqrt(s.heat[i])) } - room = room && level+p.EVSE.MinA+evPhaseMarginA <= limit + room = room && level+p.EVSE.MinA <= limit+1e-9 } switch { case !room: @@ -722,7 +744,7 @@ func fieldRoomW(c fieldCase, e *sitesim.EVSE, car *sitesim.Car) float64 { for i := range phases { worst = math.Max(worst, c.house[i]) } - roomA := math.Min(c.fuseA-evPhaseMarginA-worst, e.MaxA) + roomA := math.Min(c.fuseA-simFuseSafetyA-worst, e.MaxA) if e.LoadBalancerA > 0 { roomA = math.Min(roomA, e.LoadBalancerA-worst) } @@ -803,7 +825,8 @@ func runFieldCase(c fieldCase, trace func(string, ...any)) (*simSite, *fieldRun) s.plan = f.plan.directive case useNoPlan: run, kettle = time.Hour, []time.Duration{20 * time.Minute} - svc := &mpc.Service{LoadpointStates: func() []loadpoint.State { return s.mgr.States() }} + svc := &mpc.Service{LoadpointStates: func() []loadpoint.State { return s.mgr.States() }, + PlanningLimits: func(st loadpoint.State) (float64, float64, []float64) { return s.ctrl.PlanningLimits(st) }} // as main.go wires it s.plan = func(now time.Time) (loadpoint.Directive, bool) { d, ok := svc.SlotDirectiveAt(now) return d.LoadpointDirective(), ok @@ -830,7 +853,7 @@ func runFieldCase(c fieldCase, trace func(string, ...any)) (*simSite, *fieldRun) // Export steps up and down, never past the fuse with the car off. houseW, maxPVW := p.HouseW[0]+p.HouseW[1]+p.HouseW[2], math.Inf(1) for _, a := range c.house { - maxPVW = math.Min(maxPVW, 3*230*(c.fuseA-2*evPhaseMarginA+a)) + maxPVW = math.Min(maxPVW, 3*230*(c.fuseA-simFuseSafetyA+a)) } exports := []float64{2500, 6000, 9000, 3000} if len(c.exportW) > 0 { diff --git a/go/internal/control/sitesim_import_plan_test.go b/go/internal/control/sitesim_import_plan_test.go new file mode 100644 index 000000000..56620890e --- /dev/null +++ b/go/internal/control/sitesim_import_plan_test.go @@ -0,0 +1,182 @@ +package control + +import ( + "context" + "fmt" + "math" + "path/filepath" + "strings" + "testing" + "testing/synctest" + "time" + + "github.com/srcfl/ftw/go/internal/loadpoint" + "github.com/srcfl/ftw/go/internal/mpc" + "github.com/srcfl/ftw/go/internal/sitesim" + "github.com/srcfl/ftw/go/internal/state" +) + +// importPlanCase is the 25 October 2026 field case run closed-loop: an +// hourly import ceiling, 0.8 kW of house, a 9.6 kWh battery and a car that +// wants 19 kWh more by the deadline on a 3 × 20 A fuse. Core's planner +// replans every quarter from Core's own import hour and Core runs the plan. +type importPlanCase struct { + start, deadline time.Time + ceilingW, soc float64 + restartAt time.Duration // > 0: Core restarts then, with no record of the hour +} + +type importPlanHour struct { + start time.Time + plannedWh, metered float64 +} + +// runImportPlan returns each clock hour's import as the plan made at its +// start meant it and as the meter measured it, and the car's energy by the +// deadline as the first plan promised it and as delivered. It runs inside a +// synctest bubble. +func runImportPlan(t *testing.T, c importPlanCase) (hours []importPlanHour, promisedWh, deliveredWh float64) { + t.Helper() + zone := c.start.Location() + time.Sleep(c.start.Sub(time.Now())) // The bubble's clock becomes the plant's. + st, err := state.Open(filepath.Join(t.TempDir(), "state.db")) + if err != nil { + t.Fatal(err) + } + defer st.Close() + prices := make([]state.PricePoint, 32) + for i := range prices { + prices[i] = state.PricePoint{Zone: "SE3", SlotTsMs: c.start.Add(time.Duration(i) * 15 * time.Minute).UnixMilli(), SlotLenMin: 15, + SpotOreKwh: 40 + float64(i%4), TotalOreKwh: 90 + float64(i%4), Source: "test", FetchedAtMs: c.start.UnixMilli()} + } + if err := st.SavePrices(prices); err != nil { + t.Fatal(err) + } + svc := mpc.New(st, nil, "SE3", mpc.Params{Mode: mpc.ModeArbitrage, SoCLevels: 21, ActionLevels: 21, + CapacityWh: 9600, InitialSoC: c.soc, SoCMin: .1, SoCMax: .95, MaxChargeW: 5000, MaxDischargeW: 5000, + ChargeEfficiency: .95, DischargeEfficiency: .95}) + svc.Horizon, svc.BaseLoad = 8*time.Hour, 800 + svc.UpdateBatteryFleet([]mpc.BatteryFleetMember{{Driver: "battery", CapacityWh: 9600, MaxChargeW: 5000, MaxDischargeW: 5000}}, 9600, 5000, 5000) + svc.UpdateSiteEconomics(mpc.SiteEconomics{FuseMaxW: 13800, Timezone: zone.String()}) + + p := sitesim.New(c.start, 20) + p.EVSE = &sitesim.EVSE{Phases: 3, MinA: 6, MaxA: 16, Delay: 5 * time.Second, ReadbackLag: 10 * time.Second} + p.Car = &sitesim.Car{Phases: 3, MaxW: 11040, Plugged: true} + p.Battery = &sitesim.Battery{Phases: 3, MaxChargeW: 5000, MaxDischargeW: 5000, CapacityWh: 9600, SoC: c.soc, Latency: 2 * time.Second} + p.HouseW = [3]float64{800. / 3, 800. / 3, 800. / 3} + if c.restartAt > 0 { + p.Events = []sitesim.Event{{At: c.restartAt, Kind: sitesim.CoreRestart, For: 20 * time.Second}} + } + + const carWh, needWh = 75000., 19000. + s := &simSite{p: p, lp: simLoadpoint()} + s.plan = func(now time.Time) (loadpoint.Directive, bool) { + d, ok := svc.SlotDirectiveAt(now) + if !ok { + return loadpoint.Directive{}, false + } + return d.LoadpointDirective(), true + } + s.setup = func(s *simSite) { + s.state.PeakImportCeilingW, s.state.importHourZone = c.ceilingW, zone + s.state.Mode, s.state.UseEnergyDispatch = ModePlannerArbitrage, true + s.state.SlotDirective = func(now time.Time) (SlotDirective, bool) { + d, ok := svc.SlotDirectiveAt(now) + if !ok { + return SlotDirective{}, false + } + return SlotDirectiveFromMPC(d), true + } + s.mgr.SetSchedule(s.lp.ID, loadpoint.Schedule{SoC: .8}) + } + s.boot() + // The planner reads the hour the live limit counts, as main.go wires it. + svc.ImportCeiling = func() mpc.ImportCeiling { return PlanningImportCeiling(s.state) } + svc.Loadpoint = func(int) *mpc.LoadpointSpec { + first := s.p.Now.Truncate(15 * time.Minute) + return &mpc.LoadpointSpec{ID: s.lp.ID, CapacityWh: carWh, InitialSoC: (.8*carWh - needWh + .9*s.p.EVSE.SessionWh) / carWh, SoCMax: 1, + Levels: 21, PluggedIn: true, TargetSoC: .8, TargetSlotIdx: int(c.deadline.Sub(first)/(15*time.Minute)) - 1, + MinChargeW: s.lp.MinChargeW, MaxChargeW: s.lp.MaxChargeW, AllowedStepsW: s.lp.AllowedStepsW, ChargeEfficiency: .9} + } + plannedWh := func(plan *mpc.Plan, from time.Time) float64 { + var wh float64 + for _, a := range plan.Actions { + begin, end := max(a.ExecutionStart(), from.UnixMilli()), min(a.SlotStartMs+int64(a.SlotLenMin)*60_000, from.Add(time.Hour).UnixMilli()) + if end > begin { + wh += math.Max(0, a.GridW) * float64(end-begin) / 3.6e6 + } + } + return wh + } + + nextReplan := c.start.Add(10 * time.Second) // Telemetry first. + for s.p.Now.Before(c.deadline) { + time.Sleep(sitesim.Step) + s.step() + now := s.p.Now + if s.downUntil.IsZero() && !now.Before(nextReplan) { + svc.Tele = s.store + plan := svc.ReplanWithReason(context.Background(), "slot_elapsed") + if plan == nil { + t.Fatalf("%s: no plan: %+v", now.Format("15:04:05"), svc.PlanSnapshot()) + } + if promisedWh == 0 { + promisedWh = needWh - plan.LoadpointShortfallWh[s.lp.ID] + } + if h := importHourStart(now, zone); len(hours) == 0 || !h.Equal(hours[len(hours)-1].start) { + hours = append(hours, importPlanHour{start: h, plannedWh: plannedWh(plan, h)}) + } + nextReplan = now.Truncate(15 * time.Minute).Add(15*time.Minute + time.Second) + } + if len(hours) > 0 { + hours[len(hours)-1].metered += math.Max(0, (p.A[0]+p.A[1]+p.A[2])*p.Voltage) * sitesim.Step.Hours() + } + // The site check's own idea of when a car should charge does not + // know the plan; every other invariant holds. + if s.failure != "" && !strings.Contains(s.failure, "idle") { + t.Fatal(s.failure) + } + } + return hours, promisedWh, .9 * s.p.EVSE.SessionWh +} + +// Plan and execution agree: no clock hour imports more than the ceiling, and +// the car gets about what the first plan promised. With the battery covering +// the house the car runs from the hour's start and the promise holds within +// 3 %. Without it the car needs the hour's slack to start and Core restarts +// it each time a quarter's energy is spent: 8 %. +func TestSiteSimPlanUnderImportCeiling(t *testing.T) { + zone, err := time.LoadLocation("Europe/Stockholm") + if err != nil { + t.Fatal(err) + } + nine := time.Date(2026, 10, 25, 9, 0, 0, 0, zone) + night := time.Date(2026, 10, 24, 23, 30, 0, 0, time.UTC).In(zone) // 01:30 CEST, before 02:00 comes twice + for _, c := range []struct { + name string + c importPlanCase + promise float64 + }{ + {"battery covers the house", importPlanCase{start: nine, deadline: nine.Add(4 * time.Hour), ceilingW: 4500, soc: .5}, .03}, + {"battery empty", importPlanCase{start: nine, deadline: nine.Add(4 * time.Hour), ceilingW: 4500, soc: .1}, .08}, + {"the 25-hour day", importPlanCase{start: night, deadline: night.Add(4 * time.Hour), ceilingW: 4500, soc: .5}, .03}, + {"restart in the hour", importPlanCase{start: nine, deadline: nine.Add(2 * time.Hour), ceilingW: 4500, soc: .5, restartAt: 40 * time.Minute}, .05}, + } { + t.Run(c.name, func(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + hours, promised, delivered := runImportPlan(t, c.c) + var line string + for _, h := range hours { + line += fmt.Sprintf(" %s plan %.0f metered %.0f;", h.start.Format("15:04 MST"), h.plannedWh, h.metered) + if h.metered > c.c.ceilingW { + t.Errorf("%s: imported %.0f Wh, over the %.0f W ceiling", h.start.Format("15:04 MST"), h.metered, c.c.ceilingW) + } + } + t.Logf("car promised %.0f Wh, delivered %.0f Wh;%s", promised, delivered, line) + if math.Abs(delivered-promised) > c.promise*promised { + t.Errorf("delivered %.0f Wh against %.0f Wh promised, more than %.0f %% apart", delivered, promised, c.promise*100) + } + }) + }) + } +} diff --git a/go/internal/control/sitesim_property_test.go b/go/internal/control/sitesim_property_test.go index 057fa8c84..b498e485e 100644 --- a/go/internal/control/sitesim_property_test.go +++ b/go/internal/control/sitesim_property_test.go @@ -13,17 +13,16 @@ import ( // randomHouseKnown names the seeds that break an invariant on master, by // kind, and the finding. A listed kind that passes fails the test until its -// line goes. The fuse's own kind is never listed. Seed 2 no longer reaches -// the case where a landed cut still covers the next overage; that cost is -// pinned in TestEVFuseLandedCutCoversOnlyItsRun. +// line goes. The fuse's own kind is never listed. A landed cut covers no +// later overage (TestEVFuseLandedCutCoversOnlyItsRun). var randomHouseKnown = map[uint64]map[string]string{} // Core keeps a random house under the fuse. For each fixed seed the house -// load on each phase steps at random to anywhere up to the fuse less 2 A, +// load on each phase steps at random to anywhere up to the working limit, // with the fuse warm from the first load, while a car charges on Charge now // through a charger that answers in 2, 5 or 40 s. simSite.check holds the // fuse's invariants every plant second for an hour: the fuse never blows, -// and no phase stays over the fuse less its margin longer than the +// and no phase stays over the working limit longer than the // charger's response plus 3 s, or 3 s after a raise of Core's own. Idle // time with room is logged, not judged: the field matrix judges it with the // fuse's own memory. @@ -38,7 +37,7 @@ func TestSiteSimRandomHouse(t *testing.T) { house := func() [3]float64 { var a [3]float64 for i := range a { - a[i] = rng.Float64() * (p.Fuse.RatingA - 2) + a[i] = rng.Float64() * (p.Fuse.RatingA - simFuseSafetyA) // up to the line } return a } @@ -70,8 +69,8 @@ func TestSiteSimRandomHouse(t *testing.T) { steps++ } } - t.Logf("%.0f A fuse, charger %v, %d house steps: peak phase %.1f A, %v over %.0f A, fuse damage at most %.2f, car idle with room %v, %.1f kWh to the car", - p.Fuse.RatingA, p.EVSE.Delay, steps, s.peakA, s.overLimit, p.Fuse.RatingA-evPhaseMarginA, damage, s.idleWithRoom, p.EVSE.SessionWh/1000) + t.Logf("%.0f A fuse, charger %v, %d house steps: peak phase %.1f A, %v over %.1f A, fuse damage at most %.2f, car idle with room %v, %.1f kWh to the car", + p.Fuse.RatingA, p.EVSE.Delay, steps, s.peakA, s.overLimit, s.state.fuseLineA(), damage, s.idleWithRoom, p.EVSE.SessionWh/1000) delete(failures, failIdle) known := randomHouseKnown[seed] for _, k := range slices.Sorted(maps.Keys(failures)) { diff --git a/go/internal/control/sitesim_test.go b/go/internal/control/sitesim_test.go index 6167d172d..84316e2c0 100644 --- a/go/internal/control/sitesim_test.go +++ b/go/internal/control/sitesim_test.go @@ -20,6 +20,9 @@ import ( // simCoreEvery is control_interval_s's default. The plant steps every second. const simCoreEvery = 2 * time.Second +// simFuseSafetyA is the site's fuse.safety_margin_a, the config default. +const simFuseSafetyA = 0 + // simSite runs Core's control tick against a sitesim.Plant in main.go's // order: site import, charger tick, then battery dispatch. Core's settings // are the config defaults. Every plant second it checks the invariants and @@ -39,6 +42,20 @@ type simSite struct { every time.Duration // control_interval_s; zero: simCoreEvery field *fieldRun // the field matrix's own checks; nil outside it + // evMaxAge is the power_max_age_s the charger driver declares outside + // the field matrix; zero declares none. + evMaxAge time.Duration + // evEvery is how often the charger samples its draw, as OCPP + // MeterValues do: between samples each poll repeats the last sample + // with its own time. Zero samples on every poll. + evEvery time.Duration + // evClockLag is how far the charger's clock runs behind Core's: it + // stamps each sample that much early, as an OCPP charger stamps + // MeterValues with its own clock (or cuts the stamp to whole seconds). + evClockLag time.Duration + // meterNoise is added to each phase current Core reads, not to the + // plant's own; nil reads the plant exactly. + meterNoise func() float64 downUntil, nextTick time.Time meter sitesim.MeterReading @@ -83,7 +100,7 @@ func (s *simSite) boot() { s.state = NewState(0, 42, "meter") s.state.clock = func() time.Time { return s.p.Now } s.state.SlewRateW, s.state.MinDispatchIntervalS = 3000, 2 - s.state.SiteFuseAmps, s.state.SiteFuseVoltage, s.state.SiteFusePhases, s.state.SiteFuseSafetyA = s.p.Fuse.RatingA, s.p.Voltage, 3, .5 + s.state.SiteFuseAmps, s.state.SiteFuseVoltage, s.state.SiteFusePhases, s.state.SiteFuseSafetyA = s.p.Fuse.RatingA, s.p.Voltage, 3, simFuseSafetyA if b := s.p.Battery; b != nil { s.state.DriverLimits = map[string]PowerLimits{"battery": {MaxChargeW: b.MaxChargeW, MaxDischargeW: b.MaxDischargeW}} } @@ -215,15 +232,34 @@ func (s *simSite) step() { } if s.downUntil.IsZero() { if m := tick.Meter; m != nil { - s.emit("meter", telemetry.DerMeter, m.W, nil, map[string]any{"l1_a": m.PhaseA[0], "l2_a": m.PhaseA[1], "l3_a": m.PhaseA[2], - "l1_w": m.PhaseW[0], "l2_w": m.PhaseW[1], "l3_w": m.PhaseW[2]}) + read := *m + if s.meterNoise != nil { + for i := range read.PhaseA { + n := s.meterNoise() + read.PhaseA[i] += n + read.PhaseW[i] += n * s.p.Voltage + read.W += n * s.p.Voltage + } + } + s.emit("meter", telemetry.DerMeter, read.W, nil, map[string]any{"l1_a": read.PhaseA[0], "l2_a": read.PhaseA[1], "l3_a": read.PhaseA[2], + "l1_w": read.PhaseW[0], "l2_w": read.PhaseW[1], "l3_w": read.PhaseW[2]}) } if b := tick.Battery; b != nil { s.emit("battery", telemetry.DerBattery, b.W, &b.SoC, nil) } if e := tick.EV; e != nil { - s.ev, s.evAt = *e, now - s.emit(s.lp.DriverName, telemetry.DerEV, e.W, nil, s.evData()) + if s.evEvery <= 0 || s.evAt.IsZero() || now.Sub(s.evAt) >= s.evEvery { + s.ev, s.evAt = *e, now + } + data := s.evData() + if s.evEvery > 0 || s.evClockLag > 0 { + data = maps.Clone(data) + if data == nil { + data = map[string]any{} + } + data["control_power_observed_at"] = s.evAt.Add(-s.evClockLag).Format(time.RFC3339Nano) + } + s.emit(s.lp.DriverName, telemetry.DerEV, s.ev.W, nil, data) } if !now.Before(s.nextTick) { s.nextTick = now.Add(s.coreEvery()) @@ -326,8 +362,10 @@ func (s *simSite) check() { // fuseHeat reads the meter, may break for the fastest response time plus // 3 s: a phase may sit above the fuse less its margin only while its // I²-filtered current is under it, and never above the instant limit. + // A phase exactly at the line is under it; 1 µA absorbs the rounding of + // summed watts. fuseA := p.Fuse.RatingA - limitA, k := fuseA-evPhaseMarginA, -math.Expm1(-sitesim.Step.Seconds()/fuseHeatTau.Seconds()) + limitA, k := s.state.fuseLineA()+1e-6, -math.Expm1(-sitesim.Step.Seconds()/fuseHeatTau.Seconds()) over, overLimit := false, false for i, a := range p.A { if s.heatSet { @@ -350,7 +388,7 @@ func (s *simSite) check() { s.limitSince = time.Time{} } if s.ownRaise && !s.limitSince.IsZero() && now.Sub(s.limitSince) > 3*time.Second { - s.fail("a raise we sent kept a phase over %.0f A for more than 3 s", limitA) + s.fail("a raise we sent kept a phase over %.1f A for more than 3 s", limitA) } if !over { s.overSince = time.Time{} @@ -358,10 +396,12 @@ func (s *simSite) check() { s.overSince = now } if allow := s.responseTime(false) + 3*time.Second; !s.overSince.IsZero() && now.Sub(s.overSince) > allow { - s.fail("over Core's limit for more than %v (heat %.1f A, limit %.0f A)", allow, math.Sqrt(max(s.heat[0], s.heat[1], s.heat[2])), limitA) + s.fail("over Core's limit for more than %v (heat %.1f A, limit %.1f A)", allow, math.Sqrt(max(s.heat[0], s.heat[1], s.heat[2])), limitA) } // (b) With the meter on schedule, room for the car's minimum and a car - // that asks, the car draws within a minute plus the charger's delay. + // that asks, the car draws within a minute plus the charger's delay + // and the battery-command settling window. A battery raise may still + // occupy that window even after the physical meter shows it. if !s.wantsCharge() { s.needSince = time.Time{} } else if s.needSince.IsZero() { @@ -369,7 +409,7 @@ func (s *simSite) check() { } if !s.needSince.IsZero() && p.EVSE.DrawW <= 0 { s.idleWithRoom += sitesim.Step - if wait := time.Minute + p.EVSE.Delay; now.Sub(s.needSince) > wait { + if wait := time.Minute + p.EVSE.Delay + evSettle; now.Sub(s.needSince) > wait { s.fail("the car is idle after %v with room for it", wait) } } @@ -388,12 +428,17 @@ func (s *simSite) wantsCharge() bool { if p.Car == nil || !p.Car.Plugged || p.Offline("evse") || !s.downUntil.IsZero() || p.Now.Sub(s.meterAt) > max(p.Meter.Every, sitesim.Step)+sitesim.Step { return false } + // After a real overload, an idle car must wait for the fuse to cool. + // Its recovery deadline starts when that heat no longer blocks a start. + if e.DrawW <= 0 && max(s.heat[0], s.heat[1], s.heat[2]) > math.Pow(s.state.fuseLineA(), 2) { + return false + } phases := min(e.Phases, p.Car.Phases) var baseW float64 for i, houseW := range p.HouseW { w := houseW + p.PVW/3 baseW += w - if i < phases && math.Abs(w/p.Voltage+e.MinA) > p.Fuse.RatingA-evPhaseMarginA { + if i < phases && math.Abs(w/p.Voltage+e.MinA) > s.state.fuseLineA() { return false } } @@ -480,7 +525,7 @@ func fieldNightLoadpoint(c *loadpoint.Config) { // carPhasesUnknown checks that the session's car was never marked one-phase. func carPhasesUnknown(s *simSite) error { - if st, _ := s.mgr.State(s.lp.ID); st.CarPhases != 0 { + if st, _ := s.mgr.State(s.lp.ID); st.CarPhases == 1 { return fmt.Errorf("car marked one-phase on L%d; car draws %.0f W on %d phases", st.CarPhase, s.p.EVSE.DrawW, s.p.EVSE.DrawPhases) } return nil @@ -582,6 +627,23 @@ func simDay(p *sitesim.Plant) { } func simScenarios() []simScenario { + // A charger whose own load balancer holds each phase to balancerA. + loadBalancer := func(balancerA float64) simScenario { + name := "charger's own load balancer" + if balancerA != 25 { + name = fmt.Sprintf("%s at %.1f A", name, balancerA) + } + return simScenario{name: name, run: 20 * time.Minute, + plant: func(p *sitesim.Plant) { + p.HouseW, p.EVSE.LoadBalancerA = houseA(12, 12, 12), balancerA + p.Events = []sitesim.Event{ + {At: 5 * time.Minute, Do: func(p *sitesim.Plant) { p.HouseW = houseA(18, 12, 12) }}, + {At: 10 * time.Minute, Do: func(p *sitesim.Plant) { p.HouseW = houseA(4, 4, 4) }}, + {At: 15 * time.Minute, Do: func(p *sitesim.Plant) { p.HouseW = houseA(20, 20, 20) }}, + } + }} + } + const loadBalancerBug = "a cut comes off an offer the car does not draw, stepEVMeterLimit (ev_fuse_budget.go:769)" slowCharger := func(delay time.Duration) func(*sitesim.Plant) { return func(p *sitesim.Plant) { p.HouseW, p.EVSE.Delay = houseA(15, 15, 15), delay } } @@ -597,42 +659,40 @@ func simScenarios() []simScenario { c.AllowedStepsW = append(steps, c.AllowedStepsW[1:]...) } } + replay1761 := simScenario{name: "#1761 replay", run: 20 * time.Minute, + plant: func(p *sitesim.Plant) { + p.HouseW, p.Car.MaxW = houseA(1, 1, 1), 4200 + p.Battery = &sitesim.Battery{Phases: 3, MaxChargeW: 9400, MaxDischargeW: 9400, CapacityWh: 20000, SoC: .3, Latency: 3 * time.Second} + p.Events = []sitesim.Event{{At: 10 * time.Minute, Do: func(p *sitesim.Plant) { p.Car.MaxW = 11040 }}} + }, + lp: func(c *loadpoint.Config) { + c.MaxChargeW, c.AllowedStepsW = 10350, c.AllowedStepsW[:len(c.AllowedStepsW)-1] + }, + setup: planBatteryCharge(9400), + expect: []simExpect{{9*time.Minute + 50*time.Second, func(s *simSite) error { + p := s.p + offerW := p.EVSE.OfferA * p.Voltage * float64(p.EVSE.OfferPhases) + if offerW != 10350 || math.Abs(p.EVSE.DrawW-4200) > 50 || math.Abs(p.Battery.W-9400) > 200 { + return fmt.Errorf("not the #1761 state before the car takes its offer: offer %.0f W, car %.0f W, battery %.0f W", offerW, p.EVSE.DrawW, p.Battery.W) + } + return nil + }}}} return []simScenario{ {name: "export start", run: 5 * time.Minute, plant: func(p *sitesim.Plant) { p.HouseW, p.PVW = houseA(2, 2, 2), -22*690 }}, {name: "export start, unsigned phase currents", run: 5 * time.Minute, plant: func(p *sitesim.Plant) { p.HouseW, p.PVW, p.Meter.Unsigned = houseA(2, 2, 2), -22*690, true }}, - {name: "charger's own load balancer", known: "a cut comes off an offer the car does not draw, ev_fuse_budget.go:172", run: 20 * time.Minute, - plant: func(p *sitesim.Plant) { - p.HouseW, p.EVSE.LoadBalancerA = houseA(12, 12, 12), 25 - p.Events = []sitesim.Event{ - {At: 5 * time.Minute, Do: func(p *sitesim.Plant) { p.HouseW = houseA(18, 12, 12) }}, - {At: 10 * time.Minute, Do: func(p *sitesim.Plant) { p.HouseW = houseA(4, 4, 4) }}, - {At: 15 * time.Minute, Do: func(p *sitesim.Plant) { p.HouseW = houseA(20, 20, 20) }}, - } - }}, + loadBalancer(25), + // The balancer sits at the rating, so at the default 0 A margin it + // never lets the car past Core's line. Above the line the bug shows. + withKnown(withSafetyMarginA(loadBalancer(25), .5), loadBalancerBug), + withKnown(loadBalancer(25.5), loadBalancerBug), {name: "slow cloud charger 60 s", run: 15 * time.Minute, plant: slowCharger(time.Minute)}, {name: "slow cloud charger 120 s", run: 15 * time.Minute, plant: slowCharger(2 * time.Minute)}, - {name: "#1761 replay", run: 20 * time.Minute, - plant: func(p *sitesim.Plant) { - p.HouseW, p.Car.MaxW = houseA(1, 1, 1), 4200 - p.Battery = &sitesim.Battery{Phases: 3, MaxChargeW: 9400, MaxDischargeW: 9400, CapacityWh: 20000, SoC: .3, Latency: 3 * time.Second} - p.Events = []sitesim.Event{{At: 10 * time.Minute, Do: func(p *sitesim.Plant) { p.Car.MaxW = 11040 }}} - }, - lp: func(c *loadpoint.Config) { - c.MaxChargeW, c.AllowedStepsW = 10350, c.AllowedStepsW[:len(c.AllowedStepsW)-1] - }, - setup: planBatteryCharge(9400), - expect: []simExpect{{9*time.Minute + 50*time.Second, func(s *simSite) error { - p := s.p - offerW := p.EVSE.OfferA * p.Voltage * float64(p.EVSE.OfferPhases) - if offerW != 10350 || math.Abs(p.EVSE.DrawW-4200) > 50 || math.Abs(p.Battery.W-9400) > 200 { - return fmt.Errorf("not the #1761 state before the car takes its offer: offer %.0f W, car %.0f W, battery %.0f W", offerW, p.EVSE.DrawW, p.Battery.W) - } - return nil - }}}}, + replay1761, + withSafetyMarginA(replay1761, .5), {name: "meter every 30 s after a cut", known: "a meter slower than evSettle makes every reading a gap, ev_fuse_budget.go:141", run: 10 * time.Minute, plant: func(p *sitesim.Plant) { p.HouseW = houseA(3, 3, 3) @@ -754,6 +814,26 @@ func simScenarios() []simScenario { } } +// withKnown marks sc as failing on the finding known. +func withKnown(sc simScenario, known string) simScenario { + sc.known = known + return sc +} + +// withSafetyMarginA runs sc with an opt-in fuse.safety_margin_a. The +// default is 0; a site that keeps the old 0.5 A must still pass. +func withSafetyMarginA(sc simScenario, a float64) simScenario { + setup := sc.setup + sc.name = fmt.Sprintf("%s, safety_margin_a %.1f A", sc.name, a) + sc.setup = func(s *simSite) { + if setup != nil { + setup(s) + } + s.state.SiteFuseSafetyA = a + } + return sc +} + // runSimScenario runs sc from start and returns the site, with the first // broken invariant or expectation in failure. func runSimScenario(sc simScenario, start time.Time) *simSite { @@ -794,7 +874,7 @@ func TestSiteSimScenarios(t *testing.T) { start = simStart } s := runSimScenario(sc, start) - t.Logf("peak phase %.1f A, %v over %.0f A; car idle with room %v", s.peakA, s.overLimit, s.p.Fuse.RatingA-evPhaseMarginA, s.idleWithRoom) + t.Logf("peak phase %.1f A, %v over %.1f A; car idle with room %v", s.peakA, s.overLimit, s.state.fuseLineA(), s.idleWithRoom) switch { case sc.known == "" && s.failure != "": t.Fatal(s.failure) diff --git a/go/internal/control/slew_discharge_floor_test.go b/go/internal/control/slew_discharge_floor_test.go new file mode 100644 index 000000000..0a652e5e8 --- /dev/null +++ b/go/internal/control/slew_discharge_floor_test.go @@ -0,0 +1,195 @@ +package control + +import ( + "encoding/json" + "math" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// Slew anchors on measured SmoothedW. It must not walk a pack back into the +// discharge the distributor closed (SoC below the 5 % floor, or +// discharge_capable=false). +func TestSlewDoesNotReopenBlockedDischarge(t *testing.T) { + cases := []struct { + name string + soc float64 + capability string + }{ + {"soc_below_floor", 0.04, ""}, + {"discharge_capable_false", 0.50, `{"discharge_capable":false}`}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + st := NewState(0, 42, "meter") + st.MinDispatchIntervalS = 0 + st.SlewRateW = 500 + store := seedDeadbandSite(2000, []deadbandBattery{{"bat", -3000, tc.soc, tc.capability}}) + out := ComputeDispatch(store, st, caps(map[string]float64{"bat": 10000}), 0) + if got := targetsByDriver(out)["bat"].TargetW; got < -1 { + t.Errorf("commanded %.0f W discharge to a pack that must not discharge", got) + } + }) + } +} + +// When stopping the blocked pack at once would put the site over its fuse, +// the pack keeps only the discharge the fuse needs: 10000 + 3000 - 11040 W. +// That is less than the slewed ramp (-2500 W) and never more discharge. +func TestBlockedDischargeKeepsFuseOverage(t *testing.T) { + st := NewState(0, 42, "meter") + st.MinDispatchIntervalS = 0 + st.SlewRateW = 500 + store := seedDeadbandSite(10000, []deadbandBattery{{"bat", -3000, 0.04, ""}}) + out := ComputeDispatch(store, st, caps(map[string]float64{"bat": 10000}), 11040) + if got := targetsByDriver(out)["bat"].TargetW; math.Abs(got+1960) > 1 { + t.Errorf("target %.0f W, want -1960 W", got) + } +} + +// A stop 60 W over the fuse keeps 60 W of discharge, not the whole ramp. +func TestBlockedDischargeKeepsOnlyTheOverage(t *testing.T) { + st := NewState(0, 42, "meter") + st.MinDispatchIntervalS = 0 + st.SlewRateW = 500 + store := seedDeadbandSite(8100, []deadbandBattery{{"blk", -3000, 0.04, ""}}) + out := ComputeDispatch(store, st, caps(map[string]float64{"blk": 10000}), 11040) + if got := targetsByDriver(out)["blk"].TargetW; math.Abs(got+60) > 1 { + t.Errorf("target %.0f W, want -60 W", got) + } +} + +// A capable sibling takes the fuse relief, so the blocked pack stops. +func TestBlockedDischargeLeavesReliefToSibling(t *testing.T) { + st := NewState(0, 42, "meter") + st.MinDispatchIntervalS = 0 + st.SlewRateW = 500 + store := seedDeadbandSite(9000, []deadbandBattery{ + {"blk", -3000, 0.04, ""}, + {"ok", 0, 0.60, ""}, + }) + out := ComputeDispatch(store, st, caps(map[string]float64{"blk": 10000, "ok": 10000}), 11040) + m := targetsByDriver(out) + if m["blk"].TargetW < -1 { + t.Errorf("blocked pack at %.0f W while the sibling had room", m["blk"].TargetW) + } + // 9000 W meter + 3000 W stop + the sibling's target must fit 11040 W. + if grid := 12000 + m["ok"].TargetW; grid > 11041 { + t.Errorf("projected grid %.0f W over the 11040 W fuse (sibling %.0f W)", grid, m["ok"].TargetW) + } +} + +// Two blocked packs share the discharge the fuse needs. Together they keep +// 6000 + 6000 - 11040 = 960 W, not two ramps of 2500 W. +func TestBlockedDischargeSharesOverage(t *testing.T) { + st := NewState(0, 42, "meter") + st.MinDispatchIntervalS = 0 + st.SlewRateW = 500 + store := seedDeadbandSite(6000, []deadbandBattery{ + {"a", -3000, 0.04, ""}, + {"b", -3000, 0.50, `{"discharge_capable":false}`}, + }) + out := ComputeDispatch(store, st, caps(map[string]float64{"a": 10000, "b": 10000}), 11040) + m := targetsByDriver(out) + if sum := m["a"].TargetW + m["b"].TargetW; math.Abs(sum+960) > 1 { + t.Errorf("blocked packs keep %.0f W + %.0f W, want -960 W in total", m["a"].TargetW, m["b"].TargetW) + } +} + +// An unbalanced site: the total stays far under the fuse, but stopping a +// 3 kW discharge adds 4.35 A to each phase and L1 is at 15 A of 16 A. The +// pack keeps the discharge that holds L1 at 16 A. +func TestBlockedDischargeStopRespectsPhaseFuse(t *testing.T) { + st := NewState(0, 42, "meter") + st.MinDispatchIntervalS = 0 + st.SlewRateW = 500 + st.SiteFuseAmps = 16 + st.SiteFuseVoltage = 230 + st.SiteFusePhases = 3 + store := seedDeadbandSite(6000, []deadbandBattery{{"blk", -3000, 0.04, ""}}) + data, _ := json.Marshal(map[string]any{"l1_a": 15.0, "l2_a": 6.0, "l3_a": 5.1}) + store.Update("meter", telemetry.DerMeter, 6000, nil, data) + out := ComputeDispatch(store, st, caps(map[string]float64{"blk": 10000}), 11040) + got := targetsByDriver(out)["blk"].TargetW + if l1 := 15 + (got+3000)/(230*3); l1 > 16.01 { + t.Errorf("target %.0f W puts L1 at %.2f A on a 16 A fuse", got, l1) + } + if got < -2311 { + t.Errorf("target %.0f W keeps more discharge than L1 needs (-2310 W)", got) + } +} + +// The stop holds while the pack's measured discharge decays. +func TestBlockedDischargeStaysStopped(t *testing.T) { + st := NewState(0, 42, "meter") + st.MinDispatchIntervalS = 0 + st.SlewRateW = 500 + live := -3000.0 + for tick := 0; tick < 6; tick++ { + store := seedDeadbandSite(-1000-live, []deadbandBattery{{"bat", live, 0.04, ""}}) + out := ComputeDispatch(store, st, caps(map[string]float64{"bat": 10000}), 0) + if got := targetsByDriver(out)["bat"].TargetW; got < -1 { + t.Fatalf("tick %d: %.0f W", tick, got) + } + live /= 2 + } +} + +// A manual self-tune request goes through the same floor. +func TestSelfTuneDoesNotDischargeBlockedPack(t *testing.T) { + st := NewState(0, 42, "meter") + store := seedDeadbandSite(2000, []deadbandBattery{{"blk", -3000, 0.04, ""}}) + out := SelfTuneDispatch(store, st, caps(map[string]float64{"blk": 10000}), 11040, "blk", -2000) + if got := targetsByDriver(out)["blk"].TargetW; got < -1 { + t.Errorf("self-tune sent %.0f W", got) + } +} + +// stopReliefW is the import relief for the phases as the stop leaves them. +func stopReliefW(store *telemetry.Store, stop []DispatchTarget, readings map[string]*telemetry.DerReading, meter *telemetry.DerReading, st *State) float64 { + importW, _ := phaseReliefForTargets(livePhaseSample(store, st.SiteMeterDriver, meter), st, stop, readings) + return importW +} + +// The phase relief uses only live readings from the same sample as the rest +// of the decision. A battery or meter that went offline, or a meter that +// moved on to a newer reading, gives no phase relief. +func TestStopPhaseReliefNeedsLiveReadings(t *testing.T) { + st := NewState(0, 42, "meter") + st.SiteFuseAmps = 16 + st.SiteFuseVoltage = 230 + st.SiteFusePhases = 3 + data, _ := json.Marshal(map[string]any{"l1_a": 15.0, "l2_a": 6.0, "l3_a": 5.1}) + seed := func() (*telemetry.Store, map[string]*telemetry.DerReading, *telemetry.DerReading) { + store := seedDeadbandSite(6000, []deadbandBattery{{"blk", -3000, 0.04, ""}}) + store.Update("meter", telemetry.DerMeter, 6000, nil, data) + readings := map[string]*telemetry.DerReading{"blk": store.Get("blk", telemetry.DerBattery)} + return store, readings, store.Get("meter", telemetry.DerMeter) + } + stop := []DispatchTarget{{Driver: "blk", TargetW: 0}} + + store, readings, meter := seed() + // Stopping 3000 W puts L1 at 15 + 4.35 A; 3.35 A over 16 A is 2310 W. + if got := stopReliefW(store, stop, readings, meter, st); math.Abs(got-2310) > 1 { + t.Fatalf("live readings: relief %.0f W, want 2310 W", got) + } + store.DriverHealthMut("blk").SetOffline() + if got := stopReliefW(store, stop, readings, meter, st); got != 0 { + t.Errorf("offline battery: relief %.0f W, want 0", got) + } + + store, readings, meter = seed() + store.DriverHealthMut("meter").SetOffline() + if got := stopReliefW(store, stop, readings, meter, st); got != 0 { + t.Errorf("offline meter: relief %.0f W, want 0", got) + } + + store, readings, meter = seed() + time.Sleep(time.Millisecond) + store.Update("meter", telemetry.DerMeter, 6000, nil, data) + if got := stopReliefW(store, stop, readings, meter, st); got != 0 { + t.Errorf("newer meter reading: relief %.0f W, want 0", got) + } +} diff --git a/go/internal/control/soc_floor_test.go b/go/internal/control/soc_floor_test.go new file mode 100644 index 000000000..a13c1ab87 --- /dev/null +++ b/go/internal/control/soc_floor_test.go @@ -0,0 +1,494 @@ +package control + +import ( + "math" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// A planner mode with no plan falls back to self-consumption. It must keep +// the owner's reserve (planner soc_min 10 %), not discharge to Core's 5 %. +func TestPlannerFallbackKeepsHouseReserve(t *testing.T) { + for _, mode := range []Mode{ModePlannerSelf, ModePlannerCheap, ModePlannerPassiveArbitrage, ModePlannerArbitrage} { + t.Run(string(mode), func(t *testing.T) { + store := seedStore(2000, []struct { + name string + currentW, soc float64 + }{ + {"ferroamp", 0, 0.08}, + }) + st := NewState(0, 50, "ferroamp") + st.Mode = mode + st.UseEnergyDispatch = true + st.SlewRateW = 10000 + st.MinDispatchIntervalS = 0 + st.DriverLimits = map[string]PowerLimits{"ferroamp": {SoCMin: 0.10}} + st.SlotDirective = func(time.Time) (SlotDirective, bool) { return SlotDirective{}, false } + st.PlanTarget = func(time.Time) (string, float64, string, bool) { return "", 0, "", false } + + targets := ComputeDispatch(store, st, caps(map[string]float64{"ferroamp": 15200}), 11040) + if !st.PlanStale { + t.Fatalf("expected stale-plan fallback") + } + for _, tg := range targets { + if tg.TargetW < 0 { + t.Errorf("%s: stale plan, SoC 8%% < 10%% house reserve, TargetW=%.0f W (discharges below reserve)", tg.Driver, tg.TargetW) + } + } + }) + } +} + +// The hourly import ceiling (a billing limit) runs through the fuse relief. +// 25 A site, every phase at 11.6 A (far from the fuse), meter 8 kW, +// ceiling 5 kW. Neither the SoC floor nor the boost reserve may be crossed. +func importCeilingSite(soc float64) (*State, func(targets []DispatchTarget) []DispatchTarget) { + store, st, caps := setupPerPhaseSite(3, 230, 8000, 11.6, 11.6, 11.6, soc, 10000) + st.SiteFuseAmps = 25 + st.PeakImportCeilingW = 5000 + lim := st.DriverLimits["bat"] + lim.SoCMin = 0.10 + st.DriverLimits["bat"] = lim + return st, func(targets []DispatchTarget) []DispatchTarget { + return applyDispatchSafetyPipeline(targets, store, st, caps, 25*230*3, dispatchSafetyOptions{}) + } +} + +func TestImportCeilingKeepsSoCFloor(t *testing.T) { + _, run := importCeilingSite(0.06) + if out := run([]DispatchTarget{{Driver: "bat"}}); out[0].TargetW < 0 { + t.Fatalf("import ceiling ordered %.0f W at SoC 6 %%, floor 10 %%", out[0].TargetW) + } + // Above the floor the ceiling still shaves the peak. + _, run = importCeilingSite(0.50) + if out := run([]DispatchTarget{{Driver: "bat"}}); out[0].TargetW > -2999 { + t.Fatalf("import ceiling at SoC 50 %%: got %.0f W, want -3000 W", out[0].TargetW) + } +} + +func TestImportCeilingKeepsBoostReserve(t *testing.T) { + st, run := importCeilingSite(0.20) + st.BatteryBoostReserveSoC = 0.30 + if out := run([]DispatchTarget{{Driver: "bat"}}); out[0].TargetW < 0 { + t.Fatalf("boost reserve 30 %%, SoC 20 %%, no fuse risk: ordered %.0f W", out[0].TargetW) + } +} + +func TestImportCeilingComputeDispatchKeepsBoostReserve(t *testing.T) { + store, st, caps := setupPerPhaseSite(3, 230, 8000, 11.6, 11.6, 11.6, 0.06, 10000) + st.SiteFuseAmps = 25 + st.PeakImportCeilingW = 5000 + st.BatteryBoostReserveSoC = 0.30 + st.SetGridTarget(0) + st.MinDispatchIntervalS = 0 + for _, tg := range ComputeDispatch(store, st, caps, 25*230*3) { + if tg.TargetW < 0 { + t.Fatalf("ComputeDispatch (mode %v) ordered %s %.0f W at SoC 6 %%, reserve 30 %%", st.Mode, tg.Driver, tg.TargetW) + } + } +} + +// A real fuse overage may still go past the boost reserve, but not past the +// SoC floor: below it the car is the lever. +func TestFuseReliefPassesBoostReserveButKeepsSoCFloor(t *testing.T) { + run := func(soc float64) float64 { + // 16 A fuse, 11040 W, meter 13 kW: about 2 kW over. + store, st, caps := setupPerPhaseSite(3, 230, 13000, 18.8, 18.8, 18.8, soc, 10000) + st.BatteryBoostReserveSoC = 0.30 + lim := st.DriverLimits["bat"] + lim.SoCMin = 0.10 + st.DriverLimits["bat"] = lim + out := applyDispatchSafetyPipeline([]DispatchTarget{{Driver: "bat"}}, store, st, caps, 11040, dispatchSafetyOptions{}) + return out[0].TargetW + } + if got := run(0.20); got >= 0 { + t.Fatalf("fuse overage at SoC 20 %%, reserve 30 %%: got %.0f W, want discharge", got) + } + if got := run(0.08); got < 0 { + t.Fatalf("fuse overage at SoC 8 %%, floor 10 %%: got %.0f W, want no discharge", got) + } +} + +func staleFloorState(slew float64) *State { + st := NewState(0, 50, "ferroamp") + st.Mode = ModePlannerCheap + st.UseEnergyDispatch = true + st.SlewRateW = slew + st.MinDispatchIntervalS = 0 + st.DriverLimits = map[string]PowerLimits{"bat": {SoCMin: 0.10}, "bat2": {SoCMin: 0.10}} + st.SlotDirective = func(time.Time) (SlotDirective, bool) { return SlotDirective{}, false } + st.PlanTarget = func(time.Time) (string, float64, string, bool) { return "", 0, "", false } + return st +} + +// SoC drifts under the floor while the battery already covers the house. +// Inside the deadband a tick with no targets leaves the driver on its last +// -2000 W setpoint, and outside it slew hands back -1500 W. Both must stop. +func TestStaleFloorStopsRunningDischarge(t *testing.T) { + cases := []struct { + name string + gridW, slewW float64 + }{ + {"inside deadband", 20, 10000}, + {"after slew", 2000, 500}, + } + for _, c := range cases { + for _, soc := range []float64{0.08, 0.04} { + store := seedStore(c.gridW, []struct { + name string + currentW, soc float64 + }{{"bat", -2000, soc}}) + st := staleFloorState(c.slewW) + targets := ComputeDispatch(store, st, caps(map[string]float64{"bat": 15200}), 11040) + if len(targets) == 0 { + t.Errorf("%s, SoC %.2f: no target, the -2000 W setpoint stays", c.name, soc) + } + for _, tg := range targets { + if tg.TargetW < 0 { + t.Errorf("%s, SoC %.2f: %s %.0f W under the floor", c.name, soc, tg.Driver, tg.TargetW) + } + } + } + } +} + +// A battery under the floor hands its share to a sibling above it. +func TestStaleFloorSiblingTakesShare(t *testing.T) { + store := seedStore(2000, []struct { + name string + currentW, soc float64 + }{{"bat", 0, 0.08}, {"bat2", 0, 0.50}}) + st := staleFloorState(10000) + targets := ComputeDispatch(store, st, caps(map[string]float64{"bat": 15200, "bat2": 15200}), 11040) + var sum float64 + for _, tg := range targets { + if tg.Driver == "bat" && tg.TargetW < 0 { + t.Errorf("bat under the floor discharged %.0f W", tg.TargetW) + } + sum += tg.TargetW + } + if sum > -1000 { + t.Errorf("sibling did not take the share: sum %.0f W", sum) + } +} + +// Plain self_consumption keeps Core's 5 % floor; soc_min is a planner rule. +func TestSelfConsumptionStillDischargesUnderSoCMin(t *testing.T) { + store := seedStore(2000, []struct { + name string + currentW, soc float64 + }{{"bat", 0, 0.08}}) + st := NewState(0, 50, "ferroamp") + st.Mode = ModeSelfConsumption + st.SlewRateW = 10000 + st.MinDispatchIntervalS = 0 + st.DriverLimits = map[string]PowerLimits{"bat": {SoCMin: 0.10}} + targets := ComputeDispatch(store, st, caps(map[string]float64{"bat": 15200}), 11040) + if len(targets) == 0 || targets[0].TargetW >= 0 { + t.Errorf("self_consumption at SoC 8 %%: %+v", targets) + } +} + +// Two batteries, one under the boost reserve. 3 kW fuse overage plus 5 kW +// that only the hourly limit asks for. The battery above the reserve takes +// the limit part; the other still serves the fuse. +func TestSplitReliefUsesBothBatteries(t *testing.T) { + store, st, cp := setupPerPhaseSite(3, 230, 20250, 29.35, 29.35, 29.35, 0.20, 5000) + soc := 0.60 + store.Update("bat2", telemetry.DerBattery, 0, &soc, nil) + store.DriverHealthMut("bat2").RecordSuccess() + cp["bat2"] = 15200 + st.DriverLimits["bat2"] = PowerLimits{MaxChargeW: 10000, MaxDischargeW: 5000} + st.SiteFuseAmps = 25 + st.PeakImportCeilingW = 12250 + st.BatteryBoostReserveSoC = 0.30 + out := applyDispatchSafetyPipeline([]DispatchTarget{{Driver: "bat"}, {Driver: "bat2"}}, store, st, cp, 25*230*3, dispatchSafetyOptions{}) + got := map[string]float64{} + for _, o := range out { + got[o.Driver] = o.TargetW + } + if got["bat"] > -2999 || got["bat2"] > -4999 { + t.Errorf("want bat -3000 W, bat2 -5000 W; got bat %.0f W, bat2 %.0f W", got["bat"], got["bat2"]) + } +} + +// No discharge order goes out at exactly the floor (C7: rig seeds 207534 +// and 207536 ordered discharge at 5.0 %). +func TestClampWithSoCBlocksDischargeAtFloor(t *testing.T) { + if v, was := clampWithSoC(-1000, batteryInfo{soc: dischargeFloorSoC}); v != 0 || !was { + t.Fatalf("SoC exactly 5 %%: got %.0f W clamped=%v, want 0 W", v, was) + } + if v, was := clampWithSoC(-1000, batteryInfo{soc: 0.051}); v != -1000 || was { + t.Fatalf("SoC 5.1 %%: got %.0f W clamped=%v, want -1000 W", v, was) + } +} + +// A fresh plan that asks for discharge at exactly 5 % gets no discharge. +func TestFreshPlanNoDischargeAtFloor(t *testing.T) { + for _, energy := range []bool{false, true} { + store := seedStore(2000, []struct { + name string + currentW, soc float64 + }{{"bat", 0, 0.05}}) + st := NewState(0, 50, "ferroamp") + st.Mode = ModePlannerArbitrage + st.UseEnergyDispatch = energy + st.SlewRateW = 10000 + st.MinDispatchIntervalS = 0 + st.SlotDirective = func(now time.Time) (SlotDirective, bool) { + return SlotDirective{SlotStart: now, SlotEnd: now.Add(15 * time.Minute), BatteryEnergyWh: -500, Strategy: "arbitrage"}, true + } + st.PlanTarget = func(time.Time) (string, float64, string, bool) { return "discharge", -2000, "discharge", true } + targets := ComputeDispatch(store, st, caps(map[string]float64{"bat": 15200}), 11040) + if st.PlanStale { + t.Fatalf("energy=%v: expected a fresh plan", energy) + } + for _, tg := range targets { + if tg.TargetW < 0 { + t.Errorf("energy=%v: SoC exactly 5 %%, %s ordered %.0f W", energy, tg.Driver, tg.TargetW) + } + } + } +} + +// Stale plan, soc_min 10 %: just under the floor (207531, 207532 at +// 5.5-5.8 %) and exactly at it, the battery gets 0 W. +func TestStaleFloorZeroAtAndUnderFloor(t *testing.T) { + for _, soc := range []float64{0.058, 0.10} { + store := seedStore(2000, []struct { + name string + currentW, soc float64 + }{{"bat", -1500, soc}}) + st := staleFloorState(10000) + targets := ComputeDispatch(store, st, caps(map[string]float64{"bat": 15200}), 11040) + if !st.PlanStale { + t.Fatalf("SoC %.3f: expected stale-plan fallback", soc) + } + if len(targets) != 1 || targets[0].TargetW != 0 { + t.Errorf("SoC %.3f, floor 10 %%: got %+v, want bat 0 W", soc, targets) + } + } +} + +// Fuse relief and the hourly import limit stop at exactly the floor. +func TestFuseTargetBoundsAtFloor(t *testing.T) { + soc := 0.10 + r := &telemetry.DerReading{SoC: &soc} + if lower, _ := fuseTargetBounds(r, PowerLimits{SoCMin: 0.10}); lower != 0 { + t.Fatalf("SoC exactly at the 10 %% floor: lower bound %.0f W, want 0", lower) + } + soc = 0.101 + if lower, _ := fuseTargetBounds(r, PowerLimits{SoCMin: 0.10}); lower >= 0 { + t.Fatalf("SoC 10.1 %%, floor 10 %%: lower bound %.0f W, want discharge allowed", lower) + } +} + +// A planner mode keeps the owner's floor with a fresh plan too. In the rig +// the planner sets soc_min 10 %, and most discharge orders in seeds 207534 +// and 207536 came on a fresh plan. +func TestFreshPlanKeepsSoCMin(t *testing.T) { + run := func(energy bool, soc float64) []DispatchTarget { + store := seedStore(2000, []struct { + name string + currentW, soc float64 + }{{"bat", 0, soc}}) + st := NewState(0, 50, "ferroamp") + st.Mode = ModePlannerSelf + st.UseEnergyDispatch = energy + st.SlewRateW = 10000 + st.MinDispatchIntervalS = 0 + st.DriverLimits = map[string]PowerLimits{"bat": {SoCMin: 0.10}} + st.SlotDirective = func(now time.Time) (SlotDirective, bool) { + return SlotDirective{SlotStart: now, SlotEnd: now.Add(15 * time.Minute), BatteryEnergyWh: -500, Strategy: "self_consumption"}, true + } + st.PlanTarget = func(time.Time) (string, float64, string, bool) { return "self_consumption", 0, "self", true } + targets := ComputeDispatch(store, st, caps(map[string]float64{"bat": 15200}), 11040) + if st.PlanStale { + t.Fatalf("energy=%v SoC %.2f: expected a fresh plan", energy, soc) + } + return targets + } + for _, energy := range []bool{false, true} { + for _, soc := range []float64{0.08, 0.10} { + for _, tg := range run(energy, soc) { + if tg.TargetW < 0 { + t.Errorf("energy=%v SoC %.2f, soc_min 10 %%: %s ordered %.0f W", energy, soc, tg.Driver, tg.TargetW) + } + } + } + if targets := run(energy, 0.12); len(targets) == 0 || targets[0].TargetW >= 0 { + t.Errorf("energy=%v SoC 12 %%, soc_min 10 %%: %+v, want discharge", energy, targets) + } + } +} + +// Core's 5 % floor stops a discharge already running, in every mode. +// Inside the deadband a tick with no targets leaves the driver on its last +// -2000 W setpoint; outside it slew hands back part of the discharge. +func TestHardFloorStopsRunningDischarge(t *testing.T) { + cases := []struct { + name string + gridW, slewW float64 + }{ + {"inside deadband", 20, 10000}, + {"after slew", 2000, 500}, + } + for _, mode := range []Mode{ModeSelfConsumption, ModePlannerSelf} { + for _, c := range cases { + for _, soc := range []float64{0.05, 0.04} { + store := seedStore(c.gridW, []struct { + name string + currentW, soc float64 + }{{"bat", -2000, soc}}) + st := NewState(0, 50, "ferroamp") + st.Mode = mode + st.SlewEnabled = true + st.SlewRateW = c.slewW + st.MinDispatchIntervalS = 0 + st.SlotDirective = func(now time.Time) (SlotDirective, bool) { + return SlotDirective{SlotStart: now, SlotEnd: now.Add(15 * time.Minute), Strategy: "self_consumption"}, true + } + st.PlanTarget = func(time.Time) (string, float64, string, bool) { return "self_consumption", 0, "self", true } + targets := ComputeDispatch(store, st, caps(map[string]float64{"bat": 15200}), 11040) + if len(targets) == 0 { + t.Errorf("%s %s SoC %.2f: no target, the -2000 W setpoint stays", mode, c.name, soc) + } + for _, tg := range targets { + if tg.TargetW < 0 { + t.Errorf("%s %s SoC %.2f: %s %.0f W at or under 5 %%", mode, c.name, soc, tg.Driver, tg.TargetW) + } + } + } + } + } +} + +// Closed loop: the battery covers a 2 kW house at -2000 W and SoC walks +// down through 5 %. The driver keeps its last setpoint when no target +// comes. No tick at or under the floor may keep a discharge. +func TestHardFloorDeadbandClosedLoop(t *testing.T) { + st := NewState(0, 50, "ferroamp") + st.Mode = ModeSelfConsumption + st.SlewEnabled = true + st.SlewRateW = 3000 + st.MinDispatchIntervalS = 0 + setpoint := -2000.0 + soc := 0.0505 + bad := 0 + for tick := 0; tick < 60; tick++ { + store := seedStore(2000+setpoint, []struct { + name string + currentW, soc float64 + }{{"bat", setpoint, soc}}) + for _, tg := range ComputeDispatch(store, st, caps(map[string]float64{"bat": 15200}), 11040) { + setpoint = tg.TargetW + } + if soc <= dischargeFloorSoC && setpoint < 0 { + bad++ + } + soc -= -setpoint / 10000 / 3600 * 2 // 10 kWh, 2 s tick + } + if bad > 0 { + t.Errorf("%d of 60 ticks kept a discharge at or under 5 %% (final SoC %.4f, setpoint %.0f W)", bad, soc, setpoint) + } +} + +// Fuse relief through the full safety pipeline, no soc_min: SoC exactly +// 5 % orders no discharge; 5.1 % may. +func TestFuseReliefPipelineAtHardFloor(t *testing.T) { + run := func(soc, curW float64) float64 { + store, st, cp := setupPerPhaseSite(3, 230, 13000, 19, 19, 19, soc, 5000) + if curW != 0 { + s := soc + store.Update("bat", telemetry.DerBattery, curW, &s, nil) + } + out := applyDispatchSafetyPipeline([]DispatchTarget{{Driver: "bat"}}, store, st, cp, 16*230*3, dispatchSafetyOptions{}) + return out[0].TargetW + } + for _, cur := range []float64{0, -5000} { + if got := run(0.05, cur); got < 0 { + t.Errorf("SoC exactly 5 %% (battery at %.0f W): ordered %.0f W", cur, got) + } + } + if got := run(0.051, 0); got >= 0 { + t.Errorf("SoC 5.1 %%: got %.0f W, want discharge", got) + } +} + +// floorStandbyWalk runs 300 ticks with a pack parked at its floor that +// reads a -15 W standby draw while the grid wanders inside the 50 W +// deadband. It returns how many ticks sent an order. +func floorStandbyWalk(st *State, soc float64) int { + sent := 0 + gridW := 0.0 + seed := uint32(7) + for tick := 0; tick < 300; tick++ { + seed = seed*1664525 + 1013904223 + gridW += float64(int(seed>>24)%21 - 10) + gridW = math.Max(-45, math.Min(45, gridW)) + store := seedStore(gridW, []struct { + name string + currentW, soc float64 + }{{"bat", -15, soc}}) + if len(ComputeDispatch(store, st, caps(map[string]float64{"bat": 15200}), 11040)) > 0 { + sent++ + } + } + return sent +} + +// A pack parked at Core's 5 % floor with standby draw is not discharging. +// The deadband holds, so the PI does not chase meter noise. +func TestHardFloorStandbyRandomWalk(t *testing.T) { + for _, soc := range []float64{0.05, 0.049} { + st := NewState(0, 50, "ferroamp") + st.Mode = ModeSelfConsumption + st.SlewEnabled = true + st.SlewRateW = 3000 + st.MinDispatchIntervalS = 0 + if sent := floorStandbyWalk(st, soc); sent != 0 { + t.Errorf("SoC %.3f, -15 W standby: %d of 300 ticks sent an order, want 0", soc, sent) + } + } +} + +// The same at the owner floor, in a planner mode with a fresh plan. +func TestOwnerFloorStandbyRandomWalk(t *testing.T) { + for _, energy := range []bool{false, true} { + st := NewState(0, 50, "ferroamp") + st.Mode = ModePlannerSelf + st.UseEnergyDispatch = energy + st.SlewEnabled = true + st.SlewRateW = 3000 + st.MinDispatchIntervalS = 0 + st.DriverLimits = map[string]PowerLimits{"bat": {SoCMin: 0.10}} + st.SlotDirective = func(now time.Time) (SlotDirective, bool) { + return SlotDirective{SlotStart: now, SlotEnd: now.Add(15 * time.Minute), Strategy: "self_consumption"}, true + } + st.PlanTarget = func(time.Time) (string, float64, string, bool) { return "self_consumption", 0, "self", true } + if sent := floorStandbyWalk(st, 0.10); sent != 0 { + t.Errorf("planner_self energy=%v, SoC 10 %%, -15 W standby: %d of 300 ticks sent an order, want 0", energy, sent) + } + } +} + +// A small discharge that FTW itself ordered still stops at the floor, +// though it reads under the standby threshold: the driver would hold it. +func TestHardFloorStopsOwnSmallDischarge(t *testing.T) { + st := NewState(0, 50, "ferroamp") + st.Mode = ModeSelfConsumption + st.SlewEnabled = true + st.SlewRateW = 3000 + st.MinDispatchIntervalS = 0 + st.PrevTargets["bat"] = -30 + store := seedStore(10, []struct { + name string + currentW, soc float64 + }{{"bat", -30, 0.05}}) + targets := ComputeDispatch(store, st, caps(map[string]float64{"bat": 15200}), 11040) + if len(targets) == 0 || targets[0].TargetW < 0 { + t.Fatalf("own -30 W order at 5 %%: %+v, want a 0 W order", targets) + } +} diff --git a/go/internal/control/testdata/golden/early_exit_protections.json b/go/internal/control/testdata/golden/early_exit_protections.json index e1147757e..c8e9929d2 100644 --- a/go/internal/control/testdata/golden/early_exit_protections.json +++ b/go/internal/control/testdata/golden/early_exit_protections.json @@ -775,7 +775,7 @@ "fuse_hold_latched": false, "fuse_hold_max_charge_w": 0, "fuse_hold_max_discharge_w": 0, - "per_phase_overage_w": 2300, + "per_phase_overage_w": 2070, "effective_import_ceiling_w": 11040, "effective_export_ceiling_w": 11040, "import_ceiling_binding": false, diff --git a/go/internal/control/testdata/golden/incident_scenarios.json b/go/internal/control/testdata/golden/incident_scenarios.json index ddee0ae59..ac2192c90 100644 --- a/go/internal/control/testdata/golden/incident_scenarios.json +++ b/go/internal/control/testdata/golden/incident_scenarios.json @@ -1,7 +1,7 @@ { "family": "incident_scenarios", "generated_by": "FTW_GOLDEN_DUMP=1 go test ./internal/control/ -run TestGoldenDump", - "ftw_commit": "15f9edc3b973d7b9151e88d63575a9b7d615a0b3", + "ftw_commit": "74d77a20d58cb902f5f62325520498143e2a2d19", "record_count": 27, "records": [ { @@ -428,7 +428,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -360, + "target_w": -324, "clamped": false } ], @@ -451,9 +451,9 @@ }, "site_totals": { "battery_current_w": 0, - "battery_target_sum_w": -360, + "battery_target_sum_w": -324, "raw_grid_w": 600, - "projected_grid_w": 240 + "projected_grid_w": 276 } }, { @@ -517,13 +517,7 @@ "remaining_s": 450 } }, - "per_driver_targets": [ - { - "driver": "ferroamp", - "target_w": -60, - "clamped": false - } - ], + "per_driver_targets": [], "clamp_attributions": { "clamped_drivers": [], "all_targets_zero_clamped": false, @@ -543,9 +537,9 @@ }, "site_totals": { "battery_current_w": 0, - "battery_target_sum_w": -60, + "battery_target_sum_w": 0, "raw_grid_w": 100, - "projected_grid_w": 40 + "projected_grid_w": 100 } }, { @@ -1592,7 +1586,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": 4000.0000033333336, + "target_w": 4000, "clamped": false } ], @@ -1615,9 +1609,9 @@ }, "site_totals": { "battery_current_w": 0, - "battery_target_sum_w": 4000.0000033333336, + "battery_target_sum_w": 4000, "raw_grid_w": -2000, - "projected_grid_w": 2000.0000033333336 + "projected_grid_w": 2000 } }, { @@ -2015,7 +2009,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -420, + "target_w": -384, "clamped": false } ], @@ -2038,9 +2032,9 @@ }, "site_totals": { "battery_current_w": 0, - "battery_target_sum_w": -420, + "battery_target_sum_w": -384, "raw_grid_w": 700, - "projected_grid_w": 280 + "projected_grid_w": 316 } }, { @@ -2101,7 +2095,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -420, + "target_w": -384, "clamped": false } ], @@ -2124,9 +2118,9 @@ }, "site_totals": { "battery_current_w": 0, - "battery_target_sum_w": -420, + "battery_target_sum_w": -384, "raw_grid_w": 700, - "projected_grid_w": 280 + "projected_grid_w": 316 } }, { diff --git a/go/internal/control/testdata/golden/seeded_ev_boost.json b/go/internal/control/testdata/golden/seeded_ev_boost.json index f4fd9c77b..89eadd288 100644 --- a/go/internal/control/testdata/golden/seeded_ev_boost.json +++ b/go/internal/control/testdata/golden/seeded_ev_boost.json @@ -1913,12 +1913,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": 588, + "target_w": 0, "clamped": false }, { "driver": "sungrow", - "target_w": 588, + "target_w": 0, "clamped": false } ], @@ -1941,9 +1941,9 @@ }, "site_totals": { "battery_current_w": -3117, - "battery_target_sum_w": 1176, + "battery_target_sum_w": 0, "raw_grid_w": -1466, - "projected_grid_w": 2827 + "projected_grid_w": 1651 } }, { @@ -2506,14 +2506,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": 5000, - "clamped": true + "target_w": 3950, + "clamped": false } ], "clamp_attributions": { - "clamped_drivers": [ - "ferroamp" - ], + "clamped_drivers": [], "all_targets_zero_clamped": false, "plan_sign_intent": 1, "plan_sign_floor_zeroed": false, @@ -2531,9 +2529,9 @@ }, "site_totals": { "battery_current_w": 1982, - "battery_target_sum_w": 5000, + "battery_target_sum_w": 3950, "raw_grid_w": -2307, - "projected_grid_w": 711 + "projected_grid_w": -339 } }, { diff --git a/go/internal/control/testdata/golden/seeded_fuse.json b/go/internal/control/testdata/golden/seeded_fuse.json index 0100f4d67..5f301eb6b 100644 --- a/go/internal/control/testdata/golden/seeded_fuse.json +++ b/go/internal/control/testdata/golden/seeded_fuse.json @@ -1,7 +1,7 @@ { "family": "seeded_fuse", "generated_by": "FTW_GOLDEN_DUMP=1 go test ./internal/control/ -run TestGoldenDump", - "ftw_commit": "303f86818a95f056f68e0b652db131a33074a421", + "ftw_commit": "a11706b4", "record_count": 80, "records": [ { @@ -69,20 +69,17 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -3245.228082551051, - "clamped": true + "target_w": -1530.5555555555554, + "clamped": false }, { "driver": "sungrow", - "target_w": -3647.771917448948, - "clamped": true + "target_w": -2326.4444444444443, + "clamped": false } ], "clamp_attributions": { - "clamped_drivers": [ - "ferroamp", - "sungrow" - ], + "clamped_drivers": [], "all_targets_zero_clamped": false, "plan_sign_intent": 0, "plan_sign_floor_zeroed": false, @@ -100,9 +97,9 @@ }, "site_totals": { "battery_current_w": 1432, - "battery_target_sum_w": -6892.999999999999, + "battery_target_sum_w": -3857, "raw_grid_w": 8815, - "projected_grid_w": 490.0000000000009 + "projected_grid_w": 3526 } }, { @@ -870,13 +867,13 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -2534.523809523809, - "clamped": true + "target_w": -2017.0238095238092, + "clamped": false }, { "driver": "pixii", - "target_w": -2534.523809523809, - "clamped": true + "target_w": -2017.0238095238092, + "clamped": false }, { "driver": "sungrow", @@ -886,8 +883,6 @@ ], "clamp_attributions": { "clamped_drivers": [ - "ferroamp", - "pixii", "sungrow" ], "all_targets_zero_clamped": false, @@ -907,9 +902,9 @@ }, "site_totals": { "battery_current_w": -3018, - "battery_target_sum_w": -10069.047619047618, + "battery_target_sum_w": -9034.047619047618, "raw_grid_w": 11913, - "projected_grid_w": 4861.952380952382 + "projected_grid_w": 5896.952380952382 } }, { @@ -1688,20 +1683,17 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -3402.6442494330904, - "clamped": true + "target_w": -1518.5290322580647, + "clamped": false }, { "driver": "sungrow", - "target_w": -3145.9557505669086, - "clamped": true + "target_w": -959.0709677419356, + "clamped": false } ], "clamp_attributions": { - "clamped_drivers": [ - "ferroamp", - "sungrow" - ], + "clamped_drivers": [], "all_targets_zero_clamped": false, "plan_sign_intent": 0, "plan_sign_floor_zeroed": false, @@ -1719,9 +1711,9 @@ }, "site_totals": { "battery_current_w": 2317, - "battery_target_sum_w": -6548.5999999999985, + "battery_target_sum_w": -2477.6000000000004, "raw_grid_w": 7991, - "projected_grid_w": -874.5999999999985 + "projected_grid_w": 3196.3999999999996 } }, { @@ -1883,17 +1875,17 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -644.0000000000001, + "target_w": -208.0000000000004, "clamped": true }, { "driver": "pixii", - "target_w": -644.0000000000001, + "target_w": -208.0000000000004, "clamped": true }, { "driver": "sungrow", - "target_w": -644.0000000000001, + "target_w": -208.0000000000004, "clamped": true } ], @@ -1920,9 +1912,9 @@ }, "site_totals": { "battery_current_w": 1308, - "battery_target_sum_w": -1932.0000000000005, + "battery_target_sum_w": -624.0000000000011, "raw_grid_w": 13971, - "projected_grid_w": 10731 + "projected_grid_w": 12038.999999999998 } }, { @@ -2480,20 +2472,17 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -2820.0999999999995, - "clamped": true + "target_w": -1026.1, + "clamped": false }, { "driver": "sungrow", - "target_w": -2820.0999999999995, - "clamped": true + "target_w": -1026.1, + "clamped": false } ], "clamp_attributions": { - "clamped_drivers": [ - "ferroamp", - "sungrow" - ], + "clamped_drivers": [], "all_targets_zero_clamped": false, "plan_sign_intent": 0, "plan_sign_floor_zeroed": false, @@ -2511,9 +2500,9 @@ }, "site_totals": { "battery_current_w": 4591, - "battery_target_sum_w": -5640.199999999999, + "battery_target_sum_w": -2052.2, "raw_grid_w": 11072, - "projected_grid_w": 840.8000000000011 + "projected_grid_w": 4428.8 } }, { @@ -4800,7 +4789,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -1550, + "target_w": -265.9999999999991, "clamped": true } ], @@ -4816,7 +4805,7 @@ "fuse_ev_max_w": 0, "fuse_hold_latched": true, "fuse_hold_max_charge_w": 0, - "fuse_hold_max_discharge_w": 1550, + "fuse_hold_max_discharge_w": 265.9999999999991, "per_phase_overage_w": 1150, "effective_import_ceiling_w": 4886, "effective_export_ceiling_w": 17250, @@ -4825,9 +4814,9 @@ }, "site_totals": { "battery_current_w": -3716, - "battery_target_sum_w": -1550, + "battery_target_sum_w": -265.9999999999991, "raw_grid_w": -15970, - "projected_grid_w": -13804 + "projected_grid_w": -12520 } }, { @@ -4895,20 +4884,17 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -5000, - "clamped": true + "target_w": -3102.7, + "clamped": false }, { "driver": "sungrow", - "target_w": -5000, - "clamped": true + "target_w": -3102.7, + "clamped": false } ], "clamp_attributions": { - "clamped_drivers": [ - "ferroamp", - "sungrow" - ], + "clamped_drivers": [], "all_targets_zero_clamped": false, "plan_sign_intent": 0, "plan_sign_floor_zeroed": false, @@ -4926,9 +4912,9 @@ }, "site_totals": { "battery_current_w": 2407, - "battery_target_sum_w": -10000, + "battery_target_sum_w": -6205.4, "raw_grid_w": 14354, - "projected_grid_w": 1947 + "projected_grid_w": 5741.6 } }, { @@ -5313,20 +5299,17 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -1645.1999999999998, - "clamped": true + "target_w": -782.6999999999998, + "clamped": false }, { "driver": "sungrow", - "target_w": -1645.1999999999998, - "clamped": true + "target_w": -782.6999999999998, + "clamped": false } ], "clamp_attributions": { - "clamped_drivers": [ - "ferroamp", - "sungrow" - ], + "clamped_drivers": [], "all_targets_zero_clamped": false, "plan_sign_intent": 0, "plan_sign_floor_zeroed": false, @@ -5344,9 +5327,9 @@ }, "site_totals": { "battery_current_w": 3297, - "battery_target_sum_w": -3290.3999999999996, + "battery_target_sum_w": -1565.3999999999996, "raw_grid_w": 8104, - "projected_grid_w": 1516.6000000000004 + "projected_grid_w": 3241.6000000000004 } }, { @@ -6513,17 +6496,17 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -2088.5, + "target_w": -1508.3333333333326, "clamped": true }, { "driver": "pixii", - "target_w": -2088.5, + "target_w": -1508.3333333333326, "clamped": true }, { "driver": "sungrow", - "target_w": -2088.5, + "target_w": -1508.3333333333326, "clamped": true } ], @@ -6541,7 +6524,7 @@ "fuse_ev_max_w": 0, "fuse_hold_latched": true, "fuse_hold_max_charge_w": 0, - "fuse_hold_max_discharge_w": 6265.5, + "fuse_hold_max_discharge_w": 4524.999999999998, "per_phase_overage_w": 0, "effective_import_ceiling_w": 16905, "effective_export_ceiling_w": 16905, @@ -6550,9 +6533,9 @@ }, "site_totals": { "battery_current_w": -4249, - "battery_target_sum_w": -6265.5, + "battery_target_sum_w": -4524.999999999998, "raw_grid_w": -14716, - "projected_grid_w": -16732.5 + "projected_grid_w": -14991.999999999998 } }, { @@ -7165,7 +7148,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -3174.000000000001, + "target_w": -449.0000000000014, "clamped": true } ], @@ -7190,9 +7173,9 @@ }, "site_totals": { "battery_current_w": 2725, - "battery_target_sum_w": -3174.000000000001, + "battery_target_sum_w": -449.0000000000014, "raw_grid_w": 14295, - "projected_grid_w": 8396 + "projected_grid_w": 11120.999999999998 } }, { @@ -7917,30 +7900,34 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": 5000, - "clamped": false + "target_w": 4608.333333333333, + "clamped": true }, { "driver": "pixii", - "target_w": 5000, - "clamped": false + "target_w": 4608.333333333333, + "clamped": true }, { "driver": "sungrow", - "target_w": 5000, - "clamped": false + "target_w": 4608.333333333333, + "clamped": true } ], "clamp_attributions": { - "clamped_drivers": [], + "clamped_drivers": [ + "ferroamp", + "pixii", + "sungrow" + ], "all_targets_zero_clamped": false, "plan_sign_intent": 0, "plan_sign_floor_zeroed": false, "plan_stale": false, "fuse_saturated": false, "fuse_ev_max_w": 0, - "fuse_hold_latched": false, - "fuse_hold_max_charge_w": 0, + "fuse_hold_latched": true, + "fuse_hold_max_charge_w": 13825, "fuse_hold_max_discharge_w": 0, "per_phase_overage_w": 0, "effective_import_ceiling_w": 10350, @@ -7950,9 +7937,9 @@ }, "site_totals": { "battery_current_w": 853, - "battery_target_sum_w": 15000, + "battery_target_sum_w": 13825, "raw_grid_w": -7899, - "projected_grid_w": 6248 + "projected_grid_w": 5073 } }, { @@ -8131,17 +8118,17 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -716.6666666666667, + "target_w": -0, "clamped": true }, { "driver": "pixii", - "target_w": -716.6666666666667, + "target_w": -0, "clamped": true }, { "driver": "sungrow", - "target_w": -716.6666666666667, + "target_w": -0, "clamped": true } ], @@ -8151,26 +8138,26 @@ "pixii", "sungrow" ], - "all_targets_zero_clamped": false, + "all_targets_zero_clamped": true, "plan_sign_intent": -1, - "plan_sign_floor_zeroed": false, + "plan_sign_floor_zeroed": true, "plan_stale": false, "fuse_saturated": false, "fuse_ev_max_w": 0, "fuse_hold_latched": true, "fuse_hold_max_charge_w": 0, - "fuse_hold_max_discharge_w": 2150, + "fuse_hold_max_discharge_w": 0, "per_phase_overage_w": 1173.0000000000002, "effective_import_ceiling_w": 17250, "effective_export_ceiling_w": 17250, "import_ceiling_binding": false, - "export_ceiling_binding": true + "export_ceiling_binding": false }, "site_totals": { "battery_current_w": 1653, - "battery_target_sum_w": -2150, + "battery_target_sum_w": 0, "raw_grid_w": -13447, - "projected_grid_w": -17250 + "projected_grid_w": -15100 } } ] diff --git a/go/internal/control/testdata/golden/seeded_planner.json b/go/internal/control/testdata/golden/seeded_planner.json index 68c8f64df..63c7aacee 100644 --- a/go/internal/control/testdata/golden/seeded_planner.json +++ b/go/internal/control/testdata/golden/seeded_planner.json @@ -1,7 +1,7 @@ { "family": "seeded_planner", "generated_by": "FTW_GOLDEN_DUMP=1 go test ./internal/control/ -run TestGoldenDump", - "ftw_commit": "15f9edc3b973d7b9151e88d63575a9b7d615a0b3", + "ftw_commit": "74d77a20d58cb902f5f62325520498143e2a2d19", "record_count": 100, "records": [ { @@ -452,7 +452,7 @@ "strategy": "self_consumption", "planned_grid_w": -2028, "has_planned_grid_w": true, - "live_pv_surplus_soc_cap": 0.93, + "live_pv_surplus_soc_cap": 0.9346559122157528, "elapsed_s": 357, "remaining_s": 689 } @@ -460,7 +460,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -4189, + "target_w": -4153, "clamped": false } ], @@ -483,9 +483,9 @@ }, "site_totals": { "battery_current_w": -1651, - "battery_target_sum_w": -4189, + "battery_target_sum_w": -4153, "raw_grid_w": 4230, - "projected_grid_w": 1692 + "projected_grid_w": 1728 } }, { @@ -835,7 +835,7 @@ "strategy": "self_consumption", "planned_grid_w": 2456, "has_planned_grid_w": true, - "live_pv_surplus_soc_cap": 0.86, + "live_pv_surplus_soc_cap": 0.8587472677682063, "elapsed_s": 110, "remaining_s": 220 } @@ -1438,7 +1438,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -3376.7441889520824, + "target_w": -3376.7441860465115, "clamped": false } ], @@ -1461,9 +1461,9 @@ }, "site_totals": { "battery_current_w": -465, - "battery_target_sum_w": -3376.7441889520824, + "battery_target_sum_w": -3376.7441860465115, "raw_grid_w": 1767, - "projected_grid_w": -1144.7441889520824 + "projected_grid_w": -1144.7441860465115 } }, { @@ -1536,12 +1536,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -4275.610107612486, + "target_w": -4275.610104181533, "clamped": false }, { "driver": "sungrow", - "target_w": -4453.760528763006, + "target_w": -4453.760525189096, "clamped": false } ], @@ -1564,9 +1564,9 @@ }, "site_totals": { "battery_current_w": 460, - "battery_target_sum_w": -8729.370636375492, + "battery_target_sum_w": -8729.37062937063, "raw_grid_w": 3414, - "projected_grid_w": -5775.370636375492 + "projected_grid_w": -5775.370629370629 } }, { @@ -1825,7 +1825,7 @@ "site_fuse_voltage": 0, "site_fuse_phases": 0, "site_fuse_safety_a": 0, - "pv_surplus_absorb_soc_cap": 0.84, + "pv_surplus_absorb_soc_cap": 0.8417180764157632, "pv_surplus_absorb_threshold_w": 0 }, "slot": { @@ -1842,12 +1842,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -462.4999999999991, + "target_w": -462.5, "clamped": true }, { "driver": "sungrow", - "target_w": -462.4999999999991, + "target_w": -462.5, "clamped": true } ], @@ -1864,7 +1864,7 @@ "fuse_ev_max_w": 0, "fuse_hold_latched": true, "fuse_hold_max_charge_w": 0, - "fuse_hold_max_discharge_w": 924.9999999999982, + "fuse_hold_max_discharge_w": 925, "per_phase_overage_w": 0, "effective_import_ceiling_w": 11040, "effective_export_ceiling_w": 11040, @@ -1873,9 +1873,9 @@ }, "site_totals": { "battery_current_w": 6081, - "battery_target_sum_w": -924.9999999999982, + "battery_target_sum_w": -925, "raw_grid_w": -4034, - "projected_grid_w": -11039.999999999998 + "projected_grid_w": -11040 } }, { @@ -2628,7 +2628,7 @@ "strategy": "passive_arbitrage", "planned_grid_w": 0, "has_planned_grid_w": false, - "live_pv_surplus_soc_cap": 0.92, + "live_pv_surplus_soc_cap": 0.9169474781726529, "elapsed_s": 392, "remaining_s": 582 } @@ -3222,12 +3222,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -2222.0129032258064, + "target_w": -2208.0774193548386, "clamped": false }, { "driver": "sungrow", - "target_w": -3518.1870967741934, + "target_w": -3496.122580645161, "clamped": false } ], @@ -3250,9 +3250,9 @@ }, "site_totals": { "battery_current_w": -3768, - "battery_target_sum_w": -5740.2, + "battery_target_sum_w": -5704.2, "raw_grid_w": 3287, - "projected_grid_w": 1314.8000000000002 + "projected_grid_w": 1350.8000000000002 } }, { @@ -3319,7 +3319,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": 3855.6213055737335, + "target_w": 3855.621301775148, "clamped": false } ], @@ -3342,9 +3342,9 @@ }, "site_totals": { "battery_current_w": -1381, - "battery_target_sum_w": 3855.6213055737335, + "battery_target_sum_w": 3855.621301775148, "raw_grid_w": -1360, - "projected_grid_w": 3876.6213055737335 + "projected_grid_w": 3876.621301775148 } }, { @@ -3403,7 +3403,7 @@ "strategy": "passive_arbitrage", "planned_grid_w": -600, "has_planned_grid_w": true, - "live_pv_surplus_soc_cap": 0.77, + "live_pv_surplus_soc_cap": 0.7709719503840469, "elapsed_s": 225, "remaining_s": 623 } @@ -3691,7 +3691,7 @@ "site_fuse_voltage": 0, "site_fuse_phases": 0, "site_fuse_safety_a": 0, - "pv_surplus_absorb_soc_cap": 0.73, + "pv_surplus_absorb_soc_cap": 0.7315028887007622, "pv_surplus_absorb_threshold_w": 0 }, "slot": { @@ -3798,7 +3798,7 @@ "strategy": "passive_arbitrage", "planned_grid_w": -23, "has_planned_grid_w": true, - "live_pv_surplus_soc_cap": 0.91, + "live_pv_surplus_soc_cap": 0.9066230931563072, "elapsed_s": 184, "remaining_s": 309 } @@ -3806,12 +3806,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -1306.9935483870968, + "target_w": -1293.058064516129, "clamped": false }, { "driver": "sungrow", - "target_w": -2069.4064516129033, + "target_w": -2047.341935483871, "clamped": false } ], @@ -3834,9 +3834,9 @@ }, "site_totals": { "battery_current_w": -1868, - "battery_target_sum_w": -3376.4, + "battery_target_sum_w": -3340.4, "raw_grid_w": 2514, - "projected_grid_w": 1005.5999999999999 + "projected_grid_w": 1041.6 } }, { @@ -4000,7 +4000,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -4334.4, + "target_w": -4298.4, "clamped": false } ], @@ -4023,9 +4023,9 @@ }, "site_totals": { "battery_current_w": -1776, - "battery_target_sum_w": -4334.4, + "battery_target_sum_w": -4298.4, "raw_grid_w": 4264, - "projected_grid_w": 1705.6000000000004 + "projected_grid_w": 1741.6000000000004 } }, { @@ -4396,7 +4396,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -1459.6000000000004, + "target_w": -1423.6000000000004, "clamped": false } ], @@ -4419,9 +4419,9 @@ }, "site_totals": { "battery_current_w": 2822, - "battery_target_sum_w": -1459.6000000000004, + "battery_target_sum_w": -1423.6000000000004, "raw_grid_w": 7136, - "projected_grid_w": 2854.3999999999996 + "projected_grid_w": 2890.3999999999996 } }, { @@ -4482,7 +4482,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -4655.2, + "target_w": -3800.2, "clamped": false } ], @@ -4505,9 +4505,9 @@ }, "site_totals": { "battery_current_w": -97, - "battery_target_sum_w": -4655.2, + "battery_target_sum_w": -3800.2, "raw_grid_w": 6232, - "projected_grid_w": 1673.8000000000002 + "projected_grid_w": 2528.8 } }, { @@ -4964,7 +4964,7 @@ "strategy": "passive_arbitrage", "planned_grid_w": 0, "has_planned_grid_w": false, - "live_pv_surplus_soc_cap": 0.6, + "live_pv_surplus_soc_cap": 0.600069383012964, "elapsed_s": 82, "remaining_s": 541 } @@ -5147,7 +5147,7 @@ "site_fuse_voltage": 0, "site_fuse_phases": 0, "site_fuse_safety_a": 0, - "pv_surplus_absorb_soc_cap": 0.82, + "pv_surplus_absorb_soc_cap": 0.8170109976831964, "pv_surplus_absorb_threshold_w": 0 }, "slot": { @@ -5465,7 +5465,7 @@ "strategy": "arbitrage", "planned_grid_w": 58, "has_planned_grid_w": true, - "live_pv_surplus_soc_cap": 0.69, + "live_pv_surplus_soc_cap": 0.6876649891490324, "elapsed_s": 85, "remaining_s": 572 } @@ -5551,7 +5551,7 @@ "strategy": "self_consumption", "planned_grid_w": 0, "has_planned_grid_w": false, - "live_pv_surplus_soc_cap": 0.84, + "live_pv_surplus_soc_cap": 0.8360543416220111, "elapsed_s": 582, "remaining_s": 307 } @@ -5559,7 +5559,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -2367.4, + "target_w": -2331.4, "clamped": false } ], @@ -5582,9 +5582,9 @@ }, "site_totals": { "battery_current_w": -1912, - "battery_target_sum_w": -2367.4, + "battery_target_sum_w": -2331.4, "raw_grid_w": 759, - "projected_grid_w": 303.5999999999999 + "projected_grid_w": 339.5999999999999 } }, { @@ -5645,7 +5645,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": 46, + "target_w": 82, "clamped": false } ], @@ -5668,9 +5668,9 @@ }, "site_totals": { "battery_current_w": 3184, - "battery_target_sum_w": 46, + "battery_target_sum_w": 82, "raw_grid_w": 5230, - "projected_grid_w": 2092 + "projected_grid_w": 2128 } }, { @@ -6196,7 +6196,7 @@ "site_fuse_voltage": 0, "site_fuse_phases": 0, "site_fuse_safety_a": 0, - "pv_surplus_absorb_soc_cap": 0.72, + "pv_surplus_absorb_soc_cap": 0.720847646748382, "pv_surplus_absorb_threshold_w": 0 }, "slot": { @@ -6213,12 +6213,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": 977.8064516129032, + "target_w": 977.8064516129025, "clamped": false }, { "driver": "sungrow", - "target_w": 1548.1935483870968, + "target_w": 1548.1935483870957, "clamped": false } ], @@ -6241,9 +6241,9 @@ }, "site_totals": { "battery_current_w": -2798, - "battery_target_sum_w": 2526, + "battery_target_sum_w": 2525.999999999998, "raw_grid_w": -3154, - "projected_grid_w": 2170 + "projected_grid_w": 2169.999999999998 } }, { @@ -6302,7 +6302,7 @@ "strategy": "self_consumption", "planned_grid_w": 2519, "has_planned_grid_w": true, - "live_pv_surplus_soc_cap": 0.63, + "live_pv_surplus_soc_cap": 0.626232970478788, "elapsed_s": 270, "remaining_s": 706 } @@ -6310,7 +6310,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -4185.6, + "target_w": -4149.6, "clamped": false } ], @@ -6333,9 +6333,9 @@ }, "site_totals": { "battery_current_w": -2544, - "battery_target_sum_w": -4185.6, + "battery_target_sum_w": -4149.6, "raw_grid_w": 2736, - "projected_grid_w": 1094.3999999999996 + "projected_grid_w": 1130.3999999999996 } }, { @@ -7070,7 +7070,7 @@ "site_fuse_voltage": 0, "site_fuse_phases": 0, "site_fuse_safety_a": 0, - "pv_surplus_absorb_soc_cap": 0.86, + "pv_surplus_absorb_soc_cap": 0.8640085831651351, "pv_surplus_absorb_threshold_w": 0 }, "slot": { @@ -7079,7 +7079,7 @@ "strategy": "self_consumption", "planned_grid_w": 0, "has_planned_grid_w": false, - "live_pv_surplus_soc_cap": 0.92, + "live_pv_surplus_soc_cap": 0.9237344109254809, "elapsed_s": 253, "remaining_s": 707 } @@ -7087,12 +7087,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -4500.558904109589, + "target_w": -4476.8876712328765, "clamped": false }, { "driver": "sungrow", - "target_w": -2344.041095890411, + "target_w": -2331.7123287671234, "clamped": false } ], @@ -7115,9 +7115,9 @@ }, "site_totals": { "battery_current_w": -3460, - "battery_target_sum_w": -6844.6, + "battery_target_sum_w": -6808.6, "raw_grid_w": 5641, - "projected_grid_w": 2256.3999999999996 + "projected_grid_w": 2292.3999999999996 } }, { @@ -7196,12 +7196,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -2370.9296403164058, + "target_w": -2370.9296384238523, "clamped": false }, { "driver": "sungrow", - "target_w": -2469.71837532959, + "target_w": -2469.7183733581796, "clamped": false } ], @@ -7224,9 +7224,9 @@ }, "site_totals": { "battery_current_w": 1577, - "battery_target_sum_w": -4840.648015645996, + "battery_target_sum_w": -4840.648011782032, "raw_grid_w": 6446, - "projected_grid_w": 28.351984354004344 + "projected_grid_w": 28.351988217968028 } }, { @@ -7279,7 +7279,7 @@ "strategy": "passive_arbitrage", "planned_grid_w": 0, "has_planned_grid_w": false, - "live_pv_surplus_soc_cap": 0.82, + "live_pv_surplus_soc_cap": 0.8242908060209763, "elapsed_s": 190, "remaining_s": 608 } @@ -7364,7 +7364,7 @@ "site_fuse_voltage": 0, "site_fuse_phases": 0, "site_fuse_safety_a": 0, - "pv_surplus_absorb_soc_cap": 0.79, + "pv_surplus_absorb_soc_cap": 0.7862699024545199, "pv_surplus_absorb_threshold_w": 0 }, "slot": { @@ -7465,7 +7465,7 @@ "strategy": "self_consumption", "planned_grid_w": 0, "has_planned_grid_w": false, - "live_pv_surplus_soc_cap": 0.91, + "live_pv_surplus_soc_cap": 0.906954730436936, "elapsed_s": 479, "remaining_s": 733 } @@ -7554,7 +7554,7 @@ "site_fuse_voltage": 0, "site_fuse_phases": 0, "site_fuse_safety_a": 0, - "pv_surplus_absorb_soc_cap": 0.78, + "pv_surplus_absorb_soc_cap": 0.779134585689955, "pv_surplus_absorb_threshold_w": 0 }, "slot": { @@ -7571,12 +7571,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -3743.903874385292, + "target_w": -3743.9038763163794, "clamped": true }, { "driver": "sungrow", - "target_w": -2125.0961256147093, + "target_w": -2125.0961236836206, "clamped": true } ], @@ -7593,7 +7593,7 @@ "fuse_ev_max_w": 0, "fuse_hold_latched": true, "fuse_hold_max_charge_w": 0, - "fuse_hold_max_discharge_w": 5869.000000000001, + "fuse_hold_max_discharge_w": 5869, "per_phase_overage_w": 0, "effective_import_ceiling_w": 11040, "effective_export_ceiling_w": 11040, @@ -7602,9 +7602,9 @@ }, "site_totals": { "battery_current_w": 4842, - "battery_target_sum_w": -5869.000000000002, + "battery_target_sum_w": -5869, "raw_grid_w": -329, - "projected_grid_w": -11040.000000000002 + "projected_grid_w": -11040 } }, { @@ -7884,7 +7884,7 @@ "strategy": "self_consumption", "planned_grid_w": 0, "has_planned_grid_w": false, - "live_pv_surplus_soc_cap": 0.67, + "live_pv_surplus_soc_cap": 0.6692593883812832, "elapsed_s": 90, "remaining_s": 718 } @@ -8304,12 +8304,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -4290.6138613861385, + "target_w": -4263.524752475248, "clamped": false }, { "driver": "sungrow", - "target_w": -1411.3861386138615, + "target_w": -1402.4752475247524, "clamped": false } ], @@ -8332,9 +8332,9 @@ }, "site_totals": { "battery_current_w": -1370, - "battery_target_sum_w": -5702, + "battery_target_sum_w": -5666, "raw_grid_w": 7220, - "projected_grid_w": 2888 + "projected_grid_w": 2924 } }, { @@ -8680,7 +8680,7 @@ "strategy": "self_consumption", "planned_grid_w": 0, "has_planned_grid_w": false, - "live_pv_surplus_soc_cap": 0.79, + "live_pv_surplus_soc_cap": 0.7909721045648893, "elapsed_s": 138, "remaining_s": 562 } @@ -8688,12 +8688,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -175.64081632653063, + "target_w": -158.00816326530614, "clamped": false }, { "driver": "sungrow", - "target_w": -182.9591836734694, + "target_w": -164.59183673469389, "clamped": false } ], @@ -8716,9 +8716,9 @@ }, "site_totals": { "battery_current_w": -19, - "battery_target_sum_w": -358.6, + "battery_target_sum_w": -322.6, "raw_grid_w": 566, - "projected_grid_w": 226.39999999999998 + "projected_grid_w": 262.4 } }, { @@ -8903,12 +8903,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -1721.7999999999997, + "target_w": -1709.7999999999997, "clamped": false }, { "driver": "sungrow", - "target_w": -3443.5999999999995, + "target_w": -3419.5999999999995, "clamped": false } ], @@ -8931,9 +8931,9 @@ }, "site_totals": { "battery_current_w": -1449, - "battery_target_sum_w": -5165.4, + "battery_target_sum_w": -5129.4, "raw_grid_w": 6194, - "projected_grid_w": 2477.6000000000004 + "projected_grid_w": 2513.6000000000004 } }, { @@ -9075,7 +9075,7 @@ "site_fuse_voltage": 0, "site_fuse_phases": 0, "site_fuse_safety_a": 0, - "pv_surplus_absorb_soc_cap": 0.77, + "pv_surplus_absorb_soc_cap": 0.7655297392494168, "pv_surplus_absorb_threshold_w": 0 }, "slot": { @@ -9092,7 +9092,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -1876.2, + "target_w": -1840.2, "clamped": false } ], @@ -9115,9 +9115,9 @@ }, "site_totals": { "battery_current_w": -1491, - "battery_target_sum_w": -1876.2, + "battery_target_sum_w": -1840.2, "raw_grid_w": 642, - "projected_grid_w": 256.79999999999995 + "projected_grid_w": 292.79999999999995 } }, { @@ -9378,7 +9378,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -4768.48030502767, + "target_w": -4768.480300187617, "clamped": false } ], @@ -9401,9 +9401,9 @@ }, "site_totals": { "battery_current_w": 3711, - "battery_target_sum_w": -4768.48030502767, + "battery_target_sum_w": -4768.480300187617, "raw_grid_w": 7142, - "projected_grid_w": -1337.4803050276696 + "projected_grid_w": -1337.480300187617 } }, { @@ -9545,7 +9545,7 @@ "site_fuse_voltage": 0, "site_fuse_phases": 0, "site_fuse_safety_a": 0, - "pv_surplus_absorb_soc_cap": 0.81, + "pv_surplus_absorb_soc_cap": 0.8143981399846733, "pv_surplus_absorb_threshold_w": 0 }, "legacy_plan": { @@ -9643,7 +9643,7 @@ "site_fuse_voltage": 0, "site_fuse_phases": 0, "site_fuse_safety_a": 0, - "pv_surplus_absorb_soc_cap": 0.73, + "pv_surplus_absorb_soc_cap": 0.7275713488711221, "pv_surplus_absorb_threshold_w": 0 }, "slot": { @@ -9660,12 +9660,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": 2361.854606161241, + "target_w": 2232.923446889023, "clamped": false }, { "driver": "sungrow", - "target_w": 2460.265214751293, + "target_w": 2325.9619238427326, "clamped": false } ], @@ -9688,9 +9688,9 @@ }, "site_totals": { "battery_current_w": 4391, - "battery_target_sum_w": 4822.119820912534, + "battery_target_sum_w": 4558.885370731756, "raw_grid_w": -835, - "projected_grid_w": -403.8801790874659 + "projected_grid_w": -667.1146292682442 } }, { diff --git a/go/internal/control/testdata/golden/seeded_siblings.json b/go/internal/control/testdata/golden/seeded_siblings.json index e6339aa3a..1433c45b8 100644 --- a/go/internal/control/testdata/golden/seeded_siblings.json +++ b/go/internal/control/testdata/golden/seeded_siblings.json @@ -1,7 +1,7 @@ { "family": "seeded_siblings", "generated_by": "FTW_GOLDEN_DUMP=1 go test ./internal/control/ -run TestGoldenDump", - "ftw_commit": "b3af71859a912dcae3a512f6c1ca79a21c2bb631", + "ftw_commit": "c41dc64bb5a70b8c838e50225566995f3331fa70", "record_count": 60, "records": [ { @@ -3774,13 +3774,14 @@ }, { "driver": "sungrow", - "target_w": -3200, - "clamped": false + "target_w": -5000, + "clamped": true } ], "clamp_attributions": { "clamped_drivers": [ - "ferroamp" + "ferroamp", + "sungrow" ], "all_targets_zero_clamped": false, "plan_sign_intent": -1, @@ -3799,9 +3800,9 @@ }, "site_totals": { "battery_current_w": -1969, - "battery_target_sum_w": -3200, + "battery_target_sum_w": -5000, "raw_grid_w": 5401, - "projected_grid_w": 4170 + "projected_grid_w": 2370 } }, { diff --git a/go/internal/control/testdata/golden/slew_limiter.json b/go/internal/control/testdata/golden/slew_limiter.json index 896d7f298..10f532cbb 100644 --- a/go/internal/control/testdata/golden/slew_limiter.json +++ b/go/internal/control/testdata/golden/slew_limiter.json @@ -1,7 +1,7 @@ { "family": "slew_limiter", "generated_by": "FTW_GOLDEN_DUMP=1 go test ./internal/control/ -run TestGoldenDump", - "ftw_commit": "15f9edc3b973d7b9151e88d63575a9b7d615a0b3", + "ftw_commit": "74d77a20d58cb902f5f62325520498143e2a2d19", "record_count": 127, "records": [ { @@ -2174,7 +2174,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -1500, + "target_w": 0, "clamped": true }, { @@ -2205,9 +2205,9 @@ }, "site_totals": { "battery_current_w": -2000, - "battery_target_sum_w": -2000, + "battery_target_sum_w": -500, "raw_grid_w": 2500, - "projected_grid_w": 2500 + "projected_grid_w": 4000 } }, { @@ -3142,7 +3142,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": 0, + "target_w": -0, "clamped": true } ], @@ -3225,7 +3225,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -2760, + "target_w": -1259.9999999999995, "clamped": true } ], @@ -3250,9 +3250,9 @@ }, "site_totals": { "battery_current_w": 1500, - "battery_target_sum_w": -2760, + "battery_target_sum_w": -1259.9999999999995, "raw_grid_w": 1000, - "projected_grid_w": -3260 + "projected_grid_w": -1759.9999999999995 } }, { @@ -4432,17 +4432,17 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -778.96, + "target_w": -764.5600000000001, "clamped": false }, { "driver": "pixii", - "target_w": -778.96, + "target_w": -764.5600000000001, "clamped": false }, { "driver": "sungrow", - "target_w": -389.48, + "target_w": -382.28000000000003, "clamped": false } ], @@ -4465,9 +4465,9 @@ }, "site_totals": { "battery_current_w": 467, - "battery_target_sum_w": -1947.4, + "battery_target_sum_w": -1911.4, "raw_grid_w": 4024, - "projected_grid_w": 1609.6 + "projected_grid_w": 1645.6 } }, { @@ -5741,7 +5741,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -2985.4, + "target_w": -2949.4, "clamped": false } ], @@ -5764,9 +5764,9 @@ }, "site_totals": { "battery_current_w": -3370, - "battery_target_sum_w": -2985.4, + "battery_target_sum_w": -2949.4, "raw_grid_w": -641, - "projected_grid_w": -256.4000000000001 + "projected_grid_w": -220.4000000000001 } }, { @@ -5999,7 +5999,7 @@ }, "site_totals": { "battery_current_w": 2929, - "battery_target_sum_w": 235.00000000000017, + "battery_target_sum_w": 235.0000000000001, "raw_grid_w": 13734, "projected_grid_w": 11040 } @@ -6954,7 +6954,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -1295.6, + "target_w": -1277.6, "clamped": false }, { @@ -6984,9 +6984,9 @@ }, "site_totals": { "battery_current_w": -376, - "battery_target_sum_w": -747.5999999999999, + "battery_target_sum_w": -729.5999999999999, "raw_grid_w": 3692, - "projected_grid_w": 3320.4 + "projected_grid_w": 3338.4 } }, { @@ -8807,7 +8807,7 @@ }, { "driver": "pixii", - "target_w": 396.1600000000001, + "target_w": 410.56000000000006, "clamped": false }, { @@ -8838,9 +8838,9 @@ }, "site_totals": { "battery_current_w": 3544, - "battery_target_sum_w": 2690.16, + "battery_target_sum_w": 2704.56, "raw_grid_w": 4256, - "projected_grid_w": 3402.16 + "projected_grid_w": 3416.56 } }, { @@ -10122,7 +10122,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -1749.8, + "target_w": -1713.8, "clamped": false } ], @@ -10145,9 +10145,9 @@ }, "site_totals": { "battery_current_w": -3026, - "battery_target_sum_w": -1749.8, + "battery_target_sum_w": -1713.8, "raw_grid_w": -2127, - "projected_grid_w": -850.8 + "projected_grid_w": -814.8 } }, { @@ -10653,7 +10653,7 @@ }, { "driver": "sungrow", - "target_w": 635.7936507936507, + "target_w": 0, "clamped": false } ], @@ -10678,9 +10678,9 @@ }, "site_totals": { "battery_current_w": -2923, - "battery_target_sum_w": 491.7936507936507, + "battery_target_sum_w": -144, "raw_grid_w": -1903, - "projected_grid_w": 1511.7936507936506 + "projected_grid_w": 876 } }, { @@ -10741,7 +10741,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -4261.8, + "target_w": -4225.8, "clamped": false } ], @@ -10764,9 +10764,9 @@ }, "site_totals": { "battery_current_w": -1650, - "battery_target_sum_w": -4261.8, + "battery_target_sum_w": -4225.8, "raw_grid_w": 4353, - "projected_grid_w": 1741.1999999999998 + "projected_grid_w": 1777.1999999999998 } }, { @@ -10974,7 +10974,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -1590.9333333333332, + "target_w": -1569.2190476190474, "clamped": false }, { @@ -11004,9 +11004,9 @@ }, "site_totals": { "battery_current_w": -543, - "battery_target_sum_w": -1135.9333333333332, + "battery_target_sum_w": -1114.2190476190474, "raw_grid_w": 3491, - "projected_grid_w": 2898.0666666666666 + "projected_grid_w": 2919.780952380953 } }, { diff --git a/go/internal/control/testdata/golden/targeted_invariants.json b/go/internal/control/testdata/golden/targeted_invariants.json index 467ca3418..2b59488b9 100644 --- a/go/internal/control/testdata/golden/targeted_invariants.json +++ b/go/internal/control/testdata/golden/targeted_invariants.json @@ -1,7 +1,7 @@ { "family": "targeted_invariants", "generated_by": "FTW_GOLDEN_DUMP=1 go test ./internal/control/ -run TestGoldenDump", - "ftw_commit": "68c5b6125898bb9da70728b8921259c87e96171d", + "ftw_commit": "74d77a20d58cb902f5f62325520498143e2a2d19", "record_count": 18, "records": [ { @@ -683,7 +683,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -3360, + "target_w": -2760, "clamped": true } ], @@ -708,9 +708,9 @@ }, "site_totals": { "battery_current_w": 0, - "battery_target_sum_w": -3360, + "battery_target_sum_w": -2760, "raw_grid_w": 1000, - "projected_grid_w": -2360 + "projected_grid_w": -1760 } }, { @@ -782,7 +782,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -2240, + "target_w": -0, "clamped": true } ], @@ -790,15 +790,15 @@ "clamped_drivers": [ "ferroamp" ], - "all_targets_zero_clamped": false, + "all_targets_zero_clamped": true, "plan_sign_intent": -1, - "plan_sign_floor_zeroed": false, + "plan_sign_floor_zeroed": true, "plan_stale": false, "fuse_saturated": false, "fuse_ev_max_w": 0, "fuse_hold_latched": true, "fuse_hold_max_charge_w": 0, - "fuse_hold_max_discharge_w": 2240, + "fuse_hold_max_discharge_w": 0, "per_phase_overage_w": 920, "effective_import_ceiling_w": 11040, "effective_export_ceiling_w": 11040, @@ -807,9 +807,9 @@ }, "site_totals": { "battery_current_w": -2000, - "battery_target_sum_w": -2240, + "battery_target_sum_w": 0, "raw_grid_w": -6000, - "projected_grid_w": -6240 + "projected_grid_w": -4000 } }, { @@ -1318,12 +1318,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": 2476.129032258064, + "target_w": 2476.1290322580644, "clamped": true }, { "driver": "sungrow", - "target_w": 1563.8709677419351, + "target_w": 1563.8709677419354, "clamped": true } ], @@ -1339,7 +1339,7 @@ "fuse_saturated": true, "fuse_ev_max_w": 6000, "fuse_hold_latched": true, - "fuse_hold_max_charge_w": 4039.999999999999, + "fuse_hold_max_charge_w": 4040, "fuse_hold_max_discharge_w": 0, "per_phase_overage_w": 0, "effective_import_ceiling_w": 11040, @@ -1349,7 +1349,7 @@ }, "site_totals": { "battery_current_w": 0, - "battery_target_sum_w": 4039.999999999999, + "battery_target_sum_w": 4040, "raw_grid_w": 7000, "projected_grid_w": 11040 } @@ -1430,12 +1430,12 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -3628.7625395418668, + "target_w": -3628.7625418060206, "clamped": true }, { "driver": "sungrow", - "target_w": -3371.2374604581337, + "target_w": -3371.2374581939803, "clamped": true } ], @@ -1461,9 +1461,9 @@ }, "site_totals": { "battery_current_w": -2000, - "battery_target_sum_w": -7000, + "battery_target_sum_w": -7000.000000000001, "raw_grid_w": -3000, - "projected_grid_w": -8000 + "projected_grid_w": -8000.000000000001 } }, { @@ -1602,7 +1602,7 @@ "per_driver_targets": [ { "driver": "ferroamp", - "target_w": -540, + "target_w": -510, "clamped": false } ], @@ -1625,9 +1625,9 @@ }, "site_totals": { "battery_current_w": 0, - "battery_target_sum_w": -540, + "battery_target_sum_w": -510, "raw_grid_w": 900, - "projected_grid_w": 360 + "projected_grid_w": 390 } } ] diff --git a/go/internal/drivers/driver_init_deadline_test.go b/go/internal/drivers/driver_init_deadline_test.go new file mode 100644 index 000000000..80e32abff --- /dev/null +++ b/go/internal/drivers/driver_init_deadline_test.go @@ -0,0 +1,86 @@ +package drivers + +import ( + "context" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +const loopingInitDriver = ` +function driver_init(config) + while true do end +end +function driver_poll() return 1000 end +function driver_command(action, w, cmd) return true end +function driver_default_mode() end +` + +func shortDriverInitTimeout(t *testing.T) { + t.Helper() + old := driverInitTimeout + driverInitTimeout = 200 * time.Millisecond + t.Cleanup(func() { driverInitTimeout = old }) +} + +// A driver_init that never returns must not block Reload. Reload strips the +// caller's cancellation, so only the init bound can end the loop. +func TestLoopingDriverInitDoesNotBlockReload(t *testing.T) { + shortDriverInitTimeout(t) + path := writeTestDriver(t, loopingInitDriver) + r := NewRegistry(telemetry.NewStore()) + done := make(chan struct{}) + go func() { + r.Reload(context.Background(), []config.Driver{{Name: "loopy", Lua: path}}, false) + close(done) + }() + select { + case <-done: + case <-time.After(5 * time.Second): + t.Fatal("Reload still blocked after 5 s by a looping driver_init") + } + if names := r.Names(); len(names) != 0 { + t.Fatalf("driver whose init timed out is registered: %v", names) + } +} + +// Same at boot: Add runs with the process context, which is never cancelled +// while Core runs. A timed-out init is an init failure. +func TestLoopingDriverInitFailsAdd(t *testing.T) { + shortDriverInitTimeout(t) + path := writeTestDriver(t, loopingInitDriver) + r := NewRegistry(telemetry.NewStore()) + done := make(chan error, 1) + go func() { done <- r.Add(context.Background(), config.Driver{Name: "loopy", Lua: path}) }() + select { + case err := <-done: + if err == nil { + t.Fatal("Add returned nil for a driver_init that never finished") + } + case <-time.After(5 * time.Second): + t.Fatal("Add still blocked after 5 s by a looping driver_init") + } + if names := r.Names(); len(names) != 0 { + t.Fatalf("driver whose init timed out is registered: %v", names) + } +} + +// A caller deadline shorter than the init bound still ends driver_init. +func TestLoopingDriverInitHonoursCallerDeadline(t *testing.T) { + path := writeTestDriver(t, loopingInitDriver) + r := NewRegistry(telemetry.NewStore()) + ctx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond) + defer cancel() + done := make(chan error, 1) + go func() { done <- r.Add(ctx, config.Driver{Name: "loopy", Lua: path}) }() + select { + case err := <-done: + if err == nil { + t.Fatal("Add returned nil after the caller's deadline") + } + case <-time.After(5 * time.Second): + t.Fatal("a cancelled ctx did not stop driver_init") + } +} diff --git a/go/internal/drivers/growatt_test.go b/go/internal/drivers/growatt_test.go new file mode 100644 index 000000000..1f65dd838 --- /dev/null +++ b/go/internal/drivers/growatt_test.go @@ -0,0 +1,47 @@ +package drivers + +import ( + "context" + "path/filepath" + "testing" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// A PV-only Growatt answers its battery registers with 0. It used to emit a +// battery anyway, and Overview drew a second battery at 0 W and 0 % beside +// the site's real one (srcfl/ftw#1669). A hybrid still reports its battery. +func TestGrowattEmitsABatteryOnlyWhenOneAnswers(t *testing.T) { + for _, tc := range []struct { + name string + regs map[uint16]uint16 + battery bool + }{ + {"PV only", map[uint16]uint16{2: 25000}, false}, + {"hybrid", map[uint16]uint16{2: 25000, 1010: 100, 1013: 480, 1014: 50}, true}, + {"hybrid at 0 %", map[uint16]uint16{2: 25000, 1013: 520}, true}, + } { + t.Run(tc.name, func(t *testing.T) { + tel := telemetry.NewStore() + env := NewHostEnv("growatt", tel).WithModbus(&pixiiTestModbus{regs: tc.regs}) + d, err := NewLuaDriver(filepath.Join("..", "..", "..", "drivers", "growatt.lua"), env) + if err != nil { + t.Fatalf("load Growatt driver: %v", err) + } + t.Cleanup(d.Cleanup) + if err := d.Init(context.Background(), nil); err != nil { + t.Fatalf("init: %v", err) + } + if _, err := d.Poll(context.Background()); err != nil { + t.Fatalf("poll: %v", err) + } + if pv := tel.Get("growatt", telemetry.DerPV); pv == nil || pv.RawW != -2500 { + t.Fatalf("PV reading %+v, want -2500 W", pv) + } + bat := tel.Get("growatt", telemetry.DerBattery) + if (bat != nil) != tc.battery { + t.Fatalf("battery reading %+v, want present=%v", bat, tc.battery) + } + }) + } +} diff --git a/go/internal/drivers/lua.go b/go/internal/drivers/lua.go index 8e83c6715..c629d8ad0 100644 --- a/go/internal/drivers/lua.go +++ b/go/internal/drivers/lua.go @@ -60,6 +60,11 @@ import ( // per-driver staleness window. Tesla's 5-minute BLE proxy fits; a year does not. const maxDriverWatchdogTimeout = 15 * time.Minute +// driverInitTimeout bounds driver_init. An init that runs past it fails like +// any other init error, so a looping or hung init cannot block boot, a reload +// or a settings save. +var driverInitTimeout = 30 * time.Second + // LuaDriver wraps a running Lua VM bound to a HostEnv. type LuaDriver struct { Env *HostEnv @@ -213,7 +218,7 @@ func (d *LuaDriver) callInitLocked(ctx context.Context) error { if d.initConfig != nil { arg = goToLua(d.L, d.initConfig) } - cleanup := d.setLifecycleContext(ctx) + cleanup := d.setLuaCallContext(ctx, driverInitTimeout) defer cleanup() return d.L.CallByParam(lua.P{Fn: fn, NRet: 0, Protect: true}, arg) } diff --git a/go/internal/drivers/registry.go b/go/internal/drivers/registry.go index 5b2769d16..8fb6dc8e1 100644 --- a/go/internal/drivers/registry.go +++ b/go/internal/drivers/registry.go @@ -61,8 +61,11 @@ func (e *commandMayHaveRunError) Is(target error) bool { const ( defaultRecoveryTimeout = 5 * time.Second - defaultRetryInitial = 100 * time.Millisecond - defaultRetryMax = 30 * time.Second + // commandRunTimeout is the least time a started command gets to finish. + // The caller's deadline bounds its wait, not the device's answer. + commandRunTimeout = defaultRecoveryTimeout + defaultRetryInitial = 100 * time.Millisecond + defaultRetryMax = 30 * time.Second ) // DriverControlStatus describes the safety state that the API may expose to @@ -122,6 +125,9 @@ type Registry struct { lifecycleHook func(name string) lifecycleMu sync.Mutex lifecycleGates map[string]*lifecycleGate + spawnRetries map[string]*spawnRetry + spawnInflight map[string]struct{} + spawnRetryClosed bool } // lifecycleGate serializes every lifecycle transition for one driver name. @@ -291,6 +297,10 @@ type runningDriver struct { evPauseRevision uint64 evPauseCycleID uint64 evPausePending bool + // Default-recovery backoff bounds; zero means defaultRetryInitial and + // defaultRetryMax. Tests set them. + retryInitial time.Duration + retryMax time.Duration // Poll loop coordination cmdCh chan driverCmd defaultCh chan driverCmd @@ -359,6 +369,26 @@ func (rd *runningDriver) markDefaultRecoveryPending() { rd.statusMu.Unlock() } +// commandRunContext gives a started command at least commandRunTimeout. A +// caller deadline shorter than that ends only the caller's wait; an explicit +// cancel still stops the command. +func commandRunContext(caller context.Context) (context.Context, func()) { + deadline, ok := caller.Deadline() + if !ok || time.Until(deadline) >= commandRunTimeout { + return caller, func() {} + } + run, cancel := context.WithTimeout(context.WithoutCancel(caller), commandRunTimeout) + stop := context.AfterFunc(caller, func() { + if errors.Is(caller.Err(), context.Canceled) { + cancel() + } + }) + return run, func() { + stop() + cancel() + } +} + func (rd *runningDriver) beginCommand(ctx context.Context) (context.Context, func()) { if ctx == nil { ctx = context.Background() @@ -436,9 +466,12 @@ func (s *commandState) snapshot() (err error, started, completed bool) { return err, started, completed } +// commandContextError says whether a command that returned success was cut +// short. The caller's deadline only ends its wait, so a success that arrives +// after it still counts; an explicit cancel does not. func commandContextError(requestCtx, executionCtx context.Context) error { if requestCtx != nil { - if err := requestCtx.Err(); err != nil { + if err := requestCtx.Err(); errors.Is(err, context.Canceled) { return err } } @@ -536,7 +569,7 @@ func legacyDriverRequiresDefaultMode(path string, hasCommand bool) (bool, error) // add is the shared driver construction path. The caller must hold the // lifecycle gate for cfg.Name. startupDefault is false only for connection // probes; operational drivers use the startup safety gate. -func (r *Registry) add(ctx context.Context, cfg config.Driver, startupDefault bool) error { +func (r *Registry) add(ctx context.Context, cfg config.Driver, startupDefault bool) (err error) { if ctx == nil { ctx = context.Background() } @@ -546,6 +579,15 @@ func (r *Registry) add(ctx context.Context, cfg config.Driver, startupDefault bo return fmt.Errorf("driver %q already registered", cfg.Name) } r.mu.Unlock() + if startupDefault { + prevDelay := r.takeSpawnRetry(cfg.Name) + retryCfg := cfg + defer func() { + if err != nil { + r.scheduleSpawnRetry(retryCfg, prevDelay, err) + } + }() + } if cfg.Lua == "" { return fmt.Errorf("driver %q: must specify `lua` path", cfg.Name) @@ -856,7 +898,17 @@ func (r *Registry) runLoop(rd *runningDriver) { } var recoveryTimer *time.Timer var recoveryC <-chan time.Time - retryDelay := defaultRetryInitial + retryInitial, retryMax := rd.retryInitial, rd.retryMax + if retryInitial <= 0 { + retryInitial = defaultRetryInitial + } + if retryMax <= 0 { + retryMax = defaultRetryMax + } + retryDelay := retryInitial + // pollFailed holds while the device's reads fail. The first good poll + // after it is the device's return. + pollFailed := false clearRecoveryTimer := func() { if recoveryTimer != nil { if !recoveryTimer.Stop() { @@ -867,12 +919,12 @@ func (r *Registry) runLoop(rd *runningDriver) { } } recoveryC = nil - retryDelay = defaultRetryInitial + retryDelay = retryInitial } scheduleRecovery := func() { d := retryDelay if d <= 0 { - d = defaultRetryInitial + d = retryInitial } if recoveryTimer == nil { recoveryTimer = time.NewTimer(d) @@ -887,10 +939,10 @@ func (r *Registry) runLoop(rd *runningDriver) { } recoveryC = recoveryTimer.C rd.markDefaultRecoveryPending() - if retryDelay < defaultRetryMax { + if retryDelay < retryMax { retryDelay *= 2 - if retryDelay > defaultRetryMax { - retryDelay = defaultRetryMax + if retryDelay > retryMax { + retryDelay = retryMax } } } @@ -1073,13 +1125,21 @@ func (r *Registry) runLoop(rd *runningDriver) { // own (manual ev_resume does not require a cycle id). invalidateCommandSequence() } - commandCtx, finishCommand := rd.beginCommand(cmdCtx) + // A wallbox-cycle step keeps its caller's deadline. Its owner gives + // up at that deadline and sends no resume, so a pause that lands + // late must fail and restore the default, not latch as pending. + runCtx, finishRun := cmdCtx, func() {} + if cmd.cycleID == 0 { + runCtx, finishRun = commandRunContext(cmdCtx) + } + commandCtx, finishCommand := rd.beginCommand(runCtx) // SendDefault may close the control window after the earlier queue // check but before this actor installs activeCancel. Recheck once the // cancel hook exists; after this point a racing default cancels the // context passed to the runtime. if rejection := rd.commandRejection(cmd.controlEpoch); rejection != nil { finishCommand() + finishRun() err = rejection break } @@ -1112,6 +1172,7 @@ func (r *Registry) runLoop(rd *runningDriver) { timer.Reset(rd.env.PollInterval()) } finishCommand() + finishRun() if err == nil && cyclePause { rd.evPausePending = true rd.evPauseRevision = rd.commandRevision @@ -1133,6 +1194,8 @@ func (r *Registry) runLoop(rd *runningDriver) { pollCtx, finishPoll := rd.beginCommand(ctx) returned, err := rd.driver.Poll(pollCtx) finishPoll() + returnedFromOutage := err == nil && pollFailed + pollFailed = err != nil if r.tel != nil { r.tel.CountPoll(err != nil) } @@ -1160,6 +1223,17 @@ func (r *Registry) runLoop(rd *runningDriver) { // Re-arm at the interval the poll returned, or the standing one. interval = rd.env.nextPollInterval(returned) timer.Reset(interval) + // A default that failed while the device was away is retried + // on its first good poll, not when the backoff (up to 30 s) + // next allows. Once per return: a device that reads but still + // refuses waits for the backoff again. + if returnedFromOutage && recoveryC != nil { + if err := attemptDefault(); err != nil { + slog.Error("driver default failed on its first good poll after an outage; control remains blocked", "name", rd.cfg.Name, "err", err) + } else { + slog.Info("driver default confirmed on its first good poll after an outage", "name", rd.cfg.Name) + } + } case <-recoveryC: if err := attemptDefault(); err != nil { slog.Error("driver default recovery failed; control remains blocked", "name", rd.cfg.Name, "err", err) @@ -1206,6 +1280,7 @@ func (r *Registry) remove(name string, skipDefault bool) { // lets Restart and Reload keep the old generation and its replacement in one // serialized transition instead of opening an Add/Remove gap. func (r *Registry) removeLocked(name string, skipDefault bool) { + r.takeSpawnRetry(name) skipDefault = skipDefault || r.controlBlocked() r.mu.Lock() rd, ok := r.rec[name] @@ -1441,6 +1516,11 @@ func (r *Registry) Names() []string { // ShutdownAll stops every driver. Blocks until all poll loops exit. func (r *Registry) ShutdownAll() { + // An attempt in flight holds its gate and tears its driver down itself + // once it sees the registry closed. Wait for it. + for _, n := range r.closeSpawnRetries() { + r.acquireLifecycle(n)() + } r.mu.Lock() names := make([]string, 0, len(r.rec)) for n := range r.rec { @@ -1467,6 +1547,14 @@ func (r *Registry) Reload(ctx context.Context, newDrivers []config.Driver, troub } active = append(active, d) } + // Stop retrying a driver that failed to start and has left the config. + for _, n := range r.pendingSpawnRetries() { + if _, ok := findDriver(active, n); !ok { + release := r.acquireLifecycle(n) + r.removeLocked(n, false) + release() + } + } r.mu.Lock() troubleshootingChanged := r.troubleshootingMode != troubleshootingMode diff --git a/go/internal/drivers/registry_default_on_return_test.go b/go/internal/drivers/registry_default_on_return_test.go new file mode 100644 index 000000000..6038533a9 --- /dev/null +++ b/go/internal/drivers/registry_default_on_return_test.go @@ -0,0 +1,206 @@ +package drivers + +import ( + "context" + "errors" + "sync" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// outageRuntime is a device that can go away. While down, polls and +// defaults fail. While refusing, defaults fail but polls succeed. +type outageRuntime struct { + env *HostEnv + + mu sync.Mutex + down bool + refusing bool + events []string +} + +func (r *outageRuntime) Init(context.Context, []byte) error { return nil } +func (r *outageRuntime) Poll(context.Context) (time.Duration, error) { + r.mu.Lock() + defer r.mu.Unlock() + if r.down { + r.events = append(r.events, "poll-fail") + return 0, errors.New("modbus reconnect backoff active") + } + r.events = append(r.events, "poll-ok") + return 0, nil +} +func (r *outageRuntime) Command(context.Context, []byte) error { return nil } +func (r *outageRuntime) DefaultMode(context.Context) error { + r.mu.Lock() + defer r.mu.Unlock() + if r.down || r.refusing { + r.events = append(r.events, "default-fail") + return errors.New("default write refused") + } + r.events = append(r.events, "default-ok") + return nil +} +func (r *outageRuntime) Cleanup(context.Context) error { return nil } +func (r *outageRuntime) Discard() {} +func (r *outageRuntime) Env() *HostEnv { return r.env } + +func (r *outageRuntime) set(down, refusing bool) { + r.mu.Lock() + r.down, r.refusing = down, refusing + r.mu.Unlock() +} + +func (r *outageRuntime) snapshot() []string { + r.mu.Lock() + defer r.mu.Unlock() + return append([]string(nil), r.events...) +} + +func (r *outageRuntime) count(event string) int { + n := 0 + for _, e := range r.snapshot() { + if e == event { + n++ + } + } + return n +} + +// waitFor polls until the runtime has recorded n events of a kind. +func (r *outageRuntime) waitFor(t *testing.T, event string, n int) { + t.Helper() + deadline := time.Now().Add(2 * time.Second) + for r.count(event) < n { + if time.Now().After(deadline) { + t.Fatalf("waited for %d %q, events = %v", n, event, r.snapshot()) + } + time.Sleep(time.Millisecond) + } +} + +// startOutageDriver runs a driver that polls every 10 ms and whose own +// default-recovery timer never fires within the test, so only the poll +// path can send a default. +func startOutageDriver(t *testing.T) (*Registry, *runningDriver, *outageRuntime) { + t.Helper() + tel := telemetry.NewStore() + r := NewRegistry(tel) + runtime := &outageRuntime{env: NewHostEnv("outage", tel)} + runtime.env.PollIntervalMS = 10 + lifecycleCtx, lifecycleCancel := context.WithCancel(context.Background()) + rd := &runningDriver{ + driver: runtime, + env: runtime.env, + cfg: config.Driver{Name: "outage"}, + generation: 1, + defaultConfirmed: true, + lifecycleCtx: lifecycleCtx, + lifecycleCancel: lifecycleCancel, + cmdCh: make(chan driverCmd, 8), + defaultCh: make(chan driverCmd, 1), + stop: make(chan bool, 1), + done: make(chan struct{}), + retryInitial: time.Hour, + retryMax: time.Hour, + } + r.rec[rd.cfg.Name] = rd + go r.runLoop(rd) + t.Cleanup(func() { r.remove(rd.cfg.Name, true) }) + return r, rd, runtime +} + +// failDefaultDuringOutage takes the device away and has the watchdog's +// default fail against it, leaving recovery pending. +func failDefaultDuringOutage(t *testing.T, r *Registry, runtime *outageRuntime) { + t.Helper() + runtime.set(true, false) + runtime.waitFor(t, "poll-fail", runtime.count("poll-fail")+1) + if err := r.SendDefault(context.Background(), "outage"); err == nil { + t.Fatal("SendDefault during outage succeeded, want refusal") + } + status, _ := r.ControlStatus("outage") + if !status.Blocked || !status.RecoveryPending || status.DefaultConfirmed { + t.Fatalf("status during outage = %+v, want blocked with recovery pending", status) + } + runtime.waitFor(t, "poll-fail", runtime.count("poll-fail")+2) +} + +// eventAfterFirstGoodPoll returns the event that followed the first +// poll-ok recorded after index from. +func eventAfterFirstGoodPoll(events []string, from int) string { + for i := from; i < len(events)-1; i++ { + if events[i] == "poll-ok" { + return events[i+1] + } + } + return "" +} + +func TestDefaultSentOnFirstGoodPollAfterOutage(t *testing.T) { + r, _, runtime := startOutageDriver(t) + failDefaultDuringOutage(t, r, runtime) + returnedAt := len(runtime.snapshot()) + + runtime.set(false, false) + runtime.waitFor(t, "default-ok", 1) + if got := eventAfterFirstGoodPoll(runtime.snapshot(), returnedAt); got != "default-ok" { + t.Fatalf("event after first good poll = %q, want default-ok; events = %v", got, runtime.snapshot()) + } + okPolls := runtime.count("poll-ok") + runtime.waitFor(t, "poll-ok", okPolls+5) + if n := runtime.count("default-ok") + runtime.count("default-fail"); n != 2 { + t.Fatalf("default calls = %d, want the failed one and one on return; events = %v", n, runtime.snapshot()) + } + status, _ := r.ControlStatus("outage") + if status.Blocked || !status.DefaultConfirmed || status.RecoveryPending { + t.Fatalf("status after return = %+v, want default confirmed", status) + } +} + +func TestDefaultRetriedOncePerReturnNotPerPoll(t *testing.T) { + r, _, runtime := startOutageDriver(t) + failDefaultDuringOutage(t, r, runtime) + + // The device reads again but still refuses the default: one retry on + // the return, then the backoff timer owns it. + runtime.set(false, true) + runtime.waitFor(t, "default-fail", 2) + okPolls := runtime.count("poll-ok") + runtime.waitFor(t, "poll-ok", okPolls+10) + if n := runtime.count("default-fail"); n != 2 { + t.Fatalf("failed defaults = %d, want one during the outage and one on return; events = %v", n, runtime.snapshot()) + } + status, _ := r.ControlStatus("outage") + if !status.Blocked || !status.RecoveryPending { + t.Fatalf("status while refusing = %+v, want still blocked", status) + } + + // A second outage is a second return, so the default goes out again. + runtime.set(true, false) + runtime.waitFor(t, "poll-fail", runtime.count("poll-fail")+2) + returnedAt := len(runtime.snapshot()) + runtime.set(false, false) + runtime.waitFor(t, "default-ok", 1) + if got := eventAfterFirstGoodPoll(runtime.snapshot(), returnedAt); got != "default-ok" { + t.Fatalf("event after second return = %q, want default-ok; events = %v", got, runtime.snapshot()) + } + status, _ = r.ControlStatus("outage") + if status.Blocked || !status.DefaultConfirmed || status.RecoveryPending { + t.Fatalf("status after second return = %+v, want default confirmed", status) + } +} + +func TestGoodPollWithoutPendingRecoverySendsNoDefault(t *testing.T) { + _, _, runtime := startOutageDriver(t) + runtime.set(true, false) + runtime.waitFor(t, "poll-fail", 2) + runtime.set(false, false) + runtime.waitFor(t, "poll-ok", runtime.count("poll-ok")+3) + if n := runtime.count("default-ok") + runtime.count("default-fail"); n != 0 { + t.Fatalf("default calls = %d, want none when no default was pending; events = %v", n, runtime.snapshot()) + } +} diff --git a/go/internal/drivers/registry_slow_command_test.go b/go/internal/drivers/registry_slow_command_test.go new file mode 100644 index 000000000..f81e2e4d9 --- /dev/null +++ b/go/internal/drivers/registry_slow_command_test.go @@ -0,0 +1,138 @@ +package drivers + +import ( + "context" + "errors" + "sync/atomic" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// slowOKRuntime accepts every command, but only after delay: a device that +// is slower than the caller's wait yet answers successfully. +type slowOKRuntime struct { + env *HostEnv + delay time.Duration + commands atomic.Int32 + defaults atomic.Int32 +} + +func (r *slowOKRuntime) Init(ctx context.Context, configJSON []byte) error { return nil } +func (r *slowOKRuntime) Poll(ctx context.Context) (time.Duration, error) { + return time.Hour, nil +} +func (r *slowOKRuntime) Command(ctx context.Context, cmdJSON []byte) error { + r.commands.Add(1) + time.Sleep(r.delay) // device I/O that ignores ctx, like a Lua socket write + return nil +} +func (r *slowOKRuntime) DefaultMode(ctx context.Context) error { + r.defaults.Add(1) + return nil +} +func (r *slowOKRuntime) Cleanup(ctx context.Context) error { return nil } +func (r *slowOKRuntime) Discard() {} +func (r *slowOKRuntime) Env() *HostEnv { return r.env } + +func newSlowOKRegistry(delay time.Duration) (*Registry, *slowOKRuntime) { + tel := telemetry.NewStore() + r := NewRegistry(tel) + rt := &slowOKRuntime{env: NewHostEnv("d1", tel), delay: delay} + rd := &runningDriver{ + driver: rt, + env: rt.env, + cfg: config.Driver{Name: "d1"}, + cmdCh: make(chan driverCmd, 1), + defaultCh: make(chan driverCmd, 1), + stop: make(chan bool, 1), + done: make(chan struct{}), + } + r.rec["d1"] = rd + go r.runLoop(rd) + return r, rt +} + +func TestSlowSuccessfulCommandAfterCallerDeadlineIsSuccess(t *testing.T) { + r, rt := newSlowOKRegistry(60 * time.Millisecond) + defer r.remove("d1", true) + + var outcome atomic.Value + done := make(chan struct{}) + // Caller waits 30 ms (the tick's interval/2 deadline, scaled down); the + // device accepts the command at 60 ms. + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Millisecond) + defer cancel() + sendErr := r.SendWithOutcome(ctx, "d1", []byte(`{"action":"battery","power_w":-2000}`), func(err error) { + if err != nil { + outcome.Store(err) + } + close(done) + }) + select { + case <-done: + case <-time.After(2 * time.Second): + t.Fatal("actor never completed the command") + } + t.Logf("send err = %v", sendErr) + if v := outcome.Load(); v != nil { + t.Logf("actor outcome = %v", v.(error)) + } + if n := rt.defaults.Load(); n != 0 || outcome.Load() != nil { + t.Fatalf("device accepted the command, yet outcome = %v and driver_default_mode ran %d time(s)", outcome.Load(), n) + } +} + +// Repeats the tick: each round the device accepts the setpoint and is then +// returned to its autonomous default. +func TestSlowSuccessfulCommandKeepsControlEveryTick(t *testing.T) { + r, rt := newSlowOKRegistry(40 * time.Millisecond) + defer r.remove("d1", true) + + for i := 0; i < 3; i++ { + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Millisecond) + err := r.Send(ctx, "d1", []byte(`{"action":"battery","power_w":-2000}`)) + cancel() + t.Logf("tick %d send err = %v (may-have-run=%v)", i, err, errors.Is(err, ErrCommandMayHaveRun)) + time.Sleep(80 * time.Millisecond) // rest of the tick + } + t.Logf("commands=%d defaults=%d", rt.commands.Load(), rt.defaults.Load()) + if rt.defaults.Load() != 0 { + t.Fatalf("every accepted setpoint was followed by driver_default_mode: commands=%d defaults=%d", rt.commands.Load(), rt.defaults.Load()) + } +} + +// A wallbox-cycle pause that succeeds after its caller gave up must not latch +// as pending: the cycle owner sends no resume, so the late pause restores the +// default and a later cycle can pause again. +func TestLateWallboxCyclePauseRestoresDefaultAndDoesNotLatch(t *testing.T) { + r, rt := newSlowOKRegistry(60 * time.Millisecond) + defer r.remove("d1", true) + r.tel.DriverHealthMut("d1").RecordSuccess() + + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Millisecond) + err := r.SendEVContinuation(ctx, "d1", []byte(`{"action":"ev_pause"}`), 1) + cancel() + if !errors.Is(err, ErrCommandMayHaveRun) { + t.Fatalf("cycle 1 pause err = %v, want may-have-run", err) + } + deadline := time.Now().Add(time.Second) + for time.Now().Before(deadline) && rt.defaults.Load() == 0 { + time.Sleep(5 * time.Millisecond) + } + if got := rt.defaults.Load(); got != 1 { + t.Fatalf("defaults after late cycle pause = %d, want 1", got) + } + + ctx2, cancel2 := context.WithTimeout(context.Background(), time.Second) + err = r.SendEVContinuation(ctx2, "d1", []byte(`{"action":"ev_pause"}`), 2) + cancel2() + if err != nil { + t.Fatalf("cycle 2 pause err = %v, want nil (commands=%d defaults=%d)", err, rt.commands.Load(), rt.defaults.Load()) + } + if got := rt.commands.Load(); got != 2 { + t.Fatalf("commands = %d, want 2", got) + } +} diff --git a/go/internal/drivers/registry_spawn_retry.go b/go/internal/drivers/registry_spawn_retry.go new file mode 100644 index 000000000..3bb3574be --- /dev/null +++ b/go/internal/drivers/registry_spawn_retry.go @@ -0,0 +1,131 @@ +package drivers + +import ( + "context" + "log/slog" + "time" + + "github.com/srcfl/ftw/go/internal/config" +) + +// A driver whose start fails (a serial node not yet enumerated, a device +// still booting after a power cut) is retried with this backoff until it +// starts, leaves the config, or the registry shuts down. Running drivers are +// never torn down here. +const ( + spawnRetryMinDelay = 10 * time.Second + spawnRetryMaxDelay = 5 * time.Minute +) + +// spawnRetry is one pending retry. The map entry is the claim: an Add, +// Restart, Reload or Remove for the same name replaces or deletes it under the +// lifecycle gate, so a timer that fires late finds a different entry and stops. +type spawnRetry struct { + cfg config.Driver + delay time.Duration + attempting bool + timer *time.Timer +} + +// takeSpawnRetry removes the pending retry for name and returns the delay the +// next failure should double, or 0 when this attempt did not come from the +// retry timer. The caller holds the lifecycle gate for name. +func (r *Registry) takeSpawnRetry(name string) time.Duration { + r.mu.Lock() + defer r.mu.Unlock() + e := r.spawnRetries[name] + if e == nil { + return 0 + } + delete(r.spawnRetries, name) + e.timer.Stop() + if e.attempting { + return e.delay + } + return 0 +} + +func (r *Registry) scheduleSpawnRetry(cfg config.Driver, prev time.Duration, cause error) { + delay := spawnRetryMinDelay + if prev > 0 { + delay = min(2*prev, spawnRetryMaxDelay) + } + r.mu.Lock() + defer r.mu.Unlock() + if r.spawnRetryClosed { + return + } + if r.spawnRetries == nil { + r.spawnRetries = make(map[string]*spawnRetry) + } + e := &spawnRetry{cfg: cfg, delay: delay} + e.timer = time.AfterFunc(delay, func() { r.retrySpawn(cfg.Name, e) }) + r.spawnRetries[cfg.Name] = e + slog.Warn("driver failed to start; will retry", "name", cfg.Name, "retry_in", delay, "err", cause) +} + +func (r *Registry) retrySpawn(name string, e *spawnRetry) { + release := r.acquireLifecycle(name) + defer release() + r.mu.Lock() + if r.spawnRetries[name] != e { + r.mu.Unlock() + return + } + e.attempting = true + // add deletes the map entry before the driver reaches r.rec. Until add + // returns, the in-flight mark is what lets Reload and ShutdownAll find + // this name and wait on its gate. + if r.spawnInflight == nil { + r.spawnInflight = make(map[string]struct{}) + } + r.spawnInflight[name] = struct{}{} + r.mu.Unlock() + err := r.add(context.Background(), e.cfg, true) + r.mu.Lock() + delete(r.spawnInflight, name) + closed := r.spawnRetryClosed + r.mu.Unlock() + if err != nil { + return + } + if closed { + // Shutdown began while this attempt ran and may have missed it. + r.removeLocked(name, false) + return + } + slog.Info("driver started on retry", "name", name) +} + +// pendingSpawnRetries lists names with a scheduled retry or one in flight. +func (r *Registry) pendingSpawnRetries() []string { + r.mu.Lock() + defer r.mu.Unlock() + out := make([]string, 0, len(r.spawnRetries)+len(r.spawnInflight)) + for n := range r.spawnRetries { + out = append(out, n) + } + for n := range r.spawnInflight { + if _, dup := r.spawnRetries[n]; !dup { + out = append(out, n) + } + } + return out +} + +// closeSpawnRetries stops every pending retry, refuses new ones and returns +// the names whose attempt is still in flight. +func (r *Registry) closeSpawnRetries() []string { + r.mu.Lock() + defer r.mu.Unlock() + r.spawnRetryClosed = true + for _, e := range r.spawnRetries { + e.timer.Stop() + } + r.spawnRetries = nil + inflight := make([]string, 0, len(r.spawnInflight)) + for n := range r.spawnInflight { + inflight = append(inflight, n) + } + return inflight +} diff --git a/go/internal/drivers/registry_spawn_retry_inflight_test.go b/go/internal/drivers/registry_spawn_retry_inflight_test.go new file mode 100644 index 000000000..3234ba270 --- /dev/null +++ b/go/internal/drivers/registry_spawn_retry_inflight_test.go @@ -0,0 +1,135 @@ +package drivers + +import ( + "context" + "errors" + "slices" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// blockingSerialRegistry fails the first open, then blocks every later open +// until the test sends a result. +func blockingSerialRegistry(t *testing.T) (*Registry, chan struct{}, chan error) { + t.Helper() + entered := make(chan struct{}, 4) + result := make(chan error) + first := true + r := NewRegistry(telemetry.NewStore()) + r.SerialFactory = func(name string, c *config.SerialConfig) (SerialCap, error) { + if first { + first = false + return nil, errors.New("no such device") + } + entered <- struct{}{} + if err := <-result; err != nil { + return nil, err + } + return fakeSerial{}, nil + } + t.Cleanup(r.ShutdownAll) + return r, entered, result +} + +// startInflightRetry fails a boot Add, then runs its retry until it blocks +// in the serial open. The returned channel closes when the retry returns. +func startInflightRetry(t *testing.T, r *Registry, cfg config.Driver, entered chan struct{}) chan struct{} { + t.Helper() + if err := r.Add(context.Background(), cfg); err == nil { + t.Fatal("expected first Add to fail") + } + e := pendingSpawnRetry(r, cfg.Name) + if e == nil { + t.Fatal("no retry scheduled") + } + e.timer.Stop() + done := make(chan struct{}) + go func() { r.retrySpawn(cfg.Name, e); close(done) }() + select { + case <-entered: + case <-time.After(5 * time.Second): + t.Fatal("retry did not reach the serial open") + } + return done +} + +// returnsWithin reports whether done closes within d. +func returnsWithin(done chan struct{}, d time.Duration) bool { + select { + case <-done: + return true + case <-time.After(d): + return false + } +} + +// The user disables a failing driver while a retry runs, and the retry then +// succeeds. The disabled driver must not be left running. +func TestReloadDuringInflightRetrySuccessStopsDriver(t *testing.T) { + r, entered, result := blockingSerialRegistry(t) + cfg := serialRetryDriver(t) + done := startInflightRetry(t, r, cfg, entered) + + reloadDone := make(chan struct{}) + go func() { + disabled := cfg + disabled.Disabled = true + r.Reload(context.Background(), []config.Driver{disabled}, false) + close(reloadDone) + }() + if returnsWithin(reloadDone, 300*time.Millisecond) { + t.Error("Reload returned while the retry attempt was still in flight") + } + result <- nil + <-done + <-reloadDone + if slices.Contains(r.Names(), "p1") { + t.Fatalf("driver disabled in config is running after an in-flight retry: %v", r.Names()) + } + if e := pendingSpawnRetry(r, "p1"); e != nil { + t.Fatalf("retry scheduled for a disabled driver: %+v", e) + } +} + +// Same, but the retry fails. No further retry may be scheduled. +func TestReloadDuringInflightRetryFailureStopsRetrying(t *testing.T) { + r, entered, result := blockingSerialRegistry(t) + cfg := serialRetryDriver(t) + done := startInflightRetry(t, r, cfg, entered) + + reloadDone := make(chan struct{}) + go func() { + r.Reload(context.Background(), nil, false) + close(reloadDone) + }() + returnsWithin(reloadDone, 300*time.Millisecond) + result <- errors.New("still missing") + <-done + <-reloadDone + if e := pendingSpawnRetry(r, "p1"); e != nil { + t.Fatalf("retry still scheduled (in %v) for a driver removed from config", e.delay) + } +} + +// ShutdownAll blocks until all poll loops exit, so it must wait for a retry +// that may still start a driver. +func TestShutdownAllWaitsForInflightRetry(t *testing.T) { + r, entered, result := blockingSerialRegistry(t) + cfg := serialRetryDriver(t) + done := startInflightRetry(t, r, cfg, entered) + + shutDone := make(chan struct{}) + go func() { r.ShutdownAll(); close(shutDone) }() + if returnsWithin(shutDone, 300*time.Millisecond) { + t.Error("ShutdownAll returned while a retry attempt was still starting the driver") + } + result <- nil + <-done + <-shutDone + if slices.Contains(r.Names(), "p1") { + t.Fatalf("driver running after shutdown: %v", r.Names()) + } +} diff --git a/go/internal/drivers/registry_spawn_retry_test.go b/go/internal/drivers/registry_spawn_retry_test.go new file mode 100644 index 000000000..c0b4350af --- /dev/null +++ b/go/internal/drivers/registry_spawn_retry_test.go @@ -0,0 +1,168 @@ +package drivers + +import ( + "context" + "path/filepath" + "slices" + "sync/atomic" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +type fakeSerial struct{} + +func (fakeSerial) Read(int, time.Duration) ([]byte, error) { return nil, nil } +func (fakeSerial) Close() error { return nil } + +func serialRetryRegistry(t *testing.T, present *atomic.Bool) *Registry { + t.Helper() + r := NewRegistry(telemetry.NewStore()) + r.SerialFactory = func(name string, c *config.SerialConfig) (SerialCap, error) { + if !present.Load() { + return OpenSerial(c) // real error for a missing node + } + return fakeSerial{}, nil + } + t.Cleanup(r.ShutdownAll) + return r +} + +func serialRetryDriver(t *testing.T) config.Driver { + t.Helper() + missing := filepath.Join(t.TempDir(), "ttyUSB0") + return config.Driver{ + Name: "p1", + Lua: writeTestDriver(t, registryRestartTestDriver), + Capabilities: config.Capabilities{ + Serial: &config.SerialConfig{Address: missing, BaudRate: 115200, ReadTimeoutMS: 1000}, + }, + } +} + +// A serial site meter whose device node is missing at boot must come back +// on its own once the device appears. The boot loop calls reg.Add once. +func TestBootSpawnFailureIsRetriedWhenSerialDeviceAppears(t *testing.T) { + var present atomic.Bool + r := serialRetryRegistry(t, &present) + cfg := serialRetryDriver(t) + + // Same call the boot loop makes in cmd/ftw. + err := r.Add(context.Background(), cfg) + if err == nil { + t.Fatal("expected boot Add to fail while the device is missing") + } + t.Logf("boot Add error: %v", err) + + present.Store(true) // USB enumerates a moment later. + + deadline := time.Now().Add(15 * time.Second) + for time.Now().Before(deadline) { + if slices.Contains(r.Names(), "p1") { + return + } + time.Sleep(100 * time.Millisecond) + } + t.Fatalf("driver p1 still not registered 15s after its serial device appeared; registered=%v", r.Names()) +} + +// An explicit Reload (config save) also recovers the driver at once. +func TestReloadRecoversFailedBootSpawn(t *testing.T) { + var present atomic.Bool + r := serialRetryRegistry(t, &present) + cfg := serialRetryDriver(t) + if err := r.Add(context.Background(), cfg); err == nil { + t.Fatal("expected boot Add to fail while the device is missing") + } + present.Store(true) + r.Reload(context.Background(), []config.Driver{cfg}, false) + if !slices.Contains(r.Names(), "p1") { + t.Fatalf("Reload did not add p1; registered=%v", r.Names()) + } +} + +func pendingSpawnRetry(r *Registry, name string) *spawnRetry { + r.mu.Lock() + defer r.mu.Unlock() + return r.spawnRetries[name] +} + +// Each failed retry doubles the wait up to the cap. A user Add starts again +// from the shortest wait. Leaving the config, Remove and shutdown end retries. +func TestSpawnRetryBacksOffAndStopsWhenDriverLeaves(t *testing.T) { + var present atomic.Bool + r := serialRetryRegistry(t, &present) + cfg := serialRetryDriver(t) + if err := r.Add(context.Background(), cfg); err == nil { + t.Fatal("expected Add to fail while the device is missing") + } + + want := spawnRetryMinDelay + for range 8 { + e := pendingSpawnRetry(r, "p1") + if e == nil || e.delay != want { + t.Fatalf("retry delay = %v, want %v", e, want) + } + r.retrySpawn("p1", e) + want = min(2*want, spawnRetryMaxDelay) + } + if e := pendingSpawnRetry(r, "p1"); e == nil || e.delay != spawnRetryMaxDelay { + t.Fatalf("retry delay after many failures = %v, want cap %v", e, spawnRetryMaxDelay) + } + + // A stale timer that fires after a newer attempt must do nothing. + stale := pendingSpawnRetry(r, "p1") + if err := r.Add(context.Background(), cfg); err == nil { + t.Fatal("expected Add to fail while the device is missing") + } + if e := pendingSpawnRetry(r, "p1"); e == nil || e == stale || e.delay != spawnRetryMinDelay { + t.Fatalf("user Add did not restart backoff: %+v", e) + } + present.Store(true) + r.retrySpawn("p1", stale) + if slices.Contains(r.Names(), "p1") { + t.Fatal("stale retry started the driver") + } + + present.Store(false) + r.Reload(context.Background(), nil, false) + if e := pendingSpawnRetry(r, "p1"); e != nil { + t.Fatal("retry kept after the driver left the config") + } + + if err := r.Add(context.Background(), cfg); err == nil { + t.Fatal("expected Add to fail while the device is missing") + } + r.Remove("p1") + if e := pendingSpawnRetry(r, "p1"); e != nil { + t.Fatal("retry kept after Remove") + } + + if err := r.Add(context.Background(), cfg); err == nil { + t.Fatal("expected Add to fail while the device is missing") + } + r.ShutdownAll() + if e := pendingSpawnRetry(r, "p1"); e != nil { + t.Fatal("retry kept after ShutdownAll") + } + if err := r.Add(context.Background(), cfg); err == nil { + t.Fatal("expected Add to fail while the device is missing") + } + if e := pendingSpawnRetry(r, "p1"); e != nil { + t.Fatal("retry scheduled after ShutdownAll") + } +} + +// A probe never schedules a retry. +func TestProbeFailureIsNotRetried(t *testing.T) { + var present atomic.Bool + r := serialRetryRegistry(t, &present) + if err := r.AddProbe(context.Background(), serialRetryDriver(t)); err == nil { + t.Fatal("expected probe to fail while the device is missing") + } + if e := pendingSpawnRetry(r, "p1"); e != nil { + t.Fatal("probe failure scheduled a retry") + } +} diff --git a/go/internal/drivers/sungrow_driver_test.go b/go/internal/drivers/sungrow_driver_test.go index 86b5ac5a0..82d898d3b 100644 --- a/go/internal/drivers/sungrow_driver_test.go +++ b/go/internal/drivers/sungrow_driver_test.go @@ -3,6 +3,8 @@ package drivers import ( "context" "encoding/json" + "errors" + "fmt" "strings" "testing" "time" @@ -355,3 +357,92 @@ func TestSungrowProbesTheHybridBlockWhenTheDeviceTypeNeverAnswers(t *testing.T) "running state; the probing fallback is not working") } } + +// offlineModbus fails every read and write the way FTW's Modbus client does +// while the inverter is unreachable: the dial error the site rig logged for +// reads, and the reconnect backoff for writes. +type offlineModbus struct { + *recordingModbus + offline bool +} + +func (m *offlineModbus) Read(addr uint16, count uint16, kind int32) ([]uint16, error) { + if m.offline { + return nil, errors.New("dial tcp 127.0.0.1:502: connect: connection refused") + } + return m.recordingModbus.Read(addr, count, kind) +} + +func (m *offlineModbus) WriteSingle(addr uint16, value uint16) error { + if m.offline { + return fmt.Errorf("%w for 1.243s", ErrModbusBackoff) + } + return m.recordingModbus.WriteSingle(addr, value) +} + +func (m *offlineModbus) WriteMulti(addr uint16, values []uint16) error { + if m.offline { + return fmt.Errorf("%w for 1.243s", ErrModbusBackoff) + } + return m.recordingModbus.WriteMulti(addr, values) +} + +// An inverter that drops off the network for a while is not an inverter +// without registers. The random site rig (27 October 2026) took the Sungrow +// offline while it held a forced discharge. sungrow 1.6.0 counted the failed +// writes as the device refusing the EMS block, then reported the release as +// held without writing it, so the inverter kept discharging for up to five +// minutes after it came back. It also counted the failed reads as missing +// registers and dropped 13000, which signs the battery. +func TestSungrowOutageKeepsTheReleaseAndTheBatterySign(t *testing.T) { + tel := telemetry.NewStore() + modbus := &offlineModbus{recordingModbus: newRecordingModbus()} + modbus.regs[4999] = 0x0E0E // SH10RT + modbus.regs[13000] = 0x0004 + modbus.regs[13021] = 900 // battery power, unsigned; 13000 says discharging + modbus.regs[13022] = 550 + env := NewHostEnv("sungrow", tel).WithModbus(modbus) + env.BatteryCapacityWh = 9600 + d, err := NewLuaDriver("../../../drivers/sungrow.lua", env) + if err != nil { + t.Fatalf("load: %v", err) + } + defer d.Cleanup() + ctx := context.Background() + + if _, err := d.Poll(ctx); err != nil { + t.Fatalf("poll: %v", err) + } + cmd, _ := json.Marshal(map[string]any{"action": "battery", "power_w": -3000}) + if err := d.Command(ctx, cmd); err != nil { + t.Fatalf("forced discharge: %v", err) + } + + modbus.offline = true + for i := 0; i < 6; i++ { + untilNextFullRead(env) + _, _ = d.Poll(ctx) + if err := d.DefaultMode(); err == nil { + t.Fatalf("default mode %d reported success while nothing reached the device", i) + } + } + modbus.offline = false + + if err := d.DefaultMode(); err != nil { + t.Fatalf("default mode after the device returned: %v", err) + } + if got := modbus.regs[13049]; got != 0 { + t.Fatalf("EMS mode = %d after the device returned, want 0: the "+ + "release was skipped and the forced discharge kept running", got) + } + + untilNextFullRead(env) + if _, err := d.Poll(ctx); err != nil { + t.Fatalf("poll after the device returned: %v", err) + } + reading := tel.Get("sungrow", telemetry.DerBattery) + if reading == nil || reading.RawW != -900 { + t.Fatalf("battery reading = %+v, want -900 W: 13000, which signs it, "+ + "was dropped during the outage", reading) + } +} diff --git a/go/internal/drivers/tesla_wall_connector_test.go b/go/internal/drivers/tesla_wall_connector_test.go index 53f348627..4e79f01d8 100644 --- a/go/internal/drivers/tesla_wall_connector_test.go +++ b/go/internal/drivers/tesla_wall_connector_test.go @@ -109,14 +109,14 @@ func TestTeslaWallConnectorInitPollAndReadonlyCommands(t *testing.T) { t.Errorf("session_wh=%v, want 4200", extra["session_wh"]) } - if err := d.Command(context.Background(), []byte(`{"action":"ev_set_current","power_w":11040}`)); err != nil { - t.Fatalf("ev_set_current should ack: %v", err) - } - if err := d.Command(context.Background(), []byte(`{"action":"ev_pause"}`)); err != nil { - t.Fatalf("ev_pause should ack: %v", err) - } - if err := d.Command(context.Background(), []byte(`{"action":"ev_resume"}`)); err != nil { - t.Fatalf("ev_resume should ack: %v", err) + // The wall connector has no local setpoint; the catalog marks it + // read_only, so Core refuses control before it reaches the driver. + for _, cmd := range []string{ + `{"action":"ev_set_current","power_w":11040}`, `{"action":"ev_pause"}`, `{"action":"ev_resume"}`, + } { + if err := d.Command(context.Background(), []byte(cmd)); err == nil || !strings.Contains(err.Error(), "read_only") { + t.Fatalf("%s: err=%v, want the read_only refusal", cmd, err) + } } if err := d.DefaultMode(); err != nil { t.Fatalf("default mode: %v", err) @@ -349,15 +349,12 @@ func TestTeslaWallConnectorVitalsFailureKeepsDriver(t *testing.T) { } func TestTeslaWallConnectorPhaseFallbackToGrid(t *testing.T) { - // No usable phase voltages → grid_v × vehicle_current_a. + // No leg voltages at all → grid_v × vehicle_current_a. fake := &teslaWCFake{vitalsBody: []byte(`{ "contactor_closed": true, "vehicle_connected": true, "grid_v": 230, "vehicle_current_a": 10, - "voltageA_v": 2, - "voltageB_v": 1, - "voltageC_v": 0, "session_energy_wh": 1, "evse_state": 11 }`)} @@ -366,3 +363,42 @@ func TestTeslaWallConnectorPhaseFallbackToGrid(t *testing.T) { t.Errorf("EV W=%v, want 2300 fallback", reading.RawW) } } + +func TestTeslaWallConnectorLegsAtZeroCarryNoPower(t *testing.T) { + // The legs are measured on the car side of the contactor. Legs that + // read 0–2 V carry nothing, whatever grid_v says. + fake := &teslaWCFake{vitalsBody: []byte(`{ + "contactor_closed": true, + "vehicle_connected": true, + "grid_v": 230, + "vehicle_current_a": 10, + "voltageA_v": 2, + "voltageB_v": 1, + "voltageC_v": 0, + "session_energy_wh": 1, + "evse_state": 11 + }`)} + if reading := teslaWCPoll(t, fake, nil); reading.RawW != 0 { + t.Errorf("EV W=%v, want 0 through legs at 0 V", reading.RawW) + } +} + +func TestTeslaWallConnectorOpenContactorIsIdle(t *testing.T) { + // An idle unit reads 0.5–1 A of sensor offset on its legs with the + // contactor open; that is not load. + fake := &teslaWCFake{vitalsBody: []byte(`{ + "contactor_closed": false, + "vehicle_connected": false, + "grid_v": 230, + "vehicle_current_a": 0.8, + "currentA_a": 0.8, + "currentB_a": 0.6, + "voltageA_v": 0, + "voltageB_v": 0, + "session_energy_wh": 0, + "evse_state": 1 + }`)} + if reading := teslaWCPoll(t, fake, nil); reading.RawW != 0 { + t.Errorf("EV W=%v, want 0 with the contactor open", reading.RawW) + } +} diff --git a/go/internal/drivers/testdata/tesla_wall_connector.lua b/go/internal/drivers/testdata/tesla_wall_connector.lua index d7633fd1b..85e67b739 100644 --- a/go/internal/drivers/testdata/tesla_wall_connector.lua +++ b/go/internal/drivers/testdata/tesla_wall_connector.lua @@ -23,12 +23,13 @@ DRIVER = { host_api_min = 1, host_api_max = 1, - id = "tesla-wall-connector", + id = "tesla_wall_connector", name = "Tesla Wall Connector", manufacturer = "Tesla", - version = "0.1.0", + version = "0.1.2", protocols = { "http" }, capabilities = { "ev" }, + read_only = true, description = "Tesla Wall Connector Gen 3 on the LAN. Observation only; no cloud account.", homepage = "https://www.tesla.com/support/charging/wall-connector", authors = { "FTW contributors" }, @@ -91,11 +92,18 @@ end local function decode_json(raw) if raw == nil or raw == "" then return nil, "empty body" end - local data, err = host.json_decode(repair_json(raw)) - if data == nil then return nil, tostring(err or "decode failed") end + local ok, data = pcall(host.json_decode, repair_json(raw)) + if not ok then return nil, tostring(data) end + if data == nil then return nil, "decode failed" end return data, nil end +local function safe_http_get(url, headers) + local ok, resp, err = pcall(host.http_get, url, headers) + if not ok then return nil, tostring(resp) end + return resp, err +end + local function origin_from_config(config) if config.base_url and tostring(config.base_url) ~= "" then return tostring(config.base_url):gsub("/$", "") @@ -106,6 +114,11 @@ local function origin_from_config(config) if host:find("://", 1, true) then return host:gsub("/$", "") end + -- Optional config.port; scheme-only URL uses HTTP default 80. + local port = tonumber(config.port) + if port and port > 0 and port ~= 80 then + return "http://" .. host .. ":" .. tostring(port) + end return "http://" .. host end @@ -118,20 +131,31 @@ local function phase_power(v, a) return v * a end -local function charge_power(vitals) - if split_phase then - local w = (as_number(vitals.grid_v) or 0) * (as_number(vitals.vehicle_current_a) or 0) - if w < 0 then w = 0 end - return w +-- The contactor joins the car to the grid and closes only with a car on +-- the cable. While it is open no current reaches a car: an idle unit still +-- reads about 0.5–1 A of sensor offset on its legs, and that is not load. +-- A reply without contactor_closed falls back to vehicle_connected and +-- the EVSE state. +local function contactor_closed(state, vitals) + if vitals.contactor_closed ~= nil then return vitals.contactor_closed == true end + if vitals.vehicle_connected == false then return false end + return state == EVSE_CHARGE_LOW or state == EVSE_CHARGE_FULL +end + +local function charge_power(state, vitals) + if not contactor_closed(state, vitals) then return 0 end + local grid_w = (as_number(vitals.grid_v) or 0) * (as_number(vitals.vehicle_current_a) or 0) + if grid_w < 0 then grid_w = 0 end + if split_phase then return grid_w end + -- The legs are measured on the car side of the contactor. Only a reply + -- with no leg voltage at all falls back to the grid voltage; a leg that + -- reads 0 V carries no power. + if vitals.voltageA_v == nil and vitals.voltageB_v == nil and vitals.voltageC_v == nil then + return grid_w end - local w = phase_power(vitals.voltageA_v, vitals.currentA_a) + return phase_power(vitals.voltageA_v, vitals.currentA_a) + phase_power(vitals.voltageB_v, vitals.currentB_a) + phase_power(vitals.voltageC_v, vitals.currentC_a) - if w <= 0 then - local fallback = (as_number(vitals.grid_v) or 0) * (as_number(vitals.vehicle_current_a) or 0) - if fallback > 0 then w = fallback end - end - return w end local function active_phases(vitals) @@ -143,16 +167,11 @@ local function active_phases(vitals) return n end -local function is_charging(state, vitals) - if state == EVSE_CHARGE_LOW or state == EVSE_CHARGE_FULL then return true end - return vitals.contactor_closed == true -end - local function is_connected(state, vitals) if vitals.vehicle_connected == true then return true end return state == EVSE_CONNECTED or state == EVSE_READY or state == EVSE_HANDSHAKE or state == EVSE_FINISHED - or state == EVSE_WAIT_CAR or is_charging(state, vitals) + or state == EVSE_WAIT_CAR or contactor_closed(state, vitals) end function driver_init(config) @@ -167,7 +186,7 @@ function driver_init(config) split_phase = true end - local raw, err = host.http_get(BASE_URL .. "/api/1/version") + local raw, err = safe_http_get(BASE_URL .. "/api/1/version") if not err then local ver = decode_json(raw) if ver then @@ -184,7 +203,7 @@ end function driver_poll() if not BASE_URL then return 10000 end - local raw, err = host.http_get(BASE_URL .. "/api/1/vitals") + local raw, err = safe_http_get(BASE_URL .. "/api/1/vitals") if err then host.log("warn", "Tesla Wall Connector: vitals failed") return 10000 @@ -196,10 +215,10 @@ function driver_poll() end local state = as_number(vitals.evse_state) or EVSE_IDLE - local power_w = charge_power(vitals) + local power_w = charge_power(state, vitals) local session_wh = as_number(vitals.session_energy_wh) or 0 local connected = is_connected(state, vitals) - local charging = is_charging(state, vitals) + local charging = contactor_closed(state, vitals) local phases = active_phases(vitals) host.emit("ev", { diff --git a/go/internal/drivers/zap_test.go b/go/internal/drivers/zap_test.go index fb41f6335..602ce8ed4 100644 --- a/go/internal/drivers/zap_test.go +++ b/go/internal/drivers/zap_test.go @@ -364,3 +364,139 @@ func TestZapDoesNotUseUnsafeLegacyRESTControl(t *testing.T) { t.Fatalf("read-only Zap driver made %d REST control writes, want 0", got) } } + +// A Zap on firmware 7.x serves the local API in data model v2. These are the +// names a Zap-HAN on 7.0.3 sent: W_ac, l1_V_ac, and null for what it did not +// read. Losing them leaves FTW without a site meter. +func zapV2MeterStub(meter map[string]any) zapAPIStub { + return zapAPIStub{ + devices: map[string]any{"count": 1, "devices": []any{map[string]any{ + "type": "p1_uart", "device_type": "energy_meter", "sn": "p1m-v2", + "ders": []any{map[string]any{"type": "meter", "enabled": true, "location": "grid"}}, + }}}, + snapshots: map[string]any{"p1m-v2": map[string]any{ + "meter": meter, + "version": "v2", + "format": "p1_uart", + }}, + } +} + +func TestZapReadsMeterOnDataModelV2(t *testing.T) { + stub := zapV2MeterStub(map[string]any{ + "type": "meter", "make": "p1", "timestamp": 1793786219266, "read_time_ms": 2, + "W_ac": 2388, "Hz": nil, "apparent_VA_ac": nil, "reactive_var_ac": -114, + "total_import_Wh_ac": 8014739, "total_export_Wh_ac": 0, + "l1_V_ac": 230.1, "l1_A_ac": 8.1, "l1_W_ac": 1872, + "l2_V_ac": 231.8, "l2_A_ac": 1.9, "l2_W_ac": 451, + "l3_V_ac": 228.7, "l3_A_ac": 0.2, "l3_W_ac": 64, + }) + tel, _, _ := loadZapForTest(t, stub, nil) + + meter := tel.Get("sourceful-zap", telemetry.DerMeter) + if meter == nil || meter.RawW != 2388 { + t.Fatalf("meter = %+v, want 2388W import", meter) + } + data := readingData(t, meter) + for key, want := range map[string]float64{ + "l1_v": 230.1, "l1_a": 8.1, "l1_w": 1872, + "l2_v": 231.8, "l2_a": 1.9, "l2_w": 451, + "l3_v": 228.7, "l3_a": 0.2, "l3_w": 64, + "total_import_wh": 8014739, "import_wh": 8014739, + "total_export_wh": 0, "export_wh": 0, + } { + if data[key] != want { + t.Fatalf("meter %s = %v, want %v (%+v)", key, data[key], want, data) + } + } + if _, ok := data["freq_hz"]; ok { + t.Fatalf("Hz null must stay unknown, got %+v", data) + } +} + +func TestZapDoesNotInventZeroForNullPowerOnDataModelV2(t *testing.T) { + tel, _, _ := loadZapForTest(t, zapV2MeterStub(map[string]any{ + "type": "meter", "W_ac": nil, "l1_W_ac": 100, + }), nil) + if got := tel.Get("sourceful-zap", telemetry.DerMeter); got != nil { + t.Fatalf("W_ac null must not become a synthetic zero: %+v", got) + } +} + +// zapV2InverterStub is what a Zap on firmware 7.0.3 sent for a simulated Solis: +// solar and battery on the DC side, ratings in /api/devices, an inverter DER. +func zapV2InverterStub() zapAPIStub { + return zapAPIStub{ + devices: map[string]any{"count": 1, "devices": []any{map[string]any{ + "type": "modbus_tcp", "device_type": "inverter", "profile": "solis", "sn": "INV-V2", + "ders": []any{ + map[string]any{"type": "solar", "enabled": true, "installed_power_W_dc": 8000}, + map[string]any{"type": "battery", "enabled": true, "rated_power_W_dc": 5000, "capacity_Wh_dc": 15000}, + map[string]any{"type": "inverter", "enabled": true, "rated_power_W_ac": 20000}, + }, + }}}, + snapshots: map[string]any{"INV-V2": map[string]any{ + "solar": map[string]any{ + "type": "solar", "make": "solis", "W_ac": nil, "W_dc": -3000, + "mppt": []any{ + map[string]any{"V_dc": 300, "A_dc": -5, "W_dc": -1500}, + map[string]any{"V_dc": 290, "A_dc": -4.8, "W_dc": -1392}, + }, + "total_generation_Wh_ac": nil, "total_generation_Wh_dc": 150000, + }, + "battery": map[string]any{ + "type": "battery", "make": "solis", "W_ac": nil, "W_dc": 5000, + "V_ac": nil, "V_dc": 52, "A_ac": nil, "A_dc": 10, + "soc_nom_fract": 0.75, "soh_fract": nil, "temperature_C": 24, + "total_charge_Wh_ac": nil, "total_charge_Wh_dc": 4200000, + "total_discharge_Wh_ac": nil, "total_discharge_Wh_dc": 1800000, + }, + "inverter": map[string]any{ + "type": "inverter", "make": "solis", "W_ac": -2400, "temperature_C": 45, "rated_power_W_ac": 20000, + }, + "version": "v2", + "format": "json", + }}, + } +} + +func TestZapReadsPVAndBatteryOnDataModelV2(t *testing.T) { + tel, _, _ := loadZapForTest(t, zapV2InverterStub(), map[string]any{"read_pv": true, "read_battery": true}) + + pv := tel.Get("sourceful-zap", telemetry.DerPV) + if pv == nil || pv.RawW != -3000 { + t.Fatalf("pv = %+v, want -3000W", pv) + } + pvData := readingData(t, pv) + if pvData["rated_w"] != float64(20000) || pvData["lifetime_wh"] != float64(150000) { + t.Fatalf("pv rating and energy = %+v", pvData) + } + + bat := tel.Get("sourceful-zap", telemetry.DerBattery) + if bat == nil || bat.RawW != 5000 { + t.Fatalf("battery = %+v, want +5000W charge", bat) + } + batData := readingData(t, bat) + for key, want := range map[string]float64{ + "soc": 0.75, "v": 52, "a": 10, "temp_c": 24, "rated_w": 5000, + "total_charge_wh": 4200000, "total_discharge_wh": 1800000, + } { + if batData[key] != want { + t.Fatalf("battery %s = %v, want %v (%+v)", key, batData[key], want, batData) + } + } +} + +func TestZapDoesNotReadPVOrBatteryOnDataModelV2UnlessOptedIn(t *testing.T) { + tel, _, _ := loadZapForTest(t, zapV2InverterStub(), nil) + if got := tel.Get("sourceful-zap", telemetry.DerPV); got != nil { + t.Fatalf("Zap must not ingest PV unless opted in: %+v", got) + } + if got := tel.Get("sourceful-zap", telemetry.DerBattery); got != nil { + t.Fatalf("Zap must not ingest battery unless opted in: %+v", got) + } + count, _, ok := tel.LatestMetric("sourceful-zap", "other_resources") + if !ok || count != 2 { + t.Fatalf("other_resources = %v %v, want 2 (PV, battery); the inverter DER is not another", count, ok) + } +} diff --git a/go/internal/ftwcli/cli.go b/go/internal/ftwcli/cli.go index a50826b17..abcc30168 100644 --- a/go/internal/ftwcli/cli.go +++ b/go/internal/ftwcli/cli.go @@ -26,6 +26,7 @@ Usage: ftw rollback return to the previous release ftw backup [--output-dir DIR] make a verified full backup; DIR gets a checked copy ftw support [--output FILE] write the redacted support file + ftw password reset turn the house password off, to choose a new one ftw help Every command takes --url URL (default http://127.0.0.1:8080). @@ -114,6 +115,8 @@ func run(args []string, stdout, stderr io.Writer, e env) int { err = runBackup(args[1:], stdout, e) case "support": err = runSupport(args[1:], stdout, e) + case "password": + err = runPassword(args[1:], stdout, e) default: err = usageError("unknown command " + args[0]) } diff --git a/go/internal/ftwcli/cli_test.go b/go/internal/ftwcli/cli_test.go index 029d996a2..0ded2a14b 100644 --- a/go/internal/ftwcli/cli_test.go +++ b/go/internal/ftwcli/cli_test.go @@ -818,3 +818,28 @@ func TestUpdateReportsHealthOnlyOnceItHolds(t *testing.T) { t.Fatalf("reported after %s, before ok held", elapsed) } } + +// A forgotten house password is turned off from the box itself, and the +// page then asks for a new one. +func TestPasswordResetTurnsTheHousePasswordOff(t *testing.T) { + f, srv := newFakeCore(t) + f.on("POST", "/api/auth/password", reply(200, `{"status":"ok","lan_auth":false,"configured":false}`)) + code, out, errOut := runCLI(t, testEnv(), "password", "reset", "--url", srv.URL) + if code != exitOK || !strings.Contains(out, "The house password is off") { + t.Fatalf("reset %d %q %q", code, out, errOut) + } + if got := f.bodies; len(got) != 1 || got[0] != `{"enabled":false}` { + t.Fatalf("posted %v", got) + } + + f.on("POST", "/api/auth/password", reply(401, `{"error":"valid LAN password required"}`)) + if code, _, errOut := runCLI(t, testEnv(), "password", "reset", "--url", srv.URL); code != exitFailed || !strings.Contains(errOut, "run this on the box itself") { + t.Fatalf("from elsewhere %d %q", code, errOut) + } + + for _, args := range [][]string{{"password"}, {"password", "set"}, {"password", "reset", "now"}} { + if code, _, errOut := runCLI(t, testEnv(), args...); code != exitUsage || !strings.Contains(errOut, "ftw password reset") { + t.Fatalf("%v: exit %d, stderr %q", args, code, errOut) + } + } +} diff --git a/go/internal/ftwcli/password.go b/go/internal/ftwcli/password.go new file mode 100644 index 000000000..eba47e454 --- /dev/null +++ b/go/internal/ftwcli/password.go @@ -0,0 +1,38 @@ +package ftwcli + +import ( + "context" + "errors" + "fmt" + "io" + "net/http" +) + +// runPassword turns the house password off, for an owner who forgot it or +// finds someone else set it. Only the box itself may do this without the +// password, so it reaches Core on loopback. Afterwards the box's page asks +// for a new one. +func runPassword(args []string, out io.Writer, e env) error { + if len(args) == 0 || args[0] != "reset" { + return usageError("usage: ftw password reset [--url URL]") + } + base, err := parse(args[1:], out, "ftw password reset [--url URL]", nil) + if err != nil { + return err + } + c := newClient(base, e) + ctx := context.Background() + err = c.call(ctx, http.MethodPost, "/api/auth/password", map[string]bool{"enabled": false}, nil, e.requestTimeout) + var api *apiError + switch { + case err == nil: + fmt.Fprintln(out, "The house password is off, and every browser is signed out.") + fmt.Fprintln(out, "Open the box's page to choose a new one, under Settings → System or Settings → Access.") + return nil + case errors.As(err, &api) && api.status == http.StatusUnauthorized: + return fmt.Errorf("%s asked for the house password; run this on the box itself, against http://127.0.0.1", base) + case c.waitingForSetup(ctx): + return fmt.Errorf("Core is waiting for setup and has no house password yet; open %s/setup", base) + } + return err +} diff --git a/go/internal/ha/bridge.go b/go/internal/ha/bridge.go index 9aae52446..39c4bfbe9 100644 --- a/go/internal/ha/bridge.go +++ b/go/internal/ha/bridge.go @@ -816,6 +816,12 @@ func metricUnitAndClass(name, explicitUnit string) (unit, devClass string) { func (b *Bridge) subscribeCommands() { b.client.Subscribe(b.cmdTopic("mode"), 0, func(_ paho.Client, m paho.Message) { mode := string(m.Payload()) + // A stored broker command is replayed on every reconnect. It + // must not undo the boot rule that ends manual Charge. + if mode == string(control.ModeCharge) && m.Retained() { + slog.Warn("HA: ignored a retained Charge command; send a new command to charge") + return + } if b.cb.SetMode != nil { if err := b.cb.SetMode(mode); err != nil { slog.Warn("HA set mode failed", "err", err) diff --git a/go/internal/ha/charge_restart_test.go b/go/internal/ha/charge_restart_test.go new file mode 100644 index 000000000..03fce2067 --- /dev/null +++ b/go/internal/ha/charge_restart_test.go @@ -0,0 +1,45 @@ +package ha + +import ( + "fmt" + "testing" + + paho "github.com/eclipse/paho.mqtt.golang" +) + +type retainedModeMessage struct { + commandMessage + retained bool +} + +func (m retainedModeMessage) Retained() bool { return m.retained } + +func TestRetainedChargeCannotRestartManualCharging(t *testing.T) { + for _, tc := range []struct { + mode string + retained bool + want bool + }{ + {"charge", true, false}, + {"charge", false, true}, + {"idle", true, true}, + {"planner_passive_arbitrage", true, true}, + } { + t.Run(fmt.Sprintf("%s/retained=%t", tc.mode, tc.retained), func(t *testing.T) { + cli := &commandClient{handlers: map[string]paho.MessageHandler{}} + var called bool + b := &Bridge{topicPrefix: "ftw", client: cli, cb: CommandCallbacks{SetMode: func(mode string) error { + called = true + if mode != tc.mode { + t.Errorf("mode = %q, want %q", mode, tc.mode) + } + return nil + }}} + b.subscribeCommands() + cli.handlers[b.cmdTopic("mode")](cli, retainedModeMessage{commandMessage: commandMessage{payload: []byte(tc.mode)}, retained: tc.retained}) + if called != tc.want { + t.Fatalf("SetMode called = %v, want %v", called, tc.want) + } + }) + } +} diff --git a/go/internal/loadpoint/controller.go b/go/internal/loadpoint/controller.go index 3e9fa1dfd..9ca3488c1 100644 --- a/go/internal/loadpoint/controller.go +++ b/go/internal/loadpoint/controller.go @@ -42,6 +42,7 @@ type Controller struct { commandTimeout time.Duration wallboxCycleSeq atomic.Uint64 resumeOffers map[string]*resumeOffer // owned by the dispatch tick + unpluggedSince map[string]time.Time // owned by the dispatch tick energySamples map[string]*meteredEnergy // fuseEVMax is this tick's EV share of the site fuse. nil means no cap. @@ -1342,7 +1343,9 @@ func (c *Controller) observeTick(now time.Time, cfg Config, sample EVSample, dis c.manager.SetSurplusWithheld(cfg.ID, wasSurplus && enteringSurplusPaused) c.manager.ObserveSample(cfg.ID, sample) c.observePhases(cfg, sample, now) - c.restoreManualHoldForSession(cfg.ID) + if !c.holdInUnplugGrace(cfg.ID, sample.Connected, now) { + c.restoreManualHoldForSession(cfg.ID) + } c.evaluateBatteryBoost(cfg.ID, now, sample.Connected, dispatchAllowed && !sample.PowerUnavailable) } @@ -1432,8 +1435,12 @@ func (c *Controller) tickOne(ctx context.Context, now time.Time, lpCfg Config, s // Release any manual override when the vehicle unplugs. A // persistent "Start" hold (zero ExpiresAt) would otherwise // survive the session and silently re-apply to the next car; - // "until Stop or unplug" is the operator's mental model. - c.ClearManualHold(lpCfg.ID) + // "until Stop or unplug" is the operator's mental model. One + // reading is not an unplug: the cable must stay out for + // UnplugGrace, as the cars runtime counts it. + if since, seen := c.unpluggedSince[lpCfg.ID]; !seen || now.Sub(since) >= UnplugGrace { + c.ClearManualHold(lpCfg.ID) + } return } if !wasPlugged { @@ -2626,7 +2633,8 @@ func (c *Controller) computeCommandForBudget(now time.Time, lpCfg Config, curren } if d.LoadpointFallback[lpCfg.ID] { st, known := c.manager.State(lpCfg.ID) - if _, _, needed := DeadlineNeed(st); !known || !needed { + minW, maxW, steps := c.PlanningLimits(st) + if _, _, needed := DeadlineNeed(st, minW, maxW, steps); !known || !needed { return 0, true, false } } diff --git a/go/internal/loadpoint/controller_fuse_budget.go b/go/internal/loadpoint/controller_fuse_budget.go index 938db18e0..477cc13f5 100644 --- a/go/internal/loadpoint/controller_fuse_budget.go +++ b/go/internal/loadpoint/controller_fuse_budget.go @@ -47,13 +47,15 @@ func (c *Controller) freezeFuseBudget(now time.Time, samples map[string]EVSample st, _ := c.manager.State(cfg.ID) request := fuseRequest{cfg: c.fuseConfig(cfg, c.tickChoices[cfg.ID].phaseMode), deadline: st.TargetTime} if sample.Connected && !sample.ConnectionUnknown && c.driverCanDispatch(cfg.DriverName) { - if hold, held := c.GetManualHold(cfg.ID, now); held { + hold, held := c.GetManualHold(cfg.ID, now) + if held { request.wantW = clampManualPower(cfg, hold, c.siteFuse()) } else { request.wantW, _, _ = c.computeCommandForBudget(now, request.cfg, sample.PowerW, false) } - request.goal = request.wantW > 0 && !cfg.SurplusOnly - if _, held := c.GetManualHold(cfg.ID, now); !held { + // Charge now is a goal even on a solar-only charger. + request.goal = request.wantW > 0 && (held || !cfg.SurplusOnly) + if !held { c.batSoCArmedMu.Lock() armed := c.batSoCArmed[cfg.ID] c.batSoCArmedMu.Unlock() @@ -219,13 +221,16 @@ func (c *Controller) phaseRoomCapW(id, mode string, wantW, split, roomA float64) } // FuseConfigs returns the loadpoints with the phase mode each command names -// this tick. control.EVFuseBudget measures room on those phases, so an auto -// or 1p offer never gets three phases of room. Outside a tick, and for a -// loadpoint that gets no command, the configured mode stands. +// this tick, and the least each can order on those phases. +// control.EVFuseBudget measures room on those phases, so an auto or 1p offer +// never gets three phases of room, and an idle pool waits for room for that +// least order, not the configured three-phase minimum. Outside a tick, and +// for a loadpoint that gets no command, the configured mode stands. func (c *Controller) FuseConfigs() []Config { configs := c.manager.Configs() for i := range configs { if mode := c.tickChoices[configs[i].ID].phaseMode; mode != "" { + configs[i] = c.fuseConfig(configs[i], mode) configs[i].PhaseMode = mode } } diff --git a/go/internal/loadpoint/controller_fuse_phase_test.go b/go/internal/loadpoint/controller_fuse_phase_test.go index f5ad4fd59..3a52bb642 100644 --- a/go/internal/loadpoint/controller_fuse_phase_test.go +++ b/go/internal/loadpoint/controller_fuse_phase_test.go @@ -48,7 +48,7 @@ func TestSharedFuseGuardUsesCommandPhase(t *testing.T) { if !known || math.Abs(command.power-tc.commandW) > .01 || command.phaseMode != tc.commandMode { t.Fatalf("command=%+v, want %.0f W with phase %q", command, tc.commandW, tc.commandMode) } - if phases := PhaseFor(command.phaseMode, command.power, 3680); command.power > 0 && tc.worstA+command.power/(230*float64(phases)) > 24.001 { + if phases := PhaseFor(command.phaseMode, command.power, 3680); command.power > 0 && tc.worstA+command.power/(230*float64(phases)) > 25.001 { t.Fatalf("resolved %dp command crosses the phase working limit: %+v", phases, command) } demand := c.FuseDemands() diff --git a/go/internal/loadpoint/controller_hold_restart_test.go b/go/internal/loadpoint/controller_hold_restart_test.go index b2035d25a..c5eccf1b1 100644 --- a/go/internal/loadpoint/controller_hold_restart_test.go +++ b/go/internal/loadpoint/controller_hold_restart_test.go @@ -82,17 +82,19 @@ func TestChargeNowWithoutSessionIDResumesAfterRestart(t *testing.T) { samples["easee"] = sample tickWants(t, c, sender, now.Add(21*time.Second), 4140, false) + // The cable stays out for UnplugGrace. sample.Connected = false samples["easee"] = sample c.Tick(context.Background(), now.Add(22*time.Second)) + c.Tick(context.Background(), now.Add(22*time.Second+UnplugGrace)) sample.Connected = true samples["easee"] = sample - tickWants(t, c, sender, now.Add(23*time.Second), 0, false) + tickWants(t, c, sender, now.Add(23*time.Second+UnplugGrace), 0, false) if h, ok := c.GetManualHold("garage", now); ok { t.Fatalf("the next car inherited Charge now: %+v", h) } c, sender = bootController(t, store, samples) - tickWants(t, c, sender, now.Add(30*time.Second), 0, false) + tickWants(t, c, sender, now.Add(30*time.Second+UnplugGrace), 0, false) if h, ok := c.GetManualHold("garage", now); ok { t.Fatalf("Charge now returned after unplug and restart: %+v", h) } diff --git a/go/internal/loadpoint/controller_hold_restore.go b/go/internal/loadpoint/controller_hold_restore.go index 80bff11be..3623a3f3f 100644 --- a/go/internal/loadpoint/controller_hold_restore.go +++ b/go/internal/loadpoint/controller_hold_restore.go @@ -1,5 +1,7 @@ package loadpoint +import "time" + type manualSessionBinding struct { deviceID, sessionID string generation uint64 @@ -84,6 +86,11 @@ func (c *Controller) restoreManualHoldForSession(id string) { // A changed generation therefore means a counter reset or a new // session, even when the earlier session ID was still unknown. changed = changed || previous.generation != current.generation + if !changed && current.sessionID == "" { + // Keep the last proven session while the charger names none, + // as after an unplug, so a later proof is checked against it. + current.sessionID = previous.sessionID + } } } // A Start before the first hardware reading binds here. Once bound, a @@ -137,3 +144,58 @@ func (c *Controller) restoreManualHoldForSession(id string) { c.manager.SetManualRestoreUnconfirmed(id, false) } } + +// UnplugGrace is how long a charger must say the cable is out before the +// session ends and Charge now is cleared. The cars runtime uses it too. +const UnplugGrace = 30 * time.Second + +// holdInUnplugGrace times the unplug and reports whether a hold waits out +// UnplugGrace. An unplugged reading says nothing about the next car, so the +// caller skips the session check. A cable back within the grace keeps the +// hold for the same car; another device or session clears it. +func (c *Controller) holdInUnplugGrace(id string, connected bool, now time.Time) bool { + if connected { + if _, pending := c.unpluggedSince[id]; pending { + delete(c.unpluggedSince, id) + c.rebindAfterUnplug(id) + } + return false + } + if c.unpluggedSince == nil { + c.unpluggedSince = map[string]time.Time{} + } + since, seen := c.unpluggedSince[id] + if !seen { + since = now + c.unpluggedSince[id] = now + } + c.holdMu.Lock() + _, held := c.holds[id] + c.holdMu.Unlock() + return held && now.Sub(since) < UnplugGrace +} + +// rebindAfterUnplug compares the first reading after a short unplug with +// the hardware the hold was bound to before it. +func (c *Controller) rebindAfterUnplug(id string) { + current := c.manager.manualSessionBinding(id) + c.holdMu.Lock() + previous, bound := c.manualBindings[id] + _, held := c.holds[id] + other := bound && ((previous.deviceID != "" && current.deviceID != "" && previous.deviceID != current.deviceID) || + otherSession(previous, current)) + if held && bound && !other && current.deviceID != "" { + // The unplug may have moved the session generation. The same + // hardware and session keep the hold. restoreManualHoldForSession + // keeps the carried session while the charger names none, so a + // later session proof is checked against the one before the unplug. + if current.sessionID == "" { + current.sessionID = previous.sessionID + } + c.manualBindings[id] = current + } + c.holdMu.Unlock() + if held && other { + c.ClearManualHold(id) + } +} diff --git a/go/internal/loadpoint/controller_manual_override_surplus_test.go b/go/internal/loadpoint/controller_manual_override_surplus_test.go index beded3f35..763292815 100644 --- a/go/internal/loadpoint/controller_manual_override_surplus_test.go +++ b/go/internal/loadpoint/controller_manual_override_surplus_test.go @@ -108,11 +108,17 @@ func TestPersistentManualHoldClearedOnUnplug(t *testing.T) { t.Fatalf("hold not installed") } - // Car unplugs. + // Car unplugs. One reading is not yet an unplug. samples[cfg.DriverName] = EVSample{Connected: false} c.Tick(context.Background(), now.Add(5*time.Second)) + if _, active := c.GetManualHold(cfg.ID, now.Add(5*time.Second)); !active { + t.Fatalf("persistent hold cleared by one unplugged reading; want kept for UnplugGrace") + } - if _, active := c.GetManualHold(cfg.ID, now.Add(5*time.Second)); active { + // The cable stays out for UnplugGrace. + at := now.Add(5*time.Second + UnplugGrace) + c.Tick(context.Background(), at) + if _, active := c.GetManualHold(cfg.ID, at); active { t.Errorf("persistent hold survived unplug; want cleared") } } diff --git a/go/internal/loadpoint/controller_missing_reading_test.go b/go/internal/loadpoint/controller_missing_reading_test.go index f65aa3ac5..3265a6005 100644 --- a/go/internal/loadpoint/controller_missing_reading_test.go +++ b/go/internal/loadpoint/controller_missing_reading_test.go @@ -37,6 +37,7 @@ func TestMissingReadingDoesNotEndManualSession(t *testing.T) { } samples[cfg.DriverName] = EVSample{Connected: false} c.Tick(context.Background(), now.Add(3*time.Second)) + c.Tick(context.Background(), now.Add(3*time.Second+UnplugGrace)) if _, ok := c.GetManualHold(cfg.ID, now); ok { t.Fatal("a confirmed unplug must clear the hold") } diff --git a/go/internal/loadpoint/controller_phase_choice_test.go b/go/internal/loadpoint/controller_phase_choice_test.go index 28d818990..7b772af20 100644 --- a/go/internal/loadpoint/controller_phase_choice_test.go +++ b/go/internal/loadpoint/controller_phase_choice_test.go @@ -13,7 +13,7 @@ var threePhaseSteps = []float64{0, 1380, 1610, 1840, 2070, 2300, 2530, 2760, 299 4140, 4830, 5520, 6210, 6900, 7590, 8280, 8970, 9660, 10350, 11040} // meteredEVCap stands in for control.EVFuseBudget: the plugged cars' last -// offers plus the room the meter shows below the fuse less 1 A, on the +// offers plus the room the meter shows below the fuse, on the // phases FuseConfigs names for this tick. func meteredEVCap(c *Controller, worstA func() float64) func() (float64, bool) { return func() (float64, bool) { @@ -27,7 +27,7 @@ func meteredEVCap(c *Controller, worstA func() float64) func() (float64, bool) { offersW += math.Max(0, st.CommandedW) } } - return math.Max(0, offersW+(site.MaxAmps-1-worstA())*site.Voltage*phases), true + return math.Max(0, offersW+(site.MaxAmps-worstA())*site.Voltage*phases), true } } diff --git a/go/internal/loadpoint/controller_unplug_grace_test.go b/go/internal/loadpoint/controller_unplug_grace_test.go new file mode 100644 index 000000000..ddf9302eb --- /dev/null +++ b/go/internal/loadpoint/controller_unplug_grace_test.go @@ -0,0 +1,116 @@ +package loadpoint + +import ( + "context" + "testing" + "time" +) + +var graceCar = EVSample{Connected: true, RequestActive: true, PowerW: 4140, SessionWh: 6000, DeviceID: "dev-1", SessionID: "sess-1"} + +// unplugFlap runs Charge now at 4140 W, then one unplugged reading, then +// the given connected reading 5 s later. It returns whether the hold is +// still there and the last order. +func unplugFlap(t *testing.T, unplugged, back EVSample) (bool, float64) { + t.Helper() + now := time.Now() + cfg := holdLoadpoint() + samples := map[string]EVSample{} + sender := &fakeSender{} + c := newTestController(t, []Config{cfg}, &Directive{SlotStart: now, SlotEnd: now.Add(time.Hour)}, samples, sender) + samples[cfg.DriverName] = graceCar + c.Tick(context.Background(), now) + c.SetManualHold(cfg.ID, ManualHold{PowerW: 4140, PhaseMode: "3p", Persistent: true}) + c.Tick(context.Background(), now.Add(5*time.Second)) + if n := len(sender.calls); n == 0 || sender.calls[n-1].power != 4140 { + t.Fatalf("Charge now not dispatched: %+v", sender.calls) + } + samples[cfg.DriverName] = unplugged + c.Tick(context.Background(), now.Add(10*time.Second)) + samples[cfg.DriverName] = back + c.Tick(context.Background(), now.Add(15*time.Second)) + _, held := c.GetManualHold(cfg.ID, now.Add(15*time.Second)) + last := -1.0 + if n := len(sender.calls); n > 0 { + last = sender.calls[n-1].power + } + return held, last +} + +// One Connected=false reading between two readings of the same device and +// session does not end Charge now; cars.UnplugGrace keeps that session 30 s. +func TestOneUnpluggedReadingKeepsChargeNow(t *testing.T) { + held, last := unplugFlap(t, EVSample{Connected: false, DeviceID: "dev-1", SessionID: "sess-1"}, graceCar) + if !held || last != 4140 { + t.Fatalf("Charge now gone after one 5 s unplug reading of the same session: held=%v last order=%v W", held, last) + } +} + +// A charger that drops the session id while it reads unplugged still +// keeps Charge now when the same session comes back. +func TestUnpluggedReadingWithoutSessionKeepsChargeNow(t *testing.T) { + held, last := unplugFlap(t, EVSample{Connected: false, DeviceID: "dev-1"}, graceCar) + if !held || last != 4140 { + t.Fatalf("Charge now gone after one unplug reading without a session id: held=%v last order=%v W", held, last) + } +} + +// Another session within the grace is another car: Charge now ends and the +// plan decides, as an unplug did before. +func TestOtherSessionAfterShortUnplugClearsChargeNow(t *testing.T) { + other := graceCar + other.SessionID = "sess-2" + held, last := unplugFlap(t, EVSample{Connected: false, DeviceID: "dev-1"}, other) + if held || last != 0 { + t.Fatalf("a new session inherited Charge now: held=%v last order=%v W", held, last) + } +} + +// lateSessionFlap is unplugFlap for a charger that names the session one +// reading after the cable, as Easee Cloud does. It returns the hold and the +// last order once the session is named. +func lateSessionFlap(t *testing.T, back EVSample) (ManualHold, bool, float64) { + t.Helper() + now := time.Now() + cfg := holdLoadpoint() + samples := map[string]EVSample{} + sender := &fakeSender{} + c := newTestController(t, []Config{cfg}, &Directive{SlotStart: now, SlotEnd: now.Add(time.Hour)}, samples, sender) + samples[cfg.DriverName] = graceCar + c.Tick(context.Background(), now) + c.SetManualHold(cfg.ID, ManualHold{PowerW: 4140, PhaseMode: "3p", Persistent: true}) + c.Tick(context.Background(), now.Add(5*time.Second)) + samples[cfg.DriverName] = EVSample{Connected: false, DeviceID: "dev-1"} + c.Tick(context.Background(), now.Add(10*time.Second)) + unnamed := graceCar + unnamed.SessionID = "" + samples[cfg.DriverName] = unnamed + c.Tick(context.Background(), now.Add(12*time.Second)) + samples[cfg.DriverName] = back + c.Tick(context.Background(), now.Add(14*time.Second)) + h, held := c.GetManualHold(cfg.ID, now.Add(14*time.Second)) + last := -1.0 + if n := len(sender.calls); n > 0 { + last = sender.calls[n-1].power + } + return h, held, last +} + +// Another session named one reading after the cable came back is still +// another car: it must not inherit Charge now. +func TestLateOtherSessionAfterShortUnplugEndsChargeNow(t *testing.T) { + other := graceCar + other.SessionID = "sess-2" + h, held, last := lateSessionFlap(t, other) + if last == 4140 || (held && h.PowerW > 0) { + t.Fatalf("late other session inherited Charge now: last=%v W hold=%+v", last, h) + } +} + +// The same session named one reading late keeps Charge now. +func TestLateSameSessionAfterShortUnplugKeepsChargeNow(t *testing.T) { + h, held, last := lateSessionFlap(t, graceCar) + if !held || h.PowerW != 4140 || last != 4140 { + t.Fatalf("Charge now lost when the same session was named late: held=%v hold=%+v last=%v W", held, h, last) + } +} diff --git a/go/internal/loadpoint/deadline_need.go b/go/internal/loadpoint/deadline_need.go index dd1a8f2a6..9e9ffeae7 100644 --- a/go/internal/loadpoint/deadline_need.go +++ b/go/internal/loadpoint/deadline_need.go @@ -4,15 +4,16 @@ import ( "math" ) -// DeadlineNeed is the remaining AC energy and legal on-power of a saved goal, -// held to the session car's own charger and charge limit as the plan is. It -// does not promise future site capacity or authorize a hardware command. -func DeadlineNeed(st State) (energyWh, powerW float64, ok bool) { +// DeadlineNeed is the remaining AC energy of a saved goal, held to the car's +// charge limit, and its legal on-power within minW-maxW and steps: the limits +// the plan gets for the session (PlanningLimits). It does not promise future +// site capacity or authorize a hardware command. +func DeadlineNeed(st State, minW, maxW float64, steps []float64) (energyWh, powerW float64, ok bool) { if !st.PluggedIn || st.ChargingDeclined || st.GoalComplete || st.TargetTime.IsZero() || !finite(st.CurrentSoC) || st.CurrentSoC < 0 || st.CurrentSoC > 1 || !finite(st.TargetSoC) || st.TargetSoC < 0 || st.TargetSoC > 1 || !finite(st.VehicleCapacityWh) || st.VehicleCapacityWh <= 0 || - !finite(st.MinChargeW) || !finite(st.MaxChargeW) || st.MinChargeW <= 0 || st.MaxChargeW < st.MinChargeW { + !finite(st.MinChargeW) || !finite(st.MaxChargeW) || !(minW > 0) || st.MaxChargeW < st.MinChargeW { return 0, 0, false } // A vehicle-limit goal aims at 1, and the controller still stops at the @@ -27,7 +28,6 @@ func DeadlineNeed(st State) (energyWh, powerW float64, ok bool) { return 0, 0, false } } - minW, maxW, steps := LimitPower(st, st.AllowedStepsW) for _, step := range steps { if step >= minW { powerW = math.Max(powerW, step) diff --git a/go/internal/loadpoint/deadline_need_test.go b/go/internal/loadpoint/deadline_need_test.go index d8340dd66..b6b670a9d 100644 --- a/go/internal/loadpoint/deadline_need_test.go +++ b/go/internal/loadpoint/deadline_need_test.go @@ -29,33 +29,112 @@ func TestLimitPowerKeepsTheChargersLowestStep(t *testing.T) { } } +// deadlineNeed is the reserve's need within the limits the plan gets on a +// 3 × 25 A site, phase mode unset. +func deadlineNeed(st State) (energyWh, powerW float64, ok bool) { + cfg := Config{MinChargeW: st.MinChargeW, MaxChargeW: st.MaxChargeW, AllowedStepsW: st.AllowedStepsW} + minW, maxW, steps := PlanningLimits(st, cfg, SiteFuse{MaxAmps: 25, Voltage: 230, PhaseCnt: 3}) + return DeadlineNeed(st, minW, maxW, steps) +} + // The reserve's need and on-power hold to the session car's own charger and // its charge limit, as the plan does. func TestDeadlineNeedHoldsToTheCar(t *testing.T) { st := State{PluggedIn: true, VehicleCapacityWh: 40000, CurrentSoC: .3, TargetSoC: .8, TargetTime: time.Now().Add(3 * time.Hour), MinChargeW: 4140, MaxChargeW: 11000, AllowedStepsW: []float64{0, 4140, 7360, 11000}} - if _, w, ok := DeadlineNeed(st); !ok || w != 11000 { + if _, w, ok := deadlineNeed(st); !ok || w != 11000 { t.Fatalf("no car limit: %v %v", w, ok) } st.CarMaxW = 7400 - if _, w, ok := DeadlineNeed(st); !ok || w != 7360 { + if _, w, ok := deadlineNeed(st); !ok || w != 7360 { t.Fatalf("a 7.4 kW car: %v %v", w, ok) } st.AllowedStepsW = nil - if _, w, ok := DeadlineNeed(st); !ok || w != 7400 { + if _, w, ok := deadlineNeed(st); !ok || w != 7400 { t.Fatalf("a 7.4 kW car on a stepless charger: %v %v", w, ok) } st.CarMaxW = 3680 - if _, w, ok := DeadlineNeed(st); !ok || w != 4140 { + if _, w, ok := deadlineNeed(st); !ok || w != 4140 { t.Fatalf("a 3.7 kW car on a stepless charger: %v %v, want the charger's minimum", w, ok) } st.CarMaxW = 7400 st.CarChargeLimit = .6 - if need, _, ok := DeadlineNeed(st); !ok || math.Abs(need-(.6-.3)*40000/DefaultChargeEfficiency) > 1e-6 { + if need, _, ok := deadlineNeed(st); !ok || math.Abs(need-(.6-.3)*40000/DefaultChargeEfficiency) > 1e-6 { t.Fatalf("a 60 %% limit: %v %v", need, ok) } st.CurrentSoC = .6 - if _, _, ok := DeadlineNeed(st); ok { + if _, _, ok := deadlineNeed(st); ok { t.Fatal("a car at its own limit still needs energy") } } + +// Review of #1807: on a table with one-phase and three-phase levels, planned +// charging runs on three phases. A car whose own maximum is below the +// lowest three-phase level got a one-phase level as its on-power, which no +// three-phase order has, and was ordered 0 W. It gets that lowest level, as +// the plan gives it. A car marked one-phase gets the third of a three-phase +// level it takes, up to its own maximum, on that table and on one with +// three-phase levels only. +func TestDeadlineNeedRunsOnThePlansPhases(t *testing.T) { + var mixed, threePhase []float64 + for a := 6.0; a <= 16; a++ { + mixed, threePhase = append(mixed, a*230), append(threePhase, a*690) + } + mixed = append(append([]float64{0}, mixed...), threePhase...) + threePhase = append([]float64{0}, threePhase...) + for _, c := range []struct { + name string + minW float64 + steps []float64 + carMaxW float64 + carPhases int + want float64 + }{ + {"mixed table, a 3.7 kW car", 1380, mixed, 3680, 0, 4140}, + {"mixed table, a 3.7 kW car marked one-phase", 1380, mixed, 3680, 1, 3680}, + {"mixed table, a 2.3 kW car marked one-phase", 1380, mixed, 2300, 1, 2300}, + {"three-phase table, a 3.7 kW car", 4140, threePhase, 3680, 0, 4140}, + {"three-phase table, a 2.3 kW car marked one-phase", 4140, threePhase, 2300, 1, 2300}, + } { + st := State{PluggedIn: true, VehicleCapacityWh: 40000, CurrentSoC: .3, TargetSoC: .8, TargetTime: time.Now().Add(3 * time.Hour), + MinChargeW: c.minW, MaxChargeW: 11040, AllowedStepsW: c.steps, CarMaxW: c.carMaxW, CarPhases: c.carPhases} + if _, w, ok := deadlineNeed(st); !ok || w != c.want { + t.Errorf("%s: on-power %v ok=%v, want %v", c.name, w, ok, c.want) + } + } +} + +// The controller's limits read the loadpoint's phase mode and the site fuse +// as they stand. A three-phase site keeps a mixed table's three-phase levels; +// a 1p pin or a one-phase site keeps its one-phase levels. +func TestControllerPlanningLimitsReadPhaseModeAndSite(t *testing.T) { + mixed := []float64{0} + for _, phases := range []float64{1, 3} { + for a := 6.0; a <= 16; a++ { + mixed = append(mixed, a*230*phases) + } + } + st := State{ID: "garage", MinChargeW: 1380, MaxChargeW: 11040, AllowedStepsW: mixed, CarMaxW: 3680} + for _, c := range []struct { + name string + mode string + phases int + want float64 + }{ + {"three-phase site", "", 3, 4140}, + {"three-phase site, auto", "auto", 3, 4140}, + {"pinned 1p", "1p", 3, 3680}, + {"one-phase site", "", 1, 3680}, + } { + m := NewManager() + m.Load([]Config{{ID: "garage", DriverName: "easee", PhaseMode: c.mode, MinChargeW: 1380, MaxChargeW: 11040, AllowedStepsW: mixed}}) + ctrl := NewController(m, nil, nil, nil) + ctrl.SetSiteFuse(SiteFuse{MaxAmps: 25, Voltage: 230, PhaseCnt: c.phases}) + if _, maxW, _ := ctrl.PlanningLimits(st); maxW != c.want { + t.Errorf("%s: top %v W, want %v W", c.name, maxW, c.want) + } + } + if _, maxW, _ := (*Controller)(nil).PlanningLimits(st); maxW != 3680 { + t.Errorf("no controller: top %v W, want the charger's own table", maxW) + } +} diff --git a/go/internal/loadpoint/loadpoint.go b/go/internal/loadpoint/loadpoint.go index e532d9d3a..ef16f9de2 100644 --- a/go/internal/loadpoint/loadpoint.go +++ b/go/internal/loadpoint/loadpoint.go @@ -204,8 +204,9 @@ type State struct { CarMaxW float64 `json:"-"` CarChargeLimit float64 `json:"-"` // CarPhases is 1 when this session's car draws on one phase of a - // three-phase order, seen on the site meter, and 0 when unknown; a car - // profile's phases may set it later. CarPhase is that phase, 1-3. The + // three-phase order, seen on the site meter, 3 when it draws on all + // three, and 0 when unknown; a car profile's phases may set it later. + // CarPhase is the phase of a one-phase car, 1-3. The // plan counts the car's one-phase levels (OnePhaseCarSteps) and the // order names the current per phase it should take. CarPhases int `json:"-"` @@ -1382,8 +1383,11 @@ func (lp *loadpointRuntime) snapshot() State { CommandedPhases: lp.commandedPhases, CarPhase: lp.carPhases.phase, } - if lp.carPhases.phase > 0 { + switch { + case lp.carPhases.phase > 0: st.CarPhases = 1 + case lp.carPhases.three: + st.CarPhases = 3 } if lp.VehicleCapacityWh <= 0 { st.VehicleCapacityWh = 60000 diff --git a/go/internal/loadpoint/one_phase_car.go b/go/internal/loadpoint/one_phase_car.go index ca2c1d816..e91c6463e 100644 --- a/go/internal/loadpoint/one_phase_car.go +++ b/go/internal/loadpoint/one_phase_car.go @@ -19,10 +19,9 @@ import ( // A house load does not follow FTW's orders, and the charger's own reading // must have changed by what the phases show, so a kettle cannot mark a car // and a heat pump cannot clear one. A two-phase car moves two phases and is -// never marked. With no reading from before the car drew, as after a Core -// restart, the charger's own draw is the reference: one phase carrying all -// of it and the other two less than a third, which no three- or two-phase -// car can show. +// never marked. Without a rise there is no baseline: a house load on one +// phase and solar on the other two can look like a one-phase car, so after +// a Core restart the car stays unmarked until the next rise. const ( onePhaseStepA = 2.0 // the smallest order rise that is a test @@ -52,7 +51,7 @@ type phaseTest struct { } // carPhases is one session's evidence. phase is 1-3 once the car is seen on -// that phase alone. +// that phase alone; three is set once it is seen on all three. type carPhases struct { orderA float64 // per phase, the order the next rise is measured from // rising is set once the order has gone above orderA, and before is the @@ -67,9 +66,8 @@ type carPhases struct { lastDrawA float64 seen bool test *phaseTest - steady int // the one phase the charger's whole draw shows on, 0 none - since time.Time phase int + three bool } // phaseVerdict reads one meter reading against the baseline of a rise of @@ -113,28 +111,6 @@ func phaseVerdict(amps, baseline [3]float64, riseA, drawA float64) int { return 0 } -// steadyVerdict is the check with no baseline: the phase (1-3) that carries -// all of the charger's draw while the other two carry less than a third of -// it, or 0. drawA is the charger's draw over one phase's voltage. -func steadyVerdict(amps [3]float64, drawA float64) int { - if !finite(drawA) || drawA < 6 { - return 0 - } - phase, quiet := 0, 0 - for i, a := range amps { - switch { - case a >= drawA-1: - phase = i + 1 - case a < drawA/3-1: - quiet++ - } - } - if quiet == 2 { - return phase - } - return 0 -} - // SetSitePhases wires the site meter's phase currents for the one-phase car // check. nil turns the check off. func (c *Controller) SetSitePhases(f func() (SitePhases, bool)) { @@ -187,7 +163,7 @@ func (m *Manager) observePhases(id string, meter SitePhases, drawA, voltage floa previous, previousDrawA, seen := s.last, s.lastDrawA, s.seen s.last, s.lastDrawA, s.seen = meter.A, drawA, true if !lp.commandedKnown || lp.commandedPhases != 3 || !finite(lp.commandedW) { - s.test, s.rising, s.steady = nil, false, 0 + s.test, s.rising = nil, false return } orderA := math.Max(0, lp.commandedW) / (3 * voltage) @@ -204,12 +180,11 @@ func (m *Manager) observePhases(id string, meter SitePhases, drawA, voltage floa s.rising, s.before, s.beforeDrawA = true, previous, previousDrawA } if !s.rising || orderA-s.orderA < onePhaseStepA { - m.observeSteadyPhase(s, meter.A, drawA, now) return } s.test = &phaseTest{at: now, until: now.Add(settle + onePhaseHold), baseline: s.before, baseDrawA: s.beforeDrawA, fromA: s.orderA} - s.orderA, s.rising, s.steady = orderA, false, 0 + s.orderA, s.rising = orderA, false } t := s.test s.orderA = max(s.orderA, orderA) @@ -222,24 +197,10 @@ func (m *Manager) observePhases(id string, meter SitePhases, drawA, voltage floa if t.verdict == 0 || now.Sub(t.since) < onePhaseHold { return } - s.phase = max(t.verdict, 0) + s.phase, s.three = max(t.verdict, 0), t.verdict < 0 s.test = nil } -// observeSteadyPhase marks an unmarked car whose charger's whole draw has -// shown on one phase alone for onePhaseHold. Caller holds m.mu. -func (m *Manager) observeSteadyPhase(s *carPhases, amps [3]float64, drawA float64, now time.Time) { - if s.phase != 0 { - return - } - if v := steadyVerdict(amps, drawA); v != s.steady { - s.steady, s.since = v, now - } - if s.steady != 0 && now.Sub(s.since) >= onePhaseHold { - s.phase = s.steady - } -} - // chargerPhaseMaxA is the most current a charger offers on each phase of a // three-phase order: its maximum over three phases, within the site fuse. func chargerPhaseMaxA(cfg Config, site SiteFuse) float64 { diff --git a/go/internal/loadpoint/one_phase_car_test.go b/go/internal/loadpoint/one_phase_car_test.go index 35a268884..4d84b9e07 100644 --- a/go/internal/loadpoint/one_phase_car_test.go +++ b/go/internal/loadpoint/one_phase_car_test.go @@ -83,8 +83,8 @@ func TestOnePhaseCarMarkedFromTheMeter(t *testing.T) { return threePhase(16)(d) } run(125*time.Second, 200*time.Second, allThree, 11040) - if st := state(); st.CarPhases != 0 { - t.Fatalf("a rise on all three phases left the mark on L%d", st.CarPhase) + if st := state(); st.CarPhases != 3 || st.CarPhase != 0 { + t.Fatalf("a rise on all three phases: car phases %d on L%d, want 3", st.CarPhases, st.CarPhase) } order(4830) run(205*time.Second, 210*time.Second, still, 0) @@ -219,28 +219,38 @@ func TestHeatPumpDoesNotClearAOnePhaseCar(t *testing.T) { } } -// Review M3: after a Core restart the order is out and the car draws before -// the first reading. With no baseline, a one-phase car still shows its whole -// draw on one phase and less than a third on the others; a three-phase car -// never does. -func TestOnePhaseCarMarkedAfterARestart(t *testing.T) { +// After a Core restart the order is out and the car draws before the first +// reading, so there is no baseline. No car is marked from one reading: a +// house load on L1 and solar on L2/L3 look the same as a one-phase car. The +// next rise marks a one-phase car. +func TestOnePhaseCarMarkedOnTheRiseAfterARestart(t *testing.T) { + still := func(a [3]float64) func(time.Duration) [3]float64 { return func(time.Duration) [3]float64 { return a } } + draw := func(w float64) func(time.Duration) float64 { return func(time.Duration) float64 { return w } } for _, c := range []struct { name string amps [3]float64 drawW float64 - want int }{ - {"one-phase car at 16 A on L1, house 3/1/1 A", [3]float64{19, 1, 1}, 3680, 1}, - {"three-phase car at 16 A, house 3/1/1 A", [3]float64{19, 17, 17}, 11040, 0}, - {"two-phase car at 16 A, house 3/1/1 A", [3]float64{19, 17, 1}, 7360, 0}, + {"one-phase car at 16 A on L1, house 3/1/1 A", [3]float64{19, 1, 1}, 3680}, + {"three-phase car at 16 A, house 3/1/1 A", [3]float64{19, 17, 17}, 11040}, + {"two-phase car at 16 A, house 3/1/1 A", [3]float64{19, 17, 1}, 7360}, } { r := newPhaseRig() r.order(11040) - r.run(0, 60*time.Second, func(time.Duration) [3]float64 { return c.amps }, func(time.Duration) float64 { return c.drawW }) - if got := r.carPhases(); got != c.want { - t.Errorf("%s: car phases %d, want %d", c.name, got, c.want) + r.run(0, 60*time.Second, still(c.amps), draw(c.drawW)) + if got := r.carPhases(); got != 0 { + t.Errorf("%s: marked %d-phase with no rise", c.name, got) } } + r := newPhaseRig() + r.order(6 * 690) + r.run(0, 60*time.Second, still([3]float64{9, 1, 1}), draw(6*230)) + r.order(8 * 690) + r.run(65*time.Second, 70*time.Second, still([3]float64{9, 1, 1}), draw(6*230)) + r.run(75*time.Second, 200*time.Second, still([3]float64{11, 1, 1}), draw(8*230)) + if got := r.carPhases(); got != 1 { + t.Fatalf("a 2 A rise on L1 alone after a restart: car phases %d, want 1", got) + } } // Device-simulator rig s14 on 2e847104: a three-phase charger with the diff --git a/go/internal/loadpoint/one_phase_house_load_test.go b/go/internal/loadpoint/one_phase_house_load_test.go new file mode 100644 index 000000000..1f87be658 --- /dev/null +++ b/go/internal/loadpoint/one_phase_house_load_test.go @@ -0,0 +1,115 @@ +package loadpoint + +import ( + "context" + "encoding/json" + "testing" + "time" +) + +// A three-phase car at 6 A (4140 W, 18 A over one phase's voltage), an oven +// of 3 kW on L1 and a little PV on every phase: the meter reads +// [17, 4.5, 4.5] A. No three-phase car should be marked one-phase. +func TestHouseLoadDoesNotMarkAThreePhaseCar(t *testing.T) { + r := newPhaseRig() + r.order(4140) + r.run(0, 60*time.Second, func(time.Duration) [3]float64 { return [3]float64{17, 4.5, 4.5} }, + func(time.Duration) float64 { return 4140 }) + if got := r.carPhases(); got != 0 { + t.Errorf("three-phase car at 6 A marked as %d-phase from meter [17 4.5 4.5] A", got) + } +} + +// End to end through Controller.Tick: a planned 4140 W three-phase slot, +// the same meter and charger readings every 5 s. The order must stay near +// 4140 W. +func TestHouseLoadKeepsAPlannedThreePhaseOrder(t *testing.T) { + cfg := Config{ID: "garage", DriverName: "easee", MinChargeW: 4140, MaxChargeW: 11040, PhaseMode: "3p"} + start := time.Now() + slot := start.Add(-time.Second) + dir := Directive{SlotStart: slot, SlotEnd: slot.Add(time.Hour), LoadpointEnergyWh: map[string]float64{cfg.ID: 4140}} + var orders []float64 + send := func(_ context.Context, _ string, p []byte) error { + var payload map[string]any + _ = json.Unmarshal(p, &payload) + w, _ := payload["power_w"].(float64) + orders = append(orders, w) + return nil + } + var now time.Time + c := NewController(NewManager(), PlanFunc(func(time.Time) (Directive, bool) { return dir, true }), + TelemetryFunc(func(string) (EVSample, bool) { + return EVSample{Connected: true, RequestActive: true, PowerW: 4140, PowerAt: now, PowerMaxAge: 10 * time.Second}, true + }), send) + c.manager.Load([]Config{cfg}) + c.SetSiteFuse(SiteFuse{MaxAmps: 25, Voltage: 230, PhaseCnt: 3}) + c.SetSitePhases(func() (SitePhases, bool) { return SitePhases{A: [3]float64{17, 4.5, 4.5}, At: now}, true }) + c.manager.Observe(cfg.ID, true, 0, 0, true) + c.manager.SetSchedule(cfg.ID, Schedule{SoC: .8}) + for d := time.Duration(0); d <= 300*time.Second; d += 5 * time.Second { + now = start.Add(d) + c.Tick(context.Background(), now) + } + for _, w := range orders { + if w > 6000 { + t.Fatalf("a planned 4140 W three-phase slot went out as %.0f W", w) + } + } +} + +// The same site with a closed loop: a real three-phase car takes the order +// on all three phases, so the meter is the house [12, -1.5, -1.5] A plus the +// order's current on each phase. The order must not swing between 6 A and +// 16 A. +func TestHouseLoadDoesNotSwingAThreePhaseOrder(t *testing.T) { + cfg := Config{ID: "garage", DriverName: "easee", MinChargeW: 4140, MaxChargeW: 11040, PhaseMode: "3p"} + start := time.Now() + slot := start.Add(-time.Second) + dir := Directive{SlotStart: slot, SlotEnd: slot.Add(time.Hour), LoadpointEnergyWh: map[string]float64{cfg.ID: 4140}} + var orders []float64 + drawW := 0.0 + send := func(_ context.Context, _ string, p []byte) error { + var payload map[string]any + _ = json.Unmarshal(p, &payload) + w, _ := payload["power_w"].(float64) + orders = append(orders, w) + return nil + } + var now time.Time + c := NewController(NewManager(), PlanFunc(func(time.Time) (Directive, bool) { return dir, true }), + TelemetryFunc(func(string) (EVSample, bool) { + return EVSample{Connected: true, RequestActive: true, PowerW: drawW, PowerAt: now, PowerMaxAge: 10 * time.Second}, true + }), send) + c.manager.Load([]Config{cfg}) + c.SetSiteFuse(SiteFuse{MaxAmps: 35, Voltage: 230, PhaseCnt: 3}) + c.SetSitePhases(func() (SitePhases, bool) { + a := drawW / 690 + return SitePhases{A: [3]float64{12 + a, -1.5 + a, -1.5 + a}, At: now}, true + }) + c.manager.Observe(cfg.ID, true, 0, 0, true) + c.manager.SetSchedule(cfg.ID, Schedule{SoC: .8}) + marks, prev := 0, 0 + for d := time.Duration(0); d <= 15*time.Minute; d += 5 * time.Second { + now = start.Add(d) + c.Tick(context.Background(), now) + if len(orders) > 0 { + drawW = orders[len(orders)-1] // the car follows the order by the next tick + } + st, _ := c.manager.State(cfg.ID) + if st.CarPhases != prev { + if st.CarPhases == 1 { + marks++ + } + prev = st.CarPhases + } + } + hi := 0 + for _, w := range orders { + if w > 6000 { + hi++ + } + } + if marks > 0 || hi > 0 { + t.Fatalf("three-phase car marked one-phase %d times; %d of %d orders above 6 kW on a 4140 W plan", marks, hi, len(orders)) + } +} diff --git a/go/internal/loadpoint/planning_steps.go b/go/internal/loadpoint/planning_steps.go index 1e2614a2c..538d70bc3 100644 --- a/go/internal/loadpoint/planning_steps.go +++ b/go/internal/loadpoint/planning_steps.go @@ -73,6 +73,22 @@ func PlanningLimits(st State, cfg Config, site SiteFuse) (minW, maxW float64, st return LimitPower(st, planning) } +// PlanningLimits is PlanningLimits on this loadpoint's charger and the site +// fuse, as both stand now. The deadline reserve holds to them, as the plan +// does. Without a controller the reserve holds to the charger's own table. +func (c *Controller) PlanningLimits(st State) (minW, maxW float64, steps []float64) { + if c == nil { + return LimitPower(st, st.AllowedStepsW) + } + cfg := Config{MinChargeW: st.MinChargeW, MaxChargeW: st.MaxChargeW, AllowedStepsW: st.AllowedStepsW} + for _, lp := range c.manager.Configs() { + if lp.ID == st.ID { + cfg.PhaseMode = lp.PhaseMode + } + } + return PlanningLimits(st, cfg, c.siteFuse()) +} + // ThreePhaseCapable reports whether a charger can run three phases at 6 A // each; see threePhaseCapable. func ThreePhaseCapable(cfg Config, site SiteFuse) bool { return threePhaseCapable(cfg, site) } diff --git a/go/internal/loadpoint/surplus_hold_fuse_test.go b/go/internal/loadpoint/surplus_hold_fuse_test.go new file mode 100644 index 000000000..a63a65f0c --- /dev/null +++ b/go/internal/loadpoint/surplus_hold_fuse_test.go @@ -0,0 +1,51 @@ +package loadpoint + +import ( + "context" + "testing" + "time" +) + +func surplusHoldFuse(t *testing.T, aConnected bool) (map[string]float64, string) { + t.Helper() + now := time.Date(2026, 10, 21, 20, 0, 0, 0, time.Local) + steps := []float64{0, 4140, 5520, 6900, 8280, 9660, 11040} + configs := []Config{ + {ID: "a", DriverName: "charger-a", SurplusOnly: true, MinChargeW: 4140, MaxChargeW: 11040, AllowedStepsW: steps, PhaseMode: "3p"}, + {ID: "b", DriverName: "charger-b", SurplusOnly: true, MinChargeW: 4140, MaxChargeW: 11040, AllowedStepsW: steps, PhaseMode: "3p"}, + } + sender := &fakeSender{} + c := newTestController(t, configs, &Directive{SlotStart: now, SlotEnd: now.Add(15 * time.Minute)}, + map[string]EVSample{"charger-a": {Connected: aConnected, RequestActive: aConnected}, "charger-b": {Connected: true, RequestActive: true}}, sender) + c.SetSiteFuse(SiteFuse{MaxAmps: 25, Voltage: 230, PhaseCnt: 3}) + c.SetSiteSurplusForEV(func() (float64, bool) { return 0, true }) + c.SetFuseEVMax(func() (float64, bool) { return 8000, true }) + c.SetManualHold("b", ManualHold{PowerW: 11040, Persistent: true}) + c.Tick(context.Background(), now) + got := map[string]float64{} + for _, cmd := range sender.calls { + if cmd.action == "ev_set_current" { + got[cmd.driver] = cmd.power + } + } + st, _ := c.manager.State("b") + t.Logf("orders %v, b reason %q, frozen caps %v", got, st.CommandedReason, c.tickFuseCaps) + return got, st.CommandedReason +} + +// Two solar-only chargers, no PV surplus, EV fuse cap 8 kW. Charge now on b +// must get a share of the cap; idle solar-only a orders 0 W. +func TestChargeNowOnSurplusOnlyGetsFuseShare(t *testing.T) { + got, reason := surplusHoldFuse(t, true) + if got["charger-b"] <= 0 { + t.Fatalf("Charge now on b got %v W (%q) while a ordered %v W; cap 8000 W", got["charger-b"], reason, got["charger-a"]) + } +} + +// Control: with a unplugged, Charge now on b charges. +func TestChargeNowOnSurplusOnlyAlone(t *testing.T) { + got, reason := surplusHoldFuse(t, false) + if got["charger-b"] <= 0 { + t.Fatalf("b alone got %v W (%q)", got["charger-b"], reason) + } +} diff --git a/go/internal/localzone/localzone.go b/go/internal/localzone/localzone.go index c6555245b..3c33cc07c 100644 --- a/go/internal/localzone/localzone.go +++ b/go/internal/localzone/localzone.go @@ -1,7 +1,12 @@ -// Package localzone names the zone the box keeps its clock in. +// Package localzone keeps the household's clock: the zone every wall-clock +// rule reads, and its name for clients. package localzone import ( + "bytes" + "errors" + "fmt" + "io/fs" "os" "path/filepath" "strings" @@ -16,21 +21,90 @@ import ( // persisted clock buckets keep its DST rules, so it must be a name that // time.LoadLocation and Intl accept. UTC is valid when that is the box's // configured zone; never guess a country. -func Name() string { return name() } +// +// The name must keep the rules time.Local runs on. Clients read the box's +// household clock (a car's ready time, a goal's days) in this zone, so a name +// with other rules moves every goal by the difference. +// +// A zone the site stores (Use) wins over the host's. +func Name() string { + if household != "" { + return household + } + return name() +} + +// household is the zone the site stores, set once by Use. +var household string + +// Stored returns the zone the site stores, or "" when the box runs on the +// host's zone. +func Stored() string { return household } + +// Known reports whether the box has a household clock: a zone the site +// stores, or a host zone other than UTC. A Docker box has none until a +// client offers one. +func Known() bool { + return household != "" || !utcNames[strings.TrimPrefix(name(), "Etc/")] +} + +// Use makes the site's stored zone the household clock: time.Local, and the +// Name clients read. A Docker box runs on UTC whatever house it is in, so a +// car goal of 17:00 was met at 18:00 in a Stockholm winter until the zone +// became a fact of the site rather than of the host. +// +// Core calls it once at start, before any goroutine reads the clock: +// assigning time.Local later races every time.Now. A change takes effect +// when Core next starts. Empty keeps the host's zone and names it again. +func Use(zone string) error { + if zone == "" { + household = "" + return nil + } + loc, err := Load(zone) + if err != nil { + return err + } + time.Local = loc + household = zone + // A worker Core starts reads the same clock. + return os.Setenv("TZ", zone) +} + +// Load reads an IANA zone a site may store. "Local" and "" name no zone. +func Load(zone string) (*time.Location, error) { + if zone == "" || zone == "Local" { + return nil, errors.New("name a time zone such as Europe/Stockholm") + } + loc, err := time.LoadLocation(zone) + if err != nil { + return nil, fmt.Errorf("%q is not a time zone such as Europe/Stockholm", zone) + } + return loc, nil +} + +// SameRules reports whether a and b keep the same clock from 1970 to 2038. +func SameRules(a, b *time.Location) bool { return sameRules(a, b) } // Read once: time.Local is fixed when the process starts, so a zone changed // on the host afterwards must not rename the one the box still runs on. var name = sync.OnceValue(func() string { - return resolve(os.Getenv("TZ"), time.Local.String(), "/etc/localtime", "/etc/timezone") + return resolve(os.Getenv("TZ"), time.Local, "/etc/localtime", "/etc/timezone", zoneinfoRoots) }) -func resolve(tz, local, localtime, timezone string) string { - candidates := []string{strings.TrimPrefix(tz, ":"), local} +// zoneinfoRoots are where hosts keep the zone files a copied /etc/localtime +// comes from. +var zoneinfoRoots = []string{"/usr/share/zoneinfo", "/usr/lib/zoneinfo", "/usr/share/lib/zoneinfo"} + +func resolve(tz string, local *time.Location, localtime, timezone string, roots []string) string { + candidates := []string{strings.TrimPrefix(tz, ":"), local.String()} if target, err := filepath.EvalSymlinks(localtime); err == nil { if _, zone, ok := strings.Cut(target, "/zoneinfo/"); ok { candidates = append(candidates, zone) } } + // Debian keeps /etc/timezone beside /etc/localtime, and a container + // that mounts only the latter keeps the image's stale name. if data, err := os.ReadFile(timezone); err == nil { candidates = append(candidates, strings.TrimSpace(string(data))) } @@ -38,9 +112,105 @@ func resolve(tz, local, localtime, timezone string) string { if zone == "" || zone == "Local" { continue } - if _, err := time.LoadLocation(zone); err == nil { + if loc, err := time.LoadLocation(zone); err == nil && sameRules(loc, local) && meansWhatItSays(zone, local) { return zone } } + // A copied file, such as a bind-mounted /etc/localtime, carries the + // rules Go runs on but no name. + data, _ := os.ReadFile(localtime) + if zone := matchingZone(local, data, roots); zone != "" { + return zone + } return "UTC" } + +// matchingZone names a zone file under roots with local's rules: one with +// exactly these bytes if there is one, else the first in lexical order. Links +// and the posix/right variants repeat a zone under other names, so only +// regular Area/City files count. +func matchingZone(local *time.Location, data []byte, roots []string) string { + first := "" + for _, root := range roots { + exact := "" + _ = filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error { + if err != nil { + return nil + } + rel, _ := filepath.Rel(root, path) + rel = filepath.ToSlash(rel) + if d.IsDir() { + if rel == "posix" || rel == "right" { + return filepath.SkipDir + } + return nil + } + if !d.Type().IsRegular() || !strings.Contains(rel, "/") { + return nil + } + b, err := os.ReadFile(path) + if err != nil { + return nil + } + loc, err := time.LoadLocationFromTZData(rel, b) + if err != nil || !sameRules(loc, local) || !meansWhatItSays(rel, local) { + return nil + } + if _, err := time.LoadLocation(rel); err != nil { + return nil + } + if bytes.Equal(b, data) { + exact = rel + return filepath.SkipAll + } + if first == "" { + first = rel + } + return nil + }) + if exact != "" { + return exact + } + } + return first +} + +// utcNames are the names clients read as UTC. +var utcNames = map[string]bool{ + "UTC": true, "UCT": true, "Universal": true, "Zulu": true, + "GMT": true, "GMT0": true, "GMT+0": true, "GMT-0": true, "Greenwich": true, +} + +// meansWhatItSays rejects a UTC name for a clock that is not UTC. Debian +// links /etc/localtime to Etc/UTC, so mounting the host's file over it +// overwrites Etc/UTC itself: that name then loads the host's rules here, +// and UTC on every client. +func meansWhatItSays(name string, local *time.Location) bool { + return !utcNames[strings.TrimPrefix(name, "Etc/")] || sameRules(time.UTC, local) +} + +// sameRules reports whether a and b show the same UTC offset at every +// instant from 1970 to 2038. Older history differs between tzdata releases +// and moves no clock the box keeps. +func sameRules(a, b *time.Location) bool { + return follows(a, b) && follows(b, a) +} + +// follows reports whether b shows a's offset at the start of each of a's +// periods. Run both ways, that covers every change in either zone. +func follows(a, b *time.Location) bool { + end := time.Date(2038, 1, 1, 0, 0, 0, 0, time.UTC) + for t := time.Date(1970, 1, 1, 0, 0, 0, 0, time.UTC); t.Before(end); { + in := t.In(a) + _, want := in.Zone() + if _, got := t.In(b).Zone(); got != want { + return false + } + _, next := in.ZoneBounds() + if !next.After(t) { + return true + } + t = next + } + return true +} diff --git a/go/internal/localzone/localzone_test.go b/go/internal/localzone/localzone_test.go index ed4e35f36..9c96dc617 100644 --- a/go/internal/localzone/localzone_test.go +++ b/go/internal/localzone/localzone_test.go @@ -3,6 +3,7 @@ package localzone import ( "os" "path/filepath" + "strings" "testing" "time" _ "time/tzdata" // the binary embeds it too, so the zone names resolve as they do on a box @@ -34,41 +35,144 @@ func host(t *testing.T, linkedZone, timezoneFile string) (localtime, timezone st return localtime, timezone } +// zone loads a zone the way Go loads time.Local. +func zone(t *testing.T, name string) *time.Location { + t.Helper() + loc, err := time.LoadLocation(name) + if err != nil { + t.Fatal(err) + } + return loc +} + func TestAHostWithoutTZIsNamedByItsLocaltimeLink(t *testing.T) { localtime, timezone := host(t, "Europe/Stockholm", "") - if got := resolve("", "Local", localtime, timezone); got != "Europe/Stockholm" { + if got := resolve("", zone(t, "Europe/Stockholm"), localtime, timezone, nil); got != "Europe/Stockholm" { t.Fatalf("zone = %q, want Europe/Stockholm", got) } } func TestTZNamesTheZoneBeforeTheHostFiles(t *testing.T) { localtime, timezone := host(t, "Europe/Stockholm", "Europe/Oslo\n") - if got := resolve(":Europe/Berlin", "Europe/Berlin", localtime, timezone); got != "Europe/Berlin" { + if got := resolve(":Europe/Berlin", zone(t, "Europe/Berlin"), localtime, timezone, nil); got != "Europe/Berlin" { t.Fatalf("zone = %q, want Europe/Berlin", got) } } -func TestAnUnknownTZFallsThroughToTheHost(t *testing.T) { +// Go runs on UTC when it cannot load TZ, whatever the host files say. +func TestAnUnknownTZRunsOnUTC(t *testing.T) { localtime, timezone := host(t, "Europe/Stockholm", "") - if got := resolve("Mars/Olympus", "Mars/Olympus", localtime, timezone); got != "Europe/Stockholm" { - t.Fatalf("zone = %q, want Europe/Stockholm", got) + if got := resolve("Mars/Olympus", time.UTC, localtime, timezone, nil); got != "UTC" { + t.Fatalf("zone = %q, want UTC", got) } } func TestEtcTimezoneNamesTheZoneWhenLocaltimeIsNotALink(t *testing.T) { localtime, timezone := host(t, "", "Europe/Oslo\n") - if got := resolve("", "Local", localtime, timezone); got != "Europe/Oslo" { + if got := resolve("", zone(t, "Europe/Oslo"), localtime, timezone, nil); got != "Europe/Oslo" { t.Fatalf("zone = %q, want Europe/Oslo", got) } } func TestAHostThatNamesNoZoneIsUTC(t *testing.T) { localtime, timezone := host(t, "", "") - if got := resolve("", "Local", localtime, timezone); got != "UTC" { + if got := resolve("", time.UTC, localtime, timezone, nil); got != "UTC" { t.Fatalf("zone = %q, want UTC", got) } } +// hostZoneFile is a real tzfile from this machine's zoneinfo tree. +func hostZoneFile(t *testing.T, name string) []byte { + t.Helper() + for _, root := range zoneinfoRoots { + if b, err := os.ReadFile(filepath.Join(root, filepath.FromSlash(name))); err == nil { + return b + } + } + t.Skip("no zoneinfo tree on this host") + return nil +} + +// zoneTree fakes a zoneinfo tree: name → contents, or "->" and a link target. +func zoneTree(t *testing.T, files map[string]any) string { + t.Helper() + root := t.TempDir() + for name, body := range files { + path := filepath.Join(root, filepath.FromSlash(name)) + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if target, ok := body.(string); ok { + if err := os.Symlink(filepath.Join(root, filepath.FromSlash(strings.TrimPrefix(target, "->"))), path); err != nil { + t.Fatal(err) + } + continue + } + if err := os.WriteFile(path, body.([]byte), 0o644); err != nil { + t.Fatal(err) + } + } + return root +} + +// copied is a host whose /etc/localtime is a copy of data, as a container +// gets with -v /etc/localtime:/etc/localtime:ro, beside the image's own +// /etc/timezone. It returns the zone Go runs on and the two paths. +func copied(t *testing.T, data []byte, timezoneFile string) (local *time.Location, localtime, timezone string) { + t.Helper() + localtime, timezone = host(t, "", timezoneFile) + if err := os.WriteFile(localtime, data, 0o644); err != nil { + t.Fatal(err) + } + local, err := time.LoadLocationFromTZData("Local", data) + if err != nil { + t.Fatal(err) + } + return local, localtime, timezone +} + +// A Debian container that mounts the host's localtime keeps the image's +// "Etc/UTC" in /etc/timezone. Go runs on the copied Stockholm rules; naming +// UTC put every car's ready time an hour late on the box's clients. +func TestACopiedLocaltimeIsNamedByItsZoneFileNotAStaleTimezone(t *testing.T) { + stockholm := hostZoneFile(t, "Europe/Stockholm") + root := zoneTree(t, map[string]any{ + "Arctic/Longyearbyen": "->Europe/Stockholm", + "Etc/UTC": hostZoneFile(t, "Etc/UTC"), + "Europe/Rome": hostZoneFile(t, "Europe/Rome"), + "Europe/Stockholm": stockholm, + "posix/Europe/Stockholm": stockholm, + }) + local, localtime, timezone := copied(t, stockholm, "Etc/UTC\n") + if got := resolve("", local, localtime, timezone, []string{root}); got != "Europe/Stockholm" { + t.Fatalf("zone = %q, want Europe/Stockholm", got) + } +} + +// A file from another tzdata release differs in bytes; its rules still name it. +func TestACopiedLocaltimeIsNamedByItsRules(t *testing.T) { + root := zoneTree(t, map[string]any{ + "Europe/Rome": hostZoneFile(t, "Europe/Rome"), + "Europe/Stockholm": hostZoneFile(t, "Europe/Stockholm"), + }) + local, localtime, timezone := copied(t, hostZoneFile(t, "Europe/Berlin"), "") + if got := resolve("", local, localtime, timezone, []string{filepath.Join(root, "missing"), root}); got != "Europe/Stockholm" { + t.Fatalf("zone = %q, want Europe/Stockholm", got) + } +} + +func TestZonesWithOtherRulesDiffer(t *testing.T) { + if sameRules(zone(t, "Europe/Stockholm"), zone(t, "Europe/London")) { + t.Fatal("Stockholm and London have the same rules") + } + if sameRules(zone(t, "Europe/Stockholm"), time.UTC) { + t.Fatal("Stockholm and UTC have the same rules") + } + if !sameRules(zone(t, "Etc/UTC"), time.UTC) { + t.Fatal("Etc/UTC and UTC differ") + } +} + func TestNameIsAZoneClientsCanLoad(t *testing.T) { got := Name() if got == "Local" || got == "" { @@ -78,3 +182,75 @@ func TestNameIsAZoneClientsCanLoad(t *testing.T) { t.Fatalf("Name() = %q does not load: %v", got, err) } } + +// Debian links /etc/localtime to Etc/UTC, so a container that mounts the +// host's localtime writes the host's rules over Etc/UTC itself. The link and +// the file then both say Etc/UTC; clients would read the box's clock as UTC. +func TestAMountOverTheEtcUTCLinkIsNotUTC(t *testing.T) { + stockholm := hostZoneFile(t, "Europe/Stockholm") + root := zoneTree(t, map[string]any{ + "Etc/UTC": stockholm, + "Europe/Rome": hostZoneFile(t, "Europe/Rome"), + "Europe/Stockholm": stockholm, + }) + localtime, timezone := host(t, "", "") + if err := os.Symlink(filepath.Join(root, "Etc", "UTC"), localtime); err != nil { + t.Fatal(err) + } + local, err := time.LoadLocationFromTZData("Local", stockholm) + if err != nil { + t.Fatal(err) + } + t.Setenv("ZONEINFO", root) // Go loads Etc/UTC from the overwritten tree, as in the container + if got := resolve("", local, localtime, timezone, []string{root}); got != "Europe/Stockholm" { + t.Fatalf("zone = %q, want Europe/Stockholm", got) + } +} + +// keepClock restores the process clock a test moves. +func keepClock(t *testing.T) { + t.Helper() + local, stored := time.Local, household + t.Setenv("TZ", os.Getenv("TZ")) + t.Cleanup(func() { time.Local, household = local, stored }) +} + +// A Docker box runs on UTC. Once the site stores its zone, the box's own +// clock reads the household's hours. +func TestAStoredZoneIsTheClockOfABoxOnUTC(t *testing.T) { + keepClock(t) + time.Local = time.UTC + if err := Use("Europe/Stockholm"); err != nil { + t.Fatal(err) + } + winter := time.Date(2026, 11, 1, 16, 0, 0, 0, time.UTC) + if got := time.UnixMilli(winter.UnixMilli()).Hour(); got != 17 { + t.Fatalf("16:00Z reads %d:00 on the box, want 17:00", got) + } + if Name() != "Europe/Stockholm" || Stored() != "Europe/Stockholm" || os.Getenv("TZ") != "Europe/Stockholm" { + t.Fatalf("name %q stored %q TZ %q, want Europe/Stockholm", Name(), Stored(), os.Getenv("TZ")) + } +} + +func TestNoStoredZoneKeepsTheHostClock(t *testing.T) { + keepClock(t) + host := time.Local + if err := Use(""); err != nil { + t.Fatal(err) + } + if time.Local != host || Stored() != "" { + t.Fatalf("Use(\"\") moved the clock to %v, stored %q", time.Local, Stored()) + } +} + +func TestAZoneThatDoesNotLoadIsRefused(t *testing.T) { + keepClock(t) + for _, zone := range []string{"Local", "Mars/Olympus", "../../etc/passwd"} { + if err := Use(zone); err == nil { + t.Fatalf("Use(%q) took a zone no client can load", zone) + } + } + if Stored() != "" { + t.Fatalf("stored %q after refusals", Stored()) + } +} diff --git a/go/internal/modbus/client.go b/go/internal/modbus/client.go index 60fcbf0b2..393c537ae 100644 --- a/go/internal/modbus/client.go +++ b/go/internal/modbus/client.go @@ -21,7 +21,10 @@ import ( const ( reconnectBackoffBase = 2 * time.Second - reconnectBackoffMax = 60 * time.Second + // A device that comes back must be reached within 10 s: until then it + // may hold the last forced battery mode. A dead endpoint costs one + // connect attempt per 10 s, well under the eviction threshold below. + reconnectBackoffMax = 10 * time.Second // Many inverters accept exactly one Modbus TCP session and evict the // previous one on every new connect. Redials above this rate almost diff --git a/go/internal/modbus/client_test.go b/go/internal/modbus/client_test.go index 8050c592d..44de23120 100644 --- a/go/internal/modbus/client_test.go +++ b/go/internal/modbus/client_test.go @@ -474,10 +474,10 @@ func TestReconnectBackoffSchedule(t *testing.T) { 2: 2 * time.Second, 3: 4 * time.Second, 4: 8 * time.Second, - 5: 16 * time.Second, - 6: 32 * time.Second, - 7: 60 * time.Second, - 8: 60 * time.Second, + 5: 10 * time.Second, + 6: 10 * time.Second, + 7: 10 * time.Second, + 8: 10 * time.Second, } for failures, delay := range want { c.consecutiveTransportFailures = failures diff --git a/go/internal/mpc/charging_periods.go b/go/internal/mpc/charging_periods.go index 92a840d75..e0079e727 100644 --- a/go/internal/mpc/charging_periods.go +++ b/go/internal/mpc/charging_periods.go @@ -1,6 +1,9 @@ package mpc -import "context" +import ( + "context" + "math" +) // ChargingPeriods describes a preference, after deadline energy and physical // reserves. It never delays Core safety, manual stop, or charge completion. @@ -41,7 +44,7 @@ func (l *LoadpointSpec) departureMissWh() float64 { return 1 } -func (o *EnergyplanOptimizer) prepareEnergyplanRequest(ctx context.Context, request *externalRequest, p Params) error { +func (o *EnergyplanOptimizer) prepareEnergyplanRequest(ctx context.Context, request *externalRequest, slots []Slot, p Params) error { // Negotiate against the running worker on every request. Restart or a // binary rollback must not inherit a feature learned from another process. probeCtx, cancel := context.WithTimeout(ctx, o.cfg.Timeout) @@ -55,6 +58,31 @@ func (o *EnergyplanOptimizer) prepareEnergyplanRequest(ctx context.Context, requ request.Slots[i].Confidence = 0 } } + // A worker without demand_elapsed_hours gets the demand charges as + // before: no hour that began before the first slot, each completed day + // in already_kw. Its budget follows the charges it gets, so a charge with + // only the hour in progress leaves with its fleet budget. + if !optimizerHasFeature(info, "demand_elapsed_hours") && hasElapsedDemand(p.DemandCharges) { + p.DemandCharges = legacyDemandCharges(p.DemandCharges) + request.DemandCharges = externalDemandCharges(p.DemandCharges) + if o.timeBudget != nil { + request.Settings.TimeLimitS = math.Min(o.cfg.Timeout.Seconds()*0.8, o.timeBudget(slots, p).Seconds()) + } + } + // A worker without the hours plans past them; Core's check rejects + // such a plan and uses its own. The worker plans the hours in its fleet + // model, with the demand charges' budget. + if optimizerHasFeature(info, "hourly_import_caps") && len(slots) > 0 { + for _, h := range p.ImportHours { + if importHourPlannedH(slots, h) > 0 { + request.HourlyImportCaps = append(request.HourlyImportCaps, externalImportCap{ + StartMs: h.StartMs, EndMs: h.EndMs, MaxMeanW: p.ImportMeanW, ElapsedImportWh: h.ElapsedImportWh}) + } + } + if len(request.HourlyImportCaps) > 0 { + request.Settings.TimeLimitS = math.Min(o.cfg.Timeout.Seconds()*0.8, energyplanBudget(slots, p, true).Seconds()) + } + } loads := p.activeLoadpoints() needed := false for _, lp := range loads { diff --git a/go/internal/mpc/charging_periods_test.go b/go/internal/mpc/charging_periods_test.go index 75f73cccd..a8614e193 100644 --- a/go/internal/mpc/charging_periods_test.go +++ b/go/internal/mpc/charging_periods_test.go @@ -34,7 +34,7 @@ func TestChargingPeriodsNegotiatesEachWorkerAndKeepsLegacyWire(t *testing.T) { if r.FlexLoads[0].Charging != nil { t.Fatal("generic sidecar received an unnegotiated field") } - if err := o.prepareEnergyplanRequest(context.Background(), &r, p); err != nil { + if err := o.prepareEnergyplanRequest(context.Background(), &r, slots, p); err != nil { t.Fatal(err) } c := r.FlexLoads[0].Charging diff --git a/go/internal/mpc/deadline_directive.go b/go/internal/mpc/deadline_directive.go index 6d7005a4a..030b96cd9 100644 --- a/go/internal/mpc/deadline_directive.go +++ b/go/internal/mpc/deadline_directive.go @@ -7,6 +7,10 @@ import ( "github.com/srcfl/ftw/go/internal/loadpoint" ) +// deadlineReserveMargin is one slot at the planner's longest (hourly) price +// slot. +const deadlineReserveMargin = time.Hour + // withDeadlineReserve supplies current EV intent. It never activates or // invents an optimizer plan, battery energy, a price or a solar forecast. func (s *Service) withDeadlineReserve(now time.Time, d SlotDirective, accepted bool) (SlotDirective, bool) { @@ -14,14 +18,38 @@ func (s *Service) withDeadlineReserve(now time.Time, d SlotDirective, accepted b return d, accepted } states := s.LoadpointStates() // No control lock: the caller may hold it. + limits := s.PlanningLimits + if limits == nil { + limits = func(st loadpoint.State) (float64, float64, []float64) { + return loadpoint.LimitPower(st, st.AllowedStepsW) + } + } if !accepted { d = SlotDirective{SlotStart: now, SlotEnd: now.Add(time.Minute), EVOnly: true} } + // Each saved goal's hours at its own peak. Cars may share one fuse + // allowance, which the controller gives earliest deadline first, so a + // goal's latest start counts every goal due at or before it (#1718). + type goalHours struct { + due time.Time + hours float64 + } + var goals []goalHours + for _, st := range states { + if st.ID == "" || st.ManualActive || st.SurplusOnly { + continue + } + minW, maxW, steps := limits(st) + if need, peak, ok := loadpoint.DeadlineNeed(st, minW, maxW, steps); ok { + goals = append(goals, goalHours{due: st.TargetTime, hours: need / peak}) + } + } for _, st := range states { if _, hasBudget := d.LoadpointEnergyWh[st.ID]; hasBudget && !st.ManualActive { continue // An accepted zero budget is a price decision, too. } - need, peak, ok := loadpoint.DeadlineNeed(st) + minW, maxW, steps := limits(st) + need, peak, ok := loadpoint.DeadlineNeed(st, minW, maxW, steps) if st.ManualActive && st.PluggedIn { // Manual intent has already passed the charger's own clamps. // Reserve its ceiling, including an explicit Stop, without @@ -36,10 +64,35 @@ func (s *Service) withDeadlineReserve(now time.Time, d SlotDirective, accepted b continue } if !st.ManualActive { + // A goal with slack waits for its latest start: the time the + // remaining need of every goal due by its deadline takes at + // full power, one car after another, plus an hourly price slot. + // Before that the car holds 0 W, as at startup, so a missing + // plan does not start a car the next plan would pause (#1718). + // A running reserve keeps a second slot, so one SoC step does + // not stop and restart it. + margin := deadlineReserveMargin + if st.CommandedKnown && st.CommandedW > 0 { + margin *= 2 + } + var hours float64 + for _, g := range goals { + if !g.due.After(st.TargetTime) { + hours += g.hours + } + } + latestStart := st.TargetTime.Add(-time.Duration(hours * float64(time.Hour))).Add(-margin) + if now.Before(latestStart) { + continue + } // Near the goal use the smallest legal offer that can deliver // this minute's need, rather than reserving the charger's full - // rating for a final few Wh. - peak = loadpoint.SnapChargeW(need/d.SlotEnd.Sub(now).Hours(), st.MinChargeW, peak, st.AllowedStepsW) + // rating for a final few Wh. A charger without a table takes + // any power in its range. + if len(st.AllowedStepsW) == 0 { + steps = nil + } + peak = loadpoint.SnapChargeW(need/d.SlotEnd.Sub(now).Hours(), minW, peak, steps) } if d.LoadpointFallback == nil { d.LoadpointFallback = map[string]bool{} diff --git a/go/internal/mpc/deadline_directive_test.go b/go/internal/mpc/deadline_directive_test.go index 11da0195d..0cee1974c 100644 --- a/go/internal/mpc/deadline_directive_test.go +++ b/go/internal/mpc/deadline_directive_test.go @@ -131,3 +131,44 @@ func TestGoalReserveGivesASmallCarTheChargersMinimum(t *testing.T) { } } } + +// Review of #1807: on a table with one-phase and three-phase levels the +// reserve holds to the limits the plan gets, as planned charging runs on +// three phases. A 3.7 kW car is reserved the lowest three-phase level, at +// full need and near its goal; marked one-phase, it is reserved the third of +// a three-phase level it takes. A one-phase level was ordered 0 W. +func TestGoalReserveHoldsToThePlansLimits(t *testing.T) { + now := time.Now() + steps := []float64{0} + for _, phases := range []float64{1, 3} { + for a := 6.0; a <= 16; a++ { + steps = append(steps, a*230*phases) + } + } + site := loadpoint.SiteFuse{MaxAmps: 16, Voltage: 230, PhaseCnt: 3} + for _, c := range []struct { + name string + carPhases int + soc float64 + want float64 + }{ + {"a 3.7 kW car", 0, .3, 4140}, + {"a 3.7 kW car near its goal", 0, .7999, 4140}, + {"marked one-phase", 1, .3, 3680}, + {"marked one-phase near its goal", 1, .7999, 1380}, + } { + st := deadlineGoalState(now, "a") + st.MinChargeW, st.MaxChargeW, st.AllowedStepsW, st.CarMaxW, st.CarPhases, st.CurrentSoC = 1380, 11040, steps, 3680, c.carPhases, c.soc + st.TargetTime = now.Add(30 * time.Minute) // Within the latest start, even near the goal. + svc := &Service{ + LoadpointStates: func() []loadpoint.State { return []loadpoint.State{st} }, + PlanningLimits: func(st loadpoint.State) (float64, float64, []float64) { + return loadpoint.PlanningLimits(st, loadpoint.Config{MinChargeW: st.MinChargeW, MaxChargeW: st.MaxChargeW, AllowedStepsW: st.AllowedStepsW}, site) + }, + } + d, ok := svc.SlotDirectiveAt(now) + if !ok || !d.LoadpointFallback["a"] || d.LoadpointMaxPowerW["a"] != c.want || d.LoadpointEnergyWh["a"] <= 0 { + t.Errorf("%s: reserve %v W, %v Wh, ok=%v; want %v W", c.name, d.LoadpointMaxPowerW["a"], d.LoadpointEnergyWh["a"], ok, c.want) + } + } +} diff --git a/go/internal/mpc/deadline_runtime_test.go b/go/internal/mpc/deadline_runtime_test.go index 20d901a9b..a45bdff5b 100644 --- a/go/internal/mpc/deadline_runtime_test.go +++ b/go/internal/mpc/deadline_runtime_test.go @@ -100,6 +100,8 @@ func TestDeadlineReserveSmallRemainderLeavesRoomForOtherGoal(t *testing.T) { now := time.Now() a, b := deadlineGoalState(now, "a"), deadlineGoalState(now, "b") a.CurrentSoC = a.TargetSoC - loadpoint.DefaultChargeEfficiency/a.VehicleCapacityWh // 1 AC Wh. + // Within the latest start. + a.TargetTime = now.Add(30 * time.Minute) svc := &Service{LoadpointStates: func() []loadpoint.State { return []loadpoint.State{a, b} }} d, ok := svc.SlotDirectiveAt(now) if !ok || math.Abs(d.LoadpointEnergyWh["a"]-1) > 1e-6 || d.LoadpointMaxPowerW["a"] != a.MinChargeW || d.LoadpointMaxPowerW["b"] != b.MaxChargeW { diff --git a/go/internal/mpc/deadline_slack_test.go b/go/internal/mpc/deadline_slack_test.go new file mode 100644 index 000000000..590fc90bb --- /dev/null +++ b/go/internal/mpc/deadline_slack_test.go @@ -0,0 +1,213 @@ +package mpc + +import ( + "context" + "encoding/json" + "math" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/loadpoint" +) + +// #1718: a goal with hours of slack must not start the car at full +// power merely because no executable plan exists right now. +// Car 30 % -> 80 % of 60 kWh needs ~33.3 kWh AC, ~3 h at 11 kW; deadline in +// 10 h, so ~7 h of slack. +func TestDeadlineReserveWaitsForLatestStart(t *testing.T) { + for _, fault := range []string{"no plan", "failed replacement", "restored archive"} { + t.Run(fault, func(t *testing.T) { + now := time.Now() + mgr := loadpoint.NewManager() + cfg := loadpoint.Config{ID: "a", DriverName: "charger-a", VehicleCapacityWh: 60000, PluginSoC: .3, + MinChargeW: 1380, MaxChargeW: 11000, AllowedStepsW: []float64{0, 1380, 4140, 6900, 11000}} + mgr.Load([]loadpoint.Config{cfg}) + mgr.Observe(cfg.ID, true, 0, 0, true) + mgr.SetCurrentSoC(cfg.ID, .3) + mgr.SetTarget(cfg.ID, .8, now.Add(10*time.Hour)) + svc := &Service{LoadpointStates: mgr.States} + // The accepted plan holds 0 W now (cheap hours are at night). + zeroPlan := Plan{GeneratedAtMs: now.UnixMilli(), Actions: []Action{{SlotStartMs: now.UnixMilli(), SlotLenMin: 15, + LoadpointPowerW: map[string]float64{"a": 0}, LoadpointMaxPowerW: map[string]float64{"a": 11000}}}} + switch fault { + case "failed replacement": + svc.InstallPlan(zeroPlan, Params{}, cfg.ID) + svc.failedReplanGeneration = svc.publishedReplanGeneration + 1 + case "restored archive": + // RestoreDiagnostic sets s.last but never s.executionPlan. + p := zeroPlan + svc.last = &p + svc.lastLoadpointID = cfg.ID + } + var commanded float64 = -1 + ctrl := loadpoint.NewController(mgr, func(at time.Time) (loadpoint.Directive, bool) { + d, ok := svc.SlotDirectiveAt(at) + return d.LoadpointDirective(), ok + }, func(string) (loadpoint.EVSample, bool) { + return loadpoint.EVSample{Connected: true, RequestActive: true}, true + }, func(_ context.Context, _ string, payload []byte) error { + var cmd struct { + Action string `json:"action"` + PowerW float64 `json:"power_w"` + } + if err := json.Unmarshal(payload, &cmd); err != nil { + return err + } + if cmd.Action == "ev_set_current" { + commanded = cmd.PowerW + } + return nil + }) + ctrl.SetFuseEVMax(func() (float64, bool) { return 11000, true }) + ctrl.TickWithDispatch(context.Background(), now, true) + st, _ := mgr.State(cfg.ID) + t.Logf("fault=%s commanded=%.0f W reason=%q", fault, commanded, st.CommandedReason) + if commanded > 0 { + t.Fatalf("fault=%s: car started at %.0f W (%q) with ~7 h of slack before deadline", fault, commanded, st.CommandedReason) + } + }) + } +} + +// The reserve starts at the latest start: the need at full power plus one +// hourly slot before the deadline. A running reserve keeps a second slot, so +// one SoC step does not stop it and start it again a few minutes later. +func TestDeadlineReserveLatestStart(t *testing.T) { + now := time.Now() + base := deadlineGoalState(now, "a") // ~33.3 kWh at 11 kW: ~3.03 h. + need := (base.TargetSoC - base.CurrentSoC) * base.VehicleCapacityWh / loadpoint.DefaultChargeEfficiency + atFull := time.Duration(need / base.MaxChargeW * float64(time.Hour)) + for _, c := range []struct { + name string + slack time.Duration + running bool + want bool + }{ + {"inside the latest start", 59 * time.Minute, false, true}, + {"before the latest start", 61 * time.Minute, false, false}, + {"running, inside two slots", 119 * time.Minute, true, true}, + {"running, before two slots", 121 * time.Minute, true, false}, + } { + st := base + st.TargetTime = now.Add(atFull + c.slack) + if c.running { + st.CommandedKnown, st.CommandedW = true, 11000 + } + svc := &Service{LoadpointStates: func() []loadpoint.State { return []loadpoint.State{st} }} + d, _ := svc.SlotDirectiveAt(now) + if got := d.LoadpointFallback["a"]; got != c.want { + t.Errorf("%s: reserve %v, want %v (%+v)", c.name, got, c.want, d) + } + } +} + +// A loadpoint with min_charge_w 0 and a step table takes its legal minimum +// from the steps, so it gets the reserve within its latest start, and the +// controller orders power. +func TestDeadlineReserveZeroMinWithSteps(t *testing.T) { + now := time.Now() + st := deadlineGoalState(now, "a") + st.MinChargeW = 0 + st.TargetTime = now.Add(30 * time.Minute) + svc := &Service{LoadpointStates: func() []loadpoint.State { return []loadpoint.State{st} }} + d, ok := svc.SlotDirectiveAt(now) + if !ok || !d.LoadpointFallback["a"] || d.LoadpointMaxPowerW["a"] != 11000 { + t.Fatalf("no reserve for min 0 with steps: ok=%v %+v", ok, d) + } + + mgr := loadpoint.NewManager() + cfg := loadpoint.Config{ID: "a", DriverName: "charger-a", VehicleCapacityWh: 60000, PluginSoC: .3, + MinChargeW: 0, MaxChargeW: 11000, AllowedStepsW: []float64{0, 1380, 4140, 6900, 11000}} + mgr.Load([]loadpoint.Config{cfg}) + mgr.Observe(cfg.ID, true, 0, 0, true) + mgr.SetCurrentSoC(cfg.ID, .3) + mgr.SetTarget(cfg.ID, .8, now.Add(2*time.Hour)) + svc = &Service{LoadpointStates: mgr.States} + var commanded float64 = -1 + ctrl := loadpoint.NewController(mgr, func(at time.Time) (loadpoint.Directive, bool) { + d, ok := svc.SlotDirectiveAt(at) + return d.LoadpointDirective(), ok + }, func(string) (loadpoint.EVSample, bool) { + return loadpoint.EVSample{Connected: true, RequestActive: true}, true + }, func(_ context.Context, _ string, payload []byte) error { + var cmd struct { + Action string `json:"action"` + PowerW float64 `json:"power_w"` + } + if err := json.Unmarshal(payload, &cmd); err != nil { + return err + } + if cmd.Action == "ev_set_current" { + commanded = cmd.PowerW + } + return nil + }) + ctrl.SetFuseEVMax(func() (float64, bool) { return 11000, true }) + ctrl.TickWithDispatch(context.Background(), now, true) + if commanded <= 0 { + st, _ := mgr.State(cfg.ID) + t.Fatalf("min 0 loadpoint within its latest start got %.0f W (%q)", commanded, st.CommandedReason) + } +} + +// Two goal cars share one 11 kW EV allowance and no plan is in force. Each +// latest start counts every goal due by that car's deadline, so both goals +// are met when the allowance goes earliest deadline first, as +// freezeFuseBudget gives it, and also, for equal deadlines, when it is split +// evenly. SoC is read in whole percent, as cars report it. +func TestDeadlineReserveSharedAllowance(t *testing.T) { + t0 := time.Date(2026, 10, 27, 18, 0, 0, 0, time.UTC) + for _, c := range []struct { + name string + dueA, dueB time.Duration + edf bool + }{ + {"same deadline, earliest first", 10 * time.Hour, 10 * time.Hour, true}, + {"same deadline, even split", 10 * time.Hour, 10 * time.Hour, false}, + {"b one hour later, earliest first", 10 * time.Hour, 11 * time.Hour, true}, + {"b far later, earliest first", 4*time.Hour + 30*time.Minute, 20 * time.Hour, true}, + } { + t.Run(c.name, func(t *testing.T) { + a, b := deadlineGoalState(t0, "a"), deadlineGoalState(t0, "b") + a.TargetTime, b.TargetTime = t0.Add(c.dueA), t0.Add(c.dueB) + soc := map[string]float64{"a": .3, "b": .3} + commanded := map[string]float64{} + for now := t0; now.Before(b.TargetTime); now = now.Add(time.Minute) { + var states []loadpoint.State + for _, base := range []loadpoint.State{a, b} { // deadline order + if !now.Before(base.TargetTime) { + continue + } + cur := base + cur.CurrentSoC = math.Floor(soc[base.ID]*100+1e-9) / 100 + cur.CommandedKnown, cur.CommandedW = true, commanded[base.ID] + states = append(states, cur) + } + svc := &Service{LoadpointStates: func() []loadpoint.State { return states }} + d, _ := svc.SlotDirectiveAt(now) + var want []string + for _, st := range states { + if d.LoadpointFallback[st.ID] && d.LoadpointMaxPowerW[st.ID] > 0 && soc[st.ID] < st.TargetSoC { + want = append(want, st.ID) + } + } + clear(commanded) + room := 11000.0 + for _, id := range want { + w := min(d.LoadpointMaxPowerW[id], room) + if !c.edf { + w = min(d.LoadpointMaxPowerW[id], 11000/float64(len(want))) + } + room -= w + commanded[id] = w + soc[id] = min(.8, soc[id]+w/60*loadpoint.DefaultChargeEfficiency/60000) + } + } + for _, id := range []string{"a", "b"} { + if soc[id] < .8-1e-9 { + t.Errorf("%s missed its goal: SoC %.3f, want 0.80", id, soc[id]) + } + } + }) + } +} diff --git a/go/internal/mpc/demand.go b/go/internal/mpc/demand.go index 03cfca953..6e98d0b43 100644 --- a/go/internal/mpc/demand.go +++ b/go/internal/mpc/demand.go @@ -1,8 +1,10 @@ package mpc import ( + "cmp" "context" "log/slog" + "slices" "time" ) @@ -20,21 +22,41 @@ const ( // DemandCharge is one monthly peak-power tariff on the Energyplan wire. // PricePerKW uses the same minor currency units as slot prices per kWh. +// Each local day is one sample: a day the plan reaches is the group of its +// hours, the clock hour in progress and its completed peak among them, and +// every other completed day is a value in AlreadyKW. type DemandCharge struct { ID string PricePerKW float64 TopN int AlreadyKW []float64 Hours []DemandHour + // LegacyAlreadyKW, when set, is AlreadyKW for a worker without + // demand_elapsed_hours: every completed day, the planned ones too. + LegacyAlreadyKW []float64 } -// DemandHour is one local clock hour inside the planning horizon. +// DemandHour is one local clock hour that ends inside the planning horizon. +// ElapsedImportKWh is its import before the first slot's execution start. type DemandHour struct { StartMs int64 EndMs int64 ElapsedImportKWh float64 Weight float64 Group string + // Elapsed marks an hour only a worker that announces + // demand_elapsed_hours takes: the hour in progress when it began before + // the first slot, or a planned day's completed peak. + Elapsed bool +} + +// HourImport is the clock hour in progress as Core's live import counter +// measured it from the site meter: its import and the time the readings +// cover. +type HourImport struct { + Start time.Time + ImportedWh float64 + Covered time.Duration } type clockHour struct { @@ -102,7 +124,13 @@ func ellevioClockHours(loc *time.Location, from, to time.Time, nightWeight float return out } -func bindDemandCharges(slots []Slot, pricePerKW float64, topN int, vatPercent, nightWeight float64, loc *time.Location, now time.Time, importWh func([][2]int64) ([]float64, []int64)) []DemandCharge { +// bindDemandCharges expands the tariff to the month's clock hours. Hours +// that end inside the horizon and reach the plan go out, the hour in +// progress too when it began before the first slot. A local day the plan +// reaches sends its completed peak as an hour of its group; other completed +// days go in AlreadyKW. One history read serves both request shapes +// (legacyDemandCharges). live is Core's own count of the hour in progress. +func bindDemandCharges(slots []Slot, pricePerKW float64, topN int, vatPercent, nightWeight float64, loc *time.Location, now time.Time, importWh func([][2]int64) ([]float64, []int64), live HourImport) []DemandCharge { if pricePerKW <= 0 || !finite(pricePerKW) || len(slots) == 0 { return nil } @@ -131,10 +159,18 @@ func bindDemandCharges(slots []Slot, pricePerKW float64, topN int, vatPercent, n return nil } + type dayPeak struct { + kw, wh float64 + hour clockHour + } type monthAcc struct { id string already []float64 alreadyDay map[string]int // local day → index in already + legacy []float64 // already, with the planned days' peaks + legacyDay map[string]int + peaks map[string]dayPeak // a planned day's completed peak + planned map[string]bool // local days with planned hours hours []DemandHour elapsed [][2]int64 elapsedI []int @@ -158,7 +194,7 @@ func bindDemandCharges(slots []Slot, pricePerKW float64, topN int, vatPercent, n var lookHour []clockHour for _, hour := range hours { startMs, endMs := hour.start.UnixMilli(), hour.end.UnixMilli() - inside := startMs >= horizonStart && endMs <= horizonEnd + inside := endMs <= horizonEnd overlapsPlan := endMs > execStart && startMs < horizonEnd if inside && overlapsPlan { m := acc(hour) @@ -166,11 +202,18 @@ func bindDemandCharges(slots []Slot, pricePerKW float64, topN int, vatPercent, n continue } idx := len(m.hours) + day := hour.start.Format("2006-01-02") m.hours = append(m.hours, DemandHour{ StartMs: startMs, EndMs: endMs, Weight: hour.weight, - Group: hour.start.Format("2006-01-02"), + Group: day, + // The hour in progress, begun before the first slot. + Elapsed: startMs < horizonStart, }) + if m.planned == nil { + m.planned = make(map[string]bool) + } + m.planned[day] = true if execStart > startMs && execStart < endMs { m.elapsed = append(m.elapsed, [2]int64{startMs, execStart}) m.elapsedI = append(m.elapsedI, idx) @@ -185,16 +228,23 @@ func bindDemandCharges(slots []Slot, pricePerKW float64, topN int, vatPercent, n } } + for i := range order { + m := months[order[i]] + for _, window := range m.elapsed { + look = append(look, window) + lookKind = append(lookKind, "elapsed") + lookMonth = append(lookMonth, order[i]) + } + } + // The history of each window, or none: wh and covered stay zero. + wh, covered := make([]float64, len(look)), make([]int64, len(look)) if importWh != nil { - for i := range order { - m := months[order[i]] - for _, window := range m.elapsed { - look = append(look, window) - lookKind = append(lookKind, "elapsed") - lookMonth = append(lookMonth, order[i]) - } + if gotWh, gotCovered := importWh(look); len(gotWh) == len(look) && len(gotCovered) == len(look) { + wh, covered = gotWh, gotCovered } - wh, covered := importWh(look) + } + slot0W := max(0, slots[0].LoadW+slots[0].PVW) + if importWh != nil { if len(wh) == len(look) && len(covered) == len(look) { elapsedAt := make(map[string]int) for i, kind := range lookKind { @@ -220,6 +270,25 @@ func bindDemandCharges(slots []Slot, pricePerKW float64, topN int, vatPercent, n kw *= hour.weight } day := hour.start.Format("2006-01-02") + if j, ok := m.legacyDay[day]; ok { + m.legacy[j] = max(m.legacy[j], kw) + } else if len(m.legacy) < maxDemandAlreadyKW { + if m.legacyDay == nil { + m.legacyDay = make(map[string]int) + } + m.legacyDay[day] = len(m.legacy) + m.legacy = append(m.legacy, kw) + } + if m.planned[day] { + // The day is one sample: its peak joins its group. + if peak, ok := m.peaks[day]; !ok || kw > peak.kw { + if m.peaks == nil { + m.peaks = make(map[string]dayPeak) + } + m.peaks[day] = dayPeak{kw: kw, wh: wh[i], hour: hour} + } + continue + } if j, ok := m.alreadyDay[day]; ok { m.already[j] = max(m.already[j], kw) continue @@ -239,11 +308,50 @@ func bindDemandCharges(slots []Slot, pricePerKW float64, topN int, vatPercent, n if m == nil || n >= len(m.elapsedI) { continue } - m.hours[m.elapsedI[n]].ElapsedImportKWh = wh[i] / 1000 + if hour := &m.hours[m.elapsedI[n]]; !hour.Elapsed { + hour.ElapsedImportKWh = wh[i] / 1000 + } elapsedAt[key] = n + 1 } } } + // The hour in progress that began before the first slot, with or + // without history. + for i, kind := range lookKind { + if kind != "elapsed" { + continue + } + m := months[lookMonth[i]] + for _, idx := range m.elapsedI { + hour := &m.hours[idx] + if hour.Elapsed && hour.StartMs == look[i][0] { + hour.ElapsedImportKWh = runningHourImportWh(look[i], wh[i], covered[i], live, slot0W) / 1000 + } + } + } + for _, key := range order { + m := months[key] + if len(m.peaks) == 0 { + continue + } + days := make([]string, 0, len(m.peaks)) + for day := range m.peaks { + days = append(days, day) + } + slices.Sort(days) + for _, day := range days { + peak := m.peaks[day] + if len(m.hours) >= maxDemandHours { + m.already = append(m.already, peak.kw) + continue + } + m.hours = append(m.hours, DemandHour{ + StartMs: peak.hour.start.UnixMilli(), EndMs: peak.hour.end.UnixMilli(), + ElapsedImportKWh: peak.wh / 1000, Weight: peak.hour.weight, Group: day, Elapsed: true, + }) + } + slices.SortStableFunc(m.hours, func(a, b DemandHour) int { return cmp.Compare(a.StartMs, b.StartMs) }) + } wirePrice := pricePerKW * (1 + vatPercent/100) if !finite(wirePrice) || wirePrice < 0 { @@ -255,10 +363,14 @@ func bindDemandCharges(slots []Slot, pricePerKW float64, topN int, vatPercent, n if len(m.hours) == 0 && len(m.already) == 0 { continue } - out = append(out, DemandCharge{ + charge := DemandCharge{ ID: m.id, PricePerKW: wirePrice, TopN: topN, AlreadyKW: m.already, Hours: m.hours, - }) + } + if len(m.peaks) > 0 { + charge.LegacyAlreadyKW = m.legacy + } + out = append(out, charge) } if len(out) > 16 { out = out[:16] @@ -266,6 +378,69 @@ func bindDemandCharges(slots []Slot, pricePerKW float64, topN int, vatPercent, n return out } +// runningHourImportWh is the import of the clock hour in progress before +// the first slot's execution start, window. Core's live counter answers when +// it measures that hour, else the import history. Time neither covers counts +// at the larger of the covered mean and slot 0's forecast import, never at +// zero: an undercount would let the plan set a new peak in the hour. +func runningHourImportWh(window [2]int64, historyWh float64, historyCoveredMs int64, live HourImport, slot0W float64) float64 { + spanMs := window[1] - window[0] + wh, coveredMs := historyWh, historyCoveredMs + if !live.Start.IsZero() && live.Start.UnixMilli() == window[0] && live.Covered > 0 { + wh, coveredMs = live.ImportedWh, live.Covered.Milliseconds() + } + if !finite(wh) || wh < 0 { + wh, coveredMs = 0, 0 + } + coveredMs = min(max(coveredMs, 0), spanMs) + rateW := max(0, slot0W) + if coveredMs > 0 { + rateW = max(rateW, wh*3_600_000/float64(coveredMs)) + } + return wh + rateW*float64(spanMs-coveredMs)/3_600_000 +} + +// legacyDemandCharges is the shape a worker without demand_elapsed_hours +// takes: no hour that began before the first slot, each completed day in +// AlreadyKW. A charge left with neither goes, and a month left with +// completed days only goes after the months with planned hours, as before. +func legacyDemandCharges(charges []DemandCharge) []DemandCharge { + var out []DemandCharge + for _, c := range charges { + legacy := DemandCharge{ID: c.ID, PricePerKW: c.PricePerKW, TopN: c.TopN, AlreadyKW: c.AlreadyKW} + if c.LegacyAlreadyKW != nil { + legacy.AlreadyKW = c.LegacyAlreadyKW + } + for _, h := range c.Hours { + if !h.Elapsed { + legacy.Hours = append(legacy.Hours, h) + } + } + if len(legacy.Hours) > 0 || len(legacy.AlreadyKW) > 0 { + out = append(out, legacy) + } + } + historyOnly := func(c DemandCharge) int { + if len(c.Hours) == 0 { + return 1 + } + return 0 + } + slices.SortStableFunc(out, func(a, b DemandCharge) int { return cmp.Compare(historyOnly(a), historyOnly(b)) }) + return out +} + +// hasElapsedDemand reports whether a charge holds an hour only a worker +// that announces demand_elapsed_hours takes. +func hasElapsedDemand(charges []DemandCharge) bool { + for _, c := range charges { + if c.LegacyAlreadyKW != nil || slices.ContainsFunc(c.Hours, func(h DemandHour) bool { return h.Elapsed }) { + return true + } + } + return false +} + func slotEndMs(slot Slot) (int64, bool) { durationMs := int64(slot.LenMin) * 60_000 end := slot.StartMs + durationMs @@ -280,7 +455,11 @@ func (s *Service) demandChargesFor(site SiteEconomics, slots []Slot, now time.Ti return nil } loc := siteLocation(site.Timezone) - return bindDemandCharges(slots, site.DemandPricePerKW, site.DemandTopN, site.VATPercent, site.DemandNightWeight, loc, now, s.importEnergy) + var live HourImport + if s.HourImport != nil { + live = s.HourImport() + } + return bindDemandCharges(slots, site.DemandPricePerKW, site.DemandTopN, site.VATPercent, site.DemandNightWeight, loc, now, s.importEnergy, live) } func (s *Service) importEnergy(intervals [][2]int64) ([]float64, []int64) { diff --git a/go/internal/mpc/demand_test.go b/go/internal/mpc/demand_test.go index ddf9d4e69..3831191bb 100644 --- a/go/internal/mpc/demand_test.go +++ b/go/internal/mpc/demand_test.go @@ -3,6 +3,9 @@ package mpc import ( "context" "encoding/json" + "math" + "os" + "reflect" "strings" "testing" "time" @@ -52,7 +55,16 @@ func TestBindDemandChargesExpandsWeekdayHoursAndElapsed(t *testing.T) { } return wh, cov } - got := bindDemandCharges(slots, 7000, 3, 25, 0, loc, now, importWh) + all := bindDemandCharges(slots, 7000, 3, 25, 0, loc, now, importWh, HourImport{}) + // Today's completed peak, 06:00 (the first of equal hours), joins + // today's group for a worker that announces demand_elapsed_hours. + if len(all) != 1 || len(all[0].Hours) < 2 || !all[0].Hours[0].Elapsed || + all[0].Hours[0].StartMs != start.Add(-4*time.Hour).UnixMilli() || all[0].Hours[0].ElapsedImportKWh != 3 || + all[0].Hours[1].Elapsed { + t.Fatalf("charges=%+v", all) + } + // The rest of this test is the shape every worker takes. + got := legacyDemandCharges(all) if len(got) != 1 || got[0].ID != "weekday-high-2026-09" || got[0].TopN != 3 { t.Fatalf("charges=%+v", got) } @@ -95,7 +107,7 @@ func TestBindDemandChargesGroupsHoursByLocalDay(t *testing.T) { {StartMs: now.Add(time.Hour).UnixMilli(), LenMin: 60, PriceOre: 100}, {StartMs: time.Date(2026, 9, 10, 6, 0, 0, 0, loc).UnixMilli(), LenMin: 60, PriceOre: 100}, } - got := bindDemandCharges(slots, 7000, 3, 0, 0, loc, now, nil) + got := bindDemandCharges(slots, 7000, 3, 0, 0, loc, now, nil, HourImport{}) if len(got) != 1 || len(got[0].Hours) < 3 { t.Fatalf("charges=%+v", got) } @@ -122,11 +134,11 @@ func TestEllevioNightWeightIncludesWeekendsAndHalvesNightHours(t *testing.T) { {StartMs: time.Date(2026, 9, 5, 22, 0, 0, 0, loc).UnixMilli(), LenMin: 60, PriceOre: 80}, {StartMs: time.Date(2026, 9, 5, 23, 0, 0, 0, loc).UnixMilli(), LenMin: 60, PriceOre: 80}, } - without := bindDemandCharges(slots, 7000, 3, 0, 0, loc, now, nil) + without := bindDemandCharges(slots, 7000, 3, 0, 0, loc, now, nil, HourImport{}) if len(without) != 0 { t.Fatalf("weekday-only window should skip Saturday, got %+v", without) } - got := bindDemandCharges(slots, 7000, 3, 0, 0.5, loc, now, nil) + got := bindDemandCharges(slots, 7000, 3, 0, 0.5, loc, now, nil, HourImport{}) if len(got) != 1 || len(got[0].Hours) < 3 { t.Fatalf("ellevio hours=%+v", got) } @@ -187,11 +199,16 @@ func TestEllevioAlreadyKWWeighsNightsAndKeepsOnePeakPerDay(t *testing.T) { } return wh, cov } - got := bindDemandCharges(slots, 7000, 3, 0, 0.5, loc, now, importWh) + got := bindDemandCharges(slots, 7000, 3, 0, 0.5, loc, now, importWh, HourImport{}) if len(got) != 1 { t.Fatalf("charges=%+v", got) } - already := got[0].AlreadyKW + // Today is planned: its completed peak is an hour of its group, and + // already_kw holds 1-8 September. The earlier shape holds all nine. + if len(got[0].AlreadyKW) != 8 || !got[0].Hours[0].Elapsed { + t.Fatalf("already_kw=%v hours=%+v", got[0].AlreadyKW, got[0].Hours) + } + already := legacyDemandCharges(got)[0].AlreadyKW if len(already) != 9 { t.Fatalf("already_kw=%v, want one peak for each of 1–9 September", already) } @@ -206,21 +223,259 @@ func TestEllevioAlreadyKWWeighsNightsAndKeepsOnePeakPerDay(t *testing.T) { } } -func TestBindDemandChargesSkipsUnalignedCurrentHour(t *testing.T) { +// At 10:17 with slot 0 at 10:15 the clock hour 10-11 is in progress: it +// goes out with its import since 10:00, for a worker that announces +// demand_elapsed_hours only. +func TestBindDemandChargesSendsTheRunningHourBeforeTheFirstSlot(t *testing.T) { loc := time.UTC now := time.Date(2026, 9, 9, 10, 17, 0, 0, loc) start := time.Date(2026, 9, 9, 10, 15, 0, 0, loc) + hour := time.Date(2026, 9, 9, 10, 0, 0, 0, loc) slots := []Slot{ - {StartMs: start.UnixMilli(), ExecutionStartMs: now.UnixMilli(), LenMin: 15}, + {StartMs: start.UnixMilli(), ExecutionStartMs: now.UnixMilli(), LenMin: 15, LoadW: 1200}, {StartMs: start.Add(15 * time.Minute).UnixMilli(), LenMin: 45}, {StartMs: start.Add(time.Hour).UnixMilli(), LenMin: 60}, } - got := bindDemandCharges(slots, 7000, 1, 0, 0, loc, now, nil) - if len(got) != 1 || len(got[0].Hours) == 0 { + running := [2]int64{hour.UnixMilli(), now.UnixMilli()} + importWh := func(intervals [][2]int64) ([]float64, []int64) { + wh, cov := make([]float64, len(intervals)), make([]int64, len(intervals)) + for i, window := range intervals { + if window == running { + wh[i], cov[i] = 850, window[1]-window[0] + } + } + return wh, cov + } + got := bindDemandCharges(slots, 7000, 1, 0, 0, loc, now, importWh, HourImport{}) + if len(got) != 1 || len(got[0].Hours) != 2 { + t.Fatalf("charges=%+v", got) + } + first := got[0].Hours[0] + if first.StartMs != hour.UnixMilli() || first.EndMs != hour.Add(time.Hour).UnixMilli() || !first.Elapsed || + first.ElapsedImportKWh != 0.85 || first.Group != "2026-09-09" { + t.Fatalf("running hour=%+v; want 10:00-11:00 with 0.85 kWh, for demand_elapsed_hours", first) + } + if got[0].Hours[1].Elapsed { + t.Fatalf("11:00 is a planned hour for every worker: %+v", got[0].Hours[1]) + } + legacy := legacyDemandCharges(got) + if len(legacy) != 1 || len(legacy[0].Hours) != 1 || legacy[0].Hours[0].StartMs != start.Add(45*time.Minute).UnixMilli() { + t.Fatalf("a worker without the feature gets the hours from 11:00, got %+v", legacy) + } +} + +// The hour in progress counts Core's live import counter when it measures +// that hour, else the history. Time neither covers counts at the larger of +// the covered mean and slot 0's forecast import, never at zero. +func TestRunningHourImportCountsUncoveredTime(t *testing.T) { + hour := time.Date(2026, 9, 9, 10, 0, 0, 0, time.UTC) + window := [2]int64{hour.UnixMilli(), hour.Add(40 * time.Minute).UnixMilli()} + twenty := 20 * time.Minute + for _, c := range []struct { + name string + historyWh float64 + historyCovered time.Duration + live HourImport + slot0W, wantWh float64 + }{ + {"live counter, fully covered", 100, twenty, HourImport{Start: hour, ImportedWh: 2000, Covered: 40 * time.Minute}, 500, 2000}, + {"live counter, half covered at 3 kW", 100, twenty, HourImport{Start: hour, ImportedWh: 1000, Covered: twenty}, 500, 2000}, + {"live counter of another hour", 600, twenty, HourImport{Start: hour.Add(-time.Hour), ImportedWh: 9000, Covered: time.Hour}, 500, 1200}, + {"history half covered, forecast above its mean", 600, twenty, HourImport{}, 2400, 1400}, + {"no record at all", 0, 0, HourImport{}, 1500, 1000}, + {"no record and no house import", 0, 0, HourImport{}, 0, 0}, + } { + got := runningHourImportWh(window, c.historyWh, c.historyCovered.Milliseconds(), c.live, c.slot0W) + if math.Abs(got-c.wantWh) > 1e-9 { + t.Errorf("%s: %g Wh, want %g", c.name, got, c.wantWh) + } + } +} + +func weekdayEvening(t *testing.T, history bool) ([]Slot, func([][2]int64) ([]float64, []int64), time.Time) { + t.Helper() + loc := time.UTC + now := time.Date(2026, 9, 9, 19, 20, 0, 0, loc) // Wednesday + start := time.Date(2026, 9, 9, 19, 15, 0, 0, loc) + slots := []Slot{ + {StartMs: start.UnixMilli(), ExecutionStartMs: now.UnixMilli(), LenMin: 15, LoadW: 900, PriceOre: 100}, + {StartMs: start.Add(15 * time.Minute).UnixMilli(), LenMin: 30, LoadW: 900, PriceOre: 100}, + } + if !history { + return slots, nil, now + } + // Every completed hour imported 1 kWh plus 0.1 kWh per hour of the day, + // so other days peak at 19:00 (2.9 kWh) and today at 18:00 (2.8 kWh); + // 19:00-19:20 imported 0.7 kWh. + importWh := func(intervals [][2]int64) ([]float64, []int64) { + wh, cov := make([]float64, len(intervals)), make([]int64, len(intervals)) + for i, window := range intervals { + cov[i] = window[1] - window[0] + if cov[i] == time.Hour.Milliseconds() { + wh[i] = 1000 + 100*float64(time.UnixMilli(window[0]).UTC().Hour()) + } else { + wh[i] = 700 + } + } + return wh, cov + } + return slots, importWh, now +} + +// At 19:20 on a weekday the hour in progress is the day's last tariff hour. +// Today's group holds it and today's completed peak, so today is one +// sample and already_kw keeps the other days only. A worker without +// demand_elapsed_hours gets exactly the request it got before. +func TestBindDemandChargesPutsTodaysPeakInTodaysGroup(t *testing.T) { + slots, importWh, now := weekdayEvening(t, true) + got := bindDemandCharges(slots, 7000, 3, 0, 0, time.UTC, now, importWh, HourImport{}) + if len(got) != 1 { t.Fatalf("charges=%+v", got) } - if got[0].Hours[0].StartMs != start.Add(45*time.Minute).UnixMilli() { - t.Fatalf("current 10:00 hour is not inside the slot timeline, first hour=%+v", got[0].Hours[0]) + peak := time.Date(2026, 9, 9, 18, 0, 0, 0, time.UTC) + want := []DemandHour{ + {StartMs: peak.UnixMilli(), EndMs: peak.Add(time.Hour).UnixMilli(), ElapsedImportKWh: 2.8, Group: "2026-09-09", Elapsed: true}, + {StartMs: peak.Add(time.Hour).UnixMilli(), EndMs: peak.Add(2 * time.Hour).UnixMilli(), ElapsedImportKWh: 0.7, Group: "2026-09-09", Elapsed: true}, + } + if !reflect.DeepEqual(got[0].Hours, want) { + t.Fatalf("hours=%+v\nwant %+v", got[0].Hours, want) + } + // 1-8 September hold six weekdays at 2.9 kW; today is in its group instead. + if len(got[0].AlreadyKW) != 6 || len(got[0].LegacyAlreadyKW) != 7 { + t.Fatalf("already_kw=%v legacy=%v", got[0].AlreadyKW, got[0].LegacyAlreadyKW) + } + legacy, err := json.Marshal(externalDemandCharges(legacyDemandCharges(got))) + if err != nil { + t.Fatal(err) + } + if want := `[{"id":"weekday-high-2026-09","price_per_kw":7000,"top_n":3,"already_kw":[2.9,2.9,2.9,2.9,2.9,2.9,2.8]}]`; string(legacy) != want { + t.Fatalf("legacy demand_charges %s\nwant %s", legacy, want) + } +} + +// At 23:20 on the last day of a month under a tariff with night hours, the +// hour in progress is the month's last, so the new shape starts with that +// month. A worker without demand_elapsed_hours still gets the charges in +// their earlier order: months with planned hours first, then a month with +// completed days only. The expected JSON is what Core sent before. +func TestLegacyDemandChargesKeepTheirOrderAtTheEndOfAMonth(t *testing.T) { + now := time.Date(2026, 10, 31, 23, 20, 0, 0, time.UTC) + start := time.Date(2026, 10, 31, 23, 15, 0, 0, time.UTC) + slots := []Slot{{StartMs: start.UnixMilli(), ExecutionStartMs: now.UnixMilli(), LenMin: 45, LoadW: 900, PriceOre: 100}} + for h := range 3 { + slots = append(slots, Slot{StartMs: start.Add(45*time.Minute + time.Duration(h)*time.Hour).UnixMilli(), LenMin: 60, LoadW: 900, PriceOre: 100}) + } + importWh := func(intervals [][2]int64) ([]float64, []int64) { + wh, cov := make([]float64, len(intervals)), make([]int64, len(intervals)) + for i, window := range intervals { + cov[i] = window[1] - window[0] + if cov[i] == time.Hour.Milliseconds() { + wh[i] = 1000 + 100*float64(time.UnixMilli(window[0]).UTC().Hour()) + } else { + wh[i] = 700 + } + } + return wh, cov + } + got := bindDemandCharges(slots, 5200, 3, 25, 0.5, time.UTC, now, importWh, HourImport{}) + if len(got) != 2 || got[0].ID != "weekday-high-2026-10" || len(got[0].Hours) != 2 || len(got[0].AlreadyKW) != 30 { + t.Fatalf("charges=%+v; want October first, with today's peak and the hour in progress", got) + } + legacy, err := json.Marshal(externalDemandCharges(legacyDemandCharges(got))) + if err != nil { + t.Fatal(err) + } + november := `{"id":"weekday-high-2026-11","price_per_kw":6500,"top_n":3,"hours":[` + + `{"start_ms":1793491200000,"end_ms":1793494800000,"weight":0.5,"group":"2026-11-01"},` + + `{"start_ms":1793494800000,"end_ms":1793498400000,"weight":0.5,"group":"2026-11-01"},` + + `{"start_ms":1793498400000,"end_ms":1793502000000,"weight":0.5,"group":"2026-11-01"}]}` + october := `{"id":"weekday-high-2026-10","price_per_kw":6500,"top_n":3,"already_kw":[` + + strings.TrimSuffix(strings.Repeat("3.1,", 31), ",") + `]}` + if want := "[" + november + "," + october + "]"; string(legacy) != want { + t.Fatalf("legacy demand_charges %s\nwant %s", legacy, want) + } +} + +type demandShapeTransport struct{ elapsed bool } + +func (d *demandShapeTransport) RoundTrip(context.Context, []byte) ([]byte, error) { + features := []string{"champion", "ev_duty", "published_prices", "demand_charges"} + if d.elapsed { + features = append(features, "demand_elapsed_hours") + } + return json.Marshal(map[string]any{"name": "ftw-solver", "version": "test", "protocol_version": 1, "features": features}) +} +func (*demandShapeTransport) Close() error { return nil } + +// Core sends the hour in progress and today's completed peak only to a +// worker that announces demand_elapsed_hours. Any other worker gets the +// earlier request, byte for byte, with the budget of the charges it gets. +func TestEnergyplanRequestNegotiatesElapsedDemandHours(t *testing.T) { + transport := &demandShapeTransport{} + external := &ExternalOptimizer{cfg: ExternalOptimizerConfig{Timeout: 7 * time.Second}, transport: transport, timeBudget: energyplanTimeBudget} + o := &EnergyplanOptimizer{ExternalOptimizer: external} + _, fixture := externalTestFixture() + request := func(slots []Slot, p Params) externalRequest { + t.Helper() + r := external.buildRequest(slots, p) + r.RequestID = "fixed" + if err := o.prepareEnergyplanRequest(context.Background(), &r, slots, p); err != nil { + t.Fatal(err) + } + return r + } + wire := func(r externalRequest) string { + raw, err := json.Marshal(r.DemandCharges) + if err != nil { + t.Fatal(err) + } + return string(raw) + } + slots, importWh, now := weekdayEvening(t, true) + p := fixture + p.DemandCharges = bindDemandCharges(slots, 7000, 3, 0, 0, time.UTC, now, importWh, HourImport{}) + transport.elapsed = false + if got, want := wire(request(slots, p)), `[{"id":"weekday-high-2026-09","price_per_kw":7000,"top_n":3,"already_kw":[2.9,2.9,2.9,2.9,2.9,2.9,2.8]}]`; got != want { + t.Fatalf("old worker demand_charges %s\nwant %s", got, want) + } + transport.elapsed = true + if got, want := wire(request(slots, p)), `[{"id":"weekday-high-2026-09","price_per_kw":7000,"top_n":3,"already_kw":[2.9,2.9,2.9,2.9,2.9,2.9],"hours":[{"start_ms":1788976800000,"end_ms":1788980400000,"elapsed_import_kwh":2.8,"group":"2026-09-09"},{"start_ms":1788980400000,"end_ms":1788984000000,"elapsed_import_kwh":0.7,"group":"2026-09-09"}]}]`; got != want { + t.Fatalf("new worker demand_charges %s\nwant %s", got, want) + } + // A new site has no history: its only demand hour is the one in + // progress, counted at slot 0's forecast import. The old worker gets no + // demand charge and the battery's own budget. + slots, _, now = weekdayEvening(t, false) + p.DemandCharges = bindDemandCharges(slots, 7000, 3, 0, 0, time.UTC, now, nil, HourImport{}) + if len(p.DemandCharges) != 1 || len(p.DemandCharges[0].Hours) != 1 || p.DemandCharges[0].Hours[0].ElapsedImportKWh != 0.3 { + t.Fatalf("charges=%+v; want the hour in progress at 900 W for 20 minutes", p.DemandCharges) + } + transport.elapsed = false + if r := request(slots, p); r.DemandCharges != nil || r.Settings.TimeLimitS != energyplanSmallBudget.Seconds() { + t.Fatalf("old worker demand_charges %s, time_limit_s %g; want none and %g", wire(r), r.Settings.TimeLimitS, energyplanSmallBudget.Seconds()) + } + transport.elapsed = true + if r := request(slots, p); len(r.DemandCharges) != 1 || r.Settings.TimeLimitS != energyplanFleetBudget.Seconds() { + t.Fatalf("new worker demand_charges %s, time_limit_s %g", wire(r), r.Settings.TimeLimitS) + } +} + +// The tariff exists for either worker, so the reserve's price-only pass +// keeps away from it even when the worker took no demand charge. +func TestReservePricingKeepsAwayFromADemandTariffWithOnlyTheRunningHour(t *testing.T) { + slots, p := externalTestFixture() + p.Loadpoint = &LoadpointSpec{ID: "car", PluggedIn: true, CapacityWh: 60000, Levels: 11, SoCMax: 1, MaxChargeW: 11000, + TargetSoC: 0.5, TargetSlotIdx: 1} + reserve := Plan{Actions: []Action{{SlotStartMs: slots[0].StartMs, SlotLenMin: 60, LoadpointW: 4000}, + {SlotStartMs: slots[1].StartMs, SlotLenMin: 60}}} + p.DemandCharges = []DemandCharge{{ID: "weekday-high-2026-09", PricePerKW: 7000, TopN: 3, + Hours: []DemandHour{{StartMs: slots[0].StartMs - 900_000, EndMs: slots[0].StartMs + 2_700_000, Elapsed: true}}}} + got, err := priceReserveCharging(context.Background(), slots, p, reserve) + if err != nil { + t.Fatal(err) + } + if !reflect.DeepEqual(got, reserve) { + t.Fatalf("reserve changed: %+v", got) } } @@ -288,6 +543,58 @@ func TestNativeDemandChargesOnWeekdayHour(t *testing.T) { } } +// A worker that announces demand_elapsed_hours takes the hour in progress +// that began before slot 0 and today's completed peak in today's group, and +// Core accepts its plan. The bundled worker must announce the feature. +func TestNativeEnergyplanTakesElapsedDemandHours(t *testing.T) { + binary := os.Getenv("FTW_NATIVE_SOLVER") + if binary == "" { + t.Skip("run make native-solver-test to include the Rust worker") + } + o, err := NewEnergyplanOptimizer(binary) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = o.Close() }) + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + info, err := o.Health(ctx) + if err != nil { + t.Fatal(err) + } + if !optimizerHasFeature(info, "demand_elapsed_hours") { + t.Fatal("the bundled worker does not announce demand_elapsed_hours") + } + slots, importWh, now := weekdayEvening(t, true) + evening := time.Date(2026, 9, 9, 20, 0, 0, 0, time.UTC) + for h := range 3 { + slots = append(slots, Slot{StartMs: evening.Add(time.Duration(h) * time.Hour).UnixMilli(), LenMin: 60, LoadW: 900, PriceOre: 60}) + } + for i := range slots { + slots[i].SpotOre = 30 + slots[i].Limits = PowerLimits{MaxImportW: 11000, MaxExportW: 11000} + } + _, p := externalTestFixture() + p.DemandCharges = bindDemandCharges(slots, 7000, 3, 0, 0, time.UTC, now, importWh, HourImport{}) + plan, err := o.Optimize(ctx, slots, p) + if err != nil { + t.Fatal(err) + } + if err := ValidatePlan(slots, p, &plan); err != nil { + t.Fatal(err) + } + var req externalRequest + if err := json.Unmarshal(plan.OptimizerInput, &req); err != nil { + t.Fatal(err) + } + if len(req.DemandCharges) != 1 || len(req.DemandCharges[0].Hours) != 2 || req.DemandCharges[0].Hours[1].StartMs >= slots[0].StartMs { + t.Fatalf("demand_charges=%+v; want today's peak and the hour in progress", req.DemandCharges) + } + if plan.Solver == nil || plan.Solver.Backend != "fleet_milp_rust" { + t.Fatalf("solver %+v", plan.Solver) + } +} + func TestEnergyplanTimeBudgetUsesFleetWhenDemandCharges(t *testing.T) { slots := []Slot{{StartMs: time.Now().UnixMilli(), LenMin: 15}} p := Params{CapacityWh: 10000, DemandCharges: []DemandCharge{{ID: "weekday-high-2026-09"}}} diff --git a/go/internal/mpc/energyplan.go b/go/internal/mpc/energyplan.go index cefb6bf24..ee19f65f6 100644 --- a/go/internal/mpc/energyplan.go +++ b/go/internal/mpc/energyplan.go @@ -35,9 +35,18 @@ const ( energyplanFleetBudget = 5 * time.Second // Leave time for ValidatePlan and publication before firstSlotExpired. energyplanPublishMargin = 50 * time.Millisecond + // A worker that spends its whole budget must still answer before the + // call's deadline: 5 % of the time to it, at least this. + energyplanReplyMargin = 30 * time.Millisecond ) func energyplanTimeBudget(slots []Slot, p Params) time.Duration { + return energyplanBudget(slots, p, false) +} + +// energyplanBudget is the solve budget; hours says the request carries +// hourly import caps. +func energyplanBudget(slots []Slot, p Params, hours bool) time.Duration { batteries := len(p.Storages) if batteries == 0 && p.CapacityWh > 0 { batteries = 1 @@ -50,19 +59,22 @@ func energyplanTimeBudget(slots []Slot, p Params) time.Duration { budget = energyplanEVBudget } // 1 battery + 1 EV is 5 "assets". A 193-slot villa with demand charges - // routes to fleet MILP; 193*6 would leave it on the 500 ms budget. - if len(slots)*assets >= 193*5 || p.PVCurtailment.MinW > 0 || len(p.DemandCharges) > 0 { + // routes to fleet MILP; 193*6 would leave it on the 500 ms budget. The + // hourly import caps route there too. + if len(slots)*assets >= 193*5 || p.PVCurtailment.MinW > 0 || len(p.DemandCharges) > 0 || hours { budget = energyplanFleetBudget } - remaining := remainingFirstSlot(slots) - if remaining < energyplanSmallBudget { + return min(budget, energyplanSolveTime(remainingFirstSlot(slots)-energyplanPublishMargin)) +} + +// energyplanSolveTime is the longest limit a worker can use and still answer +// before a call deadline toDeadline away. It is zero below Core's no-call +// threshold: less than energyplanSmallBudget left in the first slot. +func energyplanSolveTime(toDeadline time.Duration) time.Duration { + if toDeadline < energyplanSmallBudget-energyplanPublishMargin { return 0 } - available := remaining - energyplanPublishMargin - if available < budget { - return available - } - return budget + return toDeadline - max(toDeadline/20, energyplanReplyMargin) } // Optimize solves the slots Service selects, including point and caution diff --git a/go/internal/mpc/energyplan_fault_test.go b/go/internal/mpc/energyplan_fault_test.go index de432e2bd..7b341ec41 100644 --- a/go/internal/mpc/energyplan_fault_test.go +++ b/go/internal/mpc/energyplan_fault_test.go @@ -57,7 +57,7 @@ func TestNativeEnergyplanEVRecoveryFallbackFaults(t *testing.T) { t.Cleanup(func() { _ = external.Close() }) wrapper := &EnergyplanOptimizer{ExternalOptimizer: external} health, err := wrapper.Health(context.Background()) - if err != nil || health.Version != "0.5.2" { + if err != nil || health.Version != "0.7.0" { t.Fatalf("bundle health=%+v err=%v", health, err) } svc := shadowTestService(t) diff --git a/go/internal/mpc/energyplan_import_caps_test.go b/go/internal/mpc/energyplan_import_caps_test.go new file mode 100644 index 000000000..816f68749 --- /dev/null +++ b/go/internal/mpc/energyplan_import_caps_test.go @@ -0,0 +1,37 @@ +package mpc + +import ( + "context" + "os" + "testing" + "time" +) + +// A worker that takes hourly_import_caps plans the 25 October field case +// itself: Core sends the hours, and the worker's plan keeps every hour and +// shows the goal short. The bundled worker skips until its pin takes them. +func TestNativeEnergyplanTakesHourlyImportCaps(t *testing.T) { + binary := os.Getenv("FTW_NATIVE_SOLVER") + if binary == "" { + t.Skip("run make native-solver-test to include the Rust worker") + } + o, err := NewEnergyplanOptimizer(binary) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = o.Close() }) + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + info, err := o.Health(ctx) + if err != nil { + t.Fatal(err) + } + if !optimizerHasFeature(info, "hourly_import_caps") { + t.Skip("this worker does not take hourly_import_caps") + } + svc, start := importCeilingFieldService(t, o, true) + plan := checkImportCeilingFieldPlan(t, svc, start) + if plan.Solver == nil || plan.Solver.Fallback { + t.Fatalf("solver %+v; want the worker's own plan", plan.Solver) + } +} diff --git a/go/internal/mpc/energyplan_test.go b/go/internal/mpc/energyplan_test.go index b0df3e747..df5bfffa4 100644 --- a/go/internal/mpc/energyplan_test.go +++ b/go/internal/mpc/energyplan_test.go @@ -95,7 +95,7 @@ func TestNativeEnergyplanDownsideAndAsyncShadow(t *testing.T) { svc := shadowTestService(t) svc.Optimizer = o info, err := svc.Optimizer.(*EnergyplanOptimizer).Health(context.Background()) - if err != nil || info.Name != "ftw-solver" || info.Version != "0.5.2" { + if err != nil || info.Name != "ftw-solver" || info.Version != "0.7.0" { t.Fatalf("bundled worker health: %+v %v", info, err) } start := time.Now().UTC().Truncate(time.Hour) @@ -373,11 +373,16 @@ func TestEnergyplanTimeBudgetRespectsRemainingSlot(t *testing.T) { if got := energyplanTimeBudget(slots, p); got != 0 { t.Fatalf("200ms remaining fleet budget=%v, want fail-fast", got) } - trimFirstSlotRemaining(slots, 2*time.Second) - got := energyplanTimeBudget(slots, p) - remaining := remainingFirstSlot(slots) - if got <= 0 || got > remaining || remaining-got != energyplanPublishMargin { - t.Fatalf("2s remaining fleet budget=%v remaining=%v", got, remaining) + // Core's call ends energyplanPublishMargin before the slot does. A + // worker that spends its whole budget must still answer before that: + // 5 % of the time left, at least 30 ms. + for _, left := range []time.Duration{2 * time.Second, 500 * time.Millisecond, energyplanFleetBudget + 60*time.Millisecond} { + trimFirstSlotRemaining(slots, left) + got, remaining := energyplanTimeBudget(slots, p), remainingFirstSlot(slots) + deadline := remaining - energyplanPublishMargin + if want := deadline - max(deadline/20, 30*time.Millisecond); got <= 0 || got != want { + t.Fatalf("%v remaining: fleet budget=%v, want %v under the %v call deadline", left, got, want, deadline) + } } small, smallParams := topologyFixture(1, 0) trimFirstSlotRemaining(small, 2*time.Second) @@ -423,13 +428,155 @@ func TestEnergyplanCapsWorkerBudgetAndWaitToRemaining(t *testing.T) { if err := json.Unmarshal(capture.last, &request); err != nil { t.Fatal(err) } - want := energyplanTimeBudget(slots, p).Seconds() - if request.Settings.TimeLimitS != want || request.Settings.TimeLimitS > 2 { - t.Fatalf("TimeLimitS=%g, want %g (≤ remaining)", request.Settings.TimeLimitS, want) - } if !capture.hasDeadline || capture.deadlineWait > 2*time.Second || capture.deadlineWait < time.Second { t.Fatalf("process wait %v, want ≤ remaining 2s", capture.deadlineWait) } + // The limit also fits the time still left before the call's deadline + // when the request is sent, which is never less than the wait seen here. + want, least := energyplanTimeBudget(slots, p).Seconds(), energyplanSolveTime(capture.deadlineWait).Seconds() + if got := request.Settings.TimeLimitS; got > want || got < least { + t.Fatalf("TimeLimitS=%g, want in [%g, %g] (≤ remaining)", got, least, want) + } +} + +// The budget counts from the frozen execution start. After a point call that +// used its whole limit, the caution call has less time before its own +// deadline, and a worker that uses its full time limit must still answer. +func TestEnergyplanLateCautionCallLimitFitsItsDeadline(t *testing.T) { + capture := &countingTransport{} + engine := &EnergyplanOptimizer{ExternalOptimizer: &ExternalOptimizer{ + cfg: ExternalOptimizerConfig{Timeout: 7 * time.Second}, + transport: capture, + timeBudget: energyplanTimeBudget, + }} + // Core's plan stands in for the point reply, so use a site it replays. + slots, p := externalTestFixture() + p.SoCLevels, p.ActionLevels = 21, 11 + _, ev := topologyFixture(0, 1) + p.Loadpoint, p.Loadpoints = ev.Loadpoint, ev.Loadpoints + const frozen = 2500 * time.Millisecond + trimFirstSlotRemaining(slots, frozen) + budget := energyplanTimeBudget(slots, p) + if budget != energyplanEVBudget { + t.Fatalf("budget=%v, want the full EV budget %v", budget, energyplanEVBudget) + } + risk := append([]Slot(nil), slots...) + risk[0].LoadW += 100 + start := time.UnixMilli(slots[0].ExecutionStart()) + var elapsed time.Duration + limits := map[string]float64{} + _, err := planWithForecastMargin(context.Background(), slots, risk, p, + func() time.Time { return start.Add(elapsed) }, + func(ctx context.Context, in []Slot, p Params) (Plan, []Slot, error) { + trips := capture.n + _, err := engine.Optimize(ctx, in, p) + if capture.n != trips+1 { + t.Fatalf("worker trips %d, want %d: %v", capture.n, trips+1, err) + } + var request externalRequest + if jsonErr := json.Unmarshal(capture.last, &request); jsonErr != nil { + t.Fatal(jsonErr) + } + if in[0].LoadW == risk[0].LoadW { + limits["caution"] = request.Settings.TimeLimitS + return Plan{}, in, err + } + limits["point"] = request.Settings.TimeLimitS + // The point worker uses its full time limit; Core's own plan + // stands in for its reply. + elapsed = budget + return corePlanWithRecovery(ctx, in, p) + }) + if err != nil { + t.Fatal(err) + } + if limits["point"] != budget.Seconds() { + t.Fatalf("point time_limit_s=%g, want the unchanged budget %g", limits["point"], budget.Seconds()) + } + left := frozen - budget - energyplanPublishMargin + want := left - max(left/20, energyplanReplyMargin) + if got := limits["caution"]; got <= 0 || got > want.Seconds() { + t.Fatalf("caution time_limit_s=%g, want in (0, %g]: the call has %v before its deadline", got, want.Seconds(), left) + } +} + +func TestEnergyplanCallWithTimeLeftKeepsItsBudget(t *testing.T) { + for _, tc := range []struct { + name string + frozen, caller time.Duration + }{ + {"full slot without a call deadline", 0, 0}, + {"frozen start near the slot end", 2500 * time.Millisecond, 0}, + {"caller deadline past the budget", 0, 2 * time.Second}, + } { + t.Run(tc.name, func(t *testing.T) { + capture := &countingTransport{} + engine := &EnergyplanOptimizer{ExternalOptimizer: &ExternalOptimizer{ + cfg: ExternalOptimizerConfig{Timeout: 7 * time.Second}, + transport: capture, + timeBudget: energyplanTimeBudget, + }} + slots, p := topologyFixture(1, 1) + if tc.frozen > 0 { + trimFirstSlotRemaining(slots, tc.frozen) + } + ctx := context.Background() + if tc.caller > 0 { + var cancel context.CancelFunc + ctx, cancel = context.WithTimeout(ctx, tc.caller) + defer cancel() + } + _, _ = engine.Optimize(ctx, slots, p) + if capture.n != 1 { + t.Fatalf("worker trips=%d", capture.n) + } + var request externalRequest + if err := json.Unmarshal(capture.last, &request); err != nil { + t.Fatal(err) + } + if want := energyplanEVBudget.Seconds(); request.Settings.TimeLimitS != want { + t.Fatalf("time_limit_s=%g, want the unchanged budget %g", request.Settings.TimeLimitS, want) + } + }) + } +} + +// The frozen start may allow a call that its own deadline no longer does. +// Below Core's no-call threshold the worker is not started; above it, the +// limit is positive and fits the deadline. +func TestEnergyplanLateCallLimitFollowsNoCallThreshold(t *testing.T) { + for _, caller := range []time.Duration{300 * time.Millisecond, 600 * time.Millisecond} { + t.Run(caller.String(), func(t *testing.T) { + capture := &countingTransport{} + engine := &EnergyplanOptimizer{ExternalOptimizer: &ExternalOptimizer{ + cfg: ExternalOptimizerConfig{Timeout: 7 * time.Second}, + transport: capture, + timeBudget: energyplanTimeBudget, + }} + slots, p := topologyFixture(1, 1) + trimFirstSlotRemaining(slots, 2500*time.Millisecond) + ctx, cancel := context.WithTimeout(context.Background(), caller) + defer cancel() + _, err := engine.Optimize(ctx, slots, p) + if caller < energyplanSmallBudget-energyplanPublishMargin { + if capture.n != 0 || err == nil || !strings.Contains(err.Error(), "below the Energyplan budget") { + t.Fatalf("worker trips=%d err=%v, want a skipped call", capture.n, err) + } + return + } + if capture.n != 1 { + t.Fatalf("worker trips=%d: %v", capture.n, err) + } + var request externalRequest + if err := json.Unmarshal(capture.last, &request); err != nil { + t.Fatal(err) + } + want := caller - max(caller/20, energyplanReplyMargin) + if got := request.Settings.TimeLimitS; got <= 0 || got > want.Seconds() { + t.Fatalf("time_limit_s=%g, want in (0, %g]", got, want.Seconds()) + } + }) + } } func TestEnergyplanLateFleetReplanDoesNotStartDoomedSolve(t *testing.T) { diff --git a/go/internal/mpc/ev_reserve.go b/go/internal/mpc/ev_reserve.go index d6cf98bc3..e9c66cd0e 100644 --- a/go/internal/mpc/ev_reserve.go +++ b/go/internal/mpc/ev_reserve.go @@ -50,6 +50,7 @@ func coreReservePlan(ctx context.Context, slots []Slot, p Params) (Plan, error) target = min(lp.TargetSoC, ceiling) * lp.CapacityWh steps = lp.normalizedSteps() } + _, importCaps := importHourCapsW(slots, p) for i, slot := range slots { if err := ctx.Err(); err != nil { return Plan{}, err @@ -62,7 +63,12 @@ func coreReservePlan(ctx context.Context, slots []Slot, p Params) (Plan, error) minBatteryW = max(minBatteryW, reservePowerForEnergy(bands[i+1].min-energy, hours, chargeEff, dischargeEff)) maxBatteryW = min(maxBatteryW, reservePowerForEnergy(bands[i+1].max-energy, hours, chargeEff, dischargeEff)) house := slot.LoadW + slot.PVW - chooseBattery := func(peak float64) (float64, bool) { + capW := math.Inf(1) + if importCaps != nil { + capW = importCaps[i] + } + // most bounds the battery from above, so it may cover the house. + chooseBattery := func(peak, most float64) (float64, bool) { low, high := reserveBatteryPowerRange(slot, p, peak) low, high = max(low, minBatteryW), min(high, maxBatteryW) if peak > 0 && lp != nil && lp.blocksBatteryToEV() { @@ -72,13 +78,16 @@ func coreReservePlan(ctx context.Context, slots []Slot, p Params) (Plan, error) return 0, false } b := min(max(0, low), high) + if b > most { + b = max(low, most) + } if !modeAllows(p.Mode, house, house+b, b) || !modeAllows(p.Mode, house+peak, house+peak+b, b) { return 0, false } return b, true } - peak, battery := 0.0, 0.0 - battery, ok := chooseBattery(0) + peak, battery, average := 0.0, 0.0, 0.0 + battery, ok := chooseBattery(0, math.Inf(1)) if !ok { return Plan{}, fmt.Errorf("slot %d: battery cannot keep the forecast within the site power limits", i) } @@ -89,13 +98,28 @@ func coreReservePlan(ctx context.Context, slots []Slot, p Params) (Plan, error) if lp.SurplusOnly && surplusOnlyExceedsHousePV(step, slot.LoadW, slot.PVW) { continue } - if b, ok := chooseBattery(step); ok { - peak, battery = step, b - break + b, ok := chooseBattery(step, math.Inf(1)) + if !ok { + continue + } + want := min(step, need/(eff*hours)) + if evAverageRoomW(capW, house+b, step) < want { + // The hour's import holds the car back: the battery + // covers the house to make room, never into export. + if covered, ok := chooseBattery(step, max(-max(0, house), capW-house-want)); ok { + b = covered + } } + avg := min(want, evAverageRoomW(capW, house+b, step)) + if !math.IsInf(capW, 1) && (avg <= 1e-6 || b < -solverZeroPowerToleranceW && house+b+avg < -50) { + // The hour leaves this step no room, or only part of the + // slot while the battery the step needs drives export. + continue + } + peak, battery, average = step, b, avg + break } } - average := min(peak, need/(eff*hours)) evEnergy += average * eff * hours energy += loadpoint.BatteryEnergyDeltaWh(battery, hours, chargeEff, dischargeEff) a := Action{ExecutionStartMs: slot.ExecutionStartMs, SlotStartMs: slot.StartMs, SlotLenMin: slot.LenMin, PriceOre: slot.PriceOre, SpotOre: slot.SpotOre, PVW: slot.PVW, LoadW: slot.LoadW, BatteryW: battery, GridW: house + battery + average, SoC: energy / p.CapacityWh, Reason: "Site power reserve"} @@ -200,6 +224,20 @@ func reserveSlotCost(slot Slot, p Params, a Action) float64 { if len(a.LoadpointMaxPowerW) == 0 { return SlotGridCostOre(slot, a.GridW*slot.DurationHours()/1000, p) } + cost := 0.0 + pulseGridW(a, func(share, gridW float64) { + cost += share * SlotGridCostOre(slot, gridW*slot.DurationHours()/1000, p) + }) + return cost +} + +// pulseGridW calls f with each pulse state's share of the slot and its grid +// flow. Without a pulse the slot is one state at its mean flow. +func pulseGridW(a Action, f func(share, gridW float64)) { + if len(a.LoadpointMaxPowerW) == 0 { + f(1, a.GridW) + return + } cuts := []float64{0, 1} totalAverage := 0.0 for id, peak := range a.LoadpointMaxPowerW { @@ -210,7 +248,6 @@ func reserveSlotCost(slot Slot, p Params, a Action) float64 { } } slices.Sort(cuts) - cost := 0.0 for i := 1; i < len(cuts); i++ { midpoint := (cuts[i-1] + cuts[i]) / 2 grid := a.GridW - totalAverage @@ -219,9 +256,8 @@ func reserveSlotCost(slot Slot, p Params, a Action) float64 { grid += peak } } - cost += (cuts[i] - cuts[i-1]) * SlotGridCostOre(slot, grid*slot.DurationHours()/1000, p) + f(cuts[i]-cuts[i-1], grid) } - return cost } // Check the minimum and maximum simultaneous EV draw. Full-interval charges diff --git a/go/internal/mpc/external_optimizer.go b/go/internal/mpc/external_optimizer.go index 3b4a4b83b..80489e278 100644 --- a/go/internal/mpc/external_optimizer.go +++ b/go/internal/mpc/external_optimizer.go @@ -50,7 +50,7 @@ type ExternalOptimizer struct { cfg ExternalOptimizerConfig transport OptimizerTransport timeBudget func([]Slot, Params) time.Duration - prepareRequest func(context.Context, *externalRequest, Params) error + prepareRequest func(context.Context, *externalRequest, []Slot, Params) error } func NewExternalOptimizer(cfg ExternalOptimizerConfig) (*ExternalOptimizer, error) { @@ -75,6 +75,8 @@ type externalRequest struct { FlexLoads []externalFlexLoad `json:"flex_loads"` ThermalLoads []map[string]any `json:"thermal_loads"` DemandCharges []externalDemandCharge `json:"demand_charges,omitempty"` + // Sent only to a worker that announces hourly_import_caps. + HourlyImportCaps []externalImportCap `json:"hourly_import_caps,omitempty"` } type externalSettings struct { @@ -131,6 +133,16 @@ type externalDemandCharge struct { Hours []externalDemandHour `json:"hours,omitempty"` } +// externalImportCap is one clock hour of the owner's import ceiling: the +// plan's import in it may reach MaxMeanW over the hour less what the hour +// had used before the plan starts. +type externalImportCap struct { + StartMs int64 `json:"start_ms"` + EndMs int64 `json:"end_ms"` + MaxMeanW float64 `json:"max_mean_w"` + ElapsedImportWh float64 `json:"elapsed_import_wh"` +} + type externalDemandHour struct { StartMs int64 `json:"start_ms"` EndMs int64 `json:"end_ms"` @@ -213,10 +225,21 @@ func (o *ExternalOptimizer) Optimize(ctx context.Context, slots []Slot, p Params } request := o.buildRequest(slots, p) if o.prepareRequest != nil { - if err := o.prepareRequest(ctx, &request, p); err != nil { + if err := o.prepareRequest(ctx, &request, slots, p); err != nil { return Plan{}, fmt.Errorf("prepare optimizer request: %w", err) } } + if deadline, ok := ctx.Deadline(); ok && o.timeBudget != nil { + // The budget counts from the frozen execution start. A later call, + // such as caution after a point solve, has less time before its own + // deadline, and a worker that uses its whole limit must answer by it. + left := time.Until(deadline) + limit := energyplanSolveTime(left) + if limit <= 0 { + return Plan{}, fmt.Errorf("remaining call time %s is below the Energyplan budget", left) + } + request.Settings.TimeLimitS = math.Min(request.Settings.TimeLimitS, limit.Seconds()) + } payload, err = json.Marshal(request) if err != nil { return Plan{}, fmt.Errorf("encode optimizer request: %w", err) @@ -350,7 +373,13 @@ func (o *ExternalOptimizer) buildRequest(slots []Slot, p Params) externalRequest SurplusOnly: lp.SurplusOnly, NoStorageToLoad: lp.blocksBatteryToEV(), }) } - for _, charge := range p.DemandCharges { + req.DemandCharges = externalDemandCharges(p.DemandCharges) + return req +} + +func externalDemandCharges(charges []DemandCharge) []externalDemandCharge { + var out []externalDemandCharge + for _, charge := range charges { wire := externalDemandCharge{ ID: charge.ID, PricePerKW: charge.PricePerKW, TopN: charge.TopN, AlreadyKW: charge.AlreadyKW, @@ -361,9 +390,9 @@ func (o *ExternalOptimizer) buildRequest(slots []Slot, p Params) externalRequest Weight: hour.Weight, Group: hour.Group, }) } - req.DemandCharges = append(req.DemandCharges, wire) + out = append(out, wire) } - return req + return out } func (r externalResponse) toPlan(slots []Slot, p Params) Plan { @@ -669,5 +698,9 @@ func validatePlan(slots []Slot, p Params, plan *Plan, vehicleRules bool) error { if math.Abs(plan.TotalCostOre-totalCost) > 0.1 { return fmt.Errorf("total cost %.4f, want %.4f", plan.TotalCostOre, totalCost) } + // Last, so an overrun hour is the only rule the plan breaks. + if err := validateImportHours(slots, p, plan.Actions); err != nil { + return &importHourOverrun{plan: *plan, err: err} + } return nil } diff --git a/go/internal/mpc/external_optimizer_test.go b/go/internal/mpc/external_optimizer_test.go index 87559cdb6..44a421243 100644 --- a/go/internal/mpc/external_optimizer_test.go +++ b/go/internal/mpc/external_optimizer_test.go @@ -19,7 +19,7 @@ func TestExternalRequestNegotiatesPublishedPricesAcrossWorkerRollback(t *testing for _, supported := range []bool{true, false, true} { transport.publishedPrices = supported request := external.buildRequest(slots, params) - if err := optimizer.prepareEnergyplanRequest(context.Background(), &request, params); err != nil { + if err := optimizer.prepareEnergyplanRequest(context.Background(), &request, slots, params); err != nil { t.Fatal(err) } raw, err := json.Marshal(request) diff --git a/go/internal/mpc/import_hours.go b/go/internal/mpc/import_hours.go new file mode 100644 index 000000000..dfc502db1 --- /dev/null +++ b/go/internal/mpc/import_hours.go @@ -0,0 +1,187 @@ +package mpc + +import ( + "fmt" + "math" + "time" +) + +// ImportCeiling is the owner's cap on each local clock hour's mean import as +// the planner gets it from Core's live import hour +// (control.PlanningImportCeiling). +type ImportCeiling struct { + W float64 // the owner's ceiling; 0 = off + MeanW float64 // the mean import a planned hour may hold + HourStart time.Time // the local clock hour in progress + ElapsedWh float64 // what that hour has used, as Core's live limit counts it +} + +// ImportHour is one local clock hour the plan spans and the import it had +// used before the plan starts. Core expands the hours from its own clock, so +// the planner needs no calendar: a local clock hour lasts an hour on the 23- +// and 25-hour days too. +type ImportHour struct { + StartMs, EndMs int64 + ElapsedImportWh float64 +} + +const hourMs = int64(time.Hour / time.Millisecond) + +// importHours lists the clock hours the slots overlap from their execution +// start: the hour in progress with what it has used, each later hour unused. +// The last may run past the horizon. +func importHours(slots []Slot, c ImportCeiling) []ImportHour { + if !(c.W > 0) || !finite(c.MeanW) || c.MeanW < 0 || !finite(c.ElapsedWh) || c.HourStart.IsZero() || len(slots) == 0 { + return nil + } + from := slots[0].ExecutionStart() + end, ok := slotEndMs(slots[len(slots)-1]) + if !ok { + return nil + } + var out []ImportHour + for start := c.HourStart.UnixMilli(); start < end; start += hourMs { + if start+hourMs <= from { + continue + } + h := ImportHour{StartMs: start, EndMs: start + hourMs} + if start == c.HourStart.UnixMilli() { + h.ElapsedImportWh = c.ElapsedWh + } + out = append(out, h) + } + return out +} + +// importLeftWh is what the plan may import in h: the hour's mean over its +// length, less what it had used. Negative once overrun. +func importLeftWh(h ImportHour, meanW float64) float64 { + return meanW*float64(h.EndMs-h.StartMs)/float64(hourMs) - h.ElapsedImportWh +} + +// importHourPlannedH is how much of h the slots plan. +func importHourPlannedH(slots []Slot, h ImportHour) float64 { + var spanH float64 + for _, s := range slots { + spanH += importHourOverlapH(s, h) + } + return spanH +} + +func importHourOverlapH(s Slot, h ImportHour) float64 { + end, ok := slotEndMs(s) + if !ok { + return 0 + } + return float64(max(0, min(end, h.EndMs)-max(s.ExecutionStart(), h.StartMs))) / float64(hourMs) +} + +// importHourCapsW spreads each hour's import left over its slots: the most a +// slot's mean import may be, without a car charging (caps) and with one +// (carCaps). Each slot keeps its own house import, so doing nothing stays +// feasible, and they share the room left evenly, never above the hour's +// mean: Core's live limit starts each hour at the mean, so it lets an even +// spread through. A car gets no house floor. In an hour the house alone +// overruns, the live limit holds the car, so no car charges there: its cap +// is negative. Nil when no hour applies. +func importHourCapsW(slots []Slot, p Params) (caps, carCaps []float64) { + if len(p.ImportHours) == 0 { + return nil, nil + } + caps, carCaps = make([]float64, len(slots)), make([]float64, len(slots)) + for i := range caps { + caps[i], carCaps[i] = math.Inf(1), math.Inf(1) + } + for _, h := range p.ImportHours { + var spanH, houseWh float64 + for _, s := range slots { + if ov := importHourOverlapH(s, h); ov > 0 { + spanH += ov + houseWh += max(0, s.LoadW+s.PVW) * ov + } + } + if spanH <= 0 { + continue + } + leftWh := max(0, importLeftWh(h, p.ImportMeanW)) + meanW, roomW := leftWh/spanH, (leftWh-houseWh)/spanH + for i, s := range slots { + if importHourOverlapH(s, h) > 0 { + house := max(0, s.LoadW+s.PVW) + caps[i] = min(caps[i], max(house, min(house+max(0, roomW), meanW))) + carCaps[i] = min(carCaps[i], min(house+roomW, meanW)) + if roomW < 0 { + carCaps[i] = math.Inf(-1) + } + } + } + } + return caps, carCaps +} + +// evAverageRoomW is the most mean EV power a slot holds under capW of mean +// import, with baseW the rest of its grid flow and the car pulsing at +// peakW. A pulse imports while it runs even when the slot exports between. +// A negative cap holds no car. +func evAverageRoomW(capW, baseW, peakW float64) float64 { + if capW < 0 { + return 0 + } + off, on := max(0, baseW), max(0, baseW+peakW) + if on <= off { + return peakW + } + return max(0, min(peakW, (capW-off)*peakW/(on-off))) +} + +// importHourOverrun is a plan that keeps every rule but a clock hour's +// import. Core plans the site itself instead; when it cannot, it publishes +// this plan, and the live limit still holds each hour. +type importHourOverrun struct { + plan Plan + err error +} + +func (e *importHourOverrun) Error() string { return e.err.Error() } + +func actionChargesCar(a Action) bool { + if a.LoadpointW > solverZeroPowerToleranceW { + return true + } + for _, w := range a.LoadpointPowerW { + if w > solverZeroPowerToleranceW { + return true + } + } + return false +} + +// validateImportHours checks the plan's import in each clock hour against +// what the hour has left. In an hour where every car is off, the house's own +// import never fails it: Core cannot cut the house. A car charges only in +// the room the hour has left, since the live limit cuts the car first. +func validateImportHours(slots []Slot, p Params, actions []Action) error { + for _, h := range p.ImportHours { + var spanH, importWh, houseWh float64 + car := false + for i, s := range slots { + ov := importHourOverlapH(s, h) + if ov <= 0 { + continue + } + spanH += ov + pulseGridW(actions[i], func(share, gridW float64) { importWh += share * max(0, gridW) * ov }) + houseWh += max(0, s.LoadW+s.PVW) * ov + car = car || actionChargesCar(actions[i]) + } + leftWh := importLeftWh(h, p.ImportMeanW) + if car { + houseWh = 0 + } + if importWh > max(leftWh, houseWh, 0)+solverGridLimitToleranceW*spanH { + return fmt.Errorf("clock hour from %s imports %.0f Wh, over the %.0f Wh it has left", + time.UnixMilli(h.StartMs).UTC().Format(time.RFC3339), importWh, leftWh) + } + } + return nil +} diff --git a/go/internal/mpc/import_hours_test.go b/go/internal/mpc/import_hours_test.go new file mode 100644 index 000000000..bd6f00f13 --- /dev/null +++ b/go/internal/mpc/import_hours_test.go @@ -0,0 +1,552 @@ +package mpc + +import ( + "context" + "encoding/json" + "fmt" + "math" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/state" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +func stockholm(t testing.TB) *time.Location { + t.Helper() + zone, err := time.LoadLocation("Europe/Stockholm") + if err != nil { + t.Fatal(err) + } + return zone +} + +// planImportWh is the plan's import in [fromMs, toMs), counting a charging +// pulse's import while it runs. +func planImportWh(plan Plan, fromMs, toMs int64) float64 { + var wh float64 + for _, a := range plan.Actions { + start := max(a.ExecutionStart(), fromMs) + end := min(a.SlotStartMs+int64(a.SlotLenMin)*60_000, toMs) + if end <= start { + continue + } + pulseGridW(a, func(share, gridW float64) { wh += share * max(0, gridW) * float64(end-start) / 3.6e6 }) + } + return wh +} + +// importCeilingFieldService is the home box at 09:00 on 25 October 2026 +// (v0.155.1-beta.1): an hourly import ceiling of 4.5 kW, about 0.8 kW of +// house load, a 9.6 kWh battery that may feed the car, and a car that wants +// 19 kWh more by 13:00 on a 3 × 20 A fuse. The plan ran the car at 4.83 kW +// from 09:00, imported 4.5-5.7 kWh an hour and called the goal met. Core held +// each hour under 4.5 kWh. A planned hour holds the ceiling less half of +// Core's 690 W rise step (control.PlanningImportCeiling). +func importCeilingFieldService(t *testing.T, optimizer PlanOptimizer, ceiling bool) (*Service, time.Time) { + t.Helper() + start := time.Date(2026, 10, 25, 9, 0, 0, 0, stockholm(t)) + st, err := state.Open(filepath.Join(t.TempDir(), "state.db")) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { st.Close() }) + prices := make([]state.PricePoint, 32) + for i := range prices { + prices[i] = state.PricePoint{ + Zone: "SE3", SlotTsMs: start.Add(time.Duration(i) * 15 * time.Minute).UnixMilli(), SlotLenMin: 15, + SpotOreKwh: 40 + float64(i%4), TotalOreKwh: 90 + float64(i%4), Source: "test", FetchedAtMs: start.UnixMilli(), + } + } + if err := st.SavePrices(prices); err != nil { + t.Fatal(err) + } + svc := New(st, nil, "SE3", Params{ + Mode: ModeArbitrage, SoCLevels: 21, ActionLevels: 21, + CapacityWh: 9600, InitialSoC: .5, SoCMin: .1, SoCMax: .95, + MaxChargeW: 5000, MaxDischargeW: 5000, ChargeEfficiency: .95, DischargeEfficiency: .95, + }) + svc.Optimizer = optimizer + svc.Horizon = 8 * time.Hour + svc.BaseLoad = 800 + svc.now = func() time.Time { return start.Add(30 * time.Second) } + tel := telemetry.NewStore() + soc := .5 + tel.Update("sungrow", telemetry.DerBattery, 0, &soc, nil) + tel.DriverHealthMut("sungrow").RecordSuccess() + svc.Tele = tel + svc.UpdateBatteryFleet([]BatteryFleetMember{{Driver: "sungrow", CapacityWh: 9600, MaxChargeW: 5000, MaxDischargeW: 5000}}, 9600, 5000, 5000) + svc.UpdateSiteEconomics(SiteEconomics{FuseMaxW: 13800, Timezone: "Europe/Stockholm"}) + svc.Loadpoint = func(int) *LoadpointSpec { + return &LoadpointSpec{ID: "carport", CapacityWh: 75000, InitialSoC: .8 - 19000./75000, SoCMax: 1, + Levels: 21, PluggedIn: true, TargetSoC: .8, TargetSlotIdx: 15, MinChargeW: 4140, MaxChargeW: 11040, + AllowedStepsW: []float64{0, 4140, 4830, 5520, 6210, 6900, 7590, 8280, 8970, 9660, 10350, 11040}, + ChargeEfficiency: .9} + } + if ceiling { + svc.ImportCeiling = func() ImportCeiling { + return ImportCeiling{W: 4500, MeanW: 4155, HourStart: start, ElapsedWh: 800 * 30 / 3600.} + } + } + return svc, start +} + +func checkImportCeilingFieldPlan(t *testing.T, svc *Service, start time.Time) Plan { + t.Helper() + if svc.Replan(context.Background()) == nil { + t.Fatalf("no plan published: %+v", svc.PlanSnapshot()) + } + plan := *svc.PlanSnapshot().Plan + for h := 0; h < 8; h++ { + from := start.Add(time.Duration(h) * time.Hour) + budget := 4155.0 + if h == 0 { + budget -= 800 * 30 / 3600. + } + if got := planImportWh(plan, from.UnixMilli(), from.Add(time.Hour).UnixMilli()); got > budget+1 { + t.Errorf("%s: plan imports %.0f Wh, over the hour's %.0f Wh", from.Format("15:04"), got, budget) + } + } + short := plan.LoadpointShortfallWh["carport"] + // The battery covers the house, so the car gets 4 h × 4.155 kW at 90 %: + // about 15 kWh of 19. + if delivered := 19000 - short; short < 1000 || delivered < 14500 { + t.Fatalf("goal by 13:00: %.0f Wh short, %.0f Wh delivered; want it flagged short with the hours' room used", short, delivered) + } + return plan +} + +func TestImportCeilingFieldCase(t *testing.T) { + t.Run("the hours stay under the ceiling and the goal is short", func(t *testing.T) { + svc, start := importCeilingFieldService(t, nil, true) + plan := checkImportCeilingFieldPlan(t, svc, start) + // The battery covers the house so the car gets the hour's room. + for _, a := range plan.Actions[:4] { + if a.LoadpointW > 0 && a.BatteryW > -700 { + t.Errorf("%s: battery %.0f W beside the car; want it covering the house", time.UnixMilli(a.SlotStartMs).In(start.Location()).Format("15:04"), a.BatteryW) + } + } + }) + t.Run("without the ceiling the plan overruns the hours", func(t *testing.T) { + // The scenario exercises the ceiling: master's plan for it. + svc, start := importCeilingFieldService(t, nil, false) + if svc.Replan(context.Background()) == nil { + t.Fatal("no plan published") + } + plan := *svc.PlanSnapshot().Plan + if got := planImportWh(plan, start.UnixMilli(), start.Add(time.Hour).UnixMilli()); got < 5000 { + t.Fatalf("09:00 imports %.0f Wh; the scenario needs a plan over the ceiling without it", got) + } + if short := plan.LoadpointShortfallWh["carport"]; short > 1 { + t.Fatalf("without the ceiling the goal is %.0f Wh short; the scenario needs a goal the fuse alone allows", short) + } + }) +} + +// The bundled worker does not announce hourly_import_caps: Core rejects its +// plan when it overruns an hour and publishes its own. +func TestNativeImportCeilingFieldCase(t *testing.T) { + worker := nativeWorker(t, 1500*time.Millisecond) + t.Cleanup(func() { _ = worker.Close() }) + svc, start := importCeilingFieldService(t, worker, true) + plan := checkImportCeilingFieldPlan(t, svc, start) + if plan.Solver == nil || !plan.Solver.Fallback || !strings.Contains(plan.Solver.FallbackReason, "clock hour") { + t.Fatalf("solver %+v; want Core's plan after the worker's overran an hour", plan.Solver) + } +} + +// A local clock hour lasts an hour on the 25-hour day: 02:00 comes twice. +func TestImportHoursOnTheLongDay(t *testing.T) { + zone := stockholm(t) + hourStart := time.Date(2026, 10, 25, 1, 0, 0, 0, zone) // CEST + now := hourStart.Add(40 * time.Minute) + slots := make([]Slot, 0, 14) + for at := now.Truncate(15 * time.Minute); at.Before(hourStart.Add(4 * time.Hour)); at = at.Add(15 * time.Minute) { + slots = append(slots, Slot{StartMs: at.UnixMilli(), LenMin: 15, LoadW: 500, PriceOre: 100}) + } + slots[0].ExecutionStartMs = now.UnixMilli() + hours := importHours(slots, ImportCeiling{W: 4500, MeanW: 4500, HourStart: hourStart, ElapsedWh: 3500}) + var got []string + for _, h := range hours { + got = append(got, time.UnixMilli(h.StartMs).In(zone).Format("15:04 MST")) + if h.EndMs-h.StartMs != hourMs { + t.Errorf("hour from %s lasts %d ms", got[len(got)-1], h.EndMs-h.StartMs) + } + } + if want := "01:00 CEST,02:00 CEST,02:00 CET,03:00 CET"; strings.Join(got, ",") != want { + t.Fatalf("hours %s, want %s", strings.Join(got, ","), want) + } + if hours[0].ElapsedImportWh != 3500 || hours[1].ElapsedImportWh != 0 || hours[2].ElapsedImportWh != 0 { + t.Fatalf("elapsed %+v", hours) + } + // Each of the two 02:00 hours holds its own 4.5 kWh. + caps, _ := importHourCapsW(slots, Params{ImportMeanW: 4500, ImportHours: hours}) + for i, s := range slots { + want := 4500.0 + if s.StartMs < hourStart.Add(time.Hour).UnixMilli() { + want = 1000 / (20. / 60) // what 01:00 has left over its last 20 minutes + } + if math.Abs(caps[i]-want) > 1e-6 { + t.Errorf("slot %s: cap %.1f W, want %.1f W", time.UnixMilli(s.StartMs).In(zone).Format("15:04 MST"), caps[i], want) + } + } +} + +// The hour in progress starts before the in-flight slot's execution and the +// last runs past the horizon: the plan's part of each may use what the hour +// has left, as Energyplan's hourly_import_caps count it. +func TestImportHoursAroundThePlan(t *testing.T) { + start := time.Date(2026, 10, 26, 9, 0, 0, 0, time.UTC) + slots := []Slot{ + {StartMs: start.Add(15 * time.Minute).UnixMilli(), ExecutionStartMs: start.Add(20 * time.Minute).UnixMilli(), LenMin: 15}, + {StartMs: start.Add(30 * time.Minute).UnixMilli(), LenMin: 15}, + } + c := ImportCeiling{W: 4690, MeanW: 4000, HourStart: start, ElapsedWh: 1000} + hours := importHours(slots, c) + if len(hours) != 1 || hours[0].StartMs != start.UnixMilli() || hours[0].EndMs != start.Add(time.Hour).UnixMilli() || hours[0].ElapsedImportWh != 1000 { + t.Fatalf("hours %+v", hours) + } + // 3 kWh left over the plan's 25 minutes of the hour. + if caps, _ := importHourCapsW(slots, Params{ImportMeanW: c.MeanW, ImportHours: hours}); math.Abs(caps[0]-3000/(25./60)) > 1e-6 { + t.Fatalf("caps %v", caps) + } + c.ElapsedWh = 5000 // overrun: only the house may import + slots[1].LoadW = 600 + if caps, carCaps := importHourCapsW(slots, Params{ImportMeanW: c.MeanW, ImportHours: importHours(slots, c)}); caps[0] != 0 || caps[1] != 600 || !math.IsInf(carCaps[0], -1) || !math.IsInf(carCaps[1], -1) { + t.Fatalf("overrun caps %v, with a car %v", caps, carCaps) + } + if importHours(slots, ImportCeiling{MeanW: 4000, HourStart: start}) != nil { + t.Fatal("no ceiling gave hours") + } + if got := importHours(slots, ImportCeiling{W: 500, HourStart: start}); len(got) != 1 { + t.Fatalf("a ceiling below the rise step gave %+v; want the hour with no room", got) + } +} + +// Each slot keeps its house import and shares the room left, never above +// the hour's mean, so the plan imports evenly as Core's live limit allows. +func TestImportHourCapsSpreadTheRoom(t *testing.T) { + start := time.Date(2026, 10, 26, 9, 0, 0, 0, time.UTC) + slots := make([]Slot, 4) + for i, w := range []float64{4000, 500, 500, -2000} { + slots[i] = Slot{StartMs: start.Add(time.Duration(i) * 15 * time.Minute).UnixMilli(), LenMin: 15, LoadW: max(0, w), PVW: min(0, w)} + } + p := Params{ImportMeanW: 3000, ImportHours: []ImportHour{{StartMs: start.UnixMilli(), EndMs: start.Add(time.Hour).UnixMilli()}}} + caps, carCaps := importHourCapsW(slots, p) + // The house takes 1.25 kWh and leaves 1.75 kWh: 1.75 kW on top of each + // slot's house, never above the 3 kW mean. The 4 kW slot keeps its house, + // but a car there gets only the mean. + for i, want := range []float64{4000, 2250, 2250, 1750} { + if math.Abs(caps[i]-want) > 1e-9 || math.Abs(carCaps[i]-min(want, 3000)) > 1e-9 { + t.Errorf("slot %d: cap %.1f W, with a car %.1f W, want %.1f W", i, caps[i], carCaps[i], want) + } + } + p.ImportMeanW = 500 // less than the house: no room for a car + caps, carCaps = importHourCapsW(slots, p) + for i, want := range []float64{4000, 500, 500, 0} { + if math.Abs(caps[i]-want) > 1e-9 || !math.IsInf(carCaps[i], -1) { + t.Errorf("overrun hour, slot %d: cap %.1f W, with a car %.1f W, want %.1f W and none", i, caps[i], carCaps[i], want) + } + } +} + +func TestValidatePlanHoldsEachClockHourToItsBudget(t *testing.T) { + start := time.Date(2026, 10, 26, 9, 0, 0, 0, time.UTC) + slots := make([]Slot, 8) + for i := range slots { + slots[i] = Slot{StartMs: start.Add(time.Duration(i) * 15 * time.Minute).UnixMilli(), LenMin: 15, LoadW: 800, PriceOre: 100, SpotOre: 50} + } + p := Params{Mode: ModeArbitrage, CapacityWh: 10000, InitialSoC: .5, SoCMin: .1, SoCMax: .9, MaxChargeW: 5000, MaxDischargeW: 5000, + ChargeEfficiency: 1, DischargeEfficiency: 1, ImportMeanW: 4500, + ImportHours: []ImportHour{ + {StartMs: start.UnixMilli(), EndMs: start.Add(time.Hour).UnixMilli()}, + {StartMs: start.Add(time.Hour).UnixMilli(), EndMs: start.Add(2 * time.Hour).UnixMilli()}, + }} + planWith := func(batteryW map[int]float64) Plan { + var plan Plan + soc := p.InitialSoC + for i, s := range slots { + b := batteryW[i] + soc += b * .25 / p.CapacityWh + a := Action{SlotStartMs: s.StartMs, SlotLenMin: 15, LoadW: s.LoadW, BatteryW: b, GridW: s.LoadW + b, SoC: soc} + a.CostOre = reserveSlotCost(s, p, a) + plan.TotalCostOre += a.CostOre + plan.Actions = append(plan.Actions, a) + } + return plan + } + // 0.8 kW + 4.9 kW in one quarter is 5.7 kW, under the fuse, but the + // hour's mean is 2.0 kW: inside the budget. + if plan := planWith(map[int]float64{2: 4900}); ValidatePlan(slots, p, &plan) != nil { + t.Fatal(ValidatePlan(slots, p, &plan)) + } + // 3.9 kW of battery charge all through 10:00 makes a 4.7 kW mean. + plan := planWith(map[int]float64{4: 3900, 5: 3900, 6: 3900, 7: 3900}) + if err := ValidatePlan(slots, p, &plan); err == nil || !strings.Contains(err.Error(), "clock hour") { + t.Fatalf("hour of 4.7 kW mean passed: %v", err) + } + // The house alone may overrun an hour that has used its budget. + p.ImportHours[0].ElapsedImportWh = 4400 + if plan := planWith(nil); ValidatePlan(slots, p, &plan) != nil { + t.Fatal(ValidatePlan(slots, p, &plan)) + } + if plan := planWith(map[int]float64{0: 1000}); ValidatePlan(slots, p, &plan) == nil { + t.Fatal("battery charge into an overrun hour passed") + } +} + +// A car pulse imports while it runs even when the slot exports between. +func TestValidatePlanCountsPulseImport(t *testing.T) { + start := time.Date(2026, 10, 26, 12, 0, 0, 0, time.UTC) + slots := make([]Slot, 4) + for i := range slots { + slots[i] = Slot{StartMs: start.Add(time.Duration(i) * 15 * time.Minute).UnixMilli(), LenMin: 15, LoadW: 500, PVW: -4500, PriceOre: 100, SpotOre: 50} + } + lp := &LoadpointSpec{ID: "car", CapacityWh: 60000, Levels: 11, SoCMax: 1, InitialSoC: .5, PluggedIn: true, + MaxChargeW: 11000, AllowedStepsW: []float64{0, 11000}, ChargeEfficiency: 1} + p := Params{Mode: ModeArbitrage, ChargeEfficiency: 1, DischargeEfficiency: 1, Loadpoint: lp, ImportMeanW: 1000, + ImportHours: []ImportHour{{StartMs: start.UnixMilli(), EndMs: start.Add(time.Hour).UnixMilli()}}} + var plan Plan + soc := lp.InitialSoC + for _, s := range slots { + // 4 kW mean: 11 kW for 4/11 of the slot imports 7 kW, else 4 kW export. + soc += 4000 * .25 / lp.CapacityWh + a := Action{SlotStartMs: s.StartMs, SlotLenMin: 15, LoadW: s.LoadW, PVW: s.PVW, GridW: 0, LoadpointW: 4000, LoadpointSoC: soc, + LoadpointPowerW: map[string]float64{"car": 4000}, LoadpointSoCByID: map[string]float64{"car": soc}, LoadpointMaxPowerW: map[string]float64{"car": 11000}} + a.CostOre = reserveSlotCost(s, p, a) + plan.TotalCostOre += a.CostOre + plan.Actions = append(plan.Actions, a) + } + // Mean grid flow is 0 W, but the pulses import 7 kW × 4/11 ≈ 2.5 kWh. + if err := ValidatePlan(slots, p, &plan); err == nil || !strings.Contains(err.Error(), "clock hour") { + t.Fatalf("pulses importing 2.5 kWh passed a 1 kWh hour: %v", err) + } +} + +// The DP charges the battery in the cheap hour only as far as the hour's +// budget leaves room, and the plan passes the same check. +func TestOptimizeKeepsEachHourUnderItsBudget(t *testing.T) { + start := time.Date(2026, 10, 26, 1, 0, 0, 0, time.UTC) + slots := make([]Slot, 8) + for i := range slots { + price := 20.0 + if i >= 4 { + price = 300 + } + slots[i] = Slot{StartMs: start.Add(time.Duration(i) * 15 * time.Minute).UnixMilli(), LenMin: 15, LoadW: 800, PriceOre: price, SpotOre: price / 2} + } + p := Params{Mode: ModeArbitrage, SoCLevels: 41, ActionLevels: 41, CapacityWh: 20000, InitialSoC: .2, SoCMin: .1, SoCMax: .95, + MaxChargeW: 8000, MaxDischargeW: 8000, ChargeEfficiency: .95, DischargeEfficiency: .95} + free := Optimize(slots, p) + if got := planImportWh(free, start.UnixMilli(), start.Add(time.Hour).UnixMilli()); got < 6000 { + t.Fatalf("without a budget the cheap hour imports %.0f Wh; the test needs it over 3 kWh", got) + } + p.ImportMeanW, p.ImportHours = 3000, []ImportHour{{StartMs: start.UnixMilli(), EndMs: start.Add(time.Hour).UnixMilli()}} + plan := Optimize(slots, p) + if err := ValidatePlan(slots, p, &plan); err != nil { + t.Fatal(err) + } + // The 400 W action grid reaches 2.8 of the 3 kWh. + if got := planImportWh(plan, start.UnixMilli(), start.Add(time.Hour).UnixMilli()); got > 3000+1e-6 || got < 2700 { + t.Fatalf("cheap hour imports %.0f Wh; want the room of its 3000 Wh used", got) + } +} + +type importCapTransport struct{ caps bool } + +func (c *importCapTransport) RoundTrip(context.Context, []byte) ([]byte, error) { + features := []string{"champion", "ev_duty", "published_prices"} + if c.caps { + features = append(features, "hourly_import_caps") + } + return json.Marshal(map[string]any{"name": "ftw-solver", "version": "test", "protocol_version": 1, "features": features}) +} +func (*importCapTransport) Close() error { return nil } + +// A worker that announces hourly_import_caps gets each hour, the mean and +// what the hour in progress had used. Any other worker, and every worker +// while the ceiling is off, gets the request it got before. +func TestEnergyplanRequestCarriesHourlyImportCaps(t *testing.T) { + slots, p := externalTestFixture() + transport := &importCapTransport{} + external := &ExternalOptimizer{cfg: ExternalOptimizerConfig{Timeout: 7 * time.Second}, transport: transport, timeBudget: energyplanTimeBudget} + o := &EnergyplanOptimizer{ExternalOptimizer: external} + request := func(p Params) string { + t.Helper() + r := external.buildRequest(slots, p) + r.RequestID = "fixed" + if err := o.prepareEnergyplanRequest(context.Background(), &r, slots, p); err != nil { + t.Fatal(err) + } + raw, err := json.Marshal(r) + if err != nil { + t.Fatal(err) + } + return string(raw) + } + off := p + on := p + on.ImportMeanW = 3810 + // The third hour lies past the slots: it is not sent. + on.ImportHours = []ImportHour{{StartMs: 0, EndMs: hourMs, ElapsedImportWh: 125}, {StartMs: hourMs, EndMs: 2 * hourMs}, {StartMs: 3 * hourMs, EndMs: 4 * hourMs}} + for _, caps := range []bool{true, false} { + transport.caps = caps + if request(on) == request(off) != !caps { + t.Fatalf("caps=%v: with the ceiling on the request changes %v", caps, request(on) != request(off)) + } + } + transport.caps = true + if got, want := request(on), `"hourly_import_caps":[{"start_ms":0,"end_ms":3600000,"max_mean_w":3810,"elapsed_import_wh":125},{"start_ms":3600000,"end_ms":7200000,"max_mean_w":3810,"elapsed_import_wh":0}]`; !strings.Contains(got, want) { + t.Fatalf("request %s\nwant %s", got, want) + } + // With the hours the worker gets the fleet budget; without them its own. + if got := request(on); !strings.Contains(got, `"time_limit_s":5,`) || !strings.Contains(request(off), `"time_limit_s":0.5,`) { + t.Fatalf("time limits: with hours %s; without %s", got, request(off)) + } + if strings.Contains(request(off), "hourly_import_caps") { + t.Fatal("the ceiling off sent caps") + } +} + +// ceilinglessOptimizer plans as a worker without the hours does: its plan +// keeps every rule but the hours, and Core's check names the hour. +type ceilinglessOptimizer struct{} + +func (ceilinglessOptimizer) Optimize(ctx context.Context, slots []Slot, p Params) (Plan, error) { + free := p + free.ImportHours, free.ImportMeanW = nil, 0 + plan, err := coreReservePlan(ctx, slots, free) + if err != nil { + return Plan{}, err + } + if err := ValidatePlan(slots, p, &plan); err != nil { + return Plan{}, fmt.Errorf("optimizer plan rejected: %w", err) + } + return plan, nil +} + +func (ceilinglessOptimizer) Close() error { return nil } + +// A worker plan that overruns an hour falls back to Core's own plan. When +// Core cannot plan the site itself, here a battery below its floor, the +// worker's plan is published as before the hours, with a warning, rather +// than no plan at all. +func TestImportCeilingWorkerPlanOverrunsAnHour(t *testing.T) { + for _, soc := range []float64{.5, .05} { + t.Run(fmt.Sprintf("battery %.0f %%", soc*100), func(t *testing.T) { + svc, start := importCeilingFieldService(t, ceilinglessOptimizer{}, true) + svc.Tele.Update("sungrow", telemetry.DerBattery, 0, &soc, nil) + if svc.Replan(context.Background()) == nil { + t.Fatalf("no plan published: %+v", svc.PlanSnapshot()) + } + snap := svc.PlanSnapshot() + got := planImportWh(*snap.Plan, start.UnixMilli(), start.Add(time.Hour).UnixMilli()) + warned := strings.Contains(strings.Join(snap.Warnings, " "), "Core cannot plan this site itself") + if soc == .5 { + if !snap.Plan.Solver.Fallback || got > 4155 || warned { + t.Fatalf("solver %+v, 09:00 imports %.0f Wh, warned %v; want Core's plan inside the hour", snap.Plan.Solver, got, warned) + } + return + } + if snap.Plan.Solver.Fallback || got <= 4155 || !warned { + t.Fatalf("solver %+v, 09:00 imports %.0f Wh, warnings %q; want the worker's plan with a warning", snap.Plan.Solver, got, snap.Warnings) + } + }) + } +} + +func TestNativeImportCeilingWhenCoreCannotPlan(t *testing.T) { + worker := nativeWorker(t, 1500*time.Millisecond) + t.Cleanup(func() { _ = worker.Close() }) + svc, _ := importCeilingFieldService(t, worker, true) + soc := .05 + svc.Tele.Update("sungrow", telemetry.DerBattery, 0, &soc, nil) + if svc.Replan(context.Background()) == nil { + t.Fatalf("no plan published: %+v", svc.PlanSnapshot()) + } + if w := strings.Join(svc.PlanSnapshot().Warnings, " "); !strings.Contains(w, "Core cannot plan this site itself") { + t.Fatalf("warnings %q", w) + } +} + +// A car step that needs the battery to discharge past the house fits the +// fuse only while it runs. When the hour makes it part-time, the battery +// would drive export beside the car: the reserve plan takes a smaller step. +func TestReservePlanUnderAnHourNeverChargesTheCarFromExport(t *testing.T) { + first := time.Date(2026, 10, 26, 10, 40, 0, 0, time.UTC) + for _, fuse := range []float64{6900, 8280} { + slots := make([]Slot, 12) + for i := range slots { + slots[i] = Slot{StartMs: first.Add(time.Duration(i) * 15 * time.Minute).UnixMilli(), LenMin: 15, LoadW: 500, PriceOre: 100, SpotOre: 50, + Limits: PowerLimits{MaxImportW: fuse, MaxExportW: fuse}} + } + p := Params{Mode: ModeArbitrage, SoCLevels: 21, ActionLevels: 21, CapacityWh: 10000, InitialSoC: .8, SoCMin: .1, SoCMax: .95, + MaxChargeW: 5000, MaxDischargeW: 5000, ChargeEfficiency: .95, DischargeEfficiency: .95, + Loadpoint: &LoadpointSpec{ID: "car", CapacityWh: 60000, InitialSoC: .3, SoCMax: 1, Levels: 21, PluggedIn: true, MinChargeW: 4140, MaxChargeW: 11040, + AllowedStepsW: []float64{0, 4140, 6900, 11040}, ChargeEfficiency: .9, TargetSoC: .5, TargetSlotIdx: 10}} + c := ImportCeiling{W: 4500, MeanW: 4155, HourStart: first.Add(-40 * time.Minute), ElapsedWh: 3800} + p.ImportHours, p.ImportMeanW = importHours(slots, c), c.MeanW + if _, used, err := corePlanWithRecovery(context.Background(), slots, p); err != nil || len(used) != len(slots) { + t.Errorf("fuse %.0f W: %d of %d slots, err %v", fuse, len(used), len(slots), err) + } + } +} + +// The house may overrun an hour that has used its budget, but a car may not +// charge there on the import the battery frees by covering the house: Core's +// live limit stops the car. The plan holds the car to the next hour. +func TestNoCarOnTheHouseFloor(t *testing.T) { + start := time.Date(2026, 10, 26, 9, 0, 0, 0, time.UTC) + slots := make([]Slot, 8) + for i := range slots { + slots[i] = Slot{StartMs: start.Add(time.Duration(i) * 15 * time.Minute).UnixMilli(), LenMin: 15, LoadW: 800, PriceOre: 100, SpotOre: 50, + Limits: PowerLimits{MaxImportW: 13800, MaxExportW: 13800}} + } + lp := &LoadpointSpec{ID: "car", CapacityWh: 60000, InitialSoC: .3, SoCMax: 1, Levels: 21, PluggedIn: true, MinChargeW: 4140, MaxChargeW: 11040, + AllowedStepsW: []float64{0, 4140, 11040}, ChargeEfficiency: .9, TargetSoC: .5, TargetSlotIdx: 7} + p := Params{Mode: ModeArbitrage, SoCLevels: 21, ActionLevels: 21, CapacityWh: 10000, InitialSoC: .8, SoCMin: .1, SoCMax: .95, + MaxChargeW: 5000, MaxDischargeW: 5000, ChargeEfficiency: 1, DischargeEfficiency: 1, Loadpoint: lp, ImportMeanW: 4155, + ImportHours: []ImportHour{ + {StartMs: start.UnixMilli(), EndMs: start.Add(time.Hour).UnixMilli(), ElapsedImportWh: 4000}, + {StartMs: start.Add(time.Hour).UnixMilli(), EndMs: start.Add(2 * time.Hour).UnixMilli()}, + }} + plan, used, err := corePlanWithRecovery(context.Background(), slots, p) + if err != nil || len(used) != len(slots) { + t.Fatalf("%d of %d slots, err %v", len(used), len(slots), err) + } + for i, a := range plan.Actions { + if i < 4 && a.LoadpointW > 0 { + t.Errorf("slot %d: car at %.0f W in the hour that has 155 Wh left for 800 W of house", i, a.LoadpointW) + } + } + if plan.Actions[4].LoadpointW <= 0 { + t.Errorf("the car waits past 10:00: %+v", plan.Actions[4]) + } + // The same car beside a battery covering the house fails the check. + a := &plan.Actions[0] + a.BatteryW, a.LoadpointW = -800, 800 + a.LoadpointPowerW, a.LoadpointMaxPowerW = map[string]float64{"car": 800}, map[string]float64{"car": 4140} + err = validateImportHours(slots, p, plan.Actions) + if err == nil || !strings.Contains(err.Error(), "clock hour") { + t.Fatalf("car on the house floor passed: %v", err) + } +} + +// The DP plans on a SoC grid. Replayed from the real SoC, the battery can run +// short of the discharge that kept an hour under its budget, here at 12:45 +// in the field case: the car gives way rather than the hour overrunning. +func TestOptimizeCarGivesWayWhenTheBatteryRunsShort(t *testing.T) { + svc, _ := importCeilingFieldService(t, nil, true) + if svc.Replan(context.Background()) == nil { + t.Fatal("no plan published") + } + svc.mu.RLock() + slots, p := append([]Slot(nil), svc.lastSlots...), svc.lastParams + svc.mu.RUnlock() + plan := Optimize(slots, p) + if err := ValidatePlan(slots, p, &plan); err != nil { + t.Fatal(err) + } +} diff --git a/go/internal/mpc/mpc.go b/go/internal/mpc/mpc.go index 7030beda1..8118f6f0c 100644 --- a/go/internal/mpc/mpc.go +++ b/go/internal/mpc/mpc.go @@ -248,6 +248,14 @@ type Params struct { // DemandCharges are peak-power tariffs Core expanded to clock hours. // Empty leaves Energyplan on energy prices only. DemandCharges []DemandCharge + + // ImportHours carry the owner's hourly import ceiling: each clock hour + // the plan spans may import ImportMeanW over its length, less what it had + // used. ValidatePlan holds every plan to them, Core's own planners spread + // each hour over its slots, and a worker that announces + // hourly_import_caps gets them on its wire. + ImportMeanW float64 + ImportHours []ImportHour } // StorageAssetSpec is one independently constrained home battery in the @@ -660,6 +668,7 @@ func OptimizeContext(ctx context.Context, slots []Slot, p Params) (Plan, error) // The mean price scales the EV deadline penalty and labels reasons. meanPrice := horizonMeanPrice(slots) + importCaps, carImportCaps := importHourCapsW(slots, p) // Action grid spans −MaxDischargeW … +MaxChargeW and always contains // 0 W. With asymmetric charge/discharge limits, an odd number of evenly @@ -866,10 +875,14 @@ func OptimizeContext(ctx context.Context, slots []Slot, p Params) (Plan, error) continue } - // Per-slot power limits. + // Per-slot power limits, and the slot's share of + // its clock hour's import budget. if !slot.Limits.allowsImport(gridW) || !slot.Limits.allowsExport(gridW) { continue } + if importCaps != nil && (gridW > importCaps[t] || evW > 0 && gridW > carImportCaps[t]) { + continue + } gridKWh := gridW * dtH / 1000.0 var cost float64 @@ -1102,6 +1115,20 @@ func OptimizeContext(ctx context.Context, slots []Slot, p Params) (Plan, error) } } gridW := loadpoint.GridW(slot.LoadW, slot.PVW, actW, evW) + // The real SoC can leave the battery short of the policy's + // discharge. The car then gives way to the hour's import, as + // Core's live limit cuts the car first. + if evW > 0 && carImportCaps != nil && gridW > carImportCaps[t] { + evW = 0 + for k := len(evSteps) - 1; k > 0; k-- { + if loadpoint.GridW(slot.LoadW, slot.PVW, actW, evSteps[k]) <= carImportCaps[t] { + evW = evSteps[k] + break + } + } + evSoc2 = evSoc + evW*dtH*evChargeEff/lp.CapacityWh + gridW = loadpoint.GridW(slot.LoadW, slot.PVW, actW, evW) + } gridKWh := gridW * dtH / 1000.0 // Report the ACTUAL expected cost using the raw (un-blended) // prices so the UI summary reflects "what we'd actually pay diff --git a/go/internal/mpc/reserve_price.go b/go/internal/mpc/reserve_price.go index b1610a2a3..bc052bb0a 100644 --- a/go/internal/mpc/reserve_price.go +++ b/go/internal/mpc/reserve_price.go @@ -73,6 +73,7 @@ func priceReserveCharging(ctx context.Context, slots []Slot, p Params, reserve P a.LoadpointSoCByID = map[string]float64{lp.ID: lp.InitialSoC} } remaining := energyWh + _, importCaps := importHourCapsW(slots, p) for _, o := range offers { if err := ctx.Err(); err != nil { return Plan{}, err @@ -82,6 +83,13 @@ func priceReserveCharging(ctx context.Context, slots []Slot, p Params, reserve P } hours := slots[o.index].DurationHours() average := min(o.peakW, remaining/(eff*hours)) + if importCaps != nil { + slot := slots[o.index] + average = min(average, evAverageRoomW(importCaps[o.index], slot.LoadW+slot.PVW+reserve.Actions[o.index].BatteryW, o.peakW)) + if average <= 1e-6 { + continue + } + } a := &priced.Actions[o.index] a.LoadpointW = average a.LoadpointPowerW[lp.ID] = average diff --git a/go/internal/mpc/service.go b/go/internal/mpc/service.go index d8fec988b..f3693768c 100644 --- a/go/internal/mpc/service.go +++ b/go/internal/mpc/service.go @@ -2,6 +2,7 @@ package mpc import ( "context" + "errors" "fmt" "log/slog" "maps" @@ -133,6 +134,14 @@ type Service struct { // watts. Nil or 0 plans against the aggregate fuse alone. Called // without s.mu; it may take the control lock. PhaseImbalanceW func() float64 + // ImportCeiling returns the owner's hourly import ceiling as Core + // enforces it. Nil or zero W plans without it. Called without s.mu; it + // may take the control lock. + ImportCeiling func() ImportCeiling + // HourImport returns the clock hour in progress as Core's live import + // counter measured it (control.MeasuredHourImport). The demand charges' + // hour in progress uses it before the import history. Nil: history only. + HourImport func() HourImport // Set before Start. Called without s.mu; must not acquire the control lock. PVExecutionAllowed func(PVCurtailment) bool Load LoadPredictor // optional — overrides flat BaseLoad @@ -160,6 +169,10 @@ type Service struct { // LoadpointStates supplies current saved goals without taking the control // lock. It is independent of forecasts, prices and battery qualification. LoadpointStates func() []loadpoint.State + // PlanningLimits is the power range and the steps the plan gets for a + // session (loadpoint.PlanningLimits); the deadline reserve holds to them + // too. Nil holds the reserve to the charger's own table (LimitPower). + PlanningLimits func(loadpoint.State) (minW, maxW float64, steps []float64) // SaveDiag is called synchronously after every successful replan that // remains the newest request, with the same Diagnostic the @@ -1694,6 +1707,10 @@ func (s *Service) runReplan(request replanRequest) *Plan { } limitNearImportForPhases(slots, now, fuseMaxW, phaseImbalanceW) limitNearImportForPhases(fallbackSlots, now, fuseMaxW, phaseImbalanceW) + var importCeiling ImportCeiling + if s.ImportCeiling != nil { + importCeiling = s.ImportCeiling() + } if s.PVCurtailmentProbe != nil { p.PVCurtailment = s.PVCurtailmentProbe() @@ -1741,6 +1758,10 @@ func (s *Service) runReplan(request replanRequest) *Plan { p.ExportFloorOreKwh = site.ExportFloorOreKwh p.PVForecastSafetyK = request.safetyK p.DemandCharges = s.demandChargesFor(site, slots, executionNow) + p.ImportHours = importHours(slots, importCeiling) + if p.ImportHours != nil { + p.ImportMeanW = importCeiling.MeanW + } if pvUncertainty != nil || s.ForecastSnapshot != nil { p.PVUncertaintyW = pvUncertaintyW } @@ -2059,6 +2080,17 @@ func (s *Service) runReplan(request replanRequest) *Plan { } plan = candidate + } else if overrun := (*importHourOverrun)(nil); errors.As(err, &overrun) && (recoveryRequired && !downsidePrimary || coreDPModelError(p) != nil) { + // Core cannot plan this site itself. The worker's plan keeps + // every rule but the hours, as before the hours existed; the + // live limit still holds each hour. + if request.wasCanceledByService() { + return s.canceledReplan(request, "primary-fallback") + } + s.recordPlanningFailure(request, "primary_optimizer", err.Error(), slots, p, optimizerErr) + plan = overrun.plan + p.ImportHours, p.ImportMeanW = nil, 0 + warnings = append(warnings, "Core cannot plan this site itself, so the plan may import more in an hour than the limit allows; Core still holds each hour to it: "+err.Error()) } else { if request.wasCanceledByService() { return s.canceledReplan(request, "primary-fallback") diff --git a/go/internal/mpc/startup_replan_test.go b/go/internal/mpc/startup_replan_test.go new file mode 100644 index 000000000..b92b1e0cf --- /dev/null +++ b/go/internal/mpc/startup_replan_test.go @@ -0,0 +1,63 @@ +package mpc + +import ( + "context" + "math" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/state" + "github.com/srcfl/ftw/go/internal/telemetry" +) + +func TestStartReplansAnArchivedPlanWithFreshSoC(t *testing.T) { + old := newCancellationTestService(t, nil) + if _, err := old.Store.ClearPrices(); err != nil { + t.Fatal(err) + } + start := time.Now().UTC().Truncate(15 * time.Minute) + var prices []state.PricePoint + for i := 0; i < 16; i++ { + prices = append(prices, state.PricePoint{ + Zone: "SE3", SlotTsMs: start.Add(time.Duration(i) * 15 * time.Minute).UnixMilli(), + SlotLenMin: 15, SpotOreKwh: 50, TotalOreKwh: 100, + Source: "test", FetchedAtMs: start.UnixMilli(), + }) + } + if err := old.Store.SavePrices(prices); err != nil { + t.Fatal(err) + } + if old.Replan(context.Background()) == nil { + t.Fatal("could not create the previous process's plan") + } + archived := old.Diagnose() + + tel := telemetry.NewStore() + soc := .4 + tel.Update("battery", telemetry.DerBattery, 0, &soc, nil) + tel.RecordDriverSuccess("battery") + svc := New(old.Store, tel, "SE3", old.Defaults) + svc.Horizon, svc.BaseLoad = old.Horizon, old.BaseLoad + svc.Interval = time.Hour + if !svc.RestoreDiagnostic(archived, time.Now(), "restored_diagnostic") { + t.Fatal("could not restore the archived plan") + } + saved := make(chan *Diagnostic, 1) + svc.SaveDiag = func(d *Diagnostic, _ string) error { + saved <- d + return nil + } + svc.Start(context.Background()) + t.Cleanup(svc.Stop) + select { + case d := <-saved: + if math.Abs(d.Params.InitialSoC-soc) > 1e-9 { + t.Fatalf("startup SoC = %v, want the fresh %v", d.Params.InitialSoC, soc) + } + if d.DecisionID == archived.DecisionID { + t.Fatal("startup reused the previous process's decision") + } + case <-time.After(5 * time.Second): + t.Fatal("startup waited for the scheduled interval instead of making a fresh plan") + } +} diff --git a/go/internal/nativeupdate/slots.go b/go/internal/nativeupdate/slots.go index 2488786ff..910c92f61 100644 --- a/go/internal/nativeupdate/slots.go +++ b/go/internal/nativeupdate/slots.go @@ -4,6 +4,7 @@ import ( "encoding/json" "errors" "fmt" + "log/slog" "os" "path/filepath" "regexp" @@ -184,11 +185,55 @@ func (m Manager) Select() (path, tag string, trial bool, err error) { tag = state.Trial } path, readErr = m.ReleaseDir(tag) + if readErr != nil && !trial { + // A damaged current release would fail every start. Run the + // previous one once as a trial; it commits as any trial does. + if previous := m.fallbackCandidateLocked(state); previous != "" { + slog.Warn("current release is damaged; starting the previous release once", + "current", state.Current, "previous", previous, "err", readErr) + // Keep a release that already failed marked, such as a + // trial that just crashed, so updates do not offer it again. + // A fallback that dies marks the previous release through + // the trial branch above, so it runs only once. + if state.LastFailed == "" { + state.LastFailed = state.Current + } + state.Trial = previous + if writeErr := m.writeLocked(state); writeErr != nil { + return writeErr + } + tag, trial = previous, true + path, readErr = m.ReleaseDir(tag) + } + } return readErr }) return } +// fallbackCandidateLocked returns the previous release when it is healthy +// and reads the same data as the damaged current release. A previous +// release that already failed is not tried again. Without a readable +// current receipt the state schema is unknown, so there is no fallback. +func (m Manager) fallbackCandidateLocked(state Slots) string { + if state.Previous == "" || state.Previous == state.LastFailed { + return "" + } + previousDir, err := m.ReleaseDir(state.Previous) + if err != nil { + return "" + } + previous, err := readReceipt(previousDir, state.Previous, runtime.GOARCH) + if err != nil { + return "" + } + current, err := readReceipt(filepath.Join(m.Root, "releases", state.Current), state.Current, runtime.GOARCH) + if err != nil || current.StateSchema != previous.StateSchema { + return "" + } + return state.Previous +} + func (m Manager) Commit(tag string) error { return m.change(func(state *Slots) error { if state.Trial != tag || tag == state.Current { diff --git a/go/internal/nativeupdate/slots_damaged_current_test.go b/go/internal/nativeupdate/slots_damaged_current_test.go new file mode 100644 index 000000000..f3955b6b3 --- /dev/null +++ b/go/internal/nativeupdate/slots_damaged_current_test.go @@ -0,0 +1,129 @@ +package nativeupdate + +import ( + "os" + "path/filepath" + "runtime" + "testing" + "time" +) + +// A damaged current release with a healthy, same-schema previous release +// beside it runs the previous release once as a trial. +func TestSelectFallsBackWhenCurrentReleaseIsDamaged(t *testing.T) { + clock := &testClock{t: time.Date(2026, 10, 27, 12, 0, 0, 0, time.UTC)} + m := committed(t, clock) // current v0.132.0, previous v0.131.0, same schema + // Past the one-hour probation window, as on a box that ran for days. + clock.t = clock.t.Add(48 * time.Hour) + if err := os.Remove(filepath.Join(m.Root, "releases", "v0.132.0", "web", "index.html")); err != nil { + t.Fatal(err) + } + _, tag, trial, err := m.Select() + if err != nil || tag != "v0.131.0" || !trial { + t.Fatalf("select %s trial=%v %v", tag, trial, err) + } + state, err := m.Read() + if err != nil { + t.Fatal(err) + } + if state.Trial != "v0.131.0" || state.LastFailed != "v0.132.0" || state.Current != "v0.132.0" { + t.Fatalf("slots after fallback: %+v", state) + } + if err := m.Commit("v0.131.0"); err != nil { + t.Fatal(err) + } + if got := startTag(t, m); got != "v0.131.0" { + t.Fatalf("after commit started %s", got) + } +} + +// A fallback trial that dies is not tried again; the box keeps failing on +// the damaged release as before instead of looping between the two. +func TestSelectFallsBackOnlyOnce(t *testing.T) { + clock := &testClock{t: time.Date(2026, 10, 27, 12, 0, 0, 0, time.UTC)} + m := committed(t, clock) + clock.t = clock.t.Add(48 * time.Hour) + if err := os.Remove(filepath.Join(m.Root, "releases", "v0.132.0", "web", "index.html")); err != nil { + t.Fatal(err) + } + if _, _, trial, err := m.Select(); err != nil || !trial { + t.Fatalf("first fallback trial=%v %v", trial, err) + } + // The trial exits before it commits. + if _, tag, trial, err := m.Select(); err == nil || trial { + t.Fatalf("second start %s trial=%v err=%v; want the damaged-release error", tag, trial, err) + } + state, err := m.Read() + if err != nil { + t.Fatal(err) + } + if state.Trial != "" || state.LastFailed != "v0.131.0" { + t.Fatalf("slots after failed fallback: %+v", state) + } +} + +// Without the current receipt the data schema is unknown, so the launcher +// does not guess. +func TestSelectDoesNotFallBackWithoutCurrentReceipt(t *testing.T) { + clock := &testClock{t: time.Date(2026, 10, 27, 12, 0, 0, 0, time.UTC)} + m := committed(t, clock) + clock.t = clock.t.Add(48 * time.Hour) + if err := os.Remove(filepath.Join(m.Root, "releases", "v0.132.0", receiptFile)); err != nil { + t.Fatal(err) + } + if _, tag, trial, err := m.Select(); err == nil || trial { + t.Fatalf("start %s trial=%v err=%v; want no fallback", tag, trial, err) + } +} + +// A trial that crashed while current was damaged stays marked after the +// fallback starts, so the update check does not offer it again. +func TestSelectFallbackKeepsTheCrashedTrialMarked(t *testing.T) { + clock := &testClock{t: time.Date(2026, 10, 27, 12, 0, 0, 0, time.UTC)} + m := committed(t, clock) + clock.t = clock.t.Add(48 * time.Hour) + release(t, m.Root, "v0.133.0") + if err := m.Prepare("v0.133.0"); err != nil { + t.Fatal(err) + } + if _, tag, trial, err := m.Select(); err != nil || tag != "v0.133.0" || !trial { + t.Fatalf("trial start %s trial=%v %v", tag, trial, err) + } + if err := os.Remove(filepath.Join(m.Root, "releases", "v0.132.0", "web", "index.html")); err != nil { + t.Fatal(err) + } + // The v0.133.0 trial exits before it commits. + _, tag, trial, err := m.Select() + if err != nil || tag != "v0.131.0" || !trial { + t.Fatalf("fallback %s trial=%v %v", tag, trial, err) + } + state, err := m.Read() + if err != nil { + t.Fatal(err) + } + if state.Trial != "v0.131.0" || state.LastFailed != "v0.133.0" { + t.Fatalf("slots after fallback: %+v", state) + } + if err := m.Commit("v0.131.0"); err != nil { + t.Fatal(err) + } + if state, _ = m.Read(); state.Current != "v0.131.0" || state.LastFailed != "v0.133.0" { + t.Fatalf("slots after commit: %+v", state) + } +} + +func TestSelectDoesNotFallBackAcrossSchemas(t *testing.T) { + clock := &testClock{t: time.Date(2026, 10, 27, 12, 0, 0, 0, time.UTC)} + m := committed(t, clock) + clock.t = clock.t.Add(48 * time.Hour) + receipt := `{"tag":"v0.131.0","arch":"` + runtime.GOARCH + `","archive_sha256":"` + string(makeDigest('a')) + `","state_schema":6}` + if err := os.WriteFile(filepath.Join(m.Root, "releases", "v0.131.0", receiptFile), []byte(receipt), 0o600); err != nil { + t.Fatal(err) + } + if err := os.Remove(filepath.Join(m.Root, "releases", "v0.132.0", "web", "index.html")); err != nil { + t.Fatal(err) + } + if _, tag, trial, err := m.Select(); err == nil || trial { + t.Fatalf("start %s trial=%v err=%v; want no fallback", tag, trial, err) + } +} diff --git a/go/internal/notifications/catalogue.go b/go/internal/notifications/catalogue.go index 8a6dd69ea..01bd1f15b 100644 --- a/go/internal/notifications/catalogue.go +++ b/go/internal/notifications/catalogue.go @@ -18,6 +18,7 @@ const ( PushUpdateInstalled = protocol.PushUpdateInstalled PushDriverOffline = protocol.PushDriverOffline PushFuseOverLimit = protocol.PushFuseOverLimit + PushControlNotFollowing = protocol.PushControlNotFollowing PushBoxUnreachable = protocol.PushBoxUnreachable PushCloudGranted = protocol.PushCloudGranted PushAgentAdded = protocol.PushAgentAdded diff --git a/go/internal/notifications/control.go b/go/internal/notifications/control.go new file mode 100644 index 000000000..7f71baf16 --- /dev/null +++ b/go/internal/notifications/control.go @@ -0,0 +1,134 @@ +package notifications + +import ( + "log/slog" + "time" + + "github.com/srcfl/ftw/go/internal/config" +) + +// ControlState is one device function in the box's answer to "Are we in +// control?", as GET /api/status serves it in control_feedback. +type ControlState struct { + Driver string + Kind string + Status string +} + +// ControlReader answers what the box's control feedback says now. main.go +// wires it to the API's own builder, so the rule and the status document +// can never disagree. +type ControlReader = func(now time.Time) []ControlState + +// controlCheckEvery is how often the control rule asks. The answer is +// worked out on demand, and the rule waits minutes before it speaks, so a +// HealthTick every few seconds would only repeat the work. +const controlCheckEvery = 30 * time.Second + +// SetControlReader wires the control feedback the control.not_following rule +// reads. nil disables the rule. +func (s *Service) SetControlReader(r ControlReader) { + if s == nil { + return + } + s.mu.Lock() + s.controlReader = r + s.mu.Unlock() +} + +// evaluateControl fires control.not_following for each device function the +// box has said is not following for the rule's threshold, once until it +// follows again. A box whose stored rules predate the kind takes its +// default, as FTW Cloud's own notices do. +func (s *Service) evaluateControl(now time.Time) { + if s == nil { + return + } + s.mu.Lock() + if s.cfg == nil || !s.cfg.Enabled || s.controlReader == nil || now.Sub(s.controlCheckedAt) < controlCheckEvery { + s.mu.Unlock() + return + } + s.controlCheckedAt = now + rule, ok := findRule(s.cfg.Events, PushControlNotFollowing) + if !ok { + rule, _ = findRule(DefaultRules(), PushControlNotFollowing) + } + if !rule.Enabled { + clear(s.controlFirstAt) + s.mu.Unlock() + return + } + reader := s.controlReader + prio := rule.Priority + if prio == 0 { + prio = s.cfg.DefaultPriority + } + s.mu.Unlock() + + // Read outside the lock: the builder takes the controller's own. + troubled := map[string]ControlState{} + for _, st := range reader(now) { + if st.Status == "not_following" { + troubled[st.Driver+"|"+st.Kind] = st + } + } + + threshold := time.Duration(rule.ThresholdS) * time.Second + if threshold == 0 { + threshold = time.Duration(DefaultThresholdS) * time.Second + } + var pending []ControlState + s.mu.Lock() + for key := range s.controlFirstAt { + if _, still := troubled[key]; !still { + delete(s.controlFirstAt, key) + delete(s.alreadyFired, PushControlNotFollowing+"|"+key) + } + } + for key, st := range troubled { + first, seen := s.controlFirstAt[key] + if !seen { + s.controlFirstAt[key] = now + continue + } + fired := PushControlNotFollowing + "|" + key + if now.Sub(first) < threshold || s.alreadyFired[fired] { + continue + } + if rule.CooldownS > 0 { + if last, ok := s.lastFired[fired]; ok && now.Sub(last) < time.Duration(rule.CooldownS)*time.Second { + continue + } + } + s.alreadyFired[fired] = true + s.lastFired[fired] = now + pending = append(pending, st) + } + s.mu.Unlock() + + for _, st := range pending { + s.deliverControl(rule, prio, st, now) + } +} + +func (s *Service) deliverControl(rule config.NotificationRule, prio int, st ControlState, now time.Time) { + args := map[string]string{"name": st.Driver} + title, body, err := RenderPush(PushControlNotFollowing, args) + if err != nil { + slog.Warn("notifications: catalogue render failed", "event", PushControlNotFollowing, "err", err) + s.bumpFailed() + s.emitDispatched(PushControlNotFollowing, st.Driver, Message{Priority: prio}, "failed", err.Error()) + return + } + s.deliver(PushControlNotFollowing, st.Driver, Message{ + Kind: PushControlNotFollowing, + Title: title, + Body: body, + Priority: prio, + Tags: splitTags(rule.Tags), + Args: args, + Device: st.Driver, + At: now, + }) +} diff --git a/go/internal/notifications/control_test.go b/go/internal/notifications/control_test.go new file mode 100644 index 000000000..01b1b2553 --- /dev/null +++ b/go/internal/notifications/control_test.go @@ -0,0 +1,113 @@ +package notifications + +import ( + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/config" +) + +// xorath, 5 Nov 2026: a battery charged 2.3 kW while FTW asked it to +// discharge 812 W, and only the box's page said so. The rule tells the owner +// once the box has said it for ten minutes, once until the device follows. +func TestControlNotFollowingTellsTheOwnerOnceAfterTenMinutes(t *testing.T) { + pub := &fakePub{} + // Stored rules from before the kind existed: its default holds, and it + // is on. + svc, clk := newSvc(pushCfg(config.NotificationRule{Type: PushDriverOffline, Enabled: false}), pub) + status := "not_following" + reads := 0 + svc.SetControlReader(func(time.Time) []ControlState { + reads++ + return []ControlState{ + {Driver: "pixii", Kind: "battery", Status: status}, + {Driver: "easee", Kind: "ev", Status: "following"}, + } + }) + step := func(d time.Duration) { + clk.advance(d) + svc.evaluateControl(clk.now()) + } + + svc.evaluateControl(clk.now()) + for i := 0; i < 19; i++ { + step(30 * time.Second) + } + if n := len(pub.Messages()); n != 0 { + t.Fatalf("before ten minutes: %d messages", n) + } + step(30 * time.Second) + msgs := pub.Messages() + if len(msgs) != 1 { + t.Fatalf("after ten minutes: %d messages, want 1", len(msgs)) + } + m := msgs[0] + if m.Kind != PushControlNotFollowing || m.Title != "A device is not following FTW" || + m.Body != "pixii does not do what FTW asks. Open FTW to see why." || m.Args["name"] != "pixii" || m.Device != "pixii" { + t.Fatalf("message = %+v", m) + } + + // Still not following: nothing more. + step(30 * time.Minute) + if n := len(pub.Messages()); n != 1 { + t.Fatalf("still not following: %d messages", n) + } + + // It follows, then fails again within the hour: the cooldown holds. + status = "following" + step(30 * time.Second) + status = "not_following" + step(30 * time.Second) + step(11 * time.Minute) + if n := len(pub.Messages()); n != 1 { + t.Fatalf("again within the cooldown: %d messages", n) + } + // Past the hour since the first, and ten minutes into this one. + step(30 * time.Minute) + if n := len(pub.Messages()); n != 2 { + t.Fatalf("again after the cooldown: %d messages, want 2", n) + } + + // A HealthTick every few seconds asks at most every 30 s. + before := reads + for i := 0; i < 10; i++ { + step(2 * time.Second) + } + if got := reads - before; got > 1 { + t.Fatalf("asked %d times in 20 s", got) + } +} + +func TestControlNotFollowingStaysQuietWhenTurnedOff(t *testing.T) { + for name, cfg := range map[string]*config.Notifications{ + "rule off": pushCfg(config.NotificationRule{Type: PushControlNotFollowing, Enabled: false}), + "notifications off": {Enabled: false, Events: []config.NotificationRule{{Type: PushControlNotFollowing, Enabled: true}}}, + } { + pub := &fakePub{} + svc, clk := newSvc(cfg, pub) + svc.SetControlReader(func(time.Time) []ControlState { + return []ControlState{{Driver: "pixii", Kind: "battery", Status: "not_following"}} + }) + for i := 0; i < 60; i++ { + clk.advance(30 * time.Second) + svc.evaluateControl(clk.now()) + } + if n := len(pub.Messages()); n != 0 { + t.Fatalf("%s: %d messages", name, n) + } + } +} + +func TestControlNotFollowingIsARuleThePhonesCanSet(t *testing.T) { + rule, ok := findRule(DefaultRules(), PushControlNotFollowing) + if !ok || !rule.Enabled || rule.ThresholdS != 600 || rule.CooldownS != 3600 { + t.Fatalf("default rule = %+v, %v", rule, ok) + } + known := false + for _, k := range KnownRuleTypes() { + known = known || k == PushControlNotFollowing + } + if !known { + t.Fatal("control.not_following is not a rule type a phone may set") + } +} diff --git a/go/internal/notifications/service.go b/go/internal/notifications/service.go index 73a443ce9..3ed363856 100644 --- a/go/internal/notifications/service.go +++ b/go/internal/notifications/service.go @@ -107,6 +107,11 @@ func DefaultRules() []config.NotificationRule { // the sentences come from the catalogue, never from templates. {Type: PushDriverOffline, Enabled: false, ThresholdS: DefaultThresholdS, Priority: 4, CooldownS: DefaultCooldownS}, {Type: PushFuseOverLimit, Enabled: false, ThresholdS: 30, Priority: 5, CooldownS: 900}, + // A device that does not do what FTW asked, for ten minutes. On by + // default (Fredrik, 5 Nov 2026): a battery that charged while FTW + // asked it to discharge said so on the box's page alone, and the + // owner looking at the phone never heard. + {Type: PushControlNotFollowing, Enabled: true, ThresholdS: DefaultThresholdS, Priority: 4, CooldownS: DefaultCooldownS}, // FTW Cloud on or off. The only rules on by default: each says the // house has started or stopped sending a readable copy of itself to // a Sourceful service, and every owner should hear that without @@ -138,7 +143,7 @@ func KnownRuleTypes() []string { EventFuseOverLimit, EventConcurrentDriversOffline, PushChargingConnected, PushChargingSessionComplete, PushChargingInterrupted, PushChargingGoalAtRisk, PushUpdateInstalled, PushDriverOffline, PushFuseOverLimit, - PushCloudGranted, PushCloudRevoked, PushAgentAdded, PushAgentRequest, + PushControlNotFollowing, PushCloudGranted, PushCloudRevoked, PushAgentAdded, PushAgentRequest, PushAgentExpiring, PushApprovalKeySet, PushOwnerAdded, PushCarWhich, } } @@ -282,9 +287,14 @@ type Service struct { // Used by the fuse_over_limit rule to enforce the threshold_s // sustained-over check, analogous to driver_offline's threshold. fuseFirstOverAt map[string]time.Time - sent uint64 - failed uint64 - now func() time.Time + // controlFirstAt[driver|kind] is when the control rule first saw a + // device function not following; controlCheckedAt when it last asked. + controlReader ControlReader + controlFirstAt map[string]time.Time + controlCheckedAt time.Time + sent uint64 + failed uint64 + now func() time.Time // charging answers what the plug-in rule needs to know about a // loadpoint; connectedSettle is how long it waits before asking. charging func(loadpointID string) (ChargingFacts, bool) @@ -301,6 +311,7 @@ func New(cfg *config.Notifications, pub Publisher, lookup DeviceLookup) *Service alreadyFired: map[string]bool{}, activeAlert: map[string]bool{}, fuseFirstOverAt: map[string]time.Time{}, + controlFirstAt: map[string]time.Time{}, now: time.Now, connectedSettle: ConnectedSettle, } @@ -398,6 +409,9 @@ func (s *Service) Subscribe(bus *events.Bus) { // Fuse rule runs off the same HealthTick cadence so brief // inrush currents don't fire without the threshold sustaining. go s.evaluateFuse(ev.Now) + // The control rule asks the box's own answer to "Are we in + // control?" on the same cadence, at most every 30 s. + go s.evaluateControl(ev.Now) }) bus.Subscribe(events.KindNotificationTest, func(e events.Event) { ev, ok := e.(events.NotificationTest) diff --git a/go/internal/ocpp/boot_limit.go b/go/internal/ocpp/boot_limit.go new file mode 100644 index 000000000..6c16b9752 --- /dev/null +++ b/go/internal/ocpp/boot_limit.go @@ -0,0 +1,144 @@ +package ocpp + +// A charger FTW steers starts at 0 A. +// +// Until a charging profile lands, a charger runs at its own default, which is +// usually its rating. FTW orders a plugged-in car every control tick, but the +// first order still takes a tick plus the charger's own response time. A car +// plugged in during that gap draws the full rating on a fuse FTW has not yet +// shared out. So when a charger a loadpoint names boots and then reports no +// car, FTW sends the same profile it sends to pause: TxDefaultProfile, id 1, +// stack level 0, 0 A for the whole charger. Its first real order has the same +// id, purpose and stack level, so it replaces this one (OCPP 1.6 §3.13.2, +// 2.0.1 K01.FR.05). + +import ( + "context" + "log/slog" + "time" +) + +// defaultBootLimitSettle is how long the connector statuses after a boot must +// stay quiet before the start limit goes out. The limit covers the whole +// charger, so one socket's Available proves nothing while another socket may +// still report a car. A charger sends its statuses one after another, each +// after FTW's reply, so on a local network the burst ends within tens of +// milliseconds. One second leaves room for a slow link and still sends the +// limit seconds before FTW's first order would take hold. +const defaultBootLimitSettle = time.Second + +// SetManagedIDs declares which approved chargers FTW orders: those a loadpoint +// names. An id listed only under ocpp.chargers is approved and metered but +// never ordered, so a 0 A start would hold it at 0 A for good. Replaces the +// previous set; a change applies at the charger's next boot. +func (h *Handler) SetManagedIDs(ids []string) { + m := make(map[string]bool, len(ids)) + for _, id := range ids { + if id != "" { + m[id] = true + } + } + h.mu.Lock() + h.managed = m + h.mu.Unlock() +} + +// maybeSendBootLimit marks a BootNotification for the 0 A start limit. The +// limit goes out once the connector statuses after it settle, and only when +// none of them showed a car (resolveBootLimit). +// +// A boot alone does not show a fresh start. After a Core restart the new +// process holds no status and no transaction, and much firmware sends its own +// BootNotification on reconnect, before FTW's identity probe, while the car +// goes on charging. A 1.6 charger reports every connector's status after an +// accepted boot (§4.9), and 2.0.1 stations send StatusNotification there too, +// so FTW waits for it. A charger that never reports one gets no start limit, +// as before this file existed. +// +// It skips a boot that answers FTW's own identity probe: that charger did not +// restart, and may be charging under FTW's last order. It also skips a boot +// when the last status FTW holds shows a car or an open transaction, on this +// socket or the one before: FTW orders that car every tick. +func (h *Handler) maybeSendBootLimit(id string) { + h.mu.Lock() + defer h.mu.Unlock() + s := h.chargersLocked(id) + s.bootLimitEpoch++ + s.bootLimitPending = h.bootLimit != nil && h.approved[id] && h.managed[id] && + s.identityProbeAttempts == 0 && !s.connected && s.transactionID < 0 +} + +// resolveBootLimit runs after each connector status. A status that shows a car +// or an open transaction on any socket cancels the start limit: FTW orders +// that car every tick, and 0 A for the whole charger would pause it. A status +// with no car starts the settle timer again, so the limit goes out only after +// the statuses have been quiet for bootLimitSettle. FTW keeps one connection +// state per charger and the last status wins, so the cancel happens here, as +// each status arrives, not when the timer fires. +func (h *Handler) resolveBootLimit(id string) { + h.mu.Lock() + defer h.mu.Unlock() + s := h.chargersLocked(id) + if !s.bootLimitPending { + return + } + if s.connected || s.transactionID >= 0 { + s.bootLimitPending = false + return + } + s.bootLimitEpoch++ + epoch := s.bootLimitEpoch + time.AfterFunc(h.bootLimitSettle, func() { h.fireBootLimit(id, epoch) }) +} + +// meterCancelsBootLimitLocked drops a pending start limit when a 1.6 meter +// value shows a car: it names a transaction, or it measures power above zero. +// After a Core restart a two-socket charger may send the running socket's +// meter values before that socket's status, and 0 A for the whole charger +// would pause its car. The caller holds h.mu. +func (s *chargerState) meterCancelsBootLimitLocked(txID *int, w float64) { + if txID != nil || w > 0 { + s.bootLimitPending = false + } +} + +// cancelBootLimit drops a pending start limit when FTW orders the charger +// during the settle wait. That order is newer, and a 0 A landing after it +// would undo it. +func (h *Handler) cancelBootLimit(id string) { + h.mu.Lock() + defer h.mu.Unlock() + if s := h.chargers[id]; s != nil { + s.bootLimitPending = false + } +} + +// fireBootLimit sends the start limit when no status, boot or socket change +// came after the timer that calls it, and still no car or transaction shows. +// The send goes through the same one-at-a-time slot as every other order. A +// refusal is logged once; the next boot tries again. +func (h *Handler) fireBootLimit(id string, epoch uint64) { + h.mu.Lock() + s := h.chargersLocked(id) + if s.bootLimitEpoch != epoch { + h.mu.Unlock() + return + } + send := h.bootLimit + if !s.bootLimitPending || !h.approved[id] || !h.managed[id] || s.connected || s.transactionID >= 0 { + send = nil + } + s.bootLimitPending = false + h.mu.Unlock() + if send != nil { + send(id) + } +} + +// sendBootLimit is the Server side of maybeSendBootLimit. +func (s *Server) sendBootLimit(id string) { + if err := s.setLimit(context.Background(), id, 0, nil); err != nil { + slog.Warn("ocpp: charger did not take the 0 A start limit; it runs at its own default until FTW's first order", + "charger", id, "err", err) + } +} diff --git a/go/internal/ocpp/boot_limit_test.go b/go/internal/ocpp/boot_limit_test.go new file mode 100644 index 000000000..869d1f43e --- /dev/null +++ b/go/internal/ocpp/boot_limit_test.go @@ -0,0 +1,492 @@ +package ocpp + +import ( + "context" + "testing" + "time" + + ocpp16 "github.com/lorenzodonini/ocpp-go/ocpp1.6" + "github.com/lorenzodonini/ocpp-go/ocpp1.6/core" + "github.com/lorenzodonini/ocpp-go/ocpp1.6/smartcharging" + "github.com/lorenzodonini/ocpp-go/ocpp1.6/types" + ocpp201 "github.com/lorenzodonini/ocpp-go/ocpp2.0.1" + "github.com/lorenzodonini/ocpp-go/ocpp2.0.1/availability" + "github.com/lorenzodonini/ocpp-go/ocpp2.0.1/provisioning" + types201 "github.com/lorenzodonini/ocpp-go/ocpp2.0.1/types" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// startManagedServer runs a 1.6 listener (and 2.0.1 when v201) where +// "garage" is named by a loadpoint and "metered" is listed only under +// ocpp.chargers. The status burst after a boot settles in testBootLimitSettle, +// well inside settle(), so a start limit that should not come has time to. +func startManagedServer(t *testing.T, v201 bool) *Server { + t.Helper() + srv := startOCPP(t, Config{ + Enabled: true, Bind: "127.0.0.1", HeartbeatIntervalS: 60, + ApprovedIDs: []string{"garage", "metered"}, + ManagedIDs: []string{"garage"}, + }, telemetry.NewStore(), v201) + srv.handler.mu.Lock() + srv.handler.bootLimitSettle = testBootLimitSettle + srv.handler.mu.Unlock() + return srv +} + +const testBootLimitSettle = 50 * time.Millisecond + +func awaitProfiles(t *testing.T, count func() int, want int) { + t.Helper() + deadline := time.Now().Add(3 * time.Second) + for count() < want { + if time.Now().After(deadline) { + t.Fatalf("got %d charging profiles, want %d", count(), want) + } + time.Sleep(5 * time.Millisecond) + } +} + +// settle gives a profile that should not come time to arrive. +func settle() { time.Sleep(300 * time.Millisecond) } + +// reportStatus16 sends what a 1.6 charger sends after an accepted boot: the +// status of connector 0 and of its connector (OCPP 1.6 §4.9). +func reportStatus16(t *testing.T, cp ocpp16.ChargePoint, status core.ChargePointStatus) { + t.Helper() + for connector, st := range []core.ChargePointStatus{core.ChargePointStatusAvailable, status} { + if _, err := cp.StatusNotification(connector, core.NoError, st); err != nil { + t.Fatal(err) + } + } +} + +// A managed charger that boots and reports no car gets a 0 A +// TxDefaultProfile, and FTW's first order uses the same id, purpose and stack +// level, so it replaces that profile instead of stacking under it. +func TestManagedChargerStartsAtZeroAmpsOnBoot(t *testing.T) { + srv := startManagedServer(t, false) + cp, fake, _ := connectCharger(t, srv, srv.cfg.Port, "garage") + settle() + if n := fake.count(); n != 0 { + t.Fatalf("boot alone sent %d profiles; want none before the charger reports its connector", n) + } + reportStatus16(t, cp, core.ChargePointStatusAvailable) + awaitProfiles(t, fake.count, 1) + + fake.mu.Lock() + boot := fake.profiles[0] + fake.mu.Unlock() + p := boot.ChargingProfile + if boot.ConnectorId != allConnectors || p.ChargingProfilePurpose != types.ChargingProfilePurposeTxDefaultProfile || + p.ChargingProfileId != ftwProfileID || p.StackLevel != ftwStackLevel { + t.Fatalf("boot profile: connector %d purpose %s id %d stack %d, want 0 TxDefaultProfile %d %d", + boot.ConnectorId, p.ChargingProfilePurpose, p.ChargingProfileId, p.StackLevel, ftwProfileID, ftwStackLevel) + } + if got := fake.lastLimit(t); got != 0 { + t.Fatalf("boot limit %v A, want 0 A", got) + } + + if err := srv.Command(context.Background(), "garage", setCurrent11kW(t)); err != nil { + t.Fatalf("order: %v", err) + } + fake.mu.Lock() + order := fake.profiles[len(fake.profiles)-1] + fake.mu.Unlock() + o := order.ChargingProfile + if order.ConnectorId != boot.ConnectorId || o.ChargingProfilePurpose != p.ChargingProfilePurpose || + o.ChargingProfileId != p.ChargingProfileId || o.StackLevel != p.StackLevel { + t.Fatalf("FTW's order does not replace the boot profile: connector %d purpose %s id %d stack %d", + order.ConnectorId, o.ChargingProfilePurpose, o.ChargingProfileId, o.StackLevel) + } + if got := fake.lastLimit(t); got != 16 { + t.Fatalf("order limit %v A, want 16 A", got) + } +} + +// FTW never orders a charger no loadpoint names, so a 0 A start would hold +// it at 0 A for good. A pending charger is not FTW's to steer at all. +func TestUnmanagedChargersGetNoBootLimit(t *testing.T) { + srv := startManagedServer(t, false) + cpManaged, managed, _ := connectCharger(t, srv, srv.cfg.Port, "garage") + cpMetered, metered, _ := connectCharger(t, srv, srv.cfg.Port, "metered") + cpPending, pending, _ := connectCharger(t, srv, srv.cfg.Port, "stranger") + for _, cp := range []ocpp16.ChargePoint{cpManaged, cpMetered, cpPending} { + reportStatus16(t, cp, core.ChargePointStatusAvailable) + } + awaitProfiles(t, managed.count, 1) + settle() + if n := metered.count(); n != 0 { + t.Errorf("charger listed only under ocpp.chargers got %d profiles, want 0", n) + } + if n := pending.count(); n != 0 { + t.Errorf("pending charger got %d profiles, want 0", n) + } +} + +// A charger that refuses the start limit gets the connector-1 retry every +// order gets, and then nothing more until it boots again. +func TestRefusedBootLimitIsNotRetried(t *testing.T) { + srv := startManagedServer(t, false) + fake := newFakeCharger() + fake.setStatus(smartcharging.ChargingProfileStatusRejected) + cp, _, err := dialChargePoint16(t, wsURL(srv.cfg.Port), "garage", nil, func(cp ocpp16.ChargePoint) { cp.SetSmartChargingHandler(fake) }) + if err != nil { + t.Fatal(err) + } + if _, err := cp.BootNotification("Dawn", "Charge Amps"); err != nil { + t.Fatal(err) + } + reportStatus16(t, cp, core.ChargePointStatusAvailable) + awaitProfiles(t, fake.count, 2) + settle() + if n := fake.count(); n != 2 { + t.Fatalf("refused start limit sent %d times, want 2 (connector 0, then 1)", n) + } +} + +// A boot on a socket that already reported a car is not a fresh start: FTW +// orders that car every tick, and 0 A would only pause it. +func TestBootWithCarAlreadyReportedGetsNoBootLimit(t *testing.T) { + srv := startManagedServer(t, false) + fake := newFakeCharger() + cp, _, err := dialChargePoint16(t, wsURL(srv.cfg.Port), "garage", nil, func(cp ocpp16.ChargePoint) { cp.SetSmartChargingHandler(fake) }) + if err != nil { + t.Fatal(err) + } + if _, err := cp.StatusNotification(1, core.NoError, core.ChargePointStatusCharging); err != nil { + t.Fatal(err) + } + if _, err := cp.BootNotification("Dawn", "Charge Amps"); err != nil { + t.Fatal(err) + } + settle() + if n := fake.count(); n != 0 { + t.Fatalf("boot with a car on the socket got %d profiles, want 0", n) + } +} + +// reconnectAndBoot drops the charger's socket, waits for FTW to see it go, +// then dials again and sends a BootNotification before any status, as much +// firmware does after a Wi-Fi drop. It returns the new socket's charger. +func reconnectAndBoot(t *testing.T, srv *Server, stop func()) (ocpp16.ChargePoint, *fakeCharger) { + t.Helper() + stop() + deadline := time.Now().Add(2 * time.Second) + for srv.Handler().IsOnline("garage") { + if time.Now().After(deadline) { + t.Fatal("FTW never saw the socket drop") + } + time.Sleep(10 * time.Millisecond) + } + fake := newFakeCharger() + cp, _, err := dialChargePoint16(t, wsURL(srv.cfg.Port), "garage", nil, func(cp ocpp16.ChargePoint) { cp.SetSmartChargingHandler(fake) }) + if err != nil { + t.Fatal(err) + } + if _, err := cp.BootNotification("Dawn", "Charge Amps"); err != nil { + t.Fatal(err) + } + return cp, fake +} + +// A Wi-Fi drop mid-session: the charger had reported Charging and taken +// FTW's 16 A. It reconnects and boots before any status. The car is most +// likely still charging, so FTW keeps its live order rather than pausing it. +func TestReconnectBootMidSessionKeepsFTWOrder(t *testing.T) { + srv := startManagedServer(t, false) + cp, fake, stop := connectCharger(t, srv, srv.cfg.Port, "garage") + reportStatus16(t, cp, core.ChargePointStatusAvailable) + awaitProfiles(t, fake.count, 1) // the real boot's 0 A + if _, err := cp.StatusNotification(1, core.NoError, core.ChargePointStatusCharging); err != nil { + t.Fatal(err) + } + if err := srv.Command(context.Background(), "garage", setCurrent11kW(t)); err != nil { + t.Fatal(err) + } + + cpAfter, after := reconnectAndBoot(t, srv, stop) + reportStatus16(t, cpAfter, core.ChargePointStatusCharging) + settle() + if n := after.count(); n != 0 { + t.Fatalf("reconnect boot mid-session got %d profile(s), last %v A; want FTW's live 16 A kept", n, after.lastLimit(t)) + } +} + +// The same reconnect after the car left: no car to pause, so the charger +// gets its start limit again. +func TestReconnectBootWithNoCarGetsBootLimit(t *testing.T) { + srv := startManagedServer(t, false) + cp, fake, stop := connectCharger(t, srv, srv.cfg.Port, "garage") + reportStatus16(t, cp, core.ChargePointStatusAvailable) + awaitProfiles(t, fake.count, 1) + + cpAfter, after := reconnectAndBoot(t, srv, stop) + reportStatus16(t, cpAfter, core.ChargePointStatusAvailable) + awaitProfiles(t, after.count, 1) + if got := after.lastLimit(t); got != 0 { + t.Fatalf("reconnect boot with no car: limit %v A, want 0 A", got) + } +} + +// After a Core restart the charger reconnects without booting, and FTW asks +// it for a BootNotification. That boot is FTW's own request, not a charger +// restart; the car may be charging on FTW's last order. +func TestBootAnsweringIdentityProbeGetsNoBootLimit(t *testing.T) { + srv := startManagedServer(t, false) + fake := newFakeCharger() + requests := make(chan string, 10) + cp, _, err := dialChargePoint16(t, wsURL(srv.cfg.Port), "garage", nil, func(cp ocpp16.ChargePoint) { + cp.SetSmartChargingHandler(fake) + cp.SetRemoteTriggerHandler(bootTrigger16{requested: requests}) + }) + if err != nil { + t.Fatal(err) + } + select { + case <-requests: + case <-time.After(3 * time.Second): + t.Fatal("FTW never asked for a BootNotification") + } + if _, err := cp.BootNotification("Dawn", "Charge Amps"); err != nil { + t.Fatal(err) + } + reportStatus16(t, cp, core.ChargePointStatusAvailable) + settle() + if n := fake.count(); n != 0 { + t.Fatalf("boot answering FTW's probe got %d profiles, want 0", n) + } +} + +// 2.0.1: the same 0 A TxDefaultProfile on EVSE 0, replaced by FTW's order. +func TestV201ManagedStationStartsAtZeroAmpsOnBoot(t *testing.T) { + srv := startManagedServer(t, true) + cs, fake := bootStationV201(t, srv) + count := func() int { fake.mu.Lock(); defer fake.mu.Unlock(); return len(fake.profiles) } + if _, err := cs.StatusNotification(types201.NewDateTime(time.Now()), availability.ConnectorStatusAvailable, 1, 1); err != nil { + t.Fatal(err) + } + awaitProfiles(t, count, 1) + + fake.mu.Lock() + boot := fake.profiles[0] + fake.mu.Unlock() + p := boot.ChargingProfile + if boot.EvseID != allEVSEs || p.ChargingProfilePurpose != types201.ChargingProfilePurposeTxDefaultProfile || + p.ID != ftwProfileID || p.StackLevel != ftwStackLevel { + t.Fatalf("boot profile: evse %d purpose %s id %d stack %d, want 0 TxDefaultProfile %d %d", + boot.EvseID, p.ChargingProfilePurpose, p.ID, p.StackLevel, ftwProfileID, ftwStackLevel) + } + if got := fake.lastLimit(t); got != 0 { + t.Fatalf("boot limit %v A, want 0 A", got) + } + if err := srv.Command(context.Background(), "garage", setCurrent11kW(t)); err != nil { + t.Fatalf("order: %v", err) + } + if got := fake.lastLimit(t); got != 16 { + t.Fatalf("order limit %v A, want 16 A", got) + } +} + +// Rig s7s-restart-short-25A on 964643e9: Core restarts during Charge now. The +// charger reconnects to the new process and sends its own BootNotification at +// once, before FTW's identity probe, then reports connector 0 Available and +// connector 1 Charging. The new process holds no status and no transaction, so +// the boot alone looks like a fresh start. FTW sent 0 A, the car went to +// SuspendedEVSE for a second, and "Charge now never stops" failed. The status +// after the boot shows the car, so no start limit goes out. +func TestCoreRestartBootMidSessionGetsNoBootLimit(t *testing.T) { + srv := startManagedServer(t, false) + fake := newFakeCharger() + cp, _, err := dialChargePoint16(t, wsURL(srv.cfg.Port), "garage", nil, func(cp ocpp16.ChargePoint) { cp.SetSmartChargingHandler(fake) }) + if err != nil { + t.Fatal(err) + } + if _, err := cp.BootNotification("Rig11", "RigCo"); err != nil { + t.Fatal(err) + } + reportStatus16(t, cp, core.ChargePointStatusCharging) + settle() + if n := fake.count(); n != 0 { + t.Fatalf("boot after a Core restart with the car charging got %d profile(s), last %v A; want none", + n, fake.lastLimit(t)) + } +} + +// bootStationV201 dials "garage" on 2.0.1 and sends a BootNotification. +func bootStationV201(t *testing.T, srv *Server) (ocpp201.ChargingStation, *fakeStationV201) { + t.Helper() + fake := newFakeStationV201() + cs, _, err := dialChargingStation201(t, wsURL(srv.cfg.PortV201), "garage", func(cs ocpp201.ChargingStation) { cs.SetSmartChargingHandler(fake) }) + if err != nil { + t.Fatal(err) + } + if _, err := cs.BootNotification(provisioning.BootReasonPowerUp, "Dawn", "Charge Amps"); err != nil { + t.Fatal(err) + } + return cs, fake +} + +// The s7s shape on 2.0.1: after a Core restart the station boots on its own +// and reports its connector Occupied. No start limit. +func TestV201CoreRestartBootWithCarGetsNoBootLimit(t *testing.T) { + srv := startManagedServer(t, true) + cs, fake := bootStationV201(t, srv) + if _, err := cs.StatusNotification(types201.NewDateTime(time.Now()), availability.ConnectorStatusOccupied, 1, 1); err != nil { + t.Fatal(err) + } + settle() + fake.mu.Lock() + n := len(fake.profiles) + fake.mu.Unlock() + if n != 0 { + t.Fatalf("boot after a Core restart with a car got %d profile(s); want none", n) + } +} + +// A Core restart while a car charges on socket 2 of a two-socket 1.6 +// charger. The charger boots on its own and reports socket 1 Available +// before socket 2 Charging. The start limit goes to connector 0, the whole +// charger, so sending it on socket 1's status would pause the car on socket +// 2. FTW waits for the status burst to settle; the car on socket 2 cancels it. +func TestDualSocketCoreRestartCarOnSocket2GetsNoBootLimit(t *testing.T) { + srv := startManagedServer(t, false) + fake := newFakeCharger() + cp, _, err := dialChargePoint16(t, wsURL(srv.cfg.Port), "garage", nil, func(cp ocpp16.ChargePoint) { cp.SetSmartChargingHandler(fake) }) + if err != nil { + t.Fatal(err) + } + if _, err := cp.BootNotification("Twin", "RigCo"); err != nil { + t.Fatal(err) + } + for connector, st := range []core.ChargePointStatus{core.ChargePointStatusAvailable, core.ChargePointStatusAvailable, core.ChargePointStatusCharging} { + if _, err := cp.StatusNotification(connector, core.NoError, st); err != nil { + t.Fatal(err) + } + } + settle() + if n := fake.count(); n != 0 { + t.Fatalf("two-socket restart with a car on socket 2 got %d profile(s), last %v A; want none", n, fake.lastLimit(t)) + } +} + +// The same charger reporting socket 2 first. FTW keeps one connection state +// per charger and the last status wins, so socket 1's later Available must +// not undo the car socket 2 showed during the burst. +func TestDualSocketCarReportedFirstStillCancelsBootLimit(t *testing.T) { + srv := startManagedServer(t, false) + fake := newFakeCharger() + cp, _, err := dialChargePoint16(t, wsURL(srv.cfg.Port), "garage", nil, func(cp ocpp16.ChargePoint) { cp.SetSmartChargingHandler(fake) }) + if err != nil { + t.Fatal(err) + } + if _, err := cp.BootNotification("Twin", "RigCo"); err != nil { + t.Fatal(err) + } + for _, c := range []struct { + connector int + status core.ChargePointStatus + }{{0, core.ChargePointStatusAvailable}, {2, core.ChargePointStatusCharging}, {1, core.ChargePointStatusAvailable}} { + if _, err := cp.StatusNotification(c.connector, core.NoError, c.status); err != nil { + t.Fatal(err) + } + } + settle() + if n := fake.count(); n != 0 { + t.Fatalf("two-socket restart with a car on socket 2 reported first got %d profile(s); want none", n) + } +} + +// FTW orders the charger while the start limit waits for the statuses to +// settle. The order is newer, so the 0 A must not land after it. +func TestOrderDuringSettleWaitCancelsBootLimit(t *testing.T) { + srv := startManagedServer(t, false) + cp, fake, _ := connectCharger(t, srv, srv.cfg.Port, "garage") + reportStatus16(t, cp, core.ChargePointStatusAvailable) + if err := srv.Command(context.Background(), "garage", setCurrent11kW(t)); err != nil { + t.Fatal(err) + } + settle() + if n, got := fake.count(), fake.lastLimit(t); n != 1 || got != 16 { + t.Fatalf("order during the settle wait: %d profile(s), last %v A; want FTW's 16 A alone", n, got) + } +} + +// 2.0.1 with two EVSEs: EVSE 1 Available, then EVSE 2 Occupied. The start +// limit goes to EVSE 0, the whole station, so it must not go out. +func TestV201TwoEVSECoreRestartCarOnEVSE2GetsNoBootLimit(t *testing.T) { + srv := startManagedServer(t, true) + cs, fake := bootStationV201(t, srv) + if _, err := cs.StatusNotification(types201.NewDateTime(time.Now()), availability.ConnectorStatusAvailable, 1, 1); err != nil { + t.Fatal(err) + } + if _, err := cs.StatusNotification(types201.NewDateTime(time.Now()), availability.ConnectorStatusOccupied, 2, 1); err != nil { + t.Fatal(err) + } + settle() + fake.mu.Lock() + n := len(fake.profiles) + fake.mu.Unlock() + if n != 0 { + t.Fatalf("two-EVSE restart with a car on EVSE 2 got %d profile(s); want none", n) + } +} + +// A Core restart while a car charges on socket 2 of a two-socket 1.6 +// charger, where socket 2's status comes only after the wait. Socket 1 +// reports Available, then meter values for socket 2's car arrive inside the +// wait. They show the car, so the start limit must not go out to the whole +// charger. The meter values may name the transaction, or carry only power. +func TestMeterValuesShowingACarDuringWaitCancelBootLimit(t *testing.T) { + tx := 7 + for _, c := range []struct { + name string + txID *int + power string + }{ + {"transaction and 7 kW", &tx, "7000"}, + {"transaction at 0 W", &tx, "0"}, + {"7 kW without transaction", nil, "7000"}, + } { + t.Run(c.name, func(t *testing.T) { + srv := startManagedServer(t, false) + srv.handler.mu.Lock() + srv.handler.bootLimitSettle = 200 * time.Millisecond + srv.handler.mu.Unlock() + cp, fake, _ := connectCharger(t, srv, srv.cfg.Port, "garage") + reportStatus16(t, cp, core.ChargePointStatusAvailable) + if _, err := cp.MeterValues(2, meterValues16(c.power), func(r *core.MeterValuesRequest) { r.TransactionId = c.txID }); err != nil { + t.Fatal(err) + } + time.Sleep(400 * time.Millisecond) + if n := fake.count(); n != 0 { + t.Fatalf("car on socket 2 shown by meter values got %d profile(s), last %v A; want none", n, fake.lastLimit(t)) + } + }) + } +} + +// Meter values with no transaction and 0 W show no car: the start limit +// still goes out. +func TestIdleMeterValuesDuringWaitKeepBootLimit(t *testing.T) { + srv := startManagedServer(t, false) + cp, fake, _ := connectCharger(t, srv, srv.cfg.Port, "garage") + reportStatus16(t, cp, core.ChargePointStatusAvailable) + if _, err := cp.MeterValues(1, meterValues16("0")); err != nil { + t.Fatal(err) + } + awaitProfiles(t, fake.count, 1) + if got := fake.lastLimit(t); got != 0 { + t.Fatalf("start limit after idle meter values = %v A; want 0", got) + } +} + +func meterValues16(powerW string) []types.MeterValue { + return []types.MeterValue{{ + Timestamp: types.NewDateTime(time.Now()), + SampledValue: []types.SampledValue{ + {Value: powerW, Measurand: types.MeasurandPowerActiveImport, Unit: types.UnitOfMeasureW}, + {Value: "12345", Measurand: types.MeasurandEnergyActiveImportRegister, Unit: types.UnitOfMeasureWh}, + }, + }} +} diff --git a/go/internal/ocpp/charging_needs.go b/go/internal/ocpp/charging_needs.go index 3e0f2f69c..27f27da2d 100644 --- a/go/internal/ocpp/charging_needs.go +++ b/go/internal/ocpp/charging_needs.go @@ -148,7 +148,7 @@ func (h *Handler) noteChargingNeeds(id string, n ChargingNeeds) { approved := h.approved[id] h.mu.Unlock() if approved && fn != nil { - fn(id, n) + h.callback(id, func() { fn(id, n) }) } } diff --git a/go/internal/ocpp/config.go b/go/internal/ocpp/config.go index c8f581ab0..242c2e1f9 100644 --- a/go/internal/ocpp/config.go +++ b/go/internal/ocpp/config.go @@ -55,6 +55,12 @@ type Config struct { // UI so an operator can adopt it, withheld from telemetry so it cannot // influence dispatch. Empty means every charger is pending. ApprovedIDs []string `yaml:"-"` + + // ManagedIDs is the part of ApprovedIDs a loadpoint names: the chargers + // FTW orders. Each starts at 0 A on boot (see boot_limit.go). Derived by + // the caller, never read from YAML, and empty while Core blocks every + // charger write (setup measurement mode). + ManagedIDs []string `yaml:"-"` } // TLSConfig points at the certificate this server presents, and optionally at diff --git a/go/internal/ocpp/control.go b/go/internal/ocpp/control.go index 2d59bab1c..0f853e52e 100644 --- a/go/internal/ocpp/control.go +++ b/go/internal/ocpp/control.go @@ -22,6 +22,7 @@ import ( "errors" "fmt" "log/slog" + "sync" "time" "github.com/lorenzodonini/ocpp-go/ocpp1.6/smartcharging" @@ -140,6 +141,11 @@ func (s *Server) Command(ctx context.Context, id string, payload []byte) (result return fmt.Errorf("ocpp: bad command payload for %s: %w", id, err) } + switch c.Action { + case "ev_set_current", "ev_pause", "ev_start", "ev_resume": + s.handler.cancelBootLimit(id) + } + switch c.Action { case "init", "deinit": // Lifecycle hooks that only mean something to a Lua VM. @@ -281,34 +287,92 @@ func (s *Server) setLimit(ctx context.Context, id string, amps float64, numberPh // refusal; the error is reserved for the cases where no verdict exists — // transport failure, cancellation, silence. func (s *Server) attemptLimit(ctx context.Context, id, alias string, amps float64, numberPhases *int, connectorID int) (profileResult, error) { - // Buffered: the library's callback must never block if we have already - // stopped waiting. - done := make(chan profileResult, 1) - - // The two versions describe the same intent with different types, so the - // request is built per dialect and the outcome normalised back. - var err error - switch version, _ := s.handler.Version(id); version { - case Version201: - err = s.sendProfileV201(alias, amps, numberPhases, connectorID, done) - default: - err = s.sendProfileV16(alias, amps, numberPhases, connectorID, done) - } - if err != nil { - return profileResult{}, fmt.Errorf("ocpp: send charging profile to %s: %w", id, err) - } - timeout := time.NewTimer(commandTimeout) defer timeout.Stop() + // One profile per charger at a time, from submission until the library + // answers it. The library keeps every queued request and sends them in + // order, so without this a stalled request pump collects one obsolete + // limit per control tick and replays them all once it clears, each a + // round trip ahead of the current one: a fuse cut would wait behind them. + // A command that cannot get the slot before its deadline is unconfirmed, + // as one whose answer did not arrive in time. + slot := s.profileSlot(id) select { - case r := <-done: - return r, nil + case slot <- struct{}{}: case <-ctx.Done(): - return profileResult{}, fmt.Errorf("ocpp: charging profile for %s cancelled: %w", id, ctx.Err()) + return profileResult{}, fmt.Errorf("ocpp: earlier charging profile for %s still unanswered: %w", id, ctx.Err()) case <-timeout.C: - return profileResult{}, fmt.Errorf("ocpp: %s did not confirm charging profile within %s", id, commandTimeout) + return profileResult{}, fmt.Errorf("ocpp: earlier charging profile for %s still unanswered after %s", id, commandTimeout) + } + var once sync.Once + release := func() { once.Do(func() { <-slot }) } + if err := ctx.Err(); err != nil { + release() + return profileResult{}, fmt.Errorf("ocpp: charging profile for %s cancelled: %w", id, err) + } + + // Buffered: the library's callback must never block if we have already + // stopped waiting. The library calls back once for every request it + // queued, on the answer, its own timeout or a disconnect. + done := make(chan profileResult, 1) + answer := func(r profileResult) { + release() + done <- r + } + + // The two versions describe the same intent with different types, so the + // request is built per dialect and the outcome normalised back. The + // library's send blocks while its request pump is stalled, so it runs on + // its own goroutine and the wait below still ends on ctx. The slot keeps + // this to one goroutine per charger. + version, _ := s.handler.Version(id) + sent := make(chan error, 1) + go func() { + var err error + if version == Version201 { + err = s.sendProfileV201(alias, amps, numberPhases, connectorID, answer) + } else { + err = s.sendProfileV16(alias, amps, numberPhases, connectorID, answer) + } + if err != nil { + // Not queued, so no answer will come. + release() + } + sent <- err + }() + + for { + select { + case err := <-sent: + if err != nil { + return profileResult{}, fmt.Errorf("ocpp: send charging profile to %s: %w", id, err) + } + sent = nil + case r := <-done: + return r, nil + case <-ctx.Done(): + return profileResult{}, fmt.Errorf("ocpp: charging profile for %s cancelled: %w", id, ctx.Err()) + case <-timeout.C: + return profileResult{}, fmt.Errorf("ocpp: %s did not confirm charging profile within %s", id, commandTimeout) + } + } +} + +// profileSlot returns the one-place channel that admits a single charging +// profile to charger id at a time. +func (s *Server) profileSlot(id string) chan struct{} { + s.profileMu.Lock() + defer s.profileMu.Unlock() + if s.profileSlots == nil { + s.profileSlots = map[string]chan struct{}{} } + slot, ok := s.profileSlots[id] + if !ok { + slot = make(chan struct{}, 1) + s.profileSlots[id] = slot + } + return slot } // profileResult is a version-neutral answer to a charging profile request, so @@ -321,7 +385,7 @@ type profileResult struct { } // sendProfileV16 issues the limit as an OCPP 1.6 TxDefaultProfile. -func (s *Server) sendProfileV16(id string, amps float64, numberPhases *int, connectorID int, done chan<- profileResult) error { +func (s *Server) sendProfileV16(id string, amps float64, numberPhases *int, connectorID int, answer func(profileResult)) error { period := types.NewChargingSchedulePeriod(scheduleStartS, amps) // Declared only when the loadpoint pinned single-phase charging. Left // unset otherwise so a charger that can switch phases keeps deciding. @@ -342,7 +406,7 @@ func (s *Server) sendProfileV16(id string, amps float64, numberPhases *int, conn r.status = string(conf.Status) r.accepted = conf.Status == smartcharging.ChargingProfileStatusAccepted } - done <- r + answer(r) }, connectorID, profile) } @@ -351,7 +415,7 @@ func (s *Server) sendProfileV16(id string, amps float64, numberPhases *int, conn // 2.0.1 carries a list of schedules rather than one, and each schedule needs // its own id; a single-entry list with a stable id keeps the meaning identical // to the 1.6 request. -func (s *Server) sendProfileV201(id string, amps float64, numberPhases *int, evseID int, done chan<- profileResult) error { +func (s *Server) sendProfileV201(id string, amps float64, numberPhases *int, evseID int, answer func(profileResult)) error { if s.csms == nil { return fmt.Errorf("ocpp: %s speaks %s but no %s listener is configured", id, Version201, Version201) } @@ -374,7 +438,7 @@ func (s *Server) sendProfileV201(id string, amps float64, numberPhases *int, evs r.status = string(conf.Status) r.accepted = conf.Status == smartcharging201.ChargingProfileStatusAccepted } - done <- r + answer(r) }, evseID, profile) } diff --git a/go/internal/ocpp/handlers.go b/go/internal/ocpp/handlers.go index 64b9ccd76..6eb327d0e 100644 --- a/go/internal/ocpp/handlers.go +++ b/go/internal/ocpp/handlers.go @@ -30,6 +30,15 @@ type Handler struct { // no driver health, no metrics — so it cannot influence dispatch. See // SetApprovedIDs. approved map[string]bool + + // managed is the approved chargers a loadpoint names, which FTW orders. + // bootLimit, set by the Server, holds each at 0 A from its boot until + // FTW's first order, once the connector statuses after the boot have + // been quiet for bootLimitSettle. See boot_limit.go. + managed map[string]bool + bootLimit func(id string) + bootLimitSettle time.Duration + nextTxID int // vehicleIdentified, when set, fires once per new vehicle identity seen @@ -48,6 +57,9 @@ type Handler struct { // it is (BootNotification) — main.go writes its row in the device // registry from it. See identity.go. identityReported func(ChargerIdentity) + // later, set by the Server, runs the three callbacks above after the + // charger's socket lock is released. See boundCall. + later func(id string, fn func()) // capabilityProbe, set by the Server, asks a charger which feature // profiles it supports (see capabilities.go). Fired from OnConnect and @@ -98,6 +110,9 @@ type chargerState struct { // transactionRef is the 2.0.1 transaction id, which is a string rather // than the int transactionID above. Empty on the 1.6 path. transactionRef string + // stoppedTxID is the last transaction this charger stopped. A late meter + // value for it must not take it up again. + stoppedTxID int // vendor and model come from BootNotification and exist so the UI can // label a charger with what it actually is rather than its URL segment. // serial is the hardware-stable half of its identity — what the device @@ -115,6 +130,11 @@ type chargerState struct { identityProbeEpoch uint64 identityProbeAttempts int identityProbeAfter time.Time + // bootLimitPending marks a boot that may get the 0 A start limit once + // the status burst after it settles with no car. bootLimitEpoch names + // the latest settle timer. See boot_limit.go. + bootLimitPending bool + bootLimitEpoch uint64 // vehicleID is the identity presented when the current/last transaction // started: the RFID idTag on 1.6, or a 2.0.1 idToken — where the token // type MacAddress (autocharge) or eMAID (ISO 15118) names the actual @@ -146,10 +166,22 @@ func NewHandler(tel *telemetry.Store, heartbeatIntervalS int) *Handler { approved: map[string]bool{}, probing: map[string]bool{}, identityProbeTiming: defaultIdentityProbeTiming, - nextTxID: 1, + nextTxID: firstTxID(time.Now()), } } +// txIDEpoch keeps clock-based transaction ids far below the 32-bit limit +// many chargers store them in. +var txIDEpoch = time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC) + +// firstTxID starts a run's transaction ids at the seconds since txIDEpoch. +// A charger keeps its transaction id while Core restarts, so a restart must +// not hand that id to another transaction. The start moves on one per +// second, and a home starts far fewer transactions than that. +func firstTxID(now time.Time) int { + return max(1, int(now.Sub(txIDEpoch)/time.Second)) +} + // SetApprovedIDs declares which charger ids are part of the site — the ids // loadpoints name in config. Everything else that connects is quarantined as // pending: accepted at the protocol level so the operator can see it in the @@ -310,7 +342,7 @@ func (h *Handler) noteVehicleID(id, vehicleID, source string, started bool) { approved := h.approved[id] h.mu.Unlock() if (changed || started) && approved && fn != nil { - fn(id, vehicleID, source) + h.callback(id, func() { fn(id, vehicleID, source) }) } } @@ -424,6 +456,7 @@ func (h *Handler) OnConnect(id string) { s.charging = false s.clearMeasuredPower() s.identityCurrent = false + s.bootLimitPending = false s.featureProfiles = "" s.steerable = nil h.cancelIdentityProbeLocked(s) @@ -445,6 +478,7 @@ func (h *Handler) OnDisconnect(id string) { delete(h.probing, id) s.online = false s.identityCurrent = false + s.bootLimitPending = false h.cancelIdentityProbeLocked(s) s.connectedKnown = false s.charging = false @@ -484,6 +518,7 @@ func (h *Handler) OnBootNotification(id string, req *core.BootNotificationReques h.noteIdentity(id) h.telHealth(id) h.maybeProbeCapability(id) + h.maybeSendBootLimit(id) return core.NewBootNotificationConfirmation( types.NewDateTime(time.Now()), h.heartbeatIntervalS, @@ -576,6 +611,7 @@ func (h *Handler) OnStatusNotification(id string, req *core.StatusNotificationRe h.pushReading(id, s) h.telSuccess(id) + h.resolveBootLimit(id) return core.NewStatusNotificationConfirmation(), nil } @@ -596,7 +632,9 @@ func (h *Handler) OnMeterValues(id string, req *core.MeterValuesRequest) (*core. measurand = types.MeasurandEnergyActiveImportRegister } val, err := strconv.ParseFloat(sv.Value, 64) - if err != nil { + // ParseFloat accepts "NaN" and "Inf". Taken up as a start + // reading, one would spoil every later session value. + if err != nil || math.IsNaN(val) || math.IsInf(val, 0) { continue } switch measurand { @@ -612,6 +650,9 @@ func (h *Handler) OnMeterValues(id string, req *core.MeterValuesRequest) (*core. if sv.Unit == types.UnitOfMeasureKWh { val *= 1000 } + if s.transactionID < 0 && sv.Phase == "" && req.TransactionId != nil { + h.takeUpTransactionLocked(id, s, *req.TransactionId, val) + } if s.transactionID >= 0 { s.sessionMeterWh = val - s.sessionStartMeterWh } @@ -619,8 +660,10 @@ func (h *Handler) OnMeterValues(id string, req *core.MeterValuesRequest) (*core. } if w, ok := meterPowerW(samples); ok { s.recordPower(w, measured, received) + s.meterCancelsBootLimitLocked(nil, w) } } + s.meterCancelsBootLimitLocked(req.TransactionId, 0) h.mu.Unlock() h.pushReading(id, s) @@ -628,6 +671,21 @@ func (h *Handler) OnMeterValues(id string, req *core.MeterValuesRequest) (*core. return core.NewMeterValuesConfirmation(), nil } +// takeUpTransactionLocked adopts a transaction that started before this run: +// the charger kept it, and its meter values name it, while Core restarted. +// The start meter reading went with the old process, so session energy +// counts from this first reading. The caller holds h.mu. +func (h *Handler) takeUpTransactionLocked(id string, s *chargerState, txID int, meterWh float64) { + if txID <= 0 || txID == s.stoppedTxID { + return + } + s.transactionID = txID + s.sessionStartMeterWh = meterWh + s.sessionMeterWh = 0 + h.nextTxID = max(h.nextTxID, txID+1) + slog.Info("OCPP running transaction taken up", "charger", id, "txid", txID, "meter_wh", meterWh) +} + func (h *Handler) OnStartTransaction(id string, req *core.StartTransactionRequest) (*core.StartTransactionConfirmation, error) { h.mu.Lock() txID := h.nextTxID @@ -658,7 +716,24 @@ func (h *Handler) OnStartTransaction(id string, req *core.StartTransactionReques func (h *Handler) OnStopTransaction(id string, req *core.StopTransactionRequest) (*core.StopTransactionConfirmation, error) { s := h.state(id) h.mu.Lock() - sessionWh := float64(req.MeterStop) - s.sessionStartMeterWh + // A stop for another transaction than the running one, such as a queued + // stop from before a restart, must not end the running one. + if s.transactionID >= 0 && req.TransactionId != s.transactionID { + current := s.transactionID + h.mu.Unlock() + slog.Warn("OCPP stop for another transaction ignored", + "charger", id, "txid", req.TransactionId, "current_txid", current) + h.telSuccess(id) + return core.NewStopTransactionConfirmation(), nil + } + // With no start and no meter value seen for the transaction, MeterStop is + // only the meter's lifetime total, not the session's energy. + known := s.transactionID >= 0 + sessionWh := s.sessionMeterWh + if known { + sessionWh = float64(req.MeterStop) - s.sessionStartMeterWh + } + s.stoppedTxID = req.TransactionId s.transactionID = -1 s.charging = false // A charger may report Available before the stop. The car is gone then. @@ -672,7 +747,9 @@ func (h *Handler) OnStopTransaction(id string, req *core.StopTransactionRequest) "charger", id, "txid", req.TransactionId, "session_wh", sessionWh, "reason", req.Reason) h.pushReading(id, s) - h.telMetric(id, "ev_session_wh", sessionWh, "Wh") + if known { + h.telMetric(id, "ev_session_wh", sessionWh, "Wh") + } h.telSuccess(id) return core.NewStopTransactionConfirmation(), nil } diff --git a/go/internal/ocpp/handlers_v201.go b/go/internal/ocpp/handlers_v201.go index b89cbfa17..fac40fb5c 100644 --- a/go/internal/ocpp/handlers_v201.go +++ b/go/internal/ocpp/handlers_v201.go @@ -67,6 +67,7 @@ func (h *handlerV201) OnBootNotification(id string, req *provisioning.BootNotifi h.setVersion(id, Version201) h.noteIdentity(id) h.telHealth(id) + h.maybeSendBootLimit(id) return provisioning.NewBootNotificationResponse( types201.NewDateTime(time.Now()), @@ -138,6 +139,7 @@ func (h *handlerV201) OnStatusNotification(id string, req *availability.StatusNo h.telSuccess(id) } h.pushReading(id, s) + h.resolveBootLimit(id) return availability.NewStatusNotificationResponse(), nil } diff --git a/go/internal/ocpp/identity.go b/go/internal/ocpp/identity.go index 5497ca3d4..34b7b492c 100644 --- a/go/internal/ocpp/identity.go +++ b/go/internal/ocpp/identity.go @@ -99,7 +99,17 @@ func (h *Handler) noteIdentity(id string) { slog.Info("ocpp: charger reported neither vendor nor serial — device identity falls back to the name it dialled with", "charger", id) } - fn(ident) + h.callback(id, func() { fn(ident) }) +} + +// callback runs fn, a callback main.go registered, outside the charger's +// socket lock. +func (h *Handler) callback(id string, fn func()) { + if h.later == nil { + fn() + return + } + h.later(id, fn) } // CurrentIdentity returns only an adopted charger's identity reported on its diff --git a/go/internal/ocpp/inbound_socket_test.go b/go/internal/ocpp/inbound_socket_test.go index a4f8de00f..6215f37bf 100644 --- a/go/internal/ocpp/inbound_socket_test.go +++ b/go/internal/ocpp/inbound_socket_test.go @@ -156,7 +156,10 @@ func TestOldSocketStatusCannotClearPauseAfterReconnect(t *testing.T) { if err := status(false); err != nil { t.Fatal(err) } - ctrl.Tick(context.Background(), time.Now()) + unplugged := time.Now() + ctrl.Tick(context.Background(), unplugged) + // The cable stays out for UnplugGrace. + ctrl.Tick(context.Background(), unplugged.Add(loadpoint.UnplugGrace)) if _, ok := ctrl.GetManualHold("garage", time.Now()); ok { t.Fatal("fresh physical unplug did not clear Pause") } diff --git a/go/internal/ocpp/profile_slot_test.go b/go/internal/ocpp/profile_slot_test.go new file mode 100644 index 000000000..a136bdac6 --- /dev/null +++ b/go/internal/ocpp/profile_slot_test.go @@ -0,0 +1,92 @@ +package ocpp + +import ( + "context" + "errors" + "fmt" + "runtime" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// While the request pump is stalled, the control loop keeps commanding each +// charger with a short deadline. Those commands must not leave a send behind +// each, or the charger is replayed every obsolete limit once the pump clears, +// one round trip each, before the current one. A fuse cut would wait behind +// them. +func TestStalledPumpDoesNotReplayObsoleteProfiles(t *testing.T) { + port, srv := startServer(t, telemetry.NewStore(), "garage", "carport") + _, carport, _ := connectCharger(t, srv, port, "carport") + _, _, _ = connectCharger(t, srv, port, "garage") + + // Hold garage's slot lock, as a stalled handler did before E2. The pump + // blocks writing garage's next request, and stops for every charger. + alias, err := srv.sockets.currentID("garage") + if err != nil { + t.Fatal(err) + } + holding := make(chan struct{}) + release := make(chan struct{}) + released := make(chan struct{}) + go func() { + defer close(released) + _, _ = boundCall(srv.sockets, alias, func(string) (bool, error) { + close(holding) + <-release + return true, nil + }) + }() + <-holding + unstall := func() { + select { + case <-release: + default: + close(release) + } + <-released + } + t.Cleanup(unstall) + + ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond) + _ = srv.Command(ctx, "garage", []byte(`{"action":"ev_set_current","power_w":4140}`)) + cancel() + + before := carport.count() + base := runtime.NumGoroutine() + const ticks = 40 + for i := 0; i < ticks; i++ { + w := 11040 + if i%2 == 1 { + w = 9000 + } + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Millisecond) + err := srv.Command(ctx, "carport", []byte(fmt.Sprintf(`{"action":"ev_set_current","power_w":%d}`, w))) + cancel() + // No verdict, not a refusal: core must not count the charger as + // refusing because another charger stalled the pump. + if !errors.Is(err, context.DeadlineExceeded) { + t.Fatalf("tick %d: err = %v, want context.DeadlineExceeded", i, err) + } + } + if grew := runtime.NumGoroutine() - base; grew > 5 { + t.Errorf("goroutines grew by %d during the stall, want at most one send per charger", grew) + } + + unstall() + // The fuse cut that follows the stall. 4140 W over three 230 V phases + // is the 6 A minimum. + ctx, cancel = context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + if err := srv.Command(ctx, "carport", []byte(`{"action":"ev_set_current","power_w":4140}`)); err != nil { + t.Fatalf("command after the stall: %v", err) + } + if got := carport.lastLimit(t); got != 6 { + t.Fatalf("carport ends at %.1f A, want 6 A", got) + } + // One profile from the stall reached the charger, then the current one. + if n := carport.count() - before; n > 2 { + t.Fatalf("carport received %d profiles for %d stalled ticks and one live command, want at most 2", n, ticks) + } +} diff --git a/go/internal/ocpp/server.go b/go/internal/ocpp/server.go index 6769bf6a5..7f214d901 100644 --- a/go/internal/ocpp/server.go +++ b/go/internal/ocpp/server.go @@ -65,6 +65,10 @@ type Server struct { done chan struct{} doneV201 chan struct{} stopOnce sync.Once + // profileSlots admits one unanswered charging profile per charger id; + // see attemptLimit. + profileMu sync.Mutex + profileSlots map[string]chan struct{} } // Start brings up the OCPP CS on the configured bind:port. It returns once @@ -132,6 +136,10 @@ func Start(ctx context.Context, cfg *Config, tel *telemetry.Store) (_ *Server, e s := &Server{cfg: cfg, cs: cs, handler: h, sockets: sockets, done: make(chan struct{})} h.identityProbe = s.requestBootNotification + h.later = sockets.later + h.SetManagedIDs(cfg.ManagedIDs) + h.bootLimit = s.sendBootLimit + h.bootLimitSettle = defaultBootLimitSettle // OCPP 2.0.1 on its own port, when configured. Same handler and therefore // the same charger state and telemetry — only the message encoding differs. diff --git a/go/internal/ocpp/socket_sessions.go b/go/internal/ocpp/socket_sessions.go index 8633d23fb..9cb8e5359 100644 --- a/go/internal/ocpp/socket_sessions.go +++ b/go/internal/ocpp/socket_sessions.go @@ -24,6 +24,10 @@ type socketSessions struct { slots map[string]*socketSlot aliases map[string]*socketBinding raw map[ws.Channel]*socketBinding + // after holds the callback work a message handler queued while its + // boundCall holds that charger's slot lock, keyed by charger id. A key + // is present only during such a call. + after map[string][]func() // The library closes the sockets in its table when it stops, but a // handshake that has passed the connection check joins that table // later and would stay open on a stopped server. Once closed is set no @@ -48,7 +52,7 @@ type socketBinding struct { func (b *socketBinding) ID() string { return b.alias } func newSocketSessions() *socketSessions { - return &socketSessions{slots: map[string]*socketSlot{}, aliases: map[string]*socketBinding{}, raw: map[ws.Channel]*socketBinding{}} + return &socketSessions{slots: map[string]*socketSlot{}, aliases: map[string]*socketBinding{}, raw: map[ws.Channel]*socketBinding{}, after: map[string][]func(){}} } func (s *socketSessions) slot(id string) *socketSlot { s.mu.Lock() @@ -239,17 +243,51 @@ func (g *guardedServer) Write(alias string, data []byte) error { // Match and mutate under one per-charger lock. SDK goroutines from a closed // socket cannot alter its replacement, including via a late BootNotification. +// +// The lock guards state only. Every write to this charger takes the same +// lock, so callback work that fn queues through later (main.go writes SQLite +// from identityReported) runs after the unlock: a stalled callback must not +// stop the commands to this charger and, through the SDK's single request +// pump, to every other charger. func boundCall[T any](s *socketSessions, alias string, fn func(string) (T, error)) (zero T, err error) { b := s.lookup(alias) if b == nil { return zero, errOldSocket } b.slot.mu.Lock() - defer b.slot.mu.Unlock() if b.slot.current != b || !b.active { + b.slot.mu.Unlock() return zero, errOldSocket } - return fn(b.Channel.ID()) + id := b.Channel.ID() + s.mu.Lock() + s.after[id] = nil + s.mu.Unlock() + defer func() { + s.mu.Lock() + after := s.after[id] + delete(s.after, id) + s.mu.Unlock() + b.slot.mu.Unlock() + for _, f := range after { + f() + } + }() + return fn(id) +} + +// later runs fn once the boundCall that holds id's slot lock releases it, or +// now when no boundCall holds it. +func (s *socketSessions) later(id string, fn func()) { + s.mu.Lock() + after, held := s.after[id] + if held { + s.after[id] = append(after, fn) + } + s.mu.Unlock() + if !held { + fn() + } } func (s *socketSessions) disconnected(alias string, fn func(string)) { b := s.lookup(alias) diff --git a/go/internal/ocpp/stalled_boot_test.go b/go/internal/ocpp/stalled_boot_test.go new file mode 100644 index 000000000..22180c39d --- /dev/null +++ b/go/internal/ocpp/stalled_boot_test.go @@ -0,0 +1,192 @@ +package ocpp + +import ( + "context" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/lorenzodonini/ocpp-go/ocpp1.6/types" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// An identityReported callback (main.go wires it to a SQLite RegisterDevice) +// that stalls during a BootNotification must not hold the charger's slot +// lock. If it did, the SDK's single request pump would block writing to that +// charger, every other charger would lose its commands, and once the 20-slot +// request channel filled, Server.Command would block past its own deadline. +func TestStalledBootCallbackDoesNotBlockCommands(t *testing.T) { + port, srv := startServer(t, telemetry.NewStore(), "garage", "carport") + _, _, _ = connectCharger(t, srv, port, "carport") + garage, _, _ := connectCharger(t, srv, port, "garage") + + entered := make(chan struct{}) + release := make(chan struct{}) + var once sync.Once + srv.Handler().SetIdentityReported(func(ident ChargerIdentity) { + if ident.ID != "garage" { + return + } + once.Do(func() { close(entered) }) + <-release // a stalled state.db write + }) + var releaseOnce sync.Once + unstall := func() { releaseOnce.Do(func() { close(release) }) } + t.Cleanup(unstall) + + go func() { _, _ = garage.BootNotification("Dawn", "Charge Amps") }() + select { + case <-entered: + case <-time.After(2 * time.Second): + t.Fatal("second boot never reached identityReported") + } + + cmd := []byte(`{"action":"ev_set_current","power_w":4140}`) + // One command to the stalled charger: the pump takes it and blocks in Write. + ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond) + _ = srv.Command(ctx, "garage", cmd) + cancel() + + // A healthy charger on the same server now gets nothing through. + ctx, cancel = context.WithTimeout(context.Background(), time.Second) + err := srv.Command(ctx, "carport", cmd) + cancel() + if err != nil { + t.Errorf("healthy charger lost its command while another charger's Boot callback stalled: %v", err) + } + + // Keep sending with a short deadline, as the control loop does each tick. + for i := 1; i <= 25; i++ { + ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond) + done := make(chan error, 1) + go func() { done <- srv.Command(ctx, "carport", cmd) }() + select { + case <-done: + cancel() + case <-time.After(2 * time.Second): + cancel() + unstall() + <-done + t.Fatalf("command %d still blocked 2 s after its 50 ms deadline", i) + } + } +} + +// A vehicleIdentified callback that stalls must not stop commands to its own +// charger, and a callback that itself commands that charger must not +// deadlock on the slot lock. +func TestStalledVehicleCallbackCanCommandItsCharger(t *testing.T) { + port, srv := startServer(t, telemetry.NewStore(), "garage") + garage, fake, _ := connectCharger(t, srv, port, "garage") + + entered := make(chan struct{}) + release := make(chan struct{}) + var calls atomic.Int32 + reentry := make(chan error, 1) + srv.Handler().SetVehicleIdentified(func(chargerID, vehicleID, source string) { + if calls.Add(1) == 1 { + close(entered) + } + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + reentry <- srv.Command(ctx, "garage", []byte(`{"action":"ev_set_current","power_w":4140}`)) + cancel() + <-release + }) + var releaseOnce sync.Once + unstall := func() { releaseOnce.Do(func() { close(release) }) } + t.Cleanup(unstall) + + go func() { _, _ = garage.StartTransaction(1, "RFID-1", 0, types.NewDateTime(time.Now())) }() + select { + case <-entered: + case <-time.After(2 * time.Second): + t.Fatal("vehicleIdentified never fired") + } + select { + case err := <-reentry: + if err != nil { + t.Fatalf("command from inside the callback failed: %v", err) + } + case <-time.After(3 * time.Second): + t.Fatal("command from inside the callback deadlocked") + } + + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + if err := srv.Command(ctx, "garage", []byte(`{"action":"ev_set_current","power_w":2300}`)); err != nil { + t.Fatalf("command to the charger whose callback stalls failed: %v", err) + } + if n := fake.count(); n < 2 { + t.Fatalf("charger received %d profiles, want at least 2", n) + } + unstall() + time.Sleep(100 * time.Millisecond) + if got := calls.Load(); got != 1 { + t.Fatalf("callback ran %d times, want 1", got) + } +} + +// Work queued through later from another goroutine while a boundCall holds +// the slot runs once, after the lock is free. With no boundCall it runs at +// once, and the queue keeps no key behind. +func TestLaterRunsAfterTheSlotIsFree(t *testing.T) { + port, srv := startServer(t, telemetry.NewStore(), "garage") + _, _, _ = connectCharger(t, srv, port, "garage") + alias, err := srv.sockets.currentID("garage") + if err != nil { + t.Fatal(err) + } + holding := make(chan struct{}) + release := make(chan struct{}) + done := make(chan struct{}) + go func() { + _, _ = boundCall(srv.sockets, alias, func(string) (bool, error) { + close(holding) + <-release + return true, nil + }) + close(done) + }() + <-holding + var ran atomic.Int32 + lockFree := make(chan bool, 1) + b := srv.sockets.lookup(alias) + srv.sockets.later("garage", func() { + ran.Add(1) + ok := b.slot.mu.TryLock() + if ok { + b.slot.mu.Unlock() + } + lockFree <- ok + }) + if ran.Load() != 0 { + t.Fatal("later ran while the slot was held") + } + close(release) + <-done + select { + case ok := <-lockFree: + if !ok { + t.Fatal("queued work ran with the slot lock held") + } + case <-time.After(time.Second): + t.Fatal("queued work was lost") + } + if got := ran.Load(); got != 1 { + t.Fatalf("queued work ran %d times, want 1", got) + } + + var now atomic.Bool + srv.sockets.later("garage", func() { now.Store(true) }) + if !now.Load() { + t.Fatal("later did not run at once with no boundCall active") + } + srv.sockets.mu.Lock() + n := len(srv.sockets.after) + srv.sockets.mu.Unlock() + if n != 0 { + t.Fatalf("after map kept %d keys", n) + } +} diff --git a/go/internal/ocpp/transaction_resume_test.go b/go/internal/ocpp/transaction_resume_test.go new file mode 100644 index 000000000..4b6ecee70 --- /dev/null +++ b/go/internal/ocpp/transaction_resume_test.go @@ -0,0 +1,210 @@ +package ocpp + +import ( + "encoding/json" + "math" + "testing" + "time" + + "github.com/lorenzodonini/ocpp-go/ocpp1.6/core" + "github.com/lorenzodonini/ocpp-go/ocpp1.6/types" + + "github.com/srcfl/ftw/go/internal/telemetry" +) + +// restartedHandler is Core just after a restart: a fresh handler and an +// adopted charger that reconnected while its car kept charging. +func restartedHandler(t *testing.T, tel *telemetry.Store, ids ...string) *Handler { + t.Helper() + h := NewHandler(tel, 60) + h.SetApprovedIDs(ids) + for _, id := range ids { + h.OnConnect(id) + h.OnStatusNotification(id, &core.StatusNotificationRequest{ConnectorId: 1, Status: core.ChargePointStatusCharging}) + } + return h +} + +func meterRegister(h *Handler, id string, tx int, wh string) { + h.OnMeterValues(id, &core.MeterValuesRequest{ConnectorId: 1, TransactionId: &tx, MeterValue: []types.MeterValue{{ + Timestamp: types.NewDateTime(time.Now()), + SampledValue: []types.SampledValue{{Value: wh, Measurand: types.MeasurandEnergyActiveImportRegister, Unit: types.UnitOfMeasureWh}}, + }}}) +} + +func readingSessionWh(t *testing.T, tel *telemetry.Store, id string) float64 { + t.Helper() + r := tel.Get(id, telemetry.DerEV) + if r == nil { + t.Fatalf("no EV reading for %s", id) + } + var d struct { + SessionWh float64 `json:"session_wh"` + } + if err := json.Unmarshal(r.Data, &d); err != nil { + t.Fatal(err) + } + return d.SessionWh +} + +// The charger keeps its transaction while Core restarts and goes on sending +// meter values for it. Core takes the transaction up from the first of them, +// so session energy counts again instead of staying at 0 until unplug. +func TestOCPP16PicksUpRunningTransactionAfterRestart(t *testing.T) { + tel := telemetry.NewStore() + h := restartedHandler(t, tel, "charger") + + meterRegister(h, "charger", 41, "15200") + meterRegister(h, "charger", 41, "15700") + + if v := h.Snapshot()["charger"]; v.TxID != 41 || v.SessionWh != 500 { + t.Fatalf("after restart: tx %d, session %.0f Wh; want tx 41, 500 Wh", v.TxID, v.SessionWh) + } + if got := readingSessionWh(t, tel, "charger"); got != 500 { + t.Fatalf("reading session_wh = %.0f, want 500", got) + } + + h.OnStopTransaction("charger", &core.StopTransactionRequest{TransactionId: 41, MeterStop: 15900}) + if v := h.Snapshot()["charger"]; v.TxID != -1 || v.SessionWh != 700 { + t.Fatalf("after stop: tx %d, session %.0f Wh; want tx -1, 700 Wh", v.TxID, v.SessionWh) + } +} + +// A stop for a transaction Core never saw start carries only the meter's +// lifetime total. That is not the session's energy. +func TestOCPP16StopOfUnseenTransactionKeepsLifetimeMeterOut(t *testing.T) { + tel := telemetry.NewStore() + h := restartedHandler(t, tel, "charger") + + h.OnStopTransaction("charger", &core.StopTransactionRequest{TransactionId: 41, MeterStop: 1234567}) + + if v := h.Snapshot()["charger"]; v.SessionWh != 0 { + t.Fatalf("session_wh = %.0f, want 0: MeterStop is the lifetime register", v.SessionWh) + } +} + +// A late meter value for a transaction that has stopped must not start it +// again: session energy would fall to 0 while the car is still plugged in. +func TestOCPP16StoppedTransactionIsNotTakenUpAgain(t *testing.T) { + tel := telemetry.NewStore() + h := restartedHandler(t, tel, "charger") + conf, err := h.OnStartTransaction("charger", &core.StartTransactionRequest{ConnectorId: 1, IdTag: "tag", MeterStart: 1000}) + if err != nil { + t.Fatal(err) + } + h.OnStopTransaction("charger", &core.StopTransactionRequest{TransactionId: conf.TransactionId, MeterStop: 8500}) + + meterRegister(h, "charger", conf.TransactionId, "8500") + + if v := h.Snapshot()["charger"]; v.TxID != -1 || v.SessionWh != 7500 { + t.Fatalf("tx %d, session %.0f Wh; want tx -1, 7500 Wh", v.TxID, v.SessionWh) + } +} + +// Every transaction id goes to one transaction, across restarts too. +func TestOCPP16TransactionIDsDoNotRepeatAfterRestart(t *testing.T) { + tel := telemetry.NewStore() + before := restartedHandler(t, tel, "a", "b") + old, err := before.OnStartTransaction("a", &core.StartTransactionRequest{ConnectorId: 1, IdTag: "tag", MeterStart: 100}) + if err != nil { + t.Fatal(err) + } + if old.TransactionId == 1 { + t.Fatalf("first id after a start is 1, which every earlier run also handed out") + } + + // Core restarts within the same second. Charger a reports its running + // transaction; then b starts one. + after := restartedHandler(t, tel, "a", "b") + meterRegister(after, "a", old.TransactionId, "400") + next, err := after.OnStartTransaction("b", &core.StartTransactionRequest{ConnectorId: 1, IdTag: "tag", MeterStart: 0}) + if err != nil { + t.Fatal(err) + } + if next.TransactionId == old.TransactionId { + t.Fatalf("b got transaction id %d, which a still holds", next.TransactionId) + } +} + +// A run starts its ids at the seconds since txIDEpoch, so a later run starts +// above the ids an earlier one used unless it started more transactions than +// it ran seconds. +func TestOCPP16FirstTransactionID(t *testing.T) { + t0 := time.Date(2026, 10, 21, 13, 1, 49, 0, time.UTC) + if firstTxID(t0.Add(20*time.Second))-firstTxID(t0) != 20 { + t.Fatalf("ids do not move on one per second") + } + if got := firstTxID(time.Unix(0, 0)); got != 1 { + t.Fatalf("clock before the epoch: first id %d, want 1", got) + } + if got := firstTxID(time.Date(2090, 1, 1, 0, 0, 0, 0, time.UTC)); got >= math.MaxInt32 { + t.Fatalf("first id %d in 2090 does not fit the 32 bits chargers store it in", got) + } +} + +// Core restarts while the charger is in transaction 42. The car finishes +// while still plugged in. The stop carries the meter's lifetime register, so +// the session must count from the reading the transaction was taken up at. +func TestOCPP16StopAfterRestartReportsSessionNotLifetimeRegister(t *testing.T) { + tel := telemetry.NewStore() + h := restartedHandler(t, tel, "charger") + + meterRegister(h, "charger", 42, "12345000") + h.OnStatusNotification("charger", &core.StatusNotificationRequest{ConnectorId: 1, Status: core.ChargePointStatusSuspendedEV}) + h.OnStopTransaction("charger", &core.StopTransactionRequest{TransactionId: 42, MeterStop: 12345678, Reason: core.ReasonEVDisconnected}) + h.OnStatusNotification("charger", &core.StatusNotificationRequest{ConnectorId: 1, Status: core.ChargePointStatusFinishing}) + + if got := readingSessionWh(t, tel, "charger"); got != 678 { + t.Fatalf("reading session_wh = %.0f, want 678", got) + } + if m, _, ok := tel.LatestMetric("charger", "ev_session_wh"); !ok || m != 678 { + t.Fatalf("ev_session_wh metric = %v (ok %v), want 678", m, ok) + } +} + +// A stop that names another transaction than the running one leaves the +// running one alone. +func TestOCPP16StopForAnotherTransactionKeepsTheRunningOne(t *testing.T) { + tel := telemetry.NewStore() + h := restartedHandler(t, tel, "charger") + conf, err := h.OnStartTransaction("charger", &core.StartTransactionRequest{ConnectorId: 1, IdTag: "card", MeterStart: 1000}) + if err != nil { + t.Fatal(err) + } + meterRegister(h, "charger", conf.TransactionId, "3000") + + if _, err := h.OnStopTransaction("charger", &core.StopTransactionRequest{TransactionId: 999, MeterStop: 9_000_000}); err != nil { + t.Fatal(err) + } + + v := h.Snapshot()["charger"] + if v.TxID != conf.TransactionId || v.SessionWh != 2000 || !v.Charging { + t.Fatalf("tx %d, session %.0f Wh, charging %v; want tx %d, 2000 Wh, charging", + v.TxID, v.SessionWh, v.Charging, conf.TransactionId) + } + if _, _, ok := tel.LatestMetric("charger", "ev_session_wh"); ok { + t.Fatal("ev_session_wh recorded for a stop of another transaction") + } +} + +// A NaN or Inf register must not become the start reading of a taken-up +// transaction. The next finite reading starts the session instead. +func TestOCPP16NonFiniteRegisterIsNotTakenUp(t *testing.T) { + for _, bad := range []string{"NaN", "Inf", "-Inf"} { + t.Run(bad, func(t *testing.T) { + tel := telemetry.NewStore() + h := restartedHandler(t, tel, "charger") + + meterRegister(h, "charger", 41, bad) + if v := h.Snapshot()["charger"]; v.TxID != -1 { + t.Fatalf("tx %d taken up from a %s register", v.TxID, bad) + } + meterRegister(h, "charger", 41, "15200") + meterRegister(h, "charger", 41, "15700") + + if got := readingSessionWh(t, tel, "charger"); got != 500 { + t.Fatalf("reading session_wh = %.0f, want 500", got) + } + }) + } +} diff --git a/go/internal/onboarding/guide.go b/go/internal/onboarding/guide.go index e4c8b8f31..db1b93672 100644 --- a/go/internal/onboarding/guide.go +++ b/go/internal/onboarding/guide.go @@ -34,7 +34,7 @@ func Guide(c *config.Config) map[string]any { add("solar", "Does this home have solar panels? If so, do you know their total kWp?", "", "kWp (convert to W for pv_rated_w)", "Capacity and roof orientation are optional estimates. Record their source and time in forecast_priors.solar_source and solar_recorded_at. They are not verified inverter control limits. Do not ask for a capacity already known from reliable device metadata.", "/setup/solar_present", "/weather/pv_rated_w", "/weather/pv_arrays") } if c.Setup == nil || !c.Setup.GoalsReviewed { - add("goals", "What should FTW do for this home: use solar, shift energy to cheap hours, keep a battery reserve, or prepare cars for departure?", "", "W, Wh; SoC as 0..1", "Read the mode catalogue in setup_schema and active mode/preferences in commissioning. Confirm runtime_timezone before any local HH:MM departure target. If it is wrong, help the owner set the host time zone and restart FTW; there is no writable configuration time-zone field. Store equipment limits and car/loadpoint configuration in this draft. After configuration is running and measurements are fresh, while measurement mode still blocks equipment control, use car_goal_request for car departure schedules and goal_request for the chosen operating mode and planning style, including explicit battery-export permission. The owner approves each request in Access on the local box or in the FTW app; read requests_list and active status to verify the result. Planner config fields do not replace these active preferences. Use Check requests and access on the local Access page to refresh approvals. Mark goals_reviewed only after applying the requested goals or the owner explicitly leaves them for later; include it in the final control-enabling review.", "/setup/goals_reviewed", "/batteries", "/vehicles", "/loadpoints", "/v2x") + add("goals", "What should FTW do for this home: use solar, shift energy to cheap hours, keep a battery reserve, or prepare cars for departure?", "", "W, Wh; SoC as 0..1", "Read the mode catalogue in setup_schema and active mode/preferences in commissioning. Confirm runtime_timezone before any local HH:MM departure target. If it is wrong, set /site/time_zone to the household's IANA zone; Core applies it when it restarts. Store equipment limits and car/loadpoint configuration in this draft. After configuration is running and measurements are fresh, while measurement mode still blocks equipment control, use car_goal_request for car departure schedules and goal_request for the chosen operating mode and planning style, including explicit battery-export permission. The owner approves each request in Access on the local box or in the FTW app; read requests_list and active status to verify the result. Planner config fields do not replace these active preferences. Use Check requests and access on the local Access page to refresh approvals. Mark goals_reviewed only after applying the requested goals or the owner explicitly leaves them for later; include it in the final control-enabling review.", "/setup/goals_reviewed", "/site/time_zone", "/batteries", "/vehicles", "/loadpoints", "/v2x") } if c.Setup == nil || !c.Setup.PriceReviewed { add("electricity_price", "Which electricity price area and import/export contract apply here?", "", "", "Read supported countries, zones and currencies from setup_schema. Ask only for missing contract terms. If unknown, continue measuring and show that price planning still needs this information.", "/price", "/setup/price_reviewed") diff --git a/go/internal/onboarding/schema.go b/go/internal/onboarding/schema.go index 1d212da6e..21d060bd4 100644 --- a/go/internal/onboarding/schema.go +++ b/go/internal/onboarding/schema.go @@ -67,6 +67,9 @@ func schemaFor(t reflect.Type) map[string]any { } func setupFieldDescription(t reflect.Type, name string) string { + if t == reflect.TypeFor[config.Site]() && name == "time_zone" { + return "The household's IANA time zone, such as Europe/Stockholm. Car goals, the import hour, daily history and savings read this clock. Empty keeps the host's zone, which is UTC in Docker. Core applies a change when it restarts." + } if t == reflect.TypeFor[config.Setup]() { return map[string]string{ "goals_reviewed": "The owner reviewed operating mode, battery reserve/export, planning style and any car/loadpoint goals. Unknown optional goals can wait.", diff --git a/go/internal/prices/prices.go b/go/internal/prices/prices.go index 973b6c11a..01cabd451 100644 --- a/go/internal/prices/prices.go +++ b/go/internal/prices/prices.go @@ -638,7 +638,13 @@ func FromConfig(cfg *config.Price, st *state.Store, fx FXConverter) *Service { sp.FX = fx p = withFallback(sp, np) case "elprisetjustnu": - p = withFallback(NewElpriser(), np) + ep := NewElpriser() + if base := strings.TrimRight(cfg.BaseURL, "/"); base != "" { + ep.BaseURL = base + p = ep + } else { + p = withFallback(ep, np) + } case "entsoe": ep := NewENTSOE(cfg.APIKey) ep.Currency = currency diff --git a/go/internal/prices/prices_test.go b/go/internal/prices/prices_test.go index e531695a4..5a1ffe59d 100644 --- a/go/internal/prices/prices_test.go +++ b/go/internal/prices/prices_test.go @@ -363,6 +363,37 @@ func TestFromConfigDefaultsZoneAndVAT(t *testing.T) { } } +// A base_url reads that server only: a day it does not have stays empty +// instead of falling back to live Nord Pool prices. +func TestFromConfigBaseURLReadsOnlyThatServer(t *testing.T) { + var paths []string + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + paths = append(paths, r.URL.Path) + if !strings.HasSuffix(r.URL.Path, "/2026/10-27_SE3.json") { + http.NotFound(w, r) + return + } + fmt.Fprint(w, `[{"SEK_per_kWh":1.25,"time_start":"2026-10-27T18:00:00+01:00","time_end":"2026-10-27T18:15:00+01:00"}]`) + })) + defer srv.Close() + s := FromConfig(&config.Price{Provider: "elprisetjustnu", BaseURL: srv.URL + "/"}, nil, nil) + if _, ok := s.Provider.(*ElpriserProvider); !ok { + t.Fatalf("provider = %T, want the elprisetjustnu provider alone", s.Provider) + } + loc := time.FixedZone("CET", 3600) + rows, err := s.Provider.Fetch(context.Background(), "SE3", time.Date(2026, 10, 27, 12, 0, 0, 0, loc)) + if err != nil || len(rows) != 1 || rows[0].SEKPerKWh != 1.25 || rows[0].SlotLenMin != 15 { + t.Fatalf("today: rows %+v err %v", rows, err) + } + rows, err = s.Provider.Fetch(context.Background(), "SE3", time.Date(2026, 10, 28, 12, 0, 0, 0, loc)) + if err != nil || len(rows) != 0 { + t.Fatalf("a day the server lacks: rows %+v err %v, want none", rows, err) + } + if paths[0] != "/2026/10-27_SE3.json" { + t.Errorf("path %q: the trailing slash doubled", paths[0]) + } +} + // ---- ENTSOE minimal checks ---- func TestENTSOERequiresAPIKey(t *testing.T) { diff --git a/go/internal/selfupdate/channel_test.go b/go/internal/selfupdate/channel_test.go new file mode 100644 index 000000000..3e4ca4056 --- /dev/null +++ b/go/internal/selfupdate/channel_test.go @@ -0,0 +1,51 @@ +package selfupdate + +import ( + "context" + "testing" +) + +// A box installed from a beta tag that never chose a channel must keep +// following beta after it installs the stable that beta promotes to. +func TestNew_BetaBoxStaysOnBetaAfterStablePromotion(t *testing.T) { + st := newMemStore() // same state.db across the restart + + // Boot 1: running a beta, no saved channel. Beta offers the promoted stable. + rls := fakeReleasesServer(t, nil, fakeRelease{tag: "v0.158.0"}) + c := newCheckerOnFakes(t, "v0.158.0-beta.3", rls, st) + info, err := c.Check(context.Background(), true) + if err != nil { + t.Fatalf("Check: %v", err) + } + t.Logf("running v0.158.0-beta.3: channel=%s latest=%s update_available=%v", info.Channel, info.Latest, info.UpdateAvailable) + if info.Channel != ChannelBeta || info.Latest != "v0.158.0" || !info.UpdateAvailable { + t.Fatalf("beta box should be offered the promoted stable: %+v", info) + } + + // Boot 2: the stable is installed; a new beta is published. + rls2 := fakeReleasesServer(t, nil, + fakeRelease{tag: "v0.159.0-beta.1", prerelease: true}, + fakeRelease{tag: "v0.158.0"}, + ) + c2 := newCheckerOnFakes(t, "v0.158.0", rls2, st) + info, err = c2.Check(context.Background(), true) + if err != nil { + t.Fatalf("Check: %v", err) + } + t.Logf("running v0.158.0: channel=%s latest=%s update_available=%v", info.Channel, info.Latest, info.UpdateAvailable) + if info.Channel != ChannelBeta { + t.Fatalf("channel flipped to %q only because the version string changed", info.Channel) + } + if info.Latest != "v0.159.0-beta.1" || !info.UpdateAvailable { + t.Fatalf("next beta not offered: %+v", info) + } +} + +// A build without a release tag implies no channel, so it saves none. +func TestNew_DevBuildSavesNoChannel(t *testing.T) { + st := newMemStore() + New(Config{CurrentVersion: "dev"}, st) + if got, ok := st.LoadConfig(channelKey); ok { + t.Fatalf("dev build saved channel %q", got) + } +} diff --git a/go/internal/selfupdate/release_order_test.go b/go/internal/selfupdate/release_order_test.go new file mode 100644 index 000000000..ad182457a --- /dev/null +++ b/go/internal/selfupdate/release_order_test.go @@ -0,0 +1,97 @@ +package selfupdate + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "strconv" + "sync/atomic" + "testing" +) + +// A stable may be promoted after newer betas ship, so GitHub lists it above +// a higher beta. Beta boxes must still get the highest version. +func TestCheck_BetaPrefersNewerBetaListedBelowPromotedStable(t *testing.T) { + releases := fakeReleasesServer(t, nil, + fakeRelease{tag: "v0.158.0"}, + fakeRelease{tag: "v0.159.0-beta.1", prerelease: true}, + fakeRelease{tag: "v0.158.0-beta.3", prerelease: true}, + ) + for _, tc := range []struct { + current string + update bool + }{ + {"v0.157.2-beta.1", true}, + {"v0.158.0-beta.3", true}, + {"v0.159.0-beta.1", false}, + } { + t.Run(tc.current, func(t *testing.T) { + c := newCheckerOnFakes(t, tc.current, releases, newMemStore()) + if err := c.SetChannel(ChannelBeta); err != nil { + t.Fatalf("SetChannel: %v", err) + } + info, err := c.Check(context.Background(), true) + if err != nil { + t.Fatalf("Check: %v", err) + } + if info.Latest != "v0.159.0-beta.1" || info.UpdateAvailable != tc.update { + t.Fatalf("latest=%q update=%v, want v0.159.0-beta.1 update=%v", info.Latest, info.UpdateAvailable, tc.update) + } + }) + } + + stable := newCheckerOnFakes(t, "v0.157.2", releases, newMemStore()) + info, err := stable.Check(context.Background(), true) + if err != nil { + t.Fatalf("stable Check: %v", err) + } + if info.Channel != ChannelStable || info.Latest != "v0.158.0" || !info.UpdateAvailable { + t.Fatalf("stable info = %+v", info) + } +} + +// pagedReleasesServer serves one page per request and links each page to +// the next, as GitHub does. +func pagedReleasesServer(t *testing.T, calls *atomic.Int32, pages ...[]fakeRelease) *httptest.Server { + t.Helper() + var srv *httptest.Server + srv = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + calls.Add(1) + page, _ := strconv.Atoi(r.URL.Query().Get("page")) + if page+1 < len(pages) { + w.Header().Set("Link", fmt.Sprintf(`<%s/releases?page=%d>; rel="next"`, srv.URL, page+1)) + } + list := make([]map[string]any, 0, len(pages[page])) + for _, rel := range pages[page] { + list = append(list, rel.json()) + } + _ = json.NewEncoder(w).Encode(list) + })) + t.Cleanup(srv.Close) + return srv +} + +func TestCheck_BetaWalksPastStableToTheFirstBetaAndStops(t *testing.T) { + var calls atomic.Int32 + releases := pagedReleasesServer(t, &calls, + []fakeRelease{{tag: "v0.158.0"}}, + []fakeRelease{{tag: "v0.159.0-beta.1", prerelease: true}}, + []fakeRelease{{tag: "v0.157.0"}}, + ) + c := newCheckerOnFakes(t, "v0.158.0-beta.3", releases, newMemStore()) + if err := c.SetChannel(ChannelBeta); err != nil { + t.Fatalf("SetChannel: %v", err) + } + info, err := c.Check(context.Background(), true) + if err != nil { + t.Fatalf("Check: %v", err) + } + if info.Latest != "v0.159.0-beta.1" || !info.UpdateAvailable { + t.Fatalf("beta info = %+v", info) + } + if got := calls.Load(); got != 2 { + t.Fatalf("pages fetched = %d, want 2: the walk stops at the first beta", got) + } +} diff --git a/go/internal/selfupdate/selfupdate.go b/go/internal/selfupdate/selfupdate.go index c7bf5581a..80e410eb1 100644 --- a/go/internal/selfupdate/selfupdate.go +++ b/go/internal/selfupdate/selfupdate.go @@ -234,6 +234,12 @@ func New(cfg Config, store Store) *Checker { } else if parsed, err := ParseChannel(persisted); err == nil { channel = parsed } + } else if isBetaTag(cfg.CurrentVersion) || isStableTag(cfg.CurrentVersion) { + // Save the channel this release implies. A beta box that later + // installs the stable its beta became must stay on beta. + if err := store.SaveConfig(channelKey, string(channel)); err != nil { + slog.Warn("selfupdate: could not save the update channel", "channel", channel, "err", err) + } } } c := &Checker{cfg: cfg, store: store} @@ -390,16 +396,33 @@ func (c *Checker) Check(ctx context.Context, force bool) (Info, error) { // resolveChannel selects the newest 0.x release on channel. Beta includes // stable releases, so beta testers converge back to a promoted stable build // instead of remaining pinned to the final prerelease. +// +// GitHub lists releases newest-created first. The release-order check +// (scripts/check-native-release-order.py) refuses a stable below a published +// stable and a beta below any published release. So the first stable listed +// is the newest stable, and nothing listed after the first beta outranks it. +// A stable may be promoted after newer betas, so on beta the walk keeps the +// higher of the first stable and the first beta. func (c *Checker) resolveChannel(ctx context.Context, channel Channel) (ghRelease, error) { - return c.fetchReleaseList(ctx, func(rel ghRelease) bool { + var best ghRelease + err := c.walkReleases(ctx, func(rel ghRelease) bool { if !strings.HasPrefix(rel.TagName, "v0.") { return false } - if channel == ChannelStable { - return !rel.Prerelease && isStableTag(rel.TagName) + stable := !rel.Prerelease && isStableTag(rel.TagName) + beta := channel != ChannelStable && rel.Prerelease && isBetaTag(rel.TagName) + if !stable && !beta { + return false + } + if best.TagName == "" || isNewer(rel.TagName, best.TagName) { + best = rel } - return !rel.Prerelease && isStableTag(rel.TagName) || rel.Prerelease && isBetaTag(rel.TagName) + return beta || channel == ChannelStable }) + if err != nil { + return ghRelease{}, err + } + return best, nil } func hasNativeAssets(rel ghRelease, arch string) bool { @@ -480,40 +503,42 @@ type ghRelease struct { } `json:"assets"` } -func (c *Checker) fetchReleaseList(ctx context.Context, accept func(ghRelease) bool) (ghRelease, error) { +// walkReleases visits published releases in list order, page by page, until +// visit returns true or the list ends. +func (c *Checker) walkReleases(ctx context.Context, visit func(ghRelease) bool) error { rawURL := c.cfg.ReleasesURL seen := make(map[string]bool) for rawURL != "" { if seen[rawURL] { - return ghRelease{}, errors.New("github releases list: repeated page") + return errors.New("github releases list: repeated page") } seen[rawURL] = true resp, err := c.getGitHub(ctx, rawURL) if err != nil { - return ghRelease{}, err + return err } if resp.StatusCode >= 400 { body, _ := io.ReadAll(io.LimitReader(resp.Body, 4<<10)) _ = resp.Body.Close() - return ghRelease{}, fmt.Errorf("github releases list %d: %s", resp.StatusCode, strings.TrimSpace(string(body))) + return fmt.Errorf("github releases list %d: %s", resp.StatusCode, strings.TrimSpace(string(body))) } var releases []ghRelease decodeErr := json.NewDecoder(io.LimitReader(resp.Body, 4<<20)).Decode(&releases) _ = resp.Body.Close() if decodeErr != nil { - return ghRelease{}, decodeErr + return decodeErr } for _, rel := range releases { - if !rel.Draft && accept(rel) { - return rel, nil + if !rel.Draft && visit(rel) { + return nil } } rawURL, err = nextReleasePage(resp.Header.Get("Link"), rawURL) if err != nil { - return ghRelease{}, err + return err } } - return ghRelease{}, nil + return nil } func nextReleasePage(linkHeader, currentURL string) (string, error) { diff --git a/go/internal/state/daily_zone.go b/go/internal/state/daily_zone.go new file mode 100644 index 000000000..a3410cdef --- /dev/null +++ b/go/internal/state/daily_zone.go @@ -0,0 +1,60 @@ +package state + +import "context" + +// dailyZonePrefix marks, in history_migrations, the household zone the cached +// local-day energy was computed on. +const dailyZonePrefix = "daily-zone:" + +// KeepDailyZone ties the cached local-day energy to the household zone. +// +// Day keys are dates on the household clock, so a day cached on another clock +// covers other hours. A new zone clears the cache, and each day is computed +// again from the UTC buckets on its next read; cleared reports that. The +// first call on a box records the zone and keeps the rows, which were cached +// on the clock it already runs on. +func (s *Store) KeepDailyZone(ctx context.Context, zone string) (cleared bool, err error) { + s.historyWriteMu.Lock() + defer s.historyWriteMu.Unlock() + tx, err := s.history.BeginTx(ctx, nil) + if err != nil { + return false, err + } + defer tx.Rollback() + rows, err := tx.QueryContext(ctx, `SELECT name FROM history_migrations WHERE name LIKE ?`, dailyZonePrefix+"%") + if err != nil { + return false, err + } + var marks []string + for rows.Next() { + var name string + if err := rows.Scan(&name); err != nil { + rows.Close() + return false, err + } + marks = append(marks, name) + } + rows.Close() + if err := rows.Err(); err != nil { + return false, err + } + want := dailyZonePrefix + zone + if len(marks) == 1 && marks[0] == want { + return false, nil + } + cleared = len(marks) > 0 + if cleared { + for _, table := range []string{"energy_daily", "energy_daily_partial", "history_daily_quality"} { + if _, err := tx.ExecContext(ctx, `DELETE FROM `+table); err != nil { + return false, err + } + } + if _, err := tx.ExecContext(ctx, `DELETE FROM history_migrations WHERE name LIKE ?`, dailyZonePrefix+"%"); err != nil { + return false, err + } + } + if _, err := tx.ExecContext(ctx, `INSERT INTO history_migrations(name) VALUES(?)`, want); err != nil { + return false, err + } + return cleared, tx.Commit() +} diff --git a/go/internal/state/daily_zone_test.go b/go/internal/state/daily_zone_test.go new file mode 100644 index 000000000..526ffbffb --- /dev/null +++ b/go/internal/state/daily_zone_test.go @@ -0,0 +1,34 @@ +package state + +import ( + "context" + "testing" +) + +// A Docker box cached its days on UTC. Once the household zone is stored, +// those dates cover other hours, so they go and are computed again. +func TestANewHouseholdZoneClearsTheCachedDays(t *testing.T) { + s := openTestStore(t) + ctx := context.Background() + if cleared, err := s.KeepDailyZone(ctx, "UTC"); err != nil || cleared { + t.Fatalf("first zone: cleared=%v err=%v, want the rows kept", cleared, err) + } + if err := s.SaveDailyEnergy("2026-10-30", DayEnergy{LoadWh: 1, Intervals: 1}); err != nil { + t.Fatal(err) + } + if cleared, err := s.KeepDailyZone(ctx, "UTC"); err != nil || cleared { + t.Fatalf("same zone: cleared=%v err=%v", cleared, err) + } + if _, ok, err := s.LoadDailyEnergy("2026-10-30"); err != nil || !ok { + t.Fatalf("same zone dropped the day: ok=%v err=%v", ok, err) + } + if cleared, err := s.KeepDailyZone(ctx, "Europe/Stockholm"); err != nil || !cleared { + t.Fatalf("new zone: cleared=%v err=%v, want the days cleared", cleared, err) + } + if _, ok, err := s.LoadDailyEnergy("2026-10-30"); err != nil || ok { + t.Fatalf("a UTC day survived the move to Stockholm: ok=%v err=%v", ok, err) + } + if cleared, err := s.KeepDailyZone(ctx, "Europe/Stockholm"); err != nil || cleared { + t.Fatalf("after the move: cleared=%v err=%v", cleared, err) + } +} diff --git a/go/internal/state/device_lookup_cache_test.go b/go/internal/state/device_lookup_cache_test.go new file mode 100644 index 000000000..279227915 --- /dev/null +++ b/go/internal/state/device_lookup_cache_test.go @@ -0,0 +1,90 @@ +package state + +import ( + "database/sql" + "errors" + "path/filepath" + "testing" + "time" +) + +func sqlLookupByDriverName(t *testing.T, s *Store, name string) string { + t.Helper() + var id string + err := s.db.QueryRow(`SELECT device_id FROM devices WHERE driver_name = ? + ORDER BY last_seen_ms DESC, device_id ASC LIMIT 1`, name).Scan(&id) + if errors.Is(err, sql.ErrNoRows) { + return "" + } + if err != nil { + t.Fatal(err) + } + return id +} + +func cacheLookupID(s *Store, name string) string { + if d := s.LookupDeviceByDriverName(name); d != nil { + return d.DeviceID + } + return "" +} + +// The cache must give the same answer as the old SQL query across +// renames, swaps, re-registration and a reopen (loadDeviceCache). +func TestLookupDeviceByDriverNameCacheMatchesSQL(t *testing.T) { + path := filepath.Join(t.TempDir(), "s.db") + s, err := Open(path) + if err != nil { + t.Fatal(err) + } + reg := func(d Device) { + t.Helper() + if _, err := s.RegisterDevice(d); err != nil { + t.Fatal(err) + } + time.Sleep(3 * time.Millisecond) + } + names := []string{"battery", "pv", "meter", "ghost"} + check := func(stage string) { + t.Helper() + for _, n := range names { + if got, want := cacheLookupID(s, n), sqlLookupByDriverName(t, s, n); got != want { + t.Fatalf("%s: %s cache=%q sql=%q", stage, n, got, want) + } + } + } + reg(Device{DriverName: "battery", Make: "Sungrow", Serial: "A1"}) + reg(Device{DriverName: "pv", Make: "Fronius", Serial: "P1"}) + check("initial") + // Hardware swap: a new battery behind the same driver name. + reg(Device{DriverName: "battery", Make: "Sungrow", Serial: "B2"}) + check("swap") + if cacheLookupID(s, "battery") != "sungrow:B2" { + t.Fatalf("swap: newest device not chosen: %q", cacheLookupID(s, "battery")) + } + // Rename: the old device moves to a new driver name. + reg(Device{DriverName: "meter", Make: "Sungrow", Serial: "A1"}) + check("rename") + // Old device returns under its old name. + reg(Device{DriverName: "battery", Make: "Sungrow", Serial: "A1"}) + check("return") + // Re-register with only a MAC as a different id. + reg(Device{DriverName: "pv", MAC: "aa:bb:cc:dd:ee:ff"}) + check("mac") + if err := s.Close(); err != nil { + t.Fatal(err) + } + s, err = Open(path) + if err != nil { + t.Fatal(err) + } + defer s.Close() + check("reopen") + // batteryModelKey still resolves through the cache after reopen. + if k := s.batteryModelKey("battery"); k != "sungrow:A1" { + t.Fatalf("batteryModelKey=%q", k) + } + if k := s.batteryModelKey("ghost"); k != "ghost" { + t.Fatalf("batteryModelKey(ghost)=%q", k) + } +} diff --git a/go/internal/state/device_lookup_pool_test.go b/go/internal/state/device_lookup_pool_test.go new file mode 100644 index 000000000..95ca14228 --- /dev/null +++ b/go/internal/state/device_lookup_pool_test.go @@ -0,0 +1,62 @@ +package state + +import ( + "context" + "database/sql" + "testing" + "time" +) + +// LookupDeviceByDriverName (called every control tick through +// forecastCurtailment.PendingDrivers) must answer from the in-memory +// deviceCache, so a busy four-connection state.db pool cannot stall the +// tick. +func TestLookupDeviceByDriverNameDoesNotWaitForStatePool(t *testing.T) { + s := openTestStore(t) + if _, err := s.RegisterDevice(Device{DriverName: "battery", Make: "Sungrow", Serial: "A1"}); err != nil { + t.Fatal(err) + } + + // Occupy every pooled connection, standing in for config writes, + // forecast archive/prune and backup. + ctx := context.Background() + var held []*sql.Conn + for i := 0; i < 4; i++ { + c, err := s.db.Conn(ctx) + if err != nil { + t.Fatal(err) + } + held = append(held, c) + } + release := func() { + for _, c := range held { + _ = c.Close() + } + held = nil + } + defer release() + + // The cache already has the answer, without touching SQL. + cacheStart := time.Now() + var cached string + for _, d := range s.CachedDevices() { + if d.DriverName == "battery" { + cached = d.DeviceID + } + } + if cached != "sungrow:A1" { + t.Fatalf("cache device id = %q", cached) + } + t.Logf("CachedDevices answered in %v", time.Since(cacheStart)) + + done := make(chan *Device, 1) + go func() { done <- s.LookupDeviceByDriverName("battery") }() + select { + case d := <-done: + t.Logf("lookup returned %+v", d) + case <-time.After(time.Second): + release() + d := <-done + t.Fatalf("LookupDeviceByDriverName waited over 1s for a state.db connection (returned %+v only after the pool was released)", d) + } +} diff --git a/go/internal/state/devices.go b/go/internal/state/devices.go index d9138469f..2fcd1a809 100644 --- a/go/internal/state/devices.go +++ b/go/internal/state/devices.go @@ -1,7 +1,6 @@ package state import ( - "database/sql" "errors" "fmt" "strings" @@ -215,21 +214,23 @@ func (s *Store) CachedDevices() []Device { // LookupDeviceByDriverName finds the most recently-seen device bound to a // given driver name. Returns nil if no device has been registered yet -// (cold start before the driver has reported its identity). +// (cold start before the driver has reported its identity). It reads the +// committed identity cache, so the control tick never waits for SQLite. func (s *Store) LookupDeviceByDriverName(name string) *Device { - row := s.db.QueryRow(`SELECT device_id, driver_name, - COALESCE(make,''), COALESCE(serial,''), COALESCE(mac,''), COALESCE(endpoint,''), - first_seen_ms, last_seen_ms - FROM devices WHERE driver_name = ? ORDER BY last_seen_ms DESC LIMIT 1`, name) - var d Device - if err := row.Scan(&d.DeviceID, &d.DriverName, &d.Make, &d.Serial, &d.MAC, &d.Endpoint, - &d.FirstSeenMs, &d.LastSeenMs); err != nil { - if errors.Is(err, sql.ErrNoRows) { - return nil + s.deviceCacheMu.RLock() + defer s.deviceCacheMu.RUnlock() + var found *Device + for _, device := range s.deviceCache { + if device.DriverName != name { + continue + } + if found == nil || device.LastSeenMs > found.LastSeenMs || + (device.LastSeenMs == found.LastSeenMs && device.DeviceID < found.DeviceID) { + d := device + found = &d } - return nil } - return &d + return found } // AllDevices returns every registered device, most recently seen first. diff --git a/go/internal/state/energy_ledger.go b/go/internal/state/energy_ledger.go index 4b4f9c5cf..84f3ee317 100644 --- a/go/internal/state/energy_ledger.go +++ b/go/internal/state/energy_ledger.go @@ -24,8 +24,11 @@ const ( // Keep recent ledger detail aligned with the legacy history hot tier. // Older energy is additive, so hourly totals preserve its contract while // bounding history growth. Daily totals remain after the hourly horizon. + // Skatteverket's battery refund is worked out hour by hour for a + // calendar year and may be claimed until three years after it ends, so + // the hours of a whole year must last four. EnergyLedgerDetailedRetention = 30 * 24 * time.Hour - EnergyLedgerRetention = 2 * 365 * 24 * time.Hour + EnergyLedgerRetention = 4 * 366 * 24 * time.Hour EnergyLedgerRollupBucketMS = int64(time.Hour / time.Millisecond) EnergyLedgerDailyBucketMS = 24 * EnergyLedgerRollupBucketMS ) @@ -505,7 +508,7 @@ func (s *Store) LoadEnergyHistoryContext(ctx context.Context, q EnergyHistoryQue } // PruneEnergyLedger keeps five-minute detail for 30 days, hourly totals for -// two years and daily totals thereafter. Energy and its source/quality remain +// four years and daily totals thereafter. Energy and its source/quality remain // intact. Each completed hour or day commits before the next begins. func (s *Store) PruneEnergyLedger(ctx context.Context, now time.Time) (hourly, daily int64, err error) { for _, tier := range []struct { diff --git a/go/internal/state/heal.go b/go/internal/state/heal.go index a13976ec8..091a7631e 100644 --- a/go/internal/state/heal.go +++ b/go/internal/state/heal.go @@ -15,12 +15,15 @@ import ( "log/slog" "os" "strings" + "time" + + "modernc.org/sqlite" ) // HealEvent records a corruption-recovery action taken at boot, for surfacing // on /api/health. Zero events means a clean boot. type HealEvent struct { - Tier string `json:"tier"` // "state" | "cache" + Tier string `json:"tier"` // "state" | "cache" | "history" Action string `json:"action"` // "rebuilt" | "restored" Detail string `json:"detail"` AtMs int64 `json:"at_ms"` @@ -29,6 +32,8 @@ type HealEvent struct { const ( tierState = "state" tierCache = "cache" + // history.db has no snapshot; a damaged file is rebuilt empty (history_sqlite.go). + tierHistory = "history" healRebuilt = "rebuilt" healRestored = "restored" @@ -202,6 +207,30 @@ func openChecked(path, tier string, nowMs int64) (*sql.DB, *HealEvent, error) { return db, ev, nil } +// HealBeforeLoad restores state.db from its snapshot when its settings cannot +// be read because the file is corrupt. Boot reads settings before Open, so a +// damaged file would otherwise stop every start before the heal in Open runs. +// The verified-good marker cannot skip this check, because it starts from a +// failed read rather than the marker. Without a usable snapshot it changes +// nothing. The caller adds the event to the Store with RecordHealEvent. +func HealBeforeLoad(path string) (*HealEvent, error) { + _, readErr := ReadConfiguration(path) + var sqliteErr *sqlite.Error + if !errors.As(readErr, &sqliteErr) || (sqliteErr.Code()&0xff != 11 && sqliteErr.Code()&0xff != 26) { + return nil, nil // readable, missing or busy: not corruption + } + if !snapshotUsable(path + ".snapshot") { + slog.Error("state: settings database is corrupt and has no usable snapshot", "path", path, "err", readErr) + return nil, nil + } + _ = os.Remove(cleanMarkerPath(path)) + db, ev, err := openChecked(path, tierState, time.Now().UnixMilli()) + if err != nil { + return nil, err + } + return ev, db.Close() +} + // quarantine renames the corrupt DB and its WAL/shm sidecars out of the way so // a fresh file can take their place. Missing sidecars are skipped. func quarantine(path string, nowMs int64) error { diff --git a/go/internal/state/heal_before_load_test.go b/go/internal/state/heal_before_load_test.go new file mode 100644 index 000000000..19654c350 --- /dev/null +++ b/go/internal/state/heal_before_load_test.go @@ -0,0 +1,37 @@ +package state + +import ( + "bytes" + "os" + "path/filepath" + "testing" +) + +// Without a usable snapshot the pre-load heal leaves a damaged state.db in +// place: the owner's settings are still in it, and Load reports the error. +func TestHealBeforeLoadWithoutSnapshotChangesNothing(t *testing.T) { + path := filepath.Join(t.TempDir(), "state.db") + damaged := bytes.Repeat([]byte{0xAB}, 8192) + if err := os.WriteFile(path, damaged, 0600); err != nil { + t.Fatal(err) + } + ev, err := HealBeforeLoad(path) + if err != nil || ev != nil { + t.Fatalf("heal = %+v, %v", ev, err) + } + got, err := os.ReadFile(path) + if err != nil || !bytes.Equal(got, damaged) { + t.Fatalf("damaged state.db was moved or changed: %v", err) + } +} + +func TestHealBeforeLoadIgnoresMissingDatabase(t *testing.T) { + path := filepath.Join(t.TempDir(), "state.db") + ev, err := HealBeforeLoad(path) + if err != nil || ev != nil { + t.Fatalf("heal = %+v, %v", ev, err) + } + if _, err := os.Stat(path); !os.IsNotExist(err) { + t.Fatalf("missing state.db was created: %v", err) + } +} diff --git a/go/internal/state/history_absorb_unreadable_test.go b/go/internal/state/history_absorb_unreadable_test.go new file mode 100644 index 000000000..12cb452f7 --- /dev/null +++ b/go/internal/state/history_absorb_unreadable_test.go @@ -0,0 +1,181 @@ +package state + +import ( + "context" + "os" + "path/filepath" + "slices" + "testing" + "time" +) + +func unreadableBucket(at time.Time, v float64) metricBucket { + return metricBucket{Driver: "meter", Metric: "grid_w", BucketSummary: BucketSummary{StartMS: at.UnixMilli(), ResolutionMS: ArchiveResolutionMS, FirstMS: at.UnixMilli(), LastMS: at.UnixMilli() + 1000, N: 1, Sum: v, Min: v, Max: v, Last: v}} +} + +// unreadableColdFixture opens a store the way main.go does, with the same +// cold folder for reads and for the boot import, so runtime readers see +// whatever the import leaves behind. +func unreadableColdFixture(t *testing.T) (*Store, string, time.Time) { + t.Helper() + s := freshStore(t) + if err := s.EnableHistoryAggregation(); err != nil { + t.Fatal(err) + } + cold := t.TempDir() + s.coldDir = cold + return s, cold, time.Now().UTC().Add(-40 * 24 * time.Hour).Truncate(24 * time.Hour) +} + +func dayFile(t *testing.T, cold string, day time.Time, suffix string) string { + t.Helper() + path := filepath.Join(cold, day.Format("2006/01"), day.Format("02")+suffix) + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + return path +} + +func assertFile(t *testing.T, path string, want bool) { + t.Helper() + _, err := os.Stat(path) + if want && err != nil { + t.Fatalf("%s must stay: %v", path, err) + } + if !want && !os.IsNotExist(err) { + t.Fatalf("%s must be gone: %v", path, err) + } +} + +func listedUnreadable(s *Store) []string { + listed, _ := s.HistoryBackend()["unreadable_cold_files"].([]string) + return listed +} + +// One damaged bucket day must not stop boot-time absorb of the other days. +// main.go exits on an AbsorbColdHistory error, and the file would stay, so +// every restart would fail the same way. The kept file must also leave the +// day's history reads working. +func TestAbsorbColdHistorySkipsUnreadableBucketFile(t *testing.T) { + damage := map[string]func(t *testing.T, path string){ + "truncated": func(t *testing.T, path string) { + info, err := os.Stat(path) + if err != nil { + t.Fatal(err) + } + if err := os.Truncate(path, info.Size()/2); err != nil { + t.Fatal(err) + } + }, + "no read permission": func(t *testing.T, path string) { + if os.Geteuid() == 0 { + t.Skip("root reads files without permission") + } + if err := os.Chmod(path, 0); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = os.Chmod(path, 0o644); _ = os.Chmod(path+unreadableColdSuffix, 0o644) }) + }, + } + for name, spoil := range damage { + t.Run(name, func(t *testing.T) { + s, cold, good := unreadableColdFixture(t) + bad := good.Add(24 * time.Hour) + hour := good.Add(3 * time.Hour) + badHour := bad.Add(5 * time.Hour) + goodPath := dayFile(t, cold, good, ".buckets.parquet") + badPath := dayFile(t, cold, bad, ".buckets.parquet") + legacyPath := dayFile(t, cold, bad, ".legacy-buckets.parquet") + if err := writeTestBuckets(goodPath, []metricBucket{unreadableBucket(hour, 10)}); err != nil { + t.Fatal(err) + } + if err := writeTestBuckets(badPath, []metricBucket{unreadableBucket(bad, 5)}); err != nil { + t.Fatal(err) + } + spoil(t, badPath) + // The older copy of the damaged day is readable and serves it. + if err := writeTestBuckets(legacyPath, []metricBucket{unreadableBucket(badHour, 7)}); err != nil { + t.Fatal(err) + } + + for boot := 1; boot <= 2; boot++ { + if err := s.AbsorbColdHistory(context.Background(), cold); err != nil { + t.Fatalf("boot %d: AbsorbColdHistory: %v", boot, err) + } + assertFile(t, goodPath, false) + assertFile(t, badPath, false) + assertFile(t, badPath+unreadableColdSuffix, true) + assertFile(t, legacyPath, true) + if want := bad.Format("2006/01/02") + ".buckets.parquet" + unreadableColdSuffix; !slices.Equal(listedUnreadable(s), []string{want}) { + t.Fatalf("boot %d: unreadable_cold_files = %v, want [%s]", boot, listedUnreadable(s), want) + } + for _, want := range []struct { + at time.Time + v float64 + }{{hour, 10}, {badHour, 7}} { + points, err := s.LoadSeriesBucketsOrRaw("meter", "grid_w", want.at.UnixMilli(), want.at.UnixMilli()+HistoryHourResolutionMS-1, 0) + if err != nil || len(points) != 1 || points[0].V != want.v { + t.Fatalf("boot %d: hour %s = %+v, %v; want one point of %v", boot, want.at, points, err, want.v) + } + } + from, to := good.UnixMilli(), bad.Add(48*time.Hour).UnixMilli() + for _, maxPoints := range []int{0, 200} { + for _, metric := range []string{"grid_w", "other_w"} { + if _, err := s.LoadSeriesBucketsOrRaw("meter", metric, from, to, maxPoints); err != nil { + t.Fatalf("boot %d: %s range over the damaged day, maxPoints=%d: %v", boot, metric, maxPoints, err) + } + } + } + now := time.Now() + if _, err := s.LoadSeriesBuckets("meter", "grid_w", now.Add(-365*24*time.Hour).UnixMilli(), now.UnixMilli(), 365); err != nil { + t.Fatalf("boot %d: year chart: %v", boot, err) + } + } + }) + } +} + +// An operator who puts back a good copy of the damaged file must get a +// clean boot. Had the legacy copy imported the same hour, the restored file +// would conflict with it and stop boot again. +func TestAbsorbColdHistoryImportsRestoredBucketFile(t *testing.T) { + s, cold, day := unreadableColdFixture(t) + at := day.Add(2 * time.Hour) + orig := dayFile(t, cold, day, ".buckets.parquet") + legacy := dayFile(t, cold, day, ".legacy-buckets.parquet") + full := []metricBucket{unreadableBucket(at, 3), unreadableBucket(at.Add(time.Minute), 4)} + if err := writeTestBuckets(orig, full); err != nil { + t.Fatal(err) + } + good, err := os.ReadFile(orig) + if err != nil { + t.Fatal(err) + } + if err := os.Truncate(orig, int64(len(good)/2)); err != nil { + t.Fatal(err) + } + if err := writeTestBuckets(legacy, full[:1]); err != nil { + t.Fatal(err) + } + if err := s.AbsorbColdHistory(context.Background(), cold); err != nil { + t.Fatalf("first boot: %v", err) + } + if err := os.Remove(orig + unreadableColdSuffix); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(orig, good, 0o644); err != nil { + t.Fatal(err) + } + if err := s.AbsorbColdHistory(context.Background(), cold); err != nil { + t.Fatalf("boot after restoring the file: %v", err) + } + assertFile(t, orig, false) + assertFile(t, legacy, false) + if listed := listedUnreadable(s); len(listed) != 0 { + t.Fatalf("unreadable_cold_files = %v after recovery", listed) + } + points, err := s.LoadSeriesBucketsOrRaw("meter", "grid_w", at.UnixMilli(), at.UnixMilli()+HistoryHourResolutionMS-1, 0) + if err != nil || len(points) != 1 || points[0].N != 2 { + t.Fatalf("restored hour = %+v, %v; want one point from both minutes", points, err) + } +} diff --git a/go/internal/state/history_heal_test.go b/go/internal/state/history_heal_test.go new file mode 100644 index 000000000..8d8e5c98e --- /dev/null +++ b/go/internal/state/history_heal_test.go @@ -0,0 +1,285 @@ +package state + +import ( + "database/sql" + "os" + "path/filepath" + "testing" +) + +func seedHistoryStore(t *testing.T) string { + t.Helper() + path := filepath.Join(t.TempDir(), "state.db") + s, err := Open(path) + if err != nil { + t.Fatalf("first open: %v", err) + } + if err := s.RecordHistory(HistoryPoint{TsMs: 1}); err != nil { + t.Fatal(err) + } + if err := s.Close(); err != nil { + t.Fatal(err) + } + return path +} + +// reopenHealed opens a store whose history.db was damaged or removed and +// checks that Core starts with an empty, bound history and reports the loss. +func reopenHealed(t *testing.T, path string, wantQuarantine bool) { + t.Helper() + s, err := Open(path) + if err != nil { + t.Fatalf("Open with damaged history.db failed: %v", err) + } + defer s.Close() + events := s.HealEvents() + if len(events) != 1 || events[0].Tier != tierHistory || events[0].Action != healRebuilt { + t.Fatalf("heal events = %+v, want one history rebuilt event", events) + } + if err := s.RecordHistory(HistoryPoint{TsMs: 2}); err != nil { + t.Fatalf("fresh history does not accept writes: %v", err) + } + active, err := s.historyConfig("history_sqlite_generation") + if err != nil || active == "" { + t.Fatalf("fresh history is not bound: %q %v", active, err) + } + moved, _ := filepath.Glob(historyDatabasePath(path) + ".corrupt-*") + if wantQuarantine && len(moved) == 0 { + t.Fatal("damaged history.db was not kept for manual recovery") + } + if err := s.Close(); err != nil { + t.Fatal(err) + } + again, err := Open(path) + if err != nil { + t.Fatalf("second open after heal: %v", err) + } + if n := len(again.HealEvents()); n != 0 { + t.Fatalf("second open reported %d heal events, want 0", n) + } + again.Close() +} + +// A torn history.db must not stop Core from opening. +func TestCorruptHistoryHealsOnOpen(t *testing.T) { + path := seedHistoryStore(t) + hist := historyDatabasePath(path) + f, err := os.OpenFile(hist, os.O_WRONLY, 0) + if err != nil { + t.Fatal(err) + } + // Overwrite the 100-byte SQLite header (torn page 1). + if _, err := f.WriteAt(make([]byte, 100), 0); err != nil { + t.Fatal(err) + } + f.Close() + os.Remove(hist + "-wal") + os.Remove(hist + "-shm") + reopenHealed(t, path, true) +} + +// Removing history.db by hand must not stop Core from opening. +func TestMissingHistoryHealsOnOpen(t *testing.T) { + path := seedHistoryStore(t) + hist := historyDatabasePath(path) + for _, p := range []string{hist, hist + "-wal", hist + "-shm"} { + os.Remove(p) + } + reopenHealed(t, path, false) +} + +// A readable history.db from another generation is a wrong file, not damage. +func TestForeignHistoryGenerationStillRefused(t *testing.T) { + path := seedHistoryStore(t) + s, err := Open(path) + if err != nil { + t.Fatal(err) + } + if err := s.SaveConfig("history_sqlite_generation", "unrelated"); err != nil { + t.Fatal(err) + } + if err := s.Close(); err != nil { + t.Fatal(err) + } + if reopened, err := Open(path); err == nil { + reopened.Close() + t.Fatal("accepted a history.db bound to another generation") + } +} + +// corruptTablePage overwrites the root page of one table inside history.db. +func corruptTablePage(t *testing.T, hist, table string) { + t.Helper() + db, err := sql.Open("sqlite", hist) + if err != nil { + t.Fatal(err) + } + var root, pageSize int64 + if err := db.QueryRow(`SELECT rootpage FROM sqlite_master WHERE name=?`, table).Scan(&root); err != nil { + t.Fatalf("rootpage %s: %v", table, err) + } + if err := db.QueryRow(`PRAGMA page_size`).Scan(&pageSize); err != nil { + t.Fatal(err) + } + if _, err := db.Exec(`PRAGMA wal_checkpoint(TRUNCATE)`); err != nil { + t.Fatal(err) + } + db.Close() + os.Remove(hist + "-wal") + os.Remove(hist + "-shm") + f, err := os.OpenFile(hist, os.O_WRONLY, 0) + if err != nil { + t.Fatal(err) + } + junk := make([]byte, pageSize) + for i := range junk { + junk[i] = 0xA5 + } + if _, err := f.WriteAt(junk, (root-1)*pageSize); err != nil { + t.Fatal(err) + } + f.Close() +} + +// A damaged data page read by a later startup query must heal too, not only +// damage seen by the schema step. +func TestCorruptHistoryDataPageHealsOnOpen(t *testing.T) { + for _, table := range []string{"history_source_cursor", "energy_ledger_meta"} { + t.Run(table, func(t *testing.T) { + path := seedHistoryStore(t) + corruptTablePage(t, historyDatabasePath(path), table) + reopenHealed(t, path, true) + }) + } +} + +// A torn half-size file heals. +func TestTruncatedHistoryHealsOnOpen(t *testing.T) { + path := seedHistoryStore(t) + hist := historyDatabasePath(path) + os.Remove(hist + "-wal") + os.Remove(hist + "-shm") + fi, err := os.Stat(hist) + if err != nil { + t.Fatal(err) + } + if err := os.Truncate(hist, fi.Size()/2); err != nil { + t.Fatal(err) + } + reopenHealed(t, path, true) +} + +// SQLite opens a 0-byte file as an empty database. With no marker it cannot +// be a wrong file from a restore, so it heals like a missing file. +func TestZeroLengthHistoryHealsOnOpen(t *testing.T) { + path := seedHistoryStore(t) + hist := historyDatabasePath(path) + os.Remove(hist + "-wal") + os.Remove(hist + "-shm") + if err := os.Truncate(hist, 0); err != nil { + t.Fatal(err) + } + reopenHealed(t, path, true) +} + +// convertedBetaStore marks a seeded site as one that converted beta history. +func convertedBetaStore(t *testing.T) string { + t.Helper() + path := seedHistoryStore(t) + s, err := Open(path) + if err != nil { + t.Fatal(err) + } + active, err := s.historyConfig("history_sqlite_generation") + if err != nil { + t.Fatal(err) + } + if err := s.SaveConfig("history_duckdb_generation", active); err != nil { + t.Fatal(err) + } + if _, err := s.history.Exec(`INSERT INTO history_migrations(name) VALUES (?)`, convertedChartSummaries); err != nil { + t.Fatal(err) + } + if err := s.Close(); err != nil { + t.Fatal(err) + } + return path +} + +// A site that already converted beta history keeps history_duckdb_generation. +// Its history must heal like any other, and the fresh file must not be taken +// for beta history that still needs conversion. +func TestConvertedBetaHistoryHealsOnOpen(t *testing.T) { + damage := map[string]func(t *testing.T, hist string){ + "missing": func(t *testing.T, hist string) { + for _, p := range []string{hist, hist + "-wal", hist + "-shm"} { + os.Remove(p) + } + }, + "header": func(t *testing.T, hist string) { + os.Remove(hist + "-wal") + os.Remove(hist + "-shm") + f, err := os.OpenFile(hist, os.O_WRONLY, 0) + if err != nil { + t.Fatal(err) + } + if _, err := f.WriteAt(make([]byte, 100), 0); err != nil { + t.Fatal(err) + } + f.Close() + }, + "data page": func(t *testing.T, hist string) { corruptTablePage(t, hist, "energy_ledger_meta") }, + "zero length": func(t *testing.T, hist string) { + os.Remove(hist + "-wal") + os.Remove(hist + "-shm") + if err := os.Truncate(hist, 0); err != nil { + t.Fatal(err) + } + }, + } + for name, apply := range damage { + t.Run(name, func(t *testing.T) { + path := convertedBetaStore(t) + apply(t, historyDatabasePath(path)) + reopenHealed(t, path, name != "missing") + s, err := Open(path) + if err != nil { + t.Fatal(err) + } + defer s.Close() + if beta, err := s.historyConfig("history_duckdb_generation"); err != nil || beta != "" { + t.Fatalf("beta key after heal = %q, %v; want cleared", beta, err) + } + }) + } +} + +// The restore path keeps refusing damage: the restored file is the only copy. +func TestRestoreGenerationCorruptStillRefused(t *testing.T) { + path := seedHistoryStore(t) + s, err := Open(path) + if err != nil { + t.Fatal(err) + } + gen, err := s.historyConfig("history_sqlite_generation") + if err != nil { + t.Fatal(err) + } + if err := s.SaveConfig("history_restore_generation", gen); err != nil { + t.Fatal(err) + } + s.Close() + hist := historyDatabasePath(path) + os.Remove(hist + "-wal") + os.Remove(hist + "-shm") + f, err := os.OpenFile(hist, os.O_WRONLY, 0) + if err != nil { + t.Fatal(err) + } + f.WriteAt(make([]byte, 100), 0) + f.Close() + if s, err := Open(path); err == nil { + s.Close() + t.Fatal("restore path healed a damaged history.db") + } +} diff --git a/go/internal/state/history_plain.go b/go/internal/state/history_plain.go index 225f2e069..bc3543488 100644 --- a/go/internal/state/history_plain.go +++ b/go/internal/state/history_plain.go @@ -9,6 +9,7 @@ import ( "math" "os" "path/filepath" + "slices" "strconv" "strings" "time" @@ -200,6 +201,13 @@ func (s *Store) RetireRawHistory(ctx context.Context) error { // goes with them, and so does a sample day whose hourly summary receipt // matches the file. Other sample days stay readable until the background // rollup summarizes them. Diagnostics files are left in place. +// +// An unreadable bucket file never stops boot. It moves aside under +// unreadableColdSuffix, out of every reader's glob, so reads for that day +// still work. It stays on disk, and HistoryBackend lists it. The day's legacy +// copy stays on disk unimported and serves reads. A good copy of the original +// restored under its normal name imports on the next boot, and the legacy +// copy then goes without a conflict. func (s *Store) AbsorbColdHistory(ctx context.Context, cold string) error { if s == nil || s.history == nil || strings.TrimSpace(cold) == "" { return nil @@ -213,19 +221,49 @@ func (s *Store) AbsorbColdHistory(ctx context.Context, cold string) error { return err } var imported int + unreadable := map[string]bool{} + // held lists files kept at their normal name: an unreadable file that + // could not move aside, or the legacy copy of an unreadable original. + held := map[string]bool{} + skip := func(path string, err error) { + unreadable[path] = true + aside := path + unreadableColdSuffix + if _, statErr := os.Lstat(aside); errors.Is(statErr, os.ErrNotExist) { + renameErr := os.Rename(path, aside) + if renameErr == nil { + slog.Warn("history bucket file unreadable; moved aside", "path", aside, "err", err) + return + } + slog.Warn("history bucket file could not move aside", "path", path, "err", renameErr) + } + slog.Warn("history bucket file unreadable; kept in place", "path", path, "err", err) + held[path] = true + } for _, path := range paths { if err := ctx.Err(); err != nil { return err } if strings.Contains(path, ".legacy-buckets.") { original := strings.Replace(path, ".legacy-buckets.parquet", ".buckets.parquet", 1) - if _, statErr := os.Stat(original); statErr == nil { + if _, statErr := os.Stat(original); statErr == nil && !unreadable[original] { + continue + } + // Importing these hours would conflict with a good copy of the + // original restored later, and stop that boot. Keep the file; + // it serves reads for the day until then. + if _, statErr := os.Lstat(original + unreadableColdSuffix); statErr == nil || unreadable[original] { + slog.Warn("history legacy bucket file kept while its original is unreadable", "path", path) + held[path] = true continue } } digest, err := historyFileHashContext(ctx, path) if err != nil { - return err + if ctx.Err() != nil { + return ctx.Err() + } + skip(path, err) + continue } receipt := "plain-archive:" + digest hours := map[string]*namedHour{} @@ -240,7 +278,11 @@ func (s *Store) AbsorbColdHistory(ctx context.Context, cold string) error { h.sources = append(h.sources, b.BucketSummary) return nil }); err != nil { - return fmt.Errorf("read %s: %w", filepath.Base(path), err) + if ctx.Err() != nil { + return ctx.Err() + } + skip(path, err) + continue } if err := s.writeHourBuckets(ctx, hours, receipt); err != nil { return err @@ -251,6 +293,27 @@ func (s *Store) AbsorbColdHistory(ctx context.Context, cold string) error { if err != nil { return err } + sampleDays := len(keep) + // Earlier boots moved unreadable files aside; list them on every boot. + aside, err := filepath.Glob(filepath.Join(cold, "[0-9][0-9][0-9][0-9]", "[0-9][0-9]", "[0-9][0-9].*"+unreadableColdSuffix)) + if err != nil { + return err + } + var kept []string + for _, path := range aside { + keep[path] = true + kept = append(kept, coldRel(path)) + } + for path := range held { + keep[path] = true + if unreadable[path] { + kept = append(kept, coldRel(path)) + } + } + slices.Sort(kept) + s.maintenanceStatusMu.Lock() + s.coldUnreadable = kept + s.maintenanceStatusMu.Unlock() if err := removeColdHistory(cold, keep); err != nil { return err } @@ -258,13 +321,22 @@ func (s *Store) AbsorbColdHistory(ctx context.Context, cold string) error { if err != nil { return err } - if len(keep) > 0 { - slog.Warn("history sample days kept until they have hourly summaries", "days", len(keep)) + if sampleDays > 0 { + slog.Warn("history sample days kept until they have hourly summaries", "days", sampleDays) } slog.Info("history cold files absorbed", "days", imported, "retired_files", removed) return nil } +// unreadableColdSuffix marks a bucket file that failed to read. The name no +// longer ends in .parquet, so no reader, importer or converter opens it. +const unreadableColdSuffix = ".unreadable" + +// coldRel names a cold file as YYYY/MM/. +func coldRel(path string) string { + return filepath.Base(filepath.Dir(filepath.Dir(path))) + "/" + filepath.Base(filepath.Dir(path)) + "/" + filepath.Base(path) +} + // A sample day counts as absorbed only through the summary receipt for this // exact file. 2.x wrote these days and never summarized them. func (s *Store) unsummarizedSampleDays(ctx context.Context, cold string) (map[string]bool, error) { diff --git a/go/internal/state/history_primary_test.go b/go/internal/state/history_primary_test.go index 305dd0e0e..c324a43bd 100644 --- a/go/internal/state/history_primary_test.go +++ b/go/internal/state/history_primary_test.go @@ -170,8 +170,8 @@ func TestHistoryWriterRetriesFailedTransaction(t *testing.T) { } } -func TestHistoryMissingOrUnboundPrimaryFails(t *testing.T) { - for _, kind := range []string{"missing", "unbound", "wrong-pending", "incomplete"} { +func TestHistoryUnboundPrimaryFails(t *testing.T) { + for _, kind := range []string{"unbound", "wrong-pending", "incomplete"} { t.Run(kind, func(t *testing.T) { path := filepath.Join(t.TempDir(), "state.db") s, err := Open(path) @@ -199,14 +199,9 @@ func TestHistoryMissingOrUnboundPrimaryFails(t *testing.T) { if err := s.Close(); err != nil { t.Fatal(err) } - if kind == "missing" { - if err := os.Remove(historyDatabasePath(path)); err != nil { - t.Fatal(err) - } - } if reopened, err := Open(path); err == nil { reopened.Close() - t.Fatal("silently accepted missing or unbound history") + t.Fatal("silently accepted unbound history") } }) } diff --git a/go/internal/state/history_sqlite.go b/go/internal/state/history_sqlite.go index 1a798e3f8..07006edde 100644 --- a/go/internal/state/history_sqlite.go +++ b/go/internal/state/history_sqlite.go @@ -9,13 +9,16 @@ import ( "fmt" "hash" "io" + "log/slog" "math" "os" "path/filepath" + "slices" "strings" "time" "github.com/google/uuid" + "modernc.org/sqlite" ) const HistoryFilename = "history.db" @@ -37,9 +40,63 @@ func ensureHistorySchema(exec func(string) error) error { return nil } +var errHistoryMissing = errors.New("primary SQLite history is missing") + +// historyDamage marks a history.db that SQLite cannot read. +type historyDamage struct{ err error } + +func (e *historyDamage) Error() string { return e.err.Error() } +func (e *historyDamage) Unwrap() error { return e.err } + +// openHistory heals a damaged, empty or missing history.db instead of refusing +// to start: history holds charts and the ledger, never control state. The old +// files move aside as .corrupt-, a fresh file binds a new generation and +// /api/health reports the loss. A readable file with another generation is +// still refused, because that is a wrong file from a restore. +func (s *Store) openHistory() error { + err := s.openHistoryOnce() + var damage *historyDamage + detail := "history.db was missing — started an empty history (charts and energy totals lost)" + switch { + case errors.As(err, &damage): + detail = "history.db was damaged — moved aside and started an empty history (charts and energy totals lost)" + case !errors.Is(err, errHistoryMissing): + return err + } + nowMs := time.Now().UnixMilli() + beta, _ := s.historyConfig("history_duckdb_generation") + slog.Error("state: "+detail, "path", s.historyPath, "err", err, "beta_generation", beta) + if err := quarantine(s.historyPath, nowMs); err != nil { + return err + } + // Both heal paths need a bound SQLite generation, so a beta key here + // belongs to a finished conversion. Left in place, the fresh file would + // be refused as beta history that still needs conversion. + if _, err := s.db.Exec(`DELETE FROM config WHERE key IN ('history_sqlite_generation','history_sqlite_pending_generation','history_duckdb_generation')`); err != nil { + return err + } + s.healEvents = append(s.healEvents, HealEvent{Tier: tierHistory, Action: healRebuilt, Detail: detail, AtMs: nowMs}) + return s.openHistoryOnce() +} + +// historyCorrupt reports whether err, or a quick_check after it, shows that +// SQLite cannot read the file. Busy, full or read-only errors are not damage. +func historyCorrupt(db *sql.DB, err error) bool { + if sqliteCorrupt(err) { + return true + } + ok, qerr := quickCheck(db) + return (qerr == nil && !ok) || sqliteCorrupt(qerr) +} + +func sqliteCorrupt(err error) bool { + var e *sqlite.Error + return errors.As(err, &e) && (e.Code()&0xff == 11 || e.Code()&0xff == 26) // SQLITE_CORRUPT, SQLITE_NOTADB +} + // The beta converter publishes a verified SQLite file before selecting its // generation. Never reopen the frozen pre-beta tables as current history. -func (s *Store) openHistory() error { +func (s *Store) openHistoryOnce() error { s.historyPath = historyDatabasePath(s.mainDBPath) active, err := s.historyConfig("history_sqlite_generation") if err != nil { @@ -61,7 +118,7 @@ func (s *Store) openHistory() error { return errors.New("beta history needs conversion: stop Core and run ftw-history-migrate; keep history.duckdb and history-hot.db") } if _, err := os.Stat(s.historyPath); errors.Is(err, os.ErrNotExist) && active != "" && restore == "" { - return errors.New("primary SQLite history is missing; restore a full backup") + return errHistoryMissing } db, err := openDurableHistory(s.historyPath) if err != nil { @@ -71,24 +128,53 @@ func (s *Store) openHistory() error { ok := false defer func() { if !ok { + // RetireRawHistory may have swapped in a new handle. + if s.history != nil && s.history != db { + s.history.Close() + } db.Close() s.history = nil } }() + // current is the open history.db handle. After a failed swap it is the + // closed old one, whose check fails without reporting damage. + current := func() *sql.DB { + if s.history != nil { + return s.history + } + return db + } + damaged := func(err error) error { + if err != nil && restore == "" && historyCorrupt(current(), err) { + return &historyDamage{err} + } + return err + } + // Steps that also read state.db or other files must not take their + // error code as proof: only a failed check of history.db itself counts. + checkedDamage := func(err error) error { + if err == nil || restore != "" { + return err + } + if ok, qerr := quickCheck(current()); (qerr == nil && !ok) || sqliteCorrupt(qerr) { + return &historyDamage{err} + } + return err + } var sourceTables int if err := db.QueryRow(`SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name IN ('history_source_energy','history_source_cursor','history_source_periods')`).Scan(&sourceTables); err != nil { - return err + return damaged(err) } if sourceTables != 0 && sourceTables != 3 { return errors.New("incomplete history source tables; restore a verified full backup") } if err := ensureHistorySchema(func(stmt string) error { _, err := db.Exec(stmt); return err }); err != nil { - return fmt.Errorf("history schema: %w", err) + return damaged(fmt.Errorf("history schema: %w", err)) } var generation string err = db.QueryRow(`SELECT name FROM history_migrations WHERE name LIKE 'generation:%'`).Scan(&generation) if err != nil && !errors.Is(err, sql.ErrNoRows) { - return err + return damaged(err) } generation = strings.TrimPrefix(generation, "generation:") if restore != "" && generation != "" && generation != restore { @@ -103,6 +189,17 @@ func (s *Store) openHistory() error { ok = true return s.openHistory() } + if active != "" && restore == "" && generation == "" { + // No marker of any kind means an empty file, such as a 0-byte + // history.db after power loss. A wrong file from a restore has one. + var markers int + if err := db.QueryRow(`SELECT COUNT(*) FROM history_migrations`).Scan(&markers); err != nil { + return damaged(err) + } + if markers == 0 { + return &historyDamage{errors.New("history.db has no generation marker")} + } + } if active != "" && restore == "" && active != generation { return errors.New("state and history generations differ; restore a full backup") } @@ -127,7 +224,7 @@ func (s *Store) openHistory() error { } var complete int if err := db.QueryRow(`SELECT COUNT(*) FROM history_migrations WHERE name='sqlite-v1'`).Scan(&complete); err != nil { - return err + return damaged(err) } if complete == 0 && s.historyMigration == nil { return errors.New("history import is incomplete; start Core to resume") @@ -140,17 +237,17 @@ func (s *Store) openHistory() error { return err } if err := s.ensureEnergyLedgerVersion(); err != nil { - return err + return damaged(err) } if err := os.Chmod(s.historyPath, 0600); err != nil { return err } if RetireRawOnOpen { if err := s.recoverLegacyCharts(context.Background(), generation); err != nil { - return err + return checkedDamage(err) } if err := s.RetireRawHistory(context.Background()); err != nil { - return err + return checkedDamage(err) } } // A process stop has no continuous sampling proof, even when it was brief. @@ -160,10 +257,10 @@ func (s *Store) openHistory() error { INSERT INTO history_source_cursor VALUES('stopped','1') ON CONFLICT(key) DO UPDATE SET value=excluded.value; UPDATE history_partial_cursor SET value=0 WHERE key IN ('last','residual_eligible'); UPDATE history_site_cursor SET value=0 WHERE key='last'`); err != nil { - return err + return damaged(err) } if err := s.refreshSourceDayCache(context.Background()); err != nil { - return err + return damaged(err) } ok = true return nil @@ -367,6 +464,11 @@ func historyFloatBits(value float64) uint64 { func (s *Store) HistoryBackend() map[string]any { info := map[string]any{"engine": "sqlite", "archive": "sqlite", "file": filepath.Base(s.historyPath), "writer": s.HistoryWriterStatus(), "migration": s.HistoryMigrationStatus(), "series_hour": s.SeriesHourBackfillStatus()} info["maintenance"] = s.HistoryMaintenanceStatus() + s.maintenanceStatusMu.Lock() + if len(s.coldUnreadable) > 0 { + info["unreadable_cold_files"] = slices.Clone(s.coldUnreadable) + } + s.maintenanceStatusMu.Unlock() if s.aggregateHistory.Load() { info["policy"] = map[string]any{"id": "sqlite-v1", "recent_resolution_ms": HistoryResolutionMS, "recent_retention_hours": int(plainTenSecondKeep.Hours()), "archive_resolution_ms": ArchiveResolutionMS, "archive_minute_days": int(plainMinuteKeep.Hours() / 24), "older_resolution_ms": HistoryHourResolutionMS, "detailed_retention_days": int(plainHourKeep.Hours() / 24)} } @@ -780,7 +882,11 @@ func requireConvertedBeta(path string) error { if err == nil && beta != "" { // An older converter may already have selected SQLite. Check its // summaries before Open can upgrade config or retire raw history. - if _, err := os.Stat(historyDatabasePath(path)); err != nil { + // A missing, empty or unreadable file has nothing left to retire; + // openHistory moves it aside and starts an empty history. + if _, err := os.Stat(historyDatabasePath(path)); errors.Is(err, os.ErrNotExist) { + return nil + } else if err != nil { return fmt.Errorf("converted beta history is unavailable; preserve the originals and restore a full backup: %w", err) } history, err := sql.Open("sqlite", ReadOnlyDatabaseURI(historyDatabasePath(path))) @@ -788,13 +894,29 @@ func requireConvertedBeta(path string) error { return err } defer history.Close() + var tables int + if err := history.QueryRowContext(ctx, `SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name='history_migrations'`).Scan(&tables); err != nil { + if sqliteCorrupt(err) { + return nil + } + return fmt.Errorf("check converted chart summaries: %w", err) + } + if tables == 0 { + return nil + } var summarized int if err := history.QueryRowContext(ctx, `SELECT COUNT(*) FROM history_migrations WHERE name=?`, convertedChartSummaries).Scan(&summarized); err != nil { + if sqliteCorrupt(err) { + return nil + } return fmt.Errorf("check converted chart summaries: %w", err) } if summarized == 0 { var raw int if err := history.QueryRowContext(ctx, `SELECT EXISTS(SELECT 1 FROM ts_samples)`).Scan(&raw); err != nil { + if sqliteCorrupt(err) { + return nil + } return fmt.Errorf("check remaining converted raw history: %w", err) } if raw != 0 { diff --git a/go/internal/state/store.go b/go/internal/state/store.go index 9ab025a89..c41697867 100644 --- a/go/internal/state/store.go +++ b/go/internal/state/store.go @@ -70,7 +70,7 @@ type Store struct { // cache.db gets no marker: it is tiny + disposable, so it is always checked. mainDBPath string - // healMu guards corrupt + verifyCancel. corrupt is set by the background + // healMu guards healEvents, corrupt and verifyCancel. corrupt is set by the background // integrity scan when it fails; Close consults it (a corrupt DB must NOT be // marked clean). verifyCancel interrupts the in-flight background scan so // Close can stop it before closing the DB — otherwise db.Close() blocks on @@ -87,6 +87,7 @@ type Store struct { maintenanceStatusMu sync.Mutex maintenanceStatus HistoryMaintenanceStatus + coldUnreadable []string // guarded by maintenanceStatusMu // offlineBackup is set by OpenBackupSource. Live Core backups yield // between copy batches so goal and control writes stay within latency // limits; the offline helper must not inherit that 100 ms live pause. @@ -430,7 +431,16 @@ func (s *Store) HealEvents() []HealEvent { if s == nil { return nil } - return s.healEvents + s.healMu.Lock() + defer s.healMu.Unlock() + return append([]HealEvent(nil), s.healEvents...) +} + +// RecordHealEvent adds a recovery made outside Open, such as HealBeforeLoad. +func (s *Store) RecordHealEvent(ev HealEvent) { + s.healMu.Lock() + s.healEvents = append(s.healEvents, ev) + s.healMu.Unlock() } // SnapshotTo writes a self-contained, defragmented copy of the database diff --git a/go/internal/state/store_test.go b/go/internal/state/store_test.go index 5b9643c36..2fdf1f1d8 100644 --- a/go/internal/state/store_test.go +++ b/go/internal/state/store_test.go @@ -715,9 +715,15 @@ func TestSnapshotToSkipsTimeSeriesTables(t *testing.T) { if err := snap.db.QueryRow(`SELECT COUNT(*) FROM sqlite_master WHERE name IN ('history_hot','history_warm','history_cold','ts_samples')`).Scan(&count); err != nil || count != 0 { t.Fatalf("compact snapshot retained history tables: %d %v", count, err) } - if partial, err := Open(dst); err == nil { - partial.Close() - t.Fatal("compact snapshot offered empty history as a full restore") + // Opened alone it starts an empty history and says so. + partial, err := Open(dst) + if err != nil { + t.Fatalf("open compact snapshot alone: %v", err) + } + events := partial.HealEvents() + partial.Close() + if len(events) != 1 || events[0].Tier != tierHistory { + t.Fatalf("compact snapshot offered empty history as a full restore: events=%+v", events) } } diff --git a/go/internal/telemetry/store.go b/go/internal/telemetry/store.go index d9dc84309..6bc61d1f7 100644 --- a/go/internal/telemetry/store.go +++ b/go/internal/telemetry/store.go @@ -5,6 +5,7 @@ import ( "fmt" "log/slog" "math" + "slices" "strings" "sync" "time" @@ -688,13 +689,49 @@ func (s *Store) SumOnlineV2XW() float64 { return sum } +// OnlineLowerW returns the lower of a driver's raw and smoothed power of +// one type, while the driver is still reporting and its latest poll +// succeeded. A held reading after a failed poll returns false. +func (s *Store) OnlineLowerW(driver string, t DerType) (float64, bool) { + s.mu.RLock() + defer s.mu.RUnlock() + r, found := s.readings[key(driver, t)] + h, known := s.health[driver] + if !found || !known || !h.TelemetryLive() || h.LastReadFailed() { + return 0, false + } + w := math.Min(r.RawW, r.SmoothedW) + return w, !math.IsNaN(w) && !math.IsInf(w, 0) +} + +// SumOnlineHigherW sums the higher of raw and smoothed power over the +// readings of these types whose drivers are still reporting. +func (s *Store) SumOnlineHigherW(types ...DerType) float64 { + s.mu.RLock() + defer s.mu.RUnlock() + var sum float64 + for _, r := range s.readings { + if h, ok := s.health[r.Driver]; ok && h.TelemetryLive() && slices.Contains(types, r.DerType) { + sum += math.Max(r.RawW, r.SmoothedW) + } + } + return sum +} + // OnlineMeterW returns a site meter's latest power and when it arrived, // while its driver is still reporting. A watchdog-offline meter returns // false, so its last value never counts as a new reading. func (s *Store) OnlineMeterW(driver string) (w float64, at time.Time, ok bool) { + return s.OnlineRawW(driver, DerMeter) +} + +// OnlineRawW returns a driver's latest raw power of one type and when it +// arrived, while the driver is still reporting. A watchdog-offline driver +// returns false. +func (s *Store) OnlineRawW(driver string, t DerType) (w float64, at time.Time, ok bool) { s.mu.RLock() defer s.mu.RUnlock() - r, found := s.readings[key(driver, DerMeter)] + r, found := s.readings[key(driver, t)] if !found || math.IsNaN(r.RawW) || math.IsInf(r.RawW, 0) { return 0, time.Time{}, false } @@ -704,6 +741,38 @@ func (s *Store) OnlineMeterW(driver string) (w float64, at time.Time, ok bool) { return r.RawW, r.UpdatedAt, true } +// OnlineMeterSample is OnlineMeterW with the reading's data, which holds +// the phase values. +func (s *Store) OnlineMeterSample(driver string) (w float64, data json.RawMessage, at time.Time, ok bool) { + s.mu.RLock() + defer s.mu.RUnlock() + r, found := s.readings[key(driver, DerMeter)] + if !found || math.IsNaN(r.RawW) || math.IsInf(r.RawW, 0) { + return 0, nil, time.Time{}, false + } + if h, known := s.health[driver]; !known || !h.TelemetryLive() { + return 0, nil, time.Time{}, false + } + return r.RawW, r.Data, r.UpdatedAt, true +} + +// OnlineRawWByType returns the latest raw power of every driver of one type +// that is still reporting. A watchdog-offline driver is left out. +func (s *Store) OnlineRawWByType(t DerType) map[string]float64 { + s.mu.RLock() + defer s.mu.RUnlock() + out := make(map[string]float64) + for _, r := range s.readings { + if r.DerType != t || math.IsNaN(r.RawW) || math.IsInf(r.RawW, 0) { + continue + } + if h, known := s.health[r.Driver]; known && h.TelemetryLive() { + out[r.Driver] = r.RawW + } + } + return out +} + // ReadingsByDriver returns all readings from one driver. func (s *Store) ReadingsByDriver(driver string) []*DerReading { s.mu.RLock() diff --git a/go/internal/telemetry/telemetry_test.go b/go/internal/telemetry/telemetry_test.go index 640d3d9f5..5bfc63876 100644 --- a/go/internal/telemetry/telemetry_test.go +++ b/go/internal/telemetry/telemetry_test.go @@ -572,6 +572,49 @@ func TestSumOnlineV2XWIsSignedAndSkipsOffline(t *testing.T) { } } +func TestOnlineLowerWNeedsALivePoll(t *testing.T) { + s := NewStore() + s.Update("charger", DerEV, 8000, nil, nil) + s.DriverHealthMut("charger").RecordSuccess() + s.Get("charger", DerEV).SmoothedW = 5000 + if w, ok := s.OnlineLowerW("charger", DerEV); !ok || w != 5000 { + t.Fatalf("got %v %v, want the lower 5000 W", w, ok) + } + s.Get("charger", DerEV).RawW = 4000 + if w, ok := s.OnlineLowerW("charger", DerEV); !ok || w != 4000 { + t.Fatalf("got %v %v, want the lower 4000 W", w, ok) + } + s.DriverHealthMut("charger").RecordError("timeout") + if _, ok := s.OnlineLowerW("charger", DerEV); ok { + t.Fatal("a reading held after a failed poll counted") + } + s.DriverHealthMut("charger").RecordSuccess() + s.DriverHealthMut("charger").SetOffline() + if _, ok := s.OnlineLowerW("charger", DerEV); ok { + t.Fatal("an offline driver's reading counted") + } + if _, ok := s.OnlineLowerW("missing", DerEV); ok { + t.Fatal("a missing reading counted") + } +} + +func TestSumOnlineHigherWSumsTheNamedTypes(t *testing.T) { + s := NewStore() + s.Update("battery", DerBattery, -3000, nil, nil) + s.Update("pv", DerPV, -2000, nil, nil) + s.Update("ev", DerEV, 7000, nil, nil) + s.Update("offline", DerBattery, -9000, nil, nil) + for _, d := range []string{"battery", "pv", "ev"} { + s.DriverHealthMut(d).RecordSuccess() + } + s.DriverHealthMut("offline").SetOffline() + s.Get("battery", DerBattery).SmoothedW = -1000 + s.Get("pv", DerPV).SmoothedW = -2500 + if got := s.SumOnlineHigherW(DerBattery, DerPV); got != -3000 { + t.Fatalf("got %v, want -1000 W battery plus -2000 W PV", got) + } +} + func TestWatchdogPerDriverOverride(t *testing.T) { s := NewStore() // Two drivers: one with the site-default tolerance, one with a @@ -705,3 +748,57 @@ func TestOnlineMeterWSkipsOfflineMeter(t *testing.T) { t.Fatal("an offline meter's last reading was returned") } } + +// A battery's raw power, like the meter's, stops counting once its driver +// goes offline. +func TestOnlineRawWSkipsOfflineBattery(t *testing.T) { + s := NewStore() + s.Update("bat", DerBattery, -3000, nil, nil) + s.DriverHealthMut("bat").RecordSuccess() + if w, _, ok := s.OnlineRawW("bat", DerBattery); !ok || w != -3000 { + t.Fatalf("online battery: %v %v, want -3000 W", w, ok) + } + if _, _, ok := s.OnlineRawW("bat", DerMeter); ok { + t.Fatal("a battery driver reported a meter reading") + } + s.DriverHealthMut("bat").SetOffline() + if _, _, ok := s.OnlineRawW("bat", DerBattery); ok { + t.Fatal("an offline battery's last reading was returned") + } +} + +// The meter's sample comes with its phase data, and both stop once the +// meter goes offline. +func TestOnlineMeterSampleSkipsOfflineMeter(t *testing.T) { + s := NewStore() + data := json.RawMessage(`{"l1_a":2.4}`) + s.Update("meter", DerMeter, 2667, nil, data) + if _, _, _, ok := s.OnlineMeterSample("meter"); ok { + t.Fatal("a meter with no health record was returned") + } + s.DriverHealthMut("meter").RecordSuccess() + w, got, at, ok := s.OnlineMeterSample("meter") + if !ok || w != 2667 || string(got) != string(data) || at.IsZero() { + t.Fatalf("online meter: %v %s %v %v", w, got, at, ok) + } + s.DriverHealthMut("meter").SetOffline() + if _, _, _, ok := s.OnlineMeterSample("meter"); ok { + t.Fatal("an offline meter's last sample was returned") + } +} + +// Every reporting driver of one type, and none that went offline. +func TestOnlineRawWByTypeSkipsOfflineDrivers(t *testing.T) { + s := NewStore() + for name, w := range map[string]float64{"a": -3000, "b": 1500} { + s.Update(name, DerBattery, w, nil, nil) + s.DriverHealthMut(name).RecordSuccess() + } + s.Update("charger", DerEV, 4140, nil, nil) + s.DriverHealthMut("charger").RecordSuccess() + s.DriverHealthMut("b").SetOffline() + got := s.OnlineRawWByType(DerBattery) + if len(got) != 1 || got["a"] != -3000 { + t.Fatalf("online batteries = %v, want only a at -3000 W", got) + } +} diff --git a/go/protocol/contract_gen.go b/go/protocol/contract_gen.go index 9df0cd81c..d87ca9394 100644 --- a/go/protocol/contract_gen.go +++ b/go/protocol/contract_gen.go @@ -272,6 +272,8 @@ var RegistryOps = map[string]string{ "site.mode.set": "ftw.mode.write", // planner.prefs.set — Set household planner preferences. "planner.prefs.set": "ftw.mode.write", + // site.time_zone.propose — Offer the phone's time zone to a box that keeps none. + "site.time_zone.propose": "ftw.setup.write", // battery.hold — Hold the battery at a fixed setpoint. "battery.hold": "ftw.dispatch.write", // loadpoint.hold — Charge the car now at a fixed current. diff --git a/go/protocol/messages.go b/go/protocol/messages.go index 9dfa42b28..b53dbbe46 100644 --- a/go/protocol/messages.go +++ b/go/protocol/messages.go @@ -42,6 +42,11 @@ const ( // permission. The box maps that permission onto a planner mode; the // caller does not name the mode. OpPlannerPrefsSet = "planner.prefs.set" + // OpSiteTimeZonePropose offers the client's time zone as the household + // clock. The box stores it only while it keeps no zone of its own, and + // restarts to run on it; a box that keeps one answers E_PRECONDITION + // with the zone it keeps in args.tz. + OpSiteTimeZonePropose = "site.time_zone.propose" // OpLoadpointHold pins one EV loadpoint to a fixed charging power, or // releases it with `clear`. The same manual hold the HTTP route // installs, reached through this door's gates instead of a verb. diff --git a/go/protocol/push_gen.go b/go/protocol/push_gen.go index 489a71065..ed3e2c7cd 100644 --- a/go/protocol/push_gen.go +++ b/go/protocol/push_gen.go @@ -15,6 +15,7 @@ const ( PushUpdateInstalled = "update.installed" PushDriverOffline = "driver.offline" PushFuseOverLimit = "fuse.over_limit" + PushControlNotFollowing = "control.not_following" PushBoxUnreachable = "box.unreachable" PushCloudGranted = "cloud.granted" PushCloudRevoked = "cloud.revoked" @@ -42,6 +43,7 @@ var PushSentences = map[string]PushSentence{ PushUpdateInstalled: {Title: "Your box updated itself", Body: "Now running {version}. Everything came back on its own."}, PushDriverOffline: {Title: "A device went quiet", Body: "{name} stopped answering."}, PushFuseOverLimit: {Title: "The house is drawing too much", Body: "{phase} is over the fuse rating."}, + PushControlNotFollowing: {Title: "A device is not following FTW", Body: "{name} does not do what FTW asks. Open FTW to see why."}, PushBoxUnreachable: {Title: "Your box is out of reach", Body: "It stopped answering. Power cut, or the internet at home is down."}, PushCloudGranted: {Title: "FTW Cloud is on", Body: "Your box now shares a copy of its readings with FTW Cloud. You can turn it off in the app or on the box."}, PushCloudRevoked: {Title: "FTW Cloud is off", Body: "Your box stopped sharing with FTW Cloud and asked it to delete its copy."}, diff --git a/go/test/site/README.md b/go/test/site/README.md index ee938f6a7..fc804b544 100644 --- a/go/test/site/README.md +++ b/go/test/site/README.md @@ -9,8 +9,11 @@ car on them. FTW talks to it over Modbus and OCPP as on a real site. Each scenario starts a fresh house and a fresh `state.db`, drives the house and FTW, and judges the run against fixed criteria from what flowed through the fuse (the house's record) and what FTW said (its API, once a second). FTW -runs `planner_self` with SE3 prices and the Core planner, or Energyplan -with `FTW_SITE_ENGINE=energyplan`, the engine a beta box runs. +runs what a beta box runs: Energyplan in `planner_passive_arbitrage`, or in +`planner_self` where a scenario needs PV surplus to charge the battery. Its +prices are pinned: the test serves 24 h of fixed prices from the quarter the +run starts in ([`prices_test.go`](prices_test.go)) through `price.base_url`, +so a run does not depend on the day or the internet. The house lives in device-simulator; this directory keeps only what is FTW's: the scenarios, FTW's configuration, the criteria and the pinned @@ -33,11 +36,11 @@ make siterig-judge DIR=/tmp/ftw-site-123 # judge a recorded run again | `FTW_SITE_PORT_BASE` | first port on 127.0.0.1 (default 18700): the house on base+0..59, FTW on base+90 and +91 | | `FTW_SITE_SHARD` | `k/n`: shard k of n, for several machines | | `FTW_SITE_OUT` | run directories, `summary.md` and `summary.json` (default: a new temporary directory) | -| `FTW_SITE_ENGINE` | the planner: `core` (default) or `energyplan`, the worker in `optimizer/native/bundle` | +| `FTW_SITE_ENGINE` | the planner: `energyplan` (default), the worker in `optimizer/native/bundle`, or `core` | | `FTW_SITE_DEVSIM_SRC` | a device-simulator checkout at the pinned commit; without it the test clones the pin into the user cache | | `FTW_SITE_FTW_BIN`, `FTW_SITE_DEVSIM_BIN` | prebuilt binaries | | `FTW_SITE_CATALOG` | the fault catalog, for `TestCatalogCoverage` | -| `FTW_SITE_JUDGE` | a results directory: `TestJudge` judges its runs again with the criteria as they are now, and rewrites their `result.json` and the summary | +| `FTW_SITE_JUDGE` | a results directory: `TestJudge` judges its runs again with the criteria as they are now, random seeds included (from each run's `plan.json`), and rewrites their `result.json` and the summary | Each run directory keeps `house/plant.jsonl` (the house once a second, with every event and mark), `house/ocpp-.jsonl` (every OCPP frame the @@ -61,7 +64,13 @@ that touch control, loadpoint, OCPP or telemetry code), `siterig/full` A criterion named in a scenario's `known` map fails on master for the finding it names. The run passes while it keeps failing, and fails once it -passes: a fix deletes its line in the same pull request. +passes: a fix deletes its line in the same pull request. The `r` +scenarios are R1-R5 of the stable plan; their known lines name the Wave 1 +item whose fix deletes them. + +A scenario with a `skip` waits for a device-simulator change the pinned +commit lacks. `TestSite` reports it as `skip` with the knob it needs; the +pull request that pins a simulator with the knob deletes the skip. ## The fault catalog @@ -71,15 +80,17 @@ its sources (`provenance` in [catalog_test.go](catalog_test.go)). ## Scenarios -"Over the limit" is the worst phase above the fuse − 1 A, measured through +"Over the limit" is the worst phase above the fuse rating, measured through the fuse. A device's response time counts as 2 s, the meter and control -tick as 2 s more. +tick as 2 s more. The random cases, `r3` and `r5a` judge a phase between +the fuse and 1.2 × the fuse as Core does: it counts only once its 60 s heat +(I² filter) is above the fuse too. | Scenario | House and steps | Criteria | Short | |---|---|---|---| | `s0-25` Idle charger | 25 A, 3 A per phase, a charger with no car, heartbeat 60 s, 300 s | Home use `measured` in 95 % of samples; no OCPP disconnect | yes | | `s0b-25` | as s0, Core's heartbeat | as s0 | | -| `s1-16`, `s1-25` Charge now | 3 A per phase, Charge now 16 A for 180 s | the room the fuse gives (25 A: 11 040 W) within 60 s; held 95 %; fuse holds; on 25 A no `fuse_limit` | yes | +| `s1-16`, `s1-25` Charge now | 3 A per phase, Charge now 16 A for 180 s | the room the fuse gives (16 A: 8 970 W; 25 A: 11 040 W) within 60 s; held 95 %; fuse holds; on 25 A no `fuse_limit` | yes | | `s2-16` Charge now during export | 16 A, 1 A house, 7 kW PV, full battery: 9.1 A export per phase | car starts within 60 s (**known**: export counted as import); fuse holds | | | `s3-16`, `s3-25` Load step | car and a 3 kW battery hold charging, then 13 A (16 A) or 21 A (25 A) per phase for 180 s | fuse holds; over the limit at most 7 s after the step | yes | | `s4-25`, `s4b-25` Saved unlock | `surplus_unlock_bat_soc` 0.9 saved, not armed; Charge now 3p, and bare (s4b) | no stops; no offer drops to 0 W; full power over the last minute | | @@ -97,8 +108,15 @@ tick as 2 s more. | `s13-25` Socket reconnect | Charge now 16 A, the charger's connection gone for 10 s | Charge now goes on after the charger returns; no unplug seen | | | `s15-25` One-phase charger, Charge now | 25 A, 3 A per phase, an Easee wired on L1 (6-16 A, steps to 3 680 W, no phase mode); Charge now on the phases `/api/loadpoints` reports, as the app sends it, 180 s | 16 A on L1 within 60 s; held 95 %; fuse holds; no `fuse_limit` | yes | | `s15b-25`, `s15c-25` | as s15, phase mode `1p` and `auto` | as s15 | | -| `s15bat-16` One-phase charger, battery rising | 16 A, 3 A per phase, the s15 charger, battery at 50 % following its EMS 8 s late; PV 2.1 kW rising every 30 s (+2.5 kW, then 1.5 kW) so Core charges the battery in steps; Charge now 6 s after the second step, while the raise is on its way | the battery order rose over 1 150 W in the 20 s before (setup check); car charging within 10 s; fuse holds; over the limit at most 3 s | | +| `s15bat-16` One-phase charger, battery rising | 16 A, 3 A per phase, the s15 charger, battery at 50 % following its EMS 8 s late, `planner_self`; PV 2.1 kW rising every 30 s (+2.5 kW, then 1.5 kW) so Core charges the battery in steps; Charge now 6 s after the second step, while the raise is on its way | the battery order rose over 1 150 W in the 20 s before (setup check); car charging within 10 s; fuse holds; over the limit at most 3 s | | | `s15p-25` One-phase charger, planned | as s15, a plan for 600 Wh by the first quarter hour 15 min out | fuse holds; 90 % of the plan by the ready time; never called not following | | | `s14-25` One-phase car | an 11 kW charger with a one-phase car, a plan it cannot meet, 20 min | 3.4–3.8 kW over the last 10 min; never called not following | | +| `r1-25` Meter silent, then offline | 25 A, 15 A per phase, battery at 50 %, Charge now 16 A; the site meter silent (Modbus timeouts) for 120 s, then gone for 90 s. **Skipped**: the pinned simulator cannot silence the site meter | the battery relieved the fuse (setup); while silent no offer rises and the battery order only moves toward 0 (**known**: A1); a 0 W order and the battery back on its own within 64 s; the meter never shows live; Charge now back within 95 s of each return and held until the next event; fuse holds | | +| `r2-25`, `r2-60-25` Battery drops out | 25 A, 20 A per phase, the battery held at a 3 kW discharge, a car taking 10 A on Charge now, the charger at once or 60 s late; the inverter gone for 180 s. The site stays under 1.2 × the fuse, where Core cuts the car on its 60 s heat budget, not at once | the battery discharged (setup); fuse holds; over 1.2 × the fuse at most the charger's delay + 7 s; the 60 s heat under 1.13 × the fuse; no offer rises while it is gone; Home use shown as estimated throughout. The time over the fuse and when Core sends the cut are recorded | | +| `r3-25` Planner floor | 25 A, 15 A per phase, battery at 12 %, planner `soc_min` 10 %, inverter floor 5 %, charger 60 s late, Charge now 16 A for 600 s | the battery reaches the floor (setup); no discharge ordered below it; the measured SoC stays at or above it; the battery order swings at most 100 W over the last 5 min; over the fuse at most the delay + 7 s; fuse holds | | +| `r4-25` Hard kill | 25 A, 12 A per phase, Charge now 16 A; once the day's Home use is known and 25 min more, the battery held at a 3 kW discharge; 5 min later SIGKILL for 20 s, and 2 min after that for 120 s while the house steps to 16 A | Charge now back at its minimum within 60 s of each restart and held until the next kill or the end; the day's Home use kept within 1 % across the first kill; session energy counts again; no state.db recovery in the log; fuse holds | | +| `r5a-25` Start beside a charging car | 25 A, 14 A per phase, battery at its 5 % floor, a car charging on its own at 16 A: 30 A per phase for 60 s before Core starts | the car charged and the site was over the fuse (setup); under the fuse within 30 s of Core answering, and over it at most 3 s after; Home use shown within 30 s; fuse holds | | +| `r5b-25` Start on a stale plan | a goal 40 → 60 % 8 h out, Core stopped for 20 min | no positive order in the first 30 s before a fresh plan; Home use shown within 30 s; fuse holds. It does not reach D6 on master: the saved plan is younger than 30 min and a fresh one lands within seconds | | +| `r5c-25` Start into a new session | Charge now, Core stopped for 10 min while the car leaves and another plugs in | Core sees the car within 60 s; Home use shown within 30 s; fuse holds. Whether Charge now carries over is recorded, not judged: ev-088 waits for the owner | | | `s16-25` Import ceiling | 25 A, house 1.3 kW, battery idle at its floor, `peak_import_ceiling_w` 4 500, Charge now 16 A: house and the car's 4 140 W minimum are above the ceiling. Starts in an hour's first minutes or waits for :56, then runs one whole clock hour and 3 min | the car starts on the hour's room before :45; every clock hour's recorded import averages at most the ceiling + 1 %; the next hour stops the car within 60 s; while held it says `import_limit` 90 % of the time and never `fuse_limit`; fuse holds | | -| `s17-25` Two equal batteries | 25 A, house 3 kW, no PV, two hybrids with 10 kWh at 74 % and 60 %, Energyplan whatever `FTW_SITE_ENGINE` says (Core DP plans one battery), a 6 h horizon (over 48 h the plan keeps the batteries for tomorrow's dearer hours). Waits up to an hour for a quarter the plan discharges, then 15 min | Energyplan's plan moves the batteries in half the polls (setup; tolerated, the day's prices decide); while the fleet moves 500 W or more, each battery takes at least a quarter of it 90 % of the time; while the plan moves them, the charge gap shrinks by over 0.2 points; fuse holds | | +| `s17-25` Two equal batteries | 25 A, house 3 kW, no PV, two hybrids with 10 kWh at 74 % and 60 %, Energyplan whatever `FTW_SITE_ENGINE` says (Core DP plans one battery), a 6 h horizon. Waits up to an hour for a quarter the plan discharges, then 15 min | Energyplan's plan moves the batteries in half the polls (setup); while the fleet moves 500 W or more, each battery takes at least a quarter of it 90 % of the time; while the plan moves them, the charge gap shrinks by over 0.2 points; fuse holds | | diff --git a/go/test/site/catalog_test.go b/go/test/site/catalog_test.go index b33feacea..55cfd9988 100644 --- a/go/test/site/catalog_test.go +++ b/go/test/site/catalog_test.go @@ -20,6 +20,7 @@ var provenance = map[string]struct { catalog []string sources []string }{ + "s18-16": {nil, []string{"https://github.com/srcfl/ftw/issues/1868"}}, "s0-25": {[]string{"home-070", "ocpp-011", "ocpp-013", "ui-052"}, []string{"https://github.com/srcfl/ftw/pull/1779", "https://github.com/srcfl/ftw/pull/1790"}}, "s0b-25": {[]string{"home-070", "ocpp-011", "ocpp-013", "ui-052"}, []string{"https://github.com/srcfl/ftw/pull/1790"}}, "s1-16": {nil, []string{"simulator test before beta 0.155.0, 21 Oct 2026"}}, @@ -51,6 +52,14 @@ var provenance = map[string]struct { "s15p-25": {nil, []string{"https://github.com/srcfl/ftw/pull/1804", "release blocker 22 Oct 2026: a one-phase charger, planned charging"}}, "s16-25": {nil, []string{"https://github.com/srcfl/ftw/pull/1828", "help report 25 Oct 2026: an idle car held at 0 W for an hour under a 4.5 kW import ceiling"}}, "s17-25": {nil, []string{"https://github.com/srcfl/ftw/pull/1814", "field report 23 Oct 2026, v0.155.1-beta.1: two equal Pixii batteries take turns"}}, + "r1-25": {[]string{"tel-001", "tel-006", "tel-007", "bat-074"}, []string{"stable plan 27 Oct 2026, R1"}}, + "r2-25": {[]string{"tel-007", "tel-022", "ev-090", "ev-075"}, []string{"stable plan 27 Oct 2026, R2"}}, + "r2-60-25": {[]string{"tel-007", "tel-022", "ev-090", "ev-075"}, []string{"stable plan 27 Oct 2026, R2"}}, + "r3-25": {[]string{"ev-099", "ev-090"}, []string{"stable plan 27 Oct 2026, R3"}}, + "r4-25": {[]string{"ev-085", "ev-091", "home-061", "home-074", "ocpp-008", "upd-051"}, []string{"stable plan 27 Oct 2026, R4", "https://github.com/srcfl/ftw/pull/1787"}}, + "r5a-25": {[]string{"ev-076", "dev-100"}, []string{"stable plan 27 Oct 2026, R5"}}, + "r5b-25": {[]string{"ev-076"}, []string{"stable plan 27 Oct 2026, R5", "https://github.com/srcfl/ftw/issues/1718"}}, + "r5c-25": {[]string{"ev-088"}, []string{"stable plan 27 Oct 2026, R5"}}, } func TestEveryScenarioSaysWhereItComesFrom(t *testing.T) { diff --git a/go/test/site/ev_battery_priority_test.go b/go/test/site/ev_battery_priority_test.go new file mode 100644 index 000000000..ef6a011e3 --- /dev/null +++ b/go/test/site/ev_battery_priority_test.go @@ -0,0 +1,79 @@ +//go:build unix + +package site + +// #1868: the car has only [0, 4140] W, while the battery can charge at +// 6000 W. The actual devices run over Modbus and OCPP; marks and the +// physical meter record let the judge see which device yielded. +func s18(c *runCtx) (map[string]any, error) { + err := with(c, stackSpec{fuseA: 16, loadsW: houseA(3), socPct: 40, + lpExtra: map[string]any{"min_charge_w": 4140, "max_charge_w": 4140, "phase_mode": "3p", "allowed_steps_w": []float64{0, 4140}}}, func(st *stack) error { + if err := st.plug(0); err != nil { + return err + } + if err := st.post("/api/battery/manual_hold", map[string]any{"direction": "charge", "power_w": 6000, "hold_s": 1800}, nil); err != nil { + return err + } + st.mark("charge_now") + if err := st.chargeNow("garage", 6, "3p", 4140); err != nil { + return err + } + sleepS(120) + st.mark("small_step") + if err := st.setLoadsA(3.1); err != nil { + return err + } + sleepS(90) + st.mark("no_room") + if err := st.setLoadsA(16); err != nil { + return err + } + sleepS(90) + st.mark("room_returns") + if err := st.setLoadsA(3); err != nil { + return err + } + sleepS(150) + st.mark("end") + return nil + }) + return map[string]any{"scenario": "s18", "fuse": 16, "battery_hold_w": 6000, "car_w": 4140}, err +} + +func judgeS18(d *runData) (metrics, []criterion) { + t0, small, none, back, end := d.marks["charge_now"], d.marks["small_step"], d.marks["no_room"], d.marks["room_returns"], d.marks["end"] + pre := d.window(small-20, small) + minor := d.window(small+10, none) + paused := d.window(back-20, back) + recovered := d.window(end-30, end) + carMin := func(w []*sample) float64 { return minOf(w, func(r *sample) float64 { return carW(r, 0) }) } + restart := d.firstT(back, func(r *sample) bool { return carW(r, 0) >= 4100 }) + batteryBefore, batteryAfter := meanOf(pre, batSigned), meanOf(d.window(none-20, none), batSigned) + // Core holds the phases at the fuse line, so the house's 0.1 A step + // lands over it until Core cuts; the I²t model judges that stretch. + // Own commands are the rest, as in the minor window. The working limit + // is the fuse rating (owner rule, 27 Oct 2026): a reading over it fails + // only while the 60 s heat is over it too, or over 1.2 x the fuse at + // once (heatOver). Readings jitter a few mA around the line. + fuse := d.metaF("fuse") + own := append(d.window(t0, small), minor...) + ownI, ownOver := maxOf(own, maxAbsI), 0 + d.warm() + for _, r := range own { + if heatOver(r, fuse) { + ownOver++ + } + } + m := metrics{"battery_before_w": batteryBefore, "battery_after_w": batteryAfter, "restart_s": restart, + "car_before_w": carMin(pre), "car_small_step_w": carMin(minor), "car_recovered_w": carMin(recovered), + "fuse_blown": anyBlown(d.samples), "max_i_before_house_overload": maxOf(d.window(t0, none), maxAbsI), + "max_i_own_commands": ownI, "own_commands_over_fuse_heat_samples": ownOver} + return m, []criterion{ + crit("setup: car and battery charge together", len(pre) > 0 && carMin(pre) >= 4100 && batteryBefore > 1000), + crit("battery yields before the minimum-step car", len(minor) > 0 && carMin(minor) >= 4100 && batteryAfter < batteryBefore-40), + crit("car stops when house leaves no room", len(paused) > 0 && maxOf(paused, func(r *sample) float64 { return carW(r, 0) }) < 100), + crit("car restarts automatically within 150 s", le(restart, 150) && len(recovered) > 0 && carMin(recovered) >= 4100), + crit("own commands stay inside the rated fuse", len(own) > 0 && ownOver == 0), + crit("fuse holds", !anyBlown(d.samples)), + } +} diff --git a/go/test/site/evaluate_test.go b/go/test/site/evaluate_test.go index 1e3caaee4..7a2f93145 100644 --- a/go/test/site/evaluate_test.go +++ b/go/test/site/evaluate_test.go @@ -42,18 +42,19 @@ type plantDevice struct { W float64 `json:"w"` // a charger's draw BatteryW float64 `json:"battery_w"` // an inverter's battery, + charging SoCPct float64 `json:"soc_pct"` - EMSCmd float64 `json:"ems_cmd"` // 170 charge, 187 discharge, 204 stop + EMSMode float64 `json:"ems_mode"` // 2 forced, as Core drives it + EMSCmd float64 `json:"ems_cmd"` // 170 charge, 187 discharge, 204 stop EMSSetW float64 `json:"ems_setpoint_w"` Connected bool `json:"connected"` + Running *bool `json:"running"` // an inverter that answers; nil when not recorded } type ftwStatus struct { - Drivers map[string]struct { - Status string `json:"status"` - } `json:"drivers"` - LoadQuality *string `json:"load_quality"` - LoadReason *string `json:"load_reason"` - LoadW *float64 `json:"load_w"` + Drivers map[string]ftwDriver `json:"drivers"` + Dispatch []ftwDispatch `json:"dispatch"` // Core's battery orders this tick + LoadQuality *string `json:"load_quality"` + LoadReason *string `json:"load_reason"` + LoadW *float64 `json:"load_w"` Energy struct { Today struct { LoadWh *float64 `json:"load_wh"` @@ -62,6 +63,18 @@ type ftwStatus struct { } `json:"energy"` } +// ftwDispatch is one battery order Core sent: + charge, - discharge. +type ftwDispatch struct { + Driver string `json:"driver"` + TargetW float64 `json:"target_w"` +} + +type ftwDriver struct { + Status string `json:"status"` + EVSessionWh *float64 `json:"ev_session_wh"` // a charger's session energy + ConsecutiveErrors *int `json:"consecutive_errors"` // failed polls in a row; 0 once a poll reads +} + type ftwLoadpoint struct { ID string `json:"id"` Plugged bool `json:"plugged_in"` @@ -69,6 +82,7 @@ type ftwLoadpoint struct { Charger struct { Known bool `json:"known"` } `json:"charger"` + ManualActive bool `json:"manual_active"` CommandedReason string `json:"commanded_reason"` Manual struct { LimitReason string `json:"limit_reason"` // the clamp behind a limited Charge now @@ -102,6 +116,7 @@ type sample struct { Bat2 plantDevice // the second, if any Status *ftwStatus LPs []ftwLoadpoint + HeatA [3]float64 // each phase's 60 s heat, filled in by warm } type frame struct { @@ -125,6 +140,7 @@ type runData struct { events []houseEventLine frames []frame // garage's charger meta map[string]any + warmed bool // every sample's HeatA is filled in } type markLine struct { @@ -340,6 +356,24 @@ func (d *runData) firstT(t0 float64, pred func(*sample) bool) *float64 { return nil } +// holdsFrom is the seconds from t0 to the start of the unbroken run of +// samples satisfying pred that lasts until t1, or nil when the last sample +// before t1 does not. A stop that begins after t0 resets it, so a car still +// running at t0 and stopped later does not count as back. +func (d *runData) holdsFrom(t0, t1 float64, pred func(*sample) bool) *float64 { + var from *float64 + for _, r := range d.window(t0, t1) { + switch { + case !pred(r): + from = nil + case from == nil: + v := r.T - t0 + from = &v + } + } + return from +} + // runsOver is the longest contiguous stretch and the total seconds with the // worst phase above limit. func (d *runData) runsOver(t0, t1, limit float64) (longest, total float64) { @@ -367,10 +401,72 @@ func (d *runData) runsOver(t0, t1, limit float64) (longest, total float64) { return round(longest, 1), round(total, 1) } +// fuseHeatTauS is the time constant of Core's fuse heat model +// (control/fuse_heat.go). +const fuseHeatTauS = 60.0 + +// warm fills in each sample's HeatA: per phase, the 60 s I² filter of Core's +// fuse heat model, on the current through the fuse in either direction, +// from the record's first sample on. +func (d *runData) warm() { + if d.warmed { + return + } + d.warmed = true + var sq [3]float64 + for i, r := range d.samples { + for k, a := range r.PhaseA { + if i == 0 { + sq[k] = a * a + } else { + sq[k] += (1 - math.Exp(-(r.T-d.samples[i-1].T)/fuseHeatTauS)) * (a*a - sq[k]) + } + r.HeatA[k] = math.Sqrt(sq[k]) + } + } +} + +// heatOver is whether a phase is over the fuse as Core's heat model judges +// it (owner rule, 27 Oct 2026): above the fuse while its 60 s heat is above +// the fuse too, or above fuseInstantFactor x the fuse. Between the fuse and +// that factor the heat is the budget. Call warm first. +func heatOver(r *sample, fuse float64) bool { + for k, a := range r.PhaseA { + a = math.Abs(a) + if a > fuse && (r.HeatA[k] > fuse || a > fuseInstantFactor*fuse) { + return true + } + } + return false +} + +// heatRunsOver is the longest stretch in [t0, t1] that heatOver holds, +// timed like runsOver. +func (d *runData) heatRunsOver(t0, t1, fuse float64) float64 { + d.warm() + longest := 0.0 + var from *float64 + var last float64 + for _, r := range d.window(t0, t1) { + t := r.T + switch { + case heatOver(r, fuse) && from == nil: + from = &t + case !heatOver(r, fuse) && from != nil: + longest, from = math.Max(longest, t-*from), nil + } + last = t + } + if from != nil { + longest = math.Max(longest, last-*from) + } + return round(longest, 1) +} + // i2Over is the seconds the worst phase's I² filter (tau 60 s) is above // limit, and the filter's peak. func (d *runData) i2Over(t0, t1, limit float64) (over, peak float64) { - const tau = 60.0 + const tau = fuseHeatTauS var y, prev float64 started := false for _, r := range d.window(t0, t1) { @@ -650,8 +746,7 @@ func le(p *float64, limit float64) bool { return p != nil && *p <= limit } func (d *runData) overage(t0, t1, fuse float64) metrics { w := d.window(t0, t1) m := metrics{"max_i": round(maxOf(w, maxAbsI), 2)} - m["over_limit_longest_s"], m["over_limit_total_s"] = d.runsOver(t0, t1, fuse-1) - m["over_fuse_longest_s"], _ = d.runsOver(t0, t1, fuse) + m["over_limit_longest_s"], m["over_limit_total_s"] = d.runsOver(t0, t1, fuse) m["fuse_damage_max"] = round(maxOf(w, damage), 4) m["fuse_blown"] = anyBlown(w) return m @@ -687,3 +782,80 @@ func (d *runData) disconnects(t0, t1 float64) []float64 { }) return out } + +// ---- the battery, the charger's orders and the log, for R1-R5 ---- + +// batOrder is Core's battery order as the inverter holds it: + charge, - +// discharge, 0 when not forced. +func batOrder(r *sample) float64 { + if r.Bat.EMSMode != 2 { + return 0 + } + switch r.Bat.EMSCmd { + case 170: + return r.Bat.EMSSetW + case 187: + return -r.Bat.EMSSetW + } + return 0 +} + +// batForced is true while Core drives the battery: forced mode with a charge +// or discharge command. +func batForced(r *sample) bool { + return r.Bat.EMSMode == 2 && (r.Bat.EMSCmd == 170 || r.Bat.EMSCmd == 187) +} + +// offers is every SetChargingProfile on connector 1 of charger 0, timed in +// Unix seconds. +func (d *runData) offers() []profile { return connector1(profiles(d.frames, 0, math.Inf(1))) } + +// offerRises counts orders sent in [t0, t1) that offer more than the order +// before them. +func (d *runData) offerRises(t0, t1 float64) int { + n := 0 + prs := d.offers() + for i := 1; i < len(prs); i++ { + if prs[i].T >= t0 && prs[i].T < t1 && prs[i].offerW() > prs[i-1].offerW()+1 { + n++ + } + } + return n +} + +// firstOffer is the seconds from t0 to the first order from t0 on that +// satisfies pred, or nil. +func (d *runData) firstOffer(t0 float64, pred func(profile) bool) *float64 { + for _, p := range d.offers() { + if p.T >= t0 && pred(p) { + v := p.T - t0 + return &v + } + } + return nil +} + +// logLines are the lines of FTW's log that hold any of subs. +func (d *runData) logLines(subs ...string) []string { + var out []string + readLines(filepath.Join(d.dir, "ftw.log"), func(b []byte) { + for _, s := range subs { + if strings.Contains(string(b), s) { + out = append(out, string(b)) + return + } + } + }) + return out +} + +// todayLoadWh is the day's Home use FTW last showed in ws, or nil. +func todayLoadWh(ws []*sample) *float64 { + var v *float64 + for _, r := range ws { + if r.Status != nil && r.Status.Energy.Today.LoadWh != nil { + v = r.Status.Energy.Today.LoadWh + } + } + return v +} diff --git a/go/test/site/prices_test.go b/go/test/site/prices_test.go new file mode 100644 index 000000000..27fa7bdcf --- /dev/null +++ b/go/test/site/prices_test.go @@ -0,0 +1,79 @@ +//go:build unix + +package site + +import ( + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "time" +) + +// pinnedSEKPerKWh is the rig's spot price for each hour after the stack +// starts: an evening peak, a cheap night, a morning peak, a flat midday and +// a second evening. A run sees the same prices from the same start on any +// day, so the plan does not change with the day's market. +var pinnedSEKPerKWh = [24]float64{ + 1.80, 1.60, 1.40, 1.10, + 0.40, 0.30, 0.25, 0.25, 0.30, 0.50, + 1.20, 1.50, 1.30, 0.90, + 0.60, 0.50, 0.45, 0.50, 0.70, 1.00, + 1.40, 1.70, 1.50, 1.10, +} + +// priceServer answers as elprisetjustnu does, in 15 min rows, with only the +// 24 h of pinned prices from the quarter it started in. Core reads it +// through price.base_url, which also turns off the live Nord Pool fallback. +type priceServer struct { + start time.Time + srv *httptest.Server +} + +func newPriceServer(now time.Time) *priceServer { + p := &priceServer{start: now.Truncate(15 * time.Minute)} + p.srv = httptest.NewServer(http.HandlerFunc(p.serve)) + return p +} + +func (p *priceServer) url() string { return p.srv.URL } + +func (p *priceServer) close() { p.srv.Close() } + +type priceRow struct { + SEKPerKWh float64 `json:"SEK_per_kWh"` + TimeStart string `json:"time_start"` + TimeEnd string `json:"time_end"` +} + +// rows are the pinned quarters that start on the given local date. +func (p *priceServer) rows(year, month, day int) []priceRow { + from := time.Date(year, time.Month(month), day, 0, 0, 0, 0, time.Local) + to := from.AddDate(0, 0, 1) + var out []priceRow + for i := 0; i < 4*len(pinnedSEKPerKWh); i++ { + t := p.start.Add(time.Duration(i) * 15 * time.Minute) + if t.Before(from) || !t.Before(to) { + continue + } + out = append(out, priceRow{SEKPerKWh: pinnedSEKPerKWh[i/4], + TimeStart: t.Format(time.RFC3339), TimeEnd: t.Add(15 * time.Minute).Format(time.RFC3339)}) + } + return out +} + +func (p *priceServer) serve(w http.ResponseWriter, r *http.Request) { + var year, month, day int + var zone string + if n, _ := fmt.Sscanf(r.URL.Path, "/%d/%d-%d_%3s.json", &year, &month, &day, &zone); n != 4 { + http.NotFound(w, r) + return + } + rows := p.rows(year, month, day) + if len(rows) == 0 { + http.NotFound(w, r) // elprisetjustnu's answer for a day it has not published + return + } + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(rows) +} diff --git a/go/test/site/random_test.go b/go/test/site/random_test.go index aeaa2b10d..3a124b728 100644 --- a/go/test/site/random_test.go +++ b/go/test/site/random_test.go @@ -9,8 +9,11 @@ import ( "math/rand" "os" "path/filepath" + "regexp" "sort" + "strconv" "strings" + "syscall" "time" ) @@ -155,7 +158,7 @@ func runPlan(c *runCtx, p randomPlan) (map[string]any, error) { case "plan": _, err = st.planTo("garage", a.SoCPct, a.GoalPct, time.Duration(a.ReadyMin*float64(time.Minute))) case "restart": - st.stopFTW() + st.stopFTW(syscall.SIGTERM) sleepS(a.DownS) err = st.startFTW() st.mark("ftw:up") @@ -211,97 +214,528 @@ func (d *runData) timing() houseTiming { return h } -// eventWindow is when the house changed on its own, and how long FTW may -// take to answer it. +// eventWindow is a stretch of time: a fault, or a Charge now hold. type eventWindow struct{ from, to float64 } +// within is whether t falls in one of ws, or up to afterS after its end. +func within(ws []eventWindow, t, afterS float64) bool { + for _, w := range ws { + if t >= w.from && t <= w.to+afterS { + return true + } + } + return false +} + +// inverterBackFrom gives, for a time t, the latest return before t (at): +// the inverter's from an outage (inverterOut) or Core's from a restart +// (down), and when the R8 count may start after it (from). Core has the inverter back at its first +// good read of it (no consecutive errors), which the Modbus reconnect +// backoff may hold back up to reconnectBackoffMaxS. After an outage, a +// default that failed while the inverter was away waits on the registry's +// retry timer, up to defaultRetryMaxS more, until #1915 (C11) sends it on +// the first good poll. A Core that starts fresh has no such wait. +func (d *runData) inverterBackFrom(inverterOut, down []eventWindow) func(t float64) (at, from float64) { + type back struct { + at, from float64 + } + var backs []back + add := func(at float64, outage bool) { + // While Core is down nothing reads the inverter: its backoff + // starts when Core is up again. + start := at + for _, w := range down { + if outage && at >= w.from && at < w.to { + start = w.to + } + } + from := start + reconnectBackoffMaxS + for _, r := range d.samples { + if r.T >= start && r.T < from && readsInverter(r) { + from = r.T + break + } + } + if outage { + from += defaultRetryMaxS + } + backs = append(backs, back{at, from}) + } + for _, w := range inverterOut { + add(w.to, true) + } + for _, w := range down { + add(w.to, false) + } + return func(t float64) (at, from float64) { + at, from = math.Inf(-1), math.Inf(-1) + for _, b := range backs { + if b.at <= t && b.at >= at { + at, from = b.at, b.from + } + } + return at, from + } +} + +// readsInverter is whether Core's last poll of the inverter succeeded. A +// status without the driver's error count says nothing, and counts as a +// read. +func readsInverter(r *sample) bool { + if r.Status == nil { + return false + } + dr, ok := r.Status.Drivers[inverterDriver] + return !ok || dr.ConsecutiveErrors == nil || *dr.ConsecutiveErrors == 0 +} + +// during is whether t falls in one of ws, up to but not at its end: at its +// end the device is back. +func during(ws []eventWindow, t float64) bool { + for _, w := range ws { + if t >= w.from && t < w.to { + return true + } + } + return false +} + +// stepWindow is a change FTW did not make, and how long FTW may take to +// answer it: until from + extraS + its lever's time + 2 s for the meter and +// the tick + 3 s. +type stepWindow struct { + plain bool // a house event that is no fault: it extends a stretch only if it raised the current + from, extraS float64 + leverS float64 // > 0: the only lever that can answer (the charger, for a car FTW did not start) + untilT float64 // > 0: the window ends here, whatever the lever (a Core restart) + noBattery bool // the step took the inverter away +} + +// What the random judge takes from the rig and from Core. +const ( + rigBatteryMaxW = 5000.0 // the simulated Sungrow's largest charge or discharge + evRestartDwellS = 35.0 // control/ev_fuse_budget.go evRestartDwell + tickS = 2.0 // the rig's site.control_interval_s + plannerFloorPct = 10.0 // Core's planner.soc_min when the config leaves it out (cmd/ftw/main.go) + + // R8: how long a battery FTW does not dispatch may hold a forced mode, + // and the Core timers that may delay it after the inverter returns. + inverterDriver = "sungrow" // the rig's inverter driver (stack_test.go) + forcedBudgetS = 10.0 + defaultRetryMaxS = 0.0 // #1915 (C11) retries the default on the first good poll after an outage +) + +// reconnectBackoffMaxS is Core's Modbus reconnect backoff cap +// (modbus/client.go reconnectBackoffMax), read from the source the rig +// builds Core from, so R8 follows Core when the cap changes (60 s, then +// 10 s with #1930). +var reconnectBackoffMaxS = coreSeconds("modbus/client.go", "reconnectBackoffMax") + +// coreSeconds reads "name = N * time.Second" from a file under go/internal. +// It panics when the line is gone: the judge has no number to fall back on. +func coreSeconds(file, name string) float64 { + b, err := os.ReadFile(filepath.Join("..", "..", "internal", file)) + if err != nil { + panic(err) + } + m := regexp.MustCompile(name + `\s*=\s*(\d+)\s*\*\s*time\.Second`).FindSubmatch(b) + if m == nil { + panic(fmt.Sprintf("no %s = N * time.Second in Core's %s", name, file)) + } + n, _ := strconv.Atoi(string(m[1])) + return float64(n) +} + +// dispatchW is Core's battery order on r, + charge; false when Core +// dispatched no battery. +func dispatchW(r *sample) (float64, bool) { + if r == nil || r.Status == nil || len(r.Status.Dispatch) == 0 { + return 0, false + } + w := 0.0 + for _, x := range r.Status.Dispatch { + w += x.TargetW + } + return w, true +} + +// worstSignedA is the phase current furthest from 0, with its direction. +func worstSignedA(r *sample) float64 { + w := 0.0 + for _, a := range r.PhaseA { + if math.Abs(a) > math.Abs(w) { + w = a + } + } + return w +} + +// chargeNowChanges says, for each Charge now in the run's plan, whether it +// asked for a power or phase mode other than the Charge now before it. +// Without a plan every Charge now counts as a change. +func (d *runData) chargeNowChanges() []bool { + b, err := os.ReadFile(filepath.Join(d.dir, "plan.json")) + if d.dir == "" || err != nil { + return nil + } + var p randomPlan + if json.Unmarshal(b, &p) != nil { + return nil + } + var out []bool + var last *ftwAction + for i, a := range p.Actions { + if a.Do != "charge_now" { + continue + } + out = append(out, last == nil || last.W != a.W || last.PhaseMode != a.PhaseMode) + last = &p.Actions[i] + } + return out +} + func judgeInvariants(d *runData) (metrics, []criterion) { fuse := d.metaF("fuse") - limit := fuse - 1 h := d.timing() t0, t1 := d.marks["start"], d.marks["end"] run := d.window(t0, t1) + d.warm() m := metrics{"charger_command_s": h.commandS, "charger_startup_s": h.startupS, "events": len(d.events)} + charger := math.Max(2, h.commandS) // the charger answers in its own time, never under 2 s + orders := d.offers() - // Windows in which an over-limit second is the house's doing. - var windows []eventWindow - faultUntil := map[float64]float64{} // a fault's start and end + // Windows in which an over-limit second is the house's doing. Only a + // meter fault adds its length to the time FTW may take: it blinds FTW. + // An inverter outage takes the battery lever away; a charger outage + // delays the charger lever (chargerLever). + var windows []stepWindow + var faults, chargerFaults, inverterOut, chargerOut []eventWindow for _, ev := range d.events { - batteryUsable := true - if r := d.at(ev.T); r != nil && r.Bat.SoCPct <= h.floorPct+1 { - batteryUsable = false - } - // The fastest lever: the battery answers in 2 s, the charger in its - // own time, never under 2 s; then the meter and tick, 2 s, and 3 s. - lever := math.Max(2, h.commandS) - if batteryUsable { - lever = 2 - } - allow := lever + 2 + 3 + w := stepWindow{from: ev.T} + until := 0.0 switch ev.Event.Do { case "meter_late", "silence", "offline", "reconnect": - allow += ev.Event.ForS - faultUntil[ev.T] = ev.T + ev.Event.ForS + until = ev.T + ev.Event.ForS case "meter_glitch": - faultUntil[ev.T] = ev.T + 5 + until = ev.T + 5 + } + if until > 0 { + f := eventWindow{ev.T, until} + faults = append(faults, f) + switch ev.Event.Device { + case "", "meter": + chargerFaults = append(chargerFaults, f) + w.extraS = ev.Event.ForS + case "garage": + chargerFaults = append(chargerFaults, f) + chargerOut = append(chargerOut, f) + case "inverter": + inverterOut = append(inverterOut, f) + w.noBattery = true + } } - windows = append(windows, eventWindow{ev.T, ev.T + allow}) + w.plain = until == 0 + windows = append(windows, w) + } + // The battery's floor: the higher of the inverter's own floor and the + // planner's soc_min. FTW may not discharge it there (check 6), so there + // it is no lever for import either. + floorPct := h.floorPct + if p := d.metaF("planner_floor_pct"); p > 0 { + floorPct = math.Max(floorPct, p) + } else { + floorPct = math.Max(floorPct, plannerFloorPct) } for _, mk := range d.allMarks { if mk.Label == "ftw:restart" { - windows = append(windows, eventWindow{mk.T, mk.T + 60}) + windows = append(windows, stepWindow{from: mk.T, untilT: mk.T + 60}) } } - explained := func(t float64) (float64, bool) { - best, ok := 0.0, false - for _, w := range windows { + // A car that starts to draw more than FTW's last order lets it (the + // charger's own default, or a lower order still on its way) is a step + // FTW did not take, and only the charger can answer it. + orderW := func(t float64) (float64, bool) { + w, ok := 0.0, false + for _, p := range orders { + if p.T > t { + break + } + w, ok = p.offerW(), true + } + return w, ok + } + carStarts := 0 + for i := 1; i < len(d.samples); i++ { + r := d.samples[i] + if r.T < t0 || r.T > t1 || carW(r, 0)-carW(d.samples[i-1], 0) <= 1000 { + continue + } + if w, ok := orderW(r.T); !ok || carW(r, 0) > w+3*volt { + windows = append(windows, stepWindow{from: r.T, leverS: charger}) + carStarts++ + } + } + m["car_starts_ftw_did_not_order"] = carStarts + // In time order, so a new step after a stretch's allowance is told + // from a window inside it (judgeStretch). + sort.SliceStable(windows, func(i, j int) bool { return windows[i].from < windows[j].from }) + + // chargerLever is how long the charger takes to answer from t: its own + // time, after the rest of any outage it is in. + chargerLever := func(t float64) float64 { + left := 0.0 + for _, w := range chargerOut { if t >= w.from && t <= w.to { - best, ok = math.Max(best, w.to), true + left = math.Max(left, w.to-t) } } - return best, ok + return charger + left + } + // lever is how long FTW's fastest lever takes, from t, to take overA off + // the worst phase: the battery, in 2 s, while its inverter runs and its + // spare power covers the overage; otherwise the charger. At its floor + // the battery may only charge less, never discharge more. The inverter + // is back at the end of its outage. + lever := func(t, overA float64, export bool) float64 { + r := d.at(t) + if r == nil || during(inverterOut, t) || (r.Bat.Running != nil && !*r.Bat.Running) { + return chargerLever(t) + } + spare := rigBatteryMaxW + r.BatW // discharge more, or charge less + switch { + case export: + spare = rigBatteryMaxW - r.BatW + case r.Bat.SoCPct <= floorPct: + spare = math.Max(0, r.BatW) + } + if spare < overA*3*volt { + return chargerLever(t) + } + return 2 + } + until := func(w stepWindow, at, overA float64, export bool) float64 { + if w.untilT > 0 { + return w.untilT + } + lv := w.leverS + switch { + case lv > 0: + lv += chargerLever(w.from) - charger + case w.noBattery: + lv = chargerLever(w.from) + default: + lv = lever(math.Max(w.from, at), overA, export) + } + return w.from + w.extraS + lv + 2 + 3 } // 1, 2: the fuse holds; over the limit only as long as an outside step - // allows, and at most 3 s after FTW's own raise. + // allows, and at most 3 s after FTW's own raise. Between the fuse and + // 1.2 x the fuse the 60 s heat is the budget (heatOver): the clock + // starts when the heat passes the fuse, as Core's cut does. A stretch + // runs on across dips under the fuse while the heat stays over it. blownAny := anyBlown(run) + // carMovedIn is whether FTW sent the car a higher order (dir +1, toward + // import) or a lower one (dir -1, toward export) in (a, b]; with over, + // only one sent while the meter showed the fuse passed. + carMovedIn := func(a, b float64, over bool, dir float64) bool { + prev, ok := 0.0, false + for _, p := range orders { + if p.T > b { + break + } + if ok && p.T > a && dir*(p.offerW()-prev) > 1 && (!over || maxAbsI(d.at(p.T)) > fuse) { + return true + } + prev, ok = p.offerW(), true + } + return false + } + // batMovedIn is the same for the battery: FTW moved its order 0.5 A a + // phase or more toward import (dir +1) or export (dir -1) while the + // inverter ran. An order moved while the inverter is out reaches no + // battery. While Core is down or has no order yet the inverter keeps + // the last one, so a missing order keeps it too. + batMovedIn := func(a, b float64, over bool, dir float64) bool { + prev, ok := 0.0, false + for _, r := range d.samples { + if r.T < a-5 { + continue + } + if r.T > b { + break + } + w, has := dispatchW(r) + if !has && ok { + continue + } + running := !during(inverterOut, r.T) && (r.Bat.Running == nil || *r.Bat.Running) + if ok && has && r.T > a && dir*(w-prev) >= 0.5*3*volt && running && (!over || maxAbsI(r) > fuse) { + return true + } + prev, ok = w, true + } + return false + } + // ownRaise is whether FTW pushed a stretch from s to end over the fuse + // itself: it moved the car or the battery toward the stretch's + // direction in (a, s], or while the stretch was over the fuse. On + // import that is a car raise; on export a car cut (owner, 28 Oct 2026), + // as a battery move toward export already was. Raising the car on + // export is the right answer. + ownRaise := func(a, s, end float64, export bool) bool { + dir := 1.0 + if export { + dir = -1 + } + return carMovedIn(a, s, false, dir) || batMovedIn(a, s, false, dir) || + carMovedIn(s, end, true, dir) || batMovedIn(s, end, true, dir) + } + // raisedAt is whether the worst phase rose 0.5 A or more in the 3 s + // from t. + raisedAt := func(t float64) bool { + b := d.at(t - 0.5) + if b == nil { + return true + } + for _, r := range d.window(t, t+3) { + if maxAbsI(r) >= maxAbsI(b)+0.5 { + return true + } + } + return false + } unexplained, longestOwn := 0.0, 0.0 var firstBad float64 - var start *sample - for i, r := range run { - over := maxAbsI(r) > limit - if over && start == nil { - start = r - } - if (!over || i == len(run)-1) && start != nil { - end := r.T - if to, ok := explained(start.T); ok { - if end > to { - unexplained = math.Max(unexplained, end-to) - if firstBad == 0 { - firstBad = to - } + bad := func(t float64) { + if firstBad == 0 { + firstBad = t + } + } + // judged lists each stretch the heat or 1.2 x the fuse put on the clock: + // from, end, clock start and allowed until, in seconds from the start, + // and whether it was FTW's own raise or a step. + judged := [][]any{} + judgeStretch := func(stretch []*sample, end float64) { + s := stretch[0].T + var clock *sample + overA, export := 0.0, false + for _, r := range stretch { + if clock == nil && heatOver(r, fuse) { + clock = r + } + if w := worstSignedA(r); math.Abs(w)-fuse > overA { + overA, export = math.Abs(w)-fuse, w < 0 + } + } + if clock == nil { + return // the heat budget covers it + } + c := clock.T + // A window explains the stretch unless FTW pushed it over the + // fuse after the window's meter delay (ownRaise). + to, explained, own := math.Inf(-1), false, false + for _, w := range windows { + if x := until(w, s, overA, export); w.from <= s && s <= x && !ownRaise(w.from+2, s, end, export) { + to, explained = math.Max(to, x), true + } + } + if !explained { // no change outside FTW: its own, import or export + own, to = true, c+3 + } + if !own && c > s { // Core answers once the heat says so + to = math.Max(to, c+lever(c, overA, export)+2+3) + } + // brokeAt is the first time from t on that the stretch is over the + // fuse; an over sample holds until the next one. + brokeAt := func(t float64) float64 { + for i, r := range stretch { + next := end + if i+1 < len(stretch) { + next = stretch[i+1].T + } + if maxAbsI(r) > fuse && next > t { + return math.Max(r.T, t) } - } else if end-start.T > 3 { - longestOwn = math.Max(longestOwn, end-start.T) - if firstBad == 0 { - firstBad = start.T + 3 + } + return math.Inf(1) + } + // heatCap is the most that windows opening from t on may add: once + // a phase's 60 s heat is over the fuse, Core's cut has its lever's + // time, and no further event extends the stretch (owner, 28 Oct + // 2026). Unbounded while the heat stays under the fuse. + heatCap := func(t float64) float64 { + for _, r := range stretch { + if r.T >= t && max(r.HeatA[0], r.HeatA[1], r.HeatA[2]) > fuse { + return r.T + lever(r.T, overA, export) + 2 + 3 } } - start = nil + return math.Inf(1) + } + // A window that opens inside the stretch before it broke its + // allowance extends it: a second step, an outage or a Core restart. + // A plain house event counts only if it raised the current. Inside + // FTW's own raise a window counts only once the raise is answered: + // after its 3 s, with the current under the fuse until the window. + // What windows add stops at the heat's cap, however many there are. + // A window that opens once the allowance has run out, with the + // current under the fuse since, is a new step: its cap runs from it. + capT := heatCap(s) + for grown := true; grown; { + grown = false + for _, w := range windows { + if w.from < s || w.from > math.Min(end, brokeAt(to)) || (own && w.from <= to) { + continue + } + if w.plain && !raisedAt(w.from) { + continue + } + if w.from > to { + capT = math.Max(capT, heatCap(w.from)) + } + if x := math.Min(until(w, w.from, overA, export), capT); x > to { + to, grown, own = x, true, false + } + } + } + kind := "step" + if own { + kind = "own" + } + judged = append(judged, []any{round(s-t0, 1), round(end-t0, 1), round(c-t0, 1), round(to-t0, 1), kind}) + switch { + case end <= to: + case own: + longestOwn = math.Max(longestOwn, end-c) + bad(to) + default: + unexplained = math.Max(unexplained, end-to) + bad(to) } } - m["over_limit_after_steps_s"], m["over_limit_own_s"] = round(unexplained, 1), round(longestOwn, 1) - - // 3: with every device reading and no fault, Home use is measured. - inFault := func(t float64) bool { - for from, to := range faultUntil { - if t >= from && t <= to+30 { - return true + from, last := -1, -1 + for i, r := range run { + if maxAbsI(r) > fuse { + if from < 0 { + from = i } + last = i } - return false + open := last == i || (from >= 0 && max(r.HeatA[0], r.HeatA[1], r.HeatA[2]) > fuse) + if (!open || i == len(run)-1) && from >= 0 { + judgeStretch(run[from:last+1], run[min(last+1, len(run)-1)].T) + from = -1 + } + } + heatPeak := 0.0 + for _, r := range run { + heatPeak = math.Max(heatPeak, max(r.HeatA[0], r.HeatA[1], r.HeatA[2])) } + m["over_limit_after_steps_s"], m["over_limit_own_s"] = round(unexplained, 1), round(longestOwn, 1) + m["heat_peak_a"], m["over_limit_judged"] = round(heatPeak, 2), judged + + // 3: with every device reading and no fault, Home use is measured. + inFault := func(t float64) bool { return within(faults, t, 30) } restartAt := []float64{t0} for _, mk := range d.allMarks { if mk.Label == "ftw:up" { @@ -346,12 +780,27 @@ func judgeInvariants(d *runData) (metrics, []criterion) { } m["home_use_estimated_while_reading_s"] = round(dishonest, 1) - // 4: during Charge now the car charges when there is room for it. + // 4: during Charge now the car charges when there is room for it. With + // room, Core waits evRestartDwell before it starts a stopped car, then + // the charger takes its own time, and a tick. The count of idle seconds + // with room starts again on each Charge now that changes the request. + // A cut FTW had to send needs no new start: the count runs only while + // the car is idle with room, so it starts when the cut car stops. A + // cut with room is FTW's doing and starts nothing. While Core is down + // the count stops. Only a fault of the meter or of the charger excuses + // an idle car. var holds []eventWindow + var graceFrom []float64 var holdFrom float64 + changes, k := d.chargeNowChanges(), 0 for _, mk := range d.allMarks { switch mk.Label { case "ftw:charge_now": + changed := k >= len(changes) || changes[k] + k++ + if holdFrom == 0 || changed { + graceFrom = append(graceFrom, mk.T) + } if holdFrom == 0 { holdFrom = mk.T } @@ -362,17 +811,49 @@ func judgeInvariants(d *runData) (metrics, []criterion) { } } } - grace := 60 + h.commandS + h.startupS + 10 + // roomA is the room below the fuse for the car, Core's line; 6 A is + // the car's minimum on three phases. + roomA := func(r *sample) float64 { return fuse - (maxAbsI(r) - carW(r, 0)/(3*volt)) } + // While Core is down nothing can start the car. + var down []eventWindow + for i, mk := range d.allMarks { + if mk.Label != "ftw:restart" { + continue + } + up := t1 + for _, n := range d.allMarks[i:] { + if n.Label == "ftw:up" { + up = n.T + break + } + } + down = append(down, eventWindow{mk.T, up}) + } + // The car may stand idle with room for Core's dwell, the charger's + // command and startup time and a tick, plus the 2 s for the meter and + // tick and 3 s the fuse budgets also give. + allowedIdle := evRestartDwellS + h.commandS + h.startupS + tickS + 2 + 3 + regrace := func(from, to float64) bool { + for _, g := range graceFrom { + if g > from && g <= to { + return true + } + } + return false + } idle, run4 := 0.0, 0.0 for _, hw := range holds { run4 = 0 - for i, r := range d.window(hw.from+grace, hw.to) { - roomA := limit - (maxAbsI(r) - carW(r, 0)/(3*volt)) - bad := carW(r, 0) < 0.5*1380 && roomA >= 7 && !inFault(r.T) && !nearRestart(r.T) + ws := d.window(hw.from, hw.to) + for i, r := range ws { + if i > 0 && regrace(ws[i-1].T, r.T) { + run4 = 0 + } + bad := carW(r, 0) < 0.5*1380 && roomA(r) >= 6 && !within(chargerFaults, r.T, 30) && !within(down, r.T, 0) if bad && i > 0 { - run4++ + run4 += r.T - ws[i-1].T idle = math.Max(idle, run4) - if idle > 30 { + if run4 > allowedIdle { note("charge now", r.T) } } else { @@ -380,7 +861,7 @@ func judgeInvariants(d *runData) (metrics, []criterion) { } } } - m["charge_now_idle_with_room_s"] = idle + m["charge_now_idle_with_room_s"], m["charge_now_idle_allowed_s"] = round(idle, 1), allowedIdle // 5: no order above the charger's 16 A per phase. over := 0 @@ -392,17 +873,19 @@ func judgeInvariants(d *runData) (metrics, []criterion) { } m["offers_over_rating"] = over - // 6: FTW does not ask the battery to discharge at its floor. + // 6: FTW does not ask the battery to discharge at its floor (floorPct). + // FTW's orders are judged, not the inverter's register, which can hold + // an order from before an outage; check 8 judges that. floorS := 0.0 for _, r := range run { - if r.Bat.EMSCmd == 187 && r.Bat.EMSSetW > 50 && r.Bat.SoCPct <= h.floorPct+0.5 { + if w, ok := dispatchW(r); ok && w < -50 && r.Bat.SoCPct <= floorPct { floorS++ if floorS > 10 { note("floor", r.T) } } } - m["discharge_at_floor_s"] = floorS + m["discharge_at_floor_s"], m["floor_pct"] = floorS, floorPct // 7: a stale meter means no raise. raises := 0 @@ -420,6 +903,45 @@ func judgeInvariants(d *runData) (metrics, []criterion) { } } m["raises_on_a_stale_meter"] = raises + + // 8: a battery FTW does not dispatch runs in its own mode (R8, C1). The + // inverter's register may hold a forced charge or discharge FTW does + // not dispatch for forcedBudgetS. After an inverter outage or a Core + // restart the count starts once Core has the inverter back (backFrom). + // While Core is stopped with SIGTERM its driver cleanup writes the + // default, so a running inverter gets the same 10 s from the stop. One + // that comes back while Core is down has its outage's allowance. The + // triage's line is 10 s from the return itself; the seconds past that + // line that Core's timers excuse are recorded, not hidden. + backFrom := d.inverterBackFrom(inverterOut, down) + forcedS, pastS, timersS, run8, from8 := 0.0, 0.0, 0.0, 0.0, 0.0 + for i, r := range run { + w, ok := dispatchW(r) + stray := math.Abs(batOrder(r)) > 50 && !within(inverterOut, r.T, 0) + if !within(down, r.T, 0) { + stray = stray && r.Status != nil && (!ok || math.Abs(w) <= 50) + } + if !stray || i == 0 { + run8 = 0 + continue + } + if run8 == 0 { + from8 = run[i-1].T + } + run8 += r.T - run[i-1].T + forcedS = math.Max(forcedS, run8) + at, from := backFrom(r.T) + switch { + case r.T > math.Max(from8, from)+forcedBudgetS: + pastS += r.T - run[i-1].T + note("forced battery", r.T) + case r.T > math.Max(from8, at)+forcedBudgetS: + timersS += r.T - run[i-1].T + } + } + m["battery_forced_without_dispatch_s"] = round(forcedS, 1) + m["battery_forced_past_allowance_s"] = round(pastS, 1) + m["battery_forced_in_core_timers_s"] = round(timersS, 1) // The first second an invariant broke, from the start, per invariant: // a kept seed is cut a minute after the one it failed on. firsts := map[string]float64{} @@ -432,10 +954,11 @@ func judgeInvariants(d *runData) (metrics, []criterion) { crit("over the limit no longer than an outside step allows", unexplained == 0), crit("over the limit at most 3 s after FTW's own raise", longestOwn == 0), crit("Home use measured while every device reads", dishonest <= 10), - crit("Charge now charges when there is room", idle <= 30), + crit("Charge now charges when there is room", idle <= allowedIdle), crit("no offer above the charger's rating", over == 0), crit("no discharge asked at the battery's floor", floorS <= 10), crit("no raise on a stale meter", raises == 0), + crit("no forced battery mode FTW does not dispatch", pastS == 0), } } @@ -488,6 +1011,7 @@ var invariantKey = map[string]string{ "no offer above the charger's rating": "rating", "no discharge asked at the battery's floor": "floor", "no raise on a stale meter": "stale meter", + "no forced battery mode FTW does not dispatch": "forced battery", } // keepFailure writes a failing random run as a case to keep: its plan with diff --git a/go/test/site/random_unit_test.go b/go/test/site/random_unit_test.go new file mode 100644 index 000000000..19fb13209 --- /dev/null +++ b/go/test/site/random_unit_test.go @@ -0,0 +1,1263 @@ +//go:build unix + +package site + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "regexp" + "strconv" + "strings" + "testing" +) + +// Unit tests for the random judge (judgeInvariants), each on the shape of +// a nightly seed that the judge called wrong (triage of 27 Oct 2026, rig +// items R2-R7). They run in every go test, without a simulator. + +const ( + limitStep = "over the limit no longer than an outside step allows" + limitOwn = "over the limit at most 3 s after FTW's own raise" + chargeNow = "Charge now charges when there is room" + atFloor = "no discharge asked at the battery's floor" +) + +// randomRun is n one-second samples on a fuse, from start at 0 to end at +// n-1: house current base per phase, the car idle, the battery at 50 % +// with its inverter running. set changes sample i. +func randomRun(n int, fuse, base float64, set func(i int, r *sample)) *runData { + d := &runData{marks: map[string]float64{"start": 0, "end": float64(n - 1)}, meta: map[string]any{"fuse": fuse}} + running := true + for i := 0; i < n; i++ { + r := &sample{T: float64(i), PhaseA: [3]float64{base, base, base}, CarW: []float64{0}, + Bat: plantDevice{SoCPct: 50, Running: &running}} + if set != nil { + set(i, r) + } + d.samples = append(d.samples, r) + } + return d +} + +// withCharger records the charger's command and startup times as the house +// wrote them, so the judge reads them as from a real run. +func withCharger(t *testing.T, d *runData, commandS, startupS float64) { + t.Helper() + dir := t.TempDir() + spec := houseSpec{Devices: []houseDevice{{ID: "garage", CommandResponseS: &commandS, StartupDelayS: &startupS}}} + b, _ := json.Marshal(spec) + if err := os.MkdirAll(filepath.Join(dir, "house"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "house", "timeline.json"), b, 0o644); err != nil { + t.Fatal(err) + } + d.dir = dir +} + +func judged(d *runData) map[string]bool { + _, cs := judgeInvariants(d) + return verdictsOf(cs) +} + +// R2, seed 207539's shape: a step the battery answers opens the stretch, +// and a Core restart 1 s into it extends what the stretch may take. A +// restart that comes after the step's time ran out excuses nothing. +func TestInvariantsExtendAStretchByEveryWindowInsideIt(t *testing.T) { + for _, c := range []struct { + name string + restartAt float64 // 0: no restart + want bool + }{ + {"restart 1 s into the stretch", 120, true}, + {"no restart", 0, false}, + {"restart after the step's time", 130, false}, + {"restart 0.2 s after the step's time", 121.9, false}, + } { + // 20 A fuse; 26 A (1.3 x) from 119 to 143 s, after a load step + // at 114.7 s. The battery's 5 kW (7.2 A) covers the 6 A overage, + // so the step alone allows 2 + 2 + 3 s. + d := randomRun(301, 20, 10, func(i int, r *sample) { + if i >= 119 && i < 144 { + r.PhaseA = [3]float64{26, 13, 12} + } + }) + withCharger(t, d, 40, 0) + d.events = []houseEventLine{{T: 114.7, Event: houseEvent{Do: "loads"}}} + if c.restartAt > 0 { + d.allMarks = []markLine{{T: c.restartAt, Label: "ftw:restart"}, {T: c.restartAt + 9, Label: "ftw:up"}} + } + if got := judged(d)[limitStep]; got != c.want { + t.Errorf("%s: %q = %v, want %v", c.name, limitStep, got, c.want) + } + } +} + +// R3, seeds 207534, 207530 and 207539: the battery answers in 2 s only +// while its inverter runs and its spare power covers the overage; otherwise +// the charger is the lever, in its own time (5 s here). +func TestInvariantsBatteryIsTheLeverOnlyWhenItCanAnswer(t *testing.T) { + for _, c := range []struct { + name string + batW float64 + running bool + want bool + }{ + {"battery idle, 7.2 A spare for a 4 A overage", 0, true, false}, + {"battery already discharging 3266 W, 2.5 A spare", -3266, true, true}, + {"inverter offline", 0, false, true}, + } { + // 16 A fuse; 20 A (1.25 x) from 50 to 59 s after a step at 49.5 s: + // 9.5 s, past the battery's 7 s and inside the charger's 10 s. + d := randomRun(101, 16, 8, func(i int, r *sample) { + r.BatW, r.Bat.Running = c.batW, &c.running + if i >= 50 && i < 59 { + r.PhaseA = [3]float64{20, 12, 12} + } + }) + withCharger(t, d, 5, 0) + d.events = []houseEventLine{{T: 49.5, Event: houseEvent{Do: "loads"}}} + if got := judged(d)[limitStep]; got != c.want { + t.Errorf("%s: %q = %v, want %v", c.name, limitStep, got, c.want) + } + } +} + +// R4, seeds 207532 and 207537: a car that starts on the charger's own +// 16 A while FTW's order is 0 A is a step FTW did not take, answered by the +// 40 s charger. The same start right after FTW raised the order to 16 A is +// FTW's own. +func TestInvariantsCarStartingOnItsOwnIsAStep(t *testing.T) { + for _, c := range []struct { + name string + frames []frame + wantStep bool + wantOwn bool + }{ + {"FTW ordered 0 A", []frame{profileAt(0, 0), profileAt(20, 0)}, true, true}, + {"FTW raised to 16 A", []frame{profileAt(0, 0), profileAt(25, 16)}, true, false}, + } { + // 16 A fuse; the car draws 11 040 W from 30 s; the meter shows + // 20 A (1.25 x) from 31 to 39 s. + d := randomRun(121, 16, 4, func(i int, r *sample) { + if i >= 30 && i < 40 { + r.CarW[0] = 11040 + } + if i >= 31 && i < 39 { + r.PhaseA = [3]float64{20, 18, 19} + } + }) + withCharger(t, d, 40, 30) + d.frames = c.frames + v := judged(d) + if v[limitStep] != c.wantStep || v[limitOwn] != c.wantOwn { + t.Errorf("%s: step %v, own %v; want %v, %v", c.name, v[limitStep], v[limitOwn], c.wantStep, c.wantOwn) + } + } +} + +// R5, seed 207536: between the fuse and 1.2 x the fuse the 60 s heat is the +// budget (owner rule, 27 Oct 2026). 27.84 A on 25 A for 15 s from 15 A +// passes; above 1.2 x it fails once the step's time is up; so does the same +// 27.84 A held past the lever's time after the heat passes the fuse, and +// 25.5 A on a fuse already warm at its rating. +func TestInvariantsJudgeBetweenFuseAndInstantByHeat(t *testing.T) { + for _, c := range []struct { + name string + base, a float64 + from, to int + want bool + }{ + {"27.84 A for 15 s", 15, 27.84, 50, 65, true}, + {"30.1 A for 15 s", 15, 30.1, 50, 65, false}, + {"27.84 A for 200 s", 15, 27.84, 50, 250, false}, + {"25.5 A for 15 s on a warm fuse", 25, 25.5, 50, 65, false}, + // The heat passes the fuse at 127 s; the battery then has 2 + 5 s. + {"27.84 A until 7 s after the heat passes the fuse", 15, 27.84, 50, 134, true}, + {"27.84 A until 8 s after the heat passes the fuse", 15, 27.84, 50, 135, false}, + } { + d := randomRun(301, 25, c.base, func(i int, r *sample) { + if i >= c.from && i < c.to { + r.PhaseA[0] = c.a + } + }) + d.events = []houseEventLine{{T: 49.5, Event: houseEvent{Do: "loads"}}} + if got := judged(d)[limitStep]; got != c.want { + t.Errorf("%s: %q = %v, want %v", c.name, limitStep, got, c.want) + } + } +} + +// R6, seeds 207533, 207536 and 207539: during Charge now an idle car with +// room is allowed Core's 35 s dwell, the charger's command and startup time +// and a tick, plus 5 s (72 s here, with a 30 s startup). The grace starts +// again on each Charge now that changes the request and on each cut FTW +// had to send, over the fuse or with no room; a cut with room does not. +// While Core is down the count stops. A fault of the meter or the charger +// excuses the car; an inverter outage does not. +func TestInvariantsChargeNowGrace(t *testing.T) { + for _, c := range []struct { + name string + startS int // the car draws from here; 0: never + overS [2]int // the car draws 16 A, 21 A on the 16 A fuse, in [from, to) + marks []markLine + frames []frame + events []houseEventLine + want bool + }{ + {"starts after 70 s", 71, [2]int{}, nil, nil, nil, true}, + {"starts after 80 s", 81, [2]int{}, nil, nil, nil, false}, + {"a changed Charge now at 60 s", 121, [2]int{}, []markLine{{T: 60, Label: "ftw:charge_now"}}, nil, nil, true}, + {"FTW cuts the car over the fuse at 60 s", 121, [2]int{10, 61}, nil, + []frame{profileAt(0, 16), profileAt(60, 0)}, nil, true}, + {"FTW cuts an idle car with room at 60 s", 121, [2]int{}, nil, + []frame{profileAt(0, 16), profileAt(60, 0)}, nil, false}, + {"FTW hunts 6 A, 0 A, 6 A with room", 0, [2]int{}, nil, + []frame{profileAt(0, 6), profileAt(15, 0), profileAt(30, 6), profileAt(45, 0), + profileAt(60, 6), profileAt(75, 0), profileAt(90, 6), profileAt(105, 0), + profileAt(120, 6), profileAt(135, 0), profileAt(150, 6), profileAt(165, 0), + profileAt(180, 6), profileAt(195, 0)}, nil, false}, + {"Core restarts at 50 s and answers at 60 s", 121, [2]int{}, + []markLine{{T: 50, Label: "ftw:restart"}, {T: 60, Label: "ftw:up"}}, nil, nil, true}, + {"the meter late for 40 s from 10 s", 140, [2]int{}, nil, nil, + []houseEventLine{{T: 10, Event: houseEvent{Do: "meter_late", ForS: 40}}}, true}, + {"the inverter offline for 40 s from 10 s", 140, [2]int{}, nil, nil, + []houseEventLine{{T: 10, Event: houseEvent{Do: "offline", Device: "inverter", ForS: 40}}}, false}, + } { + // 16 A fuse, a 5 A house: 11 A of room. Charge now at 1 s. + d := randomRun(201, 16, 5, func(i int, r *sample) { + if c.startS > 0 && i >= c.startS { + r.CarW[0] = 6900 + r.PhaseA = [3]float64{15, 15, 15} + } + if i >= c.overS[0] && i < c.overS[1] { + r.CarW[0] = 11040 + r.PhaseA = [3]float64{21, 21, 21} + } + }) + withCharger(t, d, 0, 30) + d.allMarks = append([]markLine{{T: 1, Label: "ftw:charge_now"}}, c.marks...) + d.allMarks = append(d.allMarks, markLine{T: 200, Label: "end"}) + d.frames, d.events = c.frames, c.events + if got := judged(d)[chargeNow]; got != c.want { + t.Errorf("%s: %q = %v, want %v", c.name, chargeNow, got, c.want) + } + } +} + +// R7, seeds 207534 and 207536: the floor judges FTW's own discharge orders +// (status.dispatch) against the higher of the inverter's 5 % and the +// planner's 10 %, with no extra 0.5 %. A forced discharge the inverter +// still holds while FTW dispatches nothing is not FTW's order. +func TestInvariantsFloorJudgesFTWsOrders(t *testing.T) { + for _, c := range []struct { + name string + socPct float64 + orderW float64 // FTW's dispatch; 0: none + forcedW float64 // the inverter's register + want bool + }{ + {"FTW orders 3 kW at 5.0 %", 5, -3000, 3000, false}, + {"FTW orders 3 kW at 9 %, under the planner's 10 %", 9, -3000, 3000, false}, + {"FTW orders 3 kW at 10.5 %", 10.5, -3000, 3000, true}, + {"the register holds 3 kW at 5.0 %, FTW orders nothing", 5, 0, 3000, true}, + } { + d := randomRun(61, 16, 5, func(i int, r *sample) { + r.Bat.SoCPct = c.socPct + r.Bat.EMSMode, r.Bat.EMSCmd, r.Bat.EMSSetW = 2, 187, c.forcedW + r.Status = &ftwStatus{} + if c.orderW != 0 { + r.Status.Dispatch = []ftwDispatch{{Driver: "sungrow", TargetW: c.orderW}} + } + }) + if got := judged(d)[atFloor]; got != c.want { + t.Errorf("%s: %q = %v, want %v", c.name, atFloor, got, c.want) + } + } +} + +// An over-limit stretch no outside change explains is FTW's own, whichever +// way the current flows and however it is cut up: FTW's discharge pushing +// export over the fuse, and a warm fuse held over its rating with a 1 s dip +// every 7 s, or every 4 s. A dip does not end a stretch while the heat stays +// over the fuse. All fail after 3 s. +func TestInvariantsAStretchWithNoStepIsFTWsOwn(t *testing.T) { + export := randomRun(121, 16, -10, func(i int, r *sample) { + w := 0.0 + if i >= 47 { + w, r.BatW = -5000, -5000 + } + r.Status = &ftwStatus{Dispatch: []ftwDispatch{{Driver: "sungrow", TargetW: w}}} + if i >= 50 && i < 56 { + r.PhaseA = [3]float64{-21, -21, -21} + } + }) + dips := func(every int) *runData { + return randomRun(601, 25, 25.5, func(i int, r *sample) { + if i%every == every-1 { + r.PhaseA = [3]float64{24.9, 24.9, 24.9} + } + }) + } + for name, d := range map[string]*runData{"export 6 s at 1.31 x In": export, + "600 s on a warm fuse, a dip every 7 s": dips(7), "600 s on a warm fuse, 3 s over and 1 s under": dips(4)} { + withCharger(t, d, 5, 0) + if v := judged(d); v[limitOwn] { + t.Errorf("%s: %q passed", name, limitOwn) + } + } +} + +// A window that opens inside FTW's own raise does not turn it into a step: +// FTW raises the car at 45 s and the phases sit at 31 A (1.24 x 25 A) from +// 50 to 60 s; PV moving at 51 s changes nothing, and a load step at 51 s +// that adds 1 A comes after FTW's raise broke the line. The battery is at +// its floor, so the 40 s charger would be a step's lever. +func TestInvariantsAnEventInsideFTWsRaiseLeavesItOwn(t *testing.T) { + for _, c := range []struct { + name string + event string // at 51 s; "": none + addA float64 + }{ + {"no event", "", 0}, + {"PV moves", "pv", 0}, + {"a load step adds 1 A", "loads", 1}, + } { + d := randomRun(121, 25, 10, func(i int, r *sample) { + r.Bat.SoCPct = 6 + if i >= 50 && i < 60 { + r.PhaseA = [3]float64{31, 31, 31} + } + if i >= 51 && i < 60 { + r.PhaseA[0] += c.addA + } + }) + withCharger(t, d, 40, 0) + d.frames = []frame{profileAt(40, 0), profileAt(45, 16)} + if c.event != "" { + d.events = []houseEventLine{{T: 51, Event: houseEvent{Do: c.event}}} + } + if v := judged(d); v[limitOwn] { + t.Errorf("%s: FTW's raise held 10 s at 1.24 x In and passed", c.name) + } + } +} + +// The battery is the 2 s lever for import only above the floor the floor +// check uses (the planner's 10 % here), and never while its inverter is +// out; a charger outage delays the charger. Only a meter fault adds its +// length to a step's time. 16 A fuse, a 5 s charger, 20 A on L1 (1.25 x). +func TestInvariantsLeverAndFaults(t *testing.T) { + for _, c := range []struct { + name string + socPct float64 + from, to int // 20 A on L1 in [from, to) + events []houseEventLine + want bool + }{ + {"a step at 8 %: the charger answers", 8, 50, 59, nil, true}, + {"a step at 11 %: the battery answers", 11, 50, 59, nil, false}, + {"a step while the inverter is offline", 50, 50, 59, + []houseEventLine{{T: 30, Event: houseEvent{Do: "offline", Device: "inverter", ForS: 40}}}, true}, + {"the inverter offline 40 s, 15 s over", 50, 50, 65, + []houseEventLine{{T: 45, Event: houseEvent{Do: "offline", Device: "inverter", ForS: 40}}}, false}, + {"the meter late 40 s, 15 s over", 50, 50, 65, + []houseEventLine{{T: 45, Event: houseEvent{Do: "meter_late", ForS: 40}}}, true}, + {"the charger away 20 s, 18 s over", 8, 50, 68, + []houseEventLine{{T: 45, Event: houseEvent{Do: "reconnect", Device: "garage", ForS: 20}}}, true}, + } { + d := randomRun(121, 16, 8, func(i int, r *sample) { + r.Bat.SoCPct = c.socPct + if i >= c.from && i < c.to { + r.PhaseA = [3]float64{20, 12, 12} + } + }) + withCharger(t, d, 5, 0) + d.events = append([]houseEventLine{{T: 49.5, Event: houseEvent{Do: "loads"}}}, c.events...) + if got := judged(d)[limitStep]; got != c.want { + t.Errorf("%s: %q = %v, want %v", c.name, limitStep, got, c.want) + } + } +} + +// forcedCase is an R8 run: the inverter's register holds a forced command +// from the start until forcedTo, and FTW dispatches the battery from +// dispatchFrom (0: never). An outage of the inverter (offline: it stops; a +// silence: it runs but does not answer) or a Core restart may come first. +type forcedCase struct { + name string + outage string // "offline", "silence" or "" + outFrom float64 // the outage, from outFrom + outFor float64 // for outFor + readAt float64 // Core's polls of the inverter fail from outFrom until here + stop, up float64 // a Core restart (SIGTERM at stop), if up > 0 + cmd float64 // 187 discharge, 170 charge + setW float64 + forcedTo float64 + dispatchFrom float64 + want bool +} + +func (c forcedCase) run() *runData { + // FTW dispatched the battery until the outage or the stop. + lead := c.outFrom + if c.up > 0 && (c.outage == "" || c.stop < lead) { + lead = c.stop + } + d := randomRun(1201, 16, 5, func(i int, r *sample) { + t := float64(i) + if t < c.forcedTo { + r.Bat.EMSMode, r.Bat.EMSCmd, r.Bat.EMSSetW = 2, c.cmd, c.setW + } else { + r.Bat.EMSCmd = 204 + } + if c.outage == "offline" && t >= c.outFrom && t < c.outFrom+c.outFor { + off := false + r.Bat.Running = &off + } + if c.up > 0 && t >= c.stop && t <= c.up { + return // Core is down: no reading + } + errs := 0 + if c.outage != "" && t >= c.outFrom && t < c.readAt { + errs = 1 + int(t-c.outFrom)/2 + } + r.Status = &ftwStatus{Drivers: map[string]ftwDriver{inverterDriver: {Status: "ok", ConsecutiveErrors: &errs}}} + if t < lead || (c.dispatchFrom > 0 && t >= c.dispatchFrom) { + r.Status.Dispatch = []ftwDispatch{{Driver: inverterDriver, TargetW: -c.setW}} + } + }) + if c.outage != "" { + d.events = []houseEventLine{{T: c.outFrom, Event: houseEvent{Do: c.outage, Device: "inverter", ForS: c.outFor}}} + } + if c.up > 0 { + d.allMarks = []markLine{{T: c.stop, Label: "ftw:restart"}, {T: c.up, Label: "ftw:up"}} + } + return d +} + +// R8, C1, on the shapes of the nightly seeds on master 83c2b24c. Once Core +// has the inverter back, a battery FTW does not dispatch leaves its forced +// mode within 10 s; #1915 (C11) sends the default on Core's first good +// read. The Modbus reconnect backoff (10 s since #1930) between the +// inverter's return and that read is Core's to take. +func TestInvariantsNoForcedBatteryWithoutDispatch(t *testing.T) { + const forced = "no forced battery mode FTW does not dispatch" + read207539 := 182 + reconnectBackoffMaxS - 4 + for _, c := range []forcedCase{ + // 207533: offline 576.6-644.7 s, Core reads 9 s later, 187 / + // 2668 W until FTW dispatches again 236 s after the return (C1). + {name: "207533: held until Core's 5 min retry", outage: "offline", outFrom: 100, outFor: 68, readAt: 177, + cmd: 187, setW: 2668, forcedTo: 1201, dispatchFrom: 404, want: false}, + // 207535: the same with a forced charge, 170 / 2632 W, for 235 s. + {name: "207535: a forced charge held", outage: "offline", outFrom: 100, outFor: 69, readAt: 177, + cmd: 170, setW: 2632, forcedTo: 404, want: false}, + // 207537: the inverter runs but does not answer for 46.4 s; Core + // reads 19.8 s after it answers again, the discharge holds 264 s. + {name: "207537: held after a silence", outage: "silence", outFrom: 100, outFor: 46, readAt: 166, + cmd: 187, setW: 3854, forcedTo: 1201, dispatchFrom: 410, want: false}, + // 207538: Core reads 2 s after the return; held to the run's end. + {name: "207538: held to the end", outage: "offline", outFrom: 100, outFor: 39, readAt: 141, + cmd: 187, setW: 4674, forcedTo: 1201, want: false}, + // The same outages once C1 is fixed: the default goes out within + // 10 s of Core's first good read. + {name: "default 8 s after Core reads again", outage: "offline", outFrom: 100, outFor: 68, readAt: 177, + cmd: 187, setW: 2668, forcedTo: 185, want: true}, + // 207539: offline 508.5-590.0 s, the Modbus backoff (60 s then) + // held Core's first good read back 55.7 s, and FTW dispatched 20 s + // after it. With #1915 (C11) the default goes out on that first + // read, so 20 s fails. + {name: "207539: 20 s after Core's first read", outage: "offline", outFrom: 100, outFor: 82, readAt: read207539, + cmd: 187, setW: 814, forcedTo: 1201, dispatchFrom: read207539 + 20, want: false}, + {name: "past the registry's 30 s retry", outage: "offline", outFrom: 100, outFor: 82, readAt: read207539, + cmd: 187, setW: 814, forcedTo: 1201, dispatchFrom: read207539 + 47, want: false}, + // Core never reads the inverter again: the count starts at the + // backoff's cap after the return all the same. + {name: "Core never reads it again", outage: "offline", outFrom: 100, outFor: 82, readAt: 1201, + cmd: 187, setW: 814, forcedTo: 1201, dispatchFrom: 290, want: false}, + {name: "FTW dispatches the discharge", cmd: 187, setW: 3000, forcedTo: 1201, dispatchFrom: 1, want: true}, + {name: "forced with no dispatch and no outage", cmd: 187, setW: 3000, forcedTo: 1201, dispatchFrom: 500, want: false}, + // Item 10, 207538's restart at 625.5 s: SIGTERM, the driver's + // cleanup writes 204 within a second, Core is back 14.6 s later + // and dispatches 6 s after that. + {name: "207538: SIGTERM restart writes the default", stop: 100, up: 115, cmd: 187, setW: 868, + forcedTo: 101, dispatchFrom: 121, want: true}, + // The same stop as SIGKILL: no cleanup, the discharge runs on + // while Core is down. + {name: "restart without cleanup", stop: 100, up: 115, cmd: 187, setW: 868, + forcedTo: 121, dispatchFrom: 121, want: false}, + // 207530: Core stops while the inverter is offline (888.7-916.4 + // s), so no cleanup can land. The inverter is back 2.8 s before + // Core (here 13 s), and the default lands as Core starts. + {name: "207530: restart while the inverter is offline", outage: "offline", outFrom: 90, outFor: 12, readAt: 116, + stop: 100, up: 115, cmd: 170, setW: 2983, forcedTo: 116, dispatchFrom: 125, want: true}, + // A Core that starts fresh has no default retry to wait for: 10 s + // from its first good read, not the 40 s an outage gets. + {name: "forced 8 s after Core starts", stop: 100, up: 101, cmd: 187, setW: 868, + forcedTo: 1201, dispatchFrom: 110, want: true}, + {name: "forced 23 s after Core starts", stop: 100, up: 101, cmd: 187, setW: 868, + forcedTo: 1201, dispatchFrom: 125, want: false}, + // The inverter is back at 102 while Core, stopped at 100 during the + // outage, stays down until 150. Core's first read is at 151 and the + // default lands 7 s later. While Core is down nothing reads the + // inverter, so the count cannot start before 151. + {name: "inverter back while Core is down for 48 s", outage: "offline", outFrom: 90, outFor: 12, readAt: 151, + stop: 100, up: 150, cmd: 187, setW: 3000, forcedTo: 158, dispatchFrom: 170, want: true}, + } { + if got := judged(c.run())[forced]; got != c.want { + t.Errorf("%s: %q = %v, want %v", c.name, forced, got, c.want) + } + } +} + +// R8 with two inverter outages: the count after the second starts at its +// own return and Core's first read after it, not the first outage's. +func TestInvariantsForcedBatteryCountsFromTheLatestReturn(t *testing.T) { + const forced = "no forced battery mode FTW does not dispatch" + // Outage 1 100-120, Core reads at 125 and dispatches 130-300. Outage 2 + // 300-340, Core reads at 345 and the default lands at 353. + run := func(forcedTo float64) *runData { + d := randomRun(601, 16, 5, func(i int, r *sample) { + tt := float64(i) + if tt < forcedTo { + r.Bat.EMSMode, r.Bat.EMSCmd, r.Bat.EMSSetW = 2, 187, 3000 + } else { + r.Bat.EMSCmd = 204 + } + if (tt >= 100 && tt < 120) || (tt >= 300 && tt < 340) { + off := false + r.Bat.Running = &off + } + errs := 0 + if (tt >= 100 && tt < 125) || (tt >= 300 && tt < 345) { + errs = 3 + } + r.Status = &ftwStatus{Drivers: map[string]ftwDriver{inverterDriver: {Status: "ok", ConsecutiveErrors: &errs}}} + if tt < 100 || (tt >= 130 && tt < 300) { + r.Status.Dispatch = []ftwDispatch{{Driver: inverterDriver, TargetW: -3000}} + } + }) + d.events = []houseEventLine{ + {T: 100, Event: houseEvent{Do: "offline", Device: "inverter", ForS: 20}}, + {T: 300, Event: houseEvent{Do: "offline", Device: "inverter", ForS: 40}}, + } + return d + } + if !judged(run(353))[forced] { + t.Errorf("default 8 s after Core reads again after the second outage: got fail, want pass") + } + if judged(run(1201))[forced] { + t.Errorf("forced to the end after the second outage: got pass, want fail") + } +} + +// R8 forgives the seconds Core's reconnect backoff and default retry hold +// its first write after a return (reconnectBackoffMaxS, defaultRetryMaxS). +// The triage line is 10 s from the return, so the judge records what it +// forgave beyond that line instead of hiding it. +func TestInvariantsRecordForcedSecondsInsideCoreTimers(t *testing.T) { + // 207539's shape: back at 182, Core reads 4 s before the backoff's cap + // ends (238 with 60 s) and dispatches 7 s later, inside the backoff + // alone. Forced with no dispatch until then; past 182+10 that is from + // 193 (52 s with 60 s). + read := 182 + reconnectBackoffMaxS - 4 + c := forcedCase{outage: "offline", outFrom: 100, outFor: 82, readAt: read, + cmd: 187, setW: 814, forcedTo: 1201, dispatchFrom: read + 7} + m, _ := judgeInvariants(c.run()) + if in, past, want := m["battery_forced_in_core_timers_s"], m["battery_forced_past_allowance_s"], read+6-192; in != want || past != 0.0 { + t.Errorf("in Core's timers %v s, past the allowance %v s; want %v and 0", in, past, want) + } + // Forced to 299: from 193 to the allowance's end inside the timers, + // the rest past them. + c.dispatchFrom = 300 + m, _ = judgeInvariants(c.run()) + end := read + defaultRetryMaxS + forcedBudgetS + if in, past := m["battery_forced_in_core_timers_s"], m["battery_forced_past_allowance_s"]; in != end-192 || past != 299-end { + t.Errorf("in Core's timers %v s, past the allowance %v s; want %v and %v", in, past, end-192, 299-end) + } +} + +// R8's allowances are Core's own timers. Read them from Core's source so +// they cannot drift. Once the registry retries a failed default on the +// first good poll after an outage (#1915, C11), defaultRetryMaxS must be +// 0: from then on the 30 s the judge gives today is no longer Core's. +func TestForcedAllowancesMatchCore(t *testing.T) { + src := func(p string) string { + b, err := os.ReadFile(filepath.Join("..", "..", "internal", p)) + if err != nil { + t.Fatal(err) + } + return string(b) + } + seconds := func(code, name string) float64 { + m := regexp.MustCompile(name + `\s*=\s*(\d+)\s*\*\s*time\.Second`).FindStringSubmatch(code) + if m == nil { + t.Fatalf("no %s = N * time.Second in Core's source", name) + } + n, _ := strconv.Atoi(m[1]) + return float64(n) + } + // reconnectBackoffMaxS is read from Core's source; the 207539 shapes + // below need a backoff that can hold Core's first read back past 4 s. + if reconnectBackoffMaxS < 5 { + t.Errorf("Core's reconnectBackoffMax is %v s; rework the 207539 shapes", reconnectBackoffMaxS) + } + registry := src("drivers/registry.go") + // Today the registry sends the default on a failure and on its retry + // timer: two calls. #1915 adds a third, on the first good poll. + calls := strings.Count(registry, "attemptDefault()") + switch { + case calls == 2 && defaultRetryMaxS != seconds(registry, "defaultRetryMax"): + t.Errorf("defaultRetryMaxS = %v, Core's defaultRetryMax is %v s", defaultRetryMaxS, seconds(registry, "defaultRetryMax")) + case calls > 2 && defaultRetryMaxS != 0: + t.Errorf("Core's registry calls attemptDefault() in %d places, not 2. If it now retries "+ + "the default on the first good poll after an outage (#1915, C11), set defaultRetryMaxS "+ + "to 0 and turn the R8 cases that lean on it to fail. Otherwise update this check.", calls) + case calls < 2: + t.Errorf("Core's registry calls attemptDefault() in %d places; update this check", calls) + } +} + +// TestJudge also judges the random seeds a results directory holds, from +// the plan each run wrote. +func TestRecordedRandomFindsSeedRuns(t *testing.T) { + dir := t.TempDir() + for _, seed := range []int64{7, 9} { + b, _ := json.Marshal(drawPlan(seed)) + os.MkdirAll(filepath.Join(dir, fmt.Sprintf("random-%d", seed)), 0o755) + os.WriteFile(filepath.Join(dir, fmt.Sprintf("random-%d", seed), "plan.json"), b, 0o644) + } + var got []string + for _, sc := range recordedRandom(dir) { + got = append(got, sc.name+"@"+sc.dir) + } + if len(got) != 2 || got[0] != "r7@random-7" || got[1] != "r9@random-9" { + t.Fatalf("recordedRandom = %v", got) + } +} + +// R2 after the second review of #1920: a house event that is no fault +// extends a step's stretch only if the worst phase rose 0.5 A or more +// within 3 s of it. 16 A fuse, the battery at its floor and a 40 s charger, +// so a step allows 45 s; 20 A on L1 (1.25 x) from 12 s, after a load step +// at 10 s. +func TestInvariantsOnlyARiseExtendsAStretch(t *testing.T) { + for _, c := range []struct { + name string + to int // 20 A on L1 until here + riseAt int // L1 rises to 21 A from here; 0: never + every float64 // a load event that changes nothing, this often; 0: none + want bool + }{ + {"a load event at 50 s raises L1", 80, 50, 0, true}, + {"load events every 40 s change nothing", 290, 0, 40, false}, + } { + d := randomRun(301, 16, 8, func(i int, r *sample) { + r.Bat.SoCPct = 6 + if i >= 12 && i < c.to { + r.PhaseA = [3]float64{20, 12, 12} + if c.riseAt > 0 && i >= c.riseAt { + r.PhaseA[0] = 21 + } + } + }) + withCharger(t, d, 40, 0) + d.events = []houseEventLine{{T: 10, Event: houseEvent{Do: "loads"}}} + if c.riseAt > 0 { + d.events = append(d.events, houseEventLine{T: float64(c.riseAt), Event: houseEvent{Do: "loads"}}) + } + for t0 := 50.0; c.every > 0 && t0 < 290; t0 += c.every { + d.events = append(d.events, houseEventLine{T: t0, Event: houseEvent{Do: "loads"}}) + } + if got := judged(d)[limitStep]; got != c.want { + t.Errorf("%s: %q = %v, want %v", c.name, limitStep, got, c.want) + } + } +} + +// The inverter is back at the end of its outage: its return is no step +// unless the current rises, and then the battery answers in 2 s, not the +// charger. 16 A fuse, a 40 s charger, the inverter out from 10 to 40 s; +// the outage allows until 55 s. +func TestInvariantsTheInverterIsBackAtItsReturn(t *testing.T) { + for _, c := range []struct { + name string + from, to int // 20 A on L1 in [from, to) + }{ + {"over from 12 to 80 s, the return changes nothing", 12, 80}, + {"the current rises at the return, over until 60 s", 40, 60}, + } { + d := randomRun(121, 16, 8, func(i int, r *sample) { + run := !(i >= 10 && i < 40) + r.Bat.Running = &run + if i >= c.from && i < c.to { + r.PhaseA = [3]float64{20, 12, 12} + } + }) + withCharger(t, d, 40, 0) + d.events = []houseEventLine{ + {T: 10, Event: houseEvent{Do: "offline", Device: "inverter", ForS: 30}}, + {T: 40, Event: houseEvent{Do: "online", Device: "inverter"}}, + } + if judged(d)[limitStep] { + t.Errorf("%s: %q passed", c.name, limitStep) + } + } +} + +// R4 after the second review of #1920: a car raise FTW sends after a +// step's meter delay, or after a Core restart, is FTW's own; one sent +// before the meter shows the step is part of the step. 25 A fuse, the +// battery at its floor and a 40 s charger; the phases sit at 31 A +// (1.24 x) from 50 to 80 s. +func TestInvariantsARaiseAfterAStepIsFTWsOwn(t *testing.T) { + for _, c := range []struct { + name string + raiseAt float64 + event bool // PV moves at 40 s + restart bool // Core restarts at 30 s and is back at 35 s + want bool // both limit criteria hold + }{ + {"FTW raises 5 s after PV moved", 45, true, false, false}, + {"FTW raises 1 s after PV moved", 41, true, false, true}, + {"FTW raises 15 s after Core is back", 50, false, true, false}, + } { + d := randomRun(121, 25, 10, func(i int, r *sample) { + r.Bat.SoCPct = 6 + if i >= 50 && i < 80 { + r.PhaseA = [3]float64{31, 31, 31} + r.CarW[0] = 11040 + } + }) + withCharger(t, d, 40, 0) + d.frames = []frame{profileAt(20, 0), profileAt(c.raiseAt, 16)} + if c.event { + d.events = []houseEventLine{{T: 40, Event: houseEvent{Do: "pv", Device: "inverter"}}} + } + if c.restart { + d.allMarks = []markLine{{T: 30, Label: "ftw:restart"}, {T: 35, Label: "ftw:up"}} + } + v := judged(d) + if got := v[limitStep] && v[limitOwn]; got != c.want { + t.Errorf("%s: step %v, own %v; want both %v", c.name, v[limitStep], v[limitOwn], c.want) + } + } +} + +// A Core restart explains a stretch for 60 s from the restart. 16 A fuse, +// Core restarts at 50 s and is back at 55 s; 20 A on L1 (1.25 x). +func TestInvariantsARestartExplainsAMinute(t *testing.T) { + for _, c := range []struct { + name string + from, to int + want bool + }{ + {"over from 100 to 108 s", 100, 108, true}, + {"over from 115 to 125 s", 115, 125, false}, + } { + d := randomRun(201, 16, 8, func(i int, r *sample) { + if i >= c.from && i < c.to { + r.PhaseA = [3]float64{20, 12, 12} + } + }) + withCharger(t, d, 5, 0) + d.allMarks = []markLine{{T: 50, Label: "ftw:restart"}, {T: 55, Label: "ftw:up"}} + v := judged(d) + if got := v[limitStep] && v[limitOwn]; got != c.want { + t.Errorf("%s: step %v, own %v; want both %v", c.name, v[limitStep], v[limitOwn], c.want) + } + } +} + +// For export the battery's spare is what it can charge more: a battery +// charging 4 kW has 1 kW, too little for a 4 A overage, so the 5 s charger +// answers; one discharging 4 kW has 9 kW and answers in 2 s, even at its +// floor. 16 A fuse; -20 A on L1 from 50 to 59 s after a step at 49.5 s. +func TestInvariantsBatteryLeverForExport(t *testing.T) { + for _, c := range []struct { + name string + batW float64 + socPct float64 + want bool + }{ + {"charging 4 kW", 4000, 50, true}, + {"discharging 4 kW", -4000, 50, false}, + {"idle at its floor", 0, 6, false}, + } { + d := randomRun(101, 16, -8, func(i int, r *sample) { + r.BatW, r.Bat.SoCPct = c.batW, c.socPct + if i >= 50 && i < 59 { + r.PhaseA = [3]float64{-20, -12, -12} + } + }) + withCharger(t, d, 5, 0) + d.events = []houseEventLine{{T: 49.5, Event: houseEvent{Do: "pv", Device: "inverter"}}} + if got := judged(d)[limitStep]; got != c.want { + t.Errorf("%s: %q = %v, want %v", c.name, limitStep, got, c.want) + } + } +} + +// A dip under the fuse keeps the stretch open while the heat stays over +// it, but a new step after the dip still gets its own time (2 + 5 s for +// the battery), after a step or after FTW's own raise that was cut in +// time. A step held past its time fails as a step. 16 A fuse at 15.95 A, +// so the heat stays over the fuse for a minute after a few seconds at 20 A +// on L1. +func TestInvariantsAStepAfterADipGetsItsTime(t *testing.T) { + for _, c := range []struct { + name string + first []houseEventLine // the change before 50 s; none: FTW's own + stepTo int // the step at 80 s holds 20 A until here + wantStep, wantOwn bool + }{ + {"after a step", []houseEventLine{{T: 49.5, Event: houseEvent{Do: "loads"}}}, 84, true, true}, + {"after FTW's own raise", nil, 84, true, true}, + {"held 8 s after FTW's own raise", nil, 88, false, true}, + } { + d := randomRun(201, 16, 15.95, func(i int, r *sample) { + if (i >= 50 && i < 53) || (i >= 80 && i < c.stepTo) { + r.PhaseA[0] = 20 + } + }) + withCharger(t, d, 5, 0) + d.events = append(c.first, houseEventLine{T: 80, Event: houseEvent{Do: "loads"}}) + d.warm() + if h := d.at(79).HeatA[0]; h <= 16 { + t.Fatalf("%s: heat %.2f A at 79 s, want over the fuse", c.name, h) + } + if v := judged(d); v[limitStep] != c.wantStep || v[limitOwn] != c.wantOwn { + t.Errorf("%s: step %v, own %v; want %v, %v", c.name, v[limitStep], v[limitOwn], c.wantStep, c.wantOwn) + } + } +} + +// R4 after the third review of #1920: a car or battery raise FTW sends +// while an import stretch is over the fuse is FTW's own, after a step's +// meter delay too, and a house event that changes nothing does not take +// the rise from it. One sent in a dip under the fuse, with the heat still +// over it, is allowed: the step after it gets its time. Raising the car +// into export is the right answer. 40 s charger, the battery at its +// floor. +func TestInvariantsARaiseOverTheFuseIsFTWsOwn(t *testing.T) { + for _, c := range []struct { + name string + fuse float64 + set func(i int, r *sample) + frames []frame + events []houseEventLine + want bool // both limit criteria hold + }{ + {"0 -> 16 A at 1.24 x In after PV moved", 25, func(i int, r *sample) { + if i >= 41 && i < 84 { + r.PhaseA = [3]float64{31, 31, 31} + } + if i >= 52 && i < 84 { + r.PhaseA = [3]float64{42, 42, 42} + } + }, []frame{profileAt(20, 0), profileAt(50, 16)}, + []houseEventLine{{T: 40, Event: houseEvent{Do: "pv", Device: "inverter"}}}, false}, + {"6 -> 10 A at 1.25 x In, a load event that changes nothing as it lands", 16, func(i int, r *sample) { + if i >= 11 && i < 75 { + r.PhaseA = [3]float64{20, 12, 12} + } + if i >= 32 && i < 75 { + r.PhaseA[0] = 22 + } + }, []frame{profileAt(1, 6), profileAt(30, 10)}, + []houseEventLine{{T: 10, Event: houseEvent{Do: "loads"}}, {T: 31, Event: houseEvent{Do: "loads"}}}, false}, + {"battery charge 0 -> 3 kW at 1.25 x In after a load step", 16, func(i int, r *sample) { + r.Status = &ftwStatus{Dispatch: []ftwDispatch{{Driver: "sungrow", TargetW: 0}}} + if i >= 30 { + r.Status.Dispatch[0].TargetW, r.BatW = 3000, 3000 + } + if i >= 11 && i < 55 { + r.PhaseA = [3]float64{20, 12, 12} + } + if i >= 32 && i < 55 { + r.PhaseA = [3]float64{22, 16, 16} + } + }, nil, []houseEventLine{{T: 10, Event: houseEvent{Do: "loads"}}}, false}, + {"car and battery raised in a dip under the fuse, a step after it", 16, func(i int, r *sample) { + r.Status = &ftwStatus{Dispatch: []ftwDispatch{{Driver: "sungrow", TargetW: 0}}} + if i >= 56 { + r.Status.Dispatch[0].TargetW = 2000 + } + switch { + case i >= 11 && i < 45, i >= 71 && i < 76: + r.PhaseA = [3]float64{22, 12, 12} + case i >= 45 && i < 71: + r.PhaseA = [3]float64{15, 12, 12} + } + }, []frame{profileAt(1, 6), profileAt(55, 8)}, + []houseEventLine{{T: 10, Event: houseEvent{Do: "loads"}}, {T: 70, Event: houseEvent{Do: "loads"}}}, true}, + {"0 -> 16 A into export over the fuse after PV moved", 16, func(i int, r *sample) { + r.PhaseA, r.BatW = [3]float64{-8, -8, -8}, 4000 + if i >= 11 && i < 45 { + r.PhaseA = [3]float64{-24, -12, -12} + } + }, []frame{profileAt(1, 0), profileAt(20, 16)}, + []houseEventLine{{T: 10, Event: houseEvent{Do: "pv", Device: "inverter"}}}, true}, + } { + d := randomRun(121, c.fuse, 12, func(i int, r *sample) { + r.Bat.SoCPct = 6 + c.set(i, r) + }) + withCharger(t, d, 40, 0) + d.frames, d.events = c.frames, c.events + v := judged(d) + if got := v[limitStep] && v[limitOwn]; got != c.want { + t.Errorf("%s: step %v, own %v; want both %v", c.name, v[limitStep], v[limitOwn], c.want) + } + } +} + +// R4 for the battery: a charge raise FTW sends after a Core restart is +// its own, as a car raise is (triage C2). An order moved while the +// inverter is off reaches no battery. 16 A fuse, Core restarts at 50 s +// and is back at 55 s; the order goes 0 -> 4 kW at 60 s; 21 A on L1 +// (1.31 x) from 61 to 105 s. +func TestInvariantsABatteryRaiseAfterARestartIsFTWsOwn(t *testing.T) { + for _, c := range []struct { + name string + running bool // the inverter runs from 55 to 70 s + want bool + }{ + {"the inverter runs", true, false}, + {"the inverter is off", false, true}, + } { + d := randomRun(201, 16, 8, func(i int, r *sample) { + w := 0.0 + if i >= 60 { + w = 4000 + } + r.Status = &ftwStatus{Dispatch: []ftwDispatch{{Driver: "sungrow", TargetW: w}}} + if i >= 55 && i < 70 { + run := c.running + r.Bat.Running = &run + } + if i >= 61 && c.running { + r.BatW = w + } + if i >= 61 && i < 105 { + r.PhaseA = [3]float64{21, 13, 13} + } + }) + withCharger(t, d, 5, 0) + d.allMarks = []markLine{{T: 50, Label: "ftw:restart"}, {T: 55, Label: "ftw:up"}} + v := judged(d) + if got := v[limitStep] && v[limitOwn]; got != c.want { + t.Errorf("%s: step %v, own %v; want both %v", c.name, v[limitStep], v[limitOwn], c.want) + } + } +} + +// Fourth review of #1920: what FTW's battery order did, read correctly. +// While Core is down, or before its first tick, status has no dispatch, +// but the inverter keeps the last order: a missing order is no move. A +// move counts from 0.5 A a phase (345 W), as raisedAt does, so a one- +// sample 158 W wobble of Core's closed loop (seed 207539 at 151.7 s) is +// no raise. 16 A fuse, a load step at 40 s puts 20 A on L1. +func TestInvariantsABatteryOrderMovesOnlyWhenFTWMovesIt(t *testing.T) { + order := func(r *sample, w float64) { + r.Status = &ftwStatus{Dispatch: []ftwDispatch{{Driver: "sungrow", TargetW: w}}} + } + for _, c := range []struct { + name string + set func(i int, r *sample) + down bool // Core restarts at 44 s and is back at 49 s + want bool // both limit criteria hold + }{ + // At 6 % SoC the battery charging 1 kW cannot take 4 A a phase + // off, so the charger is the lever; Core's restart extends the + // stretch to 104 s. + {"1 kW order kept while Core is down", func(i int, r *sample) { + r.BatW, r.Bat.SoCPct = 1000, 6 + order(r, 1000) + if i >= 41 && i < 80 { + r.PhaseA = [3]float64{20, 8, 8} + } + }, true, true}, + {"1 kW order missing while Core is down, then restored", func(i int, r *sample) { + r.BatW, r.Bat.SoCPct = 1000, 6 + if i < 44 || i >= 50 { + order(r, 1000) + } + if i >= 41 && i < 80 { + r.PhaseA = [3]float64{20, 8, 8} + } + }, true, true}, + {"0 W order missing while Core is down, then 1 kW", func(i int, r *sample) { + r.Bat.SoCPct = 6 + switch { + case i < 44: + order(r, 0) + case i >= 50: + order(r, 1000) + r.BatW = 1000 + } + if i >= 41 && i < 80 { + r.PhaseA = [3]float64{20, 8, 8} + } + }, true, false}, + // The battery discharges 4 999 W and answers in 2 s: allowed + // until 47 s. + {"steady discharge order", func(i int, r *sample) { + r.BatW = -4999 + order(r, -4999) + if i >= 41 && i < 46 { + r.PhaseA = [3]float64{20, 8, 8} + } + }, false, true}, + {"one 158 W wobble toward import", func(i int, r *sample) { + r.BatW = -4999 + order(r, -4999) + if i == 43 { + order(r, -4841) + } + if i >= 41 && i < 46 { + r.PhaseA = [3]float64{20, 8, 8} + } + }, false, true}, + {"one 360 W move toward import", func(i int, r *sample) { + r.BatW = -4999 + order(r, -4999) + if i == 43 { + order(r, -4639) + } + if i >= 41 && i < 46 { + r.PhaseA = [3]float64{20, 8, 8} + } + }, false, false}, + } { + d := randomRun(121, 16, 8, c.set) + withCharger(t, d, 40, 0) + d.events = []houseEventLine{{T: 40, Event: houseEvent{Do: "loads"}}} + if c.down { + d.allMarks = []markLine{{T: 44, Label: "ftw:restart"}, {T: 49, Label: "ftw:up"}} + } + m, _ := judgeInvariants(d) + v := judged(d) + if got := v[limitStep] && v[limitOwn]; got != c.want { + t.Errorf("%s: step %v, own %v; want both %v; judged %v", c.name, v[limitStep], v[limitOwn], c.want, m["over_limit_judged"]) + } + } +} + +// Export is judged by what pushes export. A car raise before an export +// stretch soaks it up, so the stretch keeps its step's time. A battery +// pushed toward export after a Core restart is FTW's own push: the +// restart's minute does not excuse it. +func TestInvariantsExportIsFTWsOwnOnlyWhenFTWPushedExport(t *testing.T) { + t.Run("car raise before an export stretch", func(t *testing.T) { + off := false + d := randomRun(121, 16, 0, func(i int, r *sample) { + r.Bat.Running = &off + if i >= 45 && i < 80 { + r.PhaseA = [3]float64{-20, -20, -20} + } + }) + withCharger(t, d, 40, 0) + d.frames = []frame{profileAt(20, 0), profileAt(43, 16)} + d.events = []houseEventLine{{T: 40, Event: houseEvent{Do: "pv", Device: "inverter"}}} + m, _ := judgeInvariants(d) + if v := judged(d); !(v[limitStep] && v[limitOwn]) { + t.Errorf("step %v, own %v; judged %v", v[limitStep], v[limitOwn], m["over_limit_judged"]) + } + }) + for _, c := range []struct { + name string + running bool // the inverter runs from 55 s + want bool + }{ + {"battery pushed 0 -> -5 kW after a restart", true, false}, + {"the same order with the inverter off", false, true}, + } { + t.Run(c.name, func(t *testing.T) { + d := randomRun(201, 16, 0, func(i int, r *sample) { + w := 0.0 + if i >= 57 { + w = -5000 + } + r.Status = &ftwStatus{Dispatch: []ftwDispatch{{Driver: "sungrow", TargetW: w}}} + if i >= 55 && !c.running { + run := false + r.Bat.Running = &run + } + if i >= 58 && c.running { + r.BatW = w + } + if i >= 58 && i < 100 { + r.PhaseA = [3]float64{-21, -10, -10} + } + }) + withCharger(t, d, 5, 0) + d.allMarks = []markLine{{T: 50, Label: "ftw:restart"}, {T: 55, Label: "ftw:up"}} + m, _ := judgeInvariants(d) + v := judged(d) + if got := v[limitStep] && v[limitOwn]; got != c.want { + t.Errorf("step %v, own %v; want both %v; judged %v", v[limitStep], v[limitOwn], c.want, m["over_limit_judged"]) + } + }) + } + // Inside an export stretch after a PV step. The battery charges at + // its maximum, so the 40 s charger is the lever. Moving its order + // toward export while the meter is over the fuse is FTW's own; the + // same move in a dip under the fuse, with a step after it, is not. + for _, c := range []struct { + name string + moveAt int // the order goes 5 -> 4 kW; 0: no move + dip bool + want bool + }{ + {"no move", 0, false, true}, + {"battery moved toward export over the fuse", 20, false, false}, + {"the same move in a dip, a step after it", 56, true, true}, + } { + t.Run(c.name, func(t *testing.T) { + d := randomRun(121, 16, -12, func(i int, r *sample) { + r.BatW = rigBatteryMaxW + w := rigBatteryMaxW + if c.moveAt > 0 && i >= c.moveAt { + w = 4000 + } + r.Status = &ftwStatus{Dispatch: []ftwDispatch{{Driver: "sungrow", TargetW: w}}} + switch { + case !c.dip && i >= 11 && i < 45: + r.PhaseA = [3]float64{-22, -12, -12} + case c.dip && (i >= 11 && i < 45 || i >= 71 && i < 76): + r.PhaseA = [3]float64{-22, -12, -12} + case c.dip && i >= 45 && i < 71: + r.PhaseA = [3]float64{-15, -12, -12} + } + }) + withCharger(t, d, 40, 0) + d.events = []houseEventLine{{T: 10, Event: houseEvent{Do: "pv", Device: "inverter"}}} + if c.dip { + d.events = append(d.events, houseEventLine{T: 70, Event: houseEvent{Do: "pv", Device: "inverter"}}) + } + m, _ := judgeInvariants(d) + v := judged(d) + if got := v[limitStep] && v[limitOwn]; got != c.want { + t.Errorf("step %v, own %v; want both %v; judged %v", v[limitStep], v[limitOwn], c.want, m["over_limit_judged"]) + } + }) + } +} + +// The line is In, not 1.2 x In: a car raise sent while the meter shows +// 1.1 x In (under the instant cut, the heat still cool) is sent over the +// line, and the stretch is FTW's own. +func TestInvariantsARaiseBetweenFuseAndInstantIsOwn(t *testing.T) { + d := randomRun(121, 25, 12, func(i int, r *sample) { + r.Bat.SoCPct = 6 + if i >= 41 && i < 84 { + r.PhaseA = [3]float64{27.5, 27.5, 27.5} + } + if i >= 52 && i < 84 { + r.PhaseA = [3]float64{42, 42, 42} + } + }) + withCharger(t, d, 40, 0) + d.frames = []frame{profileAt(20, 0), profileAt(50, 16)} + d.events = []houseEventLine{{T: 40, Event: houseEvent{Do: "pv", Device: "inverter"}}} + m, _ := judgeInvariants(d) + if v := judged(d); v[limitStep] && v[limitOwn] { + t.Errorf("a raise sent at 1.1 x In passed; judged %v", m["over_limit_judged"]) + } +} + +// Owner, 28 Oct 2026: what events add to a stretch stops at the fuse's +// heat budget. Once L1's 60 s heat passes the fuse, Core's cut has its +// lever's time (here the 5 s charger: the battery is at its floor), and +// no further event extends the stretch, however many come. 16 A fuse, +// 8 A on each phase, a load step at 10 s puts L1 at 20 A (1.25 x In) +// from 11 s; a load event every 8 s from 18 s raises it 0.5 A from a +// 1 s dip (the third review's E3). L1's heat passes the fuse at 62 s, so +// the cap is 62 + 5 + 2 + 3 = 72 s. A Core restart at 15 s, which would +// excuse a minute, is capped too: with no dips the heat passes at 61 s. +func TestInvariantsHeatCapsWhatEventsAdd(t *testing.T) { + for _, c := range []struct { + name string + to int // L1 over the fuse until here + restartAt float64 // a Core restart instead of the load events; 0: none + want bool + }{ + {"events end before the heat passes the fuse", 55, 0, true}, + {"events end inside Core's cut time", 72, 0, true}, + {"events past the heat's cap", 76, 0, false}, + {"events chained 280 s", 290, 0, false}, + {"a Core restart past the heat's cap", 74, 15, false}, + {"a Core restart before the heat passes the fuse", 55, 15, true}, + } { + d := randomRun(301, 16, 8, func(i int, r *sample) { + r.Bat.SoCPct = 6 + if i >= 11 && i < c.to { + r.PhaseA[0] = 20 + if c.restartAt == 0 && i >= 17 && (i-17)%8 == 0 { + r.PhaseA[0] = 19.5 + } + } + }) + withCharger(t, d, 5, 0) + d.events = []houseEventLine{{T: 10, Event: houseEvent{Do: "loads"}}} + if c.restartAt > 0 { + d.allMarks = []markLine{{T: c.restartAt, Label: "ftw:restart"}, {T: c.restartAt + 9, Label: "ftw:up"}} + } else { + for at := 18.0; at < float64(c.to); at += 8 { + d.events = append(d.events, houseEventLine{T: at, Event: houseEvent{Do: "loads"}}) + } + } + d.warm() + if h := d.at(61).HeatA[0]; c.restartAt == 0 && h > 16 { + t.Fatalf("%s: heat %.2f A at 61 s, want under the fuse", c.name, h) + } + if h := d.at(62).HeatA[0]; c.to > 62 && h <= 16 { + t.Fatalf("%s: heat %.2f A at 62 s, want over the fuse", c.name, h) + } + m, _ := judgeInvariants(d) + if got := judged(d)[limitStep]; got != c.want { + t.Errorf("%s: %q = %v, want %v; judged %v", c.name, limitStep, got, c.want, m["over_limit_judged"]) + } + } +} + +// Owner, 28 Oct 2026: a car cut FTW sends that pushes export over the +// fuse is FTW's own, as a battery move toward export is: after a step's +// 2 s meter delay and before the stretch, or while the stretch is over +// the fuse. A cut the meter cannot have shown the step for yet is part +// of the step. 16 A fuse, 12 A export per phase, the battery charging at +// its maximum, so the 40 s charger is the lever; PV steps at 10 s and +// L1 exports 22 A from `from` to 45 s, inside the step's 55 s. +func TestInvariantsACarCutIntoExportIsFTWsOwn(t *testing.T) { + for _, c := range []struct { + name string + from int + frames []frame + want bool // both limit criteria hold + }{ + {"no cut", 11, []frame{profileAt(1, 16)}, true}, + {"16 -> 0 A while export is over the fuse", 11, []frame{profileAt(1, 16), profileAt(20, 0)}, false}, + {"16 -> 6 A after the meter shows the step, before the stretch", 20, []frame{profileAt(1, 16), profileAt(15, 6)}, false}, + {"16 -> 6 A before the meter shows the step", 20, []frame{profileAt(1, 16), profileAt(11, 6)}, true}, + {"6 -> 16 A while export is over the fuse", 11, []frame{profileAt(1, 6), profileAt(20, 16)}, true}, + } { + d := randomRun(121, 16, -12, func(i int, r *sample) { + r.BatW = rigBatteryMaxW + r.Status = &ftwStatus{Dispatch: []ftwDispatch{{Driver: "sungrow", TargetW: rigBatteryMaxW}}} + if i >= c.from && i < 45 { + r.PhaseA[0] = -22 + } + }) + withCharger(t, d, 40, 0) + d.frames = c.frames + d.events = []houseEventLine{{T: 10, Event: houseEvent{Do: "pv", Device: "inverter"}}} + m, _ := judgeInvariants(d) + v := judged(d) + if got := v[limitStep] && v[limitOwn]; got != c.want { + t.Errorf("%s: step %v, own %v; want both %v; judged %v", c.name, v[limitStep], v[limitOwn], c.want, m["over_limit_judged"]) + } + } +} diff --git a/go/test/site/rig_matches_box_test.go b/go/test/site/rig_matches_box_test.go new file mode 100644 index 000000000..dc827e52b --- /dev/null +++ b/go/test/site/rig_matches_box_test.go @@ -0,0 +1,76 @@ +//go:build unix + +package site + +import ( + "context" + "os" + "path/filepath" + "reflect" + "strings" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/config" + "github.com/srcfl/ftw/go/internal/prices" +) + +// A beta or stable Linux box plans with Energyplan in passive or active +// arbitrage. The rig must exercise that path by default. +func TestRigRunsWhatABetaBoxRuns(t *testing.T) { + t.Setenv("FTW_SITE_ENGINE", "") + st := &stack{} + want := (&config.Planner{Enabled: true}).EngineForBuild("v0.157.2-beta.1", "linux", "arm64") + if got := st.engine(); got != want { + t.Errorf("rig default engine = %q, a v0.157.2-beta.1 linux/arm64 box runs %q", got, want) + } + files, _ := filepath.Glob("*_test.go") + arbitrage := false + for _, f := range files { + if f == "rig_matches_box_test.go" { + continue + } + b, _ := os.ReadFile(f) + if strings.Contains(string(b), "planner_passive_arbitrage") || strings.Contains(string(b), "planner_arbitrage") { + arbitrage = true + } + } + if !arbitrage { + t.Errorf("no site-rig scenario selects planner_passive_arbitrage or planner_arbitrage") + } +} + +// Two runs that start on different days, at different times, read the same +// 24 h of prices from their start, and nothing from the live market. +func TestRigPricesArePinnedToTheRunStart(t *testing.T) { + loc, err := time.LoadLocation("Europe/Stockholm") + if err != nil { + t.Skip("no time zone data:", err) + } + read := func(start time.Time) []float64 { + ps := newPriceServer(start) + defer ps.close() + svc := prices.FromConfig(&config.Price{Provider: "elprisetjustnu", Zone: "SE3", BaseURL: ps.url()}, nil, nil) + var out []float64 + next := ps.start + for _, day := range []time.Time{start, start.AddDate(0, 0, 1)} { + rows, err := svc.Provider.Fetch(context.Background(), "SE3", day.In(time.Local)) + if err != nil { + t.Fatalf("start %s, day %s: %v", start, day.Format("2006-01-02"), err) + } + for _, r := range rows { + if !r.SlotStart.Equal(next) || r.SlotLenMin != 15 { + t.Fatalf("start %s: row %s/%d min, want %s/15 min", start, r.SlotStart, r.SlotLenMin, next) + } + next = next.Add(15 * time.Minute) + out = append(out, r.SEKPerKWh) + } + } + return out + } + a := read(time.Date(2026, 10, 27, 14, 7, 0, 0, loc)) + b := read(time.Date(2026, 7, 3, 23, 52, 0, 0, loc)) + if len(a) != 96 || !reflect.DeepEqual(a, b) { + t.Fatalf("prices differ between runs:\n%v\n%v", a, b) + } +} diff --git a/go/test/site/rig_test.go b/go/test/site/rig_test.go index 35184ffb2..f507fc33d 100644 --- a/go/test/site/rig_test.go +++ b/go/test/site/rig_test.go @@ -204,12 +204,13 @@ func split(scs []scenario, n int) [][]scenario { type outcome struct { Name string `json:"name"` Dir string `json:"dir"` - Verdict string `json:"verdict"` // pass, known, fail + Verdict string `json:"verdict"` // pass, known, fail, skip DurationS float64 `json:"duration_s"` Criteria []criterion `json:"criteria"` Known []string `json:"known,omitempty"` Failures []string `json:"failures,omitempty"` Error string `json:"error,omitempty"` + Skipped string `json:"skipped,omitempty"` // what the simulator lacks } // TestSite runs the scenarios FTW_SITE names. FTW_SITE_LANES runs that many @@ -239,6 +240,16 @@ func TestSite(t *testing.T) { env := setup(t) var mu sync.Mutex var outcomes []outcome + var runnable []scenario + for _, sc := range scs { + if sc.skip == "" { + runnable = append(runnable, sc) + continue + } + t.Logf("%s skipped: %s", sc.name, sc.skip) + outcomes = append(outcomes, outcome{Name: sc.name, Dir: sc.dir, Verdict: "skip", Skipped: sc.skip}) + } + scs = runnable t.Run("lanes", func(t *testing.T) { for k, group := range split(scs, lanes) { k, group := k, group @@ -364,7 +375,7 @@ func TestJudge(t *testing.T) { t.Skip("set FTW_SITE_JUDGE to a results directory of an earlier run") } var outs []outcome - for _, sc := range selected(t, "full") { + for _, sc := range append(selected(t, "full"), recordedRandom(dir)...) { rundir := filepath.Join(dir, sc.dir) b, err := os.ReadFile(filepath.Join(rundir, "meta.json")) if err != nil { @@ -392,6 +403,25 @@ func TestJudge(t *testing.T) { writeSummary(t, dir, outs) } +// recordedRandom is each random seed recorded in dir, from the plan.json +// its run wrote, so TestJudge judges a nightly random run again. +func recordedRandom(dir string) []scenario { + files, _ := filepath.Glob(filepath.Join(dir, "random-*", "plan.json")) + sort.Strings(files) + var out []scenario + for _, f := range files { + b, err := os.ReadFile(f) + if err != nil { + continue + } + var p randomPlan + if json.Unmarshal(b, &p) == nil && p.Found == nil { + out = append(out, randomScenario(p)) + } + } + return out +} + func writeSummary(t *testing.T, out string, outs []outcome) { if len(outs) == 0 { return @@ -405,6 +435,9 @@ func writeSummary(t *testing.T, out string, outs []outcome) { if o.Error != "" { notes = append([]string{o.Error}, notes...) } + if o.Skipped != "" { + notes = append(notes, o.Skipped) + } fmt.Fprintf(&md, "| %s | %s | %s | %s |\n", o.Name, o.Verdict, (time.Duration(o.DurationS) * time.Second).String(), strings.ReplaceAll(strings.Join(notes, "; "), "|", "/")) } diff --git a/go/test/site/rig_unit_test.go b/go/test/site/rig_unit_test.go index 7e921c2ad..417bc9c98 100644 --- a/go/test/site/rig_unit_test.go +++ b/go/test/site/rig_unit_test.go @@ -4,7 +4,11 @@ package site import ( "encoding/json" + "os" + "slices" + "strings" "testing" + "time" ) // These run in every go test, without a simulator. @@ -106,20 +110,25 @@ func TestRandomPlansAreSeeded(t *testing.T) { } // An over-limit stretch right after an outside step is the house's; the -// same stretch with no step before it is FTW's own and fails after 3 s. +// same stretch with no step before it is FTW's own and fails after 3 s, +// whether or not the record shows FTW raising anything. 31 A is over +// 1.2 x the 25 A fuse, so the heat gives no budget. func TestInvariantsTellStepsFromOwnRaises(t *testing.T) { - mk := func(withStep bool) *runData { + mk := func(withStep, withRise bool) *runData { d := &runData{marks: map[string]float64{"start": 0, "end": 100}, meta: map[string]any{"fuse": 25.0}} for i := 0; i <= 100; i++ { a := 10.0 if i >= 50 && i < 56 { - a = 26 // 6 s over the 24 A limit + a = 31 // 6 s over the 25 A fuse } d.samples = append(d.samples, &sample{T: float64(i), PhaseA: [3]float64{a, a, a}, Bat: plantDevice{SoCPct: 50}}) } if withStep { d.events = append(d.events, houseEventLine{T: 49.5, Event: houseEvent{Do: "loads"}}) } + if withRise { + d.frames = []frame{profileAt(40, 0), profileAt(45, 16)} + } return d } failed := func(cs []criterion) []string { @@ -131,11 +140,279 @@ func TestInvariantsTellStepsFromOwnRaises(t *testing.T) { } return out } - if _, cs := judgeInvariants(mk(true)); len(failed(cs)) != 0 { + if _, cs := judgeInvariants(mk(true, false)); len(failed(cs)) != 0 { t.Fatalf("after a step: %v", failed(cs)) } - _, cs := judgeInvariants(mk(false)) - if f := failed(cs); len(f) != 1 || f[0] != "over the limit at most 3 s after FTW's own raise" { - t.Fatalf("without a step: %v", f) + for _, withRise := range []bool{false, true} { + _, cs := judgeInvariants(mk(false, withRise)) + if f := failed(cs); len(f) != 1 || f[0] != "over the limit at most 3 s after FTW's own raise" { + t.Fatalf("with no step, FTW's raise %v: %v", withRise, f) + } + } +} + +// The limit is the fuse rating: a phase may sit at exactly the fuse until +// the end of the run, and 0.1 A above it fails once the time budget is up +// while the fuse is already warm (heat at the fuse). From a cool fuse the +// same 20 s stays inside the heat budget. +func TestInvariantsJudgeAtTheFuse(t *testing.T) { + const fuse = 25.0 + const own = "over the limit at most 3 s after FTW's own raise" + const step = "over the limit no longer than an outside step allows" + for _, c := range []struct { + name string + base, a float64 // the level before 50 s, and from 50 s + to int // the level holds from 50 s until this second + withStep bool + want []string + }{ + {"at the fuse", fuse, fuse, 101, false, nil}, + {"at the fuse after a step", fuse, fuse, 101, true, nil}, + // 20 s is past the 3 s for FTW's own raise, and past the + // 2 + 2 + 3 s a step allows while the battery can answer. + {"0.1 A over", fuse, fuse + 0.1, 70, false, []string{own}}, + {"0.1 A over after a step", fuse, fuse + 0.1, 70, true, []string{step}}, + {"0.1 A over a cool fuse", 10, fuse + 0.1, 70, false, nil}, + {"0.1 A over a cool fuse after a step", 10, fuse + 0.1, 70, true, nil}, + } { + d := &runData{marks: map[string]float64{"start": 0, "end": 100}, meta: map[string]any{"fuse": fuse}} + // FTW raised the car's order at 45 s. + d.frames = []frame{profileAt(40, 0), profileAt(45, 16)} + if c.withStep { + d.frames = nil + } + for i := 0; i <= 100; i++ { + a := c.base + if i >= 50 && i < c.to { + a = c.a + } + d.samples = append(d.samples, &sample{T: float64(i), PhaseA: [3]float64{a, a, a}, Bat: plantDevice{SoCPct: 50}}) + } + if c.withStep { + d.events = append(d.events, houseEventLine{T: 49.5, Event: houseEvent{Do: "loads"}}) + } + var failed []string + _, cs := judgeInvariants(d) + for _, x := range cs { + if !x.OK { + failed = append(failed, x.Name) + } + } + if !slices.Equal(failed, c.want) { + t.Errorf("%s: failed %v, want %v", c.name, failed, c.want) + } + } +} + +// S1 expects the room the fuse rating gives: on 16 A with a 3 A house that +// is 13 A per phase, 8 970 W. The old line at fuse - 1 A (8 280 W) falls +// short of it. +func TestJudgeS1ExpectsTheRoomAtTheFuse(t *testing.T) { + const name = "target within 60 s" + for _, c := range []struct { + carW float64 + want bool + }{{8970, true}, {8280, false}} { + d := synth(181, map[string]float64{"charge_now": 0, "end": 180}, map[string]any{"fuse": 16.0, "house_a": 3}, + func(i int, r *sample) { + if i >= 10 { + r.CarW = []float64{c.carW} + } + }) + m, cs := judgeS1(d) + if m["target_w"] != 8970.0 { + t.Fatalf("target_w = %v, want 8970", m["target_w"]) + } + if got := verdictsOf(cs)[name]; got != c.want { + t.Errorf("car at %v W: %q = %v, want %v", c.carW, name, got, c.want) + } + } +} + +// S12 counts a cut as one on room alone while every phase is under the fuse +// by more than 0.1 A, and records fuse_limit while the worst phase has 6 A. +func TestJudgeS12MeasuresRoomAtTheFuse(t *testing.T) { + const fuse = 20.0 + for _, c := range []struct { + name string + a float64 // the worst phase when the order drops + cuts, limit int + }{ + {"at the fuse", fuse, 0, 0}, + {"0.1 A under", fuse - 0.1, 0, 0}, + {"0.5 A under", fuse - 0.5, 1, 0}, + {"5.5 A under", fuse - 5.5, 1, 0}, + {"6.5 A under", fuse - 6.5, 1, 1}, + } { + d := synth(121, map[string]float64{"plan": 0, "ready": 100, "end": 120}, map[string]any{"fuse": fuse, "plan_wh": 6600.0}, + func(i int, r *sample) { + r.PhaseA = [3]float64{c.a, c.a - 1, c.a - 2} + if i == 30 { + r.LPs = []ftwLoadpoint{{CommandedReason: "fuse_limit"}} + } + }) + d.frames = []frame{profileAt(10, 16), profileAt(20, 10)} + m, _ := judgeS12(d) + if m["cuts_with_room"] != c.cuts || m["fuse_limit_with_room_s"] != c.limit { + t.Errorf("%s: cuts %v, fuse_limit %v; want %d, %d", c.name, m["cuts_with_room"], m["fuse_limit_with_room_s"], c.cuts, c.limit) + } + } +} + +// During Charge now an idle car is a fault once the house leaves the car's +// 6 A minimum under the fuse; with less room it is not. +func TestInvariantsChargeNowRoomAtTheFuse(t *testing.T) { + const fuse = 25.0 + const name = "Charge now charges when there is room" + for _, c := range []struct { + houseA float64 + want bool + }{{fuse - 6, false}, {fuse - 5.5, true}} { + d := synth(201, map[string]float64{"start": 0, "end": 200}, map[string]any{"fuse": fuse}, + func(_ int, r *sample) { + r.PhaseA = [3]float64{c.houseA, c.houseA, c.houseA} + r.Bat.SoCPct = 50 + }) + d.allMarks = []markLine{{T: 1, Label: "ftw:charge_now"}, {T: 200, Label: "end"}} + _, cs := judgeInvariants(d) + if got := verdictsOf(cs)[name]; got != c.want { + t.Errorf("house %v A: %q = %v, want %v", c.houseA, name, got, c.want) + } + } +} + +// s18 at margin 0: Core holds the phases at the 16 A line, so the house's own +// 0.1 A step lands over it until Core cuts. That stretch is the house's, not +// an own command. The same overage at any other time is Core's and fails: +// the phases sit at the fuse from the start, so the 60 s heat is at the fuse +// and any reading over it heats it past. +func TestS18OwnCommandsSkipTheHouseStep(t *testing.T) { + const own = "own commands stay inside the rated fuse" + mk := func(overFrom float64) *runData { + d := &runData{marks: map[string]float64{"charge_now": 0, "small_step": 100, "no_room": 200, "room_returns": 300, "end": 450}, + meta: map[string]any{"fuse": 16.0}} + for i := 0; i <= 450; i++ { + a := 16.0 + if float64(i) >= overFrom && float64(i) < overFrom+4 { + a = 16.1 + } + d.samples = append(d.samples, &sample{T: float64(i), PhaseA: [3]float64{a, a, a}}) + } + return d + } + ok := func(d *runData) bool { + _, cs := judgeS18(d) + for _, c := range cs { + if c.Name == own { + return c.OK + } + } + t.Fatalf("no %q criterion", own) + return false + } + if !ok(mk(100)) { + t.Error("4 s at 16.1 A right after the house's step failed") + } + for _, at := range []float64{50, 150} { + if ok(mk(at)) { + t.Errorf("4 s at 16.1 A from %.0f s with no outside step passed", at) + } + } +} + +// s18's own commands are judged at the 16 A rating by Core's heat model +// (owner rule, 27 Oct 2026): a reading over the fuse fails only while the +// 60 s heat is over it too, or at once over 1.2 x the fuse. On master +// 575152ea the local rig failed s18 at 16.011 A against a fixed 16.01 A +// bound, with the heat at 14.95 A. +func TestS18OwnCommandsJudgedByHeat(t *testing.T) { + const own = "own commands stay inside the rated fuse" + for _, c := range []struct { + name string + baseA, overA float64 + overS int + want bool + }{ + {"16.011 A jitter for 5 s at the line", 15.99, 16.011, 5, true}, + {"16.5 A for 30 s at the line heats past the fuse", 15.99, 16.5, 30, false}, + {"16.5 A for 5 s from a cool fuse", 12, 16.5, 5, true}, + {"one reading at 19.1 A from a cool fuse", 12, 19.1, 1, true}, + {"one reading at 19.3 A, over 1.2 x the fuse", 12, 19.3, 1, false}, + } { + d := &runData{marks: map[string]float64{"charge_now": 0, "small_step": 100, "no_room": 200, "room_returns": 300, "end": 450}, + meta: map[string]any{"fuse": 16.0}} + for i := 0; i <= 450; i++ { + a := c.baseA + if i >= 50 && i < 50+c.overS { + a = c.overA + } + d.samples = append(d.samples, &sample{T: float64(i), PhaseA: [3]float64{a, a, a}}) + } + _, cs := judgeS18(d) + if got := verdictsOf(cs)[own]; got != c.want { + t.Errorf("%s: %q = %v, want %v", c.name, own, got, c.want) + } + } +} + +// s8 at margin 0: the car settles at the fuse rating, as the owner's rule +// says. A phase at exactly the fuse for the whole run passes; 0.1 A above +// it for longer than the charger's delay + 7 s fails. Before, the judge +// counted every second above fuse - 1 A, and s8-60-25 failed at 25 A. +func TestS8JudgesAtTheFuse(t *testing.T) { + const over = "no longer than the charger's delay + 7 s" + ok := func(a float64) bool { + d := &runData{marks: map[string]float64{"charge_now": 0, "end": 300}, meta: map[string]any{"fuse": 25.0, "charger_delay_s": 60.0}} + for i := 0; i <= 300; i++ { + d.samples = append(d.samples, &sample{T: float64(i), PhaseA: [3]float64{a, 15, 15}}) + } + _, cs := judgeS8(d) + for _, c := range cs { + if strings.HasSuffix(c.Name, over) { + return c.OK + } + } + t.Fatalf("no criterion ending %q", over) + return false + } + if !ok(25) { + t.Error("a phase at exactly the 25 A fuse failed") + } + if ok(25.1) { + t.Error("a phase 0.1 A over the 25 A fuse for 300 s passed") + } +} + +// A run that selects planner_self plans in self_consumption; any other run +// selects planner_passive_arbitrage and plans in passive_arbitrage. s15bat +// needs Self, where PV surplus charges the battery (#1875). +func TestRigModeMatchesItsPlannerConfig(t *testing.T) { + for _, tc := range []struct{ spec, mode, planner string }{ + {"", "planner_passive_arbitrage", "passive_arbitrage"}, + {"planner_self", "planner_self", "self_consumption"}, + } { + st := newStack(nil, lane(21700), t.TempDir(), stackSpec{fuseA: 16, mode: tc.spec}) + st.prices = newPriceServer(time.Now()) + path, err := st.ftwConfig() + st.prices.close() + if err != nil { + t.Fatal(err) + } + b, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + var cfg struct { + Planner struct { + Mode string `json:"mode"` + } `json:"planner"` + } + if err := json.Unmarshal(b, &cfg); err != nil { + t.Fatal(err) + } + if st.mode() != tc.mode || cfg.Planner.Mode != tc.planner { + t.Errorf("spec mode %q: selects %q and plans in %q, want %q and %q", + tc.spec, st.mode(), cfg.Planner.Mode, tc.mode, tc.planner) + } } } diff --git a/go/test/site/runner/README.md b/go/test/site/runner/README.md index 346d7747c..74bd8d39f 100644 --- a/go/test/site/runner/README.md +++ b/go/test/site/runner/README.md @@ -6,9 +6,30 @@ the result as a GitHub commit status on the exact commit it tested: | Status | When | What runs | |---|---|---| | `siterig/full` | each new commit on `master` | every scenario | -| `siterig/short` | the head of each open pull request from this repository that touches `go/internal/{control,loadpoint,ocpp,telemetry}` or `go/test/site` | the short set | +| `siterig/short` | the head of each open pull request from this repository that touches a watched path (below) | the short set | | `siterig/random` | once a night on `master` | that night's random seeds | +The watched paths are the control packages, every internal package they +import, and the rig itself: + +- `go/cmd/ftw` (control tick wiring) +- `go/internal/{control,loadpoint,ocpp,telemetry}` +- `go/internal/{drivers,mdnsresolve}` (driver host, command deadlines) +- `go/internal/{mpc,optimizercontract}` (plans) +- `go/internal/{gridcost,forecasting,sunpos}` (plan inputs) +- `go/internal/{config,configreload}` (limits) +- `go/internal/units` (charge steps) +- `go/internal/{cars,sitepower,events,state}` +- `drivers/` and `optimizer/native/bundle/` (the bundled drivers and solver) +- `go/test/site` + +`go/cmd/ftw` imports every package, the API and UI among them; only its own +files are watched. A change to the API or UI alone does not run the rig. + +`watchedPaths` in `runner.go` is the list; `TestTouchesWatched` pins it. +`TestWatchedPathsCoverImports` fails when a control package starts to +import an internal package that is not on the list. + A commit that already has the status is not run again. To run one again, post a new `pending` status or remove the result and wait a day. No GitHub Actions workflow runs the rig; `native-release.yml` only reads @@ -25,43 +46,76 @@ rig's timing, and the rig then fails the run as "host paused". Not the home energy box: the rig needs the CPU, and the box is production. The rig uses 100 ports per lane from `-port-base` (default 18700) on -127.0.0.1. Leave them free. +127.0.0.1: lane k uses `port-base + 100k` up to 99 above it, so three lanes +take 18700 to 18999. Leave them free. `make siterig` uses the same default, +so on the runner's host run it with `FTW_SITE_PORT_BASE=19700`. A second +runner on one host needs its own `-port-base` and `-work`. Pull requests from forks are skipped unless `-allow-forks` is set: the rig builds and runs the pull request's code on this machine. ## Install +The runner keeps its clone in `~/siterig/ftw` (`-work` plus `ftw`) and +clones it there on first start if it is missing. Build the binary from the +same clone: + ```bash gh auth login # an account that may write commit statuses on srcfl/ftw mkdir -p ~/siterig/bin -git clone https://github.com/srcfl/ftw ~/siterig/src -cd ~/siterig/src/go && go build -o ~/siterig/bin/siterig-runner ./test/site/runner +git clone https://github.com/srcfl/ftw ~/siterig/ftw +cd ~/siterig/ftw/go && go build -o ~/siterig/bin/siterig-runner ./test/site/runner ~/siterig/bin/siterig-runner -once # one pass in the foreground ``` +To rebuild after a change to this directory: + +```bash +cd ~/siterig/ftw && git pull --quiet origin master +cd go && go build -o ~/siterig/bin/siterig-runner ./test/site/runner +``` + +then restart the service. The scenarios themselves always come from the +commit under test. + macOS (launchd): ```bash -sed "s/USER/$USER/g" ~/siterig/src/go/test/site/runner/energy.ftw.siterig-runner.plist \ +sed "s/USER/$USER/g" ~/siterig/ftw/go/test/site/runner/energy.ftw.siterig-runner.plist \ > ~/Library/LaunchAgents/energy.ftw.siterig-runner.plist launchctl load ~/Library/LaunchAgents/energy.ftw.siterig-runner.plist ``` -Linux (systemd): edit `siterig-runner.service`, copy it to -`/etc/systemd/system/` and run `sudo systemctl enable --now siterig-runner`. +The log is `~/siterig/runner.log`. + +Linux, as a system service: replace `USER` in `siterig-runner.service`, +copy it to `/etc/systemd/system/` and run +`sudo systemctl enable --now siterig-runner`. The log is +`journalctl -u siterig-runner`. + +Linux, as a user service. Root is needed once, for +`loginctl enable-linger`, which keeps the service running after you log out +and starts it at boot: + +```bash +mkdir -p ~/.config/systemd/user +sed -e '/^User=/d' -e "s/USER/$USER/g" -e 's/multi-user.target/default.target/' \ + ~/siterig/ftw/go/test/site/runner/siterig-runner.service \ + > ~/.config/systemd/user/siterig-runner.service +systemctl --user daemon-reload +systemctl --user enable --now siterig-runner +sudo loginctl enable-linger "$USER" +``` -The log is `~/siterig/runner.log` (launchd) or `journalctl -u -siterig-runner` (systemd). Rebuild the binary after a change to this -directory; the scenarios themselves always come from the commit under test. +The log is `journalctl --user -u siterig-runner`. Check `HOME` and `PATH` +in the unit if your home is not `/home/$USER` or Go lives elsewhere. ## How long it takes -On an M5 MacBook (10 cores), 22 October 2026: the 23 scenarios of that -morning took 72 minutes in three lanes. With the one-phase charger cases the -set adds up to about 270 minutes, so three lanes need about 90 and four -about 70; `s12-20` alone takes up to 56, so more than four do not help. -Each lane is one `ftw` and one `simulator run`, both mostly idle. +On an M5 MacBook (10 cores), 27 October 2026: the full set took about +2 h 20 min in three lanes. `s12-20` alone took up to 56 minutes on +22 October, so more than four lanes do not help. Each lane is one `ftw` and +one `simulator run`, both mostly idle. ## Flags diff --git a/go/test/site/runner/main.go b/go/test/site/runner/main.go index 6878257c9..6a5e0bee6 100644 --- a/go/test/site/runner/main.go +++ b/go/test/site/runner/main.go @@ -1,7 +1,7 @@ // siterig-runner runs FTW's site test (go/test/site) on a machine of its own // and publishes the result as a GitHub commit status on the exact commit: // siterig/full on each new master commit, siterig/short on the head of each -// open pull request that touches control, loadpoint, OCPP or telemetry code, +// open pull request that touches code on the control path (watchedPaths), // and siterig/random for a night's random seeds. native-release.yml reads // siterig/full before it publishes a beta. See README.md. package main @@ -80,13 +80,21 @@ func (r *runner) prepare() error { // tick runs everything that is due, one job at a time. func (r *runner) tick() { stateFile := filepath.Join(r.work, "last-random") - last, _ := os.ReadFile(stateFile) - jobs, err := plan(r.gh, r.cfg, time.Now(), strings.TrimSpace(string(last))) - if err != nil { - log.Printf("plan: %v", err) - return - } - for _, j := range jobs { + // Plan again before each job: a run takes about 20 minutes, in which a + // pull request can move on or a second runner can take the same commit. + ran := map[string]bool{} + for { + last, _ := os.ReadFile(stateFile) + jobs, err := plan(r.gh, r.cfg, time.Now(), strings.TrimSpace(string(last))) + if err != nil { + log.Printf("plan: %v", err) + break + } + j, ok := next(jobs, ran) + if !ok { + break + } + ran[j.Context+" "+j.SHA] = true log.Printf("start %s", j) state, desc := r.runJob(j) log.Printf("done %s: %s, %s", j, state, desc) diff --git a/go/test/site/runner/runner.go b/go/test/site/runner/runner.go index 1a0def0c0..955fab08b 100644 --- a/go/test/site/runner/runner.go +++ b/go/test/site/runner/runner.go @@ -3,6 +3,7 @@ package main import ( "encoding/json" "fmt" + "sort" "strings" "time" ) @@ -15,12 +16,31 @@ const ( ) // watchedPaths are the parts of the tree whose changes get the short set on -// a pull request. +// a pull request: the control packages, every internal package they import, +// and the rig itself. TestWatchedPathsCoverImports keeps the imports in. var watchedPaths = []string{ + "go/cmd/ftw/", // control tick wiring "go/internal/control/", "go/internal/loadpoint/", "go/internal/ocpp/", "go/internal/telemetry/", + "go/internal/drivers/", "go/internal/mdnsresolve/", // driver host, command deadlines + "go/internal/mpc/", "go/internal/optimizercontract/", // plans + "go/internal/gridcost/", "go/internal/forecasting/", "go/internal/sunpos/", // plan inputs + "go/internal/config/", "go/internal/configreload/", // limits + "go/internal/units/", // charge steps + "go/internal/localzone/", // household clock: goal deadlines, import hour + "go/internal/cars/", "go/internal/sitepower/", + "go/internal/events/", "go/internal/state/", + "drivers/", "optimizer/native/bundle/", // bundled drivers and solver "go/test/site/", } +// controlPackages are the watched packages whose imports must be watched +// too. go/cmd/ftw is left out: it imports every package, the API and UI +// among them. +var controlPackages = []string{ + "control", "loadpoint", "ocpp", "telemetry", "drivers", "mpc", + "optimizercontract", "config", "configreload", "cars", "sitepower", +} + // job is one run of the rig on one commit. type job struct { Context string // which status it publishes @@ -45,9 +65,10 @@ func short(sha string) string { // pull is the part of a pull request the runner needs. type pull struct { - Number int `json:"number"` - Draft bool `json:"draft"` - Head struct { + Number int `json:"number"` + Draft bool `json:"draft"` + UpdatedAt string `json:"updated_at"` // RFC 3339, so it sorts as text + Head struct { SHA string `json:"sha"` Repo struct { FullName string `json:"full_name"` @@ -93,6 +114,9 @@ func plan(gh github, cfg config, now time.Time, lastRandom string) ([]job, error if err != nil { return nil, err } + // Most recently updated first: that is the work in review, and a pull + // request untouched for weeks can wait. + sort.SliceStable(pulls, func(i, j int) bool { return pulls[i].UpdatedAt > pulls[j].UpdatedAt }) for _, p := range pulls { if p.Draft && !cfg.Drafts { continue @@ -124,6 +148,17 @@ func plan(gh github, cfg config, now time.Time, lastRandom string) ([]job, error return jobs, nil } +// next is the first job not yet run in this tick. A job whose status failed +// to post is planned again; this keeps it from running twice. +func next(jobs []job, ran map[string]bool) (job, bool) { + for _, j := range jobs { + if !ran[j.Context+" "+j.SHA] { + return j, true + } + } + return job{}, false +} + func touchesWatched(files []string) bool { for _, f := range files { for _, p := range watchedPaths { diff --git a/go/test/site/runner/runner_test.go b/go/test/site/runner/runner_test.go index d05a1dd36..bf4c2ecd5 100644 --- a/go/test/site/runner/runner_test.go +++ b/go/test/site/runner/runner_test.go @@ -1,6 +1,7 @@ package main import ( + "os/exec" "strings" "testing" "time" @@ -75,6 +76,81 @@ func TestPlanRunsWhatIsDue(t *testing.T) { } } +func TestTouchesWatched(t *testing.T) { + for _, tc := range []struct { + file string + want bool + }{ + {"go/cmd/ftw/main.go", true}, + {"go/internal/control/dispatch.go", true}, + {"go/internal/loadpoint/loadpoint.go", true}, + {"go/internal/ocpp/handlers.go", true}, + {"go/internal/telemetry/store.go", true}, + {"go/internal/drivers/lua.go", true}, + {"go/internal/mpc/mpc.go", true}, + {"go/internal/optimizercontract/contract.go", true}, + {"go/internal/config/config.go", true}, + {"go/internal/configreload/apply.go", true}, + {"go/internal/cars/cars.go", true}, + {"go/internal/sitepower/sitepower.go", true}, + {"go/internal/units/charge_steps.go", true}, + {"go/internal/gridcost/gridcost.go", true}, + {"go/internal/state/store.go", true}, + {"go/internal/events/bus.go", true}, + {"drivers/sungrow.lua", true}, + {"drivers/BUNDLED_SOURCE.json", true}, + {"optimizer/native/bundle/manifest.json", true}, + {"go/test/site/rig_test.go", true}, + {"go/test/site/runner/runner.go", true}, + {"docs/architecture.md", false}, + {"web/components/theme.css", false}, + {"go/internal/api/api.go", false}, + {"go/internal/driverrepo/repo.go", false}, // shares a prefix with drivers/, not a path + {"go/internal/configx/x.go", false}, // shares a prefix with config/ + {"optimizer/native/src/main.rs", false}, // only the bundle ships + {"go/cmd/ftwctl/main.go", false}, // shares a prefix with cmd/ftw/ + {"web/drivers/list.js", false}, // drivers/ only at the root + {".changeset/a.md", false}, + } { + if got := touchesWatched([]string{tc.file}); got != tc.want { + t.Errorf("%s: %v, want %v", tc.file, got, tc.want) + } + } + if touchesWatched(nil) { + t.Error("no files: watched") + } + if !touchesWatched([]string{"README.md", "go/internal/mpc/plan.go"}) { + t.Error("one watched file among others: not watched") + } +} + +// TestWatchedPathsCoverImports fails when a control package starts to +// import an internal package that is not watched, so a change there would +// skip the rig. +func TestWatchedPathsCoverImports(t *testing.T) { + const module = "github.com/srcfl/ftw/go/internal/" + args := []string{"list", "-deps"} + for _, p := range controlPackages { + if !touchesWatched([]string{"go/internal/" + p + "/x.go"}) { + t.Errorf("control package %s is not watched", p) + } + args = append(args, module+p) + } + out, err := exec.Command("go", args...).Output() + if err != nil { + t.Fatalf("go list: %v", err) + } + for _, dep := range strings.Fields(string(out)) { + rest, ok := strings.CutPrefix(dep, module) + if !ok { + continue + } + if !touchesWatched([]string{"go/internal/" + rest + "/x.go"}) { + t.Errorf("%s imports %s, which is not watched", strings.Join(controlPackages, ", "), rest) + } + } +} + func TestDescribe(t *testing.T) { state, desc := describe([]byte(`[{"name":"s1-25","verdict":"pass"},{"name":"s2-16","verdict":"known"},{"name":"s9-25","verdict":"fail"}]`), 24*time.Minute) if state != "failure" || desc != "1 pass, 1 known, 1 fail of 3 in 24m0s: s9-25" { @@ -92,3 +168,35 @@ func TestDescribe(t *testing.T) { t.Fatalf("%d characters", len(desc)) } } + +func TestPlanTakesRecentPullFirstAndNextSkipsWhatRan(t *testing.T) { + watched := []string{"go/internal/control/dispatch.go"} + gh := fakeGitHub{ + master: "m1", + pulls: []pull{pr(3, "c3", "srcfl/ftw", false), pr(9, "c9", "srcfl/ftw", false)}, + files: map[int][]string{9: watched, 3: watched}, + contexts: map[string]map[string]bool{"m1": {ctxFull: true}}, + } + gh.pulls[0].UpdatedAt, gh.pulls[1].UpdatedAt = "2026-10-01T08:00:00Z", "2026-10-27T21:00:00Z" + jobs, err := plan(gh, config{Repo: "srcfl/ftw", Drafts: true, RandomAt: -1}, time.Now(), "") + if err != nil { + t.Fatal(err) + } + if len(jobs) != 2 || jobs[0].PR != 9 || jobs[1].PR != 3 { + t.Fatalf("jobs %v, want #9 (updated last) then #3", jobs) + } + ran := map[string]bool{} + j, ok := next(jobs, ran) + if !ok || j.PR != 9 { + t.Fatalf("first: %v %v", j, ok) + } + // #9's status failed to post, so the next plan lists it again. + ran[j.Context+" "+j.SHA] = true + if j, ok = next(jobs, ran); !ok || j.PR != 3 { + t.Fatalf("second: %v %v", j, ok) + } + ran[j.Context+" "+j.SHA] = true + if _, ok = next(jobs, ran); ok { + t.Fatal("a third job after both ran") + } +} diff --git a/go/test/site/runner/siterig-runner.service b/go/test/site/runner/siterig-runner.service index f287a597b..f7d0b079e 100644 --- a/go/test/site/runner/siterig-runner.service +++ b/go/test/site/runner/siterig-runner.service @@ -1,6 +1,7 @@ # systemd unit for siterig-runner (README.md). Replace USER and the paths, # then: sudo cp siterig-runner.service /etc/systemd/system/ && # sudo systemctl enable --now siterig-runner +# README.md also shows how to run it as a user service. [Unit] Description=FTW site test runner (siterig/short, siterig/full commit statuses) After=network-online.target diff --git a/go/test/site/scenarios_p0_test.go b/go/test/site/scenarios_p0_test.go new file mode 100644 index 000000000..571601d0b --- /dev/null +++ b/go/test/site/scenarios_p0_test.go @@ -0,0 +1,312 @@ +//go:build unix + +package site + +import ( + "encoding/json" + "os" + "path/filepath" + "syscall" + "time" +) + +// The P0 scenarios R1-R5 of the stable plan: a meter that stops answering, +// a battery that drops out, the planner's floor, a hard kill and a start +// into a running site. Their judges are in verdicts_test.go. A known line +// names the Wave 1 item whose fix deletes it. +func init() { + scenarios = append(scenarios, + scenario{name: "r1-25", dir: "r1-meter-silent-then-offline-25A", minutes: 10, run: func(c *runCtx) (map[string]any, error) { return r1(c, 25) }, + judge: judgeR1, skip: meterKnob, + known: map[string]string{"the battery order moves only toward 0 while the meter is silent": "A1: a failed poll keeps the held reading, and ComputeDispatch winds the battery against it (dispatch.go:1446-1449)"}}, + scenario{name: "r2-25", dir: "r2-battery-offline-25A", minutes: 8, run: func(c *runCtx) (map[string]any, error) { return r2(c, 25, 0) }, + judge: judgeR2}, + scenario{name: "r2-60-25", dir: "r2-battery-offline-slow60-25A", minutes: 10, run: func(c *runCtx) (map[string]any, error) { return r2(c, 25, 60) }, + judge: judgeR2}, + scenario{name: "r3-25", dir: "r3-planner-floor-25A", minutes: 13, run: func(c *runCtx) (map[string]any, error) { return r3(c, 25) }, + judge: judgeR3}, + scenario{name: "r4-25", dir: "r4-sigkill-25A", minutes: 41, run: func(c *runCtx) (map[string]any, error) { return r4(c, 25) }, + judge: judgeR4}, + scenario{name: "r5a-25", dir: "r5a-start-car-charging-25A", minutes: 5, run: func(c *runCtx) (map[string]any, error) { return r5a(c, 25) }, + judge: judgeR5a}, + scenario{name: "r5b-25", dir: "r5b-start-stale-plan-25A", minutes: 27, run: func(c *runCtx) (map[string]any, error) { return r5b(c, 25) }, + judge: judgeR5b}, + scenario{name: "r5c-25", dir: "r5c-start-new-session-25A", minutes: 15, run: func(c *runCtx) (map[string]any, error) { return r5c(c, 25) }, + judge: judgeR5c}, + ) +} + +// ---- R1. The site meter goes silent, then offline, under Charge now ---- + +// meterKnob is the device-simulator change R1 waits for. +const meterKnob = "TODO(device-simulator): `silence` and `offline` must take the site meter " + + "(device \"meter\"); at the pinned commit they answer `no device \"meter\"`. Pin a commit " + + "that has it and delete this skip." + +// r1 puts Core's battery relief on a meter that stops answering: Modbus +// timeouts for 120 s, then, two minutes after it is back, a meter gone for +// 90 s. House 15 A per phase and Charge now 16 A put the site over the fuse +// unless the battery helps. +func r1(c *runCtx, fuse float64) (map[string]any, error) { + err := with(c, stackSpec{fuseA: fuse, loadsW: houseA(15), socPct: 50}, func(st *stack) error { + if err := st.plug(0); err != nil { + return err + } + sleepS(30) + st.mark("charge_now") + if err := st.chargeNow("garage", 16, "3p", 0); err != nil { + return err + } + sleepS(90) + st.mark("silent") + if err := st.event(houseEvent{Do: "silence", Device: "meter", ForS: 120}); err != nil { + return err + } + sleepS(120) + st.mark("back") + sleepS(120) + st.mark("offline") + if err := st.event(houseEvent{Do: "offline", Device: "meter", ForS: 90}); err != nil { + return err + } + sleepS(90) + st.mark("online") + sleepS(120) + st.mark("end") + return nil + }) + return map[string]any{"scenario": "r1", "fuse": fuse, "house_a": 15, "silent_s": 120, "offline_s": 90}, err +} + +// ---- R2. The battery drops out while the car's room rests on it ---- + +// r2 holds the battery at a 3 kW discharge under a 20 A house, so the car's +// room rests on it, then takes the inverter off the site for 180 s: each +// phase rises 4-6 A at once. The car takes at most 10 A, so the site stays +// under 1.2 x the fuse, where Core's 60 s heat budget, not an instant cut, +// decides when the car stops (control/fuse_heat.go). +func r2(c *runCtx, fuse, delayS float64) (map[string]any, error) { + err := with(c, stackSpec{fuseA: fuse, loadsW: houseA(20), socPct: 60, + chargers: []charger{{delayS: delayS, carMaxA: 10}}}, func(st *stack) error { + if err := st.plug(0); err != nil { + return err + } + st.mark("plug_wait") + sleepS(delayS + 20) + st.mark("battery_hold") + if err := st.post("/api/battery/manual_hold", map[string]any{"direction": "discharge", "power_w": 3000, "hold_s": 1800}, nil); err != nil { + return err + } + st.mark("charge_now") + if err := st.chargeNow("garage", 10, "3p", 0); err != nil { + return err + } + sleepS(delayS + 90) + st.mark("offline") + if err := st.event(houseEvent{Do: "offline", Device: "inverter"}); err != nil { + return err + } + sleepS(180) + st.mark("online") + if err := st.event(houseEvent{Do: "online", Device: "inverter"}); err != nil { + return err + } + sleepS(120) + st.mark("end") + return nil + }) + return map[string]any{"scenario": "r2", "fuse": fuse, "house_a": 20, "battery_hold_w": -3000, + "charger_delay_s": delayS, "offline_s": 180}, err +} + +// ---- R3. The battery near the planner's floor under fuse pressure ---- + +// r3 starts the battery at 12 % with the planner's floor at 10 % and the +// inverter's own at 5 %. House 15 A per phase and Charge now 16 A need the +// battery's help; the charger applies a profile 60 s late. +func r3(c *runCtx, fuse float64) (map[string]any, error) { + const delayS, floor = 60, 0.10 + err := with(c, stackSpec{fuseA: fuse, loadsW: houseA(15), socPct: 12, plannerSoCMin: floor, + chargers: []charger{{delayS: delayS}}}, func(st *stack) error { + if err := st.plug(0); err != nil { + return err + } + st.mark("plug_wait") + sleepS(delayS + 20) + st.mark("charge_now") + if err := st.chargeNow("garage", 16, "3p", 0); err != nil { + return err + } + sleepS(600) + st.mark("end") + return nil + }) + return map[string]any{"scenario": "r3", "fuse": fuse, "house_a": 15, "soc_pct": 12, + "planner_floor_pct": floor * 100, "charger_delay_s": delayS}, err +} + +// ---- R4. Core killed mid-charge and mid-discharge ---- + +// r4 kills Core with SIGKILL, as a power cut does: no driver cleanup, no +// history flush, an OCPP transaction left open. House 12 A per phase, the +// battery held at a 3 kW discharge, Charge now 16 A. The first kill lasts +// 20 s; two minutes later the second lasts 120 s, and the house steps to +// 16 A per phase while Core is gone. Like s7, the run first waits until the +// day's Home use is known, and long enough after it that one restart stays +// under the 10 % unknown-time rule. The rule counts whole ledger minutes, +// so a restart whose unknown car reading crosses a minute boundary marks +// two. With the 42 s Core was down or starting, that came to 128 s of a +// 1 054 s day when the hold came 10 min after, and failed 1 run in 5. +// 25 min leaves room for it. Core holds a battery for at most +// 1800 s, so the hold starts 300 s before the first kill and outlasts both. +func r4(c *runCtx, fuse float64) (map[string]any, error) { + var serial string + err := with(c, stackSpec{fuseA: fuse, loadsW: houseA(12), socPct: 60}, func(st *stack) error { + if err := st.plug(0); err != nil { + return err + } + serial = st.serial(0) + st.mark("charge_now") + if err := st.chargeNow("garage", 16, "3p", 0); err != nil { + return err + } + if waitUntil(900*time.Second, 5*time.Second, func() bool { + var s ftwStatus + return st.get("/api/status", &s) == nil && s.Energy.Today.LoadWh != nil + }) { + st.mark("today_known") + } + sleepS(1500) + st.mark("battery_hold") + if err := st.post("/api/battery/manual_hold", map[string]any{"direction": "discharge", "power_w": 3000, "hold_s": 1800}, nil); err != nil { + return err + } + sleepS(300) + st.mark("kill1") + st.stopFTW(syscall.SIGKILL) + sleepS(20) + if err := st.startFTW(); err != nil { + return err + } + st.mark("up1") + sleepS(120) + st.mark("kill2") + st.stopFTW(syscall.SIGKILL) + st.mark("step") + if err := st.setLoadsA(16); err != nil { + return err + } + sleepS(120) + if err := st.startFTW(); err != nil { + return err + } + st.mark("up2") + sleepS(300) + st.mark("end") + return nil + }) + return map[string]any{"scenario": "r4", "fuse": fuse, "house_a": 12, "step_a": 16, "battery_hold_w": -3000, + "down_s": []float64{20, 120}, "charger": serial}, err +} + +// ---- R5. Core starts into a running site ---- + +// r5a starts Core while the car charges on its own: plugged when the house +// comes up, it starts a local transaction and draws all it can, 16 A on +// 14 A of house load. The battery sits at the inverter's 5 % floor, so it +// cannot carry the car: the site runs 30 A per phase until Core acts. +func r5a(c *runCtx, fuse float64) (map[string]any, error) { + err := with(c, stackSpec{fuseA: fuse, loadsW: houseA(14), socPct: 5, startAfterS: 60, + chargers: []charger{{plugged: true}}}, func(st *stack) error { + st.mark("ftw_up") + sleepS(180) + st.mark("end") + return nil + }) + return map[string]any{"scenario": "r5a", "fuse": fuse, "house_a": 14, "soc_pct": 5}, err +} + +// r5b saves a goal with hours of slack, so no fresh plan need charge now, +// then stops Core for 20 min and starts it on the plan it saved. For the +// first minute the rig asks for the plan once a second (plan.jsonl). On +// master it does not reach D6: the saved plan is younger than MaxPlanAge +// (30 min) and a fresh one lands within seconds, so the deadline reserve +// never acts. D6's proof is its unit repro; this run guards the start. +func r5b(c *runCtx, fuse float64) (map[string]any, error) { + err := with(c, stackSpec{fuseA: fuse, loadsW: houseA(3), socPct: 50}, func(st *stack) error { + if err := st.plug(0); err != nil { + return err + } + if _, err := st.planTo("garage", 40, 60, 8*time.Hour); err != nil { + return err + } + st.mark("plan") + sleepS(120) + st.mark("ftw_stop") + st.stopFTW(syscall.SIGTERM) + sleepS(1200) + st.mark("ftw_start") + if err := st.startFTW(); err != nil { + return err + } + st.mark("ftw_up") + f, err := os.Create(filepath.Join(st.rundir, "plan.jsonl")) + if err != nil { + return err + } + defer f.Close() + for end := time.Now().Add(60 * time.Second); time.Now().Before(end); sleepS(1) { + var p struct { + Plan *struct { + GeneratedAtMs int64 `json:"generated_at_ms"` + } `json:"plan"` + } + if st.get("/api/mpc/plan", &p) == nil && p.Plan != nil { + b, _ := json.Marshal(map[string]any{"t": now(), "generated_at_ms": p.Plan.GeneratedAtMs}) + f.Write(append(b, '\n')) + } + } + sleepS(60) + st.mark("end") + return nil + }) + return map[string]any{"scenario": "r5b", "fuse": fuse, "house_a": 3, "down_s": 1200}, err +} + +// r5c holds Charge now on one car, stops Core for 10 min, in which that car +// leaves and another plugs in, and starts Core into the new session. +func r5c(c *runCtx, fuse float64) (map[string]any, error) { + err := with(c, stackSpec{fuseA: fuse, loadsW: houseA(3), socPct: 50}, func(st *stack) error { + if err := st.plug(0); err != nil { + return err + } + st.mark("charge_now") + if err := st.chargeNow("garage", 16, "3p", 0); err != nil { + return err + } + sleepS(60) + st.mark("ftw_stop") + st.stopFTW(syscall.SIGTERM) + sleepS(300) + off, on := false, true + st.mark("unplug") + if err := st.event(houseEvent{Do: "plug", Device: "garage", On: &off}); err != nil { + return err + } + sleepS(30) + st.mark("replug") + if err := st.event(houseEvent{Do: "plug", Device: "garage", On: &on}); err != nil { + return err + } + sleepS(270) + st.mark("ftw_start") + if err := st.startFTW(); err != nil { + return err + } + st.mark("ftw_up") + sleepS(180) + st.mark("end") + return nil + }) + return map[string]any{"scenario": "r5c", "fuse": fuse, "house_a": 3, "down_s": 600}, err +} diff --git a/go/test/site/scenarios_p0_unit_test.go b/go/test/site/scenarios_p0_unit_test.go new file mode 100644 index 000000000..a23b905ca --- /dev/null +++ b/go/test/site/scenarios_p0_unit_test.go @@ -0,0 +1,459 @@ +//go:build unix + +package site + +import ( + "encoding/json" + "os" + "path/filepath" + "testing" +) + +// Unit tests for the R1-R5 judges, on records made up here. They run in +// every go test, without a simulator. + +// synth is n one-second samples from t = 0: 10 A per phase, the car idle, +// the battery not forced. set changes sample i. +func synth(n int, marks map[string]float64, meta map[string]any, set func(i int, r *sample)) *runData { + d := &runData{marks: marks, meta: meta} + for i := 0; i < n; i++ { + r := &sample{T: float64(i), PhaseA: [3]float64{10, 10, 10}, CarW: []float64{0}} + if set != nil { + set(i, r) + } + d.samples = append(d.samples, r) + } + return d +} + +// order sets the battery's EMS registers to Core's order w (+ charge). +func order(r *sample, w float64) { + switch { + case w > 0: + r.Bat.EMSMode, r.Bat.EMSCmd, r.Bat.EMSSetW = 2, 170, w + case w < 0: + r.Bat.EMSMode, r.Bat.EMSCmd, r.Bat.EMSSetW = 2, 187, -w + default: + r.Bat.EMSMode, r.Bat.EMSCmd, r.Bat.EMSSetW = 0, 204, 0 + } +} + +// profileAt is a SetChargingProfile of limitA per phase on connector 1, +// sent at t. +func profileAt(t, limitA float64) frame { + b, _ := json.Marshal([]any{2, "id", "SetChargingProfile", map[string]any{"connectorId": 1, + "csChargingProfiles": map[string]any{"chargingProfilePurpose": "TxDefaultProfile", + "chargingSchedule": map[string]any{"chargingRateUnit": "A", + "chargingSchedulePeriod": []any{map[string]any{"limit": limitA}}}}}}) + return frame{T: t, Dir: "cs->cp", Frame: b} +} + +func verdictsOf(cs []criterion) map[string]bool { + out := map[string]bool{} + for _, c := range cs { + out[c.Name] = c.OK + } + return out +} + +func TestBatOrderReadsTheForcedCommand(t *testing.T) { + for _, w := range []float64{3000, -2500, 0} { + r := &sample{} + order(r, w) + if got := batOrder(r); got != w { + t.Errorf("order %v read as %v", w, got) + } + } + r := &sample{Bat: plantDevice{EMSMode: 0, EMSCmd: 187, EMSSetW: 3000}} + if batOrder(r) != 0 || batForced(r) { + t.Fatal("a discharge command outside forced mode is no order: the inverter runs on its own") + } +} + +func TestOfferRisesCountsOnlyRaisesInTheWindow(t *testing.T) { + d := &runData{frames: []frame{profileAt(10, 8), profileAt(20, 10), profileAt(30, 6), profileAt(40, 7), profileAt(60, 16)}} + if n := d.offerRises(15, 50); n != 2 { + t.Fatalf("rises in [15, 50): %d, want 2 (8→10 A at 20 s, 6→7 A at 40 s)", n) + } + if p := d.firstOffer(25, func(p profile) bool { return p.Limit < 7 }); p == nil || *p != 5 { + t.Fatalf("first offer under 7 A from 25 s: %v, want 5 s", p) + } +} + +// r1Run is R1's record: the meter silent from 100 to 220 s and offline from +// 340 to 430 s. Core discharges 3 kW until the silence and hands the +// battery back and stands the charger down 40 s into it; the meter reads +// degraded while silent; Charge now returns 30 s after each return. orderAt +// is Core's battery order during the silence before the hand-back. +func r1Run(orderAt func(t float64) float64) *runData { + marks := map[string]float64{"charge_now": 0, "silent": 100, "back": 220, "offline": 340, "online": 430, "end": 550} + d := synth(550, marks, map[string]any{"fuse": 25.0}, func(i int, r *sample) { + t := float64(i) + switch { + case t <= 100: + order(r, -3000) + case t < 140: + order(r, orderAt(t)) + } + meter := "ok" + if (t > 102 && t < 220) || (t > 342 && t < 430) { + meter = "degraded" + } + r.Status = &ftwStatus{Drivers: map[string]ftwDriver{"meter": {Status: meter}}} + if t < 140 || (t >= 250 && t < 380) || t >= 460 { + r.CarW[0] = 11040 + } + }) + d.frames = []frame{profileAt(50, 16), profileAt(140, 0), profileAt(240, 16), profileAt(390, 0), profileAt(450, 16)} + return d +} + +func TestJudgeR1(t *testing.T) { + _, cs := judgeR1(r1Run(func(t float64) float64 { return -3000 + 50*(t-100) })) + for name, ok := range verdictsOf(cs) { + if !ok { + t.Errorf("an order easing toward 0: %q fails", name) + } + } + _, cs = judgeR1(r1Run(func(t float64) float64 { return -3000 - 100*(t-100) })) + if v := verdictsOf(cs); v["the battery order moves only toward 0 while the meter is silent"] { + t.Error("an order winding from -3 kW toward -7 kW passes") + } + _, cs = judgeR1(r1Run(func(float64) float64 { return 1000 })) + if v := verdictsOf(cs); v["the battery order moves only toward 0 while the meter is silent"] { + t.Error("an order flipping from discharge to charge passes") + } + late := r1Run(func(float64) float64 { return -3000 }) + late.frames[1] = profileAt(170, 0) + _, cs = judgeR1(late) + if v := verdictsOf(cs); v["a 0 W charger order within 64 s of the silence"] { + t.Error("a standdown 70 s into the silence passes") + } +} + +func TestJudgeR2(t *testing.T) { + marks := map[string]float64{"charge_now": 0, "offline": 100, "online": 280, "end": 400} + run := func(frames []frame) map[string]bool { + d := synth(400, marks, map[string]any{"fuse": 25.0, "charger_delay_s": 0.0}, func(i int, r *sample) { + r.BatW = -3000 + q := "measured" + if i >= 100 && i < 280 { + q = "estimated" + } + w := 5000.0 + r.Status = &ftwStatus{LoadQuality: &q, LoadW: &w} + }) + d.frames = frames + _, cs := judgeR2(d) + return verdictsOf(cs) + } + if v := run([]frame{profileAt(50, 8), profileAt(120, 6)}); !v["no EV offer rises while the battery is offline"] || + !v["Home use shown as estimated throughout the outage"] || !v["the battery discharged 2 kW or more before the drop"] { + t.Fatalf("a cut while offline: %v", v) + } + if v := run([]frame{profileAt(50, 8), profileAt(120, 10)}); v["no EV offer rises while the battery is offline"] { + t.Fatal("a raise from 8 to 10 A while the battery is offline passes") + } +} + +func TestJudgeR3(t *testing.T) { + marks := map[string]float64{"plug": 0, "charge_now": 20, "end": 400} + run := func(set func(i int, r *sample)) map[string]bool { + _, cs := judgeR3(synth(400, marks, map[string]any{"fuse": 25.0, "charger_delay_s": 60.0, "planner_floor_pct": 10.0}, set)) + return verdictsOf(cs) + } + // SoC falls from 12 % to 8 % over the first 100 s. + soc := func(i int) float64 { return 12 - 4*float64(min(i, 100))/100 } + v := run(func(i int, r *sample) { r.Bat.SoCPct = soc(i); order(r, -3000) }) + if v["no discharge ordered below the planner floor"] || !v["the battery reached the planner floor"] { + t.Fatalf("discharge down to 8 %%: %v", v) + } + v = run(func(i int, r *sample) { + r.Bat.SoCPct = soc(i) + if r.Bat.SoCPct > 10 { + order(r, -3000) + } + }) + if !v["no discharge ordered below the planner floor"] || !v["the battery order swings at most 100 W over the last 5 min"] { + t.Fatalf("stopping at the floor: %v", v) + } + v = run(func(i int, r *sample) { r.Bat.SoCPct = 10; order(r, -3000*float64(i%2)) }) + if v["the battery order swings at most 100 W over the last 5 min"] { + t.Fatal("an order hunting between 0 and -3 kW passes") + } + // No order at all, as when Core hands the battery back to the inverter, + // while it runs on from 12 % to the inverter's 5 %. + v = run(func(i int, r *sample) { r.Bat.SoCPct = 12 - 7*float64(min(i, 200))/200 }) + if !v["no discharge ordered below the planner floor"] || v["the battery stays at or above the planner floor"] { + t.Fatalf("a battery left to run to 5 %%: %v", v) + } + v = run(func(i int, r *sample) { r.Bat.SoCPct = max(10, soc(i)) }) + if !v["the battery stays at or above the planner floor"] || !v["the battery reached the planner floor"] { + t.Fatalf("a battery held at the floor: %v", v) + } +} + +func TestJudgeR4SessionEnergy(t *testing.T) { + marks := map[string]float64{"charge_now": 0, "kill1": 100, "up1": 130, "kill2": 250, "up2": 400, "end": 600} + run := func(afterWh float64) map[string]bool { + d := synth(600, marks, map[string]any{"fuse": 25.0, "charger": "CP1"}, func(i int, r *sample) { + wh := 2000.0 + if (i >= 130 && i < 250) || i >= 400 { + wh = afterWh + } + r.Status = &ftwStatus{Drivers: map[string]ftwDriver{"CP1": {Status: "ok", EVSessionWh: &wh}}} + r.CarW[0] = 11040 + }) + _, cs := judgeR4(d) + return verdictsOf(cs) + } + if v := run(0); v["session energy counts again after each restart"] { + t.Fatal("session energy stuck at 0 after the restarts passes") + } + if v := run(150); !v["session energy counts again after each restart"] || !v["Charge now back at its minimum within 60 s of each restart, and held"] { + t.Fatalf("session energy counting after the restarts: %v", v) + } +} + +func TestJudgeR5bAllowsChargingOnlyOnAFreshPlan(t *testing.T) { + dir := t.TempDir() + marks := map[string]float64{"ftw_start": 100, "ftw_up": 110, "end": 200} + // The rig sees the fresh plan, made at 115 s, at 116 s. + plan := `{"t": 111, "generated_at_ms": 40000}` + "\n" + `{"t": 116, "generated_at_ms": 115000}` + "\n" + if err := os.WriteFile(filepath.Join(dir, "plan.jsonl"), []byte(plan), 0o644); err != nil { + t.Fatal(err) + } + run := func(frames []frame) map[string]bool { + d := synth(200, marks, map[string]any{"fuse": 25.0}, nil) + d.dir, d.frames = dir, frames + _, cs := judgeR5b(d) + return verdictsOf(cs) + } + if v := run([]frame{profileAt(112, 16)}); v["no positive EV order in the first 30 s without a fresh plan"] { + t.Fatal("16 A before the fresh plan passes") + } + if v := run([]frame{profileAt(112, 0), profileAt(120, 16)}); !v["no positive EV order in the first 30 s without a fresh plan"] { + t.Fatal("0 A until the fresh plan, then 16 A, fails") + } +} + +func TestHoldsFromResetsOnALaterStop(t *testing.T) { + d := synth(100, nil, nil, func(i int, r *sample) { + if i < 20 || i >= 50 { + r.CarW[0] = 11040 + } + }) + charging := func(r *sample) bool { return carW(r, 0) >= 0.95*minW } + if p := d.holdsFrom(10, 100, charging); p == nil || *p != 40 { + t.Fatalf("charging at 10 s, stopped 20-49 s, back from 50 s: %v, want 40", p) + } + if p := d.holdsFrom(10, 40, charging); p != nil { + t.Fatalf("stopped at the window's end: %v, want nil", *p) + } +} + +// The car still draws at the restart, then Core stops it 20 s later for +// 50 s. A judge that takes the first charging sample calls it back at once. +func TestJudgeR4SeesAStopAfterTheRestart(t *testing.T) { + marks := map[string]float64{"charge_now": 0, "kill1": 100, "up1": 130, "kill2": 250, "up2": 400, "end": 600} + wh := 150.0 + d := synth(600, marks, map[string]any{"fuse": 25.0, "charger": "CP1"}, func(i int, r *sample) { + r.Status = &ftwStatus{Drivers: map[string]ftwDriver{"CP1": {Status: "ok", EVSessionWh: &wh}}} + if i < 150 || i >= 200 { + r.CarW[0] = 11040 + } + }) + m, cs := judgeR4(d) + if verdictsOf(cs)["Charge now back at its minimum within 60 s of each restart, and held"] { + t.Fatalf("a stop from 20 to 70 s after the restart passes: %v", m["charge_now_back_s"]) + } +} + +// r2Run is R2's record: the battery offline from 100 to 280 s, the site at +// 25.0 A (Core's own line, the fuse) before the drop and after the return, and +// overA per phase for the first overS s of the drop, then 20 A. +func r2Run(overA float64, overS int) *runData { + marks := map[string]float64{"charge_now": 0, "offline": 100, "online": 280, "end": 400} + d := synth(400, marks, map[string]any{"fuse": 25.0, "charger_delay_s": 0.0}, func(i int, r *sample) { + r.BatW = -3000 + q, w := "measured", 5000.0 + if i >= 100 && i < 280 { + q = "estimated" + } + r.Status = &ftwStatus{LoadQuality: &q, LoadW: &w} + switch { + case i >= 100 && i < 100+overS: + r.PhaseA = [3]float64{overA, overA, overA} + case i >= 100 && i < 280: + r.PhaseA = [3]float64{20, 20, 20} + default: + r.PhaseA = [3]float64{25.0, 25.0, 25.0} + } + }) + d.frames = []frame{profileAt(50, 8), profileAt(float64(100+overS-2), 0), profileAt(285, 8)} + return d +} + +func TestJudgeR2Overage(t *testing.T) { + const instant = "over 1.2 x the fuse no longer than the charger's delay + 7 s after the drop" + const heat = "the 60 s heat stays under 1.13 x the fuse" + // A short overage, then Core's own line after the return (the review's + // edge case), and the heat budget spent at 29.95 A for 18 s, as on + // master. + for _, c := range []struct { + a float64 + s int + }{{28, 4}, {29.95, 18}} { + _, cs := judgeR2(r2Run(c.a, c.s)) + if v := verdictsOf(cs); !v[instant] || !v[heat] { + t.Errorf("%.2f A for %d s, then Core's line: %v", c.a, c.s, v) + } + } + _, cs := judgeR2(r2Run(31, 12)) + if verdictsOf(cs)[instant] { + t.Error("31 A, over 1.2 x 25 A, for 12 s passes") + } + _, cs = judgeR2(r2Run(29.95, 180)) + if verdictsOf(cs)[heat] { + t.Error("a car never cut at 29.95 A for the whole outage passes") + } +} + +// r5aRun is R5a's record: Core starts at 60 s and answers at 70 s; the site +// runs preA per phase with the car at 16 A until 80 s, then afterA(i) with +// the car at 10 A. +func r5aRun(preA float64, afterA func(i int) float64) *runData { + marks := map[string]float64{"ftw_start": 60, "ftw_up": 70, "end": 250} + return synth(250, marks, map[string]any{"fuse": 25.0}, func(i int, r *sample) { + w := 3000.0 + r.Status = &ftwStatus{LoadW: &w} + if i < 80 { + r.PhaseA, r.CarW[0] = [3]float64{preA, preA, preA}, 11040 + return + } + a := afterA(i) + r.PhaseA, r.CarW[0] = [3]float64{a, a, a}, 6900 + }) +} + +// atFuse is Core's own line on a 25 A fuse: the rating itself. +func atFuse(int) float64 { return 25.0 } + +func TestJudgeR5a(t *testing.T) { + _, cs := judgeR5a(r5aRun(30, atFuse)) + for name, ok := range verdictsOf(cs) { + if !ok { + t.Errorf("30 A cut to Core's own 25.0 A line 10 s after it answers: %q fails", name) + } + } + _, cs = judgeR5a(r5aRun(22.75, atFuse)) + if verdictsOf(cs)["the site was over the fuse before Core started"] { + t.Error("a site at 22.75 A before Core starts passes the setup: the run proves nothing") + } +} + +// R5a judges at the fuse: exactly 25.0 A on a 25 A fuse passes, and 0.1 A +// over it for longer than the 3 s budget fails. +func TestJudgeR5aJudgesAtTheFuse(t *testing.T) { + const under, over = "under the fuse within 30 s of Core answering", "over the fuse at most 3 s once under it" + judge := func(afterA func(int) float64) map[string]bool { + _, cs := judgeR5a(r5aRun(30, afterA)) + return verdictsOf(cs) + } + if v := judge(atFuse); !v[under] || !v[over] { + t.Errorf("held at exactly 25.0 A: %v", v) + } + // Under the fuse at 80 s, then 25.1 A from 100 to 109 s. + v := judge(func(i int) float64 { + if i >= 100 && i < 110 { + return 25.1 + } + return 25.0 + }) + if v[over] { + t.Error("25.1 A for 9 s once under the fuse passes") + } + // The 3 s clock starts at the first sample at or under the fuse, not + // under fuse - 0.5 A: 24.8 A at 80 s, then 26 A from 85 to 94 s, fails. + v = judge(func(i int) float64 { + switch { + case i < 85: + return 24.8 + case i < 95: + return 26 + } + return 24 + }) + if !v[under] || v[over] { + t.Errorf("a dip to 24.8 A, then 26 A for 10 s: %v", v) + } + if v := judge(func(int) float64 { return 25.1 }); v[under] { + t.Error("held at 25.1 A after Core answers passes") + } +} + +// R3 allows the charger's delay + 7 s (67 s here) over the fuse: exactly +// 25.0 A throughout passes, and 25.1 A for 80 s fails. +func TestJudgeR3JudgesAtTheFuse(t *testing.T) { + const over = "over the fuse no longer than the charger's delay + 7 s" + marks := map[string]float64{"plug": 0, "charge_now": 20, "end": 400} + judge := func(a func(int) float64) bool { + d := synth(400, marks, map[string]any{"fuse": 25.0, "charger_delay_s": 60.0, "planner_floor_pct": 10.0}, func(i int, r *sample) { + v := a(i) + r.PhaseA = [3]float64{v, v, v} + }) + _, cs := judgeR3(d) + return verdictsOf(cs)[over] + } + if !judge(atFuse) { + t.Error("held at exactly 25.0 A fails") + } + if judge(func(i int) float64 { + if i >= 100 && i < 180 { + return 25.1 + } + return 25.0 + }) { + t.Error("25.1 A for 80 s passes") + } +} + +// Between the fuse and 1.2 x the fuse R3 and R5a allow what Core's 60 s +// heat allows (the #1888 review): a cool site may sit at 1.08 x the fuse +// past the budget while the heat stays under the fuse; above 1.2 x the +// budget holds. +func TestJudgeR3AndR5aAllowTheHeatBudget(t *testing.T) { + const r3over = "over the fuse no longer than the charger's delay + 7 s" + const r5over = "over the fuse at most 3 s once under it" + r3 := func(a float64) bool { + d := synth(400, map[string]float64{"plug": 0, "charge_now": 20, "end": 400}, + map[string]any{"fuse": 25.0, "charger_delay_s": 60.0, "planner_floor_pct": 10.0}, func(i int, r *sample) { + if i >= 100 && i < 180 { // 80 s, past the 67 s budget + r.PhaseA = [3]float64{a, a, a} + } + }) + _, cs := judgeR3(d) + return verdictsOf(cs)[r3over] + } + if !r3(27) { + t.Error("R3: 27 A for 80 s from a cool fuse fails; its heat stays under 25 A") + } + if r3(30.5) { + t.Error("R3: 30.5 A, over 1.2 x the fuse, for 80 s passes") + } + r5 := func(a float64) bool { + _, cs := judgeR5a(r5aRun(30, func(i int) float64 { + if i >= 200 && i < 215 { // 15 s, past the 3 s budget + return a + } + return 20 + })) + return verdictsOf(cs)[r5over] + } + if !r5(27) { + t.Error("R5a: 27 A for 15 s after 2 min at 20 A fails; its heat stays under 25 A") + } + if r5(30.5) { + t.Error("R5a: 30.5 A, over 1.2 x the fuse, for 15 s passes") + } +} diff --git a/go/test/site/scenarios_test.go b/go/test/site/scenarios_test.go index 0e3258df4..2388ab65b 100644 --- a/go/test/site/scenarios_test.go +++ b/go/test/site/scenarios_test.go @@ -9,6 +9,7 @@ import ( "os" "path/filepath" "strings" + "syscall" "time" ) @@ -27,9 +28,14 @@ type scenario struct { // tolerate names criteria that may fail for a known finding but need // not: random cases, where a seed may or may not reach it. tolerate map[string]string + // skip, when set, says what the pinned device-simulator lacks for this + // scenario: TestSite reports it skipped instead of running it. + skip string } var scenarios = []scenario{ + {name: "s18-16", dir: "s18-car-first-battery-charge-16A", minutes: 8, short: true, + run: s18, judge: judgeS18}, {name: "s0-25", dir: "s0-idle-charger-25A", minutes: 5.1, short: true, run: func(c *runCtx) (map[string]any, error) { return s0(c, 25, 60) }, judge: judgeS0}, {name: "s0b-25", dir: "s0b-idle-charger-default-hb-25A", minutes: 5.1, @@ -94,8 +100,7 @@ var scenarios = []scenario{ {name: "s16-25", dir: "s16-import-ceiling-idle-start-25A", minutes: 68, run: func(c *runCtx) (map[string]any, error) { return s16(c, 25) }, judge: judgeS16}, {name: "s17-25", dir: "s17-two-equal-batteries-25A", minutes: 30, - run: func(c *runCtx) (map[string]any, error) { return s17(c, 25) }, judge: judgeS17, - tolerate: map[string]string{"Energyplan's plan moves the batteries half the time": "the day's prices: within the hour's wait the plan may keep both batteries idle for later, dearer hours"}}, + run: func(c *runCtx) (map[string]any, error) { return s17(c, 25) }, judge: judgeS17}, } // runCtx is where one scenario runs. @@ -410,7 +415,7 @@ func s7(c *runCtx, fuse float64, short bool) (map[string]any, error) { sleepS(900) } st.mark("ftw_stop") - st.stopFTW() + st.stopFTW(syscall.SIGTERM) sleepS(20) st.mark("ftw_start") if err := st.startFTW(); err != nil { @@ -706,10 +711,10 @@ func judgeS12(d *runData) (metrics, []criterion) { if p.Limit > 0 && p.phasesOr3() == 1 { ones++ } - // A lower order while every phase has room is a cut on room alone, - // unless the plan itself asked for less (a pause between slots). + // A lower order while every phase is under the fuse is a cut on room + // alone, unless the plan itself asked for less (a pause between slots). if i > 0 && p.offerW() < prs[i-1].offerW()-1 { - if r := d.at(t0 + p.T); r != nil && maxAbsI(r) < fuse-1-0.5 { + if r := d.at(t0 + p.T); r != nil && maxAbsI(r) < fuse-0.1 { if after := d.at(t0 + p.T + 2); after == nil || firstLP(after).CommandedReason != "plan" { cuts++ } @@ -720,7 +725,7 @@ func judgeS12(d *runData) (metrics, []criterion) { // recorded, not judged, since a slow charger's pending order may hold it. limited := 0 for _, r := range w { - if r.LPs != nil && firstLP(r).CommandedReason == "fuse_limit" && maxAbsI(r) < fuse-1-6 { + if r.LPs != nil && firstLP(r).CommandedReason == "fuse_limit" && maxAbsI(r) < fuse-6 { limited++ } } @@ -959,9 +964,11 @@ func judgeS15p(d *runData) (metrics, []criterion) { // (15 - 3) x 230 - (1 380 + 230) = 1 150 W on its way once kept the car from // starting at all (#1805). With the battery's 2 s the car started 25 s late // when Charge now caught a raise in flight, and at once when it did not. +// The run plans in Self: in passive arbitrage the plan may keep the battery +// idle and export the PV, and then no raise is on its way (#1875). func s15bat(c *runCtx, fuse float64) (map[string]any, error) { err := with(c, stackSpec{fuseA: fuse, loadsW: houseA(3), socPct: 50, chargers: []charger{onePhaseCharger}, - lpExtra: onePhaseExtra, batteryDelayS: 8}, func(st *stack) error { + lpExtra: onePhaseExtra, batteryDelayS: 8, mode: "planner_self"}, func(st *stack) error { if err := st.plug(0); err != nil { return err } @@ -1009,7 +1016,7 @@ func judgeS15bat(d *runData) (metrics, []criterion) { start := d.firstT(t0, func(r *sample) bool { return carW(r, 0) >= 0.95*1380 }) over, longest := 0.0, 0.0 for _, r := range d.window(t0, t1) { - if maxAbsI(r) > fuse-1 { + if maxAbsI(r) > fuse { over++ longest = math.Max(longest, over) } else { @@ -1017,13 +1024,13 @@ func judgeS15bat(d *runData) (metrics, []criterion) { } } m := metrics{"battery_rise_w_20s_before": math.Round(rise), "car_start_s": roundp(start, 1), - "car_w_last60": math.Round(meanOf(d.window(t1-60, t1), func(r *sample) float64 { return carW(r, 0) })), - "over_fuse_minus1_longest_s": longest, "max_abs_i": round(maxOf(d.window(t0, t1), maxAbsI), 1)} + "car_w_last60": math.Round(meanOf(d.window(t1-60, t1), func(r *sample) float64 { return carW(r, 0) })), + "over_fuse_longest_s": longest, "max_abs_i": round(maxOf(d.window(t0, t1), maxAbsI), 1)} return m, []criterion{ crit("the battery order rose over 1 150 W in the 20 s before Charge now", rise > 1150), // the setup crit("car charging within 10 s of Charge now", le(start, 10)), crit("fuse holds", !anyBlown(d.window(t0, t1))), - crit("over fuse - 1 A at most 3 s", longest <= 3), + crit("over the fuse at most 3 s", longest <= 3), } } @@ -1145,11 +1152,11 @@ func judgeS16(d *runData) (metrics, []criterion) { // turns under the plan, one discharging while the other stood at 0 W. // Energyplan's plan puts each quarter's discharge on one battery; Core must // share it. Core DP plans one battery only, so this runs Energyplan, as a -// beta box does. A 3 kW house with no PV. Over 48 h the plan keeps the -// batteries for the dearest hours, often tomorrow's; over 6 h they hold -// more than the house needs, so it discharges now. The run still waits up -// to an hour for a quarter the plan discharges, then runs 15 min. It keeps -// the plan's order for each battery every 30 s in plan.jsonl. +// beta box does. A 3 kW house with no PV. The pinned prices open with a +// 4 h evening peak, and over a 6 h horizon the batteries hold less than +// the house needs in it, so the plan discharges now. The run still waits +// up to an hour for a quarter the plan discharges, then runs 15 min. It +// keeps the plan's order for each battery every 30 s in plan.jsonl. func s17(c *runCtx, fuse float64) (map[string]any, error) { err := with(c, stackSpec{fuseA: fuse, loadsW: [3]float64{1000, 1000, 1000}, socPct: 74, soc2Pct: 60, engine: "energyplan", horizonH: 6}, func(st *stack) error { diff --git a/go/test/site/stack_test.go b/go/test/site/stack_test.go index e45b6fc54..291baaf07 100644 --- a/go/test/site/stack_test.go +++ b/go/test/site/stack_test.go @@ -208,6 +208,7 @@ type houseDevice struct { CarMaxA float64 `json:"car_max_a,omitempty"` CarPhases int `json:"car_phases,omitempty"` WiredPhases int `json:"wired_phases,omitempty"` + Plugged bool `json:"plugged,omitempty"` } // houseEvent is one change to the house (device-simulator's runEvent). @@ -247,6 +248,7 @@ type charger struct { carPhases int // 0: as many as offered wiredPhases int // 1: wired on L1 only, kw its rating there carMaxA float64 + plugged bool // a car on it when the house starts, charging on its own } type stackSpec struct { @@ -269,10 +271,18 @@ type stackSpec struct { // charge; 0 leaves one. soc2Pct float64 // engine is the planner's: core or energyplan. Empty takes - // FTW_SITE_ENGINE, else core. + // FTW_SITE_ENGINE, else energyplan, as a beta or stable box runs. engine string + // mode is the control mode the run selects: planner_self or empty for + // planner_passive_arbitrage, as the web app chooses. + mode string // horizonH is the planner's horizon_hours; 0 leaves Core's 48 h. horizonH int + // plannerSoCMin is the planner's soc_min (a fraction); 0 leaves Core's. + plannerSoCMin float64 + // startAfterS runs the house this long before FTW starts, as a site + // runs before Core boots; FTW's start is marked ftw_start. + startAfterS float64 } type stack struct { @@ -286,6 +296,7 @@ type stack struct { sim string // the house's API base URL rec *recorder cfg string + prices *priceServer // markErr is the first mark the house did not take; the run cannot be // judged without it. markErr error @@ -340,7 +351,7 @@ func (st *stack) houseSpec() houseSpec { h.Devices = append(h.Devices, houseDevice{ID: lpIDs[i], Driver: "ocpp", OCPPURL: fmt.Sprintf("ws://ftw:%s@127.0.0.1:%d/", ocppPass, st.lane.ftwOCPP()), MaxKW: kw, Vendor: "RigCo", Model: fmt.Sprintf("Rig%.0f", kw), PhaseSwitch: c.phaseSwitch, CommandResponseS: delay, - CarPhases: c.carPhases, CarMaxA: c.carMaxA, WiredPhases: c.wiredPhases, + CarPhases: c.carPhases, CarMaxA: c.carMaxA, WiredPhases: c.wiredPhases, Plugged: c.plugged, OCPPConfig: map[string]string{"HeartbeatInterval": "60", "MeterValueSampleInterval": "10"}}) } return h @@ -353,6 +364,7 @@ func (st *stack) up() error { if err := os.MkdirAll(st.rundir, 0o755); err != nil { return err } + st.prices = newPriceServer(time.Now()) spec, _ := json.MarshalIndent(st.houseSpec(), "", " ") specPath := filepath.Join(st.rundir, "house.json") if err := os.WriteFile(specPath, spec, 0o644); err != nil { @@ -388,10 +400,14 @@ func (st *stack) up() error { return err } } + if st.spec.startAfterS > 0 { + sleepS(st.spec.startAfterS) + st.mark("ftw_start") + } if err := st.startFTW(); err != nil { return err } - if _, err := call("POST", st.ftw+"/api/mode", map[string]any{"mode": "planner_self"}, nil); err != nil { + if _, err := call("POST", st.ftw+"/api/mode", map[string]any{"mode": st.mode()}, nil); err != nil { return err } st.rec = &recorder{ftw: st.ftw, path: filepath.Join(st.rundir, "ftw.jsonl")} @@ -408,7 +424,9 @@ func (st *stack) startFTW() error { return waitHTTP(st.ftw+"/api/status", 90*time.Second) } -func (st *stack) stopFTW() { st.ps.stop("ftw", syscall.SIGTERM, 30*time.Second) } +// stopFTW stops FTW with sig: SIGTERM lets it clean up, SIGKILL is a power +// cut. +func (st *stack) stopFTW(sig syscall.Signal) { st.ps.stop("ftw", sig, 30*time.Second) } func (st *stack) down() { if st.rec != nil { @@ -417,6 +435,9 @@ func (st *stack) down() { st.ps.stop("ftw", syscall.SIGTERM, 30*time.Second) st.ps.stop("house", syscall.SIGTERM, 15*time.Second) st.ps.stopAll() + if st.prices != nil { + st.prices.close() + } } // event changes the house now. @@ -516,7 +537,23 @@ func (st *stack) engine() string { if e := os.Getenv("FTW_SITE_ENGINE"); e != "" { return e } - return "core" + return "energyplan" +} + +// mode is the control mode this run selects, and plannerMode the planner +// mode in its config that matches it. +func (st *stack) mode() string { + if st.spec.mode != "" { + return st.spec.mode + } + return "planner_passive_arbitrage" +} + +func (st *stack) plannerMode() string { + if st.mode() == "planner_self" { + return "self_consumption" + } + return "passive_arbitrage" } func (st *stack) ftwConfig() (string, error) { @@ -545,7 +582,9 @@ func (st *stack) ftwConfig() (string, error) { cfg := map[string]any{ "site": map[string]any{"name": "rig", "control_interval_s": 2, "grid_target_w": 0, "grid_tolerance_w": 42, "watchdog_timeout_s": 60}, - "fuse": map[string]any{"max_amps": st.spec.fuseA, "phases": 3, "voltage": 230}, + // safety_margin_a pinned at 0 so the rig judges Core at the rating + // whatever the default. + "fuse": map[string]any{"max_amps": st.spec.fuseA, "phases": 3, "voltage": 230, "safety_margin_a": 0}, "drivers": []map[string]any{ {"name": "meter", "lua": "drivers/sdm630.lua", "is_site_meter": true, "capabilities": modbus("meter")}, {"name": "sungrow", "lua": "drivers/sungrow.lua", "battery_capacity_wh": 10000, "capabilities": modbus("inverter")}, @@ -559,13 +598,17 @@ func (st *stack) ftwConfig() (string, error) { "device_repository": map[string]any{"enabled": false}, // A real box plans. A box without a planner runs Self, where the // battery chases the raw meter and covers the car (dispatch.go), - // which hides the stress on the fuse. - "planner": map[string]any{"enabled": true, "engine": st.engine(), "mode": "self_consumption"}, - "price": map[string]any{"provider": "elprisetjustnu", "zone": "SE3"}, + // which hides the stress on the fuse. Passive arbitrage is the + // mode the web app and a Charge restart return to. + "planner": map[string]any{"enabled": true, "engine": st.engine(), "mode": st.plannerMode()}, + "price": map[string]any{"provider": "elprisetjustnu", "zone": "SE3", "base_url": st.prices.url()}, } if st.spec.horizonH > 0 { cfg["planner"].(map[string]any)["horizon_hours"] = st.spec.horizonH } + if st.spec.plannerSoCMin > 0 { + cfg["planner"].(map[string]any)["soc_min"] = st.spec.plannerSoCMin + } if st.spec.soc2Pct > 0 { cfg["drivers"] = append(cfg["drivers"].([]map[string]any), map[string]any{"name": "sungrow2", "lua": "drivers/sungrow.lua", "battery_capacity_wh": 10000, "capabilities": modbus("inverter2")}) diff --git a/go/test/site/verdicts_test.go b/go/test/site/verdicts_test.go index 9cb180db0..0c1564f97 100644 --- a/go/test/site/verdicts_test.go +++ b/go/test/site/verdicts_test.go @@ -3,7 +3,9 @@ package site import ( + "encoding/json" "math" + "path/filepath" "sort" ) @@ -43,8 +45,11 @@ func judgeS1(d *runData) (metrics, []criterion) { fuse := d.metaF("fuse") target := maxW if fuse < 25 { - // Room on a 16 A fuse: fuse - 1 A margin - house, in 1 A steps. - target = math.Min(maxW, math.Floor(fuse-1-d.metaF("house_a"))*volt*3) + // Room on a 16 A fuse: fuse - house, in 1 A steps. Core works to + // the rating; safety_margin_a is a site's opt-in and the rig sets 0. + // The target is a floor, not the line: the battery (SoC 50) covers + // the house, so a Core that stops at fuse - 1 A also passes. + target = math.Min(maxW, math.Floor(fuse-d.metaF("house_a"))*volt*3) } m := d.carSummary(t0, t1, target) m["target_w"] = target @@ -85,11 +90,10 @@ func judgeS3(d *runData) (metrics, []criterion) { "pre_step_max_i": round(maxOf(pre, maxAbsI), 2), } m["post_step_max_i"] = round(maxOf(d.window(tStep, tBack), maxAbsI), 2) - m["over_fuse_longest_s"], m["over_fuse_total_s"] = d.runsOver(tStep, tBack, fuse) - overLongest, overTotal := d.runsOver(tStep, tBack, fuse-1) - m["over_fuse_minus1_longest_s"], m["over_fuse_minus1_total_s"] = overLongest, overTotal - m["i2_over_fuse_minus1_s"], m["i2_peak_a"] = d.i2Over(tCN, t1, fuse-1) - m["back_under_fuse_minus1_s"] = roundp(d.firstT(tStep, func(r *sample) bool { return maxAbsI(r) <= fuse-1 && r.T > tStep+1 }), 1) + overLongest, overTotal := d.runsOver(tStep, tBack, fuse) + m["over_fuse_longest_s"], m["over_fuse_total_s"] = overLongest, overTotal + m["i2_over_fuse_s"], m["i2_peak_a"] = d.i2Over(tCN, t1, fuse) + m["back_under_fuse_s"] = roundp(d.firstT(tStep, func(r *sample) bool { return maxAbsI(r) <= fuse && r.T > tStep+1 }), 1) last := d.window(tBack-60, tBack) m["post_step_car_w_last60"] = math.Round(meanOf(last, func(r *sample) float64 { return carW(r, 0) })) m["post_step_bat_w_last60"] = math.Round(meanOf(last, batSigned)) @@ -105,15 +109,12 @@ func judgeS3(d *runData) (metrics, []criterion) { // Device response + meter and tick (2 + 2 s) + 3 s. allowed := d.metaF("charger_delay_s") + 2 + 2 + 3 m["allowed_over_s"] = allowed - m["run_over_fuse_minus1_longest_s"], _ = d.runsOver(tCN, t1, fuse-1) + m["run_over_fuse_longest_s"], _ = d.runsOver(tCN, t1, fuse) m["run_max_i"] = round(maxOf(after, maxAbsI), 2) - // Before the step, layers with different margins may hold a phase - // between fuse - 1 A and fuse - 0.5 A; reported, not judged. - m["pre_step_over_fuse_minus1_longest_s"], _ = d.runsOver(tCN, tStep, fuse-1) - m["pre_step_over_fuse_minus_half_longest_s"], _ = d.runsOver(tCN, tStep, fuse-0.5) + m["pre_step_over_fuse_longest_s"], _ = d.runsOver(tCN, tStep, fuse) return m, []criterion{ crit("fuse holds", !isBlown), - crit("over fuse - 1 A no longer than the response time + 3 s after the step", overLongest <= allowed), + crit("over the fuse no longer than the response time + 3 s after the step", overLongest <= allowed), } } @@ -412,10 +413,9 @@ func judgeS8(d *runData) (metrics, []criterion) { fuse := d.metaF("fuse") t0, t1 := d.marks["charge_now"], d.marks["end"] m := d.carSummary(t0, t1, maxW) - m["over_fuse_longest_s"], m["over_fuse_total_s"] = d.runsOver(t0, t1, fuse) - overLongest, overTotal := d.runsOver(t0, t1, fuse-1) - m["over_fuse_minus1_longest_s"], m["over_fuse_minus1_total_s"] = overLongest, overTotal - m["i2_over_fuse_minus1_s"], m["i2_peak_a"] = d.i2Over(t0, t1, fuse-1) + overLongest, overTotal := d.runsOver(t0, t1, fuse) + m["over_fuse_longest_s"], m["over_fuse_total_s"] = overLongest, overTotal + m["i2_over_fuse_s"], m["i2_peak_a"] = d.i2Over(t0, t1, fuse) var prs [][2]float64 for i, p := range connector1(profiles(d.frames, t0, t1)) { if i >= 20 { @@ -433,7 +433,7 @@ func judgeS8(d *runData) (metrics, []criterion) { } return m, []criterion{ crit("fuse holds", !m["fuse_blown"].(bool)), - crit("over fuse - 1 A no longer than the charger's delay + 7 s", overLongest <= allowed), + crit("over the fuse no longer than the charger's delay + 7 s", overLongest <= allowed), } } @@ -459,8 +459,8 @@ func judgeS9(d *runData) (metrics, []criterion) { m["profiles_after_switch"] = seq return m, []criterion{ crit("fuse holds", !before["fuse_blown"].(bool) && !after["fuse_blown"].(bool)), - crit("over fuse - 1 A at most 3 s on 3p", before["over_limit_longest_s"].(float64) <= 3), - crit("over fuse - 1 A at most 3 s after the switch to 1p", after["over_limit_longest_s"].(float64) <= 3), + crit("over the fuse at most 3 s on 3p", before["over_limit_longest_s"].(float64) <= 3), + crit("over the fuse at most 3 s after the switch to 1p", after["over_limit_longest_s"].(float64) <= 3), } } @@ -500,7 +500,7 @@ func judgeS10(d *runData) (metrics, []criterion) { } return m, []criterion{ crit("fuse holds", !before["fuse_blown"].(bool) && !after["fuse_blown"].(bool)), - crit("over fuse - 1 A at most 3 s after the step", after["over_limit_longest_s"].(float64) <= 3), + crit("over the fuse at most 3 s after the step", after["over_limit_longest_s"].(float64) <= 3), } } @@ -547,7 +547,291 @@ func judgeS11(d *runData) (metrics, []criterion) { } return m, []criterion{ crit("an offer within 60 s of the restart", le(firstPosS, 60)), - crit("over fuse - 1 A at most 3 s", run["over_limit_longest_s"].(float64) <= 3), + crit("over the fuse at most 3 s", run["over_limit_longest_s"].(float64) <= 3), crit("fuse holds", !run["fuse_blown"].(bool)), } } + +// ---- R1-R5, the stable plan's P0 scenarios ---- + +// fuseInstantFactor is Core's: above this multiple of the fuse it cuts the +// car at once, with no heat budget (control/fuse_heat.go). mcbHourFactor is +// the current a miniature circuit breaker carries for an hour without a +// trip, the lower of the two ratings Core's heat model keeps under. +const fuseInstantFactor, mcbHourFactor = 1.2, 1.13 + +// judgeR1: while the meter is silent Core raises nothing and its battery +// order only moves toward 0; within the 60 s watchdog and two ticks it +// stands the charger down and hands the battery back; it never shows the +// meter as live; Charge now returns after the meter does and runs until the +// next event. +func judgeR1(d *runData) (metrics, []criterion) { + tCN, tS, tB, t1 := d.marks["charge_now"], d.marks["silent"], d.marks["back"], d.marks["end"] + tOff, tOn := d.marks["offline"], d.marks["online"] + var o0 float64 + if r := d.at(tS); r != nil { + o0 = batOrder(r) + } + wound := 0 + for _, r := range d.window(tS+2, tB) { + if o := batOrder(r); math.Abs(o) > math.Abs(o0)+50 || o*o0 < 0 { + wound++ + } + } + live := 0 + for _, w := range [][2]float64{{tS + 10, tB}, {tOff + 10, tOn}} { + for _, r := range d.window(w[0], w[1]) { + if dr, ok := status(r).Drivers["meter"]; ok && dr.Status == "ok" { + live++ + } + } + } + charging := func(r *sample) bool { return carW(r, 0) >= 0.95*minW } + standdown := d.firstOffer(tS, func(p profile) bool { return p.Limit == 0 }) + handBack := d.firstT(tS, func(r *sample) bool { return !batForced(r) }) + back1, back2 := d.holdsFrom(tB, tOff, charging), d.holdsFrom(tOn, t1, charging) + rises := d.offerRises(tS, tB) + m := metrics{"battery_order_at_silence_w": math.Round(o0), "battery_wound_samples": wound, + "offer_rises_while_silent": rises, "standdown_s": roundp(standdown, 1), "battery_hand_back_s": roundp(handBack, 1), + "meter_live_while_stale_samples": live, "charge_now_back_s": []*float64{roundp(back1, 1), roundp(back2, 1)}} + return m, []criterion{ + crit("the battery relieved the fuse when the meter fell silent", o0 <= -500), // the setup + crit("no EV offer rises while the meter is silent", rises == 0), + crit("the battery order moves only toward 0 while the meter is silent", wound == 0), + crit("a 0 W charger order within 64 s of the silence", le(standdown, 64)), + crit("the battery back on its own within 64 s of the silence", le(handBack, 64)), + crit("the meter never shows live while it is silent or offline", live == 0), + crit("Charge now back at its minimum within 95 s of each return, and held", le(back1, 95) && le(back2, 95)), + crit("fuse holds", !anyBlown(d.window(tCN, t1))), + } +} + +// judgeR2: when the battery that carried the car's room drops out, the fuse +// holds, no offer counts the battery's last -3 kW as room, and Home use says +// it is estimated (the stable plan's R2). Under 1.2 x the fuse Core spends a +// 60 s heat budget before it cuts the car (control/fuse_heat.go), so the +// overage is judged by that contract: above 1.2 x it lasts no longer than +// the charger takes to answer, and the 60 s heat stays under the 1.13 x a +// miniature circuit breaker carries for an hour. How long the site stays +// over the fuse and when Core sends the cut are recorded. +func judgeR2(d *runData) (metrics, []criterion) { + fuse, delay := d.metaF("fuse"), d.metaF("charger_delay_s") + tCN, tOff, tOn, t1 := d.marks["charge_now"], d.marks["offline"], d.marks["online"], d.marks["end"] + pre := meanOf(d.window(tOff-30, tOff), batSigned) + allowed := delay + 7 + instant, _ := d.runsOver(tOff, tOn+60, fuseInstantFactor*fuse) + overFuse, _ := d.runsOver(tOff, tOn+60, fuse) + heatOver, heatPeak := d.i2Over(tOff-60, tOn+60, mcbHourFactor*fuse) + during := summarize(d.window(tOff+15, tOn)) + marked := 0 + for k, v := range during.LoadQuality { + if k != "null" && k != "" && k != "measured" { + marked += v + } + } + rises := d.offerRises(tOff, tOn) + cut := d.firstOffer(tOff, func(p profile) bool { return p.Limit == 0 }) + m := metrics{"battery_w_before": math.Round(pre), "allowed_over_s": allowed, + "over_fuse_x1_2_longest_s": instant, "over_fuse_longest_s": overFuse, + "heat_peak_a": heatPeak, "heat_over_x1_13_s": heatOver, "cut_sent_s": roundp(cut, 1), + "offer_rises_while_offline": rises, "during": during, + "max_i_after_drop": round(maxOf(d.window(tOff, tOn+60), maxAbsI), 2)} + return m, []criterion{ + crit("the battery discharged 2 kW or more before the drop", pre <= -2000), // the setup + crit("fuse holds", !anyBlown(d.window(tCN, t1))), + crit("over 1.2 x the fuse no longer than the charger's delay + 7 s after the drop", instant <= allowed), + crit("the 60 s heat stays under 1.13 x the fuse", heatOver == 0), + crit("no EV offer rises while the battery is offline", rises == 0), + crit("Home use shown as estimated throughout the outage", + during.Samples > 0 && during.LoadWShown == during.Samples && marked == during.Samples), + } +} + +// judgeR3: Core orders no discharge below the planner's floor, its battery +// order settles, and the car answers when the battery stops. The measured +// SoC must hold the floor too: an order of 0 W also reads from a battery +// handed back to the inverter, which would run on to its own 5 %. +func judgeR3(d *runData) (metrics, []criterion) { + fuse, delay, floor := d.metaF("fuse"), d.metaF("charger_delay_s"), d.metaF("planner_floor_pct") + t0, tCN, t1 := d.marks["plug"], d.marks["charge_now"], d.marks["end"] + run := d.window(t0, t1) + below := 0 + for _, r := range run { + if batOrder(r) < -50 && r.Bat.SoCPct < floor-0.5 { + below++ + } + } + last := d.window(t1-300, t1) + swing := 0.0 + if len(last) > 0 { + lo, hi := math.Inf(1), math.Inf(-1) + for _, r := range last { + lo, hi = math.Min(lo, batOrder(r)), math.Max(hi, batOrder(r)) + } + swing = hi - lo + } + minSoC := minOf(run, func(r *sample) float64 { return r.Bat.SoCPct }) + // Core may work up to the fuse rating; safety_margin_a is a site's + // opt-in, so the overage that counts is above the fuse. Between the + // fuse and 1.2 x the fuse Core's 60 s heat is the budget (heatOver). + overInstant, _ := d.runsOver(tCN, t1, fuse) + over := d.heatRunsOver(tCN, t1, fuse) + allowed := delay + 7 + m := metrics{"discharge_below_floor_samples": below, "battery_order_swing_last5min_w": math.Round(swing), + "soc_min_pct": round(minSoC, 2), "over_fuse_longest_s": overInstant, "over_fuse_heat_longest_s": over, + "allowed_over_s": allowed, "max_i": round(maxOf(run, maxAbsI), 2)} + return m, []criterion{ + crit("the battery reached the planner floor", len(run) > 0 && minSoC <= floor+0.5), // the setup + crit("no discharge ordered below the planner floor", below <= 10), + crit("the battery stays at or above the planner floor", len(run) > 0 && minSoC >= floor-0.5), + crit("the battery order swings at most 100 W over the last 5 min", len(last) > 0 && swing <= 100), + crit("over the fuse no longer than the charger's delay + 7 s", over <= allowed), + crit("fuse holds", !anyBlown(run)), + } +} + +// judgeR4: after each hard kill Charge now is back within a minute and runs +// until the next kill or the end (a car still drawing from before the kill +// and stopped later is not back), the session energy counts again and +// state.db opens clean; the day's Home use +// survives the first kill. The second outage's 120 s would break the 10 % +// unknown-time rule on a run this short, so its day total is recorded only. +func judgeR4(d *runData) (metrics, []criterion) { + fuse := d.metaF("fuse") + tCN, k1, u1, k2, u2, t1 := d.marks["charge_now"], d.marks["kill1"], d.marks["up1"], d.marks["kill2"], d.marks["up2"], d.marks["end"] + charging := func(r *sample) bool { return carW(r, 0) >= 0.95*minW } + back1, back2 := d.holdsFrom(u1, k2, charging), d.holdsFrom(u2, t1, charging) + before, after := todayLoadWh(d.window(k1-30, k1)), todayLoadWh(d.window(u1, k2)) + nullAfter := 0 + for _, r := range d.window(u1, k2) { + if r.Status != nil && r.Status.Energy.Today.LoadWh == nil { + nullAfter++ + } + } + kept := before != nil && after != nil && nullAfter == 0 && *after >= 0.99**before + charger, _ := d.meta["charger"].(string) + session := func(t0, t1 float64) float64 { + v := 0.0 + for _, r := range d.window(t0, t1) { + if dr, ok := status(r).Drivers[charger]; ok && dr.EVSessionWh != nil { + v = math.Max(v, *dr.EVSessionWh) + } + } + return v + } + recovery := d.logLines("database corrupt", "INTEGRITY CHECK FAILED") + overAfter, _ := d.runsOver(u2, t1, fuse) + m := metrics{"charge_now_back_s": []*float64{roundp(back1, 1), roundp(back2, 1)}, + "today_load_wh_before_kill1": before, "today_load_wh_before_kill2": after, "today_load_wh_null_after_kill1": nullAfter, + "today_load_wh_end": todayLoadWh(d.window(u2, t1)), + "session_wh_before_kills": []float64{session(k1-30, k1), session(k2-30, k2)}, + "session_wh_90s_after_ups": []float64{session(u1, u1+90), session(u2, u2+90)}, + "recovery_log_lines": len(recovery), "over_fuse_longest_after_up2_s": overAfter} + return m, []criterion{ + crit("Charge now back at its minimum within 60 s of each restart, and held", le(back1, 60) && le(back2, 60)), + crit("the day's Home use kept within 1 % across the first kill", kept), + crit("session energy counts again after each restart", session(u1, u1+90) > 0 && session(u2, u2+90) > 0), + crit("state.db opens with no recovery error", len(recovery) == 0), + crit("fuse holds", !anyBlown(d.window(tCN, t1))), + } +} + +// homeUseBack is the seconds from t0 to the first sample with Home use shown. +func (d *runData) homeUseBack(t0 float64) *float64 { + return d.firstT(t0, func(r *sample) bool { return status(r).LoadW != nil }) +} + +// judgeR5a: Core starting beside a car that already draws its maximum, with +// the site over the fuse, brings the site under the limit within 30 s and +// keeps it there. The limit is the fuse: Core may work up to the rating, +// and safety_margin_a is a site's opt-in. Once under it, a reading between +// the fuse and 1.2 x the fuse counts only while the 60 s heat is over the +// fuse too (heatOver), as Core's cut does. +func judgeR5a(d *runData) (metrics, []criterion) { + fuse := d.metaF("fuse") + tS, tUp, t1 := d.marks["ftw_start"], d.marks["ftw_up"], d.marks["end"] + pre := meanOf(d.window(tS-20, tS), func(r *sample) float64 { return carW(r, 0) }) + preI := maxOf(d.window(tS-20, tS), maxAbsI) + fit := d.firstT(tUp, func(r *sample) bool { return maxAbsI(r) <= fuse }) + overAfter := 0.0 + if fit != nil { + overAfter = d.heatRunsOver(tUp+*fit, t1, fuse) + } + shown := d.homeUseBack(tUp) + m := metrics{"car_w_before_start": math.Round(pre), "under_limit_s": roundp(fit, 1), "over_after_that_longest_s": overAfter, + "home_use_shown_s": roundp(shown, 1), "max_i_before_start": round(preI, 2)} + return m, []criterion{ + crit("the car charged before Core started", pre >= minW), // the setup + crit("the site was over the fuse before Core started", preI > fuse), // the setup + crit("under the fuse within 30 s of Core answering", le(fit, 30)), + crit("over the fuse at most 3 s once under it", fit != nil && overAfter <= 3), + crit("Home use shown within 30 s of Core answering", le(shown, 30)), + crit("fuse holds", !anyBlown(d.window(tS, t1))), + } +} + +// judgeR5b: a Core that starts on a stale plan orders no charge until a +// fresh plan asks for it, at least for its first 30 s. +func judgeR5b(d *runData) (metrics, []criterion) { + tS, tUp, t1 := d.marks["ftw_start"], d.marks["ftw_up"], d.marks["end"] + fresh, polls := math.Inf(1), 0 + readLines(filepath.Join(d.dir, "plan.jsonl"), func(b []byte) { + var l struct { + T float64 `json:"t"` + GeneratedAtMs int64 `json:"generated_at_ms"` + } + if json.Unmarshal(b, &l) != nil { + return + } + polls++ + if gen := float64(l.GeneratedAtMs) / 1000; gen >= tS { + fresh = math.Min(fresh, gen) + } + }) + early := 0 + for _, p := range d.offers() { + if p.T >= tS && p.T < tUp+30 && p.T < fresh && p.Limit > 0 { + early++ + } + } + shown := d.homeUseBack(tUp) + var freshS *float64 + if !math.IsInf(fresh, 1) { + v := round(fresh-tUp, 1) + freshS = &v + } + m := metrics{"plan_polls": polls, "fresh_plan_s": freshS, "positive_orders_before_fresh_plan": early, + "home_use_shown_s": roundp(shown, 1)} + return m, []criterion{ + crit("no positive EV order in the first 30 s without a fresh plan", early == 0), + crit("Home use shown within 30 s of Core answering", le(shown, 30)), + crit("fuse holds", !anyBlown(d.window(tS, t1))), + } +} + +// judgeR5c: Core starting into a session that began while it was away sees +// the car and shows Home use. Whether the new session keeps the old car's +// Charge now waits for the owner (ev-088): recorded, not judged. +func judgeR5c(d *runData) (metrics, []criterion) { + tS, tUp, t1 := d.marks["ftw_start"], d.marks["ftw_up"], d.marks["end"] + held, seen := 0, 0 + for _, r := range d.window(tUp+30, t1) { + if r.LPs == nil { + continue + } + seen++ + if firstLP(r).ManualActive { + held++ + } + } + plugged := d.firstT(tUp, func(r *sample) bool { return r.LPs != nil && firstLP(r).Plugged }) + shown := d.homeUseBack(tUp) + m := metrics{"charge_now_kept_samples": held, "samples": seen, "plugged_s": roundp(plugged, 1), + "home_use_shown_s": roundp(shown, 1), + "car_w_mean": math.Round(meanOf(d.window(tUp+30, t1), func(r *sample) float64 { return carW(r, 0) }))} + return m, []criterion{ + crit("Core sees the car plugged in within 60 s of answering", le(plugged, 60)), + crit("Home use shown within 30 s of Core answering", le(shown, 30)), + crit("fuse holds", !anyBlown(d.window(tS, t1))), + } +} diff --git a/optimizer/native/README.md b/optimizer/native/README.md index f954123e0..def86c45d 100644 --- a/optimizer/native/README.md +++ b/optimizer/native/README.md @@ -8,7 +8,7 @@ The [Energyplan contract](../../docs/energyplan-contract.md) defines the shared planning and forecast boundary, result meanings, units and failure rules. Core owns it; Energyplan keeps an identical copy. -Energyplan 0.5.2 uses the Home Use Binary License in `bundle/LICENSE.txt`. It +Energyplan 0.7.0 uses the Home Use Binary License in `bundle/LICENSE.txt`. It permits private household use with FTW and free noncommercial redistribution for that use. Commercial use, OEM bundles, paid installation and services need a separate written license from Sourceful Labs AB. FTW's AGPL code has a @@ -29,9 +29,11 @@ optimizer/native/bundle/ftw-solver-linux-arm64 --time-limit=100ms < requests.jso The bundle requires static Linux ARM64 and Linux AMD64 workers. A macOS ARM64 worker is optional and is not a deployment target. The manifest pins their version, private source commit, sizes and SHA-256 checksums. The verifier -checks every bundled file, the host worker handshake on Linux, and the public -source boundary. Keep the license and notices with any copied or redistributed -executable. Run only a verified bundle. +checks every bundled file, the host worker handshake, and the public source +boundary. On a host the bundle has no worker for, such as macOS without the +optional worker, it checks the files and skips execution. Keep the license and +notices with any copied or redistributed executable. Run only a verified +bundle. For a captured single-battery request without scenarios, demand charges, thermal loads or PV controls, Core can replay the exact response from another @@ -56,8 +58,10 @@ explicitly, or `core` to select Core DP. Stable and development builds keep Core as the unset default; Windows and macOS have no required bundled worker. -Energyplan uses the same downside PV forecast as Core. Small requests get a -500 ms solve budget; larger fleets and PV-control requests get 5 s. +Energyplan uses the same downside PV forecast as Core. Core's call budgets +(500 ms, 1.5 s with an active car, 5 s for larger fleets and for PV-control, +demand-charge and import-cap requests, each fitted to its call's deadline) are +in the contract. Core applies PV uncertainty once when forming the downside horizon. This does not add worker scenarios or extend the solve budget by itself. The transport timeout is 7 s. After Core validates and publishes a plan, one diff --git a/optimizer/native/bundle/THIRD-PARTY-NOTICES.txt b/optimizer/native/bundle/THIRD-PARTY-NOTICES.txt index 83850bef4..6b3662c7d 100644 --- a/optimizer/native/bundle/THIRD-PARTY-NOTICES.txt +++ b/optimizer/native/bundle/THIRD-PARTY-NOTICES.txt @@ -530,7 +530,7 @@ Upstream: https://github.com/Specy/microlp Vendored from crates.io microlp 0.6.0. Upstream source commit: 22f625e4112967946e9dcd2bc87378424b2d265e Crate SHA256: a8f19803f918039a07f3c32b23716824d8f073543417bdac5b1b8619817e3674 -Sourceful modification: simplex deadline polling every 16 instead of 1000 pivots; opt-in diagnostic phase spans; a continuous-relaxation clone and fixed-variable bounds; direct adoption of complete warm starts after full bounds/domain/constraint validation; read access to a problem's variables and constraints; a time limit for later edits of a solution. +Sourceful modification: simplex deadline polling every 16 instead of 1000 pivots; opt-in diagnostic phase spans; a continuous-relaxation clone and fixed-variable bounds; direct adoption of complete warm starts after full bounds/domain/constraint validation; read access to a problem's variables and constraints; a time limit for later edits of a solution; read-only solve statistics; smaller branch & bound node storage. The upstream Apache-2.0 license and copyright notices remain in this package. diff --git a/optimizer/native/bundle/ftw-solver-darwin-arm64 b/optimizer/native/bundle/ftw-solver-darwin-arm64 deleted file mode 100755 index 05cd84aa6..000000000 Binary files a/optimizer/native/bundle/ftw-solver-darwin-arm64 and /dev/null differ diff --git a/optimizer/native/bundle/ftw-solver-linux-amd64 b/optimizer/native/bundle/ftw-solver-linux-amd64 index e3e36e525..9ef3f2dc9 100755 Binary files a/optimizer/native/bundle/ftw-solver-linux-amd64 and b/optimizer/native/bundle/ftw-solver-linux-amd64 differ diff --git a/optimizer/native/bundle/ftw-solver-linux-arm64 b/optimizer/native/bundle/ftw-solver-linux-arm64 index 5f41e41da..42cdb5607 100755 Binary files a/optimizer/native/bundle/ftw-solver-linux-arm64 and b/optimizer/native/bundle/ftw-solver-linux-arm64 differ diff --git a/optimizer/native/bundle/manifest.json b/optimizer/native/bundle/manifest.json index 02850f515..c504ea838 100644 --- a/optimizer/native/bundle/manifest.json +++ b/optimizer/native/bundle/manifest.json @@ -1,9 +1,9 @@ { "schema_version": 1, "product": "energyplan", - "version": "0.5.2", + "version": "0.7.0", "source_repository": "srcfl/energyplan", - "source_commit": "38c4872a5f646e0b822f7645ae06f3b454a8ffea", + "source_commit": "847936a15de6efed4192c2ec2d67f31f6f98ce64", "rustc": "rustc 1.95.0 (59807616e 2026-04-14)", "protocol_version": 1, "forecast_protocol_version": 1, @@ -15,10 +15,6 @@ "linux-amd64": { "path": "ftw-solver-linux-amd64", "target": "x86_64-unknown-linux-musl" - }, - "darwin-arm64": { - "path": "ftw-solver-darwin-arm64", - "target": "aarch64-apple-darwin" } }, "files": { @@ -27,8 +23,8 @@ "bytes": 4296 }, "THIRD-PARTY-NOTICES.txt": { - "sha256": "9ee090bbf54b50407b9150c4e4311f6baff76f68c738b9b17fa3f3afbc2693a7", - "bytes": 55987 + "sha256": "d6d5fa4eea968c3b480bce0078f26a62690d24732263d5f98fcc9b488dacf149", + "bytes": 56052 }, "forecast-v1.response.schema.json": { "sha256": "b7f06c00679b3798cbedd458f8b9b6bfd556e96ac442bb3820f9696c6c7bdd99", @@ -38,17 +34,13 @@ "sha256": "6b374bd19d351e568761e1050016202d78aea2fc2e6783a2e9245aa23ffb2d3f", "bytes": 15377 }, - "ftw-solver-darwin-arm64": { - "sha256": "65f819dd4f723641a74efe0a2210531f2f5f82ef64e0c98dc6318baf78c3d85f", - "bytes": 1472304 - }, "ftw-solver-linux-amd64": { - "sha256": "a7bc7c89642763c51892d361af99f60370f4cc8085cc92b02f5a4b9ee1cda260", - "bytes": 1889768 + "sha256": "67ff308aa4f30134a066898a1638f2f0436eff0a299d661af8de3a04cadb2bee", + "bytes": 2102880 }, "ftw-solver-linux-arm64": { - "sha256": "2057e3dc0f37fb5fc5a164f8717406c63fcc0d68ce1ddfe24f1a42a80e34e87f", - "bytes": 1511296 + "sha256": "a6f1609d61c38d6a8fb7b00ec695ed6bdd962c657b9fe544833c4acde0d4338b", + "bytes": 1674152 }, "rust-runtime/COPYRIGHT-library.html": { "sha256": "90567e2718bf7fd65a71a3a43c5596488e80e5f51ed02bfea6fec54458b5f3d1", diff --git a/optimizer/native/verify.py b/optimizer/native/verify.py index 354a476c9..80278b2f4 100644 --- a/optimizer/native/verify.py +++ b/optimizer/native/verify.py @@ -101,7 +101,7 @@ def main(): root = HERE / "bundle" manifest = verify_bundle(root) host = host_key() - if host is None: + if host not in manifest["artifacts"]: if not args.host_binary: print(f"Verified Energyplan {manifest['version']}: bundle integrity passed; no worker for this host, execution skipped") return diff --git a/optimizer/native/verify_test.py b/optimizer/native/verify_test.py index 6926be21b..a3f0bad35 100644 --- a/optimizer/native/verify_test.py +++ b/optimizer/native/verify_test.py @@ -1,4 +1,6 @@ """Exercise artifact corruption and accidental source disclosure boundaries.""" +import contextlib +import io import json from pathlib import Path import shutil @@ -6,6 +8,7 @@ import unittest from unittest.mock import patch +import verify from verify import HERE, verify_bundle, host_key @@ -47,6 +50,25 @@ def test_linux_only_bundle(self): manifest.write_text(json.dumps(data)) self.assertEqual(verify_bundle(self.root)["product"], "energyplan") + def test_linux_only_bundle_skips_execution_on_darwin_host(self): + self.test_linux_only_bundle() + out = io.StringIO() + with patch("verify.platform.system", return_value="Darwin"), \ + patch("verify.platform.machine", return_value="arm64"), \ + patch("verify.HERE", self.root.parent), patch("verify.check_public_tree"), \ + patch("verify.subprocess.run") as run, patch("sys.argv", ["verify.py"]), \ + contextlib.redirect_stdout(out): + verify.main() + run.assert_not_called() + self.assertIn("execution skipped", out.getvalue()) + out = io.StringIO() + with patch("verify.platform.system", return_value="Darwin"), \ + patch("verify.platform.machine", return_value="arm64"), \ + patch("verify.HERE", self.root.parent), patch("verify.check_public_tree"), \ + patch("sys.argv", ["verify.py", "--host-binary"]), contextlib.redirect_stdout(out): + verify.main() + self.assertEqual(out.getvalue(), "") + def test_missing_required_linux_platform(self): (self.root / "ftw-solver-linux-amd64").unlink() manifest = self.root / "manifest.json" diff --git a/package-lock.json b/package-lock.json index e25b60a2a..d836aec98 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "ftw", - "version": "0.156.0", + "version": "0.159.1", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "ftw", - "version": "0.156.0", + "version": "0.159.1", "license": "SEE LICENSE IN LICENSE", "devDependencies": { "@changesets/cli": "^3.0.3" diff --git a/package.json b/package.json index 36cdf331c..0b7cd3175 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "ftw", - "version": "0.156.0", + "version": "0.159.1", "private": true, "type": "module", "description": "FTW — local-first home energy coordination. Version metadata only; the runtime is Go and the mathematical planner is the compiled Energyplan worker.", diff --git a/scripts/check-native-release-order.py b/scripts/check-native-release-order.py index 619c2cffa..3e2ebac98 100644 --- a/scripts/check-native-release-order.py +++ b/scripts/check-native-release-order.py @@ -1,5 +1,9 @@ #!/usr/bin/env python3 -"""Refuse a native release older than a published native candidate.""" +"""Refuse a native release older than a published native candidate. + +A stable promotes a beta that has been tested for days, so newer betas may +exist by then. A stable is refused only when a newer stable exists. +""" import json from pathlib import Path @@ -32,7 +36,7 @@ def check(tag, pages): continue other_tag = release["tag_name"] other = rank(other_tag) - if other is not None and other > target: + if other is not None and other > target and (other[2] or not target[2]): raise ValueError(f"{tag} is older than published {other_tag}") diff --git a/scripts/test_check_native_release_order.py b/scripts/test_check_native_release_order.py index 819727272..5ebffff28 100644 --- a/scripts/test_check_native_release_order.py +++ b/scripts/test_check_native_release_order.py @@ -32,6 +32,60 @@ def test_published_stable_closes_its_beta_line(self): with self.assertRaises(ValueError): release_order.check("v0.131.0-beta.3", pages) + def test_newer_beta_does_not_block_tested_stable(self): + release_order.check("v0.158.0", [[ + {"tag_name": "v0.159.0-beta.1", "draft": False}, + {"tag_name": "v0.158.0-beta.3", "draft": False}, + ]]) + release_order.check("v0.157.2", [[ + {"tag_name": "v0.157.3-beta.1", "draft": False}, + {"tag_name": "v0.157.2-beta.1", "draft": False}, + ]]) + release_order.check("v0.158.1", [[ + {"tag_name": "v0.159.0-beta.1", "draft": False}, + {"tag_name": "v0.158.0", "draft": False}, + {"tag_name": "v0.158.1-beta.1", "draft": False}, + ]]) + + def test_rejects_stable_older_than_published_stable(self): + pages = [[ + {"tag_name": "v0.159.0", "draft": False}, + {"tag_name": "v0.158.1-beta.1", "draft": False}, + ]] + for tag in ("v0.158.1", "v0.158.0-beta.4"): + with self.subTest(tag=tag), self.assertRaises(ValueError): + release_order.check(tag, pages) + + def test_rejects_beta_older_than_published_beta(self): + with self.assertRaises(ValueError): + release_order.check("v0.158.1-beta.1", [[ + {"tag_name": "v0.159.0-beta.1", "draft": False}, + ]]) + + def test_after_late_stable_betas_stay_above_every_release(self): + # The updater stops at the first beta it lists (selfupdate.go + # resolveChannel), so no beta may rank below a published release. + pages = [[ + {"tag_name": "v0.158.0", "draft": False}, + {"tag_name": "v0.159.0-beta.1", "draft": False}, + ]] + release_order.check("v0.159.0-beta.2", pages) + release_order.check("v0.159.1-beta.1", pages) + release_order.check("v0.158.0", pages) + with self.assertRaises(ValueError): + release_order.check("v0.158.0-beta.4", pages) + + def test_newer_stable_on_a_later_page_blocks_stable(self): + with self.assertRaises(ValueError): + release_order.check("v0.158.0", [ + [{"tag_name": "v0.160.0-beta.1", "draft": False}], + [{"tag_name": "v0.159.0", "draft": False}], + ]) + release_order.check("v0.158.0", [[ + {"tag_name": "v0.159.0", "draft": True}, + {"tag_name": "v0.159.0-beta.1", "draft": False}, + ]]) + def test_drafts_do_not_block_recovery(self): release_order.check("v0.131.0-beta.1", [[ {"tag_name": "v0.131.0-beta.2", "draft": True}, diff --git a/web/app.js b/web/app.js index 6903f25b1..449342fa3 100644 --- a/web/app.js +++ b/web/app.js @@ -2350,7 +2350,7 @@ } // ---- Storage-health banner (DB corruption auto-recovered) ---- - // storage = { state, cache, last_event_ms, detail } from /api/health. + // storage = { state, cache, history, last_event_ms, detail } from /api/health. // Heal events are set once at boot and never change, so a session-scoped // dismissal (keyed on the event) hides it without losing a fresh alert // after a later restart. @@ -2358,7 +2358,8 @@ var existing = document.getElementById("storage-banner"); var bad = !!storage && ( (storage.state && storage.state !== "ok") || - (storage.cache && storage.cache !== "ok") + (storage.cache && storage.cache !== "ok") || + (storage.history && storage.history !== "ok") ); if (!bad) { if (existing) existing.remove(); return; } diff --git a/web/battery-return.js b/web/battery-return.js new file mode 100644 index 000000000..2b9c4fb8a --- /dev/null +++ b/web/battery-return.js @@ -0,0 +1,158 @@ +(function () { + 'use strict'; + + // The refund runs on Swedish calendar years. An owner can apply for the + // three years before this one, and the running year shows progress. + var firstYear = 2018; + var details = document.getElementById('battery-return-details'); + var select = document.getElementById('battery-return-year'); + var csv = document.getElementById('battery-return-csv'); + var result = document.getElementById('battery-return-result'); + var rows = document.getElementById('battery-return-rows'); + var meta = document.getElementById('battery-return-meta'); + if (!details || !select || !result || !rows) return; + var request = 0; + + var thisYear = Number(new Intl.DateTimeFormat('en', { year: 'numeric', timeZone: 'Europe/Stockholm' }).format(new Date())); + for (var year = thisYear; year >= Math.max(firstYear, thisYear - 3); year--) { + var option = document.createElement('option'); + option.value = String(year); + option.textContent = year === thisYear ? year + ' (so far)' : String(year); + select.appendChild(option); + } + select.value = String(Math.max(firstYear, thisYear - 1)); + + // Whole kWh, rounded down: the figure an owner enters never exceeds what + // the readings show. + function kWh(wh) { + return Math.floor(Math.max(0, Number(wh) || 0) / 1000).toLocaleString() + ' kWh'; + } + + function kWhExact(wh) { + return (Math.max(0, Number(wh) || 0) / 1000).toLocaleString(undefined, { maximumFractionDigits: 1 }) + ' kWh'; + } + + // "a", "a and b", "a, b and c". + function list(names) { + return names.length < 2 ? names.join('') : names.slice(0, -1).join(', ') + ' and ' + names[names.length - 1]; + } + + // The drivers that read the batteries that pass the test, once each. + function labels(data, test) { + var seen = {}; + return (data.batteries || []).filter(function (battery) { + if (!test(battery) || !battery.label || seen[battery.label]) return false; + seen[battery.label] = true; + return true; + }).map(function (battery) { return battery.label; }); + } + + // 'the driver "a"', 'the drivers "a" and "b"'. + function drivers(names) { + return (names.length > 1 ? 'the drivers ' : 'the driver ') + + list(names.map(function (name) { return '\u201c' + name + '\u201d'; })); + } + + function percent(efficiency) { + return Math.floor((Number(efficiency) || 0) * 100) + ' %'; + } + + // Why a year has no figure, from the batteries' configuration. + var problems = { + battery_size: function (data) { + var names = labels(data, function (battery) { return !battery.sized; }); + return (names.length ? 'Set the battery size of ' + drivers(names) : 'Set the battery\'s size') + + ' in Settings to get a figure. FTW needs to know how much each battery can hold.'; + }, + battery_unread: function (data) { + return 'Settings gives ' + drivers(data.unread || []) + ' a battery size, but FTW read no battery through ' + + ((data.unread || []).length > 1 ? 'them' : 'it') + ' in ' + data.year + '. FTW could count what an unread battery ' + + 'sent to the grid for another one, so there is no figure.'; + } + }; + + // How the figure was counted: each battery's size and efficiency. + function counted(batteries) { + if (batteries.length === 1) { + return ' Counted with a ' + kWhExact(batteries[0].capacity_wh) + ' battery that gave back ' + + percent(batteries[0].efficiency) + ' of its charge.'; + } + if (!batteries.length) return ''; + return ' Counted for each battery on its own: ' + list(batteries.map(function (battery) { + return kWh(battery.returned_wh) + ' from ' + battery.label + ' (' + kWhExact(battery.capacity_wh) + + ', gave back ' + percent(battery.efficiency) + ' of its charge)'; + })) + '.'; + } + + function monthName(month) { + return new Date(Date.UTC(2000, month - 1, 15)).toLocaleDateString(undefined, { month: 'long', timeZone: 'UTC' }); + } + + function cell(row, text) { + var td = document.createElement('td'); + td.textContent = text; + row.appendChild(td); + } + + function render(data) { + var complete = Number(data.complete_hours) || 0; + var hours = Number(data.hours) || 0; + var running = Number(data.year) === thisYear; + result.textContent = ''; + if (problems[data.problem]) { + result.className = 'energy-history-quality-note'; + result.textContent = problems[data.problem](data); + } else if (!complete) { + result.className = 'energy-history-quality-note'; + result.textContent = 'FTW has no complete readings for ' + data.year + '.'; + } else { + result.className = 'energy-history-quality-note' + (complete === hours && !running ? ' good' : ''); + var figure = document.createElement('strong'); + figure.textContent = kWh(data.returned_wh); + result.appendChild(figure); + result.appendChild(document.createTextNode( + ' of grid electricity went back to the grid after battery storage in ' + data.year + '. ' + + 'Readings are complete for ' + complete.toLocaleString() + ' of ' + hours.toLocaleString() + ' hours' + + (running ? ' so far; the year is still running.' : '.') + counted(data.batteries || []) + )); + } + rows.textContent = ''; + // Without a figure the months have none either. + (problems[data.problem] ? [] : data.months || []).forEach(function (month) { + if (!month.hours) return; + var row = document.createElement('tr'); + cell(row, monthName(month.month)); + cell(row, month.complete_hours ? kWh(month.returned_wh) : 'No readings'); + cell(row, Number(month.complete_hours).toLocaleString() + ' of ' + Number(month.hours).toLocaleString()); + rows.appendChild(row); + }); + if (meta) meta.textContent = problems[data.problem] ? 'No figure' : complete ? kWh(data.returned_wh) + ' in ' + data.year : 'No readings'; + } + + function load() { + var current = ++request; + var year = select.value; + if (csv) csv.href = '/api/energy/battery-return.csv?year=' + encodeURIComponent(year); + result.className = 'energy-history-quality-note'; + result.textContent = 'Working out ' + year + '…'; + rows.textContent = ''; + if (meta) meta.textContent = ''; + fetch('/api/energy/battery-return?year=' + encodeURIComponent(year)).then(function (response) { + return response.json().catch(function () { return {}; }).then(function (data) { + if (!response.ok) throw new Error(data.error || 'The figure for ' + year + ' is unavailable.'); + return data; + }); + }).then(function (data) { + if (current === request) render(data); + }).catch(function (error) { + if (current !== request) return; + result.className = 'energy-history-quality-note'; + result.textContent = error.message; + }); + } + + details.addEventListener('toggle', function () { + if (details.open) load(); + }); + select.addEventListener('change', load); +})(); diff --git a/web/battery-return.test.mjs b/web/battery-return.test.mjs new file mode 100644 index 000000000..34a235a9a --- /dev/null +++ b/web/battery-return.test.mjs @@ -0,0 +1,133 @@ +import assert from 'node:assert/strict'; +import { readFileSync } from 'node:fs'; +import test from 'node:test'; +import vm from 'node:vm'; + +const html = readFileSync(new URL('./index.html', import.meta.url), 'utf8'); +const source = readFileSync(new URL('./battery-return.js', import.meta.url), 'utf8'); + +test('History offers the battery refund figure closed, with its hourly basis', () => { + const history = html.match(/
]+open/); + assert.equal((html.match(/battery-return\.js/g) || []).length, 1); +}); + +function harness(response) { + const listeners = new Map(); + const element = (id) => ({ + id, + children: [], + className: '', + href: '', + open: false, + textContent: '', + value: '', + addEventListener(name, handler) { listeners.set(id + ':' + name, handler); }, + appendChild(child) { this.children.push(child); }, + }); + const ids = [ + 'battery-return-details', 'battery-return-year', 'battery-return-csv', + 'battery-return-result', 'battery-return-rows', 'battery-return-meta', + ]; + const elements = new Map(ids.map((id) => [id, element(id)])); + const requests = []; + vm.runInNewContext(source, { + Date, Intl, Math, Number, Object, Promise, String, encodeURIComponent, + document: { + createElement: (tag) => element(tag), + createTextNode: (text) => ({ textContent: text }), + getElementById: (id) => elements.get(id) || null, + }, + fetch: async (path) => { + requests.push(path); + return { ok: true, json: async () => response }; + }, + }); + const text = (node) => node.textContent + (node.children || []).map(text).join(''); + return { elements, listeners, requests, text }; +} + +const tick = () => new Promise((resolve) => setImmediate(resolve)); + +test('the figure loads when opened, for last year, rounded down to whole kWh', async () => { + const thisYear = Number(new Intl.DateTimeFormat('en', { year: 'numeric', timeZone: 'Europe/Stockholm' }).format(new Date())); + const months = Array.from({ length: 12 }, (_, i) => ({ month: i + 1, returned_wh: i === 0 ? 1999.9 : 0, hours: 744, complete_hours: 700 })); + const { elements, listeners, requests, text } = harness({ + year: thisYear - 1, returned_wh: 123_999.9, hours: 8760, complete_hours: 8700, months, problem: '', + batteries: [{ label: 'battery', sized: true, returned_wh: 123_999.9, capacity_wh: 10_240, efficiency: 0.8973 }], + }); + const select = elements.get('battery-return-year'); + assert.equal(select.value, String(thisYear - 1)); + assert.equal(select.children.length, 4); + assert.equal(requests.length, 0); + + elements.get('battery-return-details').open = true; + listeners.get('battery-return-details:toggle')(); + await tick(); + await tick(); + + assert.deepEqual(requests, ['/api/energy/battery-return?year=' + (thisYear - 1)]); + assert.equal(elements.get('battery-return-csv').href, '/api/energy/battery-return.csv?year=' + (thisYear - 1)); + const result = text(elements.get('battery-return-result')); + assert.match(result, /^123 kWh of grid electricity went back/); + assert.match(result, /8\D?700 of 8\D?760 hours\. Counted with a 10\D2 kWh battery that gave back 89 % of its charge\.$/); + const rows = elements.get('battery-return-rows').children; + assert.equal(rows.length, 12); + assert.equal(rows[0].children[1].textContent, '1 kWh'); + assert.equal(elements.get('battery-return-meta').textContent, '123 kWh in ' + (thisYear - 1)); +}); + +async function opened(response) { + const page = harness(response); + page.elements.get('battery-return-details').open = true; + page.listeners.get('battery-return-details:toggle')(); + await tick(); + await tick(); + return page; +} + +test('two batteries are counted each on its own', async () => { + const { elements, text } = await opened({ + year: 2025, returned_wh: 10_754.8, hours: 8760, complete_hours: 8760, months: [], problem: '', + batteries: [ + { label: 'battery', sized: true, returned_wh: 7169.9, capacity_wh: 10_000, efficiency: 0.4 }, + { label: 'garage', sized: true, returned_wh: 3584.9, capacity_wh: 5000, efficiency: 0.4 }, + ], + }); + assert.match(text(elements.get('battery-return-result')), /^10 kWh of grid electricity went back.* Counted for each battery on its own: 7 kWh from battery \(10 kWh, gave back 40 % of its charge\) and 3 kWh from garage \(5 kWh, gave back 40 % of its charge\)\.$/); +}); + +test('a year without a figure says why instead of showing zero', async () => { + for (const [problem, said, batteries, unread] of [ + ['battery_size', /^Set the battery's size in Settings to get a figure\./, [{ label: '', sized: false }], []], + ['battery_size', /^Set the battery size of the driver “garage” in Settings to get a figure\./, [{ label: 'battery', sized: true }, { label: 'garage', sized: false }], []], + ['battery_unread', /^Settings gives the driver “garage” a battery size, but FTW read no battery through it in 2025\./, [], ['garage']], + ]) { + const { elements, listeners, text } = harness({ year: 2025, returned_wh: 0, hours: 8760, complete_hours: 8000, problem, batteries, unread, months: [{ month: 1, returned_wh: 0, hours: 744, complete_hours: 0 }] }); + elements.get('battery-return-details').open = true; + listeners.get('battery-return-details:toggle')(); + await tick(); + await tick(); + assert.match(text(elements.get('battery-return-result')), said); + assert.equal(elements.get('battery-return-meta').textContent, 'No figure'); + assert.equal(elements.get('battery-return-rows').children.length, 0); + } +}); + +test('a year without complete readings says so instead of showing zero', async () => { + const { elements, listeners, text } = harness({ year: 2023, returned_wh: 0, hours: 8760, complete_hours: 0, months: [] }); + elements.get('battery-return-details').open = true; + listeners.get('battery-return-details:toggle')(); + await tick(); + await tick(); + assert.equal(text(elements.get('battery-return-result')), 'FTW has no complete readings for 2023.'); + assert.equal(elements.get('battery-return-meta').textContent, 'No readings'); +}); diff --git a/web/control-feedback.js b/web/control-feedback.js index 9fda1e31b..9d0d3c0d1 100644 --- a/web/control-feedback.js +++ b/web/control-feedback.js @@ -66,7 +66,7 @@ const REASON_TITLES = { core_limit: 'Limited by FTW', fuse_limit: 'Limited by the main fuse', fuse_cooldown: 'Waiting after a site power limit', import_limit: 'Limited by the hourly import limit', charger_limit: 'Limited by FTW’s charger setting', device_limit: 'Charger limit', offered_current_lower: 'Charger offers less current', - load_balancer_limit: 'Limited by the charger’s load balancing', + load_balancer_limit: 'Limited by the charger’s load balancing', charger_pilot_limit: 'Limited by the charger', one_phase: 'Charging on one phase', device_fault: 'Device fault', telemetry_stale: 'No fresh readings', readings_lost: 'Lost control', command_failed: 'Commands failing', command_unconfirmed: 'Command not confirmed', default_failed: 'Cannot hand back control', @@ -103,6 +103,7 @@ function sentence(row) { case 'offered_current_lower': return num(row.limit_a) && !(num(row.offered_a) && num(row.requested_a) && row.offered_a + 0.5 < row.requested_a) ? `The charger lets the car take ${amps(row.limit_a)}, below the ${amps(row.requested_a)} FTW asked for.` : `The charger offers ${amps(row.offered_a)}, below the ${amps(row.requested_a)} FTW asked for.`; + case 'charger_pilot_limit': return `The charger signals ${amps(row.pilot_a)} to the car. FTW asked for ${amps(row.requested_a)}; measured charging is ${controlPower(actual)}. The internal limiting cause is unknown.`; case 'load_balancer_limit': return `${row.make ? `${row.make} limits the charger to ${amps(row.limit_a)} (its load balancing)` : `The charger’s load balancing limits it to ${amps(row.limit_a)}`}. FTW asked for ${amps(row.requested_a)}; the car takes ${controlPower(actual)}.`; case 'one_phase': return `The car charges on ${row.charging_phase || 'one phase'} only${num(row.charging_phase_max_w) ? `, at most ${controlPower(row.charging_phase_max_w)}` : ''}. FTW plans with what it takes.`; @@ -142,6 +143,7 @@ function nextStep(row) { case 'site_meter_stale': case 'site_phase_currents_stale': return 'Check the grid meter’s connection.'; case 'device_limit': return 'Raise the limit in the charger’s app if you want faster charging.'; case 'offered_current_lower': return 'Check the charger’s load balancing and limits.'; + case 'charger_pilot_limit': return 'Check Available energy in the charger’s app for the active limit.'; case 'load_balancer_limit': return 'If this happens often, compare the charger’s load balancing settings with FTW’s main fuse setting.'; case 'observe_only': return 'Turn on control in the device settings if you want FTW to steer it.'; default: return ''; @@ -231,7 +233,9 @@ export function controlNumbers(row, live = true) { if (num(row.battery_soc)) values.push(['Battery charge', percent(row.battery_soc)]); if (num(row.tolerance_w)) values.push(['Response tolerance', `±${controlPower(row.tolerance_w)}`]); if (num(row.requested_a)) values.push(['Requested current', amps(row.requested_a)]); - if (num(row.offered_a)) values.push(['Charger offer', amps(row.offered_a)]); + if (num(row.pilot_a)) values.push(['Actual pilot current', amps(row.pilot_a)]); + if (num(row.dynamic_limit_a)) values.push(['Dynamic current setting', amps(row.dynamic_limit_a)]); + else if (num(row.offered_a)) values.push(['Reported current setting', amps(row.offered_a)]); if (num(row.device_limit_a)) values.push(['Charger limit', amps(row.device_limit_a)]); if (num(row.limit_a)) values.push([row.limited_by === 'load_balancer' ? 'Load balancing allows' : 'Charger lets the car take', amps(row.limit_a)]); const e = row.site_evidence; diff --git a/web/control-feedback.test.mjs b/web/control-feedback.test.mjs index 16cb4db5f..08df34995 100644 --- a/web/control-feedback.test.mjs +++ b/web/control-feedback.test.mjs @@ -69,7 +69,7 @@ test('expected limits stay calm and explain themselves', () => { assert.equal(balanced.tone, 'neutral'); assert.equal(controlStatus(row(lb)).text, 'The charger’s load balancing limits it to 13 A. FTW asked for 15 A; the car takes 8.8 kW.'); assert.deepEqual(controlNumbers(row(lb)).filter(([label]) => /current|offer|limit|allows/i.test(label)), [ - ['Requested current', '15 A'], ['Charger offer', '15 A'], ['Charger limit', '16 A'], ['Load balancing allows', '13 A']]); + ['Requested current', '15 A'], ['Reported current setting', '15 A'], ['Charger limit', '16 A'], ['Load balancing allows', '13 A']]); // A lower current with no cause shown says what the charger does, no more. const unexplained = {...lb, reason: 'offered_current_lower', severity: 'warning', limited_by: undefined}; assert.equal(controlStatus(row(unexplained)).text, 'The charger lets the car take 13 A, below the 15 A FTW asked for.'); @@ -181,3 +181,16 @@ test('limited positive EV offer does not explain prolonged zero delivery', () => assert.match(hour.text,/this hour’s grid import/); assert.doesNotMatch(hour.text,/fuse/); }); + +test('a confirmed lower pilot is an explained restriction with separate current values', () => { + const r = row({kind:'ev',status:'limited',reason:'charger_pilot_limit',severity:'info', + requested_a:15,offered_a:15,dynamic_limit_a:15,pilot_a:13,actual_w:8850,sent_w:10350}); + const words=controlStatus(r); + assert.match(words.title,/Limited by the charger/); + assert.match(words.text,/signals 13 A/); + assert.match(words.text,/internal limiting cause is unknown/); + const numbers=controlNumbers(r); + assert.ok(numbers.some(([label,value])=>label==='Actual pilot current'&&value==='13 A')); + assert.ok(numbers.some(([label,value])=>label==='Dynamic current setting'&&value==='15 A')); + assert.ok(!numbers.some(([label])=>label==='Charger offer')); +}); diff --git a/web/index.html b/web/index.html index 172118e91..abe9b2363 100644 --- a/web/index.html +++ b/web/index.html @@ -30,7 +30,7 @@ - + - - + + diff --git a/web/setup.js b/web/setup.js index c5289b630..11fd65e55 100644 --- a/web/setup.js +++ b/web/setup.js @@ -6,13 +6,27 @@ var TOTAL_STEPS = 8; var currentStep = 1; + // A new box has no /api/auth/status until it first starts, so a missing + // one means this wizard claims it and asks for the house password. Run + // again from Settings, the box keeps the password it has. + var firstRun = false; + fetch('/api/auth/status', {cache: 'no-store'}).then(function (r) { + firstRun = r.status === 404; + // The answer can arrive after the last step is already on screen. + var step = document.getElementById('house-password-step'); + if (step && currentStep === TOTAL_STEPS) step.hidden = !asksHousePassword(); + }).catch(function () {}); + + function asksHousePassword() { + return firstRun && !window.FTWHomeAssistant; + } window.startAssistantSetup = async function () { var btn = document.getElementById('assistant-start'); var note = document.getElementById('assistant-status'); var password = document.getElementById('assistant-password').value; - if (password.length < 10 || password !== document.getElementById('assistant-password-confirm').value) { - note.textContent = 'Use at least 10 characters and enter the same house password twice.'; + if (!password || password !== document.getElementById('assistant-password-confirm').value) { + note.textContent = 'Enter the same house password twice.'; return; } btn.disabled = true; @@ -875,6 +889,8 @@ esc(document.getElementById('fuse-voltage').value) + ' V per phase'; } html += ' · ' + esc(zone); + var clock = browserTimeZone(); + if (clock) html += ' · clock ' + esc(clock); html += ''; // Devices @@ -926,6 +942,7 @@ } document.getElementById('review-content').innerHTML = html; + document.getElementById('house-password-step').hidden = !asksHousePassword(); document.getElementById('save-error').style.display = 'none'; document.getElementById('save-actions').style.display = 'flex'; document.getElementById('save-success').style.display = 'none'; @@ -940,11 +957,23 @@ // hardware. The backend accepts this and runs with a no-op // control loop (no site meter to balance against). var btn = document.getElementById('save-btn'); + var errEl = document.getElementById('save-error'); + var cfg = buildConfig(); + var asked = asksHousePassword(); + if (asked) { + var password = document.getElementById('house-password').value; + if (!password || password !== document.getElementById('house-password-confirm').value) { + errEl.className = 'error-msg'; + errEl.textContent = 'Choose a house password, and type the same one twice.'; + errEl.style.display = 'block'; + document.getElementById('house-password').focus(); + return; + } + cfg.house_password = password; + } btn.disabled = true; btn.textContent = 'Saving...'; - var cfg = buildConfig(); - fetch('/api/config', { method: 'POST', headers: { 'Content-Type': 'application/json' }, @@ -955,6 +984,12 @@ return r.json(); }) .then(function () { + if (asked) { + document.getElementById('house-password').value = ''; + document.getElementById('house-password-confirm').value = ''; + document.getElementById('save-success-password').hidden = false; + } + document.getElementById('house-password-step').hidden = true; document.getElementById('save-actions').style.display = 'none'; document.getElementById('save-success').style.display = 'block'; setTimeout(function () { window.location.href = '/'; }, 3000); @@ -962,19 +997,31 @@ .catch(function (err) { btn.disabled = false; btn.textContent = 'Start FTW'; - var errEl = document.getElementById('save-error'); errEl.className = 'error-msg'; errEl.textContent = err.message; errEl.style.display = 'block'; }); }; + // The household clock car goals and daily history read. A Docker box runs + // on UTC, so the zone comes from the browser that sets the home up. A + // browser on UTC says nothing about the home; the box then keeps its own. + function browserTimeZone() { + try { + var tz = Intl.DateTimeFormat().resolvedOptions().timeZone || ''; + return /^(Etc\/)?(UTC|UCT|GMT|Universal|Zulu)$/.test(tz) ? '' : tz; + } catch (e) { + return ''; + } + } + function buildConfig() { var zone = document.getElementById('price-zone').value; var cfg = { site: { name: document.getElementById('site-name').value.trim() || 'My Home', + time_zone: browserTimeZone() || undefined, control_interval_s: 5, grid_target_w: 0, grid_tolerance_w: 42, @@ -1138,6 +1185,16 @@ return n; } + // A short house password is the owner's choice: a word, not a refusal. + [['assistant-password', 'assistant-password-hint'], ['house-password', 'house-password-hint']].forEach(function (ids) { + var input = document.getElementById(ids[0]); + var hint = document.getElementById(ids[1]); + input.addEventListener('input', function () { + var n = Array.from(input.value).length; + hint.textContent = n > 0 && n < 8 ? 'Short passwords are easy to guess. You can still use this one.' : ''; + }); + }); + document.getElementById('drv-ip').addEventListener('input', updateHostHint); document.getElementById('site-connection').addEventListener('change', chooseConnection); document.getElementById('price-country').addEventListener('change', connectionHelp); diff --git a/web/setup.test.mjs b/web/setup.test.mjs index 29e55f433..e5d967f15 100644 --- a/web/setup.test.mjs +++ b/web/setup.test.mjs @@ -238,3 +238,38 @@ describe("setup wizard — mDNS-first device addressing", () => { "the hint must track manual edits of the host field"); }); }); + +describe("setup wizard — the house password on a new box", () => { + it("asks for it on the last step and sends it with the settings", () => { + assert.match(HTML, /id=["']house-password-step["'][^>]*hidden/, + "the step starts hidden until the wizard knows it claims a new box"); + assert.match(HTML, /id=["']house-password["']/); + assert.match(HTML, /id=["']house-password-confirm["']/); + assert.match(JS, /cfg\.house_password\s*=\s*password/, + "the wizard sends the password with the settings it saves"); + }); + + it("asks only on a new box, never through Home Assistant", () => { + assert.match(JS, /firstRun\s*=\s*r\.status\s*===\s*404/, + "a box that already runs answers /api/auth/status and keeps its password"); + assert.match(JS, /firstRun\s*&&\s*!window\.FTWHomeAssistant/); + }); + + it("refuses only an empty or mistyped password; a short one gets a warning", () => { + assert.match(JS, /!password\s*\|\|\s*password\s*!==\s*document\.getElementById\(['"]house-password-confirm['"]\)\.value/); + assert.doesNotMatch(JS + HTML, /at least 10|minlength="10"|length\s*<\s*10/); + assert.match(JS, /Short passwords are easy to guess/); + }); +}); + +describe("setup wizard — household clock", () => { + it("stores the browser's zone as site.time_zone", () => { + assert.match(JS, /time_zone:\s*browserTimeZone\(\)/, + "a Docker box runs on UTC; the home's zone comes from the browser that sets it up"); + assert.match(JS, /resolvedOptions\(\)\.timeZone/); + }); + + it("names the clock on the review screen", () => { + assert.match(JS, /' · clock ' \+ esc\(clock\)/); + }); +}); diff --git a/web/style.css b/web/style.css index 549bc6d0f..f311eafab 100644 --- a/web/style.css +++ b/web/style.css @@ -267,6 +267,7 @@ header h1 { } .energy-history-pager button:disabled { opacity: 0.4; cursor: default; } .history-disclosure .energy-history-note { margin: 0.75rem 0 0; text-align: center; } +.history-disclosure .energy-history-note a { color: var(--text); } #view-energy > .history-row { margin-top: 0; } #view-energy > .history-row > .history-disclosure { width: 100%; } .history-numbers-disclosure .history-disclosure-body { padding: 0; } @@ -1873,6 +1874,21 @@ footer { } .access-blocked p { margin: 0 0 8px; } .access-blocked button { margin-right: 8px; } +/* The house password, asked for right where the owner needs it: a step, + not a warning. */ +.access-password { + margin: 10px 0; + padding: 10px 12px; + border: 1px solid var(--line); + border-radius: 6px; + font-size: 0.85rem; + line-height: 1.45; +} +.access-password p { margin: 0 0 8px; } +.modal-body .access-password input[type="password"] { display: block; width: 100%; max-width: 32ch; } +.access-password .btn-send { display: block; margin-top: 12px; } +.access-password .access-note { margin: 6px 0; } +.access-password a { color: var(--accent); } .access-code-row { display: flex; gap: 8px;