Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions docs/decisions/0034-rig-pyte-orphaned-stub-tolerance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# 0034 — The rig's pyte screen tolerates orphaned wide-char stubs

- **Status:** Accepted
- **Date:** 2026-11-02
- **Issue:** [#160](https://github.com/fluxvale/dashfloat/issues/160)

## Context

Issue #160 (surfaced by the #143 lane's full-menu smoke) reported the three
image scenarios — `attach` (still + gif), `drop-image` (gif) — dying inside
pyte 0.8.2 under Python 3.14: `screens.py`'s `display` property raising
`IndexError: string index out of range` at `wcwidth(char[0])`.

The mechanism, established on the lane: pyte renders every wide (2-cell)
glyph as a lead cell plus a `data=""` stub cell. When the app's redraw
reaches the terminal as two pty chunks — the first overwriting a wide
glyph's lead cell with a narrow one, the second rewriting the stub — the
stub is **orphaned** between the chunks, and `display`'s `char[0]`
IndexErrors on the empty string. The rig's poller (`wait_for` → `text()` →
`display`, deliberately outside the feed lock) can always land in that
window. The image scenarios are the locus because their 📎 rows are the
only content that is *both* wide *and* dynamically rewritten next to narrow
text; the CJK scenarios type wide text append-only, so they never orphan.

Investigation on the lane disproved the Python-3.14 attribution:

- The crash is **not** deterministic on 3.14 — the same three scenarios
passed three consecutive runs on Python 3.14.8 + pyte 0.8.2 in the
unmodified worktree. The #143 lane's reds were the race landing, not a
version gate.
- pyte's only Python-version-sensitive input is `unicodedata`
(combining classification, NFC) — and the app's byte stream contains no
combining marks, so the interpreter version cannot change what pyte
parses or renders. All widths come from the `wcwidth` package's fixed
tables, which are pure Python.
- The app's own width classifier (`DashFloat.TUI.Wrap`) agrees with
wcwidth on every glyph the scenarios emit (📎 = 2, CJK = 2, ❯ = 1), so
there is no layout disagreement to desync the stream either.

## Decision

1. The rig drives a **`RigScreen(pyte.Screen)`** whose `display` property
renders an orphaned stub as a blank cell — exactly what a real terminal
shows for a wide glyph's leftover continuation cell — instead of
indexing `char[0]`. Every other cell renders through pyte 0.8.2's own
semantics, byte-identically (verified against stock `pyte.Screen` on
narrow, wide, and end-of-row-wrap content). Both screen construction
sites — the live screen and the GIF replay screen — use it.
2. The rig venv's setup line in `capture.py`'s docstring pins its packages
exactly (`pyte==0.8.2 wcwidth==0.9.4 pillow==13.0.0`): the rig's
rendering is only reproducible across capture lanes if every lane runs
the same width tables and image encoder.

## Alternatives considered

- **Bump pyte to a release supporting Python 3.14** (the issue's first
candidate) — no such release exists: 0.8.2 (2024) is the latest, and
the fix exists only on unreleased master (whose rendering semantics
did change — grapheme clusters, per-cluster `wcswidth`; untested
against this rig's asserted frames). A git-commit pin would also make
the venv setup fragile.
- **Pin the rig venv to Python 3.13** (the issue's second candidate) —
treats a symptom the interpreter does not cause. The race exists on
every Python version; the pin would add rig tooling (a version manager
constraint or a second interpreter) and leave the crash latent for the
next loaded overnight run.
- **Upstream first** — correct long-term, but the rig must be whole now;
the shim is ~12 lines. When a pyte release finally lands, removing it
is a deliberate diff, not a free deletion: master survives empty
stubs but yields nothing for them (the row renders one char shorter),
where `RigScreen` preserves the column with a blank.

## Consequences

- The rig no longer crashes on pyte's own transient intermediate state,
on any Python version; the three image scenarios are green and the
full menu (41 invocations: every argparse scenario in primary mode plus
the secondary gif/probe modes) stays green.
- A still or GIF frame captured mid-redraw may show a one-cell blank
where a wide glyph's lead was just overwritten — honest: that is the
instant's real terminal state, and later frames carry the settled row.
- Rebuilding the venv without the pins (the old unpinned line) may
silently change rendering across lanes; the docstring setup is the
documented reproducible path.
1 change: 1 addition & 0 deletions docs/decisions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,4 @@ their replacement.
| [0031](0031-actionlint-inside-the-ci-gate.md) | actionlint inside the `mix ci` gate: sha256-pinned binaries fetched by the gate itself, no PATH lookup, no skip | Accepted |
| [0032](0032-installer-refusal-contract.md) | The installer's glibc-floor refusal contract: specific message, exit 1, nothing installed — asserted by the smokes, pinned by mix ci | Accepted |
| [0033](0033-idle-time-auto-compaction.md) | Idle-time auto-compaction: a cache-TTL-derived idle timer over the shared eviction pass | Proposed |
| [0034](0034-rig-pyte-orphaned-stub-tolerance.md) | The capture rig's pyte screen renders orphaned wide-char stubs as blanks (race tolerance, not a Python pin); the rig venv's setup pins its package versions | Accepted |
48 changes: 44 additions & 4 deletions scripts/capture.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,12 @@
Everything is pty-driven end to end: no widget dumps, no test-mode
buffers. The captures are ground truth for visual and motion claims.

One-time setup (any venv; kept outside the repo):
One-time setup (any venv; kept outside the repo). Versions are pinned
exactly — the rig's rendering is only reproducible if every capture
lane runs the same pyte/wcwidth tables and Pillow (ADR 0034):

python3 -m venv /tmp/opencode/capvenv
/tmp/opencode/capvenv/bin/pip install pyte pillow
/tmp/opencode/capvenv/bin/pip install pyte==0.8.2 wcwidth==0.9.4 pillow==13.0.0

Usage (from a worktree root, against the CURRENT tree's code):

Expand Down Expand Up @@ -247,6 +249,7 @@
import pyte
import re
import shlex
from wcwidth import wcwidth
from PIL import Image, ImageDraw, ImageFont

COLS, ROWS = 80, 24
Expand Down Expand Up @@ -851,6 +854,43 @@ def _usage_and_done(self, req, reasoning_deltas):
self.wfile.flush()


class RigScreen(pyte.Screen):
"""The rig's screen: pyte 0.8.2's `display` made race-tolerant.

pyte writes a `data=""` stub cell after every wide (2-cell) glyph.
When one pty chunk overwrites a wide glyph's lead cell with a
narrow one, the stub is orphaned until the next chunk rewrites it —
and `display`'s `wcwidth(char[0])` IndexErrors on the empty string
if the poller reads in that window (issue #160: the attach/drop
image scenarios' 📎 rows are the only dynamically rewritten wide
content, so the race clustered there). The orphan renders as the
blank a real terminal shows for the leftover continuation cell.
Python-version-independent — pyte's only 3.14-sensitive input
(unicodedata) never fires on this app's stream, which contains no
combining marks; widths come from the pinned wcwidth tables.
See ADR 0034."""

@property
def display(self) -> list[str]:
rows = []
for y in range(self.lines):
line = self.buffer[y]
cells = []
wide = False
for x in range(self.columns):
if wide: # the wide glyph's stub — skipped
wide = False
continue
data = line[x].data
if not data: # orphaned stub — a blank cell
cells.append(" ")
continue
wide = wcwidth(data[0]) == 2
cells.append(data)
rows.append("".join(cells))
return rows


class App:
"""The real TUI under a real pty, plus the capture operations."""

Expand All @@ -870,7 +910,7 @@ def __init__(self, mix_args=("run", "--no-halt"), no_ghost=False,
# (a wizard boot means the scratch config was missed) stands down.
self._wizard = wizard
self.sink = None # callable(bytes) set during read loop
self.screen = pyte.Screen(cols, rows)
self.screen = RigScreen(cols, rows)
self.stream = pyte.ByteStream(self.screen)
# The FULL byte history from boot (t, chunk): a GIF replay must
# start at boot so static chrome (borders, labels, footer, log)
Expand Down Expand Up @@ -1220,7 +1260,7 @@ def gif(self, path: str, fps=10, crop_rows=12, min_span=0.0):
cell_w, cell_h, _ = metrics
frames = []
step = 1000 / fps
replay = pyte.Screen(self.cols, self.rows)
replay = RigScreen(self.cols, self.rows)
# Freeze the reader's feed FIRST, atomically with the stream swap
# (see _frozen/_feed_lock): the replay below drives this stream in
# order; any byte the reader thread feeds concurrently reorders the
Expand Down
Loading