You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Native recordings get no field labels from the accessibility tree (only the browser's do) #1868
At record time a typed field's label is kept as evidence (field_label on the event and on the compiled step; see packages/flow/DESIGN.md, "Record-time label capture"). Only PlaywrightBackend implements focused_field_label (openadapt_flow.backend.FieldLabelBackend). Windows UIA, macOS AX and Linux AT-SPI expose the focused field's name and labelled-by relation, but their backends don't implement the seam, so native recordings fall back to the nearest OCR line at compile time.
Close: implement focused_field_label on WindowsBackend (UIA LabeledBy, then Name), MacOSBackend (AXTitleUIElement, then AXTitle/AXDescription) and LinuxBackend (AT-SPI LABELLED_BY relation, then name). Read from the focused element only, never by position. Return None when there's no label, so the OCR fallback still applies.
Test: real apps on the macOS lab VM (Chart.swift note field) and the Linux AT-SPI CI job (chart_gtk.py), asserting that the compiled type step's field_label comes from the tree.
Opened #1956 (Part of #1868): native focused_field_label on Mac/Windows/Linux with unit tests (7 passed; control fails on base).
Still needed to Close: real Chart.swift note field on the macOS lab VM and chart_gtk.py on the Linux AT-SPI CI job proving compiled field_label comes from the tree. Lab 10.0.0.53 was unreachable; claimed removed so a lab-capable pass can finish.
Activity
abrichr commentedon Nov 5, 2026
Claimed by Grok
Native backends expose focused_field_label from the a11y tree
abrichr commentedon Nov 5, 2026
From: Grok
Opened #1956 (Part of #1868): native focused_field_label on Mac/Windows/Linux with unit tests (7 passed; control fails on base).
Still needed to Close: real Chart.swift note field on the macOS lab VM and chart_gtk.py on the Linux AT-SPI CI job proving compiled field_label comes from the tree. Lab 10.0.0.53 was unreachable; claimed removed so a lab-capable pass can finish.