docs(atp): trim the test plan to 24 Docker-only scenarios

atp/README.md

@@ -11,7 +11,8 @@ The workbook is built from `ATP_template.xlsx` and keeps its two tabs:
 
 - **Info & Access** — download links, the Docker environment, the test data set
   with its verified expected results, and the notes a tester needs.
-- **Test Plan** — 56 scenarios (`F1`…`F56`, plus `F3b`; `F24` retired) with Feature ID, name, category,
+- **Test Plan** — 24 scenarios (`F1`…`F24`), all run in the Docker environment so the plan is
+  reproducible on any OS with Feature ID, name, category,
   objective, prerequisites, numbered steps, expected result and development
   status. The **Feedback** column is intentionally empty: the tester fills it.
 
@@ -24,7 +25,8 @@ anything.
 ## Regenerating the workbook
 
 The scenarios live in `scripts/gen_atp.py` next to this README; re-run it with
-`python gen_atp.py` after editing, then rebuild the archive:
+`python gen_atp.py` after editing (pass the path of `ATP_template.xlsx` to
+rebuild from the blank template; by default it rewrites the existing workbook), then rebuild the archive:
 
 ```sh
 cd atp && python scripts/make_appendices_zip.py
@@ -42,25 +44,22 @@ honours the mode at all.
 Shipped in v5.1.0:
 
 1. **macOS artifact.** ~~The `macos-latest` job is commented out.~~ v5.1.0
-   publishes an unsigned `.dmg` and `.zip` for Apple Silicon. `F3` is now a
-   native install with the Gatekeeper steps spelled out.
+   publishes an unsigned `.dmg` and `.zip` for Apple Silicon. Native installs are not
+   part of the plan: every scenario runs in Docker.
 2. **Ctrl-only shortcuts.** ~~`e.metaKey` was never tested.~~ Shortcuts follow
-   the platform, and `F53` covers Cmd on macOS. Tab navigation stays on Ctrl
+   the platform (Cmd on macOS). Tab navigation stays on Ctrl
    everywhere, because Cmd+Tab is the macOS application switcher.
-3. **Format Document was a no-op** for C/C++ — dropped, and `F24` is retired
-   (the numbering keeps the gap so every cross-reference stays valid).
+3. **Format Document was a no-op** for C/C++ — dropped, along with its scenario.
 
 Still open:
 
 4. **No Intel macOS artifact.** The x64 job targeted `macos-13`, which GitHub
    has retired; a job pinned to a retired label queues forever instead of
    failing, so v5.1.0 shipped without it. The matrix now uses `macos-15-intel`,
-   but that fix is not in the v5.1.0 tag. Until an Intel build ships, `F3b`
-   routes Intel Macs through Docker.
+   but that fix is not in the v5.1.0 tag. Intel Macs use Docker like everyone else.
 5. **The analyser binary is Linux-only.** `bin/ctrace` is an x86-64 ELF, so on
-   macOS the app runs but cannot analyse — `F54` pins the message it shows
-   instead. A Darwin build of ctrace is the real fix; until then, run `F28`
-   to `F38` on Linux.
+   macOS the app runs but cannot analyse. A Darwin build of ctrace is the real
+   fix; the plan avoids the problem by running in Docker.
 6. **Third-party analysers are unreachable.** `--invoke=cppcheck` resolves to
    the hard-coded path `/opt/homebrew/bin/cppcheck` and silently produces no
    output. Only `ctrace_stack_analyzer` is exercised by the plan.
@@ -71,4 +70,4 @@ Still open:
    `docker exec <container> xwininfo -root -tree -display :99`. Re-run that
    after any change under `appendices/docker/`.
 8. **Windows is not in the plan.** The WSL detection and setup flow
-   (`main.js`) has no scenario, because the delivery targets Linux and macOS.
+   (`main.js`) has no scenario, and the plan has no OS-specific scenario at all.

atp/appendices/README.md

@@ -7,7 +7,7 @@ release **v5.1.0**.
 ```
 appendices/
 ├── README.md            ← this file
-├── docker/              ← reproducible Linux environment (also used on Intel Macs)
+├── docker/              ← reproducible test environment, runs on any OS with Docker
 │   ├── Dockerfile
 │   ├── entrypoint.sh
 │   ├── build.sh
@@ -29,17 +29,14 @@ appendices/
 | Item | Link |
 |---|---|
 | Release under test (v5.1.0) | https://github.com/CoreTrace/coretrace-gui/releases/tag/v5.1.0 |
-| Linux AppImage | https://github.com/CoreTrace/coretrace-gui/releases/download/v5.1.0/CtraceGUI-5.1.0.AppImage |
-| Windows installer | https://github.com/CoreTrace/coretrace-gui/releases/download/v5.1.0/CtraceGUI-Setup-5.1.0.exe |
-| macOS, Apple Silicon (DMG) | https://github.com/CoreTrace/coretrace-gui/releases/download/v5.1.0/CtraceGUI-5.1.0-arm64.dmg |
-| macOS, Apple Silicon (ZIP) | https://github.com/CoreTrace/coretrace-gui/releases/download/v5.1.0/CtraceGUI-5.1.0-arm64-mac.zip |
-| macOS, Intel | not published in v5.1.0 — use the Docker setup |
 | Source code | https://github.com/CoreTrace/coretrace-gui |
 | API documentation | https://coretrace.github.io/coretrace-gui/ |
 
-## Setup A — Linux, through Docker (recommended, and the reference environment)
+## Setup — Docker (Linux, macOS or Windows)
 
-Requires Docker and about 3 GB of free disk space. The image downloads the
+The whole test plan runs in this environment, whatever your operating system.
+Requires Docker and about 3 GB of free disk space (on Windows, run the scripts
+from WSL or Git Bash; on a Mac, give Docker Desktop at least 6 GB of RAM). The image downloads the
 AppImage, extracts it, and serves the running application over noVNC, so the
 only thing you need locally is a browser.
 
@@ -72,100 +69,19 @@ hold, the application is running and the problem is the browser view: open
 <http://localhost:6080/vnc_lite.html>, or hard-reload the page (Ctrl+Shift+R) in
 case an older non-connecting version of it was cached.
 
-## Setup B — Linux, native AppImage
+## clang 20
 
-**Install clang 20 first.** The analyser compiles the source under test to LLVM
-IR before analysing it, and needs a toolchain matching the LLVM it links
-(`libclang-cpp.so.20.1`). Ubuntu 24.04 ships clang 18, which is not enough:
-it fails with `'stddef.h' file not found`.
-
-This matters more than it looks. **Without clang 20 the analyser does not
-report an error — it returns "0 diagnostics" for every file.** `F29` (clean
-file, no diagnostic) would then appear to pass while nothing is being analysed
-at all, and `F28`, `F30`, `F32` and `F33` would fail for a reason no message
-explains. Scenario **F27b** exists to catch exactly that, and must be run
-before the analysis scenarios.
-
-```sh
-wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key \
-  | sudo gpg --dearmor -o /usr/share/keyrings/llvm.gpg
-echo "deb [signed-by=/usr/share/keyrings/llvm.gpg] http://apt.llvm.org/noble/ llvm-toolchain-noble-20 main" \
-  | sudo tee /etc/apt/sources.list.d/llvm20.list
-sudo apt-get update && sudo apt-get install -y clang-20
-sudo ln -sf /usr/bin/clang-20 /usr/bin/clang
-clang --version      # must report 20.x
-```
-
-Then run the application:
-
-```sh
-wget https://github.com/CoreTrace/coretrace-gui/releases/download/v5.1.0/CtraceGUI-5.1.0.AppImage
-chmod +x CtraceGUI-5.1.0.AppImage
-./CtraceGUI-5.1.0.AppImage
-```
-
-If your distribution has no FUSE 2 runtime, run
-`./CtraceGUI-5.1.0.AppImage --appimage-extract` and start `squashfs-root/AppRun`
-instead.
-
-> The Docker environment in Setup A already contains clang 20, which is the
-> main reason to prefer it: the analysis scenarios are reproducible there
-> without touching your machine's toolchain.
-
-## Setup C — macOS, Apple Silicon (native)
-
-Download
-[CtraceGUI-5.1.0-arm64.dmg](https://github.com/CoreTrace/coretrace-gui/releases/download/v5.1.0/CtraceGUI-5.1.0-arm64.dmg),
-open it, and drag **CtraceGUI** to Applications.
-
-The build is **unsigned** — signing needs a paid Apple Developer account — so
-macOS quarantines it and refuses the first launch. That refusal is expected and
-is part of scenario **F3**; it is not a defect. Get past it one of two ways,
-depending on what macOS tells you:
-
-- *"cannot be opened because the developer cannot be verified"* →
-  **right-click** (or Control-click) the app in Applications and choose
-  **Open**, then click **Open** in the dialog. Double-clicking will not work:
-  only the Open menu item offers the override.
-- *"is damaged and can't be opened"* → that dialog has no Open button, so clear
-  the quarantine flag instead:
-
-  ```sh
-  xattr -cr /Applications/CtraceGUI.app
-  ```
-
-Either way, macOS remembers the decision and every later launch is a normal
-double-click.
-
-> On Ventura and later the app may instead appear under **System Settings →
-> Privacy & Security**; click **Open Anyway** there and authenticate.
-
-**Analysis does not run on macOS.** The bundled `ctrace` is a Linux binary, so
-**Run Analysis** reports that explicitly (scenario **F54**) instead of failing
-silently. Everything else — editor, explorer, search, terminal, assistant,
-session restore — works. Run the analysis scenarios **F28 to F38** on Linux.
-
-## Setup D — macOS, Intel
-
-v5.1.0 publishes no x64 artifact, so Intel Macs go through the Docker
-environment (scenario **F3b**):
-
-```sh
-cd appendices/docker
-./build.sh
-./run.sh
-```
-
-then open <http://localhost:6080/>. Allocate at least 6 GB of RAM to
-Docker Desktop (Settings → Resources).
+The analyser compiles the source under test to LLVM IR before analysing it and
+needs clang 20. The Docker image already contains it. Without it the analyser
+does not report an error — it returns "0 diagnostics" for every file — so
+scenario **F9** checks the toolchain before the analysis scenarios.
 
 ## Reporting an anomaly
 
 Fill the **Feedback** column of the test plan with, in this order:
 
 1. `PASS` or `FAIL`.
-2. Setup used (`Docker/Linux`, `Native/Linux`, `Native/macOS-AppleSilicon`,
-   `Docker/macOS-Intel`).
+2. Host operating system (`Linux`, `macOS`, `Windows`).
 3. For a `FAIL`: what you observed instead of the expected result, and the
    severity — **blocking** (the feature is unusable or the app crashes),
    **major** (wrong result produced), **minor** (cosmetic or wording).

atp/appendices/datasets/MANIFEST.md

@@ -29,8 +29,8 @@ toolchain that must match the LLVM it links (`libclang-cpp.so.20.1`). With an
 older clang, or none at all, **every file below reports 0 diagnostics** and no
 error is printed — `DS02` would look correct while nothing is being analysed.
 
-Verify with scenario **F27b** before running any analysis scenario. Setup A
-(Docker) already has clang 20; Setup B (native AppImage) needs it installed.
+Verify with scenario **F9** before running any analysis scenario. The Docker
+image already has clang 20.
 
 Both the analyser bundled in v5.1.0 and the v0.74.0 build that the in-app
 backend updater installs produce exactly the results above once clang 20 is

atp/appendices/docker/Dockerfile

@@ -18,7 +18,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
       libdrm2 libgbm1 libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 \
       libxrandr2 libxshmfence1 libnss3 libnspr4 libxss1 libxtst6 libnotify4 \
       libasound2t64 libsecret-1-0 libglib2.0-0t64 \
-      fonts-dejavu-core xdg-utils gnupg wget \
+      fonts-dejavu-core xdg-utils gnupg wget git \
     && rm -rf /var/lib/apt/lists/*
 
 # clang 20 is a hard runtime requirement of the analyser: it compiles the source
@@ -43,9 +43,13 @@ RUN wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key \
 # Version under test. Override with --build-arg to test another release.
 ARG CTRACE_GUI_VERSION=5.1.0
 
+# The AppImage runtime stores the magic "AI\x02" in bytes 8-10 of its ELF header.
+# Under amd64 emulation (Apple Silicon) the kernel then no longer recognises the
+# file as x86-64 and exec fails with "Exec format error", so blank those bytes.
 RUN curl -fL --retry 3 -o /tmp/CtraceGUI.AppImage \
       "https://github.com/CoreTrace/coretrace-gui/releases/download/v${CTRACE_GUI_VERSION}/CtraceGUI-${CTRACE_GUI_VERSION}.AppImage" \
     && chmod +x /tmp/CtraceGUI.AppImage \
+    && printf '\0\0\0' | dd of=/tmp/CtraceGUI.AppImage bs=1 seek=8 conv=notrunc status=none \
     && cd /opt && /tmp/CtraceGUI.AppImage --appimage-extract > /dev/null \
     && mv squashfs-root ctrace-gui \
     && rm /tmp/CtraceGUI.AppImage \
@@ -55,7 +59,9 @@ RUN curl -fL --retry 3 -o /tmp/CtraceGUI.AppImage \
 # /tmp/.X11-unix is created here because Xvfb cannot create it as a non-root user
 # and prints an alarming "ERROR" when it is missing, even though it goes on to
 # start normally — a tester should not have to judge whether that is a defect.
-RUN useradd -m -s /bin/bash tester && mkdir -p /home/tester/workspace \
+# .config exists in the image so the named volume run.sh mounts there inherits
+# tester ownership; it keeps the session across container restarts.
+RUN useradd -m -s /bin/bash tester && mkdir -p /home/tester/workspace /home/tester/.config \
     && chown -R tester:tester /home/tester \
     && mkdir -p /tmp/.X11-unix && chmod 1777 /tmp/.X11-unix
 

atp/appendices/docker/run.sh

@@ -13,5 +13,6 @@ docker run --rm -it $PLATFORM_FLAG \
   --name coretrace-gui-atp \
   -p 6080:6080 \
   -v "$DATASETS":/home/tester/workspace \
+  -v coretrace-gui-atp-config:/home/tester/.config \
   --shm-size=1g \
   coretrace-gui-atp