Skip to content

docs(atp): trim the test plan to 24 Docker-only scenarios - #99

Open
cedricroulof wants to merge 1 commit into
mainfrom
docs/atp-trim
Open

cedricroulof wants to merge 1 commit into
mainfrom
docs/atp-trim

Conversation

@cedricroulof

Copy link
Copy Markdown
Collaborator

What changed

  • Test plan cut from 57 to 24 scenarios. Trivial ones (rename, word wrap, cursor indicator, window controls, etc.) are gone; the plan keeps launch, core editing, robustness, analysis, terminal, AI assistant, session restore and git decorations.
  • No OS-specific scenario. The native AppImage and macOS scenarios are removed, so every scenario runs in the Docker environment and the plan is reproducible on Linux, macOS and Windows.
  • Renumbered F1 to F24, with cross-references updated in the workbook, both READMEs and the dataset manifest.
  • Getting started section added at the top of the Info & Access tab: get the appendices, start Docker, open the Test Plan tab.
  • Docker environment fixes:
    • the AppImage now extracts under amd64 emulation (the build failed with Exec format error on Apple Silicon);
    • git is installed, which the git decorations scenario needs;
    • the app config is kept in a named volume so session restore survives a container restart.
  • gen_atp.py no longer depends on hard-coded Windows paths.

Why

The plan was too long to demonstrate and part of it could only be run on a Mac.

Known gaps

Running the plan against v5.1.0 in Docker showed that some scenarios do not pass as written. Their expected results are left unchanged so testers report them:

  • F4: the explorer has no right-click "New file.." entry (File > New File then Save works).
  • F9 to F12, F14: every analysis shows an extra "gui execution failed" error entry.
  • F16: an unsupported file reports "No Issues Found" instead of a failure.
  • F17: the arguments field has no visible effect; an unknown tool is only logged.
  • F19: the terminal opens in the home folder, not the workspace.

Commands run

  • python atp/scripts/gen_atp.py and python atp/scripts/make_appendices_zip.py
  • ./build.sh and ./run.sh from the unzipped archive on an Apple Silicon Mac, then the scenarios through noVNC

Scenarios F13, F15 and F20 to F22 were not run in the app (F20 to F22 need an API key).

Keep the scenarios worth demonstrating, drop the OS-specific ones so the
plan is reproducible on any machine with Docker, renumber F1 to F24 and
add a Getting started section to the Info & Access tab.

Fix the Docker environment along the way: the AppImage now extracts under
amd64 emulation on Apple Silicon, git is installed for the git scenario,
and the app config lives in a named volume so session restore survives a
container restart.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant