COLDBOX-1457 Browser testing for ColdBox apps: BrowserTestCase and named route helpers by lmajano · Pull Request #708 · ColdBox/coldbox-platform

lmajano · 2026-10-01T12:00:52Z

Description

Jira Issues

Browser testing for ColdBox applications, built on TestBox's new BrowserSpec support (Ortus-Solutions/TestBox#212) and the bx-playwright BoxLang module.

coldbox.system.testing.BrowserTestCase (BoxLang only)

Extends BaseTestCase, so the app loads for route lookups.
Delegates to TestBox's BrowserSupport:
- browse(), this.playwright(), browserAvailable()
- the @browserProfile and @baseURL annotations
- the browser matchers, such as expect( page ).toSee( "Welcome" )
- automatic attachment of screenshots, traces and videos
routeURL( name, params ): the path and query, built by ColdBox's own event.route().
visitRoute( page, name, params ) visits a named route.
assertRouteIs( page, name, params ) waits until the page is on that route. Without params it matches the route pattern; with params it matches the exact path.
Browser specs skip cleanly when TestBox browser support or bx-playwright is not installed; the route helpers still run.

Logged-in tests use bx-playwright saved sessions, with no test-only login endpoints in ColdBox:

this.playwright().session( "admin", ( page ) => visitRoute( page, "login" ).fill( "Email", "[email protected]" ).fill( "Password", "secret" ).click( "Sign in" ) )
browse( ( page ) => visitRoute( page, "admin.dashboard" ), { session : "admin" } )

Fixes

event.route( "name@module" ) built links with no slash after the module entry point (bloglogin/3/). There is a unit test in RequestContextTest.
On Adobe, RequestContext.getMemento() included a this key, so RequestContextDecorator methods ran against the original context and missed its mocks. Regression specs in RequestContextDecoratorTest.

Harness and tests

Test-harness routes users.show, plus a plain login form (browserTesting.login) and browserTesting.whoami for the saved-session specs.
tests/runner.cfm skips the BoxLang-only /browser folder on other engines.
BrowserTestCaseSpec covers route paths (app and module routes, through the resourcesTest module), assertRouteIs(), a saved session created through the login form, and per-page session isolation.

Type of change

Bug Fix
New Feature
This change requires a documentation update (COLDBOX-1457 Browser Testing guide: BrowserTestCase, named routes, saved sessions and CI ortus-docs/coldbox-docs#301)

Checklist

My code follows the style guidelines of this project
I have commented my code (every new method has a docblock)
I have made corresponding changes to the documentation (COLDBOX-1457 Browser Testing guide: BrowserTestCase, named routes, saved sessions and CI ortus-docs/coldbox-docs#301)
I have added tests that prove my fix is effective or that my feature works

🤖 Generated with Claude Code

https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

event.route( "name@module" ) joined the module inherited entry point and the route pattern without a separator, so "login@BrowserTesting" built "__browser-testinglogin/3/". ModuleService stores inherited entry points without a trailing slash, so the link now joins them with a single slash. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

…dpoints GET /__browser-testing/login/:id and GET /__browser-testing/logout call the login and logout closures of moduleSettings.browserTesting. Every request gets a plain 404 unless the environment is testing, the module is enabled, a token is configured and matches the X-Browser-Testing-Token header or the token variable (compared in constant time), and the closure is set. The test harness enables it, and runs in its testing environment on 127.0.0.1. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

coldbox.system.testing.BrowserTestCase (BoxLang) extends BaseTestCase and delegates to TestBox's BrowserSupport like BrowserSpec: browse(), this.playwright(), browserAvailable(), the browserProfile and baseURL annotations, the browser matchers and the afterAll closeBrowser() hook. It adds routeURL(), visitRoute() and assertRouteIs() for named routes, and loginAs() and logout() through the BrowserTesting core module. The specs live in tests/specs/browser, which tests/runner.cfm skips on engines other than BoxLang; the browser specs skip without bx-playwright. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

github-actions · 2026-10-01T12:07:22Z

Test Results

0 tests ±0 0 ✅ ±0 0s ⏱️ ±0s
0 suites ±0 0 💤 ±0
0 files ±0 0 ❌ ±0

Results for commit af17e82. ± Comparison against base commit 0a20ed3.

♻️ This comment has been updated with latest results.

…corator mocks on Adobe BrowserTestCase registered testbox.system.browser.BrowserMatchers in its pseudo constructor, so every runner.cfm request on BoxLang failed with a 500 on released TestBox builds that do not ship testbox.system.browser yet. It now detects TestBox browser support, registers the matchers only when present, and skips browser specs with a clear reason otherwise. The route helpers still run. RequestContext.getMemento() returned the `this` reference that Adobe keeps in the variables scope. RequestContextDecorator restored it, so on Adobe the decorator's inherited methods ran with `this` pointing to the original context and mocked methods failed with "Element _MOCKRESULTS is undefined in THIS" (BrowserTestingModuleSpec header token spec). The memento now leaves it out, with regression specs in RequestContextDecoratorTest. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

lmajano · 2026-10-01T13:09:51Z

CI status on 32e827b: the remaining red jobs are not caused by this PR.

tests / Test Suites (adobe@2023, 11, true, 8)
- Failure: one error in tests.specs.web.services.ModuleServiceTest > "Still resolves a handler-only URL for a module with only a mandatory-action route": "module requested 'test-module-conventions' is not loaded" (Router.cfc:323).
- Origin: that spec and its test module came from COLDBOX-1454: fix interceptor buffer pool race between a request and its async announce thread #705, already on development. The same error appeared on COLDBOX-1454: fix interceptor buffer pool race between a request and its async announce thread #705's own CI (run 36101802801, adobe@2025).
- Evidence it is intermittent: it passes on development and on adobe@2025 in this run, and passed 4 out of 4 local adobe@2023 runs.
- This PR touches neither the spec, ModuleService nor Router. No fix exists yet; it belongs with COLDBOX-1454: fix interceptor buffer pool race between a request and its async announce thread #705.
adobe@2025 and boxlang-cfml@be jobs: CommandBox setup failed with ForgeBox "408 Request Time-out" before any test ran.

These need a re-run of the failed jobs (runs 36863474898 and 36863469751). I don't have permission to re-run them (403), so a maintainer needs to. I'll keep watching the PR.

Generated by Claude Code

Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

The test-only login endpoints were a backdoor. Logged-in browser tests now use bx-playwright saved sessions: log in through the login page once with this.playwright().session(), then browse( ..., { session : "name" } ). The test harness gets a plain login form for the saved session specs, and goes back to its development environment on 127.0.0.1. Module route specs use the resourcesTest module. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

Copilot

Copilot review overview

🟡 Changes recommended

Route helpers incorrectly skip without browser support and fail to match valid optional route variants.

Review effort: Balanced
Findings: 2

Open (2)

Remove browser-support guard from visitRoute tests · New
Match all same-named route variants in assertRouteIs · New

What changed in this PR

Adds BoxLang browser testing support to ColdBox and fixes request-context routing/decorator regressions.

Changes:

Introduces BrowserTestCase with browser and named-route helpers.
Adds browser harness routes, fixtures, and integration coverage.
Fixes module route joining and decorator memento handling.

File	Description
`AGENTS.md`	Documents BoxLang annotation and optional-value conventions.
`changelog.md`	Records the feature and fixes.
`system/testing/BrowserTestCase.bx`	Implements browser and route-testing helpers.
`system/web/context/RequestContext.cfc`	Fixes module links and memento copying.
`test-harness/config/Router.cfc`	Registers browser-test routes.
`test-harness/handlers/browserTesting.cfc`	Provides browser-test pages and login flow.
`tests/resources/browser/FakePage.bx`	Adds a browser-independent page test double.
`tests/runner.cfm`	Excludes BoxLang browser specs on other engines.
`tests/specs/browser/BrowserTestCaseSpec.bx`	Covers browser and route helpers.
`tests/specs/web/context/RequestContextDecoratorTest.cfc`	Tests decorator scope isolation.
`tests/specs/web/context/RequestContextTest.cfc`	Tests module route slash handling.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

- visitRoute() no longer skips the spec without browser support: it only visits the page it gets, and pages come from browse(), which skips. - assertRouteIs() matches every route registered with the name, so a route with optional placeholders (/posts/:id?, registered as /posts/:id and /posts) matches with and without them. Module routes use the module routing table. - The harness login form renders the logged-in page instead of relocating: the harness runs with the test controller, which intercepts relocations. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

claude added 4 commits October 1, 2026 11:58

Document browser testing support in the changelog

fd039a3

Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

lmajano mentioned this pull request Oct 1, 2026

Fix scope names resolving to web scopes in browser specs (toHaveURL, assertPathIs, cookies, forms) ortus-boxlang/bx-playwright#15

Merged

5 tasks done

This was referenced Oct 1, 2026

COLDBOX-1457 Browser Testing guide: BrowserTestCase, named routes, saved sessions and CI ortus-docs/coldbox-docs#301

Open

Skills: TestBox BrowserSpec, browser matchers, ColdBox BrowserTestCase, attachments and retries ortus-boxlang/skills#11

Merged

lmajano mentioned this pull request Oct 1, 2026

Browser testing skills: TestBox BrowserSpec and ColdBox BrowserTestCase ColdBox/skills#16

Merged

lmajano changed the title ~~Browser testing for ColdBox apps: BrowserTestCase, named route helpers and test-only loginAs()~~ COLDBOX-1457 Browser testing for ColdBox apps: BrowserTestCase, named route helpers and test-only loginAs() Oct 1, 2026

claude added 2 commits October 1, 2026 15:08

Use BoxLang annotations in BoxLang examples

f28b073

Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01UU9aLossoYUkQNwubnitBi

lmajano changed the title ~~COLDBOX-1457 Browser testing for ColdBox apps: BrowserTestCase, named route helpers and test-only loginAs()~~ COLDBOX-1457 Browser testing for ColdBox apps: BrowserTestCase and named route helpers Oct 1, 2026

lmajano requested a balanced review from Copilot October 1, 2026 20:27

Copilot started reviewing on behalf of lmajano October 1, 2026 20:28 View session

Copilot AI reviewed Oct 1, 2026

View reviewed changes

Comment thread system/testing/BrowserTestCase.bx Outdated

Comment thread system/testing/BrowserTestCase.bx Outdated

lmajano mentioned this pull request Oct 1, 2026

Sync the browser testing skills with TestBox and ColdBox ortus-boxlang/skills#12

Merged

Navigation Menu

Uh oh!

COLDBOX-1457 Browser testing for ColdBox apps: BrowserTestCase and named route helpers - #708

COLDBOX-1457 Browser testing for ColdBox apps: BrowserTestCase and named route helpers#708
lmajano wants to merge 8 commits into
developmentfrom
claude/bx-playwright-research-plan-ddfqpm

lmajano commented Oct 1, 2026 •

edited by atlassian Bot

Loading

Uh oh!

github-actions Bot commented Oct 1, 2026 •

edited

Loading

Uh oh!

lmajano commented Oct 1, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

lmajano commented Oct 1, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Jira Issues

Type of change

Checklist

Uh oh!

github-actions Bot commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

lmajano commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lmajano commented Oct 1, 2026 •

edited by atlassian Bot

Loading

github-actions Bot commented Oct 1, 2026 •

edited

Loading

lmajano commented Oct 1, 2026 •

edited

Loading