Repository navigation
CT-2322 Run Maestro E2E tests on Sauce Labs - #132
Jameson13B wants to merge 3 commits into
Conversation
🤖 AI code review (4 lenses)Expand review
Verdict: the design holds up on every lens: Convergent findings (≥2 lenses)
Unique findings
-- Jameson's Agent |
The example app hardcoded the SSO proxy at 10.0.2.2:8089 (Android) or localhost:8089 in six places. Move it to src/shared/proxy.ts and let EXPO_PUBLIC_PROXY_BASE_URL override it, so a CI build can point a Sauce Labs device at the proxy through Sauce Connect. Local development keeps the same defaults. Release builds bundle the JS (no Metro), and Android blocks cleartext http in release builds by default, so enable usesCleartextTraffic via expo-build-properties. Only the example app is affected.
a6a70fb to
12055dd
Compare
Add a workflow that builds the example app as a release APK, starts the SSO proxy on the runner, opens a Sauce Connect tunnel and runs example/maestro on a Sauce Labs Android emulator (or a real device when run manually) with maestro-runner, Sauce's community integration for Maestro flows. The APK is built in its own job, which needs no secrets, so build breaks show up on every PR even while the Sauce Labs job is skipped for lack of secrets (forks, Dependabot, or before they're configured). The runner image already has the Android SDK; setup-android@v3 fails on the removed 'tools' package, so it isn't used. The app reaches the proxy through an /etc/hosts alias, because Sauce Connect can't proxy localhost to real devices, with proxyLocalhost: allow because the alias resolves to 127.0.0.1. E2E_TESTING.md lists the required secrets.
Credentials - Redact the Sauce access key in the Maestro report instead of deleting files, fail the step if any copy is left, and only upload the report when that step succeeded. The old scrub could fail open and threw away maestro-runner.log, the main debugging file. - Pass the Sauce and MX secrets only to the steps that use them, and pin saucelabs/sauce-connect-action to a SHA since it receives the key. - Print only request lines from the SSO proxy log; its crash output includes the client credentials. - Gitignore every .mx-sso-proxy-rc* except the example file. Reliability - Pin the SSO proxy and maestro-runner in example/sauce/package.json with a lockfile, installed with --ignore-scripts, instead of a full root npm ci plus npx. - Restart the SSO proxy when it exits: version 1.3.4 dies on the first MX API error. Give the readiness probe a per-request timeout. - Only route the proxy alias through the tunnel (tunnelDomains). - Build only the x86_64 and arm64-v8a ABIs; a cold 4-ABI build took 21 minutes in CI. - Run on pushes to v4 and master so the caches warm for PRs; include the SDK build config in the path filter; keep manual runs for different targets from cancelling each other; time-box the Maestro step; turn off maestro-runner telemetry. Docs: how to run it manually (not from the Actions tab until the file is on master), valid SAUCE_REGION values, and a "When it fails" section.
12055dd to
a7e98f6
Compare
|
Closing: this was opened against the wrong repo. The React Native Connect SDK lives in mxenabled/react-native-connect-widget-sdk now; the same change is mxenabled/react-native-connect-widget-sdk#39. -- Jameson's Agent |
Refs CT-2322
Runs the example app's Maestro flows on a Sauce Labs device from GitHub Actions, instead of on a GitHub-hosted Android emulator (30+ minutes, see #117), using maestro-runner, Sauce's community integration that runs Maestro flows through Appium.
How it works
build-androidbuilds the SDK and the example app as a release APK (JS bundled, no Metro) withEXPO_PUBLIC_PROXY_BASE_URL=http://sdk-e2e-proxy:8089. No secrets needed, so it runs on every matching PR, including from forks.test-androidstarts the SSO proxy on the runner, pointssdk-e2e-proxyat127.0.0.1in/etc/hosts, opens a Sauce Connect tunnel that carries only that host, uploads the APK to App Storage and runsexample/maestro/on a Sauce Android emulator (or a real device viaworkflow_dispatch). The report is uploaded as an artifact with the Sauce access key redacted, and the APK is deleted from App Storage afterwards.Details, required secrets and a "When it fails" section are in
E2E_TESTING.md.Changes to the example app
example/src/shared/proxy.ts, overridable withEXPO_PUBLIC_PROXY_BASE_URL. Local defaults are unchanged.usesCleartextTrafficis enabled throughexpo-build-properties, so the release build can talk http to the proxy.example/sauce/holds the Appium capabilities per target and a small lockfile pinning the SSO proxy and maestro-runner.Before merging: secrets
The
test-androidjob has not run yet, because the repo has no Sauce Labs credentials. It needs these repository secrets:SAUCE_USERNAME,SAUCE_ACCESS_KEY(ideally a service user; the mobile team owns the Sauce account),MX_CLIENT_IDandMX_API_KEY(the int E2E test client from.mx-sso-proxy-rc.example.yml, or a dedicated one). Once they exist, re-run the workflow on this PR and expect to tune the flows on the first real run (for example a Chrome first-run sheet in the OAuth Custom Tab).Manual runs from the Actions tab only become available once the workflow file is on
master; until thengh workflow run e2e-sauce-labs.yml --ref <branch> -f target=android-real-deviceis the way.Acceptance criteria
10.0.2.2:8089/localhost:8089. Verified: the CI-built APK's Hermes bundle containshttp://sdk-e2e-proxy:8089; unset falls back to the old values.build-androidpasses; the manifest hasusesCleartextTraffic.</dev/nulland missing settings it exits 1 instead of prompting; the restart loop and probe behave as intended. Reachability from the Sauce device through Sauce Connect is unverified until the secrets exist.maestro-runner, and the job fails when a flow fails. Needs the secrets. The flows passmaestro-runner lint, and its exit codes were checked in the source.E2E_TESTING.mddocuments the CI setup, the required secrets and how to debug a failure.Review
Four-lens AI review posted as a comment below; everything it raised is addressed except the items marked open there (NDK download time, Chrome first-run on the OAuth tab, Sauce session-creation failures), which need a real run to judge.