Skip to content

Android: touch timeline with desktop parity - #1100

Merged
CaYatur merged 2 commits into
dev/androidfrom
android/timeline-pro
Nov 18, 2026
Merged

CaYatur merged 2 commits into
dev/androidfrom
android/timeline-pro

Conversation

@CaYatur

@CaYatur CaYatur commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

For #1054 (Phase 5).

Rebuilds the phone/tablet timeline on the desktop's shared modules so the data, the editing operations and the live output match the desktop Timeline page. Parity checklist with a status per feature: docs/android/TIMELINE.md (TR: TIMELINE.tr.md).

Shared code

  • Model, tempo map, markers, loop, timeline bank: src/shared/timeline.js
  • Split, duplicate, group copy/paste, track change, tempo edits, undo: src/shared/timeline-edit.js
  • Composition, "Use timeline", export job: src/shared/timeline-compose.js
  • Show clock: src/shared/showclock.js
  • Automation targets: src/admin/modulation-panel.js targetCatalog (loads without the panel; added to the APK assets)
  • YouTube chapters: src/shared/export-formats.js

Phone-only: the touch surface (src/mobile/shell-timeline.js, -lane.js, -forms.js) and the in-app player as the transport's clock.

Live output

"Use timeline" works as on the desktop: the engine draws the composition at the playhead every frame (transitions, opacity, fades, automation, video clips). The playhead reaches the engine as a show-clock anchor through the new SVAndroid.showClock (HOST.md updated). While the mode is on, a chip on the main screen says so and offers "Back to manual settings". The MVP's apply-scene-at-clip-boundary path is gone, so playing no longer overwrites the working scene.

In-app player as the clock

When the in-app player has a song, the song is the clock:

  • After play, a jump or a loop wrap, and while the song buffers, the playhead waits for the song. It continues from the song's position once the player reports playing. Measured on the emulator: after a seek the player played again only after ~0.3–0.5 s. A playhead that did not wait ran ahead and then jumped back.
  • "Playing" events can be missed (seen on the emulator after a loop wrap: the song played on while the playhead waited), so while waiting the playhead also asks for the player state four times a second. A state counts only if it was read after the seek was done (the seek reply carries at).
  • While playing, the player sends no events, so drift could not be seen. The playhead now asks for a fresh position twice a second through a light request (player.refresh() → playerPosition, no cover art). It moves to the song when they differ by more than 80 ms. Answers asked before a jump are dropped.
  • With the loop on, a song found outside the loop goes back in with the show clock's own wrap (showclock.wrap), and the playhead waits for it. This happens when a wrap seek came late or was lost. Measured on the emulator: the song ran 1.4 s past the loop end, and drift correction pulled the playhead out with it.
  • The player state carries at (when position was read), so a late event is corrected. On the emulator, events reached the page up to ~0.9 s late while the page was busy.
  • If the song is paused from outside (notification, headphones, end of song), the timeline pauses too. If the player does not start within 6 s, both the timeline and the player pause. Past the end of the song, or with no song, the timeline runs on its own clock.

Layouts

  • Portrait phone: dock under the visible stage; narrow track headers; toolbar that follows the selection; inspector as its own page.
  • Landscape phone: side by side (stage left, timeline full height right). A full-width dock left room for only 2 lanes on 914x411 dp.
  • Tablet (dock ≥ 900 px): wide headers with M/S/lock, inspector on the right.

Differences from the desktop (documented in the checklist)

  • With "Use timeline" on, the desktop edits the clip's local copy (timeline-target.js). On the phone, "Open scene" goes back to manual settings, applies the scene and opens Layers. Clip copies from desktop backups are kept and drawn.
  • Action clips run the phone equivalents (blackout, next scene, next/previous visualizer, next palette, next background, timeline transport). Deck, MilkDrop navigation and redundancy actions are kept in the data but do nothing.

Fixes found while testing

  • The model was rebuilt on every config patch. That would drop in-progress drags and recorded automation points. It is now rebuilt only when cfg.timeline changes from outside.
  • The fade handle hit area reached the middle of short clips, so the second tap of a double tap landed on it. It is now limited to the top 14 px, and a still tap on a handle counts as a clip tap.
  • Player sync: the old code checked a buffering flag that the player never sends (it sends a state string). The loop wrap depended on a cached state that was stale. A pause from outside right after play was swallowed.

Verification (API 36 emulator, SwiftShader)

  • node android/tools/probe.js timeline: 19 checks with real touches (adb, plus CDP touch events for pinch and double tap), all passing:
    • select; drag snapped to the bar; vertical track change; trim; undo; pinch zoom; long-press menu
    • Use timeline: engine composition via SVShowDebug.time/.composed
    • playback; inspector on double tap
    • marker, cue and GO; tempo change; loop; markers from lyrics
    • automation learn and record (4 points)
    • song as the clock: playhead 4.24 s, song 4.24 s, engine 3.86 s; pausing the player pauses the timeline; loop 0–2 s sampled ten times (the song wraps, does not stay outside the loop, playhead and song within 0.5 s in at least 7 of 10)
    • export while Use timeline is on: with "With the timeline" on, frames at two moments differ (diff 340); with it off, the manual scene (not black, different from the timeline frame)
  • That run used a build with this PR plus the follow-up PR's changes. The two exports saved only because of its gallery-name fix: the emulator's gallery had reached sv-test-song (31).mp4.
  • The loop check runs with Use timeline off. With it on, the page froze for 1.5–3 s on SwiftShader whenever a scene clip started (in-page log every 100 ms; with the mode off the ticks stay at 100 ms). No tick runs during a freeze, so the song passes the loop end. The timeline brings it back once the page runs again. Real-phone performance is Phase 6.
  • Playhead and song on the emulator: within ~0.2 s with the mode off, and ±0.4 s under load, when the audio itself ran at 64–110% of real time.
  • pages passes on this build and on the 1117e dev/android build. The earlier single failure on the favorite star did not repeat.
  • Commit-only tree: gradlew assembleDebug builds; node --test android/js-tests/*.test.js: 72 pass; npm test: 4570 pass, 1 skipped.
  • Layouts seen in portrait 1080x2400, real landscape rotation and wm size 2560x1600 (tablet).

CaYatur and others added 2 commits November 18, 2026 01:58
Rebuilds the mobile timeline on the desktop's shared modules
(timeline.js, timeline-edit.js, timeline-compose.js, showclock.js) so the
data, editing operations and output match the desktop Timeline page.

- Live output uses the desktop's "Use timeline" mode: the engine draws
  the composition at the playhead every frame (transitions, opacity,
  fades, automation, video clips). The playhead reaches the engine as a
  show-clock anchor through the new SVAndroid.showClock. The in-app
  player follows the transport (play/pause/seek, loop wrap, cue hold).
- Canvas lanes with touch gestures: select, drag (snap + magnet,
  vertical track change), trim handles outside the clip, fade handles,
  pinch zoom, fling scroll, long-press context menu, marquee
  multi-select, ruler scrubbing, marker/tempo/loop dragging.
- Desktop features: timeline bank, clip and automation tracks
  (mute/solo/lock/opacity/order), all clip types with sources and
  targets, phone equivalents of action clips, inspector fields,
  split/duplicate/copy/cut/paste/nudge/move track, markers and cues
  with GO, tempo map, loop, all snap modes, ruler units, follow, lane
  height, undo/redo, markers from lyrics, YouTube chapters, automation
  targets from the desktop catalog, learn and record.
- Layouts: dock under the visible stage in portrait, side by side in
  landscape phones, wide headers and a side inspector on tablets.
- A chip on the main screen shows when the output follows the timeline,
  with a way back to manual settings.
- Parity checklist: docs/android/TIMELINE.md (+ .tr.md).
- Probe `timeline` rewritten for the canvas: 15 checks with real touches.

Co-authored-by: caganturgut <[email protected]>
- Wait for the song after play, a jump or a loop wrap, and while it
  buffers; continue from the song's position. A state counts only if it
  was read after the seek was done (the seek reply carries `at`).
- While waiting, poll the player four times a second (a "playing" event
  can be missed); while playing, twice a second through the new light
  `playerPosition` request (`player.refresh()`), and follow the song when
  they differ by more than 80 ms.
- The player state carries `at`, so a late event is corrected.
- With the loop on, a song found outside the loop goes back in with
  showclock.wrap instead of pulling the playhead out.
- If the player does not start within 6 s, pause both; turning
  "Use timeline" off pauses the song too.
- Probe: compensated positions, loop sampled ten times (with the mode
  off: on SwiftShader the page freezes 1.5-3 s when a scene clip starts),
  exports must report saved, partial results on error.
- Docs: TIMELINE song clock and emulator precision; HOST player.refresh,
  playerPosition and `at`.

For #1054

Co-authored-by: caganturgut <[email protected]>
@CaYatur
CaYatur merged commit 9b38218 into dev/android Nov 18, 2026
7 checks passed
@CaYatur
CaYatur deleted the android/timeline-pro branch November 18, 2026 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant