Skip to content

Fall back to a 10-bit motion vector u/v map if half floats can't be rendered to - #1081

Open
Coastrnaut wants to merge 1 commit into
projectM-visualizer:masterfrom
Coastrnaut:fix/gles30-motion-vector-format
Open

Coastrnaut wants to merge 1 commit into
projectM-visualizer:masterfrom
Coastrnaut:fix/gles30-motion-vector-format

Conversation

@Coastrnaut

@Coastrnaut Coastrnaut commented Nov 13, 2026 •

Copy link
Copy Markdown

Related to #998 and #1004

Since e98fca8, projectM accepts OpenGL ES 3.0 contexts, e.g. ANGLE on Direct3D 11 as in #998. One part of the renderer still needs more than core ES 3.0: the warp pass writes the texture coordinates used for the motion vectors into a GL_RG16F texture attached to the preset framebuffer. Half-float textures are only color-renderable in OpenGL ES 3.2, or in 3.0 and 3.1 with EXT_color_buffer_float or EXT_color_buffer_half_float. Without them, the framebuffer is incomplete while the warp mesh is drawn, so nothing is rendered.

Change

MilkdropPreset checks whether half floats can be rendered to: always in desktop GL and ES 3.2, otherwise if one of the two extensions is listed. If they can't, the u/v map uses GL_RGB10_A2, which every ES 3.0 implementation can render to. The map only holds texture coordinates in 0..1, so 10 bits give the motion vector start points about one pixel of precision at 1024 pixels width.

Desktop GL and OpenGL ES 3.2 use GL_RG16F as before.

Of the other ES 3.1/3.2 features listed in #998, I found no use of vertex attribute binding (glVertexAttribBinding/glBindVertexBuffer) in the current code. Array-of-arrays in preset shaders still needs GLSL ES 3.10, which affects only the presets that use it.

Testing

Windows, RTX A6000. The NVIDIA driver always creates an OpenGL ES 3.2 context, so I tested the fallback with a local build that always takes it.

  • Fallback, precision: synthetic presets with 40×30 motion vectors over zoom, warp and translation render identically between runs. Compared with GL_RG16F, their final frames differ by a mean of about 2/255 per channel; 1.5 to 2.1% of the channels differ by more than 16, where vector endpoints moved by a pixel.
  • Real presets, fallback vs GL_RG16F: over the first 400 presets of the packs in the ES context, 66 render different frames, all with the same compile/link results and failed loads and 0 GL errors. Since each preset continues from the image the previous one left, a difference carries over to the following presets.
  • ES 3.2 without the fallback: the first 400 presets in an OpenGL ES 3.2 context give the same shader compile/link results, failed loads and frames as master, with 0 GL errors.
  • Desktop GL 3.3 core: I loaded all 14,536 presets from cream-of-the-crop, en-d, milkdrop-original and projectm-classic one after another, with the preset folders as texture search path, rendered 30 frames each with fixed audio, frame times and random seeds (the seeds were a local patch only, identical on both sides). Shader compile/link results, failed loads and frames are identical to master for every preset.
  • Unit tests: pass. There's no new unit test because the test suite doesn't create a GL context.

I couldn't test on an actual ES 3.0 implementation without the extensions, such as ANGLE's D3D11 backend.

It merges cleanly with my other open PRs and with all other open PRs.

…endered to

The warp pass writes the texture coordinates for the motion vectors into
a GL_RG16F texture attached to the preset framebuffer. OpenGL ES 3.0 and
3.1 can only render to half-float textures with EXT_color_buffer_float
or EXT_color_buffer_half_float; without them the framebuffer is
incomplete while the warp mesh is drawn.

Now that OpenGL ES 3.0 contexts are accepted, use GL_RGB10_A2 for the
map in that case. It stores the coordinates with 10 bits each, about one
pixel at 1024 pixels width. Desktop GL and OpenGL ES 3.2 are unchanged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant