Skip to content

cogl/shader: Don't keep the compilation pipeline alive - #877

Open
anthonytheinventor wants to merge 1 commit into
linuxmint:masterfrom
anthonytheinventor:fix-cogl-shader-leak
Open

anthonytheinventor wants to merge 1 commit into
linuxmint:masterfrom
anthonytheinventor:fix-cogl-shader-leak

Conversation

@anthonytheinventor

Copy link
Copy Markdown

Problem

Every ClutterShaderEffect that sets its own shader source leaks its offscreen texture when it is destroyed: one stage-sized texture, about 4 MB at 1366x768. Extensions that add these effects to windows leak GPU memory on every window open and close. With BlurCinnamon's window blur, Cinnamon's GPU memory grew by about 175 MB over 3 open/close cycles.

Cause

In cogl/cogl/deprecated/cogl-shader.c, _cogl_shader_compile_real() keeps a reference to the pipeline the shader was compiled for (shader->compilation_pipeline), only to compare its layer numbers on the next compile. That pipeline owns the user program, and the user program owns the shader, so the reference makes a cycle:

pipeline → user program → shader → compilation_pipeline (the same pipeline)

None of them is ever freed, and neither is the effect's texture held by the pipeline's layer. Separately, _cogl_shader_free() never frees the shader source string.

Fix

Store the pipeline's layer indices instead of a reference to the pipeline. The check only needs the layer indices and texture units, and texture units follow layer order, so the indices cover both. The shader frees the indices and its source string when it is freed.

Testing

On Linux Mint 22.3 with Muffin 6.6.3 (Intel HD 4000, X11). This file is the same on master, and the patch builds there without warnings.

  • 20 ClutterShaderEffects with their own shader source, created and destroyed: GPU memory +100 MB before, 0 after.
  • BlurCinnamon 2.6.1 with window blur, opening and closing a blurred window: +175 MB over 3 cycles before, 0 over 10 cycles after.
  • Rendered output is pixel-identical before and after.
  • Cogl test suite: 52/52 pass.
  • A temporary test reusing one shader across pipelines with different layers (not included) gives identical results before and after. It reaches both the reuse and recompile paths. Valgrind reports no errors in this code.

I worked on this with Claude Code and tested it on my machine.

CoglShader kept a reference to the pipeline it was last compiled for,
so it could skip recompiling when the layer numbers had not changed.
That pipeline owns the user program, which owns the shader, so the
reference made a cycle: the pipeline and its layer textures were never
freed.

ClutterShaderEffect hits this for every effect with its own shader
source. Each one leaks a stage-sized texture (about 4 MB at 1366x768),
so extensions that create these effects per window, like BlurCinnamon,
leak tens of megabytes of GPU memory each time a window is opened and
closed.

Store the layer indices instead (texture units follow layer order, so
they cover the same check), and free them and the shader source with
the shader.

Ref: klangman/BlurCinnamon#60
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant