Skip to content

Per-sprite multi-pass effects use canvas-relative UVs instead of sprite-local UVs #1407

@obiot

Description

@obiot

Problem

When a renderable has multiple post-processing effects (multi-pass via FBO ping-pong), the shader UVs are relative to the full canvas instead of the sprite. This causes UV-dependent effects (glint sweeps, scan lines, hologram patterns) to appear as "cutouts" of a full-screen effect that scroll with the camera.

Single-effect sprites are NOT affected — they use the customShader fast path which replaces the default shader during the draw call. The sprite's native texture UVs are used directly.

Root Cause

In WebGLRenderer.beginPostEffect(), when a non-camera renderable has 2+ effects, we create a canvas-sized FBO and render the sprite into it using the camera's projection. The sprite occupies a small region of the FBO at its screen position. When the effect shaders process this FBO, the UV coordinates (0..1) span the entire canvas, not the sprite.

File: packages/melonjs/src/video/webgl/webgl_renderer.jsbeginPostEffect() / endPostEffect()

What Needs to Change

Use sprite-sized FBOs with a local projection, so the sprite fills the FBO and UVs go from 0..1 across the sprite. This is how PixiJS FilterSystem works (source frame + destination frame).

The Transform Chain Challenge

In Renderable.preDraw() (packages/melonjs/src/renderable/renderable.js), beginPostEffect is called before autoTransform and the anchor offset:

save()
globalAlpha, flip, mask
beginPostEffect(this)     ← FBO bound here
autoTransform             ← sprite rotation/scale (only if non-identity)
translate(-ax, -ay)       ← anchor offset

At beginPostEffect time, currentTransform has the camera scroll + parent transforms, but NOT the sprite's own position. The sprite's position comes from Sprite.draw() which draws at this.pos.x, this.pos.y.

The Projection Math

To make the sprite fill a sprite-sized FBO:

  1. Compute screen-space bounds: getBounds() returns world-space AABB (includes pos, anchor, rotation). Transform through currentTransform (which has camera scroll) to get screen bounds:

    screenX = bounds.x * transform.a + bounds.y * transform.c + transform.tx
    screenY = bounds.x * transform.b + bounds.y * transform.d + transform.ty
    
  2. Set projection: ortho(screenX, screenX + spriteW, screenY + spriteH, screenY) — maps the sprite's screen region to NDC, so the sprite fills the FBO.

  3. Set viewport: gl.viewport(0, 0, spriteW, spriteH) — FBO pixel dimensions.

  4. In endPostEffect: intermediate ping-pong blits use ortho(0, spriteW, spriteH, 0) (sprite-local). The final blit uses ortho(0, canvasW, canvasH, 0) with the quad at (screenX, screenY, spriteW, spriteH) — this requires blitEffect to accept separate projection dimensions via projWidth/projHeight parameters.

What We Tried

Two attempts were made in the feat/post-effect-chaining branch:

  • Attempt 1: Moved beginPostEffect after all transforms, computed bounds from local (0,0,w,h) through full transform. Failed — broke the save/restore nesting and the transform was already partially consumed.
  • Attempt 2: Kept beginPostEffect in original position, computed bounds from getBounds() + currentTransform. The projection math didn't produce correct results — sprites were invisible or mispositioned.

PixiJS Reference

PixiJS v7 FilterSystem (packages/core/src/filters/FilterSystem.ts):

  • push(): gets target.getBounds(true), pads by filter padding, clips to visible area, allocates FBO from RenderTexturePool, sets projection via ProjectionSystem.update(destinationFrame, sourceFrame)
  • pop(): ping-pong with CLEAR_MODES.CLEAR for intermediate passes, CLEAR_MODES.BLEND for final pass to parent
  • Projection formula: pm.a = 2/sourceFrame.width, pm.tx = -1 - sourceFrame.x * pm.a (maps source frame to NDC)

Key Files

  • packages/melonjs/src/video/webgl/webgl_renderer.jsbeginPostEffect(), endPostEffect(), blitEffect()
  • packages/melonjs/src/renderable/renderable.jspreDraw() transform order
  • packages/melonjs/src/renderable/sprite.jsdraw() uses this.pos.x/y for draw position
  • packages/melonjs/src/video/rendertarget/render_target_pool.js — pool already supports arbitrary sizes
  • packages/melonjs/src/video/rendertarget/rendertarget.ts — base class already supports bind/unbind/resize

Acceptance Criteria

  • Per-sprite multi-pass effects use sprite-local UVs (0..1 across the sprite)
  • UV-dependent effects (hologram, glint, scan lines) stay fixed on the sprite regardless of camera position
  • Camera multi-pass effects still work correctly
  • Single-effect customShader fast path is unchanged
  • No visual regression on resize or state transitions
  • Effects like glow/blur that extend beyond sprite bounds work with configurable padding

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions