diff --git a/packages/melonjs/CHANGELOG.md b/packages/melonjs/CHANGELOG.md index 51378e2a7c..e3d7325a84 100644 --- a/packages/melonjs/CHANGELOG.md +++ b/packages/melonjs/CHANGELOG.md @@ -20,6 +20,7 @@ - Audio: a clip declared with `preload: false` is fetched when it is first played (`audio.play`, `audio.playTrack` or `autoplay`) and streamed, as with `stream: true`, so it starts as soon as enough of it has arrived ([#1784](https://github.com/melonjs/melonJS/issues/1784)). It was never fetched at all, so it never played; a clip that is never played is still never downloaded - State: the first frame of a new stage shows what it built, and the first frame after a level loaded while the game runs shows that level. That frame was drawn before the world had been stepped, and a renderable is drawn only once a step has found it on screen, so every state change showed one frame of background alone unless a transition covered it - State: a `state.change()` asked for while a transition covers the screen is made by that transition, to the last state asked for and with its arguments, as without a transition ([#1787](https://github.com/melonjs/melonJS/issues/1787)). It put up a second cover, and the first stayed on the camera for good, the screen black over a game still running; a change to a state with no transition of its own is still made at once, and takes the cover down +- Effects: the built-in effects that place a pattern place it on the sprite itself, cut from an atlas or with other effects after it, and `SwayEffect` measures its strength across the sprite ([#1407](https://github.com/melonjs/melonJS/issues/1407)). On an atlas sprite, `SwayEffect` and an anchored `WaveEffect` moved its roots and `VignetteEffect`, `RadialGradientEffect` and `ScanlineEffect`'s curve landed on the atlas; with two or more effects, every pattern, `noise_uv` and `ShineEffect`'s sweep included, sat on the canvas and slid with the camera ## [20.11.0] (melonJS 2) - _2026-10-27_ diff --git a/packages/melonjs/skills/melonjs-effects-and-shaders/SKILL.md b/packages/melonjs/skills/melonjs-effects-and-shaders/SKILL.md index 4d64cb74d1..3ac49e0542 100644 --- a/packages/melonjs/skills/melonjs-effects-and-shaders/SKILL.md +++ b/packages/melonjs/skills/melonjs-effects-and-shaders/SKILL.md @@ -138,6 +138,19 @@ as transparent, as if the sprite were drawn on its own, so they never show along its edges and the atlas needs no padding for them. Past the texture's own edge (a standalone image, or the outside of an atlas) nothing changes. +**The effects that place a pattern place it on the renderable they draw.** +`SwayEffect`'s planted row, `WaveEffect`'s `anchor`, the centre of +`VignetteEffect`, `RadialGradientEffect` and `ScanlineEffect`'s curvature all +land on the sprite itself: cut from an atlas, flipped, rotated, or with other +effects after it, the pattern sits where it sits on a sprite with a texture of +its own and that one effect. What is measured in the texture stays measured in +it: `ShineEffect`'s sweep, bands and width, `WaveEffect`'s amplitude and +frequency, `DissolveEffect`'s grain, the lines of `HologramEffect` and +`ScanlineEffect` (on an atlas sprite, fractions of the atlas, so a single shine +crosses an atlas sprite only part of the time). With other effects after them +they all stay on the sprite. `SwayEffect` measures everything across the +sprite. + **Their widths, offsets and sizes are in pixels of the texture they draw, with nothing to pass.** Each of those effects takes the size of what it draws: the sprite's whole texture (the atlas image, not the frame), the screen for a @@ -166,17 +179,18 @@ tree.addPostEffect(sway); sway.setTime(timer.getTime() / 1000); ``` -Rows are measured across the texture the effect samples, so on a sprite cut from -an atlas the anchor is a row of the atlas: it suits a standalone image or a -layer best. +Its strength (a fraction of the sprite's width), its `anchor` and how it bends +are all measured across the sprite, so a sprite cut from an atlas sways exactly +like the same art with a texture of its own. `WaveEffect`'s `anchor` is a row of +the sprite too, but its amplitude and frequency are measured across the texture, +so on an atlas sprite they are fractions of the atlas. -`RadialGradientEffect` is a procedural soft spot: solid at the centre of its -host quad, fading linearly to transparent at the edge, and elliptical for free -on a non-square quad. It is the one to reach for instead of baking a disc into -a canvas, for a pickup highlight, a trigger marker or a damage flash. Its -falloff assumes the quad samples a FULL texture rect, so pair it with a -standalone image or `Renderer.getWhitePixel()` rather than a sprite that draws -a sub-region of an atlas, where the centre would land off-quad. +`RadialGradientEffect` is a procedural soft spot: solid at the centre of the +renderable it draws, fading linearly to transparent at the edge, and elliptical +for free on a non-square one. It is the one to reach for instead of baking a +disc into a canvas, for a pickup highlight, a trigger marker or a damage flash, +on a standalone image, `Renderer.getWhitePixel()` or a sprite cut from an atlas +alike. `SquishEffect` bulges a sprite's middle out (`setBulge(0.3)`) or pinches it in (`setBulge(-0.3)`), its top and bottom rows staying put. Bulging out widens the @@ -524,6 +538,10 @@ has changes nothing). What splits the batch: - a different effect instance, or a sprite without the effect in between; - a `noise_uv` effect on sprites of different frames or sizes (its rect is per sprite); +- one of the effects that place a pattern on the sprite (`SwayEffect`, + `WaveEffect`, `ShineEffect`, `VignetteEffect`, `RadialGradientEffect`, + `ScanlineEffect`, `DissolveEffect`, `HologramEffect`): each sprite is told + where it is, so each draws on its own; - a `screen_texture` effect, which draws each sprite on its own, as each one samples a capture of everything drawn before it. @@ -603,7 +621,9 @@ For refraction, distortion and anything that samples what is already drawn: - **`screen_texture`** — a sampler kept filled with everything drawn so far (GLSL: annotate the sampler `uniform sampler2D tex : screen_texture;`) - **`screen_uv`** — this fragment's 0..1 position in that capture -- **`noise_uv`** — 0..1 across the sprite regardless of its atlas frame +- **`noise_uv`** — 0..1 across the sprite regardless of its atlas frame, and + across the renderable when it has several effects (it does not slide with + the camera) In WGSL the capture is sampled through `screen_sampler` (clamped) or `screen_sampler_repeat` (wrapping) instead of GLSL's `: screen_texture` diff --git a/packages/melonjs/src/renderable/renderable.js b/packages/melonjs/src/renderable/renderable.js index dd8356df49..8cdc329423 100644 --- a/packages/melonjs/src/renderable/renderable.js +++ b/packages/melonjs/src/renderable/renderable.js @@ -1390,6 +1390,13 @@ export default class Renderable extends Rect { renderer.translate(-ax, -ay); } + // its own transform applied: its effects learn where it is, for the + // ones that place a pattern on it (a sway's planted row, a shine's + // sweep). Optional, for a renderer that is not a `Renderer` + if (!this._postEffectManaged && this.postEffects.length > 0) { + renderer._setEffectSpace?.(this); + } + // Apply the current tint and opacity. // // Multiplied by the alpha already on the renderer, so opacity diff --git a/packages/melonjs/src/video/effects/dissolve.js b/packages/melonjs/src/video/effects/dissolve.js index 80b24e440d..b6e01add38 100644 --- a/packages/melonjs/src/video/effects/dissolve.js +++ b/packages/melonjs/src/video/effects/dissolve.js @@ -47,7 +47,8 @@ fn apply(color : vec4f, uv : vec2f) -> vec4f { if (fx.uDissolveProgress <= 0.0) { return color; } - let n = dissolveFBM(uv * 8.0); + // grain of its texture, fixed on the sprite + let n = dissolveFBM(ME_art * 8.0); if (n < fx.uDissolveProgress) { discard; } @@ -123,7 +124,8 @@ export default class DissolveEffect extends ShaderEffect { if (uDissolveProgress <= 0.0) { return color; } - float n = fbm(uv * 8.0); + // grain of its texture, fixed on the sprite + float n = fbm(ME_art * 8.0); if (n < uDissolveProgress) { discard; } diff --git a/packages/melonjs/src/video/effects/glsl_realization.js b/packages/melonjs/src/video/effects/glsl_realization.js index fc29950a17..38099e73b5 100644 --- a/packages/melonjs/src/video/effects/glsl_realization.js +++ b/packages/melonjs/src/video/effects/glsl_realization.js @@ -1,4 +1,11 @@ import quadVertex from "../webgl/shaders/quad.vert"; +import { + LOCAL_FRAGMENT_GLSL, + LOCAL_SPACE_IDENTIFIER, + localVertexDeclarationsGLSL, + localVertexStatementsGLSL, + NOISE_UV_GLSL, +} from "./local_space.js"; /* * The GLSL realization of a ShaderEffect body — the pure source-assembly @@ -20,11 +27,15 @@ import quadVertex from "../webgl/shaders/quad.vert"; * (0..1 across the screen). * - `noise_uv` — varying with a frame-local coordinate across the drawn * object (undoes atlas packing; scaled to object pixels so patterns keep - * their density when the destination is scaled). + * their density when the destination is scaled). On a pass of a + * renderable's effect chain, 0..1 across the renderable (local_space.js). * * The builtins only activate when referenced, and never when the body * carries its OWN declaration of the identifier — a shader that used these * names before this feature keeps compiling unchanged. + * + * The engine's own bodies also read `ME_local` and `ME_art`, where they are + * on the renderable they draw (see local_space.js). */ const SCREEN_TEXTURE_ANNOTATION = /\buniform\s+sampler2D\s+([A-Za-z_]\w*)\s*:\s*screen_texture(?:\((repeat|repeat-x|repeat-y|no-repeat)\))?\s*;/g; @@ -69,7 +80,8 @@ function parseShaderBuiltins(fragmentBody) { const noiseUV = NOISE_UV_IDENTIFIER.test(body) && !hasOwnDeclaration(body, "noise_uv"); const spriteFrame = SPRITE_FRAME_SWITCH.test(body); - return { body, screenTextures, screenUV, noiseUV, spriteFrame }; + const local = LOCAL_SPACE_IDENTIFIER.test(body); + return { body, screenTextures, screenUV, noiseUV, spriteFrame, local }; } /** @@ -84,9 +96,17 @@ function parseShaderBuiltins(fragmentBody) { * @internal */ function buildEffectVertex(builtins) { - if (!builtins.screenUV && !builtins.noiseUV && !builtins.spriteFrame) { + if ( + !builtins.screenUV && + !builtins.noiseUV && + !builtins.spriteFrame && + !builtins.local + ) { return quadVertex; } + // `noise_uv` and the local space both read where the vertex is on the + // renderable (local_space.js) + const space = builtins.noiseUV || builtins.local; return [ "attribute vec3 aVertex;", "attribute vec2 aRegion;", @@ -111,17 +131,15 @@ function buildEffectVertex(builtins) { "varying vec2 noise_uv;", ] : []), + ...(space ? localVertexDeclarationsGLSL(builtins) : []), "void main(void) {", " vec4 ME_clip = uProjectionMatrix * vec4(aVertex, 1.0);", " gl_Position = ME_clip;", ...(builtins.screenUV ? [" screen_uv = ME_clip.xy / ME_clip.w * 0.5 + 0.5;"] : []), - ...(builtins.noiseUV - ? [ - " noise_uv = aRegion * (ME_size_img / ME_size_obj) - ME_offset / ME_size_obj;", - ] - : []), + ...(space ? localVertexStatementsGLSL(builtins) : []), + ...(builtins.noiseUV ? [NOISE_UV_GLSL] : []), ...(builtins.spriteFrame ? [ // a side on the texture's own edge (within a hundredth of @@ -169,6 +187,7 @@ export function buildGLSLProgram(fragmentBody) { "varying vec2 vRegion;", ...(builtins.screenUV ? ["varying vec2 screen_uv;"] : []), ...(builtins.noiseUV ? ["varying vec2 noise_uv;"] : []), + ...(builtins.local ? [LOCAL_FRAGMENT_GLSL] : []), builtins.body, "void main(void) {", " vec4 texColor = texture2D(uSampler, vRegion) * vColor;", diff --git a/packages/melonjs/src/video/effects/hologram.js b/packages/melonjs/src/video/effects/hologram.js index edb30b901c..ec7c757a23 100644 --- a/packages/melonjs/src/video/effects/hologram.js +++ b/packages/melonjs/src/video/effects/hologram.js @@ -20,7 +20,8 @@ struct HologramUniforms { fn apply(color : vec4f, uv : vec2f) -> vec4f { // scan line - let scan = sin(uv.y * 200.0 + fx.uTime * 5.0) * 0.5 + 0.5; + // lines of its texture, fixed on the sprite + let scan = sin(ME_art.y * 200.0 + fx.uTime * 5.0) * 0.5 + 0.5; // flicker let flicker = 0.95 + 0.05 * sin(fx.uTime * 30.0); // color shift @@ -62,7 +63,8 @@ export default class HologramEffect extends ShaderEffect { uniform float uTime; vec4 apply(vec4 color, vec2 uv) { // scan line - float scan = sin(uv.y * 200.0 + uTime * 5.0) * 0.5 + 0.5; + // lines of its texture, fixed on the sprite + float scan = sin(ME_art.y * 200.0 + uTime * 5.0) * 0.5 + 0.5; // flicker float flicker = 0.95 + 0.05 * sin(uTime * 30.0); // color shift diff --git a/packages/melonjs/src/video/effects/local_space.js b/packages/melonjs/src/video/effects/local_space.js new file mode 100644 index 0000000000..045987a9c6 --- /dev/null +++ b/packages/melonjs/src/video/effects/local_space.js @@ -0,0 +1,264 @@ +/** + * Where an effect is on the renderable it draws: the space a built-in effect + * places its pattern in (a sway's planted row, a shine's sweep, a + * vignette's centre), and the one `noise_uv` runs across on a pass over a + * render target. + * + * An effect is handed `uv`, the coordinate it samples its texture at. That + * runs 0..1 across the renderable only for a sprite with its own texture + * drawing it alone. A sprite cut from an atlas gets the atlas's UVs, and a + * renderable with two or more effects runs them over a capture the size of + * the canvas, which gets the canvas's (#1407). So a body reads, rather than + * `uv`: + * + * - `ME_local`: 0..1 across the renderable, following its art (mirrored + * with a flip, turned with a rotation). + * - `ME_art`: the coordinate its texture has there, for a pattern measured + * in the texture (a dissolve's grain, a hologram's lines, a wave's + * phase): `uv` itself wherever `uv` is the texture's. + * - `ME_localToUV(d)`, `ME_artToUV(d)`, `ME_localToArt(d)`: a displacement + * in one of those, in another, to move where the body samples. + * + * Three modes, in `ME_mode`: + * + * - 0, the default: what `uv` always was. `ME_local` is `uv`, top to bottom + * (`ME_ydir` says when the texture runs bottom-up, as a capture does on + * WebGL), and `ME_art` is `uv`. Any path that sets nothing keeps it: a + * camera's effects, `drawLight`, a mesh, a layer. + * - 1, a renderable drawn with its one effect: `ME_local` from the vertex's + * position through the renderable's own transform (`ME_local_x`, + * `ME_local_y`, target pixels to 0..1), so it runs across the whole + * renderable even when it is drawn in several quads. `ME_art` stays `uv`, + * and `ME_to_uv` turns a local displacement into one of `uv`. + * - 2, a pass of a renderable's effect chain: `ME_local` the same way, over + * the capture, and `ME_art` the renderable's frame in its texture + * (`ME_art_rect`, origin and size) at that point. `noise_uv` is + * `ME_local` too, so it runs across the renderable as it does when the + * renderable is drawn with its one effect. + * + * The renderer works the values out for each renderable (see + * `Renderer#_setEffectSpace`) and the batchers hand them over for a pass; + * `ShaderEffect#_setLocalSpace` writes them. + * @ignore + */ + +/** + * a body reading the renderable's own space + * @ignore + * @internal + */ +export const LOCAL_SPACE_IDENTIFIER = /\bME_(?:local|art)\w*\b/; + +/** + * the modes (see the module header) + * @ignore + * @internal + */ +export const LOCAL_MODE = { uv: 0, sprite: 1, chain: 2 }; + +/** + * The CPU mirror of the WGSL `MEBuiltins` block, in floats: `noise_uv`'s + * rect first (object size, image size, offset), then the local space. + * @ignore + * @internal + */ +export const ME_LAYOUT = { + sizeObj: 0, + sizeImg: 2, + offset: 4, + mode: 6, + ydir: 7, + localX: 8, + localY: 12, + toUV: 16, + art: 20, + // floats; 96 bytes + length: 24, +}; + +/** + * the uniforms both stages of a GLSL body read + * @ignore + * @internal + */ +const GLSL_SHARED = ["uniform float ME_mode;", "uniform float ME_ydir;"]; + +/** + * The vertex-stage declarations of a GLSL body using `noise_uv` and/or the + * local space. + * @param {{noiseUV: boolean, local: boolean}} builtins - what the body uses + * @returns {string[]} the lines + * @ignore + * @internal + */ +export function localVertexDeclarationsGLSL(builtins) { + return [ + ...GLSL_SHARED, + "uniform vec3 ME_local_x;", + "uniform vec3 ME_local_y;", + ...(builtins.local + ? [ + "uniform vec4 ME_art_rect;", + "varying vec2 ME_local;", + "varying vec2 ME_art;", + ] + : []), + ]; +} + +/** + * The vertex-stage statements: `ME_px`, the vertex in the renderable's own + * space, and the varyings that come from it. + * @param {{noiseUV: boolean, local: boolean}} builtins - what the body uses + * @returns {string[]} the lines + * @ignore + * @internal + */ +export function localVertexStatementsGLSL(builtins) { + return [ + " vec3 ME_v = vec3(aVertex.xy, 1.0);", + " vec2 ME_px = vec2(dot(ME_local_x, ME_v), dot(ME_local_y, ME_v));", + ...(builtins.local + ? [ + " ME_local = ME_mode > 0.5 ? ME_px : vec2(aRegion.x, ME_ydir < 0.0 ? 1.0 - aRegion.y : aRegion.y);", + " ME_art = ME_mode > 1.5 ? ME_art_rect.xy + ME_px * ME_art_rect.zw : aRegion;", + ] + : []), + ]; +} + +/** + * the `noise_uv` statement: across the renderable on a pass of its chain, + * the frame's own rect otherwise (as it always was) + * @ignore + * @internal + */ +export const NOISE_UV_GLSL = + " noise_uv = ME_mode > 1.5 ? ME_px : aRegion * (ME_size_img / ME_size_obj) - ME_offset / ME_size_obj;"; + +/** + * The fragment-stage declarations and helpers of a GLSL body reading the + * local space. + * @ignore + * @internal + */ +export const LOCAL_FRAGMENT_GLSL = [ + ...GLSL_SHARED, + "uniform vec4 ME_to_uv;", + "uniform vec4 ME_art_rect;", + "varying vec2 ME_local;", + "varying vec2 ME_art;", + "vec2 ME_localToUV(vec2 d) {", + " if (ME_mode > 0.5) {", + " return mat2(ME_to_uv.xy, ME_to_uv.zw) * d;", + " }", + " return vec2(d.x, ME_ydir < 0.0 ? -d.y : d.y);", + "}", + "vec2 ME_artToUV(vec2 d) {", + " return ME_mode > 1.5 ? mat2(ME_to_uv.xy, ME_to_uv.zw) * (d / ME_art_rect.zw) : d;", + "}", + "vec2 ME_localToArt(vec2 d) {", + " return ME_mode > 1.5 ? d * ME_art_rect.zw : ME_localToUV(d);", + "}", +].join("\n"); + +/** + * The WGSL `MEBuiltins` struct (see ME_LAYOUT). + * @ignore + * @internal + */ +export const ME_STRUCT_WGSL = [ + "struct MEBuiltins {", + "\tsize_obj : vec2f,", + "\tsize_img : vec2f,", + "\toffset : vec2f,", + "\tmode : f32,", + "\tydir : f32,", + "\tlocal_x : vec4f,", + "\tlocal_y : vec4f,", + "\tto_uv : vec4f,", + "\tart : vec4f,", + "};", +]; + +/** + * the WGSL vertex-stage statements (see localVertexStatementsGLSL) + * @param {{noiseUV: boolean, local: boolean}} builtins - what the body uses + * @returns {string[]} the lines + * @ignore + * @internal + */ +export function localVertexStatementsWGSL(builtins) { + return [ + "\tlet meV = vec3f(aVertex.xy, 1.0);", + "\tlet mePx = vec2f(dot(ME.local_x.xyz, meV), dot(ME.local_y.xyz, meV));", + ...(builtins.local + ? [ + "\tout.vLocal = select(vec2f(aRegion.x, select(aRegion.y, 1.0 - aRegion.y, ME.ydir < 0.0)), mePx, ME.mode > 0.5);", + "\tout.vArt = select(aRegion, ME.art.xy + mePx * ME.art.zw, ME.mode > 1.5);", + ] + : []), + ...(builtins.noiseUV + ? [ + "\tout.vNoiseUV = select(aRegion * (ME.size_img / ME.size_obj) - ME.offset / ME.size_obj, mePx, ME.mode > 1.5);", + ] + : []), + ]; +} + +/** + * the WGSL module-scope privates and helpers of a body reading the local + * space + * @ignore + * @internal + */ +export const LOCAL_HELPERS_WGSL = [ + "var ME_local : vec2f;", + "var ME_art : vec2f;", + "fn ME_localToUV(d : vec2f) -> vec2f {", + "\tif (ME.mode > 0.5) {", + "\t\treturn mat2x2f(ME.to_uv.xy, ME.to_uv.zw) * d;", + "\t}", + "\treturn vec2f(d.x, select(d.y, -d.y, ME.ydir < 0.0));", + "}", + "fn ME_artToUV(d : vec2f) -> vec2f {", + "\tif (ME.mode > 1.5) {", + "\t\treturn mat2x2f(ME.to_uv.xy, ME.to_uv.zw) * (d / ME.art.zw);", + "\t}", + "\treturn d;", + "}", + "fn ME_localToArt(d : vec2f) -> vec2f {", + "\tif (ME.mode > 1.5) {", + "\t\treturn d * ME.art.zw;", + "\t}", + "\treturn ME_localToUV(d);", + "}", +]; + +/** + * What a mesh hosting a body reading the local space gives it: `uv`, as it + * always did, and displacements left as they are. A mesh has no renderable + * space of its own to offer (see mesh_splice.js). + * @ignore + * @internal + */ +export const LOCAL_MESH_SHIM_GLSL = [ + "vec2 ME_local;", + "vec2 ME_art;", + "vec2 ME_localToUV(vec2 d) { return d; }", + "vec2 ME_artToUV(vec2 d) { return d; }", + "vec2 ME_localToArt(vec2 d) { return d; }", +].join("\n"); + +/** + * the WGSL twin of LOCAL_MESH_SHIM_GLSL + * @ignore + * @internal + */ +export const LOCAL_MESH_SHIM_WGSL = [ + "var ME_local : vec2f;", + "var ME_art : vec2f;", + "fn ME_localToUV(d : vec2f) -> vec2f { return d; }", + "fn ME_artToUV(d : vec2f) -> vec2f { return d; }", + "fn ME_localToArt(d : vec2f) -> vec2f { return d; }", +].join("\n"); diff --git a/packages/melonjs/src/video/effects/mesh_splice.js b/packages/melonjs/src/video/effects/mesh_splice.js index 85604636c7..d222c61cb6 100644 --- a/packages/melonjs/src/video/effects/mesh_splice.js +++ b/packages/melonjs/src/video/effects/mesh_splice.js @@ -39,6 +39,12 @@ * @internal */ +import { + LOCAL_MESH_SHIM_GLSL, + LOCAL_MESH_SHIM_WGSL, + LOCAL_SPACE_IDENTIFIER, +} from "./local_space.js"; + /** * The identity hook each mesh shader declares, in both languages. Matching is * whitespace-tolerant because the shader pipeline reformats as it strips @@ -50,12 +56,20 @@ const HOOKS = [ pattern: /vec4\s+ME_effect\s*\(\s*vec4\s+c\s*,\s*vec2\s+uv\s*\)\s*\{[^}]*\}/, replacement: "vec4 ME_effect(vec4 c, vec2 uv) { return apply(c, uv); }", + // a mesh has no renderable space of its own to offer: a body + // reading one is given `uv`, as it always was (local_space.js) + localShim: LOCAL_MESH_SHIM_GLSL, + localReplacement: + "vec4 ME_effect(vec4 c, vec2 uv) { ME_local = uv; ME_art = uv; return apply(c, uv); }", }, { pattern: /fn\s+ME_effect\s*\(\s*c\s*:\s*vec4f\s*,\s*uv\s*:\s*vec2f\s*\)\s*->\s*vec4f\s*\{[^}]*\}/, replacement: "fn ME_effect(c : vec4f, uv : vec2f) -> vec4f { return apply(c, uv); }", + localShim: LOCAL_MESH_SHIM_WGSL, + localReplacement: + "fn ME_effect(c : vec4f, uv : vec2f) -> vec4f { ME_local = uv; ME_art = uv; return apply(c, uv); }", }, ]; @@ -160,10 +174,13 @@ export function spliceEffect(source, body) { if (typeof source !== "string" || typeof body !== "string") { return null; } + const local = LOCAL_SPACE_IDENTIFIER.test(body); for (const hook of HOOKS) { if (hook.pattern.test(source)) { return source.replace(hook.pattern, () => { - return `${body}\n${hook.replacement}`; + return local + ? `${hook.localShim}\n${body}\n${hook.localReplacement}` + : `${body}\n${hook.replacement}`; }); } } diff --git a/packages/melonjs/src/video/effects/radialGradient.js b/packages/melonjs/src/video/effects/radialGradient.js index 77e1b362b8..bb36ba4806 100644 --- a/packages/melonjs/src/video/effects/radialGradient.js +++ b/packages/melonjs/src/video/effects/radialGradient.js @@ -15,18 +15,10 @@ import ShaderEffect from "./shadereffect.js"; * fading linearly to fully transparent at the edge of the host quad. * The falloff is naturally elliptical for non-square quads. * - * **UV-space caveat.** The `apply(color, uv)` function receives - * `vRegion` — the atlas UVs of the host quad. The falloff math - * (`length(uv * 2 - 1)`) assumes those UVs span `[0, 1] × [0, 1]`, - * which is true when the quad samples a full-rect texture (a - * dedicated 1×1 white pixel, a non-atlased Sprite, an FBO blit, or - * the engine-provided light atlas used by `WebGLRenderer.drawLight`). - * If you attach this effect to a Sprite that uses a *sub-region* of a - * larger atlas, `uv` will be in `[u0..u1] × [v0..v1]` and the radial - * center will be misplaced. For atlas-based renderables, set the - * effect's `uColor`/`uIntensity` and pair it with a Sprite whose - * texture covers a full atlas, or use a Sprite created from a - * standalone image. + * It is centred on the renderable it draws, wherever that sits in its + * texture: a dedicated 1×1 white pixel, a standalone image, a sprite cut + * from an atlas, or the engine-provided light atlas used by + * `WebGLRenderer.drawLight`. * * The falloff curve is **linear** (`f = clamp(1 - d, 0, 1)`) to match * the Canvas 2D `createRadialGradient` two-stop output exactly. Output @@ -102,7 +94,7 @@ export default class RadialGradientEffect extends ShaderEffect { // recenter to [-1, 1] across the quad. The quad's own aspect // ratio handles elliptical falloffs naturally — length(c) == 1 // lies on the inscribed ellipse in world space. - vec2 c = uv * 2.0 - 1.0; + vec2 c = ME_local * 2.0 - 1.0; float d = length(c); // linear ramp matches Canvas createRadialGradient's two-stop output float f = clamp(1.0 - d, 0.0, 1.0); @@ -129,7 +121,7 @@ export default class RadialGradientEffect extends ShaderEffect { @group(3) @binding(0) var fx : RadialGradientUniforms; fn apply(color : vec4f, uv : vec2f) -> vec4f { - let c = uv * 2.0 - vec2f(1.0); + let c = ME_local * 2.0 - vec2f(1.0); let d = length(c); let f = clamp(1.0 - d, 0.0, 1.0); let k = fx.uIntensity * f * fx.uColor.a; diff --git a/packages/melonjs/src/video/effects/scanline.js b/packages/melonjs/src/video/effects/scanline.js index a6fe1f72cc..a774fbf2aa 100644 --- a/packages/melonjs/src/video/effects/scanline.js +++ b/packages/melonjs/src/video/effects/scanline.js @@ -19,26 +19,31 @@ struct ScanlineUniforms { fn apply(color : vec4f, uv : vec2f) -> vec4f { var c = color; - var coords = uv; + // where on the sprite (curved around its own centre), and the lines of + // its texture there + var spot = ME_local; + var art = ME_art; // barrel distortion (CRT curvature) if (fx.uCurvature > 0.0) { - var centered = uv * 2.0 - 1.0; + var centered = ME_local * 2.0 - 1.0; centered *= 1.0 + fx.uCurvature * dot(centered, centered); - coords = centered * 0.5 + 0.5; - if (coords.x < 0.0 || coords.x > 1.0 || coords.y < 0.0 || coords.y > 1.0) { + spot = centered * 0.5 + 0.5; + if (spot.x < 0.0 || spot.x > 1.0 || spot.y < 0.0 || spot.y > 1.0) { discard; } - c = textureSampleLevel(uTexture, uSampler, coords, 0.0) * vColor; + let shift = spot - ME_local; + art = ME_art + ME_localToArt(shift); + c = textureSampleLevel(uTexture, uSampler, uv + ME_localToUV(shift), 0.0) * vColor; } // scanlines - let line = sin(coords.y * 800.0) * 0.5 + 0.5; + let line = sin(art.y * 800.0) * 0.5 + 0.5; c = vec4f(c.rgb * (1.0 - line * fx.uScanlineOpacity), c.a); // vignette if (fx.uVignetteStrength > 0.0) { - let vig = coords * (1.0 - coords); + let vig = spot * (1.0 - spot); let vigFactor = vig.x * vig.y * 15.0; c = vec4f(c.rgb * clamp(pow(vigFactor, fx.uVignetteStrength), 0.0, 1.0), c.a); } @@ -78,26 +83,31 @@ export default class ScanlineEffect extends ShaderEffect { uniform float uCurvature; uniform float uVignetteStrength; vec4 apply(vec4 color, vec2 uv) { - vec2 coords = uv; + // where on the sprite (curved around its own centre), and the + // lines of its texture there + vec2 spot = ME_local; + vec2 art = ME_art; // barrel distortion (CRT curvature) if (uCurvature > 0.0) { - vec2 centered = uv * 2.0 - 1.0; + vec2 centered = ME_local * 2.0 - 1.0; centered *= 1.0 + uCurvature * dot(centered, centered); - coords = centered * 0.5 + 0.5; - if (coords.x < 0.0 || coords.x > 1.0 || coords.y < 0.0 || coords.y > 1.0) { + spot = centered * 0.5 + 0.5; + if (spot.x < 0.0 || spot.x > 1.0 || spot.y < 0.0 || spot.y > 1.0) { discard; } - color = texture2D(uSampler, coords) * vColor; + vec2 shift = spot - ME_local; + art = ME_art + ME_localToArt(shift); + color = texture2D(uSampler, uv + ME_localToUV(shift)) * vColor; } // scanlines - float line = sin(coords.y * 800.0) * 0.5 + 0.5; + float line = sin(art.y * 800.0) * 0.5 + 0.5; color.rgb *= 1.0 - line * uScanlineOpacity; // vignette if (uVignetteStrength > 0.0) { - vec2 vig = coords * (1.0 - coords); + vec2 vig = spot * (1.0 - spot); float vigFactor = vig.x * vig.y * 15.0; color.rgb *= clamp(pow(vigFactor, uVignetteStrength), 0.0, 1.0); } diff --git a/packages/melonjs/src/video/effects/shadereffect.js b/packages/melonjs/src/video/effects/shadereffect.js index fce6d38d0d..48da213923 100644 --- a/packages/melonjs/src/video/effects/shadereffect.js +++ b/packages/melonjs/src/video/effects/shadereffect.js @@ -7,6 +7,7 @@ import { import Texture2d from "../texture/texture2d.ts"; import GLShader from "../webgl/glshader.js"; import { buildGLSLProgram } from "./glsl_realization.js"; +import { LOCAL_MODE, ME_LAYOUT } from "./local_space.js"; import WGSLEffectRealization from "./wgsl_realization.js"; /** @@ -136,7 +137,8 @@ const UNBOUNDED_FRAME = [-1e6, -1e6, 1e6, 1e6]; * // `: screen_texture` keeps the sampler filled with everything drawn so * // far (a back-buffer copy; one screen copy per draw of the effect), * // `screen_uv` is this fragment's 0..1 screen position, and `noise_uv` - * // runs 0..1 across the sprite regardless of its atlas frame. + * // runs 0..1 across the sprite regardless of its atlas frame (and + * // across the renderable when it has several effects). * const water = new ShaderEffect(renderer, ` * uniform sampler2D uNoise; * uniform sampler2D screenTex : screen_texture; @@ -149,6 +151,21 @@ const UNBOUNDED_FRAME = [-1e6, -1e6, 1e6, 1e6]; * `); * water.setTexture("uNoise", noiseTexture.getTexture(), "repeat"); */ +/** + * whether `values` already sit in the float mirror at `at`, as the mirror + * holds them (32-bit floats) + * @ignore + * @internal + */ +function sameFloats(mirror, at, values) { + for (let i = 0; i < values.length; i++) { + if (mirror[at + i] !== Math.fround(values[i])) { + return false; + } + } + return true; +} + /** * monotonic id, so a mesh batcher can key its spliced program on the effect * that produced it without holding a reference in the cache key (#1658) @@ -311,6 +328,13 @@ export default class ShaderEffect { * @internal */ this._hasNoiseUV = program.noiseUV; + /** + * whether the body reads where it is on the renderable it draws: + * `noise_uv`, or the local space (local_space.js) + * @ignore + * @internal + */ + this._hasLocalSpace = program.noiseUV || program.builtins.local; /** * the annotation-stripped body and its builtin usage, kept so the * mesh batchers can splice this effect into the engine's own mesh @@ -377,6 +401,8 @@ export default class ShaderEffect { ] : []; this._hasNoiseUV = realization.builtins.noiseUV; + this._hasLocalSpace = + realization.builtins.noiseUV || realization.builtins.local; this.enabled = true; } @@ -606,7 +632,30 @@ export default class ShaderEffect { * @internal */ _setUVYDir(dir) { - if (this._uvYDir === dir || this.destroyed === true) { + if (this.destroyed === true) { + return; + } + // the local space's own copy, which `ME_local` runs top to bottom by + // (local_space.js) + if (this._hasLocalSpace === true && this._meYDir !== dir) { + /** + * @ignore + * @internal + */ + this._meYDir = dir; + if (typeof this.wgslRealization !== "undefined") { + const me = this.wgslRealization.meMirror; + this.wgslRealization.onUniformChange?.(); + me[ME_LAYOUT.ydir] = dir; + } else if ( + typeof this._shader !== "undefined" && + !this._shader.suspended && + typeof this._shader.uniforms.ME_ydir !== "undefined" + ) { + this._shader.setUniform("ME_ydir", dir); + } + } + if (this._uvYDir === dir) { return; } if ( @@ -629,6 +678,81 @@ export default class ShaderEffect { } } + /** + * Tell a body reading where it is on the renderable it draws (`noise_uv`, + * or the local space: see local_space.js) which renderable that is, for + * the draws about to happen. `null` puts it back to what `uv` always + * was. Called by the renderer around a renderable drawn with its one + * effect, and by the batchers around each pass of a renderable's chain; + * a no-op for any other body. Only a change is written, so draws with + * the same space keep sharing a draw. + * @param {{mode: number, localX: number[], localY: number[], toUV: number[], art: number[]}|null} space - the mode (see LOCAL_MODE), the target-pixel to 0..1 rows, the local to `uv` displacement (two columns) and the renderable's frame in its texture (origin, size); `null` for none + * @ignore + * @internal + */ + _setLocalSpace(space) { + if (this._hasLocalSpace !== true || this.destroyed === true) { + return; + } + const mode = space === null ? LOCAL_MODE.uv : space.mode; + if (typeof this.wgslRealization !== "undefined") { + const realization = this.wgslRealization; + const me = realization.meMirror; + if (space === null) { + if (me[ME_LAYOUT.mode] !== mode) { + realization.onUniformChange?.(); + me[ME_LAYOUT.mode] = mode; + } + return; + } + const L = ME_LAYOUT; + const same = + me[L.mode] === mode && + sameFloats(me, L.localX, space.localX) && + sameFloats(me, L.localY, space.localY) && + sameFloats(me, L.toUV, space.toUV) && + sameFloats(me, L.art, space.art); + if (same) { + return; + } + // the quads queued under the old space draw first (the mirror is + // uploaded when they draw) + realization.onUniformChange?.(); + me[L.mode] = mode; + me.set(space.localX, L.localX); + me.set(space.localY, L.localY); + me.set(space.toUV, L.toUV); + me.set(space.art, L.art); + return; + } + const shader = this._shader; + if (typeof shader === "undefined" || shader.suspended) { + return; + } + // guarded on the ACTIVE uniforms: the compiler drops what the body + // ends up not reading, and setUniform throws on those. GLShader only + // draws the queued quads first when a value changes + const uniforms = shader.uniforms; + if (typeof uniforms.ME_mode !== "undefined") { + shader.setUniform("ME_mode", mode); + } + if (space === null) { + return; + } + if (typeof uniforms.ME_local_x !== "undefined") { + shader.setUniform("ME_local_x", space.localX); + } + if (typeof uniforms.ME_local_y !== "undefined") { + shader.setUniform("ME_local_y", space.localY); + } + if (typeof uniforms.ME_to_uv !== "undefined") { + shader.setUniform("ME_to_uv", space.toUV); + } + if (typeof uniforms.ME_art_rect !== "undefined") { + shader.setUniform("ME_art_rect", space.art); + } + } + /** * Hand a body declaring `ME_frame` (see frame.js) the half texel of the * texture the quad about to draw comes from, which `frameTexel()` keeps diff --git a/packages/melonjs/src/video/effects/shine.js b/packages/melonjs/src/video/effects/shine.js index b43d96c730..63fe03ad75 100644 --- a/packages/melonjs/src/video/effects/shine.js +++ b/packages/melonjs/src/video/effects/shine.js @@ -21,7 +21,6 @@ struct ShineUniforms { uPulseDepth : f32, uPulseSpeed : f32, uTime : f32, - uUVYDir : f32, }; @group(3) @binding(0) var fx : ShineUniforms; @@ -31,13 +30,14 @@ fn apply(color : vec4f, uv : vec2f) -> vec4f { } // Optional brightness pulse on the base color. let pulse = (1.0 - fx.uPulseDepth) + fx.uPulseDepth * sin(fx.uTime * fx.uPulseSpeed); - // Directional UV arithmetic — an ABSOLUTE coordinate, so a flipped - // space mirrors the position (1 - uv.y), not merely the sign (see the - // ShaderEffect uUVYDir docs; the renderer feeds the space's direction) - let y = select(uv.y, 1.0 - uv.y, fx.uUVYDir < 0.0); - // Project uv along the sweep axis; tile by uShineBands (wrap-around - // distance keeps sweeps gapless — see the GLSL twin's rationale). - let pos = uv.x * cos(fx.uShineAngle) + y * sin(fx.uShineAngle); + // Project the coordinate in its texture along the sweep axis, as it + // always was (on an atlas sprite, across the atlas), top to bottom on + // every path: a capture the camera's effects read runs bottom-up on + // WebGL. Tile by uShineBands (wrap-around distance keeps sweeps + // gapless — see the GLSL twin's rationale). + let art = ME_art; + let y = select(art.y, 1.0 - art.y, ME.mode < 0.5 && ME.ydir < 0.0); + let pos = art.x * cos(fx.uShineAngle) + y * sin(fx.uShineAngle); let localX = fract(pos * fx.uShineBands); let sweep = fract(fx.uTime * fx.uShineSpeed); let d = abs(localX - sweep); @@ -59,6 +59,11 @@ fn apply(color : vec4f, uv : vec2f) -> vec4f { * "etched grooves" look of a coin's rim). An optional subtle brightness * pulse can be layered on top via `pulseDepth` — set to 0 to disable. * + * The sweep, its bands and its width are measured across the texture the + * sprite is drawn from: on a sprite cut from an atlas, across the atlas, so + * a single sweep crosses the sprite only part of the time. With other + * effects after it, it stays on the sprite. + * * The `time` uniform must be updated each frame for the animation. * @category Effects * @see {@link Renderable#addPostEffect} for usage @@ -114,18 +119,18 @@ export default class ShineEffect extends ShaderEffect { uniform float uPulseDepth; uniform float uPulseSpeed; uniform float uTime; - uniform float uUVYDir; vec4 apply(vec4 color, vec2 uv) { if (color.a == 0.0) return color; // Optional brightness pulse on the base color. float pulse = (1.0 - uPulseDepth) + uPulseDepth * sin(uTime * uPulseSpeed); - // Directional UV arithmetic — an ABSOLUTE coordinate, so a - // flipped space mirrors the position (1.0 - uv.y), not just - // the sign (see the ShaderEffect uUVYDir docs); without it, - // a vertical sweep runs backwards on bottom-up captures - float y = uUVYDir < 0.0 ? 1.0 - uv.y : uv.y; - // Project uv along the sweep axis (uShineAngle). - float pos = uv.x * cos(uShineAngle) + y * sin(uShineAngle); + // Project the coordinate in its texture along the sweep axis + // (uShineAngle), as it always was: on an atlas sprite, across + // the atlas. Top to bottom on every path: a capture the + // camera's effects read runs bottom-up on WebGL, and with + // several effects the coordinate is the sprite's frame + vec2 art = ME_art; + float y = ME_mode < 0.5 && ME_ydir < 0.0 ? 1.0 - art.y : art.y; + float pos = art.x * cos(uShineAngle) + y * sin(uShineAngle); // Tile by uShineBands so we get N parallel glints in unison. // localX and sweep both live in [0,1] tile-space; the // wrap-around distance (min(d, 1-d)) lets the glint exit @@ -160,9 +165,6 @@ export default class ShineEffect extends ShaderEffect { this.setUniform("uPulseDepth", options.pulseDepth ?? 0.0); this.setUniform("uPulseSpeed", options.pulseSpeed ?? 3.0); this.setUniform("uTime", 0.0); - // top-down default; the batchers feed the actual space direction - // per draw through _setUVYDir (bottom-up pooled captures on GL) - this.setUniform("uUVYDir", 1.0); } /** diff --git a/packages/melonjs/src/video/effects/sway.js b/packages/melonjs/src/video/effects/sway.js index ac601a58dd..dbeaef1d31 100644 --- a/packages/melonjs/src/video/effects/sway.js +++ b/packages/melonjs/src/video/effects/sway.js @@ -15,8 +15,13 @@ const TURN = Math.PI * 2; * sways most and everything from the anchor down stays put. For grass, trees, * reeds or a whole background layer: anything planted at its bottom. * + * Everything is measured across the sprite itself: its strength is a + * fraction of the sprite's width and its anchor a row of the sprite, so a + * sprite sways the same whether it has a texture of its own or is cut from + * an atlas, and whatever else it draws through. + * * It is a {@link WaveEffect} with an anchor and sway-sized defaults, so it - * shares its shader rather than adding one. The difference is the time: + * shares its shader rather than adding one. The other difference is the time: * {@link SwayEffect#setTime} wraps the phase to one turn on the CPU before * the shader sees it. A raw clock grows without bound, and a medium-precision * float, the default on most phones, runs out of digits a few minutes into a @@ -34,8 +39,8 @@ export default class SwayEffect extends WaveEffect { /** * @param {Renderer} renderer - the current renderer instance * @param {object} [options] - effect options - * @param {number} [options.strength=0.02] - how far the top moves each way, as a fraction of the texture's width - * @param {number} [options.anchor=1.0] - the row that stays put, as a fraction of the height from the top (1 = the bottom row) + * @param {number} [options.strength=0.02] - how far the top moves each way, as a fraction of the sprite's width + * @param {number} [options.anchor=1.0] - the row of the sprite that stays put, as a fraction of its height from the top (1 = the bottom row) * @param {number} [options.speed=1.5] - how fast it sways, in radians per second * @param {number} [options.frequency=3.0] - how much the sway bends along the height: low values lean the whole sprite as one, high ones ripple it */ @@ -53,6 +58,8 @@ export default class SwayEffect extends WaveEffect { * @type {number} */ this.speed = options.speed ?? 1.5; + // across the sprite, not the texture it is drawn from + this.setUniform("uAcrossSprite", 1); } /** @@ -67,7 +74,7 @@ export default class SwayEffect extends WaveEffect { /** * set how far the top moves each way - * @param {number} strength - a fraction of the texture's width + * @param {number} strength - a fraction of the sprite's width */ setStrength(strength) { this.setAmplitude(strength); diff --git a/packages/melonjs/src/video/effects/vignette.js b/packages/melonjs/src/video/effects/vignette.js index 0248283f8d..c29e231e6e 100644 --- a/packages/melonjs/src/video/effects/vignette.js +++ b/packages/melonjs/src/video/effects/vignette.js @@ -14,7 +14,8 @@ const fragment = ` uniform float uStrength; uniform float uSize; vec4 apply(vec4 color, vec2 uv) { - vec2 vig = uv * (1.0 - uv); + // across the sprite, wherever it sits in its texture + vec2 vig = ME_local * (1.0 - ME_local); float v = clamp(pow(vig.x * vig.y * uSize, uStrength), 0.0, 1.0); return vec4(color.rgb * v, color.a); } @@ -28,7 +29,8 @@ struct VignetteUniforms { @group(3) @binding(0) var fx : VignetteUniforms; fn apply(color : vec4f, uv : vec2f) -> vec4f { - let vig = uv * (1.0 - uv); + // across the sprite, wherever it sits in its texture + let vig = ME_local * (1.0 - ME_local); let v = clamp(pow(vig.x * vig.y * fx.uSize, fx.uStrength), 0.0, 1.0); return vec4f(color.rgb * v, color.a); } diff --git a/packages/melonjs/src/video/effects/wave.js b/packages/melonjs/src/video/effects/wave.js index 5fdfc46f42..4096767d1f 100644 --- a/packages/melonjs/src/video/effects/wave.js +++ b/packages/melonjs/src/video/effects/wave.js @@ -16,16 +16,21 @@ struct WaveUniforms { uFrequency : f32, uSpeed : f32, uTime : f32, - uAnchor : f32,${FRAME_WGSL_MEMBERS} + uAnchor : f32, + uAcrossSprite : f32,${FRAME_WGSL_MEMBERS} }; @group(3) @binding(0) var fx : WaveUniforms; ${FRAME_WGSL} fn apply(color : vec4f, uv : vec2f) -> vec4f { - // with an anchor, rows fade from full wave at the top to none at the anchor - let bend = select(1.0, clamp((fx.uAnchor - uv.y) / fx.uAnchor, 0.0, 1.0), fx.uAnchor > 0.0); - let wave = sin(uv.y * fx.uFrequency + fx.uTime * fx.uSpeed) * fx.uAmplitude * bend; - let distorted = vec2f(uv.x + wave, uv.y); - return frameTexel(distorted) * vColor; + // with an anchor, rows fade from full wave at the top to none at the + // anchor: rows of the sprite, wherever it sits in its texture + let bend = select(1.0, clamp((fx.uAnchor - ME_local.y) / fx.uAnchor, 0.0, 1.0), fx.uAnchor > 0.0); + // a sway measures its wave across the sprite, a wave across its texture + let across = fx.uAcrossSprite > 0.5; + let along = select(ME_art.y, ME_local.y, across); + let wave = sin(along * fx.uFrequency + fx.uTime * fx.uSpeed) * fx.uAmplitude * bend; + let shift = select(ME_artToUV(vec2f(wave, 0.0)), ME_localToUV(vec2f(wave, 0.0)), across); + return frameTexel(uv + shift) * vColor; } `; @@ -36,9 +41,11 @@ fn apply(color : vec4f, uv : vec2f) -> vec4f { * * With an `anchor`, the wave fades out toward a row instead of moving the * whole sprite: full at the top, nothing at the anchor and below. That is - * how {@link SwayEffect} leans a sprite while its feet stay planted. Rows - * are measured across the texture the effect samples, so on a sprite cut - * from an atlas the anchor is a row of the atlas. + * how {@link SwayEffect} leans a sprite while its feet stay planted. The + * anchor is a row of the sprite, wherever it sits in its texture; the + * amplitude and the frequency are measured across the texture, so on a + * sprite cut from an atlas they are fractions of the atlas ({@link SwayEffect} + * measures them across the sprite). * @category Effects * @see {@link Renderable#addPostEffect} for usage * @example @@ -55,10 +62,10 @@ export default class WaveEffect extends ShaderEffect { /** * @param {Renderer} renderer - the current renderer instance * @param {object} [options] - effect options - * @param {number} [options.amplitude=0.01] - wave displacement strength (in UV space, 0.01 = subtle) - * @param {number} [options.frequency=10.0] - number of waves across the sprite + * @param {number} [options.amplitude=0.01] - wave displacement strength, as a fraction of the texture's width (0.01 = subtle) + * @param {number} [options.frequency=10.0] - the wave's frequency down the texture's height, in radians per height (2π is one whole wave) * @param {number} [options.speed=2.0] - wave animation speed - * @param {number} [options.anchor=0] - the row the wave fades out at, as a fraction of the height from the top (0..1); 0 waves every row alike + * @param {number} [options.anchor=0] - the row of the sprite the wave fades out at, as a fraction of its height from the top (0..1); 0 waves every row alike */ constructor(renderer, options = {}) { super(renderer, { @@ -68,13 +75,20 @@ export default class WaveEffect extends ShaderEffect { uniform float uSpeed; uniform float uTime; uniform float uAnchor; + uniform float uAcrossSprite; ${FRAME_GLSL} vec4 apply(vec4 color, vec2 uv) { - // with an anchor, rows fade from full wave at the top to none at the anchor - float bend = uAnchor > 0.0 ? clamp((uAnchor - uv.y) / uAnchor, 0.0, 1.0) : 1.0; - float wave = sin(uv.y * uFrequency + uTime * uSpeed) * uAmplitude * bend; - vec2 distorted = vec2(uv.x + wave, uv.y); - return frameTexel(distorted) * vColor; + // with an anchor, rows fade from full wave at the top to none + // at the anchor: rows of the sprite, wherever it sits in its + // texture + float bend = uAnchor > 0.0 ? clamp((uAnchor - ME_local.y) / uAnchor, 0.0, 1.0) : 1.0; + // a sway measures its wave across the sprite, a wave across + // its texture + bool across = uAcrossSprite > 0.5; + float along = across ? ME_local.y : ME_art.y; + float wave = sin(along * uFrequency + uTime * uSpeed) * uAmplitude * bend; + vec2 shift = across ? ME_localToUV(vec2(wave, 0.0)) : ME_artToUV(vec2(wave, 0.0)); + return frameTexel(uv + shift) * vColor; } `, wgsl: wgslFragment, @@ -85,6 +99,9 @@ export default class WaveEffect extends ShaderEffect { this.setUniform("uSpeed", options.speed ?? 2.0); this.setUniform("uTime", 0.0); this.setUniform("uAnchor", options.anchor ?? 0); + // measured across the texture, as a wave always has been (a sway + // measures across the sprite: see SwayEffect) + this.setUniform("uAcrossSprite", 0); } /** @@ -105,7 +122,7 @@ export default class WaveEffect extends ShaderEffect { /** * set the row the wave fades out at - * @param {number} anchor - a fraction of the height from the top (0..1); 0 waves every row alike + * @param {number} anchor - a fraction of the sprite's height from the top (0..1); 0 waves every row alike */ setAnchor(anchor) { this.setUniform("uAnchor", anchor); diff --git a/packages/melonjs/src/video/effects/wgsl/parse.js b/packages/melonjs/src/video/effects/wgsl/parse.js index 5ba38929dc..1f1d9e0cb5 100644 --- a/packages/melonjs/src/video/effects/wgsl/parse.js +++ b/packages/melonjs/src/video/effects/wgsl/parse.js @@ -1,3 +1,4 @@ +import { LOCAL_SPACE_IDENTIFIER } from "../local_space.js"; import { computeUniformLayout } from "./layout.js"; /** @@ -265,6 +266,8 @@ export function parseWGSLBody(body) { // the interpolated tint (the GLSL `vColor` varying) — bodies that // re-sample the source texture reference it to re-apply the tint vColor: /\bvColor\b/.test(source) && !hasOwnDeclaration(source, "vColor"), + // where the body is on the renderable it draws (local_space.js) + local: LOCAL_SPACE_IDENTIFIER.test(source), }; if ( builtins.screenTexture && diff --git a/packages/melonjs/src/video/effects/wgsl/scaffold.js b/packages/melonjs/src/video/effects/wgsl/scaffold.js index 2b74ba51a4..0e457ebd85 100644 --- a/packages/melonjs/src/video/effects/wgsl/scaffold.js +++ b/packages/melonjs/src/video/effects/wgsl/scaffold.js @@ -15,6 +15,12 @@ * @internal */ +import { + LOCAL_HELPERS_WGSL, + localVertexStatementsWGSL, + ME_STRUCT_WGSL, +} from "../local_space.js"; + /** * reads of the frame uniform by frame.js's FRAME_WGSL helpers (not of * `fx.ME_frame_half`: no word boundary inside the longer name) @@ -45,7 +51,8 @@ export function assignBuiltinBindings(parsed) { screenSamplerClamp: -1, screenSamplerRepeat: -1, }; - if (parsed.builtins.noiseUV) { + // the `MEBuiltins` block: `noise_uv`'s rect and the local space + if (parsed.builtins.noiseUV || parsed.builtins.local) { bindings.me = next++; } if (parsed.builtins.screenTexture) { @@ -72,6 +79,8 @@ export function assignBuiltinBindings(parsed) { export function buildWGSLModule(body, parsed) { const builtins = parsed.builtins; const bindings = assignBuiltinBindings(parsed); + // `noise_uv` and the local space both read the `MEBuiltins` block + const space = builtins.noiseUV || builtins.local; // a body keeping its samples inside its frame (frame.js's FRAME_WGSL): // on a sprite the frame comes with the quad, so its reads of the // uniform become reads of the frame the vertex stage hands over. The @@ -104,16 +113,17 @@ export function buildWGSLModule(body, parsed) { scaffold.push("var ME_frame_v : vec4f;"); } if (builtins.noiseUV) { + scaffold.push("var noise_uv : vec2f;"); + } + if (space) { scaffold.push( - "var noise_uv : vec2f;", - "struct MEBuiltins {", - "\tsize_obj : vec2f,", - "\tsize_img : vec2f,", - "\toffset : vec2f,", - "};", + ...ME_STRUCT_WGSL, `@group(3) @binding(${bindings.me}) var ME : MEBuiltins;`, ); } + if (builtins.local) { + scaffold.push(...LOCAL_HELPERS_WGSL); + } if (builtins.screenTexture) { scaffold.push( `@group(3) @binding(${bindings.screenTexture}) var screen_texture : texture_2d;`, @@ -138,6 +148,9 @@ export function buildWGSLModule(body, parsed) { ...(builtins.screenUV ? ["\t@location(2) vScreenUV : vec2f,"] : []), ...(builtins.noiseUV ? ["\t@location(3) vNoiseUV : vec2f,"] : []), ...(spriteFrame ? ["\t@location(4) vFrame : vec4f,"] : []), + ...(builtins.local + ? ["\t@location(5) vLocal : vec2f,", "\t@location(6) vArt : vec2f,"] + : []), "};", "", "@vertex", @@ -161,11 +174,7 @@ export function buildWGSLModule(body, parsed) { "\tout.vScreenUV = vec2f(ndc.x * 0.5 + 0.5, 0.5 - ndc.y * 0.5);", ] : []), - ...(builtins.noiseUV - ? [ - "\tout.vNoiseUV = aRegion * (ME.size_img / ME.size_obj) - ME.offset / ME.size_obj;", - ] - : []), + ...(space ? localVertexStatementsWGSL(builtins) : []), ...(spriteFrame ? [ "\tout.vFrame = select(", @@ -185,6 +194,9 @@ export function buildWGSLModule(body, parsed) { "fn fragment_main(in : VSOut) -> @location(0) vec4f {", ...(builtins.screenUV ? ["\tscreen_uv = in.vScreenUV;"] : []), ...(builtins.noiseUV ? ["\tnoise_uv = in.vNoiseUV;"] : []), + ...(builtins.local + ? ["\tME_local = in.vLocal;", "\tME_art = in.vArt;"] + : []), ...(builtins.vColor ? ["\tvColor = in.vColor;"] : []), ...(spriteFrame ? [ diff --git a/packages/melonjs/src/video/effects/wgsl_realization.js b/packages/melonjs/src/video/effects/wgsl_realization.js index 0045eae498..2b6e97c998 100644 --- a/packages/melonjs/src/video/effects/wgsl_realization.js +++ b/packages/melonjs/src/video/effects/wgsl_realization.js @@ -1,3 +1,4 @@ +import { ME_LAYOUT } from "./local_space.js"; import { shiftGroup3, spliceEffect } from "./mesh_splice.js"; import { parseWGSLBody } from "./wgsl/parse.js"; import { buildWGSLModule } from "./wgsl/scaffold.js"; @@ -102,9 +103,10 @@ export default class WGSLEffectRealization { this.i32 = new Int32Array(this.cpu); this.u32 = new Uint32Array(this.cpu); - // CPU mirror of the engine `MEBuiltins` struct (noise_uv): size_obj - // at float 0, size_img at 2, offset at 4; 32 bytes with tail padding - this.meMirror = new Float32Array(8); + // CPU mirror of the engine `MEBuiltins` struct: the noise_uv rect + // (size_obj at float 0, size_img at 2, offset at 4) and the local + // space after it (see local_space.js for the layout) + this.meMirror = new Float32Array(ME_LAYOUT.length); /** * last user-set value per uniform name — the clone-replay source diff --git a/packages/melonjs/src/video/renderer.js b/packages/melonjs/src/video/renderer.js index 1cfcdbdda5..17fcbd7f43 100644 --- a/packages/melonjs/src/video/renderer.js +++ b/packages/melonjs/src/video/renderer.js @@ -6,6 +6,7 @@ import state from "../state/state.ts"; import { CANVAS_ONRESIZE, emit } from "../system/event.ts"; import { isAdvancedBlendMode } from "./blendmodes.js"; import { createCanvas } from "./canvas_factory.js"; +import { LOCAL_MODE } from "./effects/local_space.js"; import ToneMappingEffect, { CURVES } from "./effects/tonemap.js"; import { Gradient } from "./gradient.js"; import RenderState from "./renderstate.js"; @@ -108,6 +109,33 @@ export default class Renderer { */ this.customShader = undefined; + /** + * Where each renderable whose effect chain is under way is, for its + * passes to read (see `_setEffectSpace`); keyed by the renderable. + * @type {WeakMap} + * @ignore + * @internal + */ + this._effectSpaces = new WeakMap(); + + /** + * Where the renderable whose chain is being composited is, for the + * blits of that chain to hand its effects; `null` otherwise. + * @type {object|null} + * @ignore + * @internal + */ + this._chainSpace = null; + + /** + * Whether an effect pass's capture runs bottom-up (WebGL's + * framebuffers), which the space of a chain's passes accounts for. + * @type {boolean} + * @ignore + * @internal + */ + this._captureFlipsY = false; + /** * The render target pool for post-effect processing (ping-pong FBOs). * Initialized by GPU renderers (WebGL, WebGPU). Null on Canvas renderer. @@ -1163,6 +1191,212 @@ export default class Renderer { ); } + /** + * Tell a renderable's effects where it is, for those that place a + * pattern on it (see effects/local_space.js). Called at the end of + * `Renderable#preDraw`, once its own transform and anchor are applied, + * so the transform maps its own rectangle (`pos`, `width`, `height`) + * onto the target. + * + * Drawn with its one effect, the effect is told now, for the draws that + * follow; with a chain, its passes are, as each one is composited (see + * `_endEffectSpace`). A renderable this cannot place (no finite size, a + * perspective view, a transform that is not 2D) leaves its effects in + * the space `uv` always was. + * @param {Renderable} renderable - the renderable about to draw + * @ignore + * @internal + */ + _setEffectSpace(renderable) { + // effects only run where there are shaders (not on Canvas) + if (this.shaderLanguage === null) { + return; + } + const effects = this._effectChainFor(renderable); + if (effects.length === 0) { + return; + } + if (this._usesPostEffectFastPath(renderable, effects)) { + const space = this._effectSpaceOf(renderable, LOCAL_MODE.sprite); + if (space !== null) { + effects[0]._setLocalSpace?.(space); + } + return; + } + const space = this._effectSpaceOf(renderable, LOCAL_MODE.chain); + if (space !== null) { + this._effectSpaces.set(renderable, space); + } + } + + /** + * The other end of `_setEffectSpace`, from `endPostEffect`, for a + * renderable drawn with its one effect: that effect is put back to the + * space `uv` always was, for whatever draws with it next. + * @param {Renderable} renderable - the renderable done drawing + * @param {object[]} effects - its enabled effect chain + * @returns {boolean} true when it was drawn with its one effect + * @ignore + * @internal + */ + _endEffectSpace(renderable, effects) { + if (this._usesPostEffectFastPath(renderable, effects)) { + effects[0]._setLocalSpace?.(null); + return true; + } + return false; + } + + /** + * A chain is about to be composited: its blits hand its effects where + * the renderable is (`_chainSpace`), until `_clearChainSpace`. A + * camera's chain has no renderable space. + * @param {Renderable} renderable - the renderable whose chain it is + * @ignore + * @internal + */ + _beginChainSpace(renderable) { + this._chainSpace = renderable._postEffectManaged + ? null + : (this._effectSpaces.get(renderable) ?? null); + this._effectSpaces.delete(renderable); + } + + /** + * Done compositing a chain: blits are back to the space `uv` always was. + * @ignore + * @internal + */ + _clearChainSpace() { + this._chainSpace = null; + } + + /** + * Where a renderable is, as the effects that place a pattern on it read + * it (see effects/local_space.js): rows taking a target pixel to 0..1 + * across its rectangle, following its art; how a displacement there + * moves the coordinate the effect samples at; and, for a chain, its + * frame in its texture. `null` when it cannot be placed. + * @param {Renderable} renderable - the renderable + * @param {number} mode - LOCAL_MODE.sprite or LOCAL_MODE.chain + * @returns {object|null} the space, reused per renderable + * @ignore + * @internal + */ + _effectSpaceOf(renderable, mode) { + const w = renderable.width; + const h = renderable.height; + if (!Number.isFinite(w) || !Number.isFinite(h) || w === 0 || h === 0) { + return null; + } + // a perspective view (Camera3d) has no pixel grid to be placed on + if (this.projectionMatrix.val[11] !== 0) { + return null; + } + // target x and y must come from the 2D part alone: a 3D view + // matrix mixes the depth in + const v = this.currentTransform.val; + if (v[8] !== 0 || v[9] !== 0) { + return null; + } + const a = v[0]; + const b = v[1]; + const c = v[4]; + const d = v[5]; + const e = v[12]; + const f = v[13]; + const det = a * d - b * c; + if (Math.abs(det) < 1e-12) { + return null; + } + // the frame drawn, for a sprite: its texture, and where in it + const frame = renderable.current; + const image = renderable.image; + const sprite = + frame?.offset !== undefined && + image !== undefined && + image !== null && + renderable.offset !== undefined; + if (mode === LOCAL_MODE.sprite && !sprite) { + // drawn with its one effect, a renderable that is not a sprite + // samples what it draws in no way this can describe + return null; + } + + let space = renderable._effectSpace; + if (space === undefined) { + space = { + mode, + localX: [0, 0, 0], + localY: [0, 0, 0], + toUV: [0, 0, 0, 0], + art: [0, 0, 1, 1], + }; + /** + * @ignore + * @internal + */ + renderable._effectSpace = space; + } + space.mode = mode; + + // the inverse of the transform, then across the rectangle at `pos` + const iw = 1 / (det * w); + const ih = 1 / (det * h); + space.localX[0] = d * iw; + space.localX[1] = -c * iw; + space.localX[2] = (c * f - d * e) * iw - renderable.pos.x / w; + space.localY[0] = -b * ih; + space.localY[1] = a * ih; + space.localY[2] = (b * e - a * f) * ih - renderable.pos.y / h; + + let texW = 1; + let texH = 1; + if (sprite) { + texW = image.width || image.videoWidth || 1; + texH = image.height || image.videoHeight || 1; + } + const toUV = space.toUV; + if (mode === LOCAL_MODE.sprite) { + // across the sprite, in its texture: its frame's size, turned + // by the frame's own rotation in an atlas (see Sprite#draw) + const angle = frame.angle ?? 0; + const cos = Math.cos(-angle); + const sin = Math.sin(-angle); + toUV[0] = (w * cos) / texW; + toUV[1] = (w * sin) / texH; + toUV[2] = (-h * sin) / texW; + toUV[3] = (h * cos) / texH; + } else { + // across the renderable, in the capture: the transform, scaled + // to the capture (bottom-up on WebGL) + const canvas = this.getCanvas(); + const W = canvas.width || 1; + const H = canvas.height || 1; + const yDir = this._captureFlipsY ? -1 : 1; + toUV[0] = (a * w) / W; + toUV[1] = (yDir * b * w) / H; + toUV[2] = (c * h) / W; + toUV[3] = (yDir * d * h) / H; + } + + const art = space.art; + if (sprite) { + // the frame's whole rectangle (before any trim) in its texture + const trim = frame.trim; + art[0] = (renderable.offset.x + frame.offset.x - (trim?.x ?? 0)) / texW; + art[1] = (renderable.offset.y + frame.offset.y - (trim?.y ?? 0)) / texH; + art[2] = w / texW; + art[3] = h / texH; + } else { + art[0] = 0; + art[1] = 0; + art[2] = 1; + art[3] = 1; + } + return space; + } + /** * Begin capturing rendering to an offscreen buffer for post-effect processing. * Call endPostEffect() after rendering to blit the result to the screen. diff --git a/packages/melonjs/src/video/webgl/batchers/lit_quad_batcher.js b/packages/melonjs/src/video/webgl/batchers/lit_quad_batcher.js index 8c8dd70c2d..48eeb88743 100644 --- a/packages/melonjs/src/video/webgl/batchers/lit_quad_batcher.js +++ b/packages/melonjs/src/video/webgl/batchers/lit_quad_batcher.js @@ -720,6 +720,9 @@ export default class LitQuadBatcher extends QuadBatcher { // bottom-up capture FBO: uv.y grows upward inside apply() — see // QuadBatcher.blitTexture shader._setUVYDir?.(-1); + // a pass of a renderable's chain: where the renderable is, for the + // effects that place a pattern on it (local_space.js) + shader._setLocalSpace?.(this.renderer._chainSpace); // transform corners through the renderer transform — see // `QuadBatcher.blitTexture` for the rationale. Only caller today @@ -747,6 +750,8 @@ export default class LitQuadBatcher extends QuadBatcher { vertexData.pushSprite(vec3.x, vec3.y, 0, 1, 0, tint, 0, open, open, -1, 0); this.flush(); + // and back to the space `uv` always was, for whatever draws next + shader._setLocalSpace?.(null); // see QuadBatcher.blitTexture — unit 0 is nulled renderer-wide gl.activeTexture(gl.TEXTURE0); diff --git a/packages/melonjs/src/video/webgl/batchers/quad_batcher.js b/packages/melonjs/src/video/webgl/batchers/quad_batcher.js index 1f4767a0a2..ab33a8333b 100644 --- a/packages/melonjs/src/video/webgl/batchers/quad_batcher.js +++ b/packages/melonjs/src/video/webgl/batchers/quad_batcher.js @@ -272,6 +272,9 @@ export default class QuadBatcher extends MaterialBatcher { // below): uv.y grows UPWARD inside apply(), so directional bodies // (DropShadow) flip their y arithmetic to keep "down" down shader._setUVYDir?.(-1); + // a pass of a renderable's chain: where the renderable is, for the + // effects that place a pattern on it (local_space.js) + shader._setLocalSpace?.(this.renderer._chainSpace); // (re)bind any extra textures a ShaderEffect declared via setTexture, // to their reserved high units — after the source claims unit 0 @@ -306,6 +309,8 @@ export default class QuadBatcher extends MaterialBatcher { vertexData.pushSprite(vec3.x, vec3.y, 0, 1, 0, tint, 0, open, open); this.flush(); + // and back to the space `uv` always was, for whatever draws next + shader._setLocalSpace?.(null); // unbind the texture to prevent feedback loop on next frame, and drop // EVERY batcher's unit-0 slot — unit 0's GL binding is now null, and a diff --git a/packages/melonjs/src/video/webgl/webgl_renderer.js b/packages/melonjs/src/video/webgl/webgl_renderer.js index a6861f9c93..2205a2bcdb 100644 --- a/packages/melonjs/src/video/webgl/webgl_renderer.js +++ b/packages/melonjs/src/video/webgl/webgl_renderer.js @@ -426,6 +426,9 @@ export default class WebGLRenderer extends Renderer { */ this.currentTransform = this.renderState.currentTransform; + // an effect pass's capture is a framebuffer, read bottom-up + this._captureFlipsY = true; + /** * The current batcher used by the renderer * @type {WebGLBatcher} @@ -2144,8 +2147,9 @@ export default class WebGLRenderer extends Renderer { if (effects.length === 0) { return; } - // single effect on non-managed renderable used customShader — no FBO to unbind - if (this._usesPostEffectFastPath(renderable, effects)) { + // single effect on non-managed renderable used customShader — no FBO + // to unbind; its effect is put back to the space `uv` always was + if (this._endEffectSpace(renderable, effects)) { return; } @@ -2209,6 +2213,8 @@ export default class WebGLRenderer extends Renderer { this.toFrameTexture(); } + // the chain's passes are told where the renderable is + this._beginChainSpace(renderable); if (effects.length === 1) { this.blitEffect(rt1.texture, 0, 0, w, h, effects[0], keepBlend); } else { @@ -2247,6 +2253,7 @@ export default class WebGLRenderer extends Renderer { if (isCamera && renderable.isDefault === false) { this.disableScissor(); } + this._clearChainSpace(); // restore renderer state and projection saved in beginPostEffect this.restore(); diff --git a/packages/melonjs/src/video/webgpu/batchers/quad_batcher.js b/packages/melonjs/src/video/webgpu/batchers/quad_batcher.js index fe2fd52b05..8f1ea05449 100644 --- a/packages/melonjs/src/video/webgpu/batchers/quad_batcher.js +++ b/packages/melonjs/src/video/webgpu/batchers/quad_batcher.js @@ -569,6 +569,9 @@ export default class WebGPUQuadBatcher extends WebGPUBatcher { // and the whole texture is the frame: sampling stays unbounded. The // texture is the pass's render target, the size of the quad effect?._clearFrameRect?.(source.width ?? w, source.height ?? h); + // a pass of a renderable's chain: where the renderable is, for the + // effects that place a pattern on it (local_space.js) + effect?._setLocalSpace?.(renderer._chainSpace); const binding = effect ? prepareEffectBinding(renderer, effect) : null; const pass = renderer.ensurePass(); @@ -618,6 +621,8 @@ export default class WebGPUQuadBatcher extends WebGPUBatcher { pass.setIndexBuffer(this.indexBuffer, "uint32"); pass.drawIndexed(6); vertexData.clear(); + // drawn, its values uploaded: back to the space `uv` always was + effect?._setLocalSpace?.(null); } /** diff --git a/packages/melonjs/src/video/webgpu/effect_binding.js b/packages/melonjs/src/video/webgpu/effect_binding.js index 69ee4116c5..4eedfc9013 100644 --- a/packages/melonjs/src/video/webgpu/effect_binding.js +++ b/packages/melonjs/src/video/webgpu/effect_binding.js @@ -15,6 +15,8 @@ * @internal */ +import { ME_LAYOUT } from "../effects/local_space.js"; + /** * round up to a power-of-two alignment * @ignore @@ -25,11 +27,12 @@ function alignUp(value, alignment) { } /** - * byte size of the engine MEBuiltins struct (3×vec2f, 16-byte rounded) + * byte size of the engine MEBuiltins struct: the `noise_uv` rect and the + * local space (see local_space.js) * @ignore * @internal */ -const ME_SIZE = 32; +const ME_SIZE = ME_LAYOUT.length * 4; /** * (Re)build the device-scoped GPU state of a WGSL effect realization — @@ -60,7 +63,8 @@ export function buildEffectGPU(renderer, effect, realization) { const bindings = realization.builtinBindings; const builtins = realization.builtins; const hasUniform = realization.structSize > 0; - const hasME = builtins.noiseUV; + // `noise_uv`'s rect and the local space (local_space.js) + const hasME = builtins.noiseUV || builtins.local === true; // ---- the group-3 layout, shared per shape signature ------------------ const entries = []; @@ -93,7 +97,9 @@ export function buildEffectGPU(renderer, effect, realization) { if (hasME) { entries.push({ binding: bindings.me, - visibility: GPUShaderStage.VERTEX, + // the vertex stage works out where it is, and the local + // space's helpers turn a displacement in the fragment stage + visibility: GPUShaderStage.VERTEX | GPUShaderStage.FRAGMENT, buffer: { type: "uniform", hasDynamicOffset: true, diff --git a/packages/melonjs/src/video/webgpu/webgpu_renderer.js b/packages/melonjs/src/video/webgpu/webgpu_renderer.js index c28ffac76b..c1efb95afe 100644 --- a/packages/melonjs/src/video/webgpu/webgpu_renderer.js +++ b/packages/melonjs/src/video/webgpu/webgpu_renderer.js @@ -1872,8 +1872,9 @@ export default class WebGPURenderer extends Renderer { if (effects.length === 0) { return; } - // the fast path set customShader — nothing offscreen to composite - if (this._usesPostEffectFastPath(renderable, effects)) { + // the fast path set customShader — nothing offscreen to composite; + // its effect is put back to the space `uv` always was + if (this._endEffectSpace(renderable, effects)) { return; } @@ -1922,6 +1923,8 @@ export default class WebGPURenderer extends Renderer { this.captureFrame(); } + // the chain's passes are told where the renderable is + this._beginChainSpace(renderable); if (effects.length === 1) { this.blitEffect(rt1, 0, 0, w, h, effects[0], keepBlend); } else { @@ -1942,6 +1945,7 @@ export default class WebGPURenderer extends Renderer { if (isCamera && renderable.isDefault === false) { this.disableScissor(); } + this._clearChainSpace(); // restore renderer state and the projection saved in beginPostEffect this.restore(); diff --git a/packages/melonjs/tests/__snapshots__/effects_golden_glsl.spec.js.snap b/packages/melonjs/tests/__snapshots__/effects_golden_glsl.spec.js.snap index 087339c00c..623e11d904 100644 --- a/packages/melonjs/tests/__snapshots__/effects_golden_glsl.spec.js.snap +++ b/packages/melonjs/tests/__snapshots__/effects_golden_glsl.spec.js.snap @@ -34,11 +34,17 @@ uniform vec2 ME_size_obj; uniform vec2 ME_size_img; uniform vec2 ME_offset; varying vec2 noise_uv; +uniform float ME_mode; +uniform float ME_ydir; +uniform vec3 ME_local_x; +uniform vec3 ME_local_y; void main(void) { vec4 ME_clip = uProjectionMatrix * vec4(aVertex, 1.0); gl_Position = ME_clip; screen_uv = ME_clip.xy / ME_clip.w * 0.5 + 0.5; - noise_uv = aRegion * (ME_size_img / ME_size_obj) - ME_offset / ME_size_obj; + vec3 ME_v = vec3(aVertex.xy, 1.0); + vec2 ME_px = vec2(dot(ME_local_x, ME_v), dot(ME_local_y, ME_v)); + noise_uv = ME_mode > 1.5 ? ME_px : aRegion * (ME_size_img / ME_size_obj) - ME_offset / ME_size_obj; vColor = vec4(aColor.bgr * aColor.a, aColor.a); vRegion = aRegion; }" @@ -48,11 +54,30 @@ exports[`ShaderEffect generated-GLSL golden > a built-in effect class assembles "uniform sampler2D uSampler; varying vec4 vColor; varying vec2 vRegion; +uniform float ME_mode; +uniform float ME_ydir; +uniform vec4 ME_to_uv; +uniform vec4 ME_art_rect; +varying vec2 ME_local; +varying vec2 ME_art; +vec2 ME_localToUV(vec2 d) { + if (ME_mode > 0.5) { + return mat2(ME_to_uv.xy, ME_to_uv.zw) * d; + } + return vec2(d.x, ME_ydir < 0.0 ? -d.y : d.y); +} +vec2 ME_artToUV(vec2 d) { + return ME_mode > 1.5 ? mat2(ME_to_uv.xy, ME_to_uv.zw) * (d / ME_art_rect.zw) : d; +} +vec2 ME_localToArt(vec2 d) { + return ME_mode > 1.5 ? d * ME_art_rect.zw : ME_localToUV(d); +} uniform float uStrength; uniform float uSize; vec4 apply(vec4 color, vec2 uv) { - vec2 vig = uv * (1.0 - uv); + // across the sprite, wherever it sits in its texture + vec2 vig = ME_local * (1.0 - ME_local); float v = clamp(pow(vig.x * vig.y * uSize, uStrength), 0.0, 1.0); return vec4(color.rgb * v, color.a); } diff --git a/packages/melonjs/tests/__snapshots__/wgsl_scaffold.spec.js.snap b/packages/melonjs/tests/__snapshots__/wgsl_scaffold.spec.js.snap index 3c3e92639b..956925ded1 100644 --- a/packages/melonjs/tests/__snapshots__/wgsl_scaffold.spec.js.snap +++ b/packages/melonjs/tests/__snapshots__/wgsl_scaffold.spec.js.snap @@ -15,6 +15,12 @@ struct MEBuiltins { size_obj : vec2f, size_img : vec2f, offset : vec2f, + mode : f32, + ydir : f32, + local_x : vec4f, + local_y : vec4f, + to_uv : vec4f, + art : vec4f, }; @group(3) @binding(1) var ME : MEBuiltins; @group(3) @binding(2) var screen_texture : texture_2d; @@ -43,7 +49,9 @@ fn vertex_main( let ndc = clip.xy / clip.w; // y-down: capture row 0 = screen top under WebGPU out.vScreenUV = vec2f(ndc.x * 0.5 + 0.5, 0.5 - ndc.y * 0.5); - out.vNoiseUV = aRegion * (ME.size_img / ME.size_obj) - ME.offset / ME.size_obj; + let meV = vec3f(aVertex.xy, 1.0); + let mePx = vec2f(dot(ME.local_x.xyz, meV), dot(ME.local_y.xyz, meV)); + out.vNoiseUV = select(aRegion * (ME.size_img / ME.size_obj) - ME.offset / ME.size_obj, mePx, ME.mode > 1.5); return out; } diff --git a/packages/melonjs/tests/effect-local-space.spec.js b/packages/melonjs/tests/effect-local-space.spec.js new file mode 100644 index 0000000000..bce96827b8 --- /dev/null +++ b/packages/melonjs/tests/effect-local-space.spec.js @@ -0,0 +1,838 @@ +/** + * Effects place their pattern on the renderable they draw (#1407). + * + * An effect is handed `uv`, the coordinate it samples its texture at, which + * runs 0..1 across the renderable only for a sprite with its own texture + * drawn with that one effect. The built-in effects that place a pattern (a + * sway's planted row, a wave's anchor, a shine's sweep, a vignette's or a + * gradient's centre, a scanline's curve) placed it by `uv` all the same, so: + * - on a sprite cut from an atlas, the pattern sat on the atlas: a sway + * rocked the roots, a vignette or a shine missed the sprite altogether + * - with two or more effects, they run over a capture the size of the + * canvas, so the pattern sat on the canvas and slid as the camera moved + * (and `noise_uv` there was the canvas's too) + * + * They now read where they are on the renderable (effects/local_space.js). + * Read in pixels on WebGL: what each case draws is compared with what the + * same art draws as a sprite with its own texture and one effect, which was + * always right and is unchanged. + */ +import { afterAll, beforeAll, describe, expect, it } from "vitest"; +import { + boot, + DissolveEffect, + HologramEffect, + OutlineEffect, + RadialGradientEffect, + ScanlineEffect, + ShaderEffect, + ShineEffect, + Sprite, + SwayEffect, + TextureAtlas, + VignetteEffect, + WaveEffect, +} from "../src/index.js"; +import { LOCAL_MODE, ME_LAYOUT } from "../src/video/effects/local_space.js"; +import { spliceEffect } from "../src/video/effects/mesh_splice.js"; +import { parseWGSLBody } from "../src/video/effects/wgsl/parse.js"; +import { buildWGSLModule } from "../src/video/effects/wgsl/scaffold.js"; +import Renderer from "../src/video/renderer.js"; +import WebGLRenderer from "../src/video/webgl/webgl_renderer.js"; +import { + getWebGLRenderer, + releaseWebGLRenderer, +} from "./helpers/webgl-context.js"; + +const SIZE = 128; +const ART = 32; +// the sprite is drawn at twice its size, centred: 32..96 on the screen +const SCALE = 2; +const CENTRE = 64; +const FROM = CENTRE - (ART * SCALE) / 2; +const SPAN = ART * SCALE; +// where the art sits in the atlas, among other frames +const ATLAS_W = 112; +const ATLAS_H = 96; +const AT_X = 48; +const AT_Y = 40; + +/** + * the art: stripes across, a darker band down, a marker in a corner, with a + * clear margin left and right as swaying art has: a sample pushed past a + * frame's edge reads clear from an atlas or a capture, but repeats the edge + * from a texture of its own, which is not what is compared here + */ +const MARGIN = 6; +const paintArt = (ctx, ox, oy) => { + const left = ox + MARGIN; + const width = ART - MARGIN * 2; + ctx.fillStyle = "#e0e0e0"; + ctx.fillRect(left, oy, width, ART); + ctx.fillStyle = "#4a90d9"; + for (let x = 0; x < width; x += 6) { + ctx.fillRect(left + x, oy, 3, ART); + } + ctx.fillStyle = "#30a050"; + ctx.fillRect(left, oy + 20, width, 6); + ctx.fillStyle = "#ff3030"; + ctx.fillRect(left + 2, oy + 2, 5, 5); +}; + +// ShineEffect's body as it was before the local space, as a custom effect +const OLD_SHINE = ` + uniform vec4 uShineColor; + uniform float uShineWidth; + uniform float uShineSpeed; + uniform float uShineIntensity; + uniform float uShineAngle; + uniform float uShineBands; + uniform float uPulseDepth; + uniform float uPulseSpeed; + uniform float uTime; + uniform float uUVYDir; + vec4 apply(vec4 color, vec2 uv) { + if (color.a == 0.0) return color; + float pulse = (1.0 - uPulseDepth) + uPulseDepth * sin(uTime * uPulseSpeed); + float y = uUVYDir < 0.0 ? 1.0 - uv.y : uv.y; + float pos = uv.x * cos(uShineAngle) + y * sin(uShineAngle); + float localX = fract(pos * uShineBands); + float sweep = fract(uTime * uShineSpeed); + float d = abs(localX - sweep); + float dist = min(d, 1.0 - d); + float glint = smoothstep(uShineWidth, 0.0, dist) * uShineIntensity; + vec3 result = color.rgb * pulse + uShineColor.rgb * glint * color.a * uShineColor.a; + return vec4(result, color.a); + }`; + +describe("effects place their pattern on the renderable (WebGL)", () => { + let renderer; + let gl; + let own; + let atlas; + + beforeAll(async () => { + boot(); + renderer = await getWebGLRenderer(SIZE, SIZE); + if (renderer instanceof WebGLRenderer) { + gl = renderer.gl; + renderer.projectionMatrix.ortho(0, SIZE, SIZE, 0, -1000, 1000); + renderer.currentBatcher.setProjection(renderer.projectionMatrix); + } else { + renderer = undefined; + } + own = Renderer.createCanvas(ART, ART); + paintArt(own.getContext("2d"), 0, 0); + const sheet = Renderer.createCanvas(ATLAS_W, ATLAS_H); + const ctx = sheet.getContext("2d"); + ctx.fillStyle = "#c06090"; + ctx.fillRect(0, 0, ATLAS_W, ATLAS_H); + // the art's frame, clear around the art as its own texture is + ctx.clearRect(AT_X, AT_Y, ART, ART); + paintArt(ctx, AT_X, AT_Y); + const frame = (filename, x, y) => { + return { + filename, + frame: { x, y, w: ART, h: ART }, + rotated: false, + trimmed: false, + spriteSourceSize: { x: 0, y: 0, w: ART, h: ART }, + sourceSize: { w: ART, h: ART }, + pivot: { x: 0.5, y: 0.5 }, + }; + }; + atlas = new TextureAtlas( + { + frames: [frame("other", 0, 0), frame("art", AT_X, AT_Y)], + meta: { + app: "texturepacker", + image: "sheet.png", + size: { w: ATLAS_W, h: ATLAS_H }, + scale: "1", + }, + }, + sheet, + ); + }); + + afterAll(() => { + releaseWebGLRenderer(); + }); + + const requireWebGL = (ctx) => { + if (renderer === undefined) { + ctx.skip("WebGL renderer not available in this environment"); + } + }; + + /** the sprite: the art with its own texture, or cut from the atlas */ + const sprite = ({ fromAtlas = false, angle = 0, flip = false } = {}) => { + const s = fromAtlas + ? atlas.createSpriteFromName("art") + : new Sprite(0, 0, { image: own }); + s.pos.x = CENTRE; + s.pos.y = CENTRE; + s.scale(SCALE); + if (angle !== 0) { + s.rotate(angle); + } + if (flip) { + s.flipX(true); + } + return s; + }; + + /** the screen around the sprite, as RGB */ + const read = () => { + const out = new Uint8Array(SPAN * SPAN * 4); + gl.finish(); + gl.readPixels( + FROM, + SIZE - FROM - SPAN, + SPAN, + SPAN, + gl.RGBA, + gl.UNSIGNED_BYTE, + out, + ); + return out; + }; + + /** + * The screen around the sprite, after a draw: drawn, as a share of its + * pixels lit, so two pictures never match for both being empty. + */ + const drawnPixels = () => { + const pixels = read(); + let lit = 0; + for (let i = 0; i < pixels.length; i += 4) { + if (pixels[i] + pixels[i + 1] + pixels[i + 2] > 30) { + lit++; + } + } + expect(lit / (pixels.length / 4)).toBeGreaterThan(0.15); + return pixels; + }; + + /** draw a renderable on its own, as a container would */ + const paint = (renderable) => { + renderer.backgroundColor.setColor(0, 0, 0, 255); + renderer.clear(); + renderer.save(); + renderer.resetTransform(); + renderable.preDraw(renderer); + renderable.draw(renderer); + renderable.postDraw(renderer); + renderer.restore(); + renderer.flush(); + return drawnPixels(); + }; + + /** draw a sprite through these effects */ + const drawn = (options, ...effects) => { + const s = sprite(options); + for (const fx of effects) { + s.addPostEffect(fx); + } + return paint(s); + }; + + /** a second effect that draws what it is given */ + const passThrough = () => { + return new ShaderEffect(renderer, { + glsl: "vec4 apply(vec4 color, vec2 uv) { return color; }", + wgsl: "fn apply(color : vec4f, uv : vec2f) -> vec4f { return color; }", + }); + }; + + /** + * How far apart two pictures are: the share of pixels that differ by + * more than `tolerance` in a channel. The paths sample differently (the + * texture, or a capture of it already drawn), so rounding and an edge + * texel are allowed for, a misplaced pattern is not. + */ + const differing = (a, b, tolerance = 12) => { + let count = 0; + for (let i = 0; i < a.length; i += 4) { + if ( + Math.abs(a[i] - b[i]) > tolerance || + Math.abs(a[i + 1] - b[i + 1]) > tolerance || + Math.abs(a[i + 2] - b[i + 2]) > tolerance + ) { + count++; + } + } + return count / (a.length / 4); + }; + + // one fresh effect per draw: each draw is a different renderable + const EFFECTS = { + Sway: () => { + return new SwayEffect(renderer, { strength: 0.15, speed: 1 }).setTime(1); + }, + Wave: () => { + const fx = new WaveEffect(renderer, { + amplitude: 0.08, + frequency: 6, + speed: 1, + anchor: 1, + }); + fx.setTime(1); + return fx; + }, + Shine: () => { + const fx = new ShineEffect(renderer, { + speed: 1, + width: 0.15, + intensity: 1, + angle: 0.5, + }); + fx.setTime(0.4); + return fx; + }, + Vignette: () => { + return new VignetteEffect(renderer, { strength: 0.8, size: 12 }); + }, + RadialGradient: () => { + return new RadialGradientEffect(renderer, { intensity: 1 }); + }, + "Scanline curve": () => { + return new ScanlineEffect(renderer, { + opacity: 0, + curvature: 0.4, + vignetteStrength: 0.5, + }); + }, + }; + // patterns measured in the texture: the same on the same art, wherever + // the sprite is drawn from, but not between an atlas and a texture of + // its own (by design: their grain is the texture's) + const TEXTURE_PATTERNS = { + Dissolve: () => { + return new DissolveEffect(renderer, { progress: 0.4 }); + }, + Hologram: () => { + const fx = new HologramEffect(renderer); + fx.setTime(0.3); + return fx; + }, + "Scanline lines": () => { + return new ScanlineEffect(renderer, { opacity: 0.6 }); + }, + }; + + // a wave measures its amplitude and frequency across its texture, and a + // shine its sweep, bands and width (they stay as they were): only a + // wave's planted row is the sprite's own + const ACROSS_SPRITE = Object.keys(EFFECTS).filter((name) => { + return name !== "Wave" && name !== "Shine"; + }); + + describe("one effect on a sprite cut from an atlas", () => { + it.for(ACROSS_SPRITE)( + "%s draws as on a sprite with its own texture", + (name, ctx) => { + requireWebGL(ctx); + const make = EFFECTS[name]; + const reference = drawn({}, make()); + const fromAtlas = drawn({ fromAtlas: true }, make()); + expect(differing(reference, fromAtlas)).toBeLessThan(0.01); + }, + ); + + it.for(ACROSS_SPRITE)( + "%s draws as on its own texture when flipped", + (name, ctx) => { + requireWebGL(ctx); + const make = EFFECTS[name]; + const reference = drawn({ flip: true }, make()); + const fromAtlas = drawn({ fromAtlas: true, flip: true }, make()); + expect(differing(reference, fromAtlas)).toBeLessThan(0.01); + }, + ); + }); + + describe("a wave's anchor", () => { + /** the rows from `y0` (screen) to the sprite's bottom, as drawn */ + const rows = (pixels, y0) => { + // readPixels is bottom-up: the sprite's last rows come first + return pixels.slice(0, (SPAN - y0) * SPAN * 4); + }; + + it.for([false, true])( + "keeps the sprite's own bottom row planted (from an atlas: %s)", + (fromAtlas, ctx) => { + requireWebGL(ctx); + const still = drawn({ fromAtlas }); + const waved = drawn({ fromAtlas }, EFFECTS.Wave()); + // its last row of art (two on the screen) does not move: the + // wave fades out toward it, so the rows above move a little + const planted = SPAN - SCALE; + expect(differing(rows(still, planted), rows(waved, planted))).toBe(0); + // while the top does + expect(differing(still, waved)).toBeGreaterThan(0.05); + }, + ); + }); + + describe("a shine", () => { + /** the previous body, set as the built-in one is */ + const oldShine = () => { + const fx = new ShaderEffect(renderer, OLD_SHINE); + fx.setUniform("uShineColor", [1, 1, 1, 1]); + fx.setUniform("uShineWidth", 0.15); + fx.setUniform("uShineSpeed", 1); + fx.setUniform("uShineIntensity", 1); + fx.setUniform("uShineAngle", 0.5); + fx.setUniform("uShineBands", 3); + fx.setUniform("uPulseDepth", 0); + fx.setUniform("uPulseSpeed", 3); + fx.setUniform("uUVYDir", 1); + fx.setTime(0.4); + return fx; + }; + const shine = () => { + const fx = new ShineEffect(renderer, { + speed: 1, + width: 0.15, + intensity: 1, + angle: 0.5, + bands: 3, + }); + fx.setTime(0.4); + return fx; + }; + + it.for([false, true])( + "keeps its sweep across its texture, as it always did (from an atlas: %s)", + (fromAtlas, ctx) => { + requireWebGL(ctx); + const now = drawn({ fromAtlas }, shine()); + const before = drawn({ fromAtlas }, oldShine()); + expect(differing(now, before, 2)).toBeLessThan(0.002); + }, + ); + }); + + describe("two effects", () => { + const ALL = { ...EFFECTS, ...TEXTURE_PATTERNS }; + + it.for(Object.keys(ALL))("%s draws as it does alone", (name, ctx) => { + requireWebGL(ctx); + const make = ALL[name]; + const alone = drawn({}, make()); + const chained = drawn({}, make(), passThrough()); + expect(differing(alone, chained)).toBeLessThan(0.01); + }); + + it.for(Object.keys(ALL))( + "%s draws as it does alone, on a sprite cut from an atlas", + (name, ctx) => { + requireWebGL(ctx); + const make = ALL[name]; + const alone = drawn({ fromAtlas: true }, make()); + const chained = drawn({ fromAtlas: true }, make(), passThrough()); + expect(differing(alone, chained)).toBeLessThan(0.01); + }, + ); + + it.for(["Sway", "Shine", "Vignette", "RadialGradient"])( + "%s turns with a rotated sprite as it does alone", + (name, ctx) => { + requireWebGL(ctx); + const make = EFFECTS[name]; + const alone = drawn({ angle: 0.6 }, make()); + const chained = drawn({ angle: 0.6 }, make(), passThrough()); + // a rotated sprite's edge is filtered once more through the + // capture: a thin ring, not a misplaced pattern + expect(differing(alone, chained)).toBeLessThan(0.04); + }, + ); + + it("where the second is an outline, the first still draws as alone", (ctx) => { + requireWebGL(ctx); + const alone = drawn({}, EFFECTS.Vignette()); + const outlined = drawn( + {}, + EFFECTS.Vignette(), + new OutlineEffect(renderer, { color: [1, 0, 1], width: 1 }), + ); + // the outline adds its ring around the art; inside, the same + expect(differing(alone, outlined)).toBeLessThan(0.15); + }); + + it("noise_uv runs across the sprite, as it does with one effect", (ctx) => { + requireWebGL(ctx); + const asColour = () => { + return new ShaderEffect(renderer, { + glsl: "vec4 apply(vec4 color, vec2 uv) { return vec4(noise_uv, 0.0, 1.0) * color.a; }", + wgsl: "fn apply(color : vec4f, uv : vec2f) -> vec4f { return vec4f(noise_uv, 0.0, 1.0) * color.a; }", + }); + }; + const alone = drawn({}, asColour()); + const chained = drawn({}, asColour(), passThrough()); + expect(differing(alone, chained)).toBeLessThan(0.01); + // and it is the sprite's: 0..1 across it, left to right in red and + // top to bottom in green (sampled inside the art, past its margin) + const at = (pixels, x, y) => { + const i = ((SPAN - 1 - y) * SPAN + x) * 4; + return [pixels[i], pixels[i + 1]]; + }; + const expected = (screen) => { + return (255 * (screen + 0.5)) / SPAN; + }; + for (const [x, y] of [ + [MARGIN * SCALE + 2, 2], + [SPAN - MARGIN * SCALE - 3, SPAN - 3], + [SPAN / 2, SPAN / 2], + ]) { + const [r, g] = at(chained, x, y); + expect(Math.abs(r - expected(x))).toBeLessThan(8); + expect(Math.abs(g - expected(y))).toBeLessThan(8); + } + }); + }); + + describe("unchanged where nothing was wrong", () => { + it.for(Object.keys({ ...EFFECTS, ...TEXTURE_PATTERNS }))( + "%s on a sprite with its own texture draws as the effect always drew", + (name, ctx) => { + requireWebGL(ctx); + const make = { ...EFFECTS, ...TEXTURE_PATTERNS }[name]; + const throughSprite = drawn({}, make()); + // the same art straight through the renderer, with no + // renderable to place it on: the effect's `uv`, as always + const fx = make(); + renderer.backgroundColor.setColor(0, 0, 0, 255); + renderer.clear(); + renderer.save(); + renderer.resetTransform(); + renderer.customShader = fx; + renderer.drawImage(own, 0, 0, ART, ART, FROM, FROM, SPAN, SPAN); + renderer.customShader = undefined; + renderer.restore(); + renderer.flush(); + const asAlways = drawnPixels(); + expect(differing(throughSprite, asAlways, 2)).toBeLessThan(0.002); + }, + ); + }); + + describe("a camera's effects", () => { + /** a full screen of crossed stripes through a camera's chain */ + const throughCamera = (effect) => { + const art = Renderer.createCanvas(SIZE, SIZE); + const actx = art.getContext("2d"); + actx.fillStyle = "#909090"; + actx.fillRect(0, 0, SIZE, SIZE); + actx.fillStyle = "#2050c0"; + for (let i = 0; i < SIZE; i += 10) { + actx.fillRect(i, 0, 4, SIZE); + actx.fillRect(0, i + 3, SIZE, 3); + } + actx.fillStyle = "#ff3030"; + actx.fillRect(10, 10, 20, 20); + const camera = { + _postEffectManaged: true, + isDefault: true, + postEffects: [effect], + }; + renderer.backgroundColor.setColor(0, 0, 0, 255); + renderer.clear(); + renderer.save(); + renderer.resetTransform(); + renderer.beginPostEffect(camera); + renderer.drawImage(art, 0, 0, SIZE, SIZE, 0, 0, SIZE, SIZE); + renderer.endPostEffect(camera); + renderer.restore(); + renderer.flush(); + const out = new Uint8Array(SIZE * SIZE * 4); + gl.finish(); + gl.readPixels(0, 0, SIZE, SIZE, gl.RGBA, gl.UNSIGNED_BYTE, out); + // drawn: most of the screen is lit + let lit = 0; + for (let i = 0; i < out.length; i += 4) { + if (out[i] + out[i + 1] + out[i + 2] > 30) { + lit++; + } + } + expect(lit / (SIZE * SIZE)).toBeGreaterThan(0.5); + return out; + }; + + // the bodies as they were before the local space, as custom effects + const OLD_SCANLINE = ` + uniform float uScanlineOpacity; + uniform float uCurvature; + uniform float uVignetteStrength; + vec4 apply(vec4 color, vec2 uv) { + vec2 coords = uv; + if (uCurvature > 0.0) { + vec2 centered = uv * 2.0 - 1.0; + centered *= 1.0 + uCurvature * dot(centered, centered); + coords = centered * 0.5 + 0.5; + if (coords.x < 0.0 || coords.x > 1.0 || coords.y < 0.0 || coords.y > 1.0) { + discard; + } + color = texture2D(uSampler, coords) * vColor; + } + float line = sin(coords.y * 800.0) * 0.5 + 0.5; + color.rgb *= 1.0 - line * uScanlineOpacity; + if (uVignetteStrength > 0.0) { + vec2 vig = coords * (1.0 - coords); + float vigFactor = vig.x * vig.y * 15.0; + color.rgb *= clamp(pow(vigFactor, uVignetteStrength), 0.0, 1.0); + } + return color; + }`; + + it("draw a curved scanline exactly as before", (ctx) => { + requireWebGL(ctx); + const now = throughCamera( + new ScanlineEffect(renderer, { + opacity: 0.5, + curvature: 0.3, + vignetteStrength: 0.4, + }), + ); + const old = new ShaderEffect(renderer, OLD_SCANLINE); + old.setUniform("uScanlineOpacity", 0.5); + old.setUniform("uCurvature", 0.3); + old.setUniform("uVignetteStrength", 0.4); + const before = throughCamera(old); + expect(differing(now, before, 2)).toBeLessThan(0.002); + }); + + it("draw a slanted shine exactly as before", (ctx) => { + requireWebGL(ctx); + const shine = new ShineEffect(renderer, { + speed: 1, + width: 0.1, + intensity: 1, + angle: 1.0, + }); + shine.setTime(0.3); + const now = throughCamera(shine); + const old = new ShaderEffect(renderer, OLD_SHINE); + old.setUniform("uShineColor", [1, 1, 1, 1]); + old.setUniform("uShineWidth", 0.1); + old.setUniform("uShineSpeed", 1); + old.setUniform("uShineIntensity", 1); + old.setUniform("uShineAngle", 1.0); + old.setUniform("uShineBands", 1); + old.setUniform("uPulseDepth", 0); + old.setUniform("uPulseSpeed", 3); + old.setUniform("uUVYDir", 1); + old.setTime(0.3); + const before = throughCamera(old); + expect(differing(now, before, 2)).toBeLessThan(0.002); + }); + + it("still sweep top to bottom over WebGL's bottom-up capture", (ctx) => { + requireWebGL(ctx); + // a white screen through a camera's chain: a vertical shine a + // quarter of the way through its sweep, then a pass-through + const white = Renderer.createCanvas(SIZE, SIZE); + const wctx = white.getContext("2d"); + wctx.fillStyle = "#808080"; + wctx.fillRect(0, 0, SIZE, SIZE); + const shine = new ShineEffect(renderer, { + speed: 1, + width: 0.05, + intensity: 1, + angle: Math.PI / 2, + }); + shine.setTime(0.25); + const camera = { + _postEffectManaged: true, + isDefault: true, + postEffects: [shine, passThrough()], + }; + renderer.backgroundColor.setColor(0, 0, 0, 255); + renderer.clear(); + renderer.save(); + renderer.resetTransform(); + renderer.beginPostEffect(camera); + renderer.drawImage(white, 0, 0, SIZE, SIZE, 0, 0, SIZE, SIZE); + renderer.endPostEffect(camera); + renderer.restore(); + renderer.flush(); + // the brightest row, from the top of the screen + const column = new Uint8Array(SIZE * 4); + gl.finish(); + gl.readPixels(SIZE / 2, 0, 1, SIZE, gl.RGBA, gl.UNSIGNED_BYTE, column); + let brightest = 0; + let row = -1; + for (let i = 0; i < SIZE; i++) { + if (column[i * 4] > brightest) { + brightest = column[i * 4]; + // readPixels runs bottom-up + row = SIZE - 1 - i; + } + } + expect(brightest).toBeGreaterThan(200); + expect(Math.abs(row - SIZE / 4)).toBeLessThan(4); + }); + }); + + describe("the space", () => { + it("maps the corners of a renderable to 0..1, whatever its transform", (ctx) => { + requireWebGL(ctx); + const s = sprite({ angle: 0.7, flip: true }); + renderer.save(); + renderer.resetTransform(); + renderer.translate(13, -7); + renderer.scale(1.5, 1.5); + // what preDraw applies before telling the effects + s.addPostEffect(EFFECTS.Vignette()); + s.preDraw(renderer); + const space = s._effectSpace; + // where the corners of the art land, by the transform drawn with + const m = renderer.currentTransform.val; + const toTarget = (x, y) => { + return [m[0] * x + m[4] * y + m[12], m[1] * x + m[5] * y + m[13]]; + }; + const local = ([x, y]) => { + return [ + space.localX[0] * x + space.localX[1] * y + space.localX[2], + space.localY[0] * x + space.localY[1] * y + space.localY[2], + ]; + }; + const corners = [ + [s.pos.x, s.pos.y, 0, 0], + [s.pos.x + ART, s.pos.y, 1, 0], + [s.pos.x, s.pos.y + ART, 0, 1], + [s.pos.x + ART, s.pos.y + ART, 1, 1], + ]; + for (const [x, y, u, v] of corners) { + const [lu, lv] = local(toTarget(x, y)); + expect(lu).toBeCloseTo(u, 5); + expect(lv).toBeCloseTo(v, 5); + } + s.postDraw(renderer); + renderer.restore(); + }); + + it("leaves a renderable it cannot place in the space uv always was", (ctx) => { + requireWebGL(ctx); + // no finite size: nothing to be 0..1 across + const s = sprite(); + s.width = Infinity; + expect(renderer._effectSpaceOf(s, LOCAL_MODE.chain)).toBe(null); + // a perspective view (Camera3d) + const s2 = sprite(); + const saved = renderer.projectionMatrix.val[11]; + renderer.projectionMatrix.val[11] = -1; + expect(renderer._effectSpaceOf(s2, LOCAL_MODE.chain)).toBe(null); + renderer.projectionMatrix.val[11] = saved; + // drawn with its one effect, a renderable that is not a sprite + const notASprite = { width: 10, height: 10, pos: { x: 0, y: 0 } }; + expect(renderer._effectSpaceOf(notASprite, LOCAL_MODE.sprite)).toBe(null); + }); + + it("puts a renderable's one effect back once it is drawn", (ctx) => { + requireWebGL(ctx); + const fx = EFFECTS.Vignette(); + drawn({}, fx); + expect(fx._shader._uniformCache.ME_mode).toBe(LOCAL_MODE.uv); + }); + + it("gives a camera's chain no renderable space", () => { + const bare = Object.create(Renderer.prototype); + bare._effectSpaces = new WeakMap(); + bare._chainSpace = { mode: LOCAL_MODE.chain }; + const camera = { _postEffectManaged: true }; + bare._effectSpaces.set(camera, { mode: LOCAL_MODE.chain }); + bare._beginChainSpace(camera); + expect(bare._chainSpace).toBe(null); + }); + }); + + describe("on WebGPU (the block the WGSL bodies read)", () => { + it("assembles the local space for a built-in body", () => { + const body = ` +struct U { uStrength : f32, }; +@group(3) @binding(0) var fx : U; +fn apply(color : vec4f, uv : vec2f) -> vec4f { + let vig = ME_local * (1.0 - ME_local); + return vec4f(color.rgb * vig.x * vig.y * fx.uStrength, color.a); +}`; + const parsed = parseWGSLBody(body); + expect(parsed.ok).toBe(true); + expect(parsed.builtins.local).toBe(true); + const { code, bindings } = buildWGSLModule(body, parsed); + expect(bindings.me).toBeGreaterThan(0); + expect(code).toContain("struct MEBuiltins"); + expect(code).toContain("fn ME_localToUV"); + expect(code).toContain("out.vLocal"); + expect(code).toContain("ME_local = in.vLocal;"); + }); + + it("writes the space into the block's mirror, and only a change", () => { + const fx = Object.create(ShaderEffect.prototype); + const changes = []; + fx._hasLocalSpace = true; + fx.wgslRealization = { + meMirror: new Float32Array(ME_LAYOUT.length), + onUniformChange: () => { + changes.push("flush"); + }, + }; + const space = { + mode: LOCAL_MODE.chain, + localX: [0.5, 0, -1], + localY: [0, 0.25, -2], + toUV: [1, 0, 0, -1], + art: [0.1, 0.2, 0.3, 0.4], + }; + fx._setLocalSpace(space); + const me = fx.wgslRealization.meMirror; + expect(me[ME_LAYOUT.mode]).toBe(LOCAL_MODE.chain); + expect([...me.slice(ME_LAYOUT.localX, ME_LAYOUT.localX + 3)]).toEqual([ + 0.5, 0, -1, + ]); + expect([...me.slice(ME_LAYOUT.localY, ME_LAYOUT.localY + 3)]).toEqual([ + 0, 0.25, -2, + ]); + expect([...me.slice(ME_LAYOUT.toUV, ME_LAYOUT.toUV + 4)]).toEqual([ + 1, 0, 0, -1, + ]); + expect(me[ME_LAYOUT.art + 2]).toBeCloseTo(0.3, 6); + expect(changes).toEqual(["flush"]); + // the same again draws nothing first + fx._setLocalSpace(space); + expect(changes).toEqual(["flush"]); + // and back to `uv` + fx._setLocalSpace(null); + expect(me[ME_LAYOUT.mode]).toBe(LOCAL_MODE.uv); + expect(changes).toEqual(["flush", "flush"]); + }); + }); + + describe("on a mesh", () => { + it("gives a body reading the local space `uv`, as a mesh always did", () => { + const glsl = spliceEffect( + "vec4 ME_effect(vec4 c, vec2 uv) { return c; }\nvoid main() {}", + "vec4 apply(vec4 color, vec2 uv) { return color * ME_local.x; }", + ); + expect(glsl).toContain("vec2 ME_local;"); + expect(glsl).toContain( + "ME_local = uv; ME_art = uv; return apply(c, uv);", + ); + const wgsl = spliceEffect( + "fn ME_effect(c : vec4f, uv : vec2f) -> vec4f { return c; }", + "fn apply(color : vec4f, uv : vec2f) -> vec4f { return color * ME_local.x; }", + ); + expect(wgsl).toContain("var ME_local : vec2f;"); + expect(wgsl).toContain( + "ME_local = uv; ME_art = uv; return apply(c, uv);", + ); + // a body that does not read it is spliced as it always was + const plain = spliceEffect( + "vec4 ME_effect(vec4 c, vec2 uv) { return c; }", + "vec4 apply(vec4 color, vec2 uv) { return color; }", + ); + expect(plain).not.toContain("ME_local"); + }); + }); +}); diff --git a/packages/melonjs/tests/webgl_review_fixes.spec.js b/packages/melonjs/tests/webgl_review_fixes.spec.js index e9c9adcd25..fbd69df4bd 100644 --- a/packages/melonjs/tests/webgl_review_fixes.spec.js +++ b/packages/melonjs/tests/webgl_review_fixes.spec.js @@ -201,19 +201,19 @@ describe("WebGL setBlendMode('none')", () => { }); }); -describe("ShineEffect uUVYDir adoption", () => { - it("declares the directional uniform, defaulted top-down, and accepts the feed", (ctx) => { +describe("ShineEffect takes the space's direction", () => { + it("reads it through its local space, and accepts the feed", (ctx) => { requireWebGL(ctx, renderer); const fx = new ShineEffect(renderer, { angle: Math.PI / 2 }); try { - // active on the compiled program (the sweep uses it, so the - // compiler cannot eliminate it) — the batcher feed sites can - // flip it on the bottom-up pooled path - expect(typeof fx._shader.uniforms.uUVYDir).not.toBe("undefined"); - expect(() => { - fx._setUVYDir(-1); - fx._setUVYDir(1); - }).not.toThrow(); + // the sweep is placed in the local space (local_space.js), which + // runs top to bottom on the bottom-up pooled path by the + // direction the batchers feed — active on the compiled program + expect(typeof fx._shader.uniforms.ME_ydir).not.toBe("undefined"); + fx._setUVYDir(-1); + expect(fx._shader._uniformCache.ME_ydir).toBe(-1); + fx._setUVYDir(1); + expect(fx._shader._uniformCache.ME_ydir).toBe(1); } finally { fx.destroy(); } diff --git a/packages/melonjs/tests/webgpu_post_effect_flow.spec.js b/packages/melonjs/tests/webgpu_post_effect_flow.spec.js index d2be2ebfa0..c1d233e6b9 100644 --- a/packages/melonjs/tests/webgpu_post_effect_flow.spec.js +++ b/packages/melonjs/tests/webgpu_post_effect_flow.spec.js @@ -67,6 +67,10 @@ describe("WebGPU post-effect control flow (recorded primitives)", () => { customShader: undefined, effectProjectionStack: [], effectPassDepth: 0, + // where each renderable with a chain is, for its passes + // (Renderer#_setEffectSpace) + _effectSpaces: new WeakMap(), + _chainSpace: null, projectionMatrix: new Matrix3d(), backgroundColor: new Color(16, 32, 48), currentBatcher: null,