From 8e6f051955e2993f8bdb1e9f4e701369c46cce2c Mon Sep 17 00:00:00 2001 From: Abdelrahman Date: Sun, 9 Aug 2026 22:49:26 +0100 Subject: [PATCH] Research shaders for compositor ops --- documents/research/shader-filters.md | 1839 ++++++++++++++++++++++++++ documents/session-logs/2026-08-09.md | 59 + 2 files changed, 1898 insertions(+) create mode 100644 documents/research/shader-filters.md create mode 100644 documents/session-logs/2026-08-09.md diff --git a/documents/research/shader-filters.md b/documents/research/shader-filters.md new file mode 100644 index 0000000..2d9054b --- /dev/null +++ b/documents/research/shader-filters.md @@ -0,0 +1,1839 @@ +# Image Filter Algorithms for GPU Fragment Shaders + +Research conducted 2026-08-09. Target: Slang → SPIR-V → Vulkan, fragment-shader +node passes in the Prism compositor. + +Sources are cited inline. Where a formula comes from a specification (ASC CDL, +Rec. 709) or a reference implementation (OpenColorIO, Blender compositor, +Kyprianidis' GLSL reference), the citation is given. Where a value is a +convention rather than a specification (e.g. "typical unsharp amount"), it is +labelled as such. + +--- + +## 0. Common assumptions for all filters + +### 0.1 Pass model + +Every filter below is a fullscreen pass driven by the existing +`blit.vert`-style vertex shader (no vertex buffer; `SV_VertexID` generates the +quad/triangle). This matches `assets/blit.slang` and the node-type registry in +`documents/TEXTURE_POOL_AND_NODE_EVAL.md` §3.2. + +### 0.2 Push constants vs uniform buffer + +Vulkan guarantees only **128 bytes** of push-constant space +(`maxPushConstantsSize` minimum in the Vulkan spec). All parameter blocks below +fit comfortably within that limit — the largest is the CDL block at 48 bytes. So: + +- **Push constants** for every filter in this document. None needs more. +- A uniform buffer is only needed if a filter gains a runtime-sized table + (e.g. a precomputed Gaussian weight array longer than ~24 floats). The + recommendation below is to compute Gaussian weights **in the shader**, which + avoids the UBO entirely. + +Slang syntax (confirmed against Slang docs, "SPIR-V-Specific Functionalities" +and the Vulkan Guide "High Level Shader Language Comparison"): + +```slang +[[vk::push_constant]] +FilterData filter; + +Sampler2D input_tex; // combined image sampler → OpTypeSampledImage +``` + +Slang supports combined samplers (`Sampler2D`) directly for SPIR-V, which is +what `blit.slang` already uses. Keep that. + +### 0.3 Texel size + +Every spatial filter needs the input texel size. Two options: + +1. Pass `float2 texel_size` in push constants (explicit, works for any + sampled resolution). +2. Query in-shader: Slang exposes `GetDimensions`. + +Prefer (1). It is one `float2`, it avoids a `GetDimensions` call per fragment, +and in a compositor the node's *working* resolution may differ from the +texture's allocated resolution (texture pool reuses oversized textures — see +`TEXTURE_POOL_AND_NODE_EVAL.md` §2.6). + +### 0.4 Colour space — linear float intermediates + +**Decision: all intermediate textures are linear float (`R16G16B16A16_SFLOAT`, +`R32G32B32A32_SFLOAT` where needed).** + +Spatial filters (Gaussian, Laplacian, Sobel, sharpen, Kuwahara) are linear +operations on light. Applying them to gamma-encoded pixels gives wrong results. +Posterize and pixelize are also affected but less visibly. + +The pipeline works as follows: +- **Load**: sRGB images are linearized once at upload time by the Read node. + Float source formats (OpenEXR, HDR) are stored as-is. The destination format + is always the intermediate format. +- **All filter nodes**: operate on linear float data. No decode/encode + overhead per sample. +- **Display/output**: the final blit to an `R8G8B8A8_SRGB` swapchain lets + Vulkan encode linear→sRGB on store. No manual gamma work. + +This means no per-filter colour-space logic is needed — the pipeline guarantees +linear data everywhere between load and display. + +### 0.5 Alpha handling — premultiplied by default + +**Decision: the entire pipeline is premultiplied-alpha. Operations that need +unpremultiplied values use explicit Unpremult/Premultiplied nodes.** + +Premultiplied alpha stores `(R·A, G·A, B·A, A)`. It makes compositing +(over, add, multiply) a single operation instead of a divide-per-pixel. + +For spatial filters (blur, Laplacian, sharpen, Kuwahara), filtering RGBA +together keeps premultiplication intact — they operate directly on premultiplied +input. + +For colour operators that are mathematically defined on unpremultiplied colour +(CDL, posterize), the user inserts an **Unpremult node** before and a +**Premult node** after: + +``` +Read → Unpremult → CDL → Premult → Blur → Display +``` + +This is the Nuke model — explicit, transparent, composable. The user can +skip unpremult/premult when unnecessary (e.g., full-opacity regions), +avoiding per-node hidden cost. + +See §11 (Unpremult) and §12 (Premult) for shader implementations. + +### 0.6 Edge handling — per-node parameter + +All neighbourhood filters read outside the image at the border. Two modes: + +- **Clamp to edge** (default) — repeats the border pixel outward. Safe default, + no surprise dark borders. +- **Clamp to border** — returns transparent black `(0,0,0,0)`. Mathematically + correct for premultiplied compositing over an infinite transparent canvas. + +This is a **per-node parameter** set at node creation. It determines which +sampler state the node uses; the shader does not branch on it. Spatial filters +(Gaussian, Laplacian, Sobel, sharpen, Kuwahara) expose this parameter. Filters +that do not read neighbouring pixels (CDL, posterize, pixelize) ignore it. + +--- + +## 1. Gaussian Blur + +### 1.1 Mathematics + +#### What is a Gaussian? + +A **Gaussian** is a bell-shaped curve. It is the "normal distribution" from +statistics — the classic bell curve. In image processing, we use it as a +**weighting function**: pixels closer to the centre matter more, and pixels +further away matter less, fading smoothly to zero. + +Imagine you are standing at the centre of the image. The Gaussian tells you how +much each surrounding pixel should contribute to the result at your position. +Right next to you → full weight. A few pixels away → some weight. Far away → +almost no weight. No hard edges, just a smooth falloff. + +The formula: + +``` +G(x, y) = (1 / (2πσ²)) · exp(−(x² + y²) / (2σ²)) +``` + +In plain language: +- `x` and `y` are the horizontal and vertical distance from the centre pixel. +- `σ` (sigma) controls how wide the bell is. A small σ means a narrow bell + (only very close pixels matter → slight blur). A large σ means a wide bell + (far-away pixels matter → heavy blur). +- `exp(−...)` is the exponential function `e^(−...)`. It produces the smooth + falloff. The further you are from the centre, the more negative the exponent + becomes, and `e^(−big_number)` approaches zero. +- `1 / (2πσ²)` is a normalisation factor that ensures all the weights add up to + 1 (so the image does not get brighter or darker). + +#### Why "separable"? + +The 2D Gaussian has a special mathematical property: it can be expressed as the +product of two 1D Gaussians (one for x, one for y): + +``` +G(x, y) = G(x) · G(y) +``` + +Why does this matter? A naive 2D blur with a 31×31 kernel needs 961 texture +reads per pixel. But if we do it as two 1D passes (31 reads horizontal, then 31 +reads vertical), we need only 62 reads. That is a 15× speedup. This is what makes +real-time Gaussian blur practical. + +#### Composing blurs + +If you blur an image with σ=2 and then blur the result with σ=3, the result is +identical to blurring once with `σ = √(2² + 3²) = √13 ≈ 3.6`. This property +lets us build very large blurs from repeated small ones (useful for bloom +effects). + +### 1.2 Kernel size from sigma + +#### Infinite support — why we need to truncate + +The Gaussian curve never reaches exactly zero — it extends to infinity in both +directions (this is called "infinite support"). In practice, we must cut it off +at some radius because we cannot sample infinitely many pixels. + +The rule `radius = ceil(3 · σ)` means we keep all pixels within 3 sigma of the +centre. This captures 99.7% of the total weight (from the empirical rule of +normal distributions: ±3σ contains 99.7% of the area under the bell curve). + +- **2σ** (95.4%): cheaper but you cut off too much — visible ringing artifacts + near high-contrast edges. +- **4σ** (99.99%): negligible improvement over 3σ but 33% more taps. +- **3σ** (99.7%): the standard compromise. + +#### Renormalisation — why we divide by the sum + +After truncation, the weights no longer sum to 1 (we cut off the tails). If we +use them as-is, the image gets darker because we are "losing" some weight. The +fix: divide each weight by the actual sum of all weights in the truncated +kernel. This restores the total to 1 and keeps the image brightness unchanged. + +In the shader this appears as: `return sum / weight` where `weight` accumulates +all the Gaussian weights used. + +### 1.3 Linear-sampling optimisation + +#### What is bilinear filtering? + +When you sample a texture at a coordinate that falls between two texels (texture +pixels), the GPU can do **bilinear filtering**: it returns a weighted blend of +the four nearest texels. This is done in hardware — effectively free compared to +making an extra texture fetch. + +#### How does it help? + +In a Gaussian blur, adjacent pixels along the blur direction have weights like +`w₁ = 0.15` and `w₂ = 0.12`. Instead of sampling each separately (2 fetches), +we can sample once at a position between them and get the same result. + +The trick: sample at a weighted midpoint between the two texels: + +``` +combined_weight = w₁ + w₂ +sample_position = (t₁·w₁ + t₂·w₂) / (w₁ + w₂) // weighted average of positions +``` + +The GPU's bilinear filtering does exactly this blend for us. So instead of +sampling at `t₁` (getting texel 1) and `t₂` (getting texel 2), we sample once +at a point between them and get the same weighted result. + +This halves the number of fetches: a 9-tap filter needs only 5 fetches instead +of 9. RasterGrid measured ~60% higher throughput. + +Caveats for a compositor: +- Requires a **linear** sampler. If the node also wants point sampling for + other reasons, that conflicts. +- Requires the source to be a normal 2D texture with filtering support for its + format. `R32G32B32A32_SFLOAT` linear filtering is **not** guaranteed by the + Vulkan spec — check `VK_FORMAT_FEATURE_SAMPLED_IMAGE_FILTER_LINEAR_BIT`. + `R16G16B16A16_SFLOAT` is guaranteed. This is a real argument for making + fp16 the intermediate format. + +### 1.4 Slang implementation + +Two-pass, ping-ponged. One shader, direction supplied by push constant. + +```slang +struct BlurData { + float2 texel_size; // 1.0 / texture dimensions + float2 direction; // (1,0) pass 1, (0,1) pass 2 + float sigma; // standard deviation, pixels + int radius; // ceil(3 * sigma), clamped + uint edge_mode; // 0 = clamp to edge, 1 = clamp to border + uint pad; +}; + +[[vk::push_constant]] BlurData blur; +Sampler2D input_tex; + +[shader("fragment")] +float4 main(VSOutput input) : SV_Target { + float inv_two_sigma_sq = 1.0 / (2.0 * blur.sigma * blur.sigma); + float4 sum = input_tex.Sample(input.UV) ; + float weight = 1.0; + sum *= weight; + + for (int i = 1; i <= blur.radius; ++i) { + float w = exp(-float(i) * float(i) * inv_two_sigma_sq); + float2 off = blur.direction * blur.texel_size * float(i); + sum += (input_tex.Sample(input.UV + off) + + input_tex.Sample(input.UV - off)) * w; + weight += 2.0 * w; + } + return sum / weight; // renormalise — mandatory +} +``` + +Linear-sampled variant (halves the fetches), pairing taps `2i-1` and `2i`: + +```slang + for (int i = 1; i <= blur.radius; i += 2) { + float w0 = exp(-float(i) * float(i) * inv_two_sigma_sq); + float w1 = exp(-float(i + 1) * float(i + 1) * inv_two_sigma_sq); + float w = w0 + w1; + float t = (float(i) * w0 + float(i + 1) * w1) / w; + float2 off = blur.direction * blur.texel_size * t; + sum += (input_tex.Sample(input.UV + off) + + input_tex.Sample(input.UV - off)) * w; + weight += 2.0 * w; + } +``` + +Note the loop must handle an odd trailing tap when `radius` is even. + +### 1.5 Parameters + +| Parameter | Type | Range | Default | Note | +|-----------|------|-------|---------|------| +| `sigma` | f32 | 0.0 – 100.0 | 2.0 | Below ~0.3 the filter is a no-op; branch out | +| `radius` | i32 | derived `ceil(3σ)` | — | Clamp to a hard max (see below) | +| `direction` | f32×2 | (1,0) / (0,1) | — | Set by the host per pass | +| `texel_size` | f32×2 | — | — | `1/width, 1/height` | + +### 1.6 Performance + +- Naive 2D: `(2r+1)²` taps per pixel. At σ=10 → r=30 → **3721 taps**. Never do + this. +- Separable: `2·(2r+1)` taps → 122 taps. **30× cheaper** at σ=10. +- Separable + linear sampling: ~62 fetches. +- For very large σ, downsample first. Blurring a half-res image with σ/2 and + upsampling is visually near-identical to full-res σ and is 4× cheaper in + fetches plus 4× cheaper in fragments. Standard bloom practice. +- **Hard-cap the loop count.** A `radius` from a UI slider driving an unbounded + loop is a GPU hang risk. Cap at e.g. 64 taps per side and switch to the + downsample path beyond that. + +### 1.7 Compositor notes + +- **Two passes → needs an intermediate texture** from the texture pool. The + node evaluation loop currently assumes one output per node + (`output_count = 1`). A 2-pass blur needs either (a) a scratch texture + acquired and released within the node's evaluation, or (b) the node modelled + as two internal sub-passes. (a) is simpler and matches the pool refcount + design. +- The intermediate must be the same format and at least the same size as the + output. +- σ should scale with a "size" reference if the graph supports proxy/preview + resolutions, otherwise the preview will not match the full render. + +--- + +## 2. CDL (ASC Color Decision List) + +### 2.1 Specification + +#### What is a CDL? + +A CDL (Color Decision List) is a standard way to describe a colour grade. +Think of it as a recipe: "make the image brighter, add some warmth, increase +contrast". The ASC CDL defines exactly four operations applied in a specific +order. Every colour grading tool (Nuke, DaVinci Resolve, Blender) understands +CDL, which means a grade created in one app can be transferred to another. + +#### The SOP operations (Slope, Offset, Power) + +These three operations are applied to each colour channel (R, G, B) independently. +You can think of them as a chain of simple math operations: + +**Slope** — multiplication. `output = input · slope` +- slope > 1: makes the channel brighter (amplifies it) +- slope < 1: makes the channel darker (reduces it) +- slope = 1: no change +- Analogy: turning up the volume on one channel. + +**Offset** — addition. `output = result + offset` +- positive offset: adds brightness (lifts the channel) +- negative offset: subtracts brightness (lowers the channel) +- offset = 0: no change +- Analogy: adding a constant baseline value. + +**Power** — exponentiation. `output = result ^ power` +- This is the contrast control. It reshapes the curve non-linearly. +- power > 1: compresses highlights, expands shadows (image looks more contrasty + in the dark areas) +- power < 1: expands highlights, compresses shadows (image looks flatter) +- power = 1: no change (since x¹ = x) +- The function `x^p` is a curve that bends depending on p. For x between 0 and + 1: p > 1 pushes values lower (darker), p < 1 pushes values higher (brighter). + +Putting it together for one channel: +``` +result = (input · slope + offset) ^ power +``` + +This is called the SOP (Slope-Offset-Power) function. + +#### What does `per channel` mean? + +The `c ∈ {R, G, B}` notation means this is done separately for the red, green, +and blue channels. Each channel can have its own slope, offset, and power. This +allows colour-specific adjustments — e.g., making reds more contrasty without +affecting blues. + +#### What is `clamp`? + +`clamp(x, 0, 1)` restricts a value to the range [0, 1]. Anything below 0 becomes +0, anything above 1 becomes 1. In a display-referred (8-bit) pipeline this +prevents colour values from going out of range. In a scene-linear pipeline, +clamping destroys valid HDR data, which is why `noClampFwd` exists. + +#### Luma and Saturation + +After SOP, the CDL applies a saturation adjustment based on **luma**. + +**Luma** is a weighted sum that approximates how bright a pixel appears to the +human eye (we are more sensitive to green, less to blue): + +``` +luma = 0.2126·R + 0.7152·G + 0.0722·B +``` + +**Saturation** then pushes colours away from (or toward) this luma value: +``` +output = luma + saturation · (colour − luma) +``` + +Think of it this way: `(colour − luma)` is the colour's deviation from grey. +Multiplying by saturation scales that deviation: +- saturation = 0: the deviation becomes zero → pure grey (desaturated) +- saturation = 1: unchanged (full colour) +- saturation > 1: deviation is amplified → more vivid colours + +#### Full formula (forward, no-clamp version for scene-linear) + +``` +per channel c ∈ {R, G, B}: + t_c = in_c · slope_c + offset_c + out_c = t_c ^ power_c (skip if t_c < 0) + +luma = 0.2126·R + 0.7152·G + 0.0722·B +out_c = luma + saturation · (out_c − luma) +``` + +#### Why skip power for negative bases? + +`pow(negative_number, power)` is mathematically undefined for non-integer powers +(e.g., what is (−0.5)^0.4?). GPUs would return NaN (Not a Number), which +propagates and corrupts the image. The fix: if the value is negative, just pass +it through unchanged. This matches the official OpenColorIO behaviour. + +### 2.2 Slang implementation + +```slang +struct CdlData { + float3 slope; // 12 B + float saturation; // 4 B + float3 offset; // 12 B + float pad0; // 4 B + float3 power; // 12 B + uint clamp_mode; // 4 B 0 = noClamp, 1 = v1.2 +}; // 48 B total, std430-friendly + +[[vk::push_constant]] CdlData cdl; +Sampler2D input_tex; + +static const float3 LUMA_709 = float3(0.2126, 0.7152, 0.0722); + +float3 applyPowerNoClamp(float3 v, float3 p) { + // negative base → pass through, matching OCIO noClamp behaviour + float3 powered = pow(abs(v), p); + return select(v < 0.0, v, powered); +} + +[shader("fragment")] +float4 main(VSOutput input) : SV_Target { + float4 src = input_tex.Sample(input.UV); + float3 c = src.rgb; + + c = c * cdl.slope + cdl.offset; + + if (cdl.clamp_mode == 1) { + c = saturate(c); + c = pow(c, cdl.power); + } else { + c = applyPowerNoClamp(c, cdl.power); + } + + float luma = dot(c, LUMA_709); + c = luma + cdl.saturation * (c - luma); + + if (cdl.clamp_mode == 1) { c = saturate(c); } + + return float4(c, src.a); +} +``` + +Two correctness details: + +1. `pow(negative, x)` is **undefined** in GLSL/HLSL/SPIR-V. The `abs` + + `select` above avoids feeding a negative base to `pow`. Do not rely on the + hardware returning NaN "harmlessly". +2. Alpha is passed through untouched. OCIO does the same + (`out[3] = inAlpha`). If the graph is premultiplied, un-premultiply before + and re-premultiply after, otherwise the grade is wrong on semi-transparent + pixels. + +### 2.3 Parameters + +| Parameter | Type | Range | Default | Source | +|-----------|------|-------|---------|--------| +| `slope` | f32×3 | 0 – ∞ (practically 0 – 4) | 1.0 | ASC spec: slope ≥ 0 | +| `offset` | f32×3 | −∞ – ∞ (practically −1 – 1) | 0.0 | ASC spec | +| `power` | f32×3 | > 0 (practically 0.1 – 4) | 1.0 | ASC spec: `0 < power < ∞` | +| `saturation` | f32 | ≥ 0 (practically 0 – 4) | 1.0 | ASC spec: "values > 4 or so will probably only be used for special purposes" | + +`power` must never be exactly 0 or negative — the spec bounds it to `> 0`. +Clamp in the UI, not in the shader. + +### 2.4 Performance + +Pure per-pixel ALU. One texture fetch. Three `pow` calls dominate — `pow` is a +transcendental (log + mul + exp), roughly 4–8× the cost of a MAD on most +hardware. Worth a fast path: if `power == float3(1,1,1)` the whole SOP reduces +to a MAD, and OCIO notes its optimiser does exactly this ("if power is 1, the +optimizer is able to convert the CDL op into a pair of matrices and clamp"). + +Consider a shader specialisation constant for `power == 1` and +`saturation == 1` if profiling shows CDL nodes are hot. Probably unnecessary — +this is not a bandwidth-bound filter. + +### 2.5 Compositor notes + +- **Single pass, 1 input.** The simplest possible node. +- This is an excellent candidate for **node fusion** with other per-pixel nodes + (see `shader-architecture-patterns.md` §1 — Blender fuses per-pixel ops into + one compile unit). +- CDL values are commonly loaded from `.cdl` / `.ccc` XML files. If Prism + supports importing them, the parser belongs in host code, not the shader. + +--- + +## 3. Laplacian + +### 3.1 Kernels + +#### What is a derivative? (first, in 1D) + +A **derivative** measures how fast something changes. In 1D (a line of pixels), +the first derivative at a pixel is approximately: "how different is this pixel +from its neighbour?" If two adjacent pixels have similar colours, the derivative +is near zero. If there is a sharp jump (an edge), the derivative is large. + +#### Second derivative — the Laplacian + +The **second derivative** measures how the *rate of change itself* is changing. +In 2D (an image), the Laplacian is: + +``` +∇²f = ∂²f/∂x² + ∂²f/∂y² +``` + +This is the sum of the second derivative in x and the second derivative in y. +Intuitively: it measures whether a pixel is a **peak**, a **valley**, or on a +**slope**, relative to its neighbours. + +Key property: in flat regions (all neighbours similar), the Laplacian is **zero**. +At edges, it is **non-zero**. This makes it an edge detector. + +#### What is a kernel? + +A **kernel** (or convolution matrix) is a small grid of numbers. You centre it +over a pixel, multiply each neighbour by the corresponding number, and sum the +results. The numbers are called **weights**. The output replaces the centre pixel. + +#### The two 3×3 Laplacian kernels + +**4-neighbour (no diagonals)** — only considers pixels directly above, below, +left, and right: + +``` + 0 1 0 + 1 -4 1 + 0 1 0 +``` + +Reading this: take 1× each of the four neighbours, and −4× the centre. Sum them. +If the centre equals the average of its neighbours, the result is zero. If the +centre is very different from its neighbours, the result is large. + +**8-neighbour (with diagonals)** — also includes the four corners: + +``` + 1 1 1 + 1 -8 1 + 1 1 1 +``` + +Same idea, but uses all 8 surrounding pixels. More accurate in all directions +(isotropic) but more sensitive to noise (because it reads more pixels, any noise +in them affects the result). + +#### Why do they sum to zero? + +The weights in each kernel add up to zero: `1+1+1+1−4 = 0` and +`1+1+1+1+1+1+1+1−8 = 0`. This is why flat regions produce zero output — if every +neighbour has the same value `v`, then `4v − 4v = 0`. The kernel only responds to +differences, not to absolute values. + +#### Sharpening with the Laplacian + +The "sharpen" kernel `identity − laplacian` adds the original image back to the +Laplacian response: + +``` + 0 -1 0 +-1 5 -1 + 0 -1 0 +``` + +This is equivalent to: `output = 5·centre − (sum of 4 neighbours)`. Where there +is an edge, this amplifies the difference, making edges crisper. + +This is exactly the unsharp-mask kernel with a uniform 5-pixel blur and +amount 5 (derivation in the Wikipedia "Unsharp masking" article). See §5. + +### 3.2 Slang implementation + +```slang +struct LaplacianData { + float2 texel_size; + float scale; // output gain + uint variant; // 0 = 4-neighbour, 1 = 8-neighbour + uint edge_mode; // 0 = clamp to edge, 1 = clamp to border +}; + +[[vk::push_constant]] LaplacianData lap; +Sampler2D input_tex; + +[shader("fragment")] +float4 main(VSOutput input) : SV_Target { + float2 ts = lap.texel_size; + float3 c = input_tex.Sample(input.UV).rgb; + + float3 n = input_tex.Sample(input.UV + float2( 0, -ts.y)).rgb + + input_tex.Sample(input.UV + float2( 0, ts.y)).rgb + + input_tex.Sample(input.UV + float2(-ts.x, 0)).rgb + + input_tex.Sample(input.UV + float2( ts.x, 0)).rgb; + + float3 result; + if (lap.variant == 0) { + result = n - 4.0 * c; + } else { + float3 d = input_tex.Sample(input.UV + float2(-ts.x, -ts.y)).rgb + + input_tex.Sample(input.UV + float2( ts.x, -ts.y)).rgb + + input_tex.Sample(input.UV + float2(-ts.x, ts.y)).rgb + + input_tex.Sample(input.UV + float2( ts.x, ts.y)).rgb; + result = n + d - 8.0 * c; + } + + return float4(result * lap.scale, input_tex.Sample(input.UV).a); +} +``` + +Branching on `variant` inside the shader costs a divergence-free uniform branch +(all invocations take the same path since it comes from a push constant), so it +is essentially free. Alternatively use a Slang specialisation constant. + +### 3.3 Parameters + +| Parameter | Type | Range | Default | +|-----------|------|-------|---------| +| `variant` | u32 | 0 or 1 | 0 | +| `scale` | f32 | 0 – 10 | 1.0 | +| `texel_size` | f32×2 | — | — | + +### 3.4 Performance + +5 or 9 taps, single pass. Trivially cheap. + +**Not separable.** The Laplacian is a sum of two separable second-derivative +filters, but as a single 3×3 it is only rank-2, so a single separable pair +cannot express it. At 3×3 the point is moot — 9 taps is already minimal. + +### 3.5 Compositor notes + +- **Single pass, 1 input.** +- Output is **signed** and centred on zero. Storing it in a UNORM format + destroys the negative half. This node requires a **float or signed + intermediate format** (`R16G16B16A16_SFLOAT`). If the pool only hands out + UNORM, the node must offset by 0.5 and the downstream consumer must know. + Prefer float. +- Very noise-sensitive. A "Laplacian of Gaussian" variant (blur then Laplacian) + is the practical version, and in a node graph the user can just wire a Blur + node in front — which is the whole point of a compositor. No need to build it + into the node. + +--- + +## 4. Sobel + +### 4.1 Kernels + +#### What does Sobel detect? + +The Sobel operator detects **edges** by measuring how fast colour changes in the +horizontal direction (Gx) and the vertical direction (Gy). Think of it as +answering: "is there a brightness change left-to-right? is there one +top-to-bottom?" + +#### Understanding Gx (horizontal edges) + +``` + -1 0 +1 +Gx = -2 0 +2 + -1 0 +1 +``` + +This kernel compares the **right column** (weights +1, +2, +1) against the +**left column** (weights −1, −2, −1). The middle column is ignored (all zeros). +The top and bottom rows get weight 1; the middle row gets weight 2 (more +important — it is closer to the centre pixel). + +If the right side of the neighbourhood is bright and the left side is dark, +Gx produces a large positive value. If the left is bright and the right is dark, +Gx produces a large negative value. If both sides are equal, Gx is zero. + +In short: Gx responds to **vertical edges** (edges that run up-down, which +separate left from right). + +#### Understanding Gy (vertical edges) + +``` + -1 -2 -1 +Gy = 0 0 0 + +1 +2 +1 +``` + +Same idea but rotated: compares the **bottom row** (positive weights) against +the **top row** (negative weights). Responds to **horizontal edges** (edges +that run left-right, separating top from bottom). + +#### Magnitude and direction + +Once you have Gx and Gy, you can compute: + +**Magnitude** — how strong the edge is: +``` +G = sqrt(Gx² + Gy²) // exact (Pythagorean theorem) +G = |Gx| + |Gy| // cheaper approximation (called L1 norm) +``` + +Think of Gx and Gy as two sides of a right triangle. The magnitude is the +hypotenuse. A strong edge in any direction produces a large magnitude. + +**Direction** — which way the edge runs: +``` +θ = atan2(Gy, Gx) +``` + +This tells you the edge orientation (useful for stylised effects that respond to +edge direction). + +#### Why the middle row/column has weight 2 + +The centre pixel is closest to the pixel being processed, so it should matter +more. The `[1, 2, 1]` weighting along the detection direction is a **triangle +filter** — a simple way to give more importance to the centre and less to the +edges. This smooths the result slightly compared to using `[1, 1, 1]`. + +Direction: `θ = atan2(Gy, Gx)`. + +### 4.2 Separability + +Both kernels are separable (outer products): + +``` +Gx = [1, 2, 1]ᵀ · [1, 0, −1] +Gy = [1, 0, −1]ᵀ · [1, 2, 1] +``` + +i.e. smooth with a triangle filter perpendicular to the derivative direction, +and take a central difference along it. Confirmed in the Wikipedia derivation +and Purdue ECE438 lecture notes (Allebach, "Analysis of Sobel Edge Detector"). + +**However**: for a 3×3 kernel, separating costs *more* than it saves on a GPU. +Direct 2D = 8 taps (the centre has weight 0 in both kernels). Separated = 2 +passes × 3 taps = 6 taps, plus an intermediate texture write and read plus a +pipeline barrier. The extra bandwidth dwarfs the 2 saved taps. **Do it in one +pass.** Separation only pays off for large kernels (5×5 upward). + +### 4.3 Scharr alternative + +Sobel has ~1° of rotational-symmetry error. Scharr's optimised 3×3 kernel +reduces this to ~0.2° at identical cost (Wikipedia, OpenCV): + +``` + +3 0 -3 +3 +10 +3 +Gx = +10 0 -10 Gy = 0 0 0 + +3 0 -3 -3 -10 -3 +``` + +Worth exposing as a variant flag — it is strictly better for the same cost. + +### 4.4 Slang implementation + +```slang +struct SobelData { + float2 texel_size; + float scale; + uint output_mode; // 0 = magnitude, 1 = Gx, 2 = Gy, 3 = (Gx,Gy,angle) + uint kernel_mode; // 0 = Sobel, 1 = Scharr + uint edge_mode; // 0 = clamp to edge, 1 = clamp to border + uint pad[2]; +}; + +[[vk::push_constant]] SobelData sobel; +Sampler2D input_tex; + +float luminance(float3 c) { return dot(c, float3(0.2126, 0.7152, 0.0722)); } + +[shader("fragment")] +float4 main(VSOutput input) : SV_Target { + float2 ts = sobel.texel_size; + + // 3x3 neighbourhood, luminance + float tl = luminance(input_tex.Sample(input.UV + float2(-ts.x, -ts.y)).rgb); + float tc = luminance(input_tex.Sample(input.UV + float2( 0.0, -ts.y)).rgb); + float tr = luminance(input_tex.Sample(input.UV + float2( ts.x, -ts.y)).rgb); + float ml = luminance(input_tex.Sample(input.UV + float2(-ts.x, 0.0)).rgb); + float mr = luminance(input_tex.Sample(input.UV + float2( ts.x, 0.0)).rgb); + float bl = luminance(input_tex.Sample(input.UV + float2(-ts.x, ts.y)).rgb); + float bc = luminance(input_tex.Sample(input.UV + float2( 0.0, ts.y)).rgb); + float br = luminance(input_tex.Sample(input.UV + float2( ts.x, ts.y)).rgb); + + float k_edge = (sobel.kernel_mode == 0) ? 1.0 : 3.0; + float k_centre = (sobel.kernel_mode == 0) ? 2.0 : 10.0; + + float gx = (tr + br - tl - bl) * k_edge + (mr - ml) * k_centre; + float gy = (bl + br - tl - tr) * k_edge + (bc - tc) * k_centre; + + // normalise so a full black→white step maps to ~1.0 + float norm = 1.0 / (2.0 * k_edge + k_centre); + gx *= norm; + gy *= norm; + + float4 outc; + switch (sobel.output_mode) { + case 0: { float m = sqrt(gx*gx + gy*gy); outc = float4(m, m, m, 1.0); break; } + case 1: outc = float4(gx, gx, gx, 1.0); break; + case 2: outc = float4(gy, gy, gy, 1.0); break; + default: outc = float4(gx, gy, atan2(gy, gx), 1.0); break; + } + return outc * float4(sobel.scale, sobel.scale, sobel.scale, 1.0); +} +``` + +Design choice: operating on **luminance** (8 fetches, 8 scalar ops) vs +per-channel RGB (8 fetches, 3× the ALU, and a per-channel gradient that must +then be combined). Luminance is the conventional edge detector. Per-channel is +what Di Zenzo's multi-image gradient does and is what the structure tensor +needs (§8). Expose as a mode if needed; default to luminance. + +### 4.5 Parameters + +| Parameter | Type | Range | Default | +|-----------|------|-------|---------| +| `kernel_mode` | u32 | 0 (Sobel), 1 (Scharr) | 0 | +| `output_mode` | u32 | 0–3 | 0 (magnitude) | +| `scale` | f32 | 0 – 10 | 1.0 | + +### 4.6 Performance + +8 taps, single pass. Same class as Laplacian. The `sqrt` and optional `atan2` +are the only non-trivial ALU. Use `|Gx| + |Gy|` if profiling shows the `sqrt` +matters — it will not. + +### 4.7 Compositor notes + +- **Single pass, 1 input.** +- Gx / Gy outputs are signed → **float intermediate format required** for + modes 1–3. Magnitude mode (0) is non-negative and UNORM-safe. +- Angle output in mode 3 is in radians, range `[−π, π]` — definitely not + UNORM-safe. + +--- + +## 5. Sharpening (Unsharp Mask) + +### 5.1 Formula + +#### The intuition + +Unsharp masking is counterintuitively named — it has nothing to do with +"unsharp". The name comes from the traditional photography technique: you create +a blurred copy of the image (the "unsharp" mask), subtract it from the original +to find the edges, then add those edges back at higher strength. + +The key insight: `original − blurred` gives you **only the detail** (the edges +and fine texture), because the blur removed everything except the broad shapes. +Adding this detail back at higher contrast makes edges crisper. + +#### The formula + +``` +sharpened = original + amount · (original − blurred) +``` + +Breaking it down: +- `original − blurred` = the detail/edge information (what the blur removed) +- `amount` = how much extra detail to add (0 = no sharpening, 1 = moderate, 2+ = aggressive) +- Adding this back to the original amplifies the edges + +Equivalently: `sharpened = lerp(blurred, original, 1 + amount)` — this says "blend +beyond the original toward the detail-enhanced version". + +#### Threshold + +Without a threshold, sharpening amplifies **everything** including noise. A +threshold says: "only sharpen where the detail is significant enough": + +``` +mask = original − blurred +delta = |luma(original) − luma(blurred)| // how much detail is here? +mask *= smoothstep(0, threshold, delta) // fade in where detail > threshold +sharpened = original + amount · mask +``` + +`smoothstep(0, threshold, delta)` returns: +- 0 when delta is near 0 (flat region — no sharpening, noise suppressed) +- 1 when delta is above threshold (edge — full sharpening) +- smooth blend in between (no harsh transition) + +#### Why unsharp mask vs. Laplacian sharpen? + +A Laplacian sharpen uses a fixed 3×3 kernel — the "radius" of what counts as +"detail" is always tiny (1 pixel). Unsharp mask lets you choose the radius +through the Gaussian sigma: small σ sharpens fine detail, large σ sharpens +broader edges. This is why unsharp mask is preferred in practice. + +### 5.3 Implementation approach — two options + +**Option A: reuse the Gaussian blur node (3 passes).** +`blur_h → blur_v → combine`. The combine pass takes 2 inputs (original, +blurred) and is a trivial per-pixel op. This reuses the tuned separable blur +and is the correct architecture for a node graph. + +**Option B: fuse the vertical blur pass with the combine (2 passes).** +GEGL does exactly this — pass 1 is horizontal blur, pass 2 does vertical blur +*and* merges with the original in one shader +(`GEGL_unsharp_mask_scl_RT.glsl`: "hblur" then "vblur & merge"). Pass 2 needs +**two** input textures: the h-blurred intermediate and the original. + +Option B saves one full-screen read/write. **Recommend Option B** — the saving +is real and the shader is barely more complex. + +**Do not** use a naive 2D blur inside a single pass (the ComfyUI blueprint does +this with a nested `(2r+1)²` loop). At radius 10 that is 441 taps per pixel. + +### 5.4 Slang implementation (Option B, pass 2) + +```slang +struct SharpenData { + float2 texel_size; + float sigma; + int radius; + float amount; + float threshold; + uint edge_mode; // 0 = clamp to edge, 1 = clamp to border + uint pad; +}; + +[[vk::push_constant]] SharpenData sharp; +Sampler2D hblur_tex; // horizontally blurred intermediate +Sampler2D original_tex; // untouched source + +static const float3 LUMA_709 = float3(0.2126, 0.7152, 0.0722); + +[shader("fragment")] +float4 main(VSOutput input) : SV_Target { + // --- vertical blur of the h-blurred intermediate --- + float inv2s2 = 1.0 / (2.0 * sharp.sigma * sharp.sigma); + float4 blurred = hblur_tex.Sample(input.UV); + float wsum = 1.0; + + for (int i = 1; i <= sharp.radius; ++i) { + float w = exp(-float(i) * float(i) * inv2s2); + float2 off = float2(0.0, sharp.texel_size.y * float(i)); + blurred += (hblur_tex.Sample(input.UV + off) + + hblur_tex.Sample(input.UV - off)) * w; + wsum += 2.0 * w; + } + blurred /= wsum; + + // --- unsharp merge --- + float4 original = original_tex.Sample(input.UV); + float3 mask = original.rgb - blurred.rgb; + + if (sharp.threshold > 1e-4) { + float delta = abs(dot(original.rgb, LUMA_709) - dot(blurred.rgb, LUMA_709)); + mask *= smoothstep(0.0, sharp.threshold, delta); + } + + return float4(original.rgb + mask * sharp.amount, original.a); +} +``` + +Note: **no `saturate` on the output.** The ComfyUI shader clamps to [0,1]; +that is correct for an 8-bit display pipeline and wrong for a scene-linear +compositor, where overshoot should be preserved for downstream nodes. + +### 5.5 Parameters + +| Parameter | Type | Range | Default | Source | +|-----------|------|-------|---------|--------| +| `amount` | f32 | 0.0 – 3.0 | 0.5 – 1.0 | Wikipedia: "50–150%" typical | +| `sigma` / radius | f32 | 0.5 – 10.0 px | 1.0 | Wikipedia: "0.5 to 2 pixels" recommended | +| `threshold` | f32 | 0.0 – 0.1 | 0.0 | ComfyUI blueprint annotation | + +Special case noted by Wikipedia: **local contrast enhancement** uses a large +radius (30–100 px) with a small amount (0.05–0.2). Worth documenting in the +node's tooltip; the same shader handles it. + +### 5.6 Performance + +Same tap count as the Gaussian blur plus 1 extra fetch in pass 2. Dominated by +the blur. + +### 5.7 Compositor notes + +- **2 passes (Option B), 1 user-facing input, 1 scratch texture.** +- Pass 2 binds **2 textures** — this breaks the `input_count = 1` assumption in + the node registry. Either the registry needs per-pass descriptors, or the + sharpen node declares `input_count = 2` with the second slot filled + internally by the scratch texture. +- Overshoot means output exceeds the input range. Another argument for float + intermediates. + +--- + +## 6. Posterize + +### 6.1 Formula + +#### What is posterizing? + +Posterizing reduces the number of distinct colours in an image. A smooth +gradient becomes a series of visible bands (like a poster printed with limited +ink colours). Each band is a flat colour — all pixels in that range get snapped +to the same value. + +#### How does the math achieve this? + +A colour channel is a value from 0 to 1. Posterizing divides this range into +`steps` equal-sized buckets and snaps each pixel to its bucket's value. + +**Full-range variant** (recommended default): + +``` +out = round(in · n − 0.5) / (n − 1) +``` + +Step by step: +1. `in · n` — scale the [0, 1] range to [0, n]. Now each bucket is 1 unit wide. +2. `− 0.5` — shift so that rounding snaps to the centre of each bucket. +3. `round(...)` — snap to the nearest integer (0, 1, 2, ..., n−1). +4. `/ (n − 1)` — scale back to [0, 1]. + +Example with n=6: the output values are `0, 0.2, 0.4, 0.6, 0.8, 1.0`. Both +pure black and pure white are preserved. + +**Blender's variant**: + +``` +out = floor(in · steps) / steps +``` + +Uses `floor` instead of `round`, and divides by `steps` instead of `steps − 1`. +The output values are `0, 1/n, 2/n, ..., (n−1)/n`. Pure white (1.0) is only +reachable if the input is exactly 1.0 — any value slightly below maps to +`(n−1)/n`. This is surprising to artists. + +#### Why does this matter? + +The full-range variant is what most people expect: "posterize to 6 levels" +should give you 6 levels including both black and white. The Blender variant +technically gives you `n` buckets but the top one is unreachable for any value +below 1.0. + +### 6.2 Slang implementation + +```slang +struct PosterizeData { + float steps; // number of levels per channel + uint mode; // 0 = full-range, 1 = Blender-compatible + uint pad[2]; +}; + +[[vk::push_constant]] PosterizeData post; +Sampler2D input_tex; + +[shader("fragment")] +float4 main(VSOutput input) : SV_Target { + float4 src = input_tex.Sample(input.UV); + float n = clamp(post.steps, 2.0, 1024.0); + + float3 c; + if (post.mode == 0) { + c = round(src.rgb * n - 0.5) / (n - 1.0); + } else { + c = floor(src.rgb * n) / n; + } + return float4(c, src.a); +} +``` + +`steps` is a **float** in Blender's implementation, not an integer, which +allows smooth animation of the level count. Keep it float. + +### 6.3 Parameters + +| Parameter | Type | Range | Default | Source | +|-----------|------|-------|---------|--------| +| `steps` | f32 | 2 – 1024 | 8 | Blender clamps exactly to `[2, 1024]` | +| `mode` | u32 | 0 or 1 | 0 | — | + +### 6.4 Performance + +1 tap, ~5 ALU ops. The cheapest filter in this document. Prime fusion +candidate. + +### 6.5 Compositor notes + +- **Single pass, 1 input.** +- Alpha is passed through unquantised (Blender does the same: + `float4(..., color.a)`). Quantising alpha would produce hard matte edges. +- **Only meaningful in a bounded range.** In scene-linear float with values + above 1.0, `floor(in · n)` produces unbounded levels — the filter silently + changes behaviour. If the graph is scene-linear, posterize should either + operate after a display transform, or take explicit `range_min` / `range_max` + parameters. Worth flagging to the user. +- Quantising in linear space gives perceptually uneven bands (crowded in the + shadows). Quantising in a display-referred / gamma space gives the + perceptually even banding artists expect. This is a real design decision, not + a detail. + +--- + +## 7. Pixelize + +### 7.1 Formula + +#### What does pixelize do? + +Pixelize divides the image into a grid of square (or rectangular) blocks and +makes each block a single flat colour. The result looks like an image viewed at +extreme magnification where you can see the individual pixels. + +#### The math — coordinate snapping + +The idea: instead of sampling the texture at the exact output pixel position, +we snap to the centre of whichever block that pixel belongs to. Every pixel in +the same block samples the same texture location, producing a flat colour. + +In pixel space: + +``` +pixel = uv · resolution // convert to pixel coordinates +block_centre = (floor(pixel / block_size) + 0.5) · block_size +out = sample(input, block_centre / resolution) +``` + +Step by step: +1. `uv · resolution` — convert normalized UV [0,1] to pixel coordinates [0, width]. +2. `pixel / block_size` — divide by block size. Now each block spans 1 unit. +3. `floor(...)` — snap to the block index (integer). All pixels in the same block + get the same integer. +4. `+ 0.5` — move to the centre of the block (not the edge). Without this, + sampling happens at the block boundary, causing the whole image to shift + by half a block. +5. `· block_size` — convert back to pixel coordinates (now at the block centre). +6. `/ resolution` — back to UV coordinates for sampling. + +#### Why `+ 0.5` matters + +Without `+ 0.5`, you sample at the block's top-left corner instead of its +centre. This means: +- Each block gets the colour of the texel at its top-left corner. +- The image shifts up and to the left by half a block. +- The bottom-right row/column of blocks may sample outside the image. + +With `+ 0.5`, you sample at the block's centre, which is the most +representative point. + +### 7.2 Point sample vs block average + +Two distinct behaviours, both called "pixelize": + +**Nearest / point sample.** One tap. The block takes the colour of its +top-left-ish representative pixel. Fast, aliases badly on detailed input +(the chosen pixel is arbitrary noise). + +**Block average (true downsample).** Averages all `block_size²` pixels. Correct, +alias-free, but `block_size²` taps per fragment — at block size 32 that is +1024 taps *per output pixel*, and every pixel in the block does the same work +redundantly. Catastrophic. + +**The right implementation of block-average is a two-stage resample:** +1. Render to a texture of size `ceil(resolution / block_size)`, sampling with + a box filter (or just bilinear + mip bias). +2. Render back to full resolution with **nearest** sampling. + +This costs one small render plus one cheap upsample. This is what a compositor +should do. + +A cheap middle ground: point-sample from a **mip level** chosen as +`log2(block_size)`. One tap, and the mip chain has already done the box +averaging. Requires the input texture to have mips, which a pool-allocated +intermediate normally does not. Only viable if the pool is taught to allocate +mipped intermediates — probably not worth it. + +### 7.3 Slang implementation (single-pass, point sample) + +```slang +struct PixelizeData { + float2 resolution; // input dimensions in pixels + float2 block_size; // block dimensions in pixels (x, y — allows non-square) +}; + +[[vk::push_constant]] PixelizeData pix; +Sampler2D input_tex; // MUST use a nearest/point sampler + +[shader("fragment")] +float4 main(VSOutput input) : SV_Target { + float2 pixel = input.UV * pix.resolution; + float2 block_centre = (floor(pixel / pix.block_size) + 0.5) * pix.block_size; + return input_tex.Sample(block_centre / pix.resolution); +} +``` + +Note `block_size` is a `float2`. Non-square blocks are a legitimate artistic +control and cost nothing. + +If the sampler is linear rather than nearest, the result is subtly wrong: at +the block centre bilinear still blends 4 texels. Use a **nearest** sampler for +this node, or the two-stage approach in §7.2. + +### 7.4 Parameters + +| Parameter | Type | Range | Default | +|-----------|------|-------|---------| +| `block_size` | f32×2 | 1 – 512 px | 8, 8 | +| `resolution` | f32×2 | — | — | + +`block_size = 1` must be an exact pass-through. `floor(pixel/1) + 0.5` lands +on the texel centre, so it is — good, no special case needed. + +A non-integer `block_size` produces blocks of unequal pixel counts (some 3px, +some 4px at `block_size = 3.5`). That is expected and allows smooth animation. + +### 7.5 Performance + +1 tap, ~6 ALU. Effectively free. + +**Cache behaviour is actually excellent**, counter to intuition: every fragment +in a block reads the *same* texel, so the texture cache hit rate approaches +100%. Pixelize is cheaper than a plain blit in practice. + +### 7.6 Compositor notes + +- **Single pass, 1 input** for point-sample mode. +- **2 passes + 1 scratch texture at reduced resolution** for block-average + mode. The scratch texture is smaller than the output — the texture pool must + support requesting arbitrary sizes, which per `TEXTURE_POOL_AND_NODE_EVAL.md` + §2.6 it does. +- The block grid is anchored to the image origin. If the node graph has any + concept of image offset / bounding box, the grid should be anchored + consistently or blocks will crawl when the image moves. Add a + `float2 origin_offset` parameter if that becomes an issue. +- Requires a **nearest sampler**, unlike every other node here. The descriptor + set layout must allow per-node sampler choice. + +--- + +## 8. Kuwahara + +This is by far the most complex filter in the set. Three tiers exist, with +very different cost and quality. + +### 8.1 Tier 1 — Classic Kuwahara (4 square sub-regions) + +#### What does Kuwahara do? + +The Kuwahara filter smooths an image while **preserving edges**. Think of it as +a smart blur: in flat regions it blurs normally (removing noise), but at edges +it only blurs *along* the edge (never across it), keeping the edge sharp. + +This produces a painterly, oil-painting-like effect that has been used in art +and video for decades. + +#### How does it work? + +For each pixel, look at a square window around it (say 11×11 pixels). Divide +this window into 4 overlapping quadrants (top-left, top-right, bottom-left, +bottom-right). For each quadrant: + +1. **Compute the mean** — the average colour in that quadrant. +2. **Compute the variance** — how much the colours in that quadrant vary. + +Then output the mean of the quadrant with the **lowest variance**. + +#### Why does this preserve edges? + +Consider a pixel sitting exactly on a bright/dark edge: +- The quadrants that sit entirely on one side of the edge have low variance + (all similar colours) → they are flat, safe to blur. +- The quadrants that straddle the edge have high variance (mix of bright and + dark) → they contain the edge, so we avoid them. + +By always picking the flattest quadrant, we never blur across the edge — we +only blur along it. + +#### The math + +``` +for each of 4 quadrants q: + mean_q = (1/n) Σ colour // average colour + var_q = (1/n) Σ colour² − mean_q² // variance (spread of colours) + scalar_q = dot(var_q, luma_weights) // collapse RGB variance to one number +output = mean_q where q = argmin(scalar_q) // pick the quadrant with least variance +``` + +**Mean**: sum all colours, divide by count. This is the average. + +**Variance**: for each pixel, compute `(colour − mean)²`, then average those. A +low variance means all pixels are similar to the mean (flat region). A high +variance means pixels are very different from each other (edge or texture). + +The formula `var = mean(colour²) − mean(colour)²` is a mathematically +equivalent shortcut that avoids a separate subtraction pass. + +#### Problems with the classic version + +- **Hard selection** (picking exactly one quadrant) causes blocky artifacts — + neighbouring pixels may select different quadrants, creating visible seams. +- **Flicker** in video: a tiny change in the image can flip which quadrant is + selected, causing temporal instability. +- **Axis-aligned blocks**: square quadrants produce visible square patterns. + +### 8.2 Tier 2 — Generalized Kuwahara (8 disc sectors, weighted sum) + +#### Improving on the classic + +Two key improvements: + +1. **More regions**: instead of 4 squares, use 8 (or more) sectors arranged + like slices of a pie (a disc). This eliminates the axis-aligned blockiness + — the result is more rotationally uniform. + +2. **Weighted sum instead of hard selection**: instead of picking exactly one + sector, blend all sectors together using weights based on their variance. + Sectors with low variance (flat regions) get high weight. Sectors with high + variance (edges) get low weight. This eliminates the blocky artifacts and + flicker. + +The weight formula: + +``` +w_i = 1 / (1 + (α · σ_i²)^(q/2)) +output = Σ (w_i · mean_i) / Σ w_i +``` + +#### Understanding the weight formula + +- `σ_i²` is the variance of sector `i` (how much colours vary in that sector). +- `α` (hardness) amplifies the variance — higher α means even small variances + get heavily penalised. +- `q` (sharpness) controls how sharply the weight drops off with variance. +- When variance is low (flat region): `α·σ²` is small, the denominator is close + to 1, so `w_i` is close to 1 (high weight — this sector contributes). +- When variance is high (edge): `α·σ²` is large, the denominator is large, so + `w_i` is close to 0 (low weight — this sector is ignored). + +The final output is a weighted average of all sector means, where flat sectors +dominate and edge sectors are suppressed. +``` + +with `q` controlling the sharpness of the selection. The Pixel Composer +implementation (`sh_kuwahara.fsh`, derived from Acerola's port) uses +`q = 18` and `α = 1000`: + +```glsl +float sigma2 = s[k].r + s[k].g + s[k].b; +float w = 1.0 / (1.0 + pow(1000.0 * sigma2, 0.5 * q)); +``` + +Higher `q` → closer to hard argmin (sharper region boundaries, more artifacts). +Lower `q` → smoother blending, mushier edges. + +Kyprianidis et al. (2011, NPAR) additionally recommend **thresholding the +standard deviation before exponentiation** to avoid divide-by-zero in flat +regions and random weight selection under noise: typical values `q = 8` and +`σ_threshold = 0.02`. + +3. Papari also adds a **Gaussian radial falloff** so pixels far from the sector + centre contribute less, which removes the remaining clustering artifacts. + +### 8.3 Tier 3 — Anisotropic Kuwahara (structure-tensor guided) + +Kyprianidis, Kang & Döllner (2009, Computer Graphics Forum 28(7); GPU +implementation in GPU Pro, 2010). The disc becomes an **ellipse** whose +orientation and eccentricity follow the local image structure. This is the +version that produces the recognisable "oil painting" look with directional +brush strokes. + +**Pipeline — 3 or 4 passes:** + +**Pass 1 — structure tensor.** The structure tensor captures the **local image +geometry**: which direction does the image change most, and is it the same in all +directions? + +It uses Sobel-like gradients `fx` (horizontal change) and `fy` (vertical change) +at each pixel, then computes: + +``` +E = dot(fx, fx) // how much change is in x squared +F = dot(fx, fy) // how much x and y change together +G = dot(fy, fy) // how much change is in y squared +``` + +These 3 values (E, F, G) form a 2×2 matrix `[[E, F], [F, G]]`. This matrix +describes the local image structure. Output as `float4(E, F, F, G)`. + +Intuition: if the image has a strong vertical edge, `fx` will be large (big +change horizontally) and `fy` near zero (no change vertically along the edge). +So E will be large, G small. + +**Pass 2 — smooth the tensor.** We smooth E, F, G with a Gaussian blur. This is +important: we smooth the *tensor components*, not the image. This lets +neighbouring pixels "agree" on the local geometry. Without this, noise would +make the filter unstable. + +**Pass 3 — eigenanalysis.** We extract two properties from the smoothed tensor: + +- **Eigenvalues** (λ₁, λ₂): how strong the structure is in its two principal + directions. λ₁ is the strongest direction, λ₂ the weakest. +- **Eigenvector**: the direction of λ₁ (the dominant edge direction). +- **Anisotropy** `A = (λ₁ − λ₂) / (λ₁ + λ₂)`: 0 = isotropic (same in all + directions), 1 = fully directional (one strong direction). + +``` +phi = atan2(t.y, t.x) // orientation of the dominant direction +A = (lambda1 - lambda2) / (lambda1 + lambda2) // eccentricity +``` + +This tells the final pass: "the edge here runs at angle φ, and it is A-times +stronger in one direction than the other." + +`A ∈ [0, 1]`: 0 = isotropic, 1 = fully directional. This pass can be **fused +into pass 4** — Kyprianidis keeps it separate to amortise it, but fusing saves +a full-screen texture round trip. Fuse it. + +**Pass 4 — the filter itself.** Ellipse axes from anisotropy (Kyprianidis 2009 +§3.2, with `α` controlling eccentricity — Heckel uses `alpha = 25.0`): + +``` +a = α / (A + α) // scale along the edge direction +b = (A + α) / α // scale across it +S = diag(1/a, 1/b); R = rotation(φ) +M = S · R // maps the rotated ellipse to the unit disc +``` + +Then loop over the ellipse's bounding box, map each offset into the unit disc +via `M`, reject points outside, and accumulate into 8 sectors. + +**Polynomial weighting functions** (Kyprianidis, Semmo, Kang & Döllner 2010, +TPCG) replace the original texture-lookup weights with a closed-form +polynomial that needs no texture at all. Verbatim from Blender's +`compositor_kuwahara_anisotropic.glsl`: + +```glsl +float2 polynomial = sector_center_overlap - cross_sector_overlap * square(disk_point); +sector_weights[0] = square(max(0.0, disk_point.y + polynomial.x)); +sector_weights[2] = square(max(0.0, -disk_point.x + polynomial.y)); +sector_weights[4] = square(max(0.0, -disk_point.y + polynomial.x)); +sector_weights[6] = square(max(0.0, disk_point.x + polynomial.y)); + +float2 rotated = M_SQRT1_2 * float2(disk_point.x - disk_point.y, + disk_point.x + disk_point.y); +float2 rot_poly = sector_center_overlap - cross_sector_overlap * square(rotated); +sector_weights[1] = square(max(0.0, rotated.y + rot_poly.x)); +sector_weights[3] = square(max(0.0, -rotated.x + rot_poly.y)); +sector_weights[5] = square(max(0.0, -rotated.y + rot_poly.x)); +sector_weights[7] = square(max(0.0, rotated.x + rot_poly.y)); +``` + +The trick: 90° rotations are coordinate swaps and negations, so 4 weights come +free from one evaluation, and a single explicit 45° rotation yields the other +4. `max()` is used instead of a branch — Kyprianidis notes this explicitly +("on GPUs this function is generally available as intrinsic and much faster, +since no branching is performed"). + +Radial Gaussian component, also from Blender: + +```glsl +float radial_gaussian_weight = exp(-M_PI * disk_point_length_squared) + / sector_weights_sum; +``` + +**Two significant optimisations from Blender's implementation:** +- The ellipse is **mirror-symmetric**, so only the upper two quadrants are + iterated; each iteration processes a pixel and its mirror with the same + weight, accumulating the mirror into sector `(k + N/2) % N`. **Halves the + loop.** +- The centre pixel is accumulated once, before the loop, with weight `1/N` in + every sector. + +Pixel Composer's variant is the Papari-style isotropic version with polynomial +weights (`zeta = 2/radius`, `eta = 0`, radial term `exp(-3.125 · dot(v,v))`) — +a good simpler reference if the anisotropic version is too much for a first +implementation. + +### 8.4 Parameters + +| Parameter | Type | Range | Default | Source | +|-----------|------|-------|---------|--------| +| `radius` | i32 | 1 – 32 | 5 | Kyprianidis benchmarks at r = 3–10 | +| `sectors` (N) | u32 | 4 or 8 | 8 | Papari 2007: 8 optimal; N=4 "slightly less sharp boundaries" (Kyprianidis 2009 §4) | +| `sharpness` (q) | f32 | 1 – 24 | 8 | Kyprianidis 2011: `q = 8`; Pixel Composer uses 18 | +| `hardness` (α for weights) | f32 | 1 – 1000 | 1000 | Pixel Composer / Acerola | +| `eccentricity` (α) | f32 | 1 – 50 | 25 | Heckel uses 25; controls ellipse stretch | +| `sigma_threshold` | f32 | 0.0 – 0.1 | 0.02 | Kyprianidis 2011 §3.3.1 | +| `tensor_sigma` | f32 | 1 – 4 | 2.0 | Kyprianidis: 7×7 or 9×9 Gaussian | + +### 8.5 Slang implementation (classic tier) + +```slang +struct KuwaharaData { + float2 texel_size; // 1.0 / texture dimensions + int radius; + uint sectors; // 4 or 8 + float sharpness; // q + float hardness; // α + uint edge_mode; // 0 = clamp to edge, 1 = clamp to border +}; + +[[vk::push_constant]] KuwaharaData kyw; +Sampler2D input_tex; +``` + +### 8.7 Performance + +This is expensive and the numbers are worth stating plainly. + +Kyprianidis' own measurements (2009 paper, §4): **512×512 at 12 fps** on a +GeForce GTX 280 with `r = 3.0, N = 8`. The 2011 multi-scale CUDA version: +**42 ms for 512×512**, **150 ms for 1280×720** on a GTX 580. + +TPCG 2010 Table 1, 512×512, texture-based vs polynomial weights: + +| GPU | N=4 (texture) | N=8 (texture) | N=4 (poly) | N=8 (poly) | +|-----|---------------|---------------|------------|------------| +| GTX 285 | 10.8 ms | 36.0 ms | 11.9 ms | 40.0 ms | +| GTX 480 | 5.6 ms | 28.3 ms | 6.1 ms | 23.1 ms | +| Radeon 5850 | 17.9 ms | 41.7 ms | 18.5 ms | 35.3 ms | + +Observations: +- **N=8 is roughly 3–4× the cost of N=4.** Offer N as a user parameter. +- Polynomial weights **win on newer hardware for N=8** (GTX 480: +22.5%, + Radeon 5850: +18.1%) and lose slightly on older hardware. On any GPU from + the last decade, use polynomial weights — memory access is the bottleneck, + and the texture-based approach needs 2 lookups per kernel element at N=8. +- Modern GPUs are far faster than a GTX 480, but the **asymptotic cost is + unchanged**: `O(r²)` taps per pixel, each feeding 8 sector accumulators. + At r = 10 that is ~300 taps × 8 sectors of accumulation. + +**Register pressure is the real constraint.** 8 sectors × (mean float4 + +mean-of-squares float4 + weight float) = 8 × 9 = **72 floats of live state** +per invocation, plus loop variables. That is well past the point where +occupancy collapses on most architectures. Mitigations: +- Use `float3` instead of `float4` for the accumulators (alpha need not be + variance-tracked) → 8 × 7 = 56 floats. +- Track variance on **luminance only** rather than per-channel → 8 × 5 = 40 + floats. Quality loss is minor since variance is collapsed to a scalar anyway. +- Drop to N = 4. + +**Do not** let `radius` be unbounded from a UI slider. Blender's shader uses a +`MAX_RAD` compile-time bound with `continue`/`break` guards; Pixel Composer +uses `#define MAX_RAD 64`. Do the same — an unbounded nested loop driven by a +push constant is a hang waiting to happen. + +### 8.8 Compositor notes + +- **Classic (tier 1): 1 pass, 1 input.** Reasonable first implementation. +- **Anisotropic (tier 3): 4 passes** (tensor → blur H → blur V → filter), + **2 scratch textures** (tensor, blurred tensor), 1 user input. + The final pass needs **2 bound textures**: the original image and the + smoothed tensor. +- The tensor textures need at least `R16G16B16A16_SFLOAT`. Squared gradients + in an HDR image easily exceed the fp16 max (65504) if input values are large + — `dot(fx, fx)` on a value-100 highlight gives ~30000 per channel summed over + 3 channels. **Use `R32G32B32A32_SFLOAT` for the tensor** unless input is + known to be display-referred. +- Structure tensor smoothing being separable means passes 2 and 3 reuse the + **exact same Gaussian blur shader** as §1, on a different texture. Good + argument for making the blur shader a reusable internal pass rather than only + a user-facing node. +- Temporal stability: the weighted-sum formulation (tier 2+) is what gives + Kyprianidis' "outstanding temporal coherence… without motion estimation". + The hard-argmin classic version flickers. If Prism ever does video, tier 1 is + not acceptable. + +--- + +## 9. Unpremult + +### 9.1 Formula + +#### What is premultiplied alpha? + +Normally, a pixel stores `(R, G, B, A)` where R, G, B are the colour and A is +opacity (0 = invisible, 1 = fully visible). This is called **straight alpha**. + +**Premultiplied alpha** stores `(R·A, G·A, B·A, A)` — each colour channel is +multiplied by the alpha. A fully transparent pixel (A=0) stores `(0, 0, 0, 0)` +regardless of its R, G, B values. + +Why do this? Because compositing ("put image A over image B") becomes a single +addition instead of a division. It is the standard in professional compositing. + +#### What does Unpremult do? + +Unpremult reverses the premultiplication — it recovers the original colour +values before they were multiplied by alpha: + +``` +out.RGB = in.RGB / A // reverse the multiply +out.A = in.A // alpha unchanged +``` + +Example: a pixel with premultiplied values `(0.2, 0.1, 0.0, 0.5)` means the +original colour was `(0.2/0.5, 0.1/0.5, 0.0/0.5) = (0.4, 0.2, 0.0)` at 50% +opacity. + +#### Why the epsilon guard? + +Division by zero is undefined. When A is 0 (or very close to 0), dividing by A +would produce infinity or NaN. Since a fully transparent pixel has no meaningful +colour, we just output `(0, 0, 0)` in that case. The epsilon check (`A > 1e-6`) +prevents this. + +### 9.2 Slang implementation + +```slang +struct UnpremultData { + float epsilon; + uint pad[3]; +}; + +[[vk::push_constant]] UnpremultData up; +Sampler2D input_tex; + +[shader("fragment")] +float4 main(VSOutput input) : SV_Target { + float4 src = input_tex.Sample(input.UV); + float3 c = (src.a > up.epsilon) ? (src.rgb / src.a) : float3(0.0, 0.0, 0.0); + return float4(c, src.a); +} +``` + +### 9.3 Parameters + +| Parameter | Type | Range | Default | +|-----------|------|-------|---------| +| `epsilon` | f32 | 1e-8 – 1e-4 | 1e-6 | + +### 9.4 Compositor notes + +- **Single pass, 1 input.** No edge parameter (no neighbour reads). +- Per-pixel ALU only — the divide is the only non-trivial operation. +- Typical placement: immediately before a colour operator (CDL, posterize). +- Often paired with a Premult node after the operation. + +--- + +## 10. Premult + +### 10.1 Formula + +#### What does Premult do? + +Premult applies the premultiplication — it multiplies each colour channel by +the alpha value. This is the reverse of Unpremult: + +``` +out.RGB = in.RGB · A // multiply colour by opacity +out.A = in.A // alpha unchanged +``` + +Example: a pixel with straight values `(0.4, 0.2, 0.0, 0.5)` becomes +`(0.4·0.5, 0.2·0.5, 0.0·0.5, 0.5) = (0.2, 0.1, 0.0, 0.5)`. + +After applying a colour operation (CDL, posterize) on unpremultiplied values, +you use Premult to return the pixel to the premultiplied pipeline format. + +### 10.2 Slang implementation + +No push constants needed — the operation is parameterless. + +```slang +Sampler2D input_tex; + +[shader("fragment")] +float4 main(VSOutput input) : SV_Target { + float4 src = input_tex.Sample(input.UV); + return float4(src.rgb * src.a, src.a); +} +``` + +### 10.3 Parameters + +None. The operation is purely structural. + +### 10.4 Compositor notes + +- **Single pass, 1 input.** No edge parameter. +- One multiply per channel — cheaper than Unpremult. +- Typical placement: immediately after a colour operator, paired with a preceding + Unpremult. +- Fusion candidate: `Unpremult → CDL → Premult` can be fused into a single pass + by a smart evaluator, eliminating two full-screen round trips. + +--- + +## 11. Summary table + +| Filter | Passes | Inputs | Scratch textures | Taps/px | Push const size | Signed output | +|--------|--------|--------|------------------|---------|-----------------|---------------| +| Gaussian Blur | 2 | 1 | 1 | 2·(2r+1), or (2r+1) with linear sampling | 40 B | no | +| CDL | 1 | 1 | 0 | 1 | 48 B | possible (noClamp) | +| Laplacian | 1 | 1 | 0 | 5 or 9 | 20 B | **yes** | +| Sobel | 1 | 1 | 0 | 8 | 32 B | **yes** (Gx/Gy/angle modes) | +| Sharpen (USM) | 2 | 1 (+1 internal) | 1 | 2·(2r+1) + 1 | 40 B | **yes** (overshoot) | +| Posterize | 1 | 1 | 0 | 1 | 16 B | no | +| Pixelize (point) | 1 | 1 | 0 | 1 | 16 B | no | +| Pixelize (average) | 2 | 1 | 1 (reduced res) | 1 + box | 16 B | no | +| Kuwahara (classic) | 1 | 1 | 0 | ~(2r+1)² | 24 B | no | +| Kuwahara (anisotropic) | 4 | 1 (+2 internal) | 2 | ~πr²·N accum | 48 B | no | +| Unpremult | 1 | 1 | 0 | 1 | 16 B | no | +| Premult | 1 | 1 | 0 | 1 | 0 B | no | + +All push-constant blocks fit comfortably within the 128-byte Vulkan minimum. +**No filter in this set requires a uniform buffer.** + +### Note on push constant sizes for spatial filters + +Spatial filters include `float2 texel_size` in their push constant block, which +was accounted for in the table above. The Gaussian blur and sharpen blocks are +40 bytes (up from the 32 bytes in the original research) because they now include +the edge handling mode flag alongside `texel_size`, `sigma`, `radius`, and +`direction`/`amount`/`threshold`. + +--- + +## 12. Implications for Prism's node system + +These are the points where the filters above interact with the current design +in `documents/TEXTURE_POOL_AND_NODE_EVAL.md`. Resolved items are marked. + +### 12.1 Open: node graph architecture + +1. **`input_count` is not enough.** The registry (§3.1) has a single + `input_count` per node type. Sharpen pass 2 binds 2 textures, anisotropic + Kuwahara pass 4 binds 2. The registry needs per-pass resource signatures, or + multi-pass nodes need to be modelled as a small internal sub-graph. + +2. **Multi-pass nodes need scratch textures.** The pool's refcount model + assumes one output per node. A blur node must acquire a scratch texture, + use it as pass-1 output and pass-2 input, then release it — all within one + node's evaluation. The pool API supports this, but the evaluation loop + (§4.1) does not currently have a hook for it. + +3. **Loop bounds must be compile-time-bounded.** Gaussian radius, Kuwahara + radius, and unsharp radius all come from the UI. Every one of these loops + needs a hard `MAX_*` constant and a `break`, matching what Blender and Pixel + Composer do. + +### 12.2 Resolved decisions + +4. ~~**Intermediate format must be float.**~~ **Resolved: all intermediates are + linear float (`R16G16B16A16_SFLOAT`, `R32G32B32A32_SFLOAT` for Kuwahara + tensor).** Laplacian, Sobel (gradient modes), and sharpen all produce values + outside [0,1]. Float intermediates preserve them. The Kuwahara structure + tensor specifically needs `R32G32B32A32_SFLOAT` (squared HDR gradients exceed + fp16 max). + +5. ~~**Sampler state varies per node.**~~ **Resolved: per-node edge handling + parameter.** Spatial filters expose an edge handling mode (clamp-to-edge + default, clamp-to-border option). Pixelize uses nearest sampling. The + descriptor set layout or sampler binding must support per-node sampler choice. + +6. ~~**Alpha handling.**~~ **Resolved: premultiplied pipeline with explicit + Unpremult/Premult nodes (§9–§10).** The user inserts these before/after + colour operators that need unpremultiplied values. Spatial filters operate + directly on premultiplied input. + +7. ~~**Colour space is a graph-wide decision."~~**Resolved: all intermediates + are linear float (§0.4).** sRGB images are linearized once at load by the + Read node. The final blit to the sRGB swapchain handles display encoding. + No per-filter colour-space logic needed. + +--- + +## References + +**Specifications** +- ITU-R BT.709-6, *Parameter values for the HDTV standards* — luma + coefficients 0.2126 / 0.7152 / 0.0722. +- Autodesk CTF `ASC_CDL` operator reference — SOP + saturation styles + `v1.2_Fwd`, `v1.2_Rev`, `noClampFwd`, `noClampRev`. +- Pomfort, *An in-depth look at ASC-CDL based color controls* (2019). + +**Reference implementations** +- OpenColorIO, `src/OpenColorIO/ops/cdl/CDLOpCPU.cpp` — authoritative CDL + operator order and clamping behaviour. +- Blender compositor shaders: + `compositor_kuwahara_anisotropic.glsl`, + `compositor_kuwahara_anisotropic_compute_structure_tensor.glsl`, + `gpu_shader_compositor_posterize.glsl`. +- Jan Eric Kyprianidis, `jkyprian/gpuakf` and `jkyprian/polyakf` (GLSL + reference implementations; `tfm.glsl` for the eigenanalysis). +- GEGL unsharp mask, ported to mpv as `GEGL_unsharp_mask_scl_RT.glsl` — + 2-pass h-blur / v-blur+merge structure. +- Pixel Composer, `shaders/sh_kuwahara/sh_kuwahara.fsh` — generalized Kuwahara + with polynomial weights. + +**Papers and articles** +- Kyprianidis, Kang & Döllner (2009), *Image and Video Abstraction by + Anisotropic Kuwahara Filtering*, Computer Graphics Forum 28(7). +- Kyprianidis, Kang & Döllner (2010), *Anisotropic Kuwahara Filtering on the + GPU*, in GPU Pro, AK Peters, pp. 247–264. +- Kyprianidis, Semmo, Kang & Döllner (2010), *Anisotropic Kuwahara Filtering + with Polynomial Weighting Functions*, EG UK TPCG, pp. 25–30. +- Kyprianidis (2011), *Image and Video Abstraction by Multi-scale Anisotropic + Kuwahara Filtering*, NPAR. +- Papari, Petkov & Campisi (2007), *Artistic Edge and Corner Enhancing + Smoothing*, IEEE TIP 16(10), pp. 2449–2462. +- Daniel Rákos / RasterGrid (2010), *Efficient Gaussian blur with linear + sampling*. +- Maxime Heckel (2024), *On Crafting Painterly Shaders*. +- Wikipedia, *Sobel operator* and *Unsharp masking*. +- OpenCV documentation, *Sobel Derivatives* tutorial. + +**Slang / Vulkan** +- Vulkan Guide, *High Level Shader Language Comparison* — `[[vk::push_constant]]`, + `[[vk::binding]]`. +- Slang Documentation, *SPIR-V-Specific Functionalities* — combined samplers + (`Sampler2D` → `OpTypeSampledImage`), push-constant std430 layout. +- Slang Documentation, *Using Slang Parameter Blocks* — one push-constant + `ConstantBuffer` per entry point. diff --git a/documents/session-logs/2026-08-09.md b/documents/session-logs/2026-08-09.md new file mode 100644 index 0000000..de3aee9 --- /dev/null +++ b/documents/session-logs/2026-08-09.md @@ -0,0 +1,59 @@ +# Session Log — 2026-08-09 + +## Background colour change (blit shader) + +- User requested changing the letterbox/pillarbox background from black to neutral grey. +- Initial attempt: changed the render pass clear color to `(0.5, 0.5, 0.5, 1.0)`. This + triggered the NVIDIA validation layer warning + `BestPractices-NVIDIA-ClearColor-NotCompressed` — SRGB fast clears only work + with 0.0 or 1.0 on NVIDIA tile-based GPUs. +- Reverted the clear color and implemented the proper solution: draw a fullscreen + grey quad in the fragment shader before the texture quad. The render pass clear + stays at 0.0 (fast-compressed). +- Added `mode` field to `BlitData` push constant. Mode 0 samples the texture, mode + 1 outputs solid grey. +- User noted that a branch in the shader is free (no warp divergence since `mode` + is uniform per draw call). Agreed — no need for a separate clear pipeline. +- Changed grey from 0.5 to 0.18 (18% grey card, standard in photography/compositing). +- Fixed a Slang compilation warning by updating the profile from `spirv_1_4` to + `spirv_1_6` and explicitly declaring the required capabilities. + +## Shader filter node research + +- User requested research on: Gaussian blur, CDL, Laplacian, Sobel, sharpen, + posterize, pixelize, Kuwahara. +- Launched a research agent that produced `documents/research/shader-filters.md` + covering all filters with formulas, Slang pseudocode, parameter tables, and + performance notes. + +## Design decisions made during review + +1. **Colour space**: all intermediate textures are linear float + (`R16G16B16A16_SFLOAT`, `R32G32B32A32_SFLOAT` for Kuwahara tensor). sRGB images + are linearized once at load by the Read node. Final blit to sRGB swapchain + handles display encoding. + +2. **Alpha**: premultiplied everywhere by default. Explicit Unpremult/Premult + nodes for operations that need unpremultiplied values (Nuke model). + +3. **Edge handling**: per-node parameter, clamp-to-edge default, clamp-to-border + option. Affects sampler state, not shader branches. + +4. **Premult has no parameters**: removed the empty push constant struct. + +## Research document fixes + +- Fixed a contradictory sentence about push constant sizes and CDL block size. +- Added Unpremult (§9) and Premult (§10) sections with full implementations. +- Added `edge_mode` field to all 5 spatial filter push constant blocks (Gaussian, + Laplacian, Sobel, Sharpen, Kuwahara). +- Restructured the implications section (§12) into open items vs resolved decisions. +- Expanded all mathematics sections with plain-language explanations suitable for + someone without a strong math background. + +## Open items for next session + +- Begin implementing the actual shader nodes in Prism +- Node system needs: per-pass resource signatures, scratch texture hooks, per-node + sampler choice, compile-time-bounded loop limits +- Classic Kuwahara is the recommended first implementation (single pass)