DAGs and ImGui node graphs research

2026-06-28 13:51:16 +01:00
parent 2f837048d0
commit 832b60356b
2 changed files with 396 additions and 0 deletions
@@ -0,0 +1,184 @@
+# DAGs for Node-Based Compositing
+
+## What is a DAG?
+
+A **Directed Acyclic Graph (DAG)** is a graph where:
+
+- **Directed** — every edge has a direction (A → B means "A feeds into B")
+- **Acyclic** — no path forms a cycle; you cannot loop back to a node you've already visited
+
+In compositing, nodes are the vertices and image/data flow defines the edges. A **Read** node feeds into a **Blur** node, which feeds into a **Merge** node, etc.
+
+## Why DAGs for Compositing
+
+| Property | Benefit |
+|---|---|
+| Non-linear | Any node can be tweaked without rebuilding the whole comp |
+| Dependency-driven | Only re-evaluate nodes whose inputs changed ("dirty propagation") |
+| Parallelism | Independent branches can run concurrently |
+| Modular | Nodes are self-contained; easy to add new types |
+
+### Real-world examples
+
+- **Nuke** (Foundry) — the industry standard; everything is a DAG
+- **Fusion** (Blackmagic) — same concept, node graph as the primary interface
+- **Blender Compositor** — uses a dependency graph internally
+- **Shadertoy / MaterialX** — similar DAG concepts for shader/node graphs
+
+## Core Algorithms
+
+**BFS** (Breadth-First Search) explores a graph level by level — you visit all neighbours of a node before moving to their neighbours. Useful for shortest paths and spreading outward.
+
+**DFS** (Depth-First Search) goes deep first — you follow one path as far as it goes, then backtrack. Useful for cycle detection, pathfinding, and topological sort.
+
+### Topological Sort
+
+A **topological ordering** of a DAG is a linear sequence of all vertices such that for every edge u → v, u appears before v. This is the evaluation order for a node graph — you must process a node's inputs before processing the node itself.
+
+There are two standard approaches, one based on each traversal strategy:
+
+### 1. Kahn's Algorithm (BFS-based)
+
+Uses **in-degree** (number of incoming edges) to determine which nodes are ready to execute.
+
+```
+1. Compute in-degree for every node
+2. Queue all nodes with in-degree == 0 (no dependencies)
+3. While queue is not empty:
+   a. Dequeue node n, add to result order
+   b. For each downstream node m of n:
+      - Decrement m's in-degree
+      - If m's in-degree reaches 0, enqueue m
+4. If result count != node count → there is a cycle
+```
+
+**Why Kahn's for compositing:**
+- Naturally detects cycles (a graph editor must prevent the user from creating cycles)
+- Gives you ready-to-evaluate layers (all in-degree-0 nodes at a given step can run in parallel)
+- O(V + E) time, O(V) space
+- Easy to implement with arrays
+
+### 2. DFS-based (Post-order)
+
+```
+1. For each unvisited node, run DFS
+2. After visiting all descendants of a node, prepend it to the result
+```
+
+Simpler to code but less practical for incremental / parallel evaluation. Used more for DAG verification in build systems.
+
+## Data Structures for a DAG
+
+### Adjacency List (recommended for Prism)
+
+Store the graph as two flat arrays:
+
+```c
+// Node i's outgoing edges are adjacency[i] .. adjacency[i + 1]
+u32 *adjacency;       // flat list of edge destinations
+u32 *adjacency_begin; // start index into adjacency for each node
+u32  node_count;
+u32  edge_count;
+```
+
+Or simpler: each node stores its outputs:
+
+```c
+typedef struct PrNode PrNode;
+struct PrNode {
+    u32          id;
+    PrNodeType   type;
+    u32          input_count;     // number of inputs (incoming edges)
+    u32          input_nodes[4];  // fixed-size or pointer to array
+    u32          output_count;    // number of downstream nodes
+    u32         *output_nodes;    // allocated with arena
+    // ... data for this node type
+};
+```
+
+For a **data-oriented** approach in hot paths (graph evaluation), pack fields into parallel arrays:
+
+```c
+// SoA layout for evaluation
+u8          *node_types;     // PrNodeType for each node
+u32         *in_degrees;     // current in-degree (Kahn's state)
+u32         *topo_order;     // result of topological sort
+```
+
+### Struct-of-Arrays (SoA) Layout
+
+For the graph evaluation hot path, you can use parallel arrays instead of an array of structs:
+
+```c
+struct PrGraphEvalState {
+    u32   node_count;
+    u32  *topo_order;        // [0..node_count-1] in eval order
+    u32  *in_degrees;        // temp space for Kahn's
+    u8   *dirty_flags;       // per-node dirty bit
+    u32  *output_counts;
+    u32 **output_lists;      // adjacency
+};
+```
+
+This keeps only the data needed for traversal in cache-friendly contiguous memory.
+
+## Incremental / Dirty Evaluation
+
+For interactive use, re-running the full topological sort every frame is wasteful. Instead:
+
+1. When a node's parameter changes, mark it **dirty**
+2. Propagate the dirty flag downstream (BFS along edges)
+3. Only re-evaluate dirty nodes in topological order
+
+Alternatively, skip dirty propagation and just always eval in topo order — each node checks if its inputs are dirty or if its own parameters changed. Simpler, but does more work.
+
+## Cycles in a Graph Editor
+
+A graph editor must **prevent** the user from creating cycles in real time. Approaches:
+
+1. **Check on each edge creation:** before adding edge A → B, check if there's already a path from B to A (DFS from B). O(V + E) per edge add.
+2. **Incremental cycle detection:** more sophisticated data structures for dynamic graphs. Probably overkill for V1.
+3. **Kahn's validation:** Run Kahn's after every edit; if it doesn't produce a full ordering, reject the edit.
+
+Approach 1 (DFS reachability test) is the simplest for V1.
+
+## Evaluation Pipeline
+
+```
+User edits graph
+        │
+        ▼
+Validate no cycles  ← (reject edit if cycle detected)
+        │
+        ▼
+Topological sort    ← (Kahn's algorithm → list of node IDs)
+        │
+        ▼
+Mark dirty nodes    ← (only nodes downstream of changes)
+        │
+        ▼
+For each node in topo order:
+    If node is dirty:
+        Gather input images (from upstream node outputs)
+        Execute node (CPU or dispatch GPU shader)
+        Store output image
+        │
+        ▼
+Render final output → display
+```
+
+## Key Takeaways for Prism
+
+1. **Kahn's algorithm** is the right choice — simple, O(V+E), built-in cycle detection, parallel-layer grouping
+2. **Adjacency list** with flat arrays for the graph structure
+3. **Evaluate in topological order**; skip clean nodes for efficiency
+4. **Cycle check on edge creation** via DFS from the target node
+5. **SoA layout** for evaluation state if profiling shows cache misses on the hot path
+6. Node types (Read, Blur, Merge, etc.) can use a `PrNodeType` enum with a function dispatch table, or a union of type-specific data in the node struct
+
+## References
+
+- Kahn, A. B. (1962). "Topological sorting of large networks." *Communications of the ACM*
+- Cormen et al., *Introduction to Algorithms*, 3rd ed., Ch. 22.4 (Topological Sort)
+- Foundry Nuke documentation: [https://learn.foundry.com/nuke](https://learn.foundry.com/nuke)
+- Taskflow C++ library: [https://taskflow.github.io](https://taskflow.github.io)