Math and terminology

The function rms_scaled_softmax names the operator computed by every measured implementation: positive inverse RMS with epsilon 1e-5, stable maximum subtraction, exponentials and row normalization. It changes ordinary softmax semantics and is explicitly labeled a toy example. The NumPy code is a mathematical reference, while timing comes from the linked CUDA C++ program. Because the inverse RMS is positive and row-constant, subtracting the input maximum before scaling is mathematically valid; numerical equivalence is bounded by the stated tolerance. “Baseline” describes the nine-kernel educational implementation, not a different mathematical function or a stock-framework reference. Three phases describe this implementation, not an algorithmic lower bound. Fusion combines operations; it need not eliminate their arithmetic. Main-article speedups divide the local-fusion average latency by the megakernel average latency, where each average gives equal weight to the two session medians.

“Megakernel” means the tested cooperative retained-tile program. It has static phases and no dynamic task queue. It retains large buffers in block shared memory; small cross-block statistics still live in global memory. It does not carry shared memory from one ordinary kernel to another. The one-row plot uses the pilot-selected grid-wide version; the larger shapes use the pilot-selected row-scoped version. Sixty-four rows require two launches.

The computational graph shows whole-row mathematical operations and their value dependencies, not CUDA Graph nodes or observed launches. Its bypass edges retain the direct input to exponentiation and the direct exponential input to normalization. The storage diagram separately shows the tested three-phase implementation: local fusion recomputes exponentials in the last phase, while the megakernel retains them. Diagram labels and dependencies are recorded in figure provenance. Main timing labels round to one decimal for readability; plotted coordinates, intervals and downloadable data retain their original precision.

Graph and synchronization claims

The CUDA Graphs guide supports separating workflow definition/instantiation from replay. The measured graph preserves nine separate kernel nodes, confirmed in saved metadata. No claim is made that graphs fuse the captured kernel bodies or always improve a single-kernel launch.

CUDA 12.8.1 grid synchronization documents cooperative launches and occupancy-based sizing, matching the recorded build image's toolkit version. The custom row-scoped barrier also uses CUDA's scoped memory model: device-scope release/acquire publishes the statistics used by later phases. Admission to a cooperative residency budget is part of the implementation's correctness conditions; this code is not presented as a general safe spin barrier for an arbitrarily large ordinary grid.

Measurement claims

All plotted values are recomputed from saved raw records. The two sessions have distinct GPU UUIDs, the same frozen code/configurations/input fixture, reversed shape order and independently shuffled variant order. Main comparison bar lengths and retention-control dots average the two session medians, and whiskers span those medians; they are not confidence intervals. Retention deltas are paired by round within each session before taking its median and averaging across sessions. The detailed results preserve separate sessions, their p10–p90 variability across rounds and all raw samples. The fresh pass checks source hashes, reconstructed session quantiles, all 19 plotted aggregates, four latency-reduction annotations and article ratios. The comparison uses bars to compare total-latency magnitudes within each shape. Each panel starts at zero and uses a fitted linear scale; a visible notice and range labels disclose that axes differ, so bar lengths must not be compared across shapes. Desktop/mobile layout records verify that the largest plotted value fills at least 75% of its axis. The controls use a compact dot-and-range plot with adjacent value labels for deviations from the retained reference. They are not stacked because the measured penalties are not additive components. Code, configurations, metadata, trace exports and figure provenance accompany the selected data. The main metric is host completion time. Batched CUDA event intervals may include host supply gaps. No metric is called pure device compute time.

Slower shapes and candidates, the missed primary target, ordinary-launch controls and library-preprocessing limitations remain explicit in the supporting results. Same-binary controls support the retention mechanism in that compiled grid-wide kernel but include staging overhead; they do not explain the entire gap to the selected row-scoped kernel or local fusion. Resource/geometry differences prevent attribution of the row-scoped versus grid-wide gap solely to barrier cost. Active-block counts come from occupancy queries and represent capacity limits, not observed resident-block counts or utilization. Logical global-memory accesses are not identified with physical HBM bytes. The main post limits conclusions to warm runs, one row width and H100, and states no application-level speedup.

Fresh numerical checks cover locally available small arrays. The prior independent CPU audit covers archived full-size outputs and intermediates. The website does not claim that all full-size arrays were downloaded and revalidated during this editorial pass. A reporting bug that replaced kernel counts with zero for shapes with eviction records was corrected using saved launch metadata; timing samples were unchanged.

Agent discussion

KernelBench studies generated GPU kernels with execution and profiling feedback. The closing possibility that agents lower the cost of exploring megakernels is labeled an inference. It is not a benchmark result here, and the same reasoning applies to authoring local fusion. No claim of established production adoption or universal advantage is made.

The article retains February 22 as its original article date. Later experimental records retain their own identifiers in supporting material. The post remains a draft in the owner-only preview.