Skip to content

gemv: add optional fused GELU epilogue (NPU2)#135

Open
atassis wants to merge 1 commit into
amd:develfrom
atassis:feat/gemv-gelu-epilogue
Open

gemv: add optional fused GELU epilogue (NPU2)#135
atassis wants to merge 1 commit into
amd:develfrom
atassis:feat/gemv-gelu-epilogue

Conversation

@atassis

@atassis atassis commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Adds an opt-in epilogue parameter to GEMV. epilogue="none" (default) is byte-identical to today; epilogue="gelu" fuses a GELU (tanh approximation) over each output tile inside the producing core, folding an activation that would otherwise be a separate elementwise pass over the GEMV output.

Details

  • The GELU runs once per full m_output tile in core_body (after the matvec inner-loop has filled all rows), not per matvec call whose m_input tile can be smaller than the 16-wide activation vector.
  • No duplicated math. aie_kernels/aie2p/gelu.cc is refactored so the per-16-lane computation is a shared helper used by both the existing out-of-place gelu_tanh_approx_bf16 and a new in-place gelu_tile_bf16. In-place is aliasing-correct (a single restrict pointer, each slot read then written); reusing the out-of-place kernel with input == output would violate its restrict contract, so a dedicated in-place entry point is the correct way to share the code.
  • Since gelu.cc lives in aie2p/, the fused epilogue is NPU2-only; GEMV raises NotImplementedError if it is requested on NPU1. When enabled, the core links a (matvec, gelu) archive; the default path still links the single matvec object.

Test

test_gemv_gelu compares the fused output against a gelu(A @ B) golden across three shapes, using the same latency/bandwidth/throughput @pytest.mark.metrics as test_gemv. Verified on NPU2: 15/15 pass.

This is opt-in and default-preserving; it also composes with the existing func_prefix fusion support (#123) for use as a fused block.

@atassis atassis mentioned this pull request Jul 13, 2026
Merged
3 tasks
andrej
andrej approved these changes Jul 13, 2026
View reviewed changes

@andrej andrej left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good addition and a natural fusion opportunity. If you have any interest in this, I think it would be cool to think about if we can generalize this across other activation types as well -- sort of a generic "run this kernel in-place on the results" flag, where maybe you could pass an arbitrary Kernel object. That would be a follow-on PR of course.

@github-actions

github-actions Bot commented Jul 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

CI Test Results

3c991ad (2026_07_14_16_22_42)

IRON - CI Summary

Small

iron/operators/axpy
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0] - - ✅ 284.52
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0] - - ✅ 479.32
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0] - - ✅ 535.58
iron/operators/dequant
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32] - - ✅ 654.80
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32] - - ✅ 313.66
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32] - - ✅ 397.70
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32] - - ✅ 489.96
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32] - - ✅ 418.22
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32] - - ✅ 467.12
iron/operators/elementwise_add
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048] - - ✅ 351.22
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024] - - ✅ 438.26
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512] - - ✅ 419.66
iron/operators/elementwise_mul
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048] - - ✅ 384.76
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024] - - ✅ 472.10
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512] - - ✅ 563.76
iron/operators/gelu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 378.02
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - ✅ 636.50
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 357.00
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 350.70
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 416.70
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - ✅ 469.90
iron/operators/gemm
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1] - - ✅ 577.32
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1] - - ✅ 491.16
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] - - ✅ 82271.82
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] - - ✅ 24955.92
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1] - - ✅ 4496.04
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4] - - ✅ 6226.58
iron/operators/gemv
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128] - - ✅ 0.11
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048] - - ✅ 3.56
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024] - - ✅ 5.69
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512] - - ✅ 10.13
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024] - - ✅ 3.64
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024] - - ✅ 6.72
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024] - - ✅ 9.68
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2] - - ✅ 2.30
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2] - - ✅ 0.32
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4] - - ✅ 0.40
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8] - - ✅ 1.68
test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128] - - ❌ -
test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048] - - ❌ -
test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024] - - ❌ -
iron/operators/layer_norm
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 355.46
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - ✅ 367.80
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 437.60
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 459.72
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 418.72
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - ✅ 331.80
iron/operators/leaky_relu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01] - - ✅ 737.78
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1] - - ✅ 412.06
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25] - - ✅ 362.20
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01] - - ✅ 423.24
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01] - - ✅ 371.94
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01] - - ✅ 463.52
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01] - - ✅ 438.20
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01] - - ✅ 481.22
iron/operators/mem_copy
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048] - - ✅ 349.22
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024] - - ✅ 278.38
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024] - - ✅ 300.32
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512] - - ✅ 396.36
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512] - - ✅ 363.30
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256] - - ✅ 401.40
iron/operators/relu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 297.12
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - ✅ 321.82
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 312.26
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 620.82
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 393.28
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - ✅ 529.20
iron/operators/rms_norm
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False] - - ✅ 511.22
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True] - - ✅ 446.68
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False] - - ✅ 453.82
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True] - - ✅ 597.64
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False] - - ✅ 465.16
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True] - - ✅ 416.06
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False] - - ✅ 504.14
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True] - - ✅ 515.38
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False] - - ✅ 809.26
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True] - - ✅ 404.98
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False] - - ✅ 669.14
iron/operators/rope
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0] - - ✅ 708.48
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0] - - ✅ 687.86
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0] - - ✅ 480.10
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0] - - ✅ 439.66
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0] - - ✅ 753.06
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0] - - ✅ 358.46
iron/operators/sigmoid
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 512.66
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - ✅ 428.12
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 751.70
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 473.70
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 918.32
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - ✅ 831.86
iron/operators/silu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 424.60
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 486.62
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 439.36
iron/operators/softmax
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024] - - ✅ 494.78
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048] - - ✅ 443.04
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 460.20
iron/operators/swiglu_decode
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584] - - ✅ 16641.09
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048] - - ✅ 17079.29
iron/operators/swiglu_prefill
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False] - - ✅ 24182.40
iron/operators/tanh
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 444.22
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - ✅ 422.74
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 347.98
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 408.80
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 489.90
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - ✅ 629.98
iron/operators/transpose
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1] - - ✅ 467.24
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2] - - ✅ 1726.14
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1] - - ✅ 598.86
Phoenix - Small

IRON

Tested on 2026_07_14_16_22_42 at commit 3c991ad.

iron/operators/axpy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]✅ 5/5284.520.04n/a
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]✅ 5/5479.320.03n/a
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]✅ 5/5535.580.03n/a
iron/operators/dequant
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]✅ 5/5654.800.01n/a
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]✅ 5/5313.660.02n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]✅ 5/5397.700.01n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]✅ 5/5489.960.01n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]✅ 5/5418.220.01n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]✅ 5/5467.120.01n/a
iron/operators/elementwise_add
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5351.220.04n/a
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5438.260.03n/a
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5419.660.03n/a
iron/operators/elementwise_mul
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5384.760.04n/a
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5472.100.03n/a
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5563.760.03n/a
iron/operators/gelu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5378.020.02n/a
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5636.500.02n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5357.000.03n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5350.700.03n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5416.700.02n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5469.900.02n/a
iron/operators/gemm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5577.320.3916.70
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5491.160.4719.91
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/582271.820.31208.83
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/524955.921.01688.64
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]✅ 5/54496.041.97516.64
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]✅ 5/56226.580.2111.12
iron/operators/gemv
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]✅ 5/5n/a0.110.10
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]✅ 5/5n/a3.563.55
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]✅ 5/5n/a5.695.68
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]✅ 5/5n/a10.1310.13
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a3.643.63
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a6.726.72
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a9.689.67
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]✅ 5/5n/a2.302.30
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]✅ 5/5n/a0.320.31
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]✅ 5/5n/a0.400.39
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]✅ 5/5n/a1.681.65
test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]❌ 0/5n/an/an/a
test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]❌ 0/5n/an/an/a
test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]❌ 0/5n/an/an/a
iron/operators/layer_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5355.460.03n/a
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5367.800.02n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5437.600.02n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5459.720.02n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5418.720.02n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5331.800.03n/a
iron/operators/leaky_relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]✅ 5/5737.780.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]✅ 5/5412.060.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]✅ 5/5362.200.03n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]✅ 5/5423.240.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]✅ 5/5371.940.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]✅ 5/5463.520.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]✅ 5/5438.200.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]✅ 5/5481.220.02n/a
iron/operators/mem_copy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]✅ 5/5349.220.03n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]✅ 5/5278.380.03n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]✅ 5/5300.320.03n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]✅ 5/5396.360.02n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]✅ 5/5363.300.02n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]✅ 5/5401.400.02n/a
iron/operators/relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5297.120.03n/a
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5321.820.03n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5312.260.03n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5620.820.02n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5393.280.02n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5529.200.02n/a
iron/operators/rms_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]✅ 5/5511.220.02n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]✅ 5/5446.680.03n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]✅ 5/5453.820.02n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]✅ 5/5597.640.02n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]✅ 5/5465.160.02n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]✅ 5/5416.060.03n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]✅ 5/5504.140.02n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]✅ 5/5515.380.02n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]✅ 5/5809.260.02n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]✅ 5/5404.980.03n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]✅ 5/5669.140.01n/a
iron/operators/rope
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]✅ 5/5708.480.23n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]✅ 5/5687.860.23n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]✅ 5/5480.100.22n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]✅ 5/5439.660.18n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]✅ 5/5753.060.14n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]✅ 5/5358.460.21n/a
iron/operators/sigmoid
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5512.660.02n/a
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5428.120.02n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5751.700.02n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5473.700.02n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5918.320.02n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5831.860.01n/a
iron/operators/silu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5424.600.02n/a
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5486.620.02n/a
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5439.360.02n/a
iron/operators/softmax
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]✅ 5/5494.780.28n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]✅ 5/5443.040.32n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5460.200.33n/a
iron/operators/swiglu_decode
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]✅ 5/516641.090.00n/a
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]✅ 5/517079.290.00n/a
iron/operators/swiglu_prefill
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]✅ 5/524182.400.09n/a
iron/operators/tanh
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5444.220.02n/a
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5422.740.02n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5347.980.02n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5408.800.02n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5489.900.02n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5629.980.01n/a
iron/operators/transpose
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]✅ 5/5467.241.27n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]✅ 5/51726.141.40n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]✅ 5/5598.861.01n/a

Trends:

IRON Trends

iron/operators/axpy

test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.05 (+16.84%)0.04 (+44.53%)0.04 (+66.94%)0.04 (+99.85%)0.00 (-60.60%)302.80 (-49.96%)284.52 (-37.16%)294.00 (-40.10%)239.70 (-14.42%)25.40 (-83.31%)
e216c2f — 2026-07-13 21:54:420.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)605.10 (n/a)452.76 (n/a)490.80 (n/a)280.10 (n/a)152.20 (n/a)

test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.05 (-46.87%)0.03 (-39.12%)0.02 (-50.09%)0.02 (-12.83%)0.01 (-58.34%)604.40 (+14.73%)479.32 (+42.31%)533.30 (+100.34%)261.00 (+88.18%)135.73 (-18.66%)
e216c2f — 2026-07-13 21:54:420.09 (n/a)0.05 (n/a)0.05 (n/a)0.02 (n/a)0.03 (n/a)526.80 (n/a)336.82 (n/a)266.20 (n/a)138.70 (n/a)166.86 (n/a)

test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.05 (-22.70%)0.03 (-26.13%)0.04 (-13.19%)0.01 (-41.60%)0.02 (-1.56%)1138.20 (+71.24%)535.58 (+56.64%)336.50 (+15.20%)244.80 (+29.39%)374.12 (+101.47%)
e216c2f — 2026-07-13 21:54:420.06 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.02 (n/a)664.70 (n/a)341.92 (n/a)292.10 (n/a)189.20 (n/a)185.69 (n/a)
iron/operators/dequant

test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.02 (-8.29%)0.01 (+5.74%)0.02 (+67.65%)0.00 (-73.98%)0.01 (+41.11%)1971.30 (+284.34%)654.80 (+57.29%)293.90 (-40.34%)237.10 (+9.06%)745.66 (+479.57%)
e216c2f — 2026-07-13 21:54:420.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)512.90 (n/a)416.30 (n/a)492.60 (n/a)217.40 (n/a)128.66 (n/a)

test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.02 (-1.06%)0.02 (+51.39%)0.02 (+71.43%)0.01 (+366.36%)0.01 (-30.09%)436.70 (-78.56%)313.66 (-58.60%)260.20 (-41.66%)215.30 (+1.08%)108.44 (-85.32%)
e216c2f — 2026-07-13 21:54:420.02 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)0.01 (n/a)2036.60 (n/a)757.66 (n/a)446.00 (n/a)213.00 (n/a)738.89 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.02 (+19.28%)0.01 (+12.25%)0.01 (+6.99%)0.01 (-6.80%)0.00 (+85.36%)520.50 (+7.30%)397.70 (-6.02%)406.80 (-6.55%)261.10 (-16.15%)120.15 (+70.27%)
e216c2f — 2026-07-13 21:54:420.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)485.10 (n/a)423.18 (n/a)435.30 (n/a)311.40 (n/a)70.57 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.02 (-25.64%)0.01 (-36.17%)0.01 (-39.57%)0.01 (-41.72%)0.00 (-7.90%)723.10 (+71.59%)489.96 (+61.27%)456.50 (+65.46%)328.70 (+34.49%)143.90 (+108.99%)
e216c2f — 2026-07-13 21:54:420.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)421.40 (n/a)303.82 (n/a)275.90 (n/a)244.40 (n/a)68.85 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.02 (+4.48%)0.01 (-0.43%)0.01 (+0.62%)0.01 (-14.39%)0.01 (+24.06%)644.90 (+16.81%)418.22 (+5.14%)421.30 (-0.64%)247.00 (-4.26%)157.82 (+38.40%)
e216c2f — 2026-07-13 21:54:420.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)552.10 (n/a)397.76 (n/a)424.00 (n/a)258.00 (n/a)114.03 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.02 (+26.74%)0.01 (+11.76%)0.01 (+16.85%)0.01 (-29.26%)0.00 (+129.60%)814.60 (+41.37%)467.12 (-0.72%)399.60 (-14.41%)290.10 (-21.10%)207.96 (+166.49%)
e216c2f — 2026-07-13 21:54:420.01 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)576.20 (n/a)470.52 (n/a)466.90 (n/a)367.70 (n/a)78.04 (n/a)
iron/operators/elementwise_add

test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.05 (n/a)0.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)474.80 (n/a)351.22 (n/a)357.00 (n/a)232.40 (n/a)100.63 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.05 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)525.90 (n/a)438.26 (n/a)456.20 (n/a)258.70 (n/a)105.20 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (n/a)0.03 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)623.50 (n/a)419.66 (n/a)333.70 (n/a)287.20 (n/a)159.27 (n/a)
iron/operators/elementwise_mul

test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.06 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)641.00 (n/a)384.76 (n/a)331.00 (n/a)215.60 (n/a)170.29 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)667.40 (n/a)472.10 (n/a)427.60 (n/a)332.30 (n/a)140.30 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)997.10 (n/a)563.76 (n/a)440.50 (n/a)322.10 (n/a)268.20 (n/a)
iron/operators/gelu

test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)560.50 (n/a)378.02 (n/a)370.70 (n/a)256.30 (n/a)129.03 (n/a)

test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (n/a)0.02 (n/a)0.03 (n/a)0.00 (n/a)0.01 (n/a)1961.50 (n/a)636.50 (n/a)280.10 (n/a)240.00 (n/a)745.41 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)634.80 (n/a)357.00 (n/a)248.50 (n/a)224.80 (n/a)179.62 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)558.20 (n/a)350.70 (n/a)281.70 (n/a)241.70 (n/a)139.12 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)552.70 (n/a)416.70 (n/a)496.00 (n/a)261.80 (n/a)136.83 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)579.00 (n/a)469.90 (n/a)493.10 (n/a)271.90 (n/a)117.95 (n/a)
iron/operators/gemm

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:580.50 (+35.20%)0.39 (+18.16%)0.37 (+2.91%)0.33 (+54.63%)0.07 (+3.27%)668.20 (-35.33%)577.32 (-17.24%)601.80 (-2.84%)439.20 (-26.02%)88.62 (-52.88%)21.49 (+35.20%)16.70 (+18.16%)15.68 (+2.91%)14.12 (+54.63%)2.90 (+3.27%)
e216c2f — 2026-07-13 21:54:420.37 (n/a)0.33 (n/a)0.36 (n/a)0.21 (n/a)0.07 (n/a)1033.30 (n/a)697.62 (n/a)619.40 (n/a)593.70 (n/a)188.04 (n/a)15.89 (n/a)14.13 (n/a)15.24 (n/a)9.13 (n/a)2.81 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:580.63 (+25.96%)0.47 (+8.14%)0.46 (+12.11%)0.36 (+1.10%)0.10 (+77.01%)608.70 (-1.09%)491.16 (-5.61%)477.30 (-10.82%)348.70 (-20.61%)97.60 (+38.10%)27.06 (+25.96%)19.91 (+8.14%)19.77 (+12.11%)15.50 (+1.10%)4.42 (+77.01%)
e216c2f — 2026-07-13 21:54:420.50 (n/a)0.43 (n/a)0.41 (n/a)0.36 (n/a)0.06 (n/a)615.40 (n/a)520.34 (n/a)535.20 (n/a)439.20 (n/a)70.67 (n/a)21.49 (n/a)18.41 (n/a)17.63 (n/a)15.34 (n/a)2.50 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:580.31 (+0.20%)0.31 (+0.19%)0.31 (+0.12%)0.30 (+0.63%)0.00 (-13.56%)83213.70 (-0.62%)82271.82 (-0.20%)82318.20 (-0.12%)81224.30 (-0.20%)730.02 (-14.41%)211.51 (+0.20%)208.83 (+0.19%)208.70 (+0.12%)206.45 (+0.63%)1.86 (-13.56%)
e216c2f — 2026-07-13 21:54:420.31 (n/a)0.31 (n/a)0.31 (n/a)0.30 (n/a)0.00 (n/a)83736.50 (n/a)82433.08 (n/a)82414.90 (n/a)81386.30 (n/a)852.92 (n/a)211.09 (n/a)208.43 (n/a)208.46 (n/a)205.17 (n/a)2.15 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:581.04 (+1.11%)1.01 (-0.76%)1.01 (-1.36%)0.98 (-1.81%)0.02 (+84.20%)25649.50 (+1.84%)24955.92 (+0.79%)24992.90 (+1.38%)24224.60 (-1.10%)510.15 (+85.21%)709.19 (+1.11%)688.64 (-0.76%)687.39 (-1.36%)669.79 (-1.81%)14.12 (+84.20%)
e216c2f — 2026-07-13 21:54:421.03 (n/a)1.02 (n/a)1.02 (n/a)1.00 (n/a)0.01 (n/a)25186.40 (n/a)24761.20 (n/a)24652.60 (n/a)24493.90 (n/a)275.44 (n/a)701.39 (n/a)693.89 (n/a)696.88 (n/a)682.11 (n/a)7.66 (n/a)

test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:582.87 (-24.69%)1.97 (-24.88%)1.69 (-20.56%)1.36 (-14.64%)0.69 (-29.07%)5918.40 (+17.15%)4496.04 (+31.03%)4772.40 (+25.88%)2806.80 (+32.79%)1451.13 (+18.37%)753.14 (-24.69%)516.64 (-24.88%)442.94 (-20.56%)357.18 (-14.64%)181.94 (-29.07%)
e216c2f — 2026-07-13 21:54:423.81 (n/a)2.62 (n/a)2.13 (n/a)1.60 (n/a)0.98 (n/a)5052.20 (n/a)3431.40 (n/a)3791.10 (n/a)2113.70 (n/a)1225.96 (n/a)1000.11 (n/a)687.77 (n/a)557.60 (n/a)418.42 (n/a)256.51 (n/a)

test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:580.28 (-6.02%)0.21 (-14.20%)0.20 (-14.97%)0.16 (-8.17%)0.04 (-2.82%)7640.40 (+8.90%)6226.58 (+16.71%)6326.60 (+17.61%)4455.40 (+6.41%)1159.51 (+7.85%)15.06 (-6.02%)11.12 (-14.20%)10.61 (-14.97%)8.78 (-8.17%)2.37 (-2.82%)
e216c2f — 2026-07-13 21:54:420.30 (n/a)0.24 (n/a)0.23 (n/a)0.18 (n/a)0.05 (n/a)7016.30 (n/a)5334.86 (n/a)5379.50 (n/a)4187.00 (n/a)1075.07 (n/a)16.03 (n/a)12.97 (n/a)12.47 (n/a)9.56 (n/a)2.43 (n/a)
iron/operators/gemv

test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:580.14 (n/a)0.11 (n/a)0.11 (n/a)0.07 (n/a)0.03 (n/a)0.14 (n/a)0.10 (n/a)0.11 (n/a)0.07 (n/a)0.03 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:583.74 (n/a)3.56 (n/a)3.47 (n/a)3.42 (n/a)0.15 (n/a)3.74 (n/a)3.55 (n/a)3.47 (n/a)3.42 (n/a)0.15 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:587.07 (-5.50%)5.69 (-10.20%)5.66 (-8.92%)4.51 (-20.57%)1.01 (+37.49%)7.07 (-5.50%)5.68 (-10.20%)5.65 (-8.92%)4.51 (-20.57%)1.01 (+37.49%)
e216c2f — 2026-07-13 21:54:427.48 (n/a)6.33 (n/a)6.21 (n/a)5.68 (n/a)0.73 (n/a)7.48 (n/a)6.33 (n/a)6.21 (n/a)5.68 (n/a)0.73 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:5813.95 (+0.61%)10.13 (-5.43%)8.26 (-21.67%)8.19 (+8.25%)2.70 (-7.68%)13.94 (+0.61%)10.13 (-5.43%)8.26 (-21.67%)8.18 (+8.25%)2.70 (-7.68%)
e216c2f — 2026-07-13 21:54:4213.86 (n/a)10.71 (n/a)10.55 (n/a)7.56 (n/a)2.93 (n/a)13.85 (n/a)10.71 (n/a)10.54 (n/a)7.56 (n/a)2.93 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:583.94 (n/a)3.64 (n/a)3.64 (n/a)3.41 (n/a)0.23 (n/a)3.93 (n/a)3.63 (n/a)3.64 (n/a)3.40 (n/a)0.23 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:587.55 (+0.98%)6.72 (+1.27%)6.97 (+2.96%)5.80 (+1.41%)0.87 (+30.17%)7.55 (+0.98%)6.72 (+1.27%)6.97 (+2.96%)5.80 (+1.41%)0.87 (+30.17%)
e216c2f — 2026-07-13 21:54:427.48 (n/a)6.64 (n/a)6.77 (n/a)5.72 (n/a)0.67 (n/a)7.48 (n/a)6.63 (n/a)6.77 (n/a)5.71 (n/a)0.67 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:5813.82 (+23.61%)9.68 (-1.75%)8.58 (-19.39%)7.51 (-6.86%)2.48 (+75.29%)13.81 (+23.61%)9.67 (-1.75%)8.58 (-19.39%)7.50 (-6.86%)2.48 (+75.29%)
e216c2f — 2026-07-13 21:54:4211.18 (n/a)9.85 (n/a)10.65 (n/a)8.06 (n/a)1.41 (n/a)11.18 (n/a)9.84 (n/a)10.64 (n/a)8.06 (n/a)1.41 (n/a)

test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:582.81 (-10.96%)2.30 (-5.15%)2.53 (-10.39%)1.03 (-8.41%)0.73 (-12.11%)2.80 (-10.96%)2.30 (-5.15%)2.52 (-10.39%)1.03 (-8.41%)0.73 (-12.11%)
e216c2f — 2026-07-13 21:54:423.15 (n/a)2.42 (n/a)2.82 (n/a)1.13 (n/a)0.83 (n/a)3.14 (n/a)2.42 (n/a)2.82 (n/a)1.13 (n/a)0.83 (n/a)

test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:580.45 (-4.00%)0.32 (-1.64%)0.32 (-8.65%)0.07 (-41.04%)0.15 (+6.21%)0.44 (-4.00%)0.31 (-1.64%)0.32 (-8.65%)0.07 (-41.04%)0.15 (+6.21%)
e216c2f — 2026-07-13 21:54:420.47 (n/a)0.33 (n/a)0.35 (n/a)0.13 (n/a)0.14 (n/a)0.46 (n/a)0.32 (n/a)0.34 (n/a)0.12 (n/a)0.14 (n/a)

test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:580.60 (-4.19%)0.40 (-20.48%)0.45 (-23.49%)0.07 (-1.64%)0.20 (-17.64%)0.60 (-4.19%)0.39 (-20.48%)0.44 (-23.49%)0.07 (-1.64%)0.19 (-17.64%)
e216c2f — 2026-07-13 21:54:420.63 (n/a)0.50 (n/a)0.58 (n/a)0.08 (n/a)0.24 (n/a)0.62 (n/a)0.49 (n/a)0.58 (n/a)0.07 (n/a)0.24 (n/a)

test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3c991ad — 2026-07-14 16:18:582.59 (+14.33%)1.68 (-2.34%)1.70 (+6.72%)0.73 (-34.55%)0.88 (+96.40%)2.55 (+14.33%)1.65 (-2.34%)1.68 (+6.72%)0.72 (-34.55%)0.87 (+96.40%)
e216c2f — 2026-07-13 21:54:422.27 (n/a)1.72 (n/a)1.60 (n/a)1.11 (n/a)0.45 (n/a)2.23 (n/a)1.69 (n/a)1.57 (n/a)1.10 (n/a)0.44 (n/a)

test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

No metrics available.

test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

No metrics available.

test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

No metrics available.

iron/operators/layer_norm

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)608.40 (n/a)355.46 (n/a)299.90 (n/a)250.50 (n/a)143.68 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)529.20 (n/a)367.80 (n/a)364.10 (n/a)223.90 (n/a)117.79 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)585.10 (n/a)437.60 (n/a)487.80 (n/a)258.30 (n/a)155.02 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)599.50 (n/a)459.72 (n/a)474.70 (n/a)231.50 (n/a)139.99 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)576.60 (n/a)418.72 (n/a)475.50 (n/a)222.70 (n/a)139.91 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)366.40 (n/a)331.80 (n/a)345.70 (n/a)249.10 (n/a)48.16 (n/a)
iron/operators/leaky_relu

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (+10.78%)0.02 (-31.62%)0.02 (-47.31%)0.00 (-71.22%)0.01 (+54.14%)2003.00 (+247.50%)737.78 (+119.49%)510.60 (+89.74%)222.40 (-9.74%)717.64 (+421.44%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)576.40 (n/a)336.14 (n/a)269.10 (n/a)246.40 (n/a)137.63 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (+0.72%)0.02 (-4.85%)0.02 (+3.79%)0.01 (-30.52%)0.01 (+76.52%)624.60 (+43.92%)412.06 (+15.37%)360.40 (-3.66%)260.60 (-0.69%)162.76 (+155.63%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)434.00 (n/a)357.16 (n/a)374.10 (n/a)262.40 (n/a)63.67 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (+4.72%)0.03 (+11.89%)0.03 (+53.99%)0.02 (+2.52%)0.01 (-1.12%)531.70 (-2.44%)362.20 (-11.36%)312.00 (-35.05%)230.10 (-4.52%)140.05 (-4.93%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)545.00 (n/a)408.62 (n/a)480.40 (n/a)241.00 (n/a)147.32 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (+22.52%)0.02 (+23.89%)0.02 (+32.53%)0.01 (+181.23%)0.01 (+6.11%)710.20 (-64.44%)423.24 (-40.80%)363.60 (-24.55%)198.80 (-18.39%)212.25 (-70.82%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)0.01 (n/a)1997.20 (n/a)714.94 (n/a)481.90 (n/a)243.60 (n/a)727.43 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (+30.74%)0.02 (+18.67%)0.02 (+8.89%)0.02 (+3.70%)0.01 (+91.04%)506.70 (-3.56%)371.94 (-10.73%)373.10 (-8.15%)233.10 (-23.52%)121.36 (+42.75%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)525.40 (n/a)416.66 (n/a)406.20 (n/a)304.80 (n/a)85.01 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.02 (-35.83%)0.02 (+4.12%)0.02 (+24.63%)0.01 (+109.83%)0.00 (-62.96%)624.10 (-52.34%)463.52 (-25.96%)432.20 (-19.75%)357.30 (+55.89%)109.52 (-72.94%)
e216c2f — 2026-07-13 21:54:420.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)1309.50 (n/a)626.08 (n/a)538.60 (n/a)229.20 (n/a)404.72 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (+47.14%)0.02 (+22.56%)0.02 (+6.85%)0.01 (+11.43%)0.01 (+129.82%)575.30 (-10.26%)438.20 (-14.43%)460.80 (-6.40%)275.80 (-32.04%)121.17 (+38.39%)
e216c2f — 2026-07-13 21:54:420.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)641.10 (n/a)512.08 (n/a)492.30 (n/a)405.80 (n/a)87.55 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (+29.11%)0.02 (-6.80%)0.02 (-25.90%)0.01 (-12.58%)0.01 (+84.92%)612.70 (+14.37%)481.22 (+13.60%)525.40 (+34.96%)257.40 (-22.56%)133.77 (+49.84%)
e216c2f — 2026-07-13 21:54:420.02 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)535.70 (n/a)423.62 (n/a)389.30 (n/a)332.40 (n/a)89.28 (n/a)
iron/operators/mem_copy

test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (+1.85%)0.03 (+15.58%)0.03 (+15.66%)0.01 (-21.09%)0.01 (+17.47%)639.20 (+26.72%)349.22 (-9.01%)311.20 (-13.53%)234.50 (-1.80%)166.36 (+44.40%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)504.40 (n/a)383.80 (n/a)359.90 (n/a)238.80 (n/a)115.21 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (+2.46%)0.03 (+25.74%)0.03 (+60.05%)0.02 (+13.91%)0.01 (-9.01%)430.40 (-12.22%)278.38 (-22.04%)234.40 (-37.51%)213.10 (-2.38%)88.47 (-18.24%)
e216c2f — 2026-07-13 21:54:420.04 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)490.30 (n/a)357.08 (n/a)375.10 (n/a)218.30 (n/a)108.21 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (+16.54%)0.03 (+25.09%)0.03 (+45.76%)0.02 (+8.21%)0.01 (+8.93%)443.50 (-7.58%)300.32 (-20.13%)286.90 (-31.40%)220.40 (-14.17%)84.88 (-8.49%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)479.90 (n/a)376.00 (n/a)418.20 (n/a)256.80 (n/a)92.75 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (-24.17%)0.02 (-19.35%)0.02 (-26.13%)0.02 (-4.82%)0.01 (-32.60%)540.80 (+5.07%)396.36 (+17.34%)390.60 (+35.39%)263.90 (+31.88%)132.61 (-9.48%)
e216c2f — 2026-07-13 21:54:420.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)514.70 (n/a)337.80 (n/a)288.50 (n/a)200.10 (n/a)146.49 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (-5.28%)0.02 (+4.75%)0.02 (+17.02%)0.02 (+15.53%)0.01 (-22.81%)492.20 (-13.44%)363.30 (-9.27%)378.80 (-14.53%)246.80 (+5.61%)101.35 (-29.03%)
e216c2f — 2026-07-13 21:54:420.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)568.60 (n/a)400.42 (n/a)443.20 (n/a)233.70 (n/a)142.81 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (+17.73%)0.02 (+32.32%)0.02 (+48.41%)0.01 (+13.23%)0.01 (+45.52%)572.70 (-11.67%)401.40 (-22.71%)363.70 (-32.61%)305.00 (-15.07%)114.92 (+10.43%)
e216c2f — 2026-07-13 21:54:420.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)648.40 (n/a)519.36 (n/a)539.70 (n/a)359.10 (n/a)104.07 (n/a)
iron/operators/rms_norm

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (-4.70%)0.02 (-39.72%)0.01 (-53.90%)0.01 (-40.72%)0.01 (+75.28%)623.80 (+68.69%)511.22 (+82.08%)565.10 (+116.93%)247.30 (+4.97%)151.94 (+187.24%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.00 (n/a)369.80 (n/a)280.76 (n/a)260.50 (n/a)235.60 (n/a)52.90 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.05 (-3.44%)0.03 (+0.39%)0.03 (+23.55%)0.02 (+0.92%)0.01 (-16.47%)592.20 (-0.90%)446.68 (-4.17%)438.10 (-19.07%)260.60 (+3.58%)123.47 (-20.95%)
e216c2f — 2026-07-13 21:54:420.05 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)597.60 (n/a)466.14 (n/a)541.30 (n/a)251.60 (n/a)156.20 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (-16.68%)0.02 (-11.78%)0.02 (-4.49%)0.01 (-4.71%)0.01 (-30.88%)575.20 (+4.94%)453.82 (+8.35%)516.80 (+4.70%)297.40 (+20.02%)126.27 (-13.28%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)548.10 (n/a)418.86 (n/a)493.60 (n/a)247.80 (n/a)145.61 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.05 (-29.27%)0.02 (-34.05%)0.02 (-29.91%)0.01 (-50.97%)0.02 (-21.02%)1115.70 (+103.97%)597.64 (+73.46%)426.80 (+42.65%)209.50 (+41.36%)364.46 (+138.63%)
e216c2f — 2026-07-13 21:54:420.07 (n/a)0.04 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)547.00 (n/a)344.54 (n/a)299.20 (n/a)148.20 (n/a)152.73 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (-16.12%)0.02 (-23.03%)0.02 (-44.24%)0.01 (+12.14%)0.01 (-25.30%)612.20 (-10.84%)465.16 (+22.73%)510.30 (+79.30%)280.30 (+19.23%)142.92 (-22.46%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)686.60 (n/a)379.02 (n/a)284.60 (n/a)235.10 (n/a)184.33 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.05 (+9.39%)0.03 (-9.18%)0.02 (-32.49%)0.02 (+6.93%)0.01 (+15.45%)549.10 (-6.49%)416.06 (+12.48%)479.00 (+48.11%)214.10 (-8.58%)150.83 (+3.03%)
e216c2f — 2026-07-13 21:54:420.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)587.20 (n/a)369.90 (n/a)323.40 (n/a)234.20 (n/a)146.40 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (-6.27%)0.02 (-26.23%)0.02 (-37.17%)0.01 (-9.60%)0.01 (+8.28%)629.00 (+10.62%)504.14 (+37.97%)521.50 (+59.14%)276.80 (+6.71%)138.45 (+16.02%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.02 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)568.60 (n/a)365.40 (n/a)327.70 (n/a)259.40 (n/a)119.32 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (+0.90%)0.02 (-6.76%)0.02 (-9.77%)0.01 (-1.97%)0.00 (+8.80%)657.50 (+2.00%)515.38 (+7.73%)515.20 (+10.82%)354.60 (-0.89%)108.09 (+3.86%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)644.60 (n/a)478.40 (n/a)464.90 (n/a)357.80 (n/a)104.07 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.03 (-11.60%)0.02 (-23.66%)0.02 (-2.90%)0.00 (-80.42%)0.01 (+44.86%)2427.70 (+410.77%)809.26 (+115.21%)437.40 (+2.99%)293.20 (+13.12%)908.39 (+871.27%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)475.30 (n/a)376.04 (n/a)424.70 (n/a)259.20 (n/a)93.53 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.04 (-3.70%)0.03 (-11.04%)0.02 (-33.14%)0.02 (+26.25%)0.01 (-11.68%)533.50 (-20.79%)404.98 (+7.15%)473.70 (+49.57%)233.40 (+3.87%)137.71 (-25.33%)
e216c2f — 2026-07-13 21:54:420.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)673.50 (n/a)377.96 (n/a)316.70 (n/a)224.70 (n/a)184.43 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.02 (-8.51%)0.01 (-13.22%)0.01 (-10.09%)0.01 (-4.94%)0.01 (-4.33%)1045.90 (+5.19%)669.14 (+15.65%)606.80 (+11.22%)354.10 (+9.32%)263.74 (+5.63%)
e216c2f — 2026-07-13 21:54:420.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)994.30 (n/a)578.58 (n/a)545.60 (n/a)323.90 (n/a)249.69 (n/a)
iron/operators/rope

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.40 (+33.58%)0.23 (-4.71%)0.19 (-22.00%)0.05 (-71.64%)0.14 (+193.02%)1913.10 (+252.65%)708.48 (+68.11%)527.30 (+28.20%)246.40 (-25.13%)687.68 (+703.79%)
e216c2f — 2026-07-13 21:54:420.30 (n/a)0.24 (n/a)0.24 (n/a)0.18 (n/a)0.05 (n/a)542.50 (n/a)421.44 (n/a)411.30 (n/a)329.10 (n/a)85.56 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.36 (+2.23%)0.23 (-13.24%)0.22 (-30.02%)0.05 (-67.80%)0.13 (+33.33%)1894.80 (+210.57%)687.86 (+65.34%)456.70 (+42.90%)272.20 (-2.16%)683.41 (+310.82%)
e216c2f — 2026-07-13 21:54:420.35 (n/a)0.27 (n/a)0.31 (n/a)0.16 (n/a)0.09 (n/a)610.10 (n/a)416.02 (n/a)319.60 (n/a)278.20 (n/a)166.35 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.33 (-30.88%)0.22 (-18.77%)0.19 (-7.54%)0.17 (+8.58%)0.07 (-49.34%)584.60 (-7.91%)480.10 (+11.31%)514.40 (+8.14%)294.40 (+44.67%)111.68 (-33.65%)
e216c2f — 2026-07-13 21:54:420.48 (n/a)0.27 (n/a)0.21 (n/a)0.15 (n/a)0.13 (n/a)634.80 (n/a)431.30 (n/a)475.70 (n/a)203.50 (n/a)168.32 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.24 (+3.96%)0.18 (+9.49%)0.16 (+5.88%)0.13 (+10.47%)0.05 (+18.29%)572.50 (-9.47%)439.66 (-7.57%)464.50 (-5.55%)305.80 (-3.81%)125.53 (+0.98%)
e216c2f — 2026-07-13 21:54:420.23 (n/a)0.16 (n/a)0.15 (n/a)0.12 (n/a)0.05 (n/a)632.40 (n/a)475.66 (n/a)491.80 (n/a)317.90 (n/a)124.31 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.25 (+29.18%)0.14 (-9.55%)0.14 (-6.99%)0.04 (-69.93%)0.07 (+147.69%)1942.80 (+232.50%)753.06 (+58.30%)519.60 (+7.51%)299.70 (-22.58%)671.52 (+659.68%)
e216c2f — 2026-07-13 21:54:420.19 (n/a)0.16 (n/a)0.15 (n/a)0.13 (n/a)0.03 (n/a)584.30 (n/a)475.72 (n/a)483.30 (n/a)387.10 (n/a)88.40 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.26 (-4.54%)0.21 (+12.77%)0.23 (+61.24%)0.17 (+26.94%)0.04 (-37.91%)446.10 (-21.23%)358.46 (-17.29%)321.60 (-37.99%)283.90 (+4.76%)79.07 (-45.61%)
e216c2f — 2026-07-13 21:54:420.27 (n/a)0.19 (n/a)0.14 (n/a)0.13 (n/a)0.07 (n/a)566.30 (n/a)433.40 (n/a)518.60 (n/a)271.00 (n/a)145.36 (n/a)
iron/operators/softmax

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.43 (-2.91%)0.28 (+4.71%)0.26 (-6.38%)0.19 (+255.99%)0.09 (-37.37%)683.40 (-71.91%)494.78 (-39.89%)502.20 (+6.83%)305.40 (+3.00%)134.18 (-85.14%)
e216c2f — 2026-07-13 21:54:420.44 (n/a)0.27 (n/a)0.28 (n/a)0.05 (n/a)0.14 (n/a)2432.90 (n/a)823.14 (n/a)470.10 (n/a)296.50 (n/a)903.28 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.45 (-12.26%)0.32 (-9.98%)0.26 (-26.18%)0.22 (+17.93%)0.11 (-20.63%)587.00 (-15.21%)443.04 (+5.92%)511.30 (+35.48%)292.10 (+13.97%)132.82 (-24.96%)
e216c2f — 2026-07-13 21:54:420.51 (n/a)0.36 (n/a)0.35 (n/a)0.19 (n/a)0.13 (n/a)692.30 (n/a)418.28 (n/a)377.40 (n/a)256.30 (n/a)177.01 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.58 (+33.96%)0.33 (+6.62%)0.33 (+27.56%)0.18 (-13.10%)0.15 (+52.60%)712.90 (+15.08%)460.20 (+0.69%)401.70 (-21.60%)226.40 (-25.35%)186.85 (+35.96%)
e216c2f — 2026-07-13 21:54:420.43 (n/a)0.31 (n/a)0.26 (n/a)0.21 (n/a)0.10 (n/a)619.50 (n/a)457.04 (n/a)512.40 (n/a)303.30 (n/a)137.43 (n/a)
iron/operators/swiglu_decode

test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.00 (-16.67%)0.00 (-7.14%)0.00 (+0.00%)0.00 (+0.00%)0.00 (-25.00%)20174.68 (-11.90%)16641.09 (-9.84%)17470.80 (-21.58%)8813.66 (+31.02%)4645.21 (-32.29%)
e216c2f — 2026-07-13 21:54:420.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)22899.76 (n/a)18458.31 (n/a)22279.89 (n/a)6727.16 (n/a)6860.73 (n/a)

test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.00 (-57.14%)0.00 (-52.83%)0.00 (-58.33%)0.00 (+0.00%)0.00 (-81.62%)20049.50 (-12.14%)17079.29 (+70.22%)17339.67 (+151.91%)13745.98 (+126.98%)2264.03 (-68.42%)
e216c2f — 2026-07-13 21:54:420.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)22818.84 (n/a)10033.62 (n/a)6883.34 (n/a)6056.00 (n/a)7169.71 (n/a)
iron/operators/swiglu_prefill

test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:580.12 (-17.99%)0.09 (-16.54%)0.09 (-8.87%)0.07 (-7.62%)0.02 (-27.16%)29824.16 (+8.16%)24182.40 (+17.89%)23446.04 (+9.72%)16794.31 (+21.99%)5440.20 (-0.30%)
e216c2f — 2026-07-13 21:54:420.15 (n/a)0.11 (n/a)0.10 (n/a)0.08 (n/a)0.03 (n/a)27574.44 (n/a)20512.79 (n/a)21368.30 (n/a)13767.47 (n/a)5456.68 (n/a)
iron/operators/transpose

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:581.69 (+55.59%)1.27 (+49.68%)1.49 (+90.05%)0.67 (+7.30%)0.44 (+139.76%)784.70 (-6.81%)467.24 (-27.16%)350.80 (-47.38%)311.00 (-35.72%)201.90 (+43.83%)
e216c2f — 2026-07-13 21:54:421.08 (n/a)0.85 (n/a)0.79 (n/a)0.62 (n/a)0.18 (n/a)842.00 (n/a)641.50 (n/a)666.70 (n/a)483.80 (n/a)140.38 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:582.56 (+4.09%)1.40 (-20.46%)1.31 (-41.50%)0.29 (-6.49%)1.13 (+21.91%)3565.00 (+6.94%)1726.14 (+59.05%)802.00 (+70.97%)409.60 (-3.92%)1631.15 (+28.95%)
e216c2f — 2026-07-13 21:54:422.46 (n/a)1.76 (n/a)2.24 (n/a)0.31 (n/a)0.93 (n/a)3333.70 (n/a)1085.28 (n/a)469.10 (n/a)426.30 (n/a)1264.95 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.58 (-2.67%)1.21 (+5.65%)1.29 (+41.89%)0.66 (-20.05%)0.38 (-2.53%)792.90 (+25.08%)479.12 (-3.80%)406.80 (-29.52%)331.10 (+2.76%)189.18 (+26.62%)
4bb8427 — 2026-06-23 22:46:491.63 (n/a)1.14 (n/a)0.91 (n/a)0.83 (n/a)0.38 (n/a)633.90 (n/a)498.02 (n/a)577.20 (n/a)322.20 (n/a)149.40 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3c991ad — 2026-07-14 16:18:581.91 (+26.88%)1.01 (-6.97%)0.82 (-13.15%)0.66 (+4.77%)0.51 (+42.33%)794.90 (-4.55%)598.86 (+12.02%)637.40 (+15.14%)273.80 (-21.21%)195.40 (+1.02%)
e216c2f — 2026-07-13 21:54:421.51 (n/a)1.08 (n/a)0.95 (n/a)0.63 (n/a)0.36 (n/a)832.80 (n/a)534.60 (n/a)553.60 (n/a)347.50 (n/a)193.43 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.75 (-15.44%)1.32 (-0.50%)1.36 (+22.47%)0.77 (-19.20%)0.36 (-21.75%)678.10 (+23.76%)430.02 (-0.09%)385.10 (-18.34%)300.10 (+18.29%)146.96 (+19.46%)
4bb8427 — 2026-06-23 22:46:492.07 (n/a)1.32 (n/a)1.11 (n/a)0.96 (n/a)0.46 (n/a)547.90 (n/a)430.40 (n/a)471.60 (n/a)253.70 (n/a)123.02 (n/a)

@atassis

atassis commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

I am already sticking to a generic "run an arbitrary Kernel in-place on the output" epilogue slot, not a one-off gelu branch.

The genericity matters to me because I'm building a full NPU engine backend on XDNA2, so composing generic primitives instead of hand-fusing per model is the design rule I hold to - a generic epilogue slot fits that squarely. The harder piece I'm on right now is the resident fused dataflow: keeping the whole transformer block on-chip across dispatches, with on-chip KV and decode attention, to cut out the host round-trips - and that only holds together if the primitives stay composable. If you have a take on whether that's the right direction to push, I'd genuinely value it; you all have far more context on where the toolchain is heading.

On the PR itself: I'll rebase onto the latest devel (the conflict is just from my batched-gemv PR that merged) and clear the lint and any other CI so it lands clean. Happy to do the generic epilogue as a follow-on once this lands.

@atassis
gemv: add optional fused GELU epilogue (NPU2)
93d59b5 
Adds an opt-in `epilogue` parameter to GEMV. `epilogue="none"` (default) leaves
the output unchanged and is byte-identical to before; `epilogue="gelu"` applies a
GELU (tanh approximation) to each output tile inside the producing core, fusing an
activation that would otherwise be a separate elementwise pass over the GEMV output.

The GELU is applied once per full output tile in core_body (after the matvec
inner-loop fills all rows), not per matvec call, whose m_input tile can be smaller
than the 16-wide activation vector.

To avoid duplicating the GELU math, aie2p/gelu.cc is refactored: the per-16-lane
computation is factored into a shared helper used by both the existing out-of-place
gelu_tanh_approx_bf16 and a new in-place gelu_tile_bf16. In-place is aliasing-correct
(a single restrict pointer, each slot read then written) whereas reusing the
out-of-place kernel with input==output would violate its restrict contract. Because
gelu.cc is aie2p-only, the fused epilogue is NPU2-only; GEMV raises NotImplementedError
if requested on NPU1. When enabled the core links a (matvec, gelu) archive.

Test: test_gemv_gelu compares the fused output against a gelu(A @ B) golden across
three shapes, with the standard latency/bandwidth/throughput metrics. Verified on
NPU2 (15/15 pass).
@atassis
atassis force-pushed the feat/gemv-gelu-epilogue branch from a33ac61 to 93d59b5 Compare July 13, 2026 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrej andrej andrej approved these changes
@jgmelber jgmelber Awaiting requested review from jgmelber jgmelber is a code owner
@hunhoffe hunhoffe Awaiting requested review from hunhoffe hunhoffe is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@atassis @andrej