gpu ilp
GPU Instruction-Level Parallelism (ILP) is the compiler and hardware technique of executing multiple independent instructions from the same thread simultaneously within a GPU pipeline — complementing thread-level parallelism (TLP) by allowing each warp to issue multiple non-dependent instructions per cycle, which increases throughput when occupancy is limited and makes each thread more productive, especially in compute-bound kernels where extracting ILP from unrolled loops and independent operations can improve performance by 20-50%.
ILP vs. TLP on GPU
| Technique | What | How Parallelism Is Extracted |
|---|---|---|
| TLP (Thread-Level) | Many warps hide latency | Switch warps on stall |
| ILP (Instruction-Level) | Independent instructions in same thread | Pipeline + dual issue |
| Combined | Both | Maximum throughput |
- TLP: Need high occupancy (many active warps) → limited by registers, shared mem.
- ILP: Even with few warps, extract parallelism from instruction stream.
- Best performance: Both TLP and ILP combined.
GPU Pipeline
Instruction stream for one warp:
Cycle 1: FFMA r0, r1, r2, r3 ← FP multiply-add (4 cycle latency)
Cycle 2: FFMA r4, r5, r6, r7 ← Independent → issued next cycle
Cycle 3: FADD r8, r9, r10 ← Independent → issued next cycle
Cycle 4: FLD r11, [addr] ← Memory load (different unit)
Cycle 5: FFMA r0, r0, r12, r13 ← Depends on cycle 1 → must wait!
Instructions 1-4: All independent → 4 ILP
Instruction 5: Depends on result of 1 → no ILP (stall or switch warp)Extracting ILP Through Loop Unrolling
// Low ILP: Each iteration depends on previous sum
float sum = 0;
for (int i = 0; i < N; i++)
sum += data[i]; // sum depends on previous sum → no ILP
// High ILP: Multiple independent accumulators
float sum0 = 0, sum1 = 0, sum2 = 0, sum3 = 0;
for (int i = 0; i < N; i += 4) {
sum0 += data[i]; // Independent
sum1 += data[i+1]; // Independent
sum2 += data[i+2]; // Independent
sum3 += data[i+3]; // Independent
}
float sum = sum0 + sum1 + sum2 + sum3;
// 4-way ILP → pipeline stays full even with one warpILP and Register Pressure Trade-Off
| Unroll Factor | ILP | Registers per Thread | Occupancy | Net Effect |
|---|---|---|---|---|
| 1 (no unroll) | 1 | Low | High (many warps) | TLP-dependent |
| 2 | 2 | Medium | Medium | Better ILP |
| 4 | 4 | High | Lower | Best ILP if compute-bound |
| 8 | 8 | Very high | Low (few warps) | May hurt if memory-bound |
- More ILP → more registers → fewer warps per SM → less TLP.
- Optimal point depends on whether kernel is compute-bound or memory-bound.
- Compute-bound: More ILP helps (feed the pipeline).
- Memory-bound: More TLP helps (hide memory latency via warp switching).
Dual-Issue Capability
- Modern GPUs (Volta+): Two warp schedulers can issue to different functional units simultaneously.
- Example: FP32 instruction + memory load instruction → both from same warp, same cycle.
- Requires: Instructions use different execution units AND are independent.
Profiling ILP
# Nsight Compute: Check issued IPC (instructions per cycle per SM)
ncu --metrics sm__inst_executed_per_cycle ./my_kernel
# Theoretical max: 4 IPC (4 warp schedulers)
# Good: > 2 IPC
# Low ILP: < 1 IPC → instruction dependencies limiting throughputGPU instruction-level parallelism is the underappreciated dimension of GPU performance optimization — while most GPU programming advice focuses on occupancy and memory access patterns, extracting ILP through loop unrolling, independent accumulators, and instruction scheduling can deliver 20-50% additional throughput on compute-bound kernels, making it the optimization technique of choice when occupancy is already limited by register or shared memory constraints.