1,045 technical terms and definitions
compute pipeline gpu, general purpose gpu, gpgpu programming, dispatch compute workgroup
GPU Compute Shaders are the programmable pipeline stages that execute general-purpose parallel computations on GPU hardware outside the traditional graphics rendering pipeline…
compute pipeline vulkan, spirv shader, workgroup invocation, vulkan synchronization barrier
Vulkan Compute Shaders enable portable, hardware-agnostic GPU computing across diverse platforms (NVIDIA, AMD, Intel, mobile GPUs), leveraging SPIR-V intermediate representation…
gpgpu, general purpose gpu computing, gpu acceleration
GPU computing definition and practical boundary. uses graphics processors for general-purpose parallel work beyond raster graphics. GPUs devote large area to throughput-oriented…
cooperative kernel launch, thread block cluster, grid level synchronization, cooperative groups cuda
GPU Cooperative Groups is the CUDA programming model extension that provides flexible, hierarchical thread grouping and synchronization primitives beyond the fixed thread-block…
infrastructure
GPUDirect is the set of technologies that enable direct data paths between GPUs and external devices with minimal CPU mediation - it reduces copy hops and latency across GPU…
infrastructure
GPUDirect RDMA is the remote direct memory access capability that lets network adapters move data directly between remote and local GPU memory - it enables low-latency, zero-copy…
gpudirect networking, rdma gpu memory, zero copy gpu transfer, ib verbs gpu
GPU Direct RDMA is the data path that allows network adapters to read and write GPU memory directly without host staging. What It Covers Core concept: cuts copy overhead and host…
cuda fft optimization, cufft performance tuning, fast fourier transform gpu, frequency domain gpu
GPU FFT and Signal Processing is the parallel implementation of Fast Fourier Transform and related signal processing operations on GPUs — where cuFFT library delivers 500-2000…
graphics processing unit, hardware
GPU (Graphics Processing Unit) is the massively parallel processor that has become the primary hardware accelerator for deep learning — containing thousands of cores optimized…
instruction level parallelism gpu, gpu pipeline, gpu instruction scheduling, gpu throughput
GPU Instruction-Level Parallelism (ILP) is the compiler and hardware technique of executing multiple independent instructions from the same thread simultaneously within a GPU…
operator fusion, kernel launch overhead, fused kernel, xla fusion
Kernel fusion (also called operator fusion) is the optimization of combining several separate GPU operations into a single kernel, so that intermediate results stay in fast…
operator fusion optimization, kernel launch overhead, fused kernel computation, memory traffic reduction
GPU Kernel Fusion is the performance optimization technique that combines multiple separate GPU kernels into a single fused kernel — eliminating the overhead of multiple kernel…
cuda kernel launch, kernel fusion motivation, launch latency, gpu dispatch
GPU Kernel Launch Overhead is the fixed latency cost (typically 3-10 microseconds) incurred each time the CPU dispatches a computation kernel to the GPU — which becomes…
gpu kernel launch overhead and host-device synchronization, gpu kernel launch, kernel launch overhead, launch overhead host-device, overhead host-device synchronization, gpu kernel
GPU Kernel Launch Overhead and Host-Device Synchronization Introduction & Motivation GPU Kernel Launch Overhead and Host-Device Synchronization addresses a central problem…
kernel tuning, occupancy optimization, instruction throughput gpu
GPU Kernel Optimization is the systematic process of tuning GPU compute kernels to maximize hardware utilization and minimize execution time, addressing memory access patterns…
gpu kernel optimization techniques, gpu kernel optimization, kernel optimization techniques, gpu kernel, kernel optimization, optimization techniques
GPU Kernel Optimization Techniques Introduction & Motivation GPU Kernel Optimization Techniques addresses a central problem in ml-hardware process control and warp-utilization…
cuda kernel tuning, warp occupancy maximization, shared memory tiling, gpu memory coalescing
GPU Kernel Optimization Techniques — Systematic methods for maximizing throughput and minimizing latency of computational kernels executing on massively parallel GPU…
gpu performance analysis, occupancy analysis, nsight profiling
GPU Kernel Profiling is the systematic measurement and analysis of GPU kernel execution characteristics — occupancy, memory throughput, compute utilization, stall reasons…
gpu memory hierarchy, hbm, gddr, shared memory, gpu registers, l1 l2 cache, unified memory
GPU memory is the hierarchy that stores and moves instructions, model parameters, activations, caches, and intermediate tensors for graphics and accelerator execution. For modern…
coalesced memory access, global memory coalescing, warp memory access pattern, memory transaction efficiency
GPU Memory Coalescing is the hardware mechanism that combines multiple individual memory requests from threads within a warp (32 threads) into a single wide memory transaction…
memory access pattern gpu, global memory transaction, aligned memory access, strided access gpu
GPU Memory Coalescing is the hardware optimization where adjacent threads in a warp (32 threads) that access adjacent memory addresses have their individual memory requests…
coalesced memory access cuda, memory transaction efficiency, global memory access pattern, memory coalescing warp
GPU Memory Coalescing is the hardware mechanism that combines multiple per-thread memory accesses within a warp into fewer, wider memory transactions — achieving maximum global…
hardware
GPU memory hierarchy is the layered organization of storage levels with different capacities and latency-bandwidth characteristics - effective kernel design depends on maximizing…
gpu cache, l1 l2 cache gpu, gpu memory architecture, gpu hbm bandwidth
High-bandwidth memory (HBM) is DRAM stacked vertically and connected to its host through an interface thousands of bits wide, running beside the processor on an interposer rather…
shared memory registers, gpu cache, memory coalescing
GPU Memory Hierarchy is the multi-level storage system in GPU architectures that provides different capacity-bandwidth-latency trade-offs, from per-thread registers (fastest…
shared memory gpu tiling, global memory coalescing, texture cache gpu, register spilling gpu performance
GPU Memory Hierarchy Optimization — GPU performance is fundamentally constrained by memory bandwidth and latency, making effective utilization of the multi-level memory hierarchy…
shared memory cuda, l1 l2 cache gpu, memory bandwidth optimization, global memory access patterns
GPU Memory Hierarchy Optimization is the practice of strategically utilizing the multi-level memory system of modern GPUs — from fast but small shared memory and L1 cache (20…
cuda memory types, gpu cache optimization, shared memory optimization, gpu memory bandwidth
GPU Memory Hierarchy Optimization is the systematic tuning of data placement and access patterns across GPU's multi-level memory system to maximize bandwidth utilization…
unified memory, cuda memory, device memory
GPU Memory Management — understanding the GPU memory hierarchy and managing data transfers between host (CPU) and device (GPU) memory to avoid bottlenecks that dominate…
unified virtual memory, cuda managed memory, gpu memory allocation, pinned memory transfer
GPU Memory Management is the system-level discipline that governs how data is allocated, transferred, and accessed across the discrete address spaces of CPU (host) and GPU…
unified memory cuda, pinned memory allocation, cuda memory types, gpu memory optimization
GPU Memory Management is the systematic allocation, transfer, and optimization of data across CPU and GPU memory spaces to maximize performance and minimize overhead — where…
virtual memory gpu, cuda managed memory, gpu page fault, memory oversubscription gpu
GPU Virtual Memory and Memory Management is the system software and hardware infrastructure that provides address translation, demand paging, and memory protection for GPU…
unified virtual addressing, gpu page fault, gpu memory oversubscription
GPU Virtual Memory Management is the system of hardware and software mechanisms that provide GPUs with virtual address spaces, demand paging, memory oversubscription, and unified…
memory allocator gpu, cuda memory pool, caching allocator, pytorch memory
GPU Memory Pool Allocators are the caching memory management systems that maintain pre-allocated pools of GPU memory to eliminate the overhead of frequent cudaMalloc/cudaFree…
gpu memory pressure and host-device swapping overhead, gpu memory pressure, memory pressure host-device, pressure host-device swapping, host-device swapping overhead, gpu memory
GPU Memory Pressure and Host-Device Swapping Overhead Introduction & Motivation GPU Memory Pressure and Host-Device Swapping Overhead addresses a central problem in heterogeneous…
optimization
GPU memory utilization is the fraction of available accelerator memory actively consumed by model state, activations, and runtime buffers - it guides batch sizing and memory…
multi process service, cuda mps, gpu sharing processes, mps nvidia
GPU Multi-Process Service (MPS) is the NVIDIA runtime service that enables multiple CUDA processes to share a single GPU concurrently with improved efficiency — replacing…
nvidia mig partitioning, gpu isolation mig slices, mig compute instance profile, a100 mig configuration gpu
GPU Multi-Instance GPU (MIG) is a hardware partitioning feature introduced with NVIDIA's A100 (Ampere) architecture that divides a single physical GPU into up to seven independent…
gpu sharing, gpu virtualization multi user, time slicing gpu
GPU Multi-Tenancy is the sharing of a single physical GPU among multiple applications, users, or virtual machines, providing isolation, fairness, and efficient utilization…
nvswitch all to all, nvlink 4.0 bandwidth, nvlink c2c chip to chip, nvidia dgx h100 nvlink
NVLink and NVSwitch GPU Interconnects: High-Bandwidth All-to-All GPU Networking — specialized interconnect technology enabling 900 GB/s per-GPU communication for tightly-coupled…
occupancy calculator, warp occupancy, thread block size, sm utilization
GPU Occupancy Optimization is the process of maximizing the ratio of active warps to the maximum possible warps per Streaming Multiprocessor (SM) — achieved by carefully choosing…
sm occupancy, register pressure gpu, thread block size selection, occupancy calculator
GPU Occupancy Optimization is the performance tuning discipline that maximizes the number of active warps per Streaming Multiprocessor (SM) — measured as the ratio of active warps…
register pressure gpu, occupancy limiter, latency hiding gpu, active warps per sm
GPU Occupancy is the ratio of active warps on a Streaming Multiprocessor (SM) to the maximum number of warps the SM can support — a key performance metric that determines…
cuda occupancy calculator, register pressure gpu, shared memory occupancy, thread block sizing
GPU Occupancy Optimization is the practice of maximizing the number of active warps per Streaming Multiprocessor (SM) relative to the hardware maximum — balancing register usage…
out of memory, cuda error, gradient checkpointing, quantization, memory optimization, batch size
GPU Out of Memory (OOM) errors occur when model weights, activations, or intermediate computations exceed available GPU VRAM — a common issue in ML development that requires…
device plugin, nvidia
GPU Management in Kubernetes NVIDIA GPU Operator Automates GPU driver installation, container toolkit, and device plugins in Kubernetes. Installation Components Installed…
nvtx annotation, roofline model gpu, achieved bandwidth occupancy, gpu bottleneck analysis
GPU Performance Profiling encompasses systematic measurement and analysis of kernel execution, memory access patterns, and hardware utilization using Nsight tools, roofline…
dvfs gpu, gpu power limit, gpu frequency scaling, gpu thermal throttle
GPU Power Management and DVFS is the dynamic adjustment of GPU clock frequency and voltage to balance performance, power consumption, and thermal limits — where modern GPUs…
gpu energy efficiency, power capping, gpu tdp, thermal design power gpu
GPU Energy Efficiency and Power Management is the set of hardware and software mechanisms that dynamically control GPU power consumption to maximize performance within thermal…
nsight compute profiling, nsight systems timeline, cuda profiling tools, gpu performance analysis
GPU Profiling and Debugging is the systematic analysis of GPU application performance and correctness using specialized tools that provide detailed metrics, timeline…