cxl memory
CXL (Compute Express Link) Memory is the open standard interconnect protocol that enables cache-coherent memory expansion and sharing across CPUs, GPUs, and memory devices — allowing servers to attach additional memory pools beyond the directly-attached DDR, with CXL memory appearing as regular system memory to applications, addressing the growing gap between compute capacity and memory capacity in AI inference, in-memory databases, and HPC workloads where memory is the primary bottleneck.
Why CXL
- DDR5 channels per CPU: Limited to 8-12 channels → max ~1-2 TB per socket.
- AI inference: Large model weights need more memory than DDR can provide.
- Memory stranding: Some servers underuse memory while others are memory-starved.
- CXL: Attach additional memory devices over PCIe 5.0/6.0 physical layer → expand capacity.
CXL Protocol Types
| Type | Protocol | Purpose | Example |
|---|---|---|---|
| CXL.io | PCIe-compatible | Device discovery, configuration | All CXL devices |
| CXL.cache | Cache coherence | Device caches host memory | Smart NICs, accelerators |
| CXL.mem | Memory access | Host accesses device memory | Memory expanders |
CXL Device Types
| Type | CXL Protocols | Use Case |
|---|---|---|
| Type 1 | CXL.io + CXL.cache | Accelerators that cache host memory |
| Type 2 | CXL.io + CXL.cache + CXL.mem | GPUs, FPGAs with own memory |
| Type 3 | CXL.io + CXL.mem | Memory expanders (pure memory) |
CXL Memory Expander (Type 3) and Switch-Attached Pooling
Latency and Bandwidth
| Memory Type | Latency | Bandwidth (per channel) |
|---|---|---|
| DDR5 (local) | 70-90 ns | ~50 GB/s per channel |
| CXL 1.1 (direct attach) | 150-250 ns | ~32 GB/s (PCIe 5.0 x8) |
| CXL 2.0 (through switch) | 200-350 ns | ~32 GB/s |
| Remote NUMA (2-socket) | 120-180 ns | ~200 GB/s |
CXL for AI/ML
- LLM inference: 70B model at FP16 = 140 GB → fits in CXL-expanded memory.
- KV cache expansion: Long context (1M tokens) KV cache in CXL memory → slower but available.
- Recommendation systems: Embedding tables (TBs) in CXL memory pool.
- Tiered memory: Hot data in DDR, warm data in CXL → automatic NUMA-like tiering.
CXL memory is the most significant server architecture evolution since NUMA — by breaking the tight coupling between CPUs and their directly-attached DRAM, CXL enables flexible memory composition that can adapt to workload demands, addressing the memory capacity wall that is increasingly the bottleneck for AI inference and in-memory data processing at scales where adding more DDR channels is physically impossible.