network on chip noc

Network-on-Chip (NoC) Architecture is a scalable on-chip communication framework that replaces traditional bus-based interconnects with packet-switched networks, enabling efficient data movement in many-core and AI accelerator chips.

NoC Topology and Routing

• Mesh Topology: Regular 2D grid arrangement of routers (most common). Scales well to moderate core counts (~100s cores) with predictable performance.
• Torus Topology: Mesh with wrap-around connections on edges. Reduces diameter and improves bisection bandwidth compared to mesh.
• Ring Topology: Linear ordering of nodes. Lower area overhead but higher latency for distant cores.
• Routing Algorithms: XY routing (dimension-ordered), adaptive routing selects alternate paths based on congestion. Deadlock-free routing using virtual channels.

NoC Router Microarchitecture

• Input/Output Port Design: Each router port includes input buffers (FIFO), crossbar switch, and arbitration logic.
• Virtual Channels: Multiple independent channels per physical link prevent HOL (head-of-line) blocking and enable deadlock avoidance. Typically 4-8 VCs per port.
• Crossbar Switch: Handles simultaneous transfers between input and output ports. Area and power scale as O(n²) where n is radix.
• Arbiter Implementations: Round-robin, priority-based, or weighted arbitration for port conflicts. Critical for throughput and fairness.

Flow Control and QoS

• Wormhole Switching: Packet travels in flits. Low latency, low buffer overhead but entire packet remains in-flight during routing.
• Virtual Cut-Through: Buffers entire packet at intermediate nodes. Higher latency but enables better path optimization.
• QoS Mechanisms: Traffic class assignment, priority levels, bandwidth reservation for real-time tasks (critical for SoC interconnects).

Real-World Usage and Performance

• Many-Core CPUs: 64+ core designs require NoC for intra-cluster and inter-cluster communication.
• AI Accelerators: Tensor cores demand low-latency, high-bandwidth communication. TPU, Cerebras, and Graphcore use custom NoC designs.
• Typical Performance: 5-10 cycle latency per hop in modern implementations. Throughput limited by virtual channel bandwidth and arbitration efficiency.