condconv

CondConv (Conditionally Parameterized Convolutions) is a convolution variant where kernel weights are computed as a linear combination of expert kernels, conditioned on the input — similar to Dynamic Convolution but introduced independently by Google Brain.

How Does CondConv Work?

• Experts: n convolutional kernels (experts) {W_1, ..., W_n} with the same shape.
• Routing: Input-dependent routing weights \alpha = \sigma(r(x)) where r is a routing function.
• Combined Kernel: W = \sum_i \alpha_i W_i.
• Apply: Standard convolution with the combined kernel.
• Paper: Yang et al. (2019).

Why It Matters

• Capacity Without Depth: Increases model capacity through kernel mixture instead of adding layers.
• Efficient Scaling: Multiple experts increase expressive power with manageable compute increase.
• EfficientNet: Used in EfficientNet-EdgeTPU architectures for mobile deployment.

CondConv is mixture-of-experts for convolution kernels — blending specialized filters based on the input for adaptive feature extraction.