dynamic convolution

Dynamic Convolution is a convolution where the kernel weights are dynamically generated based on the input — rather than using fixed, learned weights. The kernel adapts to each input, providing input-dependent feature extraction.

How Does Dynamic Convolution Work?

• Attention Over Kernels: Maintain K fixed kernel candidates. Generate attention weights pi_1, ..., pi_K from the input via squeeze-excite.
• Aggregate: W_dynamic = ∑_k pi_k · W_k (weighted sum of kernel candidates).
• Apply: Use W_dynamic for standard convolution on the current input.
• Paper: Chen et al. (2020).

Why It Matters

• Adaptive: Different inputs get different effective kernels -> more expressive than static kernels.
• Lightweight: Only adds a small attention module to generate kernel weights.
• MobileNets: Particularly effective for lightweight models where increasing width is too expensive.

Dynamic Convolution is input-adaptive filtering — generating custom convolutional kernels on the fly for each input.