selective kernel networks

Selective Kernel (SK) Networks are a dynamic kernel selection mechanism that adaptively chooses different convolutional kernel sizes for different inputs — using an attention mechanism to softly combine features from multiple kernel sizes based on the input content.

How Do SK Networks Work?

• Split: Apply convolutions with different kernel sizes (e.g., 3×3 and 5×5) to the same input.
• Fuse: Add the outputs element-wise -> global average pooling -> compact feature vector.
• Select: Softmax attention over the kernel branches: $a_k = \text{softmax}(W_k z)$ for each kernel $k$.
• Aggregate: Final output = weighted sum of branches: $y = \sum_k a_k \otimes F_k$.
• Paper: Li et al. (2019).

Why It Matters

• Adaptive Receptive Field: The network learns to use small kernels for fine details and large kernels for global context, per-input.
• Content-Dependent: Different images (or different regions) get different effective kernel sizes.
• Influence: The dynamic kernel concept influenced subsequent works like CondConv and Dynamic Convolution.

SK Networks are neural networks that choose their own kernel size — dynamically adjusting the receptive field based on what the input needs.