split learning

Split Learning is a distributed learning technique that splits a neural network at a "cut layer" between the client and server — the client processes data through the lower layers (keeping data private), sends intermediate activations (smashed data) to the server, which completes the forward pass and backpropagation.

How Split Learning Works

• Client Side: Forward pass through layers 1-k on private data → produce activations a_k.
• Server Side: Receive a_k, forward through layers k+1-L, compute loss, backpropagate to layer k+1.
• Gradient Return: Server sends \nabla a_k (gradient of loss w.r.t. activations) back to client.
• Client Backward: Client backpropagates through layers 1-k using \nabla a_k.

Why It Matters

• Low Client Compute: Client only runs part of the model — suitable for resource-constrained edge devices.
• Privacy: Raw data never leaves the client — only intermediate activations are shared.
• Caveat: Intermediate activations can leak information — additional protections (noise, quantization) may be needed.

Split Learning is dividing the neural network — clients process their data through the bottom half, servers complete the computation through the top half.