token combine

Token combine is the post-expert reconstruction stage that returns routed outputs to original token order and merges multi-expert contributions - it closes the MoE routing loop and restores sequence-consistent activations.

What Is Token combine?

• Definition: Inverse mapping process that gathers expert outputs and places them back at source token indices.
• Combination Logic: Applies router weights to blend outputs when top-k routing uses multiple experts.
• Data Dependency: Relies on dispatch metadata to ensure exact correspondence between tokens and expert results.
• Runtime Position: Executed after expert compute and before downstream transformer operations.

Why Token combine Matters

• Correctness: Any index or weighting error corrupts token representations and model quality.
• Latency Contribution: Combine can become a hidden bottleneck in large expert-parallel groups.
• Memory Traffic: Inefficient scatter and gather patterns increase HBM and network overhead.
• Numerical Integrity: Weighted merge precision influences stability in mixed-precision training.
• Pipeline Balance: Fast combine is required so expert compute gains are not canceled downstream.

How It Is Used in Practice

• Inverse Indexing: Store compact permutation maps during dispatch for exact reconstruction.
• Fused Operations: Merge gather and weighting steps to reduce extra memory passes.
• Validation Suite: Test token-order restoration and top-k weighting parity against reference implementation.

Token combine is a critical correctness and performance stage in MoE execution - robust reconstruction logic ensures sparse routing produces usable and efficient transformer activations.