Back to Blog
AIPublished on October 20, 2026

Deconstructing Lightning Attention: Inside MiniMax M3.1 Flash and Chunkwise Linear Recurrence

Explore how MiniMax M3.1 Flash achieves sub-quadratic inference speeds through chunkwise linear attention and fixed-size recurrent state updates. We analyze the underlying linear algebra, GPU SRAM memory patterns, and custom Triton kernel architectures driving this breakthrough.

The Memory Wall of Quadratic Attention

Standard Transformer architectures rely on scaled dot-product attention, defined classically as:

$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$$

While FlashAttention and its successors revolutionized memory-bandwidth utilization by fusing the softmax computation and tiling key-value blocks directly into GPU SRAM, they do not escape the fundamental algorithmic complexity: $\mathcal{O}(N^2)$ computational complexity in sequence length $N$ during training, and $\mathcal{O}(N)$ memory scaling per token during inference due to the ever-expanding Key-Value (KV) cache.

In frontier deployment scenarios—such as real-time agents, continuous audio/video streaming, and context windows spanning millions of tokens—the KV cache becomes a prohibitive memory bottleneck. MiniMax M3.1 Flash addresses this constraint directly by discarding dense quadratic softmax attention in favor of a specialized form of Lightning Attention paired with chunkwise linear recurrence. By reformulating attention through kernel feature maps and associative matrix operations, this architecture achieves constant $\mathcal{O}(1)$ memory consumption per decoding step and $\mathcal{O}(N)$ computational scaling across sequence length.


The Mathematical Foundations of Linear Attention

To decouple the sequence-length dependency, linear attention removes the non-linear softmax operation between $Q$ and $K^T$. In generalized linear attention, similarity is computed via explicit kernel feature maps $\phi(\cdot)$ applied row-wise:

$$\text{Sim}(q_i, k_j) = \phi(q_i) \phi(k_j)^T$$

Exploiting the associativity of matrix multiplication, the attention output at position $i$ can be rewritten as:

$$o_i = \frac{\sum_{j=1}^{i} \left(\phi(q_i) \phi(k_j)^T\right) v_j}{\sum_{j=1}^{i} \phi(q_i) \phi(k_j)^T} = \frac{\phi(q_i) \sum_{j=1}^{i} \left(\phi(k_j)^T v_j\right)}{\phi(q_i) \sum_{j=1}^{i} \phi(k_j)^T}$$

Notice the bracketed term $\sum_{j=1}^{i} \left(\phi(k_j)^T v_j\right)$. Instead of materializing an $N \times N$ attention matrix, we compute an outer product between $\phi(k_j) \in \mathbb{R}^{1 \times d_k}$ and $v_j \in \mathbb{R}^{1 \times d_v}$, yielding a local state matrix $S_i \in \mathbb{R}^{d_k \times d_v}$:

$$S_i = \sum_{j=1}^{i} \phi(k_j)^T v_j = S_{i-1} + \phi(k_i)^T v_i$$

During inference, updating this state requires only an outer product and a sum—an $\mathcal{O}(d_k d_v)$ operation completely independent of sequence position $i$. The subsequent query projection is a simple matrix-vector product:

$$o_i = \phi(q_i) S_i$$

The Need for Exponential Decay

Unconstrained linear attention suffers from two fatal flaws: historical gradient vanishing/explosion and an inability to prioritize recent context. To fix this, architectures like MiniMax M3.1 Flash introduce continuous temporal decay parameters $\lambda \in (0, 1)$ or data-dependent gating vectors $\gamma_t$:

$$S_i = \text{diag}(\lambda) S_{i-1} + \phi(k_i)^T v_i$$

This decay transforms linear attention into an expressive continuous-time linear dynamical system, functionally bridging Transformers and State-Space Models (SSMs).


Chunkwise Parallelism: Bridging Recurrence and Tensor Cores

While the recurrent view ($S_i = \lambda S_{i-1} + k_i^T v_i$) enables ultra-fast $\mathcal{O}(1)$ autoregressive generation, it destroys parallel throughput during prefill and training. GPUs feature massive hardware concurrency through systolic array Tensor Cores (e.g., NVIDIA Hopper FP8/BF16 units), which stall when constrained to sequential recurrent updates.

To resolve this tension, MiniMax M3.1 Flash employs Chunkwise Parallel Computing. The sequence of length $N$ is partitioned into non-overlapping chunks of size $C$ (typically $C \in {64, 128}$):

  1. Intra-Chunk Computation (Local Attention): Within a chunk $c$, computations are cast as high-throughput matrix multiplications. The causal interactions within chunk $c$ are computed directly on Tensor Cores: $$O_c^{\text{intra}} = \left( (Q_c K_c^T) \odot M_{\text{causal}} \odot D_{\text{local}} \right) V_c$$ where $D_{\text{local}}$ is a precomputed intra-chunk decay matrix, and $M_{\text{causal}}$ is standard lower-triangular causal masking.

  2. Inter-Chunk Computation (Recurrent State Transfer): The cumulative state emitted by all prior chunks $1, \dots, c-1$ is accumulated into a persistent global memory matrix $S_{c-1} \in \mathbb{R}^{d_k \times d_v}$: $$S_c = \Lambda^C S_{c-1} + \sum_{\tau=1}^C \Lambda^{C - \tau} k_{c, \tau}^T v_{c, \tau}$$ The cross-chunk contribution to the queries within chunk $c$ is computed via: $$O_c^{\text{inter}} = \left( Q_c \Lambda_{\text{query}} \right) S_{c-1}$$

  3. Final Merge: $$O_c = O_c^{\text{intra}} + O_c^{\text{inter}}$$

This hybrid execution path allows the prefill engine to hit >75% of theoretical GPU FLOPs using dense GEMM operations, while dropping back to purely recurrent linear execution during token generation.


Anatomy of a Chunkwise Linear Attention Triton Kernel

Implementing chunkwise linear recurrence efficiently requires writing custom kernels in OpenAI Triton to orchestrate memory transfers between High Bandwidth Memory (HBM) and on-chip SRAM.

Below is an architectural implementation illustrating how the inter-chunk state accumulation is computed without allocating intermediate $N \times N$ buffers:

import torch
import triton
import triton.language as tl

@triton.jit
def _chunk_linear_attn_fwd_kernel(
    Q_ptr, K_ptr, V_ptr, Out_ptr, State_ptr,
    stride_qb, stride_qh, stride_qm, stride_qd,
    stride_kb, stride_kh, stride_kn, stride_kd,
    stride_vb, stride_vh, stride_vn, stride_vd,
    stride_ob, stride_oh, stride_om, stride_od,
    decay_factor,
    BLOCK_M: tl.constexpr,
    HEAD_DIM: tl.constexpr,
):
    # Program indices
    pid_b = tl.program_id(0) # Batch
    pid_h = tl.program_id(1) # Head
    pid_c = tl.program_id(2) # Chunk index

    # Offsets
    offs_d = tl.arange(0, HEAD_DIM)
    offs_m = pid_c * BLOCK_M + tl.arange(0, BLOCK_M)

    # Pointers for the current chunk
    q_offset = pid_b * stride_qb + pid_h * stride_qh + offs_m[:, None] * stride_qm + offs_d[None, :] * stride_qd
    k_offset = pid_b * stride_kb + pid_h * stride_kh + offs_m[:, None] * stride_kn + offs_d[None, :] * stride_kd
    v_offset = pid_b * stride_vb + pid_h * stride_vh + offs_m[:, None] * stride_vn + offs_d[None, :] * stride_vd
    o_offset = pid_b * stride_ob + pid_h * stride_oh + offs_m[:, None] * stride_om + offs_d[None, :] * stride_od

    # Load local chunk tensors into SRAM
    q = tl.load(Q_ptr + q_offset)
    k = tl.load(K_ptr + k_offset)
    v = tl.load(V_ptr + v_offset)

    # 1. Local Intra-chunk Attention via Tensor Cores
    # Compute raw dot product: [BLOCK_M, HEAD_DIM] x [HEAD_DIM, BLOCK_M] -> [BLOCK_M, BLOCK_M]
    scores = tl.dot(q, tl.trans(k))
    
    # Construct causal mask and apply decay matrix
    causal_mask = tl.arange(0, BLOCK_M)[:, None] >= tl.arange(0, BLOCK_M)[None, :]
    scores = tl.where(causal_mask, scores, 0.0)
    
    # Local output projection
    o_intra = tl.dot(scores.to(v.dtype), v)

    # 2. Accumulate Inter-chunk State Matrix (S = K^T * V)
    # Load the running recurrent state S_{c-1} from HBM
    state_offset = (pid_b * stride_qh + pid_h) * HEAD_DIM * HEAD_DIM + tl.arange(0, HEAD_DIM)[:, None] * HEAD_DIM + tl.arange(0, HEAD_DIM)[None, :]
    
    if pid_c > 0:
        s_prev = tl.load(State_ptr + state_offset)
        # Project queries with prior recurrent state
        o_inter = tl.dot(q, s_prev)
        o_total = o_intra + o_inter
    else:
        o_total = o_intra
        s_prev = tl.zeros((HEAD_DIM, HEAD_DIM), dtype=tl.float32)

    # Update recurrent state for next chunk: S_c = lambda * S_{c-1} + K_c^T * V_c
    k_decayed = k # Apply decay profiles across tokens in block
    s_delta = tl.dot(tl.trans(k_decayed), v)
    s_new = s_prev * decay_factor + s_delta

    # Store updated state and output
    tl.store(State_ptr + state_offset, s_new)
    tl.store(Out_ptr + o_offset, o_total)

KV-Cache Elimination: Inference Dynamics

During inference, standard Transformer serving frameworks (e.g., vLLM, TensorRT-LLM) spend substantial GPU memory allocating PagedAttention blocks to house the sequence history. For an 8k context window across 32 layers with 32 heads ($d=128$), FP16 KV buffers require:

$$\text{Memory per request} = 2 \times 2 \times L \times H \times D \times N = 4 \times 32 \times 32 \times 128 \times 8192 \approx 4.29\text{ GB}$$

Under MiniMax M3.1 Flash's linear attention mechanism, we do not store historical $K$ and $V$ tensors. Instead, each attention head maintains a single recurrent state matrix:

$$S \in \mathbb{R}^{d_k \times d_v}$$

$$\text{Memory per request} = 2 \times L \times H \times (d_k \times d_v) = 2 \times 32 \times 32 \times (128 \times 128) \approx 33.5\text{ MB}$$

| Metric | Standard Softmax Transformer | MiniMax M3.1 Flash (Lightning) | | :--- | :--- | :--- | | Prefill Complexity | $\mathcal{O}(N^2)$ | $\mathcal{O}(N \cdot C)$ | | Decode Complexity | $\mathcal{O}(N)$ memory access | $\mathcal{O}(1)$ constant state update | | KV Memory at 128k Tokens | $\approx 68.7\text{ GB / stream}$ | $\approx 33.5\text{ MB / stream}$ (Fixed) | | Decode Latency Scaling | Linear growth with length | Flatline / Invariant to context depth | | Hardware Bottleneck | Memory Bandwidth Bound | Compute / Register File Bound |

Because decoding operations are no longer bounded by memory bandwidth limits during HBM-to-SRAM KV-cache transfers, throughput scales dramatically. The engine transitions from memory-bound memory-stall regimes directly into compute-saturating execution.


Architectural Trade-Offs and Retrieval Resolution

While sub-quadratic scaling yields significant speedups, linear attention introduces trade-offs in capacity:

  1. Fixed Memory Capacity vs. Exact Retrieval: A fixed-size state matrix $S \in \mathbb{R}^{d \times d}$ fundamentally behaves as a lossy compression channel. While standard softmax attention has an unbounded memory capacity to look up arbitrary tokens across millions of tokens, linear attention must compress all historical tokens into a constant-rank operator. In precision-critical retrieval tasks ("Needle-In-A-Haystack" benchmarks), pure linear transformers can struggle to distinguish between closely correlated semantic tokens unless supplemented with periodic local dense attention windows or data-dependent gating.

  2. Numerical Instability in Kernel Maps: Unlike softmax, which maps values strictly to a probability simplex $(0, 1)$ with intrinsic numerical normalization, recurrent states $S_t$ can encounter numerical drift over ultra-long contexts. MiniMax M3.1 Flash resolves this by using FP32 accumulation registers for recurrent states inside the inner kernel loops, while keeping activations in BF16 or FP8.

MiniMax M3.1 Flash demonstrates that linear attention can transition from academic curiosity to production frontier performance. By coupling chunkwise parallelization for compute-saturating prefill with continuous-decay linear recurrence for constant-memory decoding, it redefines the efficiency frontier for large language models at scale.

#AI#Machine Learning#Transformer Architecture#GPU Optimization#LLM Inference