Gated Bidirectional Linear Attention for Generative Retrieval

In generative retrieval, bidirectional encoder attention improves quality but scales quadratically with the very long user histories seen in large-scale streaming services. GBLA is a linear-time bidirectional attention layer that extends kernelized linear attention with local causal mixing (Conv1D), sequence-level key gating for soft forgetting, and a gated RMSNorm output. A hybrid encoder interleaving self-attention and GBLA in a 1:2 ratio matches full bidirectional self-attention quality while reaching up to 8.2× single-layer speedup at a history length of 32768 on H100 GPUs, and it generalizes to public Amazon benchmarks.