Scaling Recommender Transformers to One Billion Parameters

A recipe for training generative-recommender transformers up to one billion parameters. We show that autoregressive learning on user histories decomposes naturally into feedback prediction and next-item prediction, and that this decomposition scales effectively across a wide range of transformer sizes. Deployed on a large-scale music platform serving millions of users, the model increased total listening time by +2.26% and the likelihood of user likes by +6.37% in online A/B tests.