Add a listing
    0
    Add a listing

    Tag: Next-Token Prediction

    Sep 16
    17 min read
    The Transition from Next-Token Prediction to Hierarchical Goal Planning

    For more than a decade, the foundational dogma of deep learning and generative artificial intelligence rested upon an elegant, deceptively simple statistical objective: autoregressive next-token prediction. By training multi-layer transformer architectures to calculate the conditional probability distribution of the next discrete token given an antecedent sequence of text, research laboratories produced systems with breathtaking conversational […]