Обработка языка и большие языковые модели

Language is cut into tokens, compressed into vectors, and then made to “look at” itself through attention — a loop so plain it seems excessive, yet it lifted next-token prediction all the way to a general assistant. This domain traces that path in order: bag of words, embeddings, RNNs, attention, Transformers, pre-training and alignment.