The history of large language models: a dated timeline and what it means for buyers

Most explanations of AI begin in 2022, which makes the field look like a sudden arrival. It was a decade of separate advances that happened to combine, and several of the ideas that matter most to a buyer today, such as scaling, instruction tuning and smaller efficient models, each have a date and a paper behind them. Knowing that sequence helps you judge which claims are new and which are repackaged. For the mechanics behind these steps, see our plain-English guide to how LLMs work.

What is the history of large language models in one table?

Here is the timeline, with the original paper or announcement for every row. Performance figures are the authors' own claims, reported as published.

How did language models work before neural networks?

They counted. An n-gram model, as Jurafsky and Martin's textbook explains, estimates the probability of the next word from the previous few words, using counts from a training corpus. It was useful for spelling correction and speech recognition, and it was brittle: a word sequence that never appeared in training receives zero probability, so models needed smoothing tricks to cope.

The next step that matters for buyers came with word2vec in 2013, which made it practical to learn continuous vector representations of words from very large datasets. Meaning became a position in space, and similar words landed near each other. That idea, now called an embedding, still underlies document search and retrieval.

How did attention and the transformer change things?

Between 2014 and 2018 the field found how to handle whole sentences at once. Early neural translation squeezed a sentence into a single fixed-length vector; the September 2014 attention paper let the model look back at the relevant source words instead, and sequence-to-sequence learning showed that an LSTM encoder and decoder could match phrase-based translation systems.

In June 2017 the transformer dropped recurrence entirely. Because it processes tokens in parallel, it could be trained on far more data in less time, which made the next step practical. In 2018 GPT and BERT showed that pretraining on unlabeled text, then adapting the result to a task, beat building a separate model per task. That pattern is the basis of most language model products sold today.

Why did scale and instruction tuning matter?

Scale did the first part of the work. GPT-2 reached 1.5 billion parameters in February 2019. The scaling-laws paper in January 2020 reported smooth power-law gains from more parameters, data and compute, which gave labs a reason to spend more. GPT-3 followed in May 2020 at 175 billion parameters, ten times larger than any previous non-sparse language model, and could do tasks from a few examples in its prompt.

Training method did the second part. InstructGPT (March 2022) used human feedback to make a model follow instructions, and its 1.3 billion parameter version was preferred by human raters over the 175 billion parameter GPT-3. Chinchilla, published weeks later, argued that earlier models were undertrained and that model size and training tokens should grow together. ChatGPT, announced on November 30, 2022, applied the InstructGPT recipe to dialogue and made it a free research preview. The announcement itself listed the limits: plausible-sounding but incorrect answers, and sensitivity to how a question is phrased.

What happened between 2023 and 2026?

Three trends stand out in the primary sources. Open and efficient models arrived: LLaMA reported in February 2023 that its 13 billion parameter model outperformed GPT-3 on most benchmarks, and Mixtral in January 2024 reported that activating only a fraction of a model's parameters per token could outperform or match Llama 2 70B and GPT-3.5 on the benchmarks its authors evaluated.

Multimodal input became standard. The GPT-4 report in March 2023 described a model that accepts images and text, and the April 2026 Qwen3.5-Omni report describes a single model handling audio and video as well.

Reasoning and long context moved to the center. DeepSeek-R1 in January 2025 reported reasoning gains from reinforcement learning, and the DeepSeek-V4 report in 2026 describes million-token context at much lower inference cost than its predecessor. Hallucination remains an open problem: a September 2025 paper, Why Language Models Hallucinate, argues that common benchmark scoring rewards guessing. This section reflects sources checked in October 2026, and no source here ranks one model above another.

What should a business leader learn from this history?

Four lessons follow from the dates.

1. Capability per dollar falls quickly. Stanford's 2025 AI Index reports that the inference cost of a system performing at GPT-3.5 level dropped more than 280-fold between November 2022 and October 2024. Price a project on today's rates, but do not sign a long fixed commitment at them.
2. Size is not quality. InstructGPT, Chinchilla and LLaMA each showed a smaller model matching or beating a much larger one by training it better. A buyer who equates "bigger" with "better" overpays.
3. The packaging mattered as much as the model. ChatGPT was a sibling of InstructGPT, which already existed, and what changed was the dialogue format and public access. The lesson for your own projects is that integration into the workflow often decides success, not the benchmark score.
4. Limits persist while sources multiply. Influential models have come from many different groups since 2018, and hallucination was still an open research problem in 2025. Test any model on your own work before relying on it, as our guide to testing AI before production describes.

Illustrative example: A mid-sized insurer signs a three-year committed-spend contract with one provider for a claims-summarizing workflow, priced at the rate that applied on signing day. Within two years, other models can do the same work at a fraction of that price, and the insurer cannot move because its prompts, integrations and approvals were built around one vendor's interface. The Stanford figure above concerns one capability level over two years and does not predict this case, but it shows the size of the exposure.

How do you avoid lock-in when models keep changing?

Put your own interface between your business systems and the model. Your workflow code should call a thin layer you own, with prompts, test cases and logs stored outside the vendor's platform. Switching then means pointing that layer at a different model and rerunning your tests, instead of rebuilding the integration. Our guide to model-agnostic AI architecture covers the design in detail.

Contract terms matter as much as code. Prefer usage-based or short commitments with the right to export your data, and check where each provider sends and stores your data. Models with published checkpoints, such as the DeepSeek-V4 series, whose report links to checkpoints on Hugging Face, give you an exit if a provider changes terms, though licenses differ and running a model yourself adds operating work.

How Kastling approaches this

Kastling's AI Integration & Automation work starts from the workflow, not from a favorite model. The approach is to understand the work first, connect the systems a business already runs, keep a named person in control of consequential decisions, and agree how success will be measured before building.

Model choice stays replaceable, because the history above shows how quickly the leading options change. Development is tested against real business scenarios, data access and ownership are agreed up front, and cloud deployments run on AWS or Azure. The scope and operating costs are set out in the proposal.