The design was introduced in the 2017 paper Attention Is All You Need. Earlier language models read text one step at a time and struggled with long passages. A transformer looks at the whole sequence at once and learns which parts matter for each word, and that work can be spread across many processors at the same time, which made training on very large datasets practical.
The misconception is that a transformer looks facts up. It predicts the next token from patterns learned in training, which is why it can write a confident wrong answer. That is the practical reason to connect it to your own approved documents and test it on your own work before relying on it.