The Math Behind the Magic: How do Large Language Models Work?

It’s 2026. Of course you’ve used AI by now—if you haven’t, kudos to you. Artificial Intelligence, or AI, took over our world four years ago when Large Language Models (LLMs) turned into mainstream chatbots. Since then, LLMs have become baked into even the most banal of things, like googling. What was once a hypothetical dystopia where robots ruled the world is now something I’ve heard people worry about happening within the next ten years.

But have you ever wondered about how AI actually works? How can a bunch of math write a history essay or make an app? What are tokens? What does GPT even mean?

Well, AI has been around for decades, and it can range from very simple to exceedingly complex. It can be as straightforward as using a trendline to extrapolate data, or as complex as robots that do backflips or AI agents making an app from just one prompt. It has such a broad meaning because it literally just means that a computer is doing something that would usually require human intelligence. To be more specific, most of the AI you see is part of a subset called machine learning.

In machine learning, you essentially have a huge set of numbers, known as parameters, that determine how the input, like an image or prompt, turns into the output. To get a model to do something, a program edits these numbers bit by bit based on how good the output is until the model ‘learns’ how to do something. Many machine learning models are known as neural networks because their algorithms are inspired by neurons firing in the brain, although a much simpler version of it. In general, researchers have found that more parameters in LLMs, with sufficient training, can mean more intelligence. And you can see this in the LLMs that have been used over the years: GPT-2 had hundreds of millions to just over a billion parameters, depending on the size you used, and it was pretty impressive at the time. But some of the most recent models have up to trillions of parameters, on the scale of a thousand times more than GPT-2.

Large Language Models use tokens to break down text into something they can understand. Tokens can be many different things, including a character, a word, a part of a word, an emoji, a snippet of sound, a patch of pixels, and so much more. However, it can make it easier to understand how LLMs work if you think about tokens as words. To prevent the LLM from never ending, some tokens mark things like the beginning or end of a statement. In order to do the math, tokens are converted into multi-dimensional vectors, which are essentially long lists of numbers but can also be thought of as an arrow pointing in a certain direction. Tokens are like a language for the LLM; they are both what it takes as an input and what it produces as output.

In ChatGPT, GPT stands for Generative Pre-trained Transformer. Generative pretty simply means that it generates. You probably already know that most modern AI models, and what this article is talking about, are known as “generative AI” because unlike previous AI models that could scan images for faces, transcribe your speech, or suggest a video to watch next, this family of artificial intelligence can generate tons of things like videos, websites, or books from just a simple prompt.

The pre-trained part is also pretty straightforward: it essentially means that the LLM goes through a pretraining phase before it’s taught how to be a chatbot or AI Agent. Specifically, the pretraining phase uses what’s called ‘self-supervised learning,’ where the training program uses previously collected and prepared text while the LLM tries to predict what comes next at every token. Trillions of tokens are used in the pretraining process to train the models, which is how they get good at so many different scenarios and use cases.

But the real magic happens with the Transformer. No, this isn’t a transformer like Optimus Prime. When it comes to AI, a transformer is a special type of machine learning model. Transformers are very complex, but the essence is that they have two main parts that can be called attention and a feed-forward neural network. These parts are repeated many, many times, with a new set of parameters each time it's repeated, and the repetition lets the LLM get much more intelligent than if each part only happened once. Essentially, these attention and feed-forward blocks synthesize the meaning of all the tokens that it's given into the very last token, and it uses the last token to now predict what the next token could be, and the LLM chooses one out of the ones that it predicted to go next. It is important to remember that this explanation of a transformer is simplified, but it nevertheless can give you an idea of how they work.

The purpose of the attention block is essentially to combine the meaning of tokens that are related to each other so that the LLM can understand the whole text rather than just individual words. For example, it could add the meaning of an adjective to the word that the adjective describes. In the phrase “a red lamp,” this would mean adjusting the token that represents “lamp” so that from just that new token alone you can tell that the lamp is red. It does this for each token by asking a “query” about the relationship between the token and each preceding token, and the model uses “keys” to “answer” those questions. These keys and queries are vectors calculated using parameters that are tuned during the training process. If you find it hard to understand how sets of numbers can ask questions and provide answers, don’t worry; I was just as confused. But if you want to learn more, there are many videos on YouTube that can help break it down visually and help you understand better. In fact, YouTube is what taught me how LLMs work.

Next, the tokens go through the feed-forward block, which can also go by other names such as multilayer perceptrons. Here, the tokens don’t interact with each other as they did in the attention blocks. Instead, they get processed using the parameters of the model to add new information to the tokens, based on what the parameters have learned. If you’re familiar with how older machine learning models work, like those that identify objects in pictures, each feed-forward block is like one of those applied to every token it processes. The feed-forward block can help the LLM add knowledge to the tokens that isn’t contained within the input, but instead learned through the huge amount of pretraining it goes through. For example, if you asked about the capital city of the United States, the feed-forward blocks could provide the knowledge that its name is Washington, D.C., or that it’s home to the White House. Essentially, one of the main functions of the feed-forward block is to help make use of facts and information stored throughout the LLM. 

Through many iterations of attention blocks and feed-forward blocks, all the information that the LLM needs to predict the next token gets transferred into the last token in the sequence. It then uses the values of the numbers making up the updated last token, along with more trained parameters, to create a probability distribution of what the next token should be. Finally, it picks a token based on the probability distribution, usually semirandomly, for it to output next. And then the original text, with the new token added, goes through the whole process of the transformer again and again, adding a new token each time. That’s how it’s able to make long outputs with thousands of tokens—crazy how fast it goes, right? There’s also more to how the semirandomness works, but it’s essentially why it can create different outputs from the same prompt.

Now, you may be wondering how a language model can do more than just write text. How can it search the web or control your computer? Thankfully, this part is much easier to understand. Just like the tokens I mentioned earlier that can end a statement, there are also tokens that get used to execute commands, like specialized command words. It uses sets of those commands to do stuff, like using a browser. Then it adds all the outputs of those commands back to its input, so that the transformer can decide what to do next.

Wow, that was a lot. But hopefully, the next time you ask a chatbot a question, you’ll have a better idea of what’s going on behind the scenes. Your words become tokens, the transformer processes them using what it learned during training, and it builds a response one token at a time. That process can produce some pretty impressive answers, but a convincing answer can still be wrong, so it’s worth checking important facts. You don’t need to understand every calculation to use AI, but knowing the basics can help you make sense of both what it can do and where it can go wrong.