← All posts
GENAILLMFOUNDATIONS

Generative AI: What It Is, How It Works, Why It Matters

Where generative AI came from, the four training steps behind a model like ChatGPT, and an honest separation of the hype from the substance.

Brando Koch
Brando Koch
SEPTEMBER 22, 2023 · 10 MIN READ

Three questions, answered in order: what generative AI actually is, how a model like ChatGPT is built, and why it matters to anyone who writes software or runs a business. The treatment is deliberately accessible, but nothing here is dumbed down to the point of being wrong.

Note: Sources are listed at the bottom.

What is it?

Before AI there was Programming 1.0

For most of the history of software there was one way to make a computer do something: implement the whole logic of what you want. Every branch, every rule, every edge case, written by a person. Call it Programming 1.0. It works extremely well when the logic can be written down, and it falls apart entirely when it cannot. Nobody has ever written the rules that distinguish a cat from a dog in a photograph.

What “AI” means, precisely

The loose definition is that AI is a program that can learn from data or experience. “Learn” is doing a lot of work in that sentence, so it is worth using the precise version, from Tom Mitchell in 1997 [1]:

“A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”

That definition is thirty years old and still the cleanest one available. It also tells you exactly what you need to build one: a task, a way to measure performance on that task, and experience to learn from.

Why neural networks won

Neural networks became the dominant tool because they are universal function approximation machines. Cybenko proved in 1989 that a network with a single hidden layer can approximate any continuous function to arbitrary precision [2], and backpropagation, popularized by Rumelhart, Hinton and Williams in 1986, gave us a practical way to actually find that approximation [3].

Put in Mitchell’s terms, the recipe has four parts:

Programming 2.0

Once that works, a second way of programming becomes available: instead of implementing the logic, show me examples of what you want. That is Programming 2.0, and in practice it mostly means supervised learning. It solved problems that were effectively impossible under Programming 1.0, image classification being the obvious one.

It also introduced a new bottleneck, which becomes important later: somebody has to produce those examples.

Generative AI

Generative AI is the class of AI systems that can generate new data based on the patterns and examples in existing data, and that are able to produce much more complex, higher dimensional outputs than a class label. A paragraph instead of “positive sentiment”. An image instead of “cat”.

The reason this feels like a step change rather than an increment is the occurrence of emergent abilities: capabilities that were never explicitly trained for and that simply appear once a model passes a certain scale [4]. More on those below.

What actually enabled it

Contrary to popular belief, generative AI was not an AI research breakthrough. There was no single new idea that unlocked it. Three things happened at once:

Whisper is my favourite illustration of the first point. Architecturally it is a fairly ordinary encoder-decoder transformer. What makes it remarkable is the scale and breadth of the data it was trained on. The architecture was not the story.

Programming 3.0

So the sequence looks like this:

This is the first time AI has become an easy-to-use tool for the general public. No dataset, no training loop, no ML background. You describe the task in your own language and the system attempts it. Whatever else you think about the current moment, that is a genuine discontinuity in who gets to use this technology.

How it works?

The model

Take the most famous example, ChatGPT. Unpacked, the name is a description of the thing: an instruction following, generative, pretrained transformer. The transformer architecture comes from Vaswani et al. in 2017 [5], and GPT models use only the decoder half of it.

The Transformer architecture from Vaswani et al. (2017): an encoder stack on the left and a decoder stack on the right, each repeated N times, built from multi-head attention and feed-forward layers with add-and-norm connections, fed by input and output embeddings plus positional encoding, ending in a linear layer and softmax over output probabilities.

Figure from Attention Is All You Need, Vaswani et al. (2017).

I am not going to dissect the architecture here. What is more useful for understanding why these systems behave the way they do is how they are trained, which happens in four steps.

Before self-supervision, for contrast

Before that shift, models were trained for one particular task with supervised learning. You want sentiment classification, so you feed the model “I really liked the movie, the plot was intriguing” as input and “positive sentiment” as the output, and it learns the functional dependence between them.

The catch is that the output has to be labeled by a human. To scale the data you have to scale the labeling effort, and that gets expensive fast. Large language models do two things differently: they do not train task-specific models, they train foundational ones, and the basis of those models requires no human labeling at all.

Step 1: Pretraining, the self-supervised objective

Give the model a chunk of text with the last word missing and ask it to predict the missing word. That is it. This is called next word prediction, and the model is learning to model the conditional probability of text.

Input:   Today we are going to the city to watch a ____
         conditional probability -> 0.72 Movie
                                    0.20 Play
                                    ...
Output:  Movie      (labeled by the dataset itself)

The label is free. It is already in the data. Scrape enough text and you have effectively unlimited training examples with no labeling bottleneck. Now scale this to trillions of words, and train for months on clusters of thousands of GPUs.

This step is the single most important reason behind the so-called intelligence of these systems, and it is the one people underrate because the objective sounds trivial. Ilya Sutskever has an illustration I keep reusing: imagine a detective novel where the last sentence is “and the murderer is ____”. To predict that word reliably you need to have tracked the characters, the alibis and the contradictions across the whole book. Next word prediction, done well enough at sufficient scale, forces the model to build something that behaves a lot like understanding.

Everything that follows is comparatively cheap, and it only works because of what happened here.

Step 2: Fine-tuning, the supervised objective

We taught the model to predict the next word, but that is not what we want from it. We want it to follow instructions. So the inputs are reformulated as instructions and humans provide the desired responses.

Input:   Describe to me how to measure execution time in python
Output:  Using the time module you can ...      (human labeled)

This does not require anything like the scale of step 1. A comparatively small, high quality set is enough, because the model is drawing on everything it learned about the conditional probability of language in order to do this. We are not teaching it language. We are teaching it a format.

Step 3: Reward modeling

Next we want responses that are not just correct but good, in the sense of what a human would prefer. The input here is the prompt together with a response the model generated, and the output is a score that a human assigns.

Input:   How do I debug my python app | You can use the print to...
Output:  0.2      (human labeled preference score)

The result is a separate model that has learned which responses people rate highly and which they do not.

Step 4: Reinforcement learning

The final step uses that knowledge of human preference to optimize the model itself. The input is an instruction, the model generates the response, and the reward model scores it.

Input:   How do I debug my python app
Output:  Use the logging module to...   (predicted response)
         0.8                            (predicted human preference)

We then optimize the model to produce those responses, weighted by the predicted quality score. This is why the assistant you talk to is more agreeable, more structured and more careful than a raw pretrained model would be. Much of the personality is manufactured here.

Emergent abilities

Three abilities show up that nobody explicitly trained for, and it is still genuinely a mystery why they appear when they do [4].

Step-by-step reasoning. Ask for the distance a train covers and the model lays out the speed calculation before applying it, rather than guessing an answer.

In-context learning. Tell it your pet is called Fluffy, then that Fluffy is a cat, then that she is three years old, and it uses all of it in the next answer. The model learned inside a conversation, with no weight updates involved.

Instruction following. Give it several instructions at once, “sort these numbers, then average them”, and it decomposes and executes them in order.

Compared to human intelligence

This is where I want to be careful, because the gap between what these systems do and what people assume they do is large. Gary Marcus puts it well [6]:

“While GenAI can do reflexive statistical analysis it has little to no capacity for deliberate reasoning… We are far from AGI”

The framing I find most useful is System 1 versus System 2 thinking. Generative AI has essentially mastered System 1: fast, associative, pattern-driven, the kind of response you produce without deliberating. It is largely missing System 2: slow, deliberate, checkable reasoning. Give a current model a recursive logic riddle and you can watch the difference. It produces something that has the shape of reasoning, confidently, and it is often wrong in ways that a person doing actual step-by-step deduction would not be.

Why it matters?

A significant step towards further automation

Fei-Fei Li’s summary is the one I would pick [7]:

“We seem to have reached an inflection point in the ability of machines to generate language, image, audio and more”

The consequence is an exponential rise in automation potential, and the numbers people have put on it are large enough to be worth stating carefully.

McKinsey estimates that generative AI could add $2.6 to $4.4 trillion annually across the use cases they analyzed, and their full report carries the charts behind these figures [8]. For scale, the entire GDP of the United Kingdom is around $3.1 trillion. They also estimate that half of today’s work activities could be automated somewhere between 2030 and 2060, and Goldman Sachs puts roughly two thirds of US occupations as exposed to at least some degree of automation [6].

One number in the McKinsey report stands out more than the money. Their estimate of when machines would reach median human performance in natural language understanding moved forward by about a decade. A capability they had penciled in for the 2030s arrived now. Whatever your forecasting method was, it was too slow.

Where the value lands is the uncomfortable part. About 75% of it concentrates in four areas: customer operations, marketing and sales, software engineering, and R&D [8]. Knowledge work and creative work are the most exposed, and the potential is higher for jobs requiring more education, which inverts the usual assumption about automation. Manual labour is comparatively safe, and the reason is unglamorous: there is not enough of the right data. If you work with knowledge or creative output inside a digital environment, you are the one in the blast radius.

Two more observations. None of the big players is pushing to slow this down, which is different from previous platform shifts. And the entry fee is low: a skilled software engineer can get a long way with open models and published recipes, without a research lab behind them.

Identifying the hype

Since this is the question I get asked most, here is where I would put things today.

A Venn diagram of two overlapping circles sorting generative AI claims. Not hype: significant productivity growth, and harm potential. The middle ground where the circles overlap: LLM wrapper companies, and the need for regulation. Hype: AGI, and human-like reasoning.

Not hype: significant productivity growth, and the harm potential. Both are real and both are already happening.

Hype: AGI, and human-like reasoning. See the System 2 problem above.

Middle ground: the wave of LLM wrapper companies, and the need for regulation. On the first, a company whose entire product is a thin layer over somebody else’s foundation model, with no proprietary data underneath, is one model update away from being obsolete. Some will survive by owning a workflow or a dataset. Most will not. On regulation, I do not think the position that nothing should be done is defensible, and I do not think most of the proposals currently on the table are well aimed either.

Road to a world model

The last thing worth saying is that the potential of the technology we already have is still unexplored. Without any architectural upgrade at all, the same approach can be applied to much bigger and more diverse datasets and to modalities beyond text.

A diagram captioned that the potential is still unexplored and could result in order-of-magnitude improvements. Arrows feed text, audio, video and other sensory data into a single densely connected neural network, illustrating one model trained across many modalities at once.

Text, audio, video, and other sensory data, fed into one model. That direction could produce order-of-magnitude improvements from scale and breadth alone, before anyone invents anything new.

Advice

Three things, if you take nothing else from this.

  1. Embrace generative AI. Not as an act of faith, but as an experiment you run yourself.
  2. Know how to use it, not necessarily how it works. Understanding attention heads is my job. Knowing which of your tasks it is good at is yours.
  3. Invest time into testing new tools for your industry. The space moves too fast for a one-time evaluation.

Concretely, things worth an hour of your time right now: Bing AI for summarizing long PDFs and web pages, GitHub Copilot for code, where GitHub’s own research found developers completed a task 55% faster and reported enjoying the work more [9], and Midjourney, Stable Diffusion or DALL-E 2 for images.

References

TAGS: GENAI · LLM · FOUNDATIONS