What generative AI is and how chatbots actually work

AI chatbots can write, explain and summarise in seconds, which makes them feel like they know things. Understanding what is really happening inside them helps you use them cleverly and trust them only as much as they deserve.

AI, machine learning and generative AI

• Artificial intelligence (AI) is a broad term for computer systems that do tasks we usually link with human thinking, such as recognising images or understanding language.
• Traditional machine learning is mostly about prediction: a model trained on many labelled examples answers one narrow question, such as whether an X-ray shows a tumour or whether a loan might not be repaid (MIT News).
• Generative AI is different: MIT News describes it as a machine-learning model that is trained to create new data, such as text, images, audio or code.
• Chatbots that answer questions in full sentences are built on large language models (LLMs). Image generators often use a different design called diffusion models, which refine a picture step by step.

Tokens: how a model reads text

• Language models do not read whole words the way you do. They split text into tokens, which can be whole words, parts of words or single characters (Google for Developers).
• In English, one token is roughly three quarters of a word, so about 400 tokens is around 300 words (Google for Developers).
• This is why chatbots sometimes struggle with tasks like counting letters in a word: they see chunks, not individual letters.
• Tools often have limits on how many tokens you can paste in or get back, which is why very long documents may be cut off or summarised less accurately.

Predicting the next token

• At its heart, a language model estimates how likely each possible next token is, given the text so far (Google for Developers).
• Google's example: for 'When I hear rain on my roof, I _____ in my kitchen', a model might rate 'cook soup' as the most likely ending at about 9.4 per cent and 'warm up a kettle' at about 5.2 per cent.
• The chatbot builds its answer one token at a time, each time choosing a likely continuation. This is why the same question can get slightly different answers on different tries.
• A long, fluent essay is the result of thousands of these small predictions, not of the model looking up a stored answer.

Training at huge scale

• LLMs have billions of parameters (adjustable internal numbers) and are trained on enormous amounts of text, including much of the publicly available text on the internet (MIT News).
• During training, the model learns statistical patterns in how words and sentences follow one another, which lets it produce grammar, facts, styles and even code that look right.
• A design called the transformer, introduced in 2017, made modern LLMs possible. It uses 'attention' to track how every token in a passage relates to the others (MIT News).
• Earlier methods processed text one token at a time or looked at only a few words of context. LLMs can evaluate the whole context at once, which is why today's chatbots stay on topic far better (Google for Developers).

What this means for you

• Because the model predicts patterns rather than checking facts, it can produce confident answers that are wrong. IBM notes that a generative AI does not 'know' what is true or false.
• Models can absorb biases found in their training data and repeat them, including stereotypes and false statements (MIT News).
• A chatbot is not a search engine or an encyclopaedia, even when it sounds like one. Treat it like a fast, well-read but unreliable study partner.
• The skill that matters is not just writing prompts: it is judging the output, which the next lessons cover.