Language Models Predict Words, Not Thoughts

Large language models predict the next word

Ask a colleague what a large language model does and the answer usually drifts toward intelligence, memory or magic. A simpler description works better for most workplace conversations: the software reads a stretch of text and guesses which word is likely to come next. Then it appends that word, reads the updated text and guesses again. Every paragraph it produces is a long chain of small guesses made one after another, at a speed that feels like one fluent response.

That mechanism explains most of the surprises people meet. The model has no private notebook of facts it can check, so it can state something wrong with complete polish. It has no plan it commits to before writing, so a sentence can start in one direction and land somewhere else. It does not remember your earlier meeting unless that conversation is passed along with your request. Everything it says grows out of the words already produced and the words you sent.

Framed this way, the tool becomes less intimidating and easier to use well. You are steering a very capable word predictor with clear instructions and enough context, rather than talking to a colleague with hidden intentions. That single shift in expectation changes how people write their requests and how carefully they read the replies.

Training data teaches patterns instead of rules

The learning happens long before you type anything. Engineers feed the system enormous collections of text, from books and articles to documentation and forum threads, and repeatedly ask it to guess a hidden word in a sentence. When the guess is wrong, the internal settings nudge slightly toward the answer that would have been right. Multiply that adjustment across trillions of attempts and the model absorbs the rhythms of language: grammar, tone, common facts, familiar argument shapes and the way explanations usually unfold.

Nobody hand-writes a rule stating that apostrophes go before an s. The pattern emerges instead. This is why models feel broadly knowledgeable yet oddly specific in their gaps. Whatever was frequent and well expressed in the training material tends to come back clearly, while whatever was rare, recent or private tends to come back weakly or not at all. A policy that only existed in an internal document last month may simply be invisible to the model.

The same lesson applies to skills. Writing style, reasoning habits and coding conventions show up when they were well represented in the material. Requests for a clear structure with examples help for that reason: they nudge the model toward the regions where writing looked like that.

Prompts shape answers more than people expect

Newcomers often treat the prompt as a question and the answer as a fact sheet. A better mental model treats the prompt as the entire working brief. The model receives your words, the instructions set behind the scenes and any documents attached to the request. From that bundle it produces the next word. If a detail matters and it is missing, the model fills the gap with a plausible guess, because plausibility is exactly the skill it was trained to deliver.

Small additions make large differences:

• Name the audience and the desired length.
• Ask for three alternatives instead of one polite average.
• Give a short example of the format you want.
• Attach the document the answer should follow.

Requests also get safer with a little skepticism built in. Asking the model to flag anything it is unsure about does not create real doubt, but it does surface claims that were weakly supported. Asking for the steps behind an answer makes it easier to spot a jump. Asking it to work only from the attached document narrows the space it draws from. None of these habits replace checking, yet each one turns a vague answer into a reviewable draft.

Tokens, not words, drive every calculation

Behind the scenes the model never sees letters or words. It sees tokens, meaning chunks of text that may be a whole common word, a short fragment or a piece of punctuation. Short familiar words are often a single token, while long technical terms can split into several. Numbers and unusual spellings also tend to split more finely. This is why a request costs what it costs and why reply length matters.

Tokens matter to anyone budgeting for this kind of work. Providers report the number of tokens processed and produced, and those counts decide the bill. Longer context, such as a large document sent with every question, grows the input side, while verbose answers grow the output side. Cached information can reduce repeated work when the same opening material is requested again, and the savings appear as separate read and write figures rather than one blended rate. Asking for a short answer, sending only the relevant document and reusing stable instructions are ordinary ways to keep spending predictable.

Confident-sounding answers still contain small errors

Fluency is the trickiest part of the technology for newcomers. A sentence produced by a language model carries no mark showing whether it came from solid material or from an average of related patterns. Grammar is nearly perfect, tone is even, and invented details are dressed in the same style as accurate ones. The result reads like a careful person even when it is guessing.

The practical response is a modest routine. Keep humans accountable for facts that carry consequences, such as prices, dates, legal commitments or customer promises. Ask the model to stay close to supplied sources and to say plainly when it cannot find an answer. Break large tasks into steps so one wrong assumption does not travel through an entire document. Treat the first draft as raw material rather than a finished product, the same way a team treats an early sketch from a new contractor.

Choosing a model depends on the task

Models differ in speed, cost, depth and the kind of work they handle best. A quick classification or a short summary rarely needs the largest system available, while a dense synthesis across many documents may earn the extra expense. Teams that test a few options against their own real tasks usually discover this faster than reading comparisons, and the best choice often varies by task rather than settling on one permanent favorite.

This is also where practical plumbing matters. When we built BestRelay, the API relay that publishes this article, the goal was boring reliability: each request is forwarded to the official model named in it with no substitution, across providers such as Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen and Grok. The service works with Claude Code, Codex, Gemini CLI and any OpenAI SDK, so a team keeps the interface it already knows. A summary might run on Gemini 2.5 Flash for speed while a dense analysis earns the cost of Claude Opus 5.5. Billing is pay-as-you-go from the token counts the provider reports, with cache reads and writes priced separately, and members earn a referral commission when people they invite top up. For newcomers the point is simple: when the plumbing is predictable, attention goes to the prompt and the review instead of to plumbing surprises.

Create a BestRelay account to try one yourself.