A Collection of Various AI Terms
| Term | Definition |
|---|---|
| Multimodal LLM | A large language model (LLM) that takes in more than text, such as images, audio or video, and sometimes produces them too. Usually written MLLM, or LMM (large multimodal model). A VLM (vision-language model) is the text-and-image subset. Not to be confused with MLM, which usually means masked language model, like BERT. |
| Token | A chunk of text, roughly ¾ of an English word. Varies by provider and language. |
| Context | All the input sent in one call. |
| Context window | The most tokens a model can handle in one call, counting the input plus its reply. |
| System prompt | Hidden instructions the app sends before your message to shape how the model behaves. |
| Prompt caching | The provider saves its processed version of the start of your input. If the next call starts with the same text, that part is cheaper and faster. On Anthropic: about 90% cheaper, cached for 5 minutes by default (reset on each use), with a 1-hour option. |
| Parameters/weights | The billions of numbers the model adjusts during training. Its knowledge lives there. |
| Hyperparameters | Choices people make before training: model size, number of layers, learning speed. |
| Training | Adjusting the parameters by showing the model enormous amounts of data. |
| Fine-tuning | Extra training on a smaller, focused dataset to specialize a model. |
| Synthetic data | Training data written by an AI model instead of collected from people. |
| RLHF | Reinforcement learning from human feedback. Humans score the model’s answers to prompts, usually by picking the better of two, and the model is trained toward the better ones. |
| RLAIF | Reinforcement learning from AI feedback. Same as RLHF, but an AI scores the answers, usually by checking them against written principles. Anthropic’s Constitutional AI is the best-known example. |
| Inference | Running a trained model to get an answer. The parameters stay fixed. Every message you send is inference. |
| Temperature | A setting for how random the output is. Low is predictable, high is creative. |
| Embedding model | An AI model—separate from the (multimodal) LLM—that turns text into lists of numbers, where similar meanings land close together. |
| RAG | Retrieval-augmented generation. Searching your documents for relevant pieces, usually with an embedding model, and pasting them into the context before the model answers. |
| Hallucination | When a model states something false with full confidence. Caused by training that rewards plausible guesses over “I don’t know,” and by facts that were rare or missing in its training data. |
| Agent | A model running in a loop, using tools to take actions until a task is done. |
Is it even an “LLM”?
Not everyone agrees the name fits. One side argues that “language model” implies text only, so calling a model that sees images, hears audio and watches video an “LLM” undersells it. They prefer LMM (large multimodal model) or foundation model, the term Stanford pushed in 2021.