Rayyan Ansari

← Notes

A Collection of Various AI Terms

Term Definition
Multimodal LLM A large language model (LLM) that takes in more than text, such as images, audio or video, and sometimes produces them too. Usually written MLLM, or LMM (large multimodal model). A VLM (vision-language model) is the text-and-image subset. Not to be confused with MLM, which usually means masked language model, like BERT.
Token A chunk of text, roughly ¾ of an English word. Varies by provider and language.
Context All the input sent in one call.
Context window The most tokens a model can handle in one call, counting the input plus its reply.
System prompt Hidden instructions the app sends before your message to shape how the model behaves.
Prompt caching The provider saves its processed version of the start of your input. If the next call starts with the same text, that part is cheaper and faster. On Anthropic: about 90% cheaper, cached for 5 minutes by default (reset on each use), with a 1-hour option.
Parameters/weights The billions of numbers the model adjusts during training. Its knowledge lives there.
Hyperparameters Choices people make before training: model size, number of layers, learning speed.
Training Adjusting the parameters by showing the model enormous amounts of data.
Fine-tuning Extra training on a smaller, focused dataset to specialize a model.
Synthetic data Training data written by an AI model instead of collected from people.
RLHF Reinforcement learning from human feedback. Humans score the model’s answers to prompts, usually by picking the better of two, and the model is trained toward the better ones.
RLAIF Reinforcement learning from AI feedback. Same as RLHF, but an AI scores the answers, usually by checking them against written principles. Anthropic’s Constitutional AI is the best-known example.
Inference Running a trained model to get an answer. The parameters stay fixed. Every message you send is inference.
Temperature A setting for how random the output is. Low is predictable, high is creative.
Embedding model An AI model—separate from the (multimodal) LLM—that turns text into lists of numbers, where similar meanings land close together.
RAG Retrieval-augmented generation. Searching your documents for relevant pieces, usually with an embedding model, and pasting them into the context before the model answers.
Hallucination When a model states something false with full confidence. Caused by training that rewards plausible guesses over “I don’t know,” and by facts that were rare or missing in its training data.
Agent A model running in a loop, using tools to take actions until a task is done.

Is it even an “LLM”?

Not everyone agrees the name fits. One side argues that “language model” implies text only, so calling a model that sees images, hears audio and watches video an “LLM” undersells it. They prefer LMM (large multimodal model) or foundation model, the term Stanford pushed in 2021.