AI PM Glossary
The Terms That Matter When You're Building AI Products in Production
Plain-English definitions for PMs. No PhD required.
RAG
Retrieval-Augmented Generation. A technique where an LLM retrieves relevant documents before generating a response, grounding outputs in real data rather than relying solely on training knowledge.
Fine-tuning
Training a pre-trained model on domain-specific data to improve performance on targeted tasks. Useful when RAG isn't sufficient and you need the model to internalize domain knowledge.
Embedding
A numerical vector representation of text, images, or other data. Embeddings capture semantic meaning — similar concepts have similar vectors — enabling search, clustering, and retrieval.
Vector Database
A database optimized for storing and searching embedding vectors. Used in RAG pipelines to find semantically similar content quickly. Examples: Pinecone, Weaviate, Chroma, Qdrant.
LLM
Large Language Model. A neural network trained on massive text datasets capable of generating, summarizing, translating, and reasoning about text. Examples: GPT-4, Claude, Gemini.
Foundation Model
A large model trained on broad data that can be adapted for many tasks. Foundation models are the base layer — fine-tuning, RAG, and prompting are all ways to specialize them.
Inference
Running a trained model to generate predictions or outputs. Inference cost — latency, compute, and dollars per call — is the operational challenge most teams underestimate.
Hallucination
When an LLM generates confident-sounding but factually incorrect output. A critical risk in regulated industries where accuracy is non-negotiable.
Prompt Engineering
The practice of crafting inputs to guide LLM behavior. Chain-of-thought, few-shot examples, and system prompts are all prompt engineering techniques.
Chain of Thought
A prompting technique where you instruct the model to reason step-by-step before giving a final answer. Significantly improves accuracy on complex reasoning tasks.
RLHF
Reinforcement Learning from Human Feedback. The training technique used to align LLMs with human preferences. Underlies the 'helpful, harmless, honest' behavior in most modern models.
SaMD
Software as a Medical Device. FDA's classification for software that meets the definition of a medical device. AI diagnostic tools, clinical decision support, and many health AI products fall here.
FDA 510(k)
The premarket notification pathway for medical devices that are substantially equivalent to an existing device. Many AI medical devices use this pathway.
HIPAA
Health Insurance Portability and Accountability Act. US law governing the privacy and security of protected health information. Any AI product handling patient data must comply.
MLOps
Machine Learning Operations. The practice of deploying, monitoring, and maintaining ML models in production. Covers CI/CD for models, drift detection, retraining pipelines, and observability.
Feature Store
A centralized repository for storing and serving ML features. Ensures consistency between training and serving environments and enables feature reuse across models.
Model Drift
The degradation of model performance over time as real-world data distribution shifts away from training data. Requires monitoring and periodic retraining.
Shadow Mode
Deploying a new model in parallel with an existing system, routing real traffic to both but only acting on the old system's outputs. Used to validate new models safely before cutover.
Canary Deployment
Gradually routing a small percentage of traffic to a new model version before full rollout. Reduces risk of widespread failures from model changes.
POC
Proof of Concept. A limited implementation to validate that an AI approach can work for a specific use case. Most POCs never reach production — usually due to data, integration, or organizational issues.
MVP for AI
Minimum Viable Product applied to AI. Different from software MVP — the goal is to find the minimum scope where the model's performance is good enough to deliver real value.
Build vs Buy
The decision framework for whether to build a custom AI solution or purchase/license an existing one. Depends on differentiation, data ownership, talent, and timeline.
AI Readiness
An organization's capability to successfully adopt and deploy AI — covering data maturity, technical infrastructure, talent, governance, and use case clarity.
Latency
The time between sending a request to an AI model and receiving a response. Critical for real-time applications — clinical decision support, for example, can't tolerate multi-second delays.
Token
The basic unit of text that LLMs process. Roughly 0.75 words per token. Token limits determine how much context a model can handle; token costs determine inference pricing.
Context Window
The maximum amount of text an LLM can process in a single call. Larger context windows enable more complex reasoning but increase cost and latency.
Agent
An AI system that can plan, use tools, and take multi-step actions to complete a goal. Agents combine LLMs with memory, tool use, and feedback loops.
Multimodal
AI models that process multiple types of input — text, images, audio, video — in a single model. Enables use cases like visual question answering and medical image analysis.
Synthetic Data
Artificially generated data used to train or test AI models when real data is scarce, sensitive, or expensive to label. Common in Life Sciences & Life Sciences & Healthcare AI where patient data access is restricted.
A/B Testing for ML
Running controlled experiments to compare two model versions or prompts. Different from software A/B testing — requires careful handling of feedback loops and distribution shifts.