Comparison
Machine Learning vs Pretraining
Machine Learning and Pretraining are both common AI/LLM terms but cover different ideas. Here is a quick side-by-side.
When you would reach for Machine Learning
Machine Learning comes up when the question is fundamentally about foundations.
A spam classifier learning what spam looks like from labeled examples (supervised).
When you would reach for Pretraining
Pretraining comes up when the question is fundamentally about training.
GPT-3 pretrained on ~300B tokens.
Frequently asked
What is the difference between Machine Learning and Pretraining?
Machine Learning: Machine learning is the branch of AI in which models learn patterns from data instead of being explicitly programmed. The training process adjusts model parameters to reduce error on examples. Pretraining: Pretraining is the initial training phase where an LLM learns to predict the next token on trillions of tokens of general text. It produces a base model that can be adapted later.
When should I use Machine Learning vs Pretraining?
Machine Learning is the right concept when you are focused on foundations. Pretraining applies when you are focused on training.
Are Machine Learning and Pretraining the same thing?
No. Machine Learning is foundations; Pretraining is training. They are related but address different parts of the AI stack.