The bias-variance trade-off describes a model's error: bias comes from overly simple assumptions (underfitting), variance from excessive sensitivity to the training data (overfitting). Increasing complexity lowers bias and increases variance. The goal is to find the balance that minimizes generalization error.