Test Set
A held-out dataset used to measure how well a model generalizes after training.
When is a set of photographs still an independent test?
A museum has scans of 120 separate objects. Before development, it reserves 20 objects for a final evaluation. All views of a reserved object stay together to prevent near-duplicates from appearing in training.
- Before training
Define the population, split unit and outcome. Store the test labels separately from the material used to fit models and choose settings.
- After selection
Freeze the model and preprocessing. Run the final pipeline once on the reserved objects and report both the number correct and the denominator.
- After looking
If a failure leads the team to change the model, the observations still help development. A fresh independent set is needed for a new independent performance claim.
What does the result generalize to?
A test set from one camera and one collection measures performance under those conditions. It offers little direct evidence about another museum’s lighting or object mix. Record the collection method and report weakly represented groups.

Why can a larger score be misleading?
Selecting the version that did best on the final test uses those labels indirectly, even when they were never fed to training. The selection itself gives the winning score a favorable bias.
Follow the idea
Model Evaluation · Set the test result in an evaluation design.
Cross-Validation · See how held-out folds are used during development.
Examples in the AI guides
More chapter examples (13)
Source · Scikit-learn on training, validation and held-out testing
Return to Test Set in the A–Z glossary