Active Learning

A training strategy in which the model asks for labels on the most informative examples instead of labeling everything at once.

Which scans should a curator label next?

Suppose a curator can label six new scans this afternoon. The model already distinguishes most clear etchings from photographs.

  1. Uncertain cases

    Choose two scans close to the current decision boundary. Their labels may resolve a distinction the model has trouble making.

  2. Different cases

    Choose two examples from underrepresented materials. Six nearly identical scans of the same plate would use much of the budget on one question.

  3. A sample for checking

    Reserve two randomly drawn scans to spot confident mistakes. If using these as an audit, keep their labels out of this training round.

Did requesting labels pay off?

The example allocation is illustrative. Compare the updated model with one trained on the same number of randomly chosen labels, using a separate held-out collection. Measure curator time as well as labels acquired, because a difficult attribution may take far longer than a routine one.

How It Usually Works

Common strategies ask for labels on the examples a model is least certain about, the cases that seem most diverse, or the items likely to reduce overall error the most. In practice, active learning often works alongside transfer learning or synthetic data, since teams often mix several efficiency techniques rather than relying on only one.

Follow the idea

Weak supervision uses noisy signals to propose labels.

Supervised learning can use the curator’s confirmed labels for training.

Related concepts (8)
Related Yenra articles (11)

Source · Burr Settles’ survey of active learning

Return to Active Learning in the A–Z glossary