AI weather models learn relationships in past atmospheric data and use them to forecast future states. Some now operate alongside physics-based models at major forecasting centers. The useful question is which model version performs well for your variable, location and lead time, and how that evidence connects to a real decision.
Separate the model from the whole forecasting system
Physics-based numerical weather prediction advances atmospheric equations from an estimated initial state. A learned model instead uses a trained mapping to predict later states; hybrid systems combine learned components with physical modeling. Observations, quality control, data assimilation and human interpretation still contribute to the larger forecast process.
The AIFS system paper describes training on ECMWF reanalysis and operational analyses. Reanalysis combines observations and a modeling system to estimate past atmospheric conditions consistently. That training record is a major input to the learned model, so the model's history is part of its evidence.
A chatbot that summarizes a forecast is another kind of AI use. Summarization can help explain an issued product, but it does not by itself generate an independently validated weather forecast. Check the original location, times, units and warning language after any automated summary.
Know what is operational and what is a research result
Status checked September 10, 2026: ECMWF's AIFS dataset documentation lists AIFS Single v2 and AIFS ENS v2, upgraded on May 12, 2026. The single model first became operational in February 2025 and the ensemble in July 2025. Consult that page for current versions, variables and access terms.
A single forecast provides one trajectory. An ensemble provides multiple plausible evolutions to help characterize uncertainty. ECMWF's ensemble launch account explains that AIFS operates alongside the physics-based IFS. This offers forecasters complementary guidance; agreement between systems and disagreement between members both deserve interpretation.
A research benchmark, a routinely produced forecast and an official public warning are different products. Before relying on a chart, identify the provider, version, initialization time, intended use and operational or experimental status.
Read an accuracy claim with its denominator
The GraphCast research paper reports better performance on more than 90% of 1,380 evaluated targets in its comparison with an operational deterministic system. Those targets are specific combinations of variables, levels and forecast lead times in that evaluation. The percentage is not the probability that every local forecast is correct, nor a blanket statement about every severe-weather warning.
| Question | Why it changes the conclusion |
|---|---|
| Which variable and level? | Upper-air height skill does not directly establish street-level rainfall skill. |
| Which lead time and period? | Tomorrow, day ten and a seasonal outlook are different tasks. |
| Which place and scale? | A global average can hide local or regional weaknesses. |
| Which baseline and version? | Compare against an appropriate current alternative, not an unspecified old system. |
| Which metric and reference? | Magnitude error, event detection and probability quality measure different things. |
| Which unseen cases? | Evaluation should address data separation, rare events and conditions unlike the training record. |
Read the methods, not just the headline chart. A lower average error may coexist with missed high-impact extremes. Faster model inference also differs from total delivery time, which includes obtaining observations, preparing input data and distributing results.
Compare a small example correctly
Keep forecasts issued at the same lead time for the same valid period. Compare them with observations that measure the promised quantity. An instantaneous temperature cannot be substituted for a daily maximum without changing the question.
Assess probability forecasts as probabilities
For an event probability, record the event definition and threshold before collecting results. Across many comparable cases assigned a probability near 30%, an adequately calibrated system should see the event occur roughly 30% of the time. A single event happening after a 30% forecast does not establish that the probability was wrong.
Check both calibration and usefulness: a forecast should describe uncertainty honestly and still distinguish situations with different risks. Ensemble spread is information about the modeled possibilities; its relationship to observed error must be evaluated. A tight cluster of members alone cannot prove confidence is justified.
For a consequential decision, combine the forecast with the costs of missed events and false alarms, and with the applicable professional procedures. Emergency actions should follow official warnings and local instructions rather than an informal model contest.
Make a comparison you can revisit
- Choose one location, variable, valid period and lead time.
- Record each forecast before the outcome is known, including version and source.
- Retain the observation source and quality limitations.
- Calculate the agreed metric over a meaningful sample and inspect unusual cases separately.
- Repeat the review after a model upgrade; preserve results by version.
Download the forecast-comparison record (plain text). It includes the worked arithmetic, blank records and questions for probability forecasts. Use it to organize evidence, not to extrapolate a few successes into a universal ranking.