Pedram Hassanzadeh

Abstract AI models have emerged as potential complements to physics‐based models, but their skill in capturing observed regional trends with important societal impacts remains unexplored. Here, we benchmark satellite‐era regional thermodynamic trends, including extremes, in an AI emulator (ACE2) and a hybrid model (NeuralGCM), against physics‐based models and ERA5. Both AI models capture regional temperature trends such as satellite‐era Arctic warming. ACE2 outperforms other models in capturing midlatitude vertical temperature trends. However, the AI models do not capture trends in heat extremes over the US Southwest. Furthermore, they do not capture drying trends in arid regions, but generally outperform physics‐based models. Our results show that a data‐driven AI emulator can perform comparably to, or better than, hybrid and physics‐based models in capturing regional thermodynamic trends. We also find that ACE2 learns much of the signal from .

In the climate system, extreme events and tipping points (transitions between climate attractors) are of primary importance for understanding the impacts of climate change and for designing effective adaptation and mitigation strategies. Recent extreme heat waves with severe societal consequences, as well as prolonged periods of very low renewable energy production in electricity systems, are striking examples. A key challenge in studying such phenomena is the lack of available data: these events are inherently rare, and realistic climate models are computationally expensive and highly complex. This data scarcity severely limits the applicability of traditional approaches, whether based on modelling, physics, or statistical analysis.In this talk, I will present new algorithms and theoretical approaches based on rare-event simulations, climate-model emulators, machine-learning methods for stochastic processes, and up to date blend of data and model use to estimate generalized extreme value (GEV) distribution. These methods are specifically designed to predict the probability that an extremely rare event will occur, to produce huge catalogues of dynamical trajectories leading to the event, and to use the best available historical and model data. The rare event simulation/emulator approach combines, on the one hand, state-of-the-art AI-based emulators that reproduce the full atmospheric dynamics of climate models, and, on the other hand, rare-event simulation techniques that reduce by several orders of magnitude the computational cost of sampling extremely rare events. In parallel the Bayesian GEV approach mix information from historical observation and CMIP model output to produce the best possible estimate of extreme event probabilities.To illustrate the performance of these tools, I will present results on midlatitude extreme heat waves and on extremes of renewable energy production, with a particular focus on their implications for the resilience of electricity systems.

Abstract Physics‐based atmosphere‐land models with prescribed sea surface temperature have notable successes but also biases in their ability to represent atmospheric variability compared to observations. Recently, AI emulators and hybrid models have emerged with the potential to overcome these biases, but still require systematic evaluation against metrics grounded in fundamental atmospheric dynamics. We evaluate the representation of four atmospheric variability benchmarking metrics in a fully data‐driven AI emulator (ACE2‐ERA5) and hybrid model (NeuralGCM). The hybrid model and emulator can capture the spectra of large‐scale tropical waves and extratropical eddy‐mean flow interactions, including critical levels. However, both struggle to capture the timescales associated with quasi‐biennial oscillation (QBO, ∼28 months) and Southern annular mode propagation (∼150 days). These dynamical metrics serve as an initial benchmarking tool to inform AI model development and understand their limitations, which may be essential for out‐of‐distribution applications (e.g., extrapolating to unseen climates).

By combining artificial intelligence and fluid physics, we discover a closed-form closure for 2D turbulence from small direct numerical simulation data. Large-eddy simulation with this closure is accurate and stable, reproducing direct numerical simulation statistics, including those of extremes. We also show that the new closure could be derived from a fourth-order truncated Taylor expansion. Prior analytical and artificial-intelligence-based work only found the second-order expansion, which led to unstable large-eddy simulation. The additional terms emerge only when interscale energy transfer is considered alongside standard reconstruction criterion in the sparse-equation discovery.

The Madden-Julian oscillation (MJO) is a planetary-scale, intraseasonal tropical rainfall phenomenon crucial for global weather and climate; however, its dynamics and predictability remain poorly understood. Here, we leverage deep learning (DL) to investigate the sources of MJO predictability, motivated by a central difference in MJO theories: which spatial scales are essential for driving the MJO? We first develop a deep convolutional neural network (DCNN) to forecast the MJO indices (RMM and ROMI). Our model predicts RMM and ROMI up to 21 and 33 days, respectively, achieving skills comparable to leading subseasonal-to-seasonal models such as NCEP. To identify the spatial scales most relevant for MJO forecasting, we conduct spectral analysis of the latent feature space and find that large-scale patterns dominate the learned signals. Additional experiments show that models using only large-scale signals as the input have the same skills as those using all the scales, supporting the large-scale view of the MJO. Meanwhile, we find that small-scale signals remain informative: surprisingly, models using only small-scale input can still produce skillful forecasts up to 1-2 weeks ahead. We show that this is achieved by reconstructing the large-scale envelope of the small-scale activities, which aligns with the multi-scale view of the MJO. Altogether, our findings support that large-scale patterns--whether directly included or reconstructed--may be the primary source of MJO predictability.

Generative models have recently emerged as powerful surrogates for physical systems, demonstrating increased accuracy, stability, and/or statistical fidelity. Most approaches rely on iteratively denoising a Gaussian, a choice that may not be the most effective for autoregressive prediction tasks in PDEs and dynamical systems such as climate. In this work, we benchmark generative models across diverse physical domains and tasks, and highlight the role of stochastic interpolants. By directly learning a stochastic process between current and future states, stochastic interpolants can leverage the proximity of successive physical distributions. This allows for generative models that can use fewer sampling steps and produce more accurate predictions than models relying on transporting Gaussian noise. Our experiments suggest that generative models need to balance deterministic accuracy, spectral consistency, and probabilistic calibration, and that stochastic interpolants can potentially fulfill these requirements by adjusting their sampling. This study establishes stochastic interpolants as a competitive baseline for physical emulation and gives insight into the abilities of different generative modeling frameworks.

Hurricanes threaten the petroleum industry in the United States and are expected to be influenced by climate change. This study presents an integrated framework for hurricane risk assessment of petroleum infrastructure under changing climatic conditions, calculating risk in terms of monetary loss. Variants of two synthetic probabilistic storms and one historical storm (Hurricane Ike) are simulated using the SWAN+ADCIRC model, representing a range of potential scenarios of impacts of a changing climate on hurricane forward speed and sea-level rise given uncertainties in climate projections. Model outputs inform an infrastructure impact and cascading economic loss analysis that incorporates various sources of uncertainty to estimate five types of losses sustained by petroleum facilities in surge events: land value loss, process-unit damage loss, cost of spill clean-up and repair of aboveground storage tanks, productivity loss, and civil fines. The proposed risk assessment framework is applied as a case study to seven refineries along the Houston Ship Channel (HSC), a densely-industrialized corridor in Texas. The results reveal that either an increase in mean sea level or a decrease in storm forward speed increases the maximum water elevations in the HSC for storms that produce maximum wind setup in Galveston Bay (FEMA 33 and FEMA 36), resulting in larger economic loss estimates. The role of refinery features such as storage capacity and average elevation of the refinery and its critical equipment in the refinery response to hurricane hazards is studied, and the probability distribution of refinery total loss and the loss risk profile in different hurricane scenarios are discussed. Loss estimates are presented, demonstrating the effects of hurricane forward speed and sea level on the losses for the refineries as well as the HSC. Such a framework can enable hurricane risk assessment and loss estimation for petroleum infrastructure to inform future policies and risk mitigation strategies. Potential policy implications for a region like the HSC are highlighted herein as an illustration.

Annular modes represent some of the largest sources of climate variability for Earth and Mars. Annular modes are zonally symmetric patterns of variability that represent the jet stream shifting north and south in time and are associated with anomalous occurrences of atmospheric eddies, clouds, and dust. Mars's Northern Annular Mode (MNAM) develops as anomalies of zonal-mean zonal wind emerging near the subtropics and migrating towards the pole. New anomalies appear in the subtropics and begin propagating with a period of ~150 sols, as diagnosed from the OpenMARS reanalysis. The propagation is induced by the interaction of the two leading empirical orthogonal functions that define the MNAM, which can be predicted using a pair of simple prognostic equations similar to what is done for Earth. The shifting jet stream coincides with bands of anomalous surface wind stress and encourages anomalous dust optical depth. The MNAM's internally forced periodicity on the surface wind stress combined with the seasonally forced dust cycle may help explain the inter-annual variability of global dust events, as suggested by a Monte Carlo estimate that correctly approximates the observed incidence of global dust events. Further quantification of the other modes of internal climate variability of Mars may be possible as the record of Mars's present-day climate surpasses a decade of measurement.

Neural PDE solvers offer a powerful tool for modeling complex dynamical systems, but often struggle with error accumulation over long time horizons and maintaining stability and physical consistency. We introduce a multiscale implicit neural emulator that enhances long-term prediction accuracy by conditioning on a hierarchy of lower-dimensional future state representations. Drawing inspiration from the stability properties of numerical implicit time-stepping methods, our approach leverages predictions several steps ahead in time at increasing compression rates for next-timestep refinements. By actively adjusting the temporal downsampling ratios, our design enables the model to capture dynamics across multiple granularities and enforce long-range temporal coherence. Experiments on turbulent fluid dynamics show that our method achieves high short-term accuracy and produces long-term stable forecasts, significantly outperforming autoregressive baselines while adding minimal computational overhead.

Predicting gray swan weather extremes, which are possible but so rare that they are absent from the training dataset, is a major concern for AI weather models and long-term climate emulators. An important open question is whether AI models can extrapolate from weaker weather events present in the training set to stronger, unseen weather extremes. To test this, we train independent versions of the AI weather model FourCastNet on the 1979-2015 ERA5 dataset with all data, or with Category 3-5 tropical cyclones (TCs) removed, either globally or only over the North Atlantic or Western Pacific basin. We then test these versions of FourCastNet on 2018-2023 Category 5 TCs (gray swans). All versions yield similar accuracy for global weather, but the one trained without Category 3-5 TCs cannot accurately forecast Category 5 TCs, indicating that these models cannot extrapolate from weaker storms. The versions trained without Category 3-5 TCs in one basin show some skill forecasting Category 5 TCs in that basin, suggesting that FourCastNet can generalize across tropical basins. This is encouraging and surprising because regional information is implicitly encoded in inputs. Given that current state-of-the-art AI weather and climate models have similar learning strategies, we expect our findings to apply to other models. Other types of weather extremes need to be similarly investigated. Our work demonstrates that novel learning strategies are needed for AI models to reliably provide early warning or estimated statistics for the rarest, most impactful TCs, and, possibly, other weather extremes.

Abstract Model instability remains a core challenge for data‐driven parameterizations, especially those developed with supervised algorithms, and rigorous methods to address it are lacking. Here, by integrating machine learning (ML) theory with climate physics, we demonstrate the importance of learning spatially non‐local dynamics using a 1D quasi‐biennial oscillation model with parameterized gravity waves (GW) as a testbed. While common offline metrics fail to identify shortcomings in learning non‐local dynamics, we show that the receptive field (RF) can identify instability a‐priori. We find that neural network‐based parameterizations, though predicting GW forcings from wind profiles with 99% accuracy, lead to unstable simulations when RFs are too small to capture non‐local dynamics. Additionally, we demonstrate that learning non‐local dynamics is crucial for the stability of a data‐driven spatiotemporal emulator of the zonal wind field. This work underscores the need to integrate ML theory with physics in designing data‐driven algorithms for climate modeling.

Artificial intelligence (AI) models have transformed weather forecasting, but their skill for gray swan extremes is unclear. Here, we analyze GraphCast, AIFS, and FuXi forecasts of the unprecedented 2024 Dubai storm, which had twice the training set's highest rainfall in that region. Remarkably, GraphCast and AIFS accurately forecast this event up to 8 days ahead. FuXi forecasts the event, but underestimates the rainfall. Fine-tuning and receptive field analyses suggest that these models' success stems from "translocation": learning from comparable/stronger dynamically similar events in other regions during training. Evidence of "extrapolation" (learning from weaker events) is not found. Even events within the global distribution's tail are poorly forecasted, which is not just due to data imbalance (generalization error) but also spectral bias (optimization error). These findings demonstrate the potential of AI models to forecast regional gray swans and the opportunity to improve them through understanding the mechanisms behind their successes/limitations.

Transfer learning (TL) is a powerful tool for enhancing the performance of neural networks (NNs) in applications such as weather and climate prediction and turbulence modeling. TL enables models to generalize to out-of-distribution data with minimal training data from the new system. In this study, we employ a 9-layer convolutional NN to predict the subgrid forcing in a two-layer ocean quasi-geostrophic system and examine which metrics best describe its performance and generalizability to unseen dynamical regimes. Fourier analysis of the NN kernels reveals that they learn low-pass, Gabor, and high-pass filters, regardless of whether the training data are isotropic or anisotropic. By analyzing the activation spectra, we identify why NNs fail to generalize without TL and how TL can overcome these limitations: the learned weights and biases from one dataset underestimate the out-of-distribution sample spectra as they pass through the network, leading to an underestimation of output spectra. By re-training only one layer with data from the target system, this underestimation is corrected, enabling the NN to produce predictions that match the target spectra. These findings are broadly applicable to data-driven parameterization of dynamical systems.

Physics-based eddy-viscosity and backscattering closures are widely used for large-eddy simulation (LES) of geophysical turbulence, but their key parameters are often chosen empirically. Here, we develop a semi-analytical framework for estimating these parameters in 2D geophysical turbulence. Specifically, we extend a Lilly-type scaling argument, previously used for 3D turbulence, to 2D geophysical turbulence and obtain closed-form estimates, up to an amplitude constant, for the coefficients of the Leith and Smagorinsky eddy-viscosity closures, a biharmonic eddy-viscosity closure, and the Jansen--Held backscattering closure with a prescribed backscattering fraction. The amplitude constant appears in the turbulent kinetic energy direct-cascade spectrum and can be diagnosed from a few direct numerical simulation (DNS) or eddy-resolving snapshots. For the $β$-free cases, the diagnosed amplitude constant is consistent with previous theoretical estimates based on closure, renormalization-group, and mode-coupling methods. The resulting semi-analytical parameters closely match the online-learned values obtained using ensemble Kalman inversion across several 2D geophysical turbulence setups. LES using these parameters reproduces key DNS statistics, including the tails of the vorticity distribution, and robustly outperforms dynamic Leith and Smagorinsky baselines.

Abstract We document the propagation of annular modes—zonally symmetric patterns of variability—in Mars's atmosphere using a reanalysis dataset. Mars's Northern Annular Mode (MNAM) sees anomalies of zonal‐mean zonal wind emerge near the subtropics and migrate poleward with a period of 150 days, similarly to Earth's Southern Annular Mode. The mechanism of propagation involves the interaction of the two leading empirical orthogonal functions that define the MNAM. Moreover, the propagation encourages alternating bands of surface wind stress to migrate polewards with a 150‐day period. In addition, a 150‐day periodicity in anomalous column dust optical depth most likely emerges in response to extrema of the MNAM. The combination of the impact of the MNAM's internally forced periodicity on the surface wind stress and the seasonal cycle may contribute to the inter‐annual variability of global dust events, as suggested by a Monte Carlo estimate that correctly approximates the observed incidence of global dust events.

The risk of extreme weather under climate change is of paramount importance, but remains one of the most difficult problems to study using conventional physics-based global climate models (GCMs). This is due to the high uncertainty in estimates of extreme weather return times owing to the computational cost of evolving these models for long enough to observe very rare events. AI models trained on historical reanalysis to emulate the dynamics of the global atmosphere have demonstrated both high forecast accuracy and greatly reduced computational cost. Some of these AI emulators can generate stable, decades-long trajectories, which, in conjunction with their affordability, have the potential to greatly reduce extreme weather uncertainties. However, it is impossible to validate if AI emulations can accurately estimate the risk of extreme weather events with return times longer than the historical record. In a first-of-its-kind experiment to assess this capability, we simulate 100,000 years of a stationary climate using PlaSim, a coarse resolution GCM. We then train a selection of stable AI emulators using only 100 years of data, and compare the emulated and true return times of extreme heat waves over Western Europe and the Pacific Northwest. We finally assess how the addition of a land moisture component to these AI emulators improves the accuracy of return time estimates.

AI weather models are becoming valuable tools for predicting the weather. While AI models’ general forecasts are known to be skillful, their forecast skill of extreme events is not fully understood. The 2021 Pacific Northwest (PNW) heatwave is a good case study for AI models because it falls outside of the distribution of heat waves in AI model training datasets.Here, we investigate the forecast performance of 8 AI models (AIFS, Gencast, NeuralGCM, Graphcast, Fuxi, Pangu, Fourcastnet, FourcastnetV2) of the 2021 PNW heatwave. Despite the event being out of the training dataset distribution, their forecast performance is comparable to that of a state-of-the-art numerical weather prediction model (IFS). Specifically, AI models and IFS can accurately forecast the heatwave for lead times less than 7 days.Two recent studies suggest the predictability barrier of the PNW heatwave may be due to an initial condition observation error. Leach et al. (2024) found that the 26th ensemble member of a 250 member IFS forecast accurately forecasts the heatwave 12 days in advance. Vonich and Hakim (2024) used backpropagation in Graphcast to find an optimal initial condition that leads to an accurate forecast 10 days in advance. Are these initial conditions robust across an ensemble of AI models? And do these initial conditions point to a unique solution?We find a large spread in forecast accuracy when running the 8 AI models with the Leach et al. (2024) and Vonich & Hakim (2024) initial conditions. Furthermore we ran 1000 member ensembles in NeuralGCM and find initial conditions that lead to an accurate long-term forecast are not unique. These results suggest that the improvement in forecast accuracty to certain initial conditions may not necessarily be due to the initial conditions being closer to ground truth but rather they are due to cancelation of model error.

Heatwaves are among the most impactful extreme weather events, posing significant risks to human health, ecosystems, and energy systems. Understanding the return times of these events and assessing how climate change alters their frequency and intensity are critical for effective adaptation strategies. However, the rarity of record-breaking heatwaves in observational datasets makes this task highly challenging. Climate models, while capable of simulating such rare events, require prohibitively long simulations to generate robust statistics for events with return times on the order of centuries.Our study addresses these challenges by leveraging a dual approach combining rare event simulation algorithms and AI-driven climate model emulators. Rare event algorithms, such as genetic algorithms, efficiently target the extreme trajectories leading to heatwaves while avoiding typical weather conditions, allowing for a more focused exploration of the event space. Although effective for long-duration events, these approaches are less suited to capturing shorter-term phenomena, necessitating novel methodologies for finer temporal scales.In parallel, we leverage the advancements of deep learning in climate science by training neural networks-based climate model emulators based on Vision Transformers. These emulators drastically reduce computational costs and generate realistic climate simulations, including heatwave dynamics. Here, we explore coupling emulators with a new rare event algorithm specifically designed to sample short and extreme heatwaves. We demonstrate the efficiency of this method by calculating return times for unprecedented heatwave events.In this work, we use data from PlaSim, a cheap-to-run climate model of intermediate complexity, which enables the verification of return periods spanning up to thousands of years. The next steps involve utilizing more state-of-the-art climate models at finer spatial resolutions and evaluating how the statistics of heatwaves may evolve under various climate change scenarios.

We discuss the emerging advances and opportunities at the intersection of machine learning (ML) and climate physics, highlighting the use of ML techniques, including supervised, unsupervised, and equation discovery, to accelerate climate knowledge discoveries and simulations. We delineate two distinct yet complementary aspects: ( a ) ML for climate physics and ( b ) ML for climate simulations. Although physics-free ML-based models, such as ML-based weather forecasting, have demonstrated success when data are abundant and stationary, the physics knowledge and interpretability of ML models become crucial in the small-data/nonstationary regime to ensure generalizability. Given the absence of observations, the long-term future climate falls into the small-data regime. Therefore, ML for climate physics holds a critical role in addressing the challenges of ML for climate simulations. We emphasize the need for collaboration among climate physics, ML theory, and numerical analysis to achieve reliable ML-based models for climate applications.

Climate science has been revolutionized by the combined effects of an exponential growth in computing power, which has enabled more sophisticated and higher-resolution simulations to be made of the climate system, and an exponential increase in observations since the first weather satellite was put in orbit. Big data and associated algorithms, coalesced under the field of machine learning (ML), offer the opportunity to study the physics of the climate system in ways, and with an amount of detail, that were previously infeasible. Additionally, ML can ask causal questions to determine whether one or more variables cause or affect one or more outcomes and improve prediction skills beyond classical limits. Furthermore, when paired with modelling experiments or robust research on model parameterizations, ML can accelerate computations, increasing accuracy and generating very large ensembles with a fraction of the computational cost of traditional systems. In this Review, we outline the accomplishments of ML in climate physics. We discuss how ML has been used to tackle long-standing problems in the reconstruction of observational data, representation of sub-grid-scale phenomena and climate (and weather) prediction. Finally, we consider the benefits and major challenges of exploiting ML in studying complex systems. Artificial intelligence techniques, specifically machine learning, are being increasingly applied to climate physics owing to the growing availability of big data and increasing computational power. This Review focuses on key results obtained with machine learning in reconstruction, sub-grid-scale parameterization, and weather or climate prediction. Advances in machine learning for climate physics have extended observational data records in time, space and observables, making them longer, more global and more complete. Innovative approaches that use machine learning to learn parameterizations from data or high-resolution simulations could contribute to hybrid models that will be able to provide more detailed, physically consistent simulations of the climate system. The use of machine learning has enabled classical predictability barriers to be broken in forecasts ranging from weather to phenomena at interannual scales such as the El Niño Southern Oscillation, leading to higher forecast skill at larger lead times using orders-of-magnitude less computing resource. Advances in machine learning for climate physics have extended observational data records in time, space and observables, making them longer, more global and more complete. Innovative approaches that use machine learning to learn parameterizations from data or high-resolution simulations could contribute to hybrid models that will be able to provide more detailed, physically consistent simulations of the climate system. The use of machine learning has enabled classical predictability barriers to be broken in forecasts ranging from weather to phenomena at interannual scales such as the El Niño Southern Oscillation, leading to higher forecast skill at larger lead times using orders-of-magnitude less computing resource.

Predicting gray swan weather extremes, which are possible but so rare that they are absent from the training dataset, is a major concern for AI weather models and long-term climate emulators. An important open question is whether AI models can extrapolate from weaker weather events present in the training set to stronger, unseen weather extremes. To test this, we train independent versions of the AI model FourCastNet on the 1979-2015 ERA5 dataset with all data, or with Category 3-5 tropical cyclones (TCs) removed, either globally or only over the North Atlantic or Western Pacific basin. We then test these versions of FourCastNet on 2018-2023 Category 5 TCs (gray swans). All versions yield similar accuracy for global weather, but the one trained without Category 3-5 TCs cannot accurately forecast Category 5 TCs, indicating that these models cannot extrapolate from weaker storms. The versions trained without Category 3-5 TCs in one basin show some skill forecasting Category 5 TCs in that basin, suggesting that FourCastNet can generalize across tropical basins. This is encouraging and surprising because regional information is implicitly encoded in inputs. Given that current state-of-the-art AI weather and climate models have similar learning strategies, we expect our findings to apply to other models. Other types of weather extremes need to be similarly investigated. Our work demonstrates that novel learning strategies are needed for AI models to reliably provide early warning or estimated statistics for the rarest, most impactful TCs, and, possibly, other weather extremes.

Different approaches to using data-driven methods for subgrid-scale closure modeling have emerged recently. Most of these approaches are data-hungry, and lack interpretability and out-of-distribution generalizability. Here, we use {online} learning to address parametric uncertainty of well-known physics-based large-eddy simulation (LES) closures: the Smagorinsky (Smag) and Leith eddy-viscosity models (1 free parameter) and the Jansen-Held (JH) backscattering model (2 free parameters). For 8 cases of 2D geophysical turbulence, optimal parameters are estimated, using ensemble Kalman inversion (EKI), such that for each case, the LES' energy spectrum matches that of direct numerical simulation (DNS). Only a small training dataset is needed to calculate the DNS spectra (i.e., the approach is {data-efficient}). We find the optimized parameter(s) of each closure to be constant across broad flow regimes that differ in dominant length scales, eddy/jet structures, and dynamics, suggesting that these closures are {generalizable}. In a-posteriori tests based on the enstrophy spectra and probability density functions (PDFs) of vorticity, LES with optimized closures outperform the baselines (LES with standard Smag, dynamic Smag or Leith), particularly at the tails of the PDFs (extreme events). In a-priori tests, the optimized JH significantly outperforms the baselines and optimized Smag and Leith in terms of interscale enstrophy and energy transfers (still, optimized Smag noticeably outperforms standard Smag). The results show the promise of combining advances in physics-based modeling (e.g., JH) and data-driven modeling (e.g., {online} learning with EKI) to develop data-efficient frameworks for accurate, interpretable, and generalizable closures.

An exponential growth in computing power, which has brought more sophisticated and higher resolution simulations of the climate system, and an exponential increase in observations since the first weather satellite was put in orbit, are revolutionizing climate science. Big data and associated algorithms, coalesced under the field of Machine Learning (ML), offer the opportunity to study the physics of the climate system in ways, and with an amount of detail, infeasible few years ago. The inference provided by ML has allowed to ask causal questions and improve prediction skills beyond classical barriers. Furthermore, when paired with modeling experiments or robust research in model parameterizations, ML is accelerating computations, increasing accuracy and allowing for generating very large ensembles at a fraction of the cost. In light of the urgency imposed by climate change and the rapidly growing role of ML, we review its broader accomplishments in climate physics. Decades long standing problems in observational data reconstruction, representation of sub-grid scale phenomena and climate (and weather) prediction are being tackled with new and justified optimism. Ultimately, this review aims at providing a perspective on the benefits and major challenges of exploiting ML in studying complex systems.

Machine learning (ML) techniques, especially neural networks (NNs), have shown promise in learning subgrid-scale parameterizations for climate models. However, a major problem with data-driven parameterizations, particularly those learned with supervised algorithms, is model instability. Current remedies are often ad-hoc and lack a theoretical foundation. Here, we combine ML theory and climate physics to address a source of instability in NN-based parameterization. We demonstrate the importance of learning spatially $\textit{non-local}$ dynamics using a 1D model of the quasi-biennial oscillation (QBO) with gravity wave (GW) parameterization as a testbed. While common offline metrics fail to identify shortcomings in learning non-local dynamics, we show that the concept of receptive field (RF) can identify instability a-priori. We find that NN-based parameterizations that seem to accurately predict GW forcings from wind profiles ($\mathbf{R^2 \approx 0.99}$) cause unstable simulations when RF is too small to capture the non-local dynamics, while NNs of the same size but large-enough RF are stable. We examine three broad classes of architectures, namely convolutional NNs, Fourier neural operators, and fully-connected NNs; the latter two have inherently large RFs. We also demonstrate that learning non-local dynamics is crucial for the stability and accuracy of a data-driven spatiotemporal emulator of the zonal wind field. Given the ubiquity of non-local dynamics in the climate system, we expect the use of effective RF, which can be computed for any NN architecture, to be important for many applications. This work highlights the necessity of integrating ML theory with physics to design and analyze data-driven algorithms for weather and climate modeling.

Abstract There is growing interest in discovering interpretable, closed‐form equations for subgrid‐scale (SGS) closures/parameterizations of complex processes in Earth systems. Here, we apply a common equation‐discovery technique with expansive libraries to learn closures from filtered direct numerical simulations of 2D turbulence and Rayleigh‐Bénard convection (RBC). Across common filters (e.g., Gaussian, box), we robustly discover closures of the same form for momentum and heat fluxes. These closures depend on nonlinear combinations of gradients of filtered variables, with constants that are independent of the fluid/flow properties and only depend on filter type/size. We show that these closures are the nonlinear gradient model (NGM), which is derivable analytically using Taylor‐series. Indeed, we suggest that with common (physics‐free) equation‐discovery algorithms, for many common systems/physics, discovered closures are consistent with the leading term of the Taylor‐series (except when cutoff filters are used). Like previous studies, we find that large‐eddy simulations with NGM closures are unstable, despite significant similarities between the true and NGM‐predicted fluxes (correlations >0.95). We identify two shortcomings as reasons for these instabilities: in 2D, NGM produces zero kinetic energy transfer between resolved and subgrid scales, lacking both diffusion and backscattering. In RBC, potential energy backscattering is poorly predicted. Moreover, we show that SGS fluxes diagnosed from data, presumed the “truth” for discovery, depend on filtering procedures and are not unique. Accordingly, to learn accurate, stable closures in future work, we propose several ideas around using physics‐informed libraries, loss functions, and metrics. These findings are relevant to closure modeling of any multi‐scale system.

Abstract Neural networks (NNs) are increasingly used for data‐driven subgrid‐scale parameterizations in weather and climate models. While NNs are powerful tools for learning complex non‐linear relationships from data, there are several challenges in using them for parameterizations. Three of these challenges are (a) data imbalance related to learning rare, often large‐amplitude, samples; (b) uncertainty quantification (UQ) of the predictions to provide an accuracy indicator; and (c) generalization to other climates, for example, those with different radiative forcings. Here, we examine the performance of methods for addressing these challenges using NN‐based emulators of the Whole Atmosphere Community Climate Model (WACCM) physics‐based gravity wave (GW) parameterizations as a test case. WACCM has complex, state‐of‐the‐art parameterizations for orography‐, convection‐, and front‐driven GWs. Convection‐ and orography‐driven GWs have significant data imbalance due to the absence of convection or orography in most grid points. We address data imbalance using resampling and/or weighted loss functions, enabling the successful emulation of parameterizations for all three sources. We demonstrate that three UQ methods (Bayesian NNs, variational auto‐encoders, and dropouts) provide ensemble spreads that correspond to accuracy during testing, offering criteria for identifying when an NN gives inaccurate predictions. Finally, we show that the accuracy of these NNs decreases for a warmer climate (4 × CO 2 ). However, their performance is significantly improved by applying transfer learning, for example, re‐training only one layer using ∼1% new data from the warmer climate. The findings of this study offer insights for developing reliable and generalizable data‐driven parameterizations for various processes, including (but not limited to) GWs.

Context. A magma ocean (MO) is thought to be a ubiquitous stage in the early evolution of rocky planets and exoplanets. During the lifetime of the MO, exchanges between the interior and exterior envelopes of the planet are very efficient. In particular, volatile elements that initially are contained in the solid part of the planet can be released and form a secondary outgassed atmosphere. Aims. We determine trends in the H–C–N–O–S composition and thickness of these secondary atmospheres for varying planetary sizes and MO extents, and the oxygen fugacity of MOs, which provides the main control for the atmospheric chemistry. Methods. We used a model with coupled chemical gas-gas and silicate melt-gas equilibria and mass conservation to predict the composition of an atmosphere at equilibrium with the MO depending on the planet size and the extent and redox state of the MO. We used a self-consistent mass–radius model for the rocky core to inform the structure of the planet, which we combined with an atmosphere model to predict the transit radius of lava worlds. Results. The resulting MOs have potential temperatures ranging from 1415 to 4229 K, and their outgassed atmospheres have total pressures from 3.3 to 768 bar. We find that MOs (especially the shallow ones) on small planets are generally more reduced, and are thus dominated by H 2 -rich atmospheres (whose outgassing is strengthened at low planetary mass), while larger planets and deeper MOs vary from CO to CO 2 –N 2 –SO 2 atmospheres, with increasing $\[f_{\mathrm{O}_2}\]$. In the former case, the low molecular mass of the atmosphere combined with the low gravity of the planets yields a large vertical extension of the atmosphere, while in the latter cases, secondary outgassed atmospheres on super-Earths are likely significantly shrunk. Both N and C are largely outgassed regardless of the conditions, while the S and H outgassing is strongly dependent on the $\[f_{\mathrm{O}_2}\]$, as well as on the planetary mass and MO extent for the latter. We further use these results to assess how much a secondary outgassed atmosphere may alter the mass–radius relations of rocky exoplanets.

We discuss the emerging advances and opportunities at the intersection of machine learning (ML) and climate physics, highlighting the use of ML techniques, including supervised, unsupervised, and equation discovery, to accelerate climate knowledge discoveries and simulations. We delineate two distinct yet complementary aspects: (1) ML for climate physics and (2) ML for climate simulations. While physics-free ML-based models, such as ML-based weather forecasting, have demonstrated success when data is abundant and stationary, the physics knowledge and interpretability of ML models become crucial in the small-data/non-stationary regime to ensure generalizability. Given the absence of observations, the long-term future climate falls into the small-data regime. Therefore, ML for climate physics holds a critical role in addressing the challenges of ML for climate simulations. We emphasize the need for collaboration among climate physics, ML theory, and numerical analysis to achieve reliable ML-based models for climate applications.

Abstract Convection‐generated gravity waves (CGWs) transport momentum and energy, and this momentum is a dominant driver of global features of Earth's atmosphere's general circulation (e.g., the quasi‐biennial oscillation, the pole‐to‐pole mesospheric circulation). As CGWs are not generally resolved by global weather and climate models, their effects on the circulation need to be parameterized. However, quality observations of GWs are spatiotemporally sparse, limiting understanding and preventing constraints on parameterizations. Convection‐permitting or ‐resolving simulations do generate CGWs, but validation is not possible as these simulations cannot reproduce the CGW‐forcing convection at correct times, locations, and intensities. Here, realistic convective diabatic heating, learned from full‐physics convection‐permitting Weather Research and Forecasting simulations, is predicted from weather radar observations using neural networks and a previously developed look‐up table. These heating rates are then used to force an idealized GW‐resolving dynamical model. Simulated CGWs forced in this way closely resembled those observed by the Atmospheric InfraRed Sounder in the upper stratosphere. CGW drag in these validated simulations extends 100s of kilometers away from the convective sources, highlighting errors in current gravity wave drag parameterizations due to the use of the ubiquitous single‐column approximation. Such validatable simulations have significant potential to be used to further basic understanding of CGWs, improve their parameterizations physically, and provide more restrictive constraints on tuning with confidence .

The Madden-Julian Oscillation (MJO) is a large-scale tropical phenomenon where fluctuations of clouds, rainfall, winds, and pressure propagate eastward around the globe every 30 to 90 days on average. The MJO has significant impacts on weather and climate both locally and globally. Despite its importance, forecasting the MJO remains challenging due to the limitations of traditional numerical and statistical methods. To address this, machine learning has emerged as a promising avenue for MJO forecasting (Martin et al. 2022, Silini et al. 2021, Delaunay and Christensen 2022). Apart from accurate forecasts emphasized in previous research, our study aims to get more physical insights: we build a predictive and interpretable convolution neural network (CNN) and unravel what tropical waves at which spatial scales are essential for MJO forecasting.Our CNN model takes tropical reanalysis maps as input and predicts the MJO index, achieving forecast skills comparable to NCEP Climate Forecast System (CFSv2). This level of skill is state-of-art in interpretable neural networks. To understand what information is crucial to our MJO forecast, we decompose the output of each convolution layer into tropical waves at different zonal scales. We find that the CNN focuses on large-scale patterns whose zonal scale is above 2500 km. In fact, even when fed exclusively with large-scale features as input, the CNN achieves MJO forecasts akin to the skill of the original model. Furthermore, the CNN chooses to reconstruct large-scale features from input containing solely small-scale features instead of relying directly on small scales for forecasting. This reconstruction further emphasizes the critical role of large-scale patterns in MJO predictions.In future research, we plan to perform a systematic analysis to evaluate the contribution of different tropical waves to MJO forecasting. We will also simplify the model architecture to facilitate better understanding. Additionally, we plan to incorporate more previous time steps as input memories to enhance forecast accuracy. This work represents a promising advance towards economic yet precise MJO forecasting.

Abstract Earth system models suffer from various structural and parametric errors in their representation of nonlinear, multi‐scale processes, leading to uncertainties in their long‐term projections. The effects of many of these errors (particularly those due to fast physics) can be quantified in short‐term simulations, for example, as differences between the predicted and observed states (analysis increments). With the increase in the availability of high‐quality observations and simulations, learning nudging from these increments to correct model errors has become an active research area. However, most studies focus on using neural networks, which while powerful, are hard to interpret, are data‐hungry, and poorly generalize out‐of‐distribution. Here, we show the capabilities of Model Error Discovery with Interpretability and Data Assimilation (MEDIDA), a general, data‐efficient framework that uses sparsity‐promoting equation‐discovery techniques to learn model errors from analysis increments. Using two‐layer quasi‐geostrophic turbulence as the test case, MEDIDA is shown to successfully discover various linear and nonlinear structural/parametric errors when full observations are available. Discovery from spatially sparse observations is found to require highly accurate interpolation schemes. While NNs have shown success as interpolators in recent studies, here, they are found inadequate due to their inability to accurately represent small scales, a phenomenon known as spectral bias. We show that a general remedy, adding a random Fourier feature layer to the NN, resolves this issue enabling MEDIDA to successfully discover model errors from sparse observations. These promising results suggest that with further development, MEDIDA could be scaled up to models of the Earth system and real observations.

Abstract There are different strategies for training neural networks (NNs) as subgrid‐scale parameterizations. Here, we use a 1D model of the quasi‐biennial oscillation (QBO) and gravity wave (GW) parameterizations as testbeds. A 12‐layer convolutional NN that predicts GW forcings for given wind profiles, when trained offline in a big ‐ data regime (100‐year), produces realistic QBOs once coupled to the 1D model. In contrast, offline training of this NN in a small ‐ data regime (18‐month) yields unrealistic QBOs. However, online re‐training of just two layers of this NN using ensemble Kalman inversion and only time‐averaged QBO statistics leads to parameterizations that yield realistic QBOs. Fourier analysis of these three NNs' kernels suggests why/how re‐training works and reveals that these NNs primarily learn low‐pass, high‐pass, and a combination of band‐pass filters, potentially related to the local and non‐local dynamics in GW propagation and dissipation. These findings/strategies generally apply to data‐driven parameterizations of other climate processes.

Neural networks (NNs) are increasingly used for data-driven subgrid-scale parameterization in weather and climate models. While NNs are powerful tools for learning complex nonlinear relationships from data, there are several challenges in using them for parameterizations. Three of these challenges are 1) data imbalance related to learning rare (often large-amplitude) samples; 2) uncertainty quantification (UQ) of the predictions to provide an accuracy indicator; and 3) generalization to other climates, e.g., those with higher radiative forcing. Here, we examine the performance of methods for addressing these challenges using NN-based emulators of the Whole Atmosphere Community Climate Model (WACCM) physics-based gravity wave (GW) parameterizations as the test case. WACCM has complex, state-of-the-art parameterizations for orography-, convection- and frontal-driven GWs. Convection- and orography-driven GWs have significant data imbalance due to the absence of convection or orography in many grid points. We address data imbalance using resampling and/or weighted loss functions, enabling the successful emulation of parameterizations for all three sources. We demonstrate that three UQ methods (Bayesian NNs, variational auto-encoders, and dropouts) provide ensemble spreads that correspond to accuracy during testing, offering criteria on when a NN gives inaccurate predictions. Finally, we show that the accuracy of these NNs decreases for a warmer climate (4XCO2). However, the generalization accuracy is significantly improved by applying transfer learning, e.g., re-training only one layer using ~1% new data from the warmer climate. The findings of this study offer insights for developing reliable and generalizable data-driven parameterizations for various processes, including (but not limited) to GWs.

Global climate models (GCMs) are the main tools for understanding and predicting climate change. However, due to limited numerical resolutions, these models suffer from major structural uncertainties; e.g., they cannot resolve critical processes such as small-scale eddies in atmospheric and oceanic turbulence. Thus, such small-scale processes have to be represented as a function of the resolved scales via closures (parametrization). The accuracy of these closures is particularly important for capturing climate extremes. Traditionally, such closures are based on heuristics and simplifying assumptions about the unresolved physics. Recently, supervised-learned closures, trained offline on high-fidelity data, have been shown to outperform the classical physics-based closures. However, this approach requires a significant amount of high-fidelity training data and can also lead to instabilities. Reinforcement learning is emerging as a potent alternative for developing such closures as it requires only low-order statistics and leads to stable closures. In Scientific Multi-Agent Reinforcement Learning (SMARL) computational elements serve a dual role of discretization points and learning agents. We leverage SMARL and fundamentals of turbulence physics to learn closures for prototypes of atmospheric and oceanic turbulence. The policy is trained using only the enstrophy spectrum, which is nearly invariant and can be estimated from a few high-fidelity samples (these few samples are far from enough for supervised/offline learning). We show that these closures lead to stable low-resolution simulations that, at a fraction of the cost, can reproduce the high-fidelity simulations' statistics, including the tails of the probability density functions. The results demonstrate the high potential of SMARL for closure modeling for GCMs, especially in the regime of scarce data and indirect observations.

Neural networks (NNs) are increasingly used for data-driven subgrid-scale parameterization in weather and climate models. While NNs are powerful tools for learning complex nonlinear relationships from data, there are several challenges in using them for parameterizations. Three of these challenges are 1) data imbalance related to learning rare (often large-amplitude) samples; 2) uncertainty quantification (UQ) of the predictions to provide an accuracy indicator; and 3) generalization to other climates, e.g., those with higher radiative forcing. Here, we examine the performance of methods for addressing these challenges using NN-based emulators of the Whole Atmosphere Community Climate Model (WACCM) physics-based gravity wave (GW) parameterizations as the test case. WACCM has complex, state-of-the-art parameterizations for orography-, convection- and frontal-driven GWs. Convection- and orography-driven GWs have significant data imbalance due to the absence of convection or orography in many grid points. We address data imbalance using resampling and/or weighted loss functions, enabling the successful emulation of parameterizations for all three sources. We demonstrate that three UQ methods (Bayesian NNs, variational auto-encoders, and dropouts) provide ensemble spreads that correspond to accuracy during testing, offering criteria on when a NN gives inaccurate predictions. Finally, we show that the accuracy of these NNs decreases for a warmer climate (4XCO2). However, the generalization accuracy is significantly improved by applying transfer learning, e.g., re-training only one layer using ~1% new data from the warmer climate. The findings of this study offer insights for developing reliable and generalizable data-driven parameterizations for various processes, including (but not limited) to GWs.

Earth system models suffer from various structural and parametric errors in their representation of nonlinear, multi-scale processes, leading to uncertainties in their long-term projections. The effects of many of these errors (particularly those due to fast physics) can be quantified in short-term simulations, e.g., as differences between the predicted and observed states (analysis increments). With the increase in the availability of high-quality observations and simulations, learning nudging from these increments to correct model errors has become an active research area. However, most studies focus on using neural networks, which while powerful, are hard to interpret, are data-hungry, and poorly generalize out-of-distribution. Here, we show the capabilities of Model Error Discovery with Interpretability and Data Assimilation (MEDIDA), a general, data-efficient framework that uses sparsity-promoting equation-discovery techniques to learn model errors from analysis increments. Using two-layer quasi-geostrophic turbulence as the test case, MEDIDA is shown to successfully discover various linear and nonlinear structural/parametric errors when full observations are available. Discovery from spatially sparse observations is found to require highly accurate interpolation schemes. While NNs have shown success as interpolators in recent studies, here, they are found inadequate due to their inability to accurately represent small scales, a phenomenon known as spectral bias. We show that a general remedy, adding a random Fourier feature layer to the NN, resolves this issue enabling MEDIDA to successfully discover model errors from sparse observations. These promising results suggest that with further development, MEDIDA could be scaled up to models of the Earth system and real observations.

There are different strategies for training neural networks (NNs) as subgrid-scale parameterizations. Here, we use a 1D model of the quasi-biennial oscillation (QBO) and gravity wave (GW) parameterizations as testbeds. A 12-layer convolutional NN that predicts GW forcings for given wind profiles, when trained offline in a big-data regime (100-years), produces realistic QBOs once coupled to the 1D model. In contrast, offline training of this NN in a small-data regime (18-months) yields unrealistic QBOs. However, online re-training of just two layers of this NN using ensemble Kalman inversion and only time-averaged QBO statistics leads to parameterizations that yield realistic QBOs. Fourier analysis of these three NNs' kernels suggests why/how re-training works and reveals that these NNs primarily learn low-pass, high-pass, and a combination of band-pass filters, consistent with the importance of both local and non-local dynamics in GW propagation/dissipation. These findings/strategies apply to data-driven parameterizations of other climate processes generally.

There is growing interest in discovering interpretable, closed-form equations for subgrid-scale (SGS) closures/parameterizations of complex processes in Earth system. Here, we apply a common equation-discovery technique with expansive libraries to learn closures from filtered direct numerical simulations of 2D forced turbulence and Rayleigh-Benard convection (RBC). Across common filters, we robustly discover closures of the same form for momentum and heat fluxes. These closures depend on nonlinear combinations of gradients of filtered variables (velocity, temperature), with constants that are independent of the fluid/flow properties and only depend on filter type/size. We show that these closures are the nonlinear gradient model (NGM), which is derivable analytically using Taylor-series expansions. In fact, we suggest that with common (physics-free) equation-discovery algorithms, regardless of the system/physics, discovered closures are always consistent with the Taylor-series. Like previous studies, we find that large-eddy simulations with NGM closures are unstable, despite significant similarities between the true and NGM-predicted fluxes (pattern correlations > 0.95). We identify two shortcomings as reasons for these instabilities: in 2D, NGM produces zero kinetic energy transfer between resolved and subgrid scales, lacking both diffusion and backscattering. In RBC, backscattering of potential energy is poorly predicted. Moreover, we show that SGS fluxes diagnosed from data, presumed the ‘truth’ for discovery, depend on filtering procedures and are not unique. Accordingly, to learn accurate, stable closures from high-fidelity data in future work, we propose several ideas around using physics-informed libraries, loss functions, and metrics. These findings are relevant beyond turbulence to closure modeling of any multi-scale system.

There is growing interest in discovering interpretable, closed-form equations for subgrid-scale (SGS) closures/parameterizations of complex processes in Earth systems. Here, we apply a common equation-discovery technique with expansive libraries to learn closures from filtered direct numerical simulations of 2D turbulence and Rayleigh-Bénard convection (RBC). Across common filters (e.g., Gaussian, box), we robustly discover closures of the same form for momentum and heat fluxes. These closures depend on nonlinear combinations of gradients of filtered variables, with constants that are independent of the fluid/flow properties and only depend on filter type/size. We show that these closures are the nonlinear gradient model (NGM), which is derivable analytically using Taylor-series. Indeed, we suggest that with common (physics-free) equation-discovery algorithms, for many common systems/physics, discovered closures are consistent with the leading term of the Taylor-series (except when cutoff filters are used). Like previous studies, we find that large-eddy simulations with NGM closures are unstable, despite significant similarities between the true and NGM-predicted fluxes (correlations $> 0.95$). We identify two shortcomings as reasons for these instabilities: in 2D, NGM produces zero kinetic energy transfer between resolved and subgrid scales, lacking both diffusion and backscattering. In RBC, potential energy backscattering is poorly predicted. Moreover, we show that SGS fluxes diagnosed from data, presumed the ''truth'' for discovery, depend on filtering procedures and are not unique. Accordingly, to learn accurate, stable closures in future work, we propose several ideas around using physics-informed libraries, loss functions, and metrics. These findings are relevant to closure modeling of any multi-scale system.