Celia Reina

We introduce Nonlinear GENERIC Informed Neural Networks (N-GINNs), a deep learning framework for discovering evolution equations of systems governed by the nonlinear GENERIC formalism (General Equation for Non-Equilibrium Reversible-Irreversible Coupling). Such systems exhibit coupled conservative and dissipative dynamics, and can be described via the superposition of a Hamiltonian flow and a generalized gradient flow. In contrast to existing approaches, our formulation incorporates generalized gradient flows via convex dissipation potentials, enabling the identification of a broader class of thermodynamically consistent dynamics, including systems with non-quadratic dissipation potentials. Thermodynamic structure is strongly enforced by construction through suitable reparameterizations of both the bivector operator and the dissipation potential, ensuring exact compliance with the first and second laws of thermodynamics. We validate the proposed approach on three representative examples: a harmonic oscillator coupled to a heat bath, an idealized chemical motor, and a one-dimensional viscoplastic model of Perzyna type. These results demonstrate the method's ability to accurately infer thermodynamically consistent models from data for systems incorporating both conservative and nonlinear dissipative dynamics.

We introduce Nonlinear GENERIC Informed Neural Networks (N-GINNs), a deep learning framework for discovering evolution equations of systems governed by the nonlinear GENERIC formalism (General Equation for Non-Equilibrium Reversible-Irreversible Coupling). Such systems exhibit coupled conservative and dissipative dynamics, and can be described via the superposition of a Hamiltonian flow and a generalized gradient flow. In contrast to existing approaches, our formulation incorporates generalized gradient flows via convex dissipation potentials, enabling the identification of a broader class of thermodynamically consistent dynamics, including systems with non-quadratic dissipation potentials. Thermodynamic structure is strongly enforced by construction through suitable reparameterizations of both the bivector operator and the dissipation potential, ensuring exact compliance with the first and second laws of thermodynamics. We validate the proposed approach on three representative examples: a harmonic oscillator coupled to a heat bath, an idealized chemical motor, and a one-dimensional viscoplastic model of Perzyna type. These results demonstrate the method's ability to accurately infer thermodynamically consistent models from data for systems incorporating both conservative and nonlinear dissipative dynamics.

Abstract Given a particle system obeying overdamped Langevin dynamics, we demonstrate that it is always possible to construct a thermodynamically consistent macroscopic model which obeys a gradient flow with respect to its non-equilibrium free energy. To do so, we significantly extend the recent Stochastic Thermodynamics with Internal Variables (STIV) framework, a method for producing macroscopic thermodynamic models far-from-equilibrium from the underlying mesoscopic dynamics and an approximate probability density of states parameterized with so-called internal variables. Though originally explored for Gaussian probability distributions, we here allow for an arbitrary choice of the approximate probability density while retaining a gradient flow dynamics. This greatly extends its range of applicability and automatically ensures consistency with the second law of thermodynamics, without the need for secondary verification. We demonstrate numerical convergence, in the limit of increasing internal variables, to the true probability density of states for both a multi-modal relaxation problem, a protein diffusing on a strand of DNA, and for an externally driven particle in a periodic landscape. Finally, we provide a reformulation of STIV with the quasi-equilibrium approximations in terms of the averages of observables of the mesostate, and show that these, too, obey a gradient flow.

The design of structural&functional materials for specialized applications is being fueled by rapid advancements in materials synthesis, characterization, manufacturing, with sophisticated computational materials modeling frameworks that span a wide spectrum of length&time scales in the mesoscale between atomistic&continuum approaches. This is leading towards a systems-based design methodology that will replace traditional empirical approaches, embracing the principles of the Materials Genome Initiative. However, several gaps remain in this framework as it relates to advanced structural materials:(1) limited availability&access to high-fidelity experimental&computational datasets, (2) lack of co-design of experiments&simulation aimed at computational model validation,(3) lack of on-demand access to verified and validated codes for simulation and for experimental analyses,&(4) limited opportunities for workforce training and educational outreach. These shortcomings stifle major innovations in structural materials design. This paper describes plans for a community-driven research initiative that addresses current gaps based on best-practice recommendations of leaders in mesoscale modeling, experimentation&cyberinfrastructure obtained at an NSF-sponsored workshop dedicated to this topic. The proposal is to create a hub for Mesoscale Experimentation and Simulation co-Operation (hMESO)-that will (I) provide curation and sharing of models, data,&codes, (II) foster co-design of experiments for model validation with systematic uncertainty quantification,&(III) provide a platform for education&workforce development. It will engage experimental&computational experts in mesoscale mechanics and plasticity, along with mathematicians and computer scientists with expertise in algorithms, data science, machine learning,&large-scale cyberinfrastructure initiatives.

Over the past several decades, phase field modeling has been established as a standard simulation technique for mesoscopic science, allowing for seamless boundary tracking of moving interfaces and relatively easy coupling to other physical phenomena. However, despite its widespread success, phase field modeling remains largely driven by phenomenological justifications except in a handful of instances. In this work, we leverage a recently developed statistical mechanics framework for non-equilibrium phenomena, called Stochastic Thermodynamics with Internal Variables (STIV), to provide the first derivation of a non-conservative phase field model for tracking front propagation in a one dimensional elastic medium without appeal to phenomenology or fitting to experiments or simulation data. In the resulting model, the variables obey a gradient flow with respect to a non-equilibrium free energy, although notably, the dynamics of the strain and phase variables are coupled, and while the free energy functional is non-local in the phase field variable , such non-locality deviates from the traditional form. Moreover, in the systems analyzed here, the model accurately captures stress induced nucleation of transition fronts without the need to incorporate additional physics. We find that the STIV phase field model compares favorably to Langevin simulations of the microscopic system and we provide two numerical implementations enabling one to simulate arbitrary interatomic potentials.

Given a particle system obeying overdamped Langevin dynamics, we demonstrate that it is always possible to construct a thermodynamically consistent macroscopic model which obeys a gradient flow with respect to its non-equilibrium free energy. To do so, we significantly extend the recent Stochastic Thermodynamics with Internal Variables (STIV) framework, a method for producing macroscopic thermodynamic models far-from-equilibrium from the underlying mesoscopic dynamics and an approximate probability density of states parameterized with so-called internal variables. Though originally explored for Gaussian probability distributions, we here allow for an arbitrary choice of the approximate probability density while retaining a gradient flow dynamics. This greatly extends its range of applicability and automatically ensures consistency with the second law of thermodynamics, without the need for secondary verification. We demonstrate numerical convergence, in the limit of increasing internal variables, to the true probability density of states for both a multi-modal relaxation problem, a protein diffusing on a strand of DNA, and for an externally driven particle in a periodic landscape. Finally, we provide a reformulation of STIV with the quasi-equilibrium approximations in terms of the averages of observables of the mesostate, and show that these, too, obey a gradient flow.

Thermodynamics with internal variables is a common approach in continuum mechanics to model inelastic (i.e., non-equilibrium) material behavior. It consists of enlarging the space of the state variables by introducing internal variables to eliminate the memory effects that would otherwise arise in the constitutive response when driving the system away from equilibrium. While this approach is computationally and theoretically very attractive, it currently lacks a well-established statistical mechanics foundation. As a result, internal variables are typically chosen phenomenologically and lack a direct link to the underlying atomistic or particle description. This hinders the predictability of the ensuing continuum models as well as the inverse problem of material design. In this work, we propose a machine learning approach that directly tackles these underlying issues, by learning internal variables and the evolution equations of the system, consistently with the principles of statistical mechanics and thermodynamics. The proposed approach leverages the following machine learning techniques (i) the information bottleneck (IB) method to ensure that the learned internal variables are functions of the microstates and are capable of capturing the salient feature of the microscopic distribution; (ii) conditional normalizing flows to represent arbitrary probability distributions of the microscopic states as functions of the state variables (these will be distinct from the Boltzmann distribution away from equilibrium); and (iii) Variational Onsager Neural Networks (VONNs) to guarantee thermodynamic consistency of the learned evolution equations and that the state variables are sufficient to predict the future state of the system in the absence of memory effects. The resulting framework, called IB-VONNs, is here tested on two problems on colloidal systems, governed at the microscale by overdamped Langevin dynamics. The first one is a prototypical model for a colloidal particle in an optical trap, which can be solved analytically thanks to its simplicity, and it is thus ideal to verify the framework. The second problem is a one-dimensional phase-transforming system, whose macroscopic description still lacks a statistical mechanics foundation under general conditions. The results in both cases indicate that the proposed machine learning strategy can indeed bridge statistical mechanics and thermodynamics with internal variables away from equilibrium.

The data-driven discovery of long-time macroscopic dynamics and thermodynamics of dissipative systems with particle fidelity is hampered by significant obstacles. These include the strong time-scale limitations inherent to particle simulations, the non-uniqueness of the thermodynamic potentials and operators from given macroscopic dynamics, and the need for efficient uncertainty quantification. This paper introduces Statistical-Physics Informed Epistemic Diffusion Models (SPIEDiff), a machine learning framework designed to overcome these limitations in the context of purely dissipative systems by leveraging statistical physics, conditional diffusion models, and epinets. We evaluate the proposed framework on stochastic Arrhenius particle processes and demonstrate that SPIEDiff can accurately uncover both thermodynamics and kinetics, while enabling reliable long-time macroscopic predictions using only short-time particle simulation data. SPIEDiff can deliver accurate predictions with quantified uncertainty in minutes, drastically reducing the computational demand compared to direct particle simulations, which would take days or years in the examples considered. Overall, SPIEDiff offers a robust and trustworthy pathway for the data-driven discovery of thermodynamic models.

The design of structural & functional materials for specialized applications is being fueled by rapid advancements in materials synthesis, characterization, manufacturing, with sophisticated computational materials modeling frameworks that span a wide spectrum of length & time scales in the mesoscale between atomistic & continuum approaches. This is leading towards a systems-based design methodology that will replace traditional empirical approaches, embracing the principles of the Materials Genome Initiative. However, several gaps remain in this framework as it relates to advanced structural materials:(1) limited availability & access to high-fidelity experimental & computational datasets, (2) lack of co-design of experiments & simulation aimed at computational model validation,(3) lack of on-demand access to verified and validated codes for simulation and for experimental analyses, & (4) limited opportunities for workforce training and educational outreach. These shortcomings stifle major innovations in structural materials design. This paper describes plans for a community-driven research initiative that addresses current gaps based on best-practice recommendations of leaders in mesoscale modeling, experimentation & cyberinfrastructure obtained at an NSF-sponsored workshop dedicated to this topic. The proposal is to create a hub for Mesoscale Experimentation and Simulation co-Operation (hMESO)-that will (I) provide curation and sharing of models, data, & codes, (II) foster co-design of experiments for model validation with systematic uncertainty quantification, & (III) provide a platform for education & workforce development. It will engage experimental & computational experts in mesoscale mechanics and plasticity, along with mathematicians and computer scientists with expertise in algorithms, data science, machine learning, & large-scale cyberinfrastructure initiatives.

We present Epistemic Variational Onsager Diffusion Models (EVODMs), a machine learning framework that integrates Onsager's variational principle with diffusion models to enable thermodynamically consistent learning of free energy and dissipation potentials (and associated evolution equations) from noisy, stochastic data in a robust manner. By further combining the model with Epinets, EVODMs quantify epistemic uncertainty with minimal computational cost. The framework is validated through two examples: (1) the phase transformation of a coiled-coil protein, modeled via a stochastic partial differential equation, and (2) a lattice particle process (the symmetric simple exclusion process) modeled via Kinetic Monte Carlo simulations. In both examples, we aim to discover the thermodynamic potentials that govern their dynamics in the deterministic continuum limit. EVODMs demonstrate a superior accuracy in recovering free energy and dissipation potentials from noisy data, as compared to traditional machine learning frameworks. Meanwhile, the epistemic uncertainty is quantified efficiently via Epinets and knowledge distillation. This work highlights EVODMs' potential for advancing data-driven modeling of non-equilibrium phenomena and uncertainty quantification for stochastic systems.

Thermodynamics with internal variables is a common approach in continuum mechanics to model inelastic (i.e., non-equilibrium) material behavior. While this approach is computationally and theoretically attractive, it currently lacks a well-established statistical mechanics foundation. As a result, internal variables are typically chosen phenomenologically and lack a direct link to the underlying physics which hinders the predictability of the theory. To address these challenges, we propose a machine learning approach that is consistent with the principles of statistical mechanics and thermodynamics. The proposed approach leverages the following techniques (i) the information bottleneck (IB) method to ensure that the learned internal variables are functions of the microstates and are capable of capturing the salient feature of the microscopic distribution; (ii) conditional normalizing flows to represent arbitrary probability distributions of the microscopic states as functions of the state variables; and (iii) Variational Onsager Neural Networks (VONNs) to guarantee thermodynamic consistency and Markovianity of the learned evolution equations. The resulting framework, called IB-VONNs, is tested on two problems of colloidal systems, governed at the microscale by overdamped Langevin dynamics. The first one is a prototypical model for a colloidal particle in an optical trap, which can be solved analytically, and thus ideal to verify the framework. The second problem is a one-dimensional phase-transforming system, whose macroscopic description still lacks a statistical mechanics foundation under general conditions. The results in both cases indicate that the proposed machine learning strategy can indeed bridge statistical mechanics and thermodynamics with internal variables away from equilibrium.

Over the past several decades, phase field modeling has been established as a standard simulation technique for mesoscopic science, allowing for seamless boundary tracking of moving interfaces and relatively easy coupling to other physical phenomena. However, despite its widespread success, phase field modeling remains largely driven by phenomenological justifications except in a handful of instances. In this work, we leverage a recently developed statistical mechanics framework for non-equilibrium phenomena, called Stochastic Thermodynamics with Internal Variables (STIV), to provide the first derivation of a phase field model for front propagation in a one dimensional elastic medium without appeal to phenomenology or fitting to experiments or simulation data. In the resulting model, the variables obey a gradient flow with respect to a non-equilibrium free energy, although notably, the dynamics of the strain and phase variables are coupled, and while the free energy functional is non-local in the phase field variable, it deviates from the traditional Landau-Ginzburg form. Moreover, in the systems analyzed here, the model accurately captures stress induced nucleation of transition fronts without the need to incorporate additional physics. We find that the STIV phase field model compares favorably to Langevin simulations of the microscopic system and we provide two numerical implementations enabling one to simulate arbitrary interatomic potentials.

Machine learning, with its remarkable ability for retrieving information and identifying patterns from data, has emerged as a powerful tool for discovering governing equations. It has been increasingly informed by physics, and more recently by thermodynamics, to further uncover the thermodynamic structure underlying the evolution equations, i.e., the thermodynamic potentials driving the system and the operators governing the kinetics. However, despite its great success, the inverse problem of thermodynamic model discovery from macroscopic data is in many cases non-unique, meaning that multiple pairs of potentials and operators can give rise to the same macroscopic dynamics, which significantly hinders the physical interpretability of the learned models. In this work, we propose a machine learning framework, named as Statistical-Physics-Informed Neural Networks (Stat-PINNs), which further encodes knowledge from statistical mechanics and resolves this non-uniqueness issue for the first time. The framework is here developed for purely dissipative isothermal systems. It only uses data from short-time particle simulations to learn the thermodynamic structure, which can be used to predict long-time macroscopic evolutions. We demonstrate the approach for particle systems with Arrhenius-type interactions, common to a wide range of phenomena, such as defect diffusion in solids, surface absorption and chemical reactions. Stat-PINNs can successfully recover the known analytic solution for the case with long-range interaction and discover the hitherto unknown potential and operator governing the short-range interaction cases. We compare our results with an analogous approach that solely excludes statistical mechanics, and observe that, in addition to recovering the unique thermodynamic structure, statistical mechanics relations can increase the robustness and predictability of the learning strategy.

Machine learning, with its remarkable ability for retrieving information and identifying patterns from data, has emerged as a powerful tool for discovering governing equations. It has been increasingly informed by physics, and more recently by thermodynamics, to further uncover the thermodynamic structure underlying the evolution equations, i.e., the thermodynamic potentials driving the system and the operators governing the kinetics. However, despite its great success, the inverse problem of thermodynamic model discovery from macroscopic data is in many cases non-unique, meaning that multiple pairs of potentials and operators can give rise to the same macroscopic dynamics, which significantly hinders the physical interpretability of the learned models. In this work, we propose a machine learning framework, named as Statistical-Physics-Informed Neural Networks (Stat-PINNs), which further encodes knowledge from statistical mechanics and resolves this non-uniqueness issue for the first time. The framework is here developed for purely dissipative isothermal systems. It only uses data from short-time particle simulations to learn the thermodynamic structure, which can be used to predict long-time macroscopic evolutions. We demonstrate the approach for particle systems with Arrhenius-type interactions, common to a wide range of phenomena, such as defect diffusion in solids, surface absorption and chemical reactions. Stat-PINNs can successfully recover the known analytic solution for the case with long-range interaction and discover the hitherto unknown potential and operator governing the short-range interaction cases. We compare our results with an analogous approach that solely excludes statistical mechanics, and observe that, in addition to recovering the unique thermodynamic structure, statistical mechanics relations can increase the robustness and predictability of the learning strategy.

Abstract Far-from-equilibrium phenomena are critical to all natural and engineered systems, and essential to biological processes responsible for life. For over a century and a half, since Carnot, Clausius, Maxwell, Boltzmann, and Gibbs, among many others, laid the foundation for our understanding of equilibrium processes, scientists and engineers have dreamed of an analogous treatment of nonequilibrium systems. But despite tremendous efforts, a universal theory of nonequilibrium behavior akin to equilibrium statistical mechanics and thermodynamics has evaded description. Several methodologies have proved their ability to accurately describe complex nonequilibrium systems at the macroscopic scale, but their accuracy and predictive capacity is predicated on either phenomenological kinetic equations fit to microscopic data or on running concurrent simulations at the particle level. Instead, we provide a novel framework for deriving stand-alone macroscopic thermodynamic models directly from microscopic physics without fitting in overdamped Langevin systems. The only necessary ingredient is a functional form for a parameterized, approximate density of states, in analogy to the assumption of a uniform density of states in the equilibrium microcanonical ensemble. We highlight this framework’s effectiveness by deriving analytical approximations for evolving mechanical and thermodynamic quantities in a model of coiled-coil proteins and double-stranded DNA, thus producing, to the authors’ knowledge, the first derivation of the governing equations for a phase propagating system under general loading conditions without appeal to phenomenology. The generality of our treatment allows for application to any system described by Langevin dynamics with arbitrary interaction energies and external driving, including colloidal macromolecules, hydrogels, and biopolymers.

Far-from-equilibrium phenomena are critical to all natural and engineered systems, and essential to biological processes responsible for life. For over a century and a half, since Carnot, Clausius, Maxwell, Boltzmann, and Gibbs, among many others, laid the foundation for our understanding of equilibrium processes, scientists and engineers have dreamed of an analogous treatment of non-equilibrium systems. But despite tremendous efforts, a universal theory of non-equilibrium behavior akin to equilibrium statistical mechanics and thermodynamics has evaded description. Several methodologies have proved their ability to accurately describe complex non-equilibrium systems at the macroscopic scale, but their accuracy and predictive capacity is predicated on either phenomenological kinetic equations fit to microscopic data, or on running concurrent simulations at the particle level. Instead, we provide a framework for deriving stand-alone macroscopic thermodynamics models directly from microscopic physics without fitting in overdamped Langevin systems. The only necessary ingredient is a functional form for a parameterized, approximate density of states, in analogy to the assumption of a uniform density of states in the equilibrium microcanonical ensemble. We highlight this framework's effectiveness by deriving analytical approximations for evolving mechanical and thermodynamic quantities in a model of coiled-coil proteins and double stranded DNA, thus producing, to the authors' knowledge, the first derivation of the governing equations for a phase propagating system under general loading conditions without appeal to phenomenology. The generality of our treatment allows for application to any system described by Langevin dynamics with arbitrary interaction energies and external driving, including colloidal macromolecules, hydrogels, and biopolymers.