Eric Darve

Sparse autoencoders (SAEs) extract millions of interpretable features from a language model, but flat feature inventories aren't very useful on their own. Domain concepts get mixed with generic and weakly grounded features, while related ideas are scattered across many units, and there's no way to understand relationships between features. We address this by first constructing a strict domain-specific concept universe from a large SAE inventory using contrastive activations and a multi-stage filtering process. Next, we build two aligned graph views on the filtered set: a co-occurrence graph for corpus-level conceptual structure, organized at multiple levels of granularity, and a transcoder-based mechanism graph that links source-layer and target-layer features through sparse latent pathways. Automated edge labeling then turns these graph views into readable knowledge graphs rather than unlabeled layouts. In a case study on a biology textbook, these graphs recover coherent chapter and subchapter-level structure, reveal concepts that bridge neighboring topics, and transform messy sentence-level activity containing thousands of features into compact, readable views that illustrate the model's local activity. Taken together, this reframes a flat SAE inventory as an internal knowledge graph that converts feature-level interpretability into a global map of model knowledge and enables audits of reasoning faithfulness.

Sparse autoencoders (SAEs) extract millions of interpretable features from a language model, but flat feature inventories aren't very useful on their own. Domain concepts get mixed with generic and weakly grounded features, while related ideas are scattered across many units, and there's no way to understand relationships between features. We address this by first constructing a strict domain-specific concept universe from a large SAE inventory using contrastive activations and a multi-stage filtering process. Next, we build two aligned graph views on the filtered set: a co-occurrence graph for corpus-level conceptual structure, organized at multiple levels of granularity, and a transcoder-based mechanism graph that links source-layer and target-layer features through sparse latent pathways. Automated edge labeling then turns these graph views into readable knowledge graphs rather than unlabeled layouts. In a case study on a biology textbook, these graphs recover coherent chapter and subchapter-level structure, reveal concepts that bridge neighboring topics, and transform messy sentence-level activity containing thousands of features into compact, readable views that illustrate the model's local activity. Taken together, this reframes a flat SAE inventory as an internal knowledge graph that converts feature-level interpretability into a global map of model knowledge and enables audits of reasoning faithfulness.

Physics-based 0D reduced-order models provide computationally lightweight predictions of cardiovascular flows, resolving bulk hemodynamics in fractions of a second that would take days to solve using traditional 3D finite-element techniques. However, the accuracy of 0D models is limited as a result of the dramatic simplifications made in their derivations. In this work, we use 0D parameters learned from high-fidelity 3D data to improve 0D model accuracy without sacrificing its low computational cost or interpretability. We use the resistor-quadratic resistor-inductor (RRI) model to predict pressure drops over 0D vessels and bifurcations, where the resistances and inductance (0D parameters) are predicted from the bifurcation or vessel geometry using neural networks trained on high-fidelity 3D simulations. We validate the hybrid physics-based data-driven framework in three types of patient-specific vasculature - aortic, aortofemoral, and pulmonary anatomies. Use of learned 0D parameters reduces error by at least 50% compared to baseline 0D parameters across all anatomical cohorts. The improvements are especially marked for the more complex pulmonary anatomies, where 0D models with learned parameters reduced error from 30% to 7%. Exclusion of the quadratic resistor in the RRI model improved convergence compared to using the full RRI model. The resulting hybrid model presents a means of real-time (personal laptop runtime of <2 seconds for the most complex pulmonary anatomies), interpretable, and accurate cardiovascular flow modeling, enabling digital twins that support clinical decision-making as well as cardiovascular science and engineering research.

Anomaly detection in rotating machinery is vital for maintaining industrial reliability, safety, and operational efficiency. However, developing accurate anomaly detection systems remains a significant challenge, particularly in scenarios where labeled anomalous data are limited or entirely absent during training. To address this issue, prior work introduced a transfer learning framework that estimates a key order---a weight vector over features that prioritizes certain dimensions in the anomaly scoring process---from a source domain and applies it to a target domain lacking any anomalous labels. In this study, we extend that framework to more complex and realistic scenarios where the target domain contains a limited number of known anomalous samples. Our enhanced approach estimates the key order within the target domain and integrates it with the source domain’s key order through both Bayesian-based and heuristic methods. We demonstrate that in scenarios with very limited anomalous data in the target, Bayesian-based methods for combining source and target key orders are prone to inaccuracies in estimating the variance of the target key order, leading to suboptimal performance. To mitigate this limitation, we propose a weighted linear combination strategy that improves robustness in regimes with extremely few anomalies. In contrast, when abundant anomalous data are available, the Bayesian-based approach remains preferable due to its capacity to model uncertainty more effectively. Furthermore, we introduce a heuristic based on the Kullback–Leibler (KL) divergence for source domain selection when the optimal source-target pairing is not known \textit{a priori}. Comprehensive experiments conducted on several benchmark datasets for rotating machinery validate the effectiveness of our approach. Results indicate that our proposed method of combining source and target key orders consistently outperforms source-only and target-only key orders across varying levels of anomaly availability. This work underscores the critical role of uncertainty-aware transfer learning and adaptive integration strategies in advancing anomaly detection capabilities in industrial settings characterized by scarce labeling.

Scientific reasoning increasingly requires linking structured experimental data with the unstructured literature that explains it, yet most large language model (LLM) assistants cannot reason jointly across these modalities. We introduce SpectraQuery, a hybrid natural-language query framework that integrates a relational Raman spectroscopy database with a vector-indexed scientific literature corpus using a Structured and Unstructured Query Language (SUQL)-inspired design. By combining semantic parsing with retrieval-augmented generation, SpectraQuery translates open-ended questions into coordinated SQL and literature retrieval operations, producing cited answers that unify numerical evidence with mechanistic explanation. Across SQL correctness, answer groundedness, retrieval effectiveness, and expert evaluation, SpectraQuery demonstrates strong performance: approximately 80 percent of generated SQL queries are fully correct, synthesized answers reach 93-97 percent groundedness with 10-15 retrieved passages, and battery scientists rate responses highly across accuracy, relevance, grounding, and clarity (4.1-4.6/5). These results show that hybrid retrieval architectures can meaningfully support scientific workflows by bridging data and discourse for high-volume experimental datasets.

Scientific reasoning increasingly requires linking structured experimental data with the unstructured literature that explains it, yet most large language model (LLM) assistants cannot reason jointly across these modalities. We introduce SpectraQuery, a hybrid natural-language query framework that integrates a relational Raman spectroscopy database with a vector-indexed scientific literature corpus using a Structured and Unstructured Query Language (SUQL)-inspired design. By combining semantic parsing with retrieval-augmented generation, SpectraQuery translates open-ended questions into coordinated SQL and literature retrieval operations, producing cited answers that unify numerical evidence with mechanistic explanation. Across SQL correctness, answer groundedness, retrieval effectiveness, and expert evaluation, SpectraQuery demonstrates strong performance: approximately 80 percent of generated SQL queries are fully correct, synthesized answers reach 93-97 percent groundedness with 10-15 retrieved passages, and battery scientists rate responses highly across accuracy, relevance, grounding, and clarity (4.1-4.6/5). These results show that hybrid retrieval architectures can meaningfully support scientific workflows by bridging data and discourse for high-volume experimental datasets.

Research paper on the existence of the LU factorization under different assumptions

Research paper on the existence of the LU factorization under different assumptions

We establish necessary and sufficient conditions for the existence of an LU factorization $A=LU$ for an arbitrary square matrix $A$, including singular and rank-deficient cases, without the use of row or column permutations. We prove that such a factorization exists if and only if the nullity of every leading principal submatrix is bounded by the sum of the nullities of the corresponding leading column and row blocks. While building upon the work of Okunev and Johnson, we present simpler, constructive proofs. Furthermore, we extend these results to characterize rank-revealing factorizations, providing explicit sparsity bounds for the factors $L$ and $U$. Finally, we derive analogous necessary and sufficient conditions for the existence of factorizations constrained to have unit lower or unit upper triangular factors.

Anomalies in radio-frequency (rf) stations can result in unplanned downtime and performance degradation in linear accelerators such as SLAC’s Linac Coherent Light Source (LCLS). Detecting these anomalies is challenging due to the complexity of accelerator systems, high data volume, and scarcity of labeled fault data. Prior work identified faults using beam-based detection, combining rf amplitude and beam position monitor data. Due to the simplicity of the rf amplitude data, classical methods are sufficient to identify faults, but the recall is constrained by the low-frequency and asynchronous characteristics of the data. In this work, we leverage high-frequency, time-synchronous rf phase data to enhance anomaly detection in the LCLS accelerator. Due to the complexity of phase data, classical methods fail, and we instead train deep neural networks within the Coincident Anomaly Detection (CoAD) framework. We find that applying CoAD to phase data detects nearly 3 times as many anomalies as when applied to amplitude data, while achieving broader coverage across rf stations. Furthermore, the rich structure of phase data enables us to cluster anomalies into distinct physical categories. Through the integration of auxiliary system status bits, we link clusters to specific fault signatures, providing additional granularity for uncovering the root cause of faults. We also investigate interpretability via Shapley values, confirming that the learned models focus on the most informative regions of the data and providing insight for cases where the model makes mistakes. This work demonstrates that phase-based anomaly detection for rf stations improves both diagnostic coverage and root cause analysis in accelerator systems and that deep neural networks are essential for effective analysis.

Many science and engineering problems rely on expensive computational simulations, where a multi-fidelity approach can accelerate the exploration of a parameter space. We study efficient allocation of a simulation budget using a Gaussian Process (GP) model in the binary simulation output case. This paper introduces Bernoulli Parameter Mutual Information (BPMI), a batch active learning algorithm for multi-fidelity GP classifiers. BPMI circumvents the intractability of calculating mutual information in the probability space by employing a first-order Taylor expansion of the link function. We evaluate BPMI against several baselines on two synthetic test cases and a complex, real-world application involving the simulation of a laser-ignited rocket combustor. In all experiments, BPMI demonstrates superior performance, achieving higher predictive accuracy for a fixed computational budget.

Abstract. Grounding line retreat in the Amundsen Sea Embayment (ASE) is expected to drive the largest Antarctic contribution to sea-level rise over the coming centuries. In this region, low mantle viscosity accelerates the solid Earth's viscoelastic response to ice mass loss, leading to a stabilizing feedback via bedrock uplift and local sea-level fall: effects governed by gravitation, rotation, and deformation (GRD) processes. These stabilizing effects can be enhanced by the presence of ridges and confinements, which have been identified in ASE but can only be represented by using high model resolutions. Here, we investigate how coupled ice sheet–GRD simulations respond to (i) ice sheet model resolution, (ii) GRD spatial resolution, and (iii) the coupling interval between the two systems. We consider two model setups with distinct mesh structures, surface mass balance (SMB) forcings, and basal melt parametrizations. Our findings underscore the importance of feedback mechanisms at kilometer scales and decadal to sub-decadal timescales. Resolving bedrock topography at 2 km instead of 1 km raises the projected sea level by 7.1 % in 2100 and lowers it by 18.8 % in 2350. In our most conservative setup, we find that bedrock uplift delays grounding line retreat by up to 30 years on ridges located 34 and 75 km upstream of Thwaites Glacier's current grounding line. This mechanism plays a key role in reducing Thwaites' sea-level contribution by up to 53.1 % in 2350. These findings underscore the critical need to reduce uncertainties in bedrock topography.

LUX-ZEPLIN (LZ) is a large-scale dark matter direct detection experiment that employs a time projection chamber (TPC) to observe particle interactions recorded as waveforms. In this work, we explore how unsupervised machine learning applied to waveforms can be used to characterize these interactions, with the goal of identifying anomalous events and detector pathologies. We introduce a framework for analyzing waveform shapes using dimensionality reduction. Applying this approach to single-scatter data, we cluster waveforms in the latent space constructed without explicit labels. The resulting regions in the embedding appear correlated with physically meaningful features, such as the identification of unphysical drift time events, a proxy for accidental coincidence events, with high recall (87%).

Multi-fidelity simulation is a widely used strategy to reduce the computational cost of many-query numerical simulation tasks such as uncertainty quantification, design space exploration, and design optimization. The reduced basis approach based on bi-fidelity interpolative decomposition is one such approach, which identifies a reduced basis, along with an interpolation rule in that basis, from low-fidelity samples to approximate the corresponding high-fidelity samples. However, as illustrated in the present study, when the model response is multi-modal and mode occupancy is stochastic, the assumptions underpinning this approach may not hold, thus leading to inaccurate estimates. We introduce the multi-modal interpolative decomposition method using bi-fidelity data, an extension tailored for this use case. Our work is motivated by a complex engineering application: a laser-ignited methane-oxygen rocket combustor evaluated over uncertain input parameters, exhibiting a bifurcation-like phenomenon in some regions of parameter space. Unlike the standard bi-fidelity interpolative decomposition approach, the proposed method can approximate a dataset of high-fidelity simulations for 16\% of the cost, while maintaining relatively high correlation (0.70--0.90) with parameter sensitivities.

Three-dimensional (3D) finite-element simulations of cardiovascular flows provide high-fidelity predictions to support cardiovascular medicine, but their high computational cost limits clinical practicality. Reduced-order models (ROMs) offer computationally efficient alternatives but suffer reduced accuracy, particularly at vessel bifurcations where complex flow physics are inadequately captured by standard Poiseuille flow assumptions. We present an enhanced numerical framework that integrates machine learning-predicted bifurcation coefficients into zero-dimensional (0D) hemodynamic ROMs to improve accuracy while maintaining computational efficiency. We develop a resistor-resistor-inductor (RRI) model that uses neural networks to predict pressure-flow relationships from bifurcation geometry, incorporating linear and quadratic resistances along with inductive effects. The method employs non-dimensionalization to reduce training data requirements and apriori flow split prediction for improved bifurcation characterization. We incorporate the RRI model into a 0D model using an optimization-based solution strategy. We validate the approach in isolated bifurcations and vascular trees, across Reynolds numbers from 0 to 5,500, defining ROM accuracy by comparison to 3D finite element simulation. Results demonstrate substantial accuracy improvements: averaged across all trees and Reynolds numbers, the RRI method reduces inlet pressure errors from 54 mmHg (45%) for standard 0D models to 25 mmHg (17%), while a simplified resistor-inductor (RI) variant achieves 31 mmHg (26%) error. The enhanced 0D models show particular effectiveness at high Reynolds numbers and in extensive vascular networks. This hybrid numerical approach enables accurate, real-time hemodynamic modeling for clinical decision support, uncertainty quantification, and digital twins in cardiovascular biomedical engineering.

Anomalies in radio-frequency (RF) stations can result in unplanned downtime and performance degradation in linear accelerators such as SLAC's Linac Coherent Light Source (LCLS). Detecting these anomalies is challenging due to the complexity of accelerator systems, high data volume, and scarcity of labeled fault data. Prior work identified faults using beam-based detection, combining RF amplitude and beam-position monitor data. Due to the simplicity of the RF amplitude data, classical methods are sufficient to identify faults, but the recall is constrained by the low-frequency and asynchronous characteristics of the data. In this work, we leverage high-frequency, time-synchronous RF phase data to enhance anomaly detection in the LCLS accelerator. Due to the complexity of phase data, classical methods fail, and we instead train deep neural networks within the Coincident Anomaly Detection (CoAD) framework. We find that applying CoAD to phase data detects nearly three times as many anomalies as when applied to amplitude data, while achieving broader coverage across RF stations. Furthermore, the rich structure of phase data enables us to cluster anomalies into distinct physical categories. Through the integration of auxiliary system status bits, we link clusters to specific fault signatures, providing additional granularity for uncovering the root cause of faults. We also investigate interpretability via Shapley values, confirming that the learned models focus on the most informative regions of the data and providing insight for cases where the model makes mistakes. This work demonstrates that phase-based anomaly detection for RF stations improves both diagnostic coverage and root cause analysis in accelerator systems and that deep neural networks are essential for effective analysis.

Computational Fluid Dynamics (CFD) is central to fluid mechanics, offering precise simulations of fluid behavior through partial differential equations (PDEs). Traditional CFD methods, such as those based on finite difference and finite volume schemes, are resource-consuming, especially for high-fidelity simulations of complex flows. Understanding such datasets presents unique challenges due to their high dimensionality, inherent stochasticity, and limited data availability.

Computational Fluid Dynamics (CFD) plays a pivotal role in fluid mechanics, enabling precise simulations of fluid behavior through partial differential equations (PDEs). However, traditional CFD methods are resource-intensive, particularly for high-fidelity simulations of complex flows, which are further complicated by high dimensionality, inherent stochasticity, and limited data availability. This paper addresses these challenges by proposing a data-driven approach that leverages a Gaussian Mixture Variational Autoencoder (GMVAE) to encode high-dimensional scientific data into low-dimensional, physically meaningful representations. The GMVAE learns a structured latent space where data can be categorized based on physical properties such as the Reynolds number while maintaining global physical consistency. To assess the interpretability of the learned representations, we introduce a novel metric based on graph spectral theory, quantifying the smoothness of physical quantities along the latent manifold. We validate our approach using 2D Navier-Stokes simulations of flow past a cylinder over a range of Reynolds numbers. Our results demonstrate that the GMVAE provides improved clustering, meaningful latent structure, and robust generative capabilities compared to baseline dimensionality reduction methods. This framework offers a promising direction for data-driven turbulence modeling and broader applications in computational fluid dynamics and engineering systems.

Abstract. The projected evolution of marine ice sheets is greatly affected by Gravitation, Rotation, and Deformation (GRD) effects over century timescales. In the Amundsen Sea sector, GRD effects cause viscoelastic solid Earth uplift and near-field sea-level fall, reducing the ice sheet mass loss. Spatiotemporal resolutions are critical for computational feasibility and accurately capturing solid Earth and ice sheet interactions. However, the sensitivity of coupled ice sheet and GRD models to these resolutions is not fully understood. Here, we investigate the influence of: (i) the spatial resolution of the ice sheet model, (ii) the spatial resolution of the GRD response, and (iii) the coupling interval between the ice sheet and GRD models. We consider two model setups with distinct mesh structures, surface mass balance and basal melt parameterizations. Our findings underscore the importance of feedback mechanisms at kilometer scales and decadal to sub-decadal timescales. Resolving bedrock topography at 2 km instead of 1 km results in sea-level projection differences of 7.1 % by 2100 and 18.8 % by 2350. We examine the influence of GRD effects on bedrock ridges to explain the noted sensitivities. In our most conservative setup, we find that bedrock uplift extends buttressing by up to 30 years on ridges located 34 and 75 km upstream of Thwaites' current grounding line. This mechanism plays a key role in reducing Thwaites’ sea-level contribution by up to 53.1 % in 2350. These findings underscore the critical need to reduce uncertainties in bedrock topography.

The detection of anomalous behavior of an engineered system or its components is an important task for enhancing reliability, safety, and efficiency across various engineering applications. However, designing an accurate anomaly detector can be very challenging in settings where anomalous labels are sparse or, in the worst case, missing in the training data. To mitigate this issue of the lack of anomalous labels in the domain of interest, existing approaches use transfer learning by leveraging information from anomalous samples in a closely related source domain. Although previous studies have shown good results from applying transfer learning, they do not specifically address the issue of high false-positive rates from such transfer. High false-positive rates can arise from misleading information present in uninformative/irrelevant features. Inspired by this observation, this paper focuses on identifying key input features, termed as such, due to their strong predictability in anomaly detection. A transfer learning approach is introduced that leverages the optimal <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$f_{\beta } $ </tex-math></inline-formula> score for key feature estimation. This approach involves finding a weight vector to amplify key features and attenuate uninformative inputs during prediction. We demonstrate the capabilities of our proposed anomaly detection method as a quality check for newly manufactured automotive transmissions. Given the use of frequency domain order-based features in our use case, our proposed method is also easily extensible to the anomaly detection of other rotating machinery. Based on our findings we also find that our proposed anomaly detection algorithm, utilizing precise data-driven features, outperforms detectors based on experience/heuristics-based features currently used in automotive engineering applications. More importantly, our proposed framework can work with any downstream unsupervised anomaly detection algorithm, allowing us to freely choose the best algorithm for the anomaly detection task on hand. Note to Practitioners—Unlike legacy systems that have rich data sets for both healthy and anomalous behaviors over a broad spectrum of operating states, newly designed system or design variants of existing systems have little or no performance data. This makes the task of identifying indicators of anomalous behaviors very challenging for systems without legacy information. This work introduces a machine learning framework that uses label information from related, well-studied systems, to improve anomaly detection outcomes in a new or variant (but related) system. We demonstrate the effectiveness of our transfer approach through the End of Line (EoL) anomaly detection for quality checks on new design variants of an existing automotive transmission. Inspired by feature selection techniques, our proposed framework is easy to use and offers excellent interpretability, which is crucial for achieving a zero-failure system design.

LUX-ZEPLIN (LZ) is a large-scale dark matter direct detection experiment that employs a time projection chamber (TPC) to observe particle interactions recorded as waveforms. In this work, we explore how unsupervised machine learning applied to waveforms can be used to characterize these interactions, with the goal of identifying anomalous events and detector pathologies. We introduce a framework for analyzing waveform shapes using dimensionality reduction. Applying this approach to single-scatter data, we cluster waveforms in the latent space constructed without explicit labels. The resulting regions in the embedding appear correlated with physically meaningful features, such as the identification of unphysical drift time events, a proxy for accidental coincidence events, with high recall (87%).

Reduced-order models are valuable tools in cardiovascular modeling when high-fidelity simulations are prohibitively expensive, but they often suffer reduced accuracy due to the simplifications in their formulations.One such assumption is that pressure is constant over a bifurcation.The resistor-resistor-inductor model was recently proposed to predict the pressure difference over bifurcations as a function of flow rate, using coefficients extracted from the bifurcation geometry using machine learning.We implemented the resistor-resistor-inductor model (previously only tested on isolated bifurcations) in a reducedorder cardiovascular model for an idealized test geometry and compare the results to those achieved with standard bifurcation handling.We found improved accuracy (defined by comparison to high-fidelity simulations) for both steady and transient flows.

X-ray light source facilities such as the Linac Coherence Light Source (LCLS) at SLAC National Accelerator Laboratory generate massive amounts of data that need to be analyzed quickly to inform ongoing experiments. The analysis of data streams coming from various parts of the instrument has potential to feed back into instrument operation or experiment steering. For example, shot-to-shot images of the beam profile inform on the quality of the beam delivery while downstream data read from large area detectors inform on the state of diffraction experiments carried on samples of interests at various beamlines. However, the high repetition rate and high dimensionality of these data streams make their analysis challenging, both in terms of scalability and interpretability. In this work, we propose an image monitoring and classification framework that follows a three-stage process: dimensionality reduction using principal component analysis on a matrix sketch, visualization using UMAP, and clustering using OPTICS. In the dimensionality reduction step, we combine the Priority Sampling algorithm with a modified Frequent Directions algorithm to produce a rank-adaptive accelerated matrix sketching (ARAMS) algorithm, wherein practitioners specify the target error of the sketch as opposed to the rank. Furthermore, the framework is parallel, enabling real-time analysis of the underpinning structure of the data. This framework demonstrates strong empirical performance and scalability. We explore its effectiveness on both beam profile data and diffraction data from recent LCLS experiments.

This chapter provides a concise exploration of Bayesian inference and demonstrates how recent advancements in machine learning can assist in efficient Bayesian inference within the realm of geomechanics applications. It begins with a brief overview of inverse problems, introducing the two predominant approaches for their solution: deterministic and Bayesian methods. Subsequently, the chapter delves into the challenges of Bayesian inference with physics-based forward problems, outlining the difficulties associated with their three crucial elements: the prior, the likelihood and the posterior. It details how machine learning can effectively address these challenges. Specifically, the chapter presents potential avenues to leverage machine learning: informative prior characterization with deep generative modeling; computationally inexpensive likelihood evaluation via operator learning; and efficient posterior inference in a black-box setting through multi-fidelity modeling.

Abstract Anomaly detection is an important task for complex scientific experiments and other complex systems (e.g. industrial facilities, manufacturing), where failures in a sub-system can lead to lost data, poor performance, or even damage to components. While scientific facilities generate a wealth of data, labeled anomalies may be rare (or even nonexistent), and expensive to acquire. Unsupervised approaches are therefore common and typically search for anomalies either by distance or density of examples in the input feature space (or some associated low-dimensional representation). This paper presents a novel approach called coincident learning for anomaly detection (CoAD), which is specifically designed for multi-modal tasks and identifies anomalies based on coincident behavior across two different slices of the feature space. We define an unsupervised metric, <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mrow> <mml:msub> <mml:mrow> <mml:mover> <mml:mi>F</mml:mi> <mml:mo stretchy="true">^</mml:mo> </mml:mover> </mml:mrow> <mml:mi>β</mml:mi> </mml:msub> </mml:mrow> </mml:math> , out of analogy to the supervised classification F β statistic. CoAD uses <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mrow> <mml:msub> <mml:mrow> <mml:mover> <mml:mi>F</mml:mi> <mml:mo stretchy="true">^</mml:mo> </mml:mover> </mml:mrow> <mml:mi>β</mml:mi> </mml:msub> </mml:mrow> </mml:math> to train an anomaly detection algorithm on unlabeled data , based on the expectation that anomalous behavior in one feature slice is coincident with anomalous behavior in the other. The method is illustrated using a synthetic outlier data set and a MNIST-based image data set, and is compared to prior state-of-the-art on two real-world tasks: a metal milling data set and our motivating task of identifying RF station anomalies in a particle accelerator.

Numerous applications in biology, statistics, science, and engineering require generating samples from high-dimensional probability distributions. In recent years, the Hamiltonian Monte Carlo (HMC) method has emerged as a state-of-the-art Markov chain Monte Carlo technique, exploiting the shape of such high-dimensional target distributions to efficiently generate samples. Despite its impressive empirical success and increasing popularity, its wide-scale adoption remains limited due to the high computational cost of gradient calculation. Moreover, applying this method is impossible when the gradient of the posterior cannot be computed (for example, with black-box simulators). To overcome these challenges, we propose a novel two-stage Hamiltonian Monte Carlo algorithm with a surrogate model. In this multi-fidelity algorithm, the acceptance probability is computed in the first stage via a standard HMC proposal using an inexpensive differentiable surrogate model, and if the proposal is accepted, the posterior is evaluated in the second stage using the high-fidelity (HF) numerical solver. Splitting the standard HMC algorithm into these two stages allows for approximating the gradient of the posterior efficiently, while producing accurate posterior samples by using HF numerical solvers in the second stage. We demonstrate the effectiveness of this algorithm for a range of problems, including linear and nonlinear Bayesian inverse problems with in-silico data and experimental data. The proposed algorithm is shown to seamlessly integrate with various low-fidelity and HF models, priors, and datasets. Remarkably, our proposed method outperforms the traditional HMC algorithm in both computational and statistical efficiency by several orders of magnitude, all while retaining or improving the accuracy in computed posterior statistics.

Anomaly detection is an important task in industrial applications. However, designing an accurate anomaly detector can be very challenging in settings where anomalous labels are sparse or, in the worst case, missing in the training data. To mitigate this issue of a lack of anomalous labels in the domain of interest, existing approaches use transfer learning, leveraging information from anomalous samples in a closely related domain. Although previous studies have shown good results from applying transfer learning, they do not specifically address the issue of high false-positive rates, especially in industrial settings. High false-positive rates can arise from misleading information present in uninformative features. Inspired by this observation, the paper focuses on identifying key input features—termed as such due to their strong predictability in anomaly detection. A transfer learning approach is introduced that leverages the optimal \(f_{\beta}\) score for key feature estimation. This approach involves a weight vector to amplify key features and attenuate uninformative inputs during prediction. We demonstrate the capabilities of our proposed method through an industrial application: anomaly detection for rotating machinery. Based on our findings, anomaly detection algorithms that utilize data-driven features obtained through the proposed method outperform detectors based on features identified by domain experts. More importantly, our proposed framework can work with any downstream unsupervised anomaly detection algorithm, allowing us to freely choose the best algorithm for the anomaly detection task.

Reduced-order models (ROMs) allow for the simulation of blood flow in patient-specific vasculatures without the high computational cost and wait time associated with traditional computational fluid dynamics (CFD) models. Unfortunately, due to the simplifications made in their formulations, ROMs can suffer from significantly reduced accuracy. One common simplifying assumption is the continuity of static or total pressure over vascular junctions. In many cases, this assumption has been shown to introduce significant error. We propose a model to account for this pressure difference, with the ultimate goal of increasing the accuracy of cardiovascular ROMs. Our model successfully uses a structure common in existing ROMs in conjunction with machine-learning techniques to predict the pressure difference over a vascular bifurcation. We analyze the performance of our model on steady and transient flows, testing it on three bifurcation cohorts representing three different bifurcation geometric types. We also compare the efficacy of different machine-learning techniques and two different model modalities.

The landscape of computational science and engineering is continually evolving, with the challenge of highdimensional regression problems standing as a significant hurdle in numerous scientific endeavors.Addressing this challenge, our research, funded by this award, has led to the development of an innovative computational framework known as Probabilistic Partition of Unity Networks (PPOU-Nets).This initiative represents a collaborative effort to harness the potential of mathematics and physics-informed machine learning in tackling multiscale and multiphysics problems prevalent in high-dimensional spaces.Through this work, we have proposed a novel methodology that seamlessly integrates adaptive dimensionality reduction and a mixture of experts model, thereby facilitating a more efficient and accurate approximation of complex functions.This research effort has not only advanced the state of computational science but also opened new avenues for exploration in quantum computing and beyond.This report outlines the motivation, methodology, key findings, and implications of our work, underscoring our contributions to the broader scientific community and the potential pathways for future research.The PPOU-Net framework, developed under this award, focuses on adaptive dimensionality reduction and leverages a mixture of experts model on a low-dimensional manifold, where each cluster is associated with a local polynomial.A unique training strategy utilizing the expectation maximization (EM) algorithm and gradient descent for updating the model parameters is presented.The PPOU-Nets demonstrated superior performance over traditional fully-connected neural networks in various numerical experiments and applications in quantum computing, acting as surrogate models for cost landscapes associated with variational quantum circuits.

Significant advances in utilizing deep learning for anomaly detection have been made in recent years. However, these methods largely assume the existence of a normal training set (i.e., uncontaminated by anomalies) or even a completely labeled training set. In many complex engineering systems, such as particle accelerators, labels are sparse and expensive; in order to perform anomaly detection in these cases, we must drop these assumptions and utilize a completely unsupervised method. This paper introduces the Resilient Variational Autoencoder (ResVAE), a deep generative model specifically designed for anomaly detection. ResVAE exhibits resilience to anomalies present in the training data and provides feature-level anomaly attribution. During the training process, ResVAE learns the anomaly probability for each sample as well as each individual feature, utilizing these probabilities to effectively disregard anomalous examples in the training data. We apply our proposed method to detect anomalies in the accelerator status at the SLAC Linac Coherent Light Source (LCLS). By utilizing shot-to-shot data from the beam position monitoring system, we demonstrate the exceptional capability of ResVAE in identifying various types of anomalies that are visible in the accelerator.