Wei‐keng Liao

Abstract Driven by advances in artificial intelligence and the growing availability of databases, machine learning (ML) now plays a central role in data-driven materials knowledge discovery. In studies that employ ML models, maintaining deterministic workflows is crucial for reproducibility, as it ensures that repeated runs with identical settings yield consistent and reliable predictions. Here, we conduct a determinism analysis of the recently proposed hybrid large language model-graph neural network (Hybrid-LLM-GNN) framework for material property prediction. Our study provides a comprehensive analysis of sources of non-determinism, with particular focus on variability introduced by hardware variations and software stacks across training and inference pipelines. By distinguishing different forms of determinism, we assess how discrepancies arise under varying execution conditions. In cases where results are non-deterministic, we further analyze and compare prediction differences to assess the impact of these variations. Our investigation uncovers systematic patterns in prediction discrepancies at the dataset level as well as variations at the individual sample level. Based on the findings, we identify the main sources of divergence and provide practical recommendations to improve deterministic behavior in ML-based materials property prediction workflows.

This patch release includes, Fix for issues with HDF5 iotypes for EAMXX history output

SCORPIO v1.9.2 includes the following fixes, Fix for build issues with some compilers due to missing uint64 type definition

Abstract Graph neural networks (GNNs) have proven effective in understanding and predicting diverse material properties, even when working with limited datasets. An important step in training GNN is to use an appropriate and informative graph embedding that can adequately represent the structural and compositional information in the chemical space. Current graph embeddings consist of composition and structure-agnostic element-level encodings, which are static in nature. This makes it challenging to differentiate between different compounds on the element level, especially for datasets with limited data size, thereby relying more on the complex input and architecture for model training. Here, we present a novel framework for GNN-based prediction tasks that use dynamic embedding to significantly improve the models’ predictive ability on materials properties with limited data size. We evaluated the proposed framework on multiple materials datasets across various domains to find that the model trained using dynamic embedding outperforms the models trained using conventional static embedding and features obtained using a pre-trained model. The proposed framework holds significant potential for expediting artificial intelligence (AI)-driven materials discovery.

SCORPIO v1.9.1 includes many fixes related to data compression, Fixed issues with multiple redef/enddef calls with HDF5 Manual datatype conversion, as needed, for HDF5 output Fix typos for ADIOS+ZFP compression rate Disable HDF5+ZFP compression for scalars and 5D+ variables and variables with only unlimited dimensions Adding related tests put_vars_* support for CDF5 types when using PnetCDF

Large Language Models (LLMs) can enable inverse materials discovery by generating text-encoded crystal structures from target properties.

The objective of RAPIDS2 SciDAC Institute for Computer Science and Data is to assist Office of Science (SC) application teams in overcoming computer science and data challenges in the use of DOE supercomputing resources to achieve science breakthroughs.

High-level I/O libraries, such as PnetCDF and HDF5, are commonly used by large-scale scientific applications to perform I/O tasks in parallel. These I/O libraries store the metadata of data objects in files along with their raw data. To ensure metadata consistency during parallel data object creation, they require applications to call the metadata APIs collectively using consistent metadata. Such a requirement can result in an expensive consistency check, as its cost increases with the metadata volume and the number of processes. To address this limitation, we propose a new file header format, which uses partitioned metadata blocks to enable independent data object creation and reduce the objects required for consistency check. Our performance evaluation shows that this new design achieves a scalable performance, cutting data object creation times by up to 196 × when running on 4096 MPI processes to create 5,684,800 data objects in parallel.

In this study, we present an artificial intelligence (AI)-driven framework for predicting the microstructural texture of polycrystalline materials after a specific deformation process. The microstructural texture is defined in terms of the orientation distribution function (ODF) which indicates the volume density of crystal orientations. Our approach leverages an encoder-decoder model with Long Short-Term Memory (LSTM) layers to model the relationship between processing conditions and material properties. As a case study, we apply our framework to copper, generating a dataset of 3125 unique processing parameter combinations and their corresponding ODF vectors. The resulting predictions enable the calculation of homogenized properties. Our AI-driven framework outperforms traditional material processing simulations, yielding faster results with limited error rates (< 0.3% for both the elastic matrix C and the compliance matrix S), making it a promising tool for the expedited design of microstructures with tailored properties.

This document provides a technical project report for the first six months of 2025 for the Software Tools Ecosystem Project (STEP). The mission of STEP is to enable critical software tools to proactively adapt to emerging platform technologies (such as new accelerators, storage devices, network technologies, and smart devices) and emerging application use cases (such as advanced machine learning and workflow frameworks) so that they continue to meet the needs of scientific computing and provide a strong foundation for future Advanced Scientific Computing Research activities. Our challenges include the wide breadth of our stakeholders and rapidly evolving platform technology dependencies.

High-level I/O libraries, such as HDF5 and PnetCDF, are commonly used by large-scale scientific applications to perform I/O tasks in parallel. These I/O libraries store the metadata such as data types and dimensionality along with the raw data in the same files. While these libraries are well-optimized for concurrent access to the raw data, they are designed neither to handle a large number of data objects efficiently nor to create different data objects independently by multiple processes, as they require applications to call data object creation APIs collectively with consistent metadata among all processes. Applications that process data gathered from remote sensors, such as particle collision experiments in high-energy physics, may generate data of different sizes from different sensors and desire to store them as separate data objects. For such applications, the I/O library's requirement on collective data object creation can become very expensive, as the cost of metadata consistency check increases with the metadata volume as well as the number of processes. To address this limitation, using PnetCDF as an experimental platform, we investigate solutions in this paper that abide the netCDF file format, as well as propose a new file header format that enables independent data object creation. The proposed file header consists of two sections, an index table and a list of metadata blocks. The index table contains the reference to the metadata blocks and each block stores metadata of objects that can be created collectively or independently. The new design achieves a scalable performance, cutting data object creation times by up to 582x when running on 4096 MPI processes to create 5,684,800 data objects in parallel. Additionally, the new method reduces the memory footprints, with each process requiring an amount of memory space inversely proportional to the number of processes.

Recent developments in materials science have made it possible to synthesize millions of individual nanoparticles on a chip. However, many steps in the characterization process still require extensive human input. To address this challenge, we present an automated image processing pipeline that optimizes high-throughput nanoparticle characterization using intelligent image segmentation and coordinate generation. The proposed method can rapidly analyze each image and return optimized acquisition coordinates suitable for multiple analytical STEM techniques, including 4D-STEM, EELS, and EDS. The pipeline employs computer vision and unsupervised learning to remove the image background, segment the particle into areas of interest, and generate acquisition coordinates. This approach eliminates the need for uniform grid sampling, focusing data collection on regions of interest. We validated our approach using a diverse dataset of over 900 high-resolution grayscale nanoparticle images, achieving a 96.0% success rate based on expert-validated criteria. Using established 4D-STEM acquisition times as a baseline, our method demonstrates a 25.0 to 29.1-fold reduction in total processing time. By automating this crucial preprocessing step and optimizing data acquisition, our pipeline significantly accelerates materials characterization workflows while reducing unnecessary data collection.

We investigate the impact of neutrino statistical property on cosmology and the constraints imposed by cosmological data on neutrino statistics. Cosmological data from probes such as Cosmic Microwave Background(CMB) radiation and Baryon Acoustic Oscillation(BAO) are used to constrain the statistical parameter of neutrino. This constraint is closely related to the degeneracy effects among neutrino statistical property, the sum of neutrino masses, and the Hubble constant. Our results show that purely bosonic neutrinos can be ruled out at 95\% confidence level and purely fermionic neutrinos are preferred.

This study combines Graph Neural Networks (GNNs) and Large Language Models (LLMs) to improve material property predictions. By leveraging both embeddings, this hybrid approach achieves up to a 25% improvement over GNN-only model in accuracy.

Modern data mining methods have seen a widespread and growing application in the field of materials science for regression-based predictive modeling due to their effectiveness in extracting and utilizing the hidden information from the materials datasets. However, due to the costly and time-consuming nature of the methods involved in obtaining the experimental and computational data, the majority of the materials datasets are small in size. Moreover, limited hand-engineered representations available from the raw materials data make it harder to improve the accuracy of predictive models on such small and specialized training datasets. In this paper, we introduce a novel technique that combines transfer learning (TL) and representation learning (RL) using a pre-trained deep neural network to maximize accuracy without additional computational costs on inorganic material properties. The performance of the proposed method is compared against traditional machine learning (ML), and deep neural network models trained from scratch (SC) with elemental fraction (EF) as input, more informative physical attributes (PA) as input (for a stringent comparison), as well as conventional TL and RL techniques using deep neural networks. The results demonstrate that the proposed method can improve the accuracy as compared to SC models and conventional TL and RL techniques.

Polycrystalline materials are crucial in various industries, necessitating a comprehensive understanding of the processing-structure-property-performance (PSPP) relationships. Traditional experimental methods are laborious and slow, while computational approaches predominantly address forward problems, deriving structures and properties from processing conditions. Conversely, inferring processing parameters from desired microstructures and properties remains a crucial yet challenging inverse problem due to the complex and nonlinear mappings involved. In this work, we propose a deep learning-based framework exploring non-sequential and sequential models to address two key inverse problems: predicting processing parameters from microstructures and from properties. Focusing on microstructural texture defined by the orientation distribution function (ODF), we apply our framework to copper, generating a dataset of 31,588 unique processing strain rates (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$s$</tex><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup>) in [0, 1] with corresponding ODFs and homogenized properties through simulations. Our inverse prediction results on processing parameters demonstrate high accuracy, with average test RMSEs of 0.0152 from microstructures and 0.0295 from properties. These findings validate the framework's efficacy as a tool for polycrystalline materials process design, enabling the precise determination of processing methods to achieve desired microstructures and properties.

Deep Neural Networks, as a subset of AI, outper-form in understanding complex relationships. The key to this success lies in the network's ability to adapt to problem-specific nuances. During model training, the network dynamically optimizes its weights by updating them during backpropagation while trying to minimize the value of the loss function. Throughout this process, the shaping of model weights is crucially linked to how they were initialized. In this study, we introduce the auxiliary network model, called Sup-Walsh (Support Walsh), which reorganizes weights to enhance class boundaries. We tested our approach on three publicly available datasets using popular classification models. For instance, when using AlexNet [1] on the MNIST dataset [2], integrating Sup-Walsh led to a significant increase in accuracy after first epoch from 14.61% to 78.99%. Similarly, GoogleNet [3] on the FashionMNIST dataset [4] showed a notable 31.61% accuracy difference between configurations without and with Sup-Walsh after first epoch. Across nearly all experiments, our proposed method consistently outperformed existing approaches, demonstrating its potential to improve classification accuracy. Code availability: Code is available at Efficient-Weight-Initializer.

Generative models have been widely used in the field of computer vision due to their ability to produce unseen data points. Its application has proven to be useful in various scientific domains such as materials science for generating new microstructure images that require learning nonlinear and one-to-many property-microstructure relationships. However, existing simulation-based solutions for this application are inefficient and time-consuming. Moreover, nonlinearity from the lower to higher dimensions poses considerable challenges. In this work, we propose a novel Mixture Density Network (MDN) based Quan-tized Autoencoder Network designed to generate microstructure images from only a single data point by establishing one-to-many nonlinear relationships from property to microstructure. Once the autoencoder effectively compresses spatial information into the property domain, we use the combination of produced latent vectors and property values to create a supplementary dataset for MDN. Upon completion of the model training, generative structures are extracted and merged to create a framework that generates images based on target property values (i.e., absorption values in this study). The trained MDN demonstrates proficiency in generating latent vectors within distribution, while the proposed Vector Quantized Variational Autoencoder (VQ-VAE) efficiently maps the embedding table to the latent space, generating images from property values within the range of properties observed during training. We demonstrate that our proposed model consistently outperforms the baselines with respect to generating new microstructure images having target properties and overcoming the above-mentioned challenges.

Recent progress in deep learning has significantly impacted materials science, leading to accelerated material discovery and innovation. ElemNet, a deep neural network model that predicts formation energy from elemental compositions, exemplifies the application of deep learning techniques in this field. However, the "black-box" nature of deep learning models often raises concerns about their interpretability and reliability. In this study, we propose XElemNet to explore the interpretability of ElemNet by applying a series of explainable artificial intelligence (XAI) techniques, focusing on post-hoc analysis and model transparency. The experiments with artificial binary datasets reveal ElemNet's effectiveness in predicting convex hulls of element-pair systems across periodic table groups, indicating its capability to effectively discern elemental interactions in most cases. Additionally, feature importance analysis within ElemNet highlights alignment with chemical properties of elements such as reactivity and electronegativity. XElemNet provides insights into the strengths and limitations of ElemNet and offers a potential pathway for explaining other deep learning models in materials science.

Recent innovations have made it possible to produce millions of distinct nanoparticles on a chip. These vast volumes of data are impossible to analyze manually, necessitating the development of automated tools. In previous work, we created a binary classification machine learning model to select quality nanoparticle images for downstream analysis. In this work, we show that adding a custom image preprocessing step before model training can produce significantly higher-performing models in a fraction of the time and make the model more robust to different image noise levels and microscope acquisition settings. The proposed image processing pipeline effectively cleans raw nanoparticle images, enhances key features, and allows us to use much lower resolution images and simpler neural network model architectures, resulting in higher performance and significant cost savings. Experiments demonstrate superior performance relative to our baseline, including a 15% improvement in recall and more than a 10% increase in accuracy. Given the high cost of downstream analysis, it is critical to minimize false positives in our application, and our best-performing model obtains a precision of 97.3% and weighted F-score of 95.9% on an unseen test set. Additionally, model training time is reduced from 15.5 hours to 32 seconds. We expect that adopting this pipeline for AI-driven automated nanoparticle characterization will offer a considerable speedup in the laboratory, allowing researchers to rapidly and accurately analyze much greater volumes of data and accelerate materials discovery.

Introduction Reconstructing low-level particle tracks in neutrino physics can address some of the most fundamental questions about the universe. However, processing petabytes of raw data using deep learning techniques poses a challenging problem in the field of High Energy Physics (HEP). In the Exa.TrkX Project, an illustrative HEP application, preprocessed simulation data is fed into a state-of-art Graph Neural Network (GNN) model, accelerated by GPUs. However, limited GPU memory often leads to Out-of-Memory (OOM) exceptions during training, due to the large size of models and datasets. This problem is exacerbated when deploying models on High-Performance Computing (HPC) systems designed for large-scale applications. Methods We observe a high workload imbalance issue during GNN model training caused by the irregular sizes of input graph samples in HEP datasets, contributing to OOM exceptions. We aim to scale GNNs on HPC systems, by prioritizing workload balance in graph inputs while maintaining model accuracy. Our paper introduces diverse balancing strategies aimed at decreasing the maximum GPU memory footprint and avoiding the OOM exception, across various datasets. Results Our experiments showcase memory reduction of up to 32.14% compared to the baseline. We also demonstrate the proposed strategies can avoid OOM in application. Additionally, we create a distributed multi-GPU implementation using these samplers to demonstrate the scalability of these techniques on the HEP dataset. Discussion By assessing the performance of these strategies as data loading samplers across multiple datasets, we can gauge their effectiveness in both single-GPU and distributed environments. Our experiments, conducted on datasets of varying sizes and across multiple GPUs, broaden the applicability of our work to various GNN applications that handle input datasets with irregular graph sizes.

Liquid argon time projection chamber (LArTPC) detector technology offers a wealth of high-resolution information on particle interactions, and leveraging that information to its full potential requires sophisticated automated reconstruction techniques. This article describes nugraph2, a graph neural network for low-level reconstruction of simulated neutrino interactions in a LArTPC detector. Simulated neutrino interactions in the MicroBooNE detector geometry are described as heterogeneous graphs, with energy depositions on each detector plane forming nodes on planar subgraphs. The network utilizes a multihead attention message-passing mechanism to perform background filtering and semantic labeling on these graph nodes, identifying those associated with the primary physics interaction with 98.0% efficiency and labeling them according to particle type with 94.9% efficiency. The network operates directly on detector observables across multiple two-dimensional representations but utilizes a three-dimensional-context-aware mechanism to encourage consistency between these representations. Model inference takes $0.12\text{ }\text{ }\mathrm{s}/\mathrm{event}$ on a CPU and $0.005\text{ }\text{ }\mathrm{s}/\mathrm{event}$ batched on a GPU. This architecture is designed to be a general-purpose solution for particle reconstruction in neutrino physics, with the potential for deployment across a broad range of detector technologies, and offers a core convolution engine that can be leveraged for a variety of tasks beyond the two described in this paper.

Traditionally, materials discovery has been driven more by evidence and intuition than by systematic design. However, the advent of "big data" and an exponential increase in computational power have reshaped the landscape. Today, we use simulations, artificial intelligence (AI), and machine learning (ML) to predict materials characteristics, which dramatically accelerates the discovery of novel materials. For instance, combinatorial megalibraries, where millions of distinct nanoparticles are created on a single chip, have spurred the need for automated characterization tools. This paper presents an ML model specifically developed to perform real-time binary classification of grayscale high-angle annular dark-field images of nanoparticles sourced from these megalibraries. Given the high costs associated with downstream processing errors, a primary requirement for our model was to minimize false positives while maintaining efficacy on unseen images. We elaborate on the computational challenges and our solutions, including managing memory constraints, optimizing training time, and utilizing Neural Architecture Search tools. The final model outperformed our expectations, achieving over 95% precision and a weighted F-score of more than 90% on our test data set. This paper discusses the development, challenges, and successful outcomes of this significant advancement in the application of AI and ML to materials discovery.

Summary Parallel I/O is a critical technique for moving data between compute and storage subsystems of supercomputers. With massive amounts of data produced or consumed by compute nodes, high‐performant parallel I/O is essential. I/O benchmarks play an important role in this process; however, there is a scarcity of I/O benchmarks representative of current workloads on HPC systems. Toward creating representative I/O kernels from real‐world applications, we have created h5bench , a set of I/O kernels that exercise hierarchical data format version 5 (HDF5) I/O on parallel file systems in numerous dimensions. Our focus on HDF5 is due to the parallel I/O library's heavy usage in various scientific applications running on supercomputing systems. The various tests benchmarked in the h5bench suite include I/O operations (read and write), data locality (arrays of basic data types and arrays of structures), array dimensionality (one‐dimensional arrays, two‐dimensional meshes, three‐dimensional cubes), I/O modes (synchronous and asynchronous). In this paper, we present the observed performance of h5bench executed along several of these dimensions on existing supercomputers (Cori and Summit) and pre‐exascale platforms (Perlmutter, Theta, and Polaris). h5bench measurements can be used to identify performance bottlenecks and their root causes and evaluate I/O optimizations. As the I/O patterns of h5bench are diverse and capture the I/O behaviors of various HPC applications, this study will be helpful to the broader supercomputing and I/O community.

Liquid Argon Time Projection Chamber (LArTPC) detector technology offers a wealth of high-resolution information on particle interactions, and leveraging that information to its full potential requires sophisticated automated reconstruction techniques. This article describes NuGraph2, a Graph Neural Network (GNN) for low-level reconstruction of simulated neutrino interactions in a LArTPC detector. Simulated neutrino interactions in the MicroBooNE detector geometry are described as heterogeneous graphs, with energy depositions on each detector plane forming nodes on planar subgraphs. The network utilizes a multi-head attention message-passing mechanism to perform background filtering and semantic labelling on these graph nodes, identifying those associated with the primary physics interaction with 98.0\% efficiency and labelling them according to particle type with 94.9\% efficiency. The network operates directly on detector observables across multiple 2D representations, but utilizes a 3D-context-aware mechanism to encourage consistency between these representations. Model inference takes 0.12~s/event on a CPU, and 0.005s/event batched on a GPU. This architecture is designed to be a general-purpose solution for particle reconstruction in neutrino physics, with the potential for deployment across a broad range of detector technologies, and offers a core convolution engine that can be leveraged for a variety of tasks beyond the two described in this article.

Modern data mining techniques using machine learning (ML) and deep learning (DL) algorithms have been shown to excel in the regression-based task of materials property prediction using various materials representations. In an attempt to improve the predictive performance of the deep neural network model, researchers have tried to add more layers as well as develop new architectural components to create sophisticated and deep neural network models that can aid in the training process and improve the predictive ability of the final model. However, usually, these modifications require a lot of computational resources, thereby further increasing the already large model training time, which is often not feasible, thereby limiting usage for most researchers. In this paper, we study and propose a deep neural network framework for regression-based problems comprising of fully connected layers that can work with any numerical vector-based materials representations as model input. We present a novel deep regression neural network, iBRNet, with branched skip connections and multiple schedulers, which can reduce the number of parameters used to construct the model, improve the accuracy, and decrease the training time of the predictive model. We perform the model training using composition-based numerical vectors representing the elemental fractions of the respective materials and compare their performance against other traditional ML and several known DL architectures. Using multiple datasets with varying data sizes for training and testing, We show that the proposed iBRNet models outperform the state-of-the-art ML and DL models for all data sizes. We also show that the branched structure and usage of multiple schedulers lead to fewer parameters and faster model training time with better convergence than other neural networks. Scientific contribution: The combination of multiple callback functions in deep neural networks minimizes training time and maximizes accuracy in a controlled computational environment with parametric constraints for the task of materials property prediction.

Abstract Modern data mining methods have demonstrated effectiveness in comprehending and predicting materials properties. An essential component in the process of materials discovery is to know which material(s) will possess desirable properties. For many materials properties, performing experiments and density functional theory computations are costly and time-consuming. Hence, it is challenging to build accurate predictive models for such properties using conventional data mining methods due to the small amount of available data. Here we present a framework for materials property prediction tasks using structure information that leverages graph neural network-based architecture along with deep-transfer-learning techniques to drastically improve the model’s predictive ability on diverse materials (3D/2D, inorganic/organic, computational/experimental) data. We evaluated the proposed framework in cross-property and cross-materials class scenarios using 115 datasets to find that transfer learning models outperform the models trained from scratch in 104 cases, i.e., ≈90%, with additional benefits in performance for extrapolation problems. We believe the proposed framework can be widely useful in accelerating materials discovery in materials science.

A process simulator provides valuable insights into the evolution of microstructures under various elementary processes, employing the Orientation Distribution Function (ODF) as a representation of the microstructure's texture. However, such simulations often involve complex physical computations, making them time-consuming. To address this, our study introduces an artificial intelligence (AI)-based framework to predict the microstructural texture of polycrystalline materials using a specified deformation process. As a case study, we apply our framework to copper. The dataset includes 3,125 unique processing parameter combinations and their corresponding ODF vectors generated using a process simulator. The resulting predictions enable the calculation of homogenized properties. As opposed to traditional material processing simulations, our AI-driven framework offers faster results with minimal error rates (less than 0.5%). This indicates that our approach is a promising tool for rapidly predicting processing-specific microstructures and properties, thereby offering significant improvements over conventional simulation techniques.

Large-scale parallel applications can face significant I/O performance bottlenecks, making efficient I/O crucial. This work presents a comparative study of several parallel I/O implementations in the Weather Research and Forecasting model, including PnetCDF blocking and non-blocking I/O options, netCDF4, HDF5 Log VOL, and ADIOS. For I/O methods creating files in a canonical data layout, PnetCDF's non-blocking option offers up to 2x improvement over its blocking option and up to 4.5x over HDF5 via netCDF4, demonstrating the effectiveness of the write request aggregation technique. The HDF5 Log VOL outperforms ADIOS with a 4x improvement in write performance when creating files in the log layout, although both require non-negligible time to convert the file back to canonical order for post-run analysis. From these results we extract some observations that can guide I/O strategies for modern parallel codes.

Abstract Contemporary materials science has seen an increasing application of various artificial intelligence techniques in an attempt to accelerate the materials discovery process using forward modeling for predictive analysis and inverse modeling for optimization and design. Over the last decade or so, the increasing availability of computational power and large materials datasets has led to a continuous evolution in the complexity of the techniques used to advance the frontier. In this Review, we provide a high-level overview of the evolution of artificial intelligence in contemporary materials science for the task of materials property prediction in forward modeling. Each stage of evolution is accompanied by an outline of some of the commonly used methodologies and applications. We conclude the work by providing potential future ideas for further development of artificial intelligence in materials science to facilitate the discovery, design, and deployment workflow. Graphical abstract

Prediction of the autogenous shrinkage referred to as the reduction of apparent volume of concrete under seal and isothermal conditions is of great significance in the service life analysis and design of durable concrete structures, especially with the increasing use of concrete with low water-to-cement ratios. However, due to the highly complex mechanism of autogenous shrinkage, it is hard to design accurate mechanistic models for it. Existing state-of-the-art models for autogenous shrinkage do not perform well for several reasons such as not being able to capture faster shrinkage change at early ages (swelling), coefficients used are derived using statistical optimization methods to fit certain databases only, and mechanism to identify the most influencing factors on autogenous shrinkage is not present. Moreover, it is also challenging to deploy a machine learning framework directly to perform predictive analysis due to the sparse and noisy nature of the available experimental dataset. In this paper, we study and propose a method to combine the physics-based knowledge and the predictive ability of deep regression neural networks to mitigate the shortcomings of the existing models. We introduce a novel data augmentation technique that utilizes physics based knowledge to improve the accuracy while maintaining the characteristics of autogenous shrinkage in its predictions simultaneously. Using state-of-the-art B4 model, a genetic algorithm, and a deep neural network trained using raw data for comparison, we show that the proposed methods help improve the accuracy of the model as compared to other methods. We also observe that the proposed method is able to successfully learn and predict the swelling component of the shrinkage strain curve as well, which cannot be predicted using the existing state-of-the-art models.

The development of high-speed detectors and associated data acquisition workflows in electron microscopes has enabled data generation at an unprecedented rate. It is not uncommon to generate terabytes of data per hour which can easily overwhelm computing infrastructure and downstream processing, thereby limiting the full utilization of the microscope hardware [1]. Implementing a selective approach to imaging to only acquire high-fidelity data from areas of interest is common in manual instrument operation, where bright- or dark-field STEM images are acquired first to identify regions of interest for collecting higher-dimensional spectroscopy or diffraction data. In the context of automated acquisitions workflows, this procedure becomes significantly more challenging, as it becomes necessary to quickly make decisions about regions of interest in the absence of a human operator. To that end, we have developed machine learning models working with low-fidelity dark-field STEM images of nanostructure arrays to perform a binary quality assessment to determine the highest priority areas to acquire high-fidelity data. Our models will be deployed in an automated electron microscopy system for the high-throughput analysis of nanomaterial megalibraries. Megalibraries contain arrays of 108 nanoparticles with spatially encoded size and composition gradients on a single chip, these libraries are uniquely suited for the discovery of new materials at high throughput [2]. Given the volume of materials in a megalibrary, we have designed our models to be highly effective at binary classification on unseen images while minimizing the single-inference times and the number of false positive results which would result in long latencies and high downstream processing cost. In this presentation, we will describe the computational challenges and solutions implemented in this design (e.g., memory constraints, training time, Neural Architecture Search (NAS) tools). We present the standard neural network evaluation metrics, training time, and single-inference time for the most accurate model; our best-performing model achieves a precision of >95%. We further report a weighted F-score of >90% on our test data set as a balancing metric of precision and recall for a given application. Given that false positives would incur a high downstream cost in the analysis pipeline, we chose to prioritize precision over recall by a factor of 9-to-1. This binary classification of low-fidelity data represents an important step towards fully automated electron microscopy workflows, as it introduces a methodology for autonomous decision-making in situ. The acceleration resulting from this automation effectively increases the experimental throughput, enabling the analysis of materials at an unprecedented rate [3]. Automated quality assessment on a nanoparticle megalibrary. (A) Schematic illustration of the screening process, acquiring images at nanoparticle locations. (B) Acquired high-angle annular dark-field (HAADF) images as seen by the model. Scale bars: 200 nm. (C) Illustration of the quality assessment pipeline, where each raw image is fed into the model, returning a binary classifier. If the image passes the test, the microscope can magnify and focus on that region to acquire high-fidelity datasets. Scale bars: 20 nm.

Abstract Materials design aims to identify the material features that provide optimal properties for various engineering applications, such as aerospace, automotive, and naval. One of the important but challenging problems for materials design is to discover multiple polycrystalline microstructures with optimal properties. This paper proposes an end-to-end artificial intelligence (AI)-driven microstructure optimization framework for elastic properties of materials. In this work, the microstructure is represented by the Orientation Distribution Function (ODF) that determines the volume densities of crystallographic orientations. The framework was evaluated on two crystal systems, cubic and hexagonal, for Titanium (Ti) in Joint Automated Repository for Various Integrated Simulations (JARVIS) database and is expected to be widely applicable for materials with multiple crystal systems. The proposed framework can discover multiple polycrystalline microstructures without compromising the optimal property values and saving significant computational time.

Artificial intelligence based predictive modeling has become increasingly sought-after in the field of materials science for training property prediction models due to their promising ability to extract and utilize data-driven information from materials data. However, current methods typically use limited hand-engineered fixed-length representations obtained from available composition-based information only, making model inputs a stumbling block when handling small and specialized training datasets. In this paper, we study and propose a method to perform representation learning (RL) that is both applicable and adaptive for generalized use across various domains. We introduce a RL technique that utilizes pre-activation based representations extracted from a model pre-trained using a deep neural network to maximize the accuracy. We perform model training for inorganic material properties using composition-based numerical vectors representing the elemental fractions (EF) of the materials by leveraging source models trained on large datasets to build target models on small datasets and then compare its performance against traditional machine learning (ML), deep neural network and RL-based graph neural network (GNN) models trained from scratch (SC) with EF as input, more informative physical attributes (PA) as input, as well as conventional TL/RL techniques. Using large <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\sim 345K)$</tex> datasets for source model training and small computational <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\sim 28K)$</tex> and experimental <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\sim 2K)$</tex> datasets for target model training and testing, we show that the proposed RL methods help significantly improve the accuracy of the model as compared to the SC models and conventional TL/RL techniques for all data sizes and properties by using only EF as input. We also perform a statistical significance analysis by calculating the p-value to find that the observed improvement in the accuracy of proposed RL model over SC, RL-based GNN, and conventional TL/RL models is indeed significant.

Stress-strain curves are important representations of a given material's mechanical properties, which depend primarily on the orientation of the individual crystals in the microstructure. Generating stress-strain curves from numerical methods such as the crystal plasticity finite element (CPFE) simulations is computationally intensive. As a result, it is difficult to generate complete stress-strain curves for all possible orientations of a material. In this work, we propose a bilinear stress-strain curve prediction framework for metallic alloys by integrating supervised and unsupervised deep learning methods via transfer learning principles. As a specific case-study, we focus on predicting stress-strain curves of Nickel (Ni)-based superalloys that have important applications in aerospace industry. Using a small training set of just 100 complete stress-strain curves (4,000 strain steps each) of different orientations generated by CPFE simulation code, we were able to build a model that could accurately predict stress-strain curves (<2 % error) using simple features that could be obtained by running the CPFE simulation for just a single strain step. The proposed model can thus predict the complete stress-strain curve for a given orientation of Ni-based superalloys in a fraction of a second, which amounts to a speedup of over 4000x as compared to the simulation.

Modern machine learning (ML) and deep learning (DL) techniques using high-dimensional data representations have helped accelerate the materials discovery process by efficiently detecting hidden patterns in existing datasets and linking input representations to output properties for a better understanding of the scientific phenomenon. While a deep neural network comprised of fully connected layers has been widely used for materials property prediction, simply creating a deeper model with a large number of layers often faces with vanishing gradient problem, causing a degradation in the performance, thereby limiting usage. In this paper, we study and propose architectural principles to address the question of improving the performance of model training and inference under fixed parametric constraints. Here, we present a general deep-learning framework based on branched residual learning (BRNet) with fully connected layers that can work with any numerical vector-based representation as input to build accurate models to predict materials properties. We perform model training for materials properties using numerical vectors representing different composition-based attributes of the respective materials and compare the performance of the proposed models against traditional ML and existing DL architectures. We find that the proposed models are significantly more accurate than the ML/DL models for all data sizes by using different composition-based attributes as input. Further, branched learning requires fewer parameters and results in faster model training due to better convergence during the training phase than existing neural networks, thereby efficiently building accurate models for predicting materials properties.

The objective of this project is to design, develop, and evaluate scalable software to enhance resilience, data checkpointing, program restart, and analysis. The proposed tasks are to 1) develop scalable machine learning techniques to learn temporal change patterns in a scalable and in-situ manner, and to minimize data movement and maximize learning locally closest to data; 2) design a concise data representation and indexing mechanism to capture the distribution of changes in data that can guarantee point-wise user-defined tolerable errors while reducing the data storage requirements by an order of magnitude or more; 3) develop data reduction techniques as library modules; 4) exploit local SSD for minimizing data movement in storage hierarchy; 5) develop anomaly detection algorithms that can predict corruptions based on learning of emerging patterns; 6) develop software libraries to be incorporated within widely used data formats and APIs; and 7) evaluate the proposed software using DOE scientific applications. The outcomes of the proposed work are to satisfy many synergistic data reduction and resilience requirements for large-scale data intensive applications executed on extreme-scale computing systems. The developed mechanism for error-bound data approximation is directly applicable to existing scientific applications. Through machine learning from historical events and change distribution, this work will enable anomaly detection for DOE computer facility.