← 返回 Community

Dawn M. Tilbury

Mechanical Engineering · University of Michigan  high

🏠 教授主页iD ORCID

研究方向

方向提炼待补(distill 阶段生成)。

该校申请信息 · University of Michigan

ME deadline(legacy)
申请费

近三年论文 · 48 篇 (点击展开摘要,时间倒序)

Promoting Human–Robot Team Effectiveness: Shared Mental Models and Communication Improve Team Situation Awareness and Performance
IEEE Access · 2026 · cited 0 · doi.org/10.1109/access.2026.3653446
Human-robot teaming can benefit many domains. Teams with sufficient team situation awareness may better accomplish their goals, but team situation awareness can be challenging to develop and maintain. We interpret team situation awareness as the team’s collective understanding of the whole situation at a given time. In order to determine how team situation awareness can be developed and maintained in a human-robot team, we conducted a between-subjects experiment to investigate how shared mental models and communication impact team situation awareness, and how team situation awareness relates to performance. Results from 48 subjects showed the impact of shared mental models is relative to communication. A high shared mental model improved team situation awareness and performance efficiency when there was little communication, while the level of shared mental model was inconsequential when high communication was provided. In addition, team situation awareness was positively related to performance efficiency. The findings indicate that team situation awareness can be achieved through either high communication or a high shared mental model under limited communication, which consequently allows for improved performance.
A Systematic Review of Metrics Measuring Takeover Performance in Conditionally Automated Driving
International Journal of Human-Computer Interaction · 2025 · cited 4 · doi.org/10.1080/10447318.2025.2552863
A particular concern with SAE Level 3 automation is the takeover transition from the automated vehicle to the human driver. In response, research has focused on investigating this transition. However, researchers have used a wide range of metrics to measure takeover performance. The lack of consistency in these metrics poses challenges for synthesizing findings. To address this issue, we conducted a systematic literature review of studies published between January 2009 and December 2019, focusing on the takeover performance metrics. Following prior research, we categorize these metrics into two dimensions: timeliness and quality. Additionally, we summarize the scenarios used to elicit takeover requests and analyze the corresponding maneuvers (braking, lane changing, and lane keeping). The results have shown inconsistencies in calculation and naming conventions of takeover performance metrics. Based on these findings, this study proposes several directions for standardizing definitions and terminology, and advancing toward a unified measure of takeover performance.
Estimating Situation Awareness for Human-Robot Teaming
When humans supervise multiple semi-autonomous robots while also attending to their own tasks simultaneously, they may lack the situation awareness needed to assist their robot teammates. There is a need to monitor the human’s situation awareness in real-time, so interventions can be taken to improve poor situation awareness. While prior work has developed models to estimate human situation awareness, they rely heavily on advanced machine learning models and a single source of input through eye-tracking that can pose operational challenges. We develop a real-time human situation awareness estimator based on data from a human-robot teaming experiment. The situation awareness estimator uses simple and interpretable logistic regression models that take inputs from both eye-tracking and behavioral measures. Cross-validation demonstrated the situation awareness estimator had an average accuracy of 74%. The estimator is robust to missing inputs, and can monitor human situation awareness non-intrusively in real-time.
Human-Autonomy Collaboration for Escaping Local Minima
Effective human supervision of autonomous robots in high-stakes scenarios requires efficient intervention, particularly when unmanned ground vehicles (UGVs) encounter local minima problems. This study investigates user interface designs to support human intervention in resolving such issues without a complete system takeover. We conducted a human-subjects experiment comparing two intervention methods: direct waypoint selection via mouse input and directional commands via arrow keys. Participants supervised two UGVs while simultaneously performing a secondary task, simulating real-world multitasking scenarios. Results demonstrate that mouse-based waypoint selection led to significantly more efficient UGV paths than arrow key controls and was also preferred by participants. Our findings contribute to the design of human-autonomy interfaces.
Hierarchical Sensor-Robot Control for On-Demand Sensing in a Partially Known Environment
To enable industrial robot autonomy without traditional manual programming, current approaches involve a carefully modeled environment or dedicated sensor feedback. This paper explores a novel alternative regime: on-demand sensing, in which a fleet of sensorless robots operating in un-modeled environments adapt to frequently changing repetitive tasks by requesting temporary access to a shared mobile sensor. A Hierarchical Sensor-Robot Control scheme is developed to enable an ad hoc team to cooperatively solve a task, at which point the sensor is dismissed while the robot repeats the task safely in open loop. An outer loop simultaneously optimizes the sensor pose and the parameters of an inner loop robot controller, which is encoded with potential fields. Simulation results demonstrate the algorithm converging for a realistic problem after just three outer-loop iterations.
Requirement-Driven Sharing of Manufacturing Digital Twins Along the Value Chain
Digital Twins (DTs) are key enablers of Smart Manufacturing, yet their adoption across the value chain is hindered by the lack of a standardized sharing framework. This paper addresses this challenge by identifying essential descriptive and qualitative elements of DTs based on standards and literature. Leveraging the Asset Administration Shell (AAS), it proposes a Submodel Template, which standardizes the packaging of DT models, interfaces, and computational and network requirements thus going beyond, and combining, existing AAS Submodels, i.e. for simulation models, to encapsulate the full multidimensionality of DTs. A case study on a Quality Monitoring DT (QM-DT) demonstrates the template’s ability to support seamless DT deployment, aggregation, and operation across heterogeneous manufacturing environments. Results show that the template enables structured transfer of subject matter expertise captured in DT models, real-time constraint support, and interoperability, laying the groundwork for improved DT integration and exchange.
Optimal Feed-Forward and Iterative Learning Control Framework for Enhanced Precision in Extrusion-Based Additive Manufacturing
· 2025 · cited 0 · doi.org/10.1115/msec2025-155844
Abstract Extrusion-based printing is a widely used additive manufacturing (AM) process, typically controlled by motion inputs (stage speed) with a constant extrusion rate. However, relying solely on stage speed can introduce printing resolution errors, particularly in complex pattern regions like sharp angles. This paper develops a modeling framework for line width based on extrusion rate and stage speed, using the control volume concept. An optimal feedforward control framework is then designed to compute control inputs that accurately track line width while respecting system constraints. To refine these inputs, they are adjusted to match the system’s slowest time constant, yielding the final feed-forward signal, which significantly reduces errors in complex regions. To further address modeling limitations, an iterative learning control (ILC) framework is integrated with the feed-forward approach, mitigating errors across both printing and iteration directions. Experimental results strongly align with the model, demonstrating the framework’s reliability. The combined control system effectively enhances line width precision and pattern consistency across individual prints and repeated iterations, even in intricate designs.
Sensing connections in automation, control and robotics
Nature Chemical Engineering · 2025 · cited 2 · doi.org/10.1038/s44286-025-00226-6
Local Minima Prediction using Dynamic Bayesian Filtering for UGV Navigation in Unstructured Environments
arXiv (Cornell University) · 2025 · cited 0 · doi.org/10.48550/arxiv.2505.14337
Path planning is crucial for the navigation of autonomous vehicles, yet these vehicles face challenges in complex and real-world environments. Although a global view may be provided, it is often outdated, necessitating the reliance of Unmanned Ground Vehicles (UGVs) on real-time local information. This reliance on partial information, without considering the global context, can lead to UGVs getting stuck in local minima. This paper develops a method to proactively predict local minima using Dynamic Bayesian filtering, based on the detected obstacles in the local view and the global goal. This approach aims to enhance the autonomous navigation of self-driving vehicles by allowing them to predict potential pitfalls before they get stuck, and either ask for help from a human, or re-plan an alternate trajectory.
Training Human-Robot Teams by Improving Transparency Through a Virtual Spectator Interface
After-action reviews (AARs) are professional discussions that help operators and teams enhance their task performance by analyzing completed missions with peers and professionals. Previous studies comparing different formats of AARs have focused mainly on human teams. However, the inclusion of robotic teammates brings along new challenges in understanding teammate intent and communication. Traditional AAR between human teammates may not be satisfactory for human-robot teams. To address this limitation, we propose a new training review (TR) tool, called the Virtual Spectator Interface (VSI), to enhance human-robot team performance and situational awareness (SA) in a simulated search mission. The proposed VSI primarily utilizes visual feedback to review subjects' behavior. To examine the effectiveness of VSI, we took elements from AAR to conduct our own TR, and designed a 1 <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\times 3$</tex> between-subjects experiment with experimental conditions: TR with (1) VSI, (2) screen recording, and (3) non-technology (only verbal descriptions). The results of our experiments demonstrated that the VSI did not result in significantly better team performance than other conditions. However, the TR with VSI led to more improvement in the subjects' SA over the other conditions.
A Lead-Time-Aware Decomposition Approach to Optimize Disruption Response in Supply Chains
IEEE Transactions on Automation Science and Engineering · 2025 · cited 1 · doi.org/10.1109/tase.2025.3550439
Supply chain (SC) risk management is influenced by both spatial and temporal attributes of different entities (suppliers, retailers, and customers). Each entity has given capacity and lead time to process and transport products to downstream entities. In disruptive events, lead times and capacities may vary, which affects the overall performance of SC. There have been many studies on SC disruption mitigation, but often without considering lead time and the magnitude of lateness. In this paper, we formulate a mixed integer programming (MIP) model to optimize SC operations via a routing and scheduling approach, to model the delivery time of products at different entities as they flow throughout the SC network. We minimize a weighted sum of multiple objectives that involve costs related to transportation, shortages, and delivery lateness. We further develop a Benders decomposition algorithm for speeding up the computation of the NP-hard MIP model. We also develop a discrete-event simulation framework to evaluate the performance of solutions to the MIP model under lead time uncertainty. Through extensive numerical studies, we show how the attributes of SC entities affect the performance, so that we can improve the SC design and operations under various uncertainties.
Training Human-Robot Teams by Improving Transparency Through a Virtual Spectator Interface
Deep Blue (University of Michigan) · 2025 · cited 0 · doi.org/10.7302/25266
After-action reviews (AARs) are professional discussions that help operators and teams enhance their task performance by analyzing completed missions with peers and professionals. Previous studies comparing different formats of AARs have focused mainly on human teams. However, the inclusion of robotic teammates brings along new challenges in understanding teammate intent and communication. Traditional AAR between human teammates may not be satisfactory for human-robot teams. To address this limitation, we propose a new training review (TR) tool, called the Virtual Spectator Interface (VSI), to enhance human-robot team performance and situational awareness (SA) in a simulated search mission. The proposed VSI primarily utilizes visual feedback to review subjects’ behavior. To examine the effectiveness of VSI, we took elements from AAR to conduct our own TR, and designed a 1 × 3 between-subjects experiment with experimental conditions: TR with (1) VSI, (2) screen recording, and (3) non-technology (only verbal descriptions). The results of our experiments demonstrated that the VSI did not result in significantly better team performance than other conditions. However, the TR with VSI led to more improvement in the subjects’ SA over the other conditions.
Digital Twin-Based Smart Manufacturing: Dynamic Line Reconfiguration for Disturbance Handling
IEEE Transactions on Automation Science and Engineering · 2025 · cited 9 · doi.org/10.1109/tase.2025.3563320
The increasing complexity of modern manufacturing, coupled with demand fluctuation, supply chain uncertainties, and product customization, underscores the need for manufacturing systems that can flexibly update their configurations and swiftly adapt to disturbances. However, current research falls short in providing a holistic reconfigurable manufacturing framework that seamlessly monitors system disturbances, optimizes alternative line configurations based on machine capabilities, and automates simulation evaluation for swift adaptations. This paper presents a dynamic manufacturing line reconfiguration framework to handle disturbances that result in operation time changes. The framework incorporates a system process digital twin for monitoring disturbances and triggering reconfigurations, a capability-based ontology model capturing available agent and resource options, a configuration optimizer generating optimal line configurations, and a simulation generation program initializing simulation setups and evaluating line configurations at approximately 400x real-time speed. A case study of a battery production line has been conducted to evaluate the proposed framework. In two implemented disturbance scenarios, the framework successfully recovers system throughput with limited resources, preventing the 26% and 63% throughput drops that would have occurred without a reconfiguration plan. The reconfiguration optimizer efficiently finds optimal solutions, taking an average of 0.03 seconds to find a reconfiguration plan for a manufacturing line with 51 operations and 40 available agents across 8 agent types.
GraspMixer: Hybrid of Contact Surface Sampling and Grasp Feature Mixing for Grasp Synthesis
IEEE Transactions on Automation Science and Engineering · 2025 · cited 2 · doi.org/10.1109/tase.2025.3530795
The capability of robots to rapidly adapt to new tasks without extensive reprogramming offers significant flexibility in reconfiguration of manufacturing processes to cope with unforeseen events. In modern manufacturing environments where numerous hardware and software systems exchange data with each other to perform a myriad of tasks, modularizing sub-systems and reusing commonly available information like product CAD models can increase robustness and efficiency of the reconfiguration. Yet, current approaches for robotic grasping tend to focus on standalone vision-based learning that often require either retraining to adapt to new object categories or massive dataset not available in manufacturing environments, making generalization challenging. This paper addresses the problem of exploiting available information, like CAD models, in manufacturing settings to efficiently generate a tractable set of grasps for known rigid objects, which can be directly applied to a wide class of robotic manipulations. In order to quickly produce diverse grasp configurations for arbitrary geometric models, we present GraspMixer, a combination of (1) an efficient offline sampler that utilizes specifications of a parallel-jaw gripper, and (2) a mapping function that fuses multiple features of a grasp to output a binary quality metric. During evaluation using physics-based simulations, a robotic gripper successfully executes 92.9% of all grasp configurations for 12 novel objects selected by GraspMixer. Among five different grasp sampling methods, GraspMixer also achieves the highest grasp success rate when performing table-top single object grasping under object pose uncertainty. The computation of this offline pipeline takes less than 1.0 minutes for each object without GPU hardware acceleration, which is comparable to or outperforms most of the benchmarks in the evaluation. Importantly, our framework exhibits impressive simulation-to-reality adaptation, achieving over 95% grasp success rate on previously unseen novel objects. All of these results are achieved with fewer than 10% of the samples typically used by other learning-based grasping techniques. Note to Practitioners—Modularization is a major theme in current manufacturing systems to increase efficiency. In this work, we introduce a new framework called GraspMixer, which is part of a larger manipulation and decision making architecture to enable versatile robotic manipulation in a manufacturing environment. The framework decomposes the task of reasoning about graspable local surfaces on object 3D models into sequentially connected sub-components. GraspMixer leverages information about objects and grippers, including their 3D models, materials, and inertial properties, which are available in a manufacturing environment. This enables our framework to automatically precompute grasping points on new objects that can be shared among multiple robots equipped with parallel-jaw grippers. GraspMixer synergizes with Internet of Things (IoT) and Cloud Computing platforms to efficiently scale up advanced robotic automation in manufacturing. Such a combination could provide greater flexibility in deploying advanced perception systems in a manufacturing environment to accelerate adaptation of the automation while saving computational resources of onboard processors within robots.
Local Minima Prediction using Dynamic Bayesian Filtering for UGV Navigation in Unstructured Environments
IFAC-PapersOnLine · 2025 · cited 0 · doi.org/10.1016/j.ifacol.2025.10.244
Path planning is crucial for the navigation of autonomous vehicles, yet these vehicles face challenges in complex and real-world environments. Although a global view may be provided, it is often outdated, necessitating the reliance of Unmanned Ground Vehicles (UGVs) on real-time local information. This reliance on partial information, without considering the global context, can lead to UGVs getting stuck in local minima. This paper develops a method to proactively predict local minima using Dynamic Bayesian filtering, based on the detected obstacles in the local view and the global goal. This approach aims to enhance the autonomous navigation of self-driving vehicles by allowing them to predict potential pitfalls before they get stuck, and either ask for help from a human, or re-plan an alternate trajectory.
Supply Chain Design Optimization With Heterogeneous Risk-Aware Agents
IEEE Transactions on Automation Science and Engineering · 2024 · cited 1 · doi.org/10.1109/tase.2024.3513815
Modern supply chain networks (SCN) are becoming increasingly complex, with vulnerable entities exposed to uncertain disruptions that affect local or global supply chain attributes. We model a stochastic mixed-integer program to minimize the overall cost of SCN design and operations, in response to lead-time and demand uncertainties following given probability distributions. We formulate a heterogeneous risk-aware model to trade off between cost and delay/shortage by considering different risk-attitudes amongst supply chain agents. In particular, we employ the Conditional Value-at-Risk (CVaR) as a coherent risk measure for quantifying risk while attaining solution tractability. We derive managerial insights from our numerical studies, finding the most benefit from diversifying agents in the root tier, since their disruptions affect all other tiers in the SCN. We find that as agents become more risk averse, the optimal solutions for key agents (such as assemblers), seek more backup suppliers and allocate extra capacities to achieve resiliency and reliability. Practitioners can use the outcomes of our framework and studies to guide SCN design considering heterogeneous risk attitudes between agents. Note to Practitioners—With growing uncertainties in global supply chains, inefficient responses to disruptions can lead to large penalties and long-term impacts such as customer dissatisfaction. This research is motivated by the challenges arising during the operations of supply chains under both lead-time and demand uncertainties. We employ optimization and centralized control approaches to optimize supply-chain network design as well as response strategies to disruptions, and our framework can handle heterogeneous risk preferences as it models the risk attitude of each individual entity or agent in supply chains. Our model can be utilized to completely or partially re-design resilient supply chains, to better prepare for unknown features and uncertainties. Our case study provides insights about risk-averse supply-chain designs that can reduce response cost, but increase initial investments on backups and redundancies.
Interoperability of Digital Twins: Challenges, Success Factors, and Future Research Directions
Lecture notes in computer science · 2024 · cited 25 · doi.org/10.1007/978-3-031-75390-9_3
Supporting Driver Attention Toward Potential Hazards During Takeover: A Preliminary Result
Proceedings of the Human Factors and Ergonomics Society Annual Meeting · 2024 · cited 1 · doi.org/10.1177/10711813241275917
This study investigates the impact of a supportive system on takeover transitions in conditionally automated driving (SAE Level 3). The supportive system is designed to direct drivers’ attention toward potential hazards in the environment when a takeover request occurs. The study comprises two components: (a) identifying various types of potential hazards using naturalistic driving data, and (b) conducting a driving simulator study to develop and assess a gaze guidance system based on the N-SEEV model of visual attention. Results indicate that drivers using a highly salient attention guidance system were less likely to collide with a secondary hazard during takeover transitions. This suggests that gaze guidance support is an effective approach for assisting drivers during takeover transitions.
Toward Integrated Takeover Performance Measurement: Validation of Fréchet Distance as a Takeover Performance Metric
Proceedings of the Human Factors and Ergonomics Society Annual Meeting · 2024 · cited 0 · doi.org/10.1177/10711813241266831
This study introduces and validates a new metric for assessing takeover performance in conditionally automated driving, using Fréchet Distance. Fréchet Distance is a measurement that measures the similarity between two separate curves. Thirty-two participants took part in a simulated driving experiment. Employing a 2 × 2 within-subjects design, the study compared traditional takeover performance metrics, including takeover time, time to collision, and resulting acceleration, with Fréchet Distance. Analysis results revealed similar trends between traditional metrics and Fréchet Distance. These findings suggest that Fréchet Distance can effectively measure takeover performance by integrating spatial and temporal aspects.
PRF: A Program Reuse Framework for Automated Programming by Learning from Existing Robot Programs
Robotics · 2024 · cited 0 · doi.org/10.3390/robotics13080118
This paper explores the problem of automated robot program generation from limited historical data when neither accurate geometric environmental models nor online vision feedback are available. The Program Reuse Framework (PRF) is developed, which uses expert-defined motion classes, a novel data structure introduced in this work, to learn affordances, workspaces, and skills from historical data. Historical data comprise raw robot joint trajectories and descriptions of the robot task being completed. Given new tasks, motion classes are then used again to formulate an optimization problem capable of generating new open-loop, skill-based programs to complete the tasks. To cope with a lack of geometric models, a technique to learn safe workspaces from demonstrations is developed, allowing the risk of new programs to be estimated before execution. A new learnable motion primitive for redundant manipulators is introduced, called a redundancy dynamical movement primitive, which enables new end-effector goals to be reached while mimicking the whole-arm behavior of a demonstration. A mobile manipulator part transportation task is used throughout to illustrate each step of the framework.
Iterative learning spatial height control for layerwise processes
Automatica · 2024 · cited 5 · doi.org/10.1016/j.automatica.2024.111756
Sequential Manipulation of Deformable Linear Object Networks with Endpoint Pose Measurements using Adaptive Model Predictive Control
Robotic manipulation of deformable linear objects (DLOs) is an active area of research, though emerging applications, like automotive wire harness installation, introduce constraints that have not been considered in prior work. Confined workspaces and limited visibility complicate prior assumptions of multi-robot manipulation and direct measurement of DLO configuration (state). This work focuses on single-arm manipulation of stiff DLOs (StDLOs) connected to form a DLO network (DLON), for which the measurements (output) are the endpoint poses of the DLON, which are subject to unknown dynamics during manipulation. To demonstrate feasibility of output-based control without state estimation, direct input-output dynamics are shown to exist by training neural network models on simulated trajectories. Output dynamics are then approximated with polynomials and found to contain well-known rigid body dynamics terms. A composite model consisting of a rigid body model and an online data-driven residual is developed, which predicts output dynamics more accurately than either model alone, and without prior experience with the system. An adaptive model predictive controller is developed with the composite model for DLON manipulation, which completes DLON installation tasks, both in simulation and with a physical automotive wire harness.
Heterogeneous Risk Management Using a Multi-Agent Framework for Supply Chain Disruption Response
IEEE Robotics and Automation Letters · 2024 · cited 3 · doi.org/10.1109/lra.2024.3388838
In the highly complex and stochastic global, supply chain environments, local enterprise agents seek distributed and dynamic strategies for agile responses to disruptions. Existing literature explores both centralized and distributed approaches, while most work neglects temporal dynamics and the heterogeneity of the risk management of individual agents. To address this gap, this paper presents a heterogeneous risk management mechanism to incorporate uncertainties and risk attitudes into agent communication and decision-making strategy. Hence, this approach empowers enterprises to handle disruptions in stochastic environments in a distributed way, and in particular in the context of multi-agent control and management. Through a simulated case study, we showcase the feasibility and effectiveness of the proposed approach under stochastic settings and how the decision of disruption responses changes when agents hold various risk attitudes.
A data-driven approach toward a machine- and system-level performance monitoring digital twin for production lines
Computers in Industry · 2024 · cited 25 · doi.org/10.1016/j.compind.2024.104086
Efficient performance monitoring in production systems holds paramount importance as it enables organizations to optimize their manufacturing processes, enhance productivity, and maintain a competitive edge in the market. Typically, machine and system level performance monitoring systems are investigated independently, whereas an integrated approach that considers both levels can offer valuable insights and benefits. This paper introduces a data-driven approach for evaluating and improving the performance of production lines by monitoring the performance of both individual machines and their interactions as a system. The approach begins with a rigorous methodology for classifying machine states recorded by the Manufacturing Execution System (MES) into finer-grained substates, enabling a comprehensive analysis of machine cycle time variability. Subsequently, these substates are leveraged as a foundation for constructing performance monitoring models at both the machine and system levels, employing probabilistic automata for the machine level and logistic regression for the system level. The system-level performance monitoring model is constructed to predict a Flow metric that enables the prediction of abnormal behaviors and deviations from production targets. This data-driven approach serves as a foundational ingredient of a system-level digital twin, designed to provide production lines with insights that enable proactive implementation of measures aimed at optimizing overall manufacturing efficiency. Through an industrial test case from the automotive industry, the results demonstrate the capability of performance monitoring, capturing errors within confidence intervals, and establishing predictive cause-and-effect relationships between machines within the production system.
Toward Personalized Tour-Guide Robot: Adaptive Content Planner based on Visitor's Engagement
· 2024 · cited 10 · doi.org/10.1145/3610978.3640731
In the evolving landscape of human-robot interactions, tour-guide robots are increasingly being integrated into various settings. However, the existing paradigm of these robots relies heavily on pre-recorded content, which limits effective engagement with visitors. We propose to address this issue of visitor engagement by transforming tour-guide robots into dynamic, adaptable companions that cater to individual visitor needs and preferences. Our primary objective is to enhance visitor engagement during tours through a robotic system capable of assessing and reacting to visitor preference and engagement. Leveraging this data, the system can calibrate and adapt the tour-guide robot's content in real-time to meet individual visitor preferences. Through this research, we aim to enhance the tour-guide robots' impact in delivering engaging and personalized visitor experiences by providing an adaptive tour-guide robot solution that can learn from humans' preferences and adapt its behaviors by itself.
Spot Report: An Open-Source and Real-Time Secondary Task for Human-Robot Interaction User Experiments
· 2024 · cited 7 · doi.org/10.1145/3610978.3640718
The human-robot interaction (HRI) community is interested in a range of research questions, many of which are investigated through user experiments. Robots that occasionally require human input allow for humans to engage in secondary tasks. However, few secondary tasks transmit data in real-time and are openly available, which hinders interaction with the primary task and limits the ability of the community to build upon others' research. Also, the need for a secondary task relevant to the military was identified by subject matter experts. To address these concerns, this paper presents the spot report task as an open-source secondary task with real-time communication for use in HRI experiments. The spot report task requires counting target objects in static images. This paper includes details of the spot report task and real-time communication with a primary task. We developed the spot report task considering the military domain, but the software architecture is domain-independent. We hope others can leverage the spot report task in their own user experiments. The software files are available at https://github.com/UMich-MAVRIC/SpotReport.
Sequential Manipulation of Deformable Linear Object Networks with Endpoint Pose Measurements using Adaptive Model Predictive Control
arXiv (Cornell University) · 2024 · cited 0 · doi.org/10.48550/arxiv.2402.10372
Robotic manipulation of deformable linear objects (DLOs) is an active area of research, though emerging applications, like automotive wire harness installation, introduce constraints that have not been considered in prior work. Confined workspaces and limited visibility complicate prior assumptions of multi-robot manipulation and direct measurement of DLO configuration (state). This work focuses on single-arm manipulation of stiff DLOs (StDLOs) connected to form a DLO network (DLON), for which the measurements (output) are the endpoint poses of the DLON, which are subject to unknown dynamics during manipulation. To demonstrate feasibility of output-based control without state estimation, direct input-output dynamics are shown to exist by training neural network models on simulated trajectories. Output dynamics are then approximated with polynomials and found to contain well-known rigid body dynamics terms. A composite model consisting of a rigid body model and an online data-driven residual is developed, which predicts output dynamics more accurately than either model alone, and without prior experience with the system. An adaptive model predictive controller is developed with the composite model for DLON manipulation, which completes DLON installation tasks, both in simulation and with a physical automotive wire harness.
NSF Assistant Directors-Emerging Trends and Programs
· 2024 · cited 0 · doi.org/10.18260/1-2-60-31861
Behavioral and physiological responses to takeovers in different scenarios during conditionally automated driving
Transportation Research Part F Traffic Psychology and Behaviour · 2024 · cited 24 · doi.org/10.1016/j.trf.2024.01.008
ABSTRACT
Behavioral and Physiological Responses to Takeovers in Different Scenarios during Conditionally Automated Driving
Deep Blue (University of Michigan) · 2024 · cited 0 · doi.org/10.7302/22038
A variety of takeover scenarios will happen in conditionally automated driving. Previous studies presented mixed results regarding the effects of scenarios on takeover performance. According to drivers’ strategies for takeover requests, this study selected eight representative takeover scenarios and categorized them into lane-keeping and lane-changing scenarios. To investigate the effects of scenario type and road environment (highway vs. urban) on drivers’ takeover performance and physiological responses, a driving simulation study was conducted as a mixed design with 40 participants (average age = 22.8 years). The results showed that in lane-changing scenarios, with the same sensing capability, drivers on highways had deteriorated takeover performance in the form of harsher takeover maneuvers and higher collision risk, as well as higher arousal and stress, compared to urban areas. However, such effects disappeared or even reversed in lane-keeping scenarios on the curves, where drivers on highways had smoother takeover maneuvers and lower arousal and stress. These findings will help us understand the vital roles scenario type and road environment play during takeover transitions. Our findings have implications for the design of advanced driver-assistance systems and will improve driving safety in conditionally automated driving.
Digital Twin Design and Cross Process Model Transfer for Additive Manufacturing
IFAC-PapersOnLine · 2024 · cited 1 · doi.org/10.1016/j.ifacol.2025.01.106
Digital twins serve as powerful tools for monitoring the status and anticipating errors within manufacturing processes. However, the creation of these digital twins entails a significant investment in time and resources. This paper delves into the process of crafting digital twins for material extrusion additive manufacturing. Furthermore, it investigates the application of cross model transfer learning to such digital twins, and assesses the feasibility in transferring digital twins rather than building them from the ground up.
A Framework for Modeling and Control for Extrusion-based Additive Manufacturing
IFAC-PapersOnLine · 2024 · cited 0 · doi.org/10.1016/j.ifacol.2024.12.024
Additive manufacturing (AM) processes have experienced a surge in demand, largely driven by the growing need for customization across various industries. This paradigm shift underscores the need for more robust AM processes, where precise customization is vital to meet individual requirements. However, there does not exist a structured framework for deriving models that characterize the relationship between process parameters and pattern characteristics. Additionally, determining critical inputs and outputs, as well as control strategies for in situ AM control, further compounds this challenge. This work addresses these challenges by proposing a structured approach to closing the loop for an extrusion-based AM printing process, demonstrating its transferability across multiple printers within the same AM family. The control framework was tested under scenarios of nozzle clogging and over-extrusion, demonstrating effective error mitigation and rapid convergence with appropriate controller gain designs. Moreover, The framework was successfully transferred to a different extrusion printer, showing its adaptability across varied environments and printers, offering insights into the underlying dynamics of this extrusion-based AM process, and paving the way for enhanced performance and reliability in AM.
A Distributed Approach for Agile Supply Chain Decision-Making Based on Network Attributes
IEEE Transactions on Automation Science and Engineering · 2023 · cited 9 · doi.org/10.1109/tase.2023.3339171
In recent years, the frequent occurrence of disruptions has had a negative impact on global supply chains. To stay competitive, enterprises strive to remain agile through the implementation of efficient and effective decision-making strategies in reaction to disruptions. A significant effort has been made to develop these agile disruption mitigation approaches, leveraging both centralized and distributed decision-making strategies. Though trade-offs of centralized and distributed approaches have been analyzed in existing studies, no related work has been found on understanding supply chain performance based on the network <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">attributes</i> of the disrupted supply chain entities. In this paper, we characterize supply chains from a capability and network topological perspective and investigate the use of a distributed decision-making approach based on classical multi-agent frameworks. The performance of the distributed framework is evaluated through a comprehensive case study that investigates the performance of the supply chain as a function of the network structure and agent attributes within the network in the presence of a disruption. Comparison to a centralized decision-making approach highlights trade-offs between performance, computation time, and network communication based on the decision-making strategy and network architecture. Practitioners can use the outcomes of our studies to design response strategies based on agent capabilities, network attributes, and desired supply chain performance. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This research is motivated by the challenges in determining agile decision-making strategies that enable a supply chain enterprise to adapt to disruptions while taking into account the network-based attributes of the disrupted agent and the requirements of the supply chain system. Existing approaches in the literature focus on providing one feasible decision-making strategy based on specific performance metrics. This paper investigates both centralized and distributed approaches to better understand the differences between the response strategies in the case of supplier loss. More specifically, we design a supply chain instance and conduct a case study to evaluate the performance of the centralized and distributed approaches in terms of several common performance metrics used in practice. The case study provides insights for users to select a decision-making approach based on the network attributes and agent capabilities of the supply chain. The impact of network uncertainties and risk assessment are not considered in this work. Future studies will investigate a stochastic supply chain environment and heterogeneous risk management framework in the context of agile decision-making for disrupted supply chain enterprises.
Promises and Trust Repair in UGVs
Proceedings of the Human Factors and Ergonomics Society Annual Meeting · 2023 · cited 2 · doi.org/10.1177/21695067231196235
Unmanned ground vehicles (UGVs) are autonomous robots capable of performing tasks through self- navigation and decision-making. They have the potential to replace humans in dangerous driving scenarios. However, UGVs must be viewed as trustworthy to be accepted, and like any automation, they can make mistakes that decrease human trust in them. Trust repair strategies can mitigate the consequences of trust violations, but they are not always effective. To better understand their effectiveness on UGVs, we designed a between-subjects study examining promises on a UGV’s trustworthiness. Preliminary results showed that promises had a marginal impact on overall trustworthiness but were influential in repairing benevolence but not ability or integrity. These findings have implications for the design of UGV’s and trust repair theory.
A Multi-Objective Mixed-Integer Programming Approach for Supply Chain Disruption Response with Lead-Time Awareness
Supply chain (SC) risk management is influenced by both spatial and temporal attributes of different entities (suppliers, retailers, and customers). Each entity has given capacity and lead time for processing and transporting products to downstream entities. Under disruptive events, lead time and capacities may vary, which affects the overall SC performance. There have been many studies on SC disruption mitigation, but often without considering lead time and the magnitude of lateness. In this paper, we formulate a mixed-integer programming (MIP) model to optimize SC operations via a routing and scheduling approach, to model the delivery time of products at different entities as they flow throughout the SC network. We minimize a weighted sum of multiple objectives involving costs related to transportation, shortage, and delivery lateness. We also develop a discrete-event simulation framework to evaluate the performance of solutions to the MIP model under lead time uncertainty. Via extensive numerical studies, we show how the attributes of SC entities affect the performance, so that we can improve SC design and operations under various uncertainties.
Using Economic Iterative Learning Control for Time-Optimal Control of a Redundant Manipulator
Industrial manipulators are deployed for a range of repetitive tasks in cluttered environments in which the robot must rapidly execute safe trajectories. While nominal robot models exist, true dynamic models of deployed manipulators are typically unavailable. This paper addresses the problem of generating dynamically feasible, collision-free, time-optimal kinematic reference signals for redundant manipulators with unknown dynamics. A novel economic iterative learning control approach is developed to leverage repeated task executions to learn a time-optimal control signal for an uncertain robot model. Simulation results demonstrate the performance of the approach for a 7-DOF manipulator. An experimental analysis is performed to understand the impact of the initial reference trajectory on converged performance.
An Adaptive, State-Based Framework for Fault Prediction in Rotating Equipment
Predictive modeling of industrial rotating equipment is difficult due to a number of implementation challenges. Existing approaches are not well-equipped to adapt to the range of degradation trends that industrial equipment experience and typically depend on large historical datasets for model training. In this paper, a state-based framework for modeling multi-stage degradation in rotating equipment is introduced. With this approach, machine experts can define multi-stage models of equipment health based on first principles or pre-existing knowledge in place of extensive historical data. An adaptive methodology is presented to estimate a machine's health stage history and predict faults based on real-time sensor measurements. A case study with rolling element bearings shows that the proposed approach makes fault predictions with similar accuracy rates as data-driven models trained on multiple historical faults while providing additional degradation detection capabilities.
A Multi-objective Mixed-integer Programming Approach for Supply Chain Disruption Response with Lead-Time Awareness
arXiv (Cornell University) · 2023 · cited 0 · doi.org/10.48550/arxiv.2308.02687
Supply chain (SC) risk management is influenced by both spatial and temporal attributes of different entities (suppliers, retailers, and customers). Each entity has given capacity and lead time for processing and transporting products to downstream entities. Under disruptive events, lead time and capacities may vary, which affects the overall SC performance. There have been many studies on SC disruption mitigation, but often without considering lead time and the magnitude of lateness. In this paper, we formulate a mixed-integer programming (MIP) model to optimize SC operations via a routing and scheduling approach, to model the delivery time of products at different entities as they flow throughout the SC network. We minimize a weighted sum of multiple objectives involving costs related to transportation, shortage, and delivery lateness. We also develop a discrete-event simulation framework to evaluate the performance of solutions to the MIP model under lead time uncertainty. Via extensive numerical studies, we show how the attributes of SC entities affect the performance, so that we can improve SC design and operations under various uncertainties.
Opportunities and challenges in applying reinforcement learning to robotic manipulation: An industrial case study
Manufacturing Letters · 2023 · cited 4 · doi.org/10.1016/j.mfglet.2023.08.055
Full Stack Virtual Commissioning: Requirements Framework to Bridge Gaps in Current Virtual Commissioning Process
· 2023 · cited 3 · doi.org/10.1115/msec2023-106670
Abstract Commissioning forms a crucial part of the initiation of a new manufacturing system. Commissioning activities occur after the design of a new manufacturing system or product line, but before the system is built and deployed into production. Current commissioning practices involve virtual commissioning (VC) tools with limited verification abilities, where virtual models representing a manufacturing system are built and then used in simulation. Software-In-The-Loop (SIL) simulation is applied to virtual models to measure and verify performance metrics that are important to a manufacturer. Due to inaccuracies and gaps in current VC capabilities, physical commissioning is still a common practice for complete verification and validation of a manufacturing system prior to deployment. The need for physical commissioning requires valuable time during the commissioning process. A Full Stack Virtual Commissioning process would eliminate the need for physical commissioning, allowing system integrators to more rapidly commission new manufacturing systems. In this work, we provide an initial Requirements Framework that outlines the critical technological and research gaps that must be addressed in order to achieve a transition toward a Full Stack Virtual Commissioning process. The current state-of-the-art approaches for commissioning are presented, along with an introduction to digital twin technology as an important VC enabler that is being explored within the academic and industrial domains. Lastly we highlight key benefits, important technical gaps, and critical challenges that must be addressed in order to facilitate a transition to Full Stack Virtual Commissioning.