Sean Follmer

How do people decide whether one event caused another? While previous research has focused on visual and auditory cues in causal perception, the role of touch remains underexplored. Here, we investigate how haptic feedback contributes to causal judgments across three psychophysical experiments. In Experiment 1, we introduced force-based haptic feedback to a visual launching paradigm and found that haptic information increased causal judgments compared to visual feedback alone. Experiment 2 combined vision, audio, force-based haptics, and vibrotactile haptics, revealing that additional sensory cues increase causal judgments with diminishing returns - the largest benefit comes from adding a second modality. In Experiment 3, we examined how both the number and physical realism of multisensory cues affect causal perception, finding that both factors boosted causal judgments. We present a Bayesian multimodal inference model that captures human judgments by integrating visual, auditory, and haptic information based on their relative timing, uncertainty, and realism. Taken together, these experiments show that haptic information contributes to causal judgments by shaping the multisensory evidence observers use when deciding whether one event physically caused another, including how realistic and physically coherent the event appears. More broadly, we find that temporal alignment and cross-modal coherence are key constraints for how multisensory evidence shapes causal judgments, with implications for virtual reality, robotics, and human-computer interaction systems.

How do people decide whether one event caused another? While previous research has focused on visual and auditory cues in causal perception, the role of touch remains underexplored. Here, we investigate how haptic feedback contributes to causal judgments across three psychophysical experiments. In Experiment 1, we introduced force-based haptic feedback to a visual launching paradigm and found that haptic information increased causal judgments compared to visual feedback alone. Experiment 2 combined vision, audio, force-based haptics, and vibrotactile haptics, revealing that additional sensory cues increase causal judgments with diminishing returns—the largest benefit comes from adding a second modality. In Experiment 3, we examined how both the number and physical realism of multisensory cues affect causal perception, finding that both factors boosted causal judgments. We present a Bayesian multimodal inference model that captures human judgments by integrating visual, auditory, and haptic information based on their relative timing, uncertainty, and realism. Taken together, these experiments show that haptic information contributes to causal judgments by shaping the multisensory evidence observers use when deciding whether one event physically caused another, including how realistic and physically coherent the event appears. More broadly, we find that temporal alignment and cross-modal coherence are key constraints for how multisensory evidence shapes causal judgments, with implications for virtual reality, robotics, and human-computer interaction systems.

How do people decide whether one event caused another? While previous research has focused on visual and auditory cues in causal perception, the role of touch remains underexplored. Here, we investigate how haptic feedback contributes to causal judgments across three psychophysical experiments. In Experiment 1, we introduced force-based haptic feedback to a visual launching paradigm and found that haptic information increased causal judgments compared to visual feedback alone. Experiment 2 combined vision, audio, force-based haptics, and vibrotactile haptics, revealing that additional sensory cues increase causal judgments with diminishing returns—the largest benefit comes from adding a second modality. In Experiment 3, we examined how both the number and physical realism of multisensory cues affect causal perception, finding that both factors boosted causal judgments. We present a Bayesian multimodal inference model that captures human judgments by integrating visual, auditory, and haptic information based on their relative timing, uncertainty, and realism. Taken together, these experiments show that haptic information contributes to causal judgments by shaping the multisensory evidence observers use when deciding whether one event physically caused another, including how realistic and physically coherent the event appears. More broadly, we find that temporal alignment and cross-modal coherence are key constraints for how multisensory evidence shapes causal judgments, with implications for virtual reality, robotics, and human-computer interaction systems.

How do people decide whether one event caused another? While previous research has focused on visual and auditory cues in causal perception, the role of touch remains underexplored. Here, we investigate how haptic feedback contributes to causal judgments across three psychophysical experiments. In Experiment 1, we introduced force-based haptic feedback to a visual launching paradigm and found that haptic information increased causal judgments compared to visual feedback alone. Experiment 2 combined vision, audio, force-based haptics, and vibrotactile haptics, revealing that additional sensory cues increase causal judgments with diminishing returns—the largest benefit comes from adding a second modality. In Experiment 3, we examined how both the number and physical realism of multisensory cues affect causal perception, finding that both factors boosted causal judgments. We present a Bayesian multimodal inference model that captures human judgments by integrating visual, auditory, and haptic information based on their relative timing, uncertainty, and realism. Taken together, these experiments show that haptic information contributes to causal judgments by shaping the multisensory evidence observers use when deciding whether one event physically caused another, including how realistic and physically coherent the event appears. More broadly, we find that temporal alignment and cross-modal coherence are key constraints for how multisensory evidence shapes causal judgments, with implications for virtual reality, robotics, and human-computer interaction systems.

Motivation to perform movement therapy and caregiver burnout are major challenges to post-stroke life. Serious games have been shown to support therapeutic tasks in people with stroke, but there are few activities that simultaneously support informal caregiver health, which is also impacted post-stroke. Here, we present a collaborative, mutually beneficial, serious game designed to support therapy for persons with stroke and also exercise for their informal caregivers. One player performs rehabilitative wrist movements - useful to people with stroke - and the other performs a seated march exercise - useful to informal caregivers - via pedals or a keyboard to control their avatar. We present a pilot study with 6 healthy dyads to evaluate how exercise-based input of one player, the Pseudo Caregiver (PCG), impacts motivation and emotional experience in both the PCG and Pseudo Person with Stroke (PPS). While not statistically significant, we find that PCGs Interest subscale scores trended higher when using a pedal (the exercised-based input) compared to a keyboard, regardless of game play mode. PPSs' positive affect scale scores and Competence subscale scores trended higher when their partner played collaboratively with a pedal compared to a keyboard. These trends encourage future work toward incorporating an exercise-based device, such as a pedal, to enhance the emotional and motivational experience of rehabilitative serious games for people with different movement ability levels.

Motivation to perform movement therapy and caregiver burnout are major challenges to post-stroke life. Serious games have been shown to support therapeutic tasks in people with stroke, but there are few activities that simultaneously support informal caregiver health, which is also impacted post-stroke. Here, we present a collaborative, mutually beneficial, serious game designed to support therapy for persons with stroke and also exercise for their informal caregivers. One player performs rehabilitative wrist movements - useful to people with stroke - and the other performs a seated march exercise - useful to informal caregivers - via pedals or a keyboard to control their avatar. We present a pilot study with 6 healthy dyads to evaluate how exercise-based input of one player, the Pseudo Caregiver (PCG), impacts motivation and emotional experience in both the PCG and Pseudo Person with Stroke (PPS). While not statistically significant, we find that PCGs Interest subscale scores trended higher when using a pedal (the exercised-based input) compared to a keyboard, regardless of game play mode. PPSs' positive affect scale scores and Competence subscale scores trended higher when their partner played collaboratively with a pedal compared to a keyboard. These trends encourage future work toward incorporating an exercise-based device, such as a pedal, to enhance the emotional and motivational experience of rehabilitative serious games for people with different movement ability levels.

As haptic interfaces integrate more seamlessly into wearables and everyday environments, they increasingly require actuators that are soft, thin, silent, and energy efficient. However, conventional motors and temperature-responsive polymers often struggle to deliver these properties due to their bulky form factors and high power consumption. High-Voltage Electrostatic Actuators (HVEAs), which generate force by applying an electric field to localized charge concentrations using high voltages and ultra-low currents, have recently emerged as a compelling alternative due to their fast, silent, and low-power operation within highly customizable and compliant form factors. This paper presents a focused review of HVEAs for haptics, examining four major classes: electrostatic switchable adhesives, dielectric elastomer actuators, soft electrohydraulic actuators, and electrokinetic pumps. For each class, we describe their mechanisms that enable haptic output; characterize their band widths, force densities, and spatial scalability; and evaluate their versatility for rendering cutaneous and kinesthetic feedback across wearable and world-grounded interfaces. Through this cross-technology analysis, we identify common design constraints and emerging strategies for improving ergonomics, streamlining fabrication, and integrating self-sensing. We conclude by out lining where HVEAs are uniquely positioned to advance haptic interaction and highlighting key research directions needed to translate these technologies into practical systems.

Mindfulness is the state of maintaining attention to the present moment with curiosity and openness. Existing mobile technologies to support mindfulness focus on formal practices such as meditation, requiring dedicated space and time. However, everyday mindfulness—a more flexible form of practice woven into routine activities such as washing hands or cooking—remains under-supported. To address this gap, we introduce a wearable device that adopts a sensory-driven approach to foster two key components of mindfulness, attention and curiosity, in everyday contexts. Our device amplifies sounds produced by the user's hand interactions to make them more salient, such as the sounds of hands rubbing together or fingertips sliding across surfaces. By playing back the amplified sounds to the user in real time, the device leads to a fresh perspective on mundane interactions. We conducted a preregistered in-lab study with 60 participants to evaluate the device in an everyday task: object exploration. We found that audio augmentation enhanced self-reported state mindfulness, directing user attention to auditory properties of objects that would otherwise be overlooked. Behaviorally, audio augmentation caused participants to interact with objects for a longer duration than participants who did not experience audio augmentation. We also found that participants exhibited more trial-and-error exploratory behavior patterns with audio augmentation than without, suggesting increased curiosity.

In unexplored or ill-defined environments and operating conditions, robots must be able to adapt to their surroundings. Previously studied robots have achieved such adaptibility through pneumatic shape change and added actuation complexity. Here, we present a novel shape changing robot composed of two motors and two flexible loops that can change shape and locomote, enabling interaction with different environments. By twisting and untwisting two loops against each other, we can create continuous shape change, which in turn impacts the steering and locomotion capabilities of the robot. We detail the design of a flexible loop robot that uses two motors to twist and untwist two loops against each other. Informed by previous work describing the overcurvature of bent flexible loops, we describe the order in which each loop may fold while still maintaining the locomotive abilities of the robot. Further, we built two versions of the robot with which we empirically demonstrate mechanical requirements of folding. Finally, we demonstrated the robot’s ability to locomote in a variety of environments, including on a flat surface, up a slope, and across a rough terrain.

OBJECTIVE: In creating an Agency for Healthcare Research and Quality (AHRQ) Patient Safety Learning Laboratory (PSLL), our objective has been to establish a multidisciplinary research environment focused on the safe care of pregnant women and newborns. This manuscript describes work performed under grants P30 HS023506 (obstetric focus) and R18 HS029123 (neonatal focus). METHODS: We follow AHRQ's 5-step approach to systems engineering in health care: problem analysis, design, development, implementation, and evaluation. Within this 5-step approach, methods used include interviews, focus groups, direct observation, teamwork scales, flow disruption analysis, the Systems Engineering Initiative for Patient Safety model, design thinking, and simulation-based testing of processes and prototypes. RESULTS: Grant P30 HS023506 is completed. The physical characteristics of 10 labor and delivery units were examined, finding significant heterogeneity in size, design, and organization. Task analysis revealed multiple obstacles to optimal team performance. We designed and tested a delayed cord clamping cart to address inherent ergonomic challenges. Finally, we identified common lapses in verbal communication during obstetric emergencies. Grant R18 HS029123 is ongoing. Eighteen Need Statements serve as the basis for exploratory work in mitigating threats to neonates during resuscitation, including a task analysis to determine points of intervention. We are developing (a) novel resuscitation platforms, (b) improved methods of equipment/supply organization, (c) new means of acquiring, displaying, and processing multiple data streams, and (d) innovative techniques and devices for neonatal intubation. CONCLUSIONS: The approach to systems engineering in health care supported by AHRQ's PSLL funding mechanism fosters critical thinking about safety issues by facilitating the integration of investigators with diverse, complementary expertise. By encouraging such collaboration, AHRQ's 5-step process enables important questions to be answered. The PSLL mechanism is a valuable resource for the patient safety community.

We introduce Audio Personas, enabling users to “decorate” themselves with body-anchored sounds in audio augmented reality. Like outfits, makeup, and fragrances, audio personas offer an alternative yet dynamic channel to augment face-to-face interactions. For instance, one can set their audio persona as rain sounds to reflect a bad mood, bee sounds to establish personal boundaries, or a playful “woosh” sound to mimic passing by someone like a breeze. To instantiate the concept, we implemented a headphone-based prototype with multi-user tracking and audio streaming. Our preregistered in-lab study with 64 participants showed that audio personas influenced how participants formed impressions. Individuals with positive audio personas were rated as more socially attractive, more likable, and less threatening than those with negative audio personas. Our study with audio designers revealed that audio personas were preferred in public and semi-public-private spaces for managing social impressions (e.g., personality) and signaling current states (e.g., emotions).

Sonification offers a non-visual way to understand data, with pitch-based encodings being the most common. Yet, how well people perceive slope and acceleration-key features of data trends-remains poorly understood. Drawing on people's natural abilities to perceive tempo, we introduce a novel sampling method for pitch-based sonification to enhance the perception of slope and acceleration in univariate functions. While traditional sonification methods often sample data at uniform x-spacing, yielding notes played at a fixed tempo with variable pitch intervals (Variable Pitch Interval), our approach samples at uniform y-spacing, producing notes with consistent pitch intervals but variable tempo (Variable Tempo). We conducted psychoacoustic experiments to understand slope and acceleration perception across three sampling methods: Variable Pitch Interval, Variable Tempo, and a Continuous (no sampling) baseline. In slope comparison tasks, Variable Tempo was more accurate than the other methods when modulated by the magnitude ratio between slopes. For acceleration perception, just-noticeable differences under Variable Tempo were over 13 times finer than with other methods. Participants also commonly reported higher confidence, lower mental effort, and a stronger preference for Variable Tempo compared to other methods. This work contributes models of slope and acceleration perception across pitch-based sonification techniques, introduces Variable Tempo as a novel and preferred sampling method, and provides promising initial evidence that leveraging timing can lead to more sensitive, accurate, and precise interpretation of derivative-based data features.

Electroadhesive clutches are electrically controllable switchable adhesives commonly used in soft robots and haptic user interfaces. They can form strong bonds to a wide variety of surfaces at low power consumption. However, electroadhesive clutches in the literature engage to and release from substrates several orders of magnitude slower than a traditional electrostatic model would predict. Large release times, in particular, can limit electroadhesion's usefulness in high-bandwidth applications. We develop a novel electromechanical model for electroadhesion, factoring in polarization dynamics, the drive circuitry's rise and fall times, and contact mechanics between the dielectric and substrate. We show in simulation and experimentally how different design parameters affect the engagement and release times of centimeter-scale electroadhesive clutches to metallic substrates, and we find that the model accurately captures the magnitude and trends of our experimental results. In particular, we find that higher drive frequencies, narrower substrate aspect ratios, and faster drive circuitry output stages enable significantly faster release times. The fastest clutches have engagement times less than 15 us and release times less than 875 us, which are 10x and 17.1x faster, respectively, than the best times found in prior literature on centimeter-scale electroadhesive clutches.

Statistical literacy involves understanding, interpreting, and critically evaluating statistical information in a contextually grounded way. Current instructional practices rely heavily on visual techniques, which renders them inaccessible to students who are blind or have low vision (BLV). To bridge this gap, we formed an extended co-design partnership with a statistics teacher, a teacher for students with visual impairments (TVI), and two BLV students to develop accessibility-first practices for building statistical literacy. Through several months of collaboration that included discussion, exploration, design, and evaluation, we identified specific approaches to promote comprehension and engagement. The enactive approaches we designed, using scaffolding and timely feedback, fostered insights through pattern recognition and analogical reasoning. Additionally, inquiry-based methods promoted contextually situated reasoning and reflection on how statistics can improve students’ lives and communities. We present these findings alongside participants’ experiences and discuss their implications for inclusive learning frameworks and tools.

Using vibration feedback on a touchscreen is a promising method to provide blind and low-vision (BLV) users access to graphical content. While prior studies have explored the design space of vibrotactile rendering of graphics, findings do not generalize to complex shapes, and comprehensive standards for vibrotactile graphics comparable to those for tactile graphics are yet to be defined. To address this gap, we present a computational model for non-visual vibrotactile touchscreen exploration using a partially observable Markov decision process (POMDP) framework. Preliminary simulations of a triangle-tracing task demonstrate that empirically observed exploration strategies, such as circling or crossing around a point, emerge as adaptive behaviors under this framework. The model can further incorporate factors such as memory limitations and observation uncertainty, providing a new approach for analyzing exploration behaviors influenced by environmental and user-specific variables. This framework introduces a tool to understand non-visual exploration strategies and inform vibrotactile graphics design.

3D-printing can benefit blind and low-vision (BLV) users by offering tactile access to information typically conveyed visually. Complex printed models often require users to assemble multiple components, a process that relies on visual cues for sighted individuals, while the assembly experiences of BLV users remain poorly understood. In this work, we investigate how BLV users experience and navigate 3D-printed object assembly. We conducted a user study with six BLV participants under three conditions: (1) unassisted, (2) assisted by a sighted mediator, and (3) guided by step-by-step verbal instructions. Verbal instructions helped participants identify, confirm, orient, and fasten components; but challenges locating and distinguishing joints persisted. Co-designing with BLV users identified uses for tactile markers to complement verbal cues and support spatially relevant aspects of the assembly process. Our results highlight the need for assembly instructions tailored to BLV users, and we conclude with recommendations for future research and development.

Electroadhesive clutches are electrically controllable switchable adhesives commonly used in soft robots and haptic user interfaces. They can form strong bonds to a wide variety of surfaces at low power consumption. However, electroadhesive clutches in the literature engage to and release from substrates several orders of magnitude slower than a traditional electrostatic model would predict. Large release times, in particular, can limit electroadhesion's usefulness in high-bandwidth applications. We develop a novel electromechanical model for electroadhesion, factoring in polarization dynamics, the drive circuitry's rise and fall times, and contact mechanics between the dielectric and substrate. We show in simulation and experimentally how different design parameters affect the engagement and release times of centimeter-scale electroadhesive clutches to metallic substrates, and we find that the model accurately captures the magnitude and trends of our experimental results. In particular, we find that higher drive frequencies, narrower substrate aspect ratios, and faster drive circuitry output stages enable significantly faster release times. The fastest clutches have engagement times less than 15 us and release times less than 875 us, which are 10x and 17.1x faster, respectively, than the best times found in prior literature on centimeter-scale electroadhesive clutches.

Shape-changing displays enable real-time visualization and haptic exploration of 3D surfaces. However, many shape-changing displays are composed of individually actuated rigid bodies, which makes them both mechanically complex and unable to form smooth surfaces. In this work, we build a multi-stable curved line display inspired by physical splines. By using circular splines to initialize a discrete elastic rods simulator, we can model multiple stable shapes that fit specific boundary conditions. We then generate actuation instructions based on the circular spline initialization to drive the physical display. We demonstrate our display’s ability to create 16 shapes with 8 different boundary conditions. Our display is consistent in shape output, with an average standard deviation in height of 0.75 mm or 0.47% of the display’s maximum vertical range. We also show that our model is consistent with our display, with a mean RMSE of 6.68 mm or 3.85% of the display’s maximum vertical range for shapes we could stably simulate. We then demonstrate potential scalability by simulating a multi-segment version of the system and show the display’s ability to withstand loads during contour following in haptic exploration.

Interactive data learning tools provide explorable ways for students to build intuitions about data, data representations, and statistical parameters. However, these tools rely on visual consumption and are not accessible to blind and low vision (BLV) students. In this work, we investigate opportunities to leverage active exploration, enriched with multimodal feedback and embodied interaction, to foster an understanding of the relationships among individual data values, data representations, and statistical measures. We explore these opportunities in the form of an accessible learning platform that allows students to hear and feel how statistical measures are changing in real time as they construct and manipulate physicalized data representations. We introduced the platform to four teachers of students with visual impairments (TVIs) through a two-hour-long focus group. TVIs embraced the platform’s exploratory nature and universality and recommended the consideration of additional auditory and texture-based interactions to enhance engagement.

The rise of Mixed Reality (MR) stimulates new interactive techniques that seamlessly blend the virtual and physical environments. Just as virtual content could be overlayed onto the physical world for providing adaptive user interfaces [5, 8], emergent techniques "repurpose" everyday environments and sensory cues to support the virtual content [7, 9, 13-15]. For instance, a strong wind gust in the real world, rather than being distracting to the virtual experience, can be mapped with trees swaying in MR to achieve a unifying experience [15], as shown in Figure 1. Such techniques introduce stronger immersion, but they also expose users to overlooked perceptual manipulations, where safety risks arise from misperception of real-world events. In this work, we apply a causal inference perspective to understand the harms of repurposing real-world sensory cues for MR. We argue that by viewing the MR experience as a causal inference process of interpreting cues arising from both the virtual and physical world, MR designers and researchers can gain a new lens to understand potential perceptual manipulation harms.

Interactive data learning tools provide explorable ways for students to build intuitions about data, data representations, and statistical parameters. However, these tools rely on visual consumption and are not accessible to blind and low vision (BLV) students. In this work, we investigate opportunities to leverage active exploration, enriched with multimodal feedback and embodied interaction, to foster an understanding of the relationships among individual data values, data representations, and statistical measures. We explore these opportunities in the form of an accessible learning platform that allows students to hear and feel how statistical measures are changing in real time as they construct and manipulate physicalized data representations. We introduced the platform to four teachers of students with visual impairments (TVIs) through a two-hour-long focus group. TVIs embraced the platform's exploratory nature and universality and recommended the consideration of additional auditory and texture-based interactions to enhance engagement.

Researchers who build creativity support tools (CSTs) define abstractions and software representations that align with user needs to give users the power to accomplish tasks. However, these specifications also structure and limit how users can and should think, act, and express themselves. Thus, tool designers unavoidably exert power over their users by enacting a “normative ground” through their tools. Drawing on interviews with 11 creative practitioners, tool designers, and CST researchers, we offer a definition of empowerment in the context of creative practice, build a preliminary theory of how power relationships manifest in CSTs, and explain why researchers have had trouble addressing these concepts in the past. We re-examine CST literature through a lens of power and argue that mitigating power imbalances at the level of technical design requires enabling users in both vertical movement along levels of abstraction as well as horizontal movement between tools through interoperable representations. A lens of power is one possible orientation that lets us recognize the methodological shifts required towards building “artistic support tools.”

Interacting in real environments, such as manipulating objects, involves multisensory information. However, little is known about how multisensory cue characteristics help us determine what has occurred in a scene, including whether two events were causally linked. In virtual environments, the number of sensory modalities present and levels of realism often vary. In this work, we explore what role multisensory information and physical realism play in people’s causal perception. So far, haptic cues have rarely been studied in causal perception. Here, we combined visual, auditory, and haptic cues in a psychophysical study in which participants were asked to judge whether one billiard ball caused another to move. We manipulated the temporal delay between cause and effect events, and the physical realism of each cue. While temporal delays generally decreased causal judgments, the number of multisensory cues and their physical realism increased causal judgments. We highlight the implications of this work for building immersive environments.

Shape displays are a class of haptic devices that enable whole-hand haptic exploration of 3D surfaces. However, their scalability is limited by the mechanical complexity and high cost of traditional actuator arrays. In this paper, we propose using electroadhesive auxetic skins as a strain-limiting layer to create programmable shape change in a continuous (“formable crust”) shape display. Auxetic skins are manufactured as flexible printed circuit boards with dielectric-laminated electrodes on each auxetic unit cell (AUC), using monolithic fabrication to lower cost and assembly time. By layering multiple sheets and applying a voltage between electrodes on subsequent layers, electroadhesion locks individual AUCs, achieving a maximum in-plane stiffness variation of 7.6x with a power consumption of 50 <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{\mu \mathrm{W}/\text{AUC}.}$</tex> We first characterize an individual AUC and compare results to a kinematic model. We then validate the ability of a 5x5 AUC array to actively modify its own axial and transverse stiffness. Finally, we demonstrate this array in a continuous shape display as a strain-limiting skin to programmatically modulate the shape output of an inflatable LDPE pouch. Integrating electroadhesion with auxetics enables new capabilities for scalable, low-profile, and low-power control of flexible robotic systems.

Illusory VR interaction techniques such as hand redirection work because humans use vision to adjust their motor commands during movement (e.g., reaching). Existing simulations of redirected reaching are limited, however, and have not yet incorporated important stochastic characteristics like sensorimotor noise, nor captured redirection’s effect on movement duration. In this work, we propose adapting a stochastic optimal feedback control (SOFC) model of normal reach to simulate redirection by augmenting sensory feedback at run-time. We present a summary of our simulation and validate it against user data gathered in multiple redirection conditions. We also evaluate the impacts of visual attention on the effectiveness of redirection in real users and replicate the effects in simulation. Our results show that an infinite-horizon SOFC model is able to reproduce key characteristics of redirected reaches and highlight the benefits of SOFC as a tool for simulating, evaluating, and gaining insights about redirection techniques.