Leah Chong

Abstract User needs identification is a critical early step in the engineering design process, as it defines key design opportunities and goals. Observation is one of the most widely used methods, although its effectiveness is often constrained by its time-intensive nature and susceptibility to bias. To address these challenges, this paper introduces an AI-based observation tool that analyzes videos of user activity to assist the ergonomic need identification process. With ergonomic considerations, designers can better understand and satisfy user needs, leading to novel design solutions with improved usability and inclusiveness. Using computer vision techniques, including pose estimation and object segmentation, our tool assesses user posture and interactions with objects to detect physical discomfort. A human subject experiment was conducted as a formative investigation to evaluate the tool’s impact on the user needs identification process and outcome. While the tool did not significantly affect the number or types of needs identified, it improved the level of detail and efficiency in the identification process. Additionally, when using the tool, the participants reported higher confidence in their latent need finding performance. These findings suggest that our AI-based observation tool has potential to successfully assist designers by streamlining ergonomic analysis and enhancing user-centered design practices. Future improvements will focus on expanding the tool’s capabilities to detect emotional and psychological needs and refining its user interface.

In the digital economy, data is regarded a critical resource for value creation, while digital technologies are reshaping how values are reflected and enacted in society. This transformation demands new frameworks for understanding both value creation and destruction. Yet, research in HCI reveals that these processes are far more complex than simple resource exploitation, posing significant theoretical and practical challenges. In this one-day, in-person workshop, we aim to deepen our understanding of the complexities of design practice, while envisioning the future of mindful value destruction in human-data interaction. Our goal is to provide an unapologetically honest platform for broader public discourse on the real societal, ethical and environmental impact of design and its unintended consequences.

Abstract Text-to-image models are enabling efficient design space exploration, rapidly generating images from text prompts. However, many generative AI tools are imperfect for product design applications as they are not built for the goals and requirements of product design. The unclear link between text input and image output further complicates their application. This work empirically investigates design space exploration strategies that can successfully yield product images that are feasible, novel and aesthetic – three common goals in product design. Specifically, users’ actions within the global and local editing modes, including their time spent, prompt length, mono versus multi-criteria prompts, and goal orientation of prompts, are analyzed. Key findings reveal the pivotal role of mono versus multi-criteria and goal orientation of prompts in achieving specific design goals over time and prompt length. The study recommends prioritizing the use of multi-criteria prompts for feasibility and novelty during global editing while favoring mono-criteria prompts for aesthetics during local editing. Overall, this article underscores the nuanced relationship between the AI-driven text-to-image models and their effectiveness in product design, urging designers to carefully structure prompts during different editing modes to better meet the unique demands of product design.

Abstract In human-centered design, developing a comprehensive and in-depth understanding of user experiences—empathic understanding—is paramount for designing products that truly meet human needs. Nevertheless, accurately comprehending the real underlying mental states of a large human population remains a significant challenge today. This difficulty mainly arises from the tradeoff between depth and scale of user experience research: gaining in-depth insights from a small group of users does not easily scale to a larger population, and vice versa. This paper investigates the use of large language models (LLMs) for performing mental inference tasks, specifically inferring users' underlying goals and fundamental psychological needs (FPNs). Baseline and benchmark datasets were collected from human users and designers to develop an empathic accuracy metric for measuring the mental inference performance of LLMs. The empathic accuracy of inferring goals and FPNs of different LLMs with varied zero-shot prompt engineering techniques are experimented against that of human designers. Experimental results suggest that LLMs can infer and understand the underlying goals and FPNs of users with performance comparable to that of human designers, suggesting a promising avenue for enhancing the scalability of empathic design approaches through the integration of advanced artificial intelligence technologies. This work has the potential to significantly augment the toolkit available to designers during human-centered design, enabling the development of both large-scale and in-depth understanding of users' experiences.

Abstract Current generative design tools backed by artificial intelligence (AI) primarily allow for quantitative inputs while qualitative aspects of a design, in particular aesthetics, have been shown to be considered indirectly by designers. To explore this further, controlled lab experiments were conducted to understand how designers incorporate quantitative and qualitative objectives while using generative design tools and how their behavior may affect design performance. Thirty-four participants completed a design task with quantitative and qualitative objectives with and without generative design tools. The outcomes produced using generative design tools displayed a greater aesthetic diversity and expanded a larger portion of the objective space compared to those without using a generative design tool. Participants also expressed the ability to focus on the qualitative objectives by delegating the quantitative objective to the generative design tool. This showcases the potential for high-performing generative design tools to assist human designers by alleviating part of their cognitive load when balancing multiple objectives, giving them the bandwidth to focus on other objectives not fully incorporated by the tool. In this way, leveraging the expertise of both the human designer and the generative design tool can allow for greater consideration of various objectives throughout the design process.

Abstract In human-centered design, developing a comprehensive and in-depth understanding of user experiences — empathic understanding — is paramount for designing products that truly meet human needs. Nevertheless, accurately comprehending the real underlying mental states of a large human population remains a significant challenge today. This difficulty mainly arises from the trade-off between depth and scale of user experience research: gaining in-depth insights from a small group of users does not easily scale to a larger population, and vice versa. This paper investigates the use of Large Language Models (LLMs) for performing mental inference tasks, specifically inferring users’ underlying goals and fundamental psychological needs (FPNs). Baseline and benchmark datasets were collected from human users and designers to develop an empathic accuracy metric for measuring the mental inference performance of LLMs. The empathic accuracy of inferring goals and FPNs of different LLMs with varied zero-shot prompt engineering techniques are experimented against that of human designers. Experimental results suggest that LLMs can infer and understand the underlying goals and FPNs of users with performance comparable to that of human designers, suggesting a promising avenue for enhancing the scalability of empathic design approaches through the integration of advanced artificial intelligence technologies. This work has the potential to significantly augment the toolkit available to designers during human-centered design, enabling the development of both large-scale and in-depth understanding of users’ experiences.

Abstract Text-to-image generative models have increasingly been used to assist designers during concept generation in various creative domains, such as graphic design, user interface design, and fashion design. However, their applications in engineering design remain limited due to the models’ challenges in generating images of feasible designs concepts. To address this issue, this paper introduces a method that improves the design feasibility by prompting the generation with feasible CAD images. In this work, the usefulness of this method is investigated through a case study with a bike design task using an off-the-shelf text-to-image model, Stable Diffusion 2.1. A diverse set of bike designs are produced in seven different generation settings with varying CAD image prompting weights, and these designs are evaluated on their perceived feasibility and novelty. Results demonstrate that the CAD image prompting successfully helps text-to-image models like Stable Diffusion 2.1 create visibly more feasible design images. While a general tradeoff is observed between feasibility and novelty, when the prompting weight is kept low around 0.35, the design feasibility is significantly improved while its novelty remains on par with those generated by text prompts alone. The insights from this case study offer some guidelines for selecting the appropriate CAD image prompting weight for different stages of the engineering design process. When utilized effectively, our CAD image prompting method opens doors to a wider range of applications of text-to-image models in engineering design.

Text-to-image models are enabling efficient design space exploration, rapidly generating images from text prompts. However, many generative AI tools are imperfect for product design applications as they are not built for the goals and requirements of product design. The unclear link between text input and image output further complicates their application. This work empirically investigates design space exploration strategies that can successfully yield product images that are feasible, novel, and aesthetic, which are three common goals in product design. Specifically, user actions within the global and local editing modes, including their time spent, prompt length, mono vs. multi-criteria prompts, and goal orientation of prompts, are analyzed. Key findings reveal the pivotal role of mono vs. multi-criteria and goal orientation of prompts in achieving specific design goals over time and prompt length. The study recommends prioritizing the use of multi-criteria prompts for feasibility and novelty during global editing, while favoring mono-criteria prompts for aesthetics during local editing. Overall, this paper underscores the nuanced relationship between the AI-driven text-to-image models and their effectiveness in product design, urging designers to carefully structure prompts during different editing modes to better meet the unique demands of product design.

Text-to-image generative models have increasingly been used to assist designers during concept generation in various creative domains, such as graphic design, user interface design, and fashion design. However, their applications in engineering design remain limited due to the models' challenges in generating images of feasible designs concepts. To address this issue, this paper introduces a method that improves the design feasibility by prompting the generation with feasible CAD images. In this work, the usefulness of this method is investigated through a case study with a bike design task using an off-the-shelf text-to-image model, Stable Diffusion 2.1. A diverse set of bike designs are produced in seven different generation settings with varying CAD image prompting weights, and these designs are evaluated on their perceived feasibility and novelty. Results demonstrate that the CAD image prompting successfully helps text-to-image models like Stable Diffusion 2.1 create visibly more feasible design images. While a general tradeoff is observed between feasibility and novelty, when the prompting weight is kept low around 0.35, the design feasibility is significantly improved while its novelty remains on par with those generated by text prompts alone. The insights from this case study offer some guidelines for selecting the appropriate CAD image prompting weight for different stages of the engineering design process. When utilized effectively, our CAD image prompting method opens doors to a wider range of applications of text-to-image models in engineering design.

Recent advances in artificial intelligence (AI) enable AI agents to go beyond simply supporting human activities and, instead, take more control in team decision-making. While significant literature has studied human-AI collaboration through the lens of AI as a “second opinion system,” this type of interaction is not fully representative of many human-human team collaboration scenarios, such as scenarios where each decision maker is granted equal voting rights for the team decision. In this research, we explore how imparting AI agents with equal voting rights to the human impacts human-AI decision-making and team performance. Using a human subjects experiment in which participants collaborate with two AI teammates for truss structure (aka, bridge) design, we manipulate a series of voting scenarios (e.g., AI agents outvoting the human vs. AI agents agreeing with the human) and AI performance levels (high vs. low performing). The results indicate that changes in human self-confidence are not consistent with whether the quality of the final team-voted design action is advantageous or disadvantageous relative to their own actions. The results also show that when humans are outvoted by their AI teammates, they do not show strong negative emotional reactions if the team-voted decision has an advantageous outcome. Additionally, AI performance significantly influences the human-AI team decision-making process and even one low-performing AI (i.e., an AI that is frequently incorrect) on the team can significantly deteriorate team performance. Taken together, this research provides empirical evidence on the effects of AI voting with equal decision authority on human-AI collaboration, as well as valuable insights supporting real-world applications of human-AI collaboration via voting.

In human-centered design, developing a comprehensive and in-depth understanding of user experiences, i.e., empathic understanding, is paramount for designing products that truly meet human needs. Nevertheless, accurately comprehending the real underlying mental states of a large human population remains a significant challenge today. This difficulty mainly arises from the trade-off between depth and scale of user experience research: gaining in-depth insights from a small group of users does not easily scale to a larger population, and vice versa. This paper investigates the use of Large Language Models (LLMs) for performing mental inference tasks, specifically inferring users' underlying goals and fundamental psychological needs (FPNs). Baseline and benchmark datasets were collected from human users and designers to develop an empathic accuracy metric for measuring the mental inference performance of LLMs. The empathic accuracy of inferring goals and FPNs of different LLMs with varied zero-shot prompt engineering techniques are experimented against that of human designers. Experimental results suggest that LLMs can infer and understand the underlying goals and FPNs of users with performance comparable to that of human designers, suggesting a promising avenue for enhancing the scalability of empathic design approaches through the integration of advanced artificial intelligence technologies. This work has the potential to significantly augment the toolkit available to designers during human-centered design, enabling the development of both large-scale and in-depth understanding of users' experiences.

Abstract As artificial intelligence (AI) systems become increasingly capable of performing design tasks, they are expected to be deployed to assist human designers' decision-making in a greater variety of ways. For complex design problems such as those with multiple objectives, one AI may not always perform its expected accuracy due to the complexity of decision-making, and therefore, multiple AIs may be implemented to provide design suggestions. For such assistance to be productive, human designers must develop appropriate confidence in each AI and in themselves and accept or reject AI inputs accordingly. This work conducts a human subjects experiment to examine the development of a human designer's confidence in each AI and self-confidence throughout decision-making assisted by two AIs and how these confidences influence the decision to accept AI inputs. Major findings demonstrate severe decreases in a human designer's confidence especially when working with one or more low-performing AI teammates and/or receiving negative feedback. Additionally, a human designer's decision to accept AI suggestions depends on their self-confidence and confidence in one of the two AIs. Finally, an additional AI does not increase a human designer's likelihood of conforming to AI suggestions. Therefore, in comparison to a scenario with one AI, the results in this work caution against the implementation of an additional AI to AI-assisted decision-making scenarios. The insights also inform the design and management of human–AI teams to improve the outcome of AI-assisted decision-making.

Abstract The aesthetics of a product is critical to its desirability, and can be described in terms of syntactics and semantics. Syntactic aesthetics is an objective description based on the form and configuration of a product, while semantic aesthetics is a subjective interpretation of the form and gestalt of a product. This study seeks to identify a set of syntactic attributes to describe form and understand if an individual’s preferences for a form are consistent from one product to another. Form attributes from previous literature were expanded upon to create a consistent vocabulary for syntactic aesthetics that can be used to describe multiple products. Combinations of four selected attributes are utilized to describe a diverse set of designs for two products: vases and canopies. Conjoint analysis is used to quantitatively measure the form preferences of individuals towards different combinations of attribute levels of the objects. Results from conjoint analysis applied to vase and canopy designs indicate a 61.3% consistency of individual form preferences between the products. It is hoped that this methodology can help designers develop aesthetically consistent products that align with users’ preferences by quantifying users’ aesthetic preferences towards products through a vocabulary for syntactic attributes.

Abstract With the increasing implementation of artificial intelligence (AI) in the design process, it is crucial to understand how users will accept AI-designed products. This work studies how the public currently perceives an AI's design capability as compared to a human designer's capability by conducting an online survey of 205 people via Amazon Mechanical Turk. The survey collects the respondents' perception on 16 specific bicycle design goals, demographic information, and self-reported level of design and AI/ML knowledge. Findings reveal that people think an AI would perform worse than a human designer on most design goals, particularly the goals that are user-dependent. This work also shows that the higher people's self-reported level of knowledge in design and the older they are, the more likely they are to think an AI's design capability would exceed a human designer's capability. The insights from this work add to the understanding of user acceptance of AI-designed products, as well as human designers' acceptance of AI input in human-AI teams.

The data are collected from a human subjects study in which 100 participants solve chess puzzle problems with artificial intelligence (AI) assistance. The participants are assigned to one of the two experimental conditions determined by the direction of the change in AI performance at problem 20: 1) high- to low-performing and 2) low- to high-performing. The dataset contains information about the participants' move before an AI suggestion, the goodness evaluation score of these moves, AI suggestion, feedback, and the participants' confidence in AI and self-confidence during three initial practice problems and 30 experimental problems. The dataset contains 100 CSV files, one per participant. There is opportunity for this dataset to be utilized in various domains that research human-AI collaboration scenarios such as human-computer interaction, psychology, computer science, and team management in engineering/business. Not only can the dataset enable further cognitive and behavioral analysis in human-AI collaboration contexts but also provide an experimental platform to develop and test future confidence calibration methods.