Kenji Shimada

Purpose Designing structures with low thermal expansion is critical in aerospace and space applications, especially for optical instruments that require support structures capable of withstanding extreme thermal conditions with minimal displacement. Conventionally, supports are fabricated from materials like Invar or composites that are expensive and have limited operating temperature ranges. This research aims to explore the use of multi-material (bimetallic) additive manufacturing (AM) to produce structures with controllable thermal expansion by combining materials with different coefficients of thermal expansion (CTEs). Design/methodology/approach Triangular structures were designed and printed using directed energy deposition with IN625 and SS316L metals. These bimetallic structures underwent thermal expansion testing and numerical models were developed for validation. Findings Experimental results closely matched numerical simulations with an average error of approximately 5%. This validated the use of AM for bimetallic structures and the capability of bimetallic triangular designs to control thermal expansion. Originality/value Researchers have previously developed geometric designs and material combinations with low CTEs; however, a significant CTE differential between materials is needed to achieve dissimilar thermal expansion. This allows for the structures to have controllable effective CTEs. Generally, adhesives, snap-fit joints or welds are used to combine different materials. However, limited research exists on using AM to design bimetallic structures for controllable thermal expansion. This study demonstrates the feasibility and effectiveness of this approach, leading to the development of a bimetallic support structure for an optical instrument that meets critical requirements.

ABSTRACT Metal additive manufacturing (AM) provides a pathway for creating highly optimized components that would be difficult to produce using traditional manufacturing methods. However, regardless of printing parameters or postprocessing, porosity remains a prevalent challenge in AM components because strain concentrates in the vicinity of the pore, compromising fatigue performance. This study uses finite element analysis to investigate the combined effect of applied load, pore aspect ratio, orientation, and location on strain concentration factors (ECFs) under elastic and plastic deformation. Keyhole and lack‐of‐fusion pores are idealized as prolate and oblate ellipsoids, respectively. Isolated porosity is modeled through plasticity simulations in an automated workflow. Ultimately, novel explicit formulas relating the applied strain and pore configuration to ECF are developed. The model is used to rapidly quantify the uncertainty and extreme values in ECF, where the distribution of inputs is obtained from statistics of AM builds, to advance the development of computationally assisted approaches for qualification of AM components.

Metal additive manufacturing (AM) provides a pathway for creating highly optimized components that would be difficult to produce using traditional manufacturing methods. However, regardless of printing parameters or post-processing, porosity remains a prevalent challenge in AM components because strain concentrates in the vicinity of the pore, compromising fatigue performance. This study uses finite element analysis to investigate the combined effect of applied load, pore aspect ratio, orientation, and location on strain concentration factors (ECF) under elastic and plastic deformation. Keyhole and lack-of-fusion pores are idealized as prolate and oblate ellipsoids, respectively. Isolated porosity is modeled through J 2 plasticity simulations in an automated workflow. Ultimately, novel explicit formulas relating the applied strain and pore configuration to ECF are developed. The model is used to rapidly quantify the uncertainty and extreme values in ECF, where the distribution of inputs is obtained from statistics of AM builds, to advance the development of computationally assisted approaches for qualification of AM components.

Inspecting indoor environments such as tunnels, industrial facilities, and construction sites is essential for infrastructure monitoring and maintenance. While manual inspection in these environments is often time-consuming and potentially hazardous, Unmanned Aerial Vehicles (UAVs) can improve efficiency by autonomously handling inspection tasks. Such inspection tasks usually rely on reference maps for coverage planning. However, in industrial applications, only the floor plans are typically available. The unforeseen obstacles not included in the floor plans will result in outdated reference maps and inefficient or unsafe inspection trajectories. In this work, we propose an adaptive inspection framework that integrates global coverage planning with local reactive adaptation to improve the coverage and efficiency of UAV-based inspection in partially unknown indoor environments. Experimental results in structured indoor scenarios demonstrate the effectiveness of the proposed approach in inspection efficiency and achieving high coverage rates with adaptive obstacle handling, highlighting its potential for enhancing the efficiency of indoor facility inspection.

Aerial robots can enhance construction site productivity by autonomously handling inspection and mapping tasks. However, ensuring safe navigation near human workers remains challenging. While navigation in static environments has been well studied, navigating dynamic environments remains open due to challenges in perception and planning. Payload limitations restrict the robots to using cameras with limited fields of view, resulting in unreliable perception and tracking during collision avoidance. Moreover, the rapidly changing conditions of dynamic environments can quickly make the generated optimal trajectory outdated.To address these challenges, this letter presents a comprehensive navigation framework that integrates perception, intent prediction, and planning. Our perception module detects and tracks dynamic obstacles efficiently and handles tracking loss and occlusion during collision avoidance. The proposed intent prediction module employs a Markov Decision Process (MDP) to forecast potential actions of dynamic obstacles with the possible future trajectories. Finally, a novel intent-based planning algorithm, leveraging model predictive control (MPC), is applied to generate navigation trajectories. Simulation and physical experiments demonstrate that our method improves the safety of navigation by achieving the fewest collisions compared to benchmarks.

Safe flight in dynamic environments requires unmanned aerial vehicles (UAVs) to make effective decisions when navigating cluttered spaces with moving obstacles. Traditional approaches often decompose decision-making into hierarchical modules for prediction and planning. Although these handcrafted systems can perform well in specific settings, they might fail if environmental conditions change and often require careful parameter tuning. Additionally, their solutions could be suboptimal due to the use of inaccurate mathematical model assumptions and simplifications aimed at achieving computational efficiency. To overcome these limitations, this letter introduces the NavRL framework, a deep reinforcement learning-based navigation method built on the Proximal Policy Optimization (PPO) algorithm. NavRL utilizes our carefully designed state and action representations, allowing the learned policy to make safe decisions in the presence of both static and dynamic obstacles, with zero-shot transfer from simulation to real-world flight. Furthermore, the proposed method adopts a simple but effective safety shield for the trained policy, inspired by the concept of velocity obstacles, to mitigate potential failures associated with the black-box nature of neural networks. To accelerate the convergence, we implement the training pipeline using NVIDIA Isaac Sim, enabling parallel training with thousands of quadcopters. Simulation and physical experiments show that our method ensures safe navigation in dynamic environments and results in the fewest collisions compared to benchmarks.

The production revolution has materialized through Binder jet metal 3D printing because this technology creates detailed customized parts efficiently. The process achieves better results when parts are operated near each other because it reduces disturbance to the powder material. The protection of workers in addition to their safety is funda- mental. The essential elements for enhanced performance in- clude non-harmful material selection and controlled dust reg- ulation combined with ergonomic product optimization. The entire workforce needs complete training on safe protocols for operation. We can achieve improved quality and efficiency for binder jetting through proper factor resolution which will result in successful manufacturing of complex high-value components. Basic mechanical grippers do not have enough ability to handle polished parts, as their solid force can damage the components. Our research results in a special gripper system that is suitable for robotics used for safe depowdering operations. The adaptable gripper system fits different part geometries to decrease the chance of component deformation or fracturing. A combination of 304 stainless steel wires integrated with a PLA body allows the device to achieve precise handling and control together with versatile operation. The new gripper system extends its reach into tight vertical placement spaces, achieving improved comprehensive depowdering capabilities. Our work focuses on cutting down postprocessing times as we strive to reach higher levels of productivity. The development enhances the essential nature of advanced tooling systems for the advancement of binder jetting technology and process optimization within additive manufacturing. The development of these solutions makes binder jet metal 3D printing the advanced technology for fabricating intricate high-performance components that serve multiple industrial uses. The study improves the advancement of additive manufacturing by creating new possibilities for scalable and adaptable production systems in the modern industry.

Abstract In design and manufacturing engineering, the manual interpretation of 2D technical drawings remains a significant bottleneck, impeding efficiency in tasks such as part quotation and process manufacturing. While computer vision advancements have shown promise in interpreting natural images, the translation of engineering drawings into actionable data poses unique challenges. Our research proposes a novel data-driven framework that leverages computational vision and graph learning to automate the vectorization and low-level interpretation of raster engineering drawings, bringing us closer to a CAD format. Our approach involves the vectorization of rasterized engineering drawings into a collection of lines and arcs. We construct a connectivity graph from these lines and arcs and train a graph convolutional neural network to classify entity types accurately. We concentrate on 9 distinct classes: Lines, Arrowheads, Text, Dimension Lines, Center Lines, Arcs, Hidden Lines and Contours, and Misc. We achieved a notably high performance of 98.2% validation accuracy, a 20.66% improvement compared to prior methodologies, and a maximum of 46.48% increase compared to baseline learning methods. By streamlining the interpretation of technical drawings, our framework offers a pathway to enhance efficiency in modern manufacturing processes, reducing reliance on manual labor and boosting overall productivity.

Safe flight in dynamic environments requires unmanned aerial vehicles (UAVs) to make effective decisions when navigating cluttered spaces with moving obstacles. Traditional approaches often decompose decision-making into hierarchical modules for prediction and planning. Although these handcrafted systems can perform well in specific settings, they might fail if environmental conditions change and often require careful parameter tuning. Additionally, their solutions could be suboptimal due to the use of inaccurate mathematical model assumptions and simplifications aimed at achieving computational efficiency. To overcome these limitations, this paper introduces the NavRL framework, a deep reinforcement learning-based navigation method built on the Proximal Policy Optimization (PPO) algorithm. NavRL utilizes our carefully designed state and action representations, allowing the learned policy to make safe decisions in the presence of both static and dynamic obstacles, with zero-shot transfer from simulation to real-world flight. Furthermore, the proposed method adopts a simple but effective safety shield for the trained policy, inspired by the concept of velocity obstacles, to mitigate potential failures associated with the black-box nature of neural networks. To accelerate the convergence, we implement the training pipeline using NVIDIA Isaac Sim, enabling parallel training with thousands of quadcopters. Simulation and physical experiments show that our method ensures safe navigation in dynamic environments and results in the fewest collisions compared to benchmarks.

Aerial robots can enhance construction site productivity by autonomously handling inspection and mapping tasks. However, ensuring safe navigation near human workers remains challenging. While navigation in static environments has been well studied, navigating dynamic environments remains open due to challenges in perception and planning. Payload limitations restrict the robots to using cameras with limited fields of view, resulting in unreliable perception and tracking during collision avoidance. Moreover, the rapidly changing conditions of dynamic environments can quickly make the generated optimal trajectory outdated.To address these challenges, this paper presents a comprehensive navigation framework that integrates perception, intent prediction, and planning. Our perception module detects and tracks dynamic obstacles efficiently and handles tracking loss and occlusion during collision avoidance. The proposed intent prediction module employs a Markov Decision Process (MDP) to forecast potential actions of dynamic obstacles with the possible future trajectories. Finally, a novel intent-based planning algorithm, leveraging model predictive control (MPC), is applied to generate navigation trajectories. Simulation and physical experiments demonstrate that our method improves the safety of navigation by achieving the fewest collisions compared to benchmarks.

Abstract Purpose Individual facial soft tissue properties are necessary for creating individualized finite element (FE) models to evaluate medical devices such as continuous positive airway pressure (CPAP) masks. There are no standard tools available to measure facial soft tissue elastic moduli, and techniques in literature require advanced equipment or custom parts to replicate. Methods We propose a simple and inexpensive soft tissue measurement (STM) indenter device to estimate facial soft tissue elasticity at five sites: chin, cheek near lip, below cheekbone, cheekbone, and cheek. The STM device consists of a probe with a linear actuator and force sensor, an adjustment system for probe orientation, a head support frame, and a controller. The device was validated on six ballistics gel samples and then tested on 28 subjects. Soft tissue thickness was also collected for each subject using ultrasound. Results Thickness and elastic modulus measurements were successfully collected for all subjects. The mean elastic modulus for each site is E c = 53.04 ± 20.97 kPa for the chin, E l = 16.33 ± 8.37 kPa for the cheek near lip, E bc = 27.09 ± 11.38 kPa for below cheekbone, E cb = 64.79 ± 17.12 kPa for the cheekbone, and E ch = 16.20 ± 5.09 kPa for the cheek. The thickness and elastic modulus values are in the range of previously reported values. One subject’s measured soft tissue elastic moduli and thickness were used to evaluate custom-fit CPAP mask fit in comparison to a model of that subject with arbitrary elastic moduli and thickness. The model with measured values more closely resembles in vivo leakage results. Conclusion Overall, the STM provides a first estimate of facial soft tissue elasticity and is affordable and easy to build with mostly off-the-shelf parts. These values can be used to create personalized FE models to evaluate custom-fit CPAP masks.

Small drones are an ideal tool for performing inspections within confined environments. However, due to their smaller mass, small drones are subject to greater deviations based on wind disturbances. The ability to mitigate wind disturbances for unmanned aerial vehicles (UAVs) is a field of research with many disparate existing solutions. Some require immense computational capacity while others do not guarantee explicit position control performance. The methodology proposed herein uses available onboard sensors and a shallow neural network that can select control system parameters based on a desired maximum flight error when subjected to a wind disturbance, thus providing an adaptive flight control without sacrificing flight performance. The results show that the proposed neural network can recommend parameters that improve the system performance.

Autonomous exploration in dynamic environments necessitates a planner that can proactively respond to changes and make efficient and safe decisions for robots. Although plenty of sampling-based works have shown success in exploring static environments, their inherent sampling randomness and limited utilization of previous samples often result in sub-optimal exploration efficiency. Additionally, most of these methods struggle with efficient replanning and collision avoidance in dynamic settings. To overcome these limitations, we propose the Heuristic-based Incremental Probabilistic Roadmap Exploration (HIRE) planner for UAVs exploring dynamic environments. The proposed planner adopts an incremental sampling strategy based on the probabilistic roadmap constructed by heuristic sampling toward the unexplored region next to the free space, defined as the heuristic frontier regions. The heuristic frontier regions are detected by applying a lightweight vision-based method to the different levels of the occupancy map. Moreover, our dynamic module ensures that the planner dynamically updates roadmap information based on the environment changes and avoids dynamic obstacles. Simulation and physical experiments prove that our planner can efficiently and safely explore dynamic environments. Our software<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> is available on GitHub with the experiment video<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>.

Autonomous navigation requires robots to generate trajectories for collision avoidance efficiently. Although plenty of previous works have proven successful in generating smooth and spatially collision-free trajectories, their solutions often suffer from suboptimal time efficiency and potential un-safety, particularly when accounting for uncertainties in robot perception and control. To address this issue, this paper presents the Robust Optimal Time Allocation (ROTA) framework. This framework is designed to optimize the time progress of the trajectories temporally, serving as a post-processing tool to enhance trajectory time efficiency and safety under uncertainties. In this study, we begin by formulating a non-convex optimization problem aimed at minimizing trajectory execution time while incorporating constraints on collision probability as the robot approaches obstacles. Subsequently, we introduce the concept of the trajectory braking zone and adopt the chance-constrained formulation for robust collision avoidance in the braking zones. Finally, the non-convex optimization problem is reformulated into a second-order cone programming problem to achieve real-time performance. Through simulations and physical flight experiments, we demonstrate that the proposed approach effectively reduces trajectory execution time while enabling robust collision avoidance in complex environments. Our software<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> is available on GitHub, along with the developed autonomy framework<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>, as open-source ROS packages.

Continuous Positive Airway Pressure (CPAP) is a common therapy used to treat breathing disorders such as obstructive sleep apnea. In previous work, we designed a custom-fit CPAP mask to address comfort and leakage issues patients often experience. This paper presents a method to create a finite element (FE) model to evaluate the fit of the custom-fit mask before fabrication. The model includes details such as a skull to represent the variable soft tissue thicknesses on the face, and two strap configurations, original and X. The model was tested on four subjects and results show that the X strap configuration results in a more even stress distribution, measured by standard deviation, on the face compared to the original strap, indicating better fit. The simulations also show gaps in the stress distribution that seem to correspond to areas of leakage based on two initial in vivo tests on two subjects. This simulation method proves to be a valuable tool for custom-fit mask development by allowing us to evaluate designs before fabrication.

Transient mitigation for nuclear power plants is essential for safe operation. The fourth industrial revolution brings with it the potential for data-based predictive maintenance and identifying remaining time of life for degrading components. An improvement to predictive maintenance would be to address continued operation with faulty components between the time of identification and eventual replacement. The ability to perform data analysis and contemporary digital control systems allows for data-driven control system actions. A methodology is developed herein to train a neural network that can map desired system performance and current plant component capability to control system settings. Simulations of plant transients were recorded and used to train a neural network. This neural network was tested with different target performance goals. The results show that the trained neural network recommended settings that affected the control system response so as to meet the target performance goals.

Quadrotor drone control is a popular domain for control research and reinforcement learning applications. Existing control applications for quadrotor drones can be leveraged to improve the performance of reinforcement learning agents. We propose methods for interfacing a reinforcement learning agent with a typical quadrotor drone flight controller. One method is to provide auxiliary rotor commands that adjust the output of a static PID controller. The other method is for an agent to identify continuous absolute controller parameters for the PID controller. These methods are used to train agents and evaluate their performance through simulation and compare against a typical reinforcement learning approach as well as a static PID controller. The results show that the trained agents are able to successfully mitigate wind disturbances and outperform both typical reinforcement learning agents and a typical PID controller.

The aerospace industry consistently prioritizes researching optimization methods for reducing component weight, meeting structural and thermal requirements, and enhancing product quality and efficiency. This work explores a design method that combines topology optimization (TO) and lattice generation to redesign three components: a jet engine bracket, an airplane bearing bracket, and an optical instrument mounting structure to satisfy their various structural and thermal loading requirements. Redesign and optimization methods for aircraft components such as jet engines and airplane bearing brackets have led to promising results, however, these components only have structural loading requirements. Spacecraft components such as mounting structures for optical instruments are needed for any space observation mission. Due to launch loads and the harsh space environment, they experience multiphysics loading requirements, including extreme stiffness for optical pointing precision, thermal resistance, and structural stability. The combination of loads and constraints poses challenges for the sole utilization of a single method or tool for optimizing mounting structures. Although TO and lattice generation methods are commonly used to create lightweight and optimized designs, each method has its limitations. Highly topology-optimized components may fail at unexpected loads, and many lattice generation methods are limited in controlling their geometric parameters. Combining these two methods would aid in balancing their respective shortcomings, leading to an effectively optimized component. In this study, TO software is coupled with a unique bubble-mesh-based lattice generation method that allows for variation in the following three parameters: cell size/lattice density, strut intersection rounding, and strut diameter. This coupled design process led to final designs that met the essential loading requirements of each component with the following weight reductions: mounting structure: 81.8%, jet engine bracket: 62.4%, and airplane bearing bracket: 52.5%.

Ensuring the structural integrity of window frames and detecting subtle defects, such as dents and scratches, is crucial for maintaining product quality. Traditional machine vision systems face challenges in defect identification, especially with reflective materials and varied environments. Modern machine and deep learning (DL) systems hold promise for post-installation inspections but face limitations due to data scarcity and environmental variability. Our study introduces an innovative approach to enhance DL-based defect detection, even with limited data. We present a comprehensive window frame defect detection framework incorporating optimized image enhancement, data augmentation, and a core U-Net model. We constructed five datasets using cell phones and the Spot Robot for autonomous inspection, evaluating our approach across various scenarios and lighting conditions in real-world window frame inspections. Our results demonstrate significant performance improvements over the standard U-Net model, with a notable 7.43% increase in the F1 score and 15.1% in IoU. Our approach enhances defect detection capabilities, even in challenging real-world conditions. To enhance the generalizability of this study, it would be advantageous to apply its methodology across a broader range of diverse construction sites.

Deploying autonomous robots in crowded indoor environments usually requires them to have accurate dynamic obstacle perception. Although plenty of previous works in the autonomous driving field have investigated the 3D object detection problem, the usage of dense point clouds from a heavy Light Detection and Ranging (LiDAR) sensor and their high computation cost for learning-based data processing make those methods not applicable to small robots, such as vision-based UAVs with small onboard computers. To address this issue, we propose a lightweight 3D dynamic obstacle detection and tracking (DODT) method based on an RGB-D camera, which is designed for low-power robots with limited computing power. Our method adopts a novel ensemble detection strategy, combining multiple computationally efficient but low-accuracy detectors to achieve real-time high-accuracy obstacle detection. Besides, we introduce a new feature-based data association and tracking method to prevent mismatches utilizing point clouds' statistical features. In addition, our system includes an optional and auxiliary learning-based module to enhance the obstacle detection range and dynamic obstacle identification. The proposed method is implemented in a small quadcopter, and the results show that our method can achieve the lowest position error (0.11 m) and a comparable velocity error (0.23 m/s) across the benchmarking algorithms running on the robot's onboard computer. The flight experiments prove that the tracking results from the proposed method can make the robot efficiently alter its trajectory for navigating dynamic environments. Our software is available on GitHub <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> as an open-source ROS package.

Autonomous exploration in dynamic environments necessitates a planner that can proactively respond to changes and make efficient and safe decisions for robots. Although plenty of sampling-based works have shown success in exploring static environments, their inherent sampling randomness and limited utilization of previous samples often result in sub-optimal exploration efficiency. Additionally, most of these methods struggle with efficient replanning and collision avoidance in dynamic settings. To overcome these limitations, we propose the Heuristic-based Incremental Probabilistic Roadmap Exploration (HIRE) planner for UAVs exploring dynamic environments. The proposed planner adopts an incremental sampling strategy based on the probabilistic roadmap constructed by heuristic sampling toward the unexplored region next to the free space, defined as the heuristic frontier regions. The heuristic frontier regions are detected by applying a lightweight vision-based method to the different levels of the occupancy map. Moreover, our dynamic module ensures that the planner dynamically updates roadmap information based on the environment changes and avoids dynamic obstacles. Simulation and physical experiments prove that our planner can efficiently and safely explore dynamic environments.

Autonomous navigation requires robots to generate trajectories for collision avoidance efficiently. Although plenty of previous works have proven successful in generating smooth and spatially collision-free trajectories, their solutions often suffer from suboptimal time efficiency and potential unsafety, particularly when accounting for uncertainties in robot perception and control. To address this issue, this paper presents the Robust Optimal Time Allocation (ROTA) framework. This framework is designed to optimize the time progress of the trajectories temporally, serving as a post-processing tool to enhance trajectory time efficiency and safety under uncertainties. In this study, we begin by formulating a non-convex optimization problem aimed at minimizing trajectory execution time while incorporating constraints on collision probability as the robot approaches obstacles. Subsequently, we introduce the concept of the trajectory braking zone and adopt the chance-constrained formulation for robust collision avoidance in the braking zones. Finally, the non-convex optimization problem is reformulated into a second-order cone programming problem to achieve real-time performance. Through simulations and physical flight experiments, we demonstrate that the proposed approach effectively reduces trajectory execution time while enabling robust collision avoidance in complex environments.

Detecting subtle defects in window frames, including dents and scratches, is vital for upholding product integrity and sustaining a positive brand perception. Conventional machine vision systems often struggle to identify these defects in challenging environments like construction sites. In contrast, modern vision systems leveraging machine and deep learning (DL) are emerging as potent tools, particularly for cosmetic inspections. However, the promise of DL is yet to be fully realized. A few manufacturers have established a clear strategy for AI integration in quality inspection, hindered mainly by issues like scarce clean datasets and environmental changes that compromise model accuracy. Addressing these challenges, our study presents an innovative approach that amplifies defect detection in DL models, even with constrained data resources. The paper proposes a new defect detection pipeline called InspectNet (IPT-enhanced UNET) that includes the best combination of image enhancement and augmentation techniques for pre-processing the dataset and a Unet model tuned for window frame defect detection and segmentation. Experiments were carried out using a Spot Robot doing window frame inspections . 16 variations of the dataset were constructed using different image augmentation settings. Results of the experiments revealed that, on average, across all proposed evaluation measures, Unet outperformed all other algorithms when IPT-enhanced augmentations were applied. In particular, when using the best dataset, the average Intersection over Union (IoU) values achieved were IPT-enhanced Unet, reaching 0.91 of mIoU.

To automate the welding of multiple industrial parts that are too large for conveyor belts, we designed a mobile manipulator to perform welding around large parts and move between them. In our previous paper, we devised a motion planner to optimize the mobile welding motion. In this paper, we complete the automation by proposing a framework to optimize the path traveled between these large parts and the welding paths around each part. We model it as a Traveling Salesman Problem with Neighborhoods (TSPN), where the neighborhoods represent the parts, and the points in the neighborhoods represent the robot's welding positions around the parts. As opposed to previous TSPN solvers, our method optimizes the total path, which is the sum of both paths between neighborhoods (global path) and inside each neighborhood (local path). Also, entry and exit points for each neighborhood are defined to remove unnecessary traveling back to the entry point. In a simulation test case, our method resulted in path length reduction of 13.2% compared to the method with same entry and exit and without considering the welding paths. The scalability of this method was tested by adding more parts, which resulted in a higher path length reduction of 17%. Complexity was also increased by adding obstacles, which resulted in an even higher path length reduction of 25.5%. In our ship hull parts welding simulation, we show that our method is able to calculate an optimized path that visits all parts and the welding points around the parts. It shows that with our approach, the automation of large part welding is feasible as well as optimized.

Tunnel construction using the drill-and-blast method requires the 3D measurement of the excavation front to evaluate underbreak locations. Considering the inspection and measurement task's safety, cost, and efficiency, deploying lightweight autonomous robots, such as unmanned aerial vehicles (UAV), becomes more necessary and popular. Most of the previous works use a prior map for inspection viewpoint determination and do not consider dynamic obstacles. To maximally increase the level of autonomy, this letter proposes a vision-based UAV inspection framework for dynamic tunnel environments without using a prior map. Our approach utilizes a hierarchical planning scheme, decomposing the inspection problem into different levels. The high-level decision maker first determines the task for the robot and generates the target point. Then, the mid-level path planner finds the waypoint path and optimizes the collision-free static trajectory. Finally, the static trajectory will be fed into the low-level local planner to avoid dynamic obstacles and navigate to the target point. Besides, our framework contains a novel dynamic map module that can simultaneously track dynamic obstacles and represent static obstacles based on an RGB-D camera. After inspection, the Structure-from-Motion (SfM) pipeline is applied to generate the 3D shape of the target. To our best knowledge, this is the first time autonomous inspection has been realized in unknown and dynamic tunnel environments. Our flight experiments in a real tunnel prove that our method can autonomously inspect the tunnel excavation front surface.

Abstract The current design of aerodynamic shapes, like airfoils, involves computationally intensive simulations to explore the possible design space. Usually, such design relies on the prior definition of design parameters and places restrictions on synthesizing novel shapes. In this work, we propose a data-driven shape encoding and generating method, which automatically learns representations from existing airfoils and uses the learned representations to generate new airfoils. The representations are then used in the optimization of synthesized airfoil shapes based on their aerodynamic performance. Our model is built upon VAEGAN, a neural network that combines Variational Autoencoder with Generative Adversarial Network and is trained by the gradient-based technique. Our model can (1) encode the existing airfoil into a latent vector and reconstruct the airfoil from that, (2) generate novel airfoils by randomly sampling the latent vectors and mapping the vectors to the airfoil coordinate domain, and (3) synthesize airfoils with desired aerodynamic properties by optimizing learned features via a genetic algorithm. Our experiments show that the learned features encode shape information thoroughly and comprehensively without predefined design parameters. By interpolating/extrapolating feature vectors or sampling from Gaussian noises, the model can automatically synthesize novel airfoil shapes, some of which possess competitive or even better aerodynamic properties as compared to airfoils used for model training purposes. By optimizing shapes on the learned latent domain via a genetic algorithm, synthesized airfoils can evolve to target aerodynamic properties. This demonstrates an efficient learning-based airfoil design framework that encodes and optimizes the airfoil on the latent domain and synthesizes promising airfoil candidates for required aerodynamic performance.

Navigating dynamic environments requires the robot to generate collision-free trajectories and actively avoid moving obstacles. Most previous works designed path planning algorithms based on one single map representation, such as the geometric, occupancy, or ESDF map. Although they have shown success in static environments, due to the limitation of map representation, those methods cannot reliably handle static and dynamic obstacles simultaneously. To address the problem, this paper proposes a gradient-based B-spline trajectory optimization algorithm utilizing the robot's onboard vision. The depth vision enables the robot to track and represent dynamic objects geometrically based on the voxel map. The proposed optimization first adopts the circle-based guide-point algorithm to approximate the costs and gradients for avoiding static obstacles. Then, with the vision-detected moving objects, our receding-horizon distance field is simultaneously used to prevent dynamic collisions. Finally, the iterative re-guide strategy is applied to generate the collision-free trajectory. The simulation and physical experiments prove that our method can run in real-time to navigate dynamic environments safely.

The real-time dynamic environment perception has become vital for autonomous robots in crowded spaces. Although the popular voxel-based mapping methods can efficiently represent 3D obstacles with arbitrarily complex shapes, they can hardly distinguish between static and dynamic obstacles, leading to the limited performance of obstacle avoidance. While plenty of sophisticated learning-based dynamic obstacle detection algorithms exist in autonomous driving, the quad-copter's limited computation resources cannot achieve real-time performance using those approaches. To address these issues, we propose a real-time dynamic obstacle tracking and mapping system for quadcopter obstacle avoidance using an RGB-D camera. The proposed system first utilizes a depth image with an occupancy voxel map to generate potential dynamic obstacle regions as proposals. With the obstacle region proposals, the Kalman filter and our continuity filter are applied to track each dynamic obstacle. Finally, the environment-aware trajectory prediction method is proposed based on the Markov chain using the states of tracked dynamic obstacles. We implemented the proposed system with our custom quadcopter and navigation planner. The simulation and physical experiments show that our methods can successfully track and represent obstacles in dynamic environments in real-time and safely avoid obstacles.

We present a data-driven framework to automate the vectorization and machine interpretation of 2D engineering part drawings. In industrial settings, most manufacturing engineers still rely on manual reads to identify the topological and manufacturing requirements from drawings submitted by designers. The interpretation process is laborious and time-consuming, which severely inhibits the efficiency of part quotation and manufacturing tasks. While recent advances in image-based computer vision methods have demonstrated great potential in interpreting natural images through semantic segmentation approaches, the application of such methods in parsing engineering technical drawings into semantically accurate components remains a significant challenge. The severe pixel sparsity in engineering drawings also restricts the effective featurization of image-based data-driven methods. To overcome these challenges, we propose a deep learning based framework that predicts the semantic type of each vectorized component. Taking a raster image as input, we vectorize all components through thinning, stroke tracing, and cubic bezier fitting. Then a graph of such components is generated based on the connectivity between the components. Finally, a graph convolutional neural network is trained on this graph data to identify the semantic type of each component. We test our framework in the context of semantic segmentation of text, dimension and, contour components in engineering drawings. Results show that our method yields the best performance compared to recent image, and graph-based segmentation methods.

Deploying autonomous robots in crowded indoor environments usually requires them to have accurate dynamic obstacle perception. Although plenty of previous works in the autonomous driving field have investigated the 3D object detection problem, the usage of dense point clouds from a heavy Light Detection and Ranging (LiDAR) sensor and their high computation cost for learning-based data processing make those methods not applicable to small robots, such as vision-based UAVs with small onboard computers. To address this issue, we propose a lightweight 3D dynamic obstacle detection and tracking (DODT) method based on an RGB-D camera, which is designed for low-power robots with limited computing power. Our method adopts a novel ensemble detection strategy, combining multiple computationally efficient but low-accuracy detectors to achieve real-time high-accuracy obstacle detection. Besides, we introduce a new feature-based data association and tracking method to prevent mismatches utilizing point clouds' statistical features. In addition, our system includes an optional and auxiliary learning-based module to enhance the obstacle detection range and dynamic obstacle identification. The proposed method is implemented in a small quadcopter, and the results show that our method can achieve the lowest position error (0.11m) and a comparable velocity error (0.23m/s) across the benchmarking algorithms running on the robot's onboard computer. The flight experiments prove that the tracking results from the proposed method can make the robot efficiently alter its trajectory for navigating dynamic environments. Our software is available on GitHub as an open-source ROS package.

Tunnel construction using the drill-and-blast method requires the 3D measurement of the excavation front to evaluate underbreak locations. Considering the inspection and measurement task's safety, cost, and efficiency, deploying lightweight autonomous robots, such as unmanned aerial vehicles (UAV), becomes more necessary and popular. Most of the previous works use a prior map for inspection viewpoint determination and do not consider dynamic obstacles. To maximally increase the level of autonomy, this paper proposes a vision-based UAV inspection framework for dynamic tunnel environments without using a prior map. Our approach utilizes a hierarchical planning scheme, decomposing the inspection problem into different levels. The high-level decision maker first determines the task for the robot and generates the target point. Then, the mid-level path planner finds the waypoint path and optimizes the collision-free static trajectory. Finally, the static trajectory will be fed into the low-level local planner to avoid dynamic obstacles and navigate to the target point. Besides, our framework contains a novel dynamic map module that can simultaneously track dynamic obstacles and represent static obstacles based on an RGB-D camera. After inspection, the Structure-from-Motion (SfM) pipeline is applied to generate the 3D shape of the target. To our best knowledge, this is the first time autonomous inspection has been realized in unknown and dynamic tunnel environments. Our flight experiments in a real tunnel prove that our method can autonomously inspect the tunnel excavation front surface. Our software is available on GitHub as an open-source ROS package.