Jie Gu

The ongoing developments of precision medicine demand highly personalized treatment via on-device learning. In cardiac arrhythmia detection, labeled electrocardiography (ECG) data is limited especially at the individual level. To address this, we propose a subject-dependent VICReg self-supervised learning framework with on-device training. By leveraging abundant unlabeled data, our approach enables self-training with a small amount of labeled samples, making it well-suited for wearable devices. This method enables personalized, patient-specific arrhythmia detection, offering an effective and scalable solution for on-device ECG analysis in resource-constrained devices. Experimental results show that this work achieves more than 30% latency reduction in model training, 71.4% reduction in model complexity in real-time arrhythmia detection, and up to 5.8% accuracy improvement among different patients.

This work presents a novel power distribution network (PDN) analysis using the emerging physics-informed neural network (PINNs) model. Different from conventional solver-based analysis, PINN allows rapid analysis and prediction while maintaining the physics compliance for high-fidelity analysis. An adaptive multi-objective training strategy is introduced, incorporating an interconnection matrix and layer labeling to accelerate convergence across complex PDN structures. An embedded workload vector and a transient modulator with linear superposition method extend the model’s applicability to a wide range of power scenarios. The developed method is applied to both chip-level PDN analysis and Chiplet 2.5-D chip-package co-analysis, showing high accuracy and fast runtime compared with conventional methods. The approach captures both steady-state IR droop and dynamic transient supply droop, including IR and L•di/dt noise from package and on-die PDN. Experiments on 2.5-D Chiplets with RISC-V processors and CNN accelerators show that the proposed PINN-based method achieves a 299x and 7x reduction in runtime compared to conventional EDA tools or prior work and saves up to 80% of training data than traditional neural networks models.

Nowadays, the demand of tighter tolerance components requires more accurate machine tools with volumetric compensation. In mass production, post-machining inspection of the workpiece on a Coordinate Measuring Machine (CMM) provides statistical quality control information on the variability of the manufacturing process. The dynamic face offset (DFO) is proposed to compensate the machine tool errors. The method is utilizing the part quality CMM data to determine the specific compensation parameters that are used in the machine tool controller to minimize the machine tool errors for specific part faces. The DFO does not considers the table offset parameters as when using the global offset compensation method. The table errors are incorporated within the local offsets. This paper presents the development of a DFO compensation method utilizing the measurements of the machined part(s). The DFO for 4-axis and 5-axis machine tools is estimated through a model while utilizing the computed deviations between the measured and nominal dimensions of the part. The measuring results before and after compensation are compared and demonstrate that the errors are reduced and compensated successfully.

With the rapid development of the source-grid-load-storage system and the increasing proportion of photovoltaic (PV) power generation, high-precision PV power prediction has become increasingly critical for power system energy management. To address the challenges of volatility and complexity in PV power generation, this paper proposes a hybrid prediction model based on improved grey relational analysis and deep learning. First, key meteorological features are selected using improved grey relational analysis to reduce noise interference from weakly correlated factors. Second, a BiLSTM-GRU hybrid model is constructed by combining the strengths of Bidirectional Long Short-Term Memory (BiLSTM) and Gated Recurrent Unit (GRU), leveraging BiLSTM's bidirectional temporal feature extraction capability and GRU's efficient long-term dependency modeling to enhance prediction accuracy. Validation using real-world data from a PV power plant in Xinjiang in 2019 demonstrates that the proposed model achieves a root mean square error (RMSE) of 1.09, mean absolute error (MAE) of 0.97, and coefficient of determination <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\left(R^{2}\right)$</tex> of 0.98, significantly outperforming standalone BiLSTM or GRU models. Experimental results show that the model effectively captures the temporal volatility characteristics of PV power, offering a novel approach for accurate prediction of highly variable renewable energy sources.

The high integration of photovoltaic (PV) systems introduces significant grid instability and randomness to distribution networks. Understanding PV power output is crucial for improving grid stability. To address this, this paper proposes a hybrid PV power forecasting model based on GMM (Gaussian Mixture Model) weather classification. Firstly, PV generation data is normalized and analyzed for correlations. The GMM clustering algorithm is then used to classify the data into three weather types: sunny, rainy, and cloudy. Next, a BiLSTM-Attention neural network is constructed by integrating BiLSTM (Bidirectional Long Short-Term Memory) with an attention mechanism to enhance prediction. Finally, case study comparisons demonstrate that the proposed model can effectively capture local features with high prediction variability under different weather conditions, significantly improving both the accuracy and stability of PV power forecasting.

This work presents a footstep planning chip for humanoid robot. It integrates a time-domain graph search engine for high-level 3-D footstep planning and a mixed-signal zero moment point (ZMP) gait scheduler with neural inverse kinematics, enabling efficient low-level motion control. The key contributions of this work include a time-domain graph search engine for 3-D footstep planning, featuring 3-D search capabilities, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D^{\ast } $ </tex-math></inline-formula> replanning for real-time adjustments, redundant path blocking, and efficient result readout. In addition, it introduces an energy-efficient mixed-signal ZMP gait scheduler for maintaining robot balance, along with a time-domain neural-network-based inverse kinematics module for controlling robot joints. This work is demonstrated in situ on a fully assembled robot using the 65-nm system-on-chip (SoC), achieving <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.7\times $ </tex-math></inline-formula> energy savings for graph search and an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$18.4\times $ </tex-math></inline-formula> improvement in energy efficiency for motion control compared with prior works.

With the rapid expansion of autonomous robotic systems in recent years, humanoid robots are also gaining considerable attention. However, their motion control presents more complex challenges than wheeled mobile robots. For the first time, this work presents a complete footstep planning SoC chip for humanoid robots. It includes a novel time-domain graph search engine for 3D footstep planning, along with a mixed-signal zero-moment point (ZMP) gait scheduler enhanced by neural inverse kinematics for efficient motion control. This work is demonstrated in-situ on a fully assembled robot using the 65nm system-on-chip (SoC), achieving the best-in-class performance and power including an 18.4x enhancement on energy efficiency for motion control and a 2.7x energy saving in graph search compared to previous works. The demo video is provided in https://youtu.be/kBe-aRnzmG4.

Virtual Reality and Mixed Reality systems have revolutionized consumer electronics, driving innovations in the metaverse. However, traditional VR headsets lack brain activity integration for feedback and control. This work introduces the first 65nm SoC for in situ mind imagery-based brain machine interface, seamlessly integrated into VR/MR headsets, with reduced energy consumption and AI support. The digital core of the SoC achieves state-of-the-art energy consumption <1μJ/class for computation-intensive CNN operations thanks to a novel teacher-student low-power scheme, general instruction set architecture for general programming and system-level optimizations of the design. The demonstration video for the whole system is provided at https://youtu.be/WuGlcMSSQzY.

A proactive power management (PM) scheme for mitigating dynamic supply droop is proposed with a fully integrated buck converter, an RISC-V CPU core, a real-time machine learning engine (MLE), and a safety droop guardband. The dynamic supply is modeled using an RV32IM ISA CPU and the model was simulated in a customized co-sim environment. The MLE is used to predict the supply droop on a cycle-by-cycle basis for proactive supply regulation from the integrated buck converter through a fast PWM modulation of the power switches. In addition, to deal with the misprediction of MLE and other long-term supply droops that cannot be captured by MLE, a safety droop guardband is implemented to secure the integrity of the supply voltage. The proposed proactive scheme has been implemented on a 65-nm test chip and demonstrated up to 9.9% higher CPU frequency or 9.2% higher power efficiency compared with prior fast digital LDO schemes.

This work presents a low-power physical-based ray-tracing (PBRT) rendering processor for photorealistic augmented reality (AR) rendering applications on mobile devices, referred to as mobile physical-based renderer (Mobile-PBR). By introducing inverse rendering (IR) and background clustering, Mobile-PBR enables complicated photorealistic lighting effects such as reflection, refraction, and shadow with minimum resources on mobile edge devices. The key features of this work include: 1) an ASIC rendering processor that embeds an end-to-end ray-tracing (RT) solution with IR for AR on mobile devices; 2) a reconfigurable mixed-precision processing element (PE) design supporting diverse computing tasks for both IR and RT modes; 3) background clustered field of view (FOV)-focused 3-D construction reducing conventional background scene complexity from O(nlogn) to O(1); 4) scalable partitioning scheme for complex 3-D objects with an average of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$13{\times }$ </tex-math></inline-formula> speed up on test scenes; and 5) use of global RT scheduler (GRTS) and global memory access controller (GMAC) to overcome the challenges of irregular memory access pattern and varied PE runtime with overall <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$684{\times }$ </tex-math></inline-formula> speed up compared with the baseline design. A 28-nm test chip was fabricated demonstrating 500- and 1418-frames/s/W power efficiency in IR and RT modes, respectively, achieving <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$28.8{\times }$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3.95{\times }$ </tex-math></inline-formula> higher RT rendering efficiency compared with existing ASIC solutions, and having an average performance of 25.8 frames/s on various testing scenes, enabling real-time physical-based RT rendering on mobile edge devices.

In order to meet the current requirements for green and high-quality development, it is necessary to carry out energy-saving renovation on the old existing data center computer room. In the China Mobile Information Port No. 1 Data Center Renovation Project, a comprehensive evaluation of the operating status of the computer room was carried out to address issues such as the long distance of the original precision air conditioning supply and the susceptibility to local hotspots. Computational Fluid Dynamics (CFD) simulation software 6SigmaRoom was used to simulate and study energy-saving renovation schemes. By analyzing the air flow organization and energy-saving effects of the air conditioning, the top mounted heat pipe air conditioning+enclosed cold channel scheme was selected to renovate the computer room. By simulating the power consumption within 144 hours after renovation, it was calculated that the PUE factor at the end of the air conditioning system decreased to 0.014, resulting in a power saving rate of approximately ${9 2 \%}$. The research results of this article can provide reference for energy-saving transformation in existing data center computer rooms.

This work presents a special unified compute-in-memory (CIM) processor supporting both general-purpose computing and deep neural network (DNN) operations, referred to as the general-purpose CIM (GPCIM) processor. By implementing a unique CIM macro with two different bitcell arrays and a central compute unit (CCU), GPCIM can be reconfigured to a CIM DNN accelerator or a CIM vector central processing unit (CPU). By using special reconfigurability, dataflow, and support of a customized vector instruction set, GPCIM achieves SOTA performance for end-to-end deep learning tasks with enhanced CPU efficiency and data locality. A 65 nm test chip was fabricated demonstrating a 28.3 TOPS/W DNN macro efficiency and a best-in-class peak CPU efficiency of 802 GOPS/W. Benefit from a data locality flow, 37%–55% end-to-end latency reduction on artificial intelligence (AI)-related applications is achieved by eliminating inter-core data transfer in traditional heterogeneous system-on-chip (SoC). An averaged <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$17.8{\times }$ </tex-math></inline-formula> CPU energy efficiency improvement is achieved compared with vector RISC-V CPUs in the existing machine learning (ML) SoCs.

Physics-embedded neural networks have recently gained significant interest in robotics due to the benefits of combining data-driven machine learning approaches with physics-based modeling methods for real-time control. Despite the improved accuracy over black-box neural networks, existing works have limitations in handling large ranges of system parameters, extracting latent physical parameters, and meeting real-time latency constraints. This paper proposes enhanced physics-embedded neural network models that overcome the scaling limitation of existing models and coupling issues in the extraction of hidden variables, rendering significantly improved model accuracy by more than 95%. A reinforcement learning based neural architecture search engine is developed to meet real-time latency constraints in embedded microprocessors, optimize the solution for scaling issues, and enable the efficient deployment of physics-embedded neural networks into resource-limited edge devices, with 3X searching speed compared with the exhaustive random search method.

As generative AI such as ChatGPT rapidly evolves, the increasing incidence of data misconduct such as the proliferation of counterfeit news or unauthorized use of Large Language Models (LLMs) presents a significant challenge for consumers to obtain authentic information. While new watermarking schemes are recently being proposed to protect the intellectual property (IP) of LLM, the computation cost is unfortunately too high for the targeted real-time execution on local devices. In this work, a specialized hardware-efficient watermarking computing framework is proposed enabling model authentication at local devices. By employing the proposed hardware hashing for fast lookup and pruned bitonic sorting network acceleration, the developed architecture framework enables fast and efficient watermarking of LLM on the small local devices. The proposed architecture is evaluated on Xilinx XCZU15EG FPGA, demonstrating 30x computing speed-up, making this architecture highly suitable for integration into local mobile devices. The proposed algorithm to architecture codesign framework offers a practical solution to the immediate challenges posed by LLM misuse, providing a feasible hardware solution for Intellectual Property protection in the era of generative AI.

This work presents a footstep planning SoC chip for humanoid robot. A time-domain graph search engine for 3D footstep planning and mixed-signal zero moment point (ZMP) gait scheduler with neural inverse kinematics is developed for efficient robot motion control. A 65nm SoC chip is fabricated and demonstrated in-situ on a humanoid robot with the state-of-the-art search rate and energy efficiency for humanoid robot control and footstep planning.

Virtual Reality (VR) and Mixed Reality (MR) systems, e.g., Meta Quest and Apple Vision Pro, have recently gained significant interest in consumer electronics, creating a new wave of developments in metaverse for gaming, social networking, workforce assistance, online shopping, etc. Strong technological innovations in AI computing and multi-modular human activity tracking and control have produced immersive virtual realistic user experiences. However, most existing VR headsets only rely on traditional joysticks or camera-based user gestures for input control and human tracking, missing an important source of information, namely, brain activity. Hence there is a growing interest in incorporating brain-machine interfaces (BMIs) into VR/MR systems for consumer and clinical applications [1]. As illustrated in Fig. 33.2.1, an existing VR/MR system integrated with EEG channels typically consists of a VR headset, a 16/32-channel EEG cap, a neural recording analog frontend, and a PC for signal classification. Major drawbacks of such systems include: (1) cumbersome wear and poor user appearance, (2) lack of in situ computing support for low-latency operation, (3) inability for real-time mind imagery control and feedback based on brain activity, (4) high power consumption due to AI classification. To overcome these challenges, this work introduces a mind imagery device integrated into existing VR headsets without extra wearing burden for mind-controlled BMI for a VR/MR system. The contributions of this work include: (1) an SoC supporting in situ mind imagery control for VR/MR systems, (2) seamless integration with existing VR headset and optimized selection of EEG channels to enhance user acceptance and experience, (3) a general-purpose instruction set architecture (ISA) with flexible dataflow, supporting a broad range of mind imagery operations, (4) a confusion-matrix-guided teacher-student CNN scheme to save power during AI operations, (5) sparsity enhancement on EEG signals to reduce energy. A 65nm SoC test chip is fabricated with in situ demonstrations on various mind imagery-based VR controls. While prior works address EEG-based seizure detection or similar biomedical applications [2] –[6], this work focuses on emerging BMI in a VR/MR environment. The digital core of the SoC achieves an energy consumption <1μJ/class for compute-intensive CNN operations thanks to the low-power features and system-level optimizations of the design.

As the applications of Augmented Reality (AR) or Virtual Reality (VR) expand rapidly with the growing demands on enhanced visual realism, photorealistic image generation and insertion has become an essential feature for the emerging AR applications providing real-time workplace/household visual assistance. Physical Based Ray-Tracing (PBRT) is often used where synthesized images are generated by simulating the real environment and tracing the light transportation to achieve photorealistic effects, such as reflection, refraction, soft shadows, etc. PBRT is widely used in product design, medical visualization, video games and movie effects. To enable photorealistic rendering, there is a strong demand to support ray-tracing (RT) on mobile devices [1]. However, the challenges are: (1) unstructured memory access pattern and complex control flow lead to scheduling difficulty; (2) high memory requirements exhaust the limited SRAM space on edge devices; (3) low error tolerance requires high precision for computing; (4) complex computations, such as division and square root, require significant computing resources for the edge devices. As a result, common rendering engines such as Apple ARKit, OpenGL, are mainly based on the lower cost rasterization rendering technique. Unfortunately, rasterization rendering fails to produce photorealistic synthesis as shown in Fig. 2.5.1. Few ASICs have been fabricated so far as a mobile photorealistic rendering solution solution, however, they may not support RT [2], or may suffer from low efficiency [3]. This work has developed a ray-tracing processor, which also supports inverse rendering (IR) for background extraction [4]. The key features of this work include: (1) an ASIC rendering processor that embeds an end-to-end PBRT solution with IR for AR on mobile devices, (2) a reconfigurable mixed-precision PE design supporting diverse computing tasks for both IR and RT, (3) background clustered Field of View (FOV)-focused 3D construction reducing conventional background scene complexity from O(nlogn) to O(1), (4) scalable partitioning scheme for complex 3D objects, with an average of $13 \times$ speed up on test scenes, (5) use of Global RT Scheduler (GRTS) and Global Memory Access Controller (GMAC) to overcome the challenges of irregular memory access pattern and varied PE run-time with overall $684 \times$ speedup compared with the baseline design. The 28nm test chip achieves $3.95 - 28.8 \times$ higher rendering efficiency compared with existing ASIC solutions, enabling real-time PBRT rendering on mobile edge devices.

The demand for real-time computing on edge devices from emerging applications, e.g. AI, has exploded in recent years. Lately, physics-based scientific computing has also drawn significant interests driven by the growth of real-time applications, e.g., VR, IoT, robotics, etc. Fig. 20.4.1 shows examples of real-time physics-based computation including structural deformation in photorealistic VR/MR, robot dynamic control, temperature monitoring in additive manufacturing, and real-time leak-gas tracking. Unfortunately, hardware support for numerical scientific computing on edge devices is relatively poor, hindering the use of high-accuracy, high-resolution physics-based computing in real time. Figure 20.4.1 shows an example of beam deformation analysis in VR/MR falling short of a real-time latency target using classic solvers due to the large number of iterations for convergence. Recently, ASIC solvers have been designed to solve Poisson equation-related applications with a finite difference method (FDM), but have trouble handling more complex structures [1–3]. To overcome the real-time hurdle, physics-informed neural network (PINN) or physics-informed machine learning (PIML) solutions [4–7] are being developed by the scientific community, using a data-driven approach to boost the computing efficiency of physics solvers. Figure 20.4.1 shows PINN solutions can reach 1900-10000× speedup compared with classic solvers based on Nvidia Modulus with less than 1% accuracy loss [4]. However, if numerous physics equations are to be processed by a PINN, highly diversified dataflows are needed to support a variety of PINN models, making it unfriendly to an ASIC solution. In addition, a tradeoff of speed and accuracy needs to be made between a PINN and classic numerical solutions for a specific application. To overcome these challenges, this work presents a unified physics computing unit (PhyCU) architecture supporting both PINNs and classic finite element method (FEM) solution. The highlights of PhyCU are as follows: 1) This work delivers an ASIC solution supporting inference for most major PINN models with configurable dataflow; 2) The PhyCU architecture also natively supports the classic FEM through a conjugate gradient iterative method (CG) providing a high-accuracy alternative using the same hardware; 3) Sparsity and data compression techniques for both PINN and FEM computation are developed achieving orders of magnitude latency reduction compared with a classic solution on GPU and 19.5-35.9× energy savings compared with prior ASICs.

Modern microprocessors such as CPU, GPU, and the recent deep learning accelerators exhibit significant runtime timing variation, i.e. dynamic timing slack due to the diverse instructions and programs being executed inside the processor cores. Many studies show that only in a small fraction of the system execution, e.g. 13% of the time, the processors fully occupy its dedicated clock cycle. This brings a new opportunity to enhance the processors and accelerators’ performance by exploiting the dynamic timing slack based on the instructions being executed inside the programs. This chapter presents the recent developments on the “dynamic timing enhanced computing scheme” where excessive runtime timing margin is utilized for boosting the computing performance on both microprocessors and deep learning accelerators. Simulation and test chip measurement results are presented to elaborate the benefits of the dynamic timing enhanced computing scheme in terms of performance and energy saving on modern processors. Both hardware design and software techniques using compiler optimization are presented as a holistic solution.

Tropical algebra solves complex problems with only sum and min/max operations replacing expensive multiplication and addition in linear algebra. Due to the low computing cost, tropical algebra has recently gained significant attention in a broad range of areas such as combinatorial optimization, scheduling, machine learning, etc. In this paper, we propose a generic hardware accelerator architecture for tropical algebra supporting a wide range of applications. Novel time-domain (TD) computing accelerators with special mapping, precision expansion and, unrolling techniques are proposed to further improve hardware efficiency. Test results on various tropical calculations including linear regression, dynamic programming, and neural network are shown to demonstrate an energy saving from 1.5X to 2.1X, latency saving from 2.6X to 5.2X, or an overall energy-delay-product (EDP) improvement from 3.9X-10.5X compared with conventional digital implementation manifesting the promise of the algebraic solution on low power edge devices.

This work presents a general-purpose compute-in-memory (GPCIM) processor combining DNN operations and vector CPU. Utilizing special reconfigurability, dataflow, and instruction set, the 65nm test chip demonstrates a 28.5 TOPS/W DNN macro efficiency and a best-in-class peak CPU efficiency of 802GOPS/W. Due to a data locality flow, 37% to 55% end-to-end latency improvement on AI-related applications is achieved by eliminating inter-core data transfer.

This paper presents a distributed multi-chip human activity recognition system for Virtual Reality (VR) and Augmented Reality (AR) applications. A comprehensive solution is delivered including AI core for classification, analog sensing for neural activity detection and infrared data communication for multi-chip collaboration. A $65\mathrm{~nm}$ test chip is fabricated and distributed across the body area to demonstrate the low power, low latency, and camera-free features of the target applications.

In collision processes such as Compton scattering, radiative electron capture (REC) and resonant transfer and excitation (RTE) and so on, the shape of their cross sections in the impulse approximation can be determined by the so-called Compton profile (CP), which reflects the initial distribution in the momentum space for bound electrons in atomic and molecular targets. In this paper, the pseudospectral-fitting (PSF) method is proposed to obtain the radial wave function in position space numerically, which is proved equivalent to the analytical solution, and then the radial wave function in momentum space can be calculated by the series method analytically in order to avoid the numerical difficulties in highly oscillatory integrals. Finally, taking the ns(n=1−10) orbitals of hydrogenic ions as examples, high-accuracy Compton profile has also been calculated at an arbitrary momentum with enough significant digits by using the Gauss–Legendre quadrature.