This paper presents HNPU, which is an energy-efficient DNN training processor by adopting algorithm-hardward co-design. The HNPU supports stochastic dynamic fixed-point representation and layer-wise adaptive precision searching unit for low-bit-precision training. It additionally utilizes slice-level reconfigurability and sparsity to maximize its efficiency both in DNN inference and training. Adaptive-bandwidth reconfigurab…

Generative adversarial networks (GAN) have a wide range of applications, from image style transfer to synthetic voice generation [1]. GAN applications on mobile devices, such as face-to-Emoji conversion and super-resolution imaging, enable more engaging user interaction. As shown in Fig. 7.4.1, a GAN consists of 2 competing deep neural networks (DNN): a generator and a discriminator. The discriminator is trained, while the …

Recently, deep neural network (DNN) hardware accelerators have been reported for energy-efficient deep learning (DL) acceleration [1-6]. Most prior DNN inference accelerators are trained in the cloud using public datasets; parameters are then downloaded to implement AI [1-5]. However, local DNN learning with domain-specific and private data is required meet various user preferences on edge or mobile devices. Since edge and …

Recently, deep neural networks (DNNs) are actively used for object recognition, but also for action control, so that an autonomous system, such as the robot, can perform human-like behaviors and operations. Unlike recognition tasks, real-time operation is important in action control, and it is too slow to use remote learning on a server communicating through a network. New learning techniques, such as reinforcement learning…

Recently, 3D hand-gesture recognition (HGR) has become an important feature in smart mobile devices, such as head-mounted displays (HMDs) or smartphones for AR/VR applications. A 3D HGR system in Fig. 13.4.1 enables users to interact with virtual 3D objects using depth sensing and hand tracking. However, a previous 3D HGR system, such as Hololens [1], utilized a power consuming time-of-flight (ToF) depth sensor (>2W) lim…

Deep neural network (DNN) accelerators [1-3] have been proposed to accelerate deep learning algorithms from face recognition to emotion recognition in mobile or embedded environments [3]. However, most works accelerate only the convolutional layers (CLs) or fully-connected layers (FCLs), and different DNNs, such as those containing recurrent layers (RLs) (useful for emotion recognition) have not been supported in hardware. …

Recently, deep learning with convolutional neural networks (CNNs) and recurrent neural networks (RNNs) has become universal in all-around applications. CNNs are used to support vision recognition and processing, and RNNs are able to recognize time varying entities and to support generative models. Also, combining both CNNs and RNNs can recognize time varying visual entities, such as action and gesture, and to support image …

Recently, face recognition (FR) based on always-on CIS has been investigated for the next-generation UI/UX of wearable devices. A FR system, shown in Fig. 14.6.1, was developed as a life-cycle analyzer or a personal black box, constantly recording the people we meet, along with time and place information. In addition, FR with always-on capability can be used for user authentication for secure access to his or her smart phon…

Micro robots with artificial intelligence (AI) are being investigated for many applications, such as unmanned delivery services. The robots, shown in Fig. 14.3.1, have enhanced controllers that realize AI functions, such as perception (information extraction) and cognition (decision making). Historically, controllers have been based on general-purpose CPUs, and only recently, a few perception SoCs [1-3] have been reported. …

The robots with artificial intelligence (AI) are being investigated for many unmanned systems, such as unmanned delivery services. The robots are rapidly getting smaller and faster along with the advances in robotics technology, and they have to perform more complicated tasks in dynamically changing environments. Therefore, an ultra-low-power and high-performance artificial intelligence processor (AIP) is necessary for inte…

Advanced driver assistance system (ADAS) for adaptive cruise control and collision avoidance is strongly dependent upon the robust image recognition technology such as lane detection, vehicle/pedestrian detection and traffic sign recognition. However, the conventional ADAS cannot realize more advanced collision evasion in real environment due to the absence of intelligent vehicle/pedestrian behavior analysis. Moreover, accu…

Recently, intelligent robotics and automation have been widely used for various applications such as advanced driver assistance system (ADAS), autonomous driving technology, and robot control system. These require a real-time object detection as well as artificial intelligences (AI) such as path planning, cut in detection, behavior analysis, and localization. However, the heavy computation cost of those algorithms makes it …

Wearable head-mounted display (HMD) smart devices are emerging as a smartphone substitute due to their ease of use and suitability for advanced applications, such as gaming and augmented reality (AR). Most current HMD systems suffer from: 1) a lack of rich user interfaces, 2) short battery life, and 3) heavy weight. Although current HMDs (e.g. Google Glass) use a touch panel and voice commands as the interface, such interfa…

A low-power object recognition (OR) system with intuitive gaze user interface (UI) is proposed for battery-powered smart glasses. For low-power gaze UI, we propose a low-power single-chip gaze estimation sensor, called gaze image sensor (GIS). In GIS, a novel column-parallel pupil edge detection circuit (PEDC) with new pupil edge detection algorithm XY pupil detection (XY-PD) is proposed which results in 2.9x power reductio…

Real-time augmented reality (AR) is actively studied for the future user interface and experience in high-performance head-mounted display (HMD) systems. The small battery size and limited computing power of the current HMD, however, fail to implement the real-time markerless AR in the HMD. In this paper, we propose a real-time and low-power AR processor for advanced 3D-AR HMD applications. For the high throughput, the proc…