Taiwan at the Forefront of AI Sensing

The 2022 Consumer Electronics Show (CES) attracted participation by 2279 internationally known firms and startup companies from 58 countries and territories around the globe.

Astonishing feats

In 2018, Chiou Chyou-huey, then director general of the Department of Academia–Industry Collaboration and Science Park Affairs at Taiwan’s Ministry of Science and Technology, invited 32 startup companies to form a national team called the Taiwan Tech Arena (TTA), which went to exhibit in the CES startup section Eureka Park. They set up a Taiwan Pavilion there in hopes of boosting Taiwan’s visibility in the international startup community.

Since then, Taiwan has steadily expanded its participation in the CES, and was already making a mark in its second year. In 2020, over 20 products from the Industrial Technology Research Institute (ITRI), startups, and major companies including Acer and Asustek won Innova­tion Awards. In 2022 Taiwan challenged traditional Eureka Park giants like the Netherlands and Israel by organizing 100 startups to exhibit in the section, becoming the world’s second-ranking country behind only France in terms of startup participation.

Taiwan’s innovative strength is also demonstrated by the numerous major international awards won by institutions doing innovative technological R&D through the Technology Development Programs of the Ministry of Economic Affairs.

For example, at CES 2022 ITRI exhibited ten products, of which three won Innovation Awards. All were singled out for the ways they combine artificial intelligence (AI) with sensor technology.

Chiou Chyou-huey, director general of the Department of Industrial Technology, Ministry of Economic Affairs: “The current global trend is for innovation to be done by startups. Startups are gradually replacing the role of large companies in innovative R&D. That’s my interpretation of the CES Innovation Awards.”

iPetWeaR: The first smart radar collar

If a family pet wears an iPetWeaR collar, or lies on an iPetweaR bed, the owner can check up on the animal’s heartbeat, respiration, and activity level at any time using a smartphone or other mobile device.

This cool device was created by researchers at ITRI who sewed a radar sensor antenna into a dog collar to conduct physiological monitoring through low-power radar technology. The information displayed on the owner’s mobile device is generated by a backend AI algorithm that analyses the radar signal to detect any abnormality in the pet’s health or mood. The collar also has a location finding function.

Besides winning a CES Innovation Award, this invention was named by the authoritative British tech website TechRadar as one of “the five quirkiest products we can’t wait to see.” In addition, BGR.in, a leading Indian tech website, ranked iPetWeaR alongside LG’s curved OLED TV as one of the “top five cool products” unveiled at CES.

J.C. Roger Cheng, general director of ITRI’s Service Systems Technology Center (SSTC), points out the key to the popularity of this invention: It marks the first time that ­micro radar sensing technology, which is completely harmless to pets, has been used in a pet healthcare product. The technology has now been licensed to device manufacturers, and in the not-too-distant future products will be available for use in veterinary clinics and pet hotels.

J.C. Roger Cheng, general director of ITRI’s Service Systems Technology Center: “iPetWeaR’s commercial value comes from its simplicity of operation, real-time monitoring, and humane concern for animal welfare.”

ITRI’s Service Systems Technology Center has won CES Innovation Awards for three straight years with applications for smart wearable systems.

Making a slip-free device

The reason they started with a product for pets was initially because Lin Hong-dun, a division director at the SSTC, had a big ginger cat that one day suddenly began to act crazy. As an expert in radar sensing, he thought that radar devices could be used on pets to detect where there might be a problem.

However, despite the SSTC’s great expertise in low-power radar sensing technology, there was a difficulty: Because pets have fur and are very active, devices attached to them can easily slip off. It was essential to overcome the challenges of making a device that can collect data without the need for any of the pet’s fur to be shaved off.

It was also necessary to create the backend smart analysis technology from scratch, constructing mathematical models based on the heartbeat, respiration, and activity levels of different pets.

Pets are like family members, says Roger Cheng. ­iPetWeaR’s commercial value comes from its simplicity of operation, real-time monitoring, and humane concern for animal welfare. In the future it will be possible to create electronic health records from a pet’s birth and immunizations through sickness and old age, and integrate these into comprehensive services linked with veterinary clinics and pet hotels. Moreover, the global market for pet wearable devices is not the only economic opportunity opened up by this technology. Currently ITRI is in talks with large-scale cattle farming interests in places like the US and Australia to develop livestock management devices that will track the physiological condition, milk production, and illnesses of cattle. The commercial possibilities appear unlimited.

MoMo is sitting on a pet bed with a radar antenna inside, so that his physiological condition can be monitored using low-power radar.

Angelabelle is wearing an iPetWeaR collar, enabling her owner to use an app to check on her heartbeat, respiration, and activity level.

RGB-D AI Robot: A sharp-eyed smart cobot

Another honoree of the CES 2022 Innovation Awards is a collaborative robot (cobot) developed by ITRI’s Smart Sensing and Systems Technology Center (SSSTC). It is the first robot in the world to have smart 3D vision sensors and smart object retrieval technology as standard equipment.

This smart and capable robot from Taiwan, built by Techman Robot Inc., the world’s second-largest cobot manu­facturer, is already at work on production lines.

SSSTC general director Chu Chun-hsun says that the reason the SSSTC stands out from the crowd in terms of the Internet of Things and intelligent manufacturing can be found in the “eyes” of industrial robots—image sensors.

The SSSTC used entirely new technology to develop a 3D image sensor only 14 × 8 × 3.5 millimeters in size. This makes it 168 times smaller than its predecessor. It is also 38.6 times faster, and production costs have been reduced by more than half.

This Made-in-Taiwan collaborative robot developed by the Industrial Technology Research Institute is the first robot in the world with built-in smart 3D vision sensors and AI self-learning capability.

A robotic arm with an AI brain

So how remarkable is this industrial robot in terms of global innovation?

Chen Po-lung, director of R&D at the SSSTC, explains that the human eye has no difficulty discerning both very bright and very dark objects. But image sensors do have problems with bright light reflected from metals and with dark objects that absorb a lot of the light that falls on them. In the past engineers had to spend one or two days inputting optical parameters to enable a sensor to clearly distinguish such objects. But for the RGB-D AI Robot, the team built the visual parameters directly into the sensors, raising the 3D reconstruction rate when viewing reflective materials by 2.1 times. Tests have also shown that while adjustment and calibration formerly took eight hours, they can now be completed within 15 minutes.

At present, industrial robots used on production lines around the world cannot be switched at will among different tasks, so that when a company wants to ­alter a ­production line or upgrade a robot’s functions, engin­eers may have to spend a week or two revising its software. When there is incompatibility between equipment and software from different vendors, it is necessary to employ additional engineers to make new adjustments, using up even more time and manpower.

However, after the SSSTC team optimized their algorithm, their AI robot became able to autonomously learn to pick up and place new objects. As a result, when companies want to alter production lines or change materials, a software engineer needs to spend only 15 minutes adjusting the parameters so that the robot can rapidly determine which objects to pick up and how, thereby achieving the goals of accurate workpiece identification, flexible object pickup, and smart classification.

Health monitoring by thermal sensing

Another area of sensor technology in which ITRI has great expertise is thermal imaging. This is widely used in epidemic response technology, in systems that can simultaneously measure the forehead temperatures of multiple people, making it possible to control entry into premises and assisting in contact tracing. The SSSTC has also applied this technology to the needs of residential care facilities, winning a CES 2022 Innovation Award in the process.

According to a survey on remote healthcare conducted by the SSSTC at nursing homes, long-term care facilities, and hospitals, more than 90% of patients do not want their moments of personal privacy (such as dressing and undressing) to be recorded by remote monitoring devices using color images.

SSSTC general director Chu Chun-hsun states that the only way to provide effective remote healthcare while assuring patient privacy is to use infrared imaging. However, it is not possible to accurately interpret the data acquired by the infrared sensing systems currently on the market. This was the motivation for the SSSTC to develop the “All-in-One Thermal Sensing System.”

Every year at the Consumer Electronics Show ITRI focuses on market-oriented products that address human needs. The institute has become a frequent winner of CES Innovation Awards. (courtesy of ITRI)

Supporting respite care

The advantage of using infrared imaging in patient care is that at night when the lights are off it can still be used to monitor patients, yet also assures their privacy. However, the resolution of the infrared images is not as good as color video images.

Chen Po-lung cites an example: When a person gets out of bed, thermal imaging will detect two people, one getting up and the other still lying down. This is because the sensors pick up the residual body heat left on the bed.

The technological breakthrough made by the SSSTC lies in using AI machine learning to discriminate between residual heat, environmental heat, and the heat signal from the actual patient, so that thermal imaging can achieve the required level of precision to accurately measure the patient’s breathing, heartbeat, and body temperature. For example, body temperature measurements made using SSSTC’s technology are accurate to within ± 0.1°C.

What’s most special is that the innovative thermal scanning technology developed by the SSSTC reaches the 1-megapixel level required for effective imaging, greatly reduces the purchase cost of a monitoring system, helps meet the remote diagnosis and remote healthcare needs of clinics and families, and offers a solution to the global shortage of caregivers. Moreover, in the future ITRI plans to extend this technology to health monitoring for sports and fitness activities. Using technology to serve humanity can make the world a better place.

The members of the research team at ITRI’s Smart Sensing and Systems Technology Center have an average of over ten years of experience with sensor technology, so they have a firm grasp of the key core capabilities and can stay ahead of competition from the rest of the world. Third from right in the front row is SSSTC general director Chu Chun-hsun.