Researchers developing smart ring for health care and extended reality

A team of researchers in the Penn State School of Electrical Engineering and Computer Science aims to enable health care and extended reality—which encompasses virtual, augmented and mixed reality—with their smart sensing ring, OmniRing.

The ring uses both inertial measurement unit (IMU) sensors, which can capture location, speed and rotation of the fingers, and photoplethysmography (PPG) sensors, which utilize an infrared light to measure volumetric changes in blood circulation. The IMU sensors are used to detect and track finger motion, and the PPG sensors are used to collect and analyze health metrics such as heartrate, glucose levels and oxygen saturation.

“The sensor data offers rich information to be able to capture the stress and mood of the wearer and recommend meditation and relaxation activities,” said Wormley Family Early Career Assistant Professor of Computer Science and Engineering Mahanth Gowda. “We are currently exploring deeper applications at the intersection of sensing, machine learning, and Edge Internet of Things to extend the capabilities of what can be achieved in the convenient small size of a ring.”

The research team’s paper, “One Ring to Rule Them All: An Open Source Smartring Platform for Finger Motion Analytics and Healthcare Applications,” recently won the Best Paper award in the “Edge Internet of Things Artificial Intelligence” category during the Internet of Things Design and Implementation Conference.

The paper details how, unlike smartwatches, the OmniRing doesn’t have a display and so can support a longer battery life. This makes it possible to wear the ring all the time, including while sleeping and swimming, enabling the ring to capture deeper and more intimate levels of sensing information.

“Our hope is to bring the technology to reality,” doctoral student and OmniRing hardware lead Taiting Lu said. “Because the ring can have a lot of potential to help people in the future, we don’t want to limit it to the lab but bring it to the real world.”

To meet that end, the research team became the first ones to develop a smart ring with an open-source design, meaning they have fully disclosed all details on their hardware, software and firmware. This will allow the research community to find and develop potentially interesting use cases for the technology.

Lu emphasized that the project is an accomplishment of teamwork, with every member playing a crucial role in bringing the technology to reality. The research team includes, along with Gowda and Lu, doctoral candidates Hao Zhou, Yilin Liu, Shijia Zhang and Runze Liu.

“We envision a significant role for OmniRing in the future: individuals will be able to effortlessly interact with their connected devices by utilizing hand motions,” said doctoral student and OmniRing machine learning lead Hao Zhou. “Personal health and emotional data can be selectively relayed in real-time, enabling surrounding devices to interpret the information and respond appropriately. To put it another way, our goal is to strive to ensure that OmniRing deserves its name.”

In addition to health care and XR, the ring can potentially enable sign language recognition and translation by integrating with other sensors, said the researchers.

In order to do this, Gowda’s lab is actively collaborating with Kenneth DeHaan, director of the Master of Arts in Sign Language Education program at Gallaudet University in Washington, D.C., and a member of the Deaf community.

Lu explained that his research team and DeHaan explored challenges that could be addressed with such technology. DeHaan highlighted one common situation to the team: “Often in a public setting such as a coffee shop, for instance, baristas or servers are often impatient to assist as a customer is writing down their order, demeaning the customer and their shopping experience. This is why we want to even the playing field with this technology.”

In spring 2023, Lu and his team participated in the Invent Penn State NSF I-Corps Short Course program, which helps researchers connect with and interview potential customers to learn what potential iterations to make to their technology, as well as how to best present and talk about the technology, prior to entering the market.

“The biggest benefit of the short course is that it taught me how to go about finding the needs or the difficulties of the customers,” Lu said. “We can think our product is the best and can solve all these problems, but the reality is, the first thing we need to do is listen to opinions from partners and customers and from the community, then we can make a good decision about which application of our technology makes the most sense.”

The OmniRing team also participated in the Happy Valley LaunchBox powered by PNC Bank Idea TestLab, a 4-week program that helps anyone with an idea connect with potential customers to better define the problem and a potential solution. Lu said he and his team also had the opportunity to collaborate with Alert Heart, another Penn State startup, which is developing a device capable of detecting heart attacks wirelessly and alerting the caretakers.

“I would definitely recommend the short course to other researchers,” Lu said. “It would be a good starting point if you have a product and research funding and a prototype. It gives you a lot of experience, and you can learn how to bring the technology from the lab to real life.”

Right now, OmniRing has been tested and is able to detect 3D hand and finger motions, said the researchers. The next step for the team is to program the ring to recognize the hand and finger motions specific to ASL, as well as implement the health care features like heart rate and oxygen saturation monitoring.

The research team is currently seeking collaboration opportunities for research and commercialization, and is hoping to participate in the NSF I-Corps National Teams program this summer to narrow down their market through additional customer interviews.

The study is published as part of the Proceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation.