summary: Researchers have unveiled a pioneering technology capable of real-time human emotion recognition, promising innovative applications in wearable devices and digital services.
The system, known as a Personalized Skin-Integrated Facial Interface (PSiFI), combines verbal and non-verbal cues through a self-powered stretchable sensor and efficiently processes data for wireless communication.
This breakthrough technology, supported by machine learning, accurately identifies emotions even in mask-wearing situations and has been applied to VR “digital concierge” scenarios, demonstrating the potential to personalize user experiences in smart environments. Masu. This development is a major advance towards enhancing human-machine interactions by integrating complex emotional data.
Important facts:
- Innovative emotion recognition system: A research team at UNIST has developed a multimodal system that integrates verbal and nonverbal expressions for real-time emotion recognition.
- Self-powered and extendable sensor: The PSiFI system utilizes a new self-powered sensor that facilitates the simultaneous capture and integration of diverse emotional data without external power sources.
- Practical applications in VR: This technology, demonstrated in a VR environment, provides personalized services based on the user’s emotions and shows great potential in digital concierge services and other fields.
sauce: UNIST
A breakthrough technology that can recognize human emotions in real time has been developed by Professor Jiyun Kim and his research team at UNIST’s School of Materials Science and Engineering.
This innovative technology is poised to revolutionize various industries, including next-generation wearable systems that deliver services based on emotions.
Due to the abstract and ambiguous nature of human emotions, such as emotions, moods, and emotions, understanding and accurately extracting emotional information has been a long-standing challenge.
To address this, the research team developed a multimodal human emotion recognition system that combines verbal and nonverbal expression data to efficiently utilize comprehensive emotional information.
The core of the system is the Personalized Skin-Integrated Facial Interface (PSiFI) system, which is self-powered, easy, stretchable, and transparent. This product features a first-of-its-kind bidirectional triboelectric strain and vibration sensor that enables simultaneous detection and integration of verbal and nonverbal expressive data.
The system is fully integrated with data processing circuitry for wireless data transfer, enabling real-time emotion recognition.
Utilizing machine learning algorithms, the developed technology demonstrates accurate and real-time human emotion recognition tasks even when the individual is wearing a mask. This system has also been successfully applied to digital concierge applications within virtual reality (VR) environments.
This technology utilizes the phenomenon of “triboelectric charging,” in which an object separates into positive and negative charges due to friction. In particular, the system is self-generating and does not require external power supplies or complex measurement equipment for data recognition.
Professor Kim commented, “Based on these technologies, we developed a skin-integrated face interface (PSiFI) system that can be customized to the individual.” The research team used semi-curing technology to fabricate transparent conductors for triboelectric charging electrodes. Additionally, multi-angle photography techniques were used to create a personalized mask that combines flexibility, elasticity, and transparency.
The research team successfully integrated detection of facial muscle deformation and vocal cord vibration, enabling real-time emotion recognition. The capabilities of this system were demonstrated in a virtual reality “digital concierge” application, where customized services are provided based on the user’s emotions.
Jin Pyo Lee, lead author of the study, said: “With this developed system, it is possible to implement real-time emotion recognition in just a few learning steps, without the use of complex measurement equipment. This will allow us to implement portable emotion recognition devices in the future. and the next generation of emotion-based digital platform services.”
The research team conducted real-time emotion recognition experiments and collected multimodal data such as facial muscle deformation and voice. This system showed high emotion recognition accuracy with minimal training. Its wireless and customizable nature ensures wearability and convenience.
Furthermore, we applied this system to a VR environment and used it as a “digital concierge” in various settings such as smart homes, private movie theaters, and smart offices. The system’s ability to identify an individual’s emotions in different situations allows it to provide personalized recommendations for music, movies, and books.
Professor Kim said, “For effective interaction between humans and machines, human-machine interface (HMI) devices must be able to collect diverse data types and process complex integrated information. This study demonstrates the potential of using emotion, a complex form of human information, in next-generation wearable systems.”
The research was conducted in collaboration with Professor Lee Pui See of Nanyang Technological University in Singapore, and was supported by the National Research Foundation of Korea (NRF) and the Korea Institute of Materials Research (KIMS) under the Ministry of Science, Information and Communications.
About this emotion and neurotechnology research news
author: Heo Joo Hyun
sauce: UNIST
contact: Heo Joo Hyun – UNIST
image: Image credited to Neuroscience News
Original research: Open access.
“Multimodal emotional information encoding via personalized skin-integrated wireless facial interface” by Jiyun Kim et al. nature communications
abstract
Encoding multimodal emotional information via a personalized skin-integrated wireless facial interface
Human influences such as emotions, moods, and moods are increasingly considered as important parameters to enhance the interaction of humans with various machines and systems. However, its inherently abstract and ambiguous nature makes it difficult to accurately extract and exploit emotional information.
Here, we will develop a multimodal human emotion recognition system that can efficiently utilize comprehensive emotional information by combining verbal and nonverbal expression data.
The system consists of a self-powered, easy, stretchable, transparent personalized skin-integrated facial interface (PSiFI) system that is the first bidirectional system capable of sensing and combining verbal and non-verbal expressive data. Equipped with triboelectric strain and vibration sensors. first time. It is fully integrated with data processing circuitry for wireless data transfer and can perform real-time emotion recognition.
With the help of machine learning, we demonstrated a digital concierge application in a VR environment by accurately performing various human emotion recognition tasks in real-time even while wearing a mask.