我们最近发表在《IEEE CASE 2023》的会议论文，提出了一种增强水下人机互动的新方法，解决了潜水员在水下人机交互时面临的挑战。该论文的公共第一作者郭宇芹、张镕正为南方科技大学机械与能源工程系及设计学院硕士研究生，合作作者包括南方科技大学机械与能源工程系本科生邱望宏杰，MIT机械工程系教授Harry Asada（外肢体机器人概念的开创者），本文的共同通讯作者是设计学院助理教授万芳、机械与能源工程系助理教授宋超阳。
Our study presents a novel approach to enhance human-robot interactions underwater by recognizing human intentions, addressing the challenges divers face when communicating while submerged. We developed a compact, wearable system that simultaneously detects head motion and throat vibrations, successfully translating these signals into control commands for an underwater superlimb, thus reducing the physical and mental demands on divers. These findings lay the groundwork for further research in underwater intention recognition and its applications.
doi: https://doi.org/10.1109/CASE56687.2023.10260480
在我们的研究中，我们介绍了一种创新的方法，旨在提高水下环境中的人机交互。我们的研究专注于解决潜水员在水下识别其意图的复杂挑战。这项研究对于开发智能可穿戴系统，以实现水下的人机交互具有重要意义。
In our study, we introduce a pioneering approach to enhance human-robot interactions in underwater environments. Our research is focused on addressing the complex challenge of recognizing human intentions while diving beneath the waves. This research has significant implications for the development of intelligent wearable systems for underwater human-robot interactions.
水下的挑战｜The Underwater Challenge
水下世界为潜水员提出了独特的挑战，特别是在有效表达他们的意图方面。潜水员通常需要在三维空间内保持精确的体态，同时操作设备，例如工具，这是具有体力和精神负担的工作。目前，潜水员的通信方式有限，传统的手势是在水下传达意图的最常见方式。
The underwater realm presents unique challenges for divers, particularly when it comes to expressing their intentions effectively. Divers often need to maintain precise body postures in three dimensions while operating equipment, such as tools, which is physically demanding and mentally exhausting. Current communication methods for divers are limited, and traditional hand gestures are the most common way of conveying intentions underwater.
然而，这些手势并不总是实际的，特别是当潜水员需要平衡他们的身体时，而他们的双手正在执行任务，例如工具操作。水下环境限制了潜水员的感知能力，也限制了通过传统方法进行通信，例如口头表达或面部表情。这种限制使得有必要开发水下的意图识别新方法，以便实现有效的人机交互。
However, these gestures are not always practical, especially when divers need to balance their bodies, and their hands are occupied with tasks like tool operations. The underwater environment restricts divers’ senses and the ability to communicate through conventional methods like verbal expressions or facial cues. This limitation necessitates novel solutions for recognizing human intentions underwater to enable effective human-robot interactions.
从陆地系统汲取灵感｜Drawing Inspiration from On-Land Systems
为了解决这一问题，我们从陆地系统中吸取了灵感，这些系统利用头部运动和喉咙振动来识别人的意图。例如，研究人员已经使用喉咙振动来解释喉咙切除术后模糊发音的言语，以及检测眼部运动和喉咙振动以理解患有肌萎缩侧索硬化症（ALS）的人的意图。其他人已经开发了使用惯性传感器来检测头部运动的系统，以进行意图识别。而且，一些研究人员设计了一种通过头部手势控制的轮椅，以帮助四肢瘫痪的患者。
To address this issue, we drew inspiration from on-land systems that utilize head motion and throat vibrations to recognize human intentions. For instance, researchers have used throat vibrations to interpret speech after laryngectomy surgery and have detected eye motions and throat vibrations to understand the intentions of individuals with amyotrophic lateral sclerosis (ALS). Others have developed systems that use head gestures for intention recognition in various applications.
我们认识到，这些方法在水下情景中也能够适应，特别是当潜水员需要保持平衡，并且他们的双手通常用于保持身体姿势。尽管存在挑战，例如水下环境中的嘈杂信号，但我们的目标是将头部运动和喉咙振动结合在一起，以作为一种表达意图的手段，用于控制水下可穿戴机器人，比如水下外肢体。
We recognized the potential to adapt these methods to underwater scenarios, where divers’ hands are often busy with maintaining body postures. Despite challenges, such as noisy signals in the aquatic environment, we aimed to combine head motion and throat vibration as a means of expressing intentions for controlling underwater wearable robots, like an underwater superlimb.
我们的方法｜Our Approach
我们的研究引入了一种创新的水下意图识别方法。我们设计了一种紧凑的可穿戴系统，能够同时检测潜水员的头部运动和喉咙振动。该系统旨在通过机器学习算法解释潜水员的头部运动和声音振动，以识别其意图，并将其转化为水下外肢体的控制命令。这种能力旨在减轻潜水员的体力和精神负担，使他们能够在不使用手的情况下自然地互动。
Our study introduces an innovative solution for underwater intention recognition. We designed a compact, wearable system capable of simultaneously detecting the diver’s head motion and throat vibrations. This system is engineered to facilitate multi-modal human-robot interactions with an underwater superlimb, which provides propulsion assistance for divers.
The wearable design includes a customized headband with a waterproof Inertial Measurement Unit (IMU) sensor placed on the top and a throat microphone located on the neck. By utilizing machine learning algorithms, the system interprets the diver’s head motion and vocal vibrations to recognize intentions and translate them into control commands for an underwater superlimb. This capability aims to reduce the physical and mental demands on divers, allowing them to interact naturally without using their hands.
本文贡献｜Key Contributions
我们的研究具有以下几个重要贡献：
Our study makes several noteworthy contributions:
- 创新设计：我们提出了一种新颖的水下意图识别设计，采用头部运动和喉咙振动，以适应潜水情景的紧凑形式。
Novel Design: We propose a novel design for underwater intention recognition that incorporates head motion and throat vibration in a compact form factor suitable for diving scenarios. - 实时分类算法：我们开发了一种实时分类算法，基于头部运动和音阶，用于有效识别水下潜水员的意图。
Real-Time Classification Algorithm: We developed a real-time classification algorithm based on head motion and musical scales to recognize divers’ intentions underwater effectively. - 控制可行性：我们通过控制水下外肢体原型来证明了我们方法的可行性，用于连续的水下推进辅助。
Feasibility for Control: We demonstrated the feasibility of our approach by controlling an underwater superlimb prototype with continuous motion commands for underwater propulsion assistance.
未来方向｜Future Directions
虽然我们的结果是令人鼓舞的，但我们的研究为水下意图识别和具有超肢体支持的人机交互打下了基础。扩大意图表达的词汇和改进系统的性能是正在进行和未来工作的领域。此外，我们计划探索更自然的水下意图表达方法，借鉴了头部运动和喉咙振动的成功组合。
While our results are promising, our study serves as a foundation for future research in underwater intention recognition and human-robot interactions with supernumerary support. Expanding the vocabulary for intention expression and refining the system’s performance are areas of ongoing and future work. Additionally, we plan to explore more natural expressions of intentions underwater, building upon the successful combination of head motion and throat vibrations.
结论｜Conclusion
我们的研究代表了在水下意图识别领域的重要一步，为改进水下人机交互提供了一种创新和实际的方法。通过将头部运动和喉咙振动结合起来识别人的意图，我们旨在使潜水更加高效和便捷。我们的研究结果不仅适用于潜水，还可以应用于具有言语障碍的个体以及在水下和陆地环境中各种人机交互情景。
Our research represents a significant step forward in the development of intelligent wearable systems for underwater human-robot interactions. By combining head motion and throat vibrations to recognize human intentions, we aim to make diving more efficient and accessible. The potential applications of our findings extend beyond diving, offering possibilities for individuals with vocal impairments and various human-robot interaction scenarios in aquatic and terrestrial environments.
我们的研究为水下意图识别领域的研究和发展开辟了新的道路，提供了改进水下人机交互的创新和实际方法。
In summary, our study has opened up new avenues for research and development in the field of underwater intention recognition, providing an innovative and practical approach for enhancing human-robot interactions in the underwater world.
Yuqin Guo#, Rongzheng Zhang#, Wanghongjie Qiu, Harry Asada, Fang Wan* and Chaoyang Song* (2023). “Underwater Intention Recognition using Head Motion and Throat Vibration for Supernumerary Robotic Assistance.” IEEE International Conference on Automation Science and Engineering (CASE). Auckland, New Zealand, on 26-30 August 2023.
doi: https://doi.org/10.1109/CASE56687.2023.10260480
