本论文结合基于指内视觉的软体机器人触觉设计和变分自编码机器学习技术，在非结构水下环境中实现精确的触觉感知、全向适应性和可靠抓取，该论文发表于《Advanced Intelligent Systems》，第一作者南方科技大学博士研究生郭宁，合作作者包括南方科技大学博士研究生韩旭东、刘小博、钟树乔、海洋系讲席教授林间、研究副教授周志远、机械与能源工程系讲席教授戴建生，通讯作者为设计学院助理教授万芳、机械与能源工程系助理教授宋超阳
Achieving precise tactile sensing, omni-directional adaption, and reliable grasping in unstructured aquatic environments using vision-based soft robotics and machine learning.
DOI: https://doi.org/10.1002/aisy.202300382
在水下机器人技术领域，通过增加触觉感知，已经在具有挑战性的水下抓取领域取得了重大突破。为水下机器人带来触觉感知的能力为在无结构水下环境中进行精细且自主的交互开辟了新的可能性，这对于环境、生物和海洋研究等科学活动至关重要。
In underwater robotics, a significant breakthrough has been achieved in the challenging domain of underwater grasping with the addition of tactile sensing. The ability to bring a sense of touch to aquatic robots has opened up new possibilities for conducting delicate and autonomous interactions in the unstructured underwater environment, vital for scientific activities in environmental, biological, and ocean research.
传统水下触觉感知的解决方案多采用机械方法，涉及复杂的密封技术以抵抗液体压力和腐蚀性污染。然而，这些方法通常伴随着工程灵活性和智能感知方面的权衡。本研究提出的创新方法充分利用了软件机器人和机器学习的进展，尤其是基于视觉的触觉感知领域。
Traditionally, solutions for underwater tactile sensing have taken a mechanical approach, involving complex sealing technologies to combat fluidic pressure and corrosive contamination. However, these methods often come with trade-offs in terms of engineering flexibility and intelligent perception. The innovative approach presented in this work leverages advancements in soft robotics and machine learning, particularly in the area of vision-based tactile sensing.
本文章的关键创新点在于设计的简单性，最小化了对复杂机械部件的需求，消除了对动态密封的要求。相反，采用了软触觉网络，具有被动适应性和指内视觉。这些软触觉手指与受监督的变分自动编码器（SVAE）相结合，使其能够通过水下的视觉数据学习触觉感知。
The key innovation lies in the simplicity of the design, which minimizes the need for complex mechanical components, eliminating the requirement for dynamic seals. Instead, soft finger networks are utilized, with passive adaptation and in-finger vision. These soft fingers are integrated with a Supervised Variational Autoencoder (SVAE) to enable the learning of tactile sensing through visual data underwater.
SVAE模型的潜在表示提供了一个生成性解决方案，用于在水下交互过程中推断6D的力和扭矩，实现了超过98%的触觉力预测准确度。在实验室水箱中进行的实时抓取实验强调了这种软触觉手指在水下和陆地场景中可靠且精细的抓取的有效性。
The SVAE model’s latent representations provide a generative solution for inferring forces and torques in 6D during underwater interactions, enabling an impressive tactile force prediction accuracy of over 98%. Real-time grasping experiments in a lab tank underscore the effectiveness of this soft tactile finger for reliable and delicate grasping in both underwater and on-land scenarios.
该工作还涉及模型解释性和泛化的关键方面。通过利用变分自动编码器模型的表示学习能力，研究表明，从不同环境中提取的手指变形的潜在特征呈现相似的分布。这对于维度缩减、统计推断以及卷积神经网络的威力的更深入理解有所贡献。
The work also addresses the crucial aspects of model explainability and generalization. By leveraging the variational autoencoder model’s representation learning capabilities, the research demonstrates that latent features extracted from finger deformations exhibit similar distributions in different environments. This insight into dimension reduction, statistical inference, and the power of convolutional neural networks contributes to a deeper understanding of the learned tactile representations.
尽管水下流体动力学带来的挑战不可避免地导致了从陆地到水下的触觉力预测性能下降，但添加触觉反馈有效地提高了水下抓取的可靠性。该手指的独特设计在闭合时通常会减少流体动力学引起的干扰，这是刚性结构手指通常会遇到的常见问题。
Despite the challenges posed by underwater fluid dynamics, the addition of tactile feedback significantly enhances the reliability of underwater grasping. The finger’s unique design minimizes disturbances caused by fluid dynamics, a common issue with rigid structures, ultimately leading to improved performance.
这项具有开创性的工作突显了水下机器人中触觉感知的重要性。在水下环境中，视觉感知面临挑战，触觉反馈的加入提高了抓取成功率，促进了在具有挑战性的水下情境中的更智能的操作，为未来的优化和工程改进提供了可能的解决方案。
This groundbreaking work highlights the importance of tactile perception in underwater robotics. With the potential to overcome the limitations of visual perception in underwater environments, the study underscores the significant role tactile feedback can play in improving grasping success rates and fostering more intelligent manipulation in challenging underwater scenarios.
尽管这项研究取得了显著成就，但它承认存在一些局限性，为未来的优化提供了机会。其中包括通过在手指表面添加硅皮肤层来减轻背景噪音对视觉输入的影响，通过XMem增强触觉感知，以跟踪手指变形的内部视觉，或使用修复算法来使用内部视觉来进行视觉感知。所提出的水下抓取系统尚未在浅水和深水中的远程操作车辆（ROV）上进行测试，需要进一步进行工程优化。这项工作有望推动水下机器人技术及其在多领域的科学和工业应用的发展。
While this research represents a remarkable achievement, it acknowledges several limitations, offering opportunities for future optimization. These include the mitigation of background noise in visual input, the enhancement of tactile perception through skin augmentation, and further testing on Remotely Operated Vehicles (ROVs) in both shallow and deep waters. This work promises to drive the evolution of underwater robotics and its applications in diverse scientific and industrial domains.
Ning Guo, Xudong Han, Xiaobo Liu, Shuqiao Zhong, Zhiyuan Zhou, Jian Lin, Jiansheng Dai, Fang Wan*, Chaoyang Song*, “Autoencoding a Soft Touch to Learn Grasping from On-land to Underwater,” Advanced Intelligent Systems (2023), 2300382.
DOI: https://doi.org/10.1002/aisy.202300382
