Summary
TELESAR IIは、従来のテレイグジスタンスロボットTELESARのように操縦者が遠隔地にロボットを介して入り込み、あたかも自分が遠隔地にいるかのように感じることができるだけでなく、遠隔地にいる人々がロボットではなく操縦者と実際にコミュニケーションをしているように感じることができる相互テレイグジスタンスを実現したロボットシステムです。
ロボットアームに対環境柔軟性を持たすためのインピーダンス制御、操作者に物体との接触感覚を与える遭遇型触力覚提示、ロボットへ操縦者を投影する再規制投影技術など多くの技術の統合システムとして研究開発が進められました。2005年の愛知万博に於いて、TELESAR IIを用いたデモンストレーストレーション実験が成功裏に行われ、相互テレイグジスタンスの概念とその有効性が実証されました。
DETAILS
There have been several commercial products with the name telepresence, such as Teliris telepresence videoconferencing system, Cisco telepresence, Polycom telepresence, Anybots QB telepresence robot, Texai remote presence system, Double telepresence robot, Suitable Beam remote presence system, and VGo robotic telepresence. Current commercial telepresence robots that are controlled from laptops or intelligent-pads could provide a certain sense of their presence on the side of the robot, but the remote user has a poor sense of self-presence. As for the sense of their presence, commercial products have problems that the image presented on a display is only a two-dimensional face, which is far from real, and that multi-viewpoint images are not provided; this results in the same front face being seen even when viewed from the side.
The ideal systems should be mutual telexistence systems, which provide both the sense of self-presence and their presence, i.e., the user should get a feeling of being present in the remote environment where his or her surrogate robot exists, and at the same time, the user who remotely visited the surrogate robot’s location could be seen naturally and simultaneously by several people standing around the surrogate robot, as if he actually exists there. However, almost none of the previous systems could provide both the sense of self-presence and the sense of their presence.
Figure 1 shows the conceptual sketch of an ideal mutual telexistence system using a telexistence cockpit such as TWISTER and an avatar robot. User A can observe remote environment [B] using an omnistereo camera mounted on surrogate robot A. This provides user A with a panoramic stereo view of the remote environment displayed inside the cockpit. User A controls robot A' using the telexistence master-slave control method. Cameras B' and C' mounted on the booth are controlled by the position and orientation of users B and C, respectively, relative to robot A'. Users B and C can observe different images of user A projected on robot A' by wearing their own head mounted projectors (HMP) to provide the correct perspective. Since robot A' is covered with retroreflective material, it is possible to project images from both cameras B' and C' onto the same robot while having both images viewed separately by users B and C.
Figure 1 Proposed mutual telexistence system using RPT.
A method for mutual telexistence based on the projection of real-time images of the operator onto a surrogate robot using the RPT was first proposed in 1999 [1], together with several potential applications such as transparent cockpits [2], and the feasibility of the concept was demonstrated by the construction of experimental mutual telexistence systems in 2004 [3]. In 2005, a mutual telexistence master-slave system called TELESAR II was constructed for the Aichi World Exposition.
TELESAR II is composed of three subsystems: a slave robot, a master cockpit, and a viewer system, as shown in Fig. 2.
Figure 2 Schematic diagram of mutual telexistence system TELESAR II.
The robot has two human-sized arms and hands, a torso, and a head. Its neck has two degrees-of-freedom (DOFs), by which it can rotate around its pitch and roll axes. The robot has four pairs of stereo cameras located on top of its head for a three-dimensional surround display, so that an operator can see the remote environment naturally with a sensation of presence. A microphone array and a speaker are also employed for auditory sensation and verbal communication. Each arm has seven DOFs, and each hand has five fingers with a total of eight DOFs [4].
The cockpit consists of two master arms, two master hands, the aforementioned multi-stereo display system, speakers, a microphone, and cameras for capturing the images of the operator in real time. For the operator to move smoothly, each master arm has a six-DOF structure so that the operator's elbow is free of constraints. To control the redundant seven DOFs of the anthropomorphic slave arm, a small orientation sensor is mounted on the operator's elbow. Therefore, each master arm can measure the seven-DOF motions of the corresponding slave arm, whereas the force from each slave arm is transmitted back to the corresponding master arm with six DOFs.
The most distinctive feature of TELESAR II is the use of an RPT viewer system. Both the motion and visual image of the operator are important factors to be determined for the operator to perceive existence at the place where the robot is working. In order to view the image of the operator on the slave robot as if the operator is inside the robot, the robot is covered with a retroreflective material, and the image captured by the camera in the master cockpit is projected on TELESAR II.
TELESAR II acts as a screen, and a person seeing through the RPT viewer system observes the robot as though it were the operator because of the projection of the real image of the operator on the robot. The face and chest of TELESAR II are covered with retroreflective material. A ray incident from a particular direction is reflected in the same direction from the surface of the retroreflective material. Because of the characteristics of retroreflective materials, an image is projected on the surface of TELESAR II without distortion. Since many RPT projectors are used in different directions, and different images are projected corresponding to the cameras located around the operator, the corresponding images of the operator can be viewed (Fig. 3).
Figure 3 Principle of the Retroreflective Projection Technology (RPT) and multi-projection images from different angles at the same time.
Figure 4 (left view) illustrates the telexistence surrogate robot TELESAR II as a virtual exoskeleton human amplifier of a remote operator by showing his image as if he is inside the robot, and Fig. 4 (right view) shows the operator who is telexisting in the TELESAR II robot, feeling as if he were inside the robot. Nonverbal communication such as gestures and handshakes could be performed in addition to conventional verbal communication, because a master-slave manipulation robot was used as the surrogate for a human [5]. Moreover, a person who remotely visited the surrogate robot’s location could be seen naturally and simultaneously by several people standing around the surrogate robot, so that mutual telexistence is attained.
Figure 4 Mutual telexistence using avatar robot (left) and operator at the control (right).
PDFs of the following papers are available at https://tachilab.org/publications/.
[1] S. Tachi, “Augmented Telexistence,” in Y.Ohta and H.Tamura ed., Mixed Reality - Merging Real and Virtual Worlds, Springer-Verlag, ISBN3-540-65623-5, pp. 251-260, 1999.
[2] S. Tachi, N. Kawakami, M. Inami and Y. Zaitsu, “Mutual Telexistence System Using Retro-reflective Projection Technology,” International Journal of Humanoid Robotics, vol.1, no.1, pp.45-64, 2004.
[3] S. Tachi, M. Inami and Y. Uema, “The Transparent Cockpit,” IEEE Spectrum, vol.51, no.11, pp.52-56, 2014.
[4] R. Tadakuma, Y. Asahara, H. Kajimoto, N. Kawakami and S. Tachi, “Development of Anthropomorphic Multi-D.O.F. Master-Slave Arm for Mutual Telexistence,” IEEE Transactions on Visualization and Computer Graphics, vol.11, no.6, pp.626-636, 2005.
[5] S. Tachi, N. Kawakami, H. Nii, K. Watanabe and K. Minamizawa, “TELEsarPHONE: Mutual Telexistence Master Slave Communication System based on Retroreflective Projection Technology,” SICE Journal of Control, Measurement, and System Integration, vol.1, no.5, pp.335-344, 2008.