Proceedings of EuroHaptics 2004, Munich Germany, June 5-7, 2004.
A Novel Haptic Sensor-Actuator System for Virtual Reality H. Böse1*, H. Ermert2a, A. Tunayar3, G. Monkman4, M. Baumann1, W. Khaled2a, S. Reichling2b, O. T. Bruhns2b, H. Freimuth3, and S. Egersdörfer4 1
Fraunhofer-Institut für Silicatforschung, Neunerplatz 2, 97082 Würzburg, Germany Ruhr-Universität Bochum, Institut für Hochfrequenztechnik, Gebäude IC 6/132, 44780 Bochum, Germany 2b Ruhr-Universität Bochum, Lehrstuhl für Technische Mechanik, Gebäude IA 3/26, 44780 Bochum, Germany 3 Institut für Mikrotechnik Mainz, Carl-Zeiss-Straße 18-20, 55129 Mainz, Germany 4 Fachhochschule Regensburg, Fachbereich Elektrotechnik, Prüfeninger Straße 58, 93049 Regensburg, Germany * E-mail:
[email protected] 2a
Abstract. The development of a novel system for the generation and representation of haptic information in virtual reality is described. With this system the stiffness distribution of mechanically inhomogeneous objects can be detected and made perceivable for users at distant locations. The sensor part is based on ultrasonic elastography and the actuator part utilizes the ability of electrorheological fluids to change their consistency in electric fields reversibly. Two-dimensional elastographic images or arbitrary projections of threedimensional objects generated in the sensor part can principally be represented by the actuator part, which has a flat surface above a two-dimensional array of actuator elements with individually addressable stiffness.
1 Introduction In haptics it may be distinguished between kinesthetic and tactile perception [1]. Kinesthetic perception is the registration of macroscopic forces such as the weight of an object, whereas tactile perception describes the feel of the surface properties of the object, e. g. roughness. By squeezing the object and feeling its stiffness, a complex perception mechanism corresponding to kinesthetic as well as to tactile information occurs. In the case of mechanically inhomogeneous objects, the problem of representing the stiffness information is a multidimensional one and requires sophisticated technical solutions. However, such a system would have a strong impact on medical applications and could lead to new diagnosis and treatment techniques. An example is the palpation of body organs and tissue by the physician. Palpation is the commonly used method to detect hard inclusions in soft tissue. Such regions of increased stiffness could be an indication of pathological changes in tissue. However, the possibility of directly feeling the hardness of tissue decreases corresponding to the increasing importance of techniques of minimally invasive surgery.
88
In the following, the concept of a novel sensor-actuator system and several steps of its realization are described. In its sensor part the system detects information about the stiffness distribution of mechanically inhomogeneous objects and then displays this information on a virtual object, i. e. the actuator part.
2 Concept of the Haptic System Due to the lack of commercially available technical solutions, innovative approaches are necessary to fulfill the requirements of the haptic sensor-actuator system. The technique of ultrasonic elastography in the sensor part as well as the use of electrorheological fluids in the actuator part have been identified as the basic mechanisms for the detection and representation of the stiffness information. In ultrasonic elastography small deformations are determined between successive ultrasonic image pairs, which are acquired under small axial compression of the investigated object. The derivative of the displacement is equal to the strain, which can be used to differentiate between rigid and soft regions in the object [2]. Since hard regions in the object are less deformed than soft regions, the strain can be considered as an inverse measure for the stiffness.
Fig. 1. Scheme of the haptic sensor-actuator system Electrorheological (ER) fluids are smart materials, whose consistency can be drastically, quickly and reversibly influenced by electric fields [3]. They consist of electrically polarizable particles in a non-conducting carrier liquid. The electrically induced transition changes the consistency of the ER fluid from a liquid to a gelleous state. Some approaches in which ER fluids have been exploited to generate haptic effects have already been reported [4-6]. The actuator part of our haptic system consists of a two-dimensional array of small elements filled with the ER fluid. The lateral resolution of the actuator array has been targeted to be about 2 mm, which is in rough accordance with the surface density of receptors on the human fingertip [1].
89
Fig. 1 gives an overview on the configuration of the haptic sensor-actuator system. Depicted is the ultrasound system including the transducer on the mechanically inhomogeneous object as well as the actuator part including the ER fluid in the array and the regulation unit. The latter applies voltage to each of the actuator elements individually in order to generate the corresponding stiffness. The transfer of haptic data from the sensor to the actuator part of the system can principally be achieved by direct connection or by telecommunication.
3 System Development and Preliminary Results 3.1 Sensor Part The sensor part consists of a standard ultrasound device, equipped with a 9 MHz linear array transducer and a workstation. Fig. 2a demonstrates that the hard inclusion in the object investigated is not visible in the conventional ultrasonic image (B-Scan) because the image is isoechoic. The object is a specially designed ultrasonic tissuelike agar phantom containing a harder cylindrical-shaped agar inclusion, where the ratio of Young’s moduli of the hard and soft material approaches the corresponding ratio of tumour and healthy tissue. However, in the elastogram (strain image) derived from successive B-scans, the hard inclusion can clearly be observed (see Fig. 2b). soft
(b)
(a)
soft rigid
rigid B-Scan image (hard inclusion not visible)
Elastogram (strain image)
Fig. 2. (a) Ultrasonic B-scan image and (b) elastogram (strain image) of a phantom
After acquiring a series of 2D elastograms, the 3D volume of the object is created by placing each image at the proper location in the volume. From a 3D elastogram arbitrary 2D sections can be selected in order to represent the corresponding haptic information on the actuator part of the system. In addition to hard lumps in soft tissue elastography also enables to detect calcifications in blood vessels, for example in coronary arteries. This offers the possibility of representing haptic information from inner regions of arteries which cannot be palpated directly. The prospects for a better medical diagnosis are enhanced by such methods.
90
3.2 Actuator Part The basis of the actuator part of the haptic system is a two-dimensional array of elements filled with an ER fluid. Various working mechanisms of single electrorheological actuators have been evaluated in order to find a promising actuator principle. The actuator should generate high resistance forces against the compression of its top surface and be suitable for miniaturization. For the ER fluid based haptic actuator a special design has been selected, where a flat metallic piston is vertically guided between two electrodes. The gaps between piston and electrodes are filled by the ER fluid. When pressing onto its electrically isolated head at the top, the piston is moved down, shearing the ER fluid. The force perceived by the user depends on the electric field strength applied and is proportional to the effective electrode surface. (a)
3,5
(b)
0 kV/mm 0,5 kV/mm 1 kV/mm 1,5 kV/mm 2 kV/mm 2,5 kV/mm 3 kV/mm
3
Force [N]
2,5 2 1,5 1 0,5 0 0
10
20
30
40
Piston displacement [mm]
Fig. 3. (a) Large haptic actuator and (b) results of force measurements
According to the principle described above a large modularly designed actuator has been built and used for force measurements with a special apparatus equipped with a stepper motor and a force sensor (Fig. 3a). These measurements support the evaluation of the ER fluid behaviour under conditions close to the intended application. The actuator can be modified by changing the electrode gap distance or the electrode material. An ER fluid with polymer particles containing movable ions has been synthesized for the use in the actuators. As an example Fig. 3b shows the results of such force measurements with the large model actuator. The linear increase of force with the piston displacement corresponds to the increasing effective electrode surface. On the basis of the actuator principle described above, an array of 4 x 4 small actuator elements has been designed, manufactured and tested. The electrode pairs in each element are individually addressable. Fig. 4a shows the actuator array with a single piston. Once the pistons pressed down, they do not move back to their upper position, which is due to the passive forces of the ER fluid. However, this motion is achieved by an elastic foam component fixed at the top of the actuator array (see Fig. 4b). This means that the array also has an elastic response, but only the viscous force can be influenced by the electric field. With this actuator array, well perceivable haptic effects could be demonstrated, indicating the feasibility of the whole concept.
91
As shown in Fig. 3b, field strengths of some kV/mm are required for the stiffening of the ER fluid, from which the necessary maximum voltage of about 1.5 kV can be derived. There are no commercially available components which are small and switch such voltages with very low currents. After evaluating various switching principles, a novel technology for the high-voltage switching of ER fluid based actuators has been found [7]. This innovation concerns semiconducting elements from gallium arsenide (GaAs), whose conductivity is influenced by irradiation with light. The optoelectric effect makes it possible to supply an individual voltage to each actuator element, where the information is provided by the light intensity. For the 4 x 4 actuator array described above 16 discrete GaAs elements have been mounted on a board and irradiated by light-emitting diodes (LED). By switching time-dependent voltage patterns on the LEDs dynamic haptic effects could also be generated and perceived by the user. (a)
(b)
Fig. 4. 4 x 4 actuator array (a) with a single piston and (b) with an elastic foam component
The current work concerns the design, manufacturing and assembling of an actuator array with a larger number of elements. In the next step, an 8 x 8 array has been constructed and manufactured, consisting of 64 elements with a quadratic top surface of 2.5 mm x 2.5 mm. The larger the number of actuator elements, the more relevant become manufacturing methods with a high degree of automation. For this purpose, well-established procedures of microtechnology can be applied, which has the further advantage that the small components in the actuator array have a higher accuracy. The design of the actuator array is modular, which means that the number of elements can easily be increased up to the final array size of 32 x 32 elements without changing the manufacturing method. The discrete GaAs elements used to switch the high voltage to the actuator electrodes are automatically sawn from a wafer, contacted and mounted on a board which is equipped with the required electric connections. The mounting of the GaAs elements on the board is performed by a robot. Instead of the LEDs which irradiate the GaAs elements, a new technique for the optical switching has been found. A beamer with a special optical system sheds light on the array of GaAs elements on the board, corresponding to the haptic image to be represented. This technique has been successfully evaluated. It is flexible and can easily be adapted to actuator arrays of different sizes by modifying the optical system of the beamer. The haptic data of the sensor part of the whole system generate the beamer image which directly distributes the light intensity over the area of the GaAs array. The final system will be based on this technological principle.
92
4 Conclusions and Outlook The feasibility of a haptic sensor-actuator system based on ultrasonic elastography and electrically induced stiffness variation of an ER fluid have been shown. This is demonstrated by the realisation of a system of reduced complexity. The system allows to detect the stiffness distribution of mechanically inhomogeneous objects and to represent the haptic information on the actuator part. The approach is a novel technological platform and offers far-reaching perspectives for the development of various applications in medical technology. Further benefit is expected in other branches such as the virtual presentation of goods in the internet (electronic commerce), entertainment industry or training with simulators including haptic functions. In the final steps the size of the actuator array will be extended up to 32 x 32 elements. This size allows to touch the actuator array with several fingertips simultaneously. The whole system has to be tested and evaluated with respect to its haptic performance and reliability. Further benefits could arise from the use of different communication methods for the data transfer from the sensor to the actuator part of the system.
Acknowledgements Financial support of this work by the German Federal Ministry for Education and Research (BMBF) is gratefully acknowledged.
References 1. Burdea, G.C.: Force and Touch Feedback for Virtual Reality, Wiley, New York, 1996 2. Pesavento, A.; Perrey, C.; Krueger, M.; Ermert, H.: A time-efficient and accurate strain estimation concept for ultrasonic elastography using iterative phase zero estimation. IEEE Trans. Ultrasonics, Ferroelectrics and Frequency Control, vol. 46 (1999), 1057-1067 3. Böse, H., Trendler, A.: Smart fluids and their properties – perspectives for new actuators; Proc. Euromat (2000) 1-6 4. Furusho, J., Sakaguchi, M.: New actuators using ER fluid and their applications to force display devices in virtual reality and medical treatments; Proc. 6th Int. Conf. on Electrorheological Fluids, Magnetorheo-logical Suspensions and Their Applications (M. Nakano and K. Koyama, Eds.), World Scientific, Singapore (1998) 755 5. Böse, H., Berkemeier, H.-J., Trendler, A.: Haptic system based on an electrorheological fluid; Actuator 2000, 7th Int. Conf. New Actuators, 563-566 6. Mavroidis, C., Bar-Cohen, Y., Bouzit, M.: Haptic interfaces using electrorheological fluids; Electroactive polymer (EAP) actuators as artificial muscles (Y. Bar-Cohen, Ed.), SPIE Press, Washington, 2001, 567-591 7. Monkman G. J., Meier A. and Egersdörfer S.: Opto-isolated high voltage control array. UK/European patent GB 0304769-3
93
