ATR artificial brain project: 2004 progress report

Artif Life Robotics (2005) 9:197–201 DOI 10.1007/s10015-005-0345-9

© ISAROB 2005

ORIGINAL ARTICLE Andrzej Buller · Michal Joachimczak · Juan Liu Katsunori Shimohara

ATR artificial brain project: 2004 progress report

Received and accepted: February 28, 2005

Abstract This article presents the key assumptions and current status of the ATR Artificial Brain Project being undertaken to create Volitron, a device equipped with circuitry that enables the emergence of thought. Such thought would be recognized from Volitron’s specific communication behaviors. The project consists of three complementary themes: psychodynamic architecture, brain-specific evolvable hardware, and the management of brain-building. The psychodynamic architecture is designed to develop automatically, driven by “pleasure” coming from discharges of tension gathered in special tension-accumulating devices. Tension-discharging patterns come first of all from the robot’s interactions with its care giver/provider. For the dedicated hardware, we developed qcellular-automata (qCA), in which groups of uniform logic primitives (q-cells) serve as spike-train-processing units, as well as pulsed paraneural networks (PPNN) that can be evolved, using fuzzified signals and a genetic algorithm combined with hill climbing, and converted into qCA. The psychodynamic ideas were tested using three robots: Neko, equipped with a pleasuredriven associator, Miao, equipped with MemeStorms (a special working memory in which conflicting ideas fight for access to the long-term memory and actuators), and Miao+, whose brain is equipped with a growing neural network. Key words Artificial brain · Autonomous robots · Evolvable hardware

A. Buller (*) · M. Joachimczak · J. Liu · K. Shimohara ATR International, Human Information Science Laboratories, 2-2-2 Hikaridai, Keihanna Science City, Kyoto 619-0288, Japan Tel. +81-774-95-1009; Fax +81-774-95-2653 e-mail: {buller; mjoach; juanliu; katsu}@atr.jp This work was presented in part at the 9th International Symposium on Artificial Life and Robotics, Oita, Japan, January 28–30, 2004

1 Introduction The ATR Artificial Brain Project is being conducted in the framework of the Emergent Communications Mechanisms Project, which aims to create novel technologies to reach the frontiers of new possibilities in human–machine communication.1 The target product is to be Volitron – a device equipped with circuitry that enables the emergence of thought. Such thought would be recognized from Volitron’s specific communication behaviors.2 The project consists of three complementary themes: psychodynamic architecture (PDA), brain-specific evolvable hardware (BEH), and management of brain-building (MBB). The purpose of PDA (Sect. 2) is to develop automatically all functional agencies and communication channels necessary to host emergent thought. It is also used to facilitate the work of researchers by guiding them, according to the MBB rules, to a synthesis of new agencies. Famous experiments have been conducted with simulated worlds populated by primitive evolving creatures, and their results gave a limited hope that a thinking machine could be obtained through simulated evolution.3 On the other hand, genetic programming has been successfully employed to solve defined modest-sized problems.4 Consequently, we have developed the evolutionary short-cut doctrine, according to which isolated functions are evolved separately with a fitness function with reference to the behavior of a Volitron-driven agent, using, if necessary, a simplified simulation model of the agent and its world.5 Currently, the Volitron is being developed by using a prototype of the BrainCAD, i.e., a dedicated client-server system using several workstations. The clients can be implemented as C+ + programs, as well as circuits synthesized using the NeuroMaze 3.0 Pro.6 The psychodynamic ideas were tested experimentally (Sect. 3) using simulated robots: Neko, equipped with a pleasure-driven associator, Miao, equipped with MemeStorms (a special working memory in which conflicting ideas fight for access to the long-term memory and

198

actuators), and Miao+, whose brain is equipped with a growing neural network. One day we will want to install Volitrons in the heads of life-size robots. In this case, every cubic millimeter in a robot’s head is worth its space in gold, so the use of generalpurpose microprocessors with gigabytes of memory occupied by unwanted library functions does not seem to be an efficient solution.5 BEH (Sect. 4) is intended to use only as many logic primitives as necessary and, which is more important, be evolvable.

2 Psychodynamic architecture Since empirical evidence supporting any theory of mind is limited, the ATR Artificial Brain Group had to choose the most suitable ideas and verify them through behaviors demonstrated by the developed prototypes. Our choices were primarily based on a selection of ideas proposed by Sigmund Freud, Jean Piaget, and Marvin Minsky. Based on this selection, a psychodynamic architecture (PDA) is being developed. It is assumed that the only mission of a PDA-based agent is to seek its own pleasure. Such pleasure, as Freud proposed, is a quick discharge of a psychic tension.7 Tensions are represented in PDA as states of dedicated tensionaccumulating devices. This architecture is being considered in terms of agencies constituting a certain instance of the Society of Mind.8 PDA, in the most general sense, consists of two coupled agencies: a joint tension accumulator and a joint tension discharger. The joint tension accumulator includes a sensory system, a bundle of input channels, and a set of tension-accumulating devices contributing substantially to the pleasure record. The joint tension discharger includes a set of actuators, a bundle of output channels, and a set of blocks responsible for tension-discharging behaviors. The blocks also consist of tension-accumulating devices, but they do not contribute substantially to the pleasure record. The challenging issue is to find such rules of growth for the architecture, as well as methods of training the robot where groups of tension-accumulating devices form subagencies such as working memory, long-term memories, an action planning module, and defense mechanisms regulating psychic balance. Although the ability to imitate another agent’s behaviors is currently one of the most essential factors qualifying a machine as an intelligent entity, the goal is to build imitating systems without modules dedicated to imitating defined objects and their behaviors.9 Therefore, we develop a tension-based imitation system. Volitron will have an agency that, based on a model of perceived reality, creates a model of desired reality (e.g., an image of the agent behaving in the same way as another, already perceived, agent), then generates a tension according to the difference between the created and perceived models, and rewards or punishes the agency responsible for the discharge of this tension.10 Freudian psychology does not provide any clues to the nature of psychic forces or the mechanisms of their interac-

tions. We propose representing these forces as populations of identical pieces of information that in turn represent concepts of interest. We call the pieces memes.11,12 A working memory, called MemeStorms, is a theater in which contradictory ideas represented by populations of memes fight for access to the long-term memory and the actuators. We have performed a number of simulation experiments with this kind of memory.12,13 The results were comparable with the psychological evidence on the intrinsic dynamics of judgment.14 The question that is currently most divisive in the AI community is whether intelligent minds use a world model. While one camp recognizes a world model as an indispensable part of intelligent systems,15 the idea of intelligence without representation is being promoted by the opposite camp.16 Rodney Brooks et al.17 collected psychological evidence supporting the position that humans do not build an internal model of the entire visible scene, and that there are multiple internal representations that are not mutually consistent. PDA is intended to offer a compromise solution between these two viewpoints. Volitron does not and cannot build an internal model of the entire scene because the above-mentioned MemeStorms system employs volatile swarms of memes representing only selected parts of the scene and those obtained on request from the long-term memory representing only the most necessary elements of perceived reality.10,13 The model of perceived reality is updated throughout the agent’s entire life and never becomes perfect. It is also never accessible in its entirety because of the limited capacity of the working memory. A Piagetian way of mind development18 is being pursued toward the ability of self-improvement for machine long-term memory. Another argument for representations and world models is the possibility to improve pattern recognition via a mechanism proposed by Jerome Brunner,19 i.e., comparing a new image with previous recognitions.

3 Experimental results For testing basic psychodynamic solutions one does not need to employ a walking humanoid robot. A simple twomotor mobile vehicle with a camera and speaker can demonstrate how tensions and pleasures work. Robots of such a kind, Neko, Miao, and Miao+, were used for the first experiments with tension-driven behaviors. Neko was physically constructed. It had two wheels propelled by two dedicated motors, a speaker, a color camera, and two touch sensors. The robot’s brain consisted of functional modules simulated on a cluster of three PCs and an on-board module for collision avoidance connected to the touch sensors. The tensions implemented represented boredom, excitation, fear, and a three-dimensional anxiety. Boredom increased when Neko did not perceive any object of interest, and discharged when it saw one. When the camera detected a green object, the level of excitation increased, and remained high as long as the object remained in the visual field. The level of fear became high immedi-

199

ately as a reaction to the appearance of a red object, remained high as long as the object remained in the visual field, and dropped down after the object’s disappearance. Anxiety was represented by the states of three tensionaccumulating devices: AnxietyL, AnxietyR, and AnxietyB. Each of the three parts of anxiety increased spontaneously and independently from the other. A discharge of AnxietyL, AnxietyR, or AnxietyB took place any time Neko turned left, right, or back, respectively. An arbitrary hard-wiring determined the hierarchy of Neko’s tensions. The priority list was: fear, excitation, anxiety, and boredom. The tension that achieved maximum volume suppressed the outputs of all tension-accumulating devices related to tensions located at lower positions at the list. As for the elements of the vector of anxiety, the one that first achieved maximum volume suppressed the outputs of the tension-accumulating devices related to the other two. An unsuppressed signal from a tension-accumulating device related to a tension at its maximum volume activated a dedicated functional module. The module activated by fear caused the robot to turn back and escape until the fear fell below a defined level. The module activated by excitation forced the robot to chase the exciting object. In the case of activation of the module connected to AnxietyL, AnxietyR, or AnxietyB, the robot looked left, right, or back, respectively. When the signal produced by the tensionaccumulating device representing boredom was high and unsuppressed, it went through a controlled associator connected to three functional modules: a scream generator, a driver for looking around, a driver for going forward. The associator initially activated the scream generator connected to the speaker. When for a defined time the associator did not receive a tension discharge signal, it activated the driver to look around. If a tension discharge signal still did not come, the associator activated the driver to go forward. If even then a tension discharge signal did not come, it again activated the scream generator, and so on. Signals produced by the modules could be suppressed by the signal from the on-board module for wall avoidance. Equipped as described above, Neko learned by itself how to cope with the boredom that grew when no object of interest was perceived. It could choose between producing a sound, looking around, and going forward. Indeed, going forward increased the chance of seeing an object of interest. The learning was reinforced in a psychodynamic way, i.e., by a tension discharge signal. Since tensions representing “irrational” anxiety were also accumulated, in the event of the lack of an object of interest Neko behaved as an animal in a cage, i.e., it wandered back and forth, and “nervously” looked around.20 Miao is a simulated creature living in a simulated world. Its tension-accumulation devices and functional modules cover fear-, excitation-, anxiety-, and boredom-related behaviors similar to those demonstrated by Neko, as well as hunger-related behavior and a fight between hunger and excitation. The hunger volume is a function of the state of the simulated battery. If tensions representing fear and excitation are negligible, hungry Miao activates a module that

drives it toward the battery charger. However, the activation signal must pass through a device called MemeStorm (a simplified version of MemeStorms). The signal representing excitation must also pass through MemeStorm, and only then can it activate the module that makes it chase the object of excitation. Inside MemeStorm, hunger and excitation try to suppress each other. The device has such an intrinsic dynamic that in the case of a substantial difference between competing tensions, the stronger one quickly wins, i.e., suppresses its rival and becomes high. However, when the competing tensions are close to a balance, one of the tensions wins, but after a couple of seconds loses to the other tension, with a possibility of winning again in a short time.21 A fragment of a related report states: “Miao punches the ball / it stopped punching and looks toward the battery charger / Miao turns back to the ball and punches it (though not too vigorously) / suddenly it resigns, turns, and slowly approaches the charger / It gets very close to the charger / Miao sadly looks back at the ball, then turns and starts recharging . . .”22 The word “sadly,” if treated literally, would be by all means farfetched, but what the experiment intended to show was not a “true” sadness. The point is that the robot’s gaze looked sad, and looked so not because somebody intentionally programmed a masquerade sadness, but because the robot’s brain circuitry allowed for a psychodynamic process resulting in such an expression. Miao+, like Miao, is a simulated creature living in a simulated world, with the same set of sensors, actuators, and tension-accumulating devices. However, unlike its predecessor, Miao+ has a brain that develops in a literal way. Each new sensorimotor experience adds new cells and connections to related neural network. The development is modulated by pleasure signals. The network provides control signals to the speaker and to each of the two motors. What is essential is that there is no ready-made circuitry for approaching an object of interest. A “newborn” Miao+, like a newborn human baby, has no idea of how to purposefully use its actuators, so in the face of increasing tensions it can only produce random sounds and moves. In the development of the brain of Miao+ the role of its caregiver/provider is fundamental. Hearing sequences of sounds produced by the robot, the caregiver/provider gives items she supposes it wants to have at the moment. If by accident the given item causes a discharge of dominating tension, the subsequently generated pleasure signal reinforces changes in the neural network increasing the strength of association between the tension and the recent vocal expression. This way, gradually, Miao+ learns to differentiate vocal expressions toward distinguishable sequences of sounds, each dedicated to a different object of desire. At the same time the caregiver/provider gradually learns to guess the robot’s needs properly based on the heard sequences of sounds. In other words, within the pair – Miao+ and its caregiver/provider – a common, mutually understandable language emerges. When, in this stage of the robot’s brain development, the caregiver/provider fails to give it a desired item, Miao+ has no choice but to try and get the item itself. The growing

200

Fig. 1. PPNN (pulsed para-neural networks)-based timer. For x = 1 0 0 0 . . . and D0, D1 and D2 equal to 1, 40, and 7, respectively, u = 0 0 0 1 0 000000100000001000000010000000100000000 0 0 0 0 0 0 0 0 0 0 . . . In the PPNN schemes, every nonlabelled arrow indicates a connection of zero delay

network provides the motors with senseless signals, but when, by accident (or owing to a discrete caregiver/ provider’s help), the item is touched, the related pleasure signal reinforces the changes in the network that caused the recent sequence of motor-driving signals and the locational changes of the image of the item in the robot’s visual field. This way Miao+ learns how to use its motors and camera to approach objects of desire with increasing efficiency. Having learned to approach immobile items, the robot then learns to chase mobile objects. When, having learned the art of approaching and chasing, Miao+ still faces difficulties in catching the object of desire, it “recalls” the possibility of asking its caregiver/provider to bring the object to it, so it again produces the appropriate sound sequences.22 Although the brains of Neko and Miao can be expanded to cover more and more psychodynamic functionalities, because of the necessity of manually attaching the new tension-accumulating devices and behavior-generating blocks, the constructions cannot be called psychodynamic agents. The above conclusion does not apply to Miao+, whose brain develops driven by pleasures.

4 Brain-specific hardware The psychodynamic functions can be coded in several ways. We developed pulsed para-neural networks (PPNN) as graphs consisting of processing nodes and directed edges, called axons.23 The nodes represent delayed functions operating on spike-trains, where all spikes have the same amplitude. Axons represent pure delays. Although several node functions have been investigated, this paper discusses PPNNs in which both nodes and axons receive/return spikes at discrete moments of time called clocks, such that each node returns a pulse at clock t if it received one and only one pulse at clock t - 1. PPNN graphical notation uses squares representing nodes, thin arrows representing delays equal to 0, and thick labeled arrows representing nonzero delays, where a label shows a particular number of clocks or a variable’s symbol (Fig. 1). Since a node with a constant input equal to 1 can serve as a delayed logical NOT function, while three appropriately connected nodes may constitute a delayed logical AND function (Fig. 2), PPNN appears to be universally complete, which means that for any desired manipulation on spiketrains there is an appropriate

Fig. 2. PPNN serving as a delayed AND. ut+2 = xt yt (Solution suggested by H. Eeckhaut and J. Van Campenhout26)

Fig. 3. Evolving PPNN. This example involves an evolutionary search for two unknown delays in a scheme built on two parallel timers. The evolution is accelerated by fuzzifying the output signals and combining a genetic algorithm with hill climbing, which is possible owing to the fuzzification. Evolved PPNN can easily be converted into equivalent qcellular automata

PPNN. We have found an efficient method of evolving PPNNs24 (Fig. 3). The concept of BEH investigated here is considered as a battery of 2D qcellular automatic (qCA) panels, where states of uniform q-cells represent both a given PPNN and a layout of pulses (Fig. 4). Any PPNN can easily be converted into an equivalent qCA. Ultimately, every BEH cell is to be a distinct physical entity, but we have developed two interim solutions: a universal spiketrain processor (USP) and a qCA Machine (qCAM). The USP is a dedicated digital device that directly emulates a given PPNN by using a pointer that moves along a FIFO buffer representing each delaying edge.25 The heart of the qCA Machine is an FPGA circuit that emulates a 2-dimensional array of q-cells and multiplexes the input/output signals.

5 Concluding remarks The theory underlying psychodynamic architecture proposes that if a machine were equipped with a set of devices accumulating certain “psychic tensions” and a control system that forces the machine to search for a way to discharge the tensions, then the machine might autonomously de-

201

Fig. 4. Example of a qcellular automaton (qCA). Every q-cell has its location i,j and a state defined by an integer defined by di,j,4di,j,3di,j,2di,j,1di,j,0 (a series of binary values) where di,j,0 serves as a pulse to be propagated. The group A, B, . . . , E of q-cells serves as a “flat overpass” (worked out based on [20])

velop its intelligence to eventually reach the level of “strong AI.” We performed several experiments with mobile robots equipped with controllers operating according to the various selected psychodynamic principles. In the case of the Miao+ robot, the emergence of communication and motor behaviors was observed. The universal spiketrain processor (USP) and qCA machine are complementary tools for BEH-oriented research, and they will eventually replace the obsolete CAM-brain machine (CBM).23 The PC-stations constituting our research platform will gradually be replaced with clusters of USP or FPGA-based qCAM units, and these will be replaced one day with a synthetic neural tissue based on nano-scale q-cells. Acknowledgment This research was supported by the National Institute of Information and Communications Technology of Japan (NICT), Grant 13-05.

References 1. Shimohara K (2002) Human communication project: towards full realization of the richness of natural human communication. ATR UptoDate 3:9–11

2. Buller A (2002) Volitron: on a psychodynamic robot and its four realities. In: Prince CG, Demiris Y, Maron Y, Kozima H, Balkenius C (eds) Proceedings of the 2nd International Workshop on Epigenetic Robotics, August 10–11, Edinburgh, Lund University Cognitive Studies 94, pp 17–20 3. Brooks R (2002) Flesh and machine. Pantheon, New York 4. Nolfi S, Floreano D (2000) Evolutionary robotics: the biology, intelligence, and technology of self-organizing machines. Bradford/ MIT Press, Cambridge 5. Buller A (2002) In quest of an artificial brain. Proceedings of the 5th International Conference on Human and Computer (HC2002), September 11–14, Aizu-Wakamatsu, Japan, University of Aizu, pp 195–201 6. Buller A, Joachimczak M, Liu J, et al. (2003) Neuromazes: 3dimensional Spike-train processors. 2nd WSEAS Transactions on Computers, 3:157–161 7. Freud S (1920/1990) Beyond the pleasure principle. Norton, New York 8. Minsky M (1985) The society of mind. Simon & Schuster, New York 9. Dautenhahn K, Nehaniv CL (2002) The agent-based perspective on imitation. In: Dautenhahn K, Nehaniv CL (eds) Imitation in animals and artifacts. Bradford Books/MIT Press, Cambridge, pp 1–40, Cambridge 10. Buller A, Shimohara K (2003) Artificial mind: theoretical background and research directions. 8th International Symposium on Artificial Life and Robotics (AROB 8th ’03), January 24–26, Beppu, Japan, Oita University, pp 506–509 11. Buller A, Shimohara K (1999) Decision making as a debate in the society of memes in a neural working memory. J 3D Images 13(3):77–82 12. Buller A (2002) Dynamic fuzziness. Proceedings, 7th Pacific Rim Conference on Artificial Intelligence, Tokyo, August 18–22, Springer (LNAI2417) pp 90–96, Berlin 13. Buller A, Shimohara K (2001) On the dynamics of judgment: does the butterfly effect take place in human working memory? Artif Life Robotics 5(2):88–92 14. Nowak A, Vallacher RA (1998) Dynamical social psychology. Guilford Press, New York 15. Albus JS, Meystel AM (2001) Engineering of mind: an introduction to the science of intelligent systems. Wiley, New York 16. Brooks RA (1991) Intelligence without representation. Artif Intell J 47:139–160 17. Brooks R, Breazeal (Ferrel) C, Irie R, et al. (1998) Alternative essences of intelligence. Proceedings, AAAI-98, July 26–30, AAAI Press Madison, Menlo Park, pp 961–968 18. Wadsworth BJ (1996) Piaget’s theory of cognitive and affective development: foundations of constructivism. 5th edn, AddisonWesley, Reading 19. Bruner JS, Anglin JM (1973) Beyond the information given: studies in the psychology of knowing, Norton, New York 20. Buller A (2004) From q-cell to artificial brain. Artif Life Robotics 8:89–94 21. Liu J, Buller (2004) Tensions and conflicts: toward a pleasureseeking artifact. 5th IFAC Symposium on Intelligent Autonomous Vehicles (IAV’2004), July 5–7, Instituto Superior Tecnico, Lisbon 22. Liu J, Buller A (2004) Emergent communication and motor behavior of a simulated robot. Unpublished research note, Kyoto 23. Buller A (2002) CAM-brain machines and pulsed para-neural networks: toward a hardware for future robotic on-board brains. 8th International Symposium on Artificial Life and Robotics (AROB 8th ’03), January 24–26, Beppu, Japan, Oita University, pp 490– 493 24. Liu J, Buller A (2004) Evolving spike-train processors genetic and evolutionary computation. GECCO 2004, June 26–30, Seattle, Proceedings, part II, Springer, pp 408–409, Berlin 25. Joachimczak M, Grzyb B, Jelinski D (2004) Universal spike-train processor for a high-speed simulation of pulsed para-neural networks. Neural information processing. Proceedings of the 11th International Conference, November 22–25, Calcutta, Springer, Berlin, pp 416–421 26. Eeckhaut H, Van Campenhout J (2003) Handcrafting pulsed neural networks for the CAM-brain machine. 8th International Symposium on Artificial Life and Robotics (AROB 8th ’03), January 24–26, Beppu, Japan, Oita University, pp 494–498