Connections Between Embodiment, Mindfulness, And Representation
We live in a constructed reality, built through experience and technology
Evolution of mind has always involved the evolution of the body; distributed mind will seek a distributed body. To assist in the embodiment of this connectivity of mind is part of the artist’s task.
Most of our fundamental concepts are organized in terms of one or more spatialization metaphors. Lakoff and Johnson, Metaphors We Live By. 1981
Affective cognition Value Psychomotor cognition Action
Cognitive structures are mental structures help organize to understand and remember Synthetic experiences extend our senses / perceptual system extend our cognitive structures
This poster offers insights on the artistic practice of creating virtual environments that are responsive to body motion in order to bring attention to the present. The act of being in a spatial environment where visual consequences reflect the body that triggered them are sought to help focus the mind in a way akin to what Varela described as “perceptually guided action.”
Representation of the self acts as a location marker for consciousness.
Virtual Spaces And Perception As Synthetic Experience The synthetic experience can be understood as the natural experience extended through technological means. These means are designed to immerse a person or people into a representation of reality, a reality of being immersed on a task, an imagined space, a visualization of state of mind, or a combination of them. To represent reality, technology is built around the human perceptual system that connects with the focus of that perceptual system’s attention towards the outside world. In handling reality, the human perceptual system also extends inwards, to patterns of reasoning where the structure of the body, that is, the perceptual system itself, models the organization of ideas and interaction (Johnson, 1987). Therefore, a synthetic experience has unique potential to allow for interaction with structures of the mind. I am not looking at the structure ‘from’ the cognitive process but from a design point of view. I envision aspects of awareness becoming more evident and transparent as the tracking of body gestures become widespread. The person immersed in a synthetic experience world is ideally in the present in the most optimal state of consciousness. I have tried to define synthetic experience as the one that is not there as brought up by the Universe but by human representation in its perceptual form. It is not an isolated mental experience. Rather, its focus is on centering the experience in the human perceptual system as it supports the cognitive process, rather than a replica of the position and dynamics of a three dimensional world ruled by gravity and other physical forces as we know them. As a potential result, I anticipate a non materialistic view of reality where time is spent in attention to experiencing for knowledge rather than possessing representations. In this potentiality, I foresee a sustainable future assisted by self reflection via mirroring of the environment.
Virtual Spaces as Mindfulness Practice
VIE W There can only be a difference in degree, and not a difference in kind, between matter and perception. We have seen that the degree of consciousness varies in accordance with the living being’s freedom of movement. Bergson-Deleuze Encounters: Transcendental Experience and the Thought of the Virtual, Valentine Moulard-leonard. 2009
Tracking devices help join the body to the structure and feedback mechanisms of the experience.
The visual field is covered as much as possible to create the illusion that one is in a virtual place.
Stereoscopy, parallax and visual cues are utilized to make objects present relative to the body of the person. They are used to calculate distance to the body.
Mental cognition Comprehension
C EN ES
H T P E D
Roy Ascott, 2000
Cultural heritage Affective states Memories Imagination Asynchronous communication Telepresence Many kinds of mediated content
Standard cognitive domains
Synthetic experiences --> Remapped perception --> Embodied cognition
The act of questioning is accompanied by a strange mental tension unknown to animals. This active hollow, this seminal void is the very escense of the virtual.
Remapping to the human perceptual system Sense of passage of time Sense of scale Perceivable light Sound, touch, scent Proprioceptive sense and physics
Process where one is able to monitor one’s own process of understanding.
Pierre Lévy, Becoming Virtual. 1998 Reflective consciousness through awareness of the body
Synthetic experiences and the body Embodiment techniques
Embodiment and metacognition
Cognitive maps Active mind Active body Some approaches Situated cognition Embodied cognition -Lakoff, Johnson, Turner Enactivism -Maturana, Varela, Bateson Means to connect synthetic and real Virtual Reality (VR): real in the synthetic Augmented Reality (AR): synthetic in the real
My research pays attention to theories of the self (William James, Masumi, as a self referential model), philosophy of experience (Bergson) and neurobiology findings (Damasio) in the design of synthetic environments, which encompass the body (Lakoff and Johnson) and the thought process (Varela, Damasio) on a state of mindfulness, parsing the present as emergent self (Varela). I reflect on the potential of immersive and interactive synthetic experiences to make us better at dealing with external reality through embodiment techniques. How does VR relate to this “real virtuality”? All kinds of representation ultimately lead to looking at reality, or real virtuality, to observe it and reflect upon it. Besides a simple replication of theater, VR can be utilized to immerse ourselves on visualizations, that is, realities beyond our experiential range such as micro or macro environments or abstract situations, situations that can be mapped to our body and enter our consciousness through the perceptual system in the familiar way that reality does.
Embodiment and constructivism Theory of knowledge that argues that humans generate knowledge and meaning from the interaction between their experiences and their ideas.
Reflective Consciousness Through Body Actions The importance of embodiment may arise not just from the connection between an experience and the thought process but as a model of interaction which connects conscious organisms that can handle things, navigate space, and make models in the mind, and that in turn, reasons based on such experiences. This is relevant to the creation of synthetic experiences which immerse or co-immerse the person or people in that the experience can be understood as a dynamic reflection, or an exoskeleton of the mind.
References Antle, A. Lifelong Interactions. Embodied child computer interaction: why embodiment matters, interactions, v.16 n.2, March + April 2009
Interactive and immersive synthetic experiences design or utilize the sense of embodiment of the person experiencing it. At the same time, one could say that the ideal state of the person is that of mindful attention. How are mindfulness and embodiment part of an experience? Varela, Thompson and Rosh state that mindfulness is fundamental as a method where the mind is present in it. By embodiment, they mean “reflection in which body and mind have been brought together.” The opposite of embodiment, as philosopher Thomas Nagel puts it, is a “view from nowhere.” Regarding mindfulness, in his paper “Virtual Reality for Animals” John Waterworth points out we are seldom in the present: animals are a better choice for experiencing these experiences. On the other hand, the practice of mindfulness in Buddhist traditions seeks to be in the present integrating more sophisticated “embodied mind” mental processes. Synthetic environments ideally presuppose you are parsing your thoughts to the experience as it happens. From a biological point of view, brains “operate on the basis of massive interconnections in a distributed form, so that the actual connections among ensembles of neurons change as a result of experience.” Coincidentally, for Buddhist traditions, consciousness is about the parsing of the experience into those moments of emergence. (Varela,1991) Varela et al. argue that consciousness is always changing, and perhaps the self is but a series of aggregates of emergent states. “Perception and action, sensorium and motorium, are linked together as successively emergent and mutually selecting patterns.” Mindfulness has been studied in the context of therapy, gaming, interaction design and semiotics. It relates to psychological well being as an attribute of consciousness where consciousness acts in the present reality in order to deal with it effectively regardless of other mental processes (Brown, 2003).
Bachelard, G., [The Poetics of Space]. Beacon Press, Boston (1958) Baer RA, Smith GT, Lykins E, Button D, Krietemeyer J, Sauer S, Walsh E, Duggan D, Williams JM. Construct validity of the five facet mindfulness questionnaire in meditating and nonmeditating samples. Assessment. 2008;15:329–342. Bransford, J. (2000). How people learn: Brain, mind, experience, and school (Expanded Edition). Washington, DC: National Academy Press. Brown, K. W., & Ryan, R. M. (2003). The benefits of being present: Mindfulness and its role in psychological well-being. Journal of Personality and Social Psychology, 84, 822-848. Cruz-Neira, C., Sandin, D. J., DeFanti, T. A., Surround-screen projection-based virtual reality: the design and implementation of the CAVE, ACM Press, New York, 1993. Damasio, A. Self comes to mind: Constructing the Conscious Brain. Pantheon, 2010. Deleuze, G., Guattari, F. A thousand plateaus: Capitalism and schizophrenia, Trans. B. Masumi, University of Minnesota Press, 1987 Fox, J.A., & Bailenson, J.N. (2009). Virtual self-modeling: The effects of vicarious reinforcement and identification on exercise behaviors. Media Psychology, 12: 1-25. Freedberg, D., and Gallese, V., “Motion, emotion and empathy in esthetic experience.” Trends in Cognitive Sciences, 11, 197–203 (2007).
High on his tower, as far as possible from the earth, sits a human being. He has so transformed his eyes with the aid of gigantic optical instruments, that they have become fit to penetrate the universe up to its most distant stars. Jakob Von Uexküll
For example: MBSR (Mindfulness Based Stress Reduction) is a “a structured group program that employs mindfulness meditation to alleviate suffering associated with physical, psychosomatic and psychiatric disorders.” (Grossman, 2004).
Grossman, P., Niemann, L., Schmidt, S., and Walach, H. (2004). Mindfulness-based stress reduction and health benefits: A meta-analysis. Journal of Psychosomatic Research, 57(1): 35-43. Johnson, M. (1987). The Body in the Mind. Chicago: University of Chicago Press. Lakoff, G. and Johnson, M. (1980) Metaphors We Live By, Chicago, IL: Chicago Press. Lakoff, G., Nuñez, R. Where Mathematics Comes From: How the Embodied Mind Brings Mathematics into Being. 2001. Lau, M. A., Bishop, S. R., Segal, Z. V., Buis, T., Anderson, N. D., Carlson, L., et al. (2006). The Toronto Mindfulness Scale: Development and validation. Journal of Clinical Psychology, 62, 1445–1467. Levin, S., 1992. The Problem of Pattern and Scale in Ecology: The Robert H. MacArthur Award Lecture. Ecology, Vol. 73, No. 6. Ecological Society of America, USA. pp.1944-1947-1957-1960. Lévy, P., Becoming virtual: Reality in the digital age, Plenum Press, New York, (1998). Leymarie, F., 2001. Art and Visual Perception by Rudolph Arnheim. Notes by Frederic F. Leymarie. Accessed July 2010. pp.6-7 Mason, O., & Hargreaves, I. (2001). A qualitative study of mindfulness-based cognitive therapy for depression. British Journal of Medical Psychology, 74, 197–212. Masumi, Brian. (2000) The Parable of the Cave. Moulard-Leonard, V. Bergson-Deleuze encounters : transcendental experience and the thought of the virtual. Albany, NY : State University of New York Press, 2008. Murray, J., [Hamlet on the Holodeck], The MIT Press, Massachusetts, (2001).
Some evaluation models of mindfulness measurement emphasize the acceptance of the present in a kind of de-centered intentional, reflective introspection and self observation (Lau et al, 2006). Such de-centering leads to a shift from other mental states to curiosity. (Lau et al, 2006) Mason describes this de-centering as a useful action to distance oneself from problems such as depression in times of “mind overload” (Mason, 2001). Cultivating mindfulness allows practitioners ‘to interrupt automatic patterns of conditioned behavior’ (Varela). Mindfulness is described as a “state of consciousness”, a scaffolding to develop a skill (Kabat-Zinn, 2005). The target is not the content of the thoughts per se, but the relationship of the individual to the process of thinking.” (Shapiro, 2006) Structures of intentionality involve imaginative projections. Johnson sees these recurrent structures as image schemas (1987) of gestalt where ‘we develop patterns for interacting forcefully with our environment’ handling things, navigating space and extending force (1987). Both Johnson and Varela work on the basis that patterns and schemas are dynamic and very close if not in the present. They focus on the actualization of the present for the understanding of structures of the mind, Varela through mindfulness and Johnson through pre-conceptual dimensions of experience (Johnson, 1987, Varela, 1991) For Johnson, gestalt forces learned from experiencing space physics work as constraint of coherence (1987). It is through a process of abstraction which originates in the way we inhabit the world that functional representation has evolved into more abstract structures (Zlatev, 2009). These structures respond to the need for efficient patterns to resolve the dynamic existence of human consciousness. In terms of the meaning that flows through the experience, Zlatev points out that there are multiple levels to the organization of meaning that involve enaction as embodiment. In fact, he suggests there is some degree of embodiment to all of them and therefore a continuum between perception and symbols. It details that life presupposes consciousness, which in turn presupposes sign function, which then presupposes language. The same is not true the other way around. He cites: “Representation is a flexible pattern of organism-environment interactions” (cited in Zlatev, 2009; Johnson and Lakoff 2002). Based on Maturana and Damasio’s observations, he proposes that “consciousness emerged as a biological adaptation in creatures in need of a “common currency” for multimodal perception, action and evaluation, so that attentional resources can be allocated flexibly, and evaluation can be performed efficiently via feelings, e.g. for the purpose of anticipating the results of actions.” Based on evidence that differentiate imitative capacity and intersubjectivity, Zlatev adds that consciousness “co-evolved with the capacity for bodily mimesis, the use of the body as a representational device, and that furthermore the latter gave rise to the sign function in evolution.” He concludes that through all these levels of embodied meaning, we live in “subjectively coloured but intersubjectively shared Lifeworlds”.
Mori, T., Endou, Y. and, Nakayama, A. 1996 .Fractal Analysis and Aesthetic Evaluation of Geometrically Overlapping Patterns. Textile Research Journal September vol. 66 no. 9. USA. pp. 581-586
Virtual Space Beyond The Senses: Visualization Perception of space is unique in the field of astronomy. Cognition through extended senses entail the Universe we can reach with man made devices. It has been noted that human brains recognize action in others through a mechanism called mirror neurons. How does this understanding translate to universe-size scale patterns? What kinds of schemas would be useful for embodying cosmic scale visualizations? Schemas that link the senses to each other at human scale. Yet there are events that are connected to the present in trans-scalar ways. Today our senses are extended to where our sensors are, wired to the environment literally and metaphorically. Note the landing of the Curiosity Rover on Mars which extended the eyes of humanity further over the Martian surface. Back on Earth, in the context of natural experience, concepts are used to convey meaning in visualizations. In natural experience, semantic memory is “explained in terms of embodied memory patterns”. This working memory, when used “off-line” appears to “be an example of a kind of symbolic off-loading” where “new skills become automatized, reducing cognitive load and circumventing the representational bottleneck.” (Wilson 2002) Spatial perception in large scale visualizations articulate the experience of space as a visual, proprioceptive and physical experience that is the basis for synthetic representations created to convey astronomically-sized dynamic structures that are outside of the human perceptual range. Astronomy visualizations based on real data as well as simulations are tailored to the space and time that lie within the range of human scale, in order to show relationships that are important for the understanding of large phenomena. A fluid perceptual state of mindfulness is important to notice those relationships, and tying them through embodied interaction helps to integrate complex relations in a direct manner. Perceptual cues that exploit the experiential connections of the senses to reality are used to reinforce the vivid richness of the visualizations. Vision, sound and tracked movement work together to build a full sensory experience by reconstructing the missing input such as touch and smell through the senses afforded by the media used, as well as the person experiencing it.
Morie, J. F., Meaning and Emplacement in Expressive Immersive Virtual Environments. University of East London, 2007. Orbán, G., Fiser, J., Aslin, R., and Lengyel, M. 2008 Bayesian learning of visual chunks by human observers. Proceedings of the National Academy of Sciences, USA. February 19, 2008 vol. 105 no. 7 2745-2750. The National Academy of Sciences, Stanford, USA. Panofsky, E., Perspective as Symbolic Form, translated by Christopher Wood (New York: Zone Books, 1997). Originally published in 1927. Shapiro, S. L., Carlson L. E., & Astin J. A. (2006). Mechanisms of mindfulness. Journal of Clinical Psychology, 62, 373– 386. Schulze, J., Forsberg, A., Kleppe, R. Z., and Laidlaw, D. H. 2005. Characterizing the effect of level of immersion on a 3D marking task. In Proceedings of HCI International, HCI International, 447–452. Suntinger, M., Obweger, H., Schiefer, J., 2007. The Event Tunnel: Interactive Visualization of Complex Event Streams for Business Process Pattern Analysis. 2008 IEEE Pacific Visualization Symposium. Kyoto, Japan, March 5-7. pp. 2-4-8-9 Varela, F. J., Thompson, E., & Rosch, E. (1991). The embodied mind: Cognitive science and human experience. Cambridge, MA: MIT Press, Kindle Edition. Varela, F.J., Shear, J. The view from within: First-person approaches to the study of consciousness. 1999. Ware, C. 2008. Toward a Perceptual Theory of Flow Visualization. IEEE Computer Society Press Los Alamitos, CA, USA. Waterworth, J. (2003). ‘Technology in Support of Returning: From Conscious Doing to Consciously Being’, in K. Höök, D, Benyon, A. Munro (eds.), Designing Information Spaces: The Social Navigation Approach. London: Springer London. Waterworth, J., Virtual Reality for Animals: linking concrete and abstract reasoning through action in virtual space. Proceedings CIBER@RT’96 - International Conference on VR - Valencia, Spain, 1996. Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin and Review, 9, 625-636. Yee, N. and Bailenson, J. (2007). The Proteus Effect: The Effect of Transformed Self- Representation on Behavior. Human Communication Research 33(3): 271—29. Zheng, N., and Xue, J. 2009. Unsupervised Learning for Visual Pattern Analysis. Statistical Learning and Pattern Analysis for Image and Video Processing. Springer London, London, UK. pp.16-17-33 Zlatev, J. (2009). The semiotic hierarchy: Life, consciousness, signs and language. Cognitive Semiotics, 2009(4):169–200.
Julieta Aguilera Ph.D. Candidate, Planetary Collegium University of Plymouth, UK [email protected]
Associate Director, Space Visualization Laboratory Adler Planetarium, Chicago, US [email protected]