The Kiki Bouba Effect – A deeper explanation

…. Deep inside your brain, cross-modal correspondences make you human. This finding explains so much about civilization that it is mindblowing.

It’s the strongest evidence we have that proves that people did not randomly make sounds by pointing at objects to make language. Somehow the form of that object influenced the sound that is appropriate for it. So, language is not an arbitrary mapping of sounds, symbols, and objects.

It’s also the reason why some dogs are floofs and others are woofers. I bet you can imagine the shape of both dogs.

The Kiki-Bouba effect

Which shape do you think is ‘KIKI’ and which one do you think is ‘BOUBA’?

…
…..
……
…….

What is your answer?
….
…..
……
…….

Do you think the left one is KIKI and the right one is Bouba?
..
…
…

Yeah? Or do you think it was the other way around?

No problem if you thought it was the other way around. But it is just striking that….

95% of the people think the left jagged shape is KIKI and the right bulbous shape is BOUBA!!

You just demonstrated cross-modal correspondence. Something in your brain thought the sound of Kiki matched the left shape and something in Bouba matched the other right shape.

Today, I am going to dive deep into my favorite topic in psychology and neuroscience – The Kiki-Bouba effect. It is something that may always separate us from the robots. Perhaps it holds the key to making robots more human. We don’t know to what extent this may influence AI. But for now, let us focus on how this effect manifests in us humans.

Because this article is related to my research interest, it is going to be of an academic nature. You’ll keep seeing references to this effect in other things I write such as how the effect helps us understand subtitles and audio better when together.

We often describe objects in the environment in complex ways. These descriptions contain analogies, metaphors, emotional effects, and structural and functional details about the objects. In 1929, Wolfgang Kohler demonstrated, for the first time, an inherent tendency in people to match certain sounds to shapes. He showed that people, non-arbitrarily, matched the non-sense word KIKI to a jagged star-like shape and BOUBA to a round bulbous shape [1-2].

So somehow, the word KIKI equates to a star shape, and the word BOUBA equates to a round shape. This is a cross-modal correspondence. Cross-modal correspondence refers to the interaction and association of different sensory information. We, humans, have a tendency to create cross-modal correspondences. The audio-visual correspondence of Kiki-Bouba is the sound of Kiki matching the shape of Kiki, which is the jagged star shape, for example.

Cross-modal correspondence is the reason why we can think of associating music and colors. Why some art can feel dominant while other art feels submissive. It is the pairing of information from the multiple senses we possess. This information, or stimuli features, may seem completely unrelated.

Q: So Kiki, Bouba, cross-modal? How are they related?
A: The Kiki-Bouba effect is a famous and influential example of this cross-modal correspondence. It is the pairing of sounds (words: Kiki & Bouba) to visual features (shapes: Jagged & Bulbous). But similarly, other senses also have a mapping of this sort. Touch can feel kiki with sand but bouba with cotton.

Examples:
While drinking coffee, the shape of your glass (visual feature) affects the taste through cross-modal correspondence. People consistently associate bright light (visual feature) with high-pitched sounds (auditory feature) through cross-modal correspondence.

Now, ample cross-modal and multi-sensory perception evidence has accumulated that verifies this phenomenon effect [3-4]. For example, drum cymbals typically sound Kiki; dark chocolate tastes Kiki, and bright lights appear Kiki; while a flute sounds Bouba, milk chocolate tastes Bouba and dim lights appear Bouba. An astonishing accuracy of over 90-95% is seen in neurologically normal populations, but in autistic people, the accuracy is only slightly better than that expected by chance.

The accuracy is measured based on how many people agree on which word matches with which shape. In this case, more than 90% of people agree that Kiki represents the jagged shape and Bouba represents the bulbous shape.

There are 2 popular explanations for how this cross-modal experience begins in the first place.

1. The common denominator theory. When we acquire information as we grow up, our general-purpose brain circuitry and early concepts we learn (sounds, shapes, symbols, textures, etc.) are used and re-used to process new information. As we keep processing new information, the re-used circuitry and the early associations that help us learn other things pair up and become a common denominator. Essentially, small components (aka micro-features) of 1 stimulus are similar to small components of another stimulus. Some of these denominators are learned through experience and some are extracted when you are presented with 2 new stimuli. When that happens, the brain is pattern-matching small components. In this case, the angular “K” in kiki matches the angular corners of the shape and the rounded sound “ou” in bouba matches the curves of the shape. Those features were extracted after learning about the shapes and names, or they were already present in the brain as potential micro-features learned through experiences. Both explanations are valid in research. This is the same reason why holding a warm cup of soup can temporarily reduce loneliness – the brain interprets loneliness as lack of social warmth but physical warmth counters it with the common shared denominator of “heat” that emerged from the pairing of heat & social connection that shared a neural circuit. This theory is known under the idea of embodied cognition where abstract concepts like loneliness are an outcome of simple, physical, and literal aspects of interacting with the world.
2. Hubs and spoke theory. Researchers propose the brain has these “hubs” where sensory information integrates with each other to create a “perception”. Like enjoying food is an experience that occurs through the integration of taste, smell, visuals, food sound (crunchiness), etc. They merge in the hub to create a semi-unified experience. Then there are “spokes,” which are neural paths that lead to the hub and out of it, carrying more individual data like just a memory or a single sensory experience like taste. With the hubs and spokes constantly transferring and integrating information, the brain achieves cross-modal integration where it is possible to separate the sensory experience of an event and still have a unified, holistic experience. Inside the hub, small units of sensory information are matched.

The number of experiments that have used this framework has shot up since 2005. It suggests that we are in a new paradigm of understanding perception- A well-integrated deeper linking of the senses and what they represent. Cross-modal correspondences may be unique to biology. Before we make robots that have the ability to perceive as humans do, we need to gather extensive data on how our senses influence each other. We need to find and understand the abstract realm in which the kiki-bouba phenomenon is enabled.

The idea that there is some link between all our sensory organs is not alien. The skin parchment illusion demonstrates that there is cross-talk between sensory organs, and there is feedback from the cross-talk that we perceive. In the experiment demonstrating the illusion, subjects rubbed their hands together, and the sound produced was recorded. Replaying the sound to the subjects evoked sensations of roughness as though the skin turned into parchment paper, and then altering the high frequencies of its sound amplified the rough sensation [6].

The Kiki-Bouba effect has been observed across various age groups, including infants and cultures from around the world and extensive data reliably validates the effect.
Additionally, the effect is observed across multiple types of stimuli, including wine, music, foods, walking, drawings, etc., suggesting a cross-modal ubiquity.

The effect doesn’t stop at shapes being mapped to these words. We have shapes mapped to brightness, light mapped to sound, and even movement mapped to shapes! The putative scientific standpoint is that this phenomenon is universal [7].

Typical Design Of The Kiki-Bouba Experiment

Two shapes are presented to human subjects. One shape is a rounded, bulbous closed figure, and the other shape is a jagged, pointy closed figure (refer to the first image). The subject is then presented with 2 names which are nonsense words (like Kiki & Bouba) and is asked to match each of the shapes with a word. There is generally an agreement of 90%-95% between people over which shape is which word. A number of experiments have tested various groups of people.

A variety of non-sense words such as Takete, Kiriki, Teziki, Kichiki, and Kekiti have been used as substitutes, and in addition to ‘Kiki’ and words like Maluma, Lomoro, Mamima, Muromu, and Malomu for ‘Bouba’ which validate the effect[8].

Something ‘binds’ the shape and the word in ways that it makes sense. Something deep inside our biological brain, our mind. Something abstract.

Theoretical Frameworks To Understand The Kiki-Bouba Effect

Three theoretical frameworks offer an explanation to bring together various findings with the Kiki-Bouba paradigm. The definitions are given below. The following frameworks help identify and understand the clockwork of cross-modal correspondence.

Sound Symbolism

When you read the word “thud,” can you instantly imagine it is the right sound for a dumbbell falling on the floor?

Now compare the 2 words “big” and “gargantuan,” Both refer to something large in size. But the word “gargantuan” gives that sense of hugeness, while the word big doesn’t. The reason is – big is a 3-letter word, and gargantuan is a 10-letter word. The size of the word indicates the size of the meaning. This aspect of our brain has also led to the rise of some universal words that mean the same in any language and whose meaning anyone can guess without knowing the language.

Sound symbolism is the idea that sounds naturally carry meaning independent of semantics (excluding onomatopoeias), and this becomes one of the precursors to the development of language [9]. Congruence between a sound and certain features of a referent (symbols like glyphs and letters + objects) would make that sound more likely to be used in naming that referent. This characteristic called ‘iconicity’ is largely implicated in giving people the ability to make a ‘Kiki-Bouba’ discrimination. It is evident that sound symbolism is a key design feature of the human language [10].

Synesthesia

Synesthesia is a condition where people reliably, automatically, and consistently experience atypical sensory responses in addition to regular & typical responses for certain sensory inputs [11-12]. For example, a synesthete may see each individual numerical digit as a unique color involuntarily while fully understanding the meaning of numbers as non-synesthete people do. In effect, synaesthesia is the co-perceiving of senses. A large body of research has shown that synaesthetic experiences are commonplace in perception and can shed light on how normal humans integrate perceptual experiences [13].

But, kiki-bouba is not synethesia. It is a more diluted, generalized version of it called “ideasthesia.”

Ideasthesia

Ideasthesia describes linking concepts and perception in a steady, consistent way across everything with a brain (as far as we know). In Ideasthesia, the activation of a concept (inducer) evokes a perception-like phenomenon (concurrent) [14]. Ideasthesia involves the brain linking the inducers and concurrents as a general default tendency. Think of inducers as a trigger and concurrent as a mental response. Ideasthesia is a broader phenomenon happening across the senses that explains and widens the scope of synaesthesia-like observations.

Synesthesia fails to explain the cognitive/semantic aspect of perception and many synesthetic experiences are, in fact, a manifestation of ideasthesia through some semantic link [15-16]. Think of it like ideasthesia will make you think that a sharp, salty taste is like a tire screech that pierces the ear. But synesthesia would make that sharp salty taste always feel like the color yellow, for example. The Kiki-Bouba effect is suspected to be a prime example of ideasthesia [17]. In ideasthesia, an inducer is semantic (or carries meaning as a visual or sensation) in nature, and the concurrent is sensory, whereas, in synaesthesia, both are considered to be sensory. For example, the idea of peace somehow fits the color blue in ideasthesia, but the color blue might always trigger a rosy smell for a person.

The easiest way to understand the difference between ideasthesia and synesthesia is that synesthesia is a sense-sense mapping in which the person cannot pause in the head, and it depends solely on that person’s biological makeup. And, ideasthesia is a general phenomenon of concept-sense mapping that culturally and situationally similar people will experience the same way.

Some influential Kiki Bouba research

I’d like to highlight two research findings (2006, 2013) from the above graphic. The Kiki-Bouba effect is seen in toddlers even before language develops! That is something magnificent. We are looking at an abstract realm of sensory information processing because its manifestation precedes language development. This finding got me excited!

Multi-sensory perception lies at the heart of being alive and having meaningful interactions with the environment, advancing research in this domain would elucidate the mechanisms underlying conscious experiences and information representation- coming closer to solving the sensory binding problem^[1].

Applications of this research range from devising sensory substitution remedies for learning problems in impaired people to optimizing marketing strategies, and building a psychological theory of art, and even experience designing.

Think about this:

What makes wine taste better in specific glasses?
What makes certain music better for romance?
What makes a particular color good for attention-grabbing?
What makes a certain logo better?

Found more questions like these? Do comment stating them! 

The answers largely lie in cross-modal correspondence. Just like kiki fits the star shape and bouba fits the round shape, a specific glass fits a specific wine, a specific sound fits a mood.

Summary of the Kiki-Bouba effect

1. It is a non-arbitrary matching of the word kiki with a jagged star shape and bouba with a bulbous round shape.
2. It indicates an abstract layer of feature congruence between two stimuli or concepts
3. It may be unique to biology and form the basis of human perception
4. Understanding it led to the development of cross-modal correspondences. Ways in which one sense affects the other before we perceive information from any sense.
5. Figuring out how this happens may help build robots that see the world like we do. 

Sources:

1. Kohler W. Gestalt Psychology. New York : Liveright: 1929.
2. Kohler W. Gestalt Psychology: An Introduction to new concepts in Modern Psychology. New York : Liveright; 1947.
3. Ramachandran VS, Hubbard EM. Synaesthesia – a window into perception, thought and language. J Consc Stud 2001;8:3-34.
4. Spence C. Cross-modal correspondences: A tutorial review. Attention, Perception and Psychophysics 2011;73(4):971-95.
5. Oberman, LM, Ramachandran VS. Preliminary evidence for deficits in multisensory integration in autism spectrum disorders: The mirror neuron hypothesis. Soc Neurosci 2008;3(3-4):348-55.
6. Jousmaki V, Hari R. Parchment-skin illusion: sound-biased touch. Curr Biol 1998;8(6):190-6.
7. Ramachandran, VS, Hubbard EM. The emergence of the Human mind: Some clues from synesthesia. In: Robertson LC, Sagiv N. (eds). Synesthesia: Perspectives from neuroscience. Oxford University Press; 2004. pp.147-90.
8. Nielsen AK, Rendall D. Parsing the role of consonants versus vowels in the classic Takete-Maluma phenomenon.Can J Exp Psychol 2013;67(2):153-63.
9. Lockwood G, Dingemanse M. Iconicity in the lab: a review of behavioral, developmental, and neuroimaging research into sound-symbolism. Front Psychol 2015;6:1624.
10. Hocket C. The origin of speech. Sci Am 1960;203:89-97.
11. Lupiáñez J, CallejasA. Automatic perception and synaesthesia: evidence from colour and photism naming in a stroop-negative priming task. Cortex 2006;42(2):204-12.
12. Mylopoulos, MI, RoT. Synesthesia: a colorful word with a touching sound?. Front Psychol 2013;4:763.
13. Cytowic RE, Eagleman D. In. Wednesday Is Indigo Blue: Discovering the Brain of Synesthesia.. Cambridge, MA: MIT Press; 2009.
14. Chiou R, Rich AN. The role of conceptual knowledge in understanding synaesthesia: Evaluating contemporary findings from a “hub-and-spokes” perspective. Front Psychol 2014;5;105.
15. Dixon MJ, Smilek D, Duffy PL, Zanna PM, Merikle PM. The Role of Meaning in Grapheme-Colour Synaesthesia. Cortex 2006;42:243-52.
16. Mroczko-WąsowiczA, Nikolić D. Semantic mechanisms may be responsible for developing synesthesia. Front Neurosci 2014;8:509
17. Emilio G, Oscar I, María J, Juárez-Ramos V, Rodríguez A, José L. The Kiki-Bouba Effect A Case of Personification and Ideaesthesia. J Consc Stud 2013;20(1-2):84-102.

Sources