Synesthesia

The word “synesthesia” or “synaesthesia,” has its origin in the Greek roots, syn, meaning union, and aesthesis, meaning sensation: a union of the senses.  Many researchers use the term “synesthesia” to refer to a perceptual anomaly in which a sensory stimulus associated with one perceptual modality automatically triggers another insuppressible sensory experience which is usually, but not always, associated with a different perceptual modality as when musical tones elicit the visual experience of colors (“colored-hearing”).  Other researchers consider additional unusual correspondences under the category of synesthesias, including the automatic associations of specific objects with genders, ascriptions of unique personalities to numbers, and the involuntary assignment of spatial locations to months or days of the week.  Many synesthetes experience more than one cross-modal correspondence, and others who have unusual cross-modal sensory experiences also have some non-sensory correspondences such as those mentioned above.

Researchers from fields as varied as neurology, neuroscience, psychology and aesthetics have taken an interest in the phenomenon of synesthesia.  Consideration of synesthesia has also shed light on important subjects in philosophy of mind and cognitive science.  For instance, one of the most widely discussed problems in recent philosophy of mind has been to determine how consciousness fits with respect to physical descriptions of the world.  Consciousness refers to the seemingly irreducible subjective feel of ongoing experience, or the character of what it is like.  Philosophers have attempted to reduce consciousness to properties that will ultimately be more amenable to physical characterizations such as representational or functional properties of the mind.  Some philosophers have argued that reductive theories such as representationalism and functionalism cannot account for synesthetic experience.

Another metaphysical project is to provide an account of the nature of color.  There are two main types of views on the nature of color.  Color objectivists take color to be a real feature of the external world.  Color subjectivists take color to be a mind-dependent feature of the subject (or the subject’s experience).  Synesthesia has been used as a counter-example to color objectivism.  Not everyone agrees, however, that synesthesia can be employed to this end.  Synesthesia has also been discussed in regards to the issue of what properties perceptual experiences can represent objects as having (for example, colors).  The standard view is that color experiences represent objects as having color properties, but a special kind of grapheme-color synesthesia may show that color experience can signify numerical value.  If this is right, it shows that perceptual experiences can represent so-called “high-level” properties.

Synesthesia may also be useful in arbitrating the question of how mental processing can be so efficient given the abundance of mentally stored information and the wide variety of problems that we encounter, which must each require highly specific albeit different, processing solutions.  The modular theory of mind is a theory about mental architecture and processing aimed at solving these problems.  On the modular theory, at least some processing is performed in informationally encapsulated sub-units that evolved to perform unique processing tasks.  Synesthesia has been used as support for mental modularity in several different ways.  While some argue that synesthesia is due to an extra module, others argue that synesthesia is better explained as a breakdown in the barrier that keeps information from being shared between modules.

This article begins with an overview of synesthesia followed by a discussion of synesthesia as it has been relevant to philosophers and cognitive scientists in their discussions of the nature of consciousness, color, mental architecture, and perceptual representation, as well as several other topics.

Table of Contents

1. Synesthesia
2. Consciousness
   1. Representationalism
   2. Functionalism
3. Modularity
4. Theories of Color
5. An Extraordinary Feature of Color-Grapheme Synesthesia
6. Wittgenstein’s Philosophical Psychology
7. Individuating the Senses
8. Aesthetics and “Literary Synesthesia”
9. Synesthesia and Creativity
10. References and Further Reading

1. Synesthesia

Most take synesthesia to be a relatively rare perceptual phenomenon. Reports of prevalence vary, however, from 1 in 25,000 (Cytowic, 1997) to 1 in 200 (Galton, 1880), to even 1 in 20 (Simner et al., 2006).  It typically involves inter-modal experiences such as when a sound triggers a concurrent color experience (a photism), but it can also occur within modalities.  For example, in grapheme-color synesthesia the visual experience of alpha-numeric graphemes such as of a “4” or a “g,” induces color photisms.  These color photisms may appear to the synesthete as located inside the mind, in the peri-personal space surrounding the synesthete’s body (Grossenbacher & Lovelace, 2001), or as being projected right where the inducing grapheme is situated perhaps as if a transparency were placed on top of the grapheme (Dixon, et al., 2004).  Reported cross-modal synesthesias also include olfactory-tactile (where a scent induces a tactile experience such as of smoothness), tactile-olfactory, taste-color, taste-tactile and visual-olfactory, among others.  It is not clear which of these is most common.  Some researchers report that colored-hearing is the most commonly occurring form of synesthesia (Cytowic, 1989; Harrison & Baron-Cohen, 1997), and others report that approximately 68% of synesthetes have the grapheme-color variety (Day, 2005).  Less common forms include sound-olfactory, taste-tactile and touch-olfactory.  In recent years, synesthesia researchers have increasingly been attending to associations that don’t fit the typical synesthesia profile of cross activations between sensory modalities, such as associations of specific objects with genders, ascriptions of unique personalities to particular numbers, and the involuntary assignment of spatial locations to months or days of the week.  Many synesthetes report having these unusual correspondences in addition to cross-modal associations.

Most studied synesthesias are assumed to have genetic origins (Asher et al., 2009).  It has long been noted that synesthesia tends to run in families (Galton, 1883) and the higher proportion of female synesthetes has led some to speculate that it is carried by the X chromosome (Cytowic, 1997; Ward & Simner, 2005).  However, there are also reports of acquired synesthesias induced by drugs such as LSD or mescaline (Rang & Dale, 1987) or resulting from neurologic conditions such as epilepsy, trauma or other lesion (Cytowic, 1997; Harrison & Baron-Cohen, 1997; Critchley, 1997).  Recent studies suggest it may even be brought on through training (Meier & Rothen, 2009; Proulx, 2010) or post-hypnotic suggestion (Kadosh et al., 2009).  Another hypothesis is that synesthesia may have both genetic and developmental origins.  Additionally, some researches propose that synesthesia may arise in genetically predisposed children in response to demanding learning tasks such as the development of literacy.

Up until very recently, the primary evidence for synesthesia has come from introspectively based verbal reports.  According to Harrison and Baron-Cohen (1997), synesthesia is late in being a subject of scientific interest because the previously prevailing behaviorists rejected the importance of subjective phenomena and introspective report.  Some other researchers continue to downplay the reality of synesthesia, claiming that triggered concurrents are likely ideational in character rather than perceptual (for discussion and criticism of this view see Cytowic, 1989; Harrison, 2001; Ramachandran & Hubbard, 2001a).  One hypothesis is that synesthetic ideas result from learned associations that are so vivid in the minds of synesthetes that subjects mistakenly construe them to be perceptual phenomena.  As psychologists swung from physicalism back to mentalism, however, subjective experience became more accepted as an area of scientific inquiry.  In recent years, scientists have begun to study aspects of subjectivity, such as the photisms of synesthetes, using third person methods of science.

Recent empirical work on synesthesia suggests its perceptual reality.  For example, synesthesia is thought to influence attention (Smilek et al., 2003). Moreover, synesthetes have long reported that photisms can aid with memory (Luria, 1968).  And indeed, standard memory tests show synesthetes to be better with recall where photisms would be involved (Cytowic 1997; Smilek et al., 2002).

Other studies aimed at confirming the legitimacy of synesthesia have demonstrated that genuine synesthesia can be distinguished from other common types of learned associations in that it is remarkably consistent; over time synesthetes’ sensation pairings (for example, the grapheme 4 with the color blue) remain stable across multiple testings whereas most learned associations do not.  Synesthetes tested and retested to confirm consistency of pairings on multiple occasions, at an interval of years and without warning, exhibit consistency as high as 90% (Baron-Cohen, et al., 1987).  Non-synesthete associators are not nearly as consistent.

Grouping experiments are used to distinguish between perceptual and non-perceptual features of experience (Beck, 1966; Treisman, 1982).  In common grouping experiments, subjects view a scene comprised of vertical and tilted lines.  In perception, the tilted and vertical lines appear as grouped independently.  Studies seem to show some grapheme-color synesthetes to be subject to pop-out and grouping effects based on colored photisms (Ramachandran & Hubbard, 2001a, b; Edquist et al., 2006).  If an array of 2’s in the form of a triangle are hidden within a field of distracter graphemes such as 5’s, the 2’s may “pop-out” or appear immediately and saliently in experience as forming a triangle so long as the color ascribed to the 2’s is incongruent with the color of the 5’s (Ramachandran and Hubbard, 2001b).

Some take these studies to show that, for at least some synesthetes, the concurrent colors are genuinely perceptual phenomena arising at a relatively early pre-conscious stage of visual processing, rather than associated ideas, which would arise later in processing.

Another study often cited as substantiating the perceptual reality of synesthetic photisms shows that synesthetes are subject to Stroop effects on account of color photisms.  When synesthetes were shown a hand displaying several fingers colored to match the color photism the synesthetes typically projected onto things signifying that quantity, they were quicker at identifying the actual quantity of fingers displayed than when the fingers were painted a color that was incongruent with the photism typically associated with things significant of that quantity (Ward and Sagiv, 2007).

Finally, Smilek et al. (2001) have conducted a study with a synesthete they refer to as “C,” that suggests the perceptual reality of synesthesia.  In the study, significant graphemes are presented individually against backgrounds that are either congruent or incongruent with the photism associated with the grapheme.  If graphemes really are experienced as colored, then they should be more difficult to discern by synesthetes when they are presented against congruent backgrounds.  C did indeed have difficulty discerning the grapheme on congruent but not incongruent trials.  In a similar study, C was shown a digit “2” or “4” hidden in a field of other digits.  Again, the background was either congruent or incongruent with the photism C associated with the target digit.  C had difficulty locating the target digit when the background was congruent with the target’s photism color, but not when it was incongruent.

Nevertheless, another set of recent studies could be seen as calling into question whether some of the above studies really demonstrate the perceptual reality of synesthesia.  Meier and Rothen (2009) have shown that non-synesthetes trained over several weeks to associate specific numbers and colors behave similarly to synesthetes on synesthetic Stroop studies.  The colors that the non-synesthetes were taught to associate with certain graphemes interfered with their ability to identify target graphemes.  Moreover, Kadosh et al. (2009) have shown that highly suggestible non-synesthetes report abnormal cross-modal experiences similar to congenital synesthetes and behave similarly to Smilek’s synesthete C on target identification after receiving post-hypnotic suggestions aimed to trigger grapheme-color pairings.  Some researchers conclude from these studies that genuine synesthetic experiences can be induced through training or hypnosis.  But it isn’t clear that the evidence warrants this conclusion as the results are consistent with the presence of merely strong non-perceptual associations.  In the cases of post-hypnotic suggestion, participants may simply be behaving as if they experienced genuine synesthesia.  An alternative conclusion to draw from these studies might be that Stroop and the identification studies conducted with C do not demonstrate the perceptual reality of synesthesia.  Nonetheless, it has not been established that training and hypnotism can replicate all the effects, such as the longevity of associations in “natural” synesthetes, and few doubt that synesthetes experience genuine color photisms in the presence of inducing stimuli.

For most grapheme-color synesthetes, color photisms are induced by the formal properties of the grapheme (lower synesthesia).  In some, however, color photisms can be correlated with high-level cognitive representations specifying what the grapheme is taken to represent (higher synesthesia).  Higher synesthesia can be distinguished from lower synesthesia by several testable behaviors.

First, individuals with higher synesthesia frequently have the same synesthetic experiences (for example, see the same colors) in response to multiple inducers that share meaning—for instance, 5, V, and an array of five dots may all induce a green photism (Ramachandran & Hubbard, 2001b; Ward & Sagiv, 2007).  Second, some higher-grapheme-color synesthetes will experience color photisms both when they are veridically perceiving an external numeral, and also when they are merely imagining or thinking about the numerical concept.  Dixon et al. (2000) showed one synesthete the equation “4 + 3” followed by a color patch.  Their participant was slower at naming the color of the patch when it was incongruent with the photism normally associated with the number that is the solution to the equation.  If thinking about the numerical concept alone induces a photism then we should expect that the photism would interfere with identifying the patch color.

Moreover, when an individual with higher synesthesia sees a grapheme that is ambiguous, for example a shape that resembles both a 13 and a B, he or she may mark it with different colors when it is presented in different contexts.  For instance, when the grapheme is presented in the series, “12, 13, 14,” it may induce one photism, but it may induce a different photism when it is presented in the series, A, 13, C.  This suggests that it isn’t merely the shape of the grapheme that induces the photism here, but also the ascribed semantic value (Dixon et al., 2006).  Similarly, if an array of smaller “3”s are arranged in the form of a larger “5,” an individual with higher-grapheme synesthesia may mark the figure with one color photism when attending to it as an array of “3”s, but mark it with a different color photism when attending to it as a single number “5” (Ramachandran & Hubbard, 2000).

2. Consciousness

Some contend that synesthesia presents difficulties for certain theories of mind when it comes to conscious experience, such as representationalism (Wager, 1999, 2001; Rosenberg, 2004) and functionalism (J.A. Gray, 1998, 2003, 2004, J.A. Gray et al.; 1997, 2002, 2006).  These claims are controversial and discussed in some depth in the following two sections.

a. Representationalism

Representationalism is the view that the phenomenal character of experience (or the properties responsible for “what it is like” to undergo an experience) is exhausted by, or at least supervenes on, its representational content (Chalmers, 2004).  This means that there can be no phenomenal difference in the absence of a representational difference, and, if two experiential states are indiscernible with respect to representational content, then they must have the same phenomenal character.  Reductive brands of representationalism say that the qualitative aspects of consciousness are just the properties represented in perceptual experience (that is, the representational contents).  For instance, perhaps the conscious visual sensation of a faraway aircraft travelling across the sky is just the representation of a silver object moving across a blue background (Tye, 1995, p.93).

According to Wager (1999, 2001) and Rosenberg (2004) synesthesia shows that phenomenal character does not always depend on representational content because mental states can be the same representationally, but differ when it comes to experiential character.  Wager dubs this the “extra qualia” problem (1999, p.268) noting that his objection specifically targets externalist versions of representationalism (p.276) contending that phenomenal content depends on what the world is like (such that perfect physical duplicates could differ in experiential character given that their environments differ).  Meanwhile, Rosenberg (2004, p.101) employs examples of synesthetes who see colors when feeling pain, or hearing loud noises.  According to Rosenberg, there is no difference between the representational content of the synesthete and the ordinary person: in the case of pain, they could both be representing damage to the body of, let us suppose, a certain intensity, location and duration.  Again, the examples are claimed to show that mental states with the same representational content can differ experientially.  However, others reject this sort of argument.

Alter (2006, p.4) argues that Rosenberg’s analysis overlooks plausible differences between the representational contents in question.  A synesthete who is consciously representing bodily damage as, say, orange, is representing pain differently than an ordinary person.  The nature of this representational difference might be understood in more than one way: perhaps the manner in which they represent their intentional objects differs, or, perhaps their intentional objects differ (or both).  In short, it is suggested that the synesthete and the ordinary person are not representationally the same, and it is no threat to representationalism that different kinds of experience represent differently.  To take a trivial case, the conscious difference between touching and seeing a snowball is accounted for in that they represent differently (only one represents the snowball as cold).

Turning to Wager, he considers three cases which all concern a synesthete named Cynthia who experiences extra visual qualia in the form of a red rectangle when she hears the note Middle C.  The cases vary according to the version of externalism in question.  Case 1 examines a simple casual co-variation theory of phenomenal content, case 2 a theory that mixes co-variation and teleology (such as Tye’s, 1995), while case 3 concerns a purely teleological account, (such as Dretske’s, 1995).  These cases purportedly show that synesthetic and ordinary experience can share the same contents despite the differences in qualitative character.  R. Gray’s (2001a, 2004, pp.68-9) general reply is that synesthetic experience does indeed differ representationally in that it misrepresents.

For example, instead of attributing the redness and rectangularity to Middle C, why not attribute these to a misrepresentation of a red rectangle triggered by the auditory stimulus?  Whether representationalism can supply a plausible account of misrepresentation is an open question, however, perhaps its problems with synesthesia can be resolved by discharging this explanatory debt.

Regarding case 1, perhaps there is no extra representational content had by Cynthia.  If content is determined by the co-variation of the representation and the content it tracks, then since there is no red triangle in the external world, perhaps her experience only represents Middle C, just as it does in the case of an ordinary person (Wager, 1999, p.269).  If so, then there would be a qualitative difference in the absence of a representational difference, and this version of representationalism would be refuted.  On the other hand, Wager concedes that the objection might fail if Cynthia has visually experienced red bars in the past, for then her synesthetic experience is arguably not representationally the same as that of an ordinary person hearing Middle C.  This is because it would be open to the externalist to reply that Cynthia’s experience represents the disjunction “red bar or Middle C” (p.270) thus differing from an ordinary person’s.   However, Wager then argues that a synesthete who has never seen red bars because she is congenitally blind (Blind Cynthia) would have the same representational contents as an ordinary person (they would both just represent Middle C) and yet since she would also experience extra qualia, the objection goes through after all.

In reply, R. Gray (2001a, p.342) points out that this begs the question against the externalist, since it assumes that synesthetic color experience does not depend on a background of ordinary color experience.  If this is so, there could not be a congenitally blind synesthete, since whatever internal states Blind Cynthia had would not be representing colors.  Wager has in turn acknowledged this point (2001, p.349) though he maintains that it is more natural to suppose that Blind Cynthia’s experience would nevertheless be very different.  Support for Wager’s view might be found in such examples as color blind synesthetes who report “Martian” colors inaccessible to ordinary visual perception (Ramachandran and Hubbard, 2003a).

Wager also acknowledges that case 1 overlooks theories allowing representational contents to depend on evolutionary functions, and so the possibility that the blind synesthete functions differently when processing Middle C needs to be examined.  This leads to the second and third cases.

Case 2 is designed around Tye’s hybrid theory according to which phenomenal character depends on evolutionary functions for beings that evolved, and causal co-variation for beings that did not–such as Swampman (your perfect physical duplicate who just popped into existence as a result of lightening striking in swamp material).  Wager argues that on Tye’s view Middle C triggers an internal state with the teleological function of tracking red in the congenitally blind synesthete.  Hence Tye can account for the idea that Blind Cynthia would be representing differently than an ordinary person.

However, now the problem is that it seems the externalist must, implausibly, distinguish between the phenomenal contents of the hypothetical blind synesthete and a blind Swampsynesthete (Blind Swamp Cynthia) when they each experience Middle C.  Recall that Tye’s theory does not allow teleology to be used to account for representational contents in Swampperson cases.  But if Tye falls back on causal co-variation the problem, discussed in the first case, returns.  Since the blind Swampsynesthete’s causal tracking of Middle C does not differ from that of an ordinary person, externalism seems committed to saying that their contents and experiences do not differ—that is, since Blind Swamp Cynthia’s state reliably co-varies with Middle C, not red, it cannot be a phenomenal experience of red.

This, however, is not the end of the matter.  R. Gray could try to recycle his reply that there could not be a blind synesthete (whether of swampy origins or not) since synesthesia is parasitic on ordinary color experience.  Still another response offered on behalf of Tye (Gray, 2001a, p.343) is that Wager fails to take note of the role played by “optimal” conditions in Tye’s theory.  Where optimal conditions fail to obtain, co-variation is mere misrepresentation.  But what counts as optimal and how do we know it?  Perhaps optimal conditions would fail to obtain if the co-varying relationships are one-many (that is, if an internal state co-varies with many stimuli, or, a stimulus co-varies with many internal states, Gray, 2001a, p.343).  Such may be the case for synesthetes, and if so, then synesthetic experience would misrepresent and so differ in content.  On the other hand, Wager disputes Gray’s conception of optimal conditions (2001, p.349) arguing that Tye himself accepts they can obtain in situations where co-variation is one-many.  In addition, Wager (2001, p.349) contends Blind Swamp Cynthia’s co-varying relationship is not one-many since her synesthetic state co-varies only with Middle C.  As for Gray’s claim that optimal conditions fail for the Blind Swamp Cynthia because Middle-C co-varies with too many internal states, Wager (2001, p.349) responds that optimal conditions should indeed obtain—for it is plausible that a creature with a backup visual system could have multiple independent states co-varying with, and bearing content about, a given stimulus.  To this, however, it can be replied that having primary and backup states with content says nothing about whether the content of the backup state is auditory or visual; in other words, does Blind Swamp Cynthia both hear and synesthetically see Middle C, or, does she just hear it by way of multiple brain states (cf. Gray, 2001a, pp.343-344)?  While this summary does not exhaust the debate between Wager and Gray, the upshot for case 2 seems to turn on contentious questions about optimal conditions: what are they, and how do we know when they obtain or fail to obtain?

Finally, Case 3 considers the view that phenomenal content always depends on the state’s content tracking function as determined by natural selection.  Hence, an externalist such as Dretske could maintain that the blind synesthete undergoes a misfiring of a state that is supposed to indicate the presence of red, not Middle C.  Wager’s criticism here concerns a hypothetical case whereby synesthesia comes to acquire the evolutionary function of representing Middle C while visual perception has faded from the species though audition remains normal.  This time the problem is that it seems plausible that two individuals with diverging evolutionary histories could undergo the same synesthetic experience, but according to the externalist their contents would differ (Wager, 1999, p.273).  Perhaps worse, it follows from externalism that a member of this new synesthetic species listening to Middle C would have the very same content and experience as an ordinary member of our own species.

R. Gray replies that he does not see why the externalist must agree that synesthesia has acquired an evolutionary function just because it is adaptive (2001a, p.344).  Returning to his point about cases 1 & 2, synesthesia might well result from a breakdown in the visual system, and saying that it has no function is compatible with saying that it is fitness-enhancing.  If synesthesia does not have a teleological function, then a case 3 externalist can deny that the mutated synesthete’s contents are indiscernible with respect to those of an ordinary person.

And yet even if R. Gray is right that the case for counting synesthesia as functional is inconclusive, it seems at least possible some being evolves such that it has states with the function of representing Middle C synesthetically.  Whether synesthesia is a bug or a feature depends on, as Gray acknowledges, evolutionary considerations (p.345, see also Gray, 2001b), so Wager need only appeal to the possible world in which those considerations favor his interpretation and he can have his counterexample to externalist representationalism (cf. Wager, 2001, p.348).

On the other hand, and as R. Gray notices, Wager’s strongest cases are not drawn from the real world – and so his objections likewise turn on the very sort of controversial, “thought experiments and intuitions about possibility” he aims to distance his own arguments from (Wager, 1999, p.264).  Consider that for case 3 externalists, since Swamppeople don’t have evolutionary functions, they are unconscious zombies.  Anybody who is willing to accept that outcome will probably not be troubled by Wager’s imagined examples about synesthetes.  After all, someone who thinks having no history makes one a zombie already believes that differing evolutionary histories can have a dramatic impact on the qualitative character of experience.  In short, a lot rides on whether synethesia in fact is the result of malfunction, or, the workings of a separate teleofunctional module.

Finally, the suggestion that representational properties can explain the “extra-qualia” in synesthesia courts controversy given worries about whether this is consilient with synesthetes’ self-reports (that is, would further scrutiny of the self-reports strongly support claims about additional representational content?).  There is also general uncertainty as to what evidential weight these reports ought to be granted.  Despite Ramachandran and Hubbard’s enthusiasm for the method of, “probing the introspective phenomenological reports of these subjects” (2001b, p.7, n.3), they acknowledge skepticism on the part of many psychologists about this approach.

b. Functionalism

Synesthesia might present difficulties for the functionalist theory of mind’s account of conscious experience.  Functionalism defines mental states in terms of their functions or causal roles within cognitive systems, as opposed to their intrinsic character (that is, regardless of how they are physically realized or implemented).  Here, mental states are characterized in terms of their mediation of causal relationships obtaining between sensory input, behavioral output, and each other.  For example, an itch is a state caused by, inter alia, mosquito bites, and which results in, among other things, a tendency to scratch the affected area.  As a theory of consciousness, functionalism claims that the qualitative aspects of experience are constituted by (or at least determined by) functional roles (for example, Lycan, 1987).

In a series of articles, J.A. Gray has argued that synesthesia serves as a counter-example to functionalism, as well as to Hurley and Noë’s (2003a) specific hypothesis that sensorimotor patterns best explain variations in phenomenal experience.

Hurley and Noë’s theory employs a distinction between what they call “deference” and “dominance.”  Sensory deference occurs when experiential character conforms to cortical role rather than sensory input, and dominance the reverse.  Sometimes,

nonstandard sensory inputs “defer” to cortical activity, as when the stimulation of a patient’s cheek is felt as a touch on a missing arm.  Here cortex “dominates,” in the sense that it produces the feel of the missing limb, despite the unusual input.  One explanation is that nerve impulses arriving at the cortical region designated for producing the feel of a touch on the cheek “spill over” triggering a neighboring cortical region assigned to producing sensation of the arm.  But the cortex can also “defer” to nonstandard input, as in the case of tactile qualia experienced by Braille readers corresponding to activity in the visual cortex. J.A. Gray (2003, p.193) observes that cortical deference, not dominance, is expected given functionalism, since the character of a mental state is supposed to depend on its role in mediating inputs and outputs. If that efferent-afferent mediating role changes, then the sensory character of the state should change with it.

Hurley and Noë (2003a) propose that cortical regions implicated in one sensory modality can shift to another (and, thus be dominated by input) if there are novel sensorimotor relationships available for exploitation.  For support they point out that the mere illusion of new sensorimotor relationships can trigger cortical deference.  Such is the case with phantom limb patients who can experience the illusion of seeing and moving a missing limb with the help of an appropriately placed mirror.  In time, the phantom often disappears, leading to the conjecture that the restored sensory-motor feedback loop dominates the cortex, forcing it to give up its old role of producing sensation of the missing limb.

Hurley and Noë (2003a, p.160) next raise a worry for their theory concerning synesthesia.   Perceptual inputs are “routed differently” in synesthetes, as in the case of an auditory input fed to both auditory and visual cortex in colored hearing (p.137). This is a case of intermodal cortical dominance, since the nonstandard auditory input “defers” to the visual cortex’s ordinary production of color experience.  But theirs is a theory assuming intermodal deference, that is, qualia is supposed to be determined by sensory inputs, not cortex (pp.140, 160).  It would appear that the visual cortex should not be stuck in the role of producing extra color qualia if their account is correct.

Hurley & Noë believe synesthesia raises a puzzle for any account of color experience, namely, why color experience defers to the colors of the world in some cases but not others.  For example, subjects wearing specially tinted goggles devised by Kohler at first see one side of the world as yellow, the other, blue.  However, color experience adapts and the subjects eventually report that the world looks normal once more (so a white object would still look white even as it passes through the visual field from yellow to blue).  On the other hand, synesthetic colors differ in that they “persist instead of adapting away.”

J.A. Gray points out that since colored hearing emerges early in life, there should be many opportunities for synesthetes to explore novel sensorimotor contingencies, such as conflicts between heard color names and the elicited “alien” qualia–a phenomenon reminiscent of the Stroop effect in which it takes longer to say “blue” if it’s written in red ink (Gray, et al., 2006; see also Hurley and Noë, 2003a, p.164, n.27).  Once again, why isn’t the visual cortex dominated by these sensory-motor loops and forced to cease producing the alien colors?  Gray (2003, p.193) calls this a “major obstacle” to Hurley and Noë’s theory since the visual cortex stubbornly refuses to yield to sensorimotor dominance.

In reply, Hurley and Noë have suggested that synesthetes are relatively impoverished with respect to their sensorimotor contingencies (2003a, pp.160, 165, n.27).  For example, unlike the case of normal subjects, where unconsciously processed stimuli can influence subsequent judgment, synesthetic colors need to be consciously perceived for there to be priming effects.  In short, the input-output relationships might not be robust enough to trigger cortical deference.  Elsewhere, Noë and Hurley (2003, p.195) propose that deference might fail to occur because the synesthetic function of the visual cortex is inextricably dependent on normal cortex functioning.  Whether sensorimotor accounts of experience can accommodate synesthesia is a matter of ongoing debate and cannot be decided here.

J.A. Gray, as mentioned earlier, also thinks synesthesia (specifically, colored hearing) poses a broader challenge to functionalism, since it shows that function and qualia come apart in two ways (2003, p.194).  His first argument contends that a single quale is compatible with different functions: seeing and hearing are functionally different, and yet either modality can result in exactly the same color experience (see also Gray, et al., 2002, 2006).  A second argument claims that different qualia are compatible with the same function.  Hearing is governed by only one set of input-output relationships, but gives rise to both auditory and visual qualia in the colored-hearing synesthete (Gray, 2003, p.194).

Functionalist replies to J.A. Gray et al.’s first argument (that is, that there can be functional differences in the absence of qualia differences) are canvassed by MacPherson (2007) and R. Gray (2004).  Macpherson points out (p.71) that a single quale associated with multiple functions is no threat to a “weak” functionalism not committed to the claim that functional differences necessarily imply qualia differences—qualia might be “multiply realizable” at the functional, as well as implementational level (note that qualia differences could still imply functional differences).  She continues arguing that even for “strong” functionalisms that do assert the same type of qualitative state cannot be implemented by different functions, the counter-example still fails.  Token mental states of the same type will inevitably differ in terms of some fine-grained causes and effects (for example, two persons can each have the same green visual experience even though the associated functional roles will tend to be somewhat different, for example, as green might lead to thoughts of Islam in one person, Ireland in another, ecology in still another, or envy, and so on).  In light of this, a natural way to interpret claims about functional role indiscernibility is to restrict the experience type individuating function to a “core” or perhaps “typical” or even “normal” role.  Perhaps a core role operates at a particular explanatory level—sort of as a MAC and a PC can be functionally indiscernible at the user-level while running a web browser, despite differing in terms of their underlying operating systems.  An alternative is to argue that the synesthetic “role” is really a malfunction, and so no threat to the claim that qualia differences imply normal role differences (R. Gray 2004, pp.67-8 offers a broadly similar response).

As for the other side of J.A. Gray’s challenge, namely that synesthesia shows functional indiscernibility does not imply qualia indiscernibility, Macpherson questions whether there really is qualia indiscernibility between normal and synesthetic experience (2007, p.77).  Perhaps synesthetes only imagine, rather than perceptually experience colors (Macpherson, 2007, pp.73ff.).  She also expresses doubts about experimental tests utilizing pop-out, and questions the interpretation of brain imaging studies (p.75)—for example, is an active “visual” cortex in colored hearing evidence of visual experience, or, evidence that this part of the brain has a non-visual role in synesthetes (cf. Hardcastle, 1997, p.387)?  In short, she contends there are grounds for questioning whether there is a clear case in which the experience of a synesthetic color is just like some non-synesthetic color.

Finally, although MacPherson does not make the point, J.A. Gray’s second argument is vulnerable to a response fashioned from her reply to his first argument.  Perhaps the qualia differences aren’t functionally indiscernible because core roles are not duplicated, or because the synesthetic “role” is really just a malfunction.  To make this more concrete, consider Gray’s example in which hearing the word “train” results in both hearing sound and seeing color (2003, p.194).  He claims that this shows that one-and-the-same function can have divergent qualia.  But this is a hasty inference, and conflates the local auditory uptake of a signal with divergent processing further downstream. Perhaps there are really two quite different input-output sets involved–the auditory signal is fed to both auditory and visual cortexes, after all, and so perhaps a single signal is fed into functionally distinct subsystems one of which is malfunctioning.  Malfunction or not, the functionalist could thus argue that Gray has not offered an example of a single function resulting in divergent qualia.

3. Modularity

The modular theory of mind, most notably advanced by Jerry Fodor (1983), holds that the mind is comprised of multiple sub-units or modules within which representations are processed in a manner akin to the processing of a classical computer.  Processing begins with input to a module, which is transformed into a representational output by inductive or deductive inferences called “computations.”  Modules are individuated by the functions they perform.  The mental processing underlying visual perception, auditory perception, and the like, take place in individual modules that are specially suited to performing the unique processing tasks relevant to each.  One of the main benefits of modularity is thought to be processing efficiency.  The time-cost involved if computations were to have access to all of the information stored in the mind would be considerable.  Moreover, since an organism encounters a wide variety of problems, it would have been economical for independent systems to have evolved for performing different tasks.  Some argue that synesthesia supports the modular theory.  Before discussing how synesthesia is taken as evidence for modularity, it will help to understand a bit more precisely, the important role that the concept of modularity plays in psychology.

Many, including Fodor, believe that scientific disciplines reveal the nature of natural kinds.  Natural kinds are thought to be mind-independent natural classes of phenomena that, “have many scientifically interesting properties in common over and above whatever properties define the class” (Fodor, 1983, p.46).  Those who believe that there are natural kinds commonly take things such as water, gold, zebras and penicillin to be instances of natural kinds.  If scientific disciplines reveal the nature of natural kinds, then for psychology to be a bona fide science, the mental phenomena that it takes as its objects of study would also have to be natural kinds.  For those like Fodor, who are interested in categorically delineating special sciences like psychology from more basic sciences, it must be that the laws of the special science cannot be reduced to those of the basic science.  This means that the natural kind terms used in a particular science to articulate that science’s laws cannot be replaced with terms for other more fundamental natural phenomena.  From this perspective, it is highly desirable to see whether modules meet the criteria for natural kinds.

According to Fodor, in addition to the properties that define specific types of modules, all modules share most, if not all, of the following nine scientifically interesting characteristics:  1. They are subserved by a dedicated neural architecture, that is, specific brain regions and neural structures uniquely perform each module’s task.  2. Their operations are mandatory, once a module receives a relevant input the subject cannot override or stop its processing.  3. Modules are informationally encapsulated, their processing cannot utilize information from outside of that module.  4. The information from inside the module cannot be accessed by external processing areas.  5. The processing in modules is very quick.  6. Outputs of modules are shallow and conceptually impoverished, requiring only limited expenditure of computational resources.  7. Modules have a fixed pattern of development that, like physical attributes, may most naturally be attributed to a genetic property.  8. The processing in modules is domain specific, it only responds to certain types of inputs.  9. When modules break down, they tend to do so in characteristic ways.

It counts in favor of a theory if it is able to accommodate, predict and explain some natural phenomena, including anomalous phenomena.  In this vein, some argue that the modular theory is particularly useful for explaining the perceptual anomaly of synesthesia.  But there are competing accounts for how modularity is implicated in synesthesia.  Some think that insofar synesthesia has all the hallmarks of modularity, it likely results from the presence of an extra cognitive module (Segal, 1997).  According to the extra-module thesis, synesthetes possess an extra module whose function is the mapping of, for example, sounds or graphemes (input) to color representations (output).  This grapheme-color module would, according to Segal, possess at least most of the nine scientifically interesting characteristics of modules identified by Fodor:

1. There seems to be a dedicated neural architecture, as lexical-color synesthesia appears uniquely associated with multimodal areas of the brain including the posterior inferior temporal cortex and parieto-occipital junctions (Pausenu et al., 1995).  2. Processing is mandatory, once synesthetes are presented with a lexical or grapheme stimulus the induction of a color photism is automatic and insuppressible.  3. Processing in synesthesia seems encapsulated, information that is available to the subject which might negate the effect has no effect on processing in the color-grapheme module.  4. The information and processing in the module is not made available outside of the module, for example, the synesthete does not know how the system affects mapping.  5. Since the processing in synesthesia happens pre-consciously, it meets the rapid speed requirement.  6. The outputs are shallow, they don’t involve any higher-order theoretically inferred features, just color.  7. Since synesthesia runs in families, is dominant in females, and subjects report having had it for as long as they can remember, synesthesia seems to be heritable, and this suggests that it would have a fixed pattern of development.  The features 8 and 9, domain specificity and characteristic pattern of breakdown, are the only two that Segal cannot easily attribute to the grapheme-color module.  Segal doesn’t doubt that a grapheme-color module could be found to have domain specific processing.  But on account of the rarity of synesthesia, he suspects that it may be too hard to find cases where the lexical or grapheme-color module breaks down.  Harrison and Baron-Cohen (1997) and Cytowic (1997) among others, however, note that for some, synesthesia fades with age and has been reported to disappear with stroke or trauma.

Another explanation for synesthesia that draws on the modular framework is that synesthesia is caused by a breakdown in the barriers that ordinarily keep modules and their information and processing separate (Baron-Cohen et al., 1993; Paulesu et al., 1995).  This failure of encapsulation would allow information from one module to be shared with others.  Perhaps in lexical or grapheme-color synesthesia, information is shared between the speech or text processing module and the color-processing module.  There are two hypotheses for how this might occur.  One hypothesis is that the failure of encapsulation originates with a faulty inhibitory mechanism that normally prevents information from leaking out of a module (Grossenbacher & Lovelace, 2001; Harrison & Baron-Cohen, 1997).  Alternatively, some propose that we are born without modules but sensory processes are pre-programmed to become modularized.  So infants are natural synesthetes, but during the process of normal development extra dendritic connections are paired off, resulting in the modular encapsulation typical of adult cognition (Maurer, 1993; Maurer and Mondloch 2004; see Baron-Cohen 1996 for discussion).  In synesthetes, the normal process of pairing off of extra dendritic connections fails to occur.  Kadosh et al. (2009) claim that the fact that synesthesia can be induced in non-synesthetes post-hypnotically, demonstrates that a faulty inhibitory mechanism is responsible for synesthesia rather than excessive dendritic connections; given the time frame of their study, new cortical connections could not have been established.

The modular breakdown theory may also be able to explain why synesthesia has the appearance of the nine scientifically interesting characteristics that Fodor identifies with mental modules (R. Gray, 2001b).  If this is right, then what reason is there to prefer either the breakdown theory or the extra module theory over the other?  Gray (2001b) situates this problem within the larger debate between computational and biological frameworks in psychology; he argues that the concept of function is central to settling the issue over which account of synesthesia we should prefer.  His strategy is to first determine what the most desirable view of function is.  Based on this, we can then use empirical means to arbitrate between the extra-module theory and the modular breakdown theory.

On the classical view of modularity developed by Fodor, function is elaborated in purely computational terms.  Computers are closed symbol-manipulating devices that perform tasks merely on account of the dispositions of their physical components.  We can describe the module’s performance of a task by appealing to just the local causal properties of the underlying physical mechanisms.  R. Gray thinks that it is desirable for a functional description to allow for the possibility of a breakdown.  To describe something as having broken down seems to mean understanding it as having failed to achieve its proper goal.  The purely computational/causal view of function does not seem to easily accommodate the possibility of there being a breakdown in processing.

R. Gray promotes an alternative conception of function that he feels better allows for the possibility of breakdown.  Gray’s alternative understanding is compatible with traditional local causal explanations.  But it also considers the role that a trait such as synesthesia would have in facilitating the organism’s ability to thrive in its particular external environment, its fitness utility.  Crucially, Gray finds the elaboration of modules using this theory of function to be compatible with Fodor’s requirement that a science’s kind predicates “are ones whose terms are the bound variables of proper laws” (1974, p. 506).  Assuming such an account, whether synesthesia is the result of an extra module or a breakdown in modularity will ultimately depend on how it contributes to the fitness of individuals.  According to Baron-Cohen, in order to establish that synesthesia results from a breakdown in modularity, it would have to be shown that it detracts from overall fitness.  The problem is that synesthesia has not been shown to compromise the bearer of the trait.  In contrast, Gray claims that the burden of proof lies with those who propose that synesthesia results from the presence of an extra-module to show that synesthesia is useful in a particular environment.  But at present, according to Gray, we have no reason to think that it is.  For instance, one indicator that something has some positive fitness benefit for organisms possessing it is the proliferation of that trait in a population.  But synesthesia is remarkably rare (Gray, 2001b).  Gray admits, however, that whether or not synesthesia has such a utility is an open empirical question.

4. Theories of Color

Visual perception seems to, at the very least, provide us with information about colored shapes existing in various spatial locations.  An account of the visual perception of objects should therefore include some account of the nature of color.  Some theorists working on issues pertaining to the nature of color and color experience draw on evidence from synesthesia.

Theories about the nature of color fall broadly into two categories.  On the one hand, color objectivism is the view that colors are mind-independent properties residing out in the world, for example, in objects, surfaces or the ambient light.  Typically, objectivists identify color with a physical property.  The view that color is a mind-independent physical property of the perceived world is motivated both by commonsense considerations and the phenomenology of color experience.  It is part of our commonsense or folk understanding of color, as reflected in ordinary language, that color is a property of objects.  Moreover, the experience of color is transparent, which is to say that colors appear to the subject as belonging to external perceptual objects; one doesn’t just see red, one sees a red fire hydrant or a yellow umbrella.  Color objectivism vindicates both the commonsense view of color and the phenomenology of color experience.  But some take it to be an unfortunate implication of the theory that colors are physical properties of objects, since it seems to entail that each color will be identical to a very long disjunctive chain of physical properties.  Multiple external physical conditions can all cause the same color experience both within and across individuals.  This means that popular versions of objectivism cannot identify a single unifying property behind all instances of a single color.

Subjectivist views, on the other hand, take colors to be mind-dependent properties of the subject or of his or her experience, rather than properties of the distal causal stimulus.  Subjectivist theories of color include the sense-data theory, adverbialism and certain varieties of representationalism.  The primary motivation for color subjectivism is to accommodate various types of non-veridical color experience where perceivers have the subjective experience of color in the absence of an external distal stimulus to which the color could properly be attributed.  One commonly cited example is the after-image. Some claim that the photisms of synesthetes provide another example of non-veridical non-referring color experiences (Fish, 2010; Lycan, 2006; Revonsuo, 2001).  But others argue that the door is open to regarding at least some cases of synesthesia as veridical perceptual experiences rather than hallucinations since photisms are often:  i) perceptually and cognitively beneficial, ii) subjectively like non-synesthetic experiences, and iii) fitness-enhancing.

Still, synesthesia may pose additional difficulties for objectivism.  Consider the implications for objectivism if color synesthesias were to become the rule rather than the exception.  How then would objectivism account for color photisms in cases where they are caused by externally produced sounds?  Revonsuo (2001) suggests that the view that colors can be identified with the objective disjunctive collections of physical properties that cause color experiences would have to add the changes of air pressure that produce sounds to that disjunctive collection of color properties.  This means that if synesthesia became the rule, despite the fact that nothing else about the world would have changed, physical properties that weren’t previously colored would suddenly become colored.  Revonsuo (2001) takes this to be an undesirable consequence for a theory of color.

Enactivism is a theory of perception that takes active engagement with perceptual objects along with other contextual relations to be highly relevant to perception.  Typically, enactivists take perception to consist in a direct relation between perceivers and objective properties.  Ward uses synesthesia in an argument for enactivism about color, proposing that the enactivist theory of color actually combines elements of both objectivism and subjectivism, and is therefore the only theory of color that can account for various facts about anomalous color experiences like synesthesia.

For instance, Kohler fitted normal perceivers with goggles, each of whose lenses were vertically bisected with yellow tinting on one side and blue on the other (Kohler, 1964).  When perceivers first donned the goggles, they reported anomalous color experiences consistent with the lens colors; the world appeared to be tinted yellow and blue.  But after a few weeks of wear, subjects reported that the abnormal tint adapted away.  Ward proposes that synesthetic photisms are somewhat similar to the tinted experiences of Kohler’s goggle wearers.  In both cases, the subject is aware of the fact that their anomalous color experiences are not a reliable guide to the actual colors of things around them.  The two cases are not alike, however, in one important respect.  Whereas goggle wearers’ color experiences adapt to fall in line with what they know to be true about their color experiences, synesthetes’ experiences do not.  This asymmetry calls for explanation and Ward demonstrates that the enactive theory of color provides an elegant explanation for this asymmetry.

According to Ward’s enactive view of color, “An object’s color is its property of modifying incident reflected light in a certain way.”  This is an objective property.  But, “we perceive this [objective] property by understanding the way [subjective] color appearances systematically vary with lighting conditions.”  This view explains the asymmetry noted above in the following way.  Kohler’s goggles interfere with regular color perception.  According to the enactive view of color, the tinted goggles introduce, “a complex new set of relationships between apparent colors, viewing conditions and objective color properties.”  So it is necessary for them to adapt away.  As perceivers acclimate to the fact that their color appearances no longer refer to the colors they had previously indicated, their ability to normally perceive color returns.  Ward assumes that synesthetes do not experience their color photisms as attributed to perceived objects, so they do not impact the synesthetes’ ability to veridically perceive color.  Synesthetes’ photisms fail to adapt away because they do not need to.

Another philosophical problem having to do with the nature of color concerns whether or not phenomenal color experiences are intentional.  If they are, we might wonder what sorts of properties they are capable of representing.  A popular view is that color experiences can only represent objects to have specific color or spectral reflectance properties. Matey draws on synesthesia to support the view that perceptual experiences can represent objects to have high-level properties such as having a specific  semantic value (roughly, as representing some property, thing or concept). This argument for high-level representational contents from synesthesia, it is argued, withstands several objections that can be lodged against other popular arguments such as arguments from phenomenal contrast.  The basic idea is that a special category of grapheme-color synesthesia depends on high-level properties.  In higher-grapheme-color synesthesia, perceivers mark with a particular color, graphemes that share conceptual significance such as the property of representing a number.  Matey argues that these high-level properties penetrate color experiences, and infect their contents so that the color-experiences of these synesthetes represent the objects they are projected onto as being representative of certain numbers or letters.  Matey  demonstrates that the conclusions of the argument from synesthesia may generalize to the common perceptual experiences of ordinary perceivers as well.

5. An Extraordinary Feature of Color-Grapheme Synesthesia

What the subject says about his or her own phenomenal experience usually carries great weight.  However, in the case of color-grapheme synesthesia, Macpherson urges caution (2007, p.76).  A striking and odd aspect of color-grapheme synesthesia is that it may seem to involve the simultaneous experience of different colors in exactly the same place at exactly the same time.  Consider synesthetes who claim to see both colors simultaneously: What could it be like for someone to see the grapheme 5 printed in black ink, but see it as red as well?  How are we to characterize their experience?  To Macpherson this “extraordinary feature” suggests that synesthetic colors are either radically unlike ordinary experience, or perhaps more likely, not experiences at all.  A third possibility would be to find an interpretation compatible with ordinary color experience.  For example, perhaps the synesthetic colors are analogous to a colored-transparency laid over ink (as suggested by Kim et al. 2006, p.196;  see also Cytowic 1989, pp.41, 51 and Cytowic & Eagleman 2009, p.72).  However, this analogy is unsatisfying and gives rise to further puzzlement.

One might expect that the colors would interfere with each other, for example, they should see a darker red when the 5 is printed in black ink, and a lighter red when in white.  And yet synesthetes tend to insist that the colors do not blend (Ramachandran & Hubbard 2001b, p.7, n.3) although if the ink is in the “wrong” color this can result in task performance delays analogous to Stroop-test effects and even induce discomfort (Ramachandran & Hubbard, 2003b, p.50).  Another possibility is that the overlap is imperfect, despite the denials, for example, perhaps splotches of black ink can be distinguished from the red (as proposed by Ramachandran & Hubbard 2001b, p.7, n.3).  Or, maybe there can be a “halo” or edge where the synesthetic and ordinary colors do not overlap—this might make sense of the claims of some that the synesthetic color is not “on” the number, but, as it were, “floating” somewhere between the shape and the subject.  But against these suggestions are other reports that the synesthetic and regular colors match up perfectly (Macpherson, 2007, p.76).

A second analogy from everyday experience is simultaneously seeing what is both ahead of and behind oneself by observing a room’s reflection in a window.  This, however, only recycles the problem.  In seeing a white lamp reflected in a window facing a blue expanse of water, the colors mix (for example, the reflected lamp looks to be a pale blue). Moreover, one does not undergo distinct impressions of the lamp and the region occupied by the waves overlapping with the reflected image (though of course one can alter the presentation by either focusing on the lamp or on the waves).

A third explanation draws on the claim mentioned earlier that the extra qualia can depend on top-down processing, appearing only when the shape is recognized as a letter, or as a number (as in seeing an ambiguous shape in FA5T versus 3456).  There is some reason to think that the synesthetic color can “toggle” on and off depending on whether it is recognized and attended to, as opposed to appearing as a meaningless shape in the subject’s peripheral vision (Ramachandran & Hubbard 2001a, 2001b).  Toggling might also explain reports that emphasize seeing the red, as opposed to (merely?) knowing the ink is black (cf. Ramachandran & Hubbard, 2001b, p.7, n.3).  Along these lines, Kim et al. tentatively suggest that the “dual experience” phenomenon might be explained by rapid switching modulated by changes in attention (2006, p.202).

Cytowic and Eagleman (2009, p.73), in contrast to these ruminations, deny there is anything mysterious or conceptually difficult about the dual presentation of imagined and real objects sharing exactly the same location in physical space.  They contend that the dual experience phenomenon is comparable to visualizing an imaginary apple in the same place as a real coffee cup, “you’ll see there is nothing impossible, or even particularly confusing about two objects, one real and one imagined, sharing the same coordinates.”  This dismissal, however, fails to come to terms with the conundrum.  Instead of an apple, try visualizing a perfect duplicate of the actual coffee cup in precisely the same location (for those who believe they can do this, continue visualizing additional coffee cups until the point becomes obvious).  If Cytowic and Eagleman are to be taken literally this ought to be easy.  The visualization of a contrasting color also meets a conceptual obstacle.  What does it even mean to visualize a red surface in exactly the same place as a real black surface in the absence of alternating presentations (as in binocular rivalry) or blending?

Another perplexing feature of synesthetic color experience are reports of strange “alien” colors somehow different from ordinary color experience.  These “Martian” colors may or may not indicate a special kind of color qualia inaccessible to non-synesthetes, though given the apparent causal role differences from ordinary colors when it comes to such things as “lighting, viewing geometry and chromatic context” (Noë & Hurley, 2003, p.195) this is unsurprising and even expected by broadly functionalist theories of phenomenal experience.  Ramachandran and Hubbard (2001b, pp.5, 26, 30) offer some discussion and conjectures about the underlying neural processes.

Whether the more bizarre testimony can be explained away along one (or more) of the above suggestions, or has deep implications about synesthesia, self-report, and the nature of color experience, demands further investigation by philosophers and scientists.

6. Wittgenstein’s Philosophical Psychology

Ter Hark (2009) offers a Wittgensteinian analysis of color-grapheme synesthesia, arguing that it fails to fit the contrast between perception and mental imagery, and so calls for a third category bearing only some of the logical marks of experience.  He contends that it is somewhat like a percept in that it depends on looking, has a definite beginning and end, and is affected by shifts in attention.  On the other hand, it is also somewhat like mental imagery in that it is voluntary and non-informative about the external world.

Although ter Hark cites Rich et al. (2005) for support, only 15% of their informants claimed to have full control over synesthetic experience (that is, induced by thought independent of sensory stimulation) and most (76%) characterized it as involuntary.  It would therefore seem that ter Hark’s analysis applies to only a fraction of synesthetes.  The claim that synesthetic percepts seem non-experiential because they fail to represent the world is also contestable.  Visual experience need not always be informative (for example, hallucinations, “seeing stars,” and so forth) and failing to inform us about the world is compatible with aiming to do so but misrepresenting.

7. Individuating the Senses

Synesthesia might be important when it comes to questions about the nature of the senses, how they interact, and how many of them there are.  For example, Keeley (2002) proposes that synesthesia may challenge the assumption that the various senses are, “significantly separate and independent” (p.25, n.37) and so complicate discussions about what distinguishes one sense from another.  A similar point is made by Ross who notes that synesthesia undermines his “modified property condition” (2001, p.502).  The modified property condition is supposed to be necessary for individuating the senses, and states that each sense modality specializes in detecting certain properties (2001, p.500).  As discussed in the section on representationalism, synesthesia might seem to indicate that properties usually deemed proprietary to one sense can be detected by others after all.  Meanwhile, Ross’ proposal that synesthesia be explained away as a memory association seems unpersuasive in light of the preponderance of considerations suggesting it is a genuine sensory phenomenon (see Ramachandran & Hubbard, 2001a, 2001b, 2003b; for further discussion of Ross see Gatzia, 2008).  At present, little seems to have been written by philosophers on the significance of synesthesia as concerns the individuation and interaction of the senses (though see Macpherson, 2007, O’Callaghan 1998, p.325 and R. Gray 2011, p.253, n.17).

8. Aesthetics and “Literary Synesthesia”

The use of “intersense analogy” or sense-related metaphor as a literary technique is long familiar to authors and critics (for example, a sharp taste, a loud shirt) perhaps starting with Aristotle who noticed a “sort of parallelism between what is acute or grave to hearing and what is sharp or blunt to touch” (quoted in O’Malley, 1957, p.391).  Intersense metaphors such as “the sun is silent” (Dante quoted in O’Malley, 1957, p.409) and, more recently, “sound that makes the headphones edible” (from the lyrics of a popular rock band) may be, “a basic feature of language” natural for literature to incorporate (O’Malley, 1957, p.397), and to some “an essential component in the poetic sensibility” (Götlind, 1957, p.329).  Such “literary” synesthesia is therefore an important part of aesthetic criticism, as in Hellman’s (1977, p.287) discussion of musical styles, Masson’s analysis of acoustic associations (1953, p.222) and Ueda’s evaluation of cross-modal analogies in Haiku poetry which draw attention to “strange yet harmonious” combinations (1963, p.428).

Importantly, “the writer’s use of the ‘metaphor of the senses’” (O’Malley, 1957, p.391) is not to be confused with synesthesia as a sensory phenomenon, as repeatedly noted over the years by several philosophical works on poetry and aesthetics including Downey (1912, p.490), Götlind (1957, p.328) and O’Malley (1958, p.178).  Nevertheless, there are speculations about the connection between the two (for example, Smith, 1972, p.28; O’Malley, 1957, pp.395-396) and sensory synesthesia has been put forward as an important creative source in poetry (Downey, 1912, pp.490-491; Rayan, 1969), music and film (Brougher et al., 2005), painting (Tomas, 1969; Cazeaux, 1999; Ione, 2004) and artistic development generally (Donnell & Duignan, 1977).

That not all sensory matches work aesthetically—it seems awkward to speak of a loud smell or a salty color—might be significant in suggesting ties to perceptual synesthesia.  Perhaps they have more in common than is usually suspected (Marks, 1982; Day 1996).

Synesthetic metaphor is a “human universal” found in every culture and may be an expression of our shared nature (Pinker, 2002, p.439).  Maurer and Mondloch (2004) suggest that the fact that the cross-modal parings in synesthesias tend to be the same as the sensory matches manifest in common metaphors may reveal that non-synesthete adults share cross-modal activations with synesthetes, and synesthesia is a normal feature of early development.  Matey suggests that this lends credibility to the view that the cross-wiring present in synesthetes and non-synesthetes differs in degree and so we may draw conclusions about the types of representational contents possible of normal perceivers’ experiences based on the perceptual contents of synesthetes.

9. Synesthesia and Creativity

Ramachandran and Hubbard, among others, have been developing a number of hypotheses about the explanatory value of synesthesia towards creativity, the nature of metaphor, and even the origins of language (2001b, 2003a; see also Mulvenna, 2007; Hunt, 2005).  Like synesthesia, creativity seems to consist in, “linking two seemingly unrelated realms in order to highlight a hidden deep similarity” (Ramachandran & Hubbard, 2001b, p.17).  Ramachandran and Hubbard (2001b) conjecture that greater connectivity (or perhaps the absence of inhibitory processes) between functionally discrete brain regions might facilitate creative mappings between concepts, experiences, and behaviors in both artists and synesthetes.  These ideas are controversial and although there is some evidence that synethetes are more likely to be artists (for example, Ward et al., 2008; Rothen & Meier, 2010) the links between synesthesia and creativity remain tentative and conjectural.

10. References and Further Reading

• Alter, T. (2006). Does synesthesia undermine representationalism? Psyche, 12(5).
• Asher, J.E., Lamb, J., Brocklebank, D., Cazier, J., Maestrini, E., Addis, L., … Monaco, A. (2009). A whole-genome scan and fine-mapping linkage study of auditory-visual synesthesia reveals evidence of linkage to chromosomes. American Journal of Human Genetics, 84, 279-285.
• Baron-Cohen, S. (1996). Is there a normal phase of synaesthesia in development? Psyche, 2(27).
• Baron-Cohen, S., Wyke, M.A., & Binnie, C. (1987). Hearing words and seeing colours: An experimental investigation of a case of synaesthesia. Perception, 16(6), 761-767.
• Beck, J. (1966). Effect of orientation and of shape similarity on perceptual grouping. Perception and Psychophysics, 1, 300-302.
• Baron-Cohen, S., Harrison, J., Goldstein, L., & Wyke, M.A. (1993). Coloured speech perception: Is synaesthesia what happens when modularity breaks down? Perception, 22, 419-426.
• Brougher, K., Mattis, O., Strick, J., Wiseman, A., & Zikczer, J. (2005). Visual music: Synaesthesia in art and music since 1900. London: Thames and Hudson.
• Cazeaux, C. (1999). Synaesthesia and epistemology in abstract painting. British Journal of Aesthetics, 39(3), 241-251.
• Chalmers, D. (2004). The representational character of experience. In B. Leiter (Ed.), The Future for Philosophy (pp.153-181). Oxford: Clarendon Press.
• Critchley, E.M.R. (1997). Synaesthesia: Possible mechanisms. In S. Baron-Cohen & J. Harrison (Eds.), Synaesthesia: Classical and contemporary readings (pp.259-268). Cambridge, Massachusetts: Blackwell
• Cytowic, R.E. (1989). A union of the senses. New York: Springer-Verlag.
• Cytowic, R.E. (1997). Synesthesia: Phenomenology and neuropsychology: A review of current knowledge. In S. Baron-Cohen & J. Harrison (Eds.), Synaesthesia: Classical and contemporary readings (pp.17-39). Cambridge, Massachusetts: Blackwell.
• Cytowic, R.E., & Eagleman, D. (2009). Wednesday is indigo blue: Discovering the brain of synesthesia. Cambridge: The MIT Press.
• Day, S.A. (1996). Synaesthesia and synaesthetic metaphor. Psyche, 2(32).
• Day, S.A. (2005). Some demographic and socio-cultural aspects of synesthesia. In L. Robertson & N. Sagiv (Eds.), Synesthesia: Perspectives from cognitive neuroscience (pp.11-33). Oxford: Oxford University Press.
• Dixon, M.J., Smilek, D., Cudahy, C., & Merikle, P.M. (2000). Five plus two equals yellow. Nature, 406, 365.
• Dixon, M.J., Smilek, D., & Merikle, P.M. (2004). Not all synaesthetes are created equal: Projector versus associator synaesthetes. Cognitive, Affective & Behavioral Neuroscience, 4(3), 335-343.
• Dixon, M.J., Smilek, D., Duffy, P.L., Zanna, M.P., & Merikle, P.M. (2006). The role of meaning in grapheme-colour synaesthesia. Cortex, 42(2), 243-252.
• Donnell, C.A., & Duignan, W. (1977). Synaesthesia and aesthetic education. Journal of Aesthetic Education, 11, 69-85.
• Downey, J.E. (1912). Literary Synesthesia. The Journal of Philosophy, Psychology and Scientific Methods, 9(18), 490-498.
• Dretske, F. (1995). Naturalizing the Mind. Cambridge, MA: The MIT Press.
• Edquist, J., Rich, A.N., Brinkman, C., & Mattingley, J.B. (2006). Do synaesthetic colours act as unique features in visual search? Cortex, 42(2), 222-231.
• Fish, W. (2010). Philosophy of perception: A contemporary introduction. New York: Routledge.
• Fodor, J. (1974). Special sciences, or the disunity of science as a working hypothesis. Synthese, 28, 97-115.
• Fodor, J. (1983). Modularity of mind: An essay on faculty psychology. Cambridge, MA: MIT Press.
• Galton, F. (1880). Visualized numerals. Nature, 22, 494-495.
• Galton, F. (1883). Inquiries into human faculty and its development. Dent & Sons: London.
• Gatzia, D.E. (2008). Martian colours. Philosophical Writings, 37, 3-16.
• Gray, J.A. (2003). How are qualia coupled to functions? Trends in Cognitive Sciences, 7(5), 192-194.
• Gray, J.A. (2004). Consciousness: Creeping up on the hard problem. Oxford: Oxford University Press.
• Gray, J.A., Williams, S.C.R., Nunn, J., & Baron-Cohen, S. (1997). Possible implications of synaesthesia for the question of consciousness. In S. Baron-Cohen & J. Harrison (Eds.), Synaesthesia: Classic and contemporary readings (pp.173-181). Cambridge, MA: Blackwell.
• Gray, J.A. (1998).  Creeping up on the hard question of consciousness. In S. Hameroff, A. Kaszniak & A. Scott (Eds.), Toward a science of consciousness II: The second Tucson discussions and debates (pp.279-291). Cambridge, MA: The MIT Press.
• Gray, J.A., Nunn J., & Chopping S. (2002). Implications of synaesthesia for functionalism: Theory and experiments. Journal of Consciousness Studies, 9(12), 5-31.
• Gray, J.A., Parslow, D.M., Brammer, M.J., Chopping, S.M., Vythelingum, G.N., & Ffytche, D.H. (2006). Evidence against functionalism from neuroimaging of the alien colour effect in synaesthesia. Cortex, 42(2), 309-318.
• Gray, R. (2001a). Synaesthesia and misrepresentation: A reply to Wager. Philosophical Psychology, 14(3), 339-346.
• Gray, R. (2001b). Cognitive modules, synaesthesia and the constitution of psychological natural kinds. Philosophical Psychology, 14(1), 65-82.
• Gray, R. (2004). What synaesthesia really tells us about functionalism. Journal of Consciousness Studies, 11(9), 64-69.
• Gray, R. (2011). On the nature of the senses. In F. Macpherson (Ed.), The Senses: Classic and contemporary philosophical perspectives, pp.243-260. New York: Oxford University Press.
• Götlind, E. (1957). The appreciation of poetry: A proposal of certain empirical inquiries. The Journal of Aesthetics and Art Criticism, 15(3), 322-330.
• Grossenbacher, P.G., & Lovelace, C.T. (2001). Mechanisms of synesthesia: Cognitive and physiological constraints. Trends in Cognitive Sciences, 5(1), 36-42.
• Hardcastle, V.G. (1997). When a pain is not. The Journal of Philosophy, 94(8), 381-409.
• Harrison, J.E. (2001). Synaesthesia: The strangest thing. New York: Oxford University Press.
• Harrison, J.E., & Baron-Cohen, S. (1997). Synaesthesia: A review of psychological theories. In S. Baron-Cohen & J. Harrison (Eds.), Synaesthesia: Classic and contemporary readings (pp.109-122). Cambridge, MA: Blackwell.
• Hellman, G. (1977). Symbol systems and artistic styles. The Journal of Aesthetics and Art Criticism, 35(3), 279-292.
• Hunt, H. (2005). Synaesthesia, metaphor, and consciousness: A cognitive-developmental perspective. Journal of Consciousness Studies, 12(12), 26-45.
• Hurley, S., & Noë, A. (2003a). Neural plasticity and consciousness. Biology and Philosophy, 18, 131-168.
• Hurley, S., & Noë, A. (2003b). Neural plasticity and consciousness: Reply to Block. Trends in Cognitive Sciences, 7(1), 342.
• Ione, A. (2004). Klee and Kandinsky: Polyphonic painting, chromatic chords and synaesthesia. Journal of Consciousness Studies, 11(3-4), 148-158.
• Keeley, B.L. (2002). Making sense of the senses: Individuating modalities in humans and other animals. The Journal of Philosophy, 99(1), 5-28.
• Kim, C-Y., Blake, R., & Palmeri, T.J. (2006). Perceptual interaction between real and synesthetic colors. Cortex, 42, 195-203.
• Kadosh R.C., Henik, A., Catena, A., Walsh, V., & Fuentes, L.J. (2009). Induced cross-modal synaesthetic experiences without abnormal neuronal connections. Psychological Science, 20(2), 258-265.
• Kohler, I. (1964). Formation and transformation of the perceptual world. Psychological Issues 3(4, Monogr. No. 12), 1-173.
• Lycan, W. (1987). Consciousness. Cambridge, MA: The MIT Press.
• Lycan, W. (2006). Representational theories of consciousness. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy.
• Luria, A.R. (1968). The mind of a mnemonist. New York: Basic Books.
• Macpherson, F. (2007). Synaesthesia, functionalism and phenomenology. In M. de Caro, F. Ferretti & M. Marraffa (Eds.), Cartographies of the mind: Philosophy and psychology in intersection series: Studies in brain and mind (Vol.4, pp.65-80). Dordrecht, The Netherlands: Springer.
• Marks, L.E. (1982). Synesthetic perception and poetic metaphor. Journal of experimental psychology: Human perception and performance, 8(1): 15-23.
• Masson, D.I. (1953). Vowel and consonant patterns in poetry. The Journal Aesthetics and Art Criticism, 12(2), 213-227.
• Maurer, D. (1993). Neonatal synesthesia: Implications for the processing of speech and faces. In B. de Boysson-Bardies, S. de Schonen, P. Jusczyk, P. Mcneilage & J. Morton (Eds.), Developmental neurocognition: Speech and face processing in the first year of life (pp.109-124). Dordrecht: Kluwer.
• Maurer, D., & Mondloch, C. (2004). Neonatal synesthesia: A re-evaluation. In L. Robertson & N. Sagiv (Eds.), Attention on Synesthesia: Cognition, Development and Neuroscience, (pp. 193-213). Oxford: Oxford University Press.
• Meier, B., & Rothen, N. (2009). Training grapheme-colour associations produces a synaesthetic Stroop effect, but not a conditioned synaesthetic response. Neuropsychologia, 47(4), 1208-1211.
• Mulvenna, C.M. (2007). Synaesthesia, the arts and creativity: A neurological connection. Frontiers of Neurology and Neuroscience, 22, 206-222.
• Noë, A., & Hurley, S. (2003). The deferential brain in action. Trends in Cognitive Sciences, 7(5), 195-196.
• O’Callaghan, C. (1998). Seeing what you hear: Cross-modal illusions and perception. Philosophical Issues, 18(1), 316-338.
• O’Malley, G. (1957). Literary synesthesia. The Journal of Aesthetics and Art Criticism, 15(4), 391-411.
• O’Malley, G. (1958). Shelley’s “air-prism”: The synesthetic scheme of “Alastor.” Modern Philology, 55(3), 178-187.
• Paulesu, E., Harrison, J., Baron-Cohen, S., Watson, J.D.G., Goldstein, L., Heather, J., … Frith, C.D. (1995). The physiology of coloured hearing: A PET activation study of colour-word synaesthesia, Brain 118, 661-676.
• Pettit, P. (2003). Looks red. Philosophical Issues, 13(1), 221-252.
• Pinker, S. (2002). The blank slate: The modern denial of human nature. New York: Viking.
• Proulx, M.J. (2010). Synthetic synaesthesia and sensory substitution. Consciousness and Cognition, 19(1), 501-503.
• Ramachandran, V.S., & Hubbard, E.M. (2000). Number-color synaesthesia arises from cross-wiring in the fusiform gyrus. Society for Neuroscience Abstracts, 30, 1222.
• Ramachandran, V.S., & Hubbard, E.M. (2001a). Psychophysical investigations into the neural basis of synaesthesia. Proceedings of the Royal Society of London B, 268, 979-983.
• Ramachandran, V.S., & Hubbard, E.M. (2001b). Synaesthesia: A window into perception though and language. Journal of Consciousness Studies, 8(12), 3-34.
• Ramachandran, V.S., & Hubbard, E.M. (2003a). Hearing colors, tasting shapes. Scientific American, April, 52-59.
• Ramachandran, V.S., & Hubbard, E.M. (2003b). The phenomenology of synaesthesia. Journal of Consciousness Studies, 10(8), 49-57.
• Rang, H.P., & Dale, M.M. (1987). Pharmacology. Edinburgh: Churchill Livingstone.
• Rayan, K. (1969). Edgar Allan Poe and suggestiveness. The British Journal of Aesthetics, 9, 73-79.
• Revonsuo, A. (2001). Putting color back where it belongs. Consciousness and Cognition, 10(1), 78-84.
• Rich, A.N., Bradshaw, J.L., & Mattingley, J.B. (2005). A systematic, large-scale study of synaesthesia: Implications for the role of early experience in lexical-colour associations, Cognition, 98, 53-84.
• Rosenberg, G. (2004). A place for consciousness: Probing the deep structure of the natural world. Oxford: Oxford University Press.
• Ross, P.W. (2001). Qualia and the senses. The Philosophical Quarterly, 51(205), 495-511.
• Rothen, N., & Meier, B. (2010). Higher prevalence of synaesthesia in art students. Perception, 39, 718-720.
• Segal, G.M.A. (1997). Synaesthesia: Implications for modularity of mind. In S. Baron-Cohen & J. Harrison (Eds.), Synaesthesia: Classic and contemporary readings (pp.211-223). Cambridge, MA: Blackwell.
• Simner, J., Sagiv, N., Mulvenna, C., Tsakanikos, E., Witherby, S., Fraser, C., … Ward, J. (2006). Synaesthesia: The prevalence of atypical cross-modal experiences. Perception, 35, 1024-1033.
• Smilek, D., Dixon, M.J., Cudahy, C., & Merikle, P.M. (2001). Synaesthetic photisms influence visual perception. Journal of Cognitive Neuroscience, 13, 930-936.
• Smilek. D., Dixon, M.J., Cudahy, C. & Merikle, P.M. (2002). Synesthetic color experiences influence memory. Psychological Science, 13(6), 548-552
• Smilek, D., Dixon M.J., & Merikle P.M. (2003). Synaesthetic photisms guide attention. Brain & Cognition, 53, 364-367.
• Ter Hark, M. (2009). Coloured vowels: Wittgenstein on synaesthesia and secondary meaning. Philosophia: Philosophical Quarterly of Israel, 37(4), 589-604.
• Tomas, V. (1969). Kandinsky’s theory of painting. British Journal of Aesthetics, 9, 19-38.
• Treisman, A. (1982). Perceptual grouping and attention in visual search for features and for objects. Journal of Experimental Psychology: Human perception and performance, 8(2), 194-214.
• Tye, M. (1995). Ten problems of consciousness: A representational theory of the phenomenal mind. Cambridge, MA: The MIT Press.
• Ueda, M. (1963). Basho and the poetics of “Haiku.” The Journal of Aesthetics and Art Criticism, 21(4), 423-431.
• Wager, A. (1999). The extra qualia problem: Synaesthesia and Representationalism. Philosophical Psychology, 12(3), 263-281.
• Wager, A. (2001). Synaesthesia misrepresented. Philosophical Psychology, 14(3), 347-351.
• Ward, J., & Simner, J. (2005). Is synaesthesia an X-linked dominant trait with lethality in males? Perception, 34(5), 611-623.
• Ward, J., & Sagiv, N. (2007). Synaesthesia for finger counting and dice patterns: A case of higher synaesthesia? Neurocase, 13(2), 86-93.
• Ward, J., Thompson-Lake, D., Ely, R., & Kaminski, F. (2008). Synaesthesia, creativity and art: What is the link? British Journal of Psychology, 99, 127-141.
• Wittgenstein, L. (1958/1994). Philosophical investigations. Oxford: Blackwell.

 

Author Information

Sean Allen-Hermanson
Email: hermanso@fiu.edu
Florida International University
U. S. A.

and

Jennifer Matey
Email: jmatey@fiu.edu
Florida International University
U. S. A.