Affect and Cognition in Art: Form versus Content

Stevan Harnad
Department of Psychology
Princeton University

Abstract: This is a theory about the role of affective and cognitive factors in aesthetic perception. Representational art may be either symbolic (literature) or iconic (painting). The content of a symbolic representation is a proposition about the object to which it refers. The content of an iconic representation is a resemblance to an object. The relation of the form of a symbolic representation to its content depends on arbitrary convention; that of an iconic representation is based on physical similarity. Nonrepresentational art (music, abstract painting) has a perceptible physical form, but no content. What makes representational or nonrepresentational productions art is their affective value. Only iconic form elicits affect. Nonrepresentational art does so directly; representational art does so in virtue of metaphoric juxtapositions of content on content, form on form, or form on content, each of which reveals a further form that has aesthetic (affective) value. It follows from this theory that cognitive (content) factors are secondary or incidental in aesthetic experience and that all works of art are "intentionally underdetermined."

The relation between affective and cognitive factors in art will of course vary from art-form to art-form. One would expect, for example, a bigger cognitive component in literature than in music, and in representational than in nonrepresentational art. The reason is that (in a nonmetaphorical sense) literature and representational art are about something, whereas (nonprogrammatic) music and nonrepresentational art are not.

This is not to say that a composer could not have had some cognitive theme in mind -- say, the French Revolution and ideas of liberty and brotherhood -- and even intended his music to "express" it. But that cognitive content would not be intrinsic or essential to the music in the way it would be to a novel that was about the French Revolution, if for no other reason than that one can (fully?) appreciate such music even being unaware of any connection with the French Revolution. Moreover, one must keep in mind the celebrated "intentional fallacy," according to which it is erroneous to believe that the meaning of a work of art is (wholly or largely) determined by and answerable to the artist's intentions: Once created, a work of art is what it is, and the artist, though of course a valuable heuristic source of insight about its meaning, is (paradoxically) not an absolute authority. Besides, whether or not music (literally) has a "meaning" at all is the matter of concern in this essay, entirely apart from whatever "meaningfulness" it may have for the listener (or the composer). This is why I am emphasizing (with scare-quotes) that at issue here is the literal content of a work of art and the meaning it expresses rather than its figurative or metaphorical "content" or "expressive meaning."

Suppose there is a piece of music that a composer says he wrote to express X (where X is a cognitive content like "Mankind should be free," rather than an affective content, such as "joy" or "exhilaration" -- we will return to affect later) and (for the sake of argument) suppose that the listener too agrees that the music expresses X to him. The question is whether there is a sense of "express" in which the music really expresses X that is in any way similar to the literal meaning of "express" or "mean" when we say (likewise with agreement from all parties to the locution) that the sentence "Mankind should be free" expresses or means: Mankind should be free.

That's what we literally mean by "mean" or "express," after all: what a literal statement or sentence does. Of course, there are lots of metaphorical senses to "mean" too, as in "darkening skies mean a storm is coming" (but that obviously just means that darkening skies correlate with, signal or perhaps cause storms, not that they "mean" them) or "darkening skies mean all is lost" (which really says that I take them to resemble, symbolize or portend misfortune). But these are not the literal meanings of meaning, and it's all too easy to forget this in an effusion of figurative speech. Literally speaking, clouds do not mean anything.

"Express" is a more complicated case, because the word has arrived at its current literal meanings (from "press out," to "copy" to "portray" to "represent" to "mean") by a path of metaphoric extension that passed through some of the figurative senses of "mean." One literal sense of "express" is identical to "mean," as in "That sentence expresses (means) X." But there are other ways that people can express themselves (literally) besides in words. "Express" can also be used literally to say "That grimace expresses anger." Note that this is fine if one is talking about the expression of an affective state, where the difference between "means," on the one hand, and "resembles," "correlates with," "portends," "signals" or even "symbolizes" on the other, does not really matter. Expressing cognitive content is another matter, however. [For a detailed discussion of these matters, see the work of Nelson Goodman: "The Languages of Art," "The Structure of Appearance" and "Fact, Fiction and Forecast".]

In brief, a sentence means or expresses cognitive content "symbolically." The symbols are arbitrarily chosen and get their meaning by convention (shared usage). The content of a sentence is a "proposition," which is a claim that something is the case. Propositions can be either true or false. If you say: "X [is true]" and someone else says "No, X is not true," then you couldn't both be right: A proposition cannot be (literally) both true and false, only the one or the other.

In contrast, an image "expresses" something "iconically," which means it does so in virtue of its form, its shape, rather than by arbitrary convention. This may be either because of physical resemblance or because of a natural association or correlation in space or time. Darkening skies express impending misfortune because expecting a storm resembles expecting misfortune or because bad things tend to happen when storms do. But notice that the clouds are not claiming anything by their resemblance. Clouds do not literally express anything. To speak as if they do is merely to suggest that there is a resemblance to be perceived.

FOOTNOTE: Except, of course, if one literally means that the clouds were sent as an omen, which is more like either making a literal (though far-fetched) weather report (similar to: "An approaching low pressure system means temperatures will drop") or else attributing the use of a meteorological convention to the gods. The former would still involve only an iconic association, but the latter would involve a symbolic proposition (made by the gods, not by the clouds -- just as it is ultimately the speaker who means, not the sentence).
But since anything can be perceived as resembling anything (probably in an infinite number of ways), merely "juxtaposing" iconically similar things is very different from proposing that a symbolic statement means X and X is true. There is no "truth" to be denied in mere resemblance relations themselves (although one can deny propositions about their nature or significance).

Earlier I compared literature and music, mentioning also the representational/nonrepresentational distinction. "Representation" too can be either symbolic or iconic. A verbal (literal) proposition "X" is a formal string of symbols that "represents" ("stands for," "means") the state of affairs it refers to (e.g. "The snow is white" represents the state of affairs that consists of the snow being white). An (iconic) image represents the object it resembles (e.g., a photograph of white snow represents white snow). Again, the propositional representation can be said to be true or false, but not the iconic image: An image resembles whatever it resembles (usually a lot of things) and that's all there is to it. No statement, nothing to deny (although perhaps plenty to ignore); just a state of affairs.

One can, of course, "infer" statements from images, but such statements (interpretations, really) are not literally contained in or necessitated by the images. And the artist would of course be free to deny or disavow them in the case of his own images -- or to invoke the intentional fallacy to the effect that such inferences are moot or supererogatory.

The point is well illustrated by Foucault's book about the problem of the image of a pipe accompanied by the statement "Ceci n'est pas une pipe." Such an image/statement does not give rise to a "self-referential paradox" (of the sort involved in the statement "This statement is false," which is indeed paradoxical, being true if false and false if true), because images neither refer nor have truth values. The statement part is simply either true (if it is interpreted to mean that the accompanying image is not literally a pipe), or false (if it means that the image is not one of a pipe), just as it would be false, not paradoxical, if I said that the next symbol after this statement will not be a period. Not only is the hybrid image/statement not paradoxical, but its statement part is not even self-referential (except if interpreted to be saying of itself that it is not a pipe -- likewise true and no paradox). Foucault's example really serves to underscore the difference between sentences and images, in that only the former refer, have truth value, and, literally speaking, mean anything at all. (An image cannot "mean," it can but "be.") It is also significant (and further testimony to the intentional fallacy) that the artist who actually created the image/statement in question could step aside and noncommitally allow Foucault and the rest of us to busy ourselves with drawing whatever "inferences" we please from his work.

So representational art is iconic. It represents in virtue of resemblance. This is quite different from representing in virtue of meaning, the way literature does, through propositions in a natural language. So even literature and representational art differ. Yet both can be described as having content. The content of literature is propositional. The content of representational art is iconic.

Normally, "content" is considered in relation to "form," and that applies here as well. The form of literature is strings of words; its content is the objects and states of affairs that those words are about. The form of representational art is images; its content is the objects that those images resemble (and hence what they are "about"). But what about music and nonrepresentational art? Do they have content in this sense? Are there objects they are "about" in the sense that verbal and iconic representations are about their objects? The answer is no, but that is not all there is to the content/form, representational/nonrepresentational story.

First, we have already noted significant differences between verbal and iconic representation (e.g, verbal representations are statements, iconic ones are not; iconic representations resemble the objects they are "about," verbal ones do not). What might these differences imply? Second, there seems nothing to prevent one from seeing iconic resemblances in music and abstract art too. If anything can be seen to resemble anything, then "nonrepresentational" forms must resemble (hence "represent") things too. And finally, even my treatment of "literature" as it stands makes no distinction between artistic and nonartistic literature: between novels and biographies, between poetry and prose. Suppose we accept that the contents (though not necessarily the form) of empirical or journalistic reports are not art, but simply literal descriptions of what is the case. What sets "artistic" literature apart, making it more like music and painting? For that matter, what distinguishes an iconic representation (e.g., a photograph) as art rather than the mere mechanical production of a resemblance?

It would exceed the scope of this essay to attempt to make the case convincing (see Harnad 1982, "Metaphor and Mental Duality"), but it can be argued that the critical element in all these cases is captured by the concept of metaphor. A metaphor involves the juxtaposition of two (or more) objects for a kind of mental comparison. The objects may be verbally represented, as in poetry, or they may be iconically represented, as in pictorial images; even mental concepts or images can be members of the juxtaposition, whose ultimate outcome is the revelation of a relation of resemblance, one that has some value (because most do not), cognitive, affective or both.

Now we will apply this metaphor view to each of the cases in turn: Factual prose is in general not art, because it does not (deliberately or inadvertently) reveal relationships metaphorically (if it does so at all, it does so literally, by describing them: "A resembles B in that they both [literally] have the property P"). The content of factual prose is literal and empirical: It consists of propositions that are true or false. Fiction, poetry and other literary forms, however, differ from empirical prose to the extent that they are metaphorical, verbally juxtaposing objects, concepts or images so as to reveal resemblances we somehow find valuable.

This account of metaphor certainly sounds bland; no mechanical recipe for creating or analyzing metaphors has ever been proposed (or is likely to exist). Nor is there an independent criterion for what we will find "valuable." However, this bland simplification, if it's right, does bring out one unexpected similarity between literature and the other arts, and that is its essential dependence on iconic similarity: The artistic aspect of prose (in this view) is based on representation in the iconic, nonpropositional sense. This is even more evident with poetry, where sound/sense juxtapositions are part of the art.

This does not go far enough, though, for it fails to distinguish artistic and nonartistic iconic resemblance: Is every iconic similarity metaphoric? Surely not, and again the question of value comes up. Not every revealed similarity is perceived as valuable. Which ones are? If one looks at representational art, one might be tempted to say, at a first pass, that the closer the similarity between image and object (the more faithful the resemblance), the better. But of course this is wrong. Technical expertise or verisimilitude have never been guarantors of artistic value (otherwise the camera would long ago have displaced the brush). (Even the Greeks' "mimetic" theory of art held only that art "imitates" nature, not that it "duplicates" it.)

In the end, the value of a juxtaposition turns out to be a subjective matter (which is not to say that it is arbitrary), answerable to affective judgment in a way in which objective propositional truth, for example, is not. (It is only metaphorically, not literally, that truth is beauty, beauty truth.)

This brings us to nonrepresentational resemblance: music and abstract art. What can they tell us about what art is? Again, the metaphor view may be able to unify these phenomena. It is significant that music and nonrepresentational art are really just formal -- that they do not have content, that they are not "about" anything. In ordinary iconic resemblance, two objects or images are juxtaposed to reveal similarities; the judgment as to whether or not the similarities have value is an affective one, in particular, an aesthetic one. The form emerging from the juxtaposition, particularly that of the resemblance relationship it reveals, is judged by its subjective, affective value.

The primitive relation underlying artistic judgment is the capacity of certain sensory or cognitive forms to generate what we call an aesthetic experience. In music and nonrepresentational art (according to the aesthetic theory called "formalism") what give rise to affects are the sensory (and cognitive) forms themselves, not relations between things they are "about." This is not to say that metaphorical juxtaposition is impossible with pure forms. There is of course their "programmatic" iconic resemblance to real objects; I think that usually plays a minor role. But there is also the juxtaposition of form to form, as in the relation of the familiar to the unfamiliar, theme to variation, form to transformation. These number among the factors that give music (and, presumably, nonrepresentational art) its aesthetic value.

Now consider the most fundamental juxtaposition of all: that of form and content itself. Here, too, the metaphor view and the special case of the nonrepresentational arts may be instructive. In poetic metaphor, contents are first juxtaposed and then the shared form revealed by the juxtaposition is judged for its affective value. The value of this emergent form (if any) then reflects back on its contents. In novels and representational art, the contents that are represented are juxtaposed, not only with one another, but with the contents of our minds (so to speak): with our own pertinent experiences and ideas. The aesthetic value of such a juxtaposition again depends on whether the similarities of form it reveals elicit an affective reaction, which again redounds on its contents.

In all three cases (poetry, prose and painting), the ultimate arbiter is affect, and the process seems to be one in which forms that we find affectively moving somehow elevate or complement their contents. In poetry and painting there are also, of course, more direct juxtapositions of (representing) form and (represented) content (over and above the interaction arising from the juxtaposition of content and content). The language in which something is said or the form in which it is painted will have a direct influence on the affective reaction to the content. But in music and nonrepresentational art, what is the pure form directly juxtaposed to?

The reply may sound surprising, but I will try to show that there may be some truth in it: Pure aesthetic form is juxtaposed to the ongoing, ambient contents of one's own mind at the time. The result is that the thoughts one is having while, say, listening to beautiful music, are elevated and exalted by the juxtaposition. It is here that the fact that everything is potentially similar to everything can be experienced most dramatically. For (apart from whatever ideas the program notes may have planted in our heads -- about the French Revolution, or the biographic circumstances of the composer, or the analytic features of the work) it is likely that each listener has something different going on in his head while listening to a work; and yet everyone's concurrent thoughts will be suffused with a euphoric affect in virtue of their juxtaposition with the music.

A similar though more focused phenomenon occurs with opera, in which the music elevates and complements the story (it does not quite make sense to say that the story complements the music, although I'm not quite sure why -- perhaps because it makes more sense to say form complements content rather than the other way round). The same basic form/content interaction also underlies all representational art, as discussed. And, to go from the sublime to the ridiculous, it is probably also why sentimental background music enhances the appeal of a charity ad or a soap opera on television.

One further point still needs to be considered in the relation between affective and cognitive (content) factors in art. This has to do with the cognitive activity of formal analysis. In the case of historical and biographical analysis, I think it can be agreed that such extrinsic supplementary information about a work of art is or should be only of marginal relevance if the work is really a self-contained and universal masterpiece. One should not need to know the historic circumstances of the composer, or even his intentions.

On the other hand, the information yielded by formal analysis -- and in fact, the experience arising from perceiving a work analytically -- would seem to be another question, since a work's formal structure is, in a sense, intrinsic to it. Now I suggest a hypothesis based on what has already been said: Both biographic and analytic information are largely irrelevant to the appreciation of a work. The role they may play is to keep one's attention fixed on the work long enough so its formal virtues can have their direct affective consequences. However, as with other concurrent contents, the juxtaposition itself will have exalting effects, with the result that if someone has strongly associated the beauty of a composition with an analysis he happened to do (or be given) concurrently, he may may then go on to assume a causal relationship, superstitiously attributing his appreciation of the work to the analysis itself.

My last point concerns the performing arts, which seem to have been excluded in this discussion of form and content. This too concerns the role of cognitive and affective factors in art. What could it mean to say that one appreciates an "intelligent" performance (perhaps even enjoys it more than an effusively expressive but not well thought-out one), a performance in which one has the feeling that the performer has conceptualized how he wants to interpret the piece (and why, presumably), and has then convincingly prepared and executed it accordingly?

There is something in such an observation that squares with what has been argued here, and something that does not. Yes, one wants to hear "convincing" interpretations, whatever that means. And one wants sensitive musical judgment to go into the interpretation of a work.

But one does not want to hear how the performer has "thought it out" -- only that he has felt it, in the affective, expressive sense. For surely that expressive capacity (coupled with sufficient technical expertise and stylistic experience), when juxtaposed with virtually any conceptual interpretation at all, cannot fail to produce a convincing performance. If the listener's ultimate criterion is affective, and all accompanying cognitive content merely incidental (or even distracting), then surely no less should be true of the performer, at least while he is in the thralls of an expressive performance. And might something analogous not likewise have been true of the composer himself (and his accompanying cognition), whilst he was performing his act of creation?

Lest this affectively loaded view of art be viewed as too somnambulistic, I would close with the observations of a distinguished scientist on the subject of the creative process. Louis Pasteur's famous dictum "Chance favors the prepared mind" captures just the relation between the cognitive and affective factor argued for in this essay. The cognitive burden is to prepare the mind with the pertinent knowledge and skill. This technical preparation cannot, of course, guarantee the production of something new and valuable; on the contrary, it is to a degree, paradoxically, at odds with it, threatening to make us slaves to the cognitive conventions in which we have "prepared" ourselves. Valuable innovations, however, to be recognized and appreciated, must be fashioned out of felicitous juxtapositions of the prepared contents of our minds. The juxtapositions are a matter of chance, but the judgment of their value is an affective matter. Our content-driven cognitions can only lead us in conventional or irrelevant directions. Only our form-governed affective judgment can guide and evaluate our performance. And what, finally, guides our affective judgments? Again, music is as good a model as any: Certain acoustic forms have a capacity -- in part native and in part acquired -- to elicit affects that are peculiarly aesthetic. The ear seems to have an innate affinity for the tonal syatem, and exposure to various stylistic traditions seems to selectively strengthen their affective evocativeness. It is the play of chance in such a "prepared" composer's or performer's mind that generates the novel juxtapositions of form -- both improvisatory and interpretative -- that the "prepared" listener's mind in turn perceives as having musical value on the basis of its affective accompaniment.