Sensory intermodal resonance: Towards a Neuroaesthetics of architecture

Architecture, if addressed as a discipline of art, is that one discipline that speaks to the broadest spectrum of sense modalities: to the remote senses and the close-up senses, to the tactile senses and, in particular, to bodily proprioception. The impression that a piece of architecture makes remains trifling if only the visual sense feels addressed. The vertical structure of a facade remains featureless if no musical sense of rhythm wakes up. A surface looks interesting if the haptic sense nods. A flight of stairs pleases if its sight conveys an idea of moving vertically with grace. A corridor has appropriate proportions if the acoustic sense of width and narrowness feels comfortable. An entrance is well designed if my bodily self-awareness feels welcome and honoured. The aesthetic quality of architecture thus relies on the contribution not only of senses that are directly stimulated, but of modalities also that indirectly react to the activity of other modalities. The musical sense of rhythm that applauds to the way an array of columns is composed is not stimulated acoustically, the surface needs not to be touched for looking interesting, the bodily self-awareness that feels well or badly treated by the architecture accommodating the body is not informed by the body’s self-perception, but by the neural activity of its outer senses. Perceptions triggered by the activity of alien sense modalities are irregular regarding orthodox epistemology. We find them orchestrating resonances among one another that make perception realize more than reason understands. Dealing with these resonances represents the real challenge of architectural design. Intermodal resonance, that is the thesis the lecture illustrates, is what the aesthetics of architecture relies on. In the context of neuroscience, the existence and efficacy of intermodal resonance is an empirical question. The thesis thus leaves the hermeneutical and speculative withdrawal areas of aesthetics. It does so, however, without leaving phenomenology behind. *Correspondence to: Georg Franck, Institute of Architectural Sciences, E259.1 Digital Methods in Architecture and Planning, Vienna of Technology, Treitlstrasse 3, A-1040 Wien, Vienna, Tel: +43 1 58801-27201; Fax: +43 1 5880127299; E-mail: Georg.Franck@tuwien.ac.at


Introduction
Since antiquity it is common wisdom that beauty is in the eye of the beholder. Aesthetic quality, i.e., should not be attributed to reality as thought to exist independently of experience, but to the performance of perception. The perception capable of infusing aesthetic quality must not be confused, of course, with the scanning of patterns that just informs the organism of its surroundings and its inner physiological functions. As long as perception is restricted to the processing of sensory stimuli into representations that have visual, auditory, tactile or olfactory properties, there is no reason to expect the emergence of aesthetic value or disvalue. Value, in the last analysis, is due to feelings. Feelings, in their own turn, are either pleasant or unpleasant. In order to bring forth aesthetic quality, perception has to involve the capacity of feeling.
The capacity of feeling has neural correlates as does the capacity of processing sensory input into representations. Remarkably, there is even a location in the medial orbito-frontal cortex (mOFC), the activity of which correlates significantly and reliably with reports of subjects to have the experience of beauty in both vision and audition [1]. A way of trivialising the aesthetics of architecture would thus lie in defining architectural quality as the collection of features displayed by the buildings whose perception correlates with activity at location A1 in mOFC in a statistically representative sample of subjects. Experts claiming competence in aesthetics -be they architects, critics, art historians, architectural theorists -refuse such an equation of architectural quality with popular taste. They refer to a lengthy process of validation: the selection of the classics, whose quality eventually ceases to be a question just of subjective preference 1 [2]. This process embedded in so-called discourse of architecture is slow but indeed capable of sorting out works of paradigmatic quality. No-one can claim expertise in architecture whose taste is not educated by studying the classics.
When taken as a classic discipline of the arts, architecture is that one discipline that most clearly of all speaks to all the sense modalities: to the remote senses and the close-up senses, to the tactile senses, bodily proprioception and even to the olfactory senses. The impression that the architecture makes remains flat and plain if only the sense of sight feels addressed ( Figure 1).
The vertical structure of a façade remains unattractive if no musical sense of rhythm feels invited to respond ( Figure 2).
A surface looks interesting if the haptic sense shows interest. Look how differently the bronze appeals to the haptic sense than does the marble. Regarding form, the bodies are of equal perfection, regarding eroticism, they are not. Eroticism, to remind, is the aesthetic surplus of sexuality (Figure 3,4).
A flight of stairs in a hall pleases if its sight conveys an idea of an uplifting feeling of stepping it down to welcome guests ( Figure 5).
A corridor has appropriate proportions if the acoustic sense of width and narrowness feels comfortable (Figure 6, 7) An entrance is well designed if my body's self-awareness feels treated courteous (Figure 8,9).
In contrast Figure 10.

Hypothesis I
The aesthetic quality of architecture thus relies on the contribution not only of senses that are directly stimulated, but of modalities also that are indirectly reacting to the activity of other modalities [3]. The musical sense of rhythm that applauds to the way an array of columns is composed is not stimulated acoustically, but by the activity only of the visual sense. The surface need not to be touched for looking        interesting, it suffices that the haptic sense feels called by the visual impression. The bodily self-awareness that feels well or badly treated by the architecture accommodating the body is not informed by the body's inner perception, but by the activity of its far senses.
Our first hypothesis is thus that the aesthetic quality of architecture is a function of intermodal resonance. Resonance as exemplified by the vibration of a string that brings about overtones and undertones in the surrounding wooden encasement, which itself is not mechanically affected. This response depends on the vibrational potential of the given material and shape. In the context of cognitive neuroscience, the paradigm case of resonance is mirror neurons [4]. Mirror neurons make the bodily self-feeling of an observer perceiving a conspecific to resonate with the posture and movements observed. The activity of the visual system triggers the motor program corresponding to the observed behaviour and thus lets the observer know "how it feels" from within to do what the observed body does. The proprioception, that is the point, is not stimulated internally, but reacting to the activity of the far sense. According to classical accounts of perception, the stimulus detected by the visual system has gone astray. It causes the attribution of properties to the object perceived to which no "objective" stimulus corresponds. Intermodal resonance amounts to something like a conspiracy of the senses, a collusion under the table.
The hypothesis that the aesthetic quality of architecture is a function of intermodal resonance thus includes the conjecture that this kind of conspiratorial collusion is quite general a phenomenon, characteristic of the collaboration of the sense modalities as such. This is a strong and highly risking conjecture, to be sure. It connects, however, to a field that since long has enjoyed pertinent research: to synaesthesia. And it is suited to submit an old suspicion, sweeping through aesthetic theory, to empirical test: the suspicion that perception realizes more than reason understands. Indeed, the faculty of realizing the overtones and undertones that aesthetic perception is susceptible to has something mysterious. It is not transparent, let alone obvious, to its owner. It is not even accessible to introspection. If, however, doing what it is supposed to do, its "musicality", understood figuratively, deserves to be called an intelligence inherent in perception [5]. The hypothesis thus reaches up to the question whether there is such a thing as sensual intelligence.

Hypothesis II
The conjecture concerning intermodal resonance can be submitted to empirical methods quite independently of the context of architecture. The methods to be applied depend on purely neurological considerations. Things promise to become more involved, however, when architecture is specifically at issue. In this case, perception should be immediate, i.e. not mediated by representations. Since original architecture cannot be transported into the lab, the machinery for, e.g., functional imaging, would have to be wearable and applicable in a way that does not bias the test person's jaunty experience. Accounting for conditions such as these will make the experiments, if feasible at all, laborious and costly. In order to design a more promising experimental setup, it should do with readily available, which means, with photographic, representations. It is here where the challenge lies on the part of architecture theory.
Resonance phenomena characteristically make appearance at the edges of perceptibility: in the form of subtleties that perception realizes anyhow, but understanding routinely overlooks if not made aware of them explicitly. To my astonishment and disappointment, I haven't succeeded so far to find work on the topic in searching the pertinent literature. The literature, of course, is extensive; it overrides my resources for search. So, I cannot do better than to show in kind that phenomena at the edges of perceptibility 1. do exist and play a role in architecture and 2. are accessible to photographic representation.
One of the most famous constructive problems in architecture giving rise to solutions with subtle aesthetic effects is the corner problem in the grid arrangement of columns and beams. Both the logic of construction and the aesthetic sense for regularity require that 1. the bays are of constant measure and 2. that the joints of the beams born by the columns coincide with the axes of the columns. At the corner of the building, these requirements have the awkward consequence that the architrave, the whole encompassing the beams as segments, ends at the axis and thus reaches to the middle only of the column respectively capital. Since the architrave is bent around the corner, it would cover only a quarter of the capital of the corner column if following this rule. This would be constructively nonsensical and an offence to the bodily feeling of an unconstrained upright position. There are two options to get rid of the calamity: you can either lengthen the beams or shorten the bays at the corner (Figure 11,12) The difference may seem insignificant in the simple juxtaposition of the photomontage and the non-manipulated photograph. The difference will show to be aesthetically relevant, however, when the sequence is switched quickly back and forth (to be shown in the lecture).
What do we see? What does the difference amount to? We see a slight change in the rhythm marked by the succession of the columns. The rhythm doesn't just start or break off at the corners but sets in or comes to an end with an off-beat, though unobtrusively. The step leading to the turn around the corner is slightly shorter than the regular ones.
This solution of the corner problem makes sense as soon as one assumes that the bodily presence of the architecture over there and, in particular, its way of standing speak to the proprioception of the observing body. When perceived as a standing body, the building with the shortened corner bays looks slightly but decisively more compact and muscular (Figure 13). This effect did not go unnoticed by the classicists who revived the old shape grammar for new building purposes. Schinkel's Neue Wache in Berlin is a case in point. The Neue Wache is, to say it bluntly, a police station, a building type that didn't exist in classical antiquity. In order to utilize the time-honoured vocabulary for the recently introduced building type, Schinkel resorts to the classical principle of motivating artistic expression: The harder the rule, the further go the possibilities of semantizing minute differences (Figure 14).
This exactly is what Schinkel makes use of: he returns to the more rigorous and accuracy minded alternative of resolving the corner problem: he returns to the more rigorous option 1. of solving the corner problem: lengthen the beams at the corner, thus giving expression to Prussian rigour and militarian accuracy. The reason for thus emphasizing stiffness is not the celebration of Prussian virtues, however, but the preparation of a daring break of the canonical rule. He replaces the somewhat clumping succession of metopes and triglyphs in the entablature with something totally alien to Greek temple: with "angels", lovely floating figures of winged dancers. He thus breaks the charged rigour with highest grace. A grace that comes to performance only as pointed by the contrast (Figure 15,16).
The "angels", to be sure, are not borrowed from Christian iconography, but are victories, victorious goddesses, as having appeared on Roman triumph arches. The symbolism, however, is of minor importance. What matters is how architecture succeeds in staging an uplifting experience by addressing, via the far sense of vision, the bodily proprioception of the observer (Figure 17).
All this, you may object, is old stuff, speaking to the dying breed only of the humanistically educated. What about modern architecture?
The representative of modern architecture enjoying perhaps the highest esteem is Ludwig Mies van der Rohe. Mies' most famous and paradigmatic building is the headquarters he designed for Seagram Company in New York, 1954-58. The purpose of the building was not just the accommodation of the business administration, but the endeavour of the firm to let its questionable reputation of a liquor factory (having grown big during Prohibition) behind. In order to accomplish this, the then rising star of modern architecture, Mies van der Rohe was commissioned. Money, he was told, doesn't play a primary role. Mies understood well that this was an extreme challenge. There would be no excuses for missing the rank of highest quality.
He would have to deliver an iconic building to the effect that the name Seagram will be associated with architecture instead of just liquor. And it would have to exemplify how a corporation can represent itself architecturally. Mies chose to turn standing into an architectural motive. Make the building symbolise how a corporation wants to stand: straight but relaxed, determined but resting in itself (Figure 18,19).
The appearance is radically abstract, but breath taking in its symbolisation of uprightness. In order to achieve this effect, Mies made use of the then most innovative building technology: prefabricated steel elements. Until then, high rises were built of steel, of course. The metal remained coated, however, hidden behind masonry, i.e. prevented from appearing on the surface. Mies came out with the all metal building. Since longer, he was concerned with developing the architectural language appropriate for the high-performing but very expensive material. The language, he intuited, had to account for the fact building in steel makes economic sense only by exploiting the scale economies of industrial mass production of standardised elements. All special shapes, since excessively expensive, contradict the logic of steel. The logic, he inferred, requires the rigorous minimisation of the number of elements.
This logic, so it seems, is hostile to architecture, witness the socialist panel homes. For Mies, however, the problem consisted not in the mass production of containers, but in the minimisation of the number of Figure 13. The corner contraction is behind the bullish appearance of the building, its both steadfast and relaxed way of standing  different elements that an individual high rise is constructed from. This problem, in the last analysis, confronted him with the classical corner problem. The structure of a modern office building consists, as does that of an ancient temple, in a grid of pillars and beams. In this arrangement you are confronted with the fact that the bays at the corner stop at the axis, i.e. in the middle of the corner pillar.
The options in this case are even more limited. In order to avoid exceptional shapes, both the pillars of the bearing structure and the panels of the façade have to be in the grid. There is no way of shortening the bays or of widening the panels at the corner without violating the minimisation maxim. The only way consists in setting off the façade from the structure, hanging it as a curtain wall in front of the structure and letting the corner itself empty (Figure 20).
It took Mies a dozen or so years to bring himself to this crystal clear but seemingly paradox solution. What seems paradox is a curtain wall that is not bend around the corner, but withdrawn from the structure at the corner, from its most exposed limb. The effect, however, works in very much the same way as does the shortening of the corner bay in the case of the ancient temple (Figure 21,22).
The empty corner adds a frame to the tableau of the façade to the effect that the repetitive succession of the uniform elements does not simply break off. It stops with a novel element even if this is a void. Instead of being marked with a final emphasis, the corner finds itself discretely folded away. It is a gesture that tones down the powerful monumentalism without watering down the self-confident appearance. It is a blink of an eye that self-ironically undermines the expression of arrogant dominance that such a statement of economic power all too easily assumes.
But that is not all. Aesthetically, Mies' empty corner works, as does the corner contraction of the temple, on the edge of perceptibility. Both make use of the technical predicament for addressing subtlety the proprioception of standing on the part of the embodied observer of the architecture. In both cases, that is our second hypothesis, the aesthetic gain of the slightly irregular solution lies in its key to trigger intermodal resonance. The test thus would consist in probing whether test persons who show susceptible to the differences between the Mies was a confessing admirer and pupil of Schinkel. The most famous building of Schinkel is the Old Museum vis-á-vis the once and prospective royal castle. The museum houses the royal antiques collection and was supposed to not only provide a dignified home for the treasure, but to balance also the architectural heavyweight of the castle over there. Schinkel dealt with the challenge in a spirit remarkable close to that of the Neue Wache. He forms the front vis-á-vis the castle as a mighty colonnade exceeding the temple front in the number of columns and in the very extension of the building. He then puts the colonnade on a base that is alien to the temple and, in addition, crowns the entablature in a way that would have been unthinkable in antiquity: with a row of eagles, the militaristic emblem of Prussian kingdom. He thus answers the imperial gesture of the castle with its own symbolism and display of imperial pretention (Figure 23).
Small wonder thus, that it became a paradigm for Hitler's ideas of a temple of art ( Figure 24).
In Schinkel's case, the emphasis of imperial strength and military accuracy was the preparation of a subtle break of the rule. As a support of the heraldic eagles, you should expect columns of the proverbial stiff, male, soldierly Dorian order. The reason, however, that the museum front does not evoke the association of a barracks or ministry of war is that you find columns of the Ionian, since Vitruvius called female order with the soft cushion and the gently overflowing volutes instead of the stiff Dorian conic capital. This unexpected move not only adds a foreign sculptural element, nor is it just an ironic comment on the nevertheless majestic gest of grandeur. It is the small cause that does great effect because it takes away the intimidating character of the monumentalism. It addresses a bodily self-perception precisely different from that of the stout soldier ( Figure 25). This, precisely, did not or wanted not understand Hitler and his architect Trost when replicating the Old Museum in form of the Munich Haus der Kunst. They returned to the Dorian capital or, rather, to a gross coarsening reduction thereof. Whereas the colonnade of the Haus der Kunst makes you hear the hoes crack, Schinkel's monument, though doing its duty to countervail the heavyweight of the Schloss, has the grace to blush.  At this point you may ask whether the architect was concerned at all with considerations such as these. There are no utterances reported that would document that Mies was thinking in terms of overtones and resonances. Even in textbooks and monographs celebrating Mies as both heroic innovator and classic of his own time do so without referring to multimodal perception and intermodal resonance. There is unison emphasis on the logic of construction and the exceptional rigor of this logic in Mies' oeuvre. There is very good reason that Mies very well understood his master's architecture's charm. He couldn't replicate Schinkel's way of addressing the proprioception of the observer, but he very well understood to translate it into the language of abstract architecture. Mies, too, starts with measures to increase the monumental appearance of the towering block. He sets the front back from the street, gives away a good portion of the sinfully expensive plot. He places a forecourt, reminding of the ramp to a manor. This stately gesture, however, is of double use. It increases, on the one hand, the monumental appearance of the towering building, but it does so by offering a generous service to the general public. In the neighbourhood of the Seagram there are no parks or public places where people can spend their lunch break or just a rest in open air. The forecourt is designed exactly for this purpose and treats its users in a beneficial, courteous way with inviting seating benches and refreshing water basins. The courtesy goes even further concerning the beholder of the building itself. The surface presented to the beholder is made of bronze. Bronze is the most expensive of metals suited for cladding buildings. Its distinguishing feature is that it patinates, that it grows old by assuming a mature crust, inviting to being touched. It addresses the haptic sense even if only looked at. It softens the rigor of the steel logic and gives the building a sonorous refined look.