"What I'd like to do is to build a composition."
Kurt Schwitters about his MERZBAU' project.
Whereas the recent article 'MetaDeSign'provided a general approach about the emergence of a new artistic discipline according to the technological progress, the essay about 'Sonic space, the shape of sound' intends to go deeper in the understanding of the semantic transformations operated by information processing technologies in the general cultural paradigm and in the production and conception of artefacts. Related to synesthesia the article presents the possibilities of data visualization offered by the digital medium in the domain of sonic, visual and spatial shapes as well as the multiple forms of mapping one medium into another.
***************************************************************************************************
"light, color, rhythm, image, sound; the basis, the data's for a new performance; we can call it electronic play"
Le Corbusier, in "Le poème électronique", ed. les Editions de Minuit, 1958.
In order to illustrate this inter-relation of media, its codes and signs and the construct of space, by an historical yet not digital example one can understand the design of the Philips pavilion for the world exhibition in 1958 "le poème électronique" by Le Corbusier and Iannis Xenakis not only as a result of research according to the acoustic qualities of form but as a contemporary expression extending architectural principles, and mainly the ones of functionalism, to the one of music and cinema. The superposition of the hyperbolic external shape, conceived by Iannis Xenakis with the internal one of the "stomach of a cow" conceived by Le Corbusier constitutes thus more than a contradiction to one of the main architectural principles (inside=outside / form follows function) but a de-construction of its traditional language. On one hand the application of new static construction principles based on hyperbolic mathematical functions on which Iannis Xenakis also based his composition "Concret-ph" played inside the building, can be seen as a rational approach to relate space, music and construction through science.
But on the other hand the internal shape of the pavilion create an organic space of "flows", a "cavern" covered on its internal side with image projections displaying the history of man and technology, which can be seen as a cultural approach acting on the semantic level of form.
The pavilion thus constitutes more than a futuristic odyssey but a manner to express the interrelation of technology and aesthetics in the construct of sign forms.
The investigation on " the shape of sound " thus can be understood as a research on the way we conceive and perceive the correlation between space and sound in coherence with the specificity of a medium, its codes, structures and processes. In relation to the Xenakis/Le Corbusier example, as the way we relate space to form and sound in both a parametric/rational level as on a cultural/semantic level determined by the technological state.
***************************************************************************************************
"Two: everything around us can be represented and understood through numbers Three: if you grab the numbers of any system, patterns emerge"
Quoted from the movie "Pi", by Darren Aronofsky ( Pi website )
One of the specificities of the digital medium is the reduction of all information to a binary signal, be it a picture, a text, a space or a sound - all data is recorded as a binary sequence allowing computation as defined by programming languages and communication through networks according to transmission protocols. Therefore it is the medium, which, through the logics of information processes unifies information as much on the structural as on the semantic level. Speaking of hypermedia is thus equivalent to place this programmatic interrelation inside a spatial and temporal structure between data and in consequence between the different types of textual, visual, sonic... formalizations - information. In this view structuring data is not only about making interconnections based on hyperlinks, but also about the mapping of data from one media into another.
For example: the frequency of a sound can be transcribed in a coordinate of space or a colour allowing sine-waves to be seen. The materials of sound, modelled as sine-waves, described by numbers, are seen. A composition can be expressed as a relationship between the notes, pitch and loudness changing over time. At any point in time the notes can be reduced to a sum of sine-waves, and so be described by numbers. These numbers can be assigned colours or space coordinates. When the colours or topographies are displayed music is made visible, spatial. Now, through technology, a musical moment can be captured, described in numbers and rendered in colour and space. Visual music is created. Listen with your eyes.
This kind of information mapping leads to new possibilities in the transformation and combination of data building up information, in relation to its representation as well as in its essence, by modifying the mapping (instruction/construction).This interrelation between different media nevertheless must be described through programming (structured language). In consequence structuring data is about the visualization of data, its qualification and quantification, a structural and formal shaping. The digital media based on programming - code/language thus is completely different from other media which are based mainly on metaphors or analogies in their interrelation. Furthermore this programmatic level imports its own methods, processes and codes in the representation process and thus acts as much on the structural as on the semantic/cultural level.
In the general context of production and conception of artefacts we now have to focus on the setting of new codes proper to the processes and structures of IC technologies, a re-articulation of the entire system of signs according to our technological state.
"Data visualisation is engaged in a similar reduction as it allows us to see patterns and structures behind the vast and seemingly random data sets. Thus it is possible to think of data visualisation as a new abstraction moving from the concrete to the abstract, and then again to the concrete."
Lev Manovich: "The Anti-Sublime Ideal in Data Art"
http://www.manovich.net
In order to give comprehension of data mapping being more than a simple assignment of numbers but the one of cultural inscription and setting of a system of signs, some synesthesic works may introduce to which level information design is and will be involved in the general construct of sign-forms.
***************************************************************************************************
During the Bauhaus period Wassily Kandinsky applied to his painting music composition principles and more precisely the ones of Schönberg, leading to the development of his synesthesia theory, as formulated in "On the Spiritual in Art". He described synesthesia as a phenomenon of transposition of experience from one sense modality to another. More than a transposition of musical and colour harmonies by the means of frequencies of light waves and sound waves, he investigated two essential problems: the one of dissonance and temporality. His synesthesic approach thus cannot be understood in simple correspondence schemes as the tone-colour relations but more on the level of a composition based on visual and sonic patterns in "collaboration and opposition" in order to reveal tension and emotion inside the inter-play of colours, forms and shapes.
***************************************************************************************************
In a similar way Piet Mondrian studied the problem of the visual perception of movement in his apparent static compositions. By the use of the perceptual effect of a rhythmically moving raster he wanted to add to the two dimensional painting not only the third dimension of depth but also the fourth dimension of time, as visual suggestions of movement. The synesthesia of movement in visual raster remained a central theme in Mondrian's experiments, culminating in the “Boogie Woogie” painting, showing the city of New York seen from a skyscraper while listening to jazz music. In the painting coloured areas correspond to sounds with definite pitch, whereas areas without colour correspond to sounds without any definite pitch (which Mondrian refers to as noise). The primary colours have their analogues in the pitches of the standard scale, recalling Isaac Newton's analysis of the spectrum in terms of the seven tones of the Western scale. Other relations in Mondrian's scheme are the visual notion of size, corresponding to dynamics or acoustic amplitude, and position in space, corresponding to position in time recalling the flow of cars through a maze of city streets or the complex interplay of rhythms in a jazz improvisation. Combining time-sequenced aspect of music with the visual aspect of painting Mondrian explored the perceptive and cognitive modalities of the pictorial space, of architecture and music as a general approach of sign forms. But rather than the construct of a universal language one can retain the setting of a system of assignments (signifiers) according to the cultural value of media.
***************************************************************************************************
The Tokyo national opera competition entry by Bernard Tschumi may give an understanding how far the setting of a system of correlation can be related to the construct of a system of expression (language) and thought in the general field of design. The project is planned on the transcription of music notation principles, the partition, to the spatial organisation of the building, the 5 independents "programmatic strips". While confronting the spatial organisation of functions to the temporal structure of music, Bernard Tschumi sets up an open structure ( "historical not codified articulation of functions") in order to conceive the building like a tonality, a composition extending the architectural spatial organisation to musical concepts like tempo, rhythm and melody. More than a formal play the semantic approach of the juxtaposition of cultural significations is based on the dissociation of the code and the message in order to generate "la difference".
The different examples, all drawn from non digital approaches, show until which level the setting of correlations between sonic and spatial principles within their modalities of perception and cognition are related to the cultural context and to the setting up of a system of signifiers.
***************************************************************************************************
Now information processes allows spreading these types of works, opening them to new forms of expression, like these of hypermedia. In the multitude of possible combinations, the key issue becomes the setting of a system of assignment according to the technicity of the media as on its cultural inscription. In this manner the structuring of information and the transcription of information processes in textual, visual, sonic or spatial forms thus can be defined as a relation of sense (perception) to sense (cognition) in the setting of sign forms.
" Aesthetics begins as patterns of recognition "
Friedrich Kittler in Literature, media, information systems 1997 p.130
Yet speaking of electronic space is speaking of a less physical and more mental and cognitive experience of space, the stimulation of our neural system and the constitution of behaviours. It is a new definition of space, becoming more and more independent from its direct material inscription, but increasingly bound to transmission and computation parameters (substituting parameters as gravity distance...). Constructing space therefore generates a system based on the programming of actions, behaviours and communications between the space, the objects and the user. Conceiving electronic space is therefore working on these spatial - temporal structures in relation to the cognitive and mental processes and the way they influence our actions through perception/cognition.
It was the invention of stereophony at the end of the sixties that made the experience of space through sound convincing, what we already could see in the beginning of the seventies is that "virtual reality" could be easily created - at least in an acoustic manner. Even the tactile qualities of recorded sound, and in particular its spatial properties, were so "real" that it was indeed possible to simulate spaces, bodies and movements by only the use of the hearing.
Related to the perceptive and cognitive modalities of electronic space, the exploration of 'space-music' has its origins in the experimental work of the fifties, where musicians like Varèse, Stockhausen, Xenakis, artists like Nam Jun Paik and architects like Nicolas Schöffer not only have explored audio visualization and spatialization systems in order to relate the logics of space to the logics of imaging and music but also to link it to computation technologies, synthesized and programmed music.
"Sonic space" thus can be defined as the study of systemic relations between humans and acoustic environments formed through the conscious and subliminal perceptions of the listener. In this manner the analysis of sonic space is based on the cognitive and perceptual attributes such as: front, rear, contour, rhythm, silence, density, space and volume, which are derived of analytical concepts such as: note, sonic event, sonic object, sonic signals... In consequence the setting of sonic space is not only an organization of sound in time, but also the constitution of a matrix of acoustic, visual, behavioural patterns within their spatial expression.
***************************************************************************************************
The chosen examples present only some aspects within the broad range of possibilities relating the research of sonic space to art, architecture and technology. They explore music, as a social practice intrinsically bound to time and space, a field of experimentation determining the semiotics of the digital matrix according to aesthetics.
1. audiovizualizers
http://www.audiovisualizers.com
http://www.audiovisualizers.com/toolshak/vsynths.htm
Despite its graphic design the web site is a reference of documentation dealing with visualization systems developed since the 60th, including the works of Xenakis at the IRCAM, or the fractal mathematics in its multiple derivations by Mandelbrot. It constitutes a major starting point and a good resource within the huge domain of visualization and spatialisation systems of sound.
Similar sites:
http://vhost2.zfx.com/comm/scanimate/tajchman1.html
http://www.deas.harvard.edu/~jones/cscie129/images/snd_vis/snd_vis.html
2. The Shape of Song
http://www.turbulence.org/Works/song
In tradition to the audio visualizer the online interface "the shape of song", by Martin Wattenberger (author of the famous "the map of the market") examines the relation between sonic and visual structures. The custom software in this work draws musical patterns in the form of translucent arches. Each arch connects two repeated, identical passages of a composition allowing viewers to see, literally, the shape of any composition through time. The site presents musical examples, from the deep structure of Beethoven to the crystalline beauty of Philip Glass but also offers each visitor to introduce any available midi file from the Web in order to follow the process and analyse its visualization himself.
3.The vOICe
http://ourworld.compuserve.com/homepages/Peter_Meijer/javoice.htm
Seeing with sound: seeing with your ears! This is the slogan of the voice web-site which proposes each visitor to draw his own 64 x 64, 16 grey-tone image and immediately hear the corresponding 64-voice polyphonic soundscape. The different works presented on the site by Peter Meijer, researcher at the Philips Eindhoven lab, propose multiple applications, all based on this interface from sonifying existing images, training for audiovisual synesthesia, performing on-line composing, making soundscape animations and creating spectrograms to the development of interfaces for blinds.
4. the bot
http://plagiarist.org > the bot
A bot is a computer program designed to act as a robot, gathering a variety of information from the web. Search engines, such as Yahoo and Altavista, typically use bots to gather information for their databases. A common type of bot is a spider. A spider follows links between web pages, roaming the web, crawling from document to document. theBot is a spider. When fed with a search term by a visitor, theBot performs a search for the word or phrase in a search engine. From there, it crawls to linked pages, and continues reading scraps of text from each page it visits as it goes from pages to pages. In this manner the project provides a look / listening of the networked space following its inherent logics by transcribing the meta-information into the narration of an electronic voice.
5. Space in Veda
Exonemo http://www.exonemo.com
"Space in Veda" is an on-line project by the Japanese internet group 'Exonemo', developed in the context of the exhibition 'mind the banner' and organized by the on-line magazine "shift". The project is based on the standard banner window by taking its specific position on the screen to generate a sound by assigning the xy value of the window to the amplitude value of the sinus sound wave. The change of a position of one of those windows, even by the rendered animation or by the interaction of the visitor leads to sonic composition. This "window composition" provides a good understanding of the screen-space as a sonic matrix.
6. Electrica
http://electrica.leonid.de/cgi-bin/index.cgi
Electrica is a German experimental lab in sonic interactivity on the web. The presented project explores an 'audioactive' interface as homage to the sounds of electricity in form of a sonic journey through the industrial vibrating atmosphere of closed-down transformer station in Jena constructed by a Bauhaus architect in the 30's in the east of Germany. The project is more than a graphical and sonic illustration of architecture; it explores through interactive graphics (visual code) the programmatic logics of sounds.
7. the central city
http://www.stanza.co.uk/central city
The 'central city' is a sonic environment conceived by the English artist 'Stanza' presenting an interactive experience, a collage of generative sound settings, multi-soundtracks and a graphic urban universe. Despite its metaphoric language the different experiences are all based on the users navigation and interaction generating the sound and graphic space of the screen.
8. sPACE, navigable music
The conception of the 'space navigable music' on-line project, conceived by the laboratory for architecture and urbanism LAb[au], is entirely based on the programmatic relations and the principles of data recombination in order to set up an interface intertwining sound, colour, text and space. The principle of the project is the computation of data related to the users interaction, navigation within the electronic space, where the xyz-space coordinates and t=movement parameters are used to sonic, visual and spatial instructions in order to form soundscapes, a navigable music. In this manner each user mixes in real time through its navigation in the electronic space sounds, colours and images, an interaction, which at the moment of its editing provides an animated sonic architecture fusing sound, image and space in one single media.
9.stagging strategies
http://www.p5-berlin.de/stagingStrategiesE/index.htm
In which manner users will interact in electronic space, which aren't mimetic? Starting with this question the work "stagging strategies" by the German group programm 5 examines inside electronic space the visualization of processes in order to set up necessary sensual experiences in its construction. According to this objective the different spaces propose to the users multiple interactions within the environments and the objects it constitutes, relating form space and sound in a communicational and/or computational matrix.
10.servovalve
The web site of the French multi-media artist Servovalve proposes a series of experiences relating user behaviour, mouse movements, to sound and random graphics. The different sound and image generators thus react on movement extending the XY (mouse position value) to sound frequency assignments to time and behaviour ones. The minimalist graphic lay-out of the different experience and the reduction to simple user interaction creates an immersion into a playful space of sounds and graphics.
11. ki2D/#
http://infinitoarte.com/santiago/ki2D/#
ki2D is an application for musical creation based on a 2D space allowing to modulate sine waves while moving objects on the two dimensional matrix. The positioning (x, y values) of the icons are related to the frequency value of the sound wave where the /- value is assigned to the left/right stereo effect. The application is an introduction into the correlation which can be set up between electronic space, the 2D surface of the screen, and musical behaviours through the assignment of values.
12. lim-0 on_keydown
Limiteazero is an observation and experimentation laboratory of digital communication live-forms, in visual, sound and all other expressive forms. Used tools are forms, signs, noises, words and code. Limiteazero's experimental activity is about the exploration of alternative relations between man and machine. It tries to establish an emotional approach using interface systems, instead of a logical/deductive one. The work around form is mainly focused on investigating forms and images generated by mathematical processing and through random evolution processes. 'The "random" sequence is our most used coding expression. Our interest is not mainly centred on the final product but rather on the process itself. Limit, mathematical concept based on the idea of proximity, basically used to give values to certain functions in points where no value is defined, so that they can be coherent with the nearest ones.'
***************************************************************************************************
Metalab02 History Navigator (requires the Adobe Flash Player 8+).
