Assignment:
Choose ONE of the videos and write a response on your blog that considers the video in the context of the reading. Some ideas on things you could write about:
Which of the videos did you feel most effectively employs synesthesia? Why?
How different is the experience if you separate the audio from the video (and vice versa). What’s different about the feeling?
Did any of the videos seem particularly out of sync with your impression of the sound? How could the artist have accomplished this better??
Response:
After reading through Katharina Gsoellpointner’s Digital Synesthesia: The Merge of Perceiving and Conceiving, I couldn’t help but think about synesthesia in terms of my master’s thesis, a project I will be conducting over my next (and last) year at ITP. As such, I found myself wondering which of Gsoellpointner’s categories of synesthesia research I would fall into. A Bone Conductive Instrument (working title…), my thesis project has been informed by my desire to create a dialogue between sound, art, and the human body. An auditory machine that produces acoustic intervals to be felt and internalized in a corporeal resonance, BCI aims to evoke each user’s interactive musical discovery by lessening the distance between sound and the body. By investigating sensory substitution in sound wave propagation in conductivity through bone, I hope to contribute to alternative, more accessible methods of “hearing” interactive sound art. In carrying out this project, I aspire to add a tactile dimension to sound that brings insight to deafness, and sensibility to “a holistic, bodily understanding of reality.” On the one hand, BCI is a “prototypical representation of universal concepts of perception” as well as an “object… providing synesthetic experiences for non-synesthetes,” that allows users to tap into “synesthesia as an experience.” From another perspective, however, BCI can be viewed as a metaphor that translates “one sensory modality to another” for the “creation of an individual perceptive reality... from the interoceptive senses… [not] restricted to the five exteroceptive senses.”
Similar to my perception of my project’s relationship to synesthesia, I found that Ryuichi Sakamoto + Daito Manabe "Sensing Streams" to be the most effective method of synesthesia, by using familiar visual imagery to provoke the audience’s recognition of auditory cues. Contrary to other works like Norman McClaren's Synchromy, which approaches synesthesia from a more traditional data visualization approach, Sakamoto + Manabe’s piece uses the video to enhance the audio and vice versa. The work reveals “our abilities to move in the world, to locate ourselves in space and furthermore, even to develop a concept of temporality,” often showing generative visuals indicative of the corresponding audio (i.e.: a horizontal field of green/yellow short pixelated strokes accompanied by sounds similar to those heard of a field, and illuminated dots appearing to fall and dissipate in a matrix, accompanied by rain-like sounds).
