Historical Examples
I posted this link on my research plan post last week. This article discusses the various ways that visual music has been used throughout history. Meaning, visual music has existed before technology has ever been invented. Of course, we see visual music change throughout the years, changing from stand still paintings, to computer visualizations through Aftereffects. As I have argued before, many examples of visual music do not accurately represent individual pitches, but instead represent an overarching theme or melodic line. Cinematic music does help us to paint a picture in our heads of what the music looks like, but audio alone does not aid a Deaf person’s understanding of musical elements and pitches.
https://www.theguardian.com/artanddesign/2006/jun/24/art.art
Current Examples
In this article, Bigham et al discuss the issues that speech activated devices bring forth. In particular, the main issue is the hindrance that deaf and hard of hearing people experience with these new technologies. In smart phones and computers, speech to text or speech activated actions are optional and you can operate the device without them. However, with new devices like the Amazon Echo, it begs the question: How can a deaf person use this? This example is not necessarily catered to my project in particular and does not discuss the music aspect of accessibility. However, it is important to note that certain designers are noticing issues within our tech based world and working towards a more accessible design for members of the deaf community.
Yet still, we have no evidence of technology or innovative ways to aid the deaf in understanding music, but there is evidence in helping the blind! This article mainly discusses the difficulties of reading Braille and how difficult it is for blind people to read music. This innovation relies on a software that “sings” the notes where they’d be presented on a piece of sheet music. While useful and innovative, would a music teacher not be able to do that exact thing? Also, people do not necessarily need to know music theory in order to play an instrument or compose. In fact, at times it is easier to listen to and play music with one’s eyes closed, to eliminate the distraction of sight from the melodic material. Here we see an example of an innovation that aids blind people with music. Where are the innovations that aid deaf people with music?
Here is my starting point for research on “Visual Music for the Deaf.” Surprisingly, this article was not hard to find on the ACM database. All I had to do was search “deaf music.” This article brings up the issue that some visualizers for sound have a delay or lag to them. From prior research, Petry et al have discovered that previous iterations for accessible music for the deaf did not offer a “real time” accurate representation of the sound being represented. This is similar of what I want to accomplish with my project and the interactivity element of it. If the visualizations aren’t exactly linked up with the sounds, then they are not accurate and can’t be used as a reliable source to connect sound to visuals. One quote caught my eye from this article, “Prior work has developed visual and vibrotactile sensory substitution systems that enable access to musical sounds for deaf people [2,4,7].” (Petry et. al 1). A-ha! Here starts my journey. I was struggling to find examples of music innovations made for deaf people and here they are in 3 references at the end of the article. I’ll link these as their own sources below, but I would not have found them if it was not for this article.
Fourney, D.W. and Fels, D.I. Creating access to music through visualization. Science and Technology for Humanity (TIC-STH). 2009, 939–944.
This example stemmed from the previous one about real time music visualizations and tactile responses for deaf users. I will admit that I have not fully read these articles yet, but I have looked through the examples of visualizations from these pages. It seems to me that this article is jam-packed with information about music for deaf people. This is definitely at the top of my list of reliable sources.
While not specifically related to technology, this article talks about music therapy for deaf people and common misconceptions people have about deaf people and music. It also talks about the major struggles Deaf people have in regards to the hearing community and struggling to find peace in a world where hearing people are the majority. Even though I have learned about these misconceptions and assumptions in several ASL courses I have taken at TCNJ and communicating with Deaf people, it is reassuring to have solid evidence that backs up my claims about these remarks. A particular remark made by hearing people is “Why don’t you just get a cochlear implant? I would never want to be deaf! I can’t imagine not being able to hear I’d rather DIE.” Here’s a quote that accurately represents my opinions on that, “One does not need a good quality hearing aid or a cochlear implant to enjoy of music, because most people, even those with severe to profound deafness, have residual hearing” (Sofia P. Quiñones Barfield 6). In addition to that, there are some Deaf people who do not want to hear. There is definitely a problem with mainstream society with understanding Deaf culture and understanding why some Deaf people may not want a cochlear implant or the ability to hear.
This is another article that stemmed from the first article that I found on the ACM database. Not only does it show the importance of visualizing music, but it also discusses the importance of vibrations as well. While my project may not focus on vibrations, I think that it is an interesting field to study. There is definitely more concrete evidence for visual music than tactile music, but I think that tactile music can provide leaps and bounds for the Deaf community when it comes to music.
These last 3 links were ones that I posted in my research plan. They are visualizations for music and not catered towards Deaf or hard of hearing people.
http://www.centerforvisualmusic.org/
https://www.youtube.com/watch?v=obrBAysVef0&t=66s
https://www.wired.com/2011/05/visual-music-paul-prudence/
I plan to also do more research towards color theory and how I can best represent each pitch and instrument with appropriate colors. If anyone has suggestions where I can find those sources, please share them!