For my prototype this week, I decided to work heavily on the Home Screen. This includes a title, logo, and sign up bar. I am currently working with a friend on developing a formal logo/icon to represent the app. Besides that, I will attach what I have been creating for the Home screen.
I have also been trying to get the About Page figured out– I need to have the perfect copy for this so that people will understand what the app is for, and how to use it as well as how not to use it (aka making it a safe space)
For my elevator pitch: My app is designed to connect music-lovers with local musicians/bands in their area. (how do i make this sound like it’s not a dating app???) This app will be organized by state and will allow users to find information on the bands they like including their events, socials, merchandise, and live streams. Additionally, they will be able to interact with other users/musicians in forums depending on their questions/topics of interest.
-Since I am abroad, I had no one to receive feedback from regarding my last prototype. I asked a few of my coworkers who had a few questions in response:
-how the thesaurus will work- will there be a limit of bands?
-how will the sign up work for bands? specific page and questions asked.
-what if multiple people are live streaming the same event?
-how will you implement your map/location based services when building.
These are a few questions that I will be answering as I continue to prototype and eventually build my app.
Project Objective: An action game where the player moves along three horizontal parallel lines to limit movement so that I can quicken the speed of combat game play.
RESEARCH
Pixel Art and Coding
For my research I continue to review YouTube and Lynda.com. Both of these sites have provided me with great information. On Lynda.com I am able to focus and research on specific topics as it relates to Unity. I have found this to be very useful. Some of the topics I’ve been researching include loops, general programming habits, and creating classes. Another aspect of Lynda.com which has assisted me in my research is their asset store which I’ve learned about Cinemachine.
Cinemachine is an application for 2D professional camera tools for Unity that provide access to camera control at AAA game level. By using Cinemachine it should assist with some issues I’ve been experiencing as well as save some time. I am just starting to explore this and to understand how it could enhance and improve my final project.
My exploration and research on YouTube continues and I have discovered a number of channels which are dedicated to Pixel art. One channel which I have found to provide some simple direction is Brackeys (http://patreon.com/brakceys) . Through Brackeys there are coding tutorials, tips and tricks.
I have also researched channels dedicated to the drawing of anatomy. A particularly good one is ProkoTV on YouTube. This is led by Stan Prokopenko who is an artist and teacher and has created a number of instructional art videos.
A lot of what I’ve learned is how the animation design is essential to convey the essence of your characters.
Similar to Brackeys I have also spent time researching on MortMort which is also on YouTube but is also available on mortmort.net. This person also provides numerous tips and special ways to code line which assists in breaking down the Pixel art for line development. MortMort further shares how he goes about his game development using Asperite for Pixel Art, Photoshop for digital painting, Stencyl for simple 2D games and Unity for more complicated games. I found all of this interesting and additional topics for further research.
As I continued to research further into Pixel art I found a YouTube channel called “The Verge” which has a great documentary web series on Japanese designers. This is a Toco Toco TV and is an ongoing web series. The episodes are in Japanese but you can turn on subtitles. Each of the episodes are interesting and I find some more useful than others but in general a good resource for which is needed in game design. A few of the interviews I’ve watched include Yoko Taro, director of Nier: Automata which touches on his creative process as well as why his games feel a bit different than others or as I like to say “goofy”. Another is the director of Persona 5, Katsura Hashino which touches on creating a game in the city of Tokyo and being real-world focused. Toco Toco is now Archipel as they have evolved this channel but the interviews are still known as Toco Toco.
Project Objective: An action game where the player moves along three horizontal parallel lines to limit movement so that I can quicken the speed of combat game play.
RESEARCH
Aesthetic Visual
My ongoing research has moved to the aesthetic of the game. What will the world my characters inhabit as this will help me to focus on what my characters stories and actions. What I learned is I will need to determine what actions I will be developing and then fit my aesthetic around the function.
Based on this knowledge my game will be a city setting with very linear movement which is better aligned to narrow streets, alleys and tunnels. This city setting will allow for linear game development more closely.
As I progress through this project I can determine how realistic I want this to be or if I want it to take a more goofy turn. For example when researching Mario games, you will note when you jump on an enemy it will stop and be a static object. They way Mario overcame this to make it more aesthetic so the enemy became a turtle, so when the hero jumped onto the enemy which is the turtle, it retracted into its shell.
Other games I researched that have this street, alley and city type fighting are from CapCom’s Fatal Fury and Street Fighter.
I also looked to the movies for inspiration and the movies which I felt most closely resemble what I would like to develop are the Rocky movies and Big Trouble in Little China. All of them exhibit city level contained field fighting which is what I will require to have a pleasing aesthetic for my game. Additionally they both had fantasy elements which impacted individuals going about their day-to-day lives.
Platinum Studios was founded in 2006 and they currently have 209 employees. The head office is located in Osaka, Japan. Over the last few years, they have developed NieR:Automata, Teenage Mutant Ninja Turtles: Mutants in Manhattan, Bayonetta, and Star Fox Zero and Star Fox Guard.
Platinum excels at their development of battle/combat games. They actually co-produced NieR:Automata because the other producer did not have the experience or the desire to create the required battle/combat system.
Platinum games method for development of a game is to first develop how the character is to play. In addition, the action the players can take in game design is referred to as “verbs”. It is important not to have too many verbs so the controls do not become cluttered. Verbs serve multiple functions such as flushing out the development/skills of a character. Verbs also lead to better game design.
An example of a verb in Platinum Games that highlights the above was in one of their first projects, Bayonetta. In Bayonetta, they used a well-timed dodge to slow down time. This verb was used both in combat and in the puzzles between sections. This gave the character the assumed knowledge of how to combat but also gave the game player the ability to elude or fight without actually throwing a punch.
For me I am taking Montgomery Harris’ rule of six and distilling my verbs down to six actions. I have not finalized this list as I continue to compile the list of verbs before finalizing.
I have also researched Hideaki Itsuno. He is the mind behind the later “Devil May Cry” series, which set the bar, which all other similar games are compared. Hideaki is a Japanese video game director and video game designer employed by Capcom for most of his career. He is credited as the director of Power Stone, Devil May Cry and Dragon’s Dogma series’. Although he did not direct/design the original Devil May Cry game, he did become involved with Devil May Cry 2 to steer the project back on course. He was then involved starting on Day 1 with Devil May Cry 3 and 4.
The more I considered what I wanted to achieve I realized my greatest inspiration is the game Dead Cells – simple 2D game with approximately 5 verbs relating to combat. Motion Twin developed this game, which is a French worker cooperative based in Bordeaux, France. They have been making games since 2001 and are currently working on their first PC/Console title. Dead Cell has three consistent verbs of run, dodge and jump. Two of the actions the player can take, and the game is set up so you randomly get two weapons and two skills. Thus, all those randomized weapons can fit into the verb attack and all the skills can fit into the verb skill. Because of the player is always on the feet with hundreds of options to approach a situation. Because the players’ movement is consistent, it give the players the confidence to take risks and progress through the game. This is something I appreciate and am looking to incorporate into my own game.
As discussed, my desire is to build my game utilizing Unity. Therefore, my research has been focused on the coding in Unity. This is progressing however I have yet to get the enemies working but the player can attack and run around from left to right.
To build upon this I have also visited Linda.com which provides video trainings to continue learning and understanding the basics of Unity. I am using this to better translate the movement I want to achieve with my characters. Although I was able to increase the speed of the characters, it was negatively affecting other facets of my game design so I am further researching how to address these issues.
Game Design – How To
One great resource I have found is the YouTube channel “Extra Credits”. I have viewed their whole series on making your first game and I continue to research this information. Extra Credits states your first game should be small in scope and when putting your game plan together, break it into smaller achievable chunks. They recommend having small weekly objectives to maintain a steady progressive pace to achieve them.
This week, I’ve been looking more into what programs I’ll need to use for my thesis. As I mentioned in my last post, I’m thinking of using Unity and possibly a Leap Controller device. In this week’s post, I’ll go a bit more in depth with my plan on learning these programs.
To learn more on Unity, I’m going to invest some time into watching Lynda and YouTube tutorials. Here are a couple ones from YouTube that should be helpful:
https://www.youtube.com/watch?v=Au8oX5pu5u4
https://www.youtube.com/watch?v=qX8kabgz3wo
I also plan on talking more to my alumni advisor Andy as well as Prof. Josh Fishburn and my good friend Robin who are all more familiar with Unity than I am.
Josh teaches Games I and Games II in the IMM department. Occasionally, I will sit in on his Games I class with Robin to listen to the lecture and watch Robin build his game. So, this gives me the perfect opportunity to talk to Josh and Robin about using Unity in my project.
Then, through email, I’ll continue to communicate with Andy and ask him for some helpful tricks and techniques. Maybe at some point he’ll be able to come to TCNJ to help me and other students who have been assigned to him.
As for learning how to use a Leap Motion Controller, that’s another thing I will ask Andy about as well as Dr. Nakra (who I believe has worked with Leap Controllers), and look up directions and tutorials on. I’ll also need to look into how I would be able to access a leap controller for my project.
Here are some videos I found relating to connecting a Leap Controller with Unity:
https://www.youtube.com/watch?v=8KeZxfPh8TE
https://www.youtube.com/watch?v=Wh3YK2OcI4M
https://www.youtube.com/watch?v=Pdnik0gA09g
https://www.youtube.com/watch?v=gMHtBae897I
https://www.youtube.com/watch?v=X__oClQTob0
As for right now, though, I am currently in the prototyping phase of the project and need to really consider how I should begin to execute the project itself.
I’ve talked with some of my coworkers about my project, since they’re all IMM alum to get their feedback, as well as my alum adviser. I’ve also chatted with friends and classmates just to hear their thoughts.
However, the focus on this week’s research is more so on preparation for building actual prototypes. I already have a good stash of tutorial resources in building the actual hologram prototype. Now I’m compiling a list of materials I need:
iPads (possibly Android tablets) x5?
Headphones
Maya
Materials to make environment (paper mache? 3D printer? Cardboard?)
Plexiglass or some other transparent material
CD cases for prototype
Possible arduino or raspberry pi
Motion sensors
Mini LED lights (possibly triggered by motion sensor)
This week was all about learning how I would incorporate Visual Effects (vfx) into my game. Going into this project I knew I wanted to incorporate a variety of vfx (smoke, explosions, starbursts, etc.) to make the potion crafting feel visually rewarding. As the player created increasingly difficult potions the vfx would increase in scale to reflect their difficulty.
I found several forum posts from people working in the industry and a GDC conference about their approach/the unique development of stylized vfx. Since my project has a more cartoonish aesthetic I want the vfx to reflect this.
This Game Developer’s Conference on the creation of vfx for the game Gigantic was very insightful, but unfortunately the gifs in the slides don’t play so I can’t visualize the step by step of how they created some of their vfx.
Finally, a game developer named Etienne POV created a blog post detailing how they created their stylized vfx. The vfx they break down was one that they entered into a competition Riot Games and won. This was an incredibly helpful resource because this is almost the exact style I’m looking to recreate within my own project.
For my third piece of research I wanted to get into the minds of the professionals. After gaining knowledge on the technical sides of filmmaking I wanting to gain a better understanding of the intangibles of the those who make films. One the most prominent film communities currently is the “No Film School” community. They provide a web page, social media, and youtube clips of resources for young, up and coming, and even experienced filmmakers. I recently came across this youtube video that is added below and I was able to get in the mind and creative thinking of directors from the Sundance film festival.
There were many interesting ideas and thoughts that were shared from these directors. What I enjoyed most about this film was how they gathered a group of people who bring different stories, perspectives and worldviews together for this video. One director spoke on tapping into what you know. Many young directors and filmmakers often try and overthink their stories and lose authenticy from their films because they are trying to create a world that they know nothing about. I learned to stay true to myself and stay within the things that I know. I hope to use my own experiences and thoughts to create a real and authentic film, regardless of the genres that will take place; I hope that the style and story stays true to who I am as a filmmaker.
Another idea that stood out to me is when one filmmaker said you’ll never know until you try. This concept is often being used and can easily be overlooked despite the importance that it holds. I found myself many times doubting the possibilities of my ideas and then water down my creativity because I feel as though I wasn’t able to create what I am envisioning. There will never be a perfect time or scenario so it’s best to just try and don’t fear failing because even if you fail you be able to learn from it. Growth cannot be achieved unless failure was present beforehand. This is a motto that I live by and will carry into my thesis project.
The main idea that was being talked about was the idea of staying true to who you are. Another director shared with the group that if they were all to create a film about Hurricane Katrina they will all create totally different films despite it being the same topic. The reason this is comes the fact that we all have different perspectives and journeys so we all see the world differently. What makes a great filmmaker is the ability to show the audience how you see the world from your own eyes. When a person is able to create with authenticity then it will be apparent to the viewers and in turn will make the viewers more attached and invested into your film.
The theme of this week’s research was technology. Last week I briefly mentioned becoming familiar with the Leap Motion Camera, however this week I took the opportunity to investigate further and see how the practical implementations would work.
Through reading many articles and watching quite a few demo and tutorial videos with the Leap Motion camera, it could best be described as a Kinect, only meant for PC usage and more accurate in every regard when compared to the Kinect. Essentially, it tracks hand movements and reacts to them in the applied space. There were quite a few examples of this implementation, for instance one video showed a man writing with his hands in the air and the words being reflected on the computer screen, while another showcased creating and playing with cubes in unity. Speaking of Unity, in order to create the physical model that I envision (as of now) for my project I will need to utilize Unity, and as such did some research into that platform in the form of introduction/tutorial videos to refresh my memory.
Now, while I focused most of my technological research on getting familiar with Leap Motion and Unity, I also reached out to my student advisor to see what she thought would be good for implementation and if there are other paths I’m not considering at the moment. She in turn directed me to two different areas: google cardboard and interactive displays. Now while I didn’t have quite enough time to continue researching these at the time of this blog entry, I do plan to review these this weekend and come to an understanding of the technology.
“Poems and paintings became music, and music became poems and paintings.”
While visual music is becoming more popular with the technology available today, there are many historical examples regarding visual music as well. While synesthesia is a natural human phenomena, the implementation for sound and sight linking dates back to the 19th and 20th century (The Guardian). Wassily Kandinsky, a Russian composer and visual artist, captures the beauty of music through paintings and other forms of art. He wasn’t alone in his ideas either. Many other artists implemented similar ideas into their own work, linking sound and music with visuals and paintings of the Romantic era. From these original ideas and inspirations, these artists inspired modern lighting and color for many projects available today. For example, concerts across all genres have many lights that fit the theme of the show and each song. A slow, acoustic song has dim, soft lights, whereas a powerful, high energy song has bright, fast changing lights. In a way, modern concerts are visual music. With the empowering instrumentation and bright lights, concert goers experience music in a way that isn’t tangible at home.
Before the technology for modern lighting was available, Scriabin composed his work “Prometheus: Poem of Fire 1909-10,” which was intended to involve a color keyboard, lighting up the entire concert hall with color. Scriabin had synesthesia, where he would perceive certain keys as different colors as opposed to individual notes being a different color. In Yale’s video on YouTube, they show the implementation of the color organ and how it is performed in concert.
Yet another example of live visual music is “The Firebird” and “The Rite of Spring,” which Igor Stravinsky composed for Sergei Diaghilev’s Ballets Russes company. In these pieces, Stravinsky intended for multiple forms of art to be present during the performance. Painters, dancers, and musicians combine their art forms to perform a giant art masterpiece for an interactive and immersive experience for the audience.
Current Context
Visual Music
In this video, Nahre Sol breaks down how she interprets complex chords. These chords aren’t typically seen in pop music and are usually pretty challenging to analyze and play. To give an idea of normal chords, there are two types to start off with: triads and 7th chords. A triad consists of three notes involving the root, 3rd, and 5th of the key. The root meaning the name of the chord (In a C major chord, C is the root), the 3rd being 3 notes above the root (In a C major chord, E is the 3rd), and the 5th being 5 notes above the root (In a C major chord, G is the 5th). In a 7th chord, we build on the original triad and add a 7th above the root. In a root position C major chord, a pianist will play C, E, G, and B, in that order. Of course, there are all sorts of inversions and qualities of chords to consider when referring to triads and 7th chords.
In this video, Nahre shows more complex chords, involving more notes than a standard triad or 7th chord. She shows this interpretation with color. First she plays the chord with a bowl of mixed colors seen at the top of the screen. Then she reveals the two colors used to compose the chord. In one example, she represents an A minor 6/4 chord with the color blue, and an A major chord with the color red. Finally, she plays the chord again showing the mixed colors and the musical notes laid out on sheet music, with their respective colors. This form of visualization can help beginner musicians understand complex chords and how to build them. By thinking of the chords in colors, rather than notes on a piece of sheet music, people may be able to easily understand music theory.
With the technology that is available today, it is no wonder that visual music has become so popular. In this article, Alice Vincent shows an example of visual music by Paul Prudence and how he creates these visuals. All of the visuals are created with an algorithm based on math and geometry (Wired). The visuals themselves look stunning when paired with the music. He intends for his visual music to be experienced live due to the originality of each piece and visual. As an art piece, this form of music stands as a credible source for visual art made with current technology.
In my original idea for the thesis project, I was extremely intrigued by chromesthesia, the association between sound and sight. In this video, we see a young woman playing violin with bright colors popping up on screen to represent the notes while she gives commentary. The colors blend into the cityscape and environment as if they were a natural phenomena, and not inside one person’s head. After wondering, “what does synesthesia look like,” we get this video where someone with synesthesia can accurately represent what they see through the technology.
Based on this knowledge of synesthesia, I realized that my project would be something different. I didn’t want it to look like this video. The video is beautiful and creates a wonderful understanding for what the condition looks like, but it is not clear what each specific note is. Even after spending years training my ear, I cannot identify the notes or intervals she is playing even when looking at the visuals. After reading some comments on the video, people with similar conditions only see certain colors for particular songs, or a whole orchestra may appear one whole color as opposed to multiple vibrant colors. In my head, I envisioned synesthesia to involve a rainbow of colors ranging in shape, tint, hue, saturation, etc. After seeing this video and reading many stories, I decided to leave the idea of synesthesia as an inspiration, and not the sole focus of my project.
Deafness and Music
“Sound is so powerful that it can either disempower me and my artwork or it can empower me. I chose to be empowered.”
In this enchanting TED talk, Christine Sun Kim discusses the similarities of music and sign language and the importance of social currency in the deaf community. She gives the audience some background and history on American Sign Language (ASL) and its importance today. Her entire life, she was always taught to consider sound as something separate from her. Sound was something that she would never be able to experience and she would be distanced from it. As her life went on, she realized that ASL and sound are not as different as the average person might think. She even acknowledges that she spends more time paying attention to sound etiquette and mirroring hearing people with their sounds. At one point in the talk she says, “In deaf culture, movement is equivalent to sound.” This makes sense because all signs in ASL have movement to them, therefore making “sound with them.” What’s interesting to me is that music also has movement to it, both physically and metaphorically. As a violinist moves the bow up and down, we can gauge how intensely or passionate the piece sounds. Even if we covered our ears, we can still get a general idea of the tone and texture of the piece without ever hearing it. In addition to the physical movement of the performer, music and sound has its own movement to it. The phrasing of each piece contains a special movement that changes based on who is playing the piece. A performer can make a melodic line sound melancholy, whereas another performer can make that same line sound lively and energetic.
She also mentions the similarities between music and ASL, which are surprisingly similar. ASL is visual, so we can typically see drawings or paintings relevant to the language, but we hardly see or hear any similarities between music and visual languages. One great similarity is how music and ASL cannot be fully expressed on paper. (TED talk). Christine goes over the different parameters of sign language, including body movement, facial expressions, speed, hand shape etc. All of these parameters cannot be expressed on a piece of paper like english can. “English is a linear language,” and doesn’t have as many physical parameters to consider. Music is similar to sign language in the sense that it is not a linear language. Sheet music exists, but it does not fully capture the meaning and essence of the tone and quality of the piece. She uses a piano metaphor to explain the similarities, where english is a single note and ASL is a chord constructed of the many parameters to interpret the language. Much like music, if one were to change any part of that chord, the whole meaning would change.
My idea for this project was to drive home the point that deaf people can experience music. They can experience it in a number of ways involving visual and tactile sensations. “You don’t have to be deaf to learn ASL, and you don’t have to be hearing to learn music.” Deaf people do have a voice, they just don’t use their physical one. By recognizing the similarities between sound and ASL, both hearing and deaf people can come together to bridge the gap between the two communities.
On How Deaf People Might Use Speech to Control Devices (Jeffrey P. Bigham, Raja Kushalnagar, Ting Hao Kenneth Huang, Juan Pablo Flores, Saiph Savage)
Accessible music
“If I were here playing cello, or playing on a synth, or sharing my music with you, I’d be able to show things about myself that I can’t tell you in words.”
In this inspiring and touching TED talk, Tod Machover showcases his many projects involving music technology and accessibility. He makes the point that music is much more enjoyable when you can create it yourself (TED talk). In my project, I hope to achieve part of this goal by giving deaf people a means to learn the fundamental skills for music without needing sound. We see part of the wonders with Tod’s innovation, Hyperscore, a program that allows users to coordinate lines and colors to create music. By the end of the video, Tod brings in a man named Dan, who has cerebral palsy. Even with his physical disability, everyone can see that through the power of the infrared camera, Hyperscore, and sensors, Dan was able to express himself in ways that words cannot define. This is truly an inspiring piece of work and provides a foundation for what accessible technology should aspire to be.
If technology is accessible, that means that everyone, no matter their disability, can use it. Even today, there are still devices and innovations that do not cater to people with disabilities. In addition to helping this group of people, accessible technology can also make life easier for the average user. Take music for example. The average person can learn music, but it is extremely difficult and some may struggle more than others. But, with a program similar to Hyperscore, people across a wide spectrum of diversity can create something meaningful to them and improve their quality of life.
Ad-Hoc Access to Musical Sound for Deaf Individuals (Benjamin Petry, Thavishi Illandara, Juan Pablo Forero, Suranga Nanayakkara)
An Enhanced Musical Experience for the Deaf: Design and Evaluation of a Musical Display and a Haptic Chair (Suranga Nanayakkara, Elizabeth Taylor, Lonce Wyse, S. H. Ong)
Fourney, D.W. and Fels, D.I. Creating access to music through visualization. Science and Technology for Humanity (TIC-STH). 2009, 939–944.
Musica Parlata: A Methodology to Teach Music to Blind People (Alfredo Capozzi, Roberto De Prisco, Michele Nasti, Rocco Zaccagnino)
Color Theory
Technical Implementation
Max
After spending one semester using Max, I figured this would be my first approach for technical implementation. I have been able to make arpeggiators, random number generators, and interactive music pieces from coding on Max. For those that are unaware, Max is a program that allows for interactive music and more through patches and cords. After researching on the website, Max does allow for real time graphics, which is exactly what my project calls for. Also, I can potentially do the demo section of the project with Max as well. I am definitely most comfortable and familiar with this program compared to the other solutions.
WebGL
WebGL (Web Graphics Library) was suggested by a classmate who worked with this interface in a previous class. It allows for 2D and 3D graphics in any compatible web browser. While this is an interesting web based solution, I am not confident or skilled in coding, and I fear that my lack of skill will inhibit my progress with my project.
Chrome Music Lab
The Chrome Music Lab is an excellent source for easy music creation. Just from playing around on the site, one can easily make a simple melody in a few minutes. By using color and a visual interface, it gives the user a refreshing experience with music composition. One particular experiment of interest to me is the spectrogram, which is defined as a picture of sound (Chrome Music Lab). It almost looks similar to a heat map for sound, with certain frequencies appearing more red and others appearing more blue. In terms of my project, it’s important to consider these sources that use color to represent music and sound. In regards to my project, I may want it to perform similarly to the spectrogram where the notes occur on a timeline.
Processing
As suggested by Professor Ault, my last option is Processing, a program that allows for visual art coding. Again, this is a great solution for going about this project, but I am not confident in my coding skills.
