Category: Research Updates

For this week’s review I spent my time split between more content research while diving into the technology that will make my project a reality.

For this week’s content research, I decided to change things up a bit: last week I read a bunch of articles, websites and books. It dawned on me then that there are other mediums that have a plethora of information I need, that being documentaries. Asking around and getting in touch with some professors, I found quite a few documentaries and television series that talk about ancient civilizations and the common tie between them (which I will be sure to include at the end of this update). I found those to be helpful and insightful, providing even more content and helping me think about how to go about covering the information.

Having watched and read enough information on ancient civilizations to sustain a stable prototype, I decided it was time to tackle the beast I was avoiding: the technology. Having received feedback from Professor Ault and my student adviser a few weeks ago, I decided to look into leap motion cameras. It looks daunting to say the least, however reading about the technology in articles and watching demos provided me with enough information to realize that if I could execute this the same way I envisioned, the Leap Motion camera would help deliver the exact project I want. This upcoming week I’ll reach out to both Professor Ault and my student adviser, as well as a few other people who might be able to give insightful feedback, to talk about the technology and see if there aren’t other options I should consider and really cement my idea once and for all.

Leap Motion Releases Major Tracking Update and New Demos to Show It Off

How Does the Leap Motion Controller Work?

https://www.netflix.com/title/81012415

https://www.netflix.com/title/70212989

https://www.netflix.com/title/80095868

https://www.netflix.com/title/80185801

For my second research update I decided to watch a comedy. One of the biggest parts of my film is the ability to use a wide variety of film genres. In my opinion comedy is one of the hardest genres to successfully execute. It is important that I do a lot of research when it comes to different types of genres. While I will be watching movies with all different types of genres I felt that this was the best place to start. When it comes to comedy there are many different kinds and different approaches to it. Considering what I will be working with I decided to watch and study an Adam Sandler movie. Adam Sandler is one of the funniest comedians of our generation and has found much success in the film industry so he is a great person to study.

The Adam Sandler movie that I recently studied was “Happy Gilmore”. This is one of if not his most popular movie. I learned a lot from watching this movie and did my best to pick out the parts of the film that really made it successful. One of the things that Adam Sandler does best along with most other great comedians is the flare for the dramatics. The main character of the movie is Happy Gilmore and he has anger issues. While anger issues can tend to get out of control Adam Sandler took anger issues and really dramatized it in order to gain more of a reaction from the audience. While a golfer with anger issue may yell and curse, Happy would go beyond that and throw his golf clubs a mile long, beat up on elderly people, and even fight an alligator. These scenes proved to be some of the funniest. Along with the dramatics it seemed that Adam Sandler used pain and punishment to his advantage and it was apparent that some of his intentional funny moments came at the demise of certain characters. These are all things that I would have to direct the talent on in order to execute a joke or action. This movie showed me that sometimes the funniest parts are done and not said.

When it comes to filming it was important to note of how light the scenes were. Comedies are meant to put people in a good mood and be light hearted and fun. One would be hard pressed finding a scene that is dark and gloomy. When filming for this particular genre I will be making an intentional effort of filming things in well lite areas where it brings a light feel to the scene and puts the viewer in a good mood that is uplifting and enjoyable to watch. Another element that is important is the music. I’ve noticed that there isn’t much background music playing throughout the movie. When characters are speaking it’s usually with no background music. The main reason why songs or instrumentals are being played is to set the mood or feeling for a particular scene, but overall it’s important to let the on screen talent shine and not distract viewers away from them. Additionally many of the shot were close ups. These close ups helped bring attention and focus to the characters and let the audience know that this dialogue is very important for you to pay attention to.

It was very interesting watching a movie from a director’s or producers point of view. It’s interesting how little details can really make a big difference in the outcome of the film. With any genre that I am shooting it’s important for me to set the correct mood and setting for each different genre. The producer has the power to control the viewer’s feelings and if I am able to successfully move the emotions and feelings of the viewer from clip to clip then my project will be successful.

Within this past week, my alumni advisor, Meghan McEneaney, reached out to me and gave me some valuable feedback and things to consider for my visual music project.

The first thing she suggested I look into is the Chrome Music Lab, which can be another solution for the technical aspect of the project. It is a web based program, so I may be able to design it for mobile and create a vibration feature to link multiple senses together.

For a less technical look at the project, Meghan suggested I consider the mappings of the colors. “The way people assign different shapes and colors to sound is unique and not to mention – there are an infinite amount of mappings. How will you define these mappings to make this truly accessible for everyone?” This is an excellent point Meghan makes and I’m glad she brought it to my attention. She gave me a few sources to take a look at in regards to visual music, as well as accessible music.

I’m adding these new sources to my list of sources from last week. I have plenty of research to work with for my thesis and I’m excited to show the world of accessible and visual music to everyone!

Narrative (cont.)

This week, I started out doing some research on storytelling and telling impactful stories.

https://www.linkedin.com/pulse/5-ideas-help-create-impactful-storytelling-kieran-mccann/

https://www.forbes.com/sites/skollworldforum/2013/08/01/the-5-secrets-of-storytelling-for-social-change/#5c6af04c4859

https://www.weforum.org/agenda/2018/06/storytelling-for-social-change-communication-initiative/

My message has pretty  much been established, it’s just a matter of crafting around it. Stylistically, I’m thinking of having more geometric models (because I’m a sucker for the polygon aesthetic). I was inspired by this incredible short film I’d discovered years ago and managed to dig up again: https://www.youtube.com/watch?v=b5P2Qmq9r40 in which everything was made out of origami. Making the models geometric looking would be simple enough–I would just leave everything in 1 mode in Maya. I would just have to be very careful with my mesh and think about how I’d want to weight paint my models, whether I want them to move more organically or more like paper puppets.

In brainstorming my storyline, I decided I wanted to animate trees growing and looked for tutorials to learn how to do it.  I found this great video I can reference: https://forums.autodesk.com/t5/maya-forum/maya-2018-how-to-animate-a-growing-tree/td-p/7982261.
It doesn’t show how to do the leaves, but I can try to figure that out myself or ask for help. I have several mentors I can refer to for advice in my network, like Professor Smuga, my alum adviser Raya, and some friends who are in animation.

Speaking of adviser, I’ve been in contact with Raya and chatting with her. I’ve gotten some great advice so far so I’m excited to work with her more as my project continues to take form!

Display

On another note, after deliberating with some friends, I realized that my display set up would present a couple of problems:

1. Material. I would need to find a material sturdy enough to provide a stable surface for the pedestals for my holograms to be displayed on. This means it would most likely be made out of  wood, or some kind of recycled material if I can find it. I know that wood is expensive, especially if I require a decent amount of it. This means the ideal situation would be if I could find some sort of alternative material to repurpose (which would, actually, fit in perfectly with my theme. Hmm…..HMMMmmmMM).
2. Stability. In the same vein, I recognize that I my display needs to be sturdy enough. If I’m using iPads or some other kind of monitor, I would rather them not run the risk of being knocked over. That means the pedestal set up would be a bit precarious.
3. Screens. After watching some more videos and looking into the way the holograms are projected, I realized that the brighter the screens (and the clearer the glass), the better the quality of the projections are. This means that my best bet would be using the iPads or some kind of tablet, as opposed to just a monitor. Perhaps a laptop would work as well, but the structure of laptops would be cumbersome for this purpose. The accessibility of laptops are also a tad slim. However, Professor Ault has brought up a good point that the iPads might be in short supply come presentation day. Perhaps I can get my hands on one and then use a series of smaller screens for more supporting holograms.

HANG ON I legitimately just had a new idea for my display set up. Maybe instead of using pedestals, I can create a physical environment to mimic (or contrast) the one in my holographic animation as an extension of sorts. That would solve my set up problem, and provide a sufficient justification for my use of holographic technology as a storytelling medium. I’d like to look into this idea further, and hopefully get some feedback!

This week, I have been chatting with my assigned alumni advisor, Andy. He was very helpful in suggesting what tools and programs I should use to create my thesis project.

Two things I hadn’t thought to use before that he suggested are Unity and a Leap Controller. Unity should be able to create a textured model to show a crochet pattern and the Leap Controller should be able to pick up on hand movements and directions.

I’ll have to look into them more, but I have never used either tools before. Luckily, I have friends who have experience with coding in Unity. As for the Leap Controller though, I would have to find access to one and do more research on how it is used. I did manage to find a blog that summarizes what it does: http://blog.leapmotion.com/hardware-to-software-how-does-the-leap-motion-controller-work/

 

10/20/18

Today I researched Voice user interface (VUI) and read up on the Alexa Voice Design Guide to get a better idea of how skills work. I think I jumped right into learning how to build a skill through YouTube videos without much of a research foundation. Now I am focused on some self-teaching to learn how to make this skill as efficient as possible.

I looked into the cultural aspects of my project by learning about the most popular spice blends per geographical region. This led to my first prototype idea: an Alexa skill that will give you spice recommendations based on the regional food the cook decides to make. For example, if you wanted to make a Mexican flavor inspired meal, you’d ask Alexa for her Latin American spice suggestions and she will give the top five most common spices from the area, optimally common enough that most people would already have at home.

My next step will be looking into spice blending and consulting experts.

10/23/18

I did more research on VUI frontend work and took a glimpse into backend work. This might prove to be more coding-intensive than I originally thought.

After some correspondence with Dr. Nakra and her husband, it turns out I’ll need to do much more research on how the spice rack will be executed. RFID has been ruled out at this point, and now I am looking into a possible Bluetooth-like solution.

10/25/18

In my Dynamic Web Apps course, we are learning to use databases. I decided to make my final project a virtual spice rack that keeps track of all the spices the user has in their possession. As I continue to learn more about databases, this could potentially help me with my thesis project in the future.

So, what to talk about this week? There’s a whole bunch I could cover in this second part, so let’s just talk about modding as a whole, from the activity’s past to its present community. I’ve already covered one of the first popular mods to be passed around the community titled, “Castle Smurfenstien”. This mod took the original Castle Wolfenstein and replaced the Nazi sprites with that of Smurfs.

This comedic and lighthearted tone will carry on in modding throughout the ages. The process of swapping sprites and models will live on as well. For example, in Skyrim, there is a mod that swaps the Dragon models with Thomas the Tank Engine. Yes, you heard me. Watch at your own risk. Once the immensely popular Doom came out, modding soon followed, replacing sprites with anything the user could dream up. The developers of Doom even gave a package of easily accessed game files to the community in order for them to be able to make their own levels. Mod-friendly developers have been a godsend to the community because you’ll never really stop the modders, so might as well help them. You may ask why a developer would go out of their way to allow their fans to easily replace their dragons with trains. Well, having a healthy modding community extends the lifetime of your game by spades. Heck, Fallout: New Vegas has been out for eight whole years and still has a large player base thanks to its constant release of awesome mods. Actually, a team just released a huge mod for New Vegas called Fallout: New California.

In 2007, a website called Nexusmods went live right before the release of Fallout 3. This site has since become the place that most modders go to upload and share their mods for a variety of games. This site hosts 263,928 files for 619 games from 85,195 authors and it’s open for anyone to join and download from. This place has gathered mods together for easiest access and reaches the largest player base as it can. The site has also released its own tool to make the downloading of mods much easier for the casual fans who may not know too much about files and folders. The Nexus Mod Manager allows for you to connect to your game and the site. A simple button press on the site will send the mod’s files right to your NMM and apply the files to your game. Some mod installations are as simple as two clicks. I anticipate that the mod I will be working on will be just as simple as that. I honestly can’t wait until I can share my finished mod to the world on this site and finally enter the modding community as an author.

While I wasn’t able to set aside time to find/use a VR headset to test some games, I did learn a lot about how to make a VR game in Unity. Like most things in Unity it’s easy to start, but difficult to polish. Unity has a helpful tutorial that walks you though the basics of VR development with helpful videos, explanations, code, and tips. Unity has many more VR tutorials of increasing difficulty and scope that I’ll need to review/practice on my own to get a feel for how the engine handles things.

I also realized I’ll need to take an in-depth look on how to make effects (smoke, fire, lighting, etc.) in my project and how I’d import them into Unity. I’ll also need to look up a lot of reference on how to use Unity’s physics engine as I have little to no experience with it.

I’ve started compiling a list of the various potions the player can make in the game and thinking about how they can make them. I’ve been researching various mythos on potions, spells, and magic for inspiration. These have lead to me including ingredients I never would of thought of on my own and fun nods I can make to their origin.

For the first research update I learned about video production and editing. The resourse that I used to find this information is from Lynda.com. While putting together my research plan one of the main skills that I knew I needed to learn or grow more in was in film production. Throughout the video playlist the viewers were taught of the different aspects of production and editing. I have always believed that editing was simple and the least important. These videos have changed my perception and have caused me to changed the way I approach editing videos and I have gained a new respect for this part of the creative class. The expect giving these lesson is Rob Garrott, the content manager for video production and editing at at lynda.com.

One of the first points that Rob brought the attention of the viewers is the importance of having a clear creative goal before shooting even begins. In my past experiences I would go into the field and begin filming based off all the ideas and concepts that are scattered throughout my mind. I often found myself very frustrated while filming and now I understand why. Rob often argued that pre-production is the most important part of the entire process. The example he would give is that you would never see a musician going on stage without practicing his instrument and receive a standing ovation, and the same goes for filmmakers and producers. It would be ignorant for a producers or filmmaker to go onto the field without practicing or having a plan of action. It’s important to be able to describe your vision with clearly with words. If you are having a hard time accomplishing this then it’s imperative to go back to the drawing board and work out more of the kinks that still linger.

Another interesting part of the video series was how Rob broke down the three stories that every film has. The first story that every film has is the story that you wrote and conceive; the second story is the one you shoot in your camera; the third story is the one that you edit. All three of these stories should be able to work harmoniously and create a fluid storyline that supports the vision and creative goal that was created in the very beginning of the creative process. Something else that stood out to me was when Rob was speaking on the control and power that the editor has. Editors have the power to manipulate the viewer’s perceptions by how the various elements are pieced together. While the director camera crew and providing the shots and scenes of the film it’s actually the editor who has the most control. The way a video is editing is the reason for audience reactions and emotions throughout the film. For example when watching a horror movie or thriller the way a scene is edited brings out the emotions of the viewers. While the actors and directors play a pivotal role as well it the way the shadows are presented in the room and how the music is lingering in the background and then fades out to really put the view on the edge of his or her seat.

It’s very interesting how these videos have broken down the importance of production and editing. One of the most important concepts from the videos was how editing is a creative process and like any creative process it takes trial and error and forces one to really be creative and dig down deep inside one’s self to find solutions and beauty to the film. The skills and knowledge that I have acquired from this will be very instrumental in the production of my project and will prove to play a pivotal role in how I attack this film and guide my throughout the creative process.

Building the holograms

To start building a base of resources I can reference when building the actual project, I started out researching a couple tutorials on how to make holograms.

There were a couple different methods to create the actual videos to be projected, but they all seem to have the basic idea.

How to Make a Hologram Video in PowerPoint (Yes, Way!)

https://www.instructables.com/id/How-to-make-a-hologram-video-for-DIY-3D-Hologram-P/

Sustainability/data research

Again, I put the UN sustainable development goals site into my bank of resources (https://sustainabledevelopment.un.org/?menu=1300). Since I wanted to focus on environmental sustainability, I decided to look into statistics and facts within this area. Here are a couple sites I found:

https://ourworldindata.org/forests
https://unstats.un.org/unsd/envstats/qindicators.cshtml

Professor Ault lent me a couple of books about designing for social impact that I’ve been reading for inspiration, and there are some parts in it related to sustainability. I also found this article on how data visualization can help the environment: https://blogs.scientificamerican.com/guest-blog/dust-in-the-wind-how-data-visualization-can-help-the-environment/

Whiiiiich brings me to my next point:

Narrative

My main objective, and perhaps most challenging, is developing a narrative that will be provocative, as well as take advantage of the holographic technology I plan to utilize. It can be argued that the keystone of storytelling is emotional evocation. But with the rise of social media, the internet, and readily available multimedia, creating narratives (whether it be data vis or an actual storyline) with significant emotional impact that doesn’t wear off after the story is over becomes harder. There was one study done where virtual reality was used to demonstrate the impact of pollution on coral reefs, amongst other environmental issues (https://www.huffingtonpost.co.uk/entry/how-virtual-reality-tech-will-help-the-environment_uk_59a7f038e4b010ca289a8578?guccounter=1&guce_referrer_us=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_cs=SBZq0XkjgZ3oS5dW39gLVQ). These experiences were shown to induce a more empathetic mindset in subjects, and I hope to employ a similar concept with my project.

There is also a kind of disassociation that happens when we’re bombarded with data and statistics and “end of the world!!” messages all the time. This is why I’d like to use real time data, if possible. I was having a bit of a difficult time finding useable data on global deforestation data that I can attain in real time, but after some brainstorming I realized it would perhaps be more significant if I used data a little closer to home–perhaps TCNJ. If I can get data from TCNJ, whether it be electricity use or something else, I might be able to use something like WebGL to generate graphics for data vis on the fly. Definitely something I’d like to look more into!