huangj11

Grace Hopper Celebration 2018

Early this year in September, I attended the 2018 Grace Hopper Celebration (https://ghc.anitab.org/ghc-18/) in Houston, Texas. This annual conference is a celebration of women computer scientists and women in technology. I was lucky enough to attend as a student scholar, so my advice to anyone who’s interested in attending is to apply for the scholarships! It’s so worth it.

Anyways, the conference consisted of amazing keynotes, meet and greets with powerful women in the industry, a hack-a-thon (we didn’t have time for this, but maybe next time!), workshops, talks, and the biggest career fair I’ve ever seen!

Because my event was so early, I was mostly scouting out for ideas. I split my time between the career fair and attending as many talks as I could (a difficult feat, as they took place in 2 adjacent buildings and were often at the same time). I was especially inspired by the numerous talks on technology as a tool for change. I even got to chat briefly to a UX designer from the Loon project after her panel. I mentioned that I was doing a senior thesis project and asked for any advice, and she emphasized research, especially user research.

My first thesis idea was inspired by the keynote by Jessica Matthews, but I couldn’t land on anything solid so I moved on to a different idea for my final. But it was still an incredible and empowering experience! I my passions lie in computing, animation and design, and there were definitely a fair share of people in all those industries. Looking forward to coming back in the near future!

Prototype 1 blog

I went about my prototype plan in a bit of a different order. I discovered that in order to successfully plan the physical environment to display my animations, I would need to have a clear storyboard first. For my first prototype, I created a storyboard of the animation that would play on the devices. I would utilize the fact that there will be multiple screens by having the animation cross from one device to another to create the sense of an extended world beyond the screens. Naturally, this made it a bit difficult to storyboard accurately, but I hope I conveyed it successfully!

The physical aspect of the project will involve a mountain environment constructed from discarded trash I find around campus (cans, boxes, etc.). There will be LED lights scattered across the environment that will be connected to a motion sensor. I’m thinking that could be the data visualization aspect of my project, like for example perhaps each light represents a species that has gone extinct in the past 50 years or something.

The storyboard starts with a deer grazing peacefully on the bottom-most screen.

I consulted with a couple of my classmates as well as Professor Ault and Raya, and got positive responses, mainly regarding the idea of mixing media and the storyboard. This was really beneficial in assuring that I’m on the right track after altering my idea. I’m still looking for critiques so if anyone has any feel free to let me know!

Research Update 3/3

I’ve talked with some of my coworkers about my project, since they’re all IMM alum to get their feedback, as well as my alum adviser. I’ve also chatted with friends and classmates just to hear their thoughts.

However, the focus on this week’s research is more so on preparation for building actual prototypes. I already have a good stash of tutorial resources in building the actual hologram prototype. Now I’m compiling a list of materials I need:

iPads (possibly Android tablets) x5?
Headphones
Maya
Materials to make environment (paper mache? 3D printer? Cardboard?)
Plexiglass or some other transparent material
- CD cases for prototype
Possible arduino or raspberry pi
Motion sensors
Mini LED lights (possibly triggered by motion sensor)

Prototype Plan

My first prototype will be a sketched out or physical model of my piece. This will be important for planning my next prototype so that I can successfully organize how the physical and digital aspects will affect the animation. That’s the main goal for this prototype–to get a handle on the interaction between the two mediums and how the display will make use of the space.
My second will be a storyboard of not only the animation aspects, but how that will include the physical model (similar to what I said in the above). I hope to maybe not finalize completely, but have a good base for my story and get feedback from my classmates and alum adviser. I may do several versions.
My third prototype will be actually building a hologram for my phone. If I can do that, then I know I can scale my project up. This will include building the physical device, as well as producing a simple holographic video I can use to test. This will most likely be an animation created by following the tutorial I found last week. I’m excited to see how this will turn out!

Research Update 2/3

Narrative (cont.)

This week, I started out doing some research on storytelling and telling impactful stories.

https://www.linkedin.com/pulse/5-ideas-help-create-impactful-storytelling-kieran-mccann/

https://www.forbes.com/sites/skollworldforum/2013/08/01/the-5-secrets-of-storytelling-for-social-change/#5c6af04c4859

https://www.weforum.org/agenda/2018/06/storytelling-for-social-change-communication-initiative/

My message has pretty much been established, it’s just a matter of crafting around it. Stylistically, I’m thinking of having more geometric models (because I’m a sucker for the polygon aesthetic). I was inspired by this incredible short film I’d discovered years ago and managed to dig up again: https://www.youtube.com/watch?v=b5P2Qmq9r40 in which everything was made out of origami. Making the models geometric looking would be simple enough–I would just leave everything in 1 mode in Maya. I would just have to be very careful with my mesh and think about how I’d want to weight paint my models, whether I want them to move more organically or more like paper puppets.

In brainstorming my storyline, I decided I wanted to animate trees growing and looked for tutorials to learn how to do it. I found this great video I can reference: https://forums.autodesk.com/t5/maya-forum/maya-2018-how-to-animate-a-growing-tree/td-p/7982261.
It doesn’t show how to do the leaves, but I can try to figure that out myself or ask for help. I have several mentors I can refer to for advice in my network, like Professor Smuga, my alum adviser Raya, and some friends who are in animation.

Speaking of adviser, I’ve been in contact with Raya and chatting with her. I’ve gotten some great advice so far so I’m excited to work with her more as my project continues to take form!

Display

On another note, after deliberating with some friends, I realized that my display set up would present a couple of problems:

Material. I would need to find a material sturdy enough to provide a stable surface for the pedestals for my holograms to be displayed on. This means it would most likely be made out of wood, or some kind of recycled material if I can find it. I know that wood is expensive, especially if I require a decent amount of it. This means the ideal situation would be if I could find some sort of alternative material to repurpose (which would, actually, fit in perfectly with my theme. Hmm…..HMMMmmmMM).
Stability. In the same vein, I recognize that I my display needs to be sturdy enough. If I’m using iPads or some other kind of monitor, I would rather them not run the risk of being knocked over. That means the pedestal set up would be a bit precarious.
Screens. After watching some more videos and looking into the way the holograms are projected, I realized that the brighter the screens (and the clearer the glass), the better the quality of the projections are. This means that my best bet would be using the iPads or some kind of tablet, as opposed to just a monitor. Perhaps a laptop would work as well, but the structure of laptops would be cumbersome for this purpose. The accessibility of laptops are also a tad slim. However, Professor Ault has brought up a good point that the iPads might be in short supply come presentation day. Perhaps I can get my hands on one and then use a series of smaller screens for more supporting holograms.

HANG ON I legitimately just had a new idea for my display set up. Maybe instead of using pedestals, I can create a physical environment to mimic (or contrast) the one in my holographic animation as an extension of sorts. That would solve my set up problem, and provide a sufficient justification for my use of holographic technology as a storytelling medium. I’d like to look into this idea further, and hopefully get some feedback!

Research Update 1/3

Building the holograms

To start building a base of resources I can reference when building the actual project, I started out researching a couple tutorials on how to make holograms.

There were a couple different methods to create the actual videos to be projected, but they all seem to have the basic idea.

How to Make a Hologram Video in PowerPoint (Yes, Way!)

https://www.instructables.com/id/How-to-make-a-hologram-video-for-DIY-3D-Hologram-P/

Sustainability/data research

Again, I put the UN sustainable development goals site into my bank of resources (https://sustainabledevelopment.un.org/?menu=1300). Since I wanted to focus on environmental sustainability, I decided to look into statistics and facts within this area. Here are a couple sites I found:

https://ourworldindata.org/forests
https://unstats.un.org/unsd/envstats/qindicators.cshtml

Professor Ault lent me a couple of books about designing for social impact that I’ve been reading for inspiration, and there are some parts in it related to sustainability. I also found this article on how data visualization can help the environment: https://blogs.scientificamerican.com/guest-blog/dust-in-the-wind-how-data-visualization-can-help-the-environment/

Whiiiiich brings me to my next point:

Narrative

My main objective, and perhaps most challenging, is developing a narrative that will be provocative, as well as take advantage of the holographic technology I plan to utilize. It can be argued that the keystone of storytelling is emotional evocation. But with the rise of social media, the internet, and readily available multimedia, creating narratives (whether it be data vis or an actual storyline) with significant emotional impact that doesn’t wear off after the story is over becomes harder. There was one study done where virtual reality was used to demonstrate the impact of pollution on coral reefs, amongst other environmental issues (https://www.huffingtonpost.co.uk/entry/how-virtual-reality-tech-will-help-the-environment_uk_59a7f038e4b010ca289a8578?guccounter=1&guce_referrer_us=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_cs=SBZq0XkjgZ3oS5dW39gLVQ). These experiences were shown to induce a more empathetic mindset in subjects, and I hope to employ a similar concept with my project.

There is also a kind of disassociation that happens when we’re bombarded with data and statistics and “end of the world!!” messages all the time. This is why I’d like to use real time data, if possible. I was having a bit of a difficult time finding useable data on global deforestation data that I can attain in real time, but after some brainstorming I realized it would perhaps be more significant if I used data a little closer to home–perhaps TCNJ. If I can get data from TCNJ, whether it be electricity use or something else, I might be able to use something like WebGL to generate graphics for data vis on the fly. Definitely something I’d like to look more into!

Research Plan

History of Holography

The concept of holograms, though futuristic sounding, dates back to 1948, when it was first thought of by Hungarian scientist, Dennis Gabor. Since then, it’s an idea that has been poked, prodded and expanded in the realms of science, science fiction, and art. In the mid 1970’s, embossed holograms were invented, the first example of mass produced holographic imagery. They’re the silver-rainbow stickers found on your credit cards!

I also want to take a quick moment to spew out some statistics about deforestation. As that would heavily impact my project and storyline, I feel it’s important to get some background knowledge on the topic. According to the UN’s Sustainable Development Goals site, the earth’s forests have shrunk from “4.1 billion hectares in 2000 (or 31.2 per cent of total land area) to about 4 billion hectares (30.7 per cent of total land area) in 2015.” Although that .5% may not seem like much, especially in 15 years, that’s still over 1 billion people whose livelihood depends on these forests affected, and that percentage is still decreasing. Good news, it’s decreasing at a slower pace than before. Bad news, deforestation, illegal poaching, invasive species, pollution, etc. have triggered an alarming trend in wildlife extinction. I’d like to do more research on how to best portray and combat these issues.

Sources:

History of Holography

https://sustainabledevelopment.un.org/sdg15

Current State

Although true holography like what we come to expect when we watch movies (think Treasure Planet or Gaurdians of the Galaxy or even that scene in The Hunger Games) hasn’t been achieved yet–at least commercially, there is a burst of emerging tech that’s toeing the line. The Microsoft Hololense, for instance, uses mixed reality to imitate holographic imagery filtered through smart glasses. Although holograms are, by definition, a photographic recording of light without the aid of screens, I’d say that mixed reality in general is a sort of parody of the spirit of holography. There have been practices that involve projecting onto a thin mist to create the illusion of holograms, as Professor Ault mentioned before (https://makezine.com/2014/04/18/fog-projection-combined-with-gestural-interface-to-create-hologram-touchscreen/). Similarly, I remember the Japanese “Vocaloid” craze from back when I was in middle school. Upon further research, it seems that the holographic performers were projected onto a transparent screen. So again, not a real hologram. Rather, they are considered variants of “Pepper’s Ghost” technology (name derived from John Henry Pepper, who popularized the technique).

And finally, the most widespread DIY hologram project–the CD case. This is the technique I plan on using for practicality purposes.

As of today, there are a couple companies that have cropped up to exploit the holography craze. MDH Hologram (http://musion.com/) is one of them, and is currently owned by Giovanni Palma. They also use Pepper’s Ghost technology for mainly purposes of entertainment. Liti Holographics (http://www.litiholographics.com/about.html and yes, that is the actual name of the company. I chortled.) is another company that commercializes the use of holography. They apply it more towards the corporate consumer, marketing mostly to businesses rather than entertainment consumers. This company was founded by Paul Christie and Brian Christie. Paul Christie was mentored by Steve Benton, a pioneer of holography. These people might be worth reaching out to.

To continue my research, I plan on:

Looking through the internet and our campus resources on both holographic techniques and environmental studies
I have friends in computer science and engineering who will hopefully be able to help me out with motion sensors should I need it. I’ll also talk to John Kuiphoff to see if he can help with it.
Contacting the Rainforest Connection if there’s something I think they can help with or for general fact gathering
I feel like most of my research that involves reaching out to people will be centered around my narrative, as that’s the most vague aspect of my project. I need to look into how to achieve indirect impact and ways to make the viewer’s experience meaningful.

Finalized Concept

My finalized concept is using holographic technology (aka a bunch of iPads and plexiglass) to depict a narrative that describes the issues surrounding deforestation. I plan to use several iPads that would each feature a holographic video of part of a forest. Put together, these videos would form an overarching, connected narrative that would immerse the viewer in a digital forest or data visualization. I’d like to add motion sensors that trigger different videos to play, or at the very least turn on the holograms. I may even have a larger projection on the wall (just a regular flat 2D video), but that would depend on the time frame of the project.

The company Rainforest Connection is the main source of inspiration for this idea. Founded by Topher White, it utilizes Tensorflow, a machine learning framework created by Google, and old cell phones to detect the sounds of chainsaws and alerts the locals of illegal deforestation activities. I stumbled across this site: https://www.google.com/about/stories/rainforest/
The display of the site intrigued me, and while it’s not data visualization, I liked the way they used interactivity and presentation to draw attention to their mission. What’s great is that they even have a contact that people can reach out to for project help and contributions.

I love hiking and I love nature, so I would personally be invested in a project that promotes that. I’ve also always been a storyteller and artist first since I was young, and since coming to college I’ve mostly been focusing on the technology aspect over the artistic despite being an IMM major. I’d like to be able to return to my roots (pun as an afterthought) with this project, as well as practice some 3D modeling/environmental design. It may not be directly impactful, but I think I can make it work.

My initial prototype I’d like to complete by the end of the semester is at least one holographic video complete with a preliminary model. If I can get my hands on a sensor and get it hooked up to the hologram, that would be bonus points. I’d probably do several mockups of the hologram to decide which set up I like best, as well as what style I’d like the video narrative to be in.

With that being said, I’m excited to jump in!

Revised concept

After being able to discuss it in class, it’s pretty clear that the 2 main concepts have been narrowed down to the light generating object and the hologram exhibit. While I’m leaning more towards the hologram idea as of now, I’ll still talk about both.

I’ve tried looking up tutorials on how to store energy by building portable generators, and so far the results haven’t been very promising. I’m no electrical engineer, so the scope of this project is hard to gauge, and most likely out of my abilities. There are tutorials for solar powered lights which store energy for use later, which might be worth looking into. Whether I end up going with this idea or not, I do plan on reaching out to Jessica Matthews for more information on how she managed to build her product as a college student, as well as her design process and research. Her story is both an interesting and empowering one, and I’d love to learn more!

The hologram idea seems more feasible and far easier to scale with an incremental design process. This would be my first installation sort of piece, which would be interesting. I imagine multiple iPad sized displays each mounted on a pedestal for viewers to walk around for a more immersive environment. Preferably, I’d build everything from scratch, including the videos that would be projected. This would allow me to put my 3D modeling skills to use. The idea I have now would focus on deforestation and environmental data visualization of some sort. I would love to include some form of interactive aspect that’s more than simply walking around the display, maybe motion sensors that trigger a different video or lights or something.
There are multiple tutorials out there showing how to make your own holograms with plexiglass as well as holography-friendly videos so I’m not too worried about that. The motion sensors would be more difficult, but that would come in later should I choose to go for it. The next steps would be to flesh out a “storyboard” or the content of the videos and hopefully get a working prototype.

Thesis Initial Concepts

Disclaimer, I’m writing this from Houston, Texas with the power of the last two brain cells I have left after a nonstop day at the conference. I’ve had less time than I’d like to think of my ideas, so these are very general and will need much refining in the future (of may be changed altogether…I’ll think on it more when I can actually keep my eyes open)

An app that reads business cards and automatically finds the person on LinkedIn, Instagram, stores the email as a contact…etc. Combines computer vision, app development, and UI/UX. I feel like this has been done already and it seems a bit difficult, but I’ll do more research.
Light generating object. There was this amazing keynote talk at the conference by a woman who created a soccer ball that stores the kinetic energy generated by kicking the ball and then used it to give off light in communities with issues with power accessibility. I have no idea what object it will be yet, and I have no experience in physical computing, but I still think it’s a cool concept.
Oh geez I really don’t know…giant holograms? Is that a thing? I think it’s a thing. I’ve seen videos where people make tiny ones with cd cases and iPhones.