Prototype #3 and Elevator Pitch at Art After Dark

Last week, we had the chance to show the class our third prototypes and then present those prototypes – along with our previous ones – at Art After Dark.

My third prototype just branched off of failed 2nd one. I finally got the Leap Motion Sensor and its SDK to work on my laptop. So, people in class and people who came to Art After Dark got the chance to try out the Leap. They would simply let their hands hover above the Leap and rigged projections of their hands would show up on my computer screen. Everyone I talked to really enjoyed using the Leap and are looking forward to seeing my thesis project progress.

Prototype #2

In my original prototype plan, I had said the 2nd prototype would be photographs of hand motions during crocheting. However, I was provided with a Leap Motion Controller the week before and decided to give that a go.

I attempted to setup the Leap Starter Development Kit (SDK) on my laptop and unfortunately in class last week, I couldn’t open the program or get the USB port on my computer to sense that the Leap controller was plugged in.

Although, this week, I finally managed to make the program work. I figured out that the Leap motion program senses the user’s hand (or hands), displays it on the computer screen, and automatically creates a rig for their hand(s). It’s really neat!

I’ve been trying to figure out how to get it to sense an object that the user is holding (in my case, the objects would be the crochet hook and yarn). I believe the phrase for sensing that is “tool tracking.” I wanted to figure that out for the next prototype phase, but sadly, I haven’t had any luck in finding the setting or application within the program that rigs objects. So, my next prototype, is going to be a mock up of how the interactive stitcher will look on screen. I’m going to take a screenshot of my rigged hand in the Leap motion app, and edit in the guide dialogue, the crochet hook, and the pattern. So I’ll be showing those prototypes and letting people test out the Leap Controller at Art After Dark.

Industry Events I Attended

On November 10th, I got the chance to go to the Museum of Modern Art to see the latest exhibit, Bruce Nauman’s Disappearing Acts. I would’ve written about my experience earlier, but it has been a hectic couple of weeks, plus I wanted to keep my options open if I found another industry event that sounded interesting and that I could go to.

Anyhow, here’s a picture of my IMM friends and I on our way to the MoMA :

Image may contain: Jill Merbach and Patrick Haywood, people smiling, sky, closeup and outdoor

And here’s another photo of me in front of a painting at the MoMA (I would’ve taken one in the Disappearing Acts exhibit but the lighting in that room wasn’t very good):

Image may contain: Jill Merbach, smiling

What was really neat about Disappearing Acts was that museum visitors could directly interact with parts of the exhibit. There were hidden cameras and motion sensors set up around the showroom so you could see different angles of yourself on the TV monitors sitting in the middle of the room, but you couldn’t spot where the camera was.  Then, in another showroom within the exhibit, there were small transparent walls that would sense when you were near them and would begin speaking. Relating back to my thesis project, these parts of the exhibit made me want to have the users’ hand motions be projected onto a screen and have a verbal guide leading the user through the steps of crocheting.

There was also a part of it that I enjoyed that wasn’t interactive. I stumbled upon a wall of phrases in neon lights and it was timed so each different phrase would light up at a different time. Here’s a picture I took of it:

No automatic alt text available.

If you’d like to read more about the exhibit and artist Bruce Nauman, check out the official page on the MoMA website. It’s very interesting!  https://www.moma.org/calendar/exhibitions/3852? 

Also, the week before, on November 5th, I had the chance to attend Jesse Schell’s VR and AR seminar in Mayo Concert Hall. If you didn’t get the chance to go, that was also a really neat event because Jesse Schell talked about running his own VR gaming company and gave a lot of good insight on the future of VR and AR.

I don’t clearly remember every detail of what he said/showed (I was struggling with a serious cold and migraine that night, which didn’t help) but I do remember him talking about his experience working at Disney, showing the ad for his latest VR game called “We Want You to Die”, and showing how far video chatting and interactive toys have come, and how they’re going to change the future generations way of connecting/communicating. It made me think of how my thesis project – and also everyone else’s thesis project – will positively affect and change the future of interactive innovations and connecting.

Here are a couple of pictures I took during the seminar:

Jesse Schell was quite a fun and interesting character, too. He started the seminar off by playing harmonica for the audience! It was very entertaining.

Overall, I really enjoyed both of the industry events I went to and would suggest that other people check out the Disappearing Acts exhibit (it’s still on display through the end of February 2019) and looking into Jesse Schell’s work. My advice is to always go to conventions, interactive exhibits, and seminars if you ever get the opportunity because I’ve gotten to experience many and have had an amazing time at each and every one that I’ve attended.

Prototype #1

For my first prototype last week, I drew up a “blue print” for the setup of my thesis project. I showed the screen – and what would be on the screen, the motion sensor, and the user’s hands.

I feel this will help me to better visualize my outcome and will be a good map for when I physically set up the project.

I know it’s not much of a prototype, but it’s always necessary to plan and sketch when working on a project. Plus, this is a digital fabrication type of project, which requires much coding, setting up equipment, etc. So, it would be difficult to do a whole lot within one prototype. However, for my second prototype this week, I have an actual program set up for my project (I will get more into detail about that in my next post).

The Prototype Plan

  1. For the first prototype, I will sketch out how the project will run. I’ll draw out how it’ll look on screen and how it’ll look physically on the user’s hand.
  2. For the second prototype, I will start taking photos of someone’s hand in a crocheting position to get a closer look at how I should be structuring the hand, crochet hook, and pattern within Unity.
  3. Lastly, for the third prototype, I will start drawing up code within Unity to get a feel and an idea for what I will be working with when I begin my overall project.

Research Update 3/3

This week, I’ve been looking more into what programs I’ll need to use for my thesis. As I mentioned in my last post, I’m thinking of using Unity and possibly a Leap Controller device. In this week’s post, I’ll go a bit more in depth with my plan on learning these programs.

To learn more on Unity, I’m going to invest some time into watching Lynda and YouTube tutorials. Here are a couple ones from YouTube that should be helpful:

  • https://www.youtube.com/watch?v=Au8oX5pu5u4
  • https://www.youtube.com/watch?v=qX8kabgz3wo

I also plan on talking more to my alumni advisor Andy as well as Prof. Josh Fishburn and my good friend Robin who are all more familiar with Unity than I am.

Josh teaches Games I and Games II in the IMM department. Occasionally, I will sit in on his Games I class with Robin to listen to the lecture and watch Robin build his game. So, this gives me the perfect opportunity to talk to Josh and Robin about using Unity in my project.

Then, through email, I’ll continue to communicate with Andy and ask him for some helpful tricks and techniques. Maybe at some point he’ll be able to come to TCNJ to help me and other students who have been assigned to him.

As for learning how to use a Leap Motion Controller, that’s another thing I will ask Andy about as well as Dr. Nakra (who I believe has worked with Leap Controllers), and look up directions and tutorials on. I’ll also need to look into how I would be able to access a leap controller for my project.

Here are some videos I found relating to connecting a Leap Controller with Unity:

  • https://www.youtube.com/watch?v=8KeZxfPh8TE
  • https://www.youtube.com/watch?v=Wh3YK2OcI4M
  • https://www.youtube.com/watch?v=Pdnik0gA09g
  • https://www.youtube.com/watch?v=gMHtBae897I
  • https://www.youtube.com/watch?v=X__oClQTob0

As for right now, though, I am currently in the prototyping phase of the project and need to really consider how I should begin to execute the project itself.

Research Update 2/3

This week, I have been chatting with my assigned alumni advisor, Andy. He was very helpful in suggesting what tools and programs I should use to create my thesis project.

Two things I hadn’t thought to use before that he suggested are Unity and a Leap Controller. Unity should be able to create a textured model to show a crochet pattern and the Leap Controller should be able to pick up on hand movements and directions.

I’ll have to look into them more, but I have never used either tools before. Luckily, I have friends who have experience with coding in Unity. As for the Leap Controller though, I would have to find access to one and do more research on how it is used. I did manage to find a blog that summarizes what it does: http://blog.leapmotion.com/hardware-to-software-how-does-the-leap-motion-controller-work/

 

Research Update 1/3

My interactive stitcher project is going to be quite rigorous, considering there are multiple programs I will need to learn and use in order to execute this project.

However, within a short span of research time, I was able to find many great resources that will be useful in helping me with my project. Professor Ault was helpful in suggesting that I research”computer vision.”

To define the term computer vision, it is “a field of computer science that works on enabling computers to see, identify, and process images in the same way that human vision does, and then provide appropriate output.” (- technopedia.com). It also goes hand-in-hand with artificial intelligence (AI), considering the computer has to interpret things it sees and then have its actions match whatever it’s interpreting.

Anyhow, I’ve created a condensed list below of the sources I’ve found so far and other ideas I have of how to move forward with this project:

  • Video sources: https://www.youtube.com/watch?v=UZSm7Q2bZoc  https://www.youtube.com/watch?v=h8tk0hmWB44 https://www.youtube.com/watch?v=kK0BQjItqgw&list=PLU_f8mulhsXL3JXIjRRL0rcNSsgtzsk6w&t=0s&index=6
  • Website sources: http://programmingcomputervision.com/downloads/ProgrammingComputerVision_CCdraft.pdf   http://answers.opencv.org/question/17183/recommended-hd-camera/
  • Possible Programs to Use: Python (free download), SimpleCV, and Raspberry Pi. Python is one I’m highly considering because many sources recommended that it’s a great program to use for computer vision.
  • Necessary Equipment: small HD camera, computers (in the lab)
  • Possible Professors to Talk To: John Kuiphoff, Teresa Nakra, and Josh Fishburn

 

Finalized Concept

Feedback wise, it’s been about half-and-half: half said I should go through with my interactive storybook, the other half said I should go through with my interactive stitcher.

After contemplating for some time, I decided to go through with the idea of the interactive stitcher. I think it is something that will be able to be executed within the next 7 months and it excites me because I’ve never seen anything like my idea and I want to learn a new program to create this.

Being involved in the knitting/crocheting club here at TCNJ, I can ask my fellow club members about crocheting techniques and patterns to help me with this project. I feel that’s already a good start.

Research Plan

History

A History of Crochet Patterns

Crochet is a craft that has been around since the 19th century. However, I feel it’s a craft that hasn’t been evolved since it was started. I plan on finding more articles on the history of crocheting but this is one I liked.

More things to do and research

  • What tools/programs I will need in order to create this interactive stitcher
  • What specific audience this invention will target
  • How it will be helpful to this specific audience
  • Who can I go to for help on this project
  • What specific resources will help me in completing this project