Industry Event Report

I attended the Dust or Magic panel with Jesse Schell and other speakers who discussed the future of VR, specifically its appeal towards children. I’ve had a growing interest in VR and knew quite little before attending this event. I saw it mostly as another medium through which games could be enjoyed. The panel discussed AR as well, which I’d only been exposed to through Pokemon Go before. Because of this talk, I not only have a better understanding of VR and AR, but I now know about many more applications than just games.

What struck me as most interesting was Jesse Schell’s equally educational and entertaining programs he was making at Schell Games. In the end, VR is supposed to be an exciting experience and as it is young, many people are still experiencing it for the first time. The industry is new and this leaves VR a topic for much debate of where it should head next.

Some new things I learned at the panel were the limitations of the technology. One of the main reasons why VR hasn’t taken off is because of accessibility. The Oculus Rift and other consoles that are specifically built for VR are hundreds of dollars, and out of many consumers’ price ranges. Some solutions like the Google Glass have been created, but they cannot compare to a native VR console. Another thing I learned was the view limitations with AR glasses. The field of view is only as big as the screen you have, which can lead to cutoff and an unimmersive experience. Out of VR and AR, VR is certainly further along and we will be seeing much more growth in the coming years. I was also surprised to learn that VR is on the same course as the television, something I couldn’t imagine not owning. If this takes off the same way the TV does, the future of VR sure looks bright.

Although I do not intend on using VR in my senior thesis project, this panel has definitely provided me with some insight going forth into next semester. Just because a new piece of technology hasn’t gotten big, doesn’t mean it’s not going anywhere. I’ve talked to a few people about incorporating Alexa in my project, and one of the biggest reasons for not using Alexa was privacy concerns. Voice UI is growing and is entering the homes of more and more new families. Experimentation and pioneers of the industry are the most valuable assets of new technology right now. Without experimentation, there will be no growth. This is something I will keep in mind during the development phase of my project.

Prototype 2

For my second prototype, I constructed a row of my spice rack out of cardboard and paper. I’ve noticed that I’ve been having a difficult time getting others to picture my project, so I decided to build a simple visual aid. I plan to present this at AIMM After Dark as well. My cardboard rack includes my intent to implement a tracking device that will know where every spice is on the rack. I will also explain the alternative QR-esque tracker that I discussed in class with Professor Ault.

Presenting this prototype was not my original idea. I had planned to work with Arduino and LEDs this week, but decided I really needed this visual aid for presenting purposes. I am still challenged with researching sensors and finding a way to work this into my project. This has by far been the biggest obstacle in my path. I hope to at least discover a solution to this before my third prototype.

I learned a lot from talking to Professor Ault about sensors. Instead of something involving RF, I could look into QR codes with a camera constantly tracking the position of all the codes above the spice rack. For this, I’d need some sort of tiered design instead of a stacked rack. This prototype has opened the floor for more questions and understanding from people I present this to, which has also led to more in depth discussion and contemplation on my part.

Prototype 1 Thoughts

I presented my first prototype, the published Alexa skill “Seasoning Advice” so I could conduct some user testing and gauge their interest in my theme. Immediately, I noticed the amount of variation between different people when they interacted with my skill. Some would think it through and say the actual food they were cooking (ex. pasta) then categorized it as a region (ex. Italian), while others directly said the region. An error also popped up while reading back the spices. When the user hears the list of recommended spices, Alexa then prompts the user with a yes or no question, to either repeat the list or be taken back to the main menu. There was a case where someone responded with another region invocation instead, and Alexa ended up repeating the spice list for the previous request. These interactions gave me some insight as to how feasible this skill will be in real life scenarios. What if the user doesn’t know what region to categorize their food under? I should make this skill easier to navigate as well and create more safeguards to prevent the conversation from getting stuck in an endless loop.

This week, I had the chance to talk to my thesis advisor, Nivi. With her guidance, we were able to reestablish my original goals and vision for my spice rack. As designers, we must first and foremost understand the problem in order to create a meaningful solution. Therefore this week, I worked on getting back to the basics and shifted gears towards the hardware and functionality of the spice rack. Nivi gave me a bunch of valuable advice and taught me to look through a new perspective: how might my product fail? In most basic terms, my idea is a spice rack that shows you the spices you’re looking for. It will reveal spices and herbs by flavor, region, name, and favorite with an indicator that shows each location on the spice rack. Any additional features with the latest gadgets and gizmos will be great secondary goals to challenge myself with. I must be clear with my vision to prevent myself from being distracted with what goes on top.

Prototype Plan

Prototype 1 – Alexa Seasonings Skill

My first prototype will be a published Alexa skill that will help me learn the basic Alexa skill creation workflow. I will use one of my ideas, spices sorted by region, and work it into a helpful skill that will recommend spices based on what region you are cooking from. I will use Amazon AWS, Alexa Developer, and Getstoryline.com to put this skill together.

Prototype 2 – Arduino LED coding

I will learn how to code for Arduino in this prototype and demonstrate how I can program lights to make them turn on and off at specific times. For example, there will be a designated light for each spice and if I want to know the location of a certain spice, the light will know when to light up.

Prototype 3 – Alexa-enabled Arduino LED

For this prototype, I will put all my previous ideas together to build the simplest Smart Home program possible: telling Alexa to turn on and off an LED light with Arduino. Once I get that far, I can start configuring the lights for certain regions and flavors. This will get me as close as I can to my finished project.

Looking towards the future, I will still need to implement a database to save favorites and custom spice combinations.

Research Update 3 of 3

10/29/18

This week I have a challenging research task: figuring out how to hook up an Alexa controlled LED light. I’ve done some looking around with smart home products and Bluetooth, but circled back to Arduino, something I found out about in the beginning of the semester but never got around to looking into. There are some tutorials that demonstrate linking Arduino to Alexa, but I will need an Arduino circuit board and some mini LEDs to get this going. For now, I will start on the code and do more research on LED linking tutorials. For my prototype, I’d at least like to have one LED hooked up and controlled by Alexa by simply saying, “on” or “off”. I want to continue researching Arduino and its capabilities before I decide on any hardware or look into anything else. This just might be what I’m looking for. The biggest remaining question is how the spice rack will know where each spice is on the rack. Also, I wonder if I have the coding ability necessary to put this all together on the backend.

Research 2 of 3

10/20/18

Today I researched Voice user interface (VUI) and read up on the Alexa Voice Design Guide to get a better idea of how skills work. I think I jumped right into learning how to build a skill through YouTube videos without much of a research foundation. Now I am focused on some self-teaching to learn how to make this skill as efficient as possible.

I looked into the cultural aspects of my project by learning about the most popular spice blends per geographical region. This led to my first prototype idea: an Alexa skill that will give you spice recommendations based on the regional food the cook decides to make. For example, if you wanted to make a Mexican flavor inspired meal, you’d ask Alexa for her Latin American spice suggestions and she will give the top five most common spices from the area, optimally common enough that most people would already have at home.

My next step will be looking into spice blending and consulting experts.

10/23/18

I did more research on VUI frontend work and took a glimpse into backend work. This might prove to be more coding-intensive than I originally thought.

After some correspondence with Dr. Nakra and her husband, it turns out I’ll need to do much more research on how the spice rack will be executed. RFID has been ruled out at this point, and now I am looking into a possible Bluetooth-like solution.

10/25/18

In my Dynamic Web Apps course, we are learning to use databases. I decided to make my final project a virtual spice rack that keeps track of all the spices the user has in their possession. As I continue to learn more about databases, this could potentially help me with my thesis project in the future.

Research Update 1 of 3

10/16/18

This week, I created an Amazon AWS account and Alexa skill builder account. I spent a lot of time following along with YouTube videos to work towards developing my first Alexa skill. Talking my idea through with Professor Ault helped immensely in figuring out what direction I should go. After some thought, I realized a touchscreen UI was an impractical way to implement a cooking aid. Voice would be much more fitting for a hands-free experience. I am assuming my skill will fit into the Smart Home category because I would need lighting control, but I will figure out the specifics later. The reason I am getting started with Alexa skills so early is because I want to test if it is possible for me to build what I envision on this platform. The sooner I can test it out, the faster I will learn my limitations and potential roadblocks. Also they’re giving out a free Echo Dot if I can get a loose concept published in time 🙂

I have reached out to Dr. Nakra to meet up and learn more about RFID technology, which might come in handy when I start with the physical portion of my project.

10/17/18

I’ve started building an Alexa skill based on my thesis idea. Although I’m working off example templates for now, I’m already facing some challenges with learning constant variable names and how to translate what I’m trying to do into code. I also researched the meaning of a trademark and how to avoid unknowingly publishing someone else’s protected content.

10/18/18

I met with Dr. Nakra today and we discussed RFID technology. I learned that RFID is able to pick up signals without direct contact, and since spice bottles are small and close together, it might cause a mix up with which spice ID is being picked up by the base sensor. I plan to reach out to Dr. Nakras husband to see if he has a workaround to this or another direction he can point me in.

 

Research Plan

History of the Field

  • Spices native to certain regions or continents
  • Flavor profile analysis
  • Traditional seasoning and spice blend recipes
  • Any patents or similar physical computing project?

Current State of the Field

  • App 1: virtual list of all spices in your possession
  • App 2: herb, spice, and seasoning recommendations for over 400 dishes, as well as very detailed passages for each spice (this one seems to be the most developed and informative)
  • App 3: alphabetical list of all the spices in the world with flavor profiles and their uses
  • No such physical computing project found thus far

Other Things to Research

  • Arduino
  • Autodesk Tinkercad
    • Codeblocks
    • C++
  • How to hook up code to LED lights
    • What color?
  • How to woodwork
    • Average size of spice racks
  • Any sort of open source voice recognition software
    • Alexa compatible?
  • Where will the user interface be displayed?
    • Tablet screen  physically connected to wires
    • Tablet screen or phone connected by wifi, bluetooth(?), or other wireless connection
      • If phone, where will this app be hosted?
      • Is this even possible??
  • What teachers can I reach out to for help?
    • Digital fabrication IMM course
  • Can I find some of these LED lights or electronic parts on campus?
  • Ask around which spices people use the most – conduct possible survey of college cooking/spice usage

Finalized Concept

My senior thesis is a combination of two things I am passionate about: cooking and technological innovation. I plan to make an interactive spice rack that allows cooks of all levels to explore the art of blending spices. I thought of this idea while, unsurprisingly, cooking! As a college student off the meal plan for the first time, I am finally learning the fundamentals. This has presented me with a number of challenges, including being able to layer spices correctly. I had no clue there were so many flavor profiles and ways to intermingle them. One fruit tart recipe I followed called for cardamom, a spice whose name I’ve heard, but always thought was a flowering plant found at a nursery (I was confusing carnation + mum). I was reluctant to add this to the mixture because to me, adding a spice in a sweet dessert that wasn’t cinnamon or nutmeg sounded catastrophic. Eventually, I mustered up the courage and funds to purchase cardamom and use it in the tart. It had a pleasantly warm and floral smell that married well with the flavor of sugared berries. This find led me to research other kinds of spices and their flavor profiles. Sweet, nutty, piney, bitter, herbal, spicy, woody. The list goes on and on. Spice is the heart of a meal; it can either make or break the taste, as well as shape its flavor style. Why leave spice layering to professional chefs? It’s time for a creation that will bring this life skill to the home kitchen and spice up every cook’s life.

Two of the biggest challenges I faced while cooking were timing and multitasking. Organization is key when it comes to speed. No one wants to spend valuable time scouring the cabinet for the correct spices. Chances are, I am racing the clock to leave on time or hurrying before my family gets home and finds other snacks to eat instead. I also found it hard to remember ingredients and steps listed in a recipe. I was constantly running back to my book or phone to reread a section. For these reasons, I want to create an interactive spice rack that utilizes visual cues that immediately direct my eyes to where I should be looking. If I want something hot and spicy, every spice on the rack with a matching flavor profile will light up. This way, I will have an easier time finding what I need. This feature also allows an exploratory experience, allowing the chef to truly relish in the thought process of cooking instead of rushing to get it done.

Cooking a delicious meal through exploration and discovery is one of the most satisfying experiences as an amateur cook. Recently, I purchased some riced cauliflower for the first time and decided to make a loose version of pork fried rice with ginger. The second time I made riced cauliflower, I added cilantro and lime instead for a Chipotle-inspired meal. One or two ingredients can make all the difference and drastically change the style of it. That is why I would also like to implement a sort by geographical region in my spice rack. This way, cooks can easily find the spices that are common in cultural recipes from around the world and use it as inspiration for their dishes. Along the way, if I discover a spice I like or a combination of spices that pair well with a dish, I want to have the capability to save them in the system so I have access to them for later use.

In summary, the coded interface should primarily sort spices and herbs by flavor and geographical region. It should also save user-favorited spices and groups of spices for future reference. Secondary features include making flavor blend suggestions (sweet pairs well with spicy) and spice pairing suggestions (cumin and turmeric are usually paired together). A tertiary feature I am considering is providing possible spice blend recipe combinations like Chinese 5 spice, chili powder, curry powder, apple pie spice, and baharat. This requires including less used spices on the spice rack and I am not sure yet if there will be enough space.

Next, I plan to outline hardware requirements for the interactive spice rack. Obviously, I will need to present my project with some spices. I am currently considering building a 4×3 wooden rack that will hold 12 of the most common spices and herbs I’ve come across in my research: basil, black pepper, cayenne pepper, cinnamon, cloves, cumin, ginger, nutmeg, oregano, paprika, rosemary, and thyme. This list is subject to change or expand throughout the course of my research. I will build the LED lights into the spice rack and hook them up to the coded interface that will decide whether the lights will turn on or off beneath each spice. I do not know how this will work yet, so I am still considering my options. I have begun working in Autodesk Tinkercad to learn electronics and circuits. I also need to look into Arduino hardware and see if that could be implemented in my project. As for the user interface itself, I am planning to have a screen that the user can interact with to sort the spices. Thinking about how this will function in the kitchen made me consider a possible hands free option instead. Hands free appliances from electric mixers to Amazon’s Alexa have significantly optimized cooking, so finding a way to make spices light up through voice recognition would take this project to the next level.

I definitely plan to work on the software side of this project first to make sure the sorts work. A complete baseline project would be a simple application that performs the basic flavor profile and geographical region sorts of the spices. I will later hook the code up to the lights, then hook the lights up to the spice rack. And from there, hopefully, BOOM! The good kind, not the blowing up kind.

Revised Idea

Interactive spice rack

If I’m going to pick this idea, I plan to go all out with hardware and software. 

For the software part, I will probably end up making an app or program that can perform the following features: group spices based on flavor profile, geographical region, favorites, and possibly incorporate voice recognition for a hands-free solution. I will do some more research and ask experts what they believe would be a suitable language to code this in.

On the hardware side, I’d have to make the spice rack itself which I plan to make out of wood. Beyond making a personalized pen stand, I have no prior wood shop experience. Next I’ll have to rig up the lights to the program so it can respond with visual feedback. Not even sure how this works but there’s gotta be a wire for that. Probably involves a fair amount of circuitry but it can’t be that bad, right? Lastly I’ll need a touchscreen for the user to interact with the program. Either that, or some sort of voice recognition that is hooked up to the rack. I haven’t decided if I want some sort of visual app accompaniment or just the rack with voice recognition. Or both?