Kittch AR Cooking

This is a 0-1 project to showcase Qualcomm’s latest microchip processer for AR glasses, which is also the first public-facing AR product that allows users to cook hand-free and follow instructions at the same time.

I collaborated with clients from Qualcomm & Kittch to design this brand-new experience in 6 months and led the research, design and iteration process. I also introduced spatial design tools and workflow to the company.

Context

Project Challenges

My Contribution

Owned the design from definition to UX design.
Collaborated with engineers to test in AR glasses.

Thanks to the latest technical breakthrough by Qualcomm. AR head-mounted-device can render simultaneously with mobile devices thus increasing the wearing duration. Kittch decided to collaborate with Qulacomm and Trigger XR to utilize this new technology to bring all their recipes to mixed reality. How to bring users a better experience utilizing the power of mixed reality?

Practical
This product is the 1st public-facing AR cooking.

Technical
Qualcomm provides the 1st Co-processor.

Internal
This is the1st AR headset project in the company.

Qualcomm released the latest AR processor in 2023 & Kittch decided to utilize it to bring out the future AR cooking experience.

TIME

6 months
11. 2022 - 05. 2023

TOOLS

Adobe XD
Adobe After Effect
Bezi
ShapesXR
Unity

TEAM

Me (UX Lead)
Project Manager
UX Researcher
Software Engineer
UI Designer

Research

Designed user research to define pain points for traditional cooking.

Since this is a new type of product in the market, to better understand the project goals, I collaborated with the research team to collect the top 3 cooking pain points for users following a recipe to cook.

Key Insights

"It's frustrating when I have messy or sticky fingers, and I need to touch the screen to follow the next step in the recipe."

"I simply just cannot find my phone. I have to keep going back and forth just to check the recipe"

"Cooking a complex recipe requires juggling multiple tasks and timers, and it's really hard to keep everything on track without getting overwhelmed."

Design Solutions

Pain Point 1: Messy or sticky fingers

With AR glasses, users can control the cooking instructions by air taps, instead of physical taps with messy or sticky fingers.

Mobile User Flow:
Currently, users follow video instructions on the Kittch mobile app.

When users follow instructions on mobile devices, the Kittch app provides video-based instructions. Users start by browsing different recipes, and then viewing recipe details. Once the cooking starts, users simply follow the video instructions in the full-screen mode.

AR User Flow:
This experience can be hand-free if all the interactions happen in AR.

If the entire user journey happens in AR instead of mobile, users will follow the same user journey in 3D. In this case, air taps help avoid messy interactions on mobile screens.

Find the sweet spot for AR value in the cooking experience.

Another approach to improve the experience is to design fewer buttons. I started to think maybe AR is not the single solution for all requirements. By revisiting the user journey map, AR interactions can be limited to where they are really necessary.

Introduce AR to the experience when users start to cook

When users still have clean hands, they can view recipes from mobile, which is the platform that they are more familiar with. During cooking, users can view the instructions in AR to avoid touching devices with messy hands.

Pros and Cons Comparison:
AR interactions bring new problems to the user journey.

Comparing both user flows, although AR interactions are clean and more readable, it brings new problems.
Users are not familiar with AR headset and AR interactions are hard to proceed. Moreover, the battery life of AR headsets is less than half an hour, and the device overheats easily. A shorter AR user journey is needed.

Pain point 2: Fixed location recipe

With AR glasses, instructions can be in front of users view and become more accessible and dynamic.

Design a smooth transition between the two types of panels

This fixed panel will shrink to a mini menu when users look away. While the full panel contains all the necessary information, the mini panel only contains the instructions for the current cooking step.

When users move to different places in the kitchen, an instructional panel always follows users.

I worked with prototypes to test different following modes in AR, if the AR panel follows users when they move around in the kitchen, instructions can be more accessible. However, this following panel can block users’ views sometimes, so the content in this following panel needs to be limited. I designed 2 types of panels in the experience to convey information.

/ Fixed panel: Full content
/ Following panel: limited content

Fixed panel

Following panel

The panel changes to a mini menu following users when they look away. This panel contains minimum information that users need to keep track of.

A fixed panel stays at an assigned location. It contains all the information users need for cooking.

Design Solutions

Pain point 3: Simultaneous timer control

With AR glasses, users can set up multiple timers in different physical locations and be aware of timing at any location.

Spatial timers give users a better track of spatial simultaneous timing.

I designed spatial timers to the experience that can be attached to the cooking areas. This gives users a better track of how food is being cooked in different places.

/Mobile timers: when cooking takes too long, users will take off their glasses.
/Spatial timers: when users are busy cooking, spatial timers are visually intuitive.

Design System: UI Components and Assets in mobile and AR

To allow future implementation of more recipes, a design system was designed and it includes both mobile interface elements and AR components.

Design Solutions

Outcome

Traditional Cooking Experience

  • The mobile screen is too small for text-heavy instructions

  • Messy hands make it hard for users to interact with digital screens.

New Cooking Experience

  • A big screen in AR.

  • Air tapping won’t be affected by the messy cooking.

Outcome

Success Metrics

The iteration created a more intuitive and user-friendly experience, which was measured by several factors, including mistake number per button, AR interaction number, overall rating, and the drop rate.

This experience was live on AWE 2023 and was prominently featured in Qualcomm's keynote presentations.

It is also one of the most popular experiences in the Qualcomm booth. 

This project initiated a long-term collaboration between Trigger XR and Qualcomm.

Achievement

AWE 2023 Showcase and Partnership Between Trigger XR and Qualcomm