Smart Home AR App

UX Design | Interaction Design | Voice Design | AR App

UX Design | Interaction Design | Voice Design | AR App
Adobe XD | Amazon Sumerian | Unity | Miro | Keynote
May 2021


Here's a prototype of my smart home augmented reality application.


At first, please allow me to recap the main concepts and inspirations of this AR project.

During the pandemic, working from home changes the work-life style that people used to be. I always thinking if there is an app that could keep us to have a more qualified and better work-life balance. Thus, my goal is to build up a smart home AR app, which can help users live in a more convenient and relaxed way.

With the rapid development of augmented reality, an app that could be used in our daily life could be a life-changing experience. I've collected different kinds of AR designs as inspiration.

For instance, in a smart office, users can get navigation and reserve their group room using the app. Augmented reality applications have a huge impact on e-commerce, for instance, users can pick their favorite artworks or books more easily. Besides, the 3D UI elements play an important role in XR apps, it's more immersive and interesting to interact with. Just to name a few, tons of brilliant and talented designers come up with ideas and concepts that inspired me.



As stated before in my project proposals, here's the basic user flow for a smart kitchen. Users can use the app to scan the kitchen, they could check the ingredients in the fridge, whilst recommended recipes would be provided. Users can check the weather or to-do lists for the day, etc. There would be more features that could be developed in this project.

Overall, I would willing to build up some fundamental interactive features that could be applied in the upcoming AR app. For instance, the tabs or menu for available ingredients; a scrollable menu of recommended recipes; a floating window for the weather forecast or to-do list for the day, etc. Users can scan the displays in the kitchen to get those features that could enhance their work-life balance. They could check the food, search the recipes, catch up on the daily routine. It would interesting to add solid features could support and assist users to live a better and easier life.


The anxious and steeping learning curve cannot be the reason to give up concepts. Starting from aspects I could achieve, I picked a reference as background, designed the basic UI elements and fundamental interactions as my first version prototype. It's an interesting process to twitch or rotate the floating menus to match the real-life perspective.

Indeed it has limitations to present my ideas, still, the user flow could be presented through the prototype.

I sent my first version of the prototype to my friends and colleagues, those are the feedback and initial ratings I received. They brought up some constructive feedback, those are things I would keep in mind and make changes and improvements in my further project.

While I had a hard time trying to figure out how to imply the concepts to Sumerian on such short notice. I was amazed and inspired by the Sumerian host feature, thus I came up with the idea to set up a room to present my prototype to clients/stakeholders.

Thanks to the set-up assets in the Sumerian, I set up the scene with in-wall television and projection screen. The virtual host can track the camera with basic gestures. I would build in the voice scripts in my later version. While I conducted the second round of user testing, they thought the concept of virtual presentation was creative and interesting. From my perspective, I would willing to keep learning and explore more features in Amazon Sumerian.

In my Amazon Sumerian version, I applied the scripts for the host, connected with basic gestures to make her look natural. The orbit camera can change our view of perspective in the scene. Follow by her instructions, users could tap the button of the prototype to trigger the interactions.

Then, I took a step further adding a voice control feature to my prototype in Adobe XD, also experimented with trying to insert the UI element in the Unity scene. I will keep exploring the possibilities of my smart home XR project, trying to utilize my design skills and knowledge to see how far it could be go.


In general, I'm happy I took the shot and give it a try. The whole process is challenging as well as rewarding. The most important thing I've learned in this project is that we must stay curious about new things. With the rapid development of the world, everything is changing, so as the designing tools. Especially in the XR field, everything created or about to create is definitely mind-blowing and inspiring. As designers, we need to step out of the comfort zone, get over the learning curve, and adapt to new technologies.

Also, we need to keep in mind that not all crazy ideas could be achieved at the moment, but we can still be creative and bold to present those concepts. As long as we start doing it, there are always possibilities to have a breakthrough.

Last but not the least, don’t be afraid of critiques, they help you to improve. It feels so good to see something you've created had a tiny bit of elements you have come up with before.