Face Incognito

Unity
Challenge
The brief challenges me to create an interactive immersive experience according to the theme of “transmogrify”.
Result
I built an AR Face Mask app with Unity in three weeks, from knowing nothing about Unity to being able to build an app with it.

Full Process

Primary Research

I have gathered inspirations from films series such as Black Mirror and artists like “Annegret Soltau” and “Maurizio Anzeri” to have a better understanding in technology and visual appearances. These two artists have interact with appearance with collage and sewing to create a new identity.
I took inspiration from animation artist “Cyraik” on the theme "Transmogrify".

Experiment - Wekinator

With Wekinator, I used machine learning to create interaction. With the input, I used face tracking position and for the output I used FM synth. With training the Wekinator with my position data and the different frequency of the synth wave, I will be able to change the synth wave by changing my position in front of the camera.

Image Tracking - Vuforia

One step at a time, before getting into face tracking, I started with image tracking. I use Vuforia as the platform to create a whale swimming on the image. I used an as the image material and placed the whale swimming under the ImageTarget, this will make the animation to trigger when I show the camera the image.

Face Tracking - AR Foundation

I have use AR Foundation and “Face Manager” to be a Game Object. For the mask I use Photoshop to create a png file to import it on to the material in Unity. The problem of using AR Face Tracking in Unity is it only works on iPhone X above.

Materials

I have created multiple filters to create from personal details to face collages. From the personal details so the user can see what they actually look like on social media.

Results