Custom Facial Expressions for Optimal Mocap Results

The 57 Control Names (Face IDs) received from the iPhone X may not always be suitable for all types of characters. Sometimes, you may want to exaggerate the expressions for cartoon characters and ease the expressions for realistic human characters.

With the use of the Expression Mapping panel in the Facial Mocap feature, not only can you map the 57 Control Names (Face IDs) to CrazyTalk Animator's 60 facial morphs, but you can also set the weight and limitations for the 60 face morphs.

The benefits are:

Default 60-morph capture profiles are optimized for iPhone X's true-depth camera.
Defined expression style with morph sliders can be saved as Custom Capture Profiles.
Take advantage of the Search function to quickly access the needed sliders.

User Interface Introduction

Click the Expression Mapping button to show the Expression Mapping panel.

A	Server Name	Displays the current server name.
B	Current Profile Name	Displays the current profile name. When you load a new profile, then the name will be changed as well. The default profile (.json) files exist in the following path. - Windows: C:\Program Files\Reallusion\CrazyTalk Animator 3\APData\Default\ FacialMocap\MappingProfile* - Mac: Contents\APData\Default\ FacialMocap\MappingProfile
C	Control Name	The 57 Control Name Stream (Face ID) received from iPhoneX.
D	Search Expression	Type in a keyword to display only the sliders whose names contain the same string as the keyword.
E	Face tab	In this tab, all the 60 face morphs for the facial expressions are listed. You can then map the 57 Control Names with the 60 Face Morphs in this tab.
F	Hide Zero Value Sliders	Activate this box to hide the sliders whose value is zero in order to have a cleaner panel to facilitate observation and adjustments.
G	Load and Save Profile	You can click these two buttons to save or load custom profiles with different mapping combinations of sliders.
H	Bone Sliders tab	The Jaw and Head and Eye tabs can be used to set the bone movements for the jaw, head and the eyes.
I	Zero all and Reset buttons	Click the Zero All button to set all slider values to 0. Click the Reset button to reinitialize slider values set when the profile was saved.

Fine-tuning the 57 face IDs with 60 face morph sliders

Prepare a character. Make sure it has neither expressions nor lipsyncs.
Open the Facial Mocap panel and connect to LIVE FACE.
Click the Tracking Data Inspector button to observe the Face ID stream received from iPhone X.
You may find that the corresponding facial expressions for face IDs "eyeWide_R" and "eyeWide_L" are not ideal.
Click the Expression Mapping button to show the Expression Mapping panel.
Select the Face ID "eyeWide_R" item from the Control Name drop-down list.
Switch to the Face tab and find the Eye Enlarge Right expression slider. You may check the Hide zero value slider(s) box to hide the sliders whose value is zero, for a clearer view.
Drag the expression slider to the right to enlarge the size of the right eye.
Observe the mocap result in Preview mode.
Eye Enlarge Right = 0 Eye Enlarge Right = 200
Repeat the Steps 5 to 7 for the "eyeWide_L" item in the Control Name list and click the Save button to save the combination as your custom profile.

Configuring Facial Animation Setup

Through the Facial Animation Setup panel, you can customize the character expressions from each face feature to strengthen the mocap effect.

You may exaggerate the eyes' expression for the character in this example to create a more cartoon-like facial mocap result.

Close the Facial Mocap panel. Click the Composer button on the Functional Toolbar to enter the Composer Mode.
Note:
If you are using a G1 or G2 character, then you need to convert to G2+ facial system for the G1 or G2 character before you can start this section.
Click the Facial Animation Setup button on the Functional Toolbar to open the Facial Animation Setup Panel.
Select the Right Eye from the Face Part pane.
Select the second thumbnail (Eye Enlarge Right) in the Expression list.
Press down the Deform button to enter the Deforming mode. Drag the control points to enlarge the shape of the Deform box.
Click the Mirror Options button. Choose Mirror by Current Expression and then click Apply to copy the settings to the left eye.
Select the Left Eye from the Face Part pane and then the second thumbnail (Eye Enlarge Left) in the Expression list. You will find that the left eye is symmetrically enlarged.
Close the Facial Animation Setup panel and click the Back Stage button to switch to the Stage Mode.
Open the Facial Mocap panel again and observe the mocap result in Preview mode.
Before deforming the eyes. After deforming the eyes.

Editing Character Sprites

By using the Sprite Editor panel, you can replace the Sprite element for current character expression to enhance the facial mocap result.

You may customize the mouth sprite for the character in this example, to present a different visual effect during the facial mocap.

Close the Facial Mocap panel. Click the Composer button on the Functional Toolbar to enter the Composer Mode.
Select the Mouth sprite.
Click the Sprite Editor button to open the Sprite Editor Panel.
Select the Smile Open pose and click the Replace Current Sprite button.
Choose a prepared media file and click the Open button to replace the original one. The pose is instantly replaced with the new media.
Repeat the Steps 4 and 5 for the Show Teeth pose.
Close the Sprite Editor panel and click the Back Stage button to return to the Stage Mode.
Open the Facial Mocap panel again and observe the mocap result in Preview mode. Now the character has a bigger mouth with the teeth inside.
Before replacing the mouth sprite. After replacing the mouth sprite.


Before replacing the mouth sprite.	After replacing the mouth sprite.


Eye Enlarge Right = 0	Eye Enlarge Right = 200

Custom Facial Expressions for Optimal Mocap Results (New for v3.3)

Fine-tuning the 57 face IDs with 60 face morph sliders