Open topic with navigation

Adding or Modifying Lip-sync Keys

CrazyTalk provides special lip synchronization tools to make sure that the lip movements match the words being spoken by the virtual actor. A viseme is a generic facial image that can be used to describe a particular sound. It is the visual equivalent of a phoneme, or a unit of sound in a spoken language. Using visemes, CrazyTalk provides lip synchronization which can later be adjusted for different sounds as required.

The lip-synching keys are stored in the Lips track, which belongs to the Voice track. You may open it by clicking the button in the Voice track.

ClosedAudio Track and Lips Track

The Voice track stores the audio file from the Import Audio panel or the Voice Script Library. The audio data is displayed in a wave form. You may double-click anywhere on the track to load an audio file.

Click the button on the Voice track to show the Lips track. You may manually add lip-synching keys to this track by double-clicking on the cells in this track.

ClosedAdding or Modifying Lip-sync Keys

The lip synchronization effect is applied to only a small section of the script and indicated on the Lips track by a grey box. To access the Lips Editor panel, click the Lips Editor button in the Timeline.

ClosedAdding Lip-sync Keys

You may double-click anywhere on the Lips track to synchronize the lip movements in the Lips Editor panel.

  1. Select a viseme or phonetic sound from the various options displayed.
  2. Move the Expressiveness slider to adjust the strength of the lip movements to specify the intensity.

    "Ah" sound applied

    Expressiveness = 50

    "Ah" sound applied

    Expressiveness = 100

ClosedModifying Lip-synced Keys

You may move the lip-synched keys to other parts of the script as desired, or use the data editing tools to duplicate the lip movements.

To modify a lip-synched key, double-click on the target key to adjust the lip-synching settings in the Lips Editor panel.

ClosedConstraining Lip-sync

Sometimes the Auto Motion may affect the accuracy of the lips shape. You may follow the steps below to force the lip movements to sync with the sound.

  1. Given the settings in Auto Motion change the original lips shape when pronouncing the sound "B".
    2. The lips shape is affected by the Auto Motion. When the jaw moves down, the lips shape is not correct for a "B" sound.
  2. Double-click on the lip key "B.M.P" to access the Lips Editor panel.
  3. Drag the Expressiveness slider to enhance the result if necessary.
  4. Activate the Lip-sync constraint over the Auto Motion option.
  5. Click the Apply button to blend the lip keys you have modified with the entire Auto Motion for a more accurate animation result.
    6. After modification, the lips shape is constrained for the "B" sound.
    ClosedNote:
    • Once you click the Apply button, all of the lip keys in the Auto Motion clip will be affected. You may alternatively click the Apply button in the Auto Motion Settings panel for the same result.
    • Some visemes, such as "EE", "Ih", "Ah", "Oh", and "W_OO", will not be affected by voice strength or any adjustment in the Auto Motion.