Face Tracking

The Face Tracking section of the VRC Avatar Descriptor maps facial expressions to blendshapes and animator parameters. It supports the Unified Expressions standard from VRCFaceTracking (VRCFT), and integrates with VRChat's audio-driven visemes, eye look, and eyelids.

You can use face tracking to:

• Drive face tracking expressions across multiple meshes.
• Layer full or partial Blendshapes Sets. For example, add a set that re-authors the whole face with a happier look, or a set that only changes the eyelids.
• Use an animator state behavior to switch between or layer Blendshapes Sets while VRChat handles tracking and blendshape updates outside the animator.
• Use Parameter Behaviors to combine tracking inputs and apply the result to a blendshape, a Blendshapes Set's weight, or an avatar parameter.
• Forward calculated values to the animator to keep using animator-based techniques without syncing those parameters.

The same system also supports advanced viseme and automatic eye movement setups without requiring face tracking hardware.

Getting started

The following is a simplified guide for setting up an avatar that has face tracking blendshapes already available on one or more of its meshes:

1. Select your avatar and open its VRC Avatar Descriptor.
2. Expand Face Tracking and click Enable.
3. Enable Use All Tracking Keys if the avatar supports face tracking from a webcam or OSC. Leave it unchecked for an avatar that only uses advanced visemes or automatic eye movement.
4. Use Add Mesh to assign the Skinned Mesh Renderers whose blendshapes you want to control. Face Tracking drives blendshapes with matching names equally on all meshes.
5. Configure LipSync and Eye Look to route those features through Face Tracking as needed, so their mappings are available for Auto-Fill Blendshapes.
6. Expand Blendshapes Sets. The SDK names the initial Blendshapes Set unnamed1 and gives it a Default Weight of 1. You can rename it using Blendshapes Set Name.
7. Click Auto-Fill Blendshapes to map blendshapes with names that match common face tracking standards to Tracking Keys, then review the results.
   • To fine-tune a mapping, choose a different blendshape from a Tracking Key's Blendshape dropdown, or adjust its Multiplier.
   • Not every Tracking Key needs a blendshape, and several Tracking Keys can share the same blendshape. Tracking Keys follow the Unified Expressions standard, which serves as a superset of various face tracking standards, so most avatars only support a subset of them.
8. Use Build & Test to try the avatar in VRChat with your tracking source.

You do not need custom synced parameters for basic face tracking. See Networking and synced parameters.

LipSync and Eye Look

The Avatar Descriptor's LipSync and Eye Look settings determine how those features use Face Tracking. Enable the Face Tracking section and configure its mappings when selecting these modes.

LipSync Mode

Set LipSync > Mode to Viseme Blend Shape Face Tracking to route audio-driven visemes through the Face Tracking system. Configure the viseme mappings in each Blendshapes Set.

This mode allows you to:

• Apply visemes to multiple meshes in the Face Tracking Meshes list.
• Adjust the Blendshape and Multiplier for each viseme in each Blendshapes Set.
• Balance audio-driven mouth movement with tracked mouth movement using Viseme Dampen.
• Forward individual viseme strengths to animator parameters through Parameter Behaviors.

The Viseme_* inputs are individual float strengths from 0 to 1. They are separate from the built-in Viseme animator parameter, which identifies the current viseme.

Eye Gaze Mode

Enable Eye Look and choose an Eye Gaze Mode:

Mode	Behavior
Bone Rotation	Uses the configured eye bones and rotations.
Face Tracking	Uses eye direction mappings in the Face Tracking section.
Both	Uses bone rotation and Face Tracking eye direction mappings together.

For example, use Both to rotate the eye bones while also driving blendshapes that adjust the surrounding face to match the gaze direction. Map the EyeLook* keys in your Blendshapes Sets for the blendshape-driven part of the setup.

Gaze follows eye tracking when available and uses simulated eye movement otherwise.

Eyelids Type

Set Eye Look > Eyelids Type to Face Tracking to handle eyelid closing and opening through the Face Tracking section. Map EyeClosedLeft and EyeClosedRight to the appropriate eyelid blendshapes.

Use All Tracking Keys

Use All Tracking Keys exposes the full set of Unified Expressions keys and identifies the avatar as supporting full face tracking in VRChat, including the Avatar Details UI page where it will add an icon indicating face tracking support.

Leave this unchecked if you only need advanced viseme or automatic eye movement features. With it unchecked, you can still use:

• Eye direction keys (EyeLook*).
• Blink keys (EyeClosedLeft and EyeClosedRight).
• The Voice input.
• Viseme mappings and Viseme_* inputs, when using Viseme Blend Shape Face Tracking lip sync.

For example, an avatar using multiple meshes for audio-driven visemes does not need to be marked as a full face tracking avatar.

Face Tracking Meshes

Add each Skinned Mesh Renderer that should receive face tracking blendshape updates to Face Tracking Meshes. The inspector supports up to 16 meshes.

Face Tracking matches blendshapes by exact name. A tracking key mapping or Parameter Behavior that targets a blendshape applies to every mesh in this list that contains that name. Meshes without that blendshape are unaffected.

You do not need a mesh if you only use Parameter Behaviors to forward values to the animator.

Viseme Dampen

Viseme Dampen balances audio-driven visemes with tracked mouth movement to help prevent the combined expressions from overdriving the face. It requires Use All Tracking Keys and LipSync > Mode > Viseme Blend Shape Face Tracking.

Mode	Behavior
None	Applies no dampening.
Dampen Visemes	Strong mouth tracking activity reduces associated audio-driven viseme activity.
Dampen Tracking	Strong audio-driven viseme activity reduces associated mouth tracking strength.

Use Dampen Amount (0 to 1) to adjust how strongly the selected mode reduces the other source. Test while speaking and making facial expressions to choose an amount that suits the avatar.

Blendshapes Sets

A Blendshapes Set contains mappings from tracking keys and visemes to blendshape names. Create multiple Blendshapes Sets to map the same tracking input to different blendshapes or use different Multiplier values. A Blendshapes Set can contain mappings for the whole face or just part of it.

Property	Purpose
Blendshapes Set Name	The name used by Parameter Behaviors and animator state behaviors to reference this Blendshapes Set. Give each Blendshapes Set a unique name.
Default Weight	The initial weight of the Blendshapes Set when the avatar loads, from 0 to 1. A weight of 0 disables its mappings.
Tracking Key	The tracking key or viseme that drives the mapping.
Blendshape	The blendshape name driven by the Tracking Key, across all Face Tracking Meshes containing that name.
Multiplier	Scales the strength of an individual mapping. 1 uses the input's original strength.

For example, copy a Blendshapes Set and increase its Multiplier values to exaggerate the tracked movements. A Multiplier of 10 amplifies small input movements, reaching full strength sooner. You can also create a Blendshapes Set containing only eyelid mappings and use it alongside a Blendshapes Set with mappings for the whole face.

Layering Blendshapes Sets

Multiple Blendshapes Sets can be active at the same time. When tracking mappings in multiple Blendshapes Sets drive the same blendshape, the highest value wins. Face Tracking does not add the contributions together. Viseme mappings also use the strongest contribution across Blendshapes Sets.

For example, if two tracking mappings contribute 0.3 and 0.7 to the same blendshape, the result is 0.7, not 1.0. A Blendshapes Set containing only eyelid mappings does not replace unrelated mappings from other Blendshapes Sets. The Parameter Behavior targets Limit Blendshape (Min) and Limit Blendshape (Max) can constrain the final value.

Changing Blendshapes Set weights

Use the VRC Animator Face Tracking Blendshapes Set state behavior to change the weights of one or more Blendshapes Sets when entering an animator state. Enter the Blendshapes Set Name for each entry, then configure:

• Type > Value and Apply Weight for a constant weight.
• Type > Parameter and Source Parameter to read an animator parameter when the state is entered, with optional Convert Range remapping.
• A Lerp Speed for a smooth transition to the requested weight.

For example, create Blendshapes Sets named Happy and Angry. On entering a happy state, use Apply Weight to set the weight of Happy to 1 and Angry to 0. Reverse those weights in the angry state. This lets the animator select the active Blendshapes Sets while Face Tracking samples tracking input and applies blendshapes.

To drive a Blendshapes Set's weight continuously from tracking data, it's best to use a Parameter Behavior with Apply Blend Set Weight instead of an animator state behavior. However if necessary, it's possible to trigger an animator state behavior repeatedly to update the weight continuously from the animator.

Parameter Behaviors

Parameter Behaviors combine tracking inputs and apply the result to a target. They let you perform calculations without building equivalent animator layers or blend trees.

To configure a behavior:

1. Add an entry under Parameter Behaviors.
2. Choose a target mode and enter or select its target.
3. Choose a combine mode.
4. Add keys under Input Tracking Keys, or enter a calculation in Expression for Custom mode.

Targets

Target mode	Result
Forward to Animator	Applies the result to the named Avatar Parameter, which updates matching parameters in the animator. Use a Float parameter to preserve the calculated strength.
Apply Blend Set Weight	Uses the result as the weight of the named Blendshapes Set. "Blend Set" is an abbreviation for "Blendshapes Set" here.
Apply to Blendshape	Applies the result directly to the named blendshape on all matching Face Tracking Meshes.
Limit Blendshape (Min)	Sets a lower limit for the named blendshape's final value.
Limit Blendshape (Max)	Sets an upper limit for the named blendshape's final value.

Apply to Blendshape, Limit Blendshape (Min), and Limit Blendshape (Max) use normalized values: 0 to 1 corresponds to Unity blendshape weights of 0 to 100.

For these targets, you can optionally assign a Blendshapes Set to the Parameter Behavior. The behavior then scales with that Blendshapes Set's weight, so you can adjust its strength or turn it off the same way you control Blendshapes Set weights. Leave the field empty to always apply the behavior at full strength.

Limits are useful for shapes that would clip when fully active together. For example, a Limit Blendshape (Max) behavior using 1 - JawOpen can reduce a conflicting shape's allowed strength as the jaw opens.

Combine modes

Combine mode	Calculation
Max	Takes the largest input value.
Min	Takes the smallest input value.
Add	Adds the input values.
Multiply	Multiplies the input values.
Custom	Evaluates a math expression.

For Max, Min, Add, and Multiply, each entry under Input Tracking Keys has an Invert option that replaces its value with 1 - value before combining it. Their results are clamped to 0 to 1.

For example, choose Forward to Animator, name the target CheekPuff, select Max, and add CheekPuffLeft and CheekPuffRight. The animator receives one value representing the stronger cheek puff.

Custom expressions

In Custom mode, the Expression field accepts:

• Unified Expressions tracking key names, such as EyeClosedLeft or JawOpen.
• Voice, the built-in voice volume input from 0 to 1.
• Individual viseme inputs from 0 to 1: Viseme_sil, Viseme_pp, Viseme_ff, Viseme_th, Viseme_dd, Viseme_kk, Viseme_ch, Viseme_ss, Viseme_nn, Viseme_rr, Viseme_aa, Viseme_e, Viseme_ih, Viseme_oh, and Viseme_ou.
• Numbers, parentheses, and the operators +, -, *, and /.
• The functions abs(value), min(a, b), max(a, b), and clamp(value, min, max).

Use the exact input names and capitalization shown above. Viseme inputs require Viseme Blend Shape Face Tracking lip sync. Hardware tracking keys require Use All Tracking Keys.

The following table shows example expressions:

Expression	Result
max(EyeClosedLeft, EyeClosedRight) * 0.5	Half the strength of the more closed eye.
max(CheekPuffLeft, CheekPuffRight)	The stronger cheek puff.
clamp(JawOpen * 2, 0, 1)	Amplified jaw opening, limited to full strength.
max(Voice, Viseme_aa)	The larger of voice volume and the aa viseme strength.
EyeClosedLeft - EyeClosedRight	A signed difference between the two eyelids.

With Custom and Forward to Animator, the result is clamped between -1 and 1. Results for Apply Blend Set Weight, Apply to Blendshape, Limit Blendshape (Min), and Limit Blendshape (Max) are clamped between 0 and 1.

Networking and synced parameters

VRChat synchronizes the full set of Unified Expressions tracking data to remote viewers. Remote avatars run the same native mappings and Parameter Behaviors using the received data, including forwarding calculated values to their animators. This does not consume your custom synced parameter budget.

Like NetIK (networked body motion), face tracking synchronization can be lossy, with fidelity varying based on factors such as distance from the avatar. Nearby avatars receive higher-fidelity tracking. Conceptually, network degradation resembles less accurate tracking hardware. If an avatar is set up to be flexible across a range of tracking hardware quality, it would also be flexible across a range of network conditions.

If a particular value must avoid the face tracking system's dynamic lossy synchronization, forward it to an animator parameter and enable Synced for that parameter in the avatar's Expression Parameters asset. That parameter uses the normal synced parameter budget and synchronization rules. Most face tracking setups do not need this.

Tracking sources

Selfie Expression

VRChat's Selfie Expression drives Unified Expressions using a webcam on desktop PC or the camera on Android phones and tablets. You can enable Selfie Expression in the "Camera Tracking" section of the "Tracking & IK" main menu settings page.

OSC

External tracking applications can send Unified Expressions data to /tracking/face/v1. See OSC Face Tracking for the blob format and the complete tracking key order.

Performance

Face tracking contributes two statistics to your avatar's Performance Rank: Face Tracking Operations and Forwarded Parameters.

Use Blendshapes Sets and Parameter Behaviors to drive blendshapes directly, and use Forward to Animator only for the values your animator needs. Each blendshape mapping is counted for every mesh in Face Tracking Meshes containing the target shape, so adding meshes and duplicate Blendshapes Sets increases the operation count even when some Blendshapes Sets have a Default Weight of 0.