You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Visemes, short for "visual phonemes", are blend shapes that represent the visual aspect of phonemes during human speech. They are used for lip syncing, the process of matching a character's mouth movements to spoken words.
104
104
105
+
### Viseme Blend Shape Naming
106
+
105
107
The following viseme names are defined by G4MF, as a superset of common viseme sets found in other standards linked below:
106
108
107
109
| Viseme Name | IPA Phonemes | Example Words | Description |
@@ -127,7 +129,7 @@ The following viseme names are defined by G4MF, as a superset of common viseme s
127
129
128
130
Each example word corresponds to the listed phonemes in order. Some sounds may be merged in common English accents, in which cases a language or accent is specified in the table, otherwise the examples are based on General American English as spoken in the 21st century. For RR, those sounds are allophonic in English and most languages, therefore only one example word is provided, but the specific choice may vary by accent. For a reference on what each phoneme sounds like, see the "IPA Chart" link in the "Viseme References" section below.[^1]
129
131
130
-
Usage rules:
132
+
### Viseme Blend Shape Usage Rules
131
133
132
134
- If a blend shape (morph target) exists for one of these listed visemes, it MUST have its name set to the corresponding name from the table above in order to be used as that viseme in an application's lip syncing. The names are case-sensitive. This ensures that applications can reliably identify and use the visemes by their names.
133
135
@@ -139,13 +141,15 @@ Usage rules:
139
141
140
142
- Visemes may be used for purposes other than lip syncing. For example, "VisemeWW" may be used as part of a "pog champ" facial expression.
141
143
142
-
The G4MF viseme blend shape naming is designed to be close to other existing standards. For convenience, here are some other standards and how to convert them to G4MF, with names from those standards quoted in `"`, and the names from G4MF in backticks:
144
+
### Viseme Name Conversion
145
+
146
+
The G4MF viseme blend shape naming is designed to be close to other existing standards. For convenience, here are some other standards and how to convert them to G4MF, with names from those standards quoted in `"`, and the names from G4MF in backticks (formatted as code):
143
147
144
148
- To convert from VRM lip sync visemes: Capitalize the viseme names, and add the `Viseme` prefix, then the names match.
145
149
146
150
- To convert from MPEG-4 FBA visemes: Replace "E" with `EE`, capitalize the viseme names, and add the `Viseme` prefix, then the names match.
147
151
148
-
- To convert from VRChat visemes: Replace "e" with `EE`, replace "i" with `IH`, replace "o" with `OH`, replace `u` with `OU`, capitalize the viseme names, and add the `Viseme` prefix, then the names match.
152
+
- To convert from VRChat visemes: Replace "e" with `EE`, replace "i" with `IH`, replace "o" with `OH`, replace "u" with `OU`, capitalize the viseme names, and add the `Viseme` prefix, then the names match.
149
153
150
154
- To convert from Meta Horizon visemes: Replace "E" with `EE`, replace "I" with `IH`, replace "O" with `OH`, replace "U" with `OU`, capitalize the viseme names, and add the `Viseme` prefix, then the names match. Meta Horizon matches VRChat except for capitalization.
151
155
@@ -158,12 +162,14 @@ The G4MF viseme blend shape naming is designed to be close to other existing sta
- ARPAbet Phoneme Set https://en.wikipedia.org/wiki/ARPABET and http://www.speech.cs.cmu.edu/cgi-bin/cmudict
160
164
161
-
## Face Tracking Blend Shape Naming
165
+
## Face Tracking Blend Shapes
162
166
163
167
Face tracking is the process of using a camera to track a person's facial movements, and applying those movements to a character/avatar's face. While visemes are used for lip syncing to audio, face tracking is used to mimic a much broader range of facial expressions. The `Face*` blend shapes defined by G4MF are intended to be used for this purpose, though they can also be used to adjust the mesh around the eyes when using eye tracking, or used for manual animation.
164
168
165
169
The standards for face tracking vary widely, unlike skeleton rigs and visemes which are relatively consistent. G4MF's face tracking blend shape names are designed to be a superset of all common face tracking standards. Since "Unified Expressions" is already a comprehensive standard that includes the functionality of all other common standards, G4MF's face tracking blend shape names are based on Unified Expressions, except with a `Face` prefix added to each name for clarity, and also the list of names has been alphabetized.
166
170
171
+
### Face Tracking Blend Shape Naming
172
+
167
173
The following table includes columns for many common face tracking standards, and how they map to G4MF's face tracking blend shape names. `~` indicates an indirect non-1-to-1 mapping, which is described in the Unified Expressions documentation.
168
174
169
175
| G4MF Blend Shape Names | Unified Expressions | ARKit | SRanipal | Meta Movement (Quest Pro) |
@@ -255,7 +261,7 @@ The following table includes columns for many common face tracking standards, an
255
261
256
262
This table lacks some of the nuances described in the Unified Expressions documentation for how these map to other standards. For more detailed documentation on each of these blend shapes, read the Unified Expressions documentation linked below.
257
263
258
-
Usage rules:
264
+
### Face Tracking Blend Shape Usage Rules
259
265
260
266
- If a blend shape (morph target) exists for one of these listed face tracking shapes, it MUST have its name set to the corresponding name from the table above in order to be used as that shape in an application's face tracking. The names are case-sensitive. This ensures that applications can reliably identify and use the face tracking shapes by their names.
0 commit comments