Considerations on generating facial nonmanual signals on signing avatars

Signing avatars continue to be an active field of research in Deaf-hearing communication, and while their ability to communicate the extemporaneous signing has grown significantly in the last decade, one are continues to be commented on consistently in user tests: the quality, quantity or lack-thereof of facial movement. Facial nonmanual signals are extremely important for forming legible grammatically correct signing, because they are an intrinsic part of the grammar of the language. On computer-generated avatars, they are even more important because they are a key element that keeps the avatar moving in all aspects, and thus are critical for creating synthesized signing that is lifelike and acceptable to Deaf users. This paper explores the technical and perception issues that avatars must grapple with to attain acceptability, and proposes a method for rigging and controlling the avatar that extends on prior interfaces based on the Facial Action Coding System that has been used for many signing avatars in the past. Further, issues of realism are explored that must be considered if the avatar is to avoid the creepiness that has plagued computer-generated characters in many other applications such as film. The techniques proposed are demonstrated using a state-of-the-art signing avatar that has been tested with Deaf users for acceptability.