• All submissions to this site are governed by Second Life Project Contribution Agreement. By submitting patches and other information using this site, you acknowledge that you have read, understood, and agreed to those terms.
Issue Details (XML | Word | Printable)

Key: VWR-2629
Type: New Feature New Feature
Status: Reopened Reopened
Priority: Normal Normal
Assignee: Unassigned
Reporter: Ina Centaur
Votes: 11
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
1. Second Life Viewer - VWR

Dynamic RealTime Lip-Synching

Created: 02/Oct/07 04:23 AM   Updated: 22/Jun/08 12:12 PM
Return to search
Component/s: Avatar/Character
Affects Version/s: None
Fix Version/s: None

Issue Links:
Relates
 


 Description  « Hide
Ansatz: With certain preconfigured basis vectors or morph-points assigned to key parts of lips, eyes, and eyebrows, data from voice chat can be fed directly to allow for real time client-side facial expressions and lip-synch on individual avatar baked texture maps.

Wouldn't it be cool if the viewer allowed for dynamic real time lip-syncing based on morphing 2D data according to voice?
RealAllusions' CrazyTalk provides an example: http://reallusion.com/crazytalk, and here's another one showing her the points relate to the facial bases: http://inacentaur.com/crazyina.html



 All   Comments   Change History      Sort Order: Ascending order - Click to sort in descending order
[ Permlink | « Hide ]
lauren weyland added a comment - 19/Nov/07 07:45 PM
This is a wonderful idea. I think it would move Second Life into a new realm of virtual reality. I highly recommend it

[ Permlink | « Hide ]
Rob Linden added a comment - 27/Mar/08 10:16 AM
Sorry for marking this earlier issue as a duplicate, but the later issue has a patch attached.

VWR-4794 Babble loop for voice visualization


[ Permlink | « Hide ]
Ina Centaur added a comment - 28/Mar/08 01:31 PM
The other issue has an (unimplemented) patch, but please note that different lip-synch technologies are mentioned. This issue refers to implementing CrazyTalk by RealAllusion, which also accounts for emotional nuances in facial expressions based on voice.

[ Permlink | « Hide ]
Oneironaut Escher added a comment - 22/Jun/08 02:45 AM
Random stumbling through old Jira. . . . would this be implementable now with a client side viewer solution? I think both this and speech to text are feasible in a custom client, and the nature of the two should make it easy to make progress on both simultaneously.

I think Speech-to-Text would be very useful to blend people using voice and those not. Would be nice to have a text record of meetings again. Would also increase useability for the hearing impaired.


[ Permlink | « Hide ]
Ina Centaur added a comment - 22/Jun/08 12:12 PM
Lip-sync is implemented, but not yet facial expressions!


Powered by Atlassian JIRA the Professional Issue Tracker. (Enterprise Edition, Version: 3.13.2-#335) - Bug/feature request - Atlassian news - Contact Administrators