Uploaded image for project: 'Open Development'
  1. Open Development
  2. OPEN-366

Simplify Puppetry configuration through LEAP



    • Epic
    • Status: In Progress
    • Unset
    • Resolution: Unresolved
    • None


      The Puppetry LEAP API is going be simplified, to make it easier to use and explain, but also to make it easier to add requested features.

      (1) All existing LEAP commands (move, get_camera, send_skeleton, etc) will be removed.  In their place will be two commands: set and get.  To update any Puppetry configuration value use set.  To request data from Puppetry use get.

      (2) Inverse Kinematics (IK) targets will be explicitly differentiated from parent-relative constraints by introducing two distinct Puppetry config categories: Inverse_kinematics and Joint_state.  All Inverse_kinematics transforms will implicitly be in the avatar-frame whereas all Joint_state transforms will be in the Joint's parent-frame.   For example, to set an IK target for the WristRight joint the LEAP data might look like:


      To specify a parent-relative orientation of ElbowRight the LEAP data would look something like:


      Note: Puppetry currently enforces hard-coded Joint constraints for the parent-relative transforms and going forward the enforcement of constraints will probably be removed for Joint_state settings: if the plug-in wants to bend the elbow backwards then Puppetry will abide.  As part of this overhaul Puppetry will allow setting the parent-relative position of Joints.  In short: if the plug-in wants to fully control animation details then Puppetry will allow it.  Also, Puppetry will allow per-Joint constraints to be disabled for Inverse_kinematics calculations.  The details of the API for this have yet to be figured out.

      (3) All Puppetry configuration values will be sticky: set once and they stay at that value until changed or cleared.  An exception to this rule is: when a Puppetry plug-in loses connection or is stopped the Puppetry config will be immediately cleared.

      (4) To clear a Puppetry config value set it to an empty value.  For example the following would clear the IK position target of the ElbowRight:

      {set: {IK: {mElbowRight: {pos: []}}}}

      Note: the above only clears the position portion of the IK target, not the orientation.  To clear everything set the entire value to an empty set:

      {set: {IK: {mElbowRight: {}}}}

      (5) The Puppetry config will be sticky but the consumption of the Puppetry state by other Clients can't be incrementally updated because some Clients will first encounter the stream in the middle and won't have access to previous message history.  Therefore, in traditional codec parlance: the Client will stream complete "i-frames" worth of final Joint_state (post IK) up to the Server.  The i-frames will be sent whenever anything changes in the Puppetry config, or it will be resent at some yet-to-be-determined frame rate.  All Puppeted joints must be included in each message to the server; any not included will be assumed to NOT be under Puppetry control (e.g. "cleared").  We reserve the right to enhance compression by sending p-frames between i-frames at some future date.

      (6) The Pelvis joint will be allowed to move like any other Joint.  It is "special" in the sense that its parent is the avatar-frame itself therefore it will be movable via Inverse_kinematics or Joint_state, they both represent the same thing for the Pelvis.  Movement of the Pelvis in this manner is considered "pre-IK": the Pelvis is shifted before performing the Inverse Kinematics calculations.

      (7) An additional bogus Joint will be added called Origin.  It will represent the "post-IK" transform of the avatar-frame itself.  This will be an optional transform in case a plug-in wants to offset the entire avatar after the IK calculations are complete.  It will probably have the same offset limits as regular Animation assets.

      (8) An optional "terse" format will be available for Puppetry LEAP data, where each command, configuration key, and Joint will have a valid short name: set --> s, get --> g, Inverse_kinematics --> i, Joint_state --> j, etc, mWristRight --> 80, etc.  In other words, these Puppetry data would be equivalent:




        Issue Links



              leviathan.linden Leviathan Linden
              leviathan.linden Leviathan Linden
              3 Start watching this issue