Welcome to the PoseAI documentation!

PoseAI enables developers to easily add state-of-the-art pose estimation to their applications.

This is a complete guide for using PoseAI with UE4, UE5, Unity or via the developers’ API.

Note

This project is under active development.

Contents

• Using PoseAI with Unreal Engine
  • Connecting with UE4 with our helper nodes
    • Install the PoseAI UE4 LiveLink plugin
    • NOTE: MacOS Unreal
    • Add the PoseAIMovementComponent to the player blueprint
    • Add a source at Event Begin Play, close at Event End Play”
    • Modify the animation blueprint to include LiveLink
    • Connect Pose Camera on your phone
    • Begin the stream
    • Aim your camera at the subject
    • (Optional) Use mocap events for gameplay
  • Animating a character
    • Create an Animation Blueprint
    • Add a Live Link Pose node
    • Set retargetting for PoseAI
    • (Optional) Customize retargetting
    • Compile
    • Set your character to use the animation blueprint
    • (Optional) Add the LiveLink Skeletal Animation component to Character
    • (Optional) Create a blend for Desktop camera mode
  • Recording animations
    • Setup plugin and character
    • Add the LiveLink component to your character
    • Add your character to the world
    • Record with Take Recorder
    • Open your animation and inspect
    • (Optional) Export your animation to FBX
    • [Note] Recording MetaHuman rig animations
    • Tips for packaging builds
    • Try from a C++ project
    • Configure project settings
    • Try to build cleanly
    • Dont forget to apply a LiveLink preset in game.
    • And if that is not working…
• Using PoseAI with Unreal Engine (Manual Connection)
  • Connecting to UE4 with the LiveLink GUI
    • Install the UE4 LiveLink plugin
    • NOTE: MacOS Unreal
    • Add a PoseAI LiveLink source
    • Reduce default LiveLink buffer
    • Connect Pose Camera on your phone
    • Begin the stream
    • Aim your camera at the subject
  • Animating a UE4 Character
    • Create an Animation Blueprint
    • Add a Live Link Pose node
    • Set retargetting for PoseAI
    • (Optional) Customize retargetting
    • Compile
    • Set your character to use the animation blueprint
    • (Optional) Add the LiveLink Skeletal Animation component to Character
    • (Optional) Create a blend for Desktop camera mode
  • Recording animations
    • Setup plugin and character
    • Add the LiveLink component to your character
    • Add your character to the world
    • Record with Take Recorder
    • Open your animation and inspect
    • (Optional) Export your animation to FBX
    • [Note] Recording MetaHuman rig animations
• Walkthrough of the UE4 Example
  • Install the UE4 LiveLink plugin to the engine
  • Download the example project
  • Examine the key files for LiveLink
  • Add a PoseAI LiveLink source
  • Connect Pose Camera on your phone
  • Begin the stream
  • Animate the two mannequins
  • (Optional) Reduce default LiveLink buffer
• Adding PoseAI input to Unity
  • Copy the PoseAI UnityAPI files
  • Disable redundant controllers on the PlayerArmature
  • Add a PoseAI SourceDirect component to the armature
  • Add the PoseAI AnimatorComponent
  • Enable IK Pass on the Animator asset
  • Add the PoseAICharacterController
  • Connect Pose Camera on your phone
• Connecting with the API
  • Linking to the Pose Camera app
• Performance and Troubleshooting
  • Troubleshooting connections
  • Latency and Unreal LiveLink
  • Frames per second, heating and phone performance”
  • Unreal Engine crashes
• Open BETA
• Frequently Asked Questions
  • What are the different modes?
  • Does Pose Camera stream my camera feed? What about privacy?
  • Can I use Pose Camera to record animations?
  • Can I use videos instead of the live camera feed?
  • How realtime is Pose Camera?
  • Will running motion capture slow my game?
  • How does Pose Camera compare to other Mocap systems?
  • Why doesn’t my character’s hands meet like that of my mocap subject?
  • Why does Pose Camera work better on some movements and poses?
  • Does Pose Camera track root motion (i.e. target movement in real space)?
  • Is your AI open source? Where does it come from?
  • Will you also release on Android?