Walkthrough of the UE4 Example¶
Install the UE4 LiveLink plugin to the engine¶
Install Unreal Engine 4.26 on your desktop. Install the PoseAILiveLink plugin from the Unreal Marketplace to the engine.
Download the example project¶
Download the file, unzip and add it to your Unreal projects directory.
Open the project from the Epic Games / Unreal Engine project launcher.
Examine the key files for LiveLink¶
Open the project in the Unreal Engine and launch the tutorial to see the key files for LiveLink highlighted. Examine the files and read the comments to learn how the components work for your own projects.
Note the location of the highlighted AnimBP files. You will need to set the LiveLink node in each of these files to use your phone source once you have connected (explained in a later step).
Add a PoseAI LiveLink source¶
From the Unreal Engine menu select Window/LiveLink:
From the LiveLink popup window add a Source->Pose AI App. Try the default settings first. Use either “Room” mode or “Desktop” mode (highlighted in blue) to see how the two mannequins respond in the engine. We have provided blueprint demos for both modes:
Click OK when ready. PoseAI Mobile should be added to the first panel of LiveLink:
Note the IP address (underlined in blue in the screenshot). This is likely your computer’s local network IP address (UE4 thinks so). You will use this in the next step.
Connect Pose Camera on your phone¶
Open the PoseCamera app on your phone. Go to settings (gear icon):
and input the IP address of the computer running the Unreal Engine:
This can usually be found displayed in LiveLink (but if you have multiple adapters this may not be correct). Make sure your phone and the computer are on the same local network.
Close settings on your phone. The app should show the IP address you entered at the bottom of the main screen. You may be asked to rotate your phone sideways:
Tap to connect. When successful connected the phone app should display “Unreal LiveLink” in place of the IP address, and say “tap to stream”. Tap to begin the stream and the app will switch to camera mode.
Unsuccessful? Try our troubleshooting page for tips resolving common connection issues
Begin the stream¶
Once connected the phone app should offer “Tap to Stream”. Tap and the phone should switch to camera preview mode.
Within Unreal you should see your phone’s name in the bottom LiveLink panel and will show green when it is streaming detected poses.
If the light shows yellow it means the phone has not detected your pose. For Room or Portrait ensure the full body, head to feet, is visible. For Desktop mode ensure the camera can see the head to pelvis and ideally both elbows. The segmented human icon overlaying camera preview turns blue when a subject is visible and shows which limbs are currently poseable - if it is grey you need to reposition the camera or the person:
Animate the two mannequins¶
Finally you need to select your phone in the two animation blueprints, LiveLink_Desktop_AnimBP and LiveLink_FullBody_AnimBP. Click on “Compile” to recompile the blueprint and you should see the animation live in preview and now in the editor.
The two mannequins should now animate in the editor. The green circled one is set for full body animation (i.e. Room mode). While in dekstop mode you will notice the legs do the splits (zero transform is default). The red circled one is set to use upper body animations (i.e. Desktop mode), and use a default animation for the legs.
(Optional) Reduce default LiveLink buffer¶
By default LiveLink buffers several frames of motion capture data. This can add latency to the input from Pose Camera. On a local network and with Pose Camera’s in-app smoothing, we recommend reducing the buffer to 1 frame, and only if necessary increase the buffer.
Click on the Pose AI mobile source in the top LiveLink panel and in the right details, under Buffer-Settings reduce “Max Number of Frame to Buffered” to 1.