Week 03 - Data Cleaning and Retargeting

Lab Documentation

After calibration and putting the performer in suit, we decided to do a dance take. Terrick did the leading dance and Tian followed him. We did it twice to at least get one good and one best take. We also took a video of their actual dancing as the reference video.

Then we started to do the data cleaning. It was very lucky that we got a very nice and clean take. We only had six unlabeled markers and very few gaps. The first thing we did was to change the color of bones and mesh to have a strong contrast.

We started with the easiest part, labelling those unlabeled ones. We went frame by frame at the happening points of those unlabelled data to which marker was missing. We had to watch it from many different angels to see it clearly. For some unlabeled data, we couldn’t find it correspond to any markers, so we supposed they were just extra data and removed them.

Next, we tried to fix all the data gaps. We opened the curve view, highlighted the missing part, filled them as liner or cubic and then clicked ‘find next gap’ button and did these again.

Since we didn’t have too many gaps, we fixed all of them in lab time. The data looks nice to me.

Here is the screen recording of data cleaning: https://drive.google.com/file/d/1hIKzp7DHbB1q69DLA8oAQa1JSE1zQbu3/view?usp=sharing

Retargeting and importing to UE4

For the character, I used the little elf Charlie I made in Oculus Medium for Narrative VR last semester.

I imported it into 3Ds Max first to assign new material to it and adjusted his hands to be a better symmetrical T-pose. Then I imported the file into Maya to rig. The step by step quick rig was not perfect for a baby portion figure like Charlie, so I manually moved those joints to the right place, then generated skeleton and bonded it to the skin.

Next, I sent it to motion builder. In order to get a perfect T-pose, I selected all the branches under Charlie’s hip and zeroed out the rotation. But the problem was after that, his clothes were stretched and broken into several pieces. I tried to make the clothes bigger and redid the process but it didn’t work. I think it might be the problem of Medium?

Anyway, then I characterized the hip, renamed it as Avatar. Next thing to do was to drag in the dancing data and merge it with the current scene. We also need to zero out all the rotation of the skeleton and characterize it. To bond those two skeletons, I selected the avatar, in the setting, I chose it to be driven by the imported dancing data. Click the active box and we got the animation. Plot it and we can save it as a .fbx file.

Finally, we can import the mesh and animation into Unreal. I imported the mesh, skeleton and materials first. And the second time I imported animation, linked it with the existing skeleton. Then I dragged the animation file into Unreal scene and got it played. The model was not perfect but it was a nice try to explore the whole work flow.