Adventures in Motion Capture: The Hardware

June 10, 2016

By the time you read this, the prototype for my new game Robyn HUD should be done. That is to say, it should be ready for a small group of trusted individuals to play it and give me feedback on what’s working, what’s not, and for me to observe how they play it. Robyn HUD is a 3D game (as opposed to my 2D point-n-click Sleuthhounds adventure games). One area that I already know that needs improvement, even without any feedback, is that of animation.

Hand animating 3D characters isn’t so much hard as it is time consuming. It takes a lot of time to animate a 3D character so that it moves fluidly and with believable mass as opposed to being a janky puppet on a string. When you’re developing your own game, you want to keep the time it takes to implement things as short as possible. The less time it takes to do things the more things you can do. Simple math.

So, recognizing that doing 3D character animation to a good enough level was going to be extremely time consuming, I started looking for another solution. A lot of effects movies these days use a process called motion capture to record the movements of real people and then apply those movements to 3D characters in the computer. These motion capture setups usually involve multiple special cameras that track little Day-Glo ping pong balls stuck onto the people being recorded. The cost for such a setup easily puts this solution outside of my development budget.

However, a quick internet search on “poor man’s motion capture” yielded a surprising result to me: the Microsoft Kinect. For those unfamiliar with the Kinect, it’s a special camera/sensor that plugs into Microsoft’s Xbox console. It tracks how players are moving and uses those motions to control games.

Now, some very clever people out there realized that the Kinect could be used to do motion capture and bring the movements of people into a computer for use in developing animations or games or whatever else you need motion capture for. There are a number of applications capable of doing this and I’ll probably be looking at at least some of them in the weeks to come.

Important

At the time of writing, there are two versions of the Kinect floating around. Version 1 is for the Xbox 360 and version 2 is for the more recent Xbox One. I picked up the version 2 as it was more readily available. In hindsight, given what my needs are, I probably should have gone for the older version 1. It’s still capable of performing motion capture and doesn’t seem to have the USB compatibility issues that I detail later in this post. Just something to keep in mind if you’re considering using a Kinect for motion capture.

The Kinect

The first step was to get a Kinect for myself and get it hooked up to my computer. As I mentioned, the Kinect sensor is for the Xbox console but I wanted to tie it into my desktop PC. As it happens, Microsoft has a USB 3.0 adapter for exactly this purpose. I ordered it through the Microsoft Store who sent it direct to my home with express shipping that they paid for (bonus!). And it was a quick trip over to the local Best Buy to pick up the Kinect sensor itself.

The Kinect sensor is a single unit with, what I believe, is a USB 3.0 cable coming out of it. I could be mistaken about that as it does not end in a normal USB 3.0 connector but a special connector of its own form factor. That’s why the Kinect to PC adapter is needed.

The adapter itself comes in two pieces. One piece is the adapter proper. The Kinect plugs into one side and a normal USB 3.0 cable comes out the other side and plugs into the PC. The second piece in the adapter box is a power supply that connects to the adapter and to an AC power socket. So if you’re going this route, make sure you’ve got an open AC outlet within reach as the USB 3.0 itself is not enough to meet the Kinect sensor’s power needs.

USB 3.0 SuperSpeed

Here’s where the fun begins (read “where things went horribly wrong”). This part gets a little technical but it seems to be the biggest point of failure for using the Kinect to perform motion capture.

The adapter for the Kinect plugs into an available USB 3.0 port on the PC. Now, silly me, I figured that since the Universal Serial Bus is, you know, universal that plugging the USB 3.0 connector of the adapter into a USB 3.0 port on my computer would, you know, work. Not so much. In order to operate, the Kinect sensor needs a sustained data transfer rate of 5 GBps. I don’t know exactly how fast my PC’s onboard USB 3.0 connectors are but they’re apparently less than 5 GBps. So back to the internet to do some more reading on the Kinect.

I discovered that the Kinect requires what’s referred to as USB 3.0 SuperSpeed. This is a faster version of USB 3.0 that’s able to deliver the 5 GBps transfer rate that the sensor needs. Note that there’s also a USB 3.1 out there, which I believe is even faster than USB 3.0 SuperSpeed. I have no clue how compatible USB 3.1 is with the Kinect though.

As my onboard USB 3.0 connectors weren’t fast enough to support the Kinect, I had to look to another solution. Now it’s possible to purchase a PCIe card for your PC that will give you USB 3.0 SuperSpeed ports and that’s exactly what I did. But there are a few caveats to doing this.

First of all, data moving from PCIe devices in your computer to the CPU travels along its own bus (not the kind with wheels) within the computer. This bus has its own transfer speed, usually rated in GT/s (Giga Transfers per second). So how does GT/s compare to GBps? Well, that depends on a variety of things beyond what I can detail here. The important thing to know is that there have been several versions of PCIe released.

Typically, version 1 of PCIe has a max transfer speed of 4 GBps. It’s not as fast as what the Kinect wants although some people have reported success with it. However, version 2 of PCIe has a transfer speed of 8 GBps, which is enough to satisfy the Kinect. Later versions of PCIe have even faster rates. So if you’re looking at using the version 2 Kinect for motion capture, be sure that your PC has at least a version 2 PCIe bus. Your motherboard manual may identify this by indicating a bus speed of 5 GT/s or better. My motherboard was PCIe version 2 so I felt reasonably confident I could get things to work.

Once you’ve verified that your PC is fast enough to support the transfer rate then you need to pick up a PCIe card itself that supports SuperSpeed. It should actually have the SuperSpeed term somewhere on the box. Based on my readings, it seems people have had the most success with PCIe cards released by either Intel or Vantec. I picked up the Vantec 4-Port SuperSpeed USB 3.0 PCIe Host Card w/Internal 20-Pin Connector. Basically it gives me two external USB 3.0 ports and 2 internal ports. Since I’m just using it for the Kinect the number of ports wasn’t too important (more ports typically equals more cost).

It’s important to note that Microsoft recommends plugging the Kinect into a port that doesn’t internally have anything else connected to it. For example, with my four port card, they’d say not to use any of the other three ports. This is because the more devices you have plugged in, the more things you have sharing the bus, so even though in a standalone situation your computer might be fast enough, if you have more USB 3.0 devices hanging off the same PCIe card, their sharing the bus may reduce the transfer rate allocated to the Kinect to the point where it won’t function.

eXtensible USB Host Controllers

Once I had the PCIe card installed I was ready to try the Kinect again. I had been using Microsoft’s Kinect Configuration Verifier tool to check to see if the Kinect was working. Basically, the tool goes through a bunch of tests to make sure it can detect the Kinect and get the necessary data throughput. One minor point here is that Microsoft has version locked the tool so that it only runs on Windows 8 or later. Note that the Verification tool requires that you also download and install the Kinect SDK 2 in order to operate properly.

As I’ve reported previously, I’ve tried Windows 10 and found it not suitable for my use. As a result, I went back to Windows 7, although I did keep a drive with Windows 10 around. So to test the Kinect out, I had to pop that Win 10 drive in and finish the configuration with that.

Once you have the PCIe card installed you then need to go through the driver dance. The Vantec card I got came with a basic device driver CD but oddly one that wasn’t what’s known as a USB 3.0 eXtensible Host Controller Driver. To be honest, I’m not sure why this matters but it seemingly does.

I tried the Kinect with the driver from the CD and it didn’t work. I then went to the manufacturer’s website and tried the most recent driver from there. Still didn’t work. I then tried the generic Microsoft USB controller driver and that didn’t work. In desperation, I tried an Intel driver but couldn’t get that to install because, and here’s the kicker, the Intel driver software was version locked to only run on versions of Windows earlier than 10. Whoops!

By this point I had tried a variety of configurations and wasn’t sure what was what anymore. I went into the device manager and uninstalled the USB device driver from there to hopefully get back to a starting point. To be safe, I then restarted the computer to get rid of any lingering bits.

Once the computer restarted, I went back to the device manager to confirm that the driver had been uninstalled. Lo and behold, there was a completely different driver there. This one was supplied by Microsoft but was specific for my Vantec card, instead of being the generic driver, AND it had that key USB 3.0 eXtensible Host Controller description.

I don’t know how Windows decided to put that driver in, but I’m glad that it did. When I next ran the Kinect verification tool it successfully ran through all the tests and displayed video and depth data coming from the sensor at a constant 30 frames per second. I’m certainly happy that data is coming through from the sensor properly. I just wish I knew what the magic was to get the needed driver into place. My only recommendation here is that if you get a PCIe card to add USB 3.0 SuperSpeed support to your computer that you try to get a model that’s a couple of years old to maximize the chances that Windows will already have a Microsoft supplied version of the needed driver.

Summary

I haven’t tried any actual motion capture software with the Kinect yet. However, I am happy that I’ve gotten through at least the verification tool tests. Hopefully in time for next week, I’ll have tried out at least one motion capture application and can report on that. In the meantime, if you’re looking to use the Kinect to do motion capture yourself, here’s my conclusion on what you’ll need to get the setup going:

  • Kinect version 2 sensor (i.e. the Kinect sensor for Xbox One)
  • Kinect Adapter for Windows (be sure it’s the one for the Kinect version 2 as there’s also one for the Kinect version 1)
  • a free AC socket to power the adapter
  • an onboard or PCIe USB 3.0 SuperSpeed connection (if getting a PCIe card, Intel or Vantec are generally recommended)
  • a system BUS transfer speed of 5 GT/s or better if using a PCIe card (version 2 or later of PCIe)
  • Windows 8 or later to run the verification tool
  • A USB device driver that displays as USB 3.0 eXtensible Host Controller in your computer’s device manager