Blog Archives

Easy Access to Kinect for Windows Sample Code

Microsoft released Kinect for Windows samples under an open source license.  You can find everything on CodePlex: http://kinectforwindows.codeplex.com/.  There are total of 22 unique samples in C#, C++, and Visual Basic.

Few reasons behind this release are:

  1. Easy Access -> we will continue to release our sample applications as part of our Developer Toolkit.  However, that’s a large download & install that can be cumbersome if you just want to quickly view or access code on the web
  2. Reuse The Code -> we’re releasing all the samples under an Apache 2.0 license so that you can take the code and reuse, remix, etc.  Also, we’re using a Git repository so it’s easy clone & fork if you want
  3. Get Feedback -> we will use CodePlex’s built-in feedback & discussion tools to get community input on the samples.  We want to hear from you to understand what we can do better with the samples
  4. Faster Updates -> we will be able to update samples more quickly on CodePlex (compared to Toolkit releases).  CodePlex also has a “Subscribe” feature that enables you to follow the project and get notified when something changes, a bug gets fixed, someone says something smart in the discussions, etc.  (note:  the subscription feature doesn’t actually track the smartness of a post but one can dream :-))

Browse K4W sample code right in your browser…

 

List of Samples

This is the list of all samples included in the latest Developer Toolkit release. The table lists the name of the sample, in which languages it’s available and what technologies and additional SDKs are used.

Sample

C#

C++

VB

WPF

DirectX

Additional information

Audio Basics

Yes

Yes

Yes

Yes

Yes

Available in 1.6.0

Audio Capture Raw

No

Yes

No

No

No

Available in 1.6.0

Audio Explorer

No

Yes

No

No

Yes

Available in 1.6.0

Basic Interactions

Yes

No

No

No

No

Available in 1.6.0

Color Basics

Yes

Yes

Yes

Yes

Yes

Available in 1.6.0

Depth Basics

Yes

Yes

Yes

Yes

Yes

Available in 1.6.0

Depth

No

Yes

No

No

Yes

Available in 1.6.0

Depth with Color

No

Yes

No

No

Yes

Available in 1.6.0

Face Tracking

Yes

No

No

Yes

No

Available in 1.6.0, Require Face Tracking

Face Tracking Basics

Yes

No

No

Yes

No

Available in 1.6.0, Require Face Tracking

Face Tracking Visualization

No

Yes

No

No

Yes

Available in 1.6.0, Require Face Tracking

Green Screen

Yes

Yes

No

Yes

Yes

Available in 1.6.0

Infrared Basics

Yes

Yes

No

Yes

Yes

Available in 1.6.0

Kinect Explorer

Yes

No

No

Yes

No

Available in 1.6.0

Shape Game

Yes

No

No

Yes

No

Available in 1.6.0

Skeletal Viewer

No

Yes

No

No

Yes

Available in 1.6.0

Skeleton Basics

Yes

Yes

Yes

Yes

Yes

Available in 1.6.0

Slideshow Gestures

Yes

No

No

Yes

No

Available in 1.6.0

Speech Basics

Yes

Yes

Yes

Yes

Yes

Available in 1.6.0

Tic Tac Toe

Yes

No

No

Yes

No

Available in 1.6.0

WPF D3D Interop

Yes

Yes

No

Yes

Yes

Available in 1.6.0

XNA Basics

Yes

No

No

Yes

No

Available in 1.6.0 RequireXNA

 

Requirements

  • Visual Studio 2010 or 2012, .NET 4.0 or 4.5
  • Kinect for Windows SDK and Kinect for Windows Toolkit.
  • Some samples require additional toolkit components (e.g. Face Tracking).
  • Some samples make use of additional SDKs such as DirectX, XNA etc.
  • Note: See the full List of Samples to see if any additional SDKs are required.

 

Useful Links

Advertisements

Easy Access to Kinect for Windows Sample Code

Microsoft released Kinect for Windows samples under an open source license.  You can find everything on CodePlex: http://kinectforwindows.codeplex.com/.  There are total of 22 unique samples in C#, C++, and Visual Basic.

Few reasons behind this release are:

  1. Easy Access -> we will continue to release our sample applications as part of our Developer Toolkit.  However, that’s a large download & install that can be cumbersome if you just want to quickly view or access code on the web
  2. Reuse The Code -> we’re releasing all the samples under an Apache 2.0 license so that you can take the code and reuse, remix, etc.  Also, we’re using a Git repository so it’s easy clone & fork if you want
  3. Get Feedback -> we will use CodePlex’s built-in feedback & discussion tools to get community input on the samples.  We want to hear from you to understand what we can do better with the samples
  4. Faster Updates -> we will be able to update samples more quickly on CodePlex (compared to Toolkit releases).  CodePlex also has a “Subscribe” feature that enables you to follow the project and get notified when something changes, a bug gets fixed, someone says something smart in the discussions, etc.  (note:  the subscription feature doesn’t actually track the smartness of a post but one can dream :-))

Browse K4W sample code right in your browser…

 

List of Samples

This is the list of all samples included in the latest Developer Toolkit release. The table lists the name of the sample, in which languages it’s available and what technologies and additional SDKs are used.

Sample

C#

C++

VB

WPF

DirectX

Additional information

Audio Basics

Yes

Yes

Yes

Yes

Yes

Available in 1.6.0

Audio Capture Raw

No

Yes

No

No

No

Available in 1.6.0

Audio Explorer

No

Yes

No

No

Yes

Available in 1.6.0

Basic Interactions

Yes

No

No

No

No

Available in 1.6.0

Color Basics

Yes

Yes

Yes

Yes

Yes

Available in 1.6.0

Depth Basics

Yes

Yes

Yes

Yes

Yes

Available in 1.6.0

Depth

No

Yes

No

No

Yes

Available in 1.6.0

Depth with Color

No

Yes

No

No

Yes

Available in 1.6.0

Face Tracking

Yes

No

No

Yes

No

Available in 1.6.0, Require Face Tracking

Face Tracking Basics

Yes

No

No

Yes

No

Available in 1.6.0, Require Face Tracking

Face Tracking Visualization

No

Yes

No

No

Yes

Available in 1.6.0, Require Face Tracking

Green Screen

Yes

Yes

No

Yes

Yes

Available in 1.6.0

Infrared Basics

Yes

Yes

No

Yes

Yes

Available in 1.6.0

Kinect Explorer

Yes

No

No

Yes

No

Available in 1.6.0

Shape Game

Yes

No

No

Yes

No

Available in 1.6.0

Skeletal Viewer

No

Yes

No

No

Yes

Available in 1.6.0

Skeleton Basics

Yes

Yes

Yes

Yes

Yes

Available in 1.6.0

Slideshow Gestures

Yes

No

No

Yes

No

Available in 1.6.0

Speech Basics

Yes

Yes

Yes

Yes

Yes

Available in 1.6.0

Tic Tac Toe

Yes

No

No

Yes

No

Available in 1.6.0

WPF D3D Interop

Yes

Yes

No

Yes

Yes

Available in 1.6.0

XNA Basics

Yes

No

No

Yes

No

Available in 1.6.0 RequireXNA

 

Requirements

  • Visual Studio 2010 or 2012, .NET 4.0 or 4.5
  • Kinect for Windows SDK and Kinect for Windows Toolkit.
  • Some samples require additional toolkit components (e.g. Face Tracking).
  • Some samples make use of additional SDKs such as DirectX, XNA etc.
  • Note: See the full List of Samples to see if any additional SDKs are required.

 

Useful Links

Part 2 – Getting started with Microsoft Kinect

Hi Folks,

In the first Part, I discussed about the fundamentals of Kinect hardware as well as technical details of the camera & microphone array.
Today in this part of my “Programming with Kinect” series I am going to guide you about how to setup the development environment for Kinect and writing code to get started with development using Kinect.

Note: I’ll be using “Kinect for Xbox 360” hardware with “Kinect for Windows SDK” in my posts until & unless I am not using specific features of Kinect for Windows hardware.

Installation of SDK

The Kinect SDK is a development toolkit that allows software developers to build applications using Kinect and expand the possibilities of application and user experience. Kinect Software development kit (SDK) provides interface to interact with Camera, Motors, Microphone array etc. It allows user to build applications through Visual C#, Visual Basic.Net and C++ language using Microsoft Visual Studio IDE.

Recent release of Kinect SDK (1.6) supports Visual Studio 2012 and Windows 8.

Prerequisites

Before you begin with the installation, make sure you have the following components

  • Microsoft Kinect Sensor
  • USB cable of Kinect Sensor.

Make sure your Kinect sensor is unplugged and previous SDK of Kinect (if any) is uninstalled properly.

Installation

The installation procedure is very simple and wizard base.

  • Go to Kinect for Windows developer’s page and download the latest SDK (v1.6) and Kinect Developer toolkit (I’ll discuss about it later).
  • Install the SDK you just downloaded.
  • After that Install the Kinect Toolkit.

After successful installation of both setup files you just downloaded, it’s time to connect the Kinect sensor with your PC.
Plug in your Kinect sensor with you PC and wait till the Windows install its driver.

Note: You may skip the “Searching Driver on Windows Update” part by clicking “Skip obtaining driver from Windows Update” while Windows is installing the driver. It will expedite the installation process and use the driver files that comes with the SDK you just downloaded.

Following components will be available after successful installation of Kinect SDK.

  • Kinect Developer Browser.
  • Kinect Studio.
  • Kinect Drivers.
  • Kinect Runtime.
  • Kinect Speech Recognition Language Pack (en-US).

Kinect for Windows Architecture

Below is the architecture diagram of Kinect for Windows, You can access Kinect microphones array using your standard audio API.

I am not going to discuss this in details, but just for a quick view, we have

  • Kinect Sensor (Physical Kinect Hardware)
  • Kinect Driver
  • Audio & Video Components
  • Direct X Media Object
  • Kinect API

More information regarding architecture can be found at http://msdn.microsoft.com/en-us/library/jj131023

Development Using Kinect

After we have our environment ready for software development using Kinect, let’s move towards using Kinect in our Project.

I’ll be using Windows Presentation Foundation (WPF) application type throughout my demos, but you can also use Win-Forms with Kinect.

Go ahead and start your Visual Studio.

  • Create a new WPF Application and name it appropriately (in my case it’s UmerKinectBlogDemo1). 
  • Add “References” of “Microsoft.Kinect” by right clicking on References folder in solution explorer and click “Add Reference”.
  • Expand the “Assemblies” list on your left and select “Extensions”.
  • Scroll down until you find “Microsoft.Kinect”, apply check on that and click OK.

After adding Kinect reference;

  • In your XAML code, add the following attribute in your Window markup code.Loaded=”MainWindow_Loaded”

  • Drag/Insert an Image Control from the Toolbox into your application, name & resize it appropriately as this image control will be used for showing the camera feeds from Kinect.

     

  • Go to your Code Behind file of Main page (MainWindows.xaml.cs) and add the following namespaces into your code behind fileusing Microsoft.Kinect;
    using System.Windows.Media.Imaging;
  • Create a global reference of “KinectSensor” class, this will hold the instance of our connected Kinect sensorKinectSensor myKinectSensor;
  • In your MainWindow_Loaded method, fetch the connected Kinect sensor object using the following code;if (KinectSensor.KinectSensors.Count != 0)
    {
    if (KinectSensor.KinectSensors[0].Status == KinectStatus.Connected)

myKinectSensor = KinectSensor.KinectSensors[0];
else
MessageBox.Show(“Kinect sensor is not yet ready”);
} else
MessageBox.Show(“Kinect sensor is not connected”);

The “KinectSensors” collection holds all the connected Kinect sensor to the machine, since we have only one sensor connected to our machine, we are directly fetching the one at “zero” index. I am also doing a little error handling that if the status of the connected sensor is not ready, notify the user.

Note: For better experience, you should use StatusChanged event of KinectSensors Collection.

 Getting RGB (Color Camera Stream)

The color stream from Kinect sensor can be used to display the camera view of Kinect and you can also use the same stream to apply different Image Processing algorithms to achieve your specific output.

Kinect provides different resolutions and Formats for Color stream and you can select the one that best suits your need.

Now as we have our Kinect sensor object, let’s move and add an event handler of “ColorFrameReady” and start the “ColorStream” of our Kinect sensor to get the Color Stream from Kinect.

myKinectSensor.ColorFrameReady += myKinectSensor_ColorFrameReady;
myKinectSensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);

The “ColorFrameReady” event handler is used to handle the RGB images coming from our Kinect Sensor. We have the option to enable specified Video Stream (i.e. RGB, depth data etc.) in Kinect; for instance we are enabling the Color Stream and defining the Image Format of the stream in parameter.
Kinect supports different image formats including “RGB” and “YUV” and with different FPS (frame per second) and resolutions as well. We are using RGB Format of 640X480 resolution with 30 FPS. (Ideal FPS is 30, lower FPS results in slow image propagation).

  • Insert the following code snippet into your ColorFrameReady event handler.void myKinectSensor_ColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
    {
    using (ColorImageFrame es = e.OpenColorImageFrame())
    {
    if (!(es == null))
    {
    byte[] bits = new byte[es.PixelDataLength];
    es.CopyPixelDataTo(bits);
    image1.Source = BitmapSource.Create(es.Width, es.Height, 96, 96,PixelFormats.Bgr32, null, bits, es.Width * es.BytesPerPixel);
    }
    }
    }

    The image data is encapsulated in ColorImageFrame, we have to copy the image data into array of byte to use it, the “PixelDataLength” property of that ColorImageFrame has the length of pixel data buffer of the current frame.

    After that we have to create a bitmap image from the byte array (image data) so that can show it in our Image Control.

    BitmapSource.Create method is used to do that, the parameters of the methods are;

    • pixelWidth:        The width of the bitmap.
    • pixelHeight:       The height of the bitmap.
    • dpiX:                     The horizontal dots per inch (dpi) of the bitmap.
    • dpiY:                     The vertical dots per inch (dpi) of the bitmap.
    • pixelFormat:     The pixel format of the bitmap. (BGR, CMYK, Black etc)
    • Palette:               The palette of the bitmap.
    • Pixels:                  An array of bytes that represents the content of a bitmap image.
    • Stride:                  The stride of the bitmap. (Number of bytes allocated for one scan line of
      the bitmap).

After we create a bitmap image, assign that bitmap image object as the “Source” of Image Control we added in previous step.

Let the show begin

Our minimum code to display the camera stream from Kinect sensor is almost complete.

Before we run our project, there is one that that we need to do. We have to start the Kinect itself so that we can get the streams from the Kinect; to do that, add the following line at the end of your MainWindow_Loaded method.

myKinectSensor.Start();

Now run the project by pressing F5 to see the results.

Here is the result of my Kinect sensor.

It is my table where most of my stuff is found (I am bad at organizing stuff on my table 🙂 )

Summary

In this post, I’ve discussed the installation procedure, interfacing Kinect with our application and getting Color stream from the Kinect camera.

Don’t forget to do proper exception handling into your project, especially with KinectSensor object.

The complete project for your reference can be found here.

What’s next?

In my next post, I’ll be discussing about

  • Skeleton Tracking

And more.

If you have any suggestions on topics, have questions, feedback or want to help me out, feel free to contact me by posting your comments below this post and I’ll try to help you out!

       Muhammad Umer