Multi Video Capture in Silverlight
One fine detail before you read any further: you need to have two video capture devices (e.g. web cams) in order to really have any fun with this.
One of the best features about Silverlight 4 is that you can interact with video and audio capture devices. This enables you to take pictures from a web cam, record video, or record audio in your Silverlight application and handle the media data directly.
One fun little side effect of the video capture API in Silverlight 4 is that you can enable two video capture devices simultaneously. This may or may not have any value to you and your everyday Silverlight programming, but if it did then it probably wouldn’t be any fun anyway!
If you haven’t tried out the Audio/Video capture API in Silverlight 4 yet, here is the basic idea:
That’s all the code you need to do to start viewing/recording video. Basically, CaptureSource is the key class that makes it all work for you. You use CaptureSource’s VideoCaptureDevice property to tell it which video device on your computer to use. You may have more than one, but Silverlight lets you select a default one with the CaptureDeviceConfiguration class’s GetDefaultVideoCaptureDevice method. It’s cake.
If you actually want to select a specific video device and not use the default, you can use the CaptureDeviceConfiguration’s GetAvailableVideoCaptureDevices method:
With that list of devices you can bind to it or do whatever it is you like to do with lists of VideoCaptureDevices. Typically, I bind that list to a ComboBox so that the user can select their device.
To display the video from your device on screen, call the Start() method on the CaptureSource object and use a video brush to paint it onto a surface, like a rectangle:
In the source code for this article you can see this all in action.
Now, the cool part is that you can encapsulate all of the above logic in a Silverlight UserControl and display two cameras at the same time. Essentially, two CaptureSource objects can be set up and started.
As an example, I have a main page with two Camera user controls I created on it:
And the camera controls look like this:
You can see that I’ve added a Rectangle for the video display area, a Combo box to let the user select their device, a “Turn On” button, and a warning message to display (if necessary) about the device already being in use.
CaptureSource.Start() will throw an InvalidOperationException if the same device is selected twice (once in each Camera control). I’m catching that exception and then displaying the “already in use” warning message when that scenario occurs.
When I run the application, I can select two different cameras and display them at the same time:
You’ll receive the following error if you select a device that is already in use:
Again, what is the point? Well, I’ll leave that up to you. Theoretically the ability to run two video devices in Silverlight would allow you to, say, take pictures of yourself with one device while scanning barcodes with the other. I do that all the time. Kidding! Geez!
Another interesting example is that you can achieve a composite image by layering one visual element with less than full opacity over another element - and have both elements use VisualBrushes that pull from different video sources:
I’ve just spawned a whole new genre of visual art…
One “gotcha!” of using two web cams at the same time is that your browser and operating system might not be too happy with you. If you are constantly starting capture sources, debugging without stopping devices, plugging in and removing devices, and testing multiple browsers, you may experience a “hang” (or “crash”) in your Silverlight app when you call CaptureSource.Start().
You can reproduce this problem in the Silverlight configuration tool. Right-click on your application, choose Silverlight, then choose the Webcam/Mic tab. In a crash-free scenario, your config tool will show this:
In a scenario when your operating system and Silverlight can’t communicate with your video capture devices, the above tab will “hang” indefinitely and not display anything.
To help minimize this problem with hangs and crashing, I encourage you to stop all CaptureSources when the application exits. This seems to make all those web cams and Silverlight play more happily together:
That’s it. If anything, playing with multiple capture devices simultaneously is just good clean fun. Since audio capture is very similar to video capture in Silverlight, you could probably do some crazy stereo imaging or something with audio capture using these techniques (I smell another blog post coming…).
Have fun and enjoy!
