A Developer’s Guide To Intel® RealSense™ Camera Detection Methods

Abstract

The arrival of a new and improved front-facing camera, the SR300, has necessitated changes to the Intel® RealSense™ SDK and the Intel® RealSense™ Depth Camera Manager that may prevent legacy applications from functioning. This paper provides an overview of some key aspects in developing camera-independent applications that are portable across the different front-facing cameras: Intel® RealSense™ cameras F200 and SR300. It also details several methods for detecting the set of front- and rear-facing camera devices featured in the Intel RealSense SDK. These methods include how to use the installer scripts to detect the local capture device as well as how to use the Intel RealSense SDK to detect the camera model and its configuration at runtime. This paper is intended for novice and intermediate developers who have either previously developed F200 applications and want to ensure compatibility on SR300-equipped systems or want to develop new Intel® RealSense™ applications targeting SR300’s specific features.

Introduction

The arrival of the new and improved front-facing Intel RealSense camera SR300 has introduced a number of changes to the Intel RealSense SDK as well as new considerations to maintain application compatibility across multiple SDK versions. As of the R5 2015 SDK release for Windows*, three different camera models are supported including the rear-facing Intel RealSense camera R200 and two front-facing cameras: the Intel RealSense camera F200 and the newer SR300. The SR300 brings a number of technical improvements over the legacy F200 camera, including improved tracking range, motion detection, color stream and IR sensors, and lower system resource utilization. Developers are encouraged to create new and exciting applications that take advantage of these capabilities.

However, the presence of systems with different front-facing camera models presents several unique challenges for developers. There are certain steps that should be taken to verify the presence and configuration of the Intel RealSense camera to ensure compatibility. This paper outlines the best-known methods to develop a native SR300 application and successfully migrate an existing F200 application to a SR300 platform while maintaining compatibility across both cameras models.

Detecting The Intel® RealSense™ camera During Installation

In order to ensure support, first verify which camera model is present on the host system during application install time. The Intel RealSense SDK installer script provides options to check for the presence of any of the camera models using command-line options. Unless a specific camera model is required, we recommend that you use the installer to detect orientation (front or rear facing) to maintain portability across platforms with different camera models. If targeting specific features, you can check for specific camera models (Intel RealSense cameras F200, SR300, and R200) by specifying the appropriate options. If the queried camera model is not detected, the installer will abort with an error code. The full SDK installer command list can be found on the SDK documentation website under the topic Installer Options. For reference, you can find the options related to detecting the camera as well as sample commands below.

Installer Command Option

--f200
--sr300
--f200
--r200

Force a camera model check such that the runtime is installed only when the requested camera model is detected. If the camera model is not detected, the installer aborts with status code 1633.

--front
--rear

The --front option checks for any front facing camera and --rear option for any rear facing camera.

Examples

Detect presence of any rear-facing camera and install the 3D scan runtime silently via web download:

intel_rs_sdk_runtime_websetup_YYYY.exe --rear --silent --no-progress --acceptlicense=yes --finstall=core,3ds --fnone=all

Detect presence of an F200 camera and Install the face runtime silently:

intel_rs_sdk_runtime_YYYY.exe --f200 --silent --no-progress --acceptlicense=yes --finstall=core,face3d --fnone=all

Detecting The Intel RealSense Camera Configuration at Runtime

After verifying proper camera setup at install time, verify the capture device and driver version (that is, Intel RealSense Depth Camera Manager (DCM) version) during the initialization of your application. To do this, use the provided mechanisms in the Intel RealSense SDK such as DeviceInfo and the ImplDesc structures. Note that the device information is only valid after the Init function of the SenseManager interface.

Checking the Camera Model

To check the camera model at startup, use the QueryDeviceInfo function, which returns a DeviceInfo structure. The DeviceInfo structure includes a DeviceModel member variable that includes all supported camera models available. Note that the values enumerated by the DeviceModel include predefined camera models that will change as the SDK evolves. You will want to verify that the SDK version on which you are compiling your application is recent enough to include the appropriate camera model that your application requires.

Code sample 1 illustrates how to use the QueryDeviceInfo function to retrieve the currently connected camera model in C++. Note that the device information is only valid after the Init function of the SenseManager interface.

Code Sample 1:Using DeviceInfo to check the camera model at runtime.

// Create a SenseManager instance
PXCSenseManager *sm=PXCSenseManager::CreateInstance();
// Other SenseManager configuration (say, enable streams or modules)
...
// Initialize for starting streaming.
sm->Init();
// Get the camera info
PXCCapture::DeviceInfo dinfo={};
sm->QueryCaptureManager()->QueryDevice()->QueryDeviceInfo(&dinfo);
printf_s("camera model = %d\n", dinfo.model);
// Clean up
sm->Release();

Checking The Intel RealSense Depth Camera Manager Version At Runtime

The Intel RealSense SDK also allows you to check the DCM version at runtime (in addition to the SDK runtime and individual algorithm versions). This is useful to ensure that the required Intel® RealSense™ technologies are installed. An outdated DCM may result in unexpected camera behavior, non-functional SDK features (that is, detection, tracking, and so on), or reduced performance. In addition, having the latest Gold DCM for the Intel RealSense camera SR300 is necessary to provide backward compatibility for apps designed on the F200 camera (the latest SR300 DCM must be downloaded on Window 10 machines using Windows Update). An application developed on an SDK earlier than R5 2015 for the F200 camera should verify both the camera model and DCM at startup to ensure compatibility on an SR300 machine.

In order to verify the camera driver version at runtime, use the QueryModuleDesc function, which returns the specified module’s descriptor in the ImplDesc structure. To retrieve the camera driver version, specify the capture device as the input argument to the QueryModuleDesc and retrieve the version member of the ImplDesc structure. Code sample 2 illustrates how to retrieve the camera driver version in the R5 version of the SDK using C++ code. Note that if the DCM is not installed on the host system, the QueryModuleDesc call returns a STATUS_ITEM_UNAVAILABLE error. In the event of a missing DCM or version mismatch, the recommendation is to instruct the user to download the latest version using Windows Update. For full details on how to check the SDK, camera, and algorithm versions, please reference the topic titled Checking SDK, Camera Driver, and Algorithm Versions on the SDK documentation website.

Code Sample 2:Using ImplDesc to get the algorithm and camera driver versions at runtime.

PXCSession::ImplVersion GetVersion(PXCSession *session, PXCBase *module) {
    PXCSession::ImplDesc mdesc={};
    session->QueryModuleDesc(module, &mdesc);
    return mdesc.version;
}
// sm is the PXCSenseManager instance
PXCSession::ImplVersion driver_version=GetVersion(sm->QuerySession(), sm->QueryCaptureManager()->QueryCapture());
PXCSession::ImplVersion face_version=GetVersion(sm->QuerySession(), sm->QueryFace());

Developing For Multiple Front-Facing Camera Models

Starting with the R5 2015 SDK release for Windows, a new front-facing camera model, named the Intel RealSense camera SR300, has been added to the list of supported cameras. The SR300 improves upon the Intel RealSense camera model F200 in several key ways, including increased tracking range, lower power consumption, better color quality in low light, increased SNR for the IR sensor, and more. Applications that take advantage of the SR300 capabilities can result in improved tracking quality, speed, and enhanced responsiveness over F200 applications. However, with the addition of a new camera in the marketplace comes increased development complexity in ensuring compatibility and targeting specific features in the various camera models.

This section summarize the key aspects developers must know in order to write applications that take advantage of the unique properties of the SR300 cameras or run in backward compatibility mode with only F200 features. For a more complete description of how to migrate F200 applications to SR300 applications, please read the section titled Working with Camera SR300 on the SDK documentation website.

Intel RealSense camera F200 Compatibility Mode

In order to allow older applications designed for the F200 camera to function on systems equipped with an SR300 camera, the SR300 DCM (gold or later) implements an F200 compatibility mode. It is automatically activated when a streaming request is sent by a pre-R5 application, and it allows the DCM to emulate F200 behavior. In this mode if the application calls QueryDeviceInfo, the value returned will be “F200” for the device name and model. Streaming requests from an application built on the R5 2015 or later SDK are processed natively and are able to take advantage of all SR300 features as hardware compatibility mode is disabled.

It is important to note that only one mode (native or compatibility) can be run at a time. This means that if two applications are run, one after the other, the first application will determine the state of the F200 compatibility mode. If the first application was compiled on an SDK version earlier than R5, the F200 compatibility mode will automatically be enabled regardless of the SDK version of the second application. Similarly, if the first application is compiled on R5 or later, the F200 compatibility mode will automated be deactivated and any subsequent applications will see the camera as an SR300. Thus if the first application is R5 or later (F200 compatibility mode disabled) but a subsequent application is pre-R5, the second application will not see a valid Intel RealSense camera on the system and thus will not function. This is because the pre-R5 application requires a F200 camera but the DCM is running in native SR300 mode due to the earlier application. There is currently no way to overwrite the F200 compatibility state for the later application, nor is it possible for the DCM to emulate both F200 and SR300 simultaneously.

Table 1 summarizes the resulting state of the compatibility mode when multiple Intel RealSense applications are running on the same system featuring an SR300 camera (application 1 is started before application 2 on the system):

Table 1: Intel RealSense camera F200 Compatibility Mode State Summary with Multiple Applications Running

Developing Device-Independent Applications

To accommodate the arrival of the Intel RealSense camera SR300, many of the 2015 R5 Intel RealSense SDK components have been modified to maintain compatibility and to maximize the efficiency of the SR300’s capabilities. In most cases, developers should strive to develop camera-agnostic applications that will run on any front-facing camera to ensure maximum portability across various platforms. The SDK modules and stream interfaces provide the capability to handle all of the platform differentiation if used properly. However, if developing an application that uses unique features of either the F200 or SR300, the code must identify the camera model and handle cases where the camera is not capable of those functions. This section outlines the key details to keep in mind when developing front- facing Intel RealSense applications to ensure maximum compatibility.

SDK Interface Compatibility

To maintain maximum compatibility between the F200 and SR300 cameras, use the built-in algorithm modules (face, 3DS, BGS, and so on) and the SenseManager interface to read raw streams without specifying any stream resolutions or pixel formats. This approach allows the SDK to handle the conversion automatically and minimize necessary code changes. Keep in mind that the maturity levels of the algorithms designed for SR300 may be less than those designed for F200 given that the SR300 was not supported until the 2015 R5 release. Be sure to read the SDK release notes thoroughly to understand the maturity of the various algorithms needed for your application.

In summary, the following best practices are recommended to specify a stream and read image data:

• Avoid enabling streams using specific configuration (resolution, frame rate):

  sm->EnableStream(PXCCapture::STREAM_TYPE_COLOR, 640, 480, 60);

  Instead, let the SenseManager select the appropriate configuration based on the available camera model:

  sm->EnableStream(PXCCapture::STREAM_TYPE_COLOR);

• Use the Image functions (such as AcquireAccess and ExportData) to force pixel format conversion.

  PXCImage::ImageData data;

  image->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_RGB32,&data);

• If a native pixel format is desired, be sure to handle all cases so that the code will work independent of the camera model (see SoftwareBitmapToWriteableBitmap sample in the appendix of this document).

• When accessing the camera device properties, use the device-neutral device properties as listed in Device Neutral Device Properties.

Intel RealSense SDK Incompatibilities

As of the R5 2015 Intel RealSense SDK release, there remain several APIs that will exhibit some incompatibilities between the F200 and SR300 cameras. Follow the mitigation steps outlined in Table 2 to write camera-independent code that works for any camera:

Table 2: Mitigation Steps for Front-Facing Camera Incompatibilities

Conclusion

This paper outlined several best-known practices to ensure high compatibility across multiple Intel RealSense camera models. The R5 2015 SDK release for Windows features built-in functions to mitigate compatibility. It is generally good practice to design applications to use only common features across all cameras to facilitate development time and ensure portability. If an application uses features unique to a particular camera, be sure to verify the system configuration both at install time and during runtime initialization. In order to facilitate migration of applications developed for the F200 camera to SR300 cameras, the SR300 DCM includes an F200 compatibility mode that will allow legacy applications to run seamlessly on the later-model camera. However, be aware that not updating legacy apps (pre-R5) may result in failure to run on SR300 systems running other R5 or later applications simultaneously. Finally, it is important to read all supporting SDK documentation thoroughly to understand the varying behavior of certain SDK functions with different camera models.

Resources

Intel RealSense SDK Documentation

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_devguide_introduction.html

SR300 Migration Guide

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_mgsr300_working_with_sr300.html

Appendix

SoftwareBitmapToWriteableBitmap Code Sample

// SoftwareBitmap is the UWP data type for images.
public SoftwareBitmapToWriteableBitmap(SoftwareBitmap bitmap,
WriteableBitmap bitmap2)
{
switch (bitmap.BitmapPixelFormat)
{
default:
using (var converted = SoftwareBitmap.Convert(bitmap,
BitmapPixelFormat.Rgba8))
converted.CopyToBuffer(bitmap2.PixelBuffer);
break;
case BitmapPixelFormat.Bgra8:
bitmap.CopyToBuffer(bitmap2.PixelBuffer);
break;
case BitmapPixelFormat.Gray16:
{
// See the UWP StreamViewer sample for all the code.
....
break;
}
}
}

About the Author

Tion Thomas is a software engineer in the Developer Relations Division at Intel. He helps deliver leading-edge user experiences with optimal performance and power for all types of consumer applications with a focus on perceptual computing. Tion has a passion for delivering positive user experiences with technology. He also enjoys studying gaming and immersive technologies.