Strategies for Battery-Friendly Intel® RealSense™ Applications

Download Document

Abstract

This paper presents several techniques for developing power efficient applications using the Intel® RealSense™ Software Development Kit (SDK). The SDK allows users to interact with the computer utilizing natural input such as hand gestures, facial expressions, and voice recognition. The paper is intended for novice and intermediate developers who want to develop Intel RealSense applications primarily targeting battery-powered devices such as laptops and tablets.

Acronyms and Abbreviations

Definitions

System Call– a call/request made by the application to the OS kernel. Due to the OS overhead of transitioning between user and kernel mode, reducing the number of system calls is desirable for faster system performance.

Context Switch– a procedure used to switch from one executing thread to another. Due to the OS overhead associated with updating the active thread, reducing the number of context switches is desirable for faster system performance.

Page Fault– occurs when a thread requests a page in memory that is not in the OS working set. Due to the OS overhead associated with the resolution of page faults, reducing the number of page faults is desirable for faster system performance.

Interrupt– a signal to the processor that other work requires immediate attention. Due to the OS overhead associated with transitioning between processes, reducing the number of interrupts is desirable for faster system performance.

Processor Queue– lists of threads ready to be executed. A smaller processor queue is desirable for faster system performance.

Total CPU Utilization– sum of CPU utilization from all running processes on the system

Host Process CPU Utilization– CPU utilization of just the application process

Package Power– combined power generated by CPU, GPU, and other chipset components

IA Power– power generated by Intel® Architecture-based CPU

Table of Contents

1 Introduction.

2 Importance Of Power Management In Apps.

3 Power Savings Mechanisms In RealSense™ SDK.

   3.1 Disable RealSense™ Camera When Not Needed.

   3.2 Enable Only Required RealSense™ Modules When Needed.

   3.3 Controlling Power State of the SDK Modules.

   3.4 Utilize Pause Module Feature.

      3.4.1 Power and Performance Savings from Pausing Modules.

   3.5 Conclusion.

   3.6 Resources.

   3.7 Definitions.

1 Introduction

The Intel RealSense Software Development Kit (SDK) allows developers to design applications utilizing Intel RealSense technology to enhance the way users interact with the computer. There are multiple modules built into the SDK that provide natural input mechanisms such as hands, face, and speech. Most Intel® RealSense™ hardware devices come in sleek, portable, and power sensitive form factors such as Ultrabook™ 2 in 1 (combined laptop and tablet design), clamshell, and convertible types. When designing apps for these platforms, power management is critical to ensure the best possible user experience and optimal runtime performance. This article discusses a few simple but powerful techniques to optimize power in Intel RealSense apps.

2 Importance Of Power Management In Apps

Before discussing the specific power savings mechanisms in the SDK, let’s first review why minimizing power is important. For mobile platforms that run on battery power, the longer the device can run between charges, the more desirable the user experience. Generally, we want to minimize the chance that the user will be interrupted due to the battery dying in the middle of an activity and also minimize time spent charging the device as opposed to using it. A power cost is incurred every time a piece of software leverages system resources such as CPU cycles, GPU acceleration, and I/O ports. Thus, an app can indirectly manage system power usage by managing the loads on the various system components.

The Thermal Design Power (TDP) is a term used to describe the maximum amount of heat a CPU would generate while running real-world applications. The TDP value is given in Watts (W), and it is often used to design efficient cooling solutions to keep the amount of heat present on the package to a desired level. While TDP does not reflect the peak power that a CPU can hit before failing, it is important for software developers to be aware of the figure since the CPU’s power policy is to maintain the TDP value even if it means sacrificing performance. Furthermore, this power throttling behavior is managed at the system level and is not in the application’s control.

On Intel® chipsets, several built-in power management mechanisms are designed to maintain a package power that does not exceed the TDP value for an extended amount of time. If the TDP is reached, the CPU frequency may be reduced in an effort to reduce the amount of heat output. This reduction in CPU frequency can have an adverse effect on the Intel RealSense SDK performance since the processing is performed exclusively on the CPU. It is also important to note that developers should not assume that their Intel RealSense applications are the only user process running on the host machine at any time and thus the power usage should allow some headroom for a multi-application environment. For example, if an Intel Real Sense application is targeting low power platforms with a TDP of 15W, then it is recommended that the app’s maximum package power not exceed 10W-12W.

Figure 1.Example of the package power usage by an Intel® RealSense™ application illustrating the effect of a TDP limit on the measured package power. The system reduces the resources required by app(s) to maintain the 15W package power limit.

3 Power Savings Mechanisms in the Intel® RealSense™ SDK

The Intel RealSense SDK provides several capabilities for an application to efficiently manage its power during runtime. Some of the best techniques include:

1. Disable the Intel RealSense 3D Camera when not need by the application
2. Enable SDK modules needed by the app only when required
3. Utilize the PXCPowerState interface to set the power state of SDK modules
4. Utilize the PauseModule function to suspend the module processing when possible

Each of these topics is further explained in the sections below.

3.1 Disable Intel® RealSense™ 3D Camera When Not Needed

Several early Intel RealSense applications initialized the camera at the app startup and left it running during the app’s entire runtime. However, the camera should only be running when the SDK functionality is required with either raw streams being processed or SDK modules executing some form of tracking or detection of the user. With the camera running, the DCM (Depth Camera Manager) process will consume CPU resources that impact both the package power and CPU performance. For example, on a typical 4th gen Intel® Core™ i5 processor (codenamed Haswell)-based machine, the measured overhead of DCM on the gold release of the SDK is approximately 0.2 W and 2% of CPU cycles (the DCM overhead has been reduced greatly since the alpha and beta SDK releases). Some common scenarios where it is advisable to disable the Intel RealSense 3D camera are as follows:

• Initial splash screens that display after an application is first started
• Menu screens that require only mouse and keyboard or touch controls
• Scenes utilizing Intel RealSense modules that do not require a camera stream (i.e., speech)

One thing to consider when deciding if and when to disable the camera is the duration of time for which other Intel RealSense SDK modules will be inactive in the context of the application. A time overhead is associated with reinitializing the pipeline and restarting the streams that is typically on the order of several seconds. In some cases, this time delay may disrupt the experience and a more immediate stop/restart cycle is desired. For these cases, consider using the PauseModule feature, which is detailed in section 3.4 below.

3.2 Enable Only Required Intel® RealSense™ Modules When Needed

In several cases, an application would call EnableXXXX at startup to enable all of the Intel RealSense modules that will be used throughout (where XXXX is the name of the module such as hand or face). For example, if a game used hand, face, and emotion modules at different parts of an activity, all 3 modules would be enabled during the app’s initialization and left running during all scenes. This effectively means that all three modules are running their algorithms on the camera streams simultaneously within the pipeline even if the results are not being presented to the application. This practice will incur significant power and performance overhead during runtime. The recommended approach is to only enable the module you need when it is required in the app. For example, if a face tracking mini-game is only triggered a fraction of the runtime, then the EnableFace function should not be called until right before that mini-game begins.

The table below illustrates the significant savings from avoiding the case of having multiple SDK modules running concurrently. The data below was captured when running an Intel RealSense application on the Alpha SDK release during an idle menu screen with no other SDK activity. Before deactivating the modules as an optimization, the application was running the hand, face, and emotion modules during menus requiring no user input. The performance metrics were captured using the TypePerf command line tool that is included in Windows and can be used to write a variety of system performance counters out to a command window or a specified log file (please reference the official TypePerf Documentation for more details on the tool). For definitions on metrics listed in the table, please refer to the Definitions section at the top of this document.

Table 1.System metrics illustrating savings from deactivating inactive SDK modules, including the hand, face, and emotion libraries

3.3 Controlling the Power State of the SDK Modules

The SDK provides a built-in mechanism to manage the power and performance of an app. This can be done using the PXCPowerState interface, which allows the app to explicitly define either a performance or battery state for a supported module. The default state is battery mode, which places the control in the module to determine if there is a power saving opportunity during runtime (e.g., no hand in the view of the camera). If the SDK module has a chance to do so, it may attempt to save power and performance by lowering the frame rate or using a less intensive processing algorithm. Similarly, the app can set the power state to performance that maximizes available computation resources to ensure the best possible experience. Code sample 1 below shows how to use the PXCPowerState interface to set the power state.

// session is a PXCSession instance
PXCPowerState *ps=session->QueryInstance();
ps->SetState(PXCPowerState::STATE_BATTERY);

Code sample 1.Using the PXCPowerState Interface to set the power state to battery.

It is important to note that the performance mode is intended to be used only for brief periods of time when extra quality is required due to the additional overhead incurred on the system’s resources. It is recommended that the power state remain in battery mode as much as possible. To facilitate this, the PXCPowerState interface exposes a member function called SetInactivityInterval(pxcI32 interval), which specifies a time period (in seconds) after which battery mode will be engaged if there is no activity for the module. For example, calling SetInactivityInterval(5) for the hand module will automatically change the power state to battery if there is no hand in the scene for 5 seconds or more. Code sample 2 below illustrates a typical usuage of the SetInactivityInterval function.

// ss is a PXCSession instance.
PXCPowerState *ps=ss->CreatePowerManager();
// Set the power state
ps->SetState(PXCPowerState::STATE_PERFORMANCE);
// Set the inactivity interval.
ps->SetInactivityInterval(5);
// Clean up
pp->Release();

Code sample 2.Using the PXCPowerState inactivity interval to manage the power state. This is the recommended approach when using the STATE_PERFORMANCE power state.

3.4 Utilize Pause Module Feature

The SDK provides the capability to temporarily suspend a module to save system resources at opportune times. This is done via the PauseModule function, which is a member function of the PXCSenseManager interface. If a module is in a paused state, the SDK’s pipeline no longer delivers any input samples to that module. Note that the pause capability only applies to the SDK modules and not the 3D camera or capture device. Code sample 3 below shows the base function for pausing a module.

void PauseModule(pxcUID ID, pxcBool pause);

Code sample 3.Base function for executing a pause state for a specified module.

The SDK provides helper functions to pause each individual module and are named accordingly. For example, to pause the face module, the app should invoke the following function:

void PauseFace(pxcBool pause);

Code sample 4.Function for executing a pause state for the face module.

The app can control whether a module is in a pause or resume state by changing the value of the ‘pause’ input parameter. Setting the parameter to true will pause the module while setting it to false will resume it. For example, invoking PauseFace(true) will pause the face module. To resume it, the app just needs to call PauseFace(false).

The benefit of pausing a module as opposed to disabling a module is primarily in the reduction of the restart/resume delay. When disabling an app, the restart requires the pipeline to be reinitialized which can take several seconds. However, when a module is paused, the resume is nearly instant with processing continuing during the next frame. This pausing technique is great to use when SDK activity is not needed for a short period of time. For example, if users’ hands are tracking an activity but then they press the esc key to bring up the game’s main menu, pausing the hand module would allow for seamless resume state while saving resources for the few seconds the users navigate menus via keyboard/mouse controls.

3.4.1 Power and Performance Savings from Pausing Modules

The power and performance savings from utilizing the pause feature will vary from app to app depending on the software design. In general, the amount of CPU time and power saved is a function of the relative weight of the SDK processing within the context of the app. For example, more complex apps such as games with 3D graphics and lots of audio may save less since there is still much processing to perform per frame outside of the SDK processing. When testing some of the SDK sample apps, a savings of about 1.5W-2W in package power and ~33% CPU reduction in CPU time was measured. Table 2 below shows the savings when using pause on the face module in the face_viewer.exe sample app included in the SDK package.

Table 2.CPU Power and Performance Savings from utilizing PauseModule feature for the Face_Viewer SDK sample app Pause SDK Face Sample

Unfortunately, the SDK sample apps are not representative of a full retail app due to their limited scope (single module focus) and minimal processing aside from the SDK’s. Savings will probably be less in most real-world cases. In testing a few Intel RealSense apps, pausing the hand module resulted in a minimum of approximately 8-10% CPU time and 1W of power saved. Note that in most cases, pausing a module will save less resources than completely disabling it since the pipeline is still active and only the samples are not being transmitted to the module. As mentioned in section 3.1 above, the advantage of using pause is in the much reduced delay in resuming the module processing compared to a complete disabling. Which solution is optimal should be determined on a case-by-case basis dependent on the power and performance targets, software design, execution flow, and desired use cases of an application.

3.5 Conclusion

This paper describes how to use the Intel RealSense SDK features to efficiently manage the power and performance usage of an application. We reviewed the importance and impact that power utilization can have on performance, presented several techniques to minimize power usage, and reviewed some data quantifying the savings in several cases. Ultimately, quantifying the savings is a complex endeavor due to the dependency on variables such as machine specifications, application architecture, and the Intel RealSense modules being used. It is clear however that utilizing the SDK features appropriately can make the difference between an optimized application delivering a great user experience and an inefficient one delivering inconsistent or subpar performance. This paper summarizes some of the learnings from the first early Intel RealSense applications with regards to power management and performance tuning. As developers become more familiar with the SDK and its performance profiles continue to improve, some of these techniques may become outdated and new techniques will take precedence. As a developer, the key is to be mindful that all of the SDK’s functionality has an associated cost. When designing an app, it is important to be selective when deciding which SDK modules to use and how long to make them active.

3.6 Resources

Intel® RealSense™ SDK Documentation:
https://software.intel.com/sites/landingpage/RealSense™/camera-sdk/2014gold/documentation/html/

Power Management States: P-States, C-States, and Package C-States
https://software.intel.com/en-us/articles/power-management-states-p-states-c-states-and-package-c-states#_Toc383778909

About the Author

Tion Thomas is a software engineer in the Developer Relations Division at Intel. He helps deliver leading-edge user experiences with optimal performance and power for all types of consumer applications with a focus on perceptual computing. Tion has a passion for delivering positive user experiences with technology. He also enjoys studying gaming and immersive technologies.