Face Beautification API for Intel® Graphics Technology

Download sample code [16MB]

Abstract

This paper highlights the C++ API for enabling applications to support Face Beautification, which is one of the features supported by Intel® Graphics Technology. It outlines of the list of available effects in the Face Beautification Library Version 1.0, and describes C++ API definitions and methods included in the library.

Face Beautification in Intel® Graphics Technology

The Face Beautification feature supported by Intel Graphics Technology provides the capability to automatically adjust facial landmarks. Using the current implementation of the APIs, you can create an automatic framework and develop face enhancement tools for a better user experience. Because there is a lot of information that can be extracted from an image, the API helps to implement automatic Face Beautification.

There are two methods for enabling Face Beautification in an application. The first version of Face Beautification support was available through Device Driver Interface (DD)I implementations; applications could be enabled by a call into the private DDI. Now, we have a second and simpler version, using a C++ API that can assist application development. Developers can access the C++ API via the Face Beautification static library. The Face Beautification feature set is detailed in the table below.

API Definitions

There are five APIs for Face Beautification. The current infrastructure supports one Face Beautification feature i.e. FBRedLip. There are three structures that store the input and output properties, FDFB mode feature, feature strength, and other parameters. The constructor initializes the class data members based on information provided by the structure. The first API initializes the device using fDeviceInitialization() followed by fConfiguration() for setting the device properties based on structures passed to the constructor. There is a separate API provided for Face Detection and Face Beautification (FDFB) mode - FDFBMode_Initialization(). After the device initialization and device configuration, the pipeline is executed for each frame using the ExecutionPipeline() API. The destructor is called automatically to release the memory object.

FB_API(FDFB_IN_OUT_PARAM init_file_var, FDFB_REDLIP_PARAM FBRedLip, FDFB_MODE_PARAMS FDFB_Mode_Val);
void fDeviceInitialization();
void fConfiguration();
void FDFBMode_Initialization();
void ExecutionPipeline(char* tempBuffer);
int convertFileToFaceList(std::fstream& file, std::vector<VPE_FACE_RECT>& list);
~FB_API();

The details of the structure used by the API’s is provided below:

typedef DXGI_FORMAT FDFB_FORMAT;
typedef struct FDFB_IN_OUT_PARAM
{
 FDFB_FORMAT inputFormat;
 FDFB_FORMAT outputFormat;
 UINT inputWidth;
 UINT inputHeight;
 UINT outputWidth;
 UINT outputHeight;
} FDFB_IN_OUT_PARAM;

typedef struct FDFB_REDLIP_PARAM
{
 UINT FBRedLipStrengthEnable;
 UINT FBRedLipStrength;
} FDFB_REDLIP_PARAM;

typedef struct FDFB_MODE_PARAMS
{
 GUID * pVprepOperationGUID;
 VPE_FDFB_MODE_ENUM FDFBMode;
 VPE_FD_FACE_SELECTION_CRITERIA_ENUM faceSelectionCriteria;
 std::vector<vpe_face_rect> list;
} FDFB_MODE_PARAMS;</vpe_face_rect>

Usage and Program Flow

1. The Face Beautification header file and static library (.lib) are provided. Create the project and include the header file in the additional include directory, add .lib to additional library directories, and add the name of the static library to the additional dependencies on the Input tab.
2. In the application, provide the input file, the output file, and the face file.
3. Provide or read properties into variables like input width, input height, output width, output height, input format, and output file format. If the application wants to enable FDFB mode, provide which feature is enabled and its strength. Set the face selection criteria. If face selection criteria and FB feature strength is not provided, then the driver uses the default values. Use convertFileToFaceList() to convert the face file to list format.
4. Create a class object and pack the information into structures. Call the class constructor to initialize the value of class data members.
5. Call the device initialization function fDeviceInitialization().
6. Call the device configuration function fConfiguration().
7. Call the FDFB mode initialization function FDFBMode_Initialization().
8. Execute the pipeline by calling ExecutionPipeline(char* tempbuffer). This function loops for all the frames. A new buffer is passed for every new frame; update the buffer accordingly.
9. Write the output to the output file. A class destructor is called automatically to release the memory objects.

Future Work

The current API implementation supports one FDFB feature, FBRedLip. The upcoming versions will include a larger set of Face Beautification features. If requested, face detection support will be included as well. The DDI implementation support takes virtual camera input; this feature can be extended to the C++ API as well.

About the Author

Sonal Sharma is a software application engineer working at Intel in California. Her work responsibility includes performance profiling and analysis, and CPU/GPU code optimization for media applications.