Tips and Tricks for Intel® RealSense™ Background Segmentation
The 5.0 release of the Intel® RealSense™ SDK includes improved hair and finger and hand segmentation. I also has lessened the module's power consumption and provided more usage cues, including user is too close, too far, or had an entry event. It also adds optional callback support.
However, there are still some "known issues" and tips for getting better results:
- Don't turn on any other module while using 3D Background Segmentation.
- Long dark hair segments better with a plain background of a different color.
- Black objects segment better when placing the camera's "eye level" at the user’s head.
- Avoid glossy or IR absorbing materials (glasses, watches, headphones, hats/scarves) as they degrade mask quality.
- Don't touch the background. Similarly don't have hand against face or head (or another object). Separate the targets, so they're distinct. (This can also happen at "very close" to touching.)
- Keep user in FOV. If user touches target objects and then leaves FOV, it's best to break continuity by covering the camera or moving the objects. Also try to center the user (especially away from right edge). May also need to change resolution..
- Focus on one person at a time. Mask quality degrades when multiple people are in range.
- Targets should be thicker than 2mm. Thin objects (1-2 mm) are not segmented correctly
Resources:
- Bryan's excellent code sample Background Replacement Code Sample Using Intel® RealSense™ Technology
- Tutorial on BGS: Background_Segmentation.pdf