Download PDF [1.13MB]
Introduction
In this article, we’ll explore the strides Adobe engineers have made over the last few years to enhance Photoshop using OpenGL* and OpenCL™ to increase hardware utilization. The Adobe team selected two features—Blur and Smart Sharpen—as the focus of its recent efforts because both provided less than optimal processing speed and quality. We will discuss the results on those features in this paper.
Open Standards Languages, Adobe Photoshop, and Intel
OpenGL (Open Graphic Language) has been used for years to boost rendering performance and resource utilization across platforms. OpenCL (Open Computing Language) is a royalty-free open standard for portable general purpose parallel programming across central, graphics and other processors. The OpenCL standard is a complement to the existing OpenGL APIs that adds general computation routines to OpenGL’s use of graphics processors for rendering work. OpenCL gives developers a uniform programming environment to execute code on all processing resources within a given platform.
Adobe Photoshop is a leading graphics industry application used for graphics editing and manipulation. A heavy processing and memory resource user, Photoshop is a powerful application that requires the greatest performance possible from a computer. To aid in its graphics processing capabilities, Adobe has used open standards for many generations of Photoshop. It has now been updated to take advantage of OpenCL, which allows for an even higher level of performance.
Intel provided testing for this report. Intel also makes available an array of tools and SDKs to accelerate development for visual computing. These include a Developer’s Guides for Intel® Processors (links to guides for each generation of Intel graphics processors on the page). The latest guide is the Graphics Developer's Guide for 4th Generation Intel® Core™ Processor Graphics – now includes OpenGL), the Intel® SDK for OpenCL Applications, and a web site dedicated to visual computing.
For a powerful application, like Photoshop, using open standards like OpenGL and OpenCL can improve performance and allow the processing routines to be used across platforms and with other Adobe products more easily.
Photoshop’s Use of OpenGL and OpenCL Standards
A few years ago, in Adobe’s Creative Suite* 4 release of Photoshop (Photoshop CS4), Adobe developers focused their OpenGL efforts on enhancing the Canvas and 3D interactions. They implemented Smooth Zoom, Panning, Canvas Rotate, Pixel Grid, and 3D Axis/Lights using the OpenGL API to improve the performance. Turn these features ON (in Preferences), enable “Use Graphics Processor, select the “Advanced Settings” button, select “Advanced Drawing Mode” from the dropdown menu, and enable the checkboxes for “Use Graphics Processor to Accelerate Computation” and “Use OpenCL.“ Refer to Figure 1 and Figure 2 for recommended settings in the Photoshop user interface.
Figure 1: Graphics Settings in Photoshop* Preferences Dialog
Figure 2: Select "Advanced" in the Drawing Mode Dropdown
With Photoshop CS5, the developers used OpenGL to speed up the user interface (UI) and to add the Pixel Bender plug-in. The specific UI features targeted were the Scrubby Zoom, HUD Color Picker, Color Sampling Ring, Repousse, and 3D Overlays. With these new features, the OpenGL modes were expanded to encompass basic, normal, and advanced methods.
Then in Photoshop CS6, the team enhanced content editing with standards-based features from both OpenGL and OpenCL. Developers added Adaptive Wide Angle, Liquify, Oil Paint, Puppet Warp, Lighting Effects, and 3D Enhancements using OpenGL. The OpenCL standard was used to add a Field/Iris Tilt/Shift function as well as the Blur function.
Today, with Adobe’s latest release of their Creative Suite, Creative Cloud enhances the Photoshop application even further with Smart Sharpen and the selectable modes of “Use Graphics Processor” and “Use OpenCL.”
Intel HD Graphics Devices that support OpenGL and OpenCL
The following Intel graphics devices should be enabled for OpenGL and OpenCL graphics acceleration by default:
- 4th generation Intel® Core™ processors
- Intel® HD Graphics 4200, 4400, 4600, P4600, P4700
- HD Graphics 5000
- Iris™ Graphics 5100
- Iris™ Pro Graphics 5200
- 3rd gen Intel® Core™ processors
- Intel® HD Graphics 4000, P4000
- 2nd gen Intel® Core™ processors
- Intel® HD Graphics 3000, P3000
Developing Specific Photoshop Enhancements
How did the developers achieve these results? Photoshop uses a system of layers to apply many of its advanced features to an image. Figure 3 illustrates this concept by showing three layers of a very simple image as the WHITE space. The EFFECTS, the RED and BLUE layers in the example, are separate layers in the stack. Effects that can be applied in layers include Sharpen, Blur, and even Red-eye Removal. Effects can be applied to the final image without changing the original file. These layers can also be ordered, stacked, and combined to provide a blended effect as the right side of the figure shows, e.g., combining red and blue to get PURPLE. Additionally, there are special layers called “mask layers” that allow you to restrict an effect’s application to a select region of the image.
Figure 3: Separate (Left) and Combined (Right) Layers
The image-combining aspect also applies to Photoshop textures. A Photoshop texture refers to the content of a layer that is then blended or overlaid with other layers to “texture” an image. Notice how the bricks from the image on the lower left below provide a texture to the cloak of the statue in the middle image when applied with a small percentage of opacity.
Figure 4: Example of Using Texture in Photoshop*
Adobe used an OpenGL API to enhance the Photoshop texture/layer effect. In the OpenGL Advanced Mode, the OpenCL “– int format –GL_RGBA16F_ARB” call enables the Shader Tool to apply checkerboard compositing, tone mapping, and color matching.
Sharpen the Focus
Sports photographers use the Sharpen step extensively, as just a few increments on the controls can make a world of difference in the impact of an action shot. Figure 5 demonstrates how detail can be improved by applying a Sharpen step in photo editing. Notice the text, stars, and even brush stroke detail is a little more pronounced in the “After” image on the right.
Figure 5: Original Image (Left) and After Sharpen (Right)
However, the Sharpen step can create some unwanted side effects. Details that are relatively sharp or insignificant in the original image can develop artifacts akin to boosting the image “noise,” producing a halo effect. For this release, Adobe renovated the legacy Smart Sharpen by introducing a patch-based “denoise and sharpen” algorithm implemented using the OpenCL standard. The new patch-based algorithm produces a sharpened image without any halo effect. Furthermore, the denoise step suppresses the “noise gets boosted when you sharpen” issue. Compare the images in Figure 6, Figure 7, and Figure 8 below. With this result, Adobe looks forward to using these standards to further improve all the sharpen tools.
Figure 6: Original text image
Figure 7: Image after applying legacy smart sharpen w/halo effect
Figure 8: Image after applying patch-based smart sharpen
Bringing Blur into Focus
Another editing tool function that was improved by using OpenCL was the Blur tool. There are numerous ways to emphasize and de-emphasize a portion of an image. Many qualities can be influenced at the time the photograph is captured, but a photograph’s impact can be improved, or at least changed, with some post-processing. Red-eye removal and cropping are very common post-processing tasks, but image sharpness can also be improved. Image area-specific sharpness can have a large impact.
Figure 9: Mona Lisa (Portrait of Lisa Gherardini, wife of Francesco del Giocondo) by Leonardo da Vinci
In his masterpiece Mona Lisa, Leonardo da Vinci (Figure 9) [8] emphasized his subject in the portrait by placing her image in the foreground with a somewhat out-of-focus rural landscape behind her. By blurring the background, he helped the viewer focus on his subject, which was the most important part of the painting, not the background. Following is an example of how blurring can improve a more modern image. Finding the photograph’s main theme can be difficult, so blurring helps refine the image’s theme. I took the sharpened image used in Figure 6 and further emphasized the central star in the image by applying a Blur tool, which results in the clarity of the star in the image on the right below (Figure 10). Blurring changed our perspective of the image so that the star is obviously the focus. Suffice it to say, there are lots of ways to blur an image (on purpose), and this is one of the newest ways.
Figure 10: Original Image (left) Sharpen (center), then Blur (right) added for emphasis
Adding Blur to an image is much like using a color crayon, except the mouse is the crayon and the color is the Blur feature. To apply the Blur, you select the Blur tool, size the tool “brush” (a cursor that can be sized from 1 screen pixel to the size of the entire image) to match the size of the image region you wish to blur, and then click-and-hold the mouse while “coloring or scrubbing” over the area of the image you wish to blur. The more coloring action performed, the more blur applied to the image region.
Intel Increased Graphics Performance
The exercise of adding an OpenCL Blur tool was somewhat challenging and provided a few good learning opportunities. The team wanted to balance the workload by utilizing all the possible resources on the host platform. Cross-platform support is critical including Windows* and Mac* OSs. These factors led them to OpenCL. The team ended up taking an existing blur tool in Photoshop and porting it from optimized CPU code to OpenCL kernels.
Adobe looked to reduce the complexity required of Blur that before OpenCL required multiple command queues running on multiple threads. They also experienced resource limitations, such as timeouts and out-of-memory failures, on lower-end video subsystems. Finally, platform variations, like driver stacks and the use of various compilers, would be reduced by going to the OpenCL-based solution. OpenCL allowed them to reduce these challenges by making it possible to cache a portion of an image to local memory and break the images down to smaller 2k by 2k blocks for the graphics processor. These improvements resulted in higher reliability and a 4 to 8 times faster filter time by utilizing the GPU.
Intel’s testing shows performance gains on the following Photoshop actions, as the available execution units and memory bandwidth have increased over the generations of Intel HD graphics as shown in the chart below (Figure 11).[1]
Figure 11: Photoshop* with OpenGL* Performance over Generations of Intel HD Graphics
When tests are run with the OpenGL or OpenCL features enabled and disabled, we see the routines add significant performance improvement to both the Liquify Filter and the Field Blur tools in the graphs below (Figure 12). Liquify and Blur processing times normalized in seconds to 1 GPU acceleration off/on Intel® HD 4600.[1]
Figure 12: Photoshop* tool performance with Standards On/Off
The effort was well worth it. The performance of this new Blur function when tested with the OpenCL hardware acceleration ON versus OFF had a 3x faster processing time depending on the workload and the size of the radius being blurred (Figure 13).[1]
Figure 13: Sample blur execution time (in seconds) compared
General application processing accounts for the majority of time in smaller workloads, so larger workloads show a better improvement in processing time. When OpenCL acceleration is enabled, both the CPU and the GPU are efficiently utilized, with many of the multithreaded app’s cores submitting work to the graphics processor. The graphics processing unit is utilized at a minimum 70% rate while memory utilization is 10%-36% depending on the graphics subsystem. Finally, there were no stalls in the graphics pipeline making for an improved user experience.
Summary
Adding standards-based processing routines has allowed Adobe to continue its tradition of enhancing Photoshop performance with each release. With the addition of OpenCL-based acceleration on an Intel HD Graphics device, the user experiences an improvement in performance and gains an ability to evaluate the blur filter almost real-time across the entire image. This complete image experience was not possible before OpenCL was added to these filters, and this change makes creating compelling images much more efficient. Prior to the addition of OpenCL, only a small fraction of the image could be previewed before applying the effect. Similarly, users can review their smart sharpening filter as they make adjustments full screen and get to the desired final image faster. Now with OpenCL, Photoshop is clearly better.
[1]Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance.
References and Resources
- OpenGL Specification and Other Resources at khronos.org
- OpenCL Specification and Other Resources at khronos.org
- Adobe Photoshop
- Intel® SDK for OpenCL Applications
- Intel® Visual Computing SOURCE on Intel® Developer Zone
- Developer’s Guides for Intel® Processors
- Developer’s Guide for Intel® Processor Graphics for 4th Generation Intel Core Processors
- Intel’s OpenCL videos and presentations from SIGGRAPH 2013
- Accelerating content creatio n with Intel Iris Graphics, Intel® Performance Tools and OpenCL
- Video: Adobe optimizes with OpenCL* and Intel® Graphics
- Video: Adobe Blur Tools Combined
- Video: Sharpening and perfecting your photos with Smart Sharpen
About the Authors
Tim Duncan is an Intel Engineer and is described by friends as “Mr. Gidget-Gadget.” Currently helping developers integrate technology into solutions, Tim has decades of industry experience, from chip manufacturing to systems integration. Find him on the Intel® Developer Zone as Tim Duncan (Intel)
Murali Madhanagopal is a member of the Intel Visual & Parallel Computing Group, where he is a Lead Graphics Architect. He received his M.S. in Computer Information Systems from Texas A&M University, College Station and has a bachelor’s degree in Computer Engineering from the College of Engineering Guindy, Anna University, India. Madhanagopal is responsible for developing and executing Intel’s workstation processor graphics strategy that enables ISV’s software to run efficiently on current and future processor graphics-based platforms. He is actively engaged in application and system optimization activities with industry-leading CAD, CAE, and Digital Content Creation ISVs and OEMs.
Intel, the Intel logo, and Iris are trademarks of Intel Corporation in the U.S. and/or other countries.
OpenCL and the OpenCL logo are trademarks of Apple Inc and are used by permission by Khronos.
Copyright © 2014 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.