Tablet Optimization and Touch Implementation on Codemasters’ GRID Autosport*
by Michael Coppock and Leigh Davies
Mobile gaming is big business. Estimates show high single-digit growth of mobile gaming through 2016. However with power-conserving tablets and Ultrabook™ devices, the typical mobile game is less ambitious than PC games of the same era. Codemasters, the UK-based game developer, aims to change that with GRID Autosport*. They want to bring console-level graphics to tablets with intuitive touch controls.
The collaboration between Codemasters and Intel allows GRID Autosport to take advantage of the entire 4th generation Intel® Core™ processor product line, including 4-watt Intel® Atom™ processor-based tablets. "It's really important for us to cater to the whole market," said Richard Kettlewell, a senior programmer at Codemasters’ headquarters in Southam, England. "We’ve made sure that the game has a set of comprehensive graphics options that can scale up to the top end, though still looks great if you’re using a tablet."
Herein is the story of how the team at Codemasters optimized GRID Autosport for tablets. How their ingenuity allows GRID Autosport to look great on a 4-watt Intel Atom processor-based tablet offering a visual experience that would not have looked out of place on a good PC a few years ago. While providing plenty of extras on high-end hardware such as an Iris Pro™ graphics-powered Ultrabook or PC as detailed in a previous article: Taking the Pole Position: Codemasters Leads the Pack in PC-to-Tablet Optimization with GRID Autosport. The challenge is to add touch controls to a dense front-end UI while keeping the experience seamless across tablets, Ultrabooks, and PCs.
Optimizing for Tablets
The graphics engine in GRID Autosport* is an evolution of the GRID 2 Engine. It has been heavily optimized for 4th gen Intel Core processors, typically found in Ultrabooks. GRID 2* was designed to run at Medium settings on the Intel® graphics in 4th gen Intel Core processors, which have comparable performance to the Playstation* 3 and Xbox 360*. Game performance is a stable 30FPS at a resolution of 1366x768 on a 15-watt Ultrabook. In addition to higher-quality settings for high-end gaming PCs, GRID 2 also provides several lower-quality settings. This ensures maximum compatibility on older hardware. To help make GRID 2 playable on a wide array of older and lower-power GPUs, Codemasters lowered the quality settings on existing effects, such as shadow resolution. Several post-processing features were also disabled, including features like crowds, driver models, and even particle effects that were not considered essential to the GRID 2 experience (Figure 1).
Figure 1:Features such as crowds, driver models, and particle effects were disabled on GRID 2 at lower quality settings.
To bring GRID Autosport to tablets with ultra-low voltage Intel Atom processors these lower-quality settings were used as a starting point. Initial testing was not promising. Using GRID Autosport’s built-in benchmark and the original ultra-low settings with crowd, drivers, and particles disabled, the game was running at under 20FPS. A typical frame took 53ms to render, far greater than the target of 30ms. Looking at a frame in more detail showed what was taking the most time to render (Figure 2).
Figure 2: On the left is a frame from GRID Autosport* at ultra-low quality settings. The right side shows pre-optimization render times in ms by feature.
It was apparent the traditional approach would not work. Just scaling model textures, creating more aggressive LODs, and removing geometry did not reach the required 30FPS. The removal of some effects, such as complex depth of field and motion blur in the final post-processing, meant many of the pixel shaders were doing redundant work. Work, that when removed, also took the requirement for a velocity/depth pre-pass with it. Removing the velocity/depth pre-pass was particularly important as it contributed ~30% of the total number of draw calls per frame and ~25% of the total render workload. The studio wanted to avoid writing a new shader version for every object in the game, so they looked at quickly prototyping a simpler render pass for tablets.
Fortunately, simple versions of the shaders are used in the dynamic car environment cubemap pass. This cubemap pass was not even used in ultra-low settings as the dynamic reflections had been disabled. Codemasters opted to use these shaders for the new tablet mode. As this pass was designed for a cubemap, it does not read any screen space maps. So the tablet version lost the undercar shadow, SSAO, dynamic shadows, car headlights at night, and a few other graphics features. While some would need replacement, many were not expected to be run on tablets anyway. Very early images of the game using the basic cubemap shaders are shown in Figure 3.
Figure 3:Early Images of the game using basic cubemap shaders
While a little too simplified in certain places, it gave a speedup of over 20ms per frame, nearly hitting the performance target. To fix the few shaders where screen space maps were still needed, such as the undercar shadow and the car headlights on the track at night, a new render pass was added. The new render pass was used as an override to the environment map pass. It allows the required information, such as the velocity/depth pass, to be created for just selected items like the car track, significantly reducing the number of draw calls and associated rendering.
In addition to the huge performance win, it became obvious the new shaders were a valuable debug tool. Being able to analyze the environment map shaders in detail, rather than only see them in the reflections of the car bodywork, showed many minor visual bugs and inefficiencies. Some of the shaders were reading screen space maps that were not valid and thus performing calculations that were expensive and unnecessary for reflections. The lighting was also broken, but the issue had been masked by the shaders. Fixing these issues resulted in a very impressive rendering solution for tablets. It also improved both quality and performance on higher-end systems that use these shaders for reflections.
An added bonus was a greatly reduced, total memory bandwidth requirement to render the scene. Some other smaller changes were also found. The billboard system originally did not use discard inside the pixel shader on transparent items, instead relying on blending to the scene with an alpha value of 0. Additional bandwidth was saved by disabling the color render targets when updating the depth buffer only, rather than blending with 0 alpha.
Selective use of high-frequency details like specular maps is another area where the visual quality of the tablet shaders improved. On bigger screen desktop systems, subtle per-pixel effects such as normal maps and highly specular materials can make a big difference to the visual quality. At much smaller screen sizes these high-frequency details have a much smaller impact. To save memory bandwidth, most of the shaders used for the tablet settings favor effects that impact large amounts of screen real estate over the high-frequency details. However there are a few exceptions, such as adding specular highlights to the wheels and a special ‘metallic specular’ for the car itself (Figure 4).
Figure 4: Example of specular highlights for the wheels and the special ‘metallic specular’ for the car
The key learning for Codemasters was to target the processing power in areas that would have the best visual bang for the buck. Along those lines, Codemasters reviewed the fog calculations. Originally they were done on a per-vertex level and included 1 pow, 1 exp, numerous lerps, 2 normalizes, and a sqrt function. Removing them and simplifying the math saved bandwidth and time that was applied in other areas of the game.
After the tablet optimizations were completed GRID Autosport performance exceeded the original target of 30FPS. This savings allowed Codemasters to add features that previously had not been rendered at lower-quality settings. For example, the crowd was added back, but using a simple billboard system rather than full 3D models. Also the in-car driver and the particle system were returned. The final results can be viewed in Figure 5.
Figure 5:On the top the driver model makes its return to the tablet version. On the bottom you can see the performance savings post optimization.
The Upscaling Question
Many games render some systems with large amounts of overdraw, such as particle effects, to a lower resolution surface before upscaling to fit with the rest of the scene. Autosport is no exception; a low-resolution surface is used for particle effects on both PCs and consoles. On mainstream PCs, PlayStation 3, and Xbox 360 the cost is negligible to create a downsized depth buffer for use of the particle system, then upscale the resulting particle effect back to full resolution and compose with the main scene. Tablets were expected to continue this performance win trend. However on tablets, Codemasters found the fixed cost of creating the downsized depth buffer and the upscale to be 3.5ms, effectively reducing the frame rate by 10% before drawing any particles. The game only generates large amounts of low-resolution particles when cars go off track. Codemasters decided it was better not to pay this high cost every frame. Instead, by reducing the particle counts they would get the best overall results.
Figure 6:Particle effects are rarely used on the track.
Another experiment involved a technique commonly used on console titles to improve performance. The 3D scene is rendered at a lower resolution, then is upscaled to full resolution before adding the HUD. Initial testing looked fairly promising, but the more the scene rendering was optimized, the less benefits were realized from upscaling. By the time all the optimizations were completed, it was faster to render the game at typical back-buffer resolutions rather than apply the fixed cost of an upscale.
Optimization Results
Figure 7 is a comparison of the Chicago start line, which is common to both GRID2 and GRID Autosport, taken using ultra-low settings designed for tablets. The city design changes slightly between the games so an exact comparison is impossible. But it is possible to see some of the trade-offs that have been made and the additions they have facilitated. Additions include particles and the crowd, which are clearly seen in Autosport.
Figure 7: The left frame is from GRID Autosport* at ultra-low settings. The right frame is from GRID 2. Both are the same Chicago start line.
GRID Autosport is rendered using 820 draw calls split across 18 render targets. GRID 2 required 1617 draw calls split across 49 render targets. GPU memory reads dropped from 122Mb per frame down to 53Mb. The vertex and pixel shader invocations dropped from 736K and 9.48M down to 336K and 4.17M respectively, even with the additions of the crowd and particle systems. The end result is an image that is approximately twice as fast to render with only subtle differences and is rendering more of the original content.
Adding Touch
Having the performance to run on a tablet is only one part of the story. GRID Autosport comes from a long line of PC and console titles. It was designed around using a gamepad, a keyboard/mouse, and driving peripherals. None of these options are appropriate for someone playing on a tablet. Fortunately, Codemasters has experience with tablet gaming, having created F1 Race Stars* on iOS*. So for GRID Autosport a touch overlay was added with analog zones for acceleration and steering (Figure 8).
Figure 8: The touch overlay in GRID Autosport* supports steering and acceleration.
Moreover, touch controls were added for the pause, camera, and replay modes. All the touch controls are displayed when your finger is in contact with the screen, giving important visual feedback to the user. To further aid the user when driving, there are many driving assists, which are enabled by default when in touch mode (Figure 9).
Figure 9:Available driver assist modes when using touch controls
The front end presented more of a challenge. It was designed as a large state machine, meaning that at any time the game knows which control is currently selected and which controls the user could move to next. The screens were also very densely packed with information, especially the car and race setup screen (Figure 10).
Figure 10: The densely packed menu system as designed would not work on tablets.
It was decided very early that it would be impossible to make the various elements on these screens all touch compatible. Even if technically possible, the size of the elements would be very close together on a small tablet device. Unaltered, these items would be far closer together than the minimum recommended size of 7mm with at least 2mm between each control element. Codemasters did not want to change the look and feel of the game or simplify the screens to the point the user lost customization options. An alternative design of adding a virtual game-pad onto the front-end screen was used instead (Figure 11).
Figure 11: A virtual gamepad was added to the front end so the player can easily navigate the dense menu system.
Now a front end designed to be used 10ft from a TV or on a PC monitor would still work on a 7in tablet. Additionally the tool tips for accessing a feature using a game-pad shortcut were converted to touch controls at the base of the screen (Figure 12).
Figure 12:Shortcuts were added to the front end for easier navigation.
When in touch mode, some additional tool tips were added to menus that previously did not have any in order to improve user access to the most common play modes.
Integrating the experience on a 2 in 1 device
The front end UI appears differently depending on whether you are running on keyboard, gamepad, or touch with unique tool tips for each control method (Figure 13).
Figure 13: The left shows the interface when playing on keyboard. The touch interface with the virtual gamepad is on the right.
The next question Codemasters faced was what should happen if the user is running on a 2 in 1 device or a detachable system like the ASUS T100*. If the user starts the game using the keyboard but at some point in the game the keyboard becomes inaccessible, how will the user still be able to interact with the game? The solution required multiple changes: first, the game integrated a Windows* API call that informs the game when a device changes from one mode to another. Windows broadcasts a WM_SETTINGCHANGE message when it detects a change slate mode with the string “ConvertibleSlateMode” in lParam. In case of a docking mode change, it broadcasts the same message with the string “SystemDockMode.” The API used to query the current status is GetSystemMetrics (Figure 14).
case WM_CHANGESETTING: if (wcscmp(TEXT(“ConvertibleStateMode”), (TCHAR*) lParam)==0) NotifySlateChangeMode(); break;
Figure 14: Code Copyright(c) 2014, Intel Corporation. Distributed under the Intel Sample Source Code License.
This method works when the system is running the new Windows 8 UI mode: BOOL bSlateMode = (GetSystemMetrics(SM_CONVERTIBLESLATEMODE) == 0).
This solution works very well, but has two possible drawbacks. Not all devices support the API, meaning the game might not get the WM_CHANGESETTING message, and also the call to GetSystemMetrics might return an arbitrary value regardless of the device’s actual state. Second, the user might be running on a clamshell Ultrabook and want to change from keyboard to touch without actually changing the mode.
The solution to both problems turned out to be surprisingly simple: the game remembers the current controller it is using. This is needed to correctly display the various tool tips and the touch overlay. In the Windows message loop the game simply checks the current type of device when it gets a WM_TOUCH or WM_KEYDOWN message and triggers the mode change if a different device is being used. GRID Autosport can toggle controls based on the user-preferred input device, and by calling GetSystemMetrics only on receipt of a WM_CHANGESETTING message guarantees the API has been correctly implemented.
Codemasters had one final issue when adding the touch overlays to GRID Autosport. The game has several cut scenes where the user cannot interact with the game. With the touch overlays still being displayed it gave the users an expectation that they could interact with them. To avoid confusion, and remove a degree of frustration, the touch overlay was removed during cut scenes.
A no-compromises gaming experience
Codemasters faced many challenges optimizing GRID Autosport for tablets. Traditional approaches to the problem did not work. More creative solutions were required which eventually prevailed. GRID Autosport looks nearly as good and plays as well on a 4-watt Intel Atom processor-based tablet as it does on a high end Iris Pro graphics-powered Ultrabook or PC. Innovative touch controls allow the same front end designed for TV and large display to work on a small 8-inch tablet screen.
The mobile gaming market is arguably bigger than the traditional PC and console market, and it is still growing. Why then should tablet and low-powered Ultrabook owners have to compromise quality for portability? Codemasters’ answer is they should not; players can have a top-quality game experience on the go. Especially with the versatility of the 4th gen Intel Core processor product line, including 4-watt Intel Atom processor-based tablets.
Reference and Related Links
- Global Online Gaming Market 2014
- GRID Autosport
- Taking the Pole Position: Codemasters Leads the Pack in PC-to-Tablet Optimization with GRID Autosport
- Codemasters GRID 2* on 4th Generation Intel® Core™ Processors - Game development case study
- Rendering in Codemasters’ GRID 2 and beyond:Achieving the ultimate graphics on both PC and tablet
- Developer’s Guide for Intel® Processor Graphics (For 4th Generation Intel® Core™ Processors)
Relevant Intel Links:
Corporate Brand Identity http://intelbrandcenter.tagworldwide.com/frames.cfm
Intel product names http://www.intel.com/products/processor_number/
About the Authors
Michael Coppock specializes in PC game performance and graphics and has been with Intel since 1994. He helps game companies get the most out of Intel GPUs and CPUs. Focusing on both hardware and software, he has worked on many Intel products, all the way back to the 486DX4 Overdrive processor.
Leigh Davies has worked at Intel for 10 years specializing in gaming and graphics and has had the pleasure of working with many of the leading European PC developers. Prior to joining Intel, he worked as a senior graphics programmer for a UK-based game company developing PC and consoles titles.