Designing a Multi-User Touch Interface for Windows*
Introduction
Introduced in the mid-1990s, All-In-One (AIO) computing devices have been around for many years. They were designed to cut down on the “workspace clutter” of desktop systems with all their discrete components (monitor, computer, input devices) taking up a sizeable workspace area. Early versions’ premium price and compromised performance (compared to desktop PCs) resulted in limited acceptance. However, the current generation of AIO devices does not share these shortcomings.
For the home user, models with up to 55-inch screens are commonly available. These larger screen models can be easily used as tabletop devices and offer developers the opportunity to provide novel and compelling user experiences, most notably, multi-user applications.
Unfortunately, at the time of this writing, the Windows API provides little support for touch-based applications for two or more users. The good news is, that it is not difficult for developers to implement their own methods for tracking multiple user input. They just have to ensure, when designing the user interface, that the application is easy to use and the interface reacts as the user expects it to.
In the examples given, we will look at the tools that the Windows API currently offers to support touch applications and show how this support can be implemented in a simple multi-user application.
User Interface Design
With current levels of support, touchscreens have no way of determining which user, in a multi-user (2 or more) application issued a touch command. In some cases, knowing which user issued a touch instance is not important. For example, a virtual board game could allow any touch to move any of the pieces and let the users be responsible for tracking who moves which piece.
But when it is important to know which user issued the touch command and each user requires separate controls (such as in our example in Figure 1), it is up to the developer to design the most effective way of determining which touch corresponds to which user. The normal way is to divide the touchscreen into separate areas, one for each user. It is left to the users to ensure that their touches are in their own respective areas. Note that other strategies may be devised, but they usually require some sort of elaborate heuristics to identify individual users, a topic not discussed in this article.
The Windows API
Windows currently offers two basic formats of touch inputs. The first is in the form of WM_TOUCH messages. The application must process and interpret these touch points because WM_TOUCH only provides the raw data input.
The second format for touch-based messages is in the form of WM_GESTURE, which is generated when the touch inputs match one of the Windows predefined gestures (such as pinch-zoom, touch-and-drag, etc.). This simplifies the job for the developer, as recognition of the gestures can be handled by the OS and passed on to the application in a format that is easier to interpret.
Currently, Windows only supports gestures generated by a single user/hand, so multi-user applications like our example cannot attribute a gesture to a specific user with WM_GESTURE messages.
It is important to note that coordinates provided by the APIs are relative to the entire screen (measured in 1/100th of a pixel), not to the application window. If window coordinates are required, you can use the ScreenToClient() function to do the conversion.
Our sample application
For this example, we created a simple touch-driven implementation of “Pong,” the classic arcade game. Two players are each assigned one paddle that is moved by touch input in the vertical axis only. A ball is constantly bouncing across the playing field, and when it contacts either end of the field (on the vertical axis), the opposite player is awarded a point.
The ball travels diagonally across the screen and is checked for collision after each movement. If any corner of the ball touches the top or bottom of the screen, the ball’s vertical motion is reversed. In a similar manner, the horizontal motion of the ball is reversed if it contacts a paddle or the side of the window behind it.
The ball’s motion is handled in a separate thread in the application to ensure smooth motion occurs while touch input is being processed.
Graphics rendering is handled using standard Windows GDI calls. The paddles and ball are simple rectangles and a circle, respectively, and the score is printed as text in the background. No bitmaps or other graphics assets were created for this example. The application interface is shown below, and you can see a video of the example application executing by clicking on the image.
Figure 1.Pong with a basic 2-player interface
The two-user touch interface
To interpret touch input from two users, we divide the playing area into two halves. Player 1 is on the left, and Player 2 is on the right.
Each time the WM_TOUCH event occurs, we retrieve the full list of touch points (Code Sample 1). This can range from a single input up to the maximum supported by the device. We then iterate through the list and retrieve the x and y coordinates of each touch point:
…
bret = GetTouchInputInfo(
(HTOUCHINPUT)lparam,
data.geometry_data.touch_inputs_count,
data.geometry_data.touch_inputs,
sizeof(TOUCHINPUT)
);
assert(bret != FALSE);
//-----------------------------------------------------------------
// process touch inputs
//-----------------------------------------------------------------
for(i = 0; i < data.geometry_data.touch_inputs_count; i++)
{
touch_input = &data.geometry_data.touch_inputs[i];
//-----------------------------------------------------------------
//Convert 100ths of pixels to pixels
//-----------------------------------------------------------------
x = touch_input->x / 100;
y = touch_input->y / 100;)
{…Code Sample 1.
Next, we check to see whether the touch was registered on the left or the right half of the playing field and update the appropriate paddle’s vertical position to match that of the touch point (Code Sample 2). This causes the paddles to follow the player’s finger.
…
if (x < (ULONG)((data.graphic_data.rect.right – data.graphic_data.rect.left) / 2))
{
//-----------------------------------------------------------------
// 2-player touch division
//-----------------------------------------------------------------
Data.geometry_data.p1y = y;
}else{
Data.geometry_data.p2y = y;
}
bret = move_paddles(&data);
assert(bret == TRUE);
{Code Sample 2.
Since all inputs on the screen are processed, not only will both players’ touches register simultaneously, but multiple inputs on each side are supported. However, only the most recent active touch point is used to determine the location of the paddle.
Modifying the application for four users
To make a four-player version of the game, we change the layout of the playing field and input areas.
Two additional paddles and score counters are added, one each on the top and bottom of the game window. The dividers for touch input are rearranged, forming an X shape that divides the playing field into four triangular sections (Figure 2).
Figure 2.Screen divisions for four players
Figure 3 is a flowchart illustrating how the touch input is parsed into four separate sections of the screen.
Figure 3.Parsing Four-Player Touch Input
With the playing field subdivided into four sections, determining which section of the screen a touch occurs in is more complicated (Code Sample 3).
…
if(touch_point.y > (LONG)(((float)data.graphic_data.rect.bottom / data.graphic_data.rect.right) * touch_point.x)) {
if(touch_point.y > (LONG)((((float)data.graphic_data.rect.bottom / data.graphic_data.rect.right) * -1) * touch_point.x) + data.graphic_data.rect.bottom){
data.geometry_data.p4x = touch_point.x;
}else{
data.geometry_data.p1y = touch_point.y;}
}else{
if(touch_point.y < (LONG)((((float)data.graphic_data.rect.bottom / data.graphic_data.rect.right) * -1) * touch_point.x) + data.graphic_data.rect.bottom)
{
data.geometry_data.p3x = touch_point.x;
}else{
data.geometry_data.p2y = touch_point.y;
}
}…Code Sample 3.
The new 4-player application interface is shown in Figure 4, and you can see a video of the 4-player example application running by clicking on the image.
Figure 4.4-player Pong game running on a 55-inch screen
Conclusion
For Windows 7 and later, developers can use WM_TOUCH and WM_GESTURES messages to provide multi-user touch support. By employing creative methods to determine where onscreen touch occurs, applications can easily support four simultaneous users.
About the Authors
Stevan Rogers has been with Intel for over 20 years. He specializes in systems configuration and lab management. He is a Technical Marketing Engineer for Line Of Business applications in the Platform Launch and Scale Engineering (PLSE) group.
Alden Sawtelle is currently enrolled in the Computer Information Sciences program at Portland Community College and a contingent worker for Intel. Alden specializes in the configuration and maintenance of enterprise server hardware, networking, and software in the PLSE lab.
Jamel Tayeb is a software engineer in Intel's Software and Services Group. Jamel earned a PhD in Computer Science from the Université de Valenciennes.
Intel®Developer Zone offers tools and how-to information for cross-platform app development, platform and technology information, code samples, and peer expertise to help developers innovate and succeed. Join our communities for the Internet of Things, Android*, Intel® RealSense™ Technology and Windows* to download tools, access dev kits, share ideas with like-minded developers, and participate in hackathons, contests, roadshows, and local events.
Intel, the Intel logo, and Ultrabook are trademarks of Intel Corporation in the U.S. and/or other countries.
Copyright © 2014 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.