New 10x10 Process Combines Participatory Design with Agile Product Development

By Garret Romaine

Download Article

New 10x10 Process Combines Participatory Design with Agile Product Development [PDF 588KB]

Traditional user experience (UX) testing is often a lengthy process, using dozens of test subjects and requiring considerable time both in preparations beforehand and analysis after the testing ends. This type of approach often works against Agile processes, as development teams have little time or incentive to provide guidance for the testing.

Enter Dr. Daria Loi, who oversees UX Innovation in the PC Client Group at Intel. Loi knew the traditional way of doing business was increasingly out of sync with the needs of a modern Agile team. So she came up with a new testing methodology she calls “10x10,” which limits the user population to 10 test subjects and restricts the testing cycle to 10 weeks. As Loi explained, “This process is different from the more traditional user experience process, where you develop a product to the prototype level, then heavily test with end users. Here, end users are enlisted as design partners from the very start—they participate in the design process. In fact, they have a weekly voice and consequently feel responsible for the end product. We empower them to be much more vocal with their insights.”

Hardware engineers and app developers can now get better and faster user input during the crucial early stages of a project. In the 10x10 testing methodology, teams work closely with end users over a short amount of time and rapidly gather useful information about the product as it evolves. Intel’s new process is too important to keep a secret—it’s worth sharing with all developers who want to build better products that end users can actually use.

Great Experiences are No Accident

Loi is committed to getting the user experience of Intel’s products right. Since the days of her first academic studies in architecture and industrial design, she’s been relentless in her quest for a better UX. Loi has always seen the UX role as crucial to the future of product and service development, and she is completely committed to continuously evolving her practice as necessary. “In my team I represent the ultimate voice of the end user for our key system designs,” she said. “As a UX practitioner, I have a great responsibility in this process and must evolve the way I do UX as needed. I must ensure we make the right decisions in the right time frame and context.”

In an ideal product development process, engineering, marketing and design teams work in a collaborative partnership based on continual interaction to streamline and accelerate the decision-making process. This partnership ideally relies on a solid understanding of the target audience, through ongoing input from UX and market researchers. The reality is different, however; on one hand a tight bond among all development teams is difficult to establish, and on the other hand, the time required to generate UX data is rarely aligned with the product-development schedule.

Loi has seen that the Agile development methodology is a great process to push the envelope with regard to fast iteration and solid teamwork, but end-user input has suffered. It’s hard to integrate users’ input into a team that is iterating on a short clock. Experts, like Loi, believe that Participatory Design (PD)—an established design practice where end users are directly involved in the co-design of the things and technologies they use—is a key to Agile success. With Loi’s new 10x10 process, Agile teams can experience a clear path forward that promises to improve on all of the key success factors.

Figure 1: UX testers often use notes to gather succinct feedback from test subjects and then group the comments to see if trends jump out.

A New Way Forward: 10x10

Loi and her team were recently instrumental in developing a number of reference designs for Ultrabook™ systems, including a convertible device (code-named Cove Point) based on the 3rd generation Intel® Core™ processor (Ivy Bridge), a reference design system (North Cape) with a detachable 1080p screen and 4th generation Intel® Core™ (Haswell) processor, and a detachable system with a 5th generation Intel® Core™ (Broadwell-Y) processor.

For all projects, Loi and her team sought faster integration of key findings, a tighter bond between the test subjects and the designers, engineers, and developers, and a better all-around product. As a result of this ongoing streamlining effort, Loi developed her 10x10 process to address increasingly aggressive project timeframes and crucial needs of her partner development teams.

In Loi’s 10x10 process, users were deeply involved in the development process, in a Participatory Design fashion. The actual testing took place weekly, with participants providing input on what the design and engineering teams worked out during the previous week. Loi and her team wrote up the results and reported the data as quickly as possible, in a cadence similar to the description given in Table 1.

Table 1.Daily tasks for reporting data during the 10x10 process.

A key component of the process is the selection of the best target users. Loi recruited participants with diverse demographics according to a range of parameters (age, gender, income level, employment, family composition, and device ownership, among others). Additionally, each subject represented one of the target consumer segments for that specific product.

“For these projects, I needed end users to be my design partners, so I selected subjects who expressed themselves well,” she said. “I needed assertiveness in their articulation. You want people who won’t hold back or try to please you with their feedback. They must be able to explain clearly and uncompromisingly what they mean, need, dislike, want, and desire.”

Loi met with participants for two hours every Thursday and development teams were invited to attend these weekly design critique sessions, enabling them to receive real-time feedback. Then on Friday, Loi provided summaries for all the people who didn’t attend the session, focusing on areas where the team needed to act immediately, essentially writing up the high-level action items. By Monday, her team distributed the official report to all internal stakeholders, including the executive teams.

“There was a strong cadence,” Loi said. “End users had a weekly voice, and this quickly created an interesting phenomenon: a strong sense of personal empowerment and responsibility to the end product. Because of this, they became increasingly vocal in their insights, because they felt—they actually were—part of the team. It was a quite wonderful thing to see at work and be part of.”

In the end, the final design was substantially different from the original concept, according to Loi. Some of the initial choices did not work well, according to the end users. “Each time we got strong, unanimous feedback, we addressed it right away,” Loi said of the development team.

For example, one of the chassis designs favored by the design team was the least favorite of the test subjects. After putting it to the test a couple of times and getting similarly strong negative feedback, the project leaders decided to drop that design and move in another direction.

Similarly, the original hinge design sparked a number of complaints, so designers came up with changes to test and ratify. There were also advances with the screen angle, touchpad size, and keyboard design. Testing would reveal challenges, designers would come up with alternatives, and the test team would report back so that project leaders could make informed decisions.

Striking Results

Loi knew she was on to something when the word began to spread that the test sessions were consistently revealing actionable data points. “One lead designer found so much value in the sessions that he wanted to make attendance compulsory for the entire team,” Loi said. “Some tech companies have created programs where engineers are required to directly engage in periodical user testing for similar reasons. Anything that tightens the communication between the end user and those that design technology will pay off in the end.”

Within the context she usually operates, Loi said that her 10x10 process is a big step forward. “It’s the first time we used this approach at Intel, as far as I know. If we look at the literature, Participatory Design approaches are not yet the standard tradition for Agile methodologies. I have been an active member of the Participatory Design community since the late ’90s, and I know that although the debate around PD and Agile is lively in some communities, there are not as many case studies as I would like to see—especially within large-scale product development contexts.”

Because product-development timelines are incredibly aggressive, a traditional UX-testing cycle often ends up influencing only some decisions—by the time prototypes are ready for testing, many key decisions have already been made and it is simply too late to make significant changes.

With Loi’s approach, her teams get rich feedback on a weekly basis, as the product is being designed. This makes the development team accountable for incorporating requested changes, and it builds a data-driven condition where assumptions and opinions no longer count and are no longer smart to entertain. “This means that personal preferences have less relevance and are more difficult to assert,” Loi said. “It’s a scary concept for some creators, who like to push their own views of what a product should be and do.”

All in the Family

Critics might ask about the embedded nature of the test subjects, who become important parts of the development team. When the test subjects become such an important part of the team, the tendency could be for them to “go native” and start going along with designers who face tough engineering challenges to implement good design ideas.

Loi said the result has been the opposite. “When you are part of the family, something important happens: you care, a lot. Since we elevated the end users from being sources of information to actual partners, they became empowered and responsible for the end result. Over the weeks, I noticed a growth in focus, detailing, strength, and depth. They had a strong memory of what they said in each previous session and made us accountable when they felt we did not deliver. The nuances and richness of the feedback increased over time—the last few sessions were phenomenal in that sense,” she said. “Also, it is important to note that the 10x10 process is typically complemented by other UX research, including quantitative and longitudinal studies, to triangulate data and deepen findings.”

Since Loi and her team had such fantastic data to communicate, they didn’t mind the intense time commitment required to get the data into the hands of the rest of the team. The process was intense—but with the right tools and techniques, her team quickly documented what the test subjects were saying, with high quality and dramatic impact—and Loi didn’t have to sacrifice her weekends to get a report done by Monday.

But in the Agile world, even that wait must seem interminable for some on the team. Loi knows that there will be ongoing demands to compress that reporting component even more. That’s where her daily interactions with the development team came in handy. “You cannot be an effective UX practitioner otherwise. You MUST be on the core design and development team.”

Faster, Better, and Cheaper?

Just about everyone who toils at some level in the technology world knows that there is one absolute truism in business: when it comes to faster, better, and cheaper, you get to pick two. Because the emphasis on speed is not going to change anytime soon, testing will have to innovate faster, think faster, and even fail faster.

Loi said there shouldn’t be any doubt that Intel is strongly committed to continually improving the user experience. From her perspective, that commitment expresses itself at multiple levels of the organization, from the number of UX practitioners present in the company, to the way in which designers, engineers, and developers talk about their products.

To Loi, it has been an incredible journey of evolution and progression from a UX perspective. Engineers who might have balked at slowing down for user testing now embrace the process and act as advocates. They now come to Loi and request testing. She has seen a major turnaround that she’s proud to be a part of. But self-congratulation isn’t part of her DNA.

“Once UX is embraced at the highest level of the company, there are huge implications from any perspective. Luckily, we have a lot of good practitioners. Also, most executive teams have now actively embraced user experience and routinely advocate for our work. This means a lot because it's not easy to have buy-in at such high levels. So it's pretty phenomenal, but there is still a lot of work to do. You can always do better.”

Related Content

The Human Touch: Building Ultrabook™ Applications in a Post-PC Age
Pointing the Way: Designing a Stylus-driven Device in a Mobile World
How Multi-Region User Experience Influences Touch on Ultrabook

Intel, the Intel logo, Intel Core, Look Inside, The Look Inside logo, and Ultrabook are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
Copyright © 2014. Intel Corporation. All rights reserved.