Testing, Testing

How Does Fidelity Testing Work at EPAM Continuum? Let Us Count the Ways.

Fidelity testing. It’s not a segment on the Maury Povich Show about romantic side-hustles. It is, in fact, all business. Fidelity testing is the framework EPAM Continuum has constructed to guide our clients, and ourselves, through the prototyping process.

Fidelity testing is a diligently crafted, responsible approach to evaluating our works-in-progress. It has evolved over decades of work in all kinds of industries—some highly regulated, some not so much—and has proven valuable to our innovation process and our clients’ sometimes shakable peace of mind.

Some naïve organizations, in their enthusiasm to iterate immediately, often rush too quickly into prototyping, without ever thinking about testing. Not us. We rely on five types of testing in our work, with each type adding another layer of complexity. The process, of course, isn’t linear—but it is ongoing. There isn’t a single testing phase. Different projects call out for different testing modalities at different times. Our experience has shown us that most products call for one or more of the following types of tests.

Generative Testing

Generative testing involves real users in controlled settings. We use it to understand how well, or poorly, an idea delivers on an experience. This is about desire; that is, we rely on generative testing to reveal customer and/or employee desirability.

We’re talking about a first step in defining the ideal solution (defining the “ideal state” is central to our notion of #backcasting). Concepts are ideas that solve a customer need, defined by temporarily suspending technology, funding, and personnel constraints. Ideas include clear, simple value propositions for customers, and often take the form of statements, storyboards, or sketches.

Resonance Testing

In resonance testing, we have customers and employees evaluate mock-ups to ensure that the experience we’ve created meets their needs, is relevant to them, and delivers the appropriate value. This layer helps us not just test desirability but also the market relevance.

These mock-ups are low-fidelity samples that translate the conceptual design of an experience into a testable form. Mock-ups simulate the concept realistically but require significantly less time and money than more finished simulations. Think of paper models, user-interface front ends with no code, interactive visuals, or dramatizations.

(We must note that this sort of testing should not be confused with our podcast, The Resonance Test.)

Systems Testing

Systems testing involves in-context testing of high-fidelity prototypes to elicit feedback, from clients, customers, and/or employees, on the holistic experience in the context of an entire ecosystem. This layer of testing adds organizational feasibility into the mix.

Prototypes are high-fidelity manifestations of the ideas or experiences with an authentic look and feel but quick, inexpensive, and easily reconfigurable back ends. We’re creating prototypes that are experiential models but with full experiential features.

In-Market Testing

This is the first version of an experience with real users, employing pre-set feedback loops to understand the value of different touchpoints. In-market testing is significant because it brings business viability to the fore.

Now we’re up to pilots. Bringing the first iteration of a product or experience into the marketplace can be nerve-wracking, but because we typically do the other levels of testing beforehand, we can reduce the stress. It’s a critical step in that it showcases the value of an offering to users and allows us to evaluate its potential impact in the real-world.

Ongoing Feedback

Testing is never finished. Well, it never should be. It’s always a good idea to encourage the respectful collection of ongoing feedback gathered from customers, when the product or experience is in the market. During roll-out, the goal is to make sure that the experience continues to work with customers and employees over time.

There is much to be learned after launch, and we encourage our clients to understand that innovation shouldn’t end when the final deliverables are handed off. Testing, like so much of the general prototyping process itself—and like life itself—should be iterative. Make it so.

The Method of the Madness

So why do we test so systematically?

Two reasons.

First, this framework lets us tailor the fidelity of our work. We know that simple sketches and storyboards are sufficient at the beginning, but once we get to systems testing we must work with real hardware, software, and policies. Proper fidelity testing lets us calibrate the necessary level of detail at the appropriate time.

—and—

Second, our process helps clients, because when we follow it, we’re acting responsibly with their money. We systematically increase our level of confidence in an idea, without spending any more cash than necessary in order to move forward.

Trust us, once you try fidelity testing, you'll be faithful to it forever more.