Gather Lessons, Prioritize & Integrate: Turn Test Results Into Product Change
Learn a three-step habit: record every test, rank the lessons by impact and effort, then build the best ones into your design system, defaults and flows.
In short
Turning test results into product change takes three steps: record every test, rank the lessons by impact and effort, then build the best ones into your design.
13 min readFree
What you will learn
Running a test is only half of the work. A team can test often and still waste its effort if the results sit in a folder and nobody uses them. This lesson teaches a three-step habit. First you gather what you learned from each test. Then you prioritize, which means ranking those lessons. Last you integrate, which means building the best lessons into the product. By the end, you should be able to look at a pile of past tests and decide which ones should change your design.
Key words
- Experiment
- A planned change to a product that you compare against the current version to see what happens.
- A/B test
- A test where some users see version A and others see version B, so you can compare the two.
- Hypothesis
- A clear guess written before the test, such as: if we change this, people will do that, because of this reason.
- Metric
- The number you use to judge the test, such as how many people finish a step.
- Outcome
- What actually happened: a win, a loss, or an inconclusive result.
- Inconclusive
- The test did not show a clear difference either way.
- Impact
- How much a change is likely to help users or the business.
- Effort
- How much time, work and money a change takes to build.
- Backlog
- The list of ideas and tasks that are waiting to be done.
- Design system
- A shared set of parts, such as buttons, forms, colours and rules, that a product team reuses.
- Default
- The choice a product uses when the user does nothing to change it.
- Learning repository
- One shared, searchable place where every test and its lesson is written down.
- ICE score
- A score made from three parts: Impact, Confidence and Ease.
- RICE score
- A score made from four parts: Reach, Impact, Confidence and Effort.
Why do good lessons get lost?
Teams often stop at the winner. A test wins, the change goes live, and everyone moves on to the next idea. The reason behind the win is never written down. Figr, a testing guide, compares this to a scientist who finds something new but never publishes it. The gain is there for a while, but the team cannot build on it.
Losing and flat tests are just as useful. A test that lost tells you what users do not respond to. A test that was inconclusive may mean the change was too small, the test ran too briefly, or the idea simply does not matter much to users. Without a record, a team may run the same test again months later and get the same unclear answer.
Tip
Write up a failed or flat test with the same care as a winning one. You will need it more than you expect, usually when someone new asks why the team never tried a certain idea.
Step 1: Gather
Gather means putting every experiment, test result and user observation into one shared place. Include the failures and the inconclusive results. For each item, record the hypothesis, the metric and the outcome. A learning repository is not a dumping ground, so each entry should be short, clear and easy to search later.
What does each entry need?
The hypothesis
The guess you tested and the reason you believed it. A result means little if nobody remembers why the test existed.
The metric
The number used to judge the change. Write it down before the test starts, so nobody can pick a friendlier number after seeing the result.
The outcome
Win, loss or inconclusive, with a short note on what users actually did.
The observations
Notes, quotes or clips from users who took part in research or watched the test. These explain the numbers.
Step 2: Prioritize
Prioritize means ranking your lessons so the team works on the most useful ones first. Most teams have more lessons than time. Ranking with clear rules gives an honest order instead of one based on whoever speaks loudest. The Nielsen Norman Group describes several prioritization methods that base decisions on stated criteria rather than opinion. The goal is to act on the lessons that matter most for the product goal you chose.
What are impact and effort?
The simplest method is the impact-effort grid. You place each lesson on a grid with impact on one side and effort on the other. This creates four groups that help you decide what to do first.
Quick wins
High impact and low effort. These usually go first.
Big bets
High impact and high effort. Plan them carefully and try a prototype before you commit.
Fill-ins
Low impact and low effort. Do them when there is spare time.
Money pits
Low impact and high effort. Drop them or leave them for later.
Teams often place lessons together. People can vote with coloured dots, with one colour for impact and one for effort. A designer may judge the effect on users, while an engineer judges the work needed. Keep the finished grid after the meeting, so everyone can see why each lesson landed where it did.
How do you score with ICE or RICE?
A grid is quick, but a score gives you a number to sort by. Two common scores are ICE and RICE. ICE uses Impact, Confidence and Ease, where Ease is the opposite of effort. RICE multiplies Reach, Impact and Confidence, then divides the result by Effort, where Reach means how many users the change touches. Teams with a small list often start with ICE. Teams with better user numbers often move to RICE. Some teams use the grid for a quick talk and a score for the final order.
In words: multiply Reach, Impact and Confidence, then divide the result by Effort. A lesson that reaches many users, helps them a lot, is backed by good evidence and is cheap to build gets a high score. Confidence is how sure you are. A lesson backed by several tests and user interviews deserves more confidence than a hunch. Without research, confidence is only an opinion, so the score is only as good as the evidence behind it.
- 1
List the lessons
Put every lesson from the repository on one list, including the losses and the flat results.
- 2
Pick one method
Use the impact-effort grid for a quick talk, or ICE or RICE when you need a sorted list.
- 3
Score with the same rules
Use the same scale for every lesson, so the scores can be compared.
- 4
Sort the list
Put the highest scores at the top and read the top ten out loud.
- 5
Check the goal
Ask whether the top lessons move the one goal your team chose. If not, change the order on purpose and say why.
- 6
Write the decision
Record the final order and the reason for each top lesson.
Tip
Score the lessons together in one room, and write the reason next to each score. A score with no reason is hard to defend a month later, when someone asks why a lesson was skipped.
Step 3: Integrate
Integrate means turning the top lessons into design decisions. A lesson that stays in a report changes nothing. A lesson that becomes a shared part, a default or a changed flow keeps working after the people who ran the test have moved on. This is how improvements build on each other instead of staying as separate experiments.
How do you update the design system?
A design system holds the parts a team reuses. When a lesson wins, decide where it belongs. Often it belongs in a pattern, which is a shared solution to a common problem that links to the parts it uses. One design-system guide notes that the teams who solve real problems are often the best people to add new patterns back into the shared system. GOV.UK, the UK government's design system, labels parts that still need research as experimental, so designers know which ones are not yet proven.
How do you change defaults and flows?
A default is the choice a product makes for the user. If a winning test showed a better starting choice, the default should change too. Otherwise, people who never touch settings keep seeing the old version. A flow is the set of steps a user takes to finish a task. If a test showed that a shorter flow works better, rebuild the flow with the shorter version as the standard. This lesson only shows where these changes go. How to choose the right default is taught in other lessons.
How do you re-check the results?
After you ship a change, watch the same metric you chose before the test. Check that the gain is still there a few weeks later, and that nothing else got worse. Then update the repository entry with what happened after launch. Booking.com is described in one experimentation guide as a team where each experiment's learning feeds back into the framework for the next one. That loop is what makes the learning stick.
- 1
Pick the top lessons
Start from the ranked list, not from the newest test.
- 2
Write one decision per lesson
Describe the change in plain words, such as: the sign-up form now asks only for the essential fields.
- 3
Update the shared parts
Change the pattern, component, default or flow that the lesson affects.
- 4
Tell the team
Let designers, writers and builders know what changed and why.
- 5
Set a re-check date
Choose a date to look at the metric again, and write it in the calendar.
- 6
Record the result
Add the after-launch outcome to the repository entry.
Worked example: a language app's onboarding
Imagine a language-learning app team with three past tests. The first tested a reminder that helps people keep a daily streak. The second tested a progress bar on the lesson screen. The third tested a shorter sign-up form. The team wants to decide what goes into the standard onboarding flow, which is the set of screens a new learner sees first.
- 1
Gather
The team opens its repository and finds all three tests. Each entry has a hypothesis, the metric the team chose before the test, and an outcome. The progress bar entry also has a short note from user sessions that people looked at it but did not change how they studied.
- 2
Score with ICE
The team rates the shorter sign-up as high impact, high confidence and easy to build, because the work is mostly removing fields. The reminder gets high impact and high confidence, since it showed a clear gain, and a medium ease score because it needs notification setup. The progress bar gets lower impact, decent confidence and a harder build, because it needed new data on the lesson screen.
- 3
Read the grid
On the impact-effort grid, the shorter sign-up and the reminder both land in the quick-win group. The progress bar lands in the lower group, so it is not a priority.
- 4
Integrate the winners
The team places the shorter sign-up and the reminder into the standard onboarding flow. It also updates the shared form pattern in the design system, so every new form starts with the shorter version.
- 5
Shelve the loser for now
The progress bar stays in the repository with a note: flat result, shelved, and worth a new test if the lesson screen changes. Nothing is thrown away.
- 6
Re-check
The team sets a date to look at the same metric after launch. It writes the after-launch result into each of the three entries, so the next team sees the full story.
Common mistakes to avoid
Saving only the winners
Losses and flat results show what users ignore. Without them, the team repeats old tests.
Ranking by volume of voices
The loudest person in the room is not a measure. Use the same criteria for every lesson.
Shipping and forgetting
A change with no owner and no re-check can quietly drift back to the old version.
Changing the metric afterwards
If you pick a new number after seeing the result, the lesson is not trustworthy.
Treating one test as a law
A result applies to the users, screen and context it was tested in. Check before you copy it to a new place.
Check your understanding
Quick check
Your team ran six A/B tests this quarter. Two won, one lost, and three were inconclusive. A manager wants to ship the two winners and forget the other four. What should you do instead, and why?
Quick check
A teammate proposes a change that would reach most of your users and looks like a big win. The only evidence is one conversation with a single user, and the build will take several months. Which score is the most honest?
Key takeaways
Gather
Record every test, including failures and flat results, in one shared, searchable place with the hypothesis, the metric and the outcome.
Prioritize
Rank the lessons with a simple method such as an impact-effort grid, ICE or RICE, and check the order against your product goal.
Integrate
Turn the top lessons into changes in the design system, defaults and flows, then re-check the metric and record what happened.
Good teams treat each test as a step in a chain. When the learning is gathered, ranked and built in, the next test starts from a stronger place than the last one.
Sources
- Nielsen Norman Group: 5 Prioritization Methods (impact-effort matrix, RICE and others)
- Task Prioritization in UX Design (RICE Framework), Medium
- Prioritize UX Research Recommendations: Combining Value and Pain-Driven Approaches, Medium
- Prioritizing Usability Findings for Impactful Iteration, UX Dictionary
- UX Research Intake & Prioritization: Frameworks That Keep Teams Focused
- RICE Framework Explained: How to Prioritize Features With a Scoring Model, Basic Free Tools
- RICE vs ICE: How to Use Prioritization Frameworks With User Research, Evelance
- 10 Product Prioritization Frameworks (2026): Which to Use, ProductLift
- A/B Testing Best Practices for Product Teams, Figr
- How to Build an A/B Testing Insights Repository, Convert
- Using Mental Models to Step Up Your Experimentation Game, Convert
- UX Research Repository, Lyssna
- How to research components and design patterns, Amy Hupe
- Understanding the Parts of a Design System, Workday Design, Medium
- Design system architecture: theming, experiments, and keeping it maintainable, Medium
- The Experiment Pipeline That Learns From Itself, GTM Engineer newsletter
Last updated on 17 November 2026
Related lessons
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.