Article

Making Metrics Matter: From Fresh Starts to Learning Loops

Published on June 25, 2024
Contributors

Very simple choices in the display of performance metrics can have profound impacts on user goals and motivation. How you choose to count, display, and communicate metrics can mean the difference between success and failure.

Often, the performance feedback provided by apps is about consistency, which can be very motivating - seeing past success can inspire continued adherence. However, a focus on consistency can promote perfectionism, and an all-or-nothing mindset which can end up undermining the goal.

Consider, for example, two common Behavioral Science tactics that are often at play when it comes to performance metrics: Streaks and Fresh Starts.

Streaks encourage maintaining consistent behavior, while Fresh Starts offer a reset point, like New Year’s Day, to begin anew. In the former, consistency is gained, and in the latter consistency is assumed. But both have the same underlying motivational profile - maintain that clean and perfect consistency.

However, because Fresh Starts and Streaks rely on the same psychology, they have similar ways of failing. The seeds of their own demise are built into the very premises. The pursuit of perfection can be demotivating when users inevitably fail. The difference between 99% and 100% success is significant: perfection is binary. Whether it’s losing a streak or a clean slate, any deviation from perfection can diminish motivation.

In statistics, values that can only take on 2 values are sometimes called “dummy variables.” The name is apt here. ‘Perfect’ is a dummy variable in more ways than one.

But there is a saving grace. All of these terms – fresh start, streaks, perfection, clean slate, consistency – they aren’t real. They are frames. Artifacts of the metrics we choose. Or in other words, their existence only depends on how performance is measured and communicated to the user. As a result, there are a lot of levers we can pull to keep motivation high, while preventing failures from undermining the pursuit of perfection that keeps people motivated.

This principle is important to keep in mind for all metrics that an app might track. Profound psychological influence occurs not because of objective feedback, but because of how the metric is communicated.

Below are three principles for helping users to overcome failures in order to keep motivation high.

Make success feel achievable

Sometimes, partway through a season, a Major League Baseball player will get traded to a different league whereupon their batting average is reset. It is a sort of ‘fresh start’ which presents a perfect natural experiment to better understand how a reset of metrics affects subsequent performance.

Hengchen Dai (one of the original authors of the Fresh Start Effect) found that the impact of a performance reset depended on a player’s current batting average. If a player’s batting average was significantly above average, the reset seemed to be demotivating, and the player’s subsequent performance decreased. On the other end, those with batting averages significantly below average seemed to be more motivated, and their performance increased after the reset.

After looking at various possible confounds, such as regression to the mean, and combined with various other results, Hengchen Dai comes to the conclusion that the key psychological principle impacting the change in performance is self-efficacy.

Consider how demotivating it would be to lose a 500 day streak, and how difficult it would seem to try and get back that streak. Conversely, imagine how motivating it would be for a professor to tell you that all your missed assignments are forgiven, significantly improving your grade. After such a reset, you may be very motivated to try and maintain your perfect “streak” of turned in assignments, even more so than you were at the beginning of the semester.

Performance metrics should make users feel like high performance is achievable, and not overwhelm users. This may involve clever resets when appropriate. Or, like Duolingo, it may involve getting the user to pick their own streak goal. However it is done, ensure users feel like achieving a good metric is within reach.

Make failure forgivable

The summer of 2016 was something quite different. That year I came home from college and went to a movie with my parents. Afterwards, I observed from the backseat as my dad put on Backstreet Boys and drove slowly around the parking lot so that my mom could spin PokéStops. I couldn’t believe it. What weird twisted version of the 90’s was I living in?

While not nearly as popular as it once was, Pokémon Go still has a loyal fanbase, perhaps because of the enduring power and nostalgia of Pokémon and the “catch ‘em all'' mentality (which in itself is worthy of its own article), but also because of some clever mechanisms within the game. One particular feature that has stood out is the way it incentivizes daily play, but without making a missed day feel like a punishment.

The app utilizes several streak counters to motivate daily play, but there is one central streak counter that is emphasized above the others. This counter rewards users with a unique Pokémon for every seven challenges completed within a month, regardless of whether these are consecutive days. This system motivates users to care about the streak, but without pressuring them to maintain perfection.

Within this display of performance metrics with its reward system, a missed day isn’t seen as a catastrophic failure. Indeed, resuming the game after a “lapse” may even give players a sense of Endowed Progress, where players feel like they have already made progress towards their goal that they want to take advantage of.

Other apps have used other mechanisms to manage failure. Noom and Duolingo both use Emergency Reserves where users are able to put their streak on pause, or to get forgiveness for having missed a streak. For more information about that, check out our own Aline Holzwarth’s interview with Jack Breslauer.

However you decide to manage failure, the key is to ensure it does not undermine success. Failure should be a learning opportunity, not a reset. Which brings us to our last principle.

Allow for a learning loop

A few years ago, I developed a habit called Exercise Snacking, where I performed short exercises hourly during workdays. This allowed me to incorporate significant amounts of exercise without disrupting my schedule. I tracked three metrics: hourly, daily, and weekly completion of the exercises. Importantly, I incorporated a flexibility mechanism, allowing myself to make up missed reps within the day or week. This flexibility allowed me to maintain my metrics without the pressure of immediate perfection.

This approach incorporated both of the earlier principles by being forgiving, and thus making my goal seem more achievable. But it also incorporated a third mechanism - it provided a learning loop.

I basically had three goals that varied in difficulty. The easy goal was to do all my weekly exercises within the week. The medium goal was to do all my daily exercises within the day. And the hard goal was to do all my hourly exercises within the hour. By splitting my metrics up like this, it took the pressure off me to be immediately perfect, and I could instead focus on improving the metrics over time. I paid attention to the disruptions that led me to miss my goal (such as meeting times), and adjusted my habit to make it easier to keep (such as by moving my reminder to 5 minutes before the hour so that I could exercise before a call started).

I eventually wanted to have perfect streaks of days, and perfect streaks of hours. But I wasn’t there yet, and that was ok. I was building up to them – testing what worked, finding my failure points, and designing to improve so that one day I could have perfection on all three measures. Failure wasn’t demotivating, it was telling me where I needed to improve.

I eventually burned out because (apparently) doing the same exercises every day for weeks on end isn’t very good for your body. Nevertheless, I considered it a success. I was very pleased with my ability to keep the habit during that time, and was even happier with how I looked in the mirror during this time.

The take-away is this; the idea that failure isn’t fatal but a learning experience isn’t just a mentality, but a product of how performance metrics are presented to users. Thus it is within the control of product teams to invite that approach to behavior change.

Conclusion

Designing a performance tracker involves many key decisions on how to represent and communicate metrics. Product teams must ask themselves questions such as whether the focus should be on counting days or percentage of days completed? Whether the representation should highlight successes or failures, and from what starting point? Whether to reset the metric, or start a streak over once it is completed? They must also ask themselves about the visual presentation—should it feature a bold single metric, multiple metrics, or perhaps a calendar view with color-coded indicators?

Each of these choices can have profound psychological impacts, and unfortunately there are no silver bullets.

Rather than purely focusing on specific tactics such as Fresh Start and Streaks, product teams should instead focus on whether their metric accomplishes the three principles listed above; making success feel achievable, making failure feel non-fatal, and creating a learning loop.

To be sure, these are not the only considerations when designing an interface. Other considerations, such as increasing the sense of reward upon success, and  ensuring streaks are connected to goals that actually matter, can be just as essential. Nevertheless, these three principles can go a long way to ensuring your performance metrics result in changed behavior.

References

Dai, H. (2018). A double-edged sword: How and why resetting performance metrics affects motivation and performance. Organizational Behavior and Human Decision Processes, 148, 12–29. https://doi.org/10.1016/j.obhdp.2018.06.002

‌Dai, H., Milkman, K. L., & Riis, J. (2014). The Fresh Start Effect: Temporal Landmarks Motivate Aspirational Behavior. Management Sciences; https://www.semanticscholar.org/paper/The-Fresh-Start-Effect%3A-Temporal-Landmarks-Motivate-Dai-Milkman/f2d7b7a5c53fa4ae79bc1ab03fe7be928de908db

‌Holzwarth, A. (2024, March 26). Emergency Reserves For Goal Slack: Interview With Noom’s Former Senior Director Of Product Management Jack Breslauer. Forbes. https://www.forbes.com/sites/alineholzwarth/2024/03/25/emergency-reserves-for-goal-slack-interview-with-nooms-jack-breslauer/?sh=54176a71e707

Share