You learn more by doing than you do by reading, planning, or talking about doing.
Alex Hormozi
Hey guys 👋Deekshit here with this week’s newsletter edition, and in today’s edition, I will talk about metrics. As a growth product manager, most of my day goes into tracking metrics for the various experiments I am running. My thoughts on how to define and think about metrics have changed before and after joining my company. So, I believe my learnings would open up new perspectives for you guys.Â
In this newsletter, we are going to talk aboutÂ
Metrics when one is building a featureÂ
What should we measure?Â
Defining a new KPI
Metrics when one is building a featureÂ
We need to think about metrics right from the start during the design phase of the feature. One should clearly state the impact measure while drafting up the PRDs for a feature they want to build. It’s necessary to clearly state the metrics that the PM intends to measure for the following reasons.Â
All stakeholders are clear about what impact the feature/project is driving.Â
Instrumentation of those metrics can be implemented.Â
If this is not called out clearly in the design phase, the PM might not be able to showcase the impact their project is driving or have to wait for a few more sprints before the instrumentation of the metrics is implemented.Â
What should we measure?Â
Before, releasing a feature, I always create a dashboard to track the following necessary charts.Â
App version rollouts: This gives me an understanding of the adoption of the specific app version my feature is a part of.Â
Flag rollouts: If the feature is behind a feature flag, I track the number of users that have the flag turned on for them
Check for bugs: I list down all the scenarios the feature can fail or might be causing issues and create the necessary charts to ensure that the feature does not have bugsÂ
Primary metrics: Create the charts with the primary product metrics that track the impact of the productÂ
Secondary metrics: Other charts that are associated with the feature to ensure that the feature is working as expectedÂ
Guardrail metrics: Tracking metrics we expect not to change and ensuring that there are no changes to them post the release of the feature
Defining a new KPI
I was recently assigned to drive a team in defining the KPI for one of the products in my company. Initially, I thought it was an easy task, but then I realized the number of considerations to take into account when thinking about defining a long-term KPI that the company relies on to track the impact.Â
Here are some of the considerations I have learned about while thinking about defining a KPI
Does this metric cover the actions of all users?
Is this metric consistent over time?
Can we explain the highs and lows of this metric historically?
How sensitive is this metric to changes?
Is this a metric we can improve?
Can we track this metric over a long period of time?
It was interesting how there are millions of use cases that we need to think about while defining a metric, and it’s very easy for the team to get lost in analysis and not move forward at all. One lesson I learned through this project is that it’s always good to set a temporary KPI that we can improve upon rather than going into long-winded discussions without making any real progress.Â
So, these are some of my thoughts regarding metrics and KPIs. I hope you like it! If you guys have any questions, please reply to this email, and we would love to answer them for you.Â
That’s it from this post. If you liked this post, please consider subscribing and sharing this post with your friends and earn some brownie points!