Team Intelligence For Data-Driven Product Development usenotion.

com
 Table of Contents
Acquisition p.7 Referral p.17 Revenue p.26
Conversion From Lead Source 8 Net Promoter Score 18 Average Revenue per User 27
Conversion to Signup 9 Product Invite Conversion 19 Average Revenue per Employee 28
Conversion from Trial 10 Social Media Referrals 20 Customer Lifetime Value 29
Customer Acquisition Cost 11 Monthly Recurring Revenue 30

Activation p.12 Retention p.21 Team p.31

Average Sessions per User 13 Customer Churn 22 Backlog Burn Rate 32
Average Session Time 14 Complaints per Customer 23 Cost of Development per Unit 33
MAU/WAU/DAU 15 Feature Requests 24 Cycle Time 34
Feature Utilization 16 Revenue Churn 25 Data Removal Efficiency 35
Lead Time 36
Team Health 37
Throughput 38
Total Defects 39
Work in Progress 40
Velocity 41

Team Intelligence For Data-Driven Product Development usenotion.com

 The AARRRT! of Great Product
For product people, finishing features or
dealing with bugs can take up so much time
that its hard to see the bigger picture. But
to create really loveable, effective product
for your customers, you need to understand
how the work your team is doing fits in with
the needs of your customers and the broader
vision of your company.
Data is a great way to measure and test the
effectiveness of your work, but many of us feel
like we dont have time or even know where to
start when collecting and comparing data.
KPIs FTW!
Key Performance Indicators vary across
different kinds of companies, and its
Team Intelligence For Data-Driven Product Development usenotion.com
important to think about the best fit for your
users goals. There are a number of KPIs that
are commonly tracked in many of the most
successful technology companies. You may
have come across business KPIs like ARPU or
LTV, but sometimes its hard to see how those
metrics are relevant to the process of creating
and building the product.
This guide will take you through many of
the most popular KPIs for product develop-
ment, business, and customer success. Our
goal is to teach you how measuring and track-
ing these metrics will help you to make better,
more profitable products.
 The Recipe for Great KPIs
Performance indicators need to give you
actionable information about how to improve.
To create good indicators, first assess your
objectives and your strategy to meet goals that
further the objective.
Every KPI should have a direct correlation
to performance towards the goal. The KPI
may help the team detect problems or
impediments to the goal and also help drive
awareness of actions to further the goal.
So think about how a data point based on
measuring something will tell you whether
you are reaching a goal.
Team Intelligence For Data-Driven Product Development usenotion.com
Great KPIs are:
Actionable: Create a goal for the KPI. If the
indicators value changes, youll know what
actions to take next.
Measurable: Choose KPIs that are specific
and can easily be tracked and recorded.
Comparable: Knowing how your KPIs
inter-relate can help you discover how
moving one lever can affect other outcomes.
Valid: Indicators arent really useful unless
they are based on good data and correct
assumptions. Testing can help to ensure
that the basis of your KPIs connects with the
goals youre trying to achieve.
 User-focused: KPIs work best when you
understand that improving them directly
impacts the success of your customer.
Owned: Make sure there is someone who
is accountable for each KPI so it doesnt get
lost.
Shared: KPIs are best used to help the entire
organization understand how things are
going and what issues need to be addressed.
Lagging indicators track output. They are
easy to measure but hard to improve or in-
fluence, since they deal with past work. Often
these are your financial numbers, and they
give you a concrete picture of where you are
today.
Team Intelligence For Data-Driven Product Development usenotion.com
Leading indicators tend to be predictive and
can change quickly. They help you under-
stand the direction things are going. Leading
indicators are hard to measure and easy to
influence.
Ideally, every decision-maker and stakeholder
in your business will be able to see KPIs in a
dashboard that shows where you are meeting
goals, where you are off-target, and what
factors correlate.
When a team is able to meet or achieve a KPI,
then the business goals have been met for
that particular objective. The next step is to set
the goal higher for that same KPIor focus on
a new KPI with a new objective.
How often should you measure? typically you will be reviewing those numbers at
Most KPIs need to be reviewed at least least weekly.
monthlythough weve found that data-
focused PMs are looking at these numbers We built Notion to give you an easy, beauti-
every dayto get a picture of where they are ful way to look at KPIs with your team. In the
going and what might be moving the needle. following guide, youll see the recipes weve
created to make tracking your indicators
With campaign-focused metrics, consider straightforward and simple.
the goals youre trying to hit and track often
enough to see if you need to iterate. Some
metrics wont change frequentlysuch as
Revenue per Employeeand some are tied to
data you may only be collecting periodically
such as Net Promoter Score. Your Team metrics
will be tied to your development cycle, and

Team Intelligence For Data-Driven Product Development usenotion.com

 Acquisition
Acquisition metrics help you understand which Discovering where customers hang out and
sources or channels are the most useful for what kinds of messages motivate them can be
driving customer growth. Typically, these useful for product personas. And knowing how
numbers are marketing and sales-focused, but much it costs to acquire customers is important
they are valuable for product as well. to compare with how much value the product
delivers to users.
Marketing folks will tell you about TOFU (the Top
Of the FUnnel, not a tasty soy product). Basically,
you need to identify real indicators that suggest
your efforts to bring new customers to the
product are successful.

The Metrics
 Acquisition Metrics

Conversion from Lead Source

Lead Sources are purely marketing metrics, but you
may want to keep them in mind, simply because
understanding whats driving customers to fill out
a landing page form or reply to an email can be
informative to what users expect from the product.

Unfortunately, product teams tend to ignore

marketing metrics and often vice-versa. More
communication between groups can lead to better
customer onboarding, better retention and ultimately,
better product.
How to calculate

8
 Acquisition Metrics

Conversion to Signup Rate

Conversions can be helpful to keep an eye on
since they can give you a good picture of what users
are looking for. For example, if you have great
conversions at signup but a low conversion rate
from trial, it could be that you are promising some-
thing you cant deliver on to customers, that you
are initially selling to the wrong person, or product
usability is a problem.

Based on the action you want your user to

accomplish, you can measure conversions for
How to calculate
any behaviour, but signing up and paying for your
service are key metrics.

9
 Acquisition Metrics

Conversion From Trial Rate

If you have lots of customers signing up for a 14 or
30-day trial and your Conversion from Trial rate is
low, you may want to conduct surveys or user
interviews to find out where youre going wrong.

Growth Hacker Lincoln Murphy suggests tracking

the Common Conversion Activities your success-
ful customers perform, such as Feature Utilization,
Lead Source and reading your blog or helpcenter
materials. Compare these metrics in a dashboard
like Notions to assess what actions to take.
How to calculate

10
 Acquisition Metrics

Customer Acquisition Cost

(CAC)
This metric measures how much it costs, on
average, for your company to acquire a user.
Sometimes companies, especially in ecommerce,
also measure Cost Per Action (CPA) which applies
to the cost of signup rather than the total costs
associated with acquisition, which include sales
salaries, promotions and marketing campaigns.

In general, early-stage companies will spend as little

as possible on customer acquisition until they are
How to calculate
sure they have a working product with product-
market fit. Once you are prepared to spend money
to acquire customers, youll need to be very clear on
who to spend it on, so some form of user personas
are recommended. Youll also need to understand
the LTV (customer lifetime value) so you know if
spending more makes sense.

11
 Activation
So, your marketing and sales teams are amazing
and you have tons of users signing up. All done,
right? If youre working on product, this is where
the journey actually gets interesting. By under-
standing what your users are actually doing, you
can see if they are understanding the vision you
have to make them successful, or just getting
stuck. Data can give you insight into where your
product is most useful and where its superfluous
or even annoying. With this knowledge, you can
reduce Churn and increase LTV.
The Metrics
Every company will define active and utilization
differently. A user could be active if they log
in on a regular basis, or perform an action like
installing code or making a profile.
Some products need hours of use every day to
make it useful, whereas some will work just fine
if users visit weekly or a couple times a month.
Were curious what works best for you, so drop
us a line and well incorporate your thoughts into
a future lesson in the School of Little Data.
 Activation Metrics

Average Sessions Per User

A users session can be defined as the interval of
time in which you are receiving events from that
user if the time between events is less than 30
minutes.

Average Sessions per User is a simple metric that

will help you understand how your user in engaging
with your product. In many cases, its great when a
customer is using your product frequently. Based
on the job your product does, you can speculate
how many times your customer would optimally use
How to calculate
your product, and set goals.

13
 Activation Metrics

Average Session Time

Average Session Time can give you insight into the
engagement of your customers. After all, the more
time they are on the site, the better, right?

However, AST typically includes time that your

customer is in the product but may not be using
it all. You can analyze this metric better in conjunc-
tion with Average Sessions per User and specific
product utilization metrics.

How to calculate

14
 Activation Metrics

Feature Utilization
A number of measurements can help you understand
how users are exploiting your product, and which
features they dont careor possibly knowabout.

These will tend to be somewhat specific to your

product, but generally, you will want to collect data
on how many customers used a feature, whether
they used it repetitively (x2-), or whether they con-
verted within the feature (performed the action you
designed for, such as creating a report). Typically this
metric will be most relevant for the onboarding peri-
How to calculate
od, the first 3090 days.

15
 Activation Metrics

Daily/Weekly/Monthly Active
Users (DAU/WAU/MAU)
This metric seems basic until you have to define
Active. You might imagine that the user is Active
if they visited a dashboard or installed a piece of
code. But different types of users may be measured
by different definitions of Active. Ideally, you will
define it by an action or set of actions that suggest
the customer is meaningfully using the product.

DAU, WAU, and MAU are often considered vanity

metrics. You may find it useful, though, to compare
How to calculate
the MAU/DAU ratio to see how sticky your product
is. Its harder to retain customers who only use your
product once in a while.

16
 Referral
Making your product shareable makes it much in your product or analytics from your marketing
cheaper and easier to acquire new customers. site or from social media.
So youll likely want to understand the ways your
customers are referring you. If you have sharing or referral features in-app,
you will want to understand who is activating
Some of the most successful companies rely on referrals and to whom you are being referred.
referrals within the product. Dropbox famously
grew by offering a credit for referrals, and Uber
let users give free rides to friends. Make sure
your referrals can be tracked and added to your
tracking system. This may involve tracking events

The Metrics

0-6 Detractors: customers who are likely to convey
negative impressions to their network
7-8 Passives: customers who are mildly positive to
indifferent about the product
9-10 Promoters: customers with high satisfaction
who will actively evangelize about the product
How to calculate
Referral Metrics
Net Promoter Score (NPS)
NPS was developed to give a metrically significant,
simple way of understanding customer satisfaction.
In traditional NPS surveys, you ask your customers
How likely are you to recommend our product?
on a scale from 0-10, with 10 as the most likely.
Typically, this is the only question in the survey, with
the option of providing qualitative feedback by
asking an open-ended question like Why did you
pick that number?
A principle of both Lean methodology and NPS is
closing the loop, meaning that customers who
offer feedback should be contacted and heard.
This process gives you an opportunity to make a
detractor into a promoter, and also gives you
insights into the customers experience.
Asking for an assessment of satisfaction with
versus likely to recommend has been shown to
be as or more effective to NPS.
18
 Referral Metrics

Product Invite Conversion

Many software products include a way to invite new
users, since ideally, your existing customers are the
most effective source of generating qualified new
customers.

You want to make sure that the methods youre

using to generate these referrals are effective.
Tracking how well your inproduct referral engines
are functioning can help you understand what
methods work best, what customers are promoting
you to others and how to better encourage new
How to calculate
customers to use the product.

19
 Referral Metrics

Referrals via Social Media

Typically social media stays in the marketing
domain and is certainly a vanity metric, but
keeping track of engagement numbers like shares
via Twitter or Facebook relative to product release
cycles can help you understand your customers
as well as help in thinking about ways to generate
shares within the product.

You can also think about sentiment when tracking

social media. Its great when users are engaged,
but a little less awesome when they are all unhappy.
How to calculate
On the other hand, negative sentiment can give you
visceral feedback on your product pain points.

20
 Retention
Retention can be the most important product
metric for SaaS and is often overlooked. Build-
ing retention into your product should start with
your first touch with a user, ensuring that they
have a complete understanding of the ways your
product can help them and how to achieve those
goals.
To measure retention, you will typically look at
activation metrics (i.e. how you defined an Active
User) and calculate the average instances of
activation per user or per account.
The Metrics
Another key KPI related to Retention is Churn,
which can tell you if your rate of losing customers
is a cause for concern.
Retention matters for SaaS companies far more
than for many other businesses. For example,
if you sell diamond engagement rings, you likely
dont expect or need your customers to come
back often. On the other hand, a company that
sells subscription-based marketing software will
likely lose a customer if the user doesnt return to
the product on a regular basis.
 Retention Metrics

Customer Churn
A typical solution is to calculate churn as (number
of customers who churned) divided by the sum
of (customers at the beginning of the period) +
(customers at the end of the period) divided by 2.

Unfortunately, this wont give much of a picture of

the trend of churn: if there is a significant difference
between January and March, it will be hidden. One
way to attack this problem is to calculate churn with
a daily average, then multiply over the timeframe
you want to measure.
How to calculate

Churn is a key metric that touches everyone at the

company, so experiment with different formulas
until you find the one that works best for you and
make sure everyone understands your goals.

22
 Retention Metrics

Complaints per Customer

Also known as Active Issues, this metric lets you
know how many issues customers are having at
a given time. Knowing what issues customers are
having can help you make better product decisions,
while understanding Resolution Rate (how fast
issues are resolved) can be instructive about the
kinds of solutions needed.

For example, you may simply need to improve UI

so that customers better understand how to use
a feature. On the other hand, there may be more
How to calculate
complicated issues related to product-market fit
or persistent technical issues. Using user issues in
the development process is essential to creating
products your customers love.

23
 Retention Metrics

Feature Requests
Feature Requests might be considered more of a
qualitative metric than an actionable measure. That
said, tracking requests by category can give you a
sense of how well your product is loved or even
understood.

For example, if you notice that there are persistent

feature requests for a feature you already have, you
need to do a better job of uncovering the feature for
new and existing customers. If you notice growing
feature requests in a certain category, you may want
to investigate further to find out if there is market
potential for growth in that area.

24
 Retention Metrics

Revenue Churn
Compare Customer Churn and Revenue Churn to
help you understand your user trends. For example,
you could have a net increase in revenue with an
increase in customer churn if you gained a number
of people on a $99/month plan but lost many more
customers at your $5/month level. Low-value
customers are typically more likely to churn because
theyve made less of an investment overall in your
product.

New sales dont count. You can also calculate

How to calculate
Revenue Churn in regards to certain segments,
like RC based on downgrades or RC based on
cancellations.

25
 Revenue
You love your product and making and improving
it can be fun and challenging. But at the end of
the day, you need to make money so you can eat
and pay for things like cool quadcopter drones
(or, you know, whatever YOU need). Revenue
metrics help you see how well your product is
resonating with customers based on whether
they find it valuable enough to pay for it.
The Metrics
Revenue metrics also help you plan product de-
velopment, since you dont want to build a lot of
features if no one is there to pay for them. At an
earlier-stage company, you may not be able to
tell much from a snapshot of revenue, but
comparing these metrics from month to month
can be a good way to see if you are heading in
the right direction.
 Average Revenue Per User (ARPU)
Measure the average amount of revenue each customer pro-
duces in connection with data on upgrades and downgrades
as well as Churn to understand how you can increase ARPU.

ARPU is sometimes considered a vanity metric. If youre just

dividing the sum of your MRR by your number of customers
(the simplest way to calculate ARPU), you might not learn
enough thats actionable.

To improve the way you use ARPU, consider a cohort analy-

sis, or just compare signups from the last six months versus
older customers (or whatever sample makes sense for you).
How to calculate
If ARPU for new customers is higher than for ones who have
been around a while, then you might be on a better track
than if the reverse is true. ARPU can reflect whether you are
selling to the right people at a company or if there is prod-
uct-market fit. Your ARPU needs to be high enough that you
can afford to acquire the type of customers you serve.

You will use ARPU to calculate another KPI, LTV (customer

lifetime value).

27
 Revenue Metrics

Customer Lifetime Value (LTV)

LTV gives your marketing team a baseline on CAC
(Customer Acquisition Cost). You dont want to
spend more on getting a customer than they are
worth in their lifetime. Typically, the taget ratio of
LTV:CAC is 3:1 or better.

For more accurate results, you can compare LTV of

more recent customers to your earlier customers to
see if you are trending higher.

Another interesting aspect of LTV is churn. Statisti-

How to calculate
cally, lower-value plans tend to churn more, since
the customer tends to be less invested. So lets
all charge $10,000 a month! Unfortunately, the
corollary generally tends to be fewer customers
on pricier plans. Youll need to look at the total
revenue per plan level to get a more balanced
picture.

28
 Revenue Metrics

Monthly Recurring Revenue

(MRR)
Though the more punk rock among you may
associate this acronym with Maximum Rock & Roll,
Monthly Recurring Revenue is a bedrock of business
KPIs. You know that you need to know how much
money is coming in each month, but then what?

Comparing MRR with factors like team performance,

customer satisfaction, and feature utilization can
offer useful insights and areas to test.

How to calculate

29
 Revenue Metrics

Average Revenue Per Employee

This metric can help you in two ways. First of all, you
can compare it to industry standards to see how
you compare to your competitors. For example, the
average RPE for publicly traded SaaS companies is
about $200k.

Even more importantly, it can help you to under-

stand when you have the opportunity to hire, and
what effect adding or subtracting staff has on the
overall growth of your company. As a lagging
indicator, RPE is more helpful in understanding
How to calculate
how the company is performing than as a metric to
optimize for.

30
 Team
For Product Managers, team statistics can help to
drive quality, efficiency and accountability during
product development. In the past, it was hard to
visualize the success of the development team
in relation to other metrics. After struggling with
this issue in our previous companies, we created
Notion to help you track and compare your
teams success, their morale, and effectiveness
against business metrics like revenue or custom-
er behavior.
The Metrics
KPIs such as DRE, Total Defects/Bugs, lines of
code, and other metrics are typical Product
Quality measurements. These metrics can be
helpful to understand the technical quality of
your product, but its a good idea not to become
too obsessive about metrics like these relative to
others that focus more on the larger goals you
have in delivering success to your customers.
Development teams can also look at Churn and
Customer Complaints to get a better overall
picture of product quality.
 Team Metrics

Backlog Burn Rate

Your Backlog Burn Rate tells you how fast your team
is accomplishing the tasks and stories in the back-
log. This metric gives context to your Velocity and
Sprint Burndown charts so you can determine if
your team is prepared to tackle upcoming projects.
Use this metric to help plan new features and up-
coming sprints.

Read more about Backlog Burn Rate in the School of

Little Data.

How to calculate

32
 Team Metrics

Cost of Development Per Unit

This KPI measures how much it costs you to actually
make your product, primarily in terms of staff and
software.

Use this KPI to compare with MRR and ARPU to see

if you are pre- or post-breakeven. You dont have
to know the exact salaries of your teamyou can
simply take their estimated average annual salary x
1.15 (adding 15% overhead for benefits and payroll
taxes).

How to calculate

33
 Team Metrics

Cycle Time
Cycle Time measures the completion rate of your
product. It is always a subset of Lead Time, which
measures the time from a ticket being created to
the time the work is completed. Cycle TIme mea-
sures the actual time it took to build.

Cycle TIme is the inverse of Throughput. Its related

to Lead Time by the following equation: Lead Time =
Cycle Time * WIP, according to Littles Law.

How to calculate

34
 Team Metrics

Defect Removal Efficiency

(DRE)
DRE is a percentage of the bugs identified and
corrected internally compared to the total bugs
in the complete release life cycle. So, DRE is the
percentage of bugs caught by code reviews, unit
tests, and QA.

To use a DRE effectively, you may want to classify

bugs by common sources, like Design, Build,
Requirements, or Deployment to Production. You
can also create an overall DRE to assess general
How to calculate
quality in the product.

Read more about DREs in the School of Little Data.

35
 Team Metrics

Lead Time
Lead Time is the average time it takes between the
creation of a ticket and its completion. You can cal-
culate Lead Time as Cycle Time multiplied by WIP or
as WIP divided by Throughput.

Some consider Lead Time to be the overall measure

of product development process. Being quick and
responsive to new customer stories can give a
product an edge in a competitive market.

On the other hand, good product management

How to calculate
requires thoughtful feature development rather
than answering every customers demand, so Lead
Time may be deceptive. A good practice is to
compare this metric in relation to actual customer
satisfaction or NPS.

36
 Team Metrics

Team Health
How your team actually feels about the work they
are doing can be instructive for the health and
success of the product itself.

By tracking how your team feels about product

quality, morale, and efficiency, you can correlate
team health with product development. Of course,
its a great idea to have a sense of how your team
feels for other reasons, like keeping them happy
and productive.

How to calculate
To facilitate this metric, Notion includes the Team
Polling feature, a lightweight survey tool that keeps
track of how team members assess accuracy,
defects, morale and other aspects of their work.

Read more about Team Health in the School of Little

Data.

37
 Team Metrics

Throughput
Throughput is a simple metric that compliments
Velocity when used correctly. Velocity alone doesnt
reflect the quality of work along with how much
effort it took to do, but by adding Velocity and
Throughput, youll be able to understand the real
results you are getting.

Software development doesnt ever finish so you

need to have a metric that helps you understand
how much you are getting done over time.

How to calculate
Track throughput over long periods as well as
sprints to help you ensure you are consistent and
sustainable. Throughput should be tied to your
specific goals rather than inconsistent factors like
story points, which can vary over different teams.

38
 Team Metrics

Total Defects/Bugs
Bugs are a problem for camping or cookouts and
theyre even worse when they start piling up in your
product.

Having a picture of how many bugs you have is criti-

cal to product planning and will often correlate with
Customer Complaints. This metric is a component of
DRE.

How to calculate

39
 Team Metrics

Work In Progress (WIP)

Your WIP is a simple key performance indicator
used to track how much work your team is doing at
any given moment.

Typically, too much in your WIP will lead to decreased

quality and performance for your team. Limiting
context shifting for your developers leads to better
and faster work, and limiting your WIP to essential
components can improve your overall efficiency.

Learn more about WIP in the School of Little Data.

How to calculate

40
 Team Metrics

Velocity
Velocity is a leading indicator that measures the
effort needed to deliver value. Units can consist
of features, user stories, requirements or backlog
items. Use this metric along with Throughput and
Cycle Time to evaluate your development process.

If Velocity is constant, it can be helpful in planning

sprints. For example, if you plan a sprint over a two-
week period and one of those days were a holiday,
you might estimate a 10% reduction in Velocity
(assuming youre not working weekends!)
How to calculate

Dont focus on maximizing Velocity, since your DRE

can go down significantly if youre only worrying
about what you got done, versus the quality of your
work.

Strong product management and ownership, fre-

quent and regular release cycles, and effective UX
design prior to development will improve velocity.

41
 The Map to Real Treasure
KPIs help you understand your progress. A
good Product Manager also needs to consider
the customers experience through the pro-
cess of using a product. When you decide what
KPIs to track, consider how your customers are
actually interacting with you and what would
show you that they are successfully using your
product. Dont focus on vanity metrics that look at your KPIs in relation to each other.
dont really help you understand the needs of
the customer and if youre offering the value
that theyre looking for.
When you look at your teams success, keep
in mind that your main goal is to create value
for your users. You may need to slow down if
youre generating a lot of bugs or errors that
get in the way of a good experience. You may
need to ramp up if your customers generally
need more features. And you need to under-
stand clearly how everything you are building
will address real user stories.
No one metric can tell you everything, so
We built Notion to give you a clear, sharable
visual interface to compare your KPIs and
increase product insight for your whole team.
Let us know how we can help you think about
setting up or analysing your KPIs. Now that
youve done all this learning, wed like to offer
one more recipe: The Pirates Perfect Dark &
Stormy. Dont forget to stir!

Team Intelligence For Data-Driven Product Development usenotion.com

 AARRRT!

How to calculate

Team Intelligence For Data-Driven Product Development usenotion.com