An ASO KPI should answer a business question using a defined event, population and reporting window. Keep discovery signals, listing actions and product outcomes distinct, and give every core measure a decision owner.
Your deliverable: Build a metric dictionary that connects discovery, listing choice and downstream quality.
Defined audience and event-to-opportunity measures.
PRODUCT
Did the experience deliver?
Available activation or retention evidence.
Illustrative editorial framework, not client results, industry benchmarks or a prediction. These are separate questions, not a guaranteed linked funnel.
01
Start with the decision, not the dashboard.
A useful KPI tells a team something about an objective it can act on. If the problem is discovery, inspect relevant exposure and acquisition evidence. If suitable visitors arrive but do not act, investigate the listing and audience. If new users leave immediately, the question extends beyond the store page.
Write the question above the metric. “Are we attracting suitable first-time users in the priority market?” is more useful than collecting every number available. Agree who will review the answer and what decisions it could change.
02
Separate outcomes from diagnostic signals.
Downloads or a qualified downstream action may describe the outcome your team cares about. Keyword positions and listing engagement help diagnose parts of the journey. They are not interchangeable achievements, and a better diagnostic signal does not prove the business outcome improved.
The diagram connects questions to evidence without claiming a universal funnel. Use rank tracking as an observation of search visibility, not a substitute for acquisition measurement.
03
Write a definition for each KPI.
Record the source report, event, unit, audience, date range and filters. State whether the measure counts people, devices, events or an estimate. Two charts with similar names may describe different populations.
Keep the denominator beside every rate. The conversion definitions guide explains why a percentage cannot be interpreted without knowing what was divided by what.
04
Preserve context around movement.
Compare equivalent periods and inspect important audience segments. A campaign, product release or market-mix change can move a headline number even when listing effectiveness is unchanged. Save the change timeline before drawing conclusions.
Use a change log and distinguish observation from interpretation. “Downloads increased” is an observation; “the new subtitle caused it” requires additional evidence.
05
Turn the review into an action.
End with what the evidence supports, what remains unknown and the next decision. A stable number may justify holding a change while more information arrives. A suspicious jump may require a reporting investigation rather than a celebration.
The monthly reporting guide turns this into a repeatable structure. Keep the KPI set small enough that every included measure has a clear purpose.
WORKED EXAMPLE / FICTIONAL APP
A finance app needs more than a green ranking arrow
BudgetBench is a fictional budgeting app. Its weekly report highlights improved keyword positions, but the product team cares about people completing a first budget. The ranking change may be useful, yet the report currently skips the steps between visibility and a useful customer outcome.
How to reason through it
Choose measures that correspond to different questions. Discovery signals describe whether relevant people may encounter the app. Listing interaction measures describe a defined action in a defined population. Activation or retention measures describe later behaviour. Connect the questions without pretending that every platform report can be joined at an individual-user level.
The next practical move
Write a metric dictionary with event, denominator, store, audience, source and time window. Use it to review the current report and remove misleading labels. Give each KPI a decision owner: what action might change when this measure changes? Keep diagnostic measures available, but do not present every observable number as an executive success metric.
Swipe horizontally to compare the decision map →
A finance app needs more than a green ranking arrow: illustrative decision map
Decision area
What to inspect
What it means for the work
Discovery
Are we visible for relevant intent?
Contextual search observations and reports
Listing action
Are the intended visitors taking the defined action?
Exact event and denominator
Product value
Do acquired users reach a meaningful first outcome?
Appropriate product analytics with attribution limits
The trap to avoid
Do not let a metric become a target just because it is available. A visibility score may help monitor a research set while remaining a poor measure of revenue or user satisfaction. Explain what each metric can and cannot establish.
When the situation gets more complicated
Create a definition-change log alongside the performance log. If the reporting platform changes an event, preserve the break in the time series instead of drawing an uninterrupted trend. Where different measures cannot be reconciled, display them separately. Honest discontinuities are more informative than a smooth line whose meaning changes halfway across the chart.
Reporting definition check: Google Play’s current listing-performance documentation distinguishes click intent from completed acquisitions. Preserve the actual event name in your worksheet and comparisons.
FROM METHOD TO DELIVERABLE
A metric dictionary for BudgetBench
This fictional dictionary demonstrates how to specify measures without merging incompatible sources. The product event is an illustrative instrumentation requirement, not an assertion that a store report supplies user-level attribution.
Swipe horizontally to read the complete example →
Illustrative metric dictionary. Source definitions and product instrumentation must be verified for your app.
Question and measure
Definition to preserve
Decision owner and limitation
Are relevant searches finding us? / tracked positions
Fixed query set, store, market, device context and observation date
Research owner; a position does not count installs
Are listing visitors acting? / selected listing action rate
Exact source event divided by its eligible denominator, with filters
Growth analyst; do not merge rates with different definitions
Do users reach initial value? / first-budget completion
Defined product event and eligible new-user cohort in a fixed window
Product analyst; verify tracking and attribution limits
Was the work implemented? / release status
Approved asset version compared with the visible published version
Release owner; completion is delivery evidence, not growth
Make the dictionary part of the reporting contract
For each KPI, retain the source export, event definition, counting unit, filters and timezone. State whether it counts users, devices or actions. Add an owner who can explain the definition and an action the measure could inform. If nobody can name a decision it supports, move it out of the executive summary and into diagnostic material.
Treat missing data as a finding
If first-budget completion is not reliably instrumented, label it unavailable and assign a tracking investigation. Do not substitute total sessions simply because the chart is easy to produce. Check event duplication, eligibility and timing before interpreting a newly introduced measure. The first useful deliverable may be a trustworthy baseline rather than a performance conclusion.
Avoid inventing a joined funnel
A store-level report and an app analytics cohort may describe different populations. Present them as separate evidence when they cannot be reconciled. Do not divide one source’s users by another source’s devices and label the result activation. Explain what is known at each stage and what additional access or measurement would be required to connect them.
A dashboard combines Apple downloads and Google Play install clicks under “installs.” What should happen before trend analysis?
Reveal the reasoning
Separate the events and correct the labels. Confirm the definitions for each source and historical period. A chart with an inconsistent numerator cannot become reliable through a more attractive visual design.
Your working brief
Use your own app for this exercise. Fill in the six fields below and attach the evidence behind each answer. If you cannot complete a field, write the missing question and the person who can resolve it. Do not replace unknowns with optimistic assumptions.
Business question and decision owner
Metric label and exact event
Numerator and denominator
Store, source, segment and exclusions
Window, timezone and definition changes
Interpretation limits and next action
For each field, distinguish an observation from an interpretation. Add the date and source where available, then identify the decision that follows. A colleague should be able to trace your recommendation back to the evidence without needing you to explain the entire project again.
