THE VERDICT · SUPPORTED IN THIS SAMPLE
Yes. Positive reviews can contain questions too.
In two historical review datasets, roughly one third of the reviews labelled as needing an explanation had four or five stars. Looking only at one and two star reviews would have missed many of those labelled needs.
These are selected samples, not a survey of every app user. A question does not automatically mean that the app listing needs a change.
See what I found ↓
MY QUESTION
What am I missing if I only read bad reviews?
A customer can like an app and still wonder how a feature works, what a paid version includes or where to find a setting. If I only read low ratings, I might miss those questions.
I used two historical datasets where source researchers labelled reviews for explanation needs.
Among the reviews with those labels, I counted how many had four or five stars.
I checked how many labelled needs a search limited to one and two star reviews would leave out.
A labelled need may be a question, request or problem. It does not prove the issue is unresolved or belongs in store copy.
MY THEORY VS THE RESULT
Many labelled questions came from highly rated reviews.
My theory
Useful requests for explanations would also appear in reviews giving an app four or five stars.
What I found
Both samples passed the thresholds chosen for this study. The smaller sample contains only 66 labelled needs, so I would not treat its percentage as a universal benchmark.
labelled explanation needs → check star rating
The denominator is labelled explanation needs, not all reviews or all users.
See the original result chart
Each percentage starts with reviews that the source researchers labelled as needing an explanation.
These are not percentages of all users or all positive reviews. The second sample belongs to the same research programme and is not a fully independent replication.
If I had read only one and two star reviews, I would have missed 50.6% of the labelled needs in the main sample and 59.1% in the second sample. Those missed figures include three star reviews as well as four and five star reviews.
REAL REVIEWS IN THE DATA
What kind of questions did people ask?
These are paraphrases of historical review records from the supplied dataset. The records identify a store but do not establish a reliable country for these examples. They are not proof of the apps’ current features or claims of client relationships.
GOOGLE PLAY · FOUR STARS
White Noise
A reviewer wanted to know how this edition differed from the developer’s Pro edition. That could prompt me to check whether the actual differences are clear.
APP STORE · FIVE STARS
PictureThis
A reviewer praised plant care features and asked whether friends could connect and share pictures. I would verify what the app supports before suggesting any listing copy.
GOOGLE PLAY · FOUR STARS
DuckDuckGo Private Browser
A reviewer asked what reported tracking attempts meant. That might be a prompt to explain a feature more clearly, potentially in the app or help content.
GOOGLE PLAY · FOUR STARS
Blitzer.de PRO
A reviewer asked how to turn off voice announcements. That may be better addressed in settings, onboarding or support than in the store listing.
MY ASO TAKEAWAY
Read beyond the stars, then route the question.
My ASO tip: Include reviews across rating levels when researching what customers expect and misunderstand.
My ASO strategy: Sort questions into product promise, onboarding, support and product issues. Check the actual app before deciding whether a listing or screenshot should explain something.
My ASO warning: A feature request is not proof the feature exists. Do not promise it in copy until it is supported, and do not claim a listing change improved installs from this study alone.
WHAT I COULD NOT PROVE
Here is where the evidence stops.
- Every app user: These were selected historical review samples, not a representative survey of all stores.
- Unanswered questions: A review label does not establish that the issue remains unresolved.
- Best place to fix an issue: Some records concern bugs, settings or support rather than ASO copy.
- Improved performance: I did not measure whether answering questions changed conversion or retention.
- Independent replication: Both datasets come from the same research programme, and two app identities overlap.
- Store comparisons: Some source store labels conflict with the paper’s written app list, so I would not use these samples to rank the stores.
SOURCES & METHODS
Want to check my work?
I used the public replication package for From App Features to Explanation Needs by Obaidi and colleagues. The source researchers supplied the human labels; I counted the effect of filtering reviews by stars.
I reran the supplied independent recount against the original workbook files. It checks row selection and arithmetic, not the correctness of every annotation. The analysis used the final explanation need label after a source field correction described in the original report.
The source evidence and full technical report are kept locally while redistribution rights and review text are assessed.
Read the original report and technical details
About one-third of labelled explanation needs had four or five stars.
MAIN SAMPLE
31.4%685 of 2,183 reviews labelled as needing an explanation had four or five stars.
VALIDATION SAMPLE
37.9%25 of 66 reviews labelled as needing an explanation had four or five stars.
What the percentages mean: start with all the reviews researchers labelled as needing an explanation. Then count how many gave the app four or five stars. These are not percentages of all app users, all high-star reviews, or all ASO problems.
The labels cover a broad mix of confusion, questions, requests and problems. They do not establish that every question remains unanswered or that a store listing should answer it.
The theory, in everyday English
A person can like an app enough to give it a high rating and still be unsure about a feature, a paid edition or a setting. If an ASO team uses one-star and two-star reviews as its only source of customer questions, some of that useful feedback will be invisible.
Our exact test: at least 15% of labelled explanation needs must have four or five stars in each dataset. We also required the lower end of the app-resampling range to stay above 5% in each dataset. We chose these practical thresholds before calculating the results; they are not industry standards.
The observed shares were 31.4% and 37.9%. Their app-resampling ranges were 25.9% to 37.5% and 28.3% to 50.0%. Both passed.
The ranges come from repeatedly resampling whole apps, 20,000 times per dataset. They show how much the finding changes with a different mix of the observed apps. They do not turn these selected samples into a representative survey of the app stores.
What this looks like in actual app reviews
These are paraphrases of real records, selected from the inspected sample. They are examples, not new claims about what these apps currently support.
| App and rating | What the reviewer wanted to know | Possible use for ASO research |
|---|---|---|
| White Noise · 4 stars | How does this app differ from the developer’s other Pro edition? Is there anything extra worth getting? | A question about choosing between editions can reveal a need for clearer product comparisons. Verify the actual differences first. |
| PictureThis · 5 stars | The reviewer praises the plant-care features, then asks whether friends can connect and share plant pictures. | Shows uncertainty about a feature. ASO messaging may need to clarify what is included, but the question alone does not prove the listing is unclear. |
| DuckDuckGo Private Browser · 4 stars | What do all these reported tracking attempts mean, and how significant are they? | Could inform how benefits and metrics are explained. Detailed interpretation may belong in the app or help material. |
| Blitzer.de PRO · 4 stars | The reviewer likes the update but asks how to turn off voice announcements. | Useful feedback, but this is more directly a settings, onboarding or support question. |
Traceable records: main workbook Excel rows 1012 and 384; validation workbook rows 370 and 439. Row numbers include the header. Source app key for White Noise: comtmsoftwhitenoisefull.
Not every high-star “need” is an ASO copy opportunity
A five-star Monefy Pro review asks why decimal amounts are not saved. That may require a bug fix or support. A five-star Things 3 review includes a later update saying the reviewer found list sharing. Some Apple Store reviews discuss iOS problems rather than the Apple Store app itself.
We inspected 20 high-star records from each dataset. These checks helped interpret the labels; they were not an independent human relabelling exercise. The numeric results retain the researchers’ original final annotations.
How much would a low-star-only review search miss?
In the main sample, reading only one-star and two-star reviews finds 1,079 of the 2,183 labelled needs. It misses 1,104, or 50.6%. In validation, it finds 27 of 66 and misses 39, or 59.1%.
Those missed reviews include three-star reviews as well as four-star and five-star reviews. The table separates the choices.
| Reviews you read | Main: labelled needs found | Validation: labelled needs found |
|---|---|---|
| 1 star only | 736 / 2,183 (33.7%) | 19 / 66 (28.8%) |
| 1 to 2 stars | 1,079 / 2,183 (49.4%) | 27 / 66 (40.9%) |
| 1 to 3 stars | 1,498 / 2,183 (68.6%) | 41 / 66 (62.1%) |
| 1 to 4 stars | 1,825 / 2,183 (83.6%) | 55 / 66 (83.3%) |
| 1 to 5 stars | 2,183 / 2,183 (100.0%) | 66 / 66 (100.0%) |
Even reading everything except five-star reviews leaves something out: five-star reviews contain 358 labelled needs in the main sample and 11 in validation. That is about one-sixth of all labelled needs in each sample.
What about the extra reading time?
Reading more reviews takes more time. Moving from a one-star and two-star filter to all reviews adds 2,776 reviews in the main sample and 299 in validation. It finds an additional 1,104 and 39 labelled needs, respectively. The study does not measure reading time or the business value of each need, so it cannot establish the most efficient review-selection strategy.
This is a simulation of a star-filtering rule. We did not observe what real ASO teams read, and we did not measure whether the extra reviews improved their decisions.
Why this matters for apps and ASO
ASO research often needs to understand what people expect from an app, how they describe it and what makes its offer hard to understand. A high rating does not rule out useful feedback on those questions.
The practical implication: include reviews from every star level when researching customer questions. Then decide where each issue belongs.
| What the review reveals | Where the insight may belong |
|---|---|
| Uncertainty about paid versus free editions, supported features or product differences | Check whether the app listing and screenshots explain the offer accurately. |
| Difficulty finding a setting or completing a task | Onboarding, in-app guidance or help documentation may be more suitable. |
| Broken behavior, failed payments or an account-specific problem | Product fixes or customer support. |
| A request for a feature the app does not have | Product research. Do not promise an unavailable feature in ASO copy. |
This is evidence for broadening the reviews used in ASO research. It does not prove that changing a title, screenshot or description will increase installs.
Does the finding survive closer checks?
Yes, the high-star share remains visible across these different ways of analysing the data. The percentages below always use labelled explanation needs as their denominator.
| Check | Main high-star share | Validation high-star share |
|---|---|---|
| Keep all original eligible rows | 685 / 2185 (31.4%) | 25 / 66 (37.9%) |
| Count only explicitly labelled needs | 434 / 1455 (29.8%) | 21 / 44 (47.7%) |
| Count only implicitly labelled needs | 251 / 728 (34.5%) | 4 / 22 (18.2%) |
| Keep only reviews whose source raters agreed | 619 / 1984 (31.2%) | 21 / 58 (36.2%) |
| Apple App Store records only | 551 / 1601 (34.4%) | 10 / 33 (30.3%) |
| Google Play records only | 134 / 582 (23.0%) | 15 / 33 (45.5%) |
| Give every app with a labelled need equal weight | 32.8% average within-app share | 37.5% average within-app share |
| Remove the largest contributors | 474 / 1,659 (28.6%); largest 5 apps removed | 21 / 53 (39.6%); Telegram removed |
| Remove each app, one at a time | Share ranges from 29.3% to 32.2% | Share ranges from 34.9% to 39.7% |
The largest main contributors removed were Spotify, Pinterest, Minecraft, Gmail and Google. High-star needs occurred in 103 of 120 main app groups containing any labelled need, and 8 of 10 validation groups. The result is not confined to one app.
Two validation apps also occur in the main sample
Exact app IDs identify Royal Match and AnkiMobile Flashcards in both samples. After removing both from validation, 22 of 53 remaining needs have four or five stars: 41.5%. A one-star and two-star screen would miss 33 of 53, or 62.3%.
This was an additional check added after seeing the main results. No normalized review text overlaps between the two files. Some main metadata is incomplete, so this check does not certify that every other product identity is distinct. Validation is a second sample from the same research programme, not a fully independent replication.
Store-level checks use workbook store labels. The paper’s written app/store list appears inconsistent with parts of those workbooks, so store comparisons are secondary and should not be used to rank the stores.
Where the evidence comes from
| Main sample | Validation sample | |
|---|---|---|
| Original review rows | 4,495 | 495 |
| Rows used after cleaning | 4,484 | 495 |
| Store/app-name groups | 123 | 10 |
| Reviews labelled as needing an explanation | 2,183 | 66 |
| Four-star and five-star labelled needs | 685 | 25 |
| Source sampling | Selected and filtered historical benchmark; original collection described as October 2023 | Selected apps; source paper describes up to 50 recent reviews per app as of August 2024 |
Both files come from the public replication package for From App Features to Explanation Needs: Analyzing Correlations and Predictive Potential, by Obaidi and colleagues. The source researchers supplied the human annotations. We counted how a star-rating filter would change which labelled reviews were seen.
A correction to our earlier data audit
The main workbook has two similarly relevant label columns. Our expansion audit initially counted the upstream Type column. The final human labels are in Explanation Need Type.
Before cleaning, the correct final counts are 2,310 reviews with no explanation need, 1,457 with an explicit need and 728 with an implicit need. That is 2,185 labelled needs, not 2,994. The two fields disagree in 1,294 rows. We resolved the field meaning before calculating this experiment’s star-rating results, using the source documentation and rater columns. The earlier expansion report and data package have been corrected.
How we handled duplicates and checked the arithmetic
We kept reviews with valid stars, a valid final label, and app, store and review text. All source reviews passed those basic requirements. For the main analysis we excluded five rows where the same app and review text had conflicting labels or ratings, and collapsed six repeated rows. Validation needed no exclusions.
Text normalization only adjusts Unicode form, case and whitespace for matching. Original text is retained, including encoding damage. We did not treat the source review ID as globally unique.
A second implementation read the original Excel XML directly and checked the source hashes, included row numbers, star-by-label counts and star-filter totals. All 62 checks passed. This verifies the calculations and row selection, not the accuracy of every annotation.
Technical details and exact decision rule
The primary quantity is the number of final-label 1 or 2 reviews with rating 4 or 5, divided by all final-label 1 or 2 reviews. Labels mean 0 = no need, 1 = explicit need, 2 = implicit need. The secondary missed share uses ratings 3 to 5 in the numerator.
Primary duplicate keys: store, app, normalized text, stars and final label. Conflicting stars or labels within store/app/text groups are excluded before duplicate collapse. Excluded Excel row numbers are in exclusions.csv.
Bootstrap draws resample whole store/app groups with replacement, including groups with zero needs, and recompute the pooled share. There are 20,000 draws per sample, with seeds 202611 and 202612. Reported ranges are the 2.5th and 97.5th percentiles. Equal-app results average the within-app shares among groups with at least one need.
The plan was internally timestamped before the star-by-label analysis, after the source audit and research proposal. It was not externally preregistered. Its original reference to balanced labels reflected the earlier audit and is explicitly corrected by the subsequent field decision; the frozen plan is preserved unchanged. No classifier was trained or tuned on validation.
The primary rule required at least 15% high-star needs and a lower resampling endpoint above 5% in each sample. Secondary checks are descriptive. No selection among many significance tests is presented as confirmation.
What this study establishes, and what remains open
Supported: filtering these datasets to one-star and two-star reviews discards many reviews the source researchers labelled as containing explanation needs. High-star examples include clear questions relevant to choosing or understanding apps.
Still open: how common this is across today’s app stores, how many labelled needs are unresolved, which ones belong in ASO copy, and whether addressing them changes conversion or retention.
The samples are selected, the main benchmark was filtered, and the validation set has only 66 labelled needs. Ratings are not a direct measure of happiness. The labels include feature requests, possible bugs, resolved issues and questions about the wider operating system. These limits matter when applying the result.
The original authors already investigated ratings and explanation needs. Our contribution is the explicit ASO review-filter calculation, cleaning checks and practical interpretation. We do not claim to have discovered this general relationship for the first time.
A publishable conclusion
In two historical app-review samples, four-star and five-star reviews contained 31.4% and 37.9% of the reviews labelled as needing an explanation. An ASO research process limited to one-star and two-star reviews would have missed 50.6% and 59.1% of those labelled needs. This supports including high-star reviews when researching customer questions, while checking which questions belong in listing copy, in-app guidance or support.
Download the evidence
The embedded ZIP includes the unchanged source package, analysis code, frozen plan, field correction, complete analysis rows, per-app results, exclusions, sensitivity checks, diagnostic examples and verification results. No account or external download is needed to access it.
Unzip the evidence and run python analyse.py, then python verify_and_extend.py. Python dependencies are listed in the README and environment record.
The archive includes original public review text for reproducibility. The readable report uses paraphrases and does not reproduce reviewer profiles.
Sources and credit
- Obaidi, M.; Qengaj, K.; Droste, J.; Deters, H.; Herrmann, M.; Klünder, J.; Schmid, E.; Schneider, K. (2025). Dataset: From App Features to Explanation Needs. Zenodo. CC BY 4.0. Original source archive unchanged; our derived tables and report identify the modifications.
- Companion research paper, IEEE REW 2025. Prior work, validation sampling and annotation context.
- Original annotation dataset record. Source context for the main benchmark and its final explanation-need totals.
All experiment percentages and exclusions in this report were computed from the frozen files. They are not figures copied from the companion paper.
Keep exploring
All ASO experiments · Clarify your app’s value · Improve screenshot copy · Understand search intent