Do bad reviews warn me that an app will leave the top 20?

Bad reviews rose.
Most apps stayed in the top 20.

We found nine bad-review spikes in five apps. Two were followed by an exit from the US top 20 the next day. Seven were not.

The question: does a sudden rise in bad reviews make an app more likely to lose its top-20 position?

We can show exactly what happened after these nine spikes. We do not have enough comparable cases to establish that bad reviews increased the risk of leaving the top 20. The useful ASO finding is narrower: a bad-review spike did not consistently precede an immediate exit in this sample.

A review alert and a ranking forecast are different things.

An ASO team might see a sudden wave of 1-star and 2-star reviews and worry that its app is about to disappear from the top charts. That concern is worth testing. If bad-review spikes reliably come before a ranking drop, they could provide an early warning.

Here, we tested leaving the US top-free apps chart’s top 20. We did not test keyword search rankings, downloads, or whether the app listing converts visitors into users.

Apple says ratings and reviews are among the factors used in App Store search. That statement does not establish how the top-free chart responds to a sudden burst of bad reviews. Apple’s search guidance.

After a bad-review spike

2 of 9 left the top 20

22.2% of the eligible spike observations.

Without a bad-review spike

29 of 168 left the top 20

17.3% of the eligible non-spike observations.

These are raw counts, not a fair estimate of the effect of bad reviews. The groups contain different apps and starting ranks, and the same app can appear on several days. The 4.9 percentage-point difference is not proof of increased risk.

Why we cannot turn that difference into a strong claim

We planned to compare each app’s spike days with its own non-spike days, while allowing for its starting chart position and the week. Only four apps had both kinds of eligible days, giving eight spike days and 41 comparison days.

Before collecting the reviews, we required at least five such apps, ten spike days and 20 comparison days. The main comparison did not meet those requirements. Even those minimums would have produced a small, fragile study.

Both exits started close to the cutoff.

The two apps that left the top 20 started at #16 and #19. All seven spike observations that started in the top ten remained in the top 20 the next day. That is a description of these cases, not evidence that top-ten apps are protected.

Gmail: the app left the top 20

#19 → #24

On 2 October, all eight retrieved reviews in the preceding 24 hours were bad. In the seven-day baseline, 26 of 49 were bad. The next chart placed Gmail at #24.

The sequence fits the theory. It does not show that the reviews caused the fall.

Capital One Mobile: the app stayed #1

#1 → #1

On 20 October, 21 of 41 retrieved reviews in the preceding 24 hours were bad, up from 45 of 175 in the baseline. The next chart still placed the app at #1.

A bad-review spike did not force an immediate ranking drop in this case.

Four observations had a worse numerical rank the next day, three improved, and two stayed unchanged. “Stayed in the top 20” does not mean the app’s rank never worsened. These are archived chart observations paired with reviews retrieved later.

Changing the definition did not settle the question.

We also checked stricter and looser definitions of a spike, longer waits, and different starting positions. The table shows the number of observations outside the top 20 divided by the number tested.

The looser rule, requiring a 10-point rise in the bad-review share instead of 20 points, provided enough observations for the planned adjustment. It used 65 observations from six apps: 12 spike days and 53 comparison days.

Its estimate was a 24.9 percentage-point increase in next-day exit risk. But the 95% uncertainty interval ran from a 13.5-point decrease to a 63.3-point increase. With just six apps, that interval is fragile and far too wide to support a reliable ASO forecast.

The other checks did not meet the minimum sample for the adjusted comparison. None replaces the original test. The three-day and seven-day rows use the apps with an available chart on that exact later date, so their samples differ.

What counted as a bad-review spike?

For each app in the top 20, we looked backward from the time of the chart snapshot. We compared the latest 24 hours of written reviews with the previous seven days.

A day counted as a spike only when all three conditions held:

1. There were at least three bad reviews in the latest 24 hours.
2. The bad-review count was at least twice the app’s baseline daily average.
3. The share of reviews that were bad rose by at least 20 percentage points. For example, a rise from 30% to 50% meets this part of the rule.

We also required at least 20 baseline reviews and five current-window reviews, and the retrieved history had to reach the beginning of the baseline. These are our study rules, not Apple’s ranking rules or an established ASO standard.

Then we checked the next chart

The main outcome was simple: did the app appear outside the top 20 in the next calendar day’s complete chart? The charts had to be 18 to 30 hours apart. An app absent from the complete top 100 counted as outside the top 20, not as removed from the App Store.

Where the data came from

We selected every app that appeared in the top 20 of the preserved US chart series: 57 app IDs. The archive contained 37 valid snapshots, giving 36 daily charts after keeping the latest snapshot per UTC date. Each chart had 100 unique apps in positions 1 through 100.

The chart period was 19 September through 25 October 2026. The 8 October chart was missing. We did not fill it in. We retrieved up to ten pages per app from Apple’s public US written-review feed on 27 October 2026, 11:10–11:16 UTC.

In total, 437 feed requests returned 21,750 unique written reviews from 55 apps. Two apps returned empty first pages. Thirty-seven apps reached our ten-page cap. Only 30 apps contributed usable primary observations. A large review total therefore did not translate into a large number of usable spikes.

What this data cannot tell us

• What the review feed looked like at the time. We retrieved it after the chart dates. Reviews may have been edited or removed. Their update times are not guaranteed original posting times.
• Whether every review is present. A history reaching the baseline date is a coverage check, not proof of complete history. These feeds also do not contain all star-only ratings.
• Why an app moved. We did not measure advertising, news, downloads, outages or competing apps’ activity. Any of these could affect reviews and ranks together.
• What happens in other markets or rankings. This is one US top-free chart series. It does not establish effects on keyword rankings, paid charts, other countries or smaller apps.

The relationship can be more complicated than “bad reviews, lower rank.”

A separate study by Xian Gu, Jingcun Cao and Yulin Fang examined review manipulation and filtering in China’s App Store. Its data covered 40,570 apps and 58 million reviews from November 2018 through October 2019. The authors reported that manipulated one-star reviews were associated with better rankings in the short term, before filtering changed the pattern.

That is an external finding about a different market, period and type of review. It does not establish the effect of ordinary bad reviews in our US sample. We did not classify any reviews in this experiment as manipulated, and we did not obtain or replicate that study’s underlying dataset. Journal article · Authors’ university summary.

Every counted case is included.

The evidence download contains the selected chart facts, review IDs and stars, collection times, all eligible app-days, exclusion counts, the frozen analysis rules, analysis code and independent reproduction code. Reviewer names and review text are excluded.

Extract the ZIP and run python reproduce.py with NumPy and SciPy installed to recompute the reported results without fetching fresh data.

Sources

1. AppOrbit pinned source, by ivershuo. Chart extract preserved in Experiment 10; source hashes and attribution included. Chart database: ODbL 1.0.
2. Apple US customer-review feed, example endpoint for Gmail. The acquisition log contains the exact URLs and retrieval times for all 437 requests. Live feeds change after collection.
3. Apple Developer: App Store search. Used for ASO context, not as evidence for a top-chart ranking formula.
4. Gu, X., Cao, J., & Fang, Y. Review Manipulation and Filtering on Digital Platforms. Information Systems Research, 37(2), 927–947 (2026; online publication 2025). External research, not part of our test sample.