Do developers answer most reviews with just five messages?

Five messages did not dominate the typical app.

For each app, we counted its five most-used reply texts and calculated their share of all its replies. We then took the middle result across apps, giving each app equal weight.

MHARD · 47 APP GROUPS

of replies came from the five most-used messages in the middle app.

DATAR · 100 PROJECT GROUPS

of replies came from the five most-used messages in the middle project.

The theory needed at least 50% in both samples. Neither sample passed. We kept that threshold and the wording-matching rule unchanged.

“Typical” means the median of the eligible app-group results in this sample. It does not mean the typical app across Google Play. DATAR uses repository groups; 95 of the 100 have one supplied app package ID.

There was still a lot of repeated wording.

The five-message theory is stricter than simply asking whether developers repeat themselves. An app can reuse dozens or hundreds of messages without any five accounting for most replies.

Simple example: if the same message appears ten times, it contributes one different text and nine additional occurrences. Our 43.3% and 36.3% figures use that rule. They are not percentages of replies proven unhelpful or automated.

We match supplied reply text after normalizing letter case, whitespace and Unicode form. We keep names, punctuation, links and numbers in the main test. Different texts can still express the same answer; matching texts can still be appropriate answers to different users.

The actual replies tell different stories.

These historical examples are paraphrased from the frozen files. They are not assessments of the apps’ current support.

Traceable example records: MHARD data rows 107184 (Finch), 4012 and 655 (Intellect); DATAR JSONL rows 691957, 691948 and 692084 (Sky Map), 360344 to 360346 (EasyRPG), 221508, 221562 and 221552 (AFK Redeem). Full source identifiers and selected records are in the evidence.

We inspected a fixed random selection of ten eligible groups per dataset, with up to three different review texts attached to each group’s most frequent reply. Two groups had no repeated reply, giving 56 inspected records altogether. This was assistant inspection, not independent human annotation or a support-quality score.

Why this matters for ASO research

A competitor’s reply count measures visible activity. It does not tell you how many different explanations it provides, whether each response fits the question, or whether customers get their problems solved.

The evidence supports checking the content of replies when evaluating an app’s review management. It does not support assuming that most apps survive on five canned messages.

For ASO, this concerns customer understanding and competitor review-management research. We have no evidence here that repeated wording changes keyword rankings, app-page conversion, installs or retention.

Could cleaning or small wording changes reverse the result?

No check below pushes the middle app close to the original 50% threshold.

The small-variation check uses fixed text rules, not a semantic model. It can miss real templates or merge different messages. It is a sensitivity check, not a replacement for the main calculation.

What about praise versus complaints?

Among the same 39 MHARD apps with at least 30 replies in each star band, the middle top-five share is 14.0% for one-star and two-star reviews and 21.3% for four-star and five-star reviews. Among 85 matched DATAR groups, the corresponding values are 17.6% and 17.7%.

The star-band difference does not repeat consistently across datasets. Ratings are only an approximate way to separate praise from complaints, and different sample sizes can affect concentration. We do not present this as a causal effect.

Where the numbers come from

Both sources cover Google Play. MHARD contains mental-health apps; DATAR covers selected open-source Android projects. The source review coverage ends in July 2023 and December 2023 respectively. This is historical evidence, not a current market survey.

The MHARD source publication reports 200,973 reviews. Our pinned CSV contains 200,972 data records. We use the observed file count. The source also describes cleaning and de-identification. Some supplied replies contain textual emoji names or apparent gaps in contact wording. We measure the files as supplied, not verified copies of the original live-store responses.

Duplicates and app identity

Within an app, we checked repeated review IDs and then repeated combinations of review text, date and rating. Conflicting copies were excluded; matching copies were collapsed. A duplicate-looking record is not proof of a fake review or a repeated person.

No cleaned review ID spans multiple groups within either source. No exact cross-source match was found using review text, reply text, day and stars, or review text alone when at least 50 characters long. These screens do not prove that every app or user differs between sources.

Of the 100 eligible DATAR groups, 95 have one supplied package ID and five have none. No supplied package ID is shared by two eligible projects. Restricting the test to the 95 mapped groups gives a median of 16.8%, still below 50%.

Verification and the original plan

The plan fixed the 50% threshold, minimum of 100 cleaned replies, and median across apps before the per-app results were calculated. The earlier pooled repetition audit was already known. This was an internally timestamped plan, not an external preregistration.

A separate implementation read the frozen reply extracts, repeated the cleaning and checked every eligible app’s reply count, distinct-text count and top-five count. All 161 checks passed. Source checksums and row-level exclusions are preserved.

The conclusion we can publish

Across two selected historical Google Play datasets, the five most-used reply texts account for 17.5% and 16.5% of replies in the middle eligible app group. This rejects our proposed 50% pattern in these samples. Nevertheless, repeated occurrences beyond the first of each within-app text make up 43.3% and 36.3% of eligible reply records. For ASO research, reply volume and wording variety describe different aspects of review management; neither establishes support quality by itself.

Repeated app-review replies are already documented in research, including Gao and colleagues’ work on response generation. Our contribution is the explicit top-five test, duplicate handling, cross-dataset comparison and practical ASO interpretation. This is not a claim to have discovered templates for the first time.

Data, calculations and examples

The embedded ZIP includes all 125,940 source records containing a reply, original identifiers, source hashes, the frozen analysis plan, cleaning ledger, per-app tables, checks, examples and runnable code. Generated cleaned-row copies can be recreated from those inputs.

To reproduce, unzip the evidence and run python analyse.py, followed by python verify.py. Dependencies and source attribution are in the README.

Sources and attribution

1. Wang, Q. and colleagues (2025). MHARD dataset, frozen commit 6a492efa0eaf9a27e2593a4a370349f91e1e044d. MIT license included. Companion ICWSM paper.
2. Abedini, Y.; Hajihosseini, M. H.; Heydarnoori, A. (2024). DATAR archived dataset. Archived dataset licensed CC BY 4.0. Companion MSR paper. This report and its extracted tables are secondary analysis and modifications, not the authors’ original conclusions.
3. Gao, C.; Zeng, J.; Xia, X.; Lo, D.; Lyu, M. R.; King, I. Automating App Review Response Generation. ASE 2019, arXiv version 2020. Prior work on template-based replies.

All Experiment 12 percentages were calculated from the frozen data. They are not copied from the source papers.