Shuichiro Ogawa
日本語

Notes · updated 2026-08-30 · Markdown

Can Frequency Be Asked What It Closes? (A Novelty Reassessment)

Reassessed the novelty of the handoff item "the decoupling of representation frequency from social prevalence by generative AI" against 303 items of literature (135 classic-through-roughly-2024 items, 117 items added from the most recent three years, and 51 items on the echo-chamber difference).

Contents (12)
  1. Two of the First Verdict’s Findings Did Not Hold
  2. The Core Gap Had Already Been Filled
  3. What Remains Is Only a Gap in Measurement Theory
  4. The Difference from Echo Chamber Research Holds
  5. Discovery Is No Longer Available. Only Building an Apparatus Is Left
  6. How the Question Is Framed
  7. Provisional Scoring of the Novelty Axes
  8. Rechecking the Novelty Illusions
  9. If Mat Is to Be Raised
  10. What Would Constitute a Verdict of No Novelty
  11. Candidate Venues
  12. Answer to the Handoff (Updated)

Two of the First Verdict’s Findings Did Not Hold

This note replaces a verdict already written once. Adding three more years of collection (2023–2026) across five lanes filled in two of the things the first assessment had left blank.

The first verdict read as follows. The decoupling itself is already established, and two residual points remain. First, existing manipulation research requires deceptive intent at the entry point of its definitions, so it cannot handle mass generation without malice. Second, detection takes relationships among fake personas as its evidence, so it cannot handle an accumulation of output not attributable to any persona.

Of these, the first has weakened. Horne & Gruppi (2024) built a dataset of over 7.9 million articles from 1,093 pink slime sites posing as local news. Schmitz et al. (2026) are even more direct: they recorded 84 potential cases across 11 jurisdictions in which AI agents flooded government services with applications. Most of the cases involve ordinary users acting in good faith, and only some involve malicious actors. Agents acting for citizens’ benefit are what is driving the surge in demand.

The definition itself has not moved. The framework by which Mannocci et al. (2024) tried to reconcile industry and academic definitions of coordinated inauthentic behavior still keeps inauthenticity at its core. But it can no longer be said that “the phenomenon goes undescribed.”

The second residual point has changed shape as content-only detection has matured. Puccetti et al. (2024), Dawkins et al. (2025), and Shahriar et al. (2025) attempt to judge using only the linguistic features of text, with no attribution to any persona at all. But every one of them collapses badly the moment an attacker fine-tunes the model privately. That is, the point has shifted from “lacks that unit” to “tried to have that unit, but it is fragile in practice.”

The Core Gap Had Already Been Filled

The first assessment wrote that the core of the central proposition (“exposure to AI-generated content shifts judgments of what is normal”) had not yet been directly demonstrated. That was wrong.

Experiment 3 of Glickman & Sharot (2025, Nature Human Behaviour, N=1,401) shows that showing just three Stable Diffusion-generated images of a “financial manager,” 1.5 seconds each, shifts judgments of the descriptive distribution (“who looks most like a financial manager”) toward white men (from 32.36% to 38.20%, p=0.04, with no significant difference in the control group). AlDahoul, Rahwan & Zaki (2025) obtained results in the same direction using AI-generated face images, in a pre-registered experiment, and regardless of whether AI generation was disclosed. Brady et al. (2026, a pre-registered report in Nature) followed 2,000 people on Bluesky for eight weeks and confirmed on a real platform that an engagement-optimizing feed lowers the accuracy of perceived social norms.

The naming has also been pre-empted. Schroeder et al. (2026) proposed “synthetic consensus” in a policy essay in Science. It is a proposal without empirical work behind it, but it is nearly the same idea as the provisional term synthetic prevalence that the handoff had listed.

The handoff’s “C. the generation of an artificial social prevalence” therefore cannot be established as an independent contribution. “D. the self-reinforcing loop by which a distribution that was fake becomes real” is also pinned down under a generative-AI condition by Glickman & Sharot (2025), on top of Salganik & Watts (2008) and Muchnik et al. (2013).

The effect is not uniform, however. Geber & Stahel (2026, N=1,021) report that the type of recommendation had no significant effect on perceived social norms, and Liu et al. (2025, PNAS) could not detect a short-term polarizing effect of filter-bubble manipulation across four YouTube experiments (roughly 9,000 participants in total). A discrepancy remains: direct exposure to the content of generated output works, while manipulating visibility or recommendation does not, or only weakly.

What Remains Is Only a Gap in Measurement Theory

The third residual point listed in the first assessment has, by contrast, strengthened.

Daikeler et al. (2025, Social Science Computer Review) systematically reviewed 58 data-quality frameworks that existed as of 2025, going as far as building a decision tree and an evidence gap map. What they named as gaps there were visual data and linked data. An error type for the case where the sender lies outside the population appears nowhere among the 58, and it is not even listed as a gap.

New frameworks proposed around the same time are the same. Schneck & Przepiorka (2025)‘s TEF-DBD defines six error sources, but its object is transactional data such as eBay’s, and it does not assume a non-human sender. For the total error framework of Jansson & Japec (2026), we have not checked in the full text whether it mentions non-human senders [unverified: full text not reached; abstract only].

Sentences that touch on the premise do exist. Sen et al. (2021) write in one sentence of TED-On that non-humans such as bots and businesses can act as users without representing individual members of a human population (p. 409). Weiß et al. (2025) also write in one sentence that observed behavior can be due to non-human agents such as bots (p. 928). Both still build the error structure on the correspondence between users and members of the population. In the section discussing coverage error within total survey error, Alsalti, Rohrer & Arslan (2026) write that as the use of generative AI spreads, the sampling unit may no longer correspond to a human individual, undermining the premise that an online sample represents a human population. But this is a remark of one or two sentences, and it is not formulated as a subtype of coverage error, sampling error, nonresponse error, or weighting error.

In an adjacent field, the same statement has already appeared in a top-tier journal. Westwood (2025, PNAS) showed that autonomous LLM agents can evade a survey’s quality controls in 99.8% of 6,000 trials, and stated that the premise “a consistent response is a human response” can no longer be maintained. But its object is a closed measurement apparatus that holds a roster, where the problem is posed in the form of “detecting an intruder.” That is a different way of posing the problem from trace data, which has no mechanism at all for identifying the sender.

This is as far as it can be said. As of August 2026, no framework among the 252 items structures, as an error type, the case where the sender lies outside the population. And given that Westwood (2025)‘s statement is now public, there is no guarantee this gap will stay open for long.

The Difference from Echo Chamber Research Holds

The first objection a reviewer will raise is “that’s already covered by echo chamber research.” Fifty-one items of literature were added to test whether this objection can be answered.

The conclusion is that it holds. But only conditionally.

Why It Holds

Across 24 items of echo chamber research, not one paper explicitly states the premise that a real human being stands behind each observed item. The structure of the methodology is what makes this so. Cinelli et al. (2021) take over 100 million items as the utterances of real users, Barberá et al. (2015) take 3.8 million accounts, and Bakshy et al. (2015) take 10.1 million users as their population. Because the starting point is that nodes are human, there is no reason to doubt it.

The prescriptions have the same structure. Both increasing cross-cutting exposure and bridging are interventions that change the distribution of opinions that actually exist. The lineage on misestimation of composition ratios (Ahler & Sood 2018, Nadeau et al. 1993, Herda 2010) prescribes presenting the true base rate, and Ahler (2014) showed that doing so moderates opinion from 8% to 13%. The premise of this prescription is that a true ratio exists to be corrected.

For items with no corresponding human, neither premise is satisfied. This is because the denominator cannot be defined. This is not a case of “the prescription does not work” but of the prescription’s condition of application not being met, a situation outside the framework altogether.

The negative findings are also a tailwind. The four Meta 2020 election studies (Nyhan et al. 2023, Nature; Guess et al. 2023, Science, two papers; González-Bailón et al. 2023, Science; roughly 208 million people) show that substantially changing exposure does not move attitude measures. That is, echo chamber research has itself concluded that the effect of the single variable of redistributing exposure is small, and moreover its object of manipulation has always remained “which portion of real people’s opinions is visible.”

The effect size of selective exposure itself is also not large. Hart et al. (2009)‘s meta-analysis puts it at d=0.36. A discrepancy several times larger, such as the misestimation of the LGBT proportion that Ahler & Sood (2018) measured (6% estimated as 32%), is hard to explain by this effect size alone.

The Condition: One New Threat Has Appeared

Guay, Marghetis, Wong & Landy (2025, PNAS)‘s “uncertainty-based rescaling (UBR)” is the newest threat to this proposal.

The claim runs as follows. When people estimate a proportion, they pull the judgment toward a prior expectation near 0.5. So small groups are systematically overestimated. This is a topic-independent, domain-general property of statistical processing, and it needs no explanation from xenophobia or media bias.

If this account is correct, part of the proposal is absorbed. If proportion estimation was not drawing much information from the input to begin with, there is little room for contaminating the input to make things worse. The claim that “frequency stops working as a cue for prevalence” can be met with “it was not working much in the first place.”

The Line of Rebuttal

The response is to separate the tasks.

What UBR explains is proportion estimation (what percentage there are). What Experiment 3 of Glickman & Sharot (2025) moved was typicality judgment (who looks most like a financial manager), which does not ask for a proportion. AlDahoul et al. (2025), in contrast, use numerical estimates of proportions (what percentage) as the main dependent variable. Therefore, only Experiment 3 of Glickman & Sharot (2025) directly supports this separation.

That is, the study that showed exposure working on the side of typicality is Glickman & Sharot (2025), and what UBR predicts to regress sits on the side of proportion. The two are different tasks and do not share the same conclusion.

This separation directly determines the paper’s design. Placing proportion estimation as the dependent variable gets absorbed into UBR. Placing typicality judgment, and the range of “what may be attempted” derived from it, as the dependent variable steps outside UBR’s reach. The design of candidate 2 (below) is revised accordingly.

The Connection to Generative AI Already Exists

Sharma, Liao & Xiao (2024, CHI) demonstrate that LLM conversational search promotes biased information-seeking, and Jacob et al. (2025) report a chat-chamber effect. The combination “generative AI × echo chamber” itself already exists, peer-reviewed.

But both address a bias in the user’s search, not the human-correspondence of the items being supplied. The difference is preserved.

Discovery Is No Longer Available. Only Building an Apparatus Is Left

Given the above, the path of placing novelty on “discovery” is closed. What remains is the side of a “diagnostic apparatus.”

The ordinary way of cutting the problem is “is the estimate accurate.” This is the cut used by computational social science and communication research, and the lineage going back to Google Flu Trends sits here (Lazer et al. 2014, Gayo-Avello 2013, Ruths & Pfeffer 2014). And that shelf is now fairly crowded.

In contrast, when the problem is cut with a stance that judges by function and does not ask who carried out the work, the question becomes “what does that frequency close off.” If an observed frequency is converted into a judgment of what is normal, and that further pre-emptively defines the range of what may be attempted, it has the same shape as the terminalization of evaluation (an observation at one point in time turning into grounds that close off later possibilities). Bicchieri’s (2006, 2017) empirical expectations are exactly the intermediate term of this conversion.

And this stance also explains why the two existing lines of apparatus do not work. Manipulation research requires “who intended it.” Measurement theory requires “who is in the population.” Both are judgments that depend on authorship. An accumulation of generated output has no answer to either question. Replacing that with a judgment by function is itself where the contribution sits.

The diagnosis “the reader’s literacy is insufficient” also cannot be adopted under this stance. This is not a normative preference; it is also supported by evidence. Jakesch et al. (2023) showed with over 4,600 participants that humans cannot distinguish AI-generated text, and Liang et al. (2023) showed that detectors systematically misjudge non-native speakers. That a design betting on the ability to discriminate is not supported by the numbers is also confirmed from a separate body of literature in the information environment premised on slop.

How the Question Is Framed

The rejected form (the first proposal)

For representation frequency to be valid as a cue for social prevalence, what properties must the generative process satisfy?

This form asks about the conditions for validity, and it falls back into the lineage of accuracy. Moreover, now that Alsalti et al. (2026) have put the premise into words, merely enumerating conditions would amount to confirming what is already known.

The adopted form (provisional)

When an observed representation frequency is converted into a judgment of “what is normal” and further pre-emptively defines what may be attempted, at which stage of that conversion does it stop holding, and by which property of the generative process? And can that failure be diagnosed without identifying the sender?

The second half is the crux. The condition “can be diagnosed without identifying the sender” names exactly what neither manipulation research nor measurement theory can do.

Provisional Scoring of the Novelty Axes

The final scoring is the job of paper-pipeline’s novelty-auditor, so what follows here is only a rough estimate. The change from the first assessment is noted alongside.

AxisFirstReassessedRationale
PD33Shifts the unit of the question from accuracy to terminalization. But now that Alsalti et al. (2026) have put the premise into words, the room for how it can be written has narrowed
Mat22Does not yet have a new, comparable observational surface. Remains a rearrangement of the 252 items
Met34Provides a diagnostic procedure that judges without identifying the sender. That it corresponds to nothing among Daikeler et al. (2025)‘s 58 frameworks, and is not even named as a gap, backs up the emptiness of this seat. That the 24 items of echo chamber research never once make human-correspondence explicit points in the same direction
ResNoneNot scoredNo empirical work has been done
RW33If submitted as a review paper. Realigns cognitive psychology, manipulation research, survey methodology, and generative AI research not along signal validity but along the axis of terminalization

With Met=4, the single-axis exception gate (Met≥4 or Mat≥4) is touched. The strong pair of PD×Met is also maintained.

Compared with the first assessment, the room for PD has narrowed and the room for Met has widened. The verdict is that the paper’s center of gravity should shift from the discovery of the problem to the construction of a diagnostic procedure.

Rechecking the Novelty Illusions

The first assessment fell under topic illusion, naming illusion, scale illusion, and combination illusion. After reassessment, it stands as follows.

Naming illusion (still applies, the most dangerous). Synthetic consensus was used by Schroeder et al. (2026) in Science. Coining a new term is not a contribution. Use the existing term, and make the difference on the side of the apparatus.

Topic illusion (applies to a lesser degree). The formulation no longer depends on “it’s new because it’s generative AI.” The claim that diagnosis is not possible comes from the composition of the corpus.

Scale illusion (avoidable). “One person can make a million items” is absorbed into Rao & Reiley (2012) and Musser (2023). This should not be used as grounds for novelty.

Combination illusion (has gained an axis). With the axis of terminalization now in place, this is no longer merely a juxtaposition of four bodies of literature. Whether this axis actually changes classification judgments, however, will not be known until it is written.

Framing illusion (newly requires caution). If the substance does not change and only the relabeling “from accuracy to terminalization” does, this becomes a textbook framing illusion. It must be shown with a case that classification judgments actually change.

If Mat Is to Be Raised

Met=4 touches the single-axis exception gate, but with Mat left at 2, the paper does not step outside the frame of a conceptual paper. Two candidate sources of material are listed.

Candidate 1, cross-tabulating a three-axis classification of existing cases against the reach of existing methods Classify existing cases (astroturfing, bot amplification, the fifty-cent party, review fraud, pink slime, agentic flooding, mass posting by generative AI) along three axes: presence or absence of intent, presence or absence of persona attribution, and the real existence of a population-side term. Then map how far existing detection methods and error-correction methods reach. This becomes material that supports Met, and Mat may reach 3 as well. The material is already in place in the corpus (the R1 and R3 groups).

Candidate 2, an experiment separating exposure from terminalization Existing experiments (Glickman & Sharot 2025, AlDahoul et al. 2025) measure only as far as a shift in the judgment of “what is normal.” They do not measure further, to where the range of “what may be attempted” narrows. Manipulate representation frequency using fictitious categories (fictitious eating habits, fictitious student cultures), and measure separately the estimation of descriptive norms and the breadth of options a participant would attempt themselves. If the two separate, that becomes Res. If they do not separate, the axis of terminalization turns out to have been unnecessary, and the axis is discarded along with it.

A condition now attaches to the design of the dependent variable. Given Guay et al. (2025)‘s UBR, proportion estimation (what percentage do it) should not be placed as the primary dependent variable. That would be explained away by regression toward 0.5. Typicality judgment (who typically does it) and the range of options (what one feels free to try) should be placed as the primary dependent variables, with proportion estimation measured only to control for UBR.

Candidate 1 comes first. Candidate 1 supplies the design rationale for candidate 2, and its cost of entry is also low.

What Would Constitute a Verdict of No Novelty

The falsification conditions are updated.

  • If, since Daikeler et al. (2025), a framework has appeared that newly establishes an error type for the case where the sender lies outside the population. If this appears, Met=4 collapses and the entire proposal no longer holds. Monitor this with top priority.
  • If Alsalti et al. (2026)‘s point is formulated as a subtype of coverage error in a subsequent paper.
  • If a paper appears that presents a procedure for “judging the validity of a frequency cue without identifying the sender.”
  • If an experiment already exists that measured exposure narrowing the range of “what may be attempted.” Candidate 2 disappears.
  • If Peterson (2025)‘s peer-reviewed version (AI & Society) can be read as thematizing not a model of knowledge collapse but the validity of frequency cues itself. An unconfirmed item carried over from the first assessment. [unverified: full text not reached; the full text of the peer-reviewed version has not been obtained]
  • If UBR (Guay et al. 2025) is extended to typicality judgment as well, and shown to explain the effect of exposure through regression. The rebuttal that separates the tasks would no longer be usable.
  • If the echo chamber research side produces a framework that treats the human-correspondence of observed items as an explicit variable.

Candidate Venues

  • Public Opinion Quarterly (first choice). This is the venue where Sen et al. (2021)‘s TED-On appeared and where the lineage of Groves & Lyberg (2010)‘s total survey error is well established. The form that fits best is making explicit the implicit premise carried by existing error types and showing the conditions under which it fails, along with a diagnostic procedure. Daikeler et al. (2025)‘s systematic review can be cited directly as evidence of the gap.
  • Big Data & Society (second choice). In the boyd & Crawford lineage, the point of a judgment not dependent on authorship is easy to develop as a conceptual paper.
  • New Media & Society. The connection between manipulation research and information-environment theory. There is a risk, however, of being read within the lineage of accuracy.
  • Social Science Computer Review. The venue where Daikeler et al. (2025), Schneck & Przepiorka (2025), and Weiß et al. (2025) sit side by side. Because it would share a journal with the very work that pointed out the gap, the demand to make the difference explicit would be strict.
  • ACM FAccT. For the case where empirical work is included. This can sit next to Bommasani et al. (2026)‘s algorithmic monoculture research.

AI & Society was listed in the first assessment but is downgraded here, since it would share a journal with Peterson (2025).

Answer to the Handoff (Updated)

The answer to the handoff’s most important question is rewritten from the first assessment.

Is this fully explained already by a simple synthesis of existing research? Or does novelty remain as an independent theoretical problem specific to generative AI?

The decoupling itself is already explained, and the causal path from exposure to normative perception was also directly demonstrated in 2025. The naming has also been pre-empted, in Science. Mass generation without intent also exists as a case study. No novelty remains as a discovery.

What remains is that, as of August 2026, no procedure exists for diagnosing whether a frequency cue holds without identifying the sender. This is a gap in the seat of Met, and Daikeler et al. (2025)‘s systematic review of 58 frameworks backs up that absence. Placing the question in a stance that judges by function and does not ask about authorship lets this gap stand as the unit of the question.

The handoff’s policy changes as follows.

  • The policy of placing “A. the decoupling of representation volume and population distribution” at the center is rejected (maintained from the first assessment).
  • The policy of rewriting “B. the contamination of frequency as an epistemic cue” from the side of conditions is also rejected, because it falls back into the lineage of accuracy.
  • “C. the generation of an artificial social prevalence” is not established as an independent contribution. Schroeder et al. (2026) precede it.
  • “D. the self-reinforcing loop” is also not established as an independent contribution. Glickman & Sharot (2025) have it pinned down under a generative-AI condition.
  • The new center is placed on a procedure for diagnosing, without identifying the sender, where the conversion from frequency to prevalence stops holding.

Unverified Items

  • The body text of Peterson (2025)‘s peer-reviewed version (AI & Society) has not been checked. The arXiv version (2404.03502) is a model of knowledge collapse. [unverified: full text not reached; the full text of the peer-reviewed version has not been obtained]
  • The additional collection from the most recent three years has a high proportion of pre-peer-review preprints. In particular, Schmitz et al. (2026), Ma et al. (2026), and Kim et al. (2025) have not gone through peer review (Dawkins et al. 2025 appeared in Findings of ACL 2025). Daikeler et al. (2025), Alsalti et al. (2026), Westwood (2025), Glickman & Sharot (2025), and Brady et al. (2026), which support the skeleton of the verdict, are all in peer-reviewed journals. [unverified: not attempted; whether the preprints have peer-reviewed versions has not been checked]
  • The scoring of the novelty axes is provisional. The final scoring is done by paper-pipeline’s novelty-auditor.
  • The feasibility of candidate 1 (the three-axis classification) has only been confirmed to the extent that the material is in place in the corpus’s R1 and R3 groups. Whether the classification would actually be determined uniquely has not been verified. [unverified: not attempted; the classification has not yet been carried out]

References


Author: Shuichiro Ogawa (Design Researcher / Design Consultant) About me →