C-SEO Bench: Does conversational search SEO actually work?

Conversational search SEO (C-SEO) does not work as well as people hope. Most tactics that polish copy to sound authoritative or insert statistics and citations barely improve citation rank, and in some domains they make it worse. What did work, by a wide margin, was traditional SEO that puts the document near the front of the LLM context, and even the few tactics that helped turned zero-sum once everyone copied them, with gains converging to zero. C-SEO Bench, published in the NeurIPS 2025 Datasets & Benchmarks track, measured this head-on across 2 tasks, 6 domains, and about 1.9k queries ^[1]. We covered the problem definition and metrics of GEO in the original GEO paper, and optimization methods in FeatGEO. This post covers the skeptic's side: a paper that tests that optimism with a cold eye.

Conversational search engines (CSEs) such as Perplexity, Google AI Overviews, and ChatGPT search synthesize an answer instead of returning a ranked list, and they cite only a few documents. So SEO is shifting toward C-SEO, which edits content so that "my document gets cited earlier in the answer" ^[2]. The problem is that most evidence so far comes from a single-actor assumption (only I optimize, competitors stand still) and from narrow domains. C-SEO Bench fills both gaps ^[1].

Figure 1. Best C-SEO vs. best SEO. Left: comparison across 6 domains (C-SEO near zero). Right: gains shrinking as adoption rises (SEO AUC 8.60 vs. C-SEO 1.88).

What was benchmarked

C-SEO Bench is the first multi-task, multi-domain, multi-actor benchmark. It covers the two most common tasks in real CSEs, product recommendation and question answering, each split into three domains for six domains in total. Altogether that is about 1.9k queries and about 16k documents ^[1].

Table 1. C-SEO Bench setup: 2 tasks and 6 domains ^[1].

The paper tested 10 C-SEO methods: 8 classic ones plus 2 new ones. The classic methods are Authoritative tone, Statistics, Citations, Fluency, Unique Words, Technical Terms, Simple Language, and Quotes, the variants proposed in the original GEO paper ^[2]. The paper adds two more ^[1].

• Content Improvement: bundles the eight classic variants into one comprehensive rewrite.
• LLM Guidance: inspired by the llms.txt standard, it prepends a Markdown summary to the document to guide the LLM directly ^[5].

The main CSE model is GPT-4o-mini, with replications on Claude-3.5-Haiku, o3, and o4-mini. The traditional SEO baseline does not edit content. Instead, it simulates higher retrieval ranking by forcing the document into a specific position in the context (Position 1–10) ^[1].

How the effect was measured

The measurement is intuitive. If a C-SEO method works, the LLM should cite our document earlier. So the gain for a document is defined as the difference in citation rank before and after optimization. A positive value means the document moved up ^[1].

Here ${\mathrm{rank}}_d^0$ is the baseline citation rank and ${\mathrm{rank}}_d^{\alpha }$ is the rank after applying C-SEO at adoption rate $\alpha$. The adoption rate $\alpha \in [0,1]$ is the share of all documents using the same method at the same time. A method's average gain is the mean over queries and documents ^[1].

Statistical significance uses a one-sided Wilcoxon signed-rank test, and since several methods are compared within a domain, Holm-Bonferroni correction controls for multiple testing (p<0.05) ^[1]. Finally, because the adoption rate is not known in advance in the real world, the paper summarizes a method's robustness as one number: the area under the curve (AUC) of the gain integrated from 0% to 100% adoption ^[1].

Adding authority or statistics to content barely moves ranking

The results are sobering. On GPT-4o-mini, out of 60 combinations across 6 domains × 10 methods, only 3 showed a statistically significant positive gain, and all of them were in product recommendation. LLM Guidance was significant in Retail (+0.36) and Video Games (+0.24), and Content Improvement only in Retail (+0.18). In question answering (Web, News, Debate) there was essentially no effect ^[1].

Table 2. Citation rank gains on GPT-4o-mini (bold = significant after Holm-Bonferroni correction) ^[1].

The backfire is even more striking. The classic methods were not just ineffective; in several domains they significantly lowered rank. Statistics, for example, pushed documents back by -0.80 in Debate and -0.53 in Web. In the paper's words, most C-SEO is not only "largely ineffective" but often hurts ranking ^[1]. Even in Retail with LLM Guidance, 61.0% of documents kept the same rank, and the remaining positive and negative shifts canceled each other out ^[1].

The real lever is traditional SEO

So what works? The answer is search ranking, not rhetoric in the copy. Simply moving a document to the front of the context produced overwhelming gains. Position 1 delivered a +2.77 rank gain in Retail and +1.89 in Video Games, and was statistically significant in all six domains ^[1]. Compared with the best C-SEO result, LLM Guidance in Retail (+0.36), Position 1 (+2.77) was about 7.7 times larger.

Figure 2. The power of traditional SEO. The earlier a document sits in the context, the larger the rank gain, and at least the first two positions were significant in every domain ^[1].

Figure 2 shows the gain from early positions fading quickly further down. From this, the paper concludes that the upstream stage, where search and retrieval decide document order, matters far more than editing the content. Contrary to the expectation that C-SEO would replace traditional SEO, C-SEO should be seen only as a complement to it ^[1].

When everyone uses the same GEO tactics, the gain goes to zero

Even the few working tactics are not safe. C-SEO is a competitive game, so the more players adopt the same tactic, the more the gain erodes. The paper calls this zero-sum ^[1].

Figure 3. C-SEO is a zero-sum game. As adoption rises, the average gain per adopter falls (LLM Guidance AUC 1.88 in Retail, 1.58 in Games) ^[1].

In Retail, LLM Guidance gained about +0.36 at 10% adoption but only about +0.04 at 100%, almost gone. Content Improvement likewise concentrated its gains among early adopters ^[1]. By area under the curve, LLM Guidance (Retail 1.88, Games 1.58) was more robust than Content Improvement (1.05 and 0.74), but both trend downward. One document's rise comes from another's fall, so only those who adopt first benefit, and the gain disappears when everyone does it. It is a first-mover premium, not a lasting advantage.

Switch the model and it disappears

Even the few successes on GPT-4o-mini did not reproduce on another engine. On Claude-3.5-Haiku, not one of the 10 C-SEO methods produced a significant gain, and 26 of the 30 product recommendation cases significantly lowered rank ^[1].

Table 3. On Claude-3.5-Haiku, C-SEO gains are mostly negative and only traditional SEO works ^[1].

Traditional SEO (Best SEO) still worked significantly on Haiku. In other words, front-of-context placement is a lever that holds up across engines, while content-rewriting C-SEO was a fix that happened to fit one model. That is why claims about C-SEO effects validated on a single engine are hard to trust ^[1].

Limits of this benchmark

The paper narrows its own scope and interprets its results within it ^[1].

• White hat only: adversarial or black hat manipulation such as prompt injection is out of scope. Robustness evaluation is left as a separate problem.
• Commercial closed models: the experiments rely on commercial API models, so transfer to open models or other settings is an open question.
• Interaction with SEO unexplored: synergy or interference when combining traditional SEO and C-SEO is left for future work.
• English and specific tasks: the data is limited to English content and two tasks, so generalization to other languages, cultures, and tasks has not been tested.

The TRAIL view: skepticism comes from measurement

This paper targets exactly the problem we work on every day. In AEO and GEO, results come down to "which sources did the AI cite when it built its answer," and TRAIL Search tracks those citation and share-of-voice (SoV) signals. Translated into principles from our product's point of view, the lessons of C-SEO Bench look like this.

• Don't burn budget massaging copy: recommendations that insert authority, statistics, or citations yield gains near zero or below. ROI can be negative, so we don't sell tone and rhetoric tweaks as the primary lever.
• Retrieval visibility comes first: Position 1 was about 7.7 times the best C-SEO method. The real axis is traditional search visibility, trust, and indexing that lift your document toward the top of the source pool the LLM pulls from.
• Measure engine by engine: what worked on GPT produced zero significant gains on Claude. We don't trust recommendations whose per-tactic effect hasn't been measured on each engine through TRAIL Search's multi-engine fan-out.
• Report effects with their decay by adoption: bragging about single-actor gains overstates the effect. We report the effect of our own recommendations honestly, with decay by adoption rate and significance attached.

In the end, C-SEO Bench's message is simple. In conversational search, the power to put a document up front does not come from polishing surface sentences. It comes from how much search trusts the document enough to pull it forward. Read the optimistic design optimization of FeatGEO alongside this skeptical benchmark, and it becomes clear that "how to earn search's trust" is half of GEO, just as much as "what to include."

You can continue with MMDocRAG: can AI pick evidence in multimodal document RAG?, Cutting LLM agent costs, and Adding a diversity reward to query fan-out.