Abstract
We wrote ten buying questions that name no brand (five skincare, five fashion). Each has an English version and a Hinglish version, and an independent reviewer checked that both ask the same thing. Hinglish here means Hindi and English typed in English letters. Each version ran eight times on ChatGPT, Gemini and Perplexity, geolocated to India, inside a single 57-minute window. That is 480 responses, collected with zero permanent failures for $2.16.
Prompt language changes what AI recommends, and the size of the change depends on the engine far more than on the product category. Measured against each engine's own rerun-to-rerun variation, the language effect on recommended brands is 2.2 points on ChatGPT, 7.8 points on Gemini and 23.4 points on Perplexity. Citations diverge further: Gemini attaches sources to 90.0% of English answers and 41.3% of Hinglish ones. Individual brands swing by up to 17.5 percentage points of visibility between the two languages.
Why did we run this study?
Hinglish is how many Indians type online: romanized Hindi (Hindi words typed in English letters) mixed with English words. One example is the Hinglish version of "which face wash should I buy for oily skin?": "oily skin" and "face wash" stay in English, and the rest is Hindi typed in English letters. The exact Hinglish wording of every study prompt is in the open data on GitHub (opens in a new tab). Most published research on generative engine optimization (GEO, the work of getting a brand named in AI answers) tests English prompts only.
That gap matters to anyone selling in India. If engines name different brands for a Hinglish version of the same buying question, then English-only tracking describes answers that Hinglish users never see. In the 70 sources we reviewed before collecting data, we found no controlled comparison of English and code-mixed buying prompts (questions that switch between two languages in one sentence), and no GEO measurement run from India.
How does the study work?
The core problem with measuring an AI engine is that it does not give the same answer twice. Ask the identical question again and the brand list shuffles. English and Hinglish answers will always differ somewhat. The test is whether they differ by more than the engine differs from itself.
Every result below is built on that comparison. We ran each prompt eight times per language, measured how much two same-language reruns overlap (the engine's own noise floor), then measured how much an English answer overlaps a Hinglish one. The difference between those two numbers is the Jaccard gap: how much the brand list changed, compared with how much it changes when you simply ask again. If it is zero, language does not matter.
| Factor | Value |
|---|---|
| Prompts | 10 buying prompts that name no brand, 5 skincare and 5 fashion, frozen before collection |
| Languages | English and romanized Hinglish, pairs independently reviewed for equivalence |
| Engines | ChatGPT with web search and Gemini, both captured from the consumer web apps; Perplexity Sonar (its API answer model) with live web search |
| Market | India-geolocated on every call |
| Repetitions | 8 per prompt per language per engine |
| Responses | 480, zero permanent failures, $2.16 total cost |
| Window | One 57-minute window, randomized interleaved order, fixed seed |
| Statistics | 95% ranges on every rate (Wilson intervals); p-values from checking all 1,024 ways the 10 per-prompt gaps could flip sign by chance (exact permutation test); 95% ranges on each gap from resampling the 10 prompts repeatedly (bootstrap) |
No brand name appears in any prompt, so this measures which brands the engine brings up on its own. The Hinglish versions mix the two languages the way Indian users type them (category nouns in English, function words in romanized Hindi), with no literal translation. Three of the ten pairs failed an independent review and were rewritten before any data was analyzed: one of them lacked an India anchor in its English version, which would have mixed up language with market. That review log is published with the data.
How much does the brand list change on each engine?
Beyond normal rerun noise, the brand list changed 2.2 points on ChatGPT, 7.8 on Gemini and 23.4 on Perplexity. Each engine's gap clears the one-sided test we fixed before collection, and the largest is about ten times the smallest. The engine you measure matters more than the category you measure.
Recommended-brand overlap: identical reruns vs across languages
| Engine | Same language, rerun to rerun | English vs Hinglish | Gap | p (one-sided) |
|---|---|---|---|---|
| ChatGPT | 54.7% | 52.5% | 2.2 pts | .046 |
| Gemini | 51.2% | 43.3% | 7.8 pts | .0098 |
| Perplexity | 68.3% | 44.9% | 23.4 pts | .0010 |
| Engine | Rerun baseline | Cross-language | Gap | 95% CI | p (one-sided) |
|---|---|---|---|---|---|
| ChatGPT † | 54.7% | 52.5% | 2.2 pts | 0.3 to 4.6 | .046 |
| Gemini | 51.2% | 43.3% | 7.8 pts | 2.6 to 13.9 | .0098 |
| Perplexity | 68.3% | 44.9% | 23.4 pts | 12.9 to 35.1 | .0010 |
p is the chance of seeing a gap this large if language made no difference. 95% CI is the range the gap most likely sits in. † ChatGPT is the fragile result, so we flag it: the two-sided p is .092, four of its ten per-prompt deltas are negative, and the effect is carried entirely by the skincare half (skincare 4.45 points, p = .063; fashion 0.01 points, p = .469). We describe ChatGPT as approximately language-invariant.
Perplexity is the striking case. It is at once the most self-consistent engine, with the highest rerun overlap of the engines tested at 68.3%, and the most language-sensitive. Its cross-language drop cannot be dismissed as randomness, because its randomness is the lowest in the set.
How far does Gemini's citation rate drop in Hinglish?
The starkest number in the study has nothing to do with brands. Gemini attached at least one citation to 90.0% of its English answers (72 of 80) and 41.3% of its Hinglish answers (33 of 80). Mean citation count falls from 9.4 to 5.2, and the Hinglish answers are about a third shorter (4,609 down to 3,099 characters on average). The study records the drop. It does not establish the cause, because the design cannot separate how Gemini searches from how it writes.
Share of answers carrying at least one citation
| Engine | English prompt | Hinglish prompt |
|---|---|---|
| ChatGPT | 100% (80 of 80) | 100% (80 of 80) |
| Gemini | 90% (72 of 80) | 41.3% (33 of 80) |
| Perplexity | 100% (80 of 80) | 100% (80 of 80) |
ChatGPT and Perplexity cite on every answer in both languages, so the drop is specific to Gemini. On the same prompts, in the same window, Gemini gave Hinglish users a shorter answer with fewer sources. It also answered 8 of 80 romanized Hinglish prompts in Devanagari, the script Hindi is usually written in, which the user did not type.
In August, language also changed which sites got cited. Beyond normal rerun noise, the set of cited domains changed 32.4 points on Perplexity, 22.3 on Gemini and 3.1 on ChatGPT.
Cited-domain overlap: identical reruns vs across languages
| Engine | Same language, rerun to rerun | English vs Hinglish | Gap | p (one-sided) |
|---|---|---|---|---|
| ChatGPT | 33.4% | 30.3% | 3.1 pts | .033 |
| Gemini | 33.1% | 10.8% | 22.3 pts | .0098 |
| Perplexity | 85.2% | 52.8% | 32.4 pts | .0010 |
Where engines do cite, the source mix in Indian AI shopping answers looks nothing like a classic search results page. Counting every appearance across the 4,640 citations in the corpus, youtube.com leads with 534 and reddit.com is second with 191, ahead of nykaa.com (176), and a long tail of small Indian review blogs out-cites most mainstream publishers. These are counts from the August wave, and most came from Perplexity (518 and 182); the October re-run cited a different mix of sites (see section 6). Counted instead by how many answers cite a domain at all, reddit.com leads with 106 of 480 and youtube.com follows with 100, so the ranking depends on the basis and we state which one we mean. Our guide on how to get cited by AI covers how to show up on the kinds of sources engines cite in India.
Which brands gain and lose in Hinglish?
Aggregate overlap statistics are the rigorous part, but the practical question for a brand is simpler: does my visibility change? On the language-sensitive engines, by up to 17.5 points.
Brands that gain or lose visibility in Hinglish: Perplexity
| Brand | English | Hinglish | Change |
|---|---|---|---|
| The Derma Co | 2.5% (2 of 80) | 20% (16 of 80) | +17.5 pts |
| Taneira | 15% (12 of 80) | 0% (0 of 80) | −15 pts |
| Plum | 28.7% (23 of 80) | 15% (12 of 80) | −13.7 pts |
| La Roche-Posay | 13.8% (11 of 80) | 1.3% (1 of 80) | −12.5 pts |
| Deconstruct | 3.8% (3 of 80) | 16.3% (13 of 80) | +12.5 pts |
| Minimalist | 22.5% (18 of 80) | 33.8% (27 of 80) | +11.3 pts |
| Vastranand | 10% (8 of 80) | 0% (0 of 80) | −10 pts |
| Garnier | 7.5% (6 of 80) | 17.5% (14 of 80) | +10 pts |
| Allen Solly | 1.3% (1 of 80) | 11.3% (9 of 80) | +10 pts |
| Aurelia | 0% (0 of 80) | 10% (8 of 80) | +10 pts |
A pattern worth naming, though we did not test it as a hypothesis and it should be treated as an observation: on Perplexity the brands losing ground in Hinglish skew international or premium, while the gainers skew mass-market Indian direct-to-consumer. The India-origin share of recommended brands rises in the Hinglish arm on Gemini (75.6% to 82%) and Perplexity (63.7% to 66.3%), and stays flat on ChatGPT (74.4% to 74.7%).
Brands that gain or lose visibility in Hinglish: Gemini
| Brand | English | Hinglish | Change |
|---|---|---|---|
| Dot & Key | 10% (8 of 80) | 21.3% (17 of 80) | +11.3 pts |
| CeraVe | 10% (8 of 80) | 0% (0 of 80) | −10 pts |
| Allen Solly | 5% (4 of 80) | 15% (12 of 80) | +10 pts |
| The Derma Co | 37.5% (30 of 80) | 28.7% (23 of 80) | −8.8 pts |
| Zodiac | 8.8% (7 of 80) | 0% (0 of 80) | −8.8 pts |
| Sparx | 0% (0 of 80) | 8.8% (7 of 80) | +8.8 pts |
| Minimalist | 45% (36 of 80) | 36.3% (29 of 80) | −8.7 pts |
| Deconstruct | 25% (20 of 80) | 16.3% (13 of 80) | −8.7 pts |
| Plum | 18.8% (15 of 80) | 27.5% (22 of 80) | +8.7 pts |
| Pilgrim | 10% (8 of 80) | 1.3% (1 of 80) | −8.7 pts |
Which brand gets named first also becomes less predictable across languages. On Gemini, two same-language reruns agree on the first brand 63.2% of the time, but an English and a Hinglish answer agree only 42.2% of the time. On Perplexity the same figures are 69.5% and 49.8%.
Two months later: is the gap still there?
On 13 Oct 2026 we ran the identical design again: the same ten prompts, the same engines (ChatGPT, Gemini and Perplexity), India geolocation, eight repetitions, the same brand lexicon and the same analysis code. That is 479 answers for $2.14; one Gemini cell failed five times and its English-Hinglish pair was dropped. The August numbers above are untouched. This section compares the two waves.
Brand-list gap beyond rerun noise, August and October 2026
| Engine | August 2026 gap | August p | October 2026 gap | October p |
|---|---|---|---|---|
| ChatGPT | 2.2 pts | .046 | 0.8 pts | .242 |
| Gemini | 7.8 pts | .0098 | 2 pts | .063 |
| Perplexity | 23.4 pts | .0010 | 28.2 pts | .0010 |
| Engine, wave | Rerun overlap | Cross-language | Gap | Exact p | Domain gap | Cites EN | Cites HG |
|---|---|---|---|---|---|---|---|
| ChatGPT, Aug | 54.7% | 52.5% | 2.2 pts | .046 | 3.1 pts | 100.0% | 100.0% |
| ChatGPT, Oct | 49.6% | 48.8% | 0.8 pts | .242 | 0.7 pts | 100.0% | 100.0% |
| Gemini, Aug | 51.2% | 43.3% | 7.8 pts | .0098 | 22.3 pts | 90.0% | 41.3% |
| Gemini, Oct | 41.5% | 39.5% | 2 pts | .063 | 3 pts | 73.4% | 56.3% |
| Perplexity, Aug | 68.3% | 44.9% | 23.4 pts | .0010 | 32.4 pts | 100.0% | 100.0% |
| Perplexity, Oct | 76.1% | 47.9% | 28.2 pts | .0010 | 52.6 pts | 100.0% | 100.0% |
Brand-set Jaccard overlap between same-language reruns and between the English and Hinglish versions of the same prompt; the gap is the difference, tested with the exact prompt-level sign-flip test. "Cites" is the share of answers with at least one source link. August: 14 Aug 2026, 480 answers. October: 13 Oct 2026, 479 answers.
Perplexity: still there, and larger. The brand gap grew from 23.4 to 28.2 points and the cited-domain gap from 32.4 to 52.6 points, both at p = .0010. Perplexity also became more consistent with itself: two same-language reruns now overlap 76.1%, up from 68.3%. One behaviour is new: Perplexity now writes Devanagari Hindi into answers to romanized prompts. 16 of 80 Hinglish-prompted answers were more than 30% Devanagari by letter count, which is where the study's classifier labels an answer Hindi-script; two were mostly Devanagari and 26 contained some. In August 13 contained a few Devanagari words and none crossed 30%.
Gemini: mostly closed. The August effect of 7.8 points fell to 2 points (p = .063) and the domain effect from 22.3 to 3 points (p = .120), neither below .05. The citation collapse narrowed from 48.7 to 17.1 points, and it narrowed from both sides: Gemini cited sources in 73.4% of English answers (from 90.0%) and 56.3% of Hinglish ones (from 41.3%). Gemini also became noisier: its rerun overlap fell from 51.2% to 41.5%, so any language effect now has more noise to clear.
ChatGPT: gone. 0.8 points (p = .242). In August we called the ChatGPT effect fragile and said it might not survive. It did not. ChatGPT changed in other ways: its answers are about twice as long (4,745 characters in English against 2,222) and cite 6.6 sources on average, up from 2.5.
How much do the answers themselves drift in two months?
The rerun baseline in this study measures overlap inside one hour. Two months apart, the same English question produced brand lists that overlapped 45.6% on ChatGPT, 39.6% on Gemini and 28.4% on Perplexity. Against each engine's own rerun overlap that is a drift ratio of 0.89, 0.86 and 0.4, where 1.0 would mean October matched August as closely as two reruns match each other. Perplexity is the engine that agrees most with itself on the day and least with itself across months.
The same English question, two months apart
| Engine | Reruns inside August | Reruns inside October | August vs October | Drift ratio |
|---|---|---|---|---|
| ChatGPT | 54.4% | 48.3% | 45.6% | 0.89 |
| Gemini | 50.1% | 41.7% | 39.6% | 0.86 |
| Perplexity | 69.2% | 72.5% | 28.4% | 0.4 |
Individual brands moved by more than any language effect. On Perplexity in English, The Derma Co went from 2.5% of answers to 22.5%, Cetaphil from 6.3% to 20%, while Plum fell from 28.7% to 10% and Taneira and La Roche-Posay, each in more than one answer in ten in August, were not named at all. The brands that moved with language in August mostly did not move the same way in October, so a language effect is a property of the engine in that window, not of a brand.
Which sites the engines cite changed too
In August the most-cited sites, counted once per answer, were reddit.com (106 answers) and youtube.com (100). In October amazon.in led with 132 answers, then flipkart.com (82), hindustantimes.com (76), nykaa.com (65) and thedermaco.com (64). Citations fell from 4,640 to 3,471 in all, mostly because Perplexity now attaches 10 sources to an answer instead of 19.6. The August source table on this page and in our guide to the sites AI shopping answers cite describes August.
Most-cited sites, counted once per answer
| Rank | August 2026 | Answers | October 2026 | Answers |
|---|---|---|---|---|
| 1 | reddit.com | 106 | amazon.in | 132 |
| 2 | youtube.com | 100 | flipkart.com | 82 |
| 3 | nykaa.com | 93 | hindustantimes.com | 76 |
| 4 | amazon.in | 79 | nykaa.com | 65 |
| 5 | flipkart.com | 79 | thedermaco.com | 64 |
Checks on the re-run
The brand lexicon was frozen in August. Eleven brands that October answers name in eight or more answers were not in it (Himalaya, Dr. Sheth's, Venusia, Indo Era, Lakmé, Marks & Spencer, KALINI, The Indian Garage Co, Turtle, GoSriKi and Mamaearth). With those added, the October gaps are 0.9, 2.4 and 31.4 points and the August gaps 2.1, 8 and 23.4: the same picture. The comparison above keeps the August lexicon on both waves so that August's published numbers stay exactly as published. Two prompts were collected again after the October window; the direction holds on every engine, with drift ratios of 1.04, 1.06 and 0.94.
What should brands selling in India do with this?
Track English and Hinglish separately, and keep tracking. If your AI visibility tracking asks English questions only, then on Perplexity you are measuring a different brand list from the one Hinglish users see, and on every engine the list you measured two months ago has moved. Our guide on how to measure AI visibility explains how to set up both languages and read them side by side.
Measure each engine on its own. In August ChatGPT barely moved while Perplexity moved 23.4 points; in October Gemini's gap had closed while Perplexity's had grown to 28.2. An average across engines, or a single reading, would hide the change on the engine where it matters most.
Depra AI tracks Hinglish prompts next to English ones on ChatGPT, Gemini, Perplexity, Google AI Overviews and every other AI engine you track, asked from inside India or from any of the 40 markets Depra AI serves. On Growth and above, your report shows English and Hinglish visibility side by side. If you are comparing tools, this list of AI visibility tools for D2C brands notes which ones track Hinglish. Plans and limits are on the pricing page.
How was the study verified?
The full frozen methodology, including the deviations log recording every change made after the design was fixed, is published in the repository. Four verification checks are built into the study, and each of them changed it:
- Prompt review before analysis. An independent reviewer audited all ten pairs for naturalness, equivalence and purchase intent. Three failed and were rewritten, with affected responses discarded and recollected.
- Adversarial recomputation. Six independent automated verification runs (AI agents working from raw data, without our analysis code) recomputed the headline numbers with their own code. All six reproduced them. None refuted them.
- Instrument audit. An audit of 26 brand names that double as ordinary words found three defects in our own tooling, including the brand Simple matching the everyday adjective in 35% of its hits. The verification runs caught two more, both answer-language labels. All five were fixed and the analysis re-run before publication.
- Stability rerun. Two prompts were collected again 30 to 60 minutes later. The direction holds on ChatGPT, Gemini and Perplexity, with drift ratios of 0.91, 1.05 and 1.04, where 1.0 means the later answers matched as closely as reruns inside the window.
- October re-run recomputation. Three independent automated runs recomputed the October headline numbers from the raw files with their own code, without our analysis script. All three reproduced the overlap figures, the English citation rate, the top-cited site and the two-month drift figures to the decimal. They flagged two definitions, which section 6 now states precisely: the Gemini Hinglish citation rate counts all 80 collected answers (45 cited; a strict complete-pair count gives 44 of 79), and "Devanagari" means more than 30% of letters.
One detail cuts against our own headline, so it is worth stating plainly. Twenty-six brand names in our lexicon collide with ordinary English or Hindi words, such as the brand Bata, which is spelled like a common Hinglish word that means "tell". We match those case-sensitively. If we relaxed that rule, the ChatGPT gap would inflate from 2.2 to 13.7 points through Hinglish false positives. Any error in our instrument makes the effect look smaller.
What are the limitations?
Ten prompts in two categories. These results apply to unbranded skincare and fashion buying questions only. Two fieldwork windows two months apart, so this is two snapshots, and section 6 shows how far the engines moved between them. One Hinglish register, urban and romanized, so Devanagari Hindi and other Indian languages remain untested. Logged-out default surfaces, so signed-in users may see something different. Citations are the sources an engine displays. They do not prove why it named a brand.
The effect sizes differ by engine in a way that fits differences in how each one handles a Hinglish query, but this design does not separate retrieval from generation and cannot attribute the difference to either. That is open work.
How can you reproduce it?
Everything needed to check our arithmetic is in the public repository: all 480 raw responses with their complete payloads, the frozen prompt set, the brand lexicon, the analysis code and the verification logs. The analysis is deterministic and byte-reproducible, so one command confirms a clean recomputation.
git clone https://github.com/ifham001/IndicGEO
cd IndicGEO
node analyze.mjs
shasum -a 256 data/analysis/results.json | cut -d' ' -f1
# 6f9a67de4c11f11b3e157b1f21799fce66161393d24b5a043eb4a1c395de1776Fresh collection will not reproduce that digest, and it should not: engine answers change over time, so a new run is a new measurement. Section 6 is exactly that. The October raw answers, run log, lexicon sensitivity files and the comparison script follow the same formats. They are in the repository, in the rerun-2026-10 folder (opens in a new tab). Replications that disagree with us are welcome as issues or pull requests.
How to cite this study
Depra AI Research (2026). IndicGEO Study 01: Same question, different brands. English versus Hinglish buying prompts across ChatGPT, Gemini, and Perplexity in India (Version 1.0.0, August 2026; Version 1.1.0, October 2026 re-run) [Data set]. https://github.com/ifham001/IndicGEO (opens in a new tab)
Code is MIT licensed, data and text are CC BY 4.0. Machine-readable citation metadata is in the repository's CITATION.cff, now at version 1.1.0. The October 2026 re-run files are in the rerun-2026-10 folder (opens in a new tab).



