# measure.md — did it actually work? **Version 1.1 · 2026-08-29 · Nameworthy (Whitley Row LLC) · https://nameworthy.ai/spec/** `ai.md` builds it. `verify.md` confirms the site is **readable**. This file is the part that was missing: finding out whether any of it changed what an AI actually tells your customers. **Why that gap matters.** A readability audit is not a result. Most of this industry stops at the point where the files are installed and lets the buyer infer the rest — and the distance between *"your site is now legible"* and *"AI recommends you"* is where nearly all the money is spent and almost none of the measuring happens. We published `verify.md` with that gap named in its own closing section, which was honest but incomplete. This file closes it. --- ## 0. This one is for a person, not an agent Every other file in this spec is written to be executed by an AI. **This one is not, and the reason is the whole point.** You have to ask these questions the way a customer does: in the consumer app, by hand, in a browser. An agent scripting the question would be calling an API, and an API is a different product from the app — different model versions, different search behaviour, different citations. In our own field study across four local verticals, the businesses recommended by the bare OpenAI API overlapped the consumer app's recommendations only about **8% of the time**. One in twelve. Give the model a web-search tool and the overlap roughly doubles, and it still misses most of what the app names. So: **anything convenient enough to automate is measuring something your customers do not see.** That is the trade, and it is why this file asks for your afternoon instead of your API key. --- ## 1. What this measures, and what it does not **It measures:** whether AI engines name your business when someone asks a question a customer would actually ask — and whether that survives being asked again. **It does not measure:** traffic, leads, revenue, or "AI visibility" as a single number. If you want a score, this file will disappoint you on purpose. A blended score across engines averages away the only thing you can act on. --- ## 2. Before you start — the vantage Get these wrong and the rest is decoration. You are trying to answer one question: **who does this engine name for a stranger?** 1. **Log out.** Every engine. A signed-in session carries your history, your past chats, and personalisation. It will flatter you. 2. **Use a fresh session.** Temporary chat, incognito, memory off, whichever the product offers. Never reuse a thread — the previous answer conditions the next one. 3. **Do not prime it.** No "as we discussed", no follow-up, no clarifying reply. Ask the question cold and take the **first** answer. If the engine asks a clarifying question back, record that as the result. It is a real outcome and it happens more than you would think. 4. **Be honest about location.** If the question is local, the engine may infer where you are. Do not use a VPN to sit in your own market and then call the result what a stranger sees. Record where you were. 5. **Record the date and time.** These answers change week to week. An undated observation is not a measurement. 6. **Never paste your own site in.** The moment you hand the engine your URL you are testing whether it can read a page, which `verify.md` already told you. **Write the vantage down before you start**, and use the same one every time. A before-and-after that changes vantage is two numbers that differ for two reasons. --- ## 3. Pick your questions **Ask what a customer asks, in their words, before they know your name.** The single most common self-deception here is asking questions you already win. Guard against it by writing the list **before** you look at any results, and not changing it afterwards. If the list moves, the measurement means nothing. Two kinds, and they must be counted separately: - **Unbranded** — "who is the best `` in ``", "I need `` near ``, who should I call", "how much does `` cost in ``". This is the real test. - **Branded** — "what can you tell me about ``". This tests whether the engine knows you exist at all. **Never blend the two.** A business that resolves perfectly by name and never appears unprompted has a specific, diagnosable problem, and one average hides it. Ten to fifteen unbranded questions plus two or three branded is enough to learn something. The vertical packs in this spec (`profile-law-firms.md`, `profile-med-spas.md`) list question *shapes* to work from. --- ## 4. The engines Ask every one you can. They disagree far more than people expect: across 1,984 measured runs, **any two engines named overlapping businesses about 12% of the time.** One engine's answer is not a read on the others. | Engine | Notes | |---|---| | ChatGPT | The app, logged out, temporary chat | | Google AI Overviews | The overview above ordinary results | | Google AI Mode | A different product from Overviews — record separately | | Gemini | The app | | Perplexity | Logged out | | Claude | The app | **Record each engine separately and never average them.** If you take one number away from this file, that is the one. --- ## 5. Ask more than once **One run is a coin flip.** These engines are non-deterministic by design: they generate fresh searches almost every time and the businesses they name shift run to run. Ask each question **at least twice**, and three times for the questions you care most about. Space the runs a few minutes apart, each in a new session. If the answers disagree with each other, **that is a finding, not a problem with your method.** Write down all of them. A business named in one run of three is not "named" — it is intermittent, and intermittent is the most common real state. --- ## 6. What to record For every run, four things. This is the same vocabulary we score client work against, and it is deliberately boring: - **Mention** — your business is named anywhere in the answer text. - **Recommendation** — you appear in a recommended list. Record your position in it. - **Citation** — your URL is linked as a source the answer drew from. - **Sentiment** — how the answer treats you: positive, neutral, negative. **Mention and citation are different events and the difference matters.** Being used as a source without being named is its own state — the engine read you and recommended somebody else. A flat file is fine: ``` date,engine,question,run,vantage,mentioned,recommended,position,cited,sentiment,who_was_named_instead 2026-08-27,chatgpt,"best in ",1,logged-out temp chat,no,no,,no,,"Competitor A; Competitor B; Competitor C" ``` **Always record who *was* named.** It is the most useful column in the file and the one people skip. Absence tells you nothing about what to do; the list of who won tells you what the engine thinks the answer looks like. Where you land, per engine, is a rung: **Invisible** (not in the answer, not in the sources) → **Read** (your site in the sources, you unnamed) → **Named** (in the answer) → **Recommended** (a repeating pick). There is a fifth rung above those, and it does not exist in a single measurement — see §8. --- ## 7. Eight ways to fool yourself Every one of these is a mistake we have made and had to correct. They are cheap to avoid once named and nearly invisible if you do not know to look. 1. **Substring matching.** If you search the answer text for your business name with a plain text search, a name that contains a common word will match things that are not you. Read the answers. If you script the check, match on word boundaries, not substrings. 2. **Resolve your names before declaring yourself absent.** If the business has rebranded, merged, or trades under a second name, the engine may be naming the *other* one. We once scored a business at zero across an entire panel while its current name was winning in the same runs. **Search every name the business has used before concluding it is invisible.** 3. **Branded questions are about recognition, not containment.** An answer that repeats your name back and then says it cannot find anything about you has not recognised you. Read what the answer *says*, not whether the string appears. 4. **Do not blend engines.** Covered in §4 and worth repeating, because a single average is the most tempting number in the file and the least useful. 5. **Do not blend model versions.** If an engine ships a new model between your two measurements, the change you are looking at has two causes. Record which model answered, where the product tells you, and label the comparison. 6. **Small samples swing wildly.** With a handful of observations, one question entering or leaving moves your average enormously. Do not read a trend line off a week of a small panel. Ours moved forty places in a day on four impressions, and it meant nothing. 7. **Score the whole answer.** Not the summary line, not the sources panel, not the first sentence. The artifact is the answer the customer reads. 8. **Run a positive control.** Include one business you already know is prominent in your market. If your method says *they* are invisible, your method is broken — fix it before you believe anything it says about you. --- ## 8. Now do it again in thirty days **This is the part that matters, and it is the part almost nobody does.** Change nothing. Ask the same questions, the same way, from the same vantage. Then compare — not the average, but question by question and engine by engine: - Of the places you were **named** last month, how many are you still named in? - Of the places you were **recommended**, how many held? - Did anything you fixed show up at all? **Expect to lose ground you thought you had.** In our own cross-panel measurement — 383 first-position transition pairs across six engines — the expected lifetime of an **unmanaged** first-position recommendation was roughly **one to two days**. Separately, SISTRIX measured ChatGPT replacing up to **74% of its cited sources every week** across 82,619 prompts. That is what the top rung of the ladder is for. **Nameworthy** — recommended, and still recommended a month later — is a state that cannot be observed in a single measurement and cannot be bought or faked. It only exists under repeated measurement, which is why one-time work cannot demonstrate it and one-time reports cannot claim it. **If you do only one thing from this file, do the re-run.** A single snapshot tells you what happened once. Two snapshots thirty days apart tell you whether you have a position or a coincidence. --- ## 9. Reading your own numbers honestly - **Absent everywhere, unbranded, but fine by name?** The engines know you exist and do not think you are the answer. That is a corroboration and competition problem, not a markup problem, and no amount of schema fixes it. - **Read but not named?** Your site is being retrieved and passed over. What is on the page is not quotable, or not specific enough to lift. - **Named on one engine and nowhere else?** Normal. See the 12% figure. Fix the weakest surface rather than averaging. - **Named in one run of three?** You are intermittent. That is a real, common, unstable state — not a win. **A number that made you look bad is the most valuable one in your file.** Any honest measurement produces some. If yours produced none, check your method before you celebrate — start with trap 8. --- ## 10. What this file cannot tell you It is a hand measurement, and its limits are real: - **Small samples.** Ten questions across six engines twice is 120 observations, which is enough to see a pattern and not enough to be sure of a small change. - **You are not a stranger.** You know the answer you want. That is exactly why the question list gets written first and never edited afterwards. - **It is a moment, not a trend.** Two points thirty days apart is a line through two points. - **It does not attribute revenue.** Nothing here tells you a customer arrived because of an answer. None of that makes it worthless. **A measured, dated, honest observation beats every confident claim made without one**, including ours. --- ## 11. The honest sales pitch, stated once Doing this properly for a real question set, across six engines, with repeated runs, logged, and then again every month, is genuinely tedious. That tedium is the entire product we sell: we run it, we log every answer, and we hand you the log so you can check us. **But you can absolutely do it yourself, and if you do it you will know more about your own position than most people who bought a five-thousand-dollar build.** That is not a loss leader talking. It is the same reason the rest of this spec is free: the file was never the valuable part. If you would rather see it done once before deciding, the Snapshot is free — five of your real customer questions, run twice on each of six engines, reported per engine, no card and no call: https://nameworthy.ai/start/ --- *Free and public, in full. — https://nameworthy.ai/spec/*