From a stakeholder request to a testable question

Between "how are sales doing" and SQL

An analyst almost never receives a finished question. What arrives is "take a look at churn", "why did revenue drop", "the new screen seems broken". These are not questions to the data, they are feelings, and behind each one sits a decision nobody has made yet. The work starts by pulling that decision into the open and turning the feeling into a statement you can test with a query and disprove with a number.

Three things a question needs to be testable

• A metric with a definition. Not "churn", but "the share of paying accounts with no successful payment within 30 days after the paid period ends".
• Boundaries. The period, the time zone, the countries, which test and internal accounts are excluded, whether refunds count.
• An answer criterion. What the result will be — a number, a comparison of two numbers, a list — and which value changes the decision and which does not.

Feeling: "it looks like churn went up"

Testable question:
  metric    = share of paying accounts with no renewal in a 30-day window
  breakdown = cohort by month of first payment
  filters   = RU and KZ, no internal domains, no manual refunds
  period    = 2024-01 .. 2025-06, UTC
  criterion = growth above 2 pp three months in a row

Decision: if the criterion holds, part of the acquisition budget
moves to retention. Otherwise we leave it alone.

Four questions that save you a week

1. What decision will you make based on the result? If there is no answer, the task is exploratory and it deserves a different priority.
2. What number would not surprise you? The stakeholder expectation is your first plausibility check.
3. What are we comparing against? A metric without a baseline is meaningless: the previous period, another segment, the plan.
4. Who else computes this number? If finance or the product team computes it too, your figure must either match theirs or come with an explained gap.

Where AI fits

The model works well as a sanitizer of wording. Give it the raw stakeholder request and ask it to list which definitions and boundaries are missing before the task becomes unambiguous. You get a list of gaps, and some of them you did not notice. But do not let the model close those gaps for you: it will happily invent a reasonable-sounding churn definition, you will accept it, finance counts differently, and two weeks later you are explaining the discrepancy in a meeting.

The test of a good statement is simple. Give it to two colleagues separately and ask each to describe the query they would write. If the queries differ in filters or period, the question is not testable yet.

When a question cannot be reformulated

Sometimes three rounds of rewriting still produce nothing testable. There are usually two reasons. First, the data does not exist and never will: "why do people leave for a competitor" cannot be computed from your logs, that needs a survey or interviews, and the honest analyst answer is "this question is not about data". Second, the stakeholder wants support for a choice already made. The tell is that every clarification you propose narrows the question until the opposite result stops being possible. In both cases it is better to say so out loud at the start than to deliver a beautiful chart a week later that convinces nobody of anything.

Cheat sheet

• A stakeholder request is raw material, not a task.
• Metric, boundaries, answer criterion: the minimum for testability.
• Ask what decision the result will drive.
• AI finds gaps in the wording; you supply the definitions.