Say what it does before you say what it is

Everyone tells you to lead with what your product does instead of what it is. They're right. They've been right for twenty years, and that's also exactly where every article on the subject stops.

What nobody tells you is how to find out whether your version of the sentence actually works. That's the part with arithmetic in it, and the arithmetic is unkind to small products.

The claim, quickly, because you've heard it

A reader gives your first line two seconds and about five words. In that window they're not asking what is this. They're asking is this about me, which is a different question and the only one that matters.

A collaborative platform for modern finance teams answers the first question. Match two bank feeds that disagree, in about a minute answers the second.

Same product, both sentences true. One of them makes the reader do your sorting for you.

The second sentence also sends people away. Good. Someone who reads it and thinks that's not my problem was never going to pay you, and every week they spend confused in your trial costs you support time and pollutes your numbers.

Right. That's the received wisdom, and you could have got it anywhere.

Here's the part that gets left out

You rewrite the sentence. Now what? How do you know it helped?

Most founders say they'll test it. Almost none of them can, and the reason isn't discipline. It's sample size.

To tell two conversion rates apart with 80 percent power at the usual 5 percent significance, you need roughly this many people per variant:

n  =  16 x p x (1 - p) / d^2

p = your current conversion rate
d = the improvement you want to be able to see

Chasing a fifty percent relative lift, so d = 0.5p, that simplifies to something you can do on a napkin:

n  =  64 x (1 - p) / p

Three percent conversion? That's 64 x 0.97 / 0.03, or about 2069 people per variant. Call it 4100 for the test.

The worse your conversion, the more traffic you need to improve it. Nobody's intuition works this way, which is part of the problem.

And remember what you're buying with that. A fifty percent lift. If you want to catch a ten percent improvement instead, d drops to 0.1p and the requirement goes up twenty five fold. You'd need about 52 000 visitors per variant at three percent. You don't have that. Neither do I.

What that looks like on a calendar

Same test, same effect. The only thing changing is how many people walk past.

Three hundred visitors a month is an ordinary number for a product in year two. At that traffic, the test above takes fourteen months. Your product will have changed twice, your market will have moved, and you'll have been through a full seasonal cycle before the numbers mean anything.

So here's what actually happens instead. You run it for three weeks, you see a difference, you stop. And that difference is noise. Stopping when the result pleases you isn't a small flaw in the method, it is the method, because at small samples the gap between variants wanders around and will eventually favour whichever one you're rooting for. Wait long enough and it'll favour the other one too.

I've done this. Twice, that I'll admit to.

So do this instead, in this order

Change the metric, not your patience. Signups are rare, and rare events are expensive to measure. Scroll depth past the first screen happens ten to twenty times more often, and your sample requirement falls with it. Yes, it's a worse proxy for money. It's a much better proxy for whether the sentence did its job, which was to buy you a second paragraph. Decide on scroll, then check signups afterwards as a sanity test rather than a referee.

Aim at a bigger effect. You're not hunting a two percent gain. You're trying to tell a sentence that works from one that doesn't, and that gap is usually enormous. Design for a doubling. If you need statistics to see the difference, ship neither and go do something useful.

Under a thousand visitors a month, stop testing and start asking. Five conversations beat any experiment you can run at that traffic. Show two versions to five people who have the problem and aren't your friends. Ask what they think the product does. Count how many get it right. That's a sample of five and it proves nothing statistically, but it measures comprehension directly instead of guessing at it from clicks, and comprehension is the thing you're actually trying to fix.

If you insist on testing, run it as A, then B, then A again. Four weeks each. That third block is the whole point: if B beat A and then A returns to its old level, you've got something. If A comes back higher than it started, congratulations, your traffic changed underneath you and the comparison was worthless. Nobody runs the third block. It's the only part doing any work.

Four questions before you measure anything

This takes five minutes and kills most bad first lines without a single visitor.

That last one is the one founders fight me on, and it's the most diagnostic of the four. If you can't name the visitor your first line is built to repel, you haven't written positioning. You've written a description and hoped.

When this advice is wrong

It isn't universal, and the exceptions are predictable enough to check.

You genuinely do several unrelated jobs. If four different buyers hire your product for four different reasons, a job first sentence picks one and abandons three. The answer usually isn't a broader sentence, it's separate pages. That's a bigger commitment than an afternoon's rewrite, so know what you're signing up for.

The buyer already shops by category. Someone whose accountant told them to go buy accounting software is searching the category by name. Lead with a job they haven't framed that way and you're harder to recognise, not easier. When the category is the query, the category goes in the sentence.

Your reader needs permission. Past a certain price the person reading isn't the person deciding. What they need is a sentence they can forward, and forwardable sentences carry the category, because whoever opens that email has no context at all.

The bit that'll bother you

A job first sentence commits you, in public, to a narrower product than the one in your head. Every instinct you have will argue for the broad version, and it'll dress the argument up as strategy.

It isn't strategy. It's trading readers you have today against optionality on readers you don't. Sometimes that's the right trade. It's rarely as right as it feels while you're writing.

Besides, it's a sentence. Twenty minutes to change, no migration, no announcement. I've rewritten mine twice in three years, both times narrower than before, both times the numbers went up. Stop guarding it.