Activation is a design problem

Four percent. That was the share of new trials on Katalon True Platform that ever got to the point where the product had done something useful for them. The other ninety six percent signed up, looked around, and left. The number had been on a dashboard for a while. Nobody disputed it. What we disputed was what kind of number it was.

The first reading in the room was that it was a top of funnel problem. Wrong traffic, wrong intent, people kicking tyres. That reading has the advantage of being cheap, because it moves the work to somebody else. I did not believe it, and the way I stopped believing it was not clever. I went and looked at what new users actually did once they were inside. They arrived, understood roughly nothing about what the product wanted from them, and made a reasonable decision to stop.

The definition of the activation event is worth being specific about, because half the argument about activation is really an argument about it. If the event sits too far down the product, you are measuring adoption and calling it activation. If it is too shallow, you are measuring curiosity. We picked one and stopped renegotiating it, which mattered more than picking the perfect one.

What we changed

The product was already AI-native, with seven specialised agents behind it. That was the interesting part and also the problem. A system that can do a lot is harder to start using than a system that does one thing, because the first screen has to answer a question the user has not formed yet. They do not want a tour. They want to find out whether this thing is for them, in about ninety seconds, without reading anything.

So we stopped treating the trial as a period of time and started treating it as a sequence the product was responsible for. Not a checklist widget. The actual first path: what the product does on its own before the user has typed anything, what it asks for, in what order, and what it shows back. Every one of those is a design decision, and every one of them had been made by accident, as a side effect of the order features got built.

We shipped weekly, with an A/B test running more or less continuously. That cadence is the only reason any of this is legible. When you ship monthly, you get one confounded result and a story. When you ship weekly, you get an argument you can settle.

When a number moves this much, everyone who touched it remembers their part as the cause. The honest version usually has one change doing most of the work and a long tail of small ones, and the small ones only pay off because the big one cleared the way.

The two that did nothing

Two experiments returned nothing. Flat. Not a small effect we could not detect, not a mixed result, just a line that did not move.

Both were, in hindsight, attempts to persuade rather than attempts to teach. That distinction is the whole thing. You can put copy in front of somebody explaining why the product is valuable, and it will do nothing, because they have no way to check whether you are telling the truth. The changes that worked did not explain anything. They made the product do something recognisable, fast, on the user's own material, so the judgement was theirs.

I keep the flat results in the same document as the wins. Partly because it stops the team relitigating them, and partly because two dead experiments is a low price for learning which direction the surface responds in.

Why I call it design

Activation gets filed under growth, and then it gets worked on with growth tools: emails, nudges, incentives, checklists, in-app tips. We used some of those. None of them were the mechanism. The mechanism was that the product got better at explaining itself by working, in the first minutes, without a human in the loop.

The number went from 4% to 30%. Same product, broadly the same audience. Nothing I can point to outside the product accounts for it. What changed was that the first ten minutes were designed on purpose instead of inherited.

The reason I think this generalises is not the size of the lift. It is that the lever sat somewhere unfashionable. Nobody gets promoted for reordering the first three screens. It is not a strategy problem and it does not look like one on a slide. It is closer to the work I used to do as an engineer, where the fix is usually smaller and duller than the diagnosis suggests, and the hard part is being willing to look at the boring layer before reaching for the interesting one.

We still have a ceiling. Thirty percent means seven in ten new trials leave without getting anything, and I do not have a story yet for who they are. That is the next number.