A/B Study
An A/B Study compares two versions of a page by running the same Personas through both and comparing what happened.
Creating an A/B Study
Section titled “Creating an A/B Study”Describe the study in your own words: what you’re testing and the conversion you care about. Marketrix reads the goal out of that description — every derived field stays editable, and it never invents a URL you didn’t write. Choose how Variant B is made: Another URL for two live pages, or This page, changed to describe what you changed. Marketrix then drafts and applies those changes to Variant A in the browser, reviewable before you run. Review the task every Persona runs on each version — your description itself is never shown to them — and the goal-tailored questions, then choose the Personas. See User Studies for the settings shared across every study type.
How results are compared
Section titled “How results are compared”Each respondent judges one of the two versions and answers about that one. They say where they would have stopped, or that they would have finished, and how sure they feel saying so. Respondents are split across the two versions, so the two counts come from two separate groups, and no answer is coloured by having already seen the other page. Each Persona does drive both versions itself and says which of the two it preferred. The order it sees them in alternates between Personas, so being shown second is not mistaken for being worse.
The headline is a count, not a score: how many respondents out of how many said they would have finished on each variant. Because the two groups are separate, the two counts are read as two independent proportions. A difference is called only when the gap is bigger than groups this size could produce by chance. When too few finished, or too few stopped, on either version, the comparison cannot separate them at all.
A variant is crowned only when that comparison actually separates the two. The report leads with No clear winner in three cases. The two versions might not have been equivalent to sit through. The two groups that answered them might have been too different in size for the gap to mean anything. Or the counts might be too close to tell apart. It always says which of the three it was. When only some Personas met versions that were not equivalent, the report discloses how many respondents that covers. Every run also ends with ranked recommendations and the risks of shipping each variant.
Sometimes completions are too close to call, but the Personas’ own preference does separate — each of them having driven both versions. When that happens, the report says exactly that and names the preferred version. That preference is reported beside the completion counts and never folded into them: a stated preference is not evidence that anyone would have finished.
Simulated respondents are not real visitors, so every figure is a direction to check rather than a measurement, and the report says so where it is measured. The PDF export makes it concrete: next to each variant’s count it prints the range that count would plausibly land in on another run of the same size.