Multivariate testing

Also called: MVT, multivariate test

Last updated:

Multivariate testing (MVT) runs an experiment with three or more variations, or with combinations of several independent changes tested together, rather than the two-way split of an A/B test. It measures which variation — or which combination of elements — moves the target metric the most.

How it differs from an A/B test

An A/B test compares one control against one treatment. Multivariate testing either compares many discrete variations (A/B/C/D) or factorially combines elements — say two headlines crossed with two button colours, giving four cells — to measure both the individual effects and how they interact. The trade-off is sample size: more cells split your traffic thinner, so each needs more users to reach significance, which is why MVT suits higher-traffic surfaces.

How feature flags deliver it

A multivariate flag serves one of several variations — string, number, or JSON — to each user, assigned by a deterministic hash so the split stays stable for the life of the test (the same percentage rollout bucketing, divided into more than two buckets); the rollout strategies guide covers the serving model. The flag controls assignment; your analytics measures each cell. Featureflip handles the delivery side and leaves the statistics to your analytics stack — the same division of labour described under experimentation.

Want the full picture? Read the concept guide: Rollout strategies →

Related terms

A/B testing

A/B testing serves two variations to randomly assigned groups of users and measures which one performs better on a chosen metric — turning a product decision into a controlled experiment instead of a guess.

Experimentation

Experimentation is the practice of making product decisions from controlled experiments — running A/B and multivariate tests, measuring outcomes against a hypothesis, and shipping the version the data supports rather than the one with the loudest advocate.

Percentage rollout

A percentage rollout serves a feature to a defined share of users — chosen by a deterministic hash — so the same users stay in the cohort as you ramp the percentage up.

Browse the full feature flag glossary →

Try it in your own app

Free Solo plan covers 10 flags and 2 environments. No credit card, no demo call — sign up and ship.

Get started freeSee pricing