growthinterview.org

The growth skills worth practicing when AI makes experiments cheap

What growth leaders say is changing as AI makes experiments cheap, and the five skills that matter more because of it. Each comes with a worked example you can play through. Free, no-login practice; no AI scoring.

Start with the first skill

growthinterview.org is independent. Not affiliated with any company named here, and not endorsed by anyone quoted. Last reviewed Nov 9, 2026

A 10% lift. Is it real?
Control
15.0%
600 of 4,000
Variant
16.5%
660 of 4,000

The 95% interval runs from −0.1 to +3.1 pts and includes zero; p is about 0.07. Detecting a 10% lift needs 9,254 users per arm, not 4,000.

Try your own numbers in the test readout

Five skills

Our reading of the sources, not a hiring survey: when AI makes variants, copy and analysis cheap to produce, the harder work shifts to reading results honestly, choosing what to measure and following the numbers to the outcome.

Skill 01 Read the test before you ship the winner

Looking back on a decade of growth teams, Andrew Chen writes: “Product Managers are simply expected to understand A/B testing and distribution metrics like CAC, LTV, D30, and other jargon. You interview for this.” In the same post he warns that startups rarely have enough users for statistically significant tests. Our inference: when an AI readout can declare a winner in seconds, a strong answer still checks the counts, the plan and the guardrail before agreeing.

  • Andrew Chen Andreessen Horowitz
Worked example · Experimentation · Launch decisionThe new onboarding shows a 10% lift. Would you ship it?Try it: decide whether a 10% lift from a test stopped early is real

Skill 02 Treat cheap variants as guesses, not wins

Amplitude’s Viv Magida describes teams that can produce 50 test ideas before lunch, and concludes:

“Now that velocity costs nothing, it tells you nothing.”

Viv Magida, Global Solution Architect, Amplitude · Amplitude blog, Jun 1, 2026

She is writing on a vendor’s blog about experimentation programs, not hiring. Our interview translation: when variants are free, the scarce skill is knowing how many comparisons you ran, what bar that implies, and why the best of twenty looks better than it is.

Worked example · Experimentation · AI-generated variantsOne of 20 AI-written headlines won. Would you roll it out?Try it: judge the one winner among 20 AI-written headlines

Skill 03 Split retention by cohort before blaming the product

Sandy Diao, a growth advisor who was an early growth hire at Pinterest and Descript, opens an essay on growth roles with “Fast growth has never been easier. Retention has never been harder.” That is her reading across the companies she has worked with, not a measured trend. Our lesson for interviews: when a blended retention rate moves, split it by sign-up cohort and channel before you blame, or credit, the product.

Worked example · Retention · DiagnosisRetention fell after an AI launch. Would you roll it back?Try it: find why retention fell from 40% to 27.5%

Skill 04 Choose the metric that predicts the outcome

Writing about AI products, where a first output can arrive in under a minute, Diao warns that “the original product engagement-based activation metrics may no longer predict retention.” Our inference: an activation metric has to earn its place by predicting the outcome you care about. A strong answer tests that link before making a number the team’s goal.

Worked example · Metrics · ActivationPlan generation jumped to 71%. Should it be the goal?Try it: choose an activation metric after an AI feature moved one

Skill 05 Measure the whole loop, not the step that moved

Elena Verna, Lovable’s head of growth, expects routine growth work to be automated so that people can “come up with new Growth loops in the product, work on innovative campaigns, or explore new channels.” Brian Balfour, writing as Reforge’s CEO, adds a constraint: “Product Channel Fit states that products are built to mold to channels. Channels do not adapt to your product.” Our reading: a share is a small channel of its own, and what gets shared has to work for the person who sees it. Judge a loop by what comes out the end, not by the step that moved.

  • Elena Verna Lovable
  • Brian Balfour Reforge
Worked example · Growth loops · ExperimentationAI share cards doubled sharing. Would you roll them out?Try it: decide whether doubling shares improved the loop
Free toolRead any A/B test: lift, confidence interval, p-value and the sample size you need

Sources

The quotes are what each person wrote in the linked source, in their role at the time. The interview advice around them is ours. This is an editorial reading of six public sources, not a representative hiring survey, and no quoted person endorses this site.

  1. Andrew Chen, Partner, Andreessen Horowitz, at the time. @andrewchen (Substack), “10 years after “Growth Hacking””, May 13, 2024. Open the source. Checked against the published text.
  2. Viv Magida, Global Solution Architect, Amplitude, at the time. Amplitude blog, “AI broke your experimentation program. here’s how to fix it.”, Jun 1, 2026. Open the source. Checked against the published text.
  3. Sandy Diao, Growth advisor; early growth hire at Pinterest and Descript, at the time. Growth Notes by Sandy Diao (Substack), “Why full-stack growth leaders will own the next decade”, Oct 21, 2025. Open the source. Checked against the published text.
  4. Sandy Diao, Growth advisor; early growth hire at Pinterest and Descript, at the time. Growth Notes by Sandy Diao (Substack), “AI has broken retention - here’s how to fix it and what to measure instead”, Jul 3, 2025. Open the source. Checked against the published text.
  5. Elena Verna, Head of Growth, Lovable, at the time. Elena’s Growth Scoop, “I want to eliminate my own job”, Aug 1, 2025. Open the source. Checked against the published text.
  6. Brian Balfour, Founder and CEO, Reforge, at the time. Brian Balfour (Substack), “The Four Fits: A Growth Framework for the AI Era”, Sep 25, 2025. Open the source. Checked against the published text.

Each quote is checked against the publisher’s text. None comes from audio, and none has been checked by a person listening.