Skip to main content
View all posts

Calendly Growth Experiment: Trial Start Full Page Modal

Asking for plan selection upfront reduces activation, as some users abandon the product entirely when asked to choose before experiencing core value.

Marshall Hahn
Marshall HahnSoftware Engineer
Growth Engineering
Calendly Growth Experiment: Trial Start Full Page Modal

Hypothesis

IF the product proactively tells users their 14-day free trial has started, highlight paid features, and show a persistent countdown,

THEN more users will activate and upgrade after experiencing value,

AND increase trial conversion rate by 2% without hurting retention or satisfaction.

Experiment

After completing the intro flow after sign up, new users will see a trial awareness modal with two call to action buttons that lets them upgrade or dismiss the modal and continue to the trial. Two iterations were run for this experiment, with different copy for each one. The left navigation included a countdown of days for the trial.

Iteration 1

Variant 1

Modal with two buttons:

  • Maybe later button to dismiss modal
  • Upgrade now button to open upgrade modal
iteration_1_variant_both.png

Variant 2

  • The modal from variant 1, plus
  • A left navigation trial countdown
trial_awareness_experiment_iteration_1_variant_left_nav.png

Iteration 2

Variant 1

  • Continue to free button to dismiss modal
  • Choose a plan now button to open upgrade modal
  • Value Proposition Copy
trial_awareness_experiment_iteration_2_variant_1.png

Variant 2

  • Continue to free button to dismiss modal
  • Choose a plan now button to open upgrade modal
  • Product Features Copy
trial_awareness_experiment_iteration_2_variant_2.png

Results

  • Subscriptions Created:
    • Variant 1: +19.9% (StatSig)
    • Variant 2: +23.5% (StatSig)
  • First Invitee Booked:
    • Variant 1: -5.3% (StatSig)
    • Variant 2: -5.4% (StatSig)
  • Fifth Invitee Booked:
    • Variant 1: -6.2% (StatSig)
    • Variant 2: -7.1% (StatSig)
  • Week 1 Scheduling Usage Retention:
    • Variant 1: -4.5% (StatSig)
    • Variant 2: -5.0% (StatSig)
  • Week 2 Scheduling Usage Retention:
    • Variant 1: -5.6% (StatSig)
    • Variant 2: -7.0% (StatSig)
  • Week 3 Scheduling Usage Retention:
    • Variant 1: -6.9% (StatSig)
    • Variant 2: -8.6% (StatSig)
  • Week 4 Scheduling Usage Retention:
    • Variant 1: -7.8% (StatSig)
    • Variant 2: -8.3% (StatSig)

Key Learnings

  • Asking users to choose a plan upfront means some will ultimately not choose to use Calendly at all.
  • Conversion rates may have significantly increased but it also deterred a significant number of users from using Calendly for scheduling purposes. In other words, scheduling usage retention deteriorated while also seeing an improvement in the conversion rate.