Skip to content

Hints and tips

Jump to bottom Edit New page
Eliot Sykes edited this page Oct 15, 2021 · 5 revisions

  • The old version should be given as the first attribute in the ab test:

    ab_test("ab_test_name", "old version", "new version 1", "new version 2", etc...)
    • This will help to set it as a fallback (called control) when the redis connection fails
    • Also this means that any other alternative is compared statistically with this in the dashboard

  • How to stop an experiment

    • Stop the experiment but have the experiment code in place without it running? Simply select the alternative you want from the dashboard. This will only show the alternative you have selected, while the experiment won’t count participants any more.

    • Stop the experiment permanently? You will have to remove all the ab_test code and finalization that was used for this experiment only and then delete the experiment (to remove all the data from redis) from the dashboard.

    • Stop the experiment temporarily? There is no such function currently (v 0.4.1) so as to PAUSE an experiment, but you can always select an option to show temporarily (probably the one you have set as the control) without removing the ab_test code and the experiment. When you want to retake the code simply click “Reset Data” or “Delete” from the dashboard. It will remove all the data and start the experiment from scratch with a new version number

    • Privately test experiment alternative while showing control to public?

      • Set the experiment percentages to 100% for the control, 0% for other variants. The control will always be shown to the public.
      • To privately test a variant, override using the "Force for current user" button on the split dashboard or use a querystring override.
      • Once private testing is finished, change the percentages to the desired values. The control will need to be less than 100%, and the variants greater than 0%. Be sure to "Reset Data" for the experiment in the split dashboard so the experiment is restarted and not tainted by the earlier testing.

  • Split Dashboard

    • Delete Button

      • It erases everything from the current expirement concerning the data assembled

      • Whenever it is clicked (either when an option is selected or not) if the ab_test code is in use the test will begin again with new data and a new version number.

      • In case you want to change the control attribute you should deploy your code with this change and then delete the experiment from the dashboard. When the first user runs the test you will see the experiment with the correct order.

    • Reset Data Button

      • Pretty much what Delete does but without deleting the experiment board.

    • Use This Button

      • With this button you can select the option to be used and finalize the test

    • Control Attribute

      • The control alternative is the version that all the other alternatives are compared to

    • Version Attribute

      • It refers to how many times the experiment has been reset or deleted

    • Participants Column

      • Total users that have been given that alternative

    • Non-Finished Column

      • Users that have run the test but haven’t finalized it yet. (Completed - Participants)

    • Completed Column

      • Total users that have run the test and followed every step till the test finalized (was completed)

    • Conversion Rate Column

      • The conversion rate has 2 attributes

        • The percentage that appears in black letters that is calculated by the percentage of those who took the test and finalized.

        • The percentage that appears in red or green color that is the percentage of comparison to the control alternative. This is calculated by: ( Alternatives’ Conversion / Controls’ Conversion) - 1

Add a custom footer
Add a custom sidebar
Clone this wiki locally