Hints and tips

These are a couple of things I found out (or so I may think) by using split for ab-testing. Please feel free to add or comment anything.

• The old version should be given as the first attribute in tha abTesting E.g abtest( AbTestName, OldVersion, NewVersion1, NewVersion2, etc)

  • This will help to set it as a fallback (called control) when the redis connection fails
  • Also this means that any other alternative is compared statistically with this in the dashboard

• How to stop an experiment

  • Stop the experiment but have the experiment code in place without it running? Simply select the alternative you want from the dashboard. This will only show the alternative you have selected, while the experiment won’t count participants any more.

  • Stop the experiment permanently? You will have to remove all the ab_test code and finalization that was used for this experiment only and then delete the experiment (to remove all the data from redis) from the dashboard.

  • Stop the experiment temporarily? There is no such function currently (v 0.4.1) so as to PAUSE an experiment, but you can always select an option to show temporarily (probably the one you have set as the control) without removing the ab_test code and the experiment. When you want to retake the code simply click “Reset Data” or “Delete” from the dashboard. It will remove all the data and start the experiment from scratch with a new version number

• Split Dashboard

  • Delete Button

    • It erases everything from the current expirement concerning the data assembled

    • Whenever it is clicked (either when an option is selected or not) if the ab_test code is in use the test will begin again with new data and a new version number.

    • In case you want to change the control attribute you should deploy your code with this change and then delete the experiment from the dashboard. When the first user runs the test you will see the experiment with the correct order.

  • Reset Data Button

    • Pretty much what Delete does but without deleting the experiment board.

  • Use This Button

    • With this button you can select the option to be used and finalize the test

  • Control Attribute

    • The control alternative is the version that all the other alternatives are compared to

  • Version Attribute

    • It refers to how many times the experiment has been reset or deleted

  • Participants Column

    • Total users that have been given that alternative

  • Non-Finished Column

    • Users that have run the test but haven’t finalized it yet. (Completed - Participants)

  • Completed Column

    • Total users that have run the test and followed every step till the test finalized (was completed)

  • Conversion Rate Column

    • The conversion rate has 2 attributes * The percentage that appears in black letters that is calculated by the percentage of those who took the test and finalized.

      * The percentage that appears in red or green color that is the percentage of comparison to the control alternative.
      

This is calculated by: ( Alternatives’ Conversion / Controls’ Conversion) - 1